Massachusetts-purple-equal recurrent crash

FiveM Server Development Resource Support
1

Hello,

I’m looking for some help on a quite big crash people on my server are often getting.

Stack:
GTA5_b2802.exe!sub_14161A888 (0x252)
GTA5_b2802.exe!sub_140912B1C (0x9b)
GTA5_b2802.exe!sub_140913338 (0x239)
GTA5_b2802.exe!sub_140931FC8 (0x6e)
GTA5_b2802.exe!sub_140931ECC (0x1a)
GTA5_b2802.exe!sub_14092485C (0x191)
GTA5_b2802.exe!sub_14094E0D4 (0x42)

Canary b2802 (also happened in 2699 under another name)

I’ve no clue what provoked it, even though I’ve managed to reproduce it on my server. Unfortunately, I haven’t found the underlying cause yet, so I’m hopeful someone can help.

Full dump: 1.54 GB file on MEGA

2

The crash location is inside the entity/entityset loader for interiors/mlo’s.

0x00000278e7526510 -> deref -> 0x7ff6e6eef2f8 - baseaddr(0x00007FF6E5540000) -> 0x19AF2F8
This should be rage::fwArchetype::vft or inheriting but is rage::datBase::vft.
rage::datBase->vft_0x48 will trash rax → rbx, dereferencing rbx causes an access violation.

Looking at the weird rage::datBase at 0x00000278e7526510 shows remains of a previous valid rage::fwArchetype (when the destructor is called it will emplace the base class vft again, so that object might have been unloaded and therefore destroyed)

0x00000278e7526510 + 0x18 -> rage::fwArchetype::m_nHash -> 0x1d8406a3 -> po1_05_fenfiz3 (part of a bridge, port_metadata_005_strm.ytyp)
0x00000278e7526510 + 0x3C -> rage::fwArchetype::m_fLodDistance -> 0x43160000 -> 150f -> checks out
0x00000278e7526510 + 0x4C -> rage::fwArchetype::m_fHdTextureDistance -> 0x43158000 -> 149.5f checks out
0x00000278e7526510 + 0x5C -> rage::fwArchetype::m_nflags -> 0x80000100 -> create dummy object flag not set -> should've created a CBuilding later on -> checks out

Location of the po1_05_fenfiz3 model in world: 996.25, -2611.91, 52.76
Location of crash from the provided log: 111.175110, 6630.765137, 31.790512

Am I correct in assuming that the crash only happens inside an interior / in the streaming distance of a specific interior (that interior may be in multiple places on the map)?

Are there any entities defined or imported from other 3rd party .ytyp’s that have a hash collision with the base game hash 0x1d8406a3 in that mlo?

3

Hello,

Thank you for taking the time to analyze it.
In order to reproduce it, I was teleporting between two points multiple times until it crashed. I was moving from the crash position you mentioned to an interior below the dock, so near po1_05_fenfiz3 object.

I’ve searched in all third-party ytyp + ymap for any other prop with the same hash I haven’t found any

4

Another full dump if useful 3.23 GB file on MEGA

5

Second full dump gave some more insight.

Roughly explained there is a rage::atHashMap and a rage::atPoolBase for archetypes.
The hashmap has buckets full of [hash, indexOfPoolEntry, next] (next is a linked list for collisions, just like most modern hashmaps)

That “indexOfPoolEntry” is then used to retrieve the actual rage::fwArchetype (or CBaseModelInfo, CMloModelinfo…) from the pool.

The weird part is: The hashmap in your dump has multiple entries for the same hash. The first entry points to the pool slot that contains the broken, already destructed, rage::datBase entry. This is the one retrieved and therefore causing a crash.
The other entries in the “next hierarchy” with the same hash all point to different uninitialized or unloaded pool slots.
The last two entries in the chain, with different and valid hashes, are just hashmap modulo collisions and are perfectly fine.

In your first dump 0x1D8406A3/po1_05_fenfiz3 was in that pool slot.
In the second dump it is 0xe7d7d718/ch3_13_props_combo330_34_lod.

If you go up the callstack to sub_140912B1C you can see that the game stores the actual entity’s hash it is trying to load on the stack at rsp+0x60.
This hash is the same in both crash dumps: 0x282b8abd/sf_int1_fruit.

Can you check all your third-party ytyps for 0x282b8abd/sf_int1_fruit.

A collision would explain why the pool entry is “trashed” instead of freed too.
Inside rage::fwArchetypeManager::UnregisterStreamedArchetype the game checks if the slot it is about to remove from the pool is actually the correct archetype. It skips deleting the pool entry, but still calls the rage::datBase destructor on it, which is responsible for the remaining vtable.

I also don’t recommend opening this spoiler, looking at a hashmap inside a crashdump via the visual studio “watch” widget qualifies as cursed, but below are my notes for understanding what happened

Summary 
//
// rage::fwArchetypeManager::ms_ArchetypeMap
//

*(uint64_t*)(0x00007FF6E5540000 + 0x1E63400) = 0x000001d9987a0640 -> hashmap data
*(uint16_t*)(0x00007FF6E5540000 + 0x1E63408) = 0xfe8f -> hashmap buckets

//hashmap->buckets[0xhash % mapsize]->xxx
*(uint64_t*)(0x000001d9987a0640 + (8 * (0x282b8abd % 0xfe8f))) -> 0x000001d99876e070 -> first hashmap entry

// This is the first link in the hashmap for hash 0x282b8abd, pointing to the broken pool entry
*(uint32_t*)(0x000001d99876e070 + 0) = 0x282b8abd -> sf_int1_fruit
*(uint16_t*)(0x000001d99876e070 + 4) = 0x9bae -> pool entry valid but destroyed
*(uint64_t*)(0x000001d99876e070 + 8) = 0x000001d99876aa40 -> next

// Second entry in the hashmap buckets linked list
*(uint32_t*)(0x000001d99876aa40  + 0) = 0x282b8abd -> sf_int1_fruit
*(uint16_t*)(0x000001d99876aa40 + 4) = 0xcee2 -> pool entry uninitialized or unloaded
*(uint64_t*)(0x000001d99876aa40  + 8) = 0x000001d998775450 -> next

// Third...
*(uint32_t*)(0x000001d998775450  + 0) = 0x282b8abd -> sf_int1_fruit
*(uint16_t*)(0x000001d998775450 + 4) = 0xcee2 -> pool entry uninitialized or unloaded
*(uint64_t*)(0x000001d998775450  + 8) = 0x000001d998767b90 -> next

*(uint32_t*)(0x000001d998767b90  + 0) = 0x282b8abd -> sf_int1_fruit
*(uint16_t*)(0x000001d998767b90 + 4) = 0x9bae -> pool entry valid but destroyed
*(uint64_t*)(0x000001d998767b90  + 8) = 0x000001d9987137b0 -> next

*(uint32_t*)(0x000001d9987137b0  + 0) = 0x282b8abd -> sf_int1_fruit
*(uint16_t*)(0x000001d9987137b0 + 4) = 0x95a8 -> pool entry uninitialized or unloaded
*(uint64_t*)(0x000001d9987137b0  + 8) = 0x000001d99873c0b0 -> next

*(uint32_t*)(0x000001d99873c0b0  + 0) = 0x282b8abd -> sf_int1_fruit
*(uint16_t*)(0x000001d99873c0b0 + 4) = 0x95a8 -> pool entry uninitialized or unloaded
*(uint64_t*)(0x000001d99873c0b0  + 8) = 0x000001d99874fff0 -> next

*(uint32_t*)(0x000001d99874fff0  + 0) = 0x282b8abd -> sf_int1_fruit
*(uint16_t*)(0x000001d99874fff0 + 4) = 0x95a8 -> pool entry uninitialized or unloaded
*(uint64_t*)(0x000001d99874fff0  + 8) = 0x000001d99874e400 -> next

*(uint32_t*)(0x000001d99874e400  + 0) = 0x282b8abd -> sf_int1_fruit
*(uint16_t*)(0x000001d99874e400 + 4) = 0xba41 -> pool entry uninitialized or unloaded
*(uint64_t*)(0x000001d99874e400  + 8) = 0x000001d998760a40 -> next

*(uint32_t*)(0x000001d998760a40  + 0) = 0x282b8abd -> sf_int1_fruit
*(uint16_t*)(0x000001d998760a40 + 4) = 0xba41 -> pool entry uninitialized or unloaded
*(uint64_t*)(0x000001d998760a40  + 8) = 0x000001d998765d40 -> next

*(uint32_t*)(0x000001d998765d40  + 0) = 0x282b8abd -> sf_int1_fruit
*(uint16_t*)(0x000001d998765d40 + 4) = 0x9bae -> pool entry valid but destroyed
*(uint64_t*)(0x000001d998765d40  + 8) = 0x000001d9986fae50 -> next

*(uint32_t*)(0x000001d9986fae50  + 0) = 0x282b8abd -> sf_int1_fruit
*(uint16_t*)(0x000001d9986fae50 + 4) = 0x53d9 -> pool entry uninitialized or unloaded
*(uint64_t*)(0x000001d9986fae50  + 8) = 0x000001d9987402d0 -> next

*(uint32_t*)(0x000001d9987402d0  + 0) = 0x282b8abd -> sf_int1_fruit
*(uint16_t*)(0x000001d9987402d0 + 4) = 0x9bae -> pool entry valid but destroyed
*(uint64_t*)(0x000001d9987402d0  + 8) = 0x000001d998759cc0 -> next

*(uint32_t*)(0x000001d998759cc0  + 0) = 0x282b8abd -> sf_int1_fruit
*(uint16_t*)(0x000001d998759cc0 + 4) = 0x935d -> pool entry uninitialized or unloaded
*(uint64_t*)(0x000001d998759cc0  + 8) = 0x000001d998739c00 -> next

*(uint32_t*)(0x000001d998739c00  + 0) = 0x282b8abd -> sf_int1_fruit
*(uint16_t*)(0x000001d998739c00 + 4) = 0x935d-> pool entry uninitialized or unloaded
*(uint64_t*)(0x000001d998739c00  + 8) = 0x000001d998742110 -> next

*(uint32_t*)(0x000001d998742110  + 0) = 0x282b8abd -> sf_int1_fruit
*(uint16_t*)(0x000001d998742110 + 4) = 0x9bae -> pool entry valid but destroyed
*(uint64_t*)(0x000001d998742110  + 8) = 0x000001d998713390 -> next

*(uint32_t*)(0x000001d998713390  + 0) = 0x282b8abd -> sf_int1_fruit
*(uint16_t*)(0x000001d998713390 + 4) = 0xad80 -> pool entry uninitialized or unloaded
*(uint64_t*)(0x000001d998713390  + 8) = 0x000001d998764f60 -> next

*(uint32_t*)(0x000001d998764f60  + 0) = 0x282b8abd -> sf_int1_fruit
*(uint16_t*)(0x000001d998764f60 + 4) = 0xbe93 -> pool entry uninitialized or unloaded
*(uint64_t*)(0x000001d998764f60  + 8) = 0x000001d998753e30 -> next

*(uint32_t*)(0x000001d998753e30  + 0) = 0x282b8abd -> sf_int1_fruit
*(uint16_t*)(0x000001d998753e30 + 4) = 0x935d -> pool entry uninitialized or unloaded
*(uint64_t*)(0x000001d998753e30  + 8) = 0x000001d99873aac0 -> next

*(uint32_t*)(0x000001d99873aac0  + 0) = 0x282b8abd -> sf_int1_fruit
*(uint16_t*)(0x000001d99873aac0 + 4) = 0x9449 -> pool entry uninitialized or unloaded
*(uint64_t*)(0x000001d99873aac0  + 8) = 0x000001d99870abf0 -> next

// First entry in the link of a different hash (due to hash % mapsize collisions). Expected and valid
*(uint32_t*)(0x000001d99870abf0  + 0) = 0xaa094ac3 -> cs2_09_props_combo369_14_lod
*(uint16_t*)(0x000001d99870abf0 + 4) = 0x645c -> pool entry valid for archetype cs2_09_props_combo369_14_lod
*(uint64_t*)(0x000001d99870abf0  + 8) = 0x000001d998705140 -> next

// Second different entry, also valid. Has no more next members. Chain ends here
*(uint32_t*)(0x000001d998705140  + 0) = 0x8d8c9609 -> ch3_03_props_combo48_01_lod
*(uint16_t*)(0x000001d998705140 + 4) = 0x5eb1 -> pool entry valid for archetype ch3_03_props_combo48_01_lod
*(uint64_t*)(0x000001d998705140  + 8) = 0x0000000000000000 -> no more entries


//
// rage::fwArchetypeManager::ms_ArchetypePool
//

*(uint64_t*)(0x00007FF6E5540000 + 0x1E633A0 + 0) = 0x000001d998629620 	-> rage::fwArchetypeManager::ms_ArchetypePool.m_buffer
*(uint32_t*)(0x00007FF6E5540000 + 0x1E633A0 + 8) = 64000 		-> rage::fwArchetypeManager::ms_ArchetypePool.m_count


//This is the pool entry that is returned and crashed the game due to the wrong vtable at 0x0
// 0x18 is fwArchetype::m_nHash
*(uint32_t*)(*(uint64_t*)(0x000001d998629620 + 8 * 0x9bae) + 0x0) = 0x00007ff6e6eef2f8 -> const rage::datBase::`vftable'
*(uint32_t*)(*(uint64_t*)(0x000001d998629620 + 8 * 0x9bae) + 0x18) = 0xe7d7d718 -> ch3_13_props_combo330_34_lod

*(uint32_t*)(*(uint64_t*)(0x000001d998629620 + 8 * 0xcee2) + 0x0) = 0x000001d994d6d040 -> invalid value
*(uint32_t*)(*(uint64_t*)(0x000001d998629620 + 8 * 0xcee2) + 0x18) = 0x94d6d2b0 -> invalid value

*(uint32_t*)(*(uint64_t*)(0x000001d998629620 + 8 * 0x95a8) + 0x0) = 0x000001d994d6d040 -> invalid value
*(uint32_t*)(*(uint64_t*)(0x000001d998629620 + 8 * 0x95a8) + 0x18) = 0x94d6d2b0 -> invalid value

*(uint32_t*)(*(uint64_t*)(0x000001d998629620 + 8 * 0xba41) + 0x0) = 0x0000ffff01e0007f -> invalid value
*(uint32_t*)(*(uint64_t*)(0x000001d998629620 + 8 * 0xba41) + 0x18) = 0x00000000 -> invalid value

*(uint32_t*)(*(uint64_t*)(0x000001d998629620 + 8 * 0x53d9) + 0x0) = 0x0000000000000000 -> invalid value
*(uint32_t*)(*(uint64_t*)(0x000001d998629620 + 8 * 0x53d9) + 0x18) = 0x00000000 -> invalid value

*(uint32_t*)(*(uint64_t*)(0x000001d998629620 + 8 * 0xad80) + 0x0) = 0x0000000000000000 -> invalid value
*(uint32_t*)(*(uint64_t*)(0x000001d998629620 + 8 * 0xad80) + 0x18) = 0x425adbfa -> invalid value

*(uint32_t*)(*(uint64_t*)(0x000001d998629620 + 8 * 0xbe93) + 0x0) = 0x3f2e50a2bf3a45a9 -> invalid value
*(uint32_t*)(*(uint64_t*)(0x000001d998629620 + 8 * 0xbe93) + 0x18) = 0x4013d7f2 -> invalid value

*(uint32_t*)(*(uint64_t*)(0x000001d998629620 + 8 * 0x935d) + 0x0) = 0x000001d995bf07e0 -> invalid value
*(uint32_t*)(*(uint64_t*)(0x000001d998629620 + 8 * 0x935d) + 0x18) = 0x95bf0620 -> invalid value

*(uint32_t*)(*(uint64_t*)(0x000001d998629620 + 8 * 0x9449) + 0x0) = 0x000001d998673858 -> invalid value
*(uint32_t*)(*(uint64_t*)(0x000001d998629620 + 8 * 0x9449) + 0x18) = 0x9866d308 -> invalid value

*(uint32_t*)(*(uint64_t*)(0x000001d998629620 + 8 * 0x645c) + 0x0) = 0x00007ff6e6f5f380 -> const CBaseModelInfo::`vftable'
*(uint32_t*)(*(uint64_t*)(0x000001d998629620 + 8 * 0x645c) + 0x18) = 0xaa094ac3 -> cs2_09_props_combo369_14_lod

*(uint32_t*)(*(uint64_t*)(0x000001d998629620 + 8 * 0x5eb1) + 0x0) = 0x00007ff6e6f5f380 -> const CBaseModelInfo::`vftable'
*(uint32_t*)(*(uint64_t*)(0x000001d998629620 + 8 * 0x5eb1) + 0x18) = 0x8d8c9609 -> ch3_03_props_combo48_01_lod
1 Like
6

Looks like you identified the right object 0x282b8abd/sf_int1_fruit
This resource is re-creating CBaseArchetypeDef multiple times in 2-3 ytyps for objects like sf_mp_h_yacht_side_table_01 sf_int1_fruit sf_mpapyacht_st_011 sf_int1_office2a_sideboard2 sf_prop_sf_art_box_cig_01a :face_with_spiral_eyes:

Removing these extra declarations fixed it

7

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.