Using canary? Multiple, both Yes and No Windows version: 19043.1110, Multiple System specifications: Unknown, Multiple
Server
Operating system: Windows Server 2019 Artifact version: 4304 IP address: 72.5.195.226 System specifications: Xeon E3-1270, 32GB DDR3 RAM
Incident
Summary:
Multiple players - from various geographic regions on both mainline and canary - are experiencing 166074F - california-white-muppet - crashes. Based on analyzing the dumps provided by users, it appears to be caused by the SetVehicleDoorsLocked RPC native.
Steps to reproduce:
We are unable to reproduce this in smaller scale local/staging tests, it appears with 30-50 active players, performing an unknown number of vehicle locks/unlocks which use the SetVehicleDoorsLocked RPC native.
Dump Files:
These are all full dumps ranging from 10-20 GB each.
Right - I suspect an easy fix would be to wrap the native invocations from ClientRPC in a SEH wrapper just like ScRT invocations get, as game natives may crash/bail out for a variety of reasons.
The underlying issue may be more or less nefarious, but a good start would be to SEHify this anyway.
@nta Unfortunately this crash still seems to be appearing on canary client version 4377. A full crash dump is below. Let me know if you need more of them.
Hopefully this is some random race condition somewhere and sanitizing object->GetGameObject() in ObjectToEntity would be sufficient. The first deference of the GetGameObject pointer would happen at +0x18 (rage::fwExtensionList::Get), which correlates to this dump and stack trace on github.
I can look into this, I suppose.
(Worst case scenario, SEH the entire native execution scope rather than just the call handler?)
Changing the conditional in ObjectToEntity to if (object && object->GetGameObject()) is probably sufficient given how ‘conditional’ callbacks work.
However, if ClientRPC is to be expanded upon, should we consider improving the failure logic in fivem/ClientRPC.cpp at master · citizenfx/fivem · GitHub ? For example, after X units (seconds?) of failed conditions, send a failure message/error-code to the server that can be logged, re-broadcasted, etc.