[C#] Native crash in citizen-scripting-core.dll

Crash dumps available here

  1. FXServer version:
    “FXServer-master SERVER v1.0.0.5489 win32” - flags( )

  2. What did you expect to happen
    The server should handle the exception internally

  3. What actually happens
    The server crashes

  4. Category of bug (eg. client, server, weapons, peds, native)
    Server

  5. Reproducible steps, preferably with example script(s)
    Unknown

2 Likes

5489 left supported state a month ago (May 2nd, 2022). Does this still happen on more recent builds?

This crash seems to be caused by invoking a game function (an export, maybe? haven’t been able to look at the managed call stack) from a background thread, such as an await function without using Delay to marshal back to the main thread.

That behavior is inherently unsafe, and in this case it crashed since this apparently didn’t end up holding the runtime mutex, which might be easier to fix.

2 Likes

Managed to read the managed stack trace using a WinDbg script I just wrote:

NATIVE: citizen_scripting_core!std::optional<std::vector<unsigned char,std::allocator<unsigned char> > >::operator= + 0x23
NATIVE: citizen_scripting_core!fx::TestScriptHost::SubmitBoundaryEnd + 0xde
MANAGED: System.Object:submitBoundaryEndMethod
MANAGED: .DirectScriptHost:SubmitBoundaryInternal
MANAGED: .DirectScriptHost:SubmitBoundaryEnd
MANAGED: CitizenFX.Core.BaseScript:TriggerEventInternal
MANAGED: CitizenFX.Core.BaseScript:TriggerEvent
MANAGED: Gtacnr.Server.Core.Authentication.AuthenticationController:SetPlayerAccountId
MANAGED: Gtacnr.Server.Core.Authentication.AuthenticationController:SetAccountId

TriggerEvent indeed shouldn’t be called from off-thread.

Diagnostics for this also definitely seem lacking especially as it doesn’t consistently crash either, there’s three things that could be done here:

  1. This definitely should warn and show a managed stack trace, at least the first time around.
  2. Submitting boundary should be skipped for these so it won’t race.
  3. Perhaps(?) these should get mapped to a queued event so existing code at least will be fixed (assuming any user code doesn’t expect the event to get executed immediately).
The script for future reference
function invokeScript()
{
    try {
    const { currentThread, currentProcess } = host;
    const stack = currentThread.Stack.Frames;
    let monoDomain = null;
    for (const frame of stack) {
        if (frame.toString().includes('mono_2_0_sgen!mono_jit_runtime_invoke')) {
            monoDomain = frame.LocalVariables.domain;
        }
    }

    if (!monoDomain) {
        return;
    }

    const table = monoDomain.jit_info_table;
    for (const frame of stack) {
        const addr = frame.Attributes.InstructionOffset;
        const ji = jit_info_table_find(table, addr);

        if (ji) {
            function s(a) {
                return host.memory.readString(a);
            }

            host.diagnostics.debugLog(`MANAGED: ${s(ji.d.method.klass.name_space)}.${s(ji.d.method.klass.name)}:${s(ji.d.method.name)}\n`);
        } else {
            host.diagnostics.debugLog(`NATIVE: ${frame}\n`);
        }
    }
    } catch (e) {
        host.diagnostics.debugLog(`ee? ${e.stack}\n`);
    }

}

function jit_info_table_find(table, addr) {
    let chunk_pos = jit_info_table_index(table, addr);
    let pos = jit_info_table_chunk_index(table.chunks[chunk_pos], addr);
    let ji = null;

    /* We now have a position that's very close to that of the
    first element whose end address is higher than the one
    we're looking for.  If we don't have the exact position,
    then we have a position below that one, so we'll just
    search upward until we find our element. */
    do {
        const chunk = table.chunks[chunk_pos];

        while (pos < chunk.num_elements) {
            ji = chunk.data [pos];

            ++pos;

            /*if (IS_JIT_INFO_TOMBSTONE (ji)) {
                mono_hazard_pointer_clear (hp, JIT_INFO_HAZARD_INDEX);
                continue;
            }*/
            if (addr.compareTo(ji.code_start.address) >= 0
                    && addr.compareTo(ji.code_start.address.add(ji.code_size)) < 0) {
                return ji;
            }

            /* If we find a non-tombstone element which is already
            beyond what we're looking for, we have to end the
            search. */
            if (addr.compareTo(ji.code_start.address) < 0) {
                return null;
            }
        }

        ++chunk_pos;
        pos = 0;
    } while (chunk_pos < table.num_chunks);

    return null;
}

function jit_info_table_chunk_index (chunk, addr)
{
	let left = 0, right = chunk.num_elements;

	while (left < right) {
		const pos = ((left + right) / 2) | 0;
		const ji = chunk.data [pos];
		const code_end = ji.code_start.address.add(ji.code_size);

		if (addr.compareTo(code_end) < 0)
			right = pos;
		else
			left = pos + 1;
	}

	return left;
}

function jit_info_table_index (table, addr)
{
	let left = 0, right = table.num_chunks;

	//g_assert (left < right);

	do {
		const pos = ((left + right) / 2) | 0;
		const chunk = table.chunks [pos];
		if (addr.compareTo(chunk.last_code_end.address) < 0)
			right = pos;
		else
			left = pos + 1;
	} while (left < right);
	//g_assert (left == right);

	if (left >= table.num_chunks)
		return table.num_chunks - 1;
	return left;
}
6 Likes

Ok, thank you. I’ve upgraded to the latest recommended, and I’m gonna thoroughly check my code for cases where I call a native (I don’t remember if I used any exports) without marshaling to the main thread. However, is there any way to have the FiveM code recognize where in my code I’m making that mistake? It would be really helpful

5 Likes

Ok, this definitely helped a lot, many thanks!

2 Likes