[C#] Event handler bottleneck

  1. FXServer version
    FXServer-master SERVER v1.0.0.5181 win32
    Running on Windows Server 2019 [10.0.17763.2300], running on Intel® Xeon® CPU E3-1270 v6 @ 3.80GHz

  2. What you expected to happen
    The platform for handling server events should be efficient.

  3. What actually happens
    The platform isn’t efficient and it kills server performance.

  4. Category of bug (eg. client, server, weapons, peds, native)
    Platform / C#

  5. Reproducible steps, preferably with example script(s)

Add this code to a server with a substantial amount of players, then run the command.

    public class PlatformBugTestScript : BaseScript
    {
        [Command("new-testing-command", Restricted = true)]
        private void TestingCommand([FromSource] Player source, string[] args)
        {
            Debug.WriteLine("Test handler added.");
            EventHandlers["entityCreating"] += new Action<int>(TestingTask);
        }

        [Command("new-testing-command-2", Restricted = true)]
        private void Testing2Command([FromSource] Player source, string[] args)
        {
            Debug.WriteLine("Test handler removed.");
            EventHandlers["entityCreating"] -= new Action<int>(TestingTask);
        }

        private void TestingTask(int param)
        {
            // Nothing at all here
        }
    }

I’ve chosen the entityCreating event because it’s constantly run thousands of times by players. The video below uses the code above to demonstrate that the platform behind event handlers is not performing well.

The screenshot below shows the CPU usage of the FxServer running C# scripts containing lots of event handlers. CPU details can be found above.

I bet that solving this problem in the platform will remove all those “server thread hitch warnings” that my C#-based server is suffering from, and I am 100% willing to offer my help in creating a solution because it’s mainly in my interest and probably no other big C#-based servers exist.

More hints that point to the problem being in the platform code:

Usage of DynamicInvoke

Usage of Reflection (probably the main issue)

As for the DynamicInvoke, I had posted a link to an article that had a better solution but I realized the author took that code down and didn’t reply to an email I’ve sent him. However, what about wrapping the delegate in an Action<Tn> or Func<Tn, Tr>? This would allow us to use Invoke() instead of DynamicInvoke().

As for the reflection problem, a good solution would be to have a Dictionary<MethodInfo, ParameterInfo> cache it so that we don’t have it use reflection (a major performance killer) so much.

You’re a stubborn one, aren’t you? As said many times before: confirm your assumptions before starting to blame anything or anyone. You have provided no additional info compared to the last times you’ve complained about this, nor did you test if your proposed fixes even help (which is also puzzling).

Run with the Mono log profiler, an ETW trace, or provide an actual live repro (e.g. an actual event to call “a lot” or whatever). Your assumptions are most likely wrong, especially since it supposedly only replicates ‘with a substantial amount of players’ so it probably isn’t related to invocation logic at all.

Since you’re apparently running on Windows, running and uploading an ETW trace would be a good start before trying to deduce this any further (using e.g. the Mono log profiler).

Closing this topic since it doesn’t belong in ‘bug reports’ as it’s not actionable nor reproducible. If you want to discuss this any further, first off stop trying to assign blame and actually confirm your assumptions and/or provide actual info, and secondly go and post in a normal support section like #development.

Just double checked some thoughts and I rather think it’s the use of remoting for app domain transitions (as Mono on Linux didn’t like the same trick used on client) that’s slow. Not any of your assumptions.