Hello, For over 3 months now, I have been having problems with network thread hitch warning (around 160-240 milliseconds), usually when I reach 450+ connected players. I cannot say for sure that 100% of players are disconnected because of this, but over 90% are disconnected in these events, which are usually daily, typically at 10 PM when I reach the peak of connected players. I am using the txAdmin from FXServer itself, but even with it deactivated, I still have this problem.
What’s happening? The network thread starts to hitch and over 90% of players lose their connection to the server. Usually, after the players are disconnected, the network thread hitch warning stops occurring and starts happening again when the peak is reached, but there are days when I reach 560+ simultaneous players and this still doesn’t happen.
What am I running?
- Roleplay server with the vRP framework (however, much of it has been rewritten and optimized)
- 65 scripts (not counting assets such as maps, vehicles, and clothing)
- Approximately 78 addon vehicles
- Mumble through the pma-voice resource (currently with external mumble, but the error occurred even with mumble integrated on the machine)
What have I tried?
- We changed VPS, network adapter, and nowadays we are using Windows Server 2019.
- We checked the codes looking for any leaks, but without success.
- Updated oxmysql to the latest version.
- Checked using resmon (server), but it didn’t help much.
- Tried using UIForETW, but perhaps due to lack of knowledge, FXServer did not appear in Generic Events (I have a comment about this, you can go to it by clicking here)
Almost all of my database tables have primary key and index, to further optimize queries, and I query most of the database only on player connection and disconnection, controlling most of the others through tables and objects in LUA.
I would be happy to receive help, if you have any questions, please don’t hesitate to ask down below.
7 Likes
a small hitch warning just happened, below is the .etw
2023-04-20_21-39-45_Administrator.etl (56.2 MB)
2 Likes
Another .etl uploaded, this time the drop was much bigger.
2023-04-24_20-52-37_Administrator.etl (68.4 MB)
2 Likes
We experience network thread hitches at around 150 players sometimes. I could not identify the issue and after restarting it works fine. It randomly occurres.
There is one thing i can reproduce like 90% of the time: We have a “Factory”-Script (self-made) which syncs big data between the server and the client on connect. Works fine if 1 player requests the data and gets the data. But if i run a “full sync” with “-1” (all players) i get network thread hitches and a crash all the time if the player count is above 100.
The payload is kinda big and is sent as a LatentEvent.
If you’r interested in our specs, maybe its important:
- i9 10900KF
- 128 GB RAM (100 GB allocated for the VM, Host-System is running proxmox, linux)
- Windows Server 2022
- 2x1 Gbit/s Uplink/Downlink
- NVME SSD 200 GB allocated
2 Likes
We’re having the same issue, usually with 80-110 players. I have been trying to fix this for a month. I have gone through every single resource we have and I have not found anything that’s triggering huge events to the clients. I have done profilers during before and after network thread hitch shows up in the console and all the players drop. I’ve tried UIforETW, but that doesn’t log event names any more for some reason. I’ve done wireshark packet captures during the thread hitches, but I have no idea how to analyze it. Wish we could get some help on AT LEAST how to debug the problem a little more info than “just use profiler” would help because profiler only captures svMain, what I want is info about svNetwork and svSync.
How did you get UIforETW to capture the fivem server properly? Because whenever I do it, it doesn’t show the list of events when I go to analyze it
1 Like
We also experience 90% player network timeout between 10pm-1030pm almost daily. Everyone reconnects and then its usually fine. Its very strange. Have not been able to figure it out. I honestly was wondering if its something on FiveM’s network like routine maintenance / onesync related.
2 Likes
Ours do not happen at the same time every day, it happens at varying times. That might be a thing you can look into. Does your VPS/ server provider offer automatic backups that run at a certain time every day? Also, are you using a proxy behind a DDoS Protection service at all?
2 Likes
What resources are you using ?
18 Likes
I am the same as you at 180 players at 10am - 11pm daily, the server is not completely down and no ddos, resmon from my player side in the server is just 0, 5 - 0.7. I paid a large amount of money to rewrite the platform from QBcore to ESX and change the vds server but the result is still the same
6 Likes
where and how to get whitelisted role?cannot join without it…
2 Likes
my problem cannot join server gta 5 roleplay
1 Like
network problem connetion help
1 Like
Are you sure your scripts are optimized?
Normally hitch warning are a sign that some resources taking too long to load.
2 Likes
kenapa asal nak masuk mesti ada ni eh?
1 Like
why connection failed ? please help
1 Like
The solution was to host the voice system on a dedicated one, through pma.
4 Likes
boleh tolong saya… sebab sblom nie discord sy kena dengan hacker. sekarang dah ok tp tak boleh masuk discord abrp
1 Like
hi nak minta tolong saya dari tadi tak boleh masuk dari semalam saya baru download fivem baru nak try masuk server
