What exactly happens when the crash occurs? What displays on the server console when it happens?
There are very few reasons it crashes on Linux in general so it’s interesting you’re having such trouble with it. Unfortunately due to how FiveM runs in linux there are no crash dumps or things like procdump like the windows version.
Even if it’s literally a SIGSEV in your console, post it.
The red auto reconnect message comes up. All players disconnects. 10% rejoin.
The server is started at VPS boot, the ocnsole is down, i do not know how to start at boot from script in a window. I will try to run it with console up and in a window. I cannot see the console in this manner, and do not know what it says.
This is now a serious short coming for FiveM. The uncrashable server now crashes readily. A server log system should be produced. A server dump system should be produced. The reasons they cannot should be overcome by whatever means possible, even unconventional means like a secondary watcher process should be considered.
I will do that as soon as I can.
Thank you for your productive answer, in a sea of accusations. You clearly read everything about this issue and are well-informed.
Linux Server 1337 Crashed at least 1 time last night. It had a definite uptime of over 4 hours and under 12 hours, quite similar to the three post-1317 previous versions. On to 1338, in a window this time…
It took three crashes on 1338, but i finally logged it. Below is the PuTTY log from the last regular join to the First resource load of the new restarted server. The server auto restarts after a crash, with a script. Everything above and below this point looks quite normal.
...
Sending heartbeat to live-internal.fivem.net:30110
Connecting: D a n i e l Key-Drop.pl
Sending heartbeat to live-internal.fivem.net:30110
server thread hitch warning: timer interval of 274 milliseconds
network thread hitch warning: timer interval of 174 milliseconds
network thread hitch warning: timer interval of 157 milliseconds
server thread hitch warning: timer interval of 151 milliseconds
network thread hitch warning: timer interval of 325 milliseconds
network thread hitch warning: timer interval of 152 milliseconds
network thread hitch warning: timer interval of 152 milliseconds
network thread hitch warning: timer interval of 280 milliseconds
network thread hitch warning: timer interval of 189 milliseconds
server thread hitch warning: timer interval of 152 milliseconds
server thread hitch warning: timer interval of 573 milliseconds
/home/_fivem/fx-server-data/Server-Starter-fivem.sh: line 1: 2991 Killed /home/_fivem/fx-server/run.sh +exec server.cfg +set con_disableNonTTYReads 1
FiveM server has stopped.
restarting FiveM server in 5 seconds
Starting 5M Server
Creating script environments for _cfx_internal
Found new resource...
This is the above mentioned Server-Starter-fivem.sh:
So this is telling me some external process is killing your server. I was expecting a stack dump at the very least - this is pointing at something else.
Check syslog and dmesg for anything interesting. It could be any number of things, like cron killing off your server, the server itself running out of resources, etc.
A script which works faultlessly on all the above listed mps for 5 years. One that has worked well on FiveM for several years, until 1141 when it was upgraded by a FiveM element. No harm in trying FiveM without it… thanks for your help.
I will upgrade the server to 1347 any minute now, and try a normal start without the script, and see over the next 48 hours if it continues to crash.
UPDATE: 1347 is running via the common method:
bash /home/_fivem/fx-server/run.sh +exec server.cfg
Following the rebuild to 1347 i started the server with no restarting script via the normal method: bash /home/_fivem/fx-server/run.sh +exec server.cfg
several minutes elapse here, then the red disconnect message appears at the same time on two different clients, and i presume on the clients of those playing.
i close PuTTY and relogin to check top, and the fivem server process is no where to be seen in top over several minutes. (See attached PuTTY log)
top displays system processes at 6% with .7mb of ram usage without FiveM. The last top at the base of this file is my second run without the restart script. With FiveM server running, at roughly the same time as the previous crash, showing CPU at 6% and ram at 12mb which is typical for an empty server by my observations.
1347 Crashed in an uprecident short time. Under 30 minutes with 4 players on it, previous versions have given me 4-8 hours and only crash when the population exceeds 10. Maybe just bad luck, trying it a second time to see.
But what can be assumed at this point, is that the server restarter is not to blame. A once stable FiveM server, package, and startup system now crash frequently. I have made no changes to this system since January, apart from the upgrade to the restart script as advised by a FiveM element, which works perfectly.
UPDATE: the attached…
putty_1348-1.log (180.5 KB)
I’m unfortunately not sure what to tell you since I don’t run the Linux version of FiveM. I do know that it’s not as stable and that error reporting is very …well, not there.
At this point all I can recommend is attempting the same on a different VPS. Everything seems normal based on the logs you sent (aside from a 2 year old kernel, but not the main issue here).
The only reason I can’t blame FXServer itself is because many others run on Linux with no problems at all.
Not to be repeating myself…
The server is running without the restart script. Crashed terminally without restarting. The second time this experiment has been run. Twice now FiveM 1347 has crashed without the Starter Script being active.
So what I am seeing, is that the server during a calm period where only heartbeats are registering, after a lengthy hitch session says “Killed” which the first time seems to have meant nothing. The server continues to send heatbeats, but now looks like it is failing to connect, followed by a second hitch session, and a second “Killed” which is the last thing in the log.
putty_1348-2.log (18.2 KB)
1347 not 1348^^ can’t change it 
Which raises the question of linux server logging. If it can say heartbeat, and it can say Killed, why can’t it say a lot more things? If it can’t log, why can’t it create a dump file?
Your dmesg is literally full of the server process (and other processes, such as apache2) getting killed due to being “out of memory”. It seems your server has only 512 MB of RAM, and you’re running multiple intensive server processes? This was a common amount of memory for a PC in 2003, not in 2019. Due to all the background processes going on, the server is getting killed when its memory usage is merely 128 MB, which is a perfectly normal amount of memory to be used by a game server.
Try shutting down stuff like apache2 (which takes around 70% of your server’s memory), exim4, and other things you don’t need, or get more RAM.
Even if this’d be a normal crash (which it isn’t - it’s the OS forcibly terminating the server because you’re running with only 512 MB of memory with random other processes using 70% of it), making a crash dump file on Linux would be completely futile since there’s no software that can open it due to the way Linux works with dynamic linking.
While running FiveM Server and an apache2 stress test, CPU 23%, and 323MB RAM free. Apache2 under test appears to not be occupying more than 20% of ram. I additionally have 512mb of swap which goes unused 24/7 for 3 years.
free -m appears to confirm this
The only tasks running on this server is 1 website @ 250 users a day, and 1 usually half full FiveM server. The website operates perfectly, I’ve never once seen a download or page load fail. FiveM until 1317 ran perfectly. I never once saw it crash. The VPS is largely unchanged for three years.
Presumably, your server is configured to just kill processes before using it. Linux memory management is a weird one, it’ll OOM kill long before it indicates anything as being wrong.
Of interest is that 1347 seems to fail following the hitch warning sessions. I have an identically prepared VPS that I dev FiveM on, that is currently idling the same setup, on 1244, waiting for FiveM to stop blocking the masterlisting of it. And in its log i see the same behavior of the FiveM server/package a large hitch session, perhaps of my own causing. But unlike post-1317 it does not crash as a result of this hitch session, which is currently occurring with no players on the server, none ever joined:
961-1244 hitches and survives, 1317-1347 hitches and crashes every time.
My VPS is a very stock Debian setup, as advised by the great tutorials at Digital Ocean. Usually anything said following the word Linux takes me several hours to understand and implement. There is no fancy configuration here. What ever Debian does stock so does mine.
ok, so your VPS also has very much inconsistent CPU scheduling so it’s definitely not suited for running a game server.
Checking server changes between 1244 and 1313 doesn’t show much that could potentially cause this kind of issue - does it get worse if you e.g. target your server’s port using a HTTP stress test? If so, that’s one potential point that this could’ve changed. 
I have run on three identical VPS for five years: MTA, iv:mp, jc3mp, co-op, fivemp, gta:mp, ■■■■■■■, just to name a few. At my peak I ran on 1 indentical server MTA 20+ players, ■■■■■■ 20+ players, and iv:mp on a single VPS with 4 websites at 500 users a day, and never had a single problem for over 1 year, and that was with docker running ■■■■■■.
