Desync Issue with 50 Slots Server

I am having a lot of issues running a fivem server with more than 32 slots using Onesync. My current VPS specs are

4 Core Intel Xeon with 2.3 GHZ
8 GB Ram
100 GB SSD Storage
2 GBS Internet Connection

I had tried both Google Cloud and Microsoft Azure, but seems like maybe the specs are not enough. I have seen people running 64 slots server smoothly… don’t know where I am doing wrong.

My player base doesn’t have high ping max 100 and on average between 30-60. But just the day I switched to Onesync and with only 50 slots it started getting worst. People get very low FPS, some get texture loss, others can’t even drive, and occasionally for players Fivem crash with timeout or some odd errors.

Just curious to know what Kind of Hosting would be best to solve all kinds of Desync issues. Obviously just for 64 slots or maybe even less just 50 would also do the work.

Any kind of guidance would be very helpful, I am ready to invest into an 8 Core VPS, but only if the issue is about it. Because the CPU Usage and overall memory usage never went too high for me to think it is a Low End VPS Issue.