Server specs are just fine and are not an issue.
Please use the profiler tool to diagnose your server issues. I’m positive it’s one of your scripts that are unoptimized that are causing the problems.
And if you’re using vRP or ESX, then I’m 100% positive that it’s unoptimized scripts causing the problem.