RegisterStreamingFileFromUrl not caching entries properly

FiveM Discussion Bug reports
1
  1. Client (production/canary) and FXServer version
    Client=Canary, FXServer= 3539
  2. What you expected to happen
    Each registered asset gets cached based on the URL.
  3. What actually happens
    Only a single asset, ResourceCacheEntryList::Entry, can be cached, each consecutive RequestStreamingFileFromUrl request replaces the existing entry one with the new one.
    When actually streaming, loading from a ymap or using RegisterArchetypes(…) followed by RequestModel(…), the previous asset entries will be using the latest entry. This causes issues when, for example, first requesting a .ydr file followed by a request for a .ytd because now the drawable will be using the texture dictionary data resulting in a crash.
    image
    image990×27 23.1 KB
  4. Category of bug (eg. client, server, weapons, peds, native)
    native
  5. Reproducible steps, preferably with example script(s)
    For this code to work you would also need the backend providing the files, can also post this if needed (multiple different file names / urls were used to make sure that there wasn’t any accidental hash collisions)
RegisterCommand("downloadEmpty", function(source, args, rawCommand)
    
    RegisterStreamingFileFromUrl("testobject02.ydr", "http://localhost:3000?hellomaybeornot=somerandomvalue");
    while not IsStreamingFileReady("testobject02.ydr") do 
        Wait(100)
    end
    RegisterStreamingFileFromUrl("makingsurenameisnothashedandcollided.ytd", "http://localhost:3000/ytd?thisisatest6542");
    while not IsStreamingFileReady("makingsurenameisnothashedandcollided.ytd") do 
        Wait(100)
    end
end, false)

I assume this bug might be because of the referenceHash not being filled in by the ResourceCacheDevice. fivem/RuntimeAssetNatives.cpp at master · citizenfx/fivem · GitHub
Probably because of the change to ResourceCacheDeviceV2, the old version of ResourceCacheDevice did check when “referenceHash” was empty and used the “remoteUrl” instead.
Proof that V2 is being used: image

Old version: fivem/ResourceCacheDevice.cpp at master · citizenfx/fivem · GitHub
New version V2: fivem/ResourceCacheDeviceV2.cpp at master · citizenfx/fivem · GitHub

2

I believe support for this native was intentionally skipped as there were some inherent design flaws like the header pre-download making it a bit weird and the fact that you needed to do explicit cache-busting in URLs to make any renamed asset work.

That, and literally nobody was using it at the time so demand for a function like this with its flaws evidently was extremely low.

3

So there is no chance that this will ever get fixed? Because it does work perfectly for a single asset and I think some cool stuff might be possible. The cache-busting is a little bit annoying but it’s not that difficult to work around it client side using scripts.

4

Might get reimplemented at some point with explicitly specifying a content hash or so, instead.

5

If I change the referenceHash to be equal to remoteUrl in RequestStreamingFileFromUrl and make a pull request and proof that it works, would you merge it or not?

6

Just registering my support for this - it’s also an issue I’ve come across and it’d open up a lot of interesting possibilities if this was fixed.

7

This PR fixes the issue Fix runtime streaming native issue with referenceHash by PichotM · Pull Request #1751 · citizenfx/fivem · GitHub