[Tool] Stream file assistant

FiveM Releases
,
1

Introduction

In the past, missing medium LODs often caused crashes, forcing many creators to manually duplicate files and append _hi to make things work.

Big thanks to packfile (fix for crashes when no medium LOD exists), this workaround is no longer necessary, saving significant file space.

This tool, using Python and packaged to exe, is designed to clean up those extra, now unnecessary files, helping you optimize your stream resources. :smile:

Before You Proceed

Always back up everything you plan to modify! Itā€™s a good habit to avoid any unintended issues.

Features

1. Duplicate YFT Cleaner

  • Specifically designed to clean up duplicate-pasted files.

  • Scans for and identifies _hi YFT files that were created as a workaround for missing medium LODs.

  • Provides tools to select, review, and delete unnecessary duplicates.

  • What is size margin?

    • Allows you to define a margin (in KB) for file size comparison when _hi files differ slightly from their counterparts.

2. Stream Duplicate Checker

  • Checks all stream directories for duplicate files, regardless of extension.

  • Allows users to quickly locate files and their duplicate directories via right-click context menu.

  • Simplifies the process of managing large resource libraries.

Screenshots

Duplicate YFT Cleaner

äø‹č¼‰
äø‹č¼‰1146Ɨ827 37.6 KB

Identify and manage duplicate YFT files with ease.

Stream Duplicate Checker

pic_02
pic_021145Ɨ827 18.8 KB

Locate and resolve file collisions in stream directories.

Contributions

Welcome contributions to improve this tool! Feel free to fork the repository and submit a pull request.

Acknowledgments

This tool was inspired by and references the FiveM project.

Release

21 Likes
2

What a wonderful tool. Thank you for the free release.

1 Like
3

Very nice tool, thanks :pray:

1 Like
4

Insane tool, nice work!
Didnt even know about the fix.

Only problem i see is: When you export a vehicle (for example) via ZModeler and you change the name of the vehicle for the _hi.yft inside zmodeler, then the file content is not 1:1 identical.
A lot of vehicles are using basically the same model for model.yft and model_hi.yft, but changed eg the model name or something inside zmodeler for the _hi.yft. Now the files are not identical anymore in size or content. Cant think of a way to fix that right now, maybe a small configurable ā€œsize marginā€ or something. Mostly its just a very very small difference in size
eg:

8,92 MB (9.357.750 Bytes) for model.yft
8,92 MB (9.356.697 Bytes) for model_hi.yft

But this small difference is enough to ā€œbypassā€ this code

        # Check if file contents are identical (hash) -> Line 380 of StreamFileAssistant.py
        hi_hash = self.compute_file_hash(hi_file)
        org_hash = self.compute_file_hash(original_file)
        if not hi_hash or not org_hash or hi_hash != org_hash:
            return None

maybe im completely wrong but for the couple of models ive tested, it didnt recognize them. This seemed to me like the logical reason for it :smiley:

1 Like
5

Thank you for your feedback and for pointing this out! :smile:

You are absolutely correct that small differences in file size(e.g., due to changes in some editing tools) can cause the hash comparison to fail. To address this, Iā€™ve added a new size margin feature in the latest update: Commit a7304f7. :tada:

Whatā€™s New:

  • Size Margin Option:

    • You can now configure a size margin (in KB) when scanning _hi.yft files.
    • This allow the tool to treat files with small size differences as identical.
    • For example, in your case:
      8,92 MB (9,357,750 Bytes) for model.yft
8,92 MB (9,356,697 Bytes) for model_hi.yft
      If you set the size margin to 2 KB, this will now be recognized as a duplicate.

  • Detailed Byte Difference:

    • If size margin is applied, the tool will display the exact byte difference in the results, so you can verify which files are close matches.

Thank you again for highlighting this use case!
Feel free to test it out and let me know if you encounter any other issues.

2 Likes
6

Hey awesome! Nice work!
I will check it out later
Cheers

7

Hey everyone, I have a question thatā€™s been bothering me. I used Stream Duplicate Checker, and it found a lot of duplicatesā€”things like .ymap, .ytd, and other files with the same name but in different resources.

If I understand correctly, I need to remove one of the duplicates. However, Iā€™m worried that if I remove, for example, a .ymap file from one MLO thatā€™s in a different location than another, it could cause issuesā€”like props or parts of the MLO not showing up anymore.

Does anyone have any advice or suggestions on how to handle this? Iā€™d really appreciate the help!

8

IĀ“m wondering the same thing.

9

you snappedšŸ”„