How the FO2 engine finds it's data. [Red]

[Temaperacl]

Some interesting information posted By Red here.

I've spent the better part of my free time for the last 3 days investigating how the game finds/loads it's data. This was at first for speed considerations (see the thread about slow fallout), but there's a heck of a lot of useful information for the modders which I've found and would like to share.

First, the "root data" concept: The engine accepts DATs as sources of data as we all know, and also accepts the main directory as most modders know, not precisly how though. I've noticed in various threads that some have even found this out on their own, but I don't know to what extent. The important thing to know is that the engine supports information to be either from a DAT file or from a directory. The engine doesn't care if it's a DAT or a directory, it supports both.

Now the reason I'm talking about "root data" is because the engine uses specific root directories (or DATs...) to find it's data. Here's the order:

1. patchXXX.dat
2. critter_patch
3. critter_dat
4. master_patch
5. master_dat
6. Current directory
7. If it fails to find the data it will try to continue without it as gracefully as it can.

Notice how I used _dat instead of .dat. That's to emphasise once more that it can be a directory too. These are in fact the values used in the Fallout2.cfg file as opposed to the "root data" file they point to. It's to be noted that if the file/directory doesn't exist, the engine will create it. Lastly, if two "root data" paths match, the engine won't try to look for the file in the same directory twice.

Special notes:
- patchXXX.dat: is identified as such because the engine is actually looking (in the current directory, not the Fallout directory) for patch000.dat through patch998.dat in 2's incriments. If it finds a file or directory of that name it will use it, otherwise the engine will ingore it entirely (ie: it won't try to create a patch000.dat directory and thus the engine won't try to find data in the empty directory)
- critter_patch: under normal circumstances, this points to "data" if you didn't toy with your CFG since the installation. This is why current mods work given you supply the files manually.
- critter_dat: Under normal circumstances, points to critter.dat (can be on the CD depending on the installation selection).
- master_patch: This is very important for modders. Under normal circumstances, this points to data (as critters_patch). Now this isn't such a big deal, however the game deletes all of master_patch\proto\items\*.pro and master_patch\proto\critters\*.pro. Since it normally points to "data" and this is where most modders will want to put their PRO files so they get loaded by the engine (by the critter_patch reference), this can be a problem. The most commonly used solution is simply to put the files read-only (select all files, right click, properties, select "read only", click OK and wait). An elegant solution was suggested by Haenlomal on this very forum is to put your files in the patch000.dat directory, and finally mine would be to point master_patch somwhere with no PRO files, ideally an empty, easy to figure out directory. [Extra note: the game actually builds the list of files to delete from all "roots", not just the master_patch directory, so you can't fool the game into not deleting files by pointing one of the roots to an empty directory - it even tries to delete master.dat/proto/items/XXXXXXXX.PRO (which thankfully fails) if all "roots" are DATs!]
- master_dat: Nothing much to say about this one see critter_dat.
- Current directory: First, this isn't the Fallout2 directory, but the directory in which you run the game. Under normal circustances they are the same. For those who don't quite understand, click on the Fallout2 icon, right click and "create a chorcut". Then right click on the freshly created shortcut and select "Properties", you should be on the "Shortcut" tab and see a textbox preceeded by "Start in". This is the "current directory". It's to be noted that unlike the other "roots" this one obviously can't be a DAT file, as such the engine doesn't "match" this one to other "roots". This means that even if you make ALL your "roots" point to the current directory, the game sill still looks for the file twice!

Final Words:
- For "fast" fallout: Extract everything in the Fallout base directory (as opposed to in "data"!) This way when a file is missing it'll check for the same file (which should be faster then checking another file as it'll be cached for sure). Edit Fallout2.cfg and put the full path to your fallout base directory. Then finally, remember to make your proto/items/*.pro and proto/critters/*.pro read only! With that in place the load screen now takes under 1/2 second on my p3-600, 640MB RAM (on second run - keep in mind that on the first run it can create files (wolrdmap.dat) and the cache isn't setup. The main gain of speed though was the decompression, not the "root" modifications, albeit every bit helps.).
- For modders: I'd suggest using critter_patch as your modding "base", and extract EVERYTHING into the base directory of fallout. Point master_patch to somewhere invalid as you'll most probably need those PRO files unless working on a TC or somesuch. Point everything else left to your fallout base. Rename patch000.dat to something else to keep as reference when rebuilding your mod for ditribution. This way you can name your mod dir something meaningful too! Extra note, for modders who might be thinking of avoiding the whole "replace patch000.dat" ordeal: Sadly the CFG file the game uses IS the same as the one found in the same directory as the fallout2.exe file and not the one in the current directory. This means you can't make a directory with just a link to Fallout using your own CFG file, you'll also need the executable (which makes the link pointless in the first place).

Extra notes for modders:
- mapper2 pretty much works the same way, however it does not look for patchXXX.dat!
- It saves it's data into master_patch.
To make best use of this whole ordeal, first make sure to extract everything (to make sure the patches are loaded); then point mapper2.cfg's critter_dat to the original Fallout data, point critter_patch to your mod dir and also point master_patch to your mod dir. This way it loads everything you didn't touch from the original data, without overwriting them - and writes in your mod dir; if you edited them then it will load them from your mod dir since critter_patch is loaded before critter_dat! Ain't this spiffy?

Don't hesitate to ask questions if anything's not clear.
Sure hope this helps out!