This change makes it mostly functional, having a way to both parse and merge in one step and output from the database. The issue is that the output from Parse2 doesn't match Parse OR Parse + Merge. Duplicates don't seem to be added in the first case and there are a differing number of roms in the second.
The new merging process is both parse and merge in one step using an in-memory database. The next steps are to get the data in this database and write it out properly.
This code for parsing needed to be changed because of large file handling. XmlTextReader ended up being the best solution that doesn't crash at 500 MB in. A side effect of this change is that the code for processing a given DAT is very minimal now. The entire sub-block for SL-specific DAT info doesn't need to exist anymore. Small-scale tests have given 1:1 results with previous versions but further testing is needed for large files to make sure no data is lost.
This is currently borken for reasons that I don't 100% understand. It needs to be tested further with a basic DAT to see why it doesn't seem to get every ROM.
Basically, this will output the information that's in each DAT but not in the merged and also the data that's only in every DAT. This needs testing, but it's a good start.
This code can be enabled for local testing by uncommenting and running it in place of the current code. This is the first of few commits with similar theme
The current format of the database is good for small amounts of data. The truth is that the hashes and sizes of the files are what really determine if a file is different or not. After chatting with Obiwantje about this, I'm beginning some code tests with a hash-centric database model, instead of the game/rom centric. This means that each hash can have mutliple roms attached to it, similarly a given game can have multiple sources or systems attached. It's a much cleaner system, but demands a total rewrite and new databae structure.
