Matt Nadareski
d2e96a8f24
Use DatData for everything; read header elements
...
Reading the header elements will come in handy for when built-in parse and generate code will take care of converting from any format to any format.
2016-05-16 14:28:23 -07:00
Matt Nadareski
d5b5a9a30b
More changes using DatData objects
2016-05-16 13:42:21 -07:00
Matt Nadareski
b23710c814
Consolodate the key variables since there's no need for separate ones.
2016-05-15 13:45:41 -07:00
Matt Nadareski
c723033273
Implement continue/ignore flag for software lists
2016-05-15 13:23:29 -07:00
Matt Nadareski
5448d86cc2
This caused more issues than before. Commenting out for the time being until a better solution is found.
2016-05-15 12:17:11 -07:00
Matt Nadareski
c5564a3219
[doc] minor logging update
2016-05-14 20:50:50 -07:00
Matt Nadareski
9f0f9bda94
dataarea didn't work, let's try offset
2016-05-14 19:41:07 -07:00
Matt Nadareski
f3b3b4c915
Fix new SL dats
...
New SL dats have the size in the dataarea tag and NOT the rom tag. This should fix the issues from that.
2016-05-14 14:59:40 -07:00
Matt Nadareski
741af21e72
[doc] update actual key definition
2016-05-13 22:53:17 -07:00
Matt Nadareski
e06701f08b
[doc] remove old sorting algorithm because it's not useful anymore
2016-05-13 16:55:58 -07:00
Matt Nadareski
5f7252349c
Wrong direction on logic.
2016-05-13 16:52:01 -07:00
Matt Nadareski
a10592ebf3
Maybe fixes the issue with source/system sorting?
2016-05-13 16:50:45 -07:00
Matt Nadareski
e5ebc52129
Second experimental attempt at improving merges
2016-05-13 16:18:50 -07:00
Matt Nadareski
6f1f65aedb
Revert "Experimental flip of values"
...
This reverts commit 1b3cbf6796
2016-05-13 16:17:03 -07:00
Matt Nadareski
1b3cbf6796
Experimental flip of values
2016-05-13 15:43:26 -07:00
Matt Nadareski
6f025b3dfa
Undo part of a previous change because of sorting issues
2016-05-13 12:41:06 -07:00
Matt Nadareski
ec228fc499
Slight rearrangement, Some issues popped up with ordering
2016-05-13 12:16:03 -07:00
Matt Nadareski
f463f3b767
@tractivo: change comparison to better match uniqueness
...
He brought this to my attention because somehow some items are getting compared by CRC only. This is wrong on a lot of levels, but this should be a step in the right direction.
2016-05-13 10:13:46 -07:00
Matt Nadareski
56c4a4b47c
Add code to check for SHA-1 only situations with proper sorting
2016-05-12 10:25:20 -07:00
Matt Nadareski
5224b1dabe
Change merging because blank CRCs can lead to problems
2016-05-11 15:02:33 -07:00
Matt Nadareski
c2b1cebe37
Invert the logic so that it actually makes sense
2016-05-11 12:08:17 -07:00
Matt Nadareski
2844bac565
Take advantage of new enum ordering
2016-05-11 10:57:00 -07:00
Matt Nadareski
382f74afaa
An attempt to make diffing work the way I think it should
...
Basically, if a rom is a duplicate, it can be a duplicate within a system or source or across system or source, and by hash alone or by all data matching. The four combinations make an enum now and the diff function is the only one that uses them right now. If we're in dedup mode, we want to check hashes only in diff. Otherwise, we want to check against ones that match all information. This needs field testing.
2016-05-10 20:55:51 -07:00
Matt Nadareski
d649988a04
Some hex hashes are short and need to be padded
2016-05-10 18:20:27 -07:00
Matt Nadareski
e7ede7cca2
Even more fun with zero-byte files
2016-05-06 13:12:00 -07:00
Matt Nadareski
a7f97cc419
More 0-byte file shennanigans
2016-05-06 12:56:02 -07:00
Matt Nadareski
ced41c5cde
Add skip on nodump
2016-05-06 10:59:05 -07:00
Matt Nadareski
53cfcd19ed
Fix "-" crc, md5, and sha1 hashes
2016-05-06 10:53:42 -07:00
Matt Nadareski
411115083d
Make merging more robust
2016-05-03 23:59:32 -07:00
Matt Nadareski
51dbb6cf8f
[doc] update comment
2016-05-02 15:30:39 -07:00
Matt Nadareski
dcf5aa7071
Forgot the other Parse
2016-05-02 14:08:50 -07:00
Matt Nadareski
690d17f9b5
Take care of hex-hashes
2016-05-02 14:02:43 -07:00
Matt Nadareski
842ab5f39d
Reduce the number of steps
2016-04-29 11:49:23 -07:00
Matt Nadareski
586488296b
Forgot to add fix from DB iteration of code
2016-04-28 17:13:16 -07:00
Matt Nadareski
e9a9ddad6a
Make keys more selective
2016-04-28 16:41:23 -07:00
Matt Nadareski
0897f39c52
Mono doesn't like a missing DTD
2016-04-28 13:04:32 -07:00
Matt Nadareski
fa519036ad
Remove all traces of the outdated DB merge/import
2016-04-28 12:43:56 -07:00
Matt Nadareski
ae2b8fd44d
Add more code for dictionary based merging
2016-04-28 11:31:35 -07:00
Matt Nadareski
a202dc3dbb
More changes to enable Dictionary merging
2016-04-28 11:06:27 -07:00
Matt Nadareski
8aea894aed
Fix non-deduped merging in DB-based version
2016-04-28 10:40:45 -07:00
Matt Nadareski
6d67512258
Add dictionary-based Parse for future use
2016-04-28 10:37:33 -07:00
Matt Nadareski
ed193d1f1e
Remove commented code
2016-04-27 21:53:54 -07:00
Matt Nadareski
a6536fd65e
Fix some finicky XML DATs and how the name is read
2016-04-27 21:27:41 -07:00
Matt Nadareski
820097a2ec
Rename since this is a permanent method now
2016-04-27 20:57:05 -07:00
Matt Nadareski
18369e89df
More code changes to accustom diff and AB
2016-04-27 16:57:03 -07:00
Matt Nadareski
0d22715fa4
Remove unnecessary switch
2016-04-27 11:54:27 -07:00
Matt Nadareski
40b7d64303
Fix standard merging; dedupe still broken
2016-04-27 11:44:14 -07:00
Matt Nadareski
e450eebfd7
Stage 2 changes for using in-mem database
...
This change makes it mostly functional, having a way to both parse and merge in one step and output from the database. The issue is that the output from Parse2 doesn't match Parse OR Parse + Merge. Duplicates don't seem to be added in the first case and there are a differing number of roms in the second.
2016-04-27 01:10:24 -07:00
Matt Nadareski
bce1cc5839
Add stage 1 changes for better merging
...
The new merging process is both parse and merge in one step using an in-memory database. The next steps are to get the data in this database and write it out properly.
2016-04-27 00:53:15 -07:00
Matt Nadareski
f8ed865eaa
Replace Parse with mature second iteration
...
This code for parsing needed to be changed because of large file handling. XmlTextReader ended up being the best solution that doesn't crash at 500 MB in. A side effect of this change is that the code for processing a given DAT is very minimal now. The entire sub-block for SL-specific DAT info doesn't need to exist anymore. Small-scale tests have given 1:1 results with previous versions but further testing is needed for large files to make sure no data is lost.
2016-04-25 01:59:09 -07:00
