mirror of
https://github.com/SabreTools/MPF.git
synced 2026-02-03 21:29:27 +00:00
[Request] IRD file creation for PS3 discs #348
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @SamRohod on GitHub (Oct 23, 2021).
Originally assigned to: @Deterous on GitHub.
Is your feature request related to a problem? Please describe.
Currently, creating IRD files for PS3 discs mostly requires a PS3. The program Redump2IRD (HERE) can make IRD files from a Redump .iso but needs:
1- Encrypted Disc key (D1)
2- Decrypted Disc ID (D2)
3- PIC data
Describe the solution you'd like
The program dumping the PS3 disc to the Redump .iso format with its companion IRD file. Or give the information needed to create the IRD with Redump2IRD.
1- For Encrypted Disc key (D1): Incorporate them into the program or allow the user to provide them (like PS3 Disc Dumper).
2- For Decrypted Disc ID (D2): Maybe through connection to the Redump database which I believe store it or if this could only be extracted with a PS3, set a default value of zeros or {check next section}.
If we set default value to zero's:
Encrypted Disc ID = Decrypted Disc ID
04EC7666382555DE33944D947DD64257 = 00000000000000000000000000000000
3- For PIC data: I believe that MPF already extracts the needed PIC data since v2.0 as in the changelog it says "Add PIC layerbreak extraction".
Describe alternatives you've considered
I'm not an expert but in trying to understand the PS3 Disc ID (D2) I unpacked 2758 IRD files (downloaded from the now down jonnysp.bplaced.net) and all of them have the Data2 be one of the following:
Encrypted Disc ID = Decrypted Disc ID (with Redump2IRD) = Regions of games
04F9E7546E1035B2BDC9A7547553D332 = 00000000000000FF0002000000000000 = BD50(??)
508EA396776D0928E58D57469F30D059 = 00000000000000FF0002000100000001 = Asia / Japan mostly
2D1B1AAE18366BD5CE988BF81D6012FB = 00000000000000FF0002000300000001 = US mostly
19795F9BFBD8E3B8B317EDBF6071E728 = 00000000000000FF0002000400000001 = Europe mostly
6539A28E2D4311BCD16A7A675559305E = 00000000000000FF0002000600000001 = Europe only
5EBD7719383C4A0ACE8331FA74D7263E = 00000000000000FF0002000900000001 = US only
2647567752CA7A71243A1EEAA61F50C1 = 00000000000000FF0002000A00000001 = Europe only
It seems there is a limited number of PS3 Decrypted Disc ID (D2) that might relate to regions.
Additional context
What I am really interested in is the ability to convert between Redump .iso and JB format of my backups. The process that I use:
*To make a Redump .iso from JB folder:
1- Use PS3 ISO Rebuilder to build a .iso (plain header) with the proper IRD file.
2- Use PS3Dec with the proper Disc key to encrypt the resulting .iso from step 1
*To extract a Redump .iso to JB folder:
1- Use PS3Dec with the proper Disc key to decrypt the Redump .iso
2- Use PS3 ISO Rebuilder to extract the resulting .iso from step 1
So, an IRD file is needed. Even if the D2 and PIC data were wrong, Redump2IRD would still be able to create an IRD file that allows disc verification/rebuilding from JB folder (with PS3 ISO Rebuilder). But this will contribute to the spreading of these "sudo" IRD files that are useful for disc verification/rebuilding but might have random values for D2 and PIC data. That's why I suggested that the Decrypted Disc ID (D2) is to be set to zero's if the exact value is not obtainable through a PC dump.
A more secure solution would be to use the last 8b of the IRD file in a new way. Instead of CRC and UID (LINK), the last 8b of the IRD file would be encrypted. This encrypted section will have:
1- CRC of the unencrypted part of the IRD file (the part before the last 8b)
2- MPF signature only known to SabreTools.
A closed source "Checker" tool is to be used to verify:
1- The unencrypted part of the IRD has not been corrupted/modified by checking it against the CRC in the encrypted section.
2- The encrypted section of the IRD was not corrupted/modified by checking the MPF signature.
3- The IRD was created from an actual disc using MPF by checking the MPF signature.
With this, we don't need to start a new format, just reusing the old format. Starting a new file format just for disc verification/rebuilding is just not worth it. Also, I believe that most programs that use IRD files (like PS3 ISO Rebuilder) will still work just fine since they don't use the last 8b in any important way.
Also, with this method we can easily verify that two dumps are the same by just comparing the two IRD files (because there is no UID). Increasing our confidence it's a correct dump (if they are the same).
HERE is what seems to be the code for PS3 ISO Rebuilder (might be useful)
@mnadareski commented on GitHub (Oct 18, 2023):
https://www.psx-place.com/threads/manual-ird-creation.33486/
@mnadareski commented on GitHub (Oct 18, 2023):
fc38c4e7b4/MGZ/source/ird_build.c (L74)@Deterous commented on GitHub (Oct 20, 2023):
@SamRohod do you have any more information regarding the structure of an IRD file, and how they're made?
We do not want to use the ISOTools .dll files, which is how Redump2IRD works.
@Deterous commented on GitHub (Oct 22, 2023):
File format specs, try target "Version 9"
https://psdevwiki.com/ps3/Bluray_disc#IRD_file
IRD File Format.pdf
@Deterous commented on GitHub (Oct 22, 2023):
This should provide a decent template, although it seems to be for creating IRD "Version 6":
https://github.com/jhonathanc/3k3y-Ripp3r (originally from https://web.archive.org/web/20190120115657/https://www.eurasia.nu/modules.php?name=Downloads&d_op=search&query=3k3y_ripp3r)
These resources should be enough to give a stab at this tomorrow maybe.
@SamRohod commented on GitHub (Oct 22, 2023):
@Deterous Sorry for the late response.
I'm sorry but unfortunately I'm not in anyway an expert in the matter. I just did some reading on the matter.
Here are somethings that might be useful:
Sorry for not being able to be more helpful.
@Deterous commented on GitHub (Oct 23, 2023):
Thank you @SamRohod, I have already found enough resources to give making an "IRD Library" for MPF to use a go.
I have read your idea around making the IRD file "secure", by using a closed-source checker tool, but I don't think that's a good idea.
Instead, I propose that I build the IRD Library to make reproducible IRDs, such that MPF can build an IRD, hash that IRD, then add the IRD hash to the comments section on redump.org. Anyone else who would later dump an identical disc would then create an identical IRD, a 1:1 correspondence between Redump ISO and redump-style IRD. While these IRDs would not be 3k3y-style, in that fields such as the Disc Key and Unique Identifier are now fixed and deterministic, the redump-style IRD would still be useful for ISO <-> JB, and would of course contain correct file hashes.
Hopefully this is a valid solution to the problem you stated of "bad IRDs" being created and shared.
@Deterous commented on GitHub (Nov 1, 2023):
Progress update: https://github.com/Deterous/LibIRD
Ideas for MPF:
@Deterous commented on GitHub (Nov 10, 2023):
@SamRohod I have finished making the IRD library, work will soon be done to integrate it into MPF,
towards the end of Novembernext year.@gmipf commented on GitHub (Nov 28, 2023):
You could also just use ps3dec for proper iso to iso operations instead of using PS3 Disc Dumper and the rebuilder after that. No one should need to mess with JB folder stuff in this day and age any more since Cobras wide distribution today. Of course there is also RPCS3 which still has no iso support for whatever reason.
@Deterous commented on GitHub (Nov 28, 2023):
The end goal here is for integration with NKit (work in progress) which would provide more than just JB<->ISO and ISO<->ISO. RPCS3 is its own beast, hopefully the developers can figure something out. In the mean time, JB folders remain somewhat useful for rare cases of rebuilding scene-only dumps.
@gmipf commented on GitHub (Nov 28, 2023):
Yes there I agree. Also rebuilding isos where we only have redump entries but not the actual dump, rebuilding with ird can sometimes help generating the og iso. Scene releases also sometime has modified eboot with their own intro screens.
@Deterous commented on GitHub (Feb 5, 2024):
@SamRohod
Finally, I have completed the IRD library such that non-contiguous files are hashed correctly. This is better than 3k3y ISOTools, Redump2IRD and ManaGunZ which all get these file hashes incorrect. It is also much faster than Redump2IRD for large ISOs.
I also have a command-line tool for creating IRDs, printing info about them, and even doing a diff to compare two IRD files: https://github.com/Deterous/LibIRD/releases/
You can create IRDs from just the ISO by running
irdkit.exe create game.isoor even an entire folder of ISOs.There will be a UI window added to MPF soon after MPF 3.1 releases. There will also be an option to enable IRD creation after PS3 dumping, however this won't occur by default.
@SamRohod commented on GitHub (Feb 6, 2024):
@Deterous
Thank you very much for your efforts.
There is no "issues" section on the LibIRD repository, so please excuse the use of this page to mention the following.
I did a test with the LibIRD command-line tool using the ISO of THIS GAME:
I inputted the encrypted .iso file (matching the SHA-1 from Redump) into the tool. This is what I got:
Then I downloaded the IRD file for the same game (BLUS30767) from the ManaGunZ IRD DATABASE
Then using ird_tools to extract the json files from both IRD files, I found that there are mismatches:
1- Mismatches in the values of: MGZ_SIG, MD5, EXTRA_CONFIG, UNIQUE_ID, CRC, DATA2_DEC, DATA2_ENC.
I mentioned all of them because I don't know which ones are expected to be different given the tool's goal of creating deterministic IRD files.
2- Mismatch with the file: "/PS3_GAME/USRDIR/INST/DRACRO.CPK"
I decrypted the ISO with PS3dec then checked the decrypted ISO with PS3-ISO-Rebuilder against the two IRD files. It seems the IRD created by the LibIRD tool is the incorrect one. While properly recreating the ISO using the IRD file downloaded from the ManaGunZ IRD Database can be done, the IRD file produced from the LibIRD tool can't be used with PS3-ISO-Rebuilder for this purpose.
@Deterous commented on GitHub (Feb 6, 2024):
@SamRohod
Thank you for testing it, those ISOs with split files are what needs testing and I only tested one. Likely something I've done to try "fix" non-contiguous files has caused a bad side effect on ISOs that were working. I will look into this, hopefully it's something obvious.
I've also now enabled the issues tab.
When comparing ISOs, you can also use
irdkit diff game1.ird game2.irdwhich will print the different fields.Actually, most of the values you mentioned being different are expected, it is only the file hash that is the problem.
EXTRA_CONFIG: irdkit sets this to 0x0001 so that you know the IRD was creating using LibIRD (normal IRDs have 0x0000).
UNIQUE_ID: As the name suggests, this should be unique. But LibIRD sets this to the CRC of the ISO.
CRC: This is the hash of the IRD itself, will always be different if the IRDs are different
DATA2_DEC/DATA2_ENC: LibIRD uses a fixed value for these currently.
@Deterous commented on GitHub (Feb 7, 2024):
Solved it - an error that only shows up for split files where the split file filesize is not a multiple of 2048.
I'll release a version 0.4.1 which should fix this, thanks for testing!
@SamRohod commented on GitHub (Feb 8, 2024):
@Deterous
It now works with everything I tested it on!!!!!
Regarding DATA2/Disc ID:-
1- Is it possible to parse the value from Redump.org (like it is done for the keys)? or maybe manually inputted?
since adding the hashes of these IRD files to Redump.org is a goal, we wouldn't want there to be a difference between their contents and the Redump.org database (unless there is a mistake in the database already)
2- I'm not sure what is optimal to do with the Disc ID's that end with "XXXXXXXX", which seem to be most of the discs (example exceptions 1, 2). But I have nothing to say here given that I don't even know why these X's are there. I just wanted to point them out so that we would find some standard for translating them in the IRD deterministically (especially if manual input is to be added).
Regarding PIC:-
I don't know how relevant this is, but in Redump.org there are extra 34 digits (always zeros?) that all IRD files don't include.
Thank You
@Deterous commented on GitHub (Feb 9, 2024):
Disc ID: Yes, this is something I have been considering, whether to pull from redump, logs, allow manual input. Currently, I set the Data 2 Key for all IRDs created to be a single fixed value for all single layer discs. The reason for this is reproducibility, as a disc from e.g. UK can have a Disc ID of 00000000000000FF00020004XXXXXXXX and a disc from Australia with a matching ISO can have a Disc ID of 00000000000000FF00020006XXXXXXXX So to make a 1:1 correspondence, all single layer discs have a Disc ID of 00000000000000FF00020000XXXXXXXX regardless of their region. If you want to fix this yourself, you choose the region and change that one byte. As for the X's, they exist because those last 4 bytes are unique for each disc, so are redacted
They are replaced with 00000001 as per the 3k3y standard that ISOTools does when making IRDs.
PIC: The real PIC is actually much larger on real discs, it's just that the useful information (and for PS3, the only non-zero information) is located within the first 100 or so bytes. The IRD standard fixed it to 115 bytes for whatever reason. Redump has it higher, because redump stores more than just PS3 info. In fact, BD-Rs often have non-zero data in the PIC beyond what redump makes visible.
@mnadareski commented on GitHub (Feb 19, 2024):
Not fully complete as of #647