Compare commits

..

24 Commits
3.2.0 ... 3.3.0

Author SHA1 Message Date
Matt Nadareski
de17b0c59c Bump version 2026-11-11 17:41:51 +01:00
gmipf
1663345195 Add a UDF reader and wrapper (#100)
* Add a UDF reader

Reads the Volume Recognition Sequence, the Anchor Volume Descriptor
Pointer (sector 256, then the end of the volume), the Main or Reserve
Volume Descriptor Sequence, the File Set Descriptor and the directory
hierarchy. Type 1 and Metadata Partitions are mapped, Virtual and
Sparable partitions are not.

File Entries, Allocation Extent Descriptors and directories are read in
ascending sector order with wrap-around, adjacent sectors merged into
one request. FindFileEntry follows one path and reads only the
directories on it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Add a UDF wrapper

Prints the volume and file set structures and the directory hierarchy,
and extracts the files. CreateDiscImageWrapper tries UDF after ISO 9660
for a stream at offset 0, so discs with both file systems still open as
ISO 9660.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Treat the stream position as the start of the UDF volume

The UDF reader addresses logical sectors from the start of the stream,
so the wrapper hands it a view that starts at the current position and
keeps that offset, like the other wrappers. CreateDiscImageWrapper no
longer skips UDF for a stream that is not at offset 0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Address review comments on the UDF code

- Move the descriptor tag check to UDFExtensions, with tests
- Parse Volume Structure Descriptors in their own method
- Use the existing extensions for numbers, the partition type identifier
  and printing entity identifiers
- Use Array.Exists in IsSafeName
- Make Extract non-virtual and remove the printing test
- Use collection expressions and string interpolation

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Parse File Identifier Descriptors in their own method

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Add a TODO to use the CRC from Hashing

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 17:18:23 +01:00
Matt Nadareski
5c3184baa7 Add .NET 11 support 2026-11-11 12:02:16 +01:00
gmipf
9e39a0cf16 Close the cabinet files opened for a cabinet set (#105)
* Close the cabinet files opened for a cabinet set

MicrosoftCabinet.Extract opened the cabinet file again for the set,
together with the previous and next cabinets, and never closed any of
them, so they stayed open until their streams were finalized.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Keep the original comment on the previous cabinet

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 14:19:16 -04:00
gmipf
36a67139d9 Add the UDF models (#108)
The ECMA-167 and UDF structures that the UDF reader fills in, split out
of #100. The only change is that the notes on Volume moved from the
summary into the remarks.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 13:31:46 -04:00
gmipf
33caf06bc1 Read ISO 9660 directories in volume order and add a path lookup (#99)
* Read directory extents in volume order

ParseDirectoryDescriptors now keeps one worklist of directory extents for
all volume descriptors, sorted by offset, and always reads the next
extent at or after the last one, wrapping around at the end. Each extent
is read once. The record loop moves to a shared helper.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Add a path lookup for ISO 9660 directory records

FindDirectoryRecords follows a path from the root directory of one
volume descriptor and reads only the directories on that path. Names
are compared case-insensitively, without the version suffix or a
trailing period. A file recorded in several extents returns all of its
records.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 11:00:54 -04:00
Matt Nadareski
f639f7ac24 Strip out MS-CAB extraction test even further
This will make it so that exceptions are surfaced directly into the test log and cancel right away. It effectively does the same thing as capturing the error log and redirecting it, but with many less steps.
2026-10-26 09:52:32 -04:00
gmipf
11f9bf9fd9 Stop writing after the last file of a cabinet folder (#107)
* Stop writing after the last file of a cabinet folder

When the last data block of a folder held the end of two or more files,
WriteData still wrote the rest of the block after EndFile had closed the
last file, which threw an ObjectDisposedException. It now returns there,
as it already does when the first file in a block ends the folder.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Simplify the console check in the cabinet extraction test

Extract catches the ObjectDisposedException and only writes it to
Console.Error, so the test still reads the console, but without the
try/finally. It now fails with the whole exception as its message.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 09:46:10 -04:00
gmipf
373cf2d752 Replace invalid filename characters in Wise dialog names (#106)
The file of a custom dialog is named after the dialog name from the
script, which is free text. On Windows, a name like "Install under NT?"
is not a valid filename, so writing the file threw an IOException and
stopped the extraction.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 21:32:34 -04:00
gmipf
dc18951d0b Close the Wise script and the multipart file after extraction (#104)
* Look for the Wise overlay header before opening the file again

ExtractWiseOverlay (and ExtractWise for NE) opened a second stream on
the executable for multi-part Wise installers before checking for a
Wise overlay header at all. For any other executable that stream was
never closed, so the file stayed open until the stream was finalized.
On Windows such a file cannot be deleted, so an extraction folder that
contains it is left behind in TEMP.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Close the Wise script and the multipart file after extraction

ExtractWiseOverlay (and ExtractWise for NE) never closed WiseScript.bin,
which it opens from the output directory, nor the stream with the
installer and its parts, so these files stayed open until their streams
were finalized.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Keep the original block that opens the multipart file

Close the multipart stream with a separate using declaration after the
block instead of rewriting it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 21:30:32 -04:00
gmipf
29b9d27c33 Look for the Wise overlay header before opening the file again (#103)
ExtractWiseOverlay (and ExtractWise for NE) opened a second stream on
the executable for multi-part Wise installers before checking for a
Wise overlay header at all. For any other executable that stream was
never closed, so the file stayed open until the stream was finalized.
On Windows such a file cannot be deleted, so an extraction folder that
contains it is left behind in TEMP.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 09:50:21 -04:00
gmipf
f6ade446c6 Locate the PKZIP file data instead of reading it into the model (#102)
* Locate the PKZIP file data instead of reading it into the model

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Keep only the offset of the PKZIP file data

The length is the compressed size from the local file header or the ZIP64
extended field, which the printer already shows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Keep the file data property as a comment for documentation

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 21:04:01 -04:00
gmipf
6238fcbb46 Format refresh and freq with the invariant culture (#101)
The DAT writers and the filter formatted Refresh and Freq with the
current culture, so a German or French system wrote "123,450000".
SabreTools.IO parses these values with the invariant culture.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 19:49:12 -04:00
gmipf
2f6698edac Parse the big-endian directory extent only when ambiguous (#98)
ParseDirectory parsed the big-endian extent when the extent location
was valid (both values equal) instead of when it was ambiguous, so
every directory was parsed again, once per directory level.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 21:16:14 -04:00
Matt Nadareski
04e8751f9b Update packages 2026-10-12 09:56:31 -04:00
Matt Nadareski
7262f74f7f Enable versions in publish script 2026-09-21 09:17:22 -04:00
Matt Nadareski
0b6af978ff Check for "null" literal string cases 2026-09-18 11:01:40 -04:00
Matt Nadareski
2cacaac9fd Make size an optional output for CMP and Logiqx 2026-09-18 10:56:30 -04:00
Matt Nadareski
d39d8f9b61 Allow sizeless ROMs if there's a SHA-1 or SHA-256 2026-09-18 10:54:32 -04:00
Matt Nadareski
09fd38e776 Allow sizeless ROMs if there's a SHA-1 or SHA-256 2026-09-18 10:47:36 -04:00
Matt Nadareski
db2a6f63ff Allow sizeless ROMs if there's a SHA-1 or SHA-256 2026-09-18 10:44:56 -04:00
Matt Nadareski
d9dd16fbb9 Fix OpenMSX rom attribute parsing 2026-09-18 10:34:40 -04:00
Matt Nadareski
3c0c31cf10 Use SHA-1 as placeholder name for OpenMSX 2026-09-18 10:27:46 -04:00
Matt Nadareski
d9bf16deef OpenMSX has changed a bit 2026-09-18 10:15:54 -04:00
114 changed files with 6395 additions and 280 deletions

View File

@@ -20,6 +20,7 @@ jobs:
8.0.x
9.0.x
10.0.x
11.0.x
- name: Run tests
run: dotnet test

View File

@@ -15,6 +15,7 @@ jobs:
8.0.x
9.0.x
10.0.x
11.0.x
- name: Build
run: dotnet build

4
.vscode/launch.json vendored
View File

@@ -10,7 +10,7 @@
"request": "launch",
"preLaunchTask": "build",
// If you have changed target frameworks, make sure to update the program path.
"program": "${workspaceFolder}/ExtractionTool/bin/Debug/net10.0/ExtractionTool.dll",
"program": "${workspaceFolder}/ExtractionTool/bin/Debug/net11.0/ExtractionTool.dll",
"args": [],
"cwd": "${workspaceFolder}",
// For more information about the 'console' field, see https://aka.ms/VSCode-CS-LaunchJson-Console
@@ -24,7 +24,7 @@
"request": "launch",
"preLaunchTask": "build",
// If you have changed target frameworks, make sure to update the program path.
"program": "${workspaceFolder}/InfoPrint/bin/Debug/net10.0/InfoPrint.dll",
"program": "${workspaceFolder}/InfoPrint/bin/Debug/net11.0/InfoPrint.dll",
"args": [],
"cwd": "${workspaceFolder}",
// For more information about the 'console' field, see https://aka.ms/VSCode-CS-LaunchJson-Console

View File

@@ -1,7 +1,7 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFrameworks>net20;net35;net40;net452;net462;net472;net48;netcoreapp3.1;net5.0;net6.0;net7.0;net8.0;net9.0;net10.0</TargetFrameworks>
<TargetFrameworks>net20;net35;net40;net452;net462;net472;net48;netcoreapp3.1;net5.0;net6.0;net7.0;net8.0;net9.0;net10.0;net11.0</TargetFrameworks>
<OutputType>Exe</OutputType>
<CheckEolTargetFramework>false</CheckEolTargetFramework>
<IncludeNativeLibrariesForSelfExtract>true</IncludeNativeLibrariesForSelfExtract>
@@ -10,7 +10,7 @@
<Nullable>enable</Nullable>
<SuppressTfmSupportBuildWarnings>true</SuppressTfmSupportBuildWarnings>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
<Version>3.2.0</Version>
<Version>3.3.0</Version>
</PropertyGroup>
<!-- Support All Frameworks -->
@@ -20,11 +20,11 @@
<PropertyGroup Condition="$(TargetFramework.StartsWith(`netcoreapp`)) OR $(TargetFramework.StartsWith(`net5`))">
<RuntimeIdentifiers>win-x86;win-x64;win-arm64;linux-x64;linux-arm64;osx-x64</RuntimeIdentifiers>
</PropertyGroup>
<PropertyGroup Condition="$(TargetFramework.StartsWith(`net6`)) OR $(TargetFramework.StartsWith(`net7`)) OR $(TargetFramework.StartsWith(`net8`)) OR $(TargetFramework.StartsWith(`net9`)) OR $(TargetFramework.StartsWith(`net10`))">
<PropertyGroup Condition="$(TargetFramework.StartsWith(`net6`)) OR $(TargetFramework.StartsWith(`net7`)) OR $(TargetFramework.StartsWith(`net8`)) OR $(TargetFramework.StartsWith(`net9`)) OR $(TargetFramework.StartsWith(`net10`)) OR $(TargetFramework.StartsWith(`net11`))">
<RuntimeIdentifiers>win-x86;win-x64;win-arm64;linux-x64;linux-arm64;osx-x64;osx-arm64</RuntimeIdentifiers>
</PropertyGroup>
<PropertyGroup Condition="$(RuntimeIdentifier.StartsWith(`osx-arm`))">
<TargetFrameworks>net6.0;net7.0;net8.0;net9.0;net10.0</TargetFrameworks>
<TargetFrameworks>net6.0;net7.0;net8.0;net9.0;net10.0;net11.0</TargetFrameworks>
</PropertyGroup>
<!-- These are needed for dealing with native Windows DLLs -->
@@ -48,9 +48,9 @@
</ItemGroup>
<ItemGroup>
<PackageReference Include="SabreTools.CommandLine" Version="1.5.0" />
<PackageReference Include="SabreTools.IO" Version="2.3.0" />
<PackageReference Include="System.Text.Encoding.CodePages" Version="10.0.11" Condition="!$(TargetFramework.StartsWith(`net2`)) AND !$(TargetFramework.StartsWith(`net3`)) AND !$(TargetFramework.StartsWith(`net40`)) AND !$(TargetFramework.StartsWith(`net452`))" />
<PackageReference Include="SabreTools.CommandLine" Version="1.6.0" />
<PackageReference Include="SabreTools.IO" Version="2.4.0" />
<PackageReference Include="System.Text.Encoding.CodePages" Version="11.0.0" Condition="!$(TargetFramework.StartsWith(`net2`)) AND !$(TargetFramework.StartsWith(`net3`)) AND !$(TargetFramework.StartsWith(`net40`)) AND !$(TargetFramework.StartsWith(`net452`))" />
</ItemGroup>
</Project>

View File

@@ -1,7 +1,7 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFrameworks>net20;net35;net40;net452;net462;net472;net48;netcoreapp3.1;net5.0;net6.0;net7.0;net8.0;net9.0;net10.0</TargetFrameworks>
<TargetFrameworks>net20;net35;net40;net452;net462;net472;net48;netcoreapp3.1;net5.0;net6.0;net7.0;net8.0;net9.0;net10.0;net11.0</TargetFrameworks>
<OutputType>Exe</OutputType>
<CheckEolTargetFramework>false</CheckEolTargetFramework>
<IncludeNativeLibrariesForSelfExtract>true</IncludeNativeLibrariesForSelfExtract>
@@ -10,7 +10,7 @@
<Nullable>enable</Nullable>
<SuppressTfmSupportBuildWarnings>true</SuppressTfmSupportBuildWarnings>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
<Version>3.2.0</Version>
<Version>3.3.0</Version>
</PropertyGroup>
<!-- Support All Frameworks -->
@@ -20,11 +20,11 @@
<PropertyGroup Condition="$(TargetFramework.StartsWith(`netcoreapp`)) OR $(TargetFramework.StartsWith(`net5`))">
<RuntimeIdentifiers>win-x86;win-x64;win-arm64;linux-x64;linux-arm64;osx-x64</RuntimeIdentifiers>
</PropertyGroup>
<PropertyGroup Condition="$(TargetFramework.StartsWith(`net6`)) OR $(TargetFramework.StartsWith(`net7`)) OR $(TargetFramework.StartsWith(`net8`)) OR $(TargetFramework.StartsWith(`net9`)) OR $(TargetFramework.StartsWith(`net10`))">
<PropertyGroup Condition="$(TargetFramework.StartsWith(`net6`)) OR $(TargetFramework.StartsWith(`net7`)) OR $(TargetFramework.StartsWith(`net8`)) OR $(TargetFramework.StartsWith(`net9`)) OR $(TargetFramework.StartsWith(`net10`)) OR $(TargetFramework.StartsWith(`net11`))">
<RuntimeIdentifiers>win-x86;win-x64;win-arm64;linux-x64;linux-arm64;osx-x64;osx-arm64</RuntimeIdentifiers>
</PropertyGroup>
<PropertyGroup Condition="$(RuntimeIdentifier.StartsWith(`osx-arm`))">
<TargetFrameworks>net6.0;net7.0;net8.0;net9.0;net10.0</TargetFrameworks>
<TargetFrameworks>net6.0;net7.0;net8.0;net9.0;net10.0;net11.0</TargetFrameworks>
</PropertyGroup>
<ItemGroup>
@@ -32,9 +32,9 @@
</ItemGroup>
<ItemGroup>
<PackageReference Include="SabreTools.CommandLine" Version="1.5.0" />
<PackageReference Include="SabreTools.Hashing" Version="2.1.0" />
<PackageReference Include="SabreTools.IO" Version="2.3.0" />
<PackageReference Include="SabreTools.CommandLine" Version="1.6.0" />
<PackageReference Include="SabreTools.Hashing" Version="2.2.0" />
<PackageReference Include="SabreTools.IO" Version="2.4.0" />
</ItemGroup>
</Project>

View File

@@ -1,17 +1,24 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFrameworks>net8.0;net9.0;net10.0</TargetFrameworks>
<TargetFrameworks>net8.0;net9.0;net10.0;net11.0</TargetFrameworks>
<IsPackable>false</IsPackable>
<LangVersion>latest</LangVersion>
<Nullable>enable</Nullable>
<SuppressTfmSupportBuildWarnings>true</SuppressTfmSupportBuildWarnings>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="coverlet.collector" Version="10.0.1" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.9.0" />
<PackageReference Include="coverlet.collector" Version="10.1.0">
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
<PrivateAssets>all</PrivateAssets>
</PackageReference>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.11.1" />
<PackageReference Include="xunit" Version="2.9.3" />
<PackageReference Include="xunit.runner.visualstudio" Version="4.0.0" />
<PackageReference Include="xunit.runner.visualstudio" Version="4.0.1">
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
<PrivateAssets>all</PrivateAssets>
</PackageReference>
</ItemGroup>
<ItemGroup>

View File

@@ -0,0 +1,274 @@
using SabreTools.Data.Models.UDF;
using Xunit;
namespace SabreTools.Data.Extensions.Test
{
public class UDFExtensionsTests
{
#region DecodeDCharacters
[Fact]
public void DecodeDCharacters_Null_Empty()
{
byte[]? data = null;
Assert.Equal(string.Empty, data.DecodeDCharacters());
}
[Fact]
public void DecodeDCharacters_8Bit_Decoded()
{
byte[] data = [8, 0x41, 0x62, 0xE4];
Assert.Equal("Abä", data.DecodeDCharacters());
}
[Fact]
public void DecodeDCharacters_16Bit_Decoded()
{
byte[] data = [16, 0x00, 0x41, 0x30, 0x42];
Assert.Equal("Aあ", data.DecodeDCharacters());
}
[Fact]
public void DecodeDCharacters_UnknownCompression_Null()
{
byte[] data = [1, 0x41];
Assert.Null(data.DecodeDCharacters());
}
[Fact]
public void DecodeDString_UsesLengthByte()
{
byte[] data = [8, 0x41, 0x42, 0x43, 0x00, 3];
Assert.Equal("AB", data.DecodeDString());
}
[Fact]
public void DecodeDString_ZeroLength_Empty()
{
byte[] data = [8, 0x41, 0x00, 0];
Assert.Equal(string.Empty, data.DecodeDString());
}
#endregion
#region TryGetSector
[Fact]
public void TryGetSector_Type1_StartPlusBlock()
{
var volume = CreateVolume(new Type1PartitionMap { PartitionNumber = 0 });
bool actual = volume.TryGetSector(UDFExtensions.GetKey(0, 10), out long sector);
Assert.True(actual);
Assert.Equal(1010, sector);
}
[Fact]
public void TryGetSector_Type1_OutsidePartition_False()
{
var volume = CreateVolume(new Type1PartitionMap { PartitionNumber = 0 });
bool actual = volume.TryGetSector(UDFExtensions.GetKey(0, 100), out _);
Assert.False(actual);
}
[Fact]
public void TryGetSector_UnknownPartitionReference_False()
{
var volume = CreateVolume(new Type1PartitionMap { PartitionNumber = 0 });
bool actual = volume.TryGetSector(UDFExtensions.GetKey(1, 0), out _);
Assert.False(actual);
}
[Fact]
public void TryGetSector_Metadata_ThroughMetadataFile()
{
// Metadata File at block 5 of partition 0: blocks 0-1 of the metadata partition are blocks 20-21,
// blocks 2-3 are blocks 40-41
var volume = CreateVolume(
new Type1PartitionMap { PartitionNumber = 0 },
new MetadataPartitionMap { PartitionNumber = 0, MetadataFileLocation = 5, MetadataMirrorFileLocation = 6 });
volume.FileEntries[UDFExtensions.GetKey(0, 5)] = CreateShortAdEntry((4096, 20), (4096, 40));
Assert.True(volume.TryGetSector(UDFExtensions.GetKey(1, 1), out long first));
Assert.Equal(1021, first);
Assert.True(volume.TryGetSector(UDFExtensions.GetKey(1, 2), out long second));
Assert.Equal(1040, second);
Assert.False(volume.TryGetSector(UDFExtensions.GetKey(1, 4), out _));
}
[Fact]
public void TryGetSector_Metadata_FallsBackToMirror()
{
var volume = CreateVolume(
new Type1PartitionMap { PartitionNumber = 0 },
new MetadataPartitionMap { PartitionNumber = 0, MetadataFileLocation = 5, MetadataMirrorFileLocation = 6 });
volume.FileEntries[UDFExtensions.GetKey(0, 6)] = CreateShortAdEntry((2048, 30));
Assert.True(volume.TryGetSector(UDFExtensions.GetKey(1, 0), out long sector));
Assert.Equal(1030, sector);
}
#endregion
#region GetAllocationDescriptors
[Fact]
public void GetAllocationDescriptors_Embedded_Empty()
{
var entry = new FileEntry();
entry.ICBTag.Flags = (ushort)AllocationDescriptorType.EMBEDDED;
entry.AllocationDescriptors = [1, 2, 3];
var actual = new Volume().GetAllocationDescriptors(0, entry);
Assert.NotNull(actual);
Assert.Empty(actual);
}
[Fact]
public void GetAllocationDescriptors_ZeroLengthTerminates()
{
var entry = CreateShortAdEntry((2048, 20), (0, 99), (2048, 40));
var actual = new Volume().GetAllocationDescriptors(UDFExtensions.GetKey(2, 7), entry);
Assert.NotNull(actual);
var ad = Assert.Single(actual);
Assert.Equal(20u, ad.ExtentLocation.LogicalBlockNumber);
Assert.Equal((ushort)2, ad.ExtentLocation.PartitionReferenceNumber);
}
[Fact]
public void GetAllocationDescriptors_MissingAllocationExtent_NullWithKey()
{
var entry = CreateShortAdEntry((2048, 20), ((3u << 30) | 2048, 50));
var actual = new Volume().GetAllocationDescriptors(UDFExtensions.GetKey(0, 7), entry, out long missingKey);
Assert.Null(actual);
Assert.Equal(UDFExtensions.GetKey(0, 50), missingKey);
}
[Fact]
public void GetAllocationDescriptors_AllocationExtentLoop_Null()
{
var volume = new Volume();
var entry = CreateShortAdEntry(((3u << 30) | 2048, 50));
volume.AllocationExtentDescriptors[UDFExtensions.GetKey(0, 50)] = new AllocationExtentDescriptor
{
AllocationDescriptors = CreateShortAdEntry(((3u << 30) | 2048, 50)).AllocationDescriptors,
};
var actual = volume.GetAllocationDescriptors(UDFExtensions.GetKey(0, 7), entry);
Assert.Null(actual);
}
#endregion
#region IsValidDescriptor
[Theory]
[InlineData(2)]
[InlineData(3)]
public void IsValidDescriptor_Valid_True(byte version)
{
byte[] data = CreateDescriptor(version);
Assert.True(data.IsValidDescriptor(0, data.Length, 20));
}
[Fact]
public void IsValidDescriptor_WrongChecksum_False()
{
byte[] data = CreateDescriptor(2);
data[4]++;
Assert.False(data.IsValidDescriptor(0, data.Length, 20));
}
[Fact]
public void IsValidDescriptor_UnknownVersion_False()
{
byte[] data = CreateDescriptor(4);
Assert.False(data.IsValidDescriptor(0, data.Length, 20));
}
[Fact]
public void IsValidDescriptor_WrongCrc_False()
{
byte[] data = CreateDescriptor(2);
data[16] ^= 0x01;
Assert.False(data.IsValidDescriptor(0, data.Length, 20));
}
[Fact]
public void IsValidDescriptor_CrcPastAvailable_NotChecked()
{
byte[] data = CreateDescriptor(2);
data[16] ^= 0x01;
Assert.True(data.IsValidDescriptor(0, 16, 20));
}
[Fact]
public void IsValidDescriptor_WrongLocation_False()
{
byte[] data = CreateDescriptor(2);
Assert.False(data.IsValidDescriptor(0, data.Length, 21));
}
#endregion
/// <summary>
/// Descriptor with tag location 20 whose 3 bytes after the tag have the CRC #3299 (ECMA-167 3/7.2.6)
/// </summary>
private static byte[] CreateDescriptor(byte version)
{
byte[] data = new byte[19];
data[0] = 1;
data[2] = version;
data[8] = 0x99;
data[9] = 0x32;
data[10] = 3;
data[12] = 20;
data[16] = 0x70;
data[17] = 0x6A;
data[18] = 0x77;
for (int i = 0; i < 16; i++)
{
if (i != 4)
data[4] += data[i];
}
return data;
}
/// <summary>
/// Volume with partition 0 at sector 1000 (length 50) and the given partition maps
/// </summary>
private static Volume CreateVolume(params PartitionMap[] maps)
{
return new Volume
{
VolumeDescriptorSequence =
[
new PartitionDescriptor { PartitionNumber = 0, PartitionStartingLocation = 1000, PartitionLength = 50 },
new LogicalVolumeDescriptor { LogicalBlockSize = 2048, PartitionMaps = maps },
],
};
}
/// <summary>
/// File Entry with the given short allocation descriptors (extent length, position)
/// </summary>
private static FileEntry CreateShortAdEntry(params (uint Length, uint Position)[] extents)
{
var data = new byte[extents.Length * 8];
for (int i = 0; i < extents.Length; i++)
{
for (int b = 0; b < 4; b++)
{
data[(i * 8) + b] = (byte)(extents[i].Length >> (8 * b));
data[(i * 8) + 4 + b] = (byte)(extents[i].Position >> (8 * b));
}
}
var entry = new FileEntry { AllocationDescriptors = data };
entry.ICBTag.Flags = (ushort)AllocationDescriptorType.SHORT;
return entry;
}
}
}

View File

@@ -2,7 +2,7 @@
<PropertyGroup>
<!-- Assembly Properties -->
<TargetFrameworks>net20;net35;net40;net452;net462;net472;net48;netcoreapp3.1;net5.0;net6.0;net7.0;net8.0;net9.0;net10.0;netstandard2.0;netstandard2.1</TargetFrameworks>
<TargetFrameworks>net20;net35;net40;net452;net462;net472;net48;netcoreapp3.1;net5.0;net6.0;net7.0;net8.0;net9.0;net10.0;net11.0;netstandard2.0;netstandard2.1</TargetFrameworks>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<CheckEolTargetFramework>false</CheckEolTargetFramework>
<IncludeSourceRevisionInInformationalVersion>false</IncludeSourceRevisionInInformationalVersion>
@@ -12,7 +12,7 @@
<SuppressTfmSupportBuildWarnings>true</SuppressTfmSupportBuildWarnings>
<SymbolPackageFormat>snupkg</SymbolPackageFormat>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
<Version>3.2.0</Version>
<Version>3.3.0</Version>
<!-- Package Properties -->
<Authors>Matt Nadareski</Authors>
@@ -32,7 +32,7 @@
<ItemGroup>
<PackageReference Include="NetLegacySupport.Numerics" Version="1.0.1" Condition="$(TargetFramework.StartsWith(`net2`)) OR $(TargetFramework.StartsWith(`net3`))" />
<PackageReference Include="SabreTools.IO" Version="2.3.0" />
<PackageReference Include="SabreTools.IO" Version="2.4.0" />
</ItemGroup>
<ItemGroup>

View File

@@ -0,0 +1,417 @@
using System.Collections.Generic;
using System.Text;
using SabreTools.Data.Models.UDF;
using SabreTools.Numerics.Extensions;
namespace SabreTools.Data.Extensions
{
public static class UDFExtensions
{
/// <summary>
/// Maximum number of Allocation Extent Descriptors followed for one File Entry
/// </summary>
private const int MaximumAllocationExtents = 1024;
/// <summary>
/// CRC-ITU-T (CCITT, polynomial 0x1021) table for descriptor CRCs
/// </summary>
private static readonly ushort[] CrcTable = CreateCrcTable();
/// <summary>
/// Get the dictionary key of a logical block address
/// </summary>
/// <param name="address">Logical block address</param>
/// <returns>((long)PartitionReferenceNumber &lt;&lt; 32) | LogicalBlockNumber</returns>
public static long GetKey(this LogicalBlockAddress address)
=> GetKey(address.PartitionReferenceNumber, address.LogicalBlockNumber);
/// <summary>
/// Get the dictionary key of a logical block address
/// </summary>
/// <param name="partitionReferenceNumber">Index of the partition map</param>
/// <param name="logicalBlockNumber">Logical block number within the partition</param>
/// <returns>((long)partitionReferenceNumber &lt;&lt; 32) | logicalBlockNumber</returns>
public static long GetKey(ushort partitionReferenceNumber, uint logicalBlockNumber)
=> ((long)partitionReferenceNumber << 32) | logicalBlockNumber;
/// <summary>
/// Get the extent type from an allocation descriptor's extent length
/// </summary>
public static ExtentType GetExtentType(uint extentLength)
=> (ExtentType)(extentLength >> Constants.ExtentTypeShift);
/// <summary>
/// Get the extent length in bytes from an allocation descriptor's extent length
/// </summary>
public static uint GetExtentLength(uint extentLength)
=> extentLength & Constants.ExtentLengthMask;
/// <summary>
/// Get the type of the allocation descriptors of an ICB
/// </summary>
public static AllocationDescriptorType GetAllocationDescriptorType(this ICBTag tag)
=> (AllocationDescriptorType)(tag.Flags & Constants.AllocationDescriptorTypeMask);
/// <summary>
/// Get the prevailing Logical Volume Descriptor (highest Volume Descriptor Sequence Number)
/// </summary>
/// <param name="volume">Volume to search</param>
/// <returns>Logical Volume Descriptor on success, null if none was recorded</returns>
public static LogicalVolumeDescriptor? GetLogicalVolumeDescriptor(this Volume volume)
{
LogicalVolumeDescriptor? prevailing = null;
foreach (var descriptor in volume.VolumeDescriptorSequence)
{
if (descriptor is LogicalVolumeDescriptor lvd
&& (prevailing is null || lvd.VolumeDescriptorSequenceNumber > prevailing.VolumeDescriptorSequenceNumber))
{
prevailing = lvd;
}
}
return prevailing;
}
/// <summary>
/// Get the prevailing Partition Descriptor of a partition (highest Volume Descriptor Sequence Number)
/// </summary>
/// <param name="volume">Volume to search</param>
/// <param name="partitionNumber">Partition number</param>
/// <returns>Partition Descriptor on success, null if none was recorded</returns>
public static PartitionDescriptor? GetPartitionDescriptor(this Volume volume, ushort partitionNumber)
{
PartitionDescriptor? prevailing = null;
foreach (var descriptor in volume.VolumeDescriptorSequence)
{
if (descriptor is PartitionDescriptor pd
&& pd.PartitionNumber == partitionNumber
&& (prevailing is null || pd.VolumeDescriptorSequenceNumber > prevailing.VolumeDescriptorSequenceNumber))
{
prevailing = pd;
}
}
return prevailing;
}
/// <summary>
/// Get the partition reference number of the Type 1 partition map for a partition number
/// </summary>
/// <param name="lvd">Logical Volume Descriptor holding the partition maps</param>
/// <param name="partitionNumber">Partition number</param>
/// <returns>Partition reference number on success, -1 if there is no such map</returns>
public static int GetType1PartitionReference(this LogicalVolumeDescriptor lvd, ushort partitionNumber)
{
for (int i = 0; i < lvd.PartitionMaps.Length; i++)
{
if (lvd.PartitionMaps[i] is Type1PartitionMap map && map.PartitionNumber == partitionNumber)
return i;
}
return -1;
}
/// <summary>
/// Map a logical block of a partition to a logical sector of the volume
/// Supports Type 1 partitions and Metadata Partitions (the Metadata File's File Entry must be in FileEntries)
/// </summary>
/// <param name="volume">Volume containing the Volume Descriptor Sequence</param>
/// <param name="key">Logical block address key, see GetKey</param>
/// <param name="sector">Logical sector on success</param>
/// <returns>True if the block could be mapped, false otherwise</returns>
public static bool TryGetSector(this Volume volume, long key, out long sector)
{
sector = -1;
var lvd = volume.GetLogicalVolumeDescriptor();
if (lvd is null)
return false;
int partitionReference = (int)(key >> 32);
uint block = (uint)key;
if (partitionReference < 0 || partitionReference >= lvd.PartitionMaps.Length)
return false;
var map = lvd.PartitionMaps[partitionReference];
if (map is Type1PartitionMap type1)
{
var pd = volume.GetPartitionDescriptor(type1.PartitionNumber);
if (pd is null || block >= pd.PartitionLength)
return false;
sector = (long)pd.PartitionStartingLocation + block;
return true;
}
else if (map is MetadataPartitionMap metadata)
{
// The Metadata File (or its mirror) is recorded in the Type 1 partition of the same partition number
int physical = lvd.GetType1PartitionReference(metadata.PartitionNumber);
if (physical < 0)
return false;
if (!TryGetMetadataFileBlock(volume, lvd, GetKey((ushort)physical, metadata.MetadataFileLocation), block, out long physicalKey)
&& !TryGetMetadataFileBlock(volume, lvd, GetKey((ushort)physical, metadata.MetadataMirrorFileLocation), block, out physicalKey))
{
return false;
}
return volume.TryGetSector(physicalKey, out sector);
}
// Virtual and Sparable partitions are not supported
return false;
}
/// <summary>
/// Map a logical block of a Metadata Partition through the Metadata File's allocation descriptors
/// </summary>
private static bool TryGetMetadataFileBlock(Volume volume, LogicalVolumeDescriptor lvd, long fileKey, uint block, out long physicalKey)
{
physicalKey = -1;
if (!volume.FileEntries.TryGetValue(fileKey, out var fileEntry))
return false;
var descriptors = volume.GetAllocationDescriptors(fileKey, fileEntry);
if (descriptors is null)
return false;
uint blockSize = lvd.LogicalBlockSize;
long first = 0;
foreach (var ad in descriptors)
{
long blocks = (GetExtentLength(ad.ExtentLength) + blockSize - 1) / blockSize;
if (block < first + blocks)
{
if (GetExtentType(ad.ExtentLength) != ExtentType.RECORDED_AND_ALLOCATED)
return false;
physicalKey = GetKey(ad.ExtentLocation.PartitionReferenceNumber, (uint)(ad.ExtentLocation.LogicalBlockNumber + (block - first)));
return true;
}
first += blocks;
}
return false;
}
/// <summary>
/// Get the allocation descriptors of a File Entry, including those continued in Allocation Extent Descriptors,
/// as long allocation descriptors (short allocation descriptors get the partition of the File Entry)
/// </summary>
/// <param name="volume">Volume containing the Allocation Extent Descriptors</param>
/// <param name="key">Logical block address key of the File Entry</param>
/// <param name="fileEntry">File Entry</param>
/// <returns>Allocation descriptors (empty for embedded data), null if they are not supported or incomplete</returns>
public static List<LongAllocationDescriptor>? GetAllocationDescriptors(this Volume volume, long key, FileEntry fileEntry)
=> volume.GetAllocationDescriptors(key, fileEntry, out _);
/// <summary>
/// Get the allocation descriptors of a File Entry, including those continued in Allocation Extent Descriptors,
/// as long allocation descriptors (short allocation descriptors get the partition of the File Entry)
/// </summary>
/// <param name="volume">Volume containing the Allocation Extent Descriptors</param>
/// <param name="key">Logical block address key of the File Entry</param>
/// <param name="fileEntry">File Entry</param>
/// <param name="missingKey">Key of the first Allocation Extent Descriptor that is not in the volume, -1 if none</param>
/// <returns>Allocation descriptors (empty for embedded data), null if they are not supported or incomplete</returns>
public static List<LongAllocationDescriptor>? GetAllocationDescriptors(this Volume volume, long key, FileEntry fileEntry, out long missingKey)
{
missingKey = -1;
var type = fileEntry.ICBTag.GetAllocationDescriptorType();
if (type == AllocationDescriptorType.EMBEDDED)
return [];
// Extended allocation descriptors are not allowed by UDF
if (type != AllocationDescriptorType.SHORT && type != AllocationDescriptorType.LONG)
return null;
List<LongAllocationDescriptor> result = [];
ushort partition = (ushort)(key >> 32);
byte[] descriptors = fileEntry.AllocationDescriptors;
int length = descriptors.Length;
for (int extents = 0; extents <= MaximumAllocationExtents; extents++)
{
long nextKey = -1;
int size = type == AllocationDescriptorType.SHORT ? 8 : 16;
for (int offset = 0; offset + size <= length; offset += size)
{
var ad = new LongAllocationDescriptor();
ad.ExtentLength = descriptors.ToUInt32LittleEndian(offset);
if (type == AllocationDescriptorType.SHORT)
{
ad.ExtentLocation.LogicalBlockNumber = descriptors.ToUInt32LittleEndian(offset + 4);
ad.ExtentLocation.PartitionReferenceNumber = partition;
}
else
{
ad.ExtentLocation.LogicalBlockNumber = descriptors.ToUInt32LittleEndian(offset + 4);
ad.ExtentLocation.PartitionReferenceNumber = descriptors.ToUInt16LittleEndian(offset + 8);
for (int i = 0; i < 6; i++)
{
ad.ImplementationUse[i] = descriptors[offset + 10 + i];
}
}
// A zero length terminates the allocation descriptors
if (GetExtentLength(ad.ExtentLength) == 0)
return result;
if (GetExtentType(ad.ExtentLength) == ExtentType.NEXT_EXTENT)
{
nextKey = ad.ExtentLocation.GetKey();
break;
}
result.Add(ad);
}
if (nextKey < 0)
return result;
if (!volume.AllocationExtentDescriptors.TryGetValue(nextKey, out var aed))
{
missingKey = nextKey;
return null;
}
partition = (ushort)(nextKey >> 32);
descriptors = aed.AllocationDescriptors;
length = descriptors.Length;
}
// Too many continuation extents, assume a loop
return null;
}
/// <summary>
/// Check a descriptor tag: checksum, version, CRC (when it fits into the data) and location
/// </summary>
/// <remarks>ECMA-167 3/7.2; receiving systems allow descriptor version 2 or 3 (3/7.2.2)</remarks>
/// <param name="data">Data holding the descriptor</param>
/// <param name="offset">Offset of the descriptor</param>
/// <param name="available">Number of bytes of the descriptor available from the offset</param>
/// <param name="location">Expected tag location (logical sector or logical block number)</param>
/// <returns>True if the tag is valid, false otherwise</returns>
public static bool IsValidDescriptor(this byte[] data, int offset, int available, uint location)
{
byte checksum = 0;
for (int i = 0; i < 16; i++)
{
if (i != 4)
checksum += data[offset + i];
}
if (checksum != data[offset + 4])
return false;
ushort version = data.ToUInt16LittleEndian(offset + 2);
if (version != 2 && version != 3)
return false;
ushort crcLength = data.ToUInt16LittleEndian(offset + 10);
if (16 + crcLength <= available)
{
if (ComputeCrc(data, offset + 16, crcLength) != data.ToUInt16LittleEndian(offset + 8))
return false;
}
return data.ToUInt32LittleEndian(offset + 12) == location;
}
/// <summary>
/// Decode OSTA Compressed Unicode d-characters (first byte = compression ID)
/// </summary>
/// <param name="data">Encoded characters</param>
/// <returns>Decoded string, empty for no characters, null for an unknown compression ID</returns>
public static string? DecodeDCharacters(this byte[]? data)
=> DecodeDCharacters(data, data?.Length ?? 0);
/// <summary>
/// Decode an OSTA Compressed Unicode dstring (last byte = number of used bytes)
/// </summary>
/// <param name="data">Encoded dstring field</param>
/// <returns>Decoded string, empty for no characters, null for an unknown compression ID</returns>
public static string? DecodeDString(this byte[]? data)
{
if (data is null || data.Length == 0)
return string.Empty;
int length = data[data.Length - 1];
if (length > data.Length - 1)
length = data.Length - 1;
return DecodeDCharacters(data, length);
}
/// <summary>
/// Decode the first length bytes of OSTA Compressed Unicode d-characters
/// </summary>
private static string? DecodeDCharacters(byte[]? data, int length)
{
if (data is null || length == 0)
return string.Empty;
// 254 and 255 are the same encodings, used by UDF for deleted entries
byte compressionId = data[0];
var builder = new StringBuilder(length);
if (compressionId == Constants.CompressionId8 || compressionId == 254)
{
for (int i = 1; i < length; i++)
{
builder.Append((char)data[i]);
}
}
else if (compressionId == Constants.CompressionId16 || compressionId == 255)
{
for (int i = 1; i + 1 < length; i += 2)
{
builder.Append((char)data.ToUInt16BigEndian(i));
}
}
else
{
return null;
}
return builder.ToString();
}
// TODO: Replace with CRC-16/XMODEM from Hashing once its CRC code is faster
/// <summary>
/// Compute the CRC-ITU-T of a range of bytes
/// </summary>
private static ushort ComputeCrc(byte[] data, int offset, int length)
{
ushort crc = 0;
for (int i = offset; i < offset + length; i++)
{
crc = (ushort)(CrcTable[((crc >> 8) ^ data[i]) & 0xFF] ^ (crc << 8));
}
return crc;
}
/// <summary>
/// Create the CRC-ITU-T table
/// </summary>
private static ushort[] CreateCrcTable()
{
var table = new ushort[256];
for (int i = 0; i < 256; i++)
{
ushort crc = (ushort)(i << 8);
for (int bit = 0; bit < 8; bit++)
{
crc = (crc & 0x8000) != 0 ? (ushort)((crc << 1) ^ 0x1021) : (ushort)(crc << 1);
}
table[i] = crc;
}
return table;
}
}
}

View File

@@ -192,6 +192,9 @@ namespace SabreTools.Data.Models.Metadata
/// <remarks>(baddump|nodump|good|verified) "good"</remarks>
public ItemStatus? Status { get; set; }
/// <remarks>Possibly enum; OpenMSX.RomBase</remarks>
public string? StatusMisc { get; set; }
/// <remarks>ArchiveDotOrg.File</remarks>
public string? Summation { get; set; }
@@ -353,6 +356,7 @@ namespace SabreTools.Data.Models.Metadata
obj.SpamSum = SpamSum;
obj.Start = Start;
obj.Status = Status;
obj.StatusMisc = StatusMisc;
obj.Summation = Summation;
obj.TesseractOCR = TesseractOCR;
obj.TesseractOCRConverted = TesseractOCRConverted;

View File

@@ -8,17 +8,37 @@ namespace SabreTools.Data.Models.OpenMSX
/// </summary>
public abstract class RomBase
{
[XmlElement("start")]
public string? Start { get; set; }
/// <remarks>
/// "hash" in older versions, "sha1" in newer versions
/// </remarks>
[XmlAttribute("sha1")]
[XmlElement("hash")]
public string? SHA1 { get; set; }
/// <remarks>
/// Element in older versions and DTD, attribute in newer versions
/// </remarks>
[XmlAttribute("type")]
[XmlElement("type")]
public string? Type { get; set; }
/// <remarks>SHA-1 hash</remarks>
[XmlElement("hash")]
public string? Hash { get; set; }
/// <remarks>
/// Only present in newer versions
/// </remarks>
[XmlAttribute("status")]
public string? Status { get; set; }
/// <remarks>
/// Element in older versions and DTD, attribute in newer versions
/// </remarks>
[XmlAttribute("remark")]
[XmlElement("remark")]
public string? Remark { get; set; }
/// <remarks>
/// Only present in older versions
/// </remarks>
[XmlElement("start")]
public string? Start { get; set; }
}
}

View File

@@ -6,30 +6,63 @@ namespace SabreTools.Data.Models.OpenMSX
[XmlRoot("software")]
public class Software
{
/// <remarks>
/// Element in older versions and DTD, attribute in newer versions
/// </remarks>
[Required]
[XmlAttribute("title")]
[XmlElement("title")]
public string? Title { get; set; }
[XmlElement("genmsxid")]
public string? GenMSXID { get; set; }
/// <remarks>
/// Element in older versions and DTD, attribute in newer versions
/// </remarks>
[Required]
[XmlAttribute("system")]
[XmlElement("system")]
public string? System { get; set; }
/// <remarks>
/// Element in older versions and DTD, attribute in newer versions
/// </remarks>
[Required]
[XmlAttribute("company")]
[XmlElement("company")]
public string? Company { get; set; }
/// <remarks>
/// Element in older versions and DTD, attribute in newer versions
/// </remarks>
[Required]
[XmlAttribute("year")]
[XmlElement("year")]
public string? Year { get; set; }
/// <remarks>
/// Element in older versions and DTD, attribute in newer versions
/// </remarks>
[Required]
[XmlAttribute("country")]
[XmlElement("country")]
public string? Country { get; set; }
/// <remarks>
/// Element in older versions and DTD, attribute in newer versions
/// </remarks>
[XmlAttribute("genmsxid")]
[XmlElement("genmsxid")]
public string? GenMSXID { get; set; }
/// <remarks>
/// Present in older versions
/// </remarks>
[XmlElement("dump")]
public Dump[]? Dump { get; set; }
/// <remarks>
/// Present in newer versions
/// </remarks>
[XmlElement("rom")]
public RomBase[]? Rom { get; set; }
}
}

View File

@@ -22,7 +22,13 @@ namespace SabreTools.Data.Models.PKZIP
/// File data, appears after the encryption header
/// if it exists or after the local file header otherwise
/// </summary>
public byte[] FileData { get; set; } = [];
/// Not read into the model, see <see cref="FileDataOffset"/>
// public byte[] FileData { get; set; } = [];
/// <summary>
/// Offset of the file data in the stream
/// </summary>
public long FileDataOffset { get; set; }
/// <summary>
/// Data descriptors, appears after the file data

View File

@@ -2,7 +2,7 @@
<PropertyGroup>
<!-- Assembly Properties -->
<TargetFrameworks>net20;net35;net40;net452;net462;net472;net48;netcoreapp3.1;net5.0;net6.0;net7.0;net8.0;net9.0;net10.0;netstandard2.0;netstandard2.1</TargetFrameworks>
<TargetFrameworks>net20;net35;net40;net452;net462;net472;net48;netcoreapp3.1;net5.0;net6.0;net7.0;net8.0;net9.0;net10.0;net11.0;netstandard2.0;netstandard2.1</TargetFrameworks>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<CheckEolTargetFramework>false</CheckEolTargetFramework>
<IncludeSourceRevisionInInformationalVersion>false</IncludeSourceRevisionInInformationalVersion>
@@ -12,7 +12,7 @@
<SuppressTfmSupportBuildWarnings>true</SuppressTfmSupportBuildWarnings>
<SymbolPackageFormat>snupkg</SymbolPackageFormat>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
<Version>3.2.0</Version>
<Version>3.3.0</Version>
<!-- Package Properties -->
<Authors>Matt Nadareski</Authors>
@@ -32,7 +32,7 @@
<ItemGroup>
<PackageReference Include="Newtonsoft.Json" Version="13.0.4" />
<PackageReference Include="SabreTools.IO" Version="2.3.0" />
<PackageReference Include="SabreTools.IO" Version="2.4.0" />
</ItemGroup>
</Project>

View File

@@ -0,0 +1,24 @@
namespace SabreTools.Data.Models.UDF
{
/// <summary>
/// UDF Allocation Extent Descriptor (4/14.5), continues the allocation descriptors of a File Entry
/// </summary>
/// <see href="https://ecma-international.org/wp-content/uploads/ECMA-167_3rd_edition_june_1997.pdf"/>
public sealed class AllocationExtentDescriptor : Descriptor
{
/// <summary>
/// Logical block of the previous allocation extent
/// </summary>
public uint PreviousAllocationExtentLocation { get; set; }
/// <summary>
/// Length of the allocation descriptors in bytes
/// </summary>
public uint LengthOfAllocationDescriptors { get; set; }
/// <summary>
/// Allocation descriptors, of the type given in the owning File Entry's ICB tag flags
/// </summary>
public byte[] AllocationDescriptors { get; set; } = [];
}
}

View File

@@ -0,0 +1,24 @@
namespace SabreTools.Data.Models.UDF
{
/// <summary>
/// UDF Anchor Volume Descriptor Pointer (3/10.2), recorded at sector 256 (and at the end of the volume)
/// </summary>
/// <see href="https://ecma-international.org/wp-content/uploads/ECMA-167_3rd_edition_june_1997.pdf"/>
public sealed class AnchorVolumeDescriptorPointer : Descriptor
{
/// <summary>
/// Extent of the Main Volume Descriptor Sequence
/// </summary>
public ExtentDescriptor MainVolumeDescriptorSequenceExtent { get; set; } = new();
/// <summary>
/// Extent of the Reserve Volume Descriptor Sequence
/// </summary>
public ExtentDescriptor ReserveVolumeDescriptorSequenceExtent { get; set; } = new();
/// <summary>
/// Reserved, 480 bytes of 0x00
/// </summary>
public byte[] Reserved { get; set; } = new byte[480];
}
}

View File

@@ -0,0 +1,20 @@
namespace SabreTools.Data.Models.UDF
{
/// <summary>
/// UDF Character Set Specification (charspec, 1/7.2.1)
/// UDF requires CS0 with the information "OSTA Compressed Unicode"
/// </summary>
/// <see href="https://ecma-international.org/wp-content/uploads/ECMA-167_3rd_edition_june_1997.pdf"/>
public sealed class CharacterSetSpecification
{
/// <summary>
/// Character set type (0 = CS0)
/// </summary>
public byte CharacterSetType { get; set; }
/// <summary>
/// Character set information, padded with 0x00
/// </summary>
public byte[] CharacterSetInformation { get; set; } = new byte[63];
}
}

View File

@@ -0,0 +1,126 @@
namespace SabreTools.Data.Models.UDF
{
/// <summary>
/// UDF / ECMA-167 constant values and arrays
/// </summary>
/// <see href="https://ecma-international.org/wp-content/uploads/ECMA-167_3rd_edition_june_1997.pdf"/>
/// <see href="http://www.osta.org/specs/pdf/udf260.pdf"/>
public static class Constants
{
#region Volume Recognition Constants
/// <summary>
/// Size of a logical sector on optical media
/// </summary>
public const int SectorSize = 2048;
/// <summary>
/// Sector of the first Volume Structure Descriptor (after the 32KiB System Area)
/// </summary>
public const int VolumeRecognitionSequenceSector = 16;
/// <summary>
/// Sector of the first Anchor Volume Descriptor Pointer
/// </summary>
public const int AnchorVolumeDescriptorPointerSector = 256;
/// <summary>
/// Beginning Extended Area Descriptor identifier, "BEA01"
/// </summary>
public static readonly byte[] BeginningExtendedAreaDescriptor = [0x42, 0x45, 0x41, 0x30, 0x31];
/// <summary>
/// NSR Descriptor identifier for ECMA-167 2nd edition (UDF 1.02 to 1.50), "NSR02"
/// </summary>
public static readonly byte[] NSR02 = [0x4E, 0x53, 0x52, 0x30, 0x32];
/// <summary>
/// NSR Descriptor identifier for ECMA-167 3rd edition (UDF 2.00 and later), "NSR03"
/// </summary>
public static readonly byte[] NSR03 = [0x4E, 0x53, 0x52, 0x30, 0x33];
/// <summary>
/// Terminating Extended Area Descriptor identifier, "TEA01"
/// </summary>
public static readonly byte[] TerminatingExtendedAreaDescriptor = [0x54, 0x45, 0x41, 0x30, 0x31];
/// <summary>
/// ISO 9660 volume descriptor identifier, "CD001" (bridge discs)
/// </summary>
public static readonly byte[] CD001 = [0x43, 0x44, 0x30, 0x30, 0x31];
/// <summary>
/// Boot Descriptor identifier, "BOOT2"
/// </summary>
public static readonly byte[] BOOT2 = [0x42, 0x4F, 0x4F, 0x54, 0x32];
/// <summary>
/// ECMA-168 volume descriptor identifier, "CDW02"
/// </summary>
public static readonly byte[] CDW02 = [0x43, 0x44, 0x57, 0x30, 0x32];
#endregion
#region Partition Map Constants
/// <summary>
/// Type 1 Partition Map type
/// </summary>
public const byte Type1PartitionMap = 1;
/// <summary>
/// Type 2 Partition Map type
/// </summary>
public const byte Type2PartitionMap = 2;
/// <summary>
/// Partition type identifier of a UDF Metadata Partition (UDF 2.50 and later)
/// </summary>
public const string MetadataPartitionIdentifier = "*UDF Metadata Partition";
/// <summary>
/// Partition type identifier of a UDF Sparable Partition
/// </summary>
public const string SparablePartitionIdentifier = "*UDF Sparable Partition";
/// <summary>
/// Partition type identifier of a UDF Virtual Partition
/// </summary>
public const string VirtualPartitionIdentifier = "*UDF Virtual Partition";
#endregion
#region Character Set Constants
/// <summary>
/// OSTA Compressed Unicode, 8 bits per character
/// </summary>
public const byte CompressionId8 = 8;
/// <summary>
/// OSTA Compressed Unicode, 16 bits per character (big-endian)
/// </summary>
public const byte CompressionId16 = 16;
#endregion
#region Allocation Descriptor Constants
/// <summary>
/// Mask of the extent length in an allocation descriptor (the top two bits hold the extent type)
/// </summary>
public const uint ExtentLengthMask = 0x3FFFFFFF;
/// <summary>
/// Shift of the extent type in an allocation descriptor's extent length
/// </summary>
public const int ExtentTypeShift = 30;
/// <summary>
/// Mask of the allocation descriptor type in an ICB tag's flags
/// </summary>
public const ushort AllocationDescriptorTypeMask = 0x0007;
#endregion
}
}

View File

@@ -0,0 +1,14 @@
namespace SabreTools.Data.Models.UDF
{
/// <summary>
/// Abstract UDF tagged descriptor, all start with a Descriptor Tag
/// </summary>
/// <see href="https://ecma-international.org/wp-content/uploads/ECMA-167_3rd_edition_june_1997.pdf"/>
public abstract class Descriptor
{
/// <summary>
/// Descriptor tag
/// </summary>
public DescriptorTag DescriptorTag { get; set; } = new();
}
}

View File

@@ -0,0 +1,50 @@
namespace SabreTools.Data.Models.UDF
{
/// <summary>
/// UDF Descriptor Tag (3/7.2), the first 16 bytes of every tagged descriptor
/// </summary>
/// <see href="https://ecma-international.org/wp-content/uploads/ECMA-167_3rd_edition_june_1997.pdf"/>
public sealed class DescriptorTag
{
/// <summary>
/// Type of the descriptor
/// </summary>
public TagIdentifier TagIdentifier { get; set; }
/// <summary>
/// Version of the descriptor, 2 (ECMA-167 2nd edition) or 3 (3rd edition)
/// </summary>
public ushort DescriptorVersion { get; set; }
/// <summary>
/// Sum modulo 256 of bytes 0-3 and 5-15 of the tag
/// </summary>
public byte TagChecksum { get; set; }
/// <summary>
/// Reserved, 0x00
/// </summary>
public byte Reserved { get; set; }
/// <summary>
/// Identification of a set of descriptors
/// </summary>
public ushort TagSerialNumber { get; set; }
/// <summary>
/// CRC-CCITT of the DescriptorCRCLength bytes following the tag
/// </summary>
public ushort DescriptorCRC { get; set; }
/// <summary>
/// Number of bytes after the tag covered by the DescriptorCRC
/// </summary>
public ushort DescriptorCRCLength { get; set; }
/// <summary>
/// Number of the logical sector (volume structures) or logical block (file structures)
/// in which the descriptor is recorded
/// </summary>
public uint TagLocation { get; set; }
}
}

View File

@@ -0,0 +1,24 @@
namespace SabreTools.Data.Models.UDF
{
/// <summary>
/// UDF Entity Identifier (regid, 1/7.4)
/// </summary>
/// <see href="https://ecma-international.org/wp-content/uploads/ECMA-167_3rd_edition_june_1997.pdf"/>
public sealed class EntityIdentifier
{
/// <summary>
/// Flags (bit 0: dirty, bit 1: protected)
/// </summary>
public byte Flags { get; set; }
/// <summary>
/// Identifier, padded with 0x00
/// </summary>
public byte[] Identifier { get; set; } = new byte[23];
/// <summary>
/// Identifier suffix (UDF domain identifiers: revision as a little-endian ushort, then domain flags)
/// </summary>
public byte[] IdentifierSuffix { get; set; } = new byte[8];
}
}

View File

@@ -0,0 +1,306 @@
using System;
namespace SabreTools.Data.Models.UDF
{
/// <summary>
/// Enum for DescriptorTag.TagIdentifier
/// </summary>
/// <see href="https://ecma-international.org/wp-content/uploads/ECMA-167_3rd_edition_june_1997.pdf"/>
public enum TagIdentifier : ushort
{
/// <summary>
/// Primary Volume Descriptor (3/10.1)
/// </summary>
PRIMARY_VOLUME_DESCRIPTOR = 1,
/// <summary>
/// Anchor Volume Descriptor Pointer (3/10.2)
/// </summary>
ANCHOR_VOLUME_DESCRIPTOR_POINTER = 2,
/// <summary>
/// Volume Descriptor Pointer (3/10.3)
/// </summary>
VOLUME_DESCRIPTOR_POINTER = 3,
/// <summary>
/// Implementation Use Volume Descriptor (3/10.4)
/// </summary>
IMPLEMENTATION_USE_VOLUME_DESCRIPTOR = 4,
/// <summary>
/// Partition Descriptor (3/10.5)
/// </summary>
PARTITION_DESCRIPTOR = 5,
/// <summary>
/// Logical Volume Descriptor (3/10.6)
/// </summary>
LOGICAL_VOLUME_DESCRIPTOR = 6,
/// <summary>
/// Unallocated Space Descriptor (3/10.8)
/// </summary>
UNALLOCATED_SPACE_DESCRIPTOR = 7,
/// <summary>
/// Terminating Descriptor (3/10.9)
/// </summary>
TERMINATING_DESCRIPTOR = 8,
/// <summary>
/// Logical Volume Integrity Descriptor (3/10.10)
/// </summary>
LOGICAL_VOLUME_INTEGRITY_DESCRIPTOR = 9,
/// <summary>
/// File Set Descriptor (4/14.1)
/// </summary>
FILE_SET_DESCRIPTOR = 256,
/// <summary>
/// File Identifier Descriptor (4/14.4)
/// </summary>
FILE_IDENTIFIER_DESCRIPTOR = 257,
/// <summary>
/// Allocation Extent Descriptor (4/14.5)
/// </summary>
ALLOCATION_EXTENT_DESCRIPTOR = 258,
/// <summary>
/// Indirect Entry (4/14.7)
/// </summary>
INDIRECT_ENTRY = 259,
/// <summary>
/// Terminal Entry (4/14.8)
/// </summary>
TERMINAL_ENTRY = 260,
/// <summary>
/// File Entry (4/14.9)
/// </summary>
FILE_ENTRY = 261,
/// <summary>
/// Extended Attribute Header Descriptor (4/14.10.1)
/// </summary>
EXTENDED_ATTRIBUTE_HEADER_DESCRIPTOR = 262,
/// <summary>
/// Unallocated Space Entry (4/14.11)
/// </summary>
UNALLOCATED_SPACE_ENTRY = 263,
/// <summary>
/// Space Bitmap Descriptor (4/14.12)
/// </summary>
SPACE_BITMAP_DESCRIPTOR = 264,
/// <summary>
/// Partition Integrity Entry (4/14.13)
/// </summary>
PARTITION_INTEGRITY_ENTRY = 265,
/// <summary>
/// Extended File Entry (4/14.17)
/// </summary>
EXTENDED_FILE_ENTRY = 266,
}
/// <summary>
/// Enum for FileIdentifierDescriptor.FileCharacteristics (4/14.4.3)
/// Bits 5-7: Reserved: 0
/// </summary>
/// <see href="https://ecma-international.org/wp-content/uploads/ECMA-167_3rd_edition_june_1997.pdf"/>
[Flags]
public enum FileCharacteristics : byte
{
/// <summary>
/// No characteristics set
/// </summary>
NONE = 0x00,
/// <summary>
/// The existence of the file shall not be made known to the user (hidden)
/// </summary>
EXISTENCE = 0x01,
/// <summary>
/// The file is a directory
/// </summary>
DIRECTORY = 0x02,
/// <summary>
/// The descriptor shall be ignored (deleted)
/// </summary>
DELETED = 0x04,
/// <summary>
/// The ICB identifies the parent directory
/// </summary>
PARENT = 0x08,
/// <summary>
/// The file contains implementation use data (UDF: stream directory metadata)
/// </summary>
METADATA = 0x10,
}
/// <summary>
/// Enum for ICBTag.FileType (4/14.6.6, UDF 2.3.5.4)
/// </summary>
/// <see href="https://ecma-international.org/wp-content/uploads/ECMA-167_3rd_edition_june_1997.pdf"/>
public enum FileType : byte
{
/// <summary>
/// Not specified
/// </summary>
UNSPECIFIED = 0,
/// <summary>
/// Unallocated Space Entry
/// </summary>
UNALLOCATED_SPACE_ENTRY = 1,
/// <summary>
/// Partition Integrity Entry
/// </summary>
PARTITION_INTEGRITY_ENTRY = 2,
/// <summary>
/// Indirect Entry
/// </summary>
INDIRECT_ENTRY = 3,
/// <summary>
/// Directory
/// </summary>
DIRECTORY = 4,
/// <summary>
/// Sequence of bytes (a regular file)
/// </summary>
FILE = 5,
/// <summary>
/// Block special device
/// </summary>
BLOCK_DEVICE = 6,
/// <summary>
/// Character special device
/// </summary>
CHARACTER_DEVICE = 7,
/// <summary>
/// Extended attributes
/// </summary>
EXTENDED_ATTRIBUTES = 8,
/// <summary>
/// FIFO file
/// </summary>
FIFO = 9,
/// <summary>
/// Socket
/// </summary>
SOCKET = 10,
/// <summary>
/// Terminal Entry
/// </summary>
TERMINAL_ENTRY = 11,
/// <summary>
/// Symbolic link
/// </summary>
SYMBOLIC_LINK = 12,
/// <summary>
/// Stream directory
/// </summary>
STREAM_DIRECTORY = 13,
/// <summary>
/// UDF Virtual Allocation Table
/// </summary>
VIRTUAL_ALLOCATION_TABLE = 248,
/// <summary>
/// UDF Real-Time file
/// </summary>
REAL_TIME_FILE = 249,
/// <summary>
/// UDF Metadata File
/// </summary>
METADATA_FILE = 250,
/// <summary>
/// UDF Metadata Mirror File
/// </summary>
METADATA_MIRROR_FILE = 251,
/// <summary>
/// UDF Metadata Bitmap File
/// </summary>
METADATA_BITMAP_FILE = 252,
}
/// <summary>
/// Enum for the allocation descriptor type in ICBTag.Flags (4/14.6.8)
/// </summary>
/// <see href="https://ecma-international.org/wp-content/uploads/ECMA-167_3rd_edition_june_1997.pdf"/>
public enum AllocationDescriptorType : byte
{
/// <summary>
/// Short Allocation Descriptors (4/14.14.1)
/// </summary>
SHORT = 0,
/// <summary>
/// Long Allocation Descriptors (4/14.14.2)
/// </summary>
LONG = 1,
/// <summary>
/// Extended Allocation Descriptors (4/14.14.3)
/// </summary>
EXTENDED = 2,
/// <summary>
/// The file data is embedded in the allocation descriptors field
/// </summary>
EMBEDDED = 3,
}
/// <summary>
/// Enum for the extent type in the top two bits of an allocation descriptor's extent length (4/14.14.1.1)
/// </summary>
/// <see href="https://ecma-international.org/wp-content/uploads/ECMA-167_3rd_edition_june_1997.pdf"/>
public enum ExtentType : byte
{
/// <summary>
/// Extent recorded and allocated
/// </summary>
RECORDED_AND_ALLOCATED = 0,
/// <summary>
/// Extent not recorded but allocated
/// </summary>
NOT_RECORDED_BUT_ALLOCATED = 1,
/// <summary>
/// Extent not recorded and not allocated
/// </summary>
NOT_RECORDED_NOT_ALLOCATED = 2,
/// <summary>
/// The extent is the next extent of allocation descriptors
/// </summary>
NEXT_EXTENT = 3,
}
}

View File

@@ -0,0 +1,30 @@
namespace SabreTools.Data.Models.UDF
{
/// <summary>
/// UDF Extended File Entry (4/14.17, UDF 2.00 and later), a File Entry with object size,
/// creation time and a stream directory
/// </summary>
/// <see href="https://ecma-international.org/wp-content/uploads/ECMA-167_3rd_edition_june_1997.pdf"/>
public sealed class ExtendedFileEntry : FileEntry
{
/// <summary>
/// Size of the file and its streams in bytes
/// </summary>
public ulong ObjectSize { get; set; }
/// <summary>
/// Creation date and time
/// </summary>
public Timestamp CreationDateAndTime { get; set; } = new();
/// <summary>
/// Reserved, 0x00000000
/// </summary>
public uint Reserved { get; set; }
/// <summary>
/// ICB of the stream directory, length 0 if none
/// </summary>
public LongAllocationDescriptor StreamDirectoryICB { get; set; } = new();
}
}

View File

@@ -0,0 +1,19 @@
namespace SabreTools.Data.Models.UDF
{
/// <summary>
/// UDF Extent Descriptor (extent_ad, 3/7.1), an extent of logical sectors
/// </summary>
/// <see href="https://ecma-international.org/wp-content/uploads/ECMA-167_3rd_edition_june_1997.pdf"/>
public sealed class ExtentDescriptor
{
/// <summary>
/// Length of the extent in bytes
/// </summary>
public uint ExtentLength { get; set; }
/// <summary>
/// Logical sector number of the start of the extent
/// </summary>
public uint ExtentLocation { get; set; }
}
}

View File

@@ -0,0 +1,115 @@
namespace SabreTools.Data.Models.UDF
{
/// <summary>
/// UDF File Entry (4/14.9), the ICB describing a file or directory and where its data is recorded
/// </summary>
/// <see href="https://ecma-international.org/wp-content/uploads/ECMA-167_3rd_edition_june_1997.pdf"/>
public class FileEntry : Descriptor
{
/// <summary>
/// ICB tag
/// </summary>
public ICBTag ICBTag { get; set; } = new();
/// <summary>
/// User ID
/// </summary>
public uint Uid { get; set; }
/// <summary>
/// Group ID
/// </summary>
public uint Gid { get; set; }
/// <summary>
/// Permissions
/// </summary>
public uint Permissions { get; set; }
/// <summary>
/// Number of File Identifier Descriptors identifying this ICB
/// </summary>
public ushort FileLinkCount { get; set; }
/// <summary>
/// Record format
/// </summary>
public byte RecordFormat { get; set; }
/// <summary>
/// Record display attributes
/// </summary>
public byte RecordDisplayAttributes { get; set; }
/// <summary>
/// Record length
/// </summary>
public uint RecordLength { get; set; }
/// <summary>
/// Size of the file in bytes
/// </summary>
public ulong InformationLength { get; set; }
/// <summary>
/// Number of recorded logical blocks
/// </summary>
public ulong LogicalBlocksRecorded { get; set; }
/// <summary>
/// Access date and time
/// </summary>
public Timestamp AccessDateAndTime { get; set; } = new();
/// <summary>
/// Modification date and time
/// </summary>
public Timestamp ModificationDateAndTime { get; set; } = new();
/// <summary>
/// Attribute date and time
/// </summary>
public Timestamp AttributeDateAndTime { get; set; } = new();
/// <summary>
/// Checkpoint
/// </summary>
public uint Checkpoint { get; set; }
/// <summary>
/// ICB of the extended attributes, length 0 if none
/// </summary>
public LongAllocationDescriptor ExtendedAttributeICB { get; set; } = new();
/// <summary>
/// Implementation identifier
/// </summary>
public EntityIdentifier ImplementationIdentifier { get; set; } = new();
/// <summary>
/// Unique ID
/// </summary>
public ulong UniqueID { get; set; }
/// <summary>
/// Length of the extended attributes in bytes
/// </summary>
public uint LengthOfExtendedAttributes { get; set; }
/// <summary>
/// Length of the allocation descriptors in bytes
/// </summary>
public uint LengthOfAllocationDescriptors { get; set; }
/// <summary>
/// Extended attributes
/// </summary>
public byte[] ExtendedAttributes { get; set; } = [];
/// <summary>
/// Allocation descriptors of the type given in the ICB tag flags,
/// or the file data itself if the type is AllocationDescriptorType.EMBEDDED
/// </summary>
public byte[] AllocationDescriptors { get; set; } = [];
}
}

View File

@@ -0,0 +1,49 @@
namespace SabreTools.Data.Models.UDF
{
/// <summary>
/// UDF File Identifier Descriptor (4/14.4), a directory entry naming a file or directory and pointing to its ICB
/// </summary>
/// <see href="https://ecma-international.org/wp-content/uploads/ECMA-167_3rd_edition_june_1997.pdf"/>
public sealed class FileIdentifierDescriptor : Descriptor
{
/// <summary>
/// File version number
/// </summary>
public ushort FileVersionNumber { get; set; }
/// <summary>
/// File characteristics
/// </summary>
public FileCharacteristics FileCharacteristics { get; set; }
/// <summary>
/// Length of the file identifier in bytes
/// </summary>
public byte LengthOfFileIdentifier { get; set; }
/// <summary>
/// ICB of the file or directory
/// </summary>
public LongAllocationDescriptor ICB { get; set; } = new();
/// <summary>
/// Length of the implementation use in bytes
/// </summary>
public ushort LengthOfImplementationUse { get; set; }
/// <summary>
/// Implementation use
/// </summary>
public byte[] ImplementationUse { get; set; } = [];
/// <summary>
/// File identifier, d-characters (first byte = compression ID), empty for the parent directory
/// </summary>
public byte[] FileIdentifier { get; set; } = [];
/// <summary>
/// Padding to a multiple of 4 bytes
/// </summary>
public byte[] Padding { get; set; } = [];
}
}

View File

@@ -0,0 +1,99 @@
namespace SabreTools.Data.Models.UDF
{
/// <summary>
/// UDF File Set Descriptor (4/14.1), the location of the root directory
/// </summary>
/// <see href="https://ecma-international.org/wp-content/uploads/ECMA-167_3rd_edition_june_1997.pdf"/>
public sealed class FileSetDescriptor : Descriptor
{
/// <summary>
/// Recording date and time
/// </summary>
public Timestamp RecordingDateAndTime { get; set; } = new();
/// <summary>
/// Interchange level
/// </summary>
public ushort InterchangeLevel { get; set; }
/// <summary>
/// Maximum interchange level
/// </summary>
public ushort MaximumInterchangeLevel { get; set; }
/// <summary>
/// Character set list
/// </summary>
public uint CharacterSetList { get; set; }
/// <summary>
/// Maximum character set list
/// </summary>
public uint MaximumCharacterSetList { get; set; }
/// <summary>
/// File set number
/// </summary>
public uint FileSetNumber { get; set; }
/// <summary>
/// File Set Descriptor number
/// </summary>
public uint FileSetDescriptorNumber { get; set; }
/// <summary>
/// Logical volume identifier character set
/// </summary>
public CharacterSetSpecification LogicalVolumeIdentifierCharacterSet { get; set; } = new();
/// <summary>
/// Logical volume identifier, dstring (last byte = length)
/// </summary>
public byte[] LogicalVolumeIdentifier { get; set; } = new byte[128];
/// <summary>
/// File set character set
/// </summary>
public CharacterSetSpecification FileSetCharacterSet { get; set; } = new();
/// <summary>
/// File set identifier, dstring (last byte = length)
/// </summary>
public byte[] FileSetIdentifier { get; set; } = new byte[32];
/// <summary>
/// Copyright file identifier, dstring (last byte = length)
/// </summary>
public byte[] CopyrightFileIdentifier { get; set; } = new byte[32];
/// <summary>
/// Abstract file identifier, dstring (last byte = length)
/// </summary>
public byte[] AbstractFileIdentifier { get; set; } = new byte[32];
/// <summary>
/// ICB of the root directory
/// </summary>
public LongAllocationDescriptor RootDirectoryICB { get; set; } = new();
/// <summary>
/// Domain identifier
/// </summary>
public EntityIdentifier DomainIdentifier { get; set; } = new();
/// <summary>
/// Next extent of File Set Descriptors, length 0 if none
/// </summary>
public LongAllocationDescriptor NextExtent { get; set; } = new();
/// <summary>
/// ICB of the system stream directory (ECMA-167 3rd edition), length 0 if none
/// </summary>
public LongAllocationDescriptor SystemStreamDirectoryICB { get; set; } = new();
/// <summary>
/// Reserved, 32 bytes of 0x00
/// </summary>
public byte[] Reserved { get; set; } = new byte[32];
}
}

View File

@@ -0,0 +1,15 @@
namespace SabreTools.Data.Models.UDF
{
/// <summary>
/// UDF tagged descriptor of a type that is not parsed further
/// (e.g. Implementation Use Volume Descriptor, Unallocated Space Descriptor, Terminating Descriptor)
/// </summary>
/// <see href="https://ecma-international.org/wp-content/uploads/ECMA-167_3rd_edition_june_1997.pdf"/>
public sealed class GenericDescriptor : Descriptor
{
/// <summary>
/// Contents of the descriptor after the tag, up to the end of the logical sector
/// </summary>
public byte[] Data { get; set; } = [];
}
}

View File

@@ -0,0 +1,49 @@
namespace SabreTools.Data.Models.UDF
{
/// <summary>
/// UDF Information Control Block Tag (icbtag, 4/14.6)
/// </summary>
/// <see href="https://ecma-international.org/wp-content/uploads/ECMA-167_3rd_edition_june_1997.pdf"/>
public sealed class ICBTag
{
/// <summary>
/// Prior recorded number of direct entries
/// </summary>
public uint PriorRecordedNumberOfDirectEntries { get; set; }
/// <summary>
/// Strategy type (UDF: 4, or 4096 for write-once media)
/// </summary>
public ushort StrategyType { get; set; }
/// <summary>
/// Strategy parameter, 2 bytes
/// </summary>
public byte[] StrategyParameter { get; set; } = new byte[2];
/// <summary>
/// Maximum number of entries
/// </summary>
public ushort MaximumNumberOfEntries { get; set; }
/// <summary>
/// Reserved, 0x00
/// </summary>
public byte Reserved { get; set; }
/// <summary>
/// File type
/// </summary>
public FileType FileType { get; set; }
/// <summary>
/// Parent ICB location
/// </summary>
public LogicalBlockAddress ParentICBLocation { get; set; } = new();
/// <summary>
/// Flags, the lower 3 bits hold the AllocationDescriptorType
/// </summary>
public ushort Flags { get; set; }
}
}

View File

@@ -0,0 +1,19 @@
namespace SabreTools.Data.Models.UDF
{
/// <summary>
/// UDF Recorded Address (lb_addr, 4/7.1), a logical block within a partition
/// </summary>
/// <see href="https://ecma-international.org/wp-content/uploads/ECMA-167_3rd_edition_june_1997.pdf"/>
public sealed class LogicalBlockAddress
{
/// <summary>
/// Logical block number, relative to the start of the partition
/// </summary>
public uint LogicalBlockNumber { get; set; }
/// <summary>
/// Index of the partition map in the Logical Volume Descriptor
/// </summary>
public ushort PartitionReferenceNumber { get; set; }
}
}

View File

@@ -0,0 +1,69 @@
namespace SabreTools.Data.Models.UDF
{
/// <summary>
/// UDF Logical Volume Descriptor (3/10.6), block size, partition maps and the location of the File Set Descriptor
/// </summary>
/// <see href="https://ecma-international.org/wp-content/uploads/ECMA-167_3rd_edition_june_1997.pdf"/>
public sealed class LogicalVolumeDescriptor : Descriptor
{
/// <summary>
/// Volume Descriptor Sequence Number, the highest number prevails
/// </summary>
public uint VolumeDescriptorSequenceNumber { get; set; }
/// <summary>
/// Descriptor character set
/// </summary>
public CharacterSetSpecification DescriptorCharacterSet { get; set; } = new();
/// <summary>
/// Logical volume identifier, dstring (last byte = length)
/// </summary>
public byte[] LogicalVolumeIdentifier { get; set; } = new byte[128];
/// <summary>
/// Size of a logical block in bytes
/// </summary>
public uint LogicalBlockSize { get; set; }
/// <summary>
/// Domain identifier ("*OSTA UDF Compliant", suffix holds the UDF revision)
/// </summary>
public EntityIdentifier DomainIdentifier { get; set; } = new();
/// <summary>
/// Logical volume contents use (UDF: extent of the File Set Descriptor)
/// </summary>
public LongAllocationDescriptor LogicalVolumeContentsUse { get; set; } = new();
/// <summary>
/// Length of the partition maps in bytes
/// </summary>
public uint MapTableLength { get; set; }
/// <summary>
/// Number of partition maps
/// </summary>
public uint NumberOfPartitionMaps { get; set; }
/// <summary>
/// Implementation identifier
/// </summary>
public EntityIdentifier ImplementationIdentifier { get; set; } = new();
/// <summary>
/// Implementation use, 128 bytes
/// </summary>
public byte[] ImplementationUse { get; set; } = new byte[128];
/// <summary>
/// Extent of the Logical Volume Integrity Sequence
/// </summary>
public ExtentDescriptor IntegritySequenceExtent { get; set; } = new();
/// <summary>
/// Partition maps, a partition reference number is the index into this array
/// </summary>
public PartitionMap[] PartitionMaps { get; set; } = [];
}
}

View File

@@ -0,0 +1,25 @@
namespace SabreTools.Data.Models.UDF
{
/// <summary>
/// UDF Long Allocation Descriptor (long_ad, 4/14.14.2), an extent within any partition of the logical volume
/// </summary>
/// <see href="https://ecma-international.org/wp-content/uploads/ECMA-167_3rd_edition_june_1997.pdf"/>
public sealed class LongAllocationDescriptor
{
/// <summary>
/// Length of the extent in bytes (lower 30 bits) and the ExtentType (upper 2 bits)
/// A length of 0 terminates a sequence of allocation descriptors
/// </summary>
public uint ExtentLength { get; set; }
/// <summary>
/// Logical block address of the start of the extent
/// </summary>
public LogicalBlockAddress ExtentLocation { get; set; } = new();
/// <summary>
/// Implementation use (UDF: flags and unique ID bits)
/// </summary>
public byte[] ImplementationUse { get; set; } = new byte[6];
}
}

View File

@@ -0,0 +1,45 @@
namespace SabreTools.Data.Models.UDF
{
/// <summary>
/// UDF Metadata Partition Map (UDF 2.2.10, UDF 2.50 and later), a partition whose logical blocks
/// are the contents of the Metadata File recorded in the underlying partition
/// </summary>
/// <see href="http://www.osta.org/specs/pdf/udf260.pdf"/>
public sealed class MetadataPartitionMap : Type2PartitionMap
{
/// <summary>
/// Logical block of the Metadata File's File Entry in the underlying partition
/// </summary>
public uint MetadataFileLocation { get; set; }
/// <summary>
/// Logical block of the Metadata Mirror File's File Entry in the underlying partition
/// </summary>
public uint MetadataMirrorFileLocation { get; set; }
/// <summary>
/// Logical block of the Metadata Bitmap File's File Entry, 0xFFFFFFFF if not recorded
/// </summary>
public uint MetadataBitmapFileLocation { get; set; }
/// <summary>
/// Allocation unit size in logical blocks
/// </summary>
public uint AllocationUnitSize { get; set; }
/// <summary>
/// Alignment unit size in logical blocks
/// </summary>
public ushort AlignmentUnitSize { get; set; }
/// <summary>
/// Flags (bit 0: duplicate metadata flag)
/// </summary>
public byte Flags { get; set; }
/// <summary>
/// Reserved, 5 bytes of 0x00
/// </summary>
public byte[] Reserved2 { get; set; } = new byte[5];
}
}

View File

@@ -0,0 +1,64 @@
namespace SabreTools.Data.Models.UDF
{
/// <summary>
/// UDF Partition Descriptor (3/10.5), the physical location of a partition
/// </summary>
/// <see href="https://ecma-international.org/wp-content/uploads/ECMA-167_3rd_edition_june_1997.pdf"/>
public sealed class PartitionDescriptor : Descriptor
{
/// <summary>
/// Volume Descriptor Sequence Number, the highest number prevails
/// </summary>
public uint VolumeDescriptorSequenceNumber { get; set; }
/// <summary>
/// Partition flags (bit 0: space is allocated)
/// </summary>
public ushort PartitionFlags { get; set; }
/// <summary>
/// Partition number, referenced by the partition maps
/// </summary>
public ushort PartitionNumber { get; set; }
/// <summary>
/// Partition contents ("+NSR02" or "+NSR03" for UDF)
/// </summary>
public EntityIdentifier PartitionContents { get; set; } = new();
/// <summary>
/// Partition contents use (Partition Header Descriptor for NSR), 128 bytes
/// </summary>
public byte[] PartitionContentsUse { get; set; } = new byte[128];
/// <summary>
/// Access type (1 = read-only, 2 = write-once, 3 = rewritable, 4 = overwritable)
/// </summary>
public uint AccessType { get; set; }
/// <summary>
/// Logical sector number of the first sector of the partition
/// </summary>
public uint PartitionStartingLocation { get; set; }
/// <summary>
/// Length of the partition in logical sectors
/// </summary>
public uint PartitionLength { get; set; }
/// <summary>
/// Implementation identifier
/// </summary>
public EntityIdentifier ImplementationIdentifier { get; set; } = new();
/// <summary>
/// Implementation use, 128 bytes
/// </summary>
public byte[] ImplementationUse { get; set; } = new byte[128];
/// <summary>
/// Reserved, 156 bytes of 0x00
/// </summary>
public byte[] Reserved { get; set; } = new byte[156];
}
}

View File

@@ -0,0 +1,19 @@
namespace SabreTools.Data.Models.UDF
{
/// <summary>
/// Abstract UDF Partition Map (3/10.7)
/// </summary>
/// <see href="https://ecma-international.org/wp-content/uploads/ECMA-167_3rd_edition_june_1997.pdf"/>
public abstract class PartitionMap
{
/// <summary>
/// Partition map type (1 or 2)
/// </summary>
public byte PartitionMapType { get; set; }
/// <summary>
/// Length of the partition map in bytes
/// </summary>
public byte PartitionMapLength { get; set; }
}
}

View File

@@ -0,0 +1,114 @@
namespace SabreTools.Data.Models.UDF
{
/// <summary>
/// UDF Primary Volume Descriptor (3/10.1)
/// </summary>
/// <see href="https://ecma-international.org/wp-content/uploads/ECMA-167_3rd_edition_june_1997.pdf"/>
public sealed class PrimaryVolumeDescriptor : Descriptor
{
/// <summary>
/// Volume Descriptor Sequence Number, the highest number prevails
/// </summary>
public uint VolumeDescriptorSequenceNumber { get; set; }
/// <summary>
/// Primary Volume Descriptor Number
/// </summary>
public uint PrimaryVolumeDescriptorNumber { get; set; }
/// <summary>
/// Volume identifier, dstring (last byte = length)
/// </summary>
public byte[] VolumeIdentifier { get; set; } = new byte[32];
/// <summary>
/// Volume sequence number
/// </summary>
public ushort VolumeSequenceNumber { get; set; }
/// <summary>
/// Maximum volume sequence number
/// </summary>
public ushort MaximumVolumeSequenceNumber { get; set; }
/// <summary>
/// Interchange level
/// </summary>
public ushort InterchangeLevel { get; set; }
/// <summary>
/// Maximum interchange level
/// </summary>
public ushort MaximumInterchangeLevel { get; set; }
/// <summary>
/// Character set list
/// </summary>
public uint CharacterSetList { get; set; }
/// <summary>
/// Maximum character set list
/// </summary>
public uint MaximumCharacterSetList { get; set; }
/// <summary>
/// Volume set identifier, dstring (last byte = length)
/// </summary>
public byte[] VolumeSetIdentifier { get; set; } = new byte[128];
/// <summary>
/// Descriptor character set
/// </summary>
public CharacterSetSpecification DescriptorCharacterSet { get; set; } = new();
/// <summary>
/// Explanatory character set
/// </summary>
public CharacterSetSpecification ExplanatoryCharacterSet { get; set; } = new();
/// <summary>
/// Extent of the volume abstract
/// </summary>
public ExtentDescriptor VolumeAbstract { get; set; } = new();
/// <summary>
/// Extent of the volume copyright notice
/// </summary>
public ExtentDescriptor VolumeCopyrightNotice { get; set; } = new();
/// <summary>
/// Application identifier
/// </summary>
public EntityIdentifier ApplicationIdentifier { get; set; } = new();
/// <summary>
/// Recording date and time
/// </summary>
public Timestamp RecordingDateAndTime { get; set; } = new();
/// <summary>
/// Implementation identifier
/// </summary>
public EntityIdentifier ImplementationIdentifier { get; set; } = new();
/// <summary>
/// Implementation use, 64 bytes
/// </summary>
public byte[] ImplementationUse { get; set; } = new byte[64];
/// <summary>
/// Predecessor Volume Descriptor Sequence location
/// </summary>
public uint PredecessorVolumeDescriptorSequenceLocation { get; set; }
/// <summary>
/// Flags (bit 0: volume set identification is common to all volumes of the set)
/// </summary>
public ushort Flags { get; set; }
/// <summary>
/// Reserved, 22 bytes of 0x00
/// </summary>
public byte[] Reserved { get; set; } = new byte[22];
}
}

View File

@@ -0,0 +1,20 @@
namespace SabreTools.Data.Models.UDF
{
/// <summary>
/// UDF Short Allocation Descriptor (short_ad, 4/14.14.1), an extent within the partition of the descriptor
/// </summary>
/// <see href="https://ecma-international.org/wp-content/uploads/ECMA-167_3rd_edition_june_1997.pdf"/>
public sealed class ShortAllocationDescriptor
{
/// <summary>
/// Length of the extent in bytes (lower 30 bits) and the ExtentType (upper 2 bits)
/// A length of 0 terminates a sequence of allocation descriptors
/// </summary>
public uint ExtentLength { get; set; }
/// <summary>
/// Logical block number of the start of the extent
/// </summary>
public uint ExtentPosition { get; set; }
}
}

View File

@@ -0,0 +1,60 @@
namespace SabreTools.Data.Models.UDF
{
/// <summary>
/// UDF Timestamp (1/7.3)
/// </summary>
/// <see href="https://ecma-international.org/wp-content/uploads/ECMA-167_3rd_edition_june_1997.pdf"/>
public sealed class Timestamp
{
/// <summary>
/// Type (upper 4 bits, 1 = local time) and time zone offset from UTC in minutes
/// (lower 12 bits, signed, -2047 = not specified)
/// </summary>
public ushort TypeAndTimezone { get; set; }
/// <summary>
/// Year, 1 to 9999
/// </summary>
public short Year { get; set; }
/// <summary>
/// Month, 1 to 12
/// </summary>
public byte Month { get; set; }
/// <summary>
/// Day, 1 to 31
/// </summary>
public byte Day { get; set; }
/// <summary>
/// Hour, 0 to 23
/// </summary>
public byte Hour { get; set; }
/// <summary>
/// Minute, 0 to 59
/// </summary>
public byte Minute { get; set; }
/// <summary>
/// Second, 0 to 59
/// </summary>
public byte Second { get; set; }
/// <summary>
/// Centiseconds, 0 to 99
/// </summary>
public byte Centiseconds { get; set; }
/// <summary>
/// Hundreds of microseconds, 0 to 99
/// </summary>
public byte HundredsOfMicroseconds { get; set; }
/// <summary>
/// Microseconds, 0 to 99
/// </summary>
public byte Microseconds { get; set; }
}
}

View File

@@ -0,0 +1,19 @@
namespace SabreTools.Data.Models.UDF
{
/// <summary>
/// UDF Type 1 Partition Map (3/10.7.2), logical blocks map 1:1 onto a Partition Descriptor's sectors
/// </summary>
/// <see href="https://ecma-international.org/wp-content/uploads/ECMA-167_3rd_edition_june_1997.pdf"/>
public sealed class Type1PartitionMap : PartitionMap
{
/// <summary>
/// Volume sequence number of the volume containing the partition
/// </summary>
public ushort VolumeSequenceNumber { get; set; }
/// <summary>
/// Partition number of the Partition Descriptor
/// </summary>
public ushort PartitionNumber { get; set; }
}
}

View File

@@ -0,0 +1,37 @@
namespace SabreTools.Data.Models.UDF
{
/// <summary>
/// UDF Type 2 Partition Map (3/10.7.3, UDF 2.2.8-2.2.10), a partition identified by its partition type identifier
/// (e.g. "*UDF Virtual Partition", "*UDF Sparable Partition", "*UDF Metadata Partition")
/// </summary>
/// <see href="https://ecma-international.org/wp-content/uploads/ECMA-167_3rd_edition_june_1997.pdf"/>
/// <see href="http://www.osta.org/specs/pdf/udf260.pdf"/>
public class Type2PartitionMap : PartitionMap
{
/// <summary>
/// Reserved, 2 bytes of 0x00
/// </summary>
public byte[] Reserved { get; set; } = new byte[2];
/// <summary>
/// Partition type identifier
/// </summary>
public EntityIdentifier PartitionTypeIdentifier { get; set; } = new();
/// <summary>
/// Volume sequence number of the volume containing the partition
/// </summary>
public ushort VolumeSequenceNumber { get; set; }
/// <summary>
/// Partition number of the underlying Partition Descriptor
/// </summary>
public ushort PartitionNumber { get; set; }
/// <summary>
/// Partition type specific data after the partition number, 24 bytes
/// Empty for partition types that are parsed further (MetadataPartitionMap)
/// </summary>
public byte[] PartitionTypeData { get; set; } = [];
}
}

View File

@@ -0,0 +1,52 @@
using System.Collections.Generic;
namespace SabreTools.Data.Models.UDF
{
/// <summary>
/// A volume (or disc image) with a UDF / ECMA-167 file system
/// </summary>
/// <remarks>
/// Logical sectors and logical blocks are 2048 bytes on optical media
/// File system structures are addressed by logical block address (partition reference number + logical block);
/// dictionaries are keyed by ((long)PartitionReferenceNumber &lt;&lt; 32) | LogicalBlockNumber
/// </remarks>
/// <see href="https://ecma-international.org/wp-content/uploads/ECMA-167_3rd_edition_june_1997.pdf"/>
/// <see href="http://www.osta.org/specs/pdf/udf260.pdf"/>
public sealed class Volume
{
/// <summary>
/// Volume Recognition Sequence starting at sector 16
/// </summary>
public VolumeStructureDescriptor[] VolumeRecognitionSequence { get; set; } = [];
/// <summary>
/// Anchor Volume Descriptor Pointer
/// </summary>
public AnchorVolumeDescriptorPointer? AnchorVolumeDescriptorPointer { get; set; }
/// <summary>
/// Volume Descriptor Sequence (Primary, Partition, Logical Volume, ... up to the Terminating Descriptor)
/// </summary>
public Descriptor[] VolumeDescriptorSequence { get; set; } = [];
/// <summary>
/// File Set Descriptor
/// </summary>
public FileSetDescriptor? FileSetDescriptor { get; set; }
/// <summary>
/// File Entries (and Extended File Entries) by logical block address
/// </summary>
public Dictionary<long, FileEntry> FileEntries { get; set; } = [];
/// <summary>
/// Allocation Extent Descriptors by logical block address
/// </summary>
public Dictionary<long, AllocationExtentDescriptor> AllocationExtentDescriptors { get; set; } = [];
/// <summary>
/// File Identifier Descriptors of each directory, by logical block address of the directory's File Entry
/// </summary>
public Dictionary<long, FileIdentifierDescriptor[]> Directories { get; set; } = [];
}
}

View File

@@ -0,0 +1,24 @@
namespace SabreTools.Data.Models.UDF
{
/// <summary>
/// UDF Volume Descriptor Pointer (3/10.3), continues the Volume Descriptor Sequence in another extent
/// </summary>
/// <see href="https://ecma-international.org/wp-content/uploads/ECMA-167_3rd_edition_june_1997.pdf"/>
public sealed class VolumeDescriptorPointer : Descriptor
{
/// <summary>
/// Volume Descriptor Sequence Number
/// </summary>
public uint VolumeDescriptorSequenceNumber { get; set; }
/// <summary>
/// Extent of the next part of the Volume Descriptor Sequence
/// </summary>
public ExtentDescriptor NextVolumeDescriptorSequenceExtent { get; set; } = new();
/// <summary>
/// Reserved, 484 bytes of 0x00
/// </summary>
public byte[] Reserved { get; set; } = new byte[484];
}
}

View File

@@ -0,0 +1,30 @@
namespace SabreTools.Data.Models.UDF
{
/// <summary>
/// UDF Volume Structure Descriptor (2/9.1), one logical sector of the Volume Recognition Sequence
/// starting at sector 16, e.g. "BEA01", "NSR02"/"NSR03", "TEA01", or "CD001" on bridge discs
/// </summary>
/// <see href="https://ecma-international.org/wp-content/uploads/ECMA-167_3rd_edition_june_1997.pdf"/>
public sealed class VolumeStructureDescriptor
{
/// <summary>
/// Structure type
/// </summary>
public byte StructureType { get; set; }
/// <summary>
/// Standard identifier, 5 bytes
/// </summary>
public byte[] StandardIdentifier { get; set; } = new byte[5];
/// <summary>
/// Structure version
/// </summary>
public byte StructureVersion { get; set; }
/// <summary>
/// Structure data, the rest of the logical sector
/// </summary>
public byte[] StructureData { get; set; } = [];
}
}

View File

@@ -839,6 +839,7 @@ namespace SabreTools.Metadata.DatFiles.Test
SpamSum = HashType.SpamSum.ZeroString,
Start = "start",
Status = Data.Models.Metadata.ItemStatus.Good,
StatusMisc = "status",
Summation = "summation",
TesseractOCR = "ocr",
TesseractOCRConverted = "ocr_converted",
@@ -1516,6 +1517,7 @@ namespace SabreTools.Metadata.DatFiles.Test
Assert.Equal(HashType.SpamSum.ZeroString, rom.SpamSum);
Assert.Equal("start", rom.Start);
Assert.Equal(Data.Models.Metadata.ItemStatus.Good, rom.Status);
Assert.Equal("status", rom.StatusMisc);
Assert.Equal("summation", rom.Summation);
Assert.Equal("ocr", rom.TesseractOCR);
Assert.Equal("ocr_converted", rom.TesseractOCRConverted);

View File

@@ -1066,6 +1066,7 @@ namespace SabreTools.Metadata.DatFiles.Test
Assert.Equal(HashType.SpamSum.ZeroString, rom.SpamSum);
Assert.Equal("start", rom.Start);
Assert.Equal(Data.Models.Metadata.ItemStatus.Good, rom.Status);
Assert.Equal("status", rom.StatusMisc);
Assert.Equal("summation", rom.Summation);
Assert.Equal("ocr", rom.TesseractOCR);
Assert.Equal("ocr_converted", rom.TesseractOCRConverted);

View File

@@ -1,18 +1,25 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFrameworks>net8.0;net9.0;net10.0</TargetFrameworks>
<TargetFrameworks>net8.0;net9.0;net10.0;net11.0</TargetFrameworks>
<IsPackable>false</IsPackable>
<LangVersion>latest</LangVersion>
<Nullable>enable</Nullable>
<SuppressTfmSupportBuildWarnings>true</SuppressTfmSupportBuildWarnings>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="coverlet.collector" Version="10.0.1" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.9.0" />
<PackageReference Include="SabreTools.Hashing" Version="2.1.0" />
<PackageReference Include="coverlet.collector" Version="10.1.0">
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
<PrivateAssets>all</PrivateAssets>
</PackageReference>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.11.1" />
<PackageReference Include="SabreTools.Hashing" Version="2.2.0" />
<PackageReference Include="xunit" Version="2.9.3" />
<PackageReference Include="xunit.runner.visualstudio" Version="4.0.0" />
<PackageReference Include="xunit.runner.visualstudio" Version="4.0.1">
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
<PrivateAssets>all</PrivateAssets>
</PackageReference>
</ItemGroup>
<ItemGroup>

View File

@@ -684,6 +684,14 @@ namespace SabreTools.Metadata.DatFiles
if (rom.CRC32 != "null")
return rom;
// If the item has a SHA-1 but no size
if (rom.Size is null && rom.SHA1 is not null && rom.SHA1 != "null")
return rom;
// If the item has a SHA-256 but no size
if (rom.Size is null && rom.SHA256 is not null && rom.SHA256 != "null")
return rom;
// If the Rom has "null" characteristics, ensure all fields
rom.Name = rom.Name == "null" ? "-" : rom.Name;
rom.Size = 0;
@@ -983,7 +991,7 @@ namespace SabreTools.Metadata.DatFiles
{
// If we have a 0-size or blank rom, then we ignore
long? size = rom.Size;
if (size == 0 || size is null)
if ((size == 0 || size is null) && rom.SHA1 is null && rom.SHA256 is null)
{
string itemString = JsonConvert.SerializeObject(datItem, Formatting.None);
_logger.Verbose($"Item '{itemString}' was skipped because it had an invalid size");

View File

@@ -88,7 +88,9 @@ namespace SabreTools.Metadata.DatFiles.Formats
case Rom rom:
if (string.IsNullOrEmpty(rom.Name))
missingFields.Add(nameof(Data.Models.Metadata.Rom.Name));
if (rom.Size is null || rom.Size < 0)
if ((rom.Size is null || rom.Size < 0)
&& (rom.SHA1 is null || rom.SHA1 == "null")
&& (rom.SHA256 is null || rom.SHA1 == "null"))
missingFields.Add(nameof(Data.Models.Metadata.Rom.Size));
if (string.IsNullOrEmpty(rom.BLAKE3)
&& string.IsNullOrEmpty(rom.CRC16)

View File

@@ -282,7 +282,9 @@ namespace SabreTools.Metadata.DatFiles.Formats
case Rom rom:
if (string.IsNullOrEmpty(rom.Name))
missingFields.Add(nameof(Data.Models.Metadata.Rom.Name));
if (rom.Size is null || rom.Size < 0)
if ((rom.Size is null || rom.Size < 0)
&& (rom.SHA1 is null || rom.SHA1 == "null")
&& (rom.SHA256 is null || rom.SHA1 == "null"))
missingFields.Add(nameof(Data.Models.Metadata.Rom.Size));
if (string.IsNullOrEmpty(rom.BLAKE3)
&& string.IsNullOrEmpty(rom.CRC16)

View File

@@ -29,17 +29,22 @@ namespace SabreTools.Metadata.DatFiles.Formats
<![CDATA[
The softwaredb.xml file contains information about rom mapper types
- Copyright 2003 Nicolas Beyaert (Initial Database)
- Copyright 2004-2013 BlueMSX Team
- Copyright 2005-2023 openMSX Team
- Generation MSXIDs by www.generation-msx.nl
Copyright 2003 Nicolas Beyaert (Initial Database)
Copyright 2004-2013 BlueMSX Team
Copyright 2005-2026 openMSX Team
- Thanks go out to:
- - Generation MSX/Sylvester for the incredible source of information
- p_gimeno and diedel for their help adding and valdiating ROM additions
Generation MSXIDs by www.generation-msx.nl
Thanks go out to:
- Generation MSX / Sylvester for the incredible source of information
- p_gimeno and diedel for their help adding and validating ROM additions
- GDX for additional ROM info and validations and corrections
- TFH / Filehunter for his sharing of information
- MP83 for deep dives into software and corrections
- bsittler for preservation and sharing of information
Important: This file is auto generated using a database.
The Database is located at https://romdb.vampier.net/
]]>";
#endregion

View File

@@ -93,7 +93,9 @@ namespace SabreTools.Metadata.DatFiles.Formats
case Rom rom:
if (string.IsNullOrEmpty(rom.Name))
missingFields.Add(nameof(Data.Models.Metadata.Rom.Name));
if (rom.Size is null || rom.Size < 0)
if ((rom.Size is null || rom.Size < 0)
&& (rom.SHA1 is null || rom.SHA1 == "null")
&& (rom.SHA256 is null || rom.SHA1 == "null"))
missingFields.Add(nameof(Data.Models.Metadata.Rom.Size));
if (string.IsNullOrEmpty(rom.CRC32)
&& string.IsNullOrEmpty(rom.MD5)

View File

@@ -2,7 +2,7 @@
<PropertyGroup>
<!-- Assembly Properties -->
<TargetFrameworks>net20;net35;net40;net452;net462;net472;net48;netcoreapp3.1;net5.0;net6.0;net7.0;net8.0;net9.0;net10.0;netstandard2.0;netstandard2.1</TargetFrameworks>
<TargetFrameworks>net20;net35;net40;net452;net462;net472;net48;netcoreapp3.1;net5.0;net6.0;net7.0;net8.0;net9.0;net10.0;net11.0;netstandard2.0;netstandard2.1</TargetFrameworks>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<CheckEolTargetFramework>false</CheckEolTargetFramework>
<IncludeSourceRevisionInInformationalVersion>false</IncludeSourceRevisionInInformationalVersion>
@@ -12,7 +12,7 @@
<SuppressTfmSupportBuildWarnings>true</SuppressTfmSupportBuildWarnings>
<SymbolPackageFormat>snupkg</SymbolPackageFormat>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
<Version>3.2.0</Version>
<Version>3.3.0</Version>
<!-- Package Properties -->
<Authors>Matt Nadareski</Authors>
@@ -46,8 +46,8 @@
<ItemGroup>
<PackageReference Include="MinValueTupleBridge" Version="0.2.1" Condition="$(TargetFramework.StartsWith(`net2`)) OR $(TargetFramework.StartsWith(`net3`))" />
<PackageReference Include="Newtonsoft.Json" Version="13.0.4" />
<PackageReference Include="SabreTools.Hashing" Version="2.1.0" />
<PackageReference Include="SabreTools.IO" Version="2.3.0" />
<PackageReference Include="SabreTools.Hashing" Version="2.2.0" />
<PackageReference Include="SabreTools.IO" Version="2.4.0" />
<PackageReference Include="System.ValueTuple" Version="4.5.0" Condition="$(TargetFramework.StartsWith(`net40`)) OR $(TargetFramework.StartsWith(`net45`))" />
<PackageReference Include="System.ValueTuple" Version="4.6.2" Condition="$(TargetFramework.StartsWith(`net46`)) OR $(TargetFramework.StartsWith(`net47`)) OR $(TargetFramework.StartsWith(`net48`))" />
</ItemGroup>

View File

@@ -1,18 +1,25 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFrameworks>net8.0;net9.0;net10.0</TargetFrameworks>
<TargetFrameworks>net8.0;net9.0;net10.0;net11.0</TargetFrameworks>
<IsPackable>false</IsPackable>
<LangVersion>latest</LangVersion>
<Nullable>enable</Nullable>
<SuppressTfmSupportBuildWarnings>true</SuppressTfmSupportBuildWarnings>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="coverlet.collector" Version="10.0.1" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.9.0" />
<PackageReference Include="SabreTools.Hashing" Version="2.1.0" />
<PackageReference Include="coverlet.collector" Version="10.1.0">
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
<PrivateAssets>all</PrivateAssets>
</PackageReference>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.11.1" />
<PackageReference Include="SabreTools.Hashing" Version="2.2.0" />
<PackageReference Include="xunit" Version="2.9.3" />
<PackageReference Include="xunit.runner.visualstudio" Version="4.0.0" />
<PackageReference Include="xunit.runner.visualstudio" Version="4.0.1">
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
<PrivateAssets>all</PrivateAssets>
</PackageReference>
</ItemGroup>
<ItemGroup>

View File

@@ -442,6 +442,12 @@ namespace SabreTools.Metadata.DatItems.Formats
set => _internal.Status = value;
}
public string? StatusMisc
{
get => _internal.StatusMisc;
set => _internal.StatusMisc = value;
}
public string? Summation
{
get => _internal.Summation;

View File

@@ -2,7 +2,7 @@
<PropertyGroup>
<!-- Assembly Properties -->
<TargetFrameworks>net20;net35;net40;net452;net462;net472;net48;netcoreapp3.1;net5.0;net6.0;net7.0;net8.0;net9.0;net10.0;netstandard2.0;netstandard2.1</TargetFrameworks>
<TargetFrameworks>net20;net35;net40;net452;net462;net472;net48;netcoreapp3.1;net5.0;net6.0;net7.0;net8.0;net9.0;net10.0;net11.0;netstandard2.0;netstandard2.1</TargetFrameworks>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<CheckEolTargetFramework>false</CheckEolTargetFramework>
<IncludeSourceRevisionInInformationalVersion>false</IncludeSourceRevisionInInformationalVersion>
@@ -12,7 +12,7 @@
<SuppressTfmSupportBuildWarnings>true</SuppressTfmSupportBuildWarnings>
<SymbolPackageFormat>snupkg</SymbolPackageFormat>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
<Version>3.2.0</Version>
<Version>3.3.0</Version>
<!-- Package Properties -->
<Authors>Matt Nadareski</Authors>
@@ -32,8 +32,8 @@
<ItemGroup>
<PackageReference Include="Newtonsoft.Json" Version="13.0.4" />
<PackageReference Include="SabreTools.Hashing" Version="2.1.0" />
<PackageReference Include="SabreTools.IO" Version="2.3.0" />
<PackageReference Include="SabreTools.Hashing" Version="2.2.0" />
<PackageReference Include="SabreTools.IO" Version="2.4.0" />
</ItemGroup>
<ItemGroup>

View File

@@ -1,4 +1,5 @@
using System;
using System.Globalization;
using SabreTools.Data.Models.Metadata;
using SabreTools.Hashing;
using Xunit;
@@ -1022,6 +1023,20 @@ namespace SabreTools.Metadata.Filter.Test
Assert.True(actual);
}
[Theory]
[InlineData("en-US")]
[InlineData("de-DE")]
[InlineData("fr-FR")]
public void Matches_DisplayRefresh_CurrentCultureIgnored(string culture)
{
CultureInfo.CurrentCulture = new CultureInfo(culture);
var filter = new FilterObject("display.refresh", "123.45", Operation.Equals);
Display obj = new Display { Refresh = 123.45 };
bool actual = filter.Matches(obj);
Assert.True(actual);
}
#endregion
#region Driver
@@ -1661,6 +1676,7 @@ namespace SabreTools.Metadata.Filter.Test
[InlineData("rom.spamsum", "3::")]
[InlineData("rom.start", "start")]
[InlineData("rom.status", "good")]
[InlineData("rom.status_misc", "status")]
[InlineData("rom.summation", "summation")]
[InlineData("rom.ocr", "ocr")]
[InlineData("rom.tesseractocr", "ocr")]
@@ -1785,6 +1801,7 @@ namespace SabreTools.Metadata.Filter.Test
SpamSum = HashType.SpamSum.ZeroString,
Start = "start",
Status = ItemStatus.Good,
StatusMisc = "status",
Summation = "summation",
TesseractOCR = "ocr",
TesseractOCRConverted = "ocr_converted",
@@ -2083,6 +2100,20 @@ namespace SabreTools.Metadata.Filter.Test
Assert.True(actual);
}
[Theory]
[InlineData("en-US")]
[InlineData("de-DE")]
[InlineData("fr-FR")]
public void Matches_VideoRefresh_CurrentCultureIgnored(string culture)
{
CultureInfo.CurrentCulture = new CultureInfo(culture);
var filter = new FilterObject("video.refresh", "123.45", Operation.Equals);
Video obj = new Video { Refresh = 123.45 };
bool actual = filter.Matches(obj);
Assert.True(actual);
}
#endregion
}
}

View File

@@ -1,10 +1,11 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFrameworks>net8.0;net9.0;net10.0</TargetFrameworks>
<TargetFrameworks>net8.0;net9.0;net10.0;net11.0</TargetFrameworks>
<IsPackable>false</IsPackable>
<LangVersion>latest</LangVersion>
<Nullable>enable</Nullable>
<SuppressTfmSupportBuildWarnings>true</SuppressTfmSupportBuildWarnings>
</PropertyGroup>
<ItemGroup>
@@ -18,10 +19,16 @@
</ItemGroup>
<ItemGroup>
<PackageReference Include="coverlet.collector" Version="10.0.1" />
<PackageReference Include="coverlet.collector" Version="10.1.0">
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
<PrivateAssets>all</PrivateAssets>
</PackageReference>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.9.0" />
<PackageReference Include="xunit" Version="2.9.3" />
<PackageReference Include="xunit.runner.visualstudio" Version="4.0.0" />
<PackageReference Include="xunit.runner.visualstudio" Version="4.0.1">
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
<PrivateAssets>all</PrivateAssets>
</PackageReference>
</ItemGroup>
<ItemGroup>

View File

@@ -625,6 +625,7 @@ namespace SabreTools.Metadata.Filter
"spamsum",
"start",
"status",
"status_misc",
"summation",
"tesseractocr",
"tesseractocrconverted",

View File

@@ -1,4 +1,5 @@
using System;
using System.Globalization;
using System.Text.RegularExpressions;
using SabreTools.Data.Extensions;
using SabreTools.Data.Models.Metadata;
@@ -1134,7 +1135,7 @@ namespace SabreTools.Metadata.Filter
return true;
case "refresh":
case "freq":
checkValue = obj.Refresh?.ToString();
checkValue = obj.Refresh?.ToString(CultureInfo.InvariantCulture);
return true;
case "rotate":
case "orientation":
@@ -2099,6 +2100,9 @@ namespace SabreTools.Metadata.Filter
case "status":
checkValue = obj.Status?.AsStringValue();
return true;
case "status_misc":
checkValue = obj.StatusMisc;
return true;
case "summation":
checkValue = obj.Summation;
return true;
@@ -2509,7 +2513,7 @@ namespace SabreTools.Metadata.Filter
return true;
case "refresh":
case "freq":
checkValue = obj.Refresh?.ToString();
checkValue = obj.Refresh?.ToString(CultureInfo.InvariantCulture);
return true;
case "displaytype":
case "screen":

View File

@@ -2,7 +2,7 @@
<PropertyGroup>
<!-- Assembly Properties -->
<TargetFrameworks>net20;net35;net40;net452;net462;net472;net48;netcoreapp3.1;net5.0;net6.0;net7.0;net8.0;net9.0;net10.0;netstandard2.0;netstandard2.1</TargetFrameworks>
<TargetFrameworks>net20;net35;net40;net452;net462;net472;net48;netcoreapp3.1;net5.0;net6.0;net7.0;net8.0;net9.0;net10.0;net11.0;netstandard2.0;netstandard2.1</TargetFrameworks>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<CheckEolTargetFramework>false</CheckEolTargetFramework>
<IncludeSourceRevisionInInformationalVersion>false</IncludeSourceRevisionInInformationalVersion>
@@ -12,7 +12,7 @@
<SuppressTfmSupportBuildWarnings>true</SuppressTfmSupportBuildWarnings>
<SymbolPackageFormat>snupkg</SymbolPackageFormat>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
<Version>3.2.0</Version>
<Version>3.3.0</Version>
<!-- Package Properties -->
<Authors>Matt Nadareski</Authors>
@@ -36,7 +36,7 @@
</ItemGroup>
<ItemGroup>
<PackageReference Include="SabreTools.IO" Version="2.3.0" />
<PackageReference Include="SabreTools.IO" Version="2.4.0" />
</ItemGroup>
</Project>

View File

@@ -2,7 +2,7 @@
<PropertyGroup>
<!-- Assembly Properties -->
<TargetFrameworks>net20;net35;net40;net452;net462;net472;net48;netcoreapp3.1;net5.0;net6.0;net7.0;net8.0;net9.0;net10.0;netstandard2.0;netstandard2.1</TargetFrameworks>
<TargetFrameworks>net20;net35;net40;net452;net462;net472;net48;netcoreapp3.1;net5.0;net6.0;net7.0;net8.0;net9.0;net10.0;net11.0;netstandard2.0;netstandard2.1</TargetFrameworks>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<CheckEolTargetFramework>false</CheckEolTargetFramework>
<IncludeSourceRevisionInInformationalVersion>false</IncludeSourceRevisionInInformationalVersion>
@@ -13,7 +13,7 @@
<SuppressTfmSupportBuildWarnings>true</SuppressTfmSupportBuildWarnings>
<SymbolPackageFormat>snupkg</SymbolPackageFormat>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
<Version>3.2.0</Version>
<Version>3.3.0</Version>
<!-- Package Properties -->
<Authors>Matt Nadareski</Authors>
@@ -37,8 +37,8 @@
</ItemGroup>
<ItemGroup>
<PackageReference Include="SabreTools.Hashing" Version="2.1.0" />
<PackageReference Include="SabreTools.IO" Version="2.3.0" />
<PackageReference Include="SabreTools.Hashing" Version="2.2.0" />
<PackageReference Include="SabreTools.IO" Version="2.4.0" />
</ItemGroup>
</Project>

View File

@@ -1,17 +1,23 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFrameworks>net8.0;net9.0;net10.0</TargetFrameworks>
<TargetFrameworks>net8.0;net9.0;net10.0;net11.0</TargetFrameworks>
<IsPackable>false</IsPackable>
<LangVersion>latest</LangVersion>
<Nullable>enable</Nullable>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="coverlet.collector" Version="10.0.1" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.9.0" />
<PackageReference Include="coverlet.collector" Version="10.1.0">
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
<PrivateAssets>all</PrivateAssets>
</PackageReference>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.11.1" />
<PackageReference Include="xunit" Version="2.9.3" />
<PackageReference Include="xunit.runner.visualstudio" Version="4.0.0" />
<PackageReference Include="xunit.runner.visualstudio" Version="4.0.1">
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
<PrivateAssets>all</PrivateAssets>
</PackageReference>
</ItemGroup>
<ItemGroup>

View File

@@ -2,7 +2,7 @@
<PropertyGroup>
<!-- Assembly Properties -->
<TargetFrameworks>net20;net35;net40;net452;net462;net472;net48;netcoreapp3.1;net5.0;net6.0;net7.0;net8.0;net9.0;net10.0;netstandard2.0;netstandard2.1</TargetFrameworks>
<TargetFrameworks>net20;net35;net40;net452;net462;net472;net48;netcoreapp3.1;net5.0;net6.0;net7.0;net8.0;net9.0;net10.0;net11.0;netstandard2.0;netstandard2.1</TargetFrameworks>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<CheckEolTargetFramework>false</CheckEolTargetFramework>
<IncludeSourceRevisionInInformationalVersion>false</IncludeSourceRevisionInInformationalVersion>
@@ -12,7 +12,7 @@
<SuppressTfmSupportBuildWarnings>true</SuppressTfmSupportBuildWarnings>
<SymbolPackageFormat>snupkg</SymbolPackageFormat>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
<Version>3.2.0</Version>
<Version>3.3.0</Version>
<!-- Package Properties -->
<Authors>Matt Nadareski</Authors>

View File

@@ -40,11 +40,19 @@ namespace SabreTools.Serialization.CrossModel.Test
Content = "content",
};
var standalone_rom = new Data.Models.OpenMSX.Rom
{
SHA1 = "sha1",
Type = "type",
Status = "status",
Remark = "remark",
};
var rom = new Data.Models.OpenMSX.Rom
{
Start = "start",
Type = "type",
Hash = "hash",
SHA1 = "hash",
Remark = "remark",
};
@@ -59,7 +67,7 @@ namespace SabreTools.Serialization.CrossModel.Test
{
Start = "start",
Type = "type",
Hash = "hash",
SHA1 = "hash",
Remark = "remark",
};
@@ -74,7 +82,7 @@ namespace SabreTools.Serialization.CrossModel.Test
{
Start = "start",
Type = "type",
Hash = "hash",
SHA1 = "hash",
Remark = "remark",
};
@@ -94,6 +102,7 @@ namespace SabreTools.Serialization.CrossModel.Test
Year = "year",
Country = "country",
Dump = [dump_rom, dump_megarom, dump_sccpluscart],
Rom = [standalone_rom],
};
return new Data.Models.OpenMSX.SoftwareDb
@@ -122,6 +131,10 @@ namespace SabreTools.Serialization.CrossModel.Test
{
Validate(dump);
}
Assert.NotNull(software.Rom);
var rom = Assert.Single(software.Rom);
ValidateStandalone(rom);
}
/// <summary>
@@ -154,7 +167,19 @@ namespace SabreTools.Serialization.CrossModel.Test
Assert.NotNull(rombase);
Assert.Equal("start", rombase.Start);
Assert.Equal("type", rombase.Type);
Assert.Equal("hash", rombase.Hash);
Assert.Equal("hash", rombase.SHA1);
Assert.Equal("remark", rombase.Remark);
}
/// <summary>
/// Validate a RomBase
/// </summary>
private static void ValidateStandalone(Data.Models.OpenMSX.RomBase? rombase)
{
Assert.NotNull(rombase);
Assert.Equal("sha1", rombase.SHA1);
Assert.Equal("type", rombase.Type);
Assert.Equal("status", rombase.Status);
Assert.Equal("remark", rombase.Remark);
}
}

View File

@@ -1,18 +1,25 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFrameworks>net8.0;net9.0;net10.0</TargetFrameworks>
<TargetFrameworks>net8.0;net9.0;net10.0;net11.0</TargetFrameworks>
<IsPackable>false</IsPackable>
<LangVersion>latest</LangVersion>
<Nullable>enable</Nullable>
<SuppressTfmSupportBuildWarnings>true</SuppressTfmSupportBuildWarnings>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="coverlet.collector" Version="10.0.1" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.9.0" />
<PackageReference Include="SabreTools.Hashing" Version="2.1.0" />
<PackageReference Include="coverlet.collector" Version="10.1.0">
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
<PrivateAssets>all</PrivateAssets>
</PackageReference>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.11.1" />
<PackageReference Include="SabreTools.Hashing" Version="2.2.0" />
<PackageReference Include="xunit" Version="2.9.3" />
<PackageReference Include="xunit.runner.visualstudio" Version="4.0.0" />
<PackageReference Include="xunit.runner.visualstudio" Version="4.0.1">
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
<PrivateAssets>all</PrivateAssets>
</PackageReference>
</ItemGroup>
<ItemGroup>

View File

@@ -41,17 +41,21 @@ namespace SabreTools.Serialization.CrossModel
var game = new Software
{
Title = item.Name,
GenMSXID = item.GenMSXID,
System = item.System,
Company = item.Company,
Year = item.Year,
Country = item.Country,
GenMSXID = item.GenMSXID,
};
var dumps = item.Dump;
if (dumps is not null && dumps.Length > 0)
game.Dump = Array.ConvertAll(dumps, ConvertFromInternalModel);
var roms = item.Rom;
if (roms is not null && roms.Length > 0)
game.Rom = Array.ConvertAll(roms, ConvertRomFromInternalModel);
return game;
}
@@ -90,10 +94,11 @@ namespace SabreTools.Serialization.CrossModel
{
var megaRom = new MegaRom
{
Start = item.Start,
SHA1 = item.SHA1,
Type = item.OpenMSXType,
Hash = item.SHA1,
Status = item.StatusMisc,
Remark = item.Remark,
Start = item.Start,
};
return megaRom;
}
@@ -118,10 +123,11 @@ namespace SabreTools.Serialization.CrossModel
{
var rom = new Rom
{
Start = item.Start,
SHA1 = item.SHA1,
Type = item.OpenMSXType,
Hash = item.SHA1,
Status = item.StatusMisc,
Remark = item.Remark,
Start = item.Start,
};
return rom;
}
@@ -133,10 +139,11 @@ namespace SabreTools.Serialization.CrossModel
{
var sccPlusCart = new SCCPlusCart
{
Start = item.Start,
SHA1 = item.SHA1,
Type = item.OpenMSXType,
Hash = item.SHA1,
Status = item.StatusMisc,
Remark = item.Remark,
Start = item.Start,
};
return sccPlusCart;
}

View File

@@ -52,6 +52,9 @@ namespace SabreTools.Serialization.CrossModel
if (item.Dump is not null && item.Dump.Length > 0)
machine.Dump = Array.ConvertAll(item.Dump, ConvertToInternalModel);
if (item.Rom is not null && item.Rom.Length > 0)
machine.Rom = Array.ConvertAll(item.Rom, ConvertToInternalModel);
return machine;
}
@@ -112,10 +115,13 @@ namespace SabreTools.Serialization.CrossModel
{
var rom = new Data.Models.Metadata.Rom
{
Start = item.Start,
// TODO: Figure out a better placeholder
Name = item.SHA1,
SHA1 = item.SHA1,
OpenMSXType = item.Type,
SHA1 = item.Hash,
StatusMisc = item.Status,
Remark = item.Remark,
Start = item.Start,
};
return rom;
}

View File

@@ -2,7 +2,7 @@
<PropertyGroup>
<!-- Assembly Properties -->
<TargetFrameworks>net20;net35;net40;net452;net462;net472;net48;netcoreapp3.1;net5.0;net6.0;net7.0;net8.0;net9.0;net10.0;netstandard2.0;netstandard2.1</TargetFrameworks>
<TargetFrameworks>net20;net35;net40;net452;net462;net472;net48;netcoreapp3.1;net5.0;net6.0;net7.0;net8.0;net9.0;net10.0;net11.0;netstandard2.0;netstandard2.1</TargetFrameworks>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<CheckEolTargetFramework>false</CheckEolTargetFramework>
<IncludeSourceRevisionInInformationalVersion>false</IncludeSourceRevisionInInformationalVersion>
@@ -12,7 +12,7 @@
<SuppressTfmSupportBuildWarnings>true</SuppressTfmSupportBuildWarnings>
<SymbolPackageFormat>snupkg</SymbolPackageFormat>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
<Version>3.2.0</Version>
<Version>3.3.0</Version>
<!-- Package Properties -->
<Authors>Matt Nadareski</Authors>
@@ -31,7 +31,7 @@
</ItemGroup>
<ItemGroup>
<PackageReference Include="SabreTools.Hashing" Version="2.1.0" />
<PackageReference Include="SabreTools.Hashing" Version="2.2.0" />
</ItemGroup>
<ItemGroup>

View File

@@ -1,3 +1,4 @@
using System.Globalization;
using System.IO;
using System.Linq;
using Xunit;
@@ -171,6 +172,25 @@ namespace SabreTools.Serialization.Readers.Test
Validate(newGame);
}
[Theory]
[InlineData("en-US")]
[InlineData("de-DE")]
[InlineData("fr-FR")]
public void SerializeFreq_CurrentCultureIgnored(string culture)
{
CultureInfo.CurrentCulture = new CultureInfo(culture);
var serializer = new Writers.ClrMamePro();
// Serialize to stream
Stream? actual = serializer.SerializeStream(Build(game: false));
Assert.NotNull(actual);
// Validate the freq value
string dat = new StreamReader(actual).ReadToEnd();
Assert.Contains("123.450000", dat);
Assert.DoesNotContain("123,45", dat);
}
/// <summary>
/// Build model for serialization and deserialization
/// </summary>

View File

@@ -1,11 +1,20 @@
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using SabreTools.Data.Models.ISO9660;
using SabreTools.Numerics;
using Xunit;
namespace SabreTools.Serialization.Readers.Test
{
public class ISO9660Tests
{
/// <summary>
/// File flags of a plain file record
/// </summary>
private const FileFlags NoFlags = 0;
[Fact]
public void NullArray_Null()
{
@@ -68,5 +77,224 @@ namespace SabreTools.Serialization.Readers.Test
var actual = deserializer.Deserialize(data);
Assert.Null(actual);
}
[Fact]
public void ParseDirectory_AmbiguousExtentLocation_ParsesBothExtents()
{
// Little-endian location points to sector 1, big-endian location to sector 2
byte[] image = new byte[4 * 2048];
WriteDirectory(image, 1);
WriteDirectory(image, 2);
var dr = new DirectoryRecord
{
ExtentLocation = new BothInt32(1, 2),
ExtentLength = 2048,
FileFlags = FileFlags.DIRECTORY,
};
var actual = ISO9660.ParseDirectory(new MemoryStream(image), 2048, 2048, dr, false);
Assert.NotNull(actual);
Assert.True(actual.ContainsKey(1));
Assert.True(actual.ContainsKey(2));
}
[Fact]
public void ParseDirectory_ConsistentExtentLocation_ReadsEachDirectoryOnce()
{
// Sector 1 holds directory A at sector 2, which holds directory B at sector 3
byte[] image = new byte[5 * 2048];
WriteDirectory(image, 1, ("A", 2, FileFlags.DIRECTORY));
WriteDirectory(image, 2, ("B", 3, FileFlags.DIRECTORY));
WriteDirectory(image, 3);
var dr = new DirectoryRecord
{
ExtentLocation = 1,
ExtentLength = 2048,
FileFlags = FileFlags.DIRECTORY,
};
var stream = new SeekCountingStream(image);
var actual = ISO9660.ParseDirectory(stream, 2048, 2048, dr, false);
Assert.NotNull(actual);
Assert.Equal(3, actual.Count);
Assert.Equal(1, stream.SeeksTo(1 * 2048));
Assert.Equal(1, stream.SeeksTo(2 * 2048));
Assert.Equal(1, stream.SeeksTo(3 * 2048));
}
[Fact]
public void ParseDirectoryDescriptors_ReadsExtentsInVolumeOrder()
{
// Root at sector 20 holds A at sector 30 and B at sector 22, A holds C at sector 21
byte[] image = new byte[32 * 2048];
WriteDirectory(image, 20, ("A", 30, FileFlags.DIRECTORY), ("B", 22, FileFlags.DIRECTORY));
WriteDirectory(image, 30, ("C", 21, FileFlags.DIRECTORY));
WriteDirectory(image, 22);
WriteDirectory(image, 21);
var stream = new SeekCountingStream(image);
var actual = ISO9660.ParseDirectoryDescriptors(stream, 2048, [CreatePrimaryVolumeDescriptor(20)]);
Assert.NotNull(actual);
Assert.Equal(new[] { 20, 21, 22, 30 }, actual.Keys.OrderBy(k => k));
Assert.Equal(new long[] { 20 * 2048, 22 * 2048, 30 * 2048, 21 * 2048 }, stream.SeekOrder);
}
[Fact]
public void FindDirectoryRecords_NestedPath_ReadsOnlyDirectoriesOnPath()
{
// Root at sector 20 holds OTHER at sector 22 and SUB at sector 30, SUB holds FILE.TXT;1 at sector 25
byte[] image = new byte[32 * 2048];
WriteDirectory(image, 20, ("OTHER", 22, FileFlags.DIRECTORY), ("SUB", 30, FileFlags.DIRECTORY));
WriteDirectory(image, 22);
WriteDirectory(image, 30, ("FILE.TXT;1", 25, NoFlags));
var stream = new SeekCountingStream(image);
var actual = ISO9660.FindDirectoryRecords(stream, 2048, CreatePrimaryVolumeDescriptor(20), "sub/file.txt");
Assert.NotNull(actual);
Assert.Equal(25, Assert.Single(actual).ExtentLocation.LittleEndian);
Assert.Equal(new long[] { 20 * 2048, 30 * 2048 }, stream.SeekOrder);
}
[Fact]
public void FindDirectoryRecords_MissingPath_Null()
{
byte[] image = new byte[32 * 2048];
WriteDirectory(image, 20, ("SUB", 30, FileFlags.DIRECTORY));
WriteDirectory(image, 30);
var actual = ISO9660.FindDirectoryRecords(new MemoryStream(image), 2048, CreatePrimaryVolumeDescriptor(20), "SUB/MISSING.TXT");
Assert.Null(actual);
}
[Fact]
public void FindDirectoryRecords_MultiExtentFile_ReturnsAllRecords()
{
// BIG.BIN;1 is recorded in two extents, at sectors 25 and 26
byte[] image = new byte[32 * 2048];
WriteDirectory(image, 20, ("BIG.BIN;1", 25, FileFlags.MULTI_EXTENT), ("BIG.BIN;1", 26, NoFlags), ("NEXT.BIN;1", 27, NoFlags));
var actual = ISO9660.FindDirectoryRecords(new MemoryStream(image), 2048, CreatePrimaryVolumeDescriptor(20), "BIG.BIN");
Assert.NotNull(actual);
Assert.Equal(new[] { 25, 26 }, actual.Select(r => r.ExtentLocation.LittleEndian));
}
[Fact]
public void FindDirectoryRecords_SupplementaryVolumeDescriptor_UsesUnicodeNames()
{
byte[] image = new byte[32 * 2048];
WriteDirectory(image, 20, Encoding.BigEndianUnicode, ("Sub", 30, FileFlags.DIRECTORY));
WriteDirectory(image, 30, Encoding.BigEndianUnicode, ("File.txt;1", 25, NoFlags));
var svd = new SupplementaryVolumeDescriptor
{
LogicalBlockSize = 2048,
RootDirectoryRecord = CreatePrimaryVolumeDescriptor(20).RootDirectoryRecord,
};
var actual = ISO9660.FindDirectoryRecords(new MemoryStream(image), 2048, svd, "Sub/File.txt");
Assert.NotNull(actual);
Assert.Equal(25, Assert.Single(actual).ExtentLocation.LittleEndian);
}
/// <summary>
/// Create a primary volume descriptor with 2048-byte blocks and a one-sector root directory
/// </summary>
private static PrimaryVolumeDescriptor CreatePrimaryVolumeDescriptor(int rootSector)
{
return new PrimaryVolumeDescriptor
{
LogicalBlockSize = 2048,
RootDirectoryRecord = new DirectoryRecord
{
ExtentLocation = rootSector,
ExtentLength = 2048,
FileFlags = FileFlags.DIRECTORY,
},
};
}
/// <summary>
/// Write a one-sector directory extent with self and parent records and the given entries
/// </summary>
private static void WriteDirectory(byte[] image, int sector, params (string Name, int Location, FileFlags Flags)[] entries)
=> WriteDirectory(image, sector, Encoding.ASCII, entries);
/// <summary>
/// Write a one-sector directory extent with self and parent records and the given entries
/// </summary>
private static void WriteDirectory(byte[] image, int sector, Encoding encoding, params (string Name, int Location, FileFlags Flags)[] entries)
{
int offset = sector * 2048;
offset = WriteRecord(image, offset, Constants.CurrentDirectory, sector, FileFlags.DIRECTORY);
offset = WriteRecord(image, offset, Constants.ParentDirectory, sector, FileFlags.DIRECTORY);
foreach (var (name, location, flags) in entries)
{
offset = WriteRecord(image, offset, encoding.GetBytes(name), location, flags);
}
}
/// <summary>
/// Write a directory record for a one-sector extent, returns the offset after the record
/// </summary>
private static int WriteRecord(byte[] image, int offset, byte[] identifier, int location, FileFlags flags)
{
int length = 33 + identifier.Length + (identifier.Length % 2 == 0 ? 1 : 0);
image[offset] = (byte)length;
WriteBothEndian(image, offset + 2, location);
WriteBothEndian(image, offset + 10, 2048);
image[offset + 25] = (byte)flags;
image[offset + 28] = 1;
image[offset + 31] = 1;
image[offset + 32] = (byte)identifier.Length;
identifier.CopyTo(image, offset + 33);
return offset + length;
}
/// <summary>
/// Write a 32-bit value in both-endian format
/// </summary>
private static void WriteBothEndian(byte[] image, int offset, int value)
{
for (int i = 0; i < 4; i++)
{
image[offset + i] = (byte)(value >> (8 * i));
image[offset + 7 - i] = (byte)(value >> (8 * i));
}
}
/// <summary>
/// Memory stream that records absolute seeks
/// </summary>
private sealed class SeekCountingStream : MemoryStream
{
private readonly Dictionary<long, int> _seeks = [];
/// <summary>
/// Offsets of all absolute seeks, in order
/// </summary>
public List<long> SeekOrder { get; } = [];
public SeekCountingStream(byte[] buffer) : base(buffer) { }
public int SeeksTo(long offset)
=> _seeks.TryGetValue(offset, out int count) ? count : 0;
public override long Seek(long offset, SeekOrigin loc)
{
if (loc == SeekOrigin.Begin)
{
_seeks[offset] = SeeksTo(offset) + 1;
SeekOrder.Add(offset);
}
return base.Seek(offset, loc);
}
}
}
}

View File

@@ -1,3 +1,4 @@
using System.Globalization;
using System.IO;
using System.Linq;
using Xunit;
@@ -125,6 +126,25 @@ namespace SabreTools.Serialization.Readers.Test
Validate(newGame);
}
[Theory]
[InlineData("en-US")]
[InlineData("de-DE")]
[InlineData("fr-FR")]
public void SerializeRefresh_CurrentCultureIgnored(string culture)
{
CultureInfo.CurrentCulture = new CultureInfo(culture);
var serializer = new Writers.Listxml();
// Serialize to stream
Stream? actual = serializer.SerializeStream(Build(game: false));
Assert.NotNull(actual);
// Validate the refresh values
string xml = new StreamReader(actual).ReadToEnd();
Assert.Contains("refresh=\"123.450000\"", xml);
Assert.DoesNotContain("123,45", xml);
}
/// <summary>
/// Build model for serialization and deserialization
/// </summary>

View File

@@ -1,3 +1,4 @@
using System.Globalization;
using System.IO;
using System.Linq;
using Xunit;
@@ -121,6 +122,25 @@ namespace SabreTools.Serialization.Readers.Test
Validate(newGame);
}
[Theory]
[InlineData("en-US")]
[InlineData("de-DE")]
[InlineData("fr-FR")]
public void SerializeRefresh_CurrentCultureIgnored(string culture)
{
CultureInfo.CurrentCulture = new CultureInfo(culture);
var serializer = new Writers.M1();
// Serialize to stream
Stream? actual = serializer.SerializeStream(Build(game: false));
Assert.NotNull(actual);
// Validate the refresh values
string xml = new StreamReader(actual).ReadToEnd();
Assert.Contains("refresh=\"123.450000\"", xml);
Assert.DoesNotContain("123,45", xml);
}
/// <summary>
/// Build model for serialization and deserialization
/// </summary>

View File

@@ -1,3 +1,4 @@
using System.Globalization;
using System.IO;
using System.Linq;
using Xunit;
@@ -121,6 +122,25 @@ namespace SabreTools.Serialization.Readers.Test
Validate(newGame);
}
[Theory]
[InlineData("en-US")]
[InlineData("de-DE")]
[InlineData("fr-FR")]
public void SerializeRefresh_CurrentCultureIgnored(string culture)
{
CultureInfo.CurrentCulture = new CultureInfo(culture);
var serializer = new Writers.Mess();
// Serialize to stream
Stream? actual = serializer.SerializeStream(Build(game: false));
Assert.NotNull(actual);
// Validate the refresh values
string xml = new StreamReader(actual).ReadToEnd();
Assert.Contains("refresh=\"123.450000\"", xml);
Assert.DoesNotContain("123,45", xml);
}
/// <summary>
/// Build model for serialization and deserialization
/// </summary>

View File

@@ -106,11 +106,19 @@ namespace SabreTools.Serialization.Readers.Test
Content = "content",
};
var standalone_rom = new Data.Models.OpenMSX.Rom
{
SHA1 = "sha1",
Type = "type",
Status = "status",
Remark = "remark",
};
var rom = new Data.Models.OpenMSX.Rom
{
Start = "start",
Type = "type",
Hash = "hash",
SHA1 = "hash",
Remark = "remark",
};
@@ -125,7 +133,7 @@ namespace SabreTools.Serialization.Readers.Test
{
Start = "start",
Type = "type",
Hash = "hash",
SHA1 = "hash",
Remark = "remark",
};
@@ -140,7 +148,7 @@ namespace SabreTools.Serialization.Readers.Test
{
Start = "start",
Type = "type",
Hash = "hash",
SHA1 = "hash",
Remark = "remark",
};
@@ -160,6 +168,7 @@ namespace SabreTools.Serialization.Readers.Test
Year = "year",
Country = "country",
Dump = [dump_rom, dump_megarom, dump_sccpluscart],
Rom = [standalone_rom],
};
return new Data.Models.OpenMSX.SoftwareDb
@@ -188,6 +197,10 @@ namespace SabreTools.Serialization.Readers.Test
{
Validate(dump);
}
Assert.NotNull(software.Rom);
var rom = Assert.Single(software.Rom);
ValidateStandalone(rom);
}
/// <summary>
@@ -220,7 +233,19 @@ namespace SabreTools.Serialization.Readers.Test
Assert.NotNull(rombase);
Assert.Equal("start", rombase.Start);
Assert.Equal("type", rombase.Type);
Assert.Equal("hash", rombase.Hash);
Assert.Equal("hash", rombase.SHA1);
Assert.Equal("remark", rombase.Remark);
}
/// <summary>
/// Validate a RomBase
/// </summary>
private static void ValidateStandalone(Data.Models.OpenMSX.RomBase? rombase)
{
Assert.NotNull(rombase);
Assert.Equal("sha1", rombase.SHA1);
Assert.Equal("type", rombase.Type);
Assert.Equal("status", rombase.Status);
Assert.Equal("remark", rombase.Remark);
}
}

View File

@@ -1,4 +1,5 @@
using System.IO;
using System.IO.Compression;
using System.Linq;
using Xunit;
@@ -68,5 +69,48 @@ namespace SabreTools.Serialization.Readers.Test
var actual = deserializer.Deserialize(data);
Assert.Null(actual);
}
[Fact]
public void ValidStream_FileDataLocated()
{
Stream? data = CreateArchive();
var deserializer = new PKZIP();
var actual = deserializer.Deserialize(data);
Assert.NotNull(actual);
var localFile = Assert.Single(actual.LocalFiles!);
var header = localFile.LocalFileHeader;
Assert.Equal(30 + header.FileNameLength + header.ExtraFieldLength, localFile.FileDataOffset);
Assert.Single(actual.CentralDirectoryHeaders!);
}
[Fact]
public void TruncatedFileData_Null()
{
byte[] archive = CreateArchive().ToArray();
Stream? data = new MemoryStream(archive, 0, 512);
var deserializer = new PKZIP();
var actual = deserializer.Deserialize(data);
Assert.Null(actual);
}
/// <summary>
/// Create an archive with one stored file of 1000 bytes
/// </summary>
private static MemoryStream CreateArchive()
{
var stream = new MemoryStream();
using (var archive = new ZipArchive(stream, ZipArchiveMode.Create, leaveOpen: true))
{
var entry = archive.CreateEntry("file.bin", CompressionLevel.NoCompression);
using var entryStream = entry.Open();
entryStream.Write([.. Enumerable.Range(0, 1000).Select(i => (byte)i)], 0, 1000);
}
stream.Position = 0;
return stream;
}
}
}

View File

@@ -1,17 +1,24 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFrameworks>net8.0;net9.0;net10.0</TargetFrameworks>
<TargetFrameworks>net8.0;net9.0;net10.0;net11.0</TargetFrameworks>
<IsPackable>false</IsPackable>
<LangVersion>latest</LangVersion>
<Nullable>enable</Nullable>
<SuppressTfmSupportBuildWarnings>true</SuppressTfmSupportBuildWarnings>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="coverlet.collector" Version="10.0.1" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.9.0" />
<PackageReference Include="coverlet.collector" Version="10.1.0">
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
<PrivateAssets>all</PrivateAssets>
</PackageReference>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.11.1" />
<PackageReference Include="xunit" Version="2.9.3" />
<PackageReference Include="xunit.runner.visualstudio" Version="4.0.0" />
<PackageReference Include="xunit.runner.visualstudio" Version="4.0.1">
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
<PrivateAssets>all</PrivateAssets>
</PackageReference>
</ItemGroup>
<ItemGroup>

View File

@@ -0,0 +1,294 @@
using System;
using System.Text;
namespace SabreTools.Serialization.Readers.Test
{
/// <summary>
/// Builds a minimal UDF 1.02 image in memory (2048-byte sectors, one Type 1 partition at sector 260)
/// </summary>
/// <remarks>
/// Partition blocks:
/// 0 File Set Descriptor
/// 2 root directory File Entry, contents in block 3:
/// parent, "README.TXT" (embedded data), "deleted.txt" (deleted), "Sub" (directory),
/// "Media" (last entry, no padding)
/// 4 "README.TXT" File Entry, data "hello world" embedded
/// 5 "Sub" File Entry, contents in block 6: parent, "data.bin" (16-bit name)
/// 7 "data.bin" File Entry, long_ad to block 8 (2048 bytes), then an Allocation Extent Descriptor
/// in block 10 with a long_ad to block 9 (952 bytes)
/// 8-9 "data.bin" data (byte i = i % 251)
/// 11 "Media" File Entry, one short_ad of 100 bytes allocated but not recorded
/// </remarks>
internal static class UDFImageBuilder
{
public const int SectorSize = 2048;
public const int PartitionStart = 260;
public const int PartitionLength = 40;
public const int Sectors = 300;
public const int DataBinLength = 3000;
/// <summary>
/// Build the image
/// </summary>
/// <param name="corruptRootEntry">True to break the checksum of the root directory File Entry</param>
/// <param name="corruptRootEntryCrc">True to change a byte of the root directory File Entry that only the CRC covers</param>
/// <param name="emptyPartitionMap">True to record the partition map with length 0 and a map count of 3</param>
/// <param name="pointerLoop">True to replace the Terminating Descriptor by a Volume Descriptor Pointer back to the start of the sequence</param>
/// <param name="subName">Name of the "Sub" directory in the root directory</param>
/// <param name="subLoop">True to add a directory "loop" to "Sub" that points back to the root directory</param>
/// <param name="mediaPastEnd">True to record "Media" in a partition block past the end of the image</param>
public static byte[] Build(bool corruptRootEntry = false,
bool corruptRootEntryCrc = false,
bool emptyPartitionMap = false,
bool pointerLoop = false,
string subName = "Sub",
bool subLoop = false,
bool mediaPastEnd = false)
{
var image = new byte[Sectors * SectorSize];
// Volume Recognition Sequence
WriteStructure(image, 16, "BEA01");
WriteStructure(image, 17, "NSR02");
WriteStructure(image, 18, "TEA01");
// Anchor Volume Descriptor Pointer: main and reserve sequence at sector 32, 4 sectors
int avdp = 256 * SectorSize;
WriteUInt32(image, avdp + 16, 4 * SectorSize);
WriteUInt32(image, avdp + 20, 32);
WriteUInt32(image, avdp + 24, 4 * SectorSize);
WriteUInt32(image, avdp + 28, 32);
WriteTag(image, avdp, 2, 256, 512);
// Primary Volume Descriptor
int pvd = 32 * SectorSize;
WriteDString(image, pvd + 24, 32, "TESTVOLUME");
WriteTag(image, pvd, 1, 32, 512);
// Partition Descriptor: partition 0 at sector 260
int pd = 33 * SectorSize;
WriteUInt32(image, pd + 16, 1);
image[pd + 22] = 0;
WriteUInt32(image, pd + 188, PartitionStart);
WriteUInt32(image, pd + 192, mediaPastEnd ? PartitionLength + 20 : PartitionLength);
WriteTag(image, pd, 5, 33, 512);
// Logical Volume Descriptor: block size, domain, File Set Descriptor at block 0, one Type 1 map
int lvd = 34 * SectorSize;
WriteUInt32(image, lvd + 16, 2);
WriteDString(image, lvd + 84, 128, "TESTVOLUME");
WriteUInt32(image, lvd + 212, SectorSize);
WriteAscii(image, lvd + 217, "*OSTA UDF Compliant");
image[lvd + 240] = 0x02;
image[lvd + 241] = 0x01;
WriteLongAd(image, lvd + 248, SectorSize, 0);
WriteUInt32(image, lvd + 264, 6);
WriteUInt32(image, lvd + 268, emptyPartitionMap ? 3 : 1);
image[lvd + 440] = 1;
image[lvd + 441] = (byte)(emptyPartitionMap ? 0 : 6);
image[lvd + 442] = 1;
WriteTag(image, lvd, 6, 34, 446);
// Terminating Descriptor, or a Volume Descriptor Pointer back to the start of the sequence
int td = 35 * SectorSize;
if (pointerLoop)
{
WriteUInt32(image, td + 20, 4 * SectorSize);
WriteUInt32(image, td + 24, 32);
WriteTag(image, td, 3, 35, 512);
}
else
{
WriteTag(image, td, 8, 35, 512);
}
// File Set Descriptor: root directory ICB at block 2
int fsd = Block(0);
WriteDString(image, fsd + 304, 32, "TESTSET");
WriteLongAd(image, fsd + 400, SectorSize, 2);
WriteTag(image, fsd, 256, 0, 512);
// Root directory contents
int root = Block(3);
int length = 0;
length += WriteFid(image, root + length, 3, 0x0A, 2, []);
length += WriteFid(image, root + length, 3, 0x00, 4, Name8("README.TXT"));
length += WriteFid(image, root + length, 3, 0x04, 4, Name8("deleted.txt"));
length += WriteFid(image, root + length, 3, 0x02, 5, Name8(subName));
length += WriteFid(image, root + length, 3, 0x00, 11, Name8("Media"));
WriteFileEntry(image, 2, 4, 0, length, ShortAd(length, 3));
if (corruptRootEntry)
image[Block(2) + 4] ^= 0xFF;
if (corruptRootEntryCrc)
image[Block(2) + 36] ^= 0xFF;
// "README.TXT", embedded
WriteFileEntry(image, 4, 5, 3, 11, Encoding.ASCII.GetBytes("hello world"));
// "Sub" contents
int sub = Block(6);
length = 0;
length += WriteFid(image, sub + length, 6, 0x0A, 2, []);
length += WriteFid(image, sub + length, 6, 0x00, 7, Name16("data.bin"));
if (subLoop)
length += WriteFid(image, sub + length, 6, 0x02, 2, Name8("loop"));
WriteFileEntry(image, 5, 4, 0, length, ShortAd(length, 6));
// "data.bin", long allocation descriptors continued in an Allocation Extent Descriptor
var ads = new byte[32];
WriteLongAd(ads, 0, SectorSize, 8);
WriteLongAd(ads, 16, (3u << 30) | SectorSize, 10);
WriteFileEntry(image, 7, 5, 1, DataBinLength, ads);
for (int i = 0; i < DataBinLength; i++)
{
image[Block(8) + i] = (byte)(i % 251);
}
int aed = Block(10);
WriteUInt32(image, aed + 20, 16);
WriteLongAd(image, aed + 24, DataBinLength - SectorSize, 9);
WriteTag(image, aed, 258, 10, 40);
// "Media", allocated but not recorded, or recorded past the end of the image
if (mediaPastEnd)
WriteFileEntry(image, 11, 5, 0, 100, ShortAd(100, PartitionLength + 10));
else
WriteFileEntry(image, 11, 5, 0, 100, ShortAd((1u << 30) | 100, 0));
return image;
}
/// <summary>
/// Offset of a partition block in the image
/// </summary>
public static int Block(int block) => (PartitionStart + block) * SectorSize;
private static void WriteStructure(byte[] image, int sector, string identifier)
{
int offset = sector * SectorSize;
image[offset] = 0;
WriteAscii(image, offset + 1, identifier);
image[offset + 6] = 1;
}
private static int WriteFid(byte[] image, int offset, uint block, byte characteristics, uint icb, byte[] name)
{
image[offset + 16] = 1;
image[offset + 18] = characteristics;
image[offset + 19] = (byte)name.Length;
WriteLongAd(image, offset + 20, SectorSize, icb);
Array.Copy(name, 0, image, offset + 38, name.Length);
int length = (38 + name.Length + 3) & ~3;
WriteTag(image, offset, 257, block, length);
return length;
}
private static void WriteFileEntry(byte[] image, int block, byte fileType, ushort adType, long informationLength, byte[] ads)
{
int offset = Block(block);
image[offset + 16 + 4] = 4;
image[offset + 16 + 11] = fileType;
image[offset + 16 + 18] = (byte)adType;
WriteUInt32(image, offset + 56, (uint)informationLength);
WriteUInt32(image, offset + 172, (uint)ads.Length);
Array.Copy(ads, 0, image, offset + 176, ads.Length);
WriteTag(image, offset, 261, (uint)block, 176 + ads.Length);
}
private static byte[] ShortAd(uint length, uint position)
{
var ad = new byte[8];
WriteUInt32(ad, 0, length);
WriteUInt32(ad, 4, position);
return ad;
}
private static byte[] ShortAd(int length, uint position) => ShortAd((uint)length, position);
private static void WriteLongAd(byte[] data, int offset, uint length, uint block)
{
WriteUInt32(data, offset, length);
WriteUInt32(data, offset + 4, block);
}
private static void WriteLongAd(byte[] data, int offset, int length, uint block)
=> WriteLongAd(data, offset, (uint)length, block);
private static byte[] Name8(string name)
{
var bytes = new byte[name.Length + 1];
bytes[0] = 8;
Encoding.ASCII.GetBytes(name, 0, name.Length, bytes, 1);
return bytes;
}
private static byte[] Name16(string name)
{
var bytes = new byte[(name.Length * 2) + 1];
bytes[0] = 16;
Encoding.BigEndianUnicode.GetBytes(name, 0, name.Length, bytes, 1);
return bytes;
}
private static void WriteDString(byte[] image, int offset, int fieldLength, string value)
{
image[offset] = 8;
WriteAscii(image, offset + 1, value);
image[offset + fieldLength - 1] = (byte)(value.Length + 1);
}
private static void WriteAscii(byte[] image, int offset, string value)
=> Encoding.ASCII.GetBytes(value, 0, value.Length, image, offset);
private static void WriteUInt32(byte[] data, int offset, uint value)
{
data[offset] = (byte)value;
data[offset + 1] = (byte)(value >> 8);
data[offset + 2] = (byte)(value >> 16);
data[offset + 3] = (byte)(value >> 24);
}
private static void WriteUInt32(byte[] data, int offset, int value) => WriteUInt32(data, offset, (uint)value);
/// <summary>
/// Write a descriptor tag covering the given descriptor length (version 2, CRC, checksum)
/// </summary>
private static void WriteTag(byte[] image, int offset, ushort identifier, uint location, int length)
{
image[offset] = (byte)identifier;
image[offset + 1] = (byte)(identifier >> 8);
image[offset + 2] = 2;
ushort crc = Crc(image, offset + 16, length - 16);
image[offset + 8] = (byte)crc;
image[offset + 9] = (byte)(crc >> 8);
image[offset + 10] = (byte)(length - 16);
image[offset + 11] = (byte)((length - 16) >> 8);
WriteUInt32(image, offset + 12, location);
byte checksum = 0;
for (int i = 0; i < 16; i++)
{
if (i != 4)
checksum += image[offset + i];
}
image[offset + 4] = checksum;
}
private static ushort Crc(byte[] data, int offset, int length)
{
ushort crc = 0;
for (int i = offset; i < offset + length; i++)
{
crc ^= (ushort)(data[i] << 8);
for (int bit = 0; bit < 8; bit++)
{
crc = (crc & 0x8000) != 0 ? (ushort)((crc << 1) ^ 0x1021) : (ushort)(crc << 1);
}
}
return crc;
}
}
}

View File

@@ -0,0 +1,289 @@
using System.IO;
using System.Linq;
using System.Text;
using SabreTools.Data.Extensions;
using SabreTools.Data.Models.UDF;
using Xunit;
namespace SabreTools.Serialization.Readers.Test
{
public class UDFTests
{
[Fact]
public void NullArray_Null()
{
byte[]? data = null;
int offset = 0;
var deserializer = new UDF();
var actual = deserializer.Deserialize(data, offset);
Assert.Null(actual);
}
[Fact]
public void EmptyArray_Null()
{
byte[]? data = [];
int offset = 0;
var deserializer = new UDF();
var actual = deserializer.Deserialize(data, offset);
Assert.Null(actual);
}
[Fact]
public void InvalidArray_Null()
{
byte[]? data = [.. Enumerable.Repeat<byte>(0xFF, 1024)];
int offset = 0;
var deserializer = new UDF();
var actual = deserializer.Deserialize(data, offset);
Assert.Null(actual);
}
[Fact]
public void NullStream_Null()
{
Stream? data = null;
var deserializer = new UDF();
var actual = deserializer.Deserialize(data);
Assert.Null(actual);
}
[Fact]
public void EmptyStream_Null()
{
Stream? data = new MemoryStream([]);
var deserializer = new UDF();
var actual = deserializer.Deserialize(data);
Assert.Null(actual);
}
[Fact]
public void InvalidStream_Null()
{
Stream? data = new MemoryStream([.. Enumerable.Repeat<byte>(0xFF, 1024)]);
var deserializer = new UDF();
var actual = deserializer.Deserialize(data);
Assert.Null(actual);
}
[Fact]
public void ValidImage_VolumeStructures()
{
var volume = new UDF().Deserialize(new MemoryStream(UDFImageBuilder.Build()));
Assert.NotNull(volume);
Assert.Equal(3, volume.VolumeRecognitionSequence.Length);
Assert.NotNull(volume.AnchorVolumeDescriptorPointer);
Assert.Equal(4, volume.VolumeDescriptorSequence.Length);
var terminator = Assert.IsType<GenericDescriptor>(volume.VolumeDescriptorSequence[3]);
Assert.Equal(TagIdentifier.TERMINATING_DESCRIPTOR, terminator.DescriptorTag.TagIdentifier);
var lvd = volume.GetLogicalVolumeDescriptor();
Assert.NotNull(lvd);
Assert.Equal("TESTVOLUME", lvd.LogicalVolumeIdentifier.DecodeDString());
Assert.IsType<Type1PartitionMap>(Assert.Single(lvd.PartitionMaps));
Assert.NotNull(volume.FileSetDescriptor);
Assert.Equal("TESTSET", volume.FileSetDescriptor.FileSetIdentifier.DecodeDString());
Assert.Equal(2u, volume.FileSetDescriptor.RootDirectoryICB.ExtentLocation.LogicalBlockNumber);
}
[Fact]
public void ValidImage_DirectoryHierarchy()
{
var volume = new UDF().Deserialize(new MemoryStream(UDFImageBuilder.Build()));
Assert.NotNull(volume);
// Root, "README.TXT", "Sub", "data.bin", "Media"; the deleted entry is not followed
Assert.Equal(5, volume.FileEntries.Count);
Assert.Equal(2, volume.Directories.Count);
Assert.Single(volume.AllocationExtentDescriptors);
// The last root entry has no padding and ends exactly at the end of the directory
var root = volume.Directories[UDFExtensions.GetKey(0, 2)];
Assert.Equal(5, root.Length);
Assert.Equal("Media", root[4].FileIdentifier.DecodeDCharacters());
Assert.Empty(root[4].Padding);
var sub = volume.Directories[UDFExtensions.GetKey(0, 5)];
Assert.Equal("data.bin", sub[1].FileIdentifier.DecodeDCharacters());
}
[Fact]
public void FindFileEntry_EmbeddedData()
{
var data = new MemoryStream(UDFImageBuilder.Build());
var volume = new UDF().Deserialize(data);
Assert.NotNull(volume);
var entry = UDF.FindFileEntry(data, volume, "README.TXT", out long key);
Assert.NotNull(entry);
Assert.Equal(UDFExtensions.GetKey(0, 4), key);
Assert.Equal(AllocationDescriptorType.EMBEDDED, entry.ICBTag.GetAllocationDescriptorType());
Assert.Equal("hello world", Encoding.ASCII.GetString(entry.AllocationDescriptors, 0, (int)entry.InformationLength));
}
[Fact]
public void FindFileEntry_CaseInsensitive()
{
var data = new MemoryStream(UDFImageBuilder.Build());
var volume = new UDF().Deserialize(data);
Assert.NotNull(volume);
var entry = UDF.FindFileEntry(data, volume, "/sub\\DATA.BIN", out long key);
Assert.NotNull(entry);
Assert.Equal(UDFExtensions.GetKey(0, 7), key);
}
[Fact]
public void FindFileEntry_DeletedOrMissing_Null()
{
var data = new MemoryStream(UDFImageBuilder.Build());
var volume = new UDF().Deserialize(data);
Assert.NotNull(volume);
Assert.Null(UDF.FindFileEntry(data, volume, "deleted.txt", out long deletedKey));
Assert.Equal(-1, deletedKey);
Assert.Null(UDF.FindFileEntry(data, volume, "Sub/missing.bin", out _));
Assert.Null(UDF.FindFileEntry(data, volume, "README.TXT/data.bin", out _));
}
[Fact]
public void FindFileEntry_WithoutDirectories_ReadsOnlyThePath()
{
var data = new MemoryStream(UDFImageBuilder.Build());
var volume = new UDF().Deserialize(data, includeDirectories: false);
Assert.NotNull(volume);
Assert.Empty(volume.FileEntries);
Assert.Empty(volume.Directories);
var entry = UDF.FindFileEntry(data, volume, "Sub/data.bin", out long key);
Assert.NotNull(entry);
Assert.Equal(UDFExtensions.GetKey(0, 7), key);
// Root, "Sub" and "data.bin" only
Assert.Equal(3, volume.FileEntries.Count);
Assert.Equal(2, volume.Directories.Count);
Assert.False(volume.FileEntries.ContainsKey(UDFExtensions.GetKey(0, 4)));
}
[Fact]
public void AllocationExtent_Followed()
{
byte[] image = UDFImageBuilder.Build();
var data = new MemoryStream(image);
var volume = new UDF().Deserialize(data);
Assert.NotNull(volume);
long key = UDFExtensions.GetKey(0, 7);
var descriptors = volume.GetAllocationDescriptors(key, volume.FileEntries[key]);
Assert.NotNull(descriptors);
Assert.Equal(2, descriptors.Count);
Assert.Equal(8u, descriptors[0].ExtentLocation.LogicalBlockNumber);
Assert.Equal(9u, descriptors[1].ExtentLocation.LogicalBlockNumber);
Assert.Equal((uint)UDFImageBuilder.DataBinLength, UDFExtensions.GetExtentLength(descriptors[0].ExtentLength) + UDFExtensions.GetExtentLength(descriptors[1].ExtentLength));
Assert.True(volume.TryGetSector(descriptors[1].ExtentLocation.GetKey(), out long sector));
Assert.Equal(UDFImageBuilder.PartitionStart + 9, sector);
}
[Fact]
public void NotRecordedExtent_Kept()
{
var volume = new UDF().Deserialize(new MemoryStream(UDFImageBuilder.Build()));
Assert.NotNull(volume);
long key = UDFExtensions.GetKey(0, 11);
var descriptors = volume.GetAllocationDescriptors(key, volume.FileEntries[key]);
Assert.NotNull(descriptors);
var ad = Assert.Single(descriptors);
Assert.Equal(ExtentType.NOT_RECORDED_BUT_ALLOCATED, UDFExtensions.GetExtentType(ad.ExtentLength));
Assert.Equal(100u, UDFExtensions.GetExtentLength(ad.ExtentLength));
}
[Fact]
public void CorruptRootEntry_NoDirectories()
{
var volume = new UDF().Deserialize(new MemoryStream(UDFImageBuilder.Build(corruptRootEntry: true)));
Assert.NotNull(volume);
Assert.NotNull(volume.FileSetDescriptor);
Assert.Empty(volume.FileEntries);
Assert.Empty(volume.Directories);
}
[Fact]
public void CorruptRootEntryCrc_NoDirectories()
{
var volume = new UDF().Deserialize(new MemoryStream(UDFImageBuilder.Build(corruptRootEntryCrc: true)));
Assert.NotNull(volume);
Assert.NotNull(volume.FileSetDescriptor);
Assert.Empty(volume.FileEntries);
Assert.Empty(volume.Directories);
}
[Fact]
public void EmptyPartitionMap_EndsMapTable()
{
var data = new MemoryStream(UDFImageBuilder.Build(emptyPartitionMap: true));
var anchor = UDF.ParseAnchorVolumeDescriptorPointer(data, Constants.AnchorVolumeDescriptorPointerSector);
Assert.NotNull(anchor);
var sequence = UDF.ParseVolumeDescriptorSequence(data, anchor.MainVolumeDescriptorSequenceExtent);
var lvd = Assert.Single(sequence.OfType<LogicalVolumeDescriptor>());
Assert.Empty(lvd.PartitionMaps);
}
[Fact]
public void VolumeDescriptorPointerLoop_Stops()
{
var volume = new UDF().Deserialize(new MemoryStream(UDFImageBuilder.Build(pointerLoop: true)));
Assert.NotNull(volume);
Assert.Equal(5, volume.FileEntries.Count);
}
[Fact]
public void UnreadableSector_OtherEntriesRead()
{
// The "Sub" File Entry (block 5) cannot be read; "README.TXT" (block 4) is read in the same request
byte[] image = UDFImageBuilder.Build();
long bad = UDFImageBuilder.Block(5);
var volume = new UDF().Deserialize(new ThrowingStream(image, bad, bad + UDFImageBuilder.SectorSize));
// Root, "README.TXT" and "Media"
long[] expected = [UDFExtensions.GetKey(0, 2), UDFExtensions.GetKey(0, 4), UDFExtensions.GetKey(0, 11)];
Assert.NotNull(volume);
Assert.Equal(expected, volume.FileEntries.Keys.OrderBy(k => k));
Assert.Single(volume.Directories);
}
/// <summary>
/// Stream that throws an IOException for reads touching a range, like a drive with an unreadable sector
/// </summary>
private sealed class ThrowingStream(byte[] data, long badStart, long badEnd) : MemoryStream(data)
{
public override int Read(byte[] buffer, int offset, int count)
{
if (Position < badEnd && Position + count > badStart)
throw new IOException("Unreadable sector");
return base.Read(buffer, offset, count);
}
}
}
}

View File

@@ -1,5 +1,7 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.Text;
using SabreTools.Data.Extensions;
using SabreTools.Data.Models.ISO9660;
using SabreTools.IO.Extensions;
@@ -589,6 +591,10 @@ namespace SabreTools.Serialization.Readers
/// <summary>
/// Parse a Stream into a map of sector numbers to Directory
/// </summary>
/// <remarks>
/// The directory trees of all volume descriptors are read together, in the order of their extents on the volume:
/// always the next known extent at or after the last one read, wrapping around to the lowest one at the end
/// </remarks>
/// <param name="data">Stream to parse</param>
/// <param name="sectorLength">Number of bytes in a logical sector (usually 2048)</param>
/// <param name="vd">Set of volume descriptors for a volume</param>
@@ -596,6 +602,8 @@ namespace SabreTools.Serialization.Readers
public static Dictionary<int, FileExtent>? ParseDirectoryDescriptors(Stream data, short sectorLength, VolumeDescriptor[] vdSet)
{
var directories = new Dictionary<int, FileExtent>();
var parsed = new Dictionary<long, bool>();
var pending = new List<PendingExtent>();
foreach (VolumeDescriptor vd in vdSet)
{
DirectoryRecord rootDirectoryRecord;
@@ -609,17 +617,25 @@ namespace SabreTools.Serialization.Readers
// Determine logical block size
short blockLength = vd.GetLogicalBlockSize(sectorLength);
// Parse the root directory pointed to from the base volume descriptor
var descriptors = ParseDirectory(data, sectorLength, blockLength, rootDirectoryRecord, false);
if (descriptors is null || descriptors.Count == 0)
// Queue the root directory pointed to from the base volume descriptor
QueueExtent(data, blockLength, rootDirectoryRecord, pending);
}
// Read the queued extents in volume order
long position = 0;
while (pending.Count > 0)
{
int index = FindNextExtent(pending, position);
var extent = pending[index];
pending.RemoveAt(index);
position = extent.Offset + extent.Length;
// Each extent is read once
if (parsed.ContainsKey(extent.Offset))
continue;
// Merge dictionaries
foreach (var kvp in descriptors)
{
if (!directories.ContainsKey(kvp.Key))
directories.Add(kvp.Key, kvp.Value);
}
parsed[extent.Offset] = true;
ParseExtent(data, sectorLength, extent, directories, pending);
}
// Return error (null) if no valid directory descriptors were found
@@ -629,6 +645,333 @@ namespace SabreTools.Serialization.Readers
return directories;
}
/// <summary>
/// Queue the extent a directory record points to, if it needs to be read
/// </summary>
/// <param name="data">Stream to parse</param>
/// <param name="blockLength">Number of bytes in a logical block (usually 2048)</param>
/// <param name="dr">Directory record pointing to the extent</param>
/// <param name="pending">Queued extents, sorted by offset</param>
private static void QueueExtent(Stream data, short blockLength, DirectoryRecord dr, List<PendingExtent> pending)
{
// Only directories and files with an extended attribute record are read
if ((dr.FileFlags & FileFlags.DIRECTORY) != FileFlags.DIRECTORY && dr.ExtendedAttributeRecordLength == 0)
return;
QueueExtent(data, blockLength, dr, false, pending);
// If the extent location field is ambiguous, also queue the big-endian extent
if (!dr.ExtentLocation.IsValid)
QueueExtent(data, blockLength, dr, true, pending);
}
/// <summary>
/// Queue the extent a directory record points to, using one endianness
/// </summary>
/// <param name="data">Stream to parse</param>
/// <param name="blockLength">Number of bytes in a logical block (usually 2048)</param>
/// <param name="dr">Directory record pointing to the extent</param>
/// <param name="bigEndian">True if the Big Endian extent location/length should be used</param>
/// <param name="pending">Queued extents, sorted by offset</param>
private static void QueueExtent(Stream data, short blockLength, DirectoryRecord dr, bool bigEndian, List<PendingExtent> pending)
{
if (!TryGetExtent(data, blockLength, dr, bigEndian, out int extentLocation, out long extentOffset, out uint extentLength))
return;
var extent = new PendingExtent(extentLocation, extentOffset, extentLength, blockLength, dr);
int index = pending.BinarySearch(extent, PendingExtentComparer.Instance);
pending.Insert(index < 0 ? ~index : index, extent);
}
/// <summary>
/// Find the first queued extent at or after a position, or the first one if there is none
/// </summary>
/// <param name="pending">Queued extents, sorted by offset</param>
/// <param name="position">Offset just after the last extent read</param>
/// <returns>Index of the next extent to read</returns>
private static int FindNextExtent(List<PendingExtent> pending, long position)
{
var probe = new PendingExtent(0, position, 0, 0, null);
int index = pending.BinarySearch(probe, PendingExtentComparer.Instance);
if (index < 0)
{
index = ~index;
}
else
{
while (index > 0 && pending[index - 1].Offset == position)
{
index--;
}
}
return index < pending.Count ? index : 0;
}
/// <summary>
/// Parse one queued extent and queue the extents its directory records point to
/// </summary>
/// <param name="data">Stream to parse</param>
/// <param name="sectorLength">Number of bytes in a logical sector (usually 2048)</param>
/// <param name="extent">Extent to parse</param>
/// <param name="directories">Map of sector numbers to parsed extents</param>
/// <param name="pending">Queued extents, sorted by offset</param>
private static void ParseExtent(Stream data, short sectorLength, PendingExtent extent, Dictionary<int, FileExtent> directories, List<PendingExtent> pending)
{
var dr = extent.Record!;
// Move stream to extent location
data.SeekIfPossible(extent.Offset, SeekOrigin.Begin);
// Check if the current extent is a directory
if ((dr.FileFlags & FileFlags.DIRECTORY) == FileFlags.DIRECTORY)
{
// Read all directory records in this directory
var records = ParseDirectoryRecords(data, sectorLength, extent.Length);
if (records is null)
return;
// Add current directory to dictionary
var currentDirectory = new DirectoryExtent();
currentDirectory.DirectoryRecords = [.. records];
if (!directories.ContainsKey(extent.Location))
directories.Add(extent.Location, currentDirectory);
// Queue all child extents
foreach (var record in records)
{
// Don't traverse to parent or self
if (record.FileIdentifier.EqualsExactly(Constants.CurrentDirectory) || record.FileIdentifier.EqualsExactly(Constants.ParentDirectory))
continue;
QueueExtent(data, extent.BlockLength, record, pending);
}
}
else if (dr.ExtendedAttributeRecordLength > 0)
{
// Extent is a file, parse the Extended Attribute Record
var fileExtent = new FileExtent();
var ear = ParseExtendedAttributeRecord(data);
if (ear is not null)
fileExtent.ExtendedAttributeRecord = ear;
// Put the file extent is the dictionary
if (!directories.ContainsKey(extent.Location))
directories.Add(extent.Location, fileExtent);
}
}
/// <summary>
/// Get the extent a directory record points to, using one endianness
/// </summary>
/// <param name="data">Stream to parse</param>
/// <param name="blockLength">Number of bytes in a logical block (usually 2048)</param>
/// <param name="dr">Directory record pointing to the extent</param>
/// <param name="bigEndian">True if the Big Endian extent location/length should be used</param>
/// <param name="extentLocation">Logical block number of the extent</param>
/// <param name="extentOffset">Offset of the extent in the stream</param>
/// <param name="extentLength">Number of bytes in the extent</param>
/// <returns>True if the extent is valid, false otherwise</returns>
private static bool TryGetExtent(Stream data, short blockLength, DirectoryRecord dr, bool bigEndian, out int extentLocation, out long extentOffset, out uint extentLength)
{
// Use provided extent endinanness
extentLocation = bigEndian ? dr.ExtentLocation.BigEndian : dr.ExtentLocation.LittleEndian;
extentLength = bigEndian ? dr.ExtentLength.BigEndian : dr.ExtentLength.LittleEndian;
extentOffset = extentLocation * (long)blockLength;
long extentFinal = extentOffset + extentLength;
// Deal with extent length ambiguity
if (!dr.ExtentLength.IsValid)
{
// If provided extent length is invalid, use the other value
if (extentLength <= 0 || extentFinal > data.Length)
extentLength = bigEndian ? dr.ExtentLength.LittleEndian : dr.ExtentLength.BigEndian;
}
// Validate extent length
return extentLength > 0 && extentFinal <= data.Length;
}
/// <summary>
/// Parse the directory records of one directory extent
/// </summary>
/// <param name="data">Stream to parse, positioned at the start of the directory extent</param>
/// <param name="sectorLength">Number of bytes in a logical sector (usually 2048)</param>
/// <param name="extentLength">Number of bytes in the directory extent</param>
/// <returns>Filled list of DirectoryRecord on success, null if the directory is invalid</returns>
private static List<DirectoryRecord>? ParseDirectoryRecords(Stream data, short sectorLength, uint extentLength)
{
// Start of directory should not be 0
int firstRecordLength = data.PeekByteValue();
if (firstRecordLength == 0)
return null;
// Read all directory records in this directory
var records = new List<DirectoryRecord>();
uint pos = 0;
while (pos < extentLength)
{
// Peek next byte to check whether the next record length is not greater than the end of the dir extent
var recordLength = data.PeekByteValue();
// If record length of 0x00, next record begins in next sector
if (recordLength == 0)
{
int paddingLength = (int)((uint)sectorLength - (pos % (uint)sectorLength));
pos += (uint)paddingLength;
_ = data.ReadBytes(paddingLength);
// Finish parsing records if end reached
if (pos >= extentLength)
break;
// Start of sector should not be 0, ignore entire directory
int nextRecordLength = data.PeekByteValue();
if (nextRecordLength <= 0)
return null;
continue;
}
// Ensure record will end in this extent
// TODO: Smartly detect record length for invalid record lengths
pos += recordLength;
if (pos > extentLength)
break;
// Get the next directory record
var directoryRecord = ParseDirectoryRecord(data, false);
// Compare recordLength with number of bytes in directoryRecord and ignore the directory if mismatch
var readLength = 33 + directoryRecord.FileIdentifier.Length + (directoryRecord.PaddingField is null ? 0 : 1) + directoryRecord.SystemUse.Length;
if (readLength != recordLength)
return null;
records.Add(directoryRecord);
}
return records;
}
/// <summary>
/// Find the directory records of one path, reading only the directories along the path
/// </summary>
/// <remarks>
/// Names are compared case-insensitively and without a version suffix (";1") or a trailing period.
/// A file recorded in several extents returns one directory record per extent, in recorded order.
/// </remarks>
/// <param name="data">Stream to parse</param>
/// <param name="sectorLength">Number of bytes in a logical sector (usually 2048)</param>
/// <param name="vd">Primary or supplementary volume descriptor whose directory hierarchy is searched</param>
/// <param name="path">Path from the root directory, separated by '/' or '\'</param>
/// <returns>Filled array of DirectoryRecord on success, null if the path was not found</returns>
public static DirectoryRecord[]? FindDirectoryRecords(Stream data, short sectorLength, VolumeDescriptor vd, string path)
{
DirectoryRecord current;
if (vd is PrimaryVolumeDescriptor pvd)
current = pvd.RootDirectoryRecord;
else if (vd is SupplementaryVolumeDescriptor svd)
current = svd.RootDirectoryRecord;
else
return null;
// Determine logical block size
short blockLength = vd.GetLogicalBlockSize(sectorLength);
// TODO: Better encoding detection (EscapeSequences)
var encoding = vd is SupplementaryVolumeDescriptor ? Encoding.BigEndianUnicode : Encoding.UTF8;
string[] names = path.Split(['/', '\\'], StringSplitOptions.RemoveEmptyEntries);
if (names.Length == 0)
return [current];
for (int i = 0; i < names.Length; i++)
{
// Only a directory can contain the next name
if ((current.FileFlags & FileFlags.DIRECTORY) != FileFlags.DIRECTORY)
return null;
// If the extent location field is ambiguous, also search the big-endian directory extent
var found = FindInDirectory(data, sectorLength, blockLength, current, false, encoding, names[i]);
if (found is null && !current.ExtentLocation.IsValid)
found = FindInDirectory(data, sectorLength, blockLength, current, true, encoding, names[i]);
if (found is null)
return null;
if (i == names.Length - 1)
return found;
current = found[0];
}
return null;
}
/// <summary>
/// Find the directory records of one name in one directory
/// </summary>
/// <param name="data">Stream to parse</param>
/// <param name="sectorLength">Number of bytes in a logical sector (usually 2048)</param>
/// <param name="blockLength">Number of bytes in a logical block (usually 2048)</param>
/// <param name="dr">Directory record pointing to the directory extent</param>
/// <param name="bigEndian">True if the Big Endian extent location/length should be used</param>
/// <param name="encoding">Encoding of the file identifiers</param>
/// <param name="name">Name to find</param>
/// <returns>Filled array of DirectoryRecord on success, null if the name was not found</returns>
private static DirectoryRecord[]? FindInDirectory(Stream data, short sectorLength, short blockLength, DirectoryRecord dr, bool bigEndian, Encoding encoding, string name)
{
if (!TryGetExtent(data, blockLength, dr, bigEndian, out _, out long extentOffset, out uint extentLength))
return null;
// Move stream to directory location
data.SeekIfPossible(extentOffset, SeekOrigin.Begin);
var records = ParseDirectoryRecords(data, sectorLength, extentLength);
if (records is null)
return null;
string wanted = TrimIdentifier(name);
for (int i = 0; i < records.Count; i++)
{
// Don't match parent or self
if (records[i].FileIdentifier.EqualsExactly(Constants.CurrentDirectory) || records[i].FileIdentifier.EqualsExactly(Constants.ParentDirectory))
continue;
if (!string.Equals(TrimIdentifier(encoding.GetString(records[i].FileIdentifier)), wanted, StringComparison.OrdinalIgnoreCase))
continue;
// A multi-extent file continues in the following records, up to the final one
var matches = new List<DirectoryRecord> { records[i] };
while ((records[i].FileFlags & FileFlags.MULTI_EXTENT) == FileFlags.MULTI_EXTENT && i + 1 < records.Count)
{
i++;
matches.Add(records[i]);
}
return [.. matches];
}
return null;
}
/// <summary>
/// Remove the version suffix and a trailing period from a file identifier
/// </summary>
/// <param name="identifier">File identifier as recorded or requested</param>
/// <returns>File identifier without version suffix and trailing period</returns>
private static string TrimIdentifier(string identifier)
{
int versionSeparator = identifier.LastIndexOf(';');
if (versionSeparator >= 0)
identifier = identifier.Substring(0, versionSeparator);
if (identifier.EndsWith("."))
identifier = identifier.Substring(0, identifier.Length - 1);
return identifier;
}
/// <summary>
/// Parse a Stream into a map of sector numbers to Directory
/// </summary>
@@ -671,54 +1014,10 @@ namespace SabreTools.Serialization.Readers
if (dr.FileFlags.HasFlag(FileFlags.DIRECTORY))
#endif
{
// Start of directory should not be 0
int firstRecordLength = data.PeekByteValue();
if (firstRecordLength == 0)
return null;
// Read all directory records in this directory
var records = new List<DirectoryRecord>();
uint pos = 0;
while (pos < extentLength)
{
// Peek next byte to check whether the next record length is not greater than the end of the dir extent
var recordLength = data.PeekByteValue();
// If record length of 0x00, next record begins in next sector
if (recordLength == 0)
{
int paddingLength = (int)((uint)sectorLength - (pos % (uint)sectorLength));
pos += (uint)paddingLength;
_ = data.ReadBytes(paddingLength);
// Finish parsing records if end reached
if (pos >= extentLength)
break;
// Start of sector should not be 0, ignore entire directory
int nextRecordLength = data.PeekByteValue();
if (nextRecordLength <= 0)
return null;
continue;
}
// Ensure record will end in this extent
// TODO: Smartly detect record length for invalid record lengths
pos += recordLength;
if (pos > extentLength)
break;
// Get the next directory record
var directoryRecord = ParseDirectoryRecord(data, false);
// Compare recordLength with number of bytes in directoryRecord and return null if mismatch
var readLength = 33 + directoryRecord.FileIdentifier.Length + (directoryRecord.PaddingField is null ? 0 : 1) + directoryRecord.SystemUse.Length;
if (readLength != recordLength)
return null;
records.Add(directoryRecord);
}
var records = ParseDirectoryRecords(data, sectorLength, extentLength);
if (records is null)
return null;
// Add current directory to dictionary
var currentDirectory = new DirectoryExtent();
@@ -770,7 +1069,7 @@ namespace SabreTools.Serialization.Readers
}
// If the extent location field is ambiguous, also parse the big-endian directory extent
if (!bigEndian && dr.ExtentLocation.IsValid)
if (!bigEndian && !dr.ExtentLocation.IsValid)
{
var bigEndianDir = ParseDirectory(data, sectorLength, blockLength, dr, true);
if (bigEndianDir is not null)
@@ -936,5 +1235,60 @@ namespace SabreTools.Serialization.Readers
return obj;
}
/// <summary>
/// Extent queued for reading
/// </summary>
private sealed class PendingExtent
{
/// <summary>
/// Logical block number of the extent
/// </summary>
public int Location { get; }
/// <summary>
/// Byte offset of the extent
/// </summary>
public long Offset { get; }
/// <summary>
/// Length of the extent in bytes
/// </summary>
public uint Length { get; }
/// <summary>
/// Number of bytes in a logical block of the volume
/// </summary>
public short BlockLength { get; }
/// <summary>
/// Directory record pointing to the extent
/// </summary>
public DirectoryRecord? Record { get; }
public PendingExtent(int location, long offset, uint length, short blockLength, DirectoryRecord? record)
{
Location = location;
Offset = offset;
Length = length;
BlockLength = blockLength;
Record = record;
}
}
/// <summary>
/// Orders queued extents by offset
/// </summary>
private sealed class PendingExtentComparer : IComparer<PendingExtent>
{
/// <summary>
/// Shared instance
/// </summary>
public static readonly PendingExtentComparer Instance = new();
/// <inheritdoc/>
public int Compare(PendingExtent? x, PendingExtent? y)
=> (x?.Offset ?? 0).CompareTo(y?.Offset ?? 0);
}
}
}

View File

@@ -226,6 +226,11 @@ namespace SabreTools.Serialization.Readers
else
return null;
obj.SHA1 = reader.GetAttribute("sha1");
obj.Type = reader.GetAttribute("type");
obj.Status = reader.GetAttribute("status");
obj.Remark = reader.GetAttribute("remark");
// Handle empty elements
if (reader.IsEmptyElement)
return obj;
@@ -253,11 +258,12 @@ namespace SabreTools.Serialization.Readers
switch (reader.Name)
{
case "start":
if (obj.Start is not null && Debug)
case "hash":
case "sha1":
if (obj.SHA1 is not null && Debug)
Console.WriteLine($"'{reader.Name}' element already found, overwriting");
obj.Start = reader.ReadElementContentAsString();
obj.SHA1 = reader.ReadElementContentAsString();
break;
case "type":
if (obj.Type is not null && Debug)
@@ -265,11 +271,11 @@ namespace SabreTools.Serialization.Readers
obj.Type = reader.ReadElementContentAsString();
break;
case "hash":
if (obj.Hash is not null && Debug)
case "status":
if (obj.Status is not null && Debug)
Console.WriteLine($"'{reader.Name}' element already found, overwriting");
obj.Hash = reader.ReadElementContentAsString();
obj.Status = reader.ReadElementContentAsString();
break;
case "remark":
if (obj.Remark is not null && Debug)
@@ -277,6 +283,12 @@ namespace SabreTools.Serialization.Readers
obj.Remark = reader.ReadElementContentAsString();
break;
case "start":
if (obj.Start is not null && Debug)
Console.WriteLine($"'{reader.Name}' element already found, overwriting");
obj.Start = reader.ReadElementContentAsString();
break;
default:
if (Debug) Console.Error.WriteLine($"Element '{reader.Name}' is not recognized");
@@ -297,11 +309,19 @@ namespace SabreTools.Serialization.Readers
{
var obj = new Software();
obj.Title = reader.GetAttribute("title");
obj.System = reader.GetAttribute("system");
obj.Company = reader.GetAttribute("company");
obj.Year = reader.GetAttribute("year");
obj.Country = reader.GetAttribute("country");
obj.GenMSXID = reader.GetAttribute("genmsxid");
// Handle empty elements
if (reader.IsEmptyElement)
return obj;
List<Dump> dumps = [];
List<RomBase> roms = [];
reader.Read();
while (!reader.EOF)
@@ -369,6 +389,13 @@ namespace SabreTools.Serialization.Readers
reader.Skip();
break;
case "rom":
var rom = ParseRomBase(reader);
if (rom is not null)
roms.Add(rom);
reader.Skip();
break;
default:
if (Debug) Console.Error.WriteLine($"Element '{reader.Name}' is not recognized");
@@ -379,6 +406,8 @@ namespace SabreTools.Serialization.Readers
if (dumps.Count > 0)
obj.Dump = [.. dumps];
if (roms.Count > 0)
obj.Rom = [.. roms];
return obj;
}

View File

@@ -472,13 +472,13 @@ namespace SabreTools.Serialization.Readers
#region File Data
// Try to read the file data
var fileData = data.ReadBytes((int)compressedSize);
if (fileData.Length < (long)compressedSize)
// The file data is skipped, so a large archive is not read into memory
if (compressedSize > (ulong)(data.Length - data.Position))
return null;
// Set the file data
obj.FileData = fileData;
// Set the location of the file data
obj.FileDataOffset = data.Position;
data.SeekIfPossible((long)compressedSize, SeekOrigin.Current);
#endregion

View File

@@ -2,7 +2,7 @@
<PropertyGroup>
<!-- Assembly Properties -->
<TargetFrameworks>net20;net35;net40;net452;net462;net472;net48;netcoreapp3.1;net5.0;net6.0;net7.0;net8.0;net9.0;net10.0;netstandard2.0;netstandard2.1</TargetFrameworks>
<TargetFrameworks>net20;net35;net40;net452;net462;net472;net48;netcoreapp3.1;net5.0;net6.0;net7.0;net8.0;net9.0;net10.0;net11.0;netstandard2.0;netstandard2.1</TargetFrameworks>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<CheckEolTargetFramework>false</CheckEolTargetFramework>
<IncludeSourceRevisionInInformationalVersion>false</IncludeSourceRevisionInInformationalVersion>
@@ -12,7 +12,7 @@
<SuppressTfmSupportBuildWarnings>true</SuppressTfmSupportBuildWarnings>
<SymbolPackageFormat>snupkg</SymbolPackageFormat>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
<Version>3.2.0</Version>
<Version>3.3.0</Version>
<!-- Package Properties -->
<Authors>Matt Nadareski</Authors>
@@ -36,8 +36,8 @@
<ItemGroup>
<PackageReference Include="Newtonsoft.Json" Version="13.0.4" />
<PackageReference Include="SabreTools.Hashing" Version="2.1.0" />
<PackageReference Include="SabreTools.IO" Version="2.3.0" />
<PackageReference Include="SabreTools.Hashing" Version="2.2.0" />
<PackageReference Include="SabreTools.IO" Version="2.4.0" />
</ItemGroup>
<ItemGroup>

File diff suppressed because it is too large Load Diff

View File

@@ -1,17 +1,24 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFrameworks>net8.0;net9.0;net10.0</TargetFrameworks>
<TargetFrameworks>net8.0;net9.0;net10.0;net11.0</TargetFrameworks>
<IsPackable>false</IsPackable>
<LangVersion>latest</LangVersion>
<Nullable>enable</Nullable>
<SuppressTfmSupportBuildWarnings>true</SuppressTfmSupportBuildWarnings>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="coverlet.collector" Version="10.0.1" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.9.0" />
<PackageReference Include="coverlet.collector" Version="10.1.0">
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
<PrivateAssets>all</PrivateAssets>
</PackageReference>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.11.1" />
<PackageReference Include="xunit" Version="2.9.3" />
<PackageReference Include="xunit.runner.visualstudio" Version="4.0.0" />
<PackageReference Include="xunit.runner.visualstudio" Version="4.0.1">
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
<PrivateAssets>all</PrivateAssets>
</PackageReference>
</ItemGroup>
<ItemGroup>

View File

@@ -1,3 +1,4 @@
using System.Globalization;
using System.IO;
using System.Text;
using SabreTools.Data.Extensions;
@@ -261,7 +262,7 @@ namespace SabreTools.Serialization.Writers
{
writer.WriteStartElement("rom");
writer.WriteRequiredAttributeString("name", rom.Name, throwOnError: true);
writer.WriteRequiredAttributeString("size", rom.Size?.ToString(), throwOnError: true);
writer.WriteOptionalAttributeString("size", rom.Size?.ToString());
writer.WriteOptionalAttributeString("crc16", rom.CRC16);
writer.WriteOptionalAttributeString("crc", rom.CRC);
writer.WriteOptionalAttributeString("crc64", rom.CRC64);
@@ -419,7 +420,7 @@ namespace SabreTools.Serialization.Writers
writer.WriteOptionalAttributeString("y", video.Y.ToString());
writer.WriteOptionalAttributeString("aspectx", video.AspectX.ToString());
writer.WriteOptionalAttributeString("aspecty", video.AspectY.ToString());
writer.WriteOptionalAttributeString("freq", video.Freq?.ToString("0.000000"));
writer.WriteOptionalAttributeString("freq", video.Freq?.ToString("0.000000", CultureInfo.InvariantCulture));
writer.WriteEndElement(); // video
}
}

View File

@@ -1,3 +1,4 @@
using System.Globalization;
using System.IO;
using System.Text;
using System.Xml;
@@ -395,7 +396,7 @@ namespace SabreTools.Serialization.Writers
writer.WriteOptionalAttributeString("flipx", obj.FlipX.FromYesNo());
writer.WriteOptionalAttributeString("width", obj.Width?.ToString());
writer.WriteOptionalAttributeString("height", obj.Height?.ToString());
writer.WriteRequiredAttributeString("refresh", obj.Refresh?.ToString("0.000000"));
writer.WriteRequiredAttributeString("refresh", obj.Refresh?.ToString("0.000000", CultureInfo.InvariantCulture));
writer.WriteOptionalAttributeString("pixclock", obj.PixClock?.ToString());
writer.WriteOptionalAttributeString("htotal", obj.HTotal?.ToString());
writer.WriteOptionalAttributeString("hbend", obj.HBEnd?.ToString());
@@ -843,7 +844,7 @@ namespace SabreTools.Serialization.Writers
writer.WriteOptionalAttributeString("height", obj.Height.ToString());
writer.WriteOptionalAttributeString("aspectx", obj.AspectX.ToString());
writer.WriteOptionalAttributeString("aspecty", obj.AspectY.ToString());
writer.WriteRequiredAttributeString("refresh", obj.Refresh?.ToString("0.000000"));
writer.WriteRequiredAttributeString("refresh", obj.Refresh?.ToString("0.000000", CultureInfo.InvariantCulture));
writer.WriteEndElement();
}

View File

@@ -517,7 +517,7 @@ namespace SabreTools.Serialization.Writers
writer.WriteStartElement("rom");
writer.WriteRequiredAttributeString("name", obj.Name);
writer.WriteRequiredAttributeString("size", obj.Size?.ToString());
writer.WriteOptionalAttributeString("size", obj.Size?.ToString());
writer.WriteOptionalAttributeString("crc16", obj.CRC16);
writer.WriteOptionalAttributeString("crc", obj.CRC);
writer.WriteOptionalAttributeString("crc64", obj.CRC64);

View File

@@ -1,3 +1,4 @@
using System.Globalization;
using System.IO;
using System.Text;
using System.Xml;
@@ -393,7 +394,7 @@ namespace SabreTools.Serialization.Writers
writer.WriteOptionalAttributeString("flipx", obj.FlipX.FromYesNo());
writer.WriteOptionalAttributeString("width", obj.Width?.ToString());
writer.WriteOptionalAttributeString("height", obj.Height?.ToString());
writer.WriteRequiredAttributeString("refresh", obj.Refresh?.ToString("0.000000"));
writer.WriteRequiredAttributeString("refresh", obj.Refresh?.ToString("0.000000", CultureInfo.InvariantCulture));
writer.WriteOptionalAttributeString("pixclock", obj.PixClock?.ToString());
writer.WriteOptionalAttributeString("htotal", obj.HTotal?.ToString());
writer.WriteOptionalAttributeString("hbend", obj.HBEnd?.ToString());
@@ -841,7 +842,7 @@ namespace SabreTools.Serialization.Writers
writer.WriteOptionalAttributeString("height", obj.Height.ToString());
writer.WriteOptionalAttributeString("aspectx", obj.AspectX.ToString());
writer.WriteOptionalAttributeString("aspecty", obj.AspectY.ToString());
writer.WriteRequiredAttributeString("refresh", obj.Refresh?.ToString("0.000000"));
writer.WriteRequiredAttributeString("refresh", obj.Refresh?.ToString("0.000000", CultureInfo.InvariantCulture));
writer.WriteEndElement();
}

View File

@@ -1,3 +1,4 @@
using System.Globalization;
using System.IO;
using System.Text;
using System.Xml;
@@ -393,7 +394,7 @@ namespace SabreTools.Serialization.Writers
writer.WriteOptionalAttributeString("flipx", obj.FlipX.FromYesNo());
writer.WriteOptionalAttributeString("width", obj.Width?.ToString());
writer.WriteOptionalAttributeString("height", obj.Height?.ToString());
writer.WriteRequiredAttributeString("refresh", obj.Refresh?.ToString("0.000000"));
writer.WriteRequiredAttributeString("refresh", obj.Refresh?.ToString("0.000000", CultureInfo.InvariantCulture));
writer.WriteOptionalAttributeString("pixclock", obj.PixClock?.ToString());
writer.WriteOptionalAttributeString("htotal", obj.HTotal?.ToString());
writer.WriteOptionalAttributeString("hbend", obj.HBEnd?.ToString());
@@ -841,7 +842,7 @@ namespace SabreTools.Serialization.Writers
writer.WriteOptionalAttributeString("height", obj.Height.ToString());
writer.WriteOptionalAttributeString("aspectx", obj.AspectX.ToString());
writer.WriteOptionalAttributeString("aspecty", obj.AspectY.ToString());
writer.WriteRequiredAttributeString("refresh", obj.Refresh?.ToString("0.000000"));
writer.WriteRequiredAttributeString("refresh", obj.Refresh?.ToString("0.000000", CultureInfo.InvariantCulture));
writer.WriteEndElement();
}

View File

@@ -141,10 +141,11 @@ namespace SabreTools.Serialization.Writers
else
return;
writer.WriteOptionalElementString("start", obj.Start);
writer.WriteOptionalElementString("sha1", obj.SHA1);
writer.WriteOptionalElementString("type", obj.Type);
writer.WriteOptionalElementString("hash", obj.Hash);
writer.WriteOptionalElementString("status", obj.Status);
writer.WriteOptionalElementString("remark", obj.Remark);
writer.WriteOptionalElementString("start", obj.Start);
writer.WriteEndElement();
}
@@ -158,12 +159,12 @@ namespace SabreTools.Serialization.Writers
{
writer.WriteStartElement("software");
writer.WriteRequiredElementString("title", obj.Title);
writer.WriteOptionalElementString("genmsxid", obj.GenMSXID);
writer.WriteRequiredElementString("system", obj.System);
writer.WriteRequiredElementString("company", obj.Company);
writer.WriteRequiredElementString("year", obj.Year);
writer.WriteRequiredElementString("country", obj.Country);
writer.WriteRequiredAttributeString("title", obj.Title);
writer.WriteRequiredAttributeString("system", obj.System);
writer.WriteRequiredAttributeString("company", obj.Company);
writer.WriteRequiredAttributeString("country", obj.Country);
writer.WriteRequiredAttributeString("year", obj.Year);
writer.WriteOptionalAttributeString("genmsxid", obj.GenMSXID);
if (obj.Dump is not null && obj.Dump.Length > 0)
{
@@ -173,6 +174,14 @@ namespace SabreTools.Serialization.Writers
}
}
if (obj.Rom is not null && obj.Rom.Length > 0)
{
foreach (var rom in obj.Rom)
{
WriteRomBase(rom, writer);
}
}
writer.WriteEndElement();
}
}

View File

@@ -2,7 +2,7 @@
<PropertyGroup>
<!-- Assembly Properties -->
<TargetFrameworks>net20;net35;net40;net452;net462;net472;net48;netcoreapp3.1;net5.0;net6.0;net7.0;net8.0;net9.0;net10.0;netstandard2.0;netstandard2.1</TargetFrameworks>
<TargetFrameworks>net20;net35;net40;net452;net462;net472;net48;netcoreapp3.1;net5.0;net6.0;net7.0;net8.0;net9.0;net10.0;net11.0;netstandard2.0;netstandard2.1</TargetFrameworks>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<CheckEolTargetFramework>false</CheckEolTargetFramework>
<IncludeSourceRevisionInInformationalVersion>false</IncludeSourceRevisionInInformationalVersion>
@@ -12,7 +12,7 @@
<SuppressTfmSupportBuildWarnings>true</SuppressTfmSupportBuildWarnings>
<SymbolPackageFormat>snupkg</SymbolPackageFormat>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
<Version>3.2.0</Version>
<Version>3.3.0</Version>
<!-- Package Properties -->
<Authors>Matt Nadareski</Authors>
@@ -32,8 +32,8 @@
<ItemGroup>
<PackageReference Include="Newtonsoft.Json" Version="13.0.4" />
<PackageReference Include="SabreTools.Hashing" Version="2.1.0" />
<PackageReference Include="SabreTools.IO" Version="2.3.0" />
<PackageReference Include="SabreTools.Hashing" Version="2.2.0" />
<PackageReference Include="SabreTools.IO" Version="2.4.0" />
</ItemGroup>
<ItemGroup>

View File

@@ -2,7 +2,7 @@
<PropertyGroup>
<!-- Assembly Properties -->
<TargetFrameworks>net20;net35;net40;net452;net462;net472;net48;netcoreapp3.1;net5.0;net6.0;net7.0;net8.0;net9.0;net10.0;netstandard2.0;netstandard2.1</TargetFrameworks>
<TargetFrameworks>net20;net35;net40;net452;net462;net472;net48;netcoreapp3.1;net5.0;net6.0;net7.0;net8.0;net9.0;net10.0;net11.0;netstandard2.0;netstandard2.1</TargetFrameworks>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<CheckEolTargetFramework>false</CheckEolTargetFramework>
<IncludeSourceRevisionInInformationalVersion>false</IncludeSourceRevisionInInformationalVersion>
@@ -12,7 +12,7 @@
<SuppressTfmSupportBuildWarnings>true</SuppressTfmSupportBuildWarnings>
<SymbolPackageFormat>snupkg</SymbolPackageFormat>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
<Version>3.2.0</Version>
<Version>3.3.0</Version>
<!-- Package Properties -->
<Authors>Matt Nadareski</Authors>
@@ -58,14 +58,14 @@
</ItemGroup>
<ItemGroup>
<PackageReference Include="GrindCore" Version="0.9.0" />
<PackageReference Include="GrindCore.SharpCompress" Version="0.50.5" Condition="!$(TargetFramework.StartsWith(`net2`)) AND !$(TargetFramework.StartsWith(`net3`)) AND !$(TargetFramework.StartsWith(`net40`)) AND !$(TargetFramework.StartsWith(`net452`))" />
<PackageReference Include="GrindCore" Version="0.10.0" />
<PackageReference Include="GrindCore.SharpCompress" Version="0.50.8" Condition="!$(TargetFramework.StartsWith(`net2`)) AND !$(TargetFramework.StartsWith(`net3`)) AND !$(TargetFramework.StartsWith(`net40`)) AND !$(TargetFramework.StartsWith(`net452`))" />
<PackageReference Include="MinValueTupleBridge" Version="0.2.1" Condition="$(TargetFramework.StartsWith(`net2`)) OR $(TargetFramework.StartsWith(`net3`))" />
<PackageReference Include="Net30.LinqBridge" Version="1.6.0.2" Condition="$(TargetFramework.StartsWith(`net2`))" />
<PackageReference Include="NetLegacySupport.Numerics" Version="1.0.1" Condition="$(TargetFramework.StartsWith(`net2`)) OR $(TargetFramework.StartsWith(`net3`))" />
<PackageReference Include="Newtonsoft.Json" Version="13.0.4" />
<PackageReference Include="SabreTools.Hashing" Version="2.1.0" />
<PackageReference Include="SabreTools.IO" Version="2.3.0" />
<PackageReference Include="SabreTools.Hashing" Version="2.2.0" />
<PackageReference Include="SabreTools.IO" Version="2.4.0" />
<PackageReference Include="System.ValueTuple" Version="4.5.0" Condition="$(TargetFramework.StartsWith(`net40`)) OR $(TargetFramework.StartsWith(`net45`))" />
<PackageReference Include="System.ValueTuple" Version="4.6.2" Condition="$(TargetFramework.StartsWith(`net46`)) OR $(TargetFramework.StartsWith(`net47`)) OR $(TargetFramework.StartsWith(`net48`))" />
</ItemGroup>

View File

@@ -1,3 +1,4 @@
using System;
using System.IO;
using System.Linq;
using Xunit;
@@ -56,5 +57,66 @@ namespace SabreTools.Wrappers.Test
var actual = MicrosoftCabinet.Create(data);
Assert.Null(actual);
}
#region Extract
[Fact]
public void Extract_LastBlockEndsTwoFiles()
{
// One MS-ZIP block that holds both files
string path = Path.Combine(Environment.CurrentDirectory, "TestData", "MicrosoftCabinet", "mszip.cab");
var cabinet = MicrosoftCabinet.Create(File.ReadAllBytes(path), 0);
Assert.NotNull(cabinet);
string outputDirectory = Path.Combine(Path.GetTempPath(), Path.GetRandomFileName());
cabinet.Extract(outputDirectory, includeDebug: true);
Assert.Equal("Hello, ", File.ReadAllText(Path.Combine(outputDirectory, "a.txt")));
Assert.Equal("World!", File.ReadAllText(Path.Combine(outputDirectory, "b.txt")));
Directory.Delete(outputDirectory, recursive: true);
}
[Theory]
[InlineData("single.cab", null)]
[InlineData("part1.cab", null)]
[InlineData("part2.cab", null)]
[InlineData("part1.cab", "part2.cab")]
public void Extract_FilesNotKeptOpen(string filename, string? invalidPart)
{
// Copy of the test cabinets, so they can be checked and deleted afterwards
string directory = Path.Combine(Path.GetTempPath(), Path.GetRandomFileName());
string outputDirectory = Path.Combine(directory, "output");
Directory.CreateDirectory(directory);
foreach (string file in Directory.GetFiles(Path.Combine(Environment.CurrentDirectory, "TestData", "MicrosoftCabinet")))
{
File.Copy(file, Path.Combine(directory, Path.GetFileName(file)));
}
if (invalidPart is not null)
File.WriteAllBytes(Path.Combine(directory, invalidPart), [.. Enumerable.Repeat<byte>(0xFF, 1024)]);
try
{
using (var stream = File.Open(Path.Combine(directory, filename), FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
var cabinet = MicrosoftCabinet.Create(stream);
Assert.NotNull(cabinet);
cabinet.Extract(outputDirectory, includeDebug: false);
}
// Opening a file exclusively fails while another handle to it is still open
foreach (string file in Directory.GetFiles(directory))
{
using var exclusive = File.Open(file, FileMode.Open, FileAccess.Read, FileShare.None);
Assert.True(exclusive.CanRead);
}
}
finally
{
Directory.Delete(directory, recursive: true);
}
}
#endregion
}
}

View File

@@ -56,5 +56,32 @@ namespace SabreTools.Wrappers.Test
var actual = PortableExecutable.Create(data);
Assert.Null(actual);
}
[Fact]
public void ExtractWiseOverlay_NotWise_FileNotKeptOpen()
{
// Any executable that is not a Wise installer, here a copy of this test assembly
string path = Path.Combine(Path.GetTempPath(), Path.GetRandomFileName());
string outputDirectory = Path.Combine(Path.GetTempPath(), Path.GetRandomFileName());
File.Copy(typeof(PortableExecutableTests).Assembly.Location, path);
try
{
using (var stream = File.Open(path, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
var pe = PortableExecutable.Create(stream);
Assert.NotNull(pe);
Assert.False(pe.ExtractWiseOverlay(outputDirectory, includeDebug: false));
}
// Opening the file exclusively fails while another handle to it is still open
using var exclusive = File.Open(path, FileMode.Open, FileAccess.Read, FileShare.None);
Assert.True(exclusive.CanRead);
}
finally
{
File.Delete(path);
}
}
}
}

View File

@@ -1,10 +1,11 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFrameworks>net8.0;net9.0;net10.0</TargetFrameworks>
<TargetFrameworks>net8.0;net9.0;net10.0;net11.0</TargetFrameworks>
<IsPackable>false</IsPackable>
<LangVersion>latest</LangVersion>
<Nullable>enable</Nullable>
<SuppressTfmSupportBuildWarnings>true</SuppressTfmSupportBuildWarnings>
</PropertyGroup>
<ItemGroup>
@@ -18,11 +19,21 @@
</ItemGroup>
<ItemGroup>
<PackageReference Include="coverlet.collector" Version="10.0.1" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.9.0" />
<PackageReference Include="coverlet.collector" Version="10.1.0">
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
<PrivateAssets>all</PrivateAssets>
</PackageReference>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.11.1" />
<PackageReference Include="Newtonsoft.Json" Version="13.0.4" />
<PackageReference Include="xunit" Version="2.9.3" />
<PackageReference Include="xunit.runner.visualstudio" Version="4.0.0" />
<PackageReference Include="xunit.runner.visualstudio" Version="4.0.1">
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
<PrivateAssets>all</PrivateAssets>
</PackageReference>
</ItemGroup>
<ItemGroup>
<Compile Include="..\SabreTools.Serialization.Readers.Test\UDFImageBuilder.cs" Link="UDFImageBuilder.cs" />
</ItemGroup>
<ItemGroup>

Some files were not shown because too many files have changed in this diff Show More