docs/Commodore/SPYNE.TXT

*** SPY (SPYne containers)
*** Document revision: 1.3
*** Last updated: March 11, 2004
*** Compiler/Editor: Peter Schepers
*** Contributors/sources: John Iannetta

  Created  by  John  Iannetta  around  1995,  these  are  mostly  found  in
Compuserve's CBM applications forum, but do appear in the COMP.BINARIES.CBM
newsgroup as well. It is a self-extracting file, and does  not  employ  any
compression. The word "SPYNE" does not represent any acronym, but is  meant
to represent a "spine", with files linked together like a human spine.

  This format has some similarities to LNX containers in that all the files
are simply stored (with no compression), one after the other, and are  byte
aligned to take up multiples of 254 bytes (256 on a real 1541).  This  does
result in a little dead space at the end of each file  stored,  but  its  a
small price to pay for how easy it is to construct and break up files on  a
real C64/1541. 

  SPYne containers do not  contain  a  BASIC  program  (like  LNX)  at  the
beginning telling you to "Use XXX to dissolve this file", since SPYnes  are
self-extracting, and are meant to be loaded ",8,1" on a real C64. There  is
no specific signature to determine if the file is actually a  SPYne.  SPYne
containers can store up to 144 files.

  Each file inside the container can have up to 65535 blocks. However,  the
container itself is limited to (best case) 65519 blocks total (65535  minus
15 blocks for extraction code, and a  minimum  of  1  block  for  directory
entries, 1-8 entries). Worst case for the container size  is  65502  blocks
(15 blocks for extraction and 18 blocks for a directory of 137-144 files).

  This many not make sense at first, but how can  you  load  a  SPYne  file
which exceeds 258 blocks (the maximum memory of the C64)? SPYne  files  can
be as large as 65535 blocks, and it would  seem  impossible  to  work  with
files this large. The answer  is  that  the  SPYne  file  loads  at  $02A7,
trapping the necessary vectors to prevent it from loading the entire  file,
but just the extraction code. Neat trick!

  The first 15 blocks (up to offset 3809 or $0EE1) of the file contain  the
self-extracting code. It  is  an  auto-running  application,  with  a  load
address of $02A7.

      00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F        ASCII
      -----------------------------------------------   ----------------
0000: A7 02 20 A0 E5 A9 0A 8D 1E D0 A9 16 8D 18 D0 A9   <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>Щ<EFBFBD><D0A9><EFBFBD>Щ
0010: 04 8D 1F D0 20 44 E5 A9 D8 85 FC A0 00 84 FB A2   <20><><EFBFBD><EFBFBD><EFBFBD>D<EFBFBD><44>؅<EFBFBD><D885><EFBFBD><EFBFBD><EFBFBD><EFBFBD>
0020: 04 A9 0D 91 FB C8 D0 FB E6 FC CA D0 F6 20 A5 FF   <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>

  Starting at offset $0EE2 (3810) is the central directory. This address is
exactly 15 blocks into the file (15*254=3810). Each directory entry  is  32
bytes long, with the last two of these being "filler", and unused.

      00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F        ASCII
      -----------------------------------------------   ----------------
0EE2: 82 00 00 30 32 2E 44 49 47 49 54 41 4C 20 4D 41   <20><><EFBFBD>02.DIGITAL<41>MA
0EF2: 47 49 43 00 00 00 00 88 18 B5 FF 00 2B 00 00 00   GIC<49><43><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>+<2B><><EFBFBD>
0F02: 82 00 00 30 39 2E 2D 2D 2D 3E 20 42 59 20 3C 2D   <20><><EFBFBD>09.---><3E>BY<42><-
0F12: 2D 2D 2D 00 00 00 00 00 7A 44 FF 00 2D 00 00 00   ---<2D><><EFBFBD><EFBFBD><EFBFBD>zD<7A><44>-<2D><><EFBFBD>
0F22: 82 00 00 30 33 2E 2D 3E 20 46 4F 52 43 45 53 20   <20><><EFBFBD>03.-><3E>FORCES<45>
0F32: 3C 2D 2D 00 00 00 00 11 BE 89 FF 00 20 00 00 00   <--<2D><><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>
0F42: 82 00 00 30 37 2E 2D 3E 20 4F 46 20 45 56 49 4C   <20><><EFBFBD>07.-><3E>OF<4F>EVIL
0F52: 20 3C 2D 00 00 00 00 4F 78 11 FF 00 26 00 00 00   <20><-<2D><><EFBFBD><EFBFBD>Ox<4F><78><EFBFBD>&<26><><EFBFBD>
0F62: 82 00 00 30 34 2E 2D 2D 2D 2D 2D 2D 2D 2D 2D 2D   <20><><EFBFBD>04.----------
0F72: 2D 2D 2D 00 00 00 00 40 58 D2 FF 00 28 00 00 00   ---<2D><><EFBFBD><EFBFBD>@X<><58><EFBFBD>(<28><><EFBFBD>
0F82: 82 00 00 30 31 2E 20 52 45 4C 45 41 53 45 44 20   <20><><EFBFBD>01.<2E>RELEASED<45>
0F92: 4F 4E A0 00 00 00 00 05 D5 0F FF 00 29 00 00 00   ON<4F><4E><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>)<29><><EFBFBD>
0FA2: 82 00 00 30 35 2E 20 4A 41 4E 55 41 52 59 20 31   <20><><EFBFBD>05.<2E>JANUARY<52>1
0FB2: 53 54 20 00 00 00 00 F7 FA EC FF 00 24 00 00 00   ST<53><54><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>$<24><><EFBFBD>
0FC2: 82 00 00 30 38 2E 20 20 20 31 39 39 36 A0 A0 A0   <20><><EFBFBD>08.<2E><><EFBFBD>1996<39><36><EFBFBD>
0FD2: A0 A0 A0 00 00 00 00 39 71 C8 FF 00 36 00 82 00   <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>9q<39><71><EFBFBD>6<EFBFBD><36><EFBFBD>
0FE2: 00 30 36 2E 20 28 4D 4F 52 45 20 4C 49 4B 45 29   <20>06.<2E>(MORE<52>LIKE)
0FF2: A0 00 00 00 00 40 33 CF FF 00 32 00 00 00 82 00   <20><><EFBFBD><EFBFBD><EFBFBD>@3<><33><EFBFBD>2<EFBFBD><32><EFBFBD><EFBFBD><EFBFBD>
1002: 00 31 30 2E 20 28 31 32 2F 32 38 2F 39 35 21 29   <20>10.<2E>(12/28/95!)
1012: A0 00 00 00 00 AD 1C 0F FF 00 2B 00 00 00 82 00   <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>+<2B><><EFBFBD><EFBFBD><EFBFBD>
1022: 00 44 49 47 49 54 41 4C 20 4E 4F 54 45 A0 A0 A0   <20>DIGITAL<41>NOTE<54><45><EFBFBD>
1032: A0 00 00 00 00 49 B0 CB 00 00 14 00 00 00 00 00   <20><><EFBFBD><EFBFBD><EFBFBD>I<EFBFBD><49><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>

     Byte:   $00: File type
                   supported values for this location are:
                    $81 - SEQ
                     82 - PRG
                     83 - USR
           01-02: Undefined (00's for now)
           03-12: Filename (padded with $A0's)
           13-16: Undefined (00's for now)
           17-18: File checksum (in low/high byte format)
              19: LSU byte (see "INTRO.TXT"  document  for  description  of
                  "LSU")
              1A: Last file marker
                    FF - more files in container
                    00 - last file in container
              1B: Set to $00
           1C-1D: File sector count (in low/high byte format)
           1E-1F: Not used (and not present on a 254-byte boundary)

  File type support is limited to valid files only. SPYnes  cannot  contain
write-protected, "splat" or DEL files. Without DEL type support, this means
that many "separator" files in the directory cannot be included. REL  files
are partially supported, as only  the  data  portion  is  copied,  and  its
filetype is converted to USR format.

  The checksum at byte offset 17-18 is a simple  16-bit  addition  (without
carry) of the entire  file,  but  does  not  include  the  directory  entry
information.

  Looking at the above HEX dump, at address $0FE0 a  new  filename  starts.
This is two bytes earlier than expected. This occurs  because  the  address
$0FE0 is at a "254-byte boundary address", meaning it is divisible by  254,
with no remainder. Only in the directory area is this  boundary  important,
and when it happens, the last two "filler" bytes  in  the  directory  entry
don't exist.

  SPYne supports two extraction methods, fast and slow. The fast method  is
destructive in that is just breaks up the file into sections, only works on
a 1541/1571 drive, and has no checksum verification. The slow method should
work on any device, is not destructive to the original file, and  uses  the
checksum in the directory entry to verify the data integrity.

  File data starts in the next block following the end  of  the  directory.
You only know where the first file's data starts  once  the  directory  has
been completely read. Since the extraction code takes  up  15  blocks,  and
each 8 files take up another block, we can calculate how many  blocks  into
the container the actual file data starts.

  If we have 11 files, we have  two  directory  blocks  (one  full  with  8
entries, one partial with 3  entries).  Therefore  the  first  file's  data
starts at block 17 (17*254 = 4318, or $10DE). The  second  file's  starting
position can be calculated by adding the block  count  of  the  first  file
(multiplied by 254), and adding this to the first files starting position.