Cracks, emulators, and Lords of Conquest

In post #1, Odingalt wrote:

"My question is, are all of my floppy drives (I have three different drives that are calibrated and seem to work with all my other games) suffering from weak write heads, or is  it possible that these cracked versions missed something in the copy protection?" 

Uncle Bernie answers:

Floppy disk drives in a poor state (sloppy drive belts, dirty heads, bad electrolytic capacitors) can cause problems with floppy disk reads. These issues can be fixed, though. Plenty of advice found here on Applefritter and elsewhere.

What can't be fixed is sloppy "cracks". Back in the day (1980s) these "crackers" typically were teens and modified the code of the games to dodge the copy protection. There was a sort of arms race between the copiers, the crackers, and the software publishers. Some games have subtle coding which makes them unplayable in later stages of the game if the copy protection code has been tampered with.

The most famous and well documented example for this method comes from the Atari 8-bit game "Alterate Reality", which never has been successfully cracked (IIRC the author has published the unprotected code a few years ago, so the game is preserved for posterity, which is good). If the copy protection is not there, the avatar in the game just gets weaker (Scurvy) and sometimes, FBI agents appear out of the nowhere (in the game) and then the game is over. Very cunning !

In "Prince of Persia" for the Apple II, if the copy protection has been tampered with, the game also can't be solved. IIRC, it involves a magic potion which is required to complete the game but with no original copy protection it turns into poison and kills the avatar. But I'm not sure, this is from memory.

So it seems we have made a case against using "cracked" games. They were a nuisance back in the day and they are like the "gift that keeps giving" STD (Herpes virus). Avoid them. Originals for many great Apple games can be bought cheap off Ebay. Some, however, are heinously expensive. Others are not really Originals but were made with a Catweasel or Kryoflux or similar specialized hardware. These also are suspect because nobody can guarantee that the copy protection was replicated correctly. Although the software preservation foundations do the best they can.

The whole topic is really nasty. They never should have used floppy disk copy protections, but other schemes, which also work well to thwart software pirates. Some Chess software i.e. occasionally asks for a certain move in a certain opening which is documented in the booklet that came with the original.

Thanks to the copyprotection madness of the 1980s we now have great difficulty to preserve old software titles (the floppy disks deteriorate) and the ugly consequences ("cracked" games) also pervert the fun with these vintage computers.

As far as I am concerned, the very existence of Apple floppy disk copy protections have delayed the release of my Apple-1 floppy disk controller for more than a year now. I just lack enough original Apple II games to test it on the Apple II to prove it's functionally exactly the same thing as the Disk II system. I started to buy original games off Ebay but this only gets me so far.

- Uncle Bernie

UncleBernie wrote:

Thanks to the copyprotection madness of the 1980s we now have great difficulty to preserve old software titles (the floppy disks deteriorate) and the ugly consequences ("cracked" games) also pervert the fun with these vintage computers.

Back in the day, lots of us "cracked" software just for curiousity.  Stuff like,  "What's it doing?  Can I build my own levels in Miner 2049er?  Can I fix the keyboard bug in Prince of Persia?  Can I extract and print a map of Ultima's Underworld?  Etc, etc."

Uncle Bernie, you specifically reawakened my old hacker-vibes when you wrote (here) that the Disk ][ controller isn't connected to the R/W signal on the bus...which the processor can't "write" data to it.  I didn't know that because I had learned to program the disk drive using a Micro-Sci A2 controller, which is a lot easier to program.  I've been experimenting with the Disk ][ ever since, and I'm really hoping to attain something special.

So, you should ask, what's my point?

I want to devise ways of reproducing un-hacked/un-cracked original disks.  I have no idea how successful that will be in the long run, but I've successfully made un-cracked replica disks of Prince of Persia and Bank Street Music Writer where the original copy-protection code is intact, but the replica impersonates the original signature well enough to satisfy the original scheme.

I haven't learned to replicate the hard-to-copy signature (yet) but I have successfully replaced the error sector with an error-free sector whose data patterns happily satisfy the original copy protection as illustrated in the diagram below.  See Don Worth's book here for more details of this particular scheme, but essentially the copy-protection scheme would watch for the start pattern of E7 E7 E7, then it would briefly switch-on Q6 to load the write-protect status into the data register, then quickly switch-off Q6 again.  That tricks the Disk ][ to start reading in the middle of the next 'nibble' which enables it to read bits that would ordinarily be hidden between 'nibbles' on disk.

So I created this "impersonation sector" that contains valid data, but whose recording patterns on disk can fool/satisfy the copy-protection scheme as described in this illustration below (start at #1 the red bubble)

TripleE7 impersonation.png

More thoughts later...

UncleBernie wrote:

In "Prince of Persia" for the Apple II, if the copy protection has been tampered with, the game also can't be solved. IIRC, it involves a magic potion which is required to complete the game but with no original copy protection it turns into poison and kills the avatar. But I'm not sure, this is from memory.

The source code has been published for  Prince of Persia.  Here are some things I noticed while using the source code to help me replicate the disk:

• The boot track has a extra address mark for Volume=$08 Track=$1C Sector=$59 encoded with nonstandard preamble D6 AA 96.  AFAIK the program doesn't test for this signature, but it resembles another 1980's copy-protection scheme so it might be a ruse to distract hackers.
• Triple-E7 signature at boot, as described in the prevoius post.  This source code is in Purple.S
• Sectors on track 1 were "encrypted" using a scramble routine whose state variables were later used to initialize game variables.  Any modifications that affect those variables will lead to 
• Isolated variables like REDFLAG79 were hidden by surreptitiously switching to Auxilliary Memory.
• A final copy-protection check is inserted at line 402 of the main game loop in TopCtrl.S between two critical instructions: the previous instruction is necessary for game play to work at all, and the following instruction is necessary to advance to the next level.

Yellowcheck.png

In post #3, S.Elliott wrote:

Uncle Bernie, you specifically reawakened my old hacker-vibes when you wrote (here) that the Disk ][ controller isn't connected to the R/W signal on the bus...which the processor can't "write" data to it. 

Uncle Bernie comments:

Back in the day (just after the Disk II system came out) we sat in the computer club in a restaurant backroom and pondered over its schematics which can be found in the Apple DOS Handbook. We agreed that it can't work due to lack of the R/!W signal, so we suspected an error in the schematic, omitting the signal my mistake. The only (lucky) member with an Apple II would not give us his Disk II card to "investigate". At least he gave us photocopies of the schematics from his  brand new DOS Handbook to allow discussion by the "experts" on how this "miracle" worked. But without poking around in the real hardware we could not fathom it. What a pity ! I had just started to design my own version of the Apple II with less ICs but the project stalled when my home made "keyboard" using contact springs made out of beer can sheet metal strips did almost work, but not all the time . . . lesson learned, you need a good reliable keyboard, or the computer will drive you mad. IIRC this was Winter 1978/79 and stand alone computer keyboards could not be bought . . .  and I was a poor student with almost no money. The pain with the elusive keyboards at that time was so bad that some members designed capacitive keyboards based on PCBs, with the "keys" and their legends etched into the copper. These contraptions had no moving parts and also almost worked.

Many of the computer club members were electronics or IT professionals . . . and even they could not fathom the trick Woz had used. For those who are curious, it's described  in my post the "here" link in the post #3 links to.

I think the Disk II system is Woz' finest achievement and I stand in awe how he could have figured it out beginning with a blank sheet and make it work. My reverse engineering of it over the decades revealed layers upon layers of mind boggling subtle tricks and features in it, and was not easy. The interaction of the hardware and the software exploiting quirks of the 6502 instruction cycle timing is amazing ! 

Just a few years ago I succeeded in implementing a Woz Machine using a GAL16V8 or PAL and a 74LS299, and this lead to my floppy disk controller for the Apple-1 (a thread is here on Applefritter). Due to the 74LS299 having asynchronous clear rather than the synchronous clear of the 74LS323, and due to the product term limits of the PAL, I had to change a lot of things to make it fit. So far I am not absolutely sure that my version of the "Woz Machine" could digest ALL known Apple II disk copy protections. I have some theoretical lines of thoughts that it could, but the proof is yet out. More work to do ! I need to integrate it into a real Apple II and try a lot of real copy protected Apple II games. I have started to by a few off Ebay, but so far could not score "Prince of Persia" and "Drol". The asking price for original disks of the latter were just too high. I am not a collector of software, and so I don't want to waste a lot of money on buying mere test specimen for my ongoing testing and development work. I wonder how the "Software Preservation Society" gets their original disks. Donations maybe. I don't know. Before I have definite proof that my implementation of the Woz Machine is perfect, I can't release its design to the public. This work is too precious for me to risk tainting it with issues that still could be lurking in it.

The designers at Apple (the corporation) of the "Integrated Woz Machine" must have faced much the same risks. So far I have found no really good "datasheet" for the IWM on the web. I do have some. One of them tells me about a higher sample rate on RD DATA . . . hmmm. Sounds risky as it certainly changes the behaviour of the IWM vs. the Disk II if "fuzzy" sectors with intentional phase jitter are used in a copy protection. As the designers of such protections need to introduce some safety margins for their protection check (due to phase jitter and speed variations of the magnetic media itself) these subtle differences may or may not have an effect on the copy protection check outcome.

I know that the "Software Preservation Society" does competent work with their Kryofluxed images, in other words, they know what they are doing, but I decided to develop my own hardware and software to make flux change timing based floppy disk images and to interpret / restore the results. This is an ongoing project which I started and stopped several times over the past 30 years. Oh, the hardware is very simple, it's just 3 TTLs and one GAL16V8 and still has a flux resolution of 50ns (can't push the LSTTLs any faster, and I think it's not needed to get faster anyways). This is the read channel which goes into a DOS notebook printer port. Adding a write channel would be one GAL 16V8 and one TTL more. (You can see I have plenty of long obsolete ICs and long obsolete PC-ish DOS based computers). For me it's a fun project to fill my days and exercise my brain. Otherwise I'd buy a "Greaseweasel" which harnesses the powers of a modern microcontroller to do essentially the same thing with only one IC.  But I wanted to do it with 1980s technology and still be cheap and only have a few 1980s era ICs on the card. The real "magick" happens in the notebook computer, though.

So while the floppy disk copy protections of the 1970s and 1980s caused us a lot of hassles back in the day, being retired, we now have a topic we can investigate as a nice pastime which exercises our aged brains.

Is it duplicate work and effort ? Certainly, it is.   The "Software Preservation Society" did all the work already. But frankly, I do not even take a look on their ".woz" file format before my own work, based on my own ideas, is done.  This is a self-imposed "clean room" technique the industry uses for development of compatible products without  infringing on third party trade secrets. I know that this legal topic is totally irrelevant for hobbyists like us, but it has another feature: if you don't try to understand how the "competitor" does it, then you have to invent your own ways. And you might dodge some of their mistakes (the downside, of course, is to add your own mistakes). But the real interesting consequence of this is that once you compare the emulations of the two different systems in vivo and find them to behave the same, then you have a very good basis to claim that both systems got it right. If, however, differences would show up, in this case, different behaviour of emulated copy protections, then this is a clue that there still is some imperfection lurking somewhere in the system.

IMHO, doing this work on "cracked" disks is a complete and meaningless waste of time. It does not help to preserve the original bit patterns of the past for posterity. And this should be the mission. The more independent work is done in this field, the higher is the chance that the preserved bit patterns are uncompromized by possible imperfections in the hardware and software system used in the preservation effort.

In other words, we need more than one "Software Preservation Society" and more than one hardware and software system for the job. We need lots of individuals doing that work with a diverse set of hardware and software. There is not much time left, for many floppy disks it's already too late, the magnetic coating is falling off, the hub hole reinforcing ring is falling off, the friction has increased due to the lube or whatever material they used to that effect has deteriorated, and fungus / mold has eaten away parts of the disk surface.  And it does not get better over the decades !

Comments invited !

- Uncle Bernie

UncleBernie wrote:

Is it duplicate work and effort ? Certainly, it is.   The "Software Preservation Society" did all the work already. But frankly, I do not even take a look on their ".woz" file format before my own work, based on my own ideas, is done.  This is a self-imposed "clean room" technique the industry uses for development of compatible products without  infringing on third party trade secrets. I know that this legal topic is totally irrelevant for hobbyists like us, but it has another feature: if you don't try to understand how the "competitor" does it, then you have to invent your own ways. And you might dodge some of their mistakes (the downside, of course, is to add your own mistakes). But the real interesting consequence of this is that once you compare the emulations of the two different systems in vivo and find them to behave the same, then you have a very good basis to claim that both systems got it right. If, however, differences would show up, in this case, different behaviour of emulated copy protections, then this is a clue that there still is some imperfection lurking somewhere in the system.

 

You raised lots of relevant (and provocative) ideas that I want to keep pondering, but this paragraph is especially helpful encouragement for us to scrutinize the Disk II and try not to be tainted by common wisdom and  existing myths.  We don't need to keep ourselves "in the dark" entirely, but we do need to exercise a healthy skepticism.

Consider the mythical belief that "the Disk II can't read more than three 0-bits in a row."  Anyone who has directly experimented with recording directly to track $00 has undoubtedly observed evidence that suggests-but-does-not-prove that the Disk II can't read more than three 0-bits in a row.  Many programmers undoubtedly accepted that conclusion on the face of it because it fit the evidence they had observed, and didn't test further nor obtain more evidence to prove (or disprove) the limit.  Consequently, non-programmers generally accept it because the expert conclusion is based on evidence...though not conclusive evidence.

To test that mythical belief, what would occur if I use a nibble editor to write the patterns  C0 FF EE 96  and  C0 C0 AA 96  onto track $22 of a floppy disk?  The C0 nibble contains six 0-bits in a row, twice as many as the mythical maximum, so we would not expect these patterns to be reproduced accurately.  Indeed, some variants of the myth suggest that exceeding three 0-bits will cause the Disk II to produce wildly unreliable patterns until it recovers.

So, here's the log of an experiement to test it.

Step 1: Load a formatted track into Locksmith's nibble editor, and locate the data area of sector 0.  Press C to enter "change mode" and enter eight copies of the nibble pattern  C0 FF EE 96, followed by eight copies of the nibble pattern C0 C0 AA 96.   Here's what it looks like before writing:

C0FFEE C0C0AA before writing.png

Step 2: Press CTRL+W to write the pattern onto track $22 of a disk.  For good measure, presss CTRL+W and write it again onto track $21 so we'll have two copies for more coverage.

Step 3: Press CTRL+R to read the contents of track $22 into the nibble editor.  Press CTRL+S to run Locksmith's default LPL routines to identify the start and end boundaries.  Then press D to list sector offsets, locate sector 0, and scroll it into view.  (Optional: if the columns aren't aligned the way you want, use CTRL+D to delete individual nibbles until the data is aligned the way you want it.  Or you could read the track again and hope it will be aligned the way you want, just by luck.)

Step 4: Count how many instances of C0 FF EE were reproduced on track $22.

C0FFEE C0C0AA read from track 22.png

Tallies for track $22:

• C0 FF EE pattern has six 0-bits in a row, which the Disk II reproduced with 75% accuracy.
• C0 C0 AA pattern has two groups of six 0-bits in a row, which the Disk II reproduced with 63% accuracy.
• Sync versions of C0 FF EE and C0 C0 AA have eight 0-bits in a row, which the Disk II reproduced with 0% accuracy.

Track $21 yielded similar results, just swapping the numbers between the first two categories.

Surprised?  These results sure don't align with the premise that the Disk II can't read more than three 0-bits in a row.  But it does support your premise that it's worthwhile to independently research these things and compare our results.  I suggest you try this experiment yourself, with varying media quality.(PS: The disk in these pictures is a Wabash Pinnacle Series.  Better quality disks perform measurably better.  Try some, please!)