Apple II DISK Emulator using STM32

Yes, the speed looks perfect. Does your STM work with Locksmith's Fast Backup (NOT the ordinary slow Backup) especially when acting as a writing drive? Along with V option for writing with verification? 

No, not Certify. Fast Backup is the fastest diskette copy. Once loaded enter [V] RETURN to switch on verification after writing process. "12" means copying is to be done from drive one to drive 2. "21" means the source is drive 2 and destination is drive 1. "12V" means that along with copying a verification of written tracks is also done on the fly.

Excellent for Fast backup. I can assure you Apple2's accept much lower drive rpm than the defined 2%. My advice is to assume the largest buffer you've seen in any woz and play it at the convenient rate about 4uS per bit, you can try that for every disk image, it must work. If the buffer is less than your defined size, just fill it with noise/gap, and don't bother with buffer resizing. One comment on the WOZ files, they are usually "artificially" reviewed/modified by humans or software and unlike A2R do not necessarily represent the physical diskette recording. The WOZ purpose is to use the software with (drive) emulators.

Is everything else apart from weak bits working? Does your STM now work with 99.9% of the disk images available on the net?

I think for the weak/random bits you have to give up the DMA and start bit banging bits. 

In post #159, 'retro_devices' wrote:

" I think for the weak/random bits you have to give up the DMA and start bit banging bits. "

Uncle Bernie comments:

I think it's too soon to address the 'weak bit' problem right now. I've recommended in a post above to first try and catalog ALL recommended WOZ file images if they load and run fine with this emulator. I think 'weak bits' are less important until everything else works.

"bit banging" to the port is not necessary for emulation of 'weak bits'. Knowing where they are, an interrupt can be timed to fire sufficiently long before the DMA reads them, and then the interrupt routine would put certain plausible 'weak bit' patterns onto the track buffer at the known place. These must change once every revolution. Note that the extreme case are unformatted tracks where everything being read is a total random mess. All these thing are not difficult to program, but time consuming, to implement and test all cases, and I would not do that before everything else related to reading tracks is stable and works. The rationale behind this is that if the track buffer machinery changes, for any reason, most likely you can throw all your 'weak bit' code into the trash, and all the work on it is wasted.

- Uncle Bernie

After copying a diskette with Locksmith is the copy working?

In post #166, 'VIBR' wrote:

'" This is my poor score card  "

Uncle Bernie comments:

Don't give up, the fact that you can parse WOZ files and boot DOS 3.3. means that great work already has been done !

Ignore DOS 3.2. because this came before the upgrade of the DISK II card with different PROMs. DOS 3.3 came with a new set of PROMs which changed the state machine on the DISK II card and the GCR encoding.  This increased the number of sectors per track from 13 to 16. A conversion program was provided to read the old floppy disks and turn them into the new format. I don't know if DOS 3.2.1. itself would ever boot on the modified DISK II card with the new PROMs, maybe somebody in the know can chime in here.

This is what I would do:

Look for WOZ files of older titles with less sophisticated copy protecttion schemes. Use the 'Tome" book as a guidance. Find some titles which work (this will boost your confidence) and research which copy protections they use. Make a spreadsheet, line items are software titles, colums are copy protection methods. Put in a 'F' for 'fail' or a 'P' for 'pass' in the intersections. A pattern will emerge which will guide you through the debugging process. Note that a lot of the later titles are protected by one (or more) of a group of commercially sold protection methods, and maybe two of them dominated the marketplace for Apple II copy protections. So if you solve this riddle for one of the titles using a given protection, a LOT of other titles you struggled with will change from 'F' to 'P'.  Look into the '4AM' cracks ... they come with a disk image file and most have an additional bonus file which contains a discussion which protection was used and how it was cracked. These '4AM' files are a treasure trove for researchers of Apple II copy protections, despite the crack of course removed it. But you can check the '4AM' repository for the same titles your floppy emulator has trouble with the WOZ files, and in most cases, '4AM' will tell you which copy protection was used. This will get your work up to speed !

As a general advice, try to find a simple protection which relies on 'synchronized tracks' first, but no 'spiral tracks' yet. Develop your firmware until this works, which essentially means you got the loading of the track buffers and the timing of the stepping action between the tracks right. This is very, very important. Once tracks are switched, you cannot allow your emulator to lose the perfect synchronization with the (virtual) rotating 'floppy disk'. If you slip the timing by more than  a few bytes (6400 * 0.01 = 64 bytes ??? ... 1% RPM error) then all bets are off (it won't work, and your platform is wonky ... don't build any other copy protection method emulation on this sand ;-)

Once this works, proceed to 'fat tracks' and once that works, to 'spiral tracks'. Always do regression tests.

Once your emulator can do that, you may proceed with the 'weak bits'. I think you NEVER would need to indulge into the details of ANY of the 'modified GCR encoding' or 'weird format' copy protection techniques because the bit cell stream technique should provide all of that automatically, without any extra coding effort.

Still, a lot of work to be done. All I can do is to give you guidance in which direction to proceed, based on my experience with my own floppy emulator development for the Atari 8-bit (which was never commercialized, I did that because I saw my floppy disks deteriorating). On the Atari, exact timing of everything is essential. I used a 8-bit timer running forever which emulated the rotation of the floppy disk with 0.5% accuracy. This turned out to be good enough even for the most aggressive 'synchronized track' protections. If your floppy emu has deficiencies there, fix these first.

- Uncle Bernie

In post #179, 'VIBR' wrote:

" I have done some investigation and it appears that when the emulator is switching the data track to fast it does not work but when I add between 5 to 8 ms delay (like the normal coil switching) it start working again... "

" Is this normal ? "

Uncle Bernie comments:

Well, it is not "normal" for a floppy disk image to behave like that UNLESS it has a copy protection which looks for the timing between tracks to be right. The so-called 'synchronized track' protections take this to the max, as ALL tracks are synchronized and are being checked for proper radial alignment, but actually, TWO synchronized tracks are enough. The chance to accidentially "hit" the right alignment by a copy program NOT instructed to do the alignment is very low. If I'd design a 'synchronized track' copy protection it would only use two tracks, spread apart, and all the other tracks would not look aligned. This would slow down software pirates by a few hours, or maybe days ;-) . . . if you wanted to really sabotage their efforts then relax the check such that they have a maybe 30% chance to get a functional pirated copy despite using no alignment copy mode. So they would think the copy is OK and send it to their peers. Who would try to make a 2nd generation copy for their peers and fail. And then complain. Which costs time, effort, and postage. A similar rationale applies to other copy protections: use a weak one and a strong one. The weak one, if not copied properly, would crash the program upon booting. But if the weak one was copied correctly (i.e. by LOCKSMITH), and the strong one would not be copied perfectly,  the game would work, but have a dead counter which self-destructs the floppy disk contents after X loads. As this would happen downstream along the software pirate's distribution chain, it will cause a lot of complaints and a lot of effort to remedy. I'm mentioning this not as a recipe to build floppy disk copy protections (as they are a thing of the past, and floppy disks went the way of the dinosaur), but there are lessons to be learned here on how to test floppy disk emulators dealing with images of copy protected floppy disks. Add a indicator flag which tells you if the game tries to write to the floppy disk. This may be harmless (such as keeping a scoreboard or allowing to save the game state) but it also could mean the game tries to self-destruct.

I already did explain here on an earlier post how to treat the change of tracks properly. Your ST32 can do interrupt-on-change for the PH0 ...  PH3 stepper motor phases to detect when the software initiated a track change, but there will be some mechanical inertia before the head starts moving and even more delay until the RDDATA pattern gets disturbed (as the head moved away from that track). Until that point in time after a stepper motor phase change you MUST provide the correct RDDATA pattern of the old track. You can use the time to load the next track into a new track buffer, though. But you can't overwrite the old track buffer as the RDDATA stream must continue to flow for a small amount of time. The next thing is that you need to start the DMA out of your new track buffer at the right place which is the rotational angle you can calculate from the  'forever running' timer that rolls over all 200ms, or once per revolution.

If you did not implement this track change mechanism in the right way, it's no wonder that many copy protected floppy disk images won't load.

- Uncle Bernie

In post #174, 'VIBR' wrote:

" It seems that the appleSauce version of Commando is not the same as the one from 4am ..."

Uncle Bernie comments:

Of course ! It has to be like that because:

- AppleSauce makes WOZ files which contain the original copyprotection as faithfully as possible. The rules are:

Do do not make any changes to the contents of the floppy disk image.

No code patches, no manipulations.

- 4AM is a very sophisticated "cracker" who makes "cracks" and produces disk images which do not contain the original copy protection anymore. These disk images are supposed to be fit to be copied with some very primitive utility which otherwise is unable to make a functional copy of the original disk. Making the "cracked" version involves code changes to the contents of the floppy disk, so these "4AM cracks" are NOT a faithful reproduction of the original floppy disk. But "4AM" tries to keep everything in the program intact, including the title screens, and does NOT put his "tag" in like many teenage crackers back in the 1980s did (and ruining the experience. You do not want a mutilated title screen spelling out "Cracked by the Green Mental Midget. For more cracks, call the safehouse, phone number ...", wherein the phone number may have been a BBS, no way to find out almost half a century later.)

Now here is a dirty little secret about "cracking": in some cases the copy protection was so tough to crack that the "cracker" could not remove it completely by manipulating the code alone. It may have been necessary to put some alternate "weird format" on the floppy disk which is easier to copy but fools the copyprotection checks into thinking it's an original. So standard Apple DOS3.3 copy utilities could not copy these "cracked" disks and some "defeated" version of LOCKSMITH - or the like - was needed. With "defeated" I mean that that version could not copy the original floppy disk with the original, tough copy protection anymore. But you can see the manipulations done by the "cracker" in this case as a "downgrade" of the original copy protection to make it copyable by the "defeated" tools.

Whether this conundrum applies to 4AM cracks, I can't say, as I did not look into them too much. I was more interested in the "extras" where 4AM describes the copy protection and how he cracked it, and not in the cracked version of the program itself. This of course was driven by trying to study which Apple II copy protections where there, back in the 1980s and early 1990s. Then I found the "Tome" book and my thirst for information was satisfied. At least for the moment.

You should not confuse or compare WOZ images coming from AppleSauce and 4AM images. I recommended the latter for you such that you can look at the "extras" explaining how the copy protection on the WOZ file works. This was thought to help you with debugging your floppy emu firmware. If your floppy emu can't support loading 4AM crack disk images, and the crack won't run, something more serious is wrong in your code. But don't give up ... all the "secrets" about Apple II copy protections from half a Century ago ain't no secrets anymore, they are public knowledge that can be found on the internet. So you have all the know-how right there. The trick is to a) find these docs (I gave you the hint with 4AM) and b) apply the knowledge gained.

Good luck !

- Uncle Bernie

This is all above my pay grade. But if you are reading up on 4am's write up on the E7 protection, make sure you read the revised one (which maybe you are, again, above my pay grade). At least I'm assuming he revised it.

In a nutshell, about 3 years ago it was discovered that his E7 cracking method wasn't 100% right. As I understand it, his original thoughts on how it worked were wrong, but by a fluke worked only on standard disk II controllers, and would fail on IWMs. It took a while before it was caught that it was failing on IWM machines and caused him to go back and reinvestigate the whole thing. 

In post #177, 'nick3092' wrote:

" ... but by a fluke worked only on standard disk II controllers, and would fail on IWMs. It took a while before it was caught that it was failing on IWM machines and caused him to go back and reinvestigate the whole thing."

Uncle Bernie comments:

Since a few weeks I have a 100% "in lockstep" RTL for the IWM Rev B., and I can confirm that the IWM is a quite different animal than the original DISK II controller, even in the 'legacy mode' (my name for it) where it was supposed to be DISK II controller compatible. This is very bad news.

The 'read state diagram' of the IWM is a closed loop over 14 or 16 states and it never looks at the state of the shift register MSB, and never has a "wait for RDDATA pulse" wait state (a state looping onto itself). This is very bad news for copyprotection compatibility.

The message here for 'VIBR' is: do not work with a IWM based Apple. Use an original DISK II card only.

I intend to modify my IWM such that it is 100% compatible with the DISK II und all conditions (but only as long as it is in 'legacy mode'). This extra logic does not fit in the current PLD I'm working with as an intermediate step (Altera EP900). So my whole IWM substitution project gets more and more time consuming ... this (and a few other problems which need immediate attention / time to remedy) is the primary reason why I can't pay too much attention this thread at the moment. I'm just overloaded.

The worst crisis at the moment is with my Italian Parsley which gets more and more little white spots. At least I have now identified the culprits, two types of very tiny insects. They seem to suck the "blood" of the plant. The trick is to find some chemical compound which will kill them but is harmless for me when I then later eat the Parsley. Caffeine comes to mind - the shrubs who have it in their leaves are absolutely deady for many sucking insects. But how do I get Caffeine into the Parsley "bloodstream" (well, the plant equivalent of blood). These science projects can quickly get out of hand and suck you in !

- Uncle Bernie

In post #180, 'VIBR' wrote:

" Looking at the Phase timings, if I wait all phase to be low with some delay, I might have some issue with Spiral track...

the other way would be to, wait to have only 1 phase ON  and have a delay of 12ms, which is the max timing of phase without overlaps (the last phase has a duration of 36ms). doing so I have 24  (36 -12) ms to load the track which is by far enough. 

But I am sure Uncle Bernie will have an opinion :)"

Uncle Bernie's opinion:

I posted some comments already how to tackle the head movement problem. The point is: you need to model the inertia of the head properly. If a stepper motor phase is changed when the head is at standstill, the RDDATA stream of the current track will continue for a while, until the head finally starts moving and the signal coming from the old track faints and gets lost. Likewise, when arriving on the new track, it will take a short while before the RDDATA stream of that new track stabilizes. This is true for all "normal" floppy disks (no copy protection).

I'm not sure what really happens with 'spiral tracks' or 'fat tracks' because I never had the time to look at their RDDATA streams. Is there a gap in the RDDATA stream filled with trash / instable bits or not, if the head moves ?  We can't easily tell anything because we don't know what the software does in terms of reading the DISK II controller while the head is moving. Will it allow for some random trash and look for a SYNC sequence (or any sequence which does the same, synchronizing to 'disk nibble frames'. Or will it expect a continous stream of nicely stable RDDATA bits ?  I think the latter is possible if the head is only being moved by a very tiny amount. You can actually gently and carefully "wiggle" the head assembly a little bit with your fingers while reading, and reading is not disrupted (if you are ham fisted / fat fingered, don't try that, it might damage the head assembly). This experiment gives a clue that indeed, very small head movements may allow a continous, non-disturbed RDDATA stream.

This requires more investigation with a real floppy disk having such a copy protection. Just hook your logic analyzer up to RDDATA and the stepper motor phases and investigate what happens.

Also be aware that head movement algorithms (stepper motor phase sequences) vary.  You need to be able to detect all cases and have an algorithm which calculates on which quarter track the head actually is at any given point in time. The algorithms I've seen in some Apple II emulators dealing with head movement based on stepper motor phases are quite sophisticated. And they are best copied, not re-invented ;-)

The weird pdf from GFTO you have found and mentioned in the above post #173  has a lot of info about which titles use tricky stepping.

(Oh and I'm still curious about what that Gazette really is ... they use the English language but have ads from small German one-man businesses catering to hackers (i.e. selling annotated ROM listings) in the German language ! Who would pay for that ad ? Makes no sense to me. The whole weird pdf looks bogus for that reason. But the article on Apple II protections was a great read !

- Uncle Bernie

VIBR wrote:

When the head is reading 0x0D in the adress field it put 0x0D in $002E and the protection start checking for the 0x96 sequence after D5 and wait for E7E7E7. 

Then It desync the latch of the Controller card and It expect to read someting different from the upcoming E7 because of the timing bits.

Important but subtle detail: if you're using a Disk II controller (state machine) then make sure the emulated drive is signalling that the disk is WRITE PROTECTED for most reliable results

In order to "desync the latch" the E7 copy-proteciton routine puts the controller in WP-SENSE mode.  The original game disk was permanently write-protected (no notch), the WP-SENSE mode causes the data register to immediately fill with 1-bits (ie: "full").  If the drive does not return a WRITE PROTECTED signal, the data register will be filled with 0-bits.  (ie: "empty")  Apparently that's sufficient to upset the signature check.

Some years ago I adapted the copy-protcetion code from Prince of Persia into a stand-alone E7 signature-check program.  In my tests on an Apple //e with Disk II controller it was 100% reliable with WRITE PROTECTED disks, but only about 50% with WRITE ENABLED disks.

I have no idea whether write-protect affects IWM the same way, since it uses separate registers for status vs data.

In post #183, "S.Elliott" wrote:

" In order to "desync the latch" the E7 copy-proteciton routine puts the controller in WP-SENSE mode.  The original game disk was permanently write-protected (no notch), the WP-SENSE mode causes the data register to immediately fill with 1-bits (ie: "full").  If the drive does not return a WRITE PROTECTED signal, the data register will be filled with 0-bits.  (ie: "empty")  Apparently that's sufficient to upset the signature check."

Uncle Bernie comments:

True that. In general, it is VERY important that the floppy emu will produce the right WRITE PROTECT flag for any given software title. Original, professionally mastered floppy disks mostly were write protected (no notch on the edge of the floppy disk) and many copy protections check for that - pirates using LOCKSMITH and the like had to have a notch (no write protect) to make the copy, and if they did not put a write protect sticker back before test booting the copy, all sorts of more or less evil things could happen. The more stupid copy protection routines make the boot crash when the write protect was missing, which, alas, is obvious, but the more "evil" ones ("evil" from the standpoint of the pirate ;-) would  allow the game to run perfectly for that boot, after overwriting a sector with critical code, which would make the game unplayable at the second boot event. The bottom line here is: always emulate the write protection properly. This involves not allowing any writes to a write protected disk.

Just to add a little detail S.Elliott did not mention: the manipulation of the DISK II controller state machine during E7 checks does not only check the write protect status line but also brings the state machine (aka "Woz Machine") into a specific state that does not change. When the mode is switched back to "READ MODE" (L7 = L6 = 0) then there is NO reset or restart of the read state machine. It just continues with that state it was in the write protect sense mode. And continues from there. The actual trajectory through the read state diagram will depend on whether the write protect check did set or clear the MSB of the shift register. So reactivating the read mode at just the right point in time, combined with a RDDATA pulse or not, can lead to several different outcomes as seen in the shift register. 

The IWM does not work like that. As I mentioned in my post #181 above, its read state machine (meaning the bit cell centering part, which substitutes the WOZ machine seen in the DISK II controller) does not look at the MSB of the shift register at any time, and worse, it continues reading while the write protect status is checked. The IWM does not use the read data shift register running backwards (right shift) to capture the write protect sense status line. Instead, it just implements another I/O port where the write protect sense line can be read. In some Apple products the SENSE input of the IWM serves other duties than sensing the write protect. I did not look into that, but it may be used as a sort of handshake signal for some more sophisticated I/O protocol. Can't prove it but this may have been the motivation why the IWM designers did move the write protect sense away from the read data shift register, into another functional block.

Which of course is yet another difference to the original DISK II controller, and will inevitable make certain copy protections fail on the Apple IIc (which uses the IWM), despite of an original and intact disk that loads fine on an Apple II, II+ or IIe using the DISK II controller.

So we can state that the IWM is ***NOT*** 100% compatible with the DISK II controller. Their "IWM specifications" which we can find on the web nowadays say otherwise. They seem to believe the IWM is fully compatible with the DISK II controller. Which is wishful thinking.

- Uncle Bernie

UncleBernie wrote:

So we can state that the IWM is ***NOT*** 100% compatible with the DISK II controller. Their "IWM specifications" which we can find on the web nowadays say otherwise. They seem to believe the IWM is fully compatible with the DISK II controller. Which is wishful thinking.

So far it is not "we", it is mainly you who is stating that and as a human yourself you could be wrong too. Is there any practical proof to that? Which protections fail on //c and/or //GS? 

In post #186, 'retro-devices' wrote:

" So far it is not "we", it is mainly you who is stating that and as a human yourself you could be wrong too. Is there any practical proof to that? Which protections fail on //c and/or //GS ? "

Uncle Bernie answers:

Look into the Apple II "Prince of Persia" story as written up by Jordan Mechner himself (alas, I can't find it anymore, so it's up to you to find it in the vast wastelands of the internet). According to Mechner (and this is from my memory, so take it with a grain of salt), he had developed a very tough copy protection for "Prince of Persia" which worked robustly on Apple IIe but turned out to fail on the Apple IIc. He wrote that this problem forced him to weaken the copy protection check, so it would work on both machines. IMHO, this weaking may have enabled "4AM" to do his cracking work, but it is know known that 4AM's theory about the inner workings of this copy protection was a little bit off from what really happens. "S.Elliott" has looked into that and developed an alternate bit cell sequence which also fools the "Prince of Persia" copy protection. Here is the link to that exploit:

https://www.applefritter.com/content/two-disk-replica-prince-persia

This story by Jordan Mechner caused me to suspect (since I started to look into the IWM) that the IWM may behave different in legacy read mode where it should be 100% DISK II compatible. So I was aware of the possibility of a difference in the various state machines and paid attention to every detail while I was reverse-engineering the IWM. It turned out that the designers of the IWM have split the functions of the "Woz Machine" of the DISK II into two independent state machines: a bit cell alignment state machine (which centers the sampling point of the RDDATA input in the mid of the bit cell, and initiates the shift command to the shift register), and a read port control state machine which controls the read data latch (not present in the DISK II), the "freeze" of the data seen by the CPU, the clear of the shift register, and the MSB behaviour in all configurations of the IWM. The key difference to the "Woz Machine" is that the first of the two state machines runs always and never stops / freezes. The "freeze" effect for the completed disk nibble byte with MSB set is done by the second state machine. So unlike the "Woz Machine", the IWM continues the bit cell centering and shifting process even during the "freeze" period without any stoppage. The write protect sense of the IWM also does not affect this process. And this inevitably makes a difference for the E7 protection checks which  manipulate the "Woz Machine" state by briefly changing the mode to write protect sense. The IWM ignores that as far as the ongoing read process is concerned.

There also is a mathematical / algorithmic proof of "state machine equivalence" which failed, so there is a difference. This manifests as a test vector fail of test vectors which were generated from the IWM substitute netlist when the same vectors are applied to the "Woz Machine". The vectors pass when applied to the real IWM in the test rig. What this proves is that the RTL I have for the read process of the IWM is 100% identical to the read process in the real IWM, with the caveat that the real IWM might have more (hidden) functions the algorithm is not looking for, as it only looks to cover all functions of the RTL. But if this test vector set which passes on the real IWM fails on the "Woz Machine", which it did, then it is proven that the two state machines have a difference (and are not state machine equivalent).

This does not imply that specific copy protections which work on the Apple IIe must fail on the Apple IIc. Programmers can work around the subtle differences between the IWM and the DISK II controller. Like Jordan Mechner did, according to his own story as written up by himself.

Furthermore, it was not a "human" (me)  who found the difference. It was a machine / algorithm which found  a difference. And this CAD suite never failed in more than 30 years when it comes to such proofs. Of course, as explained above, it can only check for equivalence of the set of functions it extracts from the netlist which was synthesized from known RTL. For the comparision of two known RTLs it is 100% exact. But when the RTL was based on reverse engineering and the "equivalence test" is done against real hardware, then there may be false positives in the sense that it claims "state machine equivalence" when there is none (but the subset of functions described in the RTL is equivalent). However, the theory behind these equivalence checks also says that there never, ever will be false negatives (test vectors fail despite state machines being equivalent). Since I got a "negative" = "non equivalent" judgement from this automated process, I am 100% certain that IWM is not state machine equivalent to the DISK II "Woz Machine", even if limited to legacy read modes, and hence, they are different and can't be 100% compatible.

Hope this clarifies the subject. I will publish more details when my IWM reverse engineering / substitute work has progressed more, but in its own thread, and not here ... this thread here deals with a floppy emu project, and my well meant advice was to avoid the IWM during this development work.

- Uncle Bernie

UncleBernie wrote:

Look into the Apple II "Prince of Persia" story as written up by Jordan Mechner himself (alas, I can't find it anymore, so it's up to you to find it in the vast wastelands of the internet). According to Mechner (and this is from my memory, so take it with a grain of salt), he had developed a very tough copy protection for "Prince of Persia" which worked robustly on Apple IIe but turned out to fail on the Apple IIc. He wrote that this problem forced him to weaken the copy protection check, so it would work on both machines.

 

You can't find it because the protection was written not by Jordan Mechner but by Roland Gustafsson for Brøderbund, and the //c or IWM was never mentioned as a problem:

https://fabiensanglard.net/prince_of_persia/pop_boot.php

In post #191, 'VIBR' wrote:

" Discussing on Discord with some Guys from Applecause team (Disk Blitz, Keyfrazer,...), they put back on the right track. 

and in the end Retro_Devices was right, SPI can not be the solution... bit bang must be used... "

Uncle Bernie comments:

Ouch !

To me this looks like a severe flaw in the whole WOZ file concept, which brings complications for floppy disk emulators (other than software only Apple II emulators, these are not so much affected, as in there, the "bit banging" is cheap to implement).

Frankly, I did not expect that the very competent Applesauce team would be satisfied with cutting off the track at non-byte boundaries. I don't think this is necessary, as every track, even if made with a professional mastering machine, would have a "write splice area" which which the writing was stopped and the write current was turned off. This write splice area should be easy to detect for original, professionally mastered floppy disks. As the data there is undefined, it is highly unlikely that any copy protection would rely on reading data from that write splice area --- the reaction of different floppy disk drives in the field would be different. But of course, I could be wrong with this conjecture ... never underestimate the wickedness or stupidity of copy protection developers. In this context, with "stupidity" I mean clever concepts which rely on unstable phenomena, and then generate a flood of complaints from customers who have bought an original game floppy disk which would not work on their computer. Things like this can bankrupt a software company ...

The point I want to make here is that once the write splice area is detected, it could be trimmed in length to the nearest byte boundary, and this would allow a clean WOZ file track image which has full byte boundaries only.

This would have two benefits:

First, floppy disk emulators could use DMA which is much more powerful than bit banging ... the problem with bit banging is that it sucks up all the CPU activity and then, how do you handle stepper motor phase change events and reloading of the next track(s) from the flash memory card ? Keep in mind that the RDDATA stream must flow, all the time, without interruptions, even under track changes (at least for those copy protections which use this kind of tricks).

Second, re-creating of the physical floppy disk from a WOZ file would be easier, if the location and size of the write splice area is known.

Unfortunately I can't use Discord to reach out to the Applesauce team. I signed up for it with a bogus email address but found out it would swamp me with garbage and lose track when I go offline (I have no internet at home, and must go to a public WiFi place, this has nothing to do with money, because the gas to get there costs me more than internet at home would, it's just for self-protection). I think that Discord is only good for people who have smartphones which are online all the time, and not worried about the epigones of the Gestapo tracking and monitoring them 24/7. THIS is the fundamental issue with all social media on the internet. They are honeypots to suck you in and to spy on you 24/7, to sell your 'digital self image' to third parties, including foreign and domestic crime organizations. No good, at least for me !

This said, I'm sorry I can't reach out to the Applesauce team for discussion of this issue and possible improvements to their WOZ file format.

But I think that if you can identify where the write splice area is in the current WOZ file format, you could do the manipulation of the track to yield clean byte boundary cutoffs with a purpose written tool running under Linux. Use this tool to convert the original WOZ file to a VIB file which then is fed into your floppy emu. This would allow you to keep your DMA.

The point is that I don't see a way to make a bit-banged floppy emu work. The ST32 is no multithreading / multicore CPU, no ?

(Oh, and now I think I have a hunch why the BMOW Floppy Emu has a quite sophistcated CPLD/FPGA on it).

- Uncle Bernie

Hi Vincent -

I've read your post #193 and understand your reasoning. However, I think if you go with bit banging, your work will be harder and forever locked to the ST32 version you are using (not portable without starting from scratch). It's your decision what you do, as it's your project and your time, but you have to weigh all factors before you make a decision.

I am sure that if we could detect the write splice area of each track, it would be easy to remove or add 1...3 bit cells in this splice area to have only whole byte bounday aligned tracks in the track buffer. With proper wrap around on byte boundaries you could use your DMA / SPI based concept, which is more elegant and easier to port than any bit banging concept would be.

I don't think that this small manipulation of the write splice area by no more than 3 bit cells (or 12 microseconds) could have any effect on any copy protection. Unless that copy protection would look at the write splice and have some magical algorithm to detect if it is the original write splice. I think that would be a hopeless attempt, but I may be mistaken. I do have reasons to believe nobody ever did that, however. Because some 40 years ago I tried to generate unstable bits in floppy disk data sectors by using write splices. Which never yielded any reproducible or useful results the copy protection check would swallow as "good". We ended up to solve the problem by putting a second incarnation of the unstable sector in the track, which was read from the original disk, and only a few bits in it were flipping and flopping. Since we read the various incarnations from the original disk, the copy protection had to accept them, naturally. This exploit had the added benefit that we did not need to upgrade the copy chip we were selling at the time ;-)

For the Apple II, the situation is a little bit different as there are those copy protections which involve synchronized tracks and all sorts of stepper motor phasing tricks. But here is my rationale why we still could dare to add or delete a few bits in the write splice area:

If we make the hypothesis that these "synchronized" tracks were written as "fat tracks" - if such a thing really exists physically, I did not check my specimen disks with magnasee fluid yet - then this implies a wider write head, and the consequence would be that the contents of the quarter tracks covering that fat track would be perfectly aligned. In this case, you would use ONE track image for all these quarter tracks. The WOZ file format supports this method.

But if we make the the hypothesis that two (or more) "synchronized" tracks have been written using a normal write head, or were written by a 80 track drive, then any pair of these tracks cannot be perfectly synchronized (do the math: 12 microseconds for 3 bit cells vs. 200 000 microseconds per revolution at nominal RPMs). It is impossible to have perfectly synchronized tracks if they are written track by track, just due to wow and flutter which is inevitable with the spindle motor speed control and the non-constant friction effects etc. So in these cases, the copy protection check must allow a certain skew between these synchronized tracks. Which allows us to do the +/- 3 bit cell manipulation again.

The only copy protection which could puke on the proposed manipulation (other than the attempt to check the write splice area itself, which I dicussed above and don't fear) is one where a real  'spiral track' would produce a continous RDDATA stream over several revolutions of the media, and every revolution providing new and valid data. Theoretically, such a spiral could cover the whole area of the floppy disk, with no interruptions. Much like a vinyl record for music (remember these ?). But I can't say if such a copy protection really existed for the Apple II. The AppleSauce team should know more. Ask them. Note the fine point I am making ... spiral track with no interruptions of the RDDATA stream, ever, and no slips / resync action necessary while reading for more than one revolution. Spiral tracks could also be done with interruptions of the data stream, all they would need to do is to throw in a few SYNC bytes to recover. I can't say what they really did back in the day. What I know is that industry standard mastering machines could NOT generate  a spiral track continously because they used a hardware track buffer  that had a limited length and needed reloading when exhausted. The upside of this was that they were able to control the write process in a very fine grained way ... including "density frequency modulation". There were tricks which were played on PLLs of floppy disk controllers which timing wise look like fishbones. The target was i.e. the Western Digital FDC in the Atari ST computers, which had a digital PLL that could be made to do weird things with these tricks. Absolutely impossible for a copy program running on the Atari ST and using the same FDC to duplicate such a copy protection. Happy Computers solved this with their HART custom chip. But I don't think they sold enough of them to be a threat to the software industry. But back to the Apple spiral disk protections. Such a uninterrupted RDDATA flow spiral track of several revolutions of length could be produced on an Apple II itself, using the unmodified DISK II system. The great difficulty for a copy program would be to properly analyze the spiral to later reproduce it. So that mythical thing could exist. And if it exists, any WOZ file and any floppy emu would struggle to replicate it. The WOZ format probably could do it ... but analyzing such a continuous spiral would be hell. The copy protection read algorithm could use encrypted clues in the data stream coming from the DISK II controller to 'servo' the read head in the proper way, hereby guaranteeing successful read even under RPM variations. And the spiral as such could wiggle around in unpredictable ways. Again, note this is my conjecture / hypothesis / fantasy (you name it) based on the toxic thoughts I have in my mind. This is how I would design a spiral track protection. It would be incredibly difficult to reproduce this even with our current technology. The open question is if any real world Apple II software title ever used such a tough copy protection. Ask the AppleSauce team.

Once you have collected the information about the above topics, you can make a decision on which method to use for your floppy emu based on facts. Before you have all facts together, I would not recommend you to throw away all your work and start from scratch (using bit banging). I still hope my proposed bit cell manipulation method could make the DMA approach viable.

- Uncle Bernie