Apple II DISK Emulator using STM32

Hi Vincent -

the connection of the WRDATA to the LS74 seen in your post #97 is correct (the rule is, never work with signals coming from a longer cable without conditioning them with a Schmitt Trigger --- this will save you a lot of trouble).

The answer your question of post #98, the DISK II floppy drives are stripped down to the bare bones and they saved a 'toggle flipflop' there by providing the magnetic flux changes directly from the DISK II controller.

This means that for every "1" bit cell, the WRDATA must toggle (normal industry standard floppy disk drives expect an impulse instead).  The circuit with the first D-FF U7A and the XOR reverses this toggle process and produces a "1" for every change of WRDATA. The 2nd D-FF U7B captures this bit cell value and stores it until the next C_OUT clock. The SPI will sample the bit cell value one half C_OUT clock period later (on the negative edge of COUT, where the D-FF U7B does not change).

If you capture this data stream with the SPI you should see a "1" bit for every bit cell that was written as "1" to the DISK II card's shift register. You should be able to identify the SYNC bytes which are 1111111100 1111111100 ... so these are 10 bits wide and not 8 bits as a normal byte. The read process will swallow the extra "0" bits but you must keep them and later send them through the read channel, so the synchronisation process works.

When WR_REQ is deasserted, the U7B is kept in a clear state and this produces a stream of "0" bytes .... be aware that these will overwrite anything that was written after one emulated revolution of the floppy disk, as you have a circular DMA process.

This is intentional. Let me explain why:

The proposed method of implementing the write channel has the advantage that the ST32 software needs only to move small quantities of data (i.e. one sector) at a time and with its 84 MHz clock and 100+ MIPS it should be able to do that purely in software with no DMA peripherals, and easily stay ahead the 1 MHz clock 6502 pumping that data with something like 0.2 ot 0.5 MIPS, and 8-bit MIPS at that.

Another, more important, advantage is that the master "read track buffer" stays intact and is only manipulated by the ST32 software. The relative "angle" of the track in that buffer stays the same, when measured to other tracks. The "sectors" never move. An important feature of real floppy disks, and exploited by many copy protections ("synchonized / aligned tracks").

If you can't figure out how to do that in software, there is another option, which involves adding hardware. You could add a 74LS157, which has four 2:1 multiplexers inside, only two of which would be used. WR_REQ would go to the "Select Input" and the multiplexers would be wired such that when WR_REQ is deasserted (read mode) the SPI clock and data for the write channel would come from the read channel. When WR_REQ is asserted (write mode) the SPI clock and data for the write channel would come from the 74LS393 and the change detector. The net effect is that once this machinery is in place, the revolving read track buffer would be copied automatically into the revolving write track buffer. And any real write coming from WR_REQ would magically end up at the right place in that write track buffer. No fine adjustment / clever software algorithm to determine the right place needed.

But as nice as this appears, like with many "easy looking" plans, there is a devil's hoof (more than one):

First, any sector written will persist in the write track buffer for only for one revolution, and then jump back to the previous state. You would need to correct that by flipping the read and write track buffers just at the right time, such that the "new" track becomes the read track buffer, where the just written sector (or track) data persists. This "flipping at the right time" may be tricky. You need to know what was written, and when, and where, and do the flip just at the right time to capture the correct, current state of the track that just has been modified. If anything goes wrong with the timing of that, you get garbage, a mix between the old and the new state, or some of the track gone missing.

Second, due to the latency / delay in the SPI / DMA data pipelines, the relative angular position of the involved track buffers will slowly move from the original situation. You need to have some code which detects and corrects for that angular error. This can be tricky ... what are you looking for ? A sequence of bytes ? That may be shifted bitwise ? Or may have been overwritten ? Can you extract the exact position shift from the DMA machinery ? How would you correct for the error down to the individual bit cell ? Questions, Questions ...

... which I never have solved. In my floppy disk emulator of ~ 30 years ago there was a hardware track buffer (a SRAM) with a circular address counter, and single bytes would come out in read mode and single bytes would go in, when WRITE GATE was asserted, to the same byte position. It also made a virtual index pulse when the address counter rolled over. The Western Digital floppy disk controller (I think it was a WD2793, but not sure, has been a long time, it supported FM and MFM modulation) used that index pulse so start / stop track formatting so all was nicely aligned. When the head was moved, the track buffer was flushed to main memory ... stepper motor step delay and the head settling time delay was enough to stash away the old track and to load the new track. From a PC running under DOS, of course. Enough RAM to hold a complete disk image. Disk images were stored on the hard drive. It worked fine, even with the slow PCs of the time being. Today I would use the "Ratweasel" hardware and software which relies on the higher data rate and ECP parallel port of a more modern PC or notebook. Alas, even those are obsolete and hard to find in good condition today.

The ST32 based floppy emu leverages the immense power of a modern microcontroller and certainly is fast enough to handle everything. But it relies on complex inner machinery like the DMA channels, the FIFOs, the SPI peripherals and all of this data path has latency / pipeline delays which may be tricky to account and compensate for in software, to avoid corruption of the disk image. But speedwise, it could also emulate the whole Apple II and provide live video, if it had enough RAM (the basic Apple II had 48 kBytes of RAM, but to run all software you would need 128 kBytes plus some for the emulator itself). But we don't want to go there ... we want just a floppy emu. I'm quite sure you can do it, but you need to tackle the fine details and complications from the peripherals you have in the ST32. The more powerful they get, the more surprises they have lurking in them.

- Uncle Bernie

VIBR wrote:

typical signature D5AA96 & D5AAAD...Well, I am progressing but clearly I am not done with the wirting process.

Are you ready with the reading emulation? These signatures can be different, why don't you try to detect sector gaps instead?

... because you seem to have progressed further than I thought with the write process. But I made a short writeup of my thoughts about the write process which I don't want to throw in the digital trashcan. Here it is:

The method to copy individual sectors from the write buffer to the read buffer needs no complications. It can be done on a byte level, without bit shifts, because the possible maximum error of a 7 bit shift due to data path latency is far less than the 1% (max 3%) of rotational speed error of a typical floppy disk.

SHORT SUMMARY HOW SECTOR WRITES WORK (different from full track format !):

IIRC, during sector write, the RWTS will "look" for the right sector header which has the form:

$D5 $AA $96 .... the "prolog", followed by eight disk nybble byte which are FM encoded byte pairs:

1 D7 1 D5 1 D3 1 D1

1 D6 1 D4 1 D2 1 D0

the first byte of the pair, shifted one bit to the left with a "1" shifted in ANDed with the 2nd byte gives you the final byte value, here is an example of 6502 assembly code (from memory, so take it with a grain of salt):

LDA BYTE1

SEC

ROL A

AND BYTE2

The eight disk nybble bytes after the $D5 $AA $96 prolog hence yield four de-nybbelized bytes which are:

Volume

Track

Sector

Checksum

This is followed by an epilog:

$DE $AA $EB

after that there begins a GAP. The RWTS takes a while to de-nybbelize the information and to compare if the volume, track, sector, checksum is OK, and to prepare for write. It then turns on WR_REQ and writes five SYNC bytes (1111111100 1111111100 1111111100 1111111100 1111111100) before it writes the data field prolog $D5 $AA $AD, the 342 disk bytes for the 256 sector data bytes (6/2 encoded), the checksum, and the epilog, as above,$DE $AA $EB.

It then puts a final byte into the DISK II shift register (IIRC, all 11111111) and deasserts WR_REQ. Normally, these last "1" bits don't all make it to the floppy disk, some get cut off by the end of write. And there is an ugly write splice there, some random garbage, but only on the real floppy disk.

HOW TO IDENTIFY WRITTEN DATA SEQUENCE

With my proposed (separate) write track buffer, the end-of-write is the first zero bit after a sector has been written and captured. Easy to detect. By searching backwards from the approximate begin of the write for the first zero byte you can detect the start-of-write.

What is left to do is to find out where in the read track buffer this sequence of bytes needs to be copied to. If you capture the position of the forever revolving timer upon WR_REQ assertion you can calculate the place in the read track buffer accurately enough to know where the data byte sequence from the write track buffer must be copied in. The forever rotating timer must be exactly in lockstep with the revolving read DMA ring buffer to make this possible. The slightest error will accumulate and wreak havoc. I don't have the time to dig that deep into the ST32 datasheet to tell you how to accomplish that. Maybe you need to restart the timer when the DMA ring buffer wraps around. Or maybe you can look into the state of the DMA channel to find out - but beware, if a FIFO is involved, there will be DMA bursts and this will warp the time axis.

As you have correctly stated in your last posts, the write block detection and copy process must take into account possible "wrap around" situations in both buffers. But I think this is quite trivial to handle if you work with software only and don't try to use DMA hardware to make the move. It boils down to increment and decrement pointers and check for the boundary cases (index 0 and index of last byte of ring buffer) before doing that.

NO DECODING NEEDED (except for analysis during debugging)

Note that for the detection of the begin and end of the disk nybble data field in the write buffer you do not need to decode anything, just treat it as a sequence of bytes. And when you got your positioning into the read track buffer right, based on said timer, you just can copy it bytewise ... but don't copy any of the leading or trailing zero bytes in the final revision. You may copy ONE leading or trailing zero byte each during the debugging phase so you can easily see where it was copied to.

BEWARE OF ZERO BYTES

But for general purpose, all zero bytes do not exist in regular bit cell streams of Apple II disk images, EXCEPT maybe for some copy protections where instable bits must be emulated. I can't say much about this - ask the Applesauce guys - but I dimly seem to to remember I once saw a code piece in some Apple II emulator (I don't rememember which) where the DISK II controller emulation module looked for a sequence of "0" bit cells and then inserted a random "1" after three or more of these "0" bit cells.

This obviously was meant for the emulation of "instable" floppy disk bits used in some copy protections. I'm not sure if the use of random bits is the best approach to deal with it ... because I know I could construct a copy protection and a copy protection check which would produce unstable floppy disk bits but could not be foiled by said technique.  It again boils down to 'plesiochronous' systems. You can construct a sequence of bit cells where a 'clock slip' is provoked at some known locations. The exact point in time will depend on rotational speed of the floppy disk, and the jitter of the analog channel. So it can't be guaranteed exactly where the 'event' will happen. But what you know is that a "1" bit may be 'seen' earlier or later, and if 'seen' later, a "0" bit may be inserted. But except for this 'event', the bit cell sequence will be predictable and can be checked to meet expectations. Swindlers who try to trick this copy protection with a random bit cell state will not succeed. So here we have a (hypothetical) example, or "thought experiment", which proves that random bit cells in floppy disk emulators may not cut it.

The only question is if there ever were Apple II copy protections, back in the 1980s and 1990s which had that level of sophistication. We need to keep in mind that if a simple method to trick the check into accepting the emulation as an original floppy disk (which long has been eaten by mold and/or is decomposing) works for 90% of the titles, we are good. All the few exceptions could be handled by applying cracks. The "4AM cracks" are probably the best because they try to preserve the original software as much as possible. This does not apply to most of the lesser quality  1980's cracks which were done by teenage computer nerds who had to leave their "greetings" all over the place or removed title screens completely. From a standpoint of software preservation / emulation for eternity, such examples which have been mutilated by these teenage software pirates are worthless. Unless no originals could be found and properly imaged by specialized hardware (Applesauce).

So far my thoughts for today.

- Uncle Bernie

For debugging purposes I suggest you to attach your "emulator" in parallel to the real floppy drive/diskette and log only the writinig of one track to see what you receive in your buffer compared to the real track written on the physical diskette. Leave the reading channel connected to the real floppy drive. Just "sniff" the WRDATA.

In post #107, "VIBR" wrote:

"Is it worth to increase the frequency of TIM4 (currently 2Mhz) to 4Mhz ? and have COUT to Q3 (8)"

"EUREKA BAM !!!!! IT IS FREAKING WORKING !!!For the first time Locksmith approved the writing !!!I need to do some investigation sharing the bearsking among the nations before the bear is dead."

Uncle Bernie comments:

Congrats for reaching this important milestone (Locksmith approves the written track). But believe me, more work is ahead of you ... the devil is always lurking in the details. But you made good progress for the time invested.

As to the question of increasing clock speed for the TIM4, you should do that. You may go as high as possible. You could feed it 4 MHz, 8 MHz, 16 Mhz, 32 MHz, ... and then use Q3, Q4, Q5, Q6. (I did not check the datasheet if the 74LS393 could run at 32 Mhz, but there is always a 74S393 or 74AS393 option. Avoid CMOS like HC and HCT as long as you are using the plugboard. A properly designed PCB could take CMOS).

The benefit you get from higher clock speeds is smaller phase jumps on the SPI clock, so it's easier to analyze. You also will get fewer metastability events. Same reason why Apple increased the sampling speed when they designed the IWM. Except that they were dealing with RDDATA, and not with WRDATA. Going higher than 8 Mhz may not be necessary.

- Uncle Bernie 

VIBR wrote:

However, looking at the datasheat, currently I count til 4 and I use Qc (2Mhz), I can go easily to count to 8 Qd (4Mhz) but above, I will need another 74LS like the 74LS08.

I would rather to go to 4mhz and use Qd 

Wrong. You can count to 256 by cascading the other half of the 393 that you are already using. Qd must be wored to COUNT of the other half.

In post #110, "VIBR" wrote:

"Do you have somewhere a schema to protect the circuit against inverting the Apple 20 pin connector consequence (+12V -> +5V) ?"

Not yet. But I need to work out such a scheme anyways for my own purposes. I will send you a schematic when ready, but don't hold your breath. I'm working on many projects at the moment, not all are electronics, and have very little bandwidth left over.

In post #111, "retro_devices" wrote:

"You can count to 256 by cascading the other half of the 393 that you are already using. Qd must be wored to COUNT of the other half."

Correct. The QD output of counter 1 must be wired to the clock input of counter 2 to cascade the counters. This is why I recommended the 74LS393 ... I have foreseen the case that we may want more counter stages. But when you use cascaded counters, do not forget to also wire the CLR inputs together. Otherwise, only half of the counter would get cleared, which would cause that digital PLL not to work as expected. (Just for the nit-pickers and the curious, it's not a true PLL yet as it lacks a VCO and a loop filter, but it does the same work, except for having more phase jitter due to the fixed clock frequency. But it's good enough for the application. A lot of the so-called "data separators" in the early 1970's and 1980's floppy disk controllers used just a simple 4-bit counter in the same way, to make a sampling clock in the middle of a bit cell. So it's an old and proven method.)

- Uncle Bernie

In post #14, 'VIBR' wrote:

" When I copy a track using Locksmith the start of the track can be in a middle of a previous sector.  "

Uncle Bernie comments:

I don't know Locksmith, never used it, but I think that UNLESS you tell it (somehow) to align the tracks, it will start writing the track at a random location. This was the common procedure with  most track based copy programs. Aligning the tracks is possible but involves extra head movements to "look" at a reference track to which all the other tracks will get aligned. Of course, the software must compensate for the head movement times.

This increases the time expenditure to make a copy by 2 ... 3 fold so the default option was to do no track alignment. Only for copy protected disks which used track alignment aka "synchronized tracks" etc. (it has many names) there was a switch to turn that extra function on.

I think (but not 100% sure) that the best approach would be to detect if a whole track is being written or if only single sectors are being written (as usual for DOS 3.3). For single sectors, copy them to the right place in the "read track buffer". For whole tracks, discard the "read track buffer" and use the newly written track as the new read track buffer. I suggested to make this different approaches before in this thread. And I also suggested to make the single sector writes work first. All copy programs like Locksmith always reformat / write whole tracks and this is a complication. Honestly, I don't think a floppy emu must be able to receive newly made disk images produced with such a copy program. These programs (Locksmith and the like) were meant to duplicate copyprotected disks and although they were popular back in the 1980s, (for obvious reasons ;-) they never were perfect and whatever images of copyprotected floppy disks you might be able to make, the result will be inferior to WOZ images made with a flux engine and the quite sophisticated tool chain involved. A copy made with Locksmith or any other software only solution relying on the original DISK II hardware always must be doubted - is it really a faithful, perfect reproduction  of the original copy protection scheme ?

The ability of a floppy emu to take sector writes from DOS however is a must.

Oh, and don't worry about bit shifts. The captured disk nybbles in the write track buffer indeed can start at any bit position. There is no need to correct anything or shift them back, as long as the integrity of the whole bit cell stream is intact. You analysis tools, however, must be able to handle the situation. The best way is to pipe the bit cell stream into a Woz Machine emulation and watch for the re-assembled / re-aligned disk nybbles falling out of it. This will also remove all the superflous "0" bits in the SYNC bytes. Which on the emulator image of course must stay as is !

- Uncle Bernie

The more detection of patterns you implement in your firmware the worse and more partial this device will be. Locksmith is great and was written by real masters of Apple2 and assembly language. And was used back then (by me and my friends) more than DOS 3.3. itself, considering many software was not realeased on (standard) DOS at all. Its Fast Backup is still faster in copying a diskette than the contemporary buggy applesauce with its overclocked ARM for example. 

If you follow that path your "emulator" is doomed. Try to emulate a diskette. It has no logic to discard anything, except once it overlaps due to circular tracks . Just store everything on a given track within some 200 mS FIFO and be able to play it. Overlap/overwrite (this is an effective discard) when wirting past 200 mS. Keep precise with sufficient resultion "angular position" pointer, e.g. time, or counter whatever you call it common for all tracks because the emualted diskette is one physical disc. Do not interpret any data. Leave that to controller+Apple2 software. If you wish make your own file/image format that is more convenient to you. If your chosen STM controller has no sufficient RAM, try to add or use better MCU with more RAM.

In post #120, 'retro_devices' wrote in response to 'VIBR's post #119:

" If you follow that path your "emulator" is doomed. "

Uncle Bernie agrees with  'retro_devices'. Don't try to get too smart and don't try to scan for headers etc. to do "housekeeping work". You may look for them in your test setup (I suppose you can look at the track buffers of the ST32 using a PC or notebook) to analyze things, as I mentioned in this thread, but don't use that in your floppy emulator firmware itself. These header sequences exist in standard formats made by DOS 3.3  but may be modified or completely absent in copy protected disks. So relying on them to do any track buffer housekeeping is shooting yourself in the foot (and dooming your project).

What I meant with my recommendations is to not reach for the moon too early. Solve the simple case first - DOS 3.3. writes to a pre-formatted disk image. This means you only need to process small chunks of data written to the write track buffer and then to copy these chunks of data, without any decoding, bit shifting, interpretation, to the right section of the read track buffer while the copying process zeros out the bytes in the write track buffer it  took out for copying into the read buffer. This leaves zeros in the write track buffer for all work done.

This should be almost trivial to code EXCEPT how to find out where in the read track buffer (location) that data chunk needs to go. And this is strictly a function of your DMA channel latency and what is in the FIFO between the DMA and the SPI. You need to figure out how to determine that location from any state you may be able to capture from the read DMA channel by a WR_REQ initiated interrupt routine. Without looking at the contents of the read buffer. Without trying to "understand" the format.

If this is not possible you need a side channel, in hardware, to measure that latency. Which greatly complicates the hardware. Just to measure DMA latency. What a mess ! (Seems 21st Century super-MCUs have their perils because they are not very predictable ... maybe. Can't say as  I don't work with ST32, but I see the possibility of unpredictability, if they got too smart).

Full track writes as done by Locksmith are a different process and need different handling of the track buffers. AFAIK, Locksmith will write a complete track at once, and then go to read mode to verify it. If it finds something odd or wrong, it will retry. I think there is no way to catch up with that write data rate using a ST32 software copy loop, so you somehow need to flip the buffers. One approach you could try is to turn the write buffer into the read buffer once Locksmith deasserts WR_REQ. So Locksmith will "see" what it has written, with almost no delay, and might be happy.

None of this should require decoding and understanding Apple II GCR formatting. It should be possible to treat everything just as a stream of bytes. But it seems that you always need two track buffers, one for reading, one for writing, because synchronizing two SPI DMA channels such that they work on one track buffer, although this at first glance seems to be the right way to do it, opens a whole can of worms, read: lots and lots of possible complications.

So better keep it simple. If the ST32 is too advanced for the project, you can always go back to the 100% hardware based track buffer and a small 8-bit PIC processor for the housekeeping of the track changes, etc.

- Uncle Bernie

Think of a track as time. You have 200mS revolution time (approx, varying with rotation deviations from drive to drive). Divide it if you wish in 1, 2 or 4  microsecond parts, that either have flux change or not. 

In post #122, 'VIBR' wrote:

" The full track write with locksmith is not an issue, the issue is the circular buffer length (6656 13*512) and the length written by locksmith (6450 approx).

The 200 Bytes is creating the mess, and I can not predict the size of the written track, the only solution is see is discarding the content of the 200 bytes remaining with 0 when writing to SD Card."

Uncle Bernie comments:

Here you have the culprit for your troubles: excess length of your track buffer. Don't do that unless necessary for an image of a copyprotected disk that has a copyprotection which calls for a higher bit cell density ---> more bytes of track buffer needed.

To help understanding track buffer size:

it depends on whether you work with

(a) 'clean', synthetic disk images or with

(b) 'dirty' ones that came from an original disk, or with

(c) images from certain copy protected disks.

Let me explain these three cases:

GENERAL THEORY

The NTSC Apple II has a 14.3181818 MHz master clock and the typical CPU cycle has 14 such clocks (a few cycles are 'longer' but this does not matter for the floppy disk system and can be ignored for floppy emulators). So one CPU cycle has 0.9778 us (microseconds).

One ideal bit cell (as coming out of the master clock derived Q3 clock controlled DISK II card) has 4 CPU cycles, so its duration is 3.911 us, or, seen as a bit cell clock frequency, 255.7 kHz. (it would be great if you could match this frequency as close as possible for your read channel).

The rotational speed of the ideal DISK II is 300 RPM, or 5 revolutions per second, 200 ms (milliseconds) per revolution.

Into these 200 ms, 6392 disk bytes ("disk nybble") of 8 bit cells of 3.911us do fit.

CASE A):

So for case a) you need a track buffer size of 6392 disk bytes. Not more, not less.

Note that this case a) was meant for 'clean', synthetic disk images where everything is ideal, and crystal controlled, and the "floppy drive" runs exactly at 300 RPM. Which is never the case in the real world, with real floppy disk drives. For this reason, all programs which format a track (and this includes DOS 3.3 when formatting a disk, or Locksmith writing whole tracks) include automatic adaption to the actual number of bytes available per revolution. They start with long gaps between the sectors (or in case of Locksmith, one long splice gap), write the track, verify the track if its end has overwritten the beginning, and if so, then try again while reducing gap size in small decrements until the track fits and readback verifies it's correct. This means if your floppy emu offers only 6392 bytes of track buffer, any DOS or other tool will do the work to make their output fit into that buffer size. If you increase or decrease the track buffer size by a few percent (to mimic the expected variations and differences in RPMs of different DISK II floppy disk drives), it still will work. IIRC, the adjustment tolerance target for RPM was +/- 1% and the whole disk system was designed to be able to handle +/- 3% before its inner workings fall apart and fail to work.

CASE B):

These RPM tolerances explain case b): when a disk image is made from a real floppy disk, which had inevitable RPM deviations from the ideal, and assuming a fixed bit cell clock frequency, the proper emulation of that track will lead to slightly different track buffer sizes. What really matters is the exact number of bytes in the track, treated as a ring buffer, after N bytes where it must "roll over" seamlessly, with no time delay. With standard DMA channels this would be impossible unless a FIFO is present, because the software would need to watch for the DMA being finished and then - as quickly as possible - restart the DMA at the beginning of the track buffer, before the FIFO runs dry and the output data stream has a gap. Such a gap would kill the whole concept. The ST32 claims that its DMA supports circular buffers, so this should never happen.

This case b) must be handled by allowing programmable track buffer sizes. The WOZ files contain information on how many disk bytes are in each track. A floppy disk emulator must stick to this number.

CASE C):

Case c) is for nasty copy protections which change the bit cell size. This can be done either by changing the clock for the write process (this is how the mastering machines handled it (IIRC, FORMASTER called this "density frequency modulation", and they were proud to have that capability, which was not available with their competitors, at least not in that form), or it can be produced by changing the RPMs of the floppy drives, if the bit cell density of a whole track had to be changed - manipulation of RPMs is not as powerful as "density frequency modulation".

I can give you an example from the 8-bit Atari world: the normal number of sectors on a track for the Atari 810 floppy disk drive, which was FM aka "single density", was 18 sectors of 128 bytes. So the software pirate scene was stunned when games protected by 21 sectors per track did appear (I don't remember which titles, it happened more than 40 years ago).

At first, this copy protection appeared to be invincible. Until the "Happy Computer Company" which produced the infamous (from point of view of software publishers) "Happy Enhancement" for the Atari 810 added a way to reduce the RPM from 288 to 270, and then the 21 sectors could be written. "H.W.A." mode, or "Happy Wins Again" was the slogan for that mod. Other makers of Atari floppy speeders / enhancements quickly followed suit. One of them, the "1050 TURBO", had the speed change transistor on board and came with a small cable ending in a "microhook" grabber, so solderless installation was possible.

But back to the Apple II: at about the same time (early 1980s, before 1985), when the "density frequency modulation" protection schemes did appear, they also started to appear on Apple II floppy disks. I don't know if "Locksmith" ever "defeated" them. But the WOZ images have a way to support emulation of such higher density tracks, by providing the option for larger track buffer sizes. Which, IIRC, was not present in earlier versions of the WOZ image specification. And from my background in such copy protection schemes based on "density frequency modulation" on other computers (I have no experience with them on the Apple, though) I have doubts that even the current version of the WOZ file spec could properly support emulation of all possible "density frequency modulation" tricks. I think it may be beneficial to adjust the bit cell frequency (of the SPI clock in the read channel) such that a track with more bytes still has 200 ms per "revolution".

But here is the key point: floppy disk emulators (and their image file formats, like WOZ) only need to support copy protection schemes which actually did exist on Apple II floppy disks back in the day. Anyone could "cook up" a very toxic brine of copy protection nowadays by using a powerful flux engine, such as my "Ratweasel" sure could do, if I wanted to waste my time on coding the algorithms. But it is pointless to do so - real floppy disks are long obsolete, and the "New Old Stock" floppy disks we can occasionally buy also may have deteriorated to a point where they are useless. Plasticizers and lubricants disappear over the decades and mold also loves to "eat" them - I think its the glue which bonds the magnetic dust to the disk which gets eaten by the mold. (Your floppy disks may be "alive" and "crawling" with dangerous microorganisms, eeeek !). So I consider it a purely academic exercise to cook up new copy protections for floppy disks. Could be fun, though.

So far my comments / notes / background info on track buffers and how they relate to real world phenomena found with real floppy disks. Hope this gets your floppy disk emu project back on track !

- Uncle Bernie

Many here (including myself) are following your progress on this.  This project has shown technical knowledge far beyond what I personally have delved into -especially with Disk ][,  but it's of high interest & importance to many.

Congrat's on your progress and I look forward to your final result & success.

I still think that the project might have been more successful if programmed mainly in ARM assembly language. From the past I have stm32nulceo evaluation board for testing but it is with STM32F429ZI...

In post #130, 'VIBR' wrote:

" Retro_device and Uncle Bernie were instrumental to make this happen, still a lot to be done. 

 I will try to make a first prototype pcb so that people can start playing with it "

Uncle Bernie comments / advises:

Great progress was made indeed, as evidenced by the example ween running  in post #128.

If you look at the start date of this thread (2nd June 2024) there were only 5 weeks to reach this milestone.

About "still a lot to be done":

You are now at a point where you should check a lot of WOZ images of copyprotected disks on your emulator to see if everything works. Of particular importance are the "fat track" and "spiral track" protections. If these work, you have proof that your hardware choice is sound, and that your track buffer structure can handle this. Alas, I can't tell you which software titles to choose as a test suite to cover all protections. I do have a set of original Accolade "Comics" disks which are said to contain a "fat track" protection but it was a recent acquisition and I had no time to analyse it yet. So your best course of action is to ask others lurking on Applefritter if they could compile a list of test cases for you. The WOZ files can be found by searching for "Woz-A-Day Collection".

As for your plan to make a prototype PCB, I would advise against doing this prematurely, before you ran these tests. If you discover a case where the hardware needs to be modified / upgraded (i.e. a larger ST32) then your effort on the PCB would be wasted, and you will have created a bunch of disappointed and angry early adopters. So please do the tests first to get confidence that the hardware platform choice is right. This would also allow me to catch up with the design of the protection circuits. We could also put much of the logic into a PLD, such as a 22V10, or redesign the TTL stuff to avoid the nasty 74LS123 oneshots. IMHO these are NOT a good solution to make building the emulator by hobbyists foolproof. Without having an oscilloscope to check the timing, there is a risk of failure lurking. For my Apple-1 kits, I had to hand select all the 74123 and the timing R's and C's in a test rig while taking measurements on an oscilloscope. Just to guarantee the timing to be right. Otherwise it could have been a disaster. I think the clocks the ST32 (possibly) could produce from its crystal timebase could be harnessed to toss the 74123 out, and to have a solution that is robust and foolproof.

BTW, your choice to use an internal timer of the ST32 to make a finer timing for the read channel as seen in your post #127 was sound. The important thing here is that the emulated floppy disk ALWAYS should have a 200ms track repetition period, preferably +/- 1% tolerance, regardless of how many bit cells are in the track. The is the most faithful solution to emulate a real floppy disk, but requires some math to calculate and adjust everything. For read only it's trivial to do, but when you allow writes to such a higher or lower density track, it gets more complicated. This is yet unexplored territory ... and may bear some surprises and extra work for the copy process from the write channel track buffer to the read channel track buffer (the latter of which is the "timing reference" for the emulated floppy disk).

Recently, while scavenging the internet for ideas, I also have found a cheaper way to do the SD card interface, see here:

Micro_SD_adapter_A.jpg

No need to buy expensive "shields" with SMD based connectors to use micro SD cards. The adapters are abundant and a dime a dozen. Most people have some left because they come with many products as an accessory. Those who threw them in the trash have to buy them, though. But it's still cheaper than these "shields". The trick is to solder a 2.54mm raster pin header row to the contact pads on the backside of he adapter. All of them except the last one fit perfectly. And that last one, despite being a bit off center, still is doable:

Micro_SD_adapter_B.jpg

This DIY adapter for micro SD cards is not only much cheaper, but much more flexible for the mechanical arrangement / enclosure of the floppy emu. With the flat pin header row it would stick up, perpendicular to the PCB, and with a 90 degree bent pin header it would be parallel to the PCB. Several rows of pads could be provided on the PCB to allow installation as needed to fit into any conceivable enclosure.

So far my comments for today.

- Uncle Bernie

You'd better make decent working prptotype and then start optimizing it by reducing the number of ICs with PLDs. The external  ICs could have been eliminated by using lower level assembly language ARM programming for the diskette I/O. 

What you have built in post #139 is a digital oneshot. Which is a valid and good approach. But you should never leave tristate buffer outputs float, if used as a gate. The always must be a pullup resistor (for Z->H) or a pulldown resistor (for Z->L). Sizing will depend on the load. For a pulldown use 330 Ohms or less if the load is a single LSTTL input.

A similar approach based on a counter can be used to substitute the other oneshot. The 74LS393 has two independent counters in it.

I need some time to digest your schematics, and I'm very busy right now as I have found yet another bug in the IWM, at least it looks like it. Not a bug in my substitute, mind you, but a bug in the real IC as used by Apple. These are nasty surprises because they throw the 'state machine' equivalence tests off track. I need to develop the theory how the bug came about, and this costs time. More in my IWM thread. Will post info there when I have more time.

As for the PLD substitute, it may be possible to put all TTLs into one PLD. A 22V10 has 10 macrocells, so it can make 10 output functions. A EP610 has 16 macrocells, and there are larger ones.

The benefit of TTLs is that no builder of your floppy emu needs to source a programmed PLD. The downside of TTL is that circuit changes are tricky, so people can't update  / upgrade / experiment / hack the emu without cutting PCB traces, piggybacking ICs, and adding flight wires. It's a mess, as you can see on your plugboard. PLDs can be changed by just reprogramming them, for which a programmer is needed. BUT, you already have a part in your emu which needs dedicted programming hardware, the ST32. So the "programmer" problem is already here and the best solution is that you make a kit with the pre-programmed ICs and the naked PCB. The other components can be sourced by the builder. This would keep weight of the parcel (postage) and customs fees down. Making the project "open source" as you do at the moment is a noble intent but I believe it does not help the typical builder interested in the floppy emu to get one. The PCB alone is  a big headache as it only will be "cheap" if ordered in larger numbers. Even at JLCPCB the "prototype" runs of small PCBs in small quantities are pretty expensive. If you offer a kit and order, say, 50 PCBs, one of them may cost only $2 ... $3 (just an estimate based on the Apple-1 ACI card). Note that PCB costs depend on area and number of drills. This is why a TTL based solution will drive up PCB costs. The PLD solution will allow for the smallest possible PCB and hence, may be cheaper, quicker to build, and more easily modified.

For the moment I'd recommend you to focus on testing the various WOZ files with the existing hardware, and not start changing the hardware yet. There may be surprises during the testing and these may lead to hardware changes, and then, the optimization efforts on the hardware may be obsoleted. You have a working platform, use it for the tests, and keep a protocol what works and what does not work. Do not attempt to fix everything after one WOZ file fails ... get all tests done so you have a complete picture and then analyze the causation(s) and find the fixes. Much more efficient as you will see common patterns / groups of problems which need to be addressed in the same way (per group). Unless everything works with no such surprises ... but I doubt you are that lucky. This is a type of project which is difficult because you don't start from a clean slate where you can write all the specifications from scratch. This is the ideal case where everybody would expect turn-key success. But instead, you have the case where you venture into uncharted or poorly charted territory, all these vintage floppy disk protections, full of surprises, and you have to adapt to these unknowns.

Much the same situation I have with my IWM reverse engineering project. The IWM is the most poorly documented and most poorly understood custom IC in the whole Apple II world. Full of surprises. But it's out there, as a fact (or, better, "artifact") and so I have to adapt my work such that it reproduces all the bugs of the original IWM, too. Which is nasty and costs a lot of time (now 1.5 x over time budget), but it's inevitable to get a faithful reproduction. I now think that peeling off the mask layers and reconstructing the transistor level netlist may have been the more efficient approach. But hindsight is 20:20, as thay say. You have the same "black box" issue with the WOZ files.

- Uncle Bernie

VIBR wrote:

- in the woz spec, size of the track is given in the track chunk, do the emu has to adjust the circular buffer size according to this ?

No, the size must be equal to the time of one revolution , e.g.  about 200 mS or some 50000 bits for 5.25" media. If the data is less (which should always be the case) fill the remaining with zeroes or 3470 noise and just "play" the whole 200mS, e.g. your circular buffer.

- In woz 2, the size of the track is in bit and can be not a mutiple of 8, how should it be handle ? 

Overlaps with previous qestion -- fill the rest with zeros.

- When moving from one track to another, what is the best approach ? with the adjacent track I can record the current index, should I simulate a delay like the genuine disk II is doing ? 

No. 

In post #141, 'VIBR' asked:

"- in the woz spec, size of the track is given in the track chunk, do the emu has to adjust the circular buffer size according to this ?"

YES. Adjust the track buffer size to the number of bytes in the track (round up to full bytes). Adjust your SPI clock such that the track buffer 'revolves' in 200ms. This is the best way to be as close to a real floppy disk drive as possible.

"- In woz 2, the size of the track is in bit and can be not a mutiple of 8, how should it be handle ?"

Fill the last byte in the track buffer with zeros, as suggested by retro_devices in post #142. Be aware however, that this may have unwanted side effects unless the WOZ file generator stopped the track at a place where inserting the zeros has no unwanted side effect. This is guaranteed if the bit cell sequence just before the stop has generated a complete floppy disk byte in the DISK II shift register, which is defined by the MSB being set. Any sequence of zero bit cells after such a complete floppy disk byte has no effect on the byte stream "seen" by the software running on the Apple II, except for a small timing delay, which I think is too small to be noticed. If the WOZ file generator did not obey that rule, then adding zeros may cause problems and you would need to copy bit cells from the beginning of the track image to the end, until the condition is fulfilled, and then add the zeros, if needed to fill the last byte. Of course, then the wrap around of the track buffer also would need adjustment. I hope that the Applesauce team did it right ... so that the track always ends with the write splice, where there is no usable data anyways (professionally mastered floppy disks typically have only one write splice per track). In this case there is no concern, and there are no complications, the zero fill will work.

"- When moving from one track to another, what is the best approach ? with the adjacent track I can record the current index, should I simulate a delay like the genuine disk II is doing ?"

Do not simulate a delay (yet). It is a big complication that can cause more trouble for debugging than it's worth. But it may be necessary for some copy protections to work, which is just a worst case conjecture of mine. If this turns out to be necessary, you need to (crudely) emulate the inertia of the floppy disk head. This means when you detect a change of the stepper motor phases, you need to continue providing the RDDATA stream from the current track for a given number of bytes, and then switch to the RDDATA stream of the final track where the head would be positioned after all the stepping ceases. For "fat track" protections this should be trivial as the track buffer contents is identical. In other words, you continue to play the same track buffer, with no changes or interruptions of any kind, essentially ignoring the head movement. But for "spiral track" protections you need to move to the next (partial) track which will be different, so you need to change track buffers there. At the right moment. And this may be the tricky part ... you should always know where you are as you have a timer always running in lockstep (with the same "RPM", 200ms) as the emulated floppy disk turns. So a simple calculation from the timer will tell you the byte index within any track buffer, where you are. The tricky part is the inertia I mentioned (after how many bytes you change to the next track buffer, this should be a constant in all cases). The big question I can't answer is whether any existing "spiral track" copy protection did give stepper motor phase commands while continuing to read (and check) data bytes until they would fall apart due to the head actually moving. You can study this process (and measure the inertia) using any real, normally formatted floppy disk. Just read a track bytewise to know the sequence and then give a step command at a known point, and record the bytes coming out. The first few bytes should be identical to the known sequence but at some point there will be a difference / garbage. This is the number of bytes which still can be read due to the inertia of the head. Be warned that some Apple II emulator code I have seen does horribly complicated calculations involving ramp up and ramp down of the magnetic fields, calculates the forces, and then with the mass of the head assembly calculates head speed etc. I can't say if this extreme (attempted) fidelity of emulating the head positioning process is really necessary. Being lazy (= efficient) I'd avoid that unless proven to be absolutely necessary. This proof would be a) lots of copyprotections not working unless the head positioning process is so "accurate" and, b), the software title has no 4AM crack available, and c) the software title is an absolute must-be-emulated. I can't say if these three criteria apply to any known software title for the Apple II. In general, I am in favor of the KISS principle (keep it simple and stupid) and it is quite possible that those who coded the sophisticated physics model of DISK II head movement went too far. (I know these types, the same sort who build atomic clocks for the livingroom, which will not deviate more than a few seconds per human lifespan).

"I need to have a working reference like an online emulator where I can have trace & debug ? any suggestion ?"

There is no online emulator I know of able to do this. What you would need is an open source Apple II emulator which you can hack so it will produce trace files for each floppy disk byte coming out of the emulated DISK II controller (as read by the 6502). The Applesauce website recommends the EPPLE II emulator to support WOZ files, but I did not look into it. You can hack it to produce the trace files and then compare to what your floppy emulator does produce.

Don't get discouraged. Proceed step by step, and investigate the easier / earlier copy protections first. If nothing works, there most likely is a fundamental flaw in your code which affects all these titles in the same way.

Good luck !

- Uncle Bernie

Keep the bit period of 4uS and make the buffer equal to the largest track bit size, e.g 51008 bits. This will make "diskette" rotation period of about 204 mS which is well within physical drive's 2% tolerance. In fact Locksmith Speed test has an option to adjust the drive speed to "Recommended" which in practice is slower than 300rpm to be able to accomodate more bits.

If you run the Locksmith's disk Speed test with STM does it work like with a physical drive and diskette?

In post #145, 'retro_devices' wrote:

" Keep the bit period of 4uS and make the buffer equal to the largest track bit size, e.g 51008 bits. This will make "diskette" rotation period of about 204 mS which is well within physical drive's 2% tolerance. "

Uncle Bernie comments:

Seems we get into a situation where several 'cooks' spoil the soup by not agreeing about proper procedure. Which is the following:

- for all READ operations, the proper procedure is to adjust the SPI clock / bit cell rate such that one track takes exactly 200 ms ... as the real, ideally adjusted drive does. With more than the 'normal' amount of bytes in the track, you need a higher clock rate. This mimics exactly the situation where a copy protection used a mastering machine which could write higher bit cell densities. The floppy disk drive of the user ALWAYS would be assumed to run at 300 RPM. Unless you mimic this behavior by adjusting the bit cell clock rate, your floppy emulator will NOT emulate this situation properly.

- for WRITE operations involving longer than normal tracks (in terms of number of bytes on the track) you need to increase the track buffer size such that it can hold all the bytes needed. This is because unlike a floppy disk mastering machine, the Apple II can't change bit cell density - the clock of the DISK II controller is not programmable. LOCKSMITH could not write copy protection involving 'longer tracks' unless the RPMs were dialed down for the writing process. The same trick was used for the Atari copyprotections with 'long tracks' where the RPM had to be dialed down from 288 RPM to 270 RPM (-6.25 %), hard at the limit for later readback at 288 RPM.

The bottom line here is that there it no 'fixed' track buffer length and no 'fixed' READ mode bit cell frequency which could serve all situations. You could of course reserve space for the largest track buffer that could happen, but then program the DMA such that the 'empty' or 'unused' bytes in it are not presented to the Apple II during READ mode.

- Uncle Bernie

How fast? That is an indication of the problem with your firmware. Could you post a screenshot and tell wich range was selected.