60fps Video on a CGA? - The GlyphBlaster

GlyphBlaster

The GlyphBlaster in action - is that a cat in text mode??

 There's a certain genre of retro-hardware hack that everyone recognizes is technically cheating, but has a certain audaciousness to it that merits respect regardless. I hope that this one falls into that category!

A few years ago, I was impressed by TheRasteri's PiPU project - basically, a Raspberry Pi Pico-powered mapper for the NES. Of course, it can play DOOM:

The Raspberry Pi Pico has reinvigorated the retro-computing hobby quite a bit, enabling such devices as the PicoGUS, PicoMEM, PicoIDE, and most recently, the OneROM.  It was the latter device that got me thinking.

I recently used a OneROM to replace the font ROM on my CGA card, and subsequently I was able to replace IBM's classic 8x8 font with a custom one. Here's Frogblock by Polyducks:

The "Frogblock" font loaded onto a OneROM replacing the CGA's font ROM

The programmability of the OneROM intrigued me - especially its plugin system.  But I wanted to understand more about the Pico in general, and learn how to program its powerful PIO mode.  After successfully completing one Pico-based project, my IBM PCjr wireless keyboard to USB adapter, I decided my second project would be a CGA font ROM replacement.

My idea, in short, is to use the font ROM readout as a 1bpp, pixel-addressable framebuffer. 

In text mode, the IBM CGA card reads the font ROM on every character clock (in 40 column mode, it reads it twice...). This gives us a regular character clock, based on the falling edge of the ROM socket's /CE pin. We just need a single external synchronization signal - VSYNC, in this case.

Another of my goals for this project was to leave the CGA card otherwise unmodified, so I used a test clip to attach to the the exposed VSYNC pin at the back of the CGA's DE9 connector. My new Dupont crimper came in handy here.

The idea is to start a new frame at the end of a VSYNC pulse, with a PIO program waiting on the /CE pin.  It will then read bytes from a FIFO fed from our framebuffer via DMA.  We can fill this buffer with static images, or even video - in fact, since the Pico 2 W has Wi-Fi, one of my goals was to stream 1bpp video wirelessly to the CGA.

I needed a good name for this though.  I originally started calling it the PicoCGA, but that carried the implication it was an entire video card instead of an add-on for one.  I have to thank wbcbz7 on the Retro Pico Hardware discord for suggesting the name 'GlyphBlaster'.

Hacking up the Prototype

First things first, I needed some kind of adapter to fit the Pico into the ROM socket. I had some PCBs left over from my bus sniffing project that were designed to fit into an 8087 socket. Luckily, the DIP-40 and this ROM's DIP-24 socket have the same width, so this PCB was compatible. 

The initial GlyphBlaster prototype installed on a CGA card

The initial prototype ignores address lines entirely, just patching in the 8 data lines and the /CE chip enable line. The green wire is our VSYNC.  

You may be familiar with the CGA's more typically stated resolution of 640x200; this gives us an 8x8 font with 80 columns and 25 rows. But accounting for the entire NTSC field size, including front porch, back porch, and blanking periods, we arrive at an effective resolution of 912x262. 

For simplicity, I preformatted an image as a 912x262 bitmap. It was the classic bulb of the Mandelbrot Set fractal:

The 912x262 formatted Mandelbrot bitmap

After some trial and error, we were able to see this on the screen:

The Mandelbrot bitmap being drawn on a monitor through the CGA font ROM socket!

The colored bars are foreground attributes left over from booting GLaBIOS. Pardon the messy workbench in the reflection!

Streaming Video

With the basic prototype working, it was time to try to stream Bad Apple to it, in the spirit of 8088 Domination, of course. The initial prototype was written in Rust, using rp-hal.  To more easily take advantage of the Pico 2 W's Wi-Fi module, I decided to switch to Embassy, which has a nice task-based async executor. I found Embassy very easy to get up and running - the initial switch was just changing a few lines in my Cargo.toml and adding a macro declaration on my main function.

I have a solid networking background, but I had never designed a video streaming protocol before.  I decided to start very simple - UDP packets containing just a short header and then the 1bpp frame data.  We encode the frame number the packet belongs to, the starting and ending scanline the packet represents, a one-byte compression method (0 for uncompressed, initially), then finally the scanline data.  

Since we are sending the full display field of 912 hdots, each scanline is 114 bytes. I needed to keep under a 1500 byte MTU to avoid fragmentation, so I kept things simple and packed 10 scanlines into a packet.

UDP over Wi-Fi in an dense RF environment like a high-rise apartment building is somewhat unreliable to begin with, and fact that the Pico has no external antenna support does not help matters. I ended up purchasing a small travel router that sits right next to my IBM 5150 in order to get a reliable signal we can stream video over.

Converting the Video

Despite Bad Apple's notoriety as a 1bpp display demo on just about every platform conceivable, it is not actually 1bpp video - the character outlines are anti-aliased and there are several scenes with grayscale shades. 

Additionally, the CGA card's display resolution of 640x200 is severely compressed horizontally, so we need to squish it down. Using 40 column text mode allows a more squarish 320x200, but I decided playing video in 80 column mode was funnier.

The first step was to dump all the frames from the source video into a directory:

ffmpeg -i touhou bad apple smile.mp4 ./output_frames/frame_%06d.png

Then, using ImageMagick, we resized the frames down to 640x200, and converted them to 1bpp using an ordered dither:

magick mogrify -path ./resized_frames -resize 640x200! -colorspace Gray -ordered-dither o8x8 -type bilevel -depth 1 ./output_frames/*

Although my streaming application would later handle this without re-encoding, I then padded each image out to 912x262:

magick mogrify -path ./final_frames -format png -background black -gravity center -extent 912x262 ./resized_frames/*

This produces a frame formatted for the CGA's 912x262 native display field timings:

A final 912x262 Bad Apple frame

Transmitting the Video

At 912x262x1bpp, each frame of uncompressed video is about 30KB. At Bad Apple's 25fps, this is 5.97Mbps, which is not an insignificant amount of data to send to a microcontroller.  So I implemented a very simple RLE scheme, which yielded an average 75% compression rate on our 10-scanline packets.

To avoid dumping a whole frame's worth of UDP packets on the Pico all at once, the client streamer application paces out the UDP packets in a rough estimate of when they'd be needed to decode a frame.  Since each packet is tagged with a starting scanline and ending scanline, they can be received out-of-order up until the point a packet for a new frame is received and the Pico page-flips. The Pico starts each frame with a copy of the last frame, so any missed packets cause minor video glitches at worst.

How well does it work? Pretty well!

The main issue at the moment is that having the GlyphBlaster installed makes the system unusable, since we can't see anything on the screen in text mode that the GlyphBlaster isn't drawing.  We need a way for the GlyphBlaster to emulate the font ROM, or the lazy route, which would be to pass through those duties to the existing font ROM.

ROM Passthrough

To enable ROM passthrough, I soldered on  a mid-air ROM socket.  Even as a prototype, this was pretty sketchy.

A ROM socket added in mid-air with a ton of bodge wires. Spot the disconnected pin!

This stubbornly refused to work until I realized something important - always read the data sheets - this series of Mostek ROM has an internal address latch triggered by the falling edge of /CE.  You cannot simply pull /CE low.  With the PIO code adjusted to trigger the ROM's /CE pin, we were able to read the existing ROM contents, with the address lines passed directly to the ROM.

There's just one problem - we want to combine the Pico's framebuffer with the ROM data, so that the graphics appear as an overlay on screen.  Unfortunately, PIO, although powerful, lacks any sort of ALU or bitwise boolean operations, so you cannot say, XOR the contents of the ROM with our framebuffer data.  You can, however OR it, by using the inverted ROM data as the value written to the Pico's PIO pindirs register.  This means that bits that are 1 returned from the ROM will cause the Pico's output lines to float - and the Picos' internal pull-ups are enough to pull those values up to 1. 

Now the system is usable, and we have the option of displaying graphics over text mode, which is otherwise impossible to do on a CGA:

Displaying graphics OR'd with text mode by abusing the Pico's internal pull-ups.

I implemented a number of toys that don't require any Wi-Fi connectivity. First up, I thought it would be funny to replicate the classic bouncing DVD logo.

The DVD logo bouncing around a CGA text mode screen at 60fps

And then I couldn't resist taking a playful jab at the Amiga:

What would be neat is if we could actually interact with the display in some fashion.  There's currently no way for the host to communicate with the GlyphBlaster, or vice versa.  However, there are some useful signals we could tap into that might be fun. One candidate is the CURSOR pin of the Motorola MC6845.  This pin goes high when the cursor should be output in a particular character cell. 

It's somewhat simple to save the current DMA address via a PIO program after triggering on the rising edge of the CURSOR signal, and work out the position on screen from there. 

Many people fondly recall a desktop toy called Neko, that involved a cat following the mouse cursor around and other tricks.  I decided to make a PC version as an homage to the famous PC Booter game AlleyCat. 

I used my emulator, MartyPC, to dump various sprites from the game using my memory visualization tool. 

The cat will follow the text-mode cursor around, sitting down and entering an idle animation where he turns his head, blinks his eyes, and wiggles his tail. When the cursor moves to a new location and stays still for a moment, the cat will then proceed to the new cursor location, chasing after it if it happens to move during the chase.

Here's a video demonstration:

There are number of possible sync sources for the GlyphBlaster to use.  Another promising one is the light pen strobe directly from a light pen - this allows the cat to chase the pen around the screen as well:

The cat chasing the light pen around the screen

There are some intriguing possibilities besides being a novelty toy for your retro computer.  If we tie in the address lines to the Pico, the Pico can read the contents of the text-mode screen - this could make it a very useful, driver and OS independent screen-reading device. 

I've designed what is surely to be the first of several revisions of the GlyphBlaster's production PCB - although I don't know if I have any plans to sell this as a commercial product. 

I've already determined that I don't need the passthrough socket to the ROM at all - but of course, you inevitably realize these things a few days after you've submitted your PCB order.  With a single added frame of latency, the Pico can emulate the font ROM entirely, which opens up the possibility of unlimited software fonts and font animation as well.

The GlyphBlaster PCB rendered in KiCad's 3D View

What started out as something I intended to be a weekend-long hack has turned into something that may actually have some value as a general Pico-powered add-on. I doubt it will become as essential as say, the PicoGUS, but we'll just have to see where it goes.

I will update this blog when I have more news about the GlyphBlaster. If you're interested in the most recent updates, dev-blogging, and other random retro chaos, follow me on Mastodon.