Project summaries for completed and in-progress projects below. Enjoy!
- Framerate Analysis Tooling - Apr 2022
- HP Thin Client DOS Gaming PC - Feb 2021
- DOS Gaming on MiSTer FPGA - Feb 2021
- Remote Play Xbox Series S/X Games on PC - Jan 2021
- Lenovo Tiny Series Emulation Station - May 2020
- Porting Commander Keen to PlayStation Vita - In Progress
- Porting Commander Keen to WebAssembly - Nov 2019
- Learning About Webassembly - Nov 2019
- Raspberry Pi Retrogaming - Oct 2018
- Atari ST on the Web - Jun 2018
- Shooting and Sharing VR Photos - Nov 2016
- Transferring Files Between PC and ST - Mar 2016
- Playing Downloaded Games on Real ST Hardware - Sep 2015
- Watch Mobile Content on the Go - Dec 2014
- Ghost Blogging - Jun 2014
Framerate Analysis Tooling
|Goal||Build my own “Digital Foundry” style tooling to analyze videogame performance|
- Write my own performance analysis tooling. Look at captured videogame footage and analyze framerate, torn frames, resolution etc
- Run some real performance comparisons (e.g. Xbox Series X vs PS5 performance)
- Technical learning! Build some proficiency in videogame technical analysis and machine vision
After some background research, I selected OpenCV as the foundational project library. As a specialised framework for computer vision and image analysis, it’s a great project fit.
First, I worked on difference detection between video frames. If two frames show zero difference, that means we have a duplicate frame (i.e. a frame drop vs the target frame rate).
With diff detection in place, I was able to do some rudimentary framerate analysis on The Legend of Zelda: Breath of the Wild and (after some more iteration) add performance monitoring and graphing.
Next, I investigated detection of torn frames. This is when the renderer doesn’t have enough time to complete an entire frame, so a partial frame is rendered to the screen instead (with the remaining row gaps filled in with data from the last frame). I enhanced diff detection and was able to reliably detect torn frames in Gran Turismo 5 (a game known for significant image tearing).
My next focus was resolution counting. Videogames often dynamically reduce rendering solution to keep frames “within budget” (i.e. rendered fast enough to hit the desired 30fps or 60fps target). This is known as Dynamic Resolution Scaling (DRS). The frame is then upscaled back to the target resolution (e.g. 1080p or 4K). It’s an effective approach, but reduces image quality vs “native resolution” rendering. Artifacts (such as jagged edges) appear in the image, and it’s possible to exploit these to identify the original render resolution of the frame.
As a first pass, I was able to find all edges in each frame. I was then able find “countable edges” - straight line edges that can be pixel counted to identify the original render resolution of the frame.
Unfortunately the automation didn’t work. Despite hours or tuning, the edge detection algorithm wasn’t able to distinguish between polygon edges (which are suitable for counting) and straight line edges found within filtered textures (which aren’t). The counts are unreliable, so I’ve parked this for now.
Next, I looked at framerate counting for compressed video. Video compression is typically lossy and introduces image artifacts e.g. pixelation, macroblocking etc. These artifacts make it very challenging to detect duplicate frames. 100% identical frames rendered by the game engine will appear different in compressed video.
In short, compressed video broke my diff detection. Every single frame is perceived as unique - no frame drops detected. To address this, I needed to adopt a more statistical approach. Instead of looking for zero difference between frames, I looked at difference trend:
- Hypothesis: because video frames from a 3D engine are temporally connected, we would usually expect frame difference trend to be relatively consistent between frames. Significant trend divergence is aberration - and might indicates a frame drop
To test the hypothesis, I created a time series of frame-to-frame difference (Euclidean difference) for compressed video with known frame drops:
See those downspikes in the graph above? Those are the frame drops. And they’re easy to identify. This is great news!
A few updates later, I was able to complete framerate analysis for the Starfield Official Gameplay Reveal.
Finally, I looked at high quality video export from OpenCV. Unfortunately the built-in OpenCV VideoWriter is not super configurable - and video output is relatively low bitrate and blocky. Instead I chose to output results as PNG and then re-assemble into video using ffmpeg.
- Sampling framerate from captured gameplay video
- Identifying duplicate frames and torn frames
- Graphics and performance graphing in OpenCV
- How to accommodate compression noise in captured video
- High quality video export from OpenCV
- Side-by-side comparison of 2 (or 3) videos
- Resolution counting revisited
You can also find the journey chronicled on Twitter here.
HP Thin Client DOS Gaming PC
|Status||Completed July 2021|
|Goal||Build a small and light gaming PC for DOS and Windows 98 gaming|
After DOS gaming on MiSTer, I was left wanting for more. The MiSTer felt underpowered, I wanted something a little faster to play Half-Life, Unreal and other Windows 98 classics.
In search of a retro PC (but short on apartment space), I purchased a HP Compaq t5710 thin client.
Despite the 2004 build, the hardware is very well suited to DOS and Windows 98 gaming. Sound Blaster and Ad-Lib audio works even in DOS mode, making this a great all-in-one unit.
I’m super happy with this build. The t5710 is now my go to retro PC – I don’t need anything more!
- HP Compaq t5710 Review – Great for DOS and Windows 98 Gaming? - 11 Jul 2021
- HP Compaq t5710 – How To Install Windows 98 from USB Flash Drive with Easy2Boot - 11 Jul 2021
- HP Compaq t5710 – Using a PS/2 Splitter Cable - 13 Nov 2021
DOS Gaming on MiSTer FPGA
|Status||Completed March 2021|
|Goal||Play DOS and Windows 95 games on the MiSTer FPGA platform|
First on the list: the 80486 PC. I’ve many fond memories of DOS and Windows 95 gaming, messing with IRQ settings, tuning
autoexec.bat to get things working “just so”. Luckily for me, DOS gaming is very well served on MiSTer via the ao486 core.
After a lot of tinkering, I was able to setup file sharing, install a Memory Manager, configure the Sound Blaster, add CD-ROM and Mouse support, configure Roland MIDI and install Windows 95. A great result!
- MiSTer AO486 Core Part 1 – Getting Started - 06 Feb 2021
- MiSTer AO486 Core Part 2 – Sharing Files With MiSTerFS - 14 Feb 2021
- MiSTer AO486 Core Part 3 – Managing Memory - 13 Mar 2021
- MiSTer AO486 Core Part 4 – Sound and Music Setup - 13 Mar 2021
- MiSTer AO486 Core Part 5 – Adding CD-ROM Support - 13 Mar 2021
- MiSTer AO486 Core Part 6 – Mouse Support - 13 Mar 2021
- MiSTer AO486 Core Part 7 – Quick Start DOS Image - 18 Mar 2021
- MiSTer AO486 Core Part 8 – Roland MT-32, Sound Canvas Audio via MidiLink - 28 Mar 2021
- MiSTer AO486 Core Part 9 – Installing Windows 95 - 28 Mar 2021
Remote Play Xbox Series S/X Games on PC
|Status||Completed January 2021|
|Goal||Stream / Remote Play Xbox Series S/X games from my Windows PC|
With the Xbox One, I often used the Xbox Console Companion app to remote play games on PC. Frustratingly, this feature was silently dropped for Xbox Series S/X – the Console Companion app just doesn’t work for the new consoles.
Looking for a solution, I was able to install a hidden “Xbox Game Streaming (Test App)” from the Windows 10 Store, and start streaming again. Full details here. Case closed!
- How To Remote Play Xbox Series S/X Games On Your PC - 24 Jan 2021
Lenovo Tiny Series Emulation Station
|Status||Completed May 2020|
|Goal||Repurpose a Lenovo Tiny Series business desktop as a small and quiet retrogaming PC|
For many years I’ve been on quest to find the perfect emulation station. Something small and quiet, hidden under the TV, with a wireless controller and enough horsepower for high quality retrogaming.
Two years ago I built and custom-configured a Raspberry Pi with RetroPie. A great device, but not really powerful enough for 32-bit consoles like PlayStation or Saturn. I need something with a bit more grunt, reasonable price (~$100) and good Wife-Acceptance-Factor (no tower PC!)
After some investigation, I purchased a Lenovo ThinkCenter M93p Tiny Desktop from eBay for ~$100.
The form factor looks great under a TV and emulation performance is also great. Even Dreamcast and Gamecube run well, a significant step up from the Raspberry Pi. Full write up here.
Porting Commander Keen to PlayStation Vita
|Goal||Bring the DOS classic Commander Keen to PlayStation Vita.|
I’ve wanted to explore cross-compilation (i.e. building code on one platform for execution on another) for a while. I’m also passionate about bringing classic retro games to a new audience.
With the advent of h-encore, pretty much any PlayStation Vita can now be unlocked to run homebrew code. There’s also the Vita SDK development framework available to help create working binaries. So the Vita seems like a great target platform.
I’m aiming to bring Commander Keen (in the form of CloneKeen) to the Vita. The source code is C/C++ and uses the multiplatform SDL development library, so should be relatively portable. I’m also looking for improvement opportunities over the orignal release (e.g. level editor, increased resolution).
So far I have the game engine running, but need to add controller support.
- Game engine successfully running on a real Vita - 10 Nov 2019
Porting Commander Keen to WebAssembly
|Status||Completed November 2019|
|Goal||Port Commander Keen to WebAssembly, making it playable in the Browser|
I’ve wanted to find a creative outlet for my recently-acquired WebAssembly skills. I’m also passionate about bringing classic retro games to a new audience.
I used Emscripten to port Chocolate Keen to WebAssembly. The initial goal was to refactor the entire code base to Emscripten Loops but this proved too much effort. In the end I found a working (but slower) solution using Emterpreter.
In future, I could refactor towards Emscripten Loops to improve performance and add on-screen controls for mobile devices.
- Porting Commander Keen / CloneKeen to the web browser with #Emscripten. Great progress so far! - 20 Oct 2019
- Commander Keen in a Web Browser? - 28 Oct 2019
- The Commander Keen WebAssembly port is done! - 02 Nov 2019
Learning About WebAssembly
|Status||Completed November 2019|
|Goal||Learn about WebAssembly. Understand how to make C/C++ code run in the Browser. Code some “Hello World” examples and document my findings|
I’ve wanted to explore the WebAssembly ecosystem for a while. I learn best by sharing what I learn, so I’ve written up my findings as WebAssembly Lessons:
- Lesson 1: WebAssembly Hello World - 30 Nov 2019
- Lesson 2: Graphics with SDL - 01 Dec 2019
- Lesson 3: Emscripten Loops - 03 Dec 2019
- Lesson 4: File System Access - 08 Dec 2019
Raspberry Pi Retrogaming
|Status||Completed October 2018|
|Goal||Build a cheap, small and light device to retrogame on the big screen|
I selected the Raspberry Pi and RetroPie as the base for my build. After a week of on/off tweaking, I found optimal settings and documented the full setup here. For Phase 2, I configured Moonlight Game Streaming to allow low latency streaming of more demanding titles from my gaming PC.
Overall very pleased with the result, and pleasantly surprised by how well Moonlight Game Streaming works.
- RetroPie Ultimate Setup Guide - 08 Oct 2018
- Stream PC Games to your RetroPie with GameStream and Moonlight - 08 Oct 2018
Atari ST on the Web
|Status||Completed June 2018|
|Goal||Connect my venerable Atari ST to the World Wide Web. Browse some sites!|
The Atari ST was released in 1985. Is it usable on the modern Web today? How far can I push the experience?
Starting with a NetUSBee Ethernet adapter, I installed TOS drivers and the STinG TCP/IP stack. I also used an ET4000 graphics card to push the resolution to a (somewhat) modern 1024x768 in 256 colours.
With those (and a successful TCP/IP ping) in place, I downloaded and configured Crystal Atari Browser (CAB).
The Web experience is slow and doesn’t support CSS - which rules out a lot of content. However I was able to make older sites work.
In search of a better experience: it might be possible to use a proxy server to strip CSS and shrink images. Or use a text-based browser like Lynx. Browsh also looks interesting. Lots of avenues for future investigation!
Shooting and Sharing VR Photos
|Status||Completed November 2016|
|Goal||Learn how to shoot, interactively view and share VR photos|
More formally, VR photography (Virtual Reality photography) is the art of capturing or creating a complete scene as a single image, as viewed when rotating about a single central position
I want to learn how to shoot, interactively view and share my own VR photos. Here’s what I found:
- Equirectangular Projection is the most common VR image format. Another popular option is Cubic Projection and you can easily convert between the two
- The Google Street View mobile app (iOS, Android) is a cheap and easy way to shoot Equirectangular projections
- Since Equirectangular images are heavily distorted, you need a “viewer” program to interact with them
- Windows has built-in support via the .pano file extension. It’s relatively easy to create your own
- Web viewers are also popular. Here’s how to publish VR photos to Photosyth, Facebook and YouTube
- Publishing VR Panoramas to Facebook - 12 Oct 2016
- Installing Hugin and Panotools on Windows - 17 Oct 2016
- Convert Equirectangular Projection to Cube Faces - 18 Oct 2016
- Publishing VR Panoramas to Photosynth - 28 Oct 2016
- What are .pano files? - 29 Oct 2016
- Convert Equirectangular Projection to .Pano File - 30 Oct 2016
- All You Need to Know about 360 VR Photos - 02 Nov 2016
- Publishing VR Panoramas to YouTube - 08 Sep 2019
Transferring Files Between PC and ST
|Status||Completed March 2016|
|Goal||Explore the various ways to share files between PC and Atari ST|
Can my Atari ST interoperate with modern PCs? What are the best ways to send files back and forth?
I explored several ways to transfer files between PC and Atari ST, including:
- Using an SD card and Ultrasatan to Transfer Files from PC to ST - 08 Mar 2016
- Using Ethernet to Transfer Files from PC to ST - 07 Mar 2016
- Using PARCP-USB to Transfer Files from PC to ST - 24 Feb 2016
- Using Serial Cable and ZMODEM to Transfer Files from PC to ST - 13 Feb 2016
- Using Serial Cable and Ghostlink to Transfer Files from PC to ST - 06 Feb 2016
- Using Floppy Disk to Transfer Files from PC to ST - 30 Jan 2016
I also took a detour into file compression, and learned how to split large files into chunks that can be individually copied using (for example) floppy disks:
Playing Downloaded Games on Real ST Hardware
|Status||Completed September 2015|
|Goal||Download an Atari ST game image from the web and play it on my real ST|
My old ST disks are dead! Can I download replacement disk images from the web and play on my physical ST?
I explored several options, including floppy disk images and hard disk adaptations:
- Playing Downloaded Games on a Real Atari ST - 18 Sep 2015
- Use Your PC to Create a Bootable Atari ST Game Disk - 22 Sep 2015
Watch Mobile Content on the Go
|Status||Completed December 2014|
|Goal||Download video content to my mobile device so I can watch it on the go|
Update Dec 2019: YouTube has emerged as the dominant platform for Web video. In future, it makes sense to look at the YouTube API. Can I automatically download every video tagged to the “Watch Later” playlist for offline consumption?
The work commute is a great time to catch up on news, vlogs and other Web videos. But cellphone signal is way too patchy for reliable streaming - especially in the subway.
I explored use of the Pocket API and PlexSync to download and push web videos to my phone:
- I Can’t Stream Video on the Commute. How Can I Fix It? - 17 Dec 2014
- Download Pocket Videos to Your PC - 03 Jan 2015
- Sync Web Videos to Your Tablet or Phone, Watch Them Offline - 01 Feb 2015
|Status||Completed June 2014|
|Goal||Learn how to use the Ghost blogging platform|
I’m interested in Ghost as a crisp, clean, lightweight blogging platform. I took some time to learn about the platform, setup my own developmemt environment and even write my own theme. Findings below.
- Getting to Grips With Ghost Themes - 24 Apr 2014
- Setting Up a Ghost Dev Environment - 17 May 2014
- Create Your Own Ghost Theme in 5 Minutes - 10 Jun 2014
- It’s Done! 8-Bit Memories Ghost Theme - 21 Jun 2014
- Things I Wish I’d Known Before Creating My Own Ghost Theme - 23 Jun 2014