💾 Archived View for tilde.team › ~khuxkm › gemlog › video_to_gbc.gmi captured on 2023-06-16 at 19:48:07. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2021-11-30)
-=-=-=-=-=-=-
Published: 2020-11-22
Tags: khuxkm-codes
I was inspired by jfh's gemlog post of over a year ago, in which they converted video to ascii art and then into a GIF. I thought that looked pretty neat, and wanted to try my own spin on it. Hence, the GameBoy Camera-esque video generator becomes a thing!
So a while ago (March of this year, in fact), I wrote a simple webtoy that would let you take a photo. It would then take that photo and, by the power of JavaScript, attempt to mangle that image into something resembling a shot from a GameBoy Camera, circa 1998.
That webtoy, if you're interested (takes you to HTTP land!)
After reading jfh's post, my immediate next thought was "what if I could do this, but with my GameBoy Camera party trick?". So, a few days later (I had schoolwork to finish, okay?) I got to work coding it.
The original webtoy was written in JavaScript, which, while being perfectly fit for a silly little web toy I wrote in the span of an afternoon, isn't really fit for a terminal setup, let alone fit for use in an actual program. I took a picture of myself, sat down with the original source in one tab and my WebSSH terminal in another, and started working on porting the logic.
The original code, to handle some limitations of working within the web framework, looked a little like this (comments in asterisks are clarifications, outside of asterisks is what's actually in the code):
// the bread and butter of the app, processes picture:
function snap() {
// draw image, scaled to 128x112, to staging and retrieve ImageData
// *staging is a separate 128x112 canvas, used to ensure that nearest-neighbor interpolation is used instead of bicubic*
ctxs.drawImage(feed,0,0,128,112);
var imgData = ctxs.getImageData(0,0,128,112);
// each pixel is represented by 4 bytes (RGBA)
for (var i=0;i<imgData.data.length;i+=4) {
imgData.data[i+3]=255; // opaque
// emphasize green, lower red and blue, to get a reasonable grayscale.
let gsc = .2126*imgData.data[i]+.7152*imgData.data[i+1]+.0722*imgData.data[i+2];
// apply contrast and brightness controls
// *getContrast and getBrightness were JS functions that converted the contrast and brightness inputs into usable values*
gsc = getContrast()*(gsc-128)+128+getBrightness();
// clamp to closest GB palette color
// *this one doesn't need that much explanation, it's literally a function that finds the closest gray of the 4 shades the GB could produce*
gsc = findGBC(gsc);
// use that gray!
imgData.data[i]=gsc;
imgData.data[i+1]=gsc;
imgData.data[i+2]=gsc;
}
// put ImageData back into staging...
ctxs.putImageData(imgData,0,0);
// ...and draw staging on the main canvas, scaled back up
ctx.drawImage(staging,0,0,640,560);
}
An entirely separate canvas had to be used for staging? Are you for real? Luckily, Python should be able to handle this a bit easier.
...Unfortunately, using Python lead to even more confusing things.
My original implementation of the `findGBC` function from the original code (which I ended up renaming `closest_gameboy_shade`, since that was clearer), looked like this:
def closest_gameboy_shade(c): colors = [0,85,170,255] dist = 10000000000000 # sufficiently large number i = -1 for co in colors: if abs(co-c)<dist: dist=abs(co-c) i=co return co
If you can already see the problem, good. You should be able to see the problem, because it's so dumb that I cannot figure out for the life of me how I didn't notice it when I was writing it.
My output picture was all white, and after fixing a completely different problem, I finally realized what the issue was when I added some debug print statements and turned the contrast to 0%. I should return `i` from the function, not `co`. What was that other problem, you may ask? Well:
The GameBoy Camera took pictures at 128x112, so I resize the picture to 128x112. Unfortunately, this lead to my picture being squished. At first I couldn't figure out why this was, and then I came to a sudden realization:
My camera doesn't take pictures at an 8:7 aspect ratio.
So I wrote some code to figure out how to crop the image:
def get_width(height): return round((128/112)*height) # - later - im = Image.open(args.image) (width, height) = (im.width, im.height) new_width = get_width(height) left = (width/2)-(new_width/2) right = (width/2)+(new_width/2) imc = im.crop((left,0,right,height)) # now use the cropped image to resize to 128x112 imr = imc.resize((128,112))
And boom! The picture isn't squished anymore.
I wanted to start by running a picture through it, just to test and see what parameters I might want to give. After some tweaking, I decided on 140% contrast and a brightness of +16 luminance for this picture of my face. Obviously, for each video, you'd want to try and figure out what parameters would look best.
This part was simple. Use ffmpeg to convert an entire video into frames, then convert each frame to a GameBoy Camera-esque picture, then reassemble the frames into a video. The solution I ended up creating is really, *really* slow (a 3 minute video takes over 15 minutes to "render"), but it works.
I'm pretty proud of the end result.
