Solving Ancient Riddles with Neural Networks

A page of the Voynich manuscript showing a number of plant like drawings and several paragraphs in an unknown language or encrypted text
Could the Voynich Manuscript just be early world building? Image in public domain.

I, like just about everyone who has ever heard of it, have been fascinated by the Voynich manuscript for years. The idea of an eldritch textbook, written in an encrypted script and with baffling, other worldly diagrams and drawings is ripe for all manner of conspiracy and conjecture. That it is over half a century old and has managed to survive relatively intact just fuels those fires.

Personally, though, I don’t subscribe to any of the extra-terrestrial, spiritual or religious interpretations for the book. Honestly, whilst any of those would be ground breaking and revolutionary to our understanding of the universe, they also dismiss something far more interesting. The Voynich manuscripts could be some of the earliest genre fiction on the planet! That idea is genuinely more exciting to me; to find definitive proof that people 600 years ago were just as happy inventing fictional worlds, in entirety, as I am today.

Of course, ancient fiction is well documented. We have plenty of examples  from much earlier in our history, but the Voynich manuscripts are somehow more interesting then the likes of Lucian of Samosata’s True History, at least in one particular aspect. They don’t tell the tale of some great mythology or legend, at least not one which is still known, and they appear to have been created by a single author, given how consistent the writing and art style is throughout the 200+ pages. To me, that would make this veritable tome less “just a story” and more a body of work akin to Tolkien’s Middle-Earth; early proof of actual world-building, for no reason other than fun. That we may have found a 12th Century Tolkien or Martin* is far more exciting to me, personally, then any of the more grandiose theories.

With that said, it may not be that long before the Voynich manuscript finally gives up some secrets. In one of the more interesting applications of neural-network AI I’ve seen, fellows at the University of Alberta have recently been targeting the text of the manuscript. The big issue with decoding the text isn’t just the encryption; with computers we should be able to make some headway on that front. However, you can only crack an encryption system if you have some idea of what the unencrypted message will look like. For that, we need to at least know the language which was initially encrypted. Of course, if the manuscript truly was created by a 12th Century Tolkien then it may have been written in a fictional tongue, making the whole exercise fruitless. Still, this is the very mystery that Greg Kondrak, with aforementioned AI in tow, may have managed to crack.

Having trained the AI on finding lingual patterns within text (not actually deriving meaning, but recognising the mathematical models that make up a language’s semantics and syntax) it was fed the Voynich manuscript. The result: Hebrew, with a high degree of certainty, appears to be the underlying language. That’s completely amazing to me. That a computer can teach itself enough about human linguistics that it can derive the language of a block of text based on glyph placement and frequency is astonishing, but that it can then use that same logic to decode gibberish into the underlying root language is mind blowing.

Of course, the result does come with some major caveats, the first being that this is still a best guess. The AI has found a pattern, that is certain; a pattern which closely matches archaic Hebrew. But until we can decode the text and find that it makes sense in Hebrew it doesn’t get us much closer. Here, Kondrak has also made some headway, believing that the text might be using a simple alphagram system. An alphagram takes a word and reorders the letters into alphabetical order, a fairly simple form of pseudo-encryption. For example, the word “example” would become “aeelmpx”; not instantly recognisable but not the hardest riddle to solve either.

Based on that hunch, the AI has run the text through a decryption algorithm and, again, seems to have hit pay dirt. A large amount of the output words are recognisably Hebrew, with more than 80% matching known Hebrew text. Unfortunately, the AI is no good at translating ancient Hebrew, and the few sentences they have tried don’t make much sense, but it’s a promising start. Hopefully, with the help of scholars more versed in the ancient language, some light may finally be shed on just what, exactly, the Voynich manuscript is or was. And that would be pretty darn awesome!

* Or, indeed, an ancient dungeon master!

The Marvel-ous Collection: A Beginning

I’m a pretty big fan of the Marvel Cinematic Universe, so it felt a bit ridiculous when I was given Guardians of the Galaxy: Volume 2 for Christmas. To be clear, the gift wasn’t ridiculous; it’s a fantastic film and one I’ve been excited to rewatch since seeing it in the cinema. The ridiculous part was that this officially marked the start of my Marvel Bluray collection. That’s right, I might be a huge fan of the franchise and own a fairly sizeable solid-media movie collection, but I’m almost entirely absent the MCU!

I say almost, because in truth I do own both Guardians of the Galaxy (now Volume 1, I guess) and Captain America: The Winter Soldier on DVD, but for a 17 film franchise (at time of writing) that’s pretty meagre. Part of that reason is the Bluray dilemma: ultimately, I don’t care that much about the increased resolution for most films, but I definitely care about the extra features. As Bluray has become the de facto release location for collector’s editions and special features, I was increasingly left behind, waiting for both an excuse to buy a Bluray player and then, later, for prices to drop back to the realms of sanity.

Luckily, 2017 saw both goals achieved. Whilst Blurays remain expensive (Marvel’s particularly so), they’re now at an acceptable premium above the respective DVD release, so with bonus featurettes, content and a better picture quality they feel somehow more worthwhile. At the same time, Marvel finally released a collected set for both Phase One and Phase Two, something I find bizarre has taken half a decade. I mean, what other purpose does the marking of “phases” serve then to artificially create film sets? At any rate, the result was a sudden galvanisation to fill in the blanks and finally own some of my favourite superhero films.

Unfortunately, a quick look at the contents of the collected sets left me a little cold. Yes, there are new bonus scenes, animatics and fun Agent Coulson introductions for each of the films, but they also lack a number of key special features from previous releases, especially the big documentaries. As a result, I’ve thrown in the towel! If Marvel/Disney can’t get their act together and release a definitive edition of the MCU then I’ll just create one myself.

The first hurdle was finding out what variations existed, what the actual differences were and then weighing up the pros and cons. Luckily, Reddit came to my aid (after Google summarily failed) with a raft of suggestions for comparison websites geared towards just this kind of task.

Since then, I’ve been slowly going through the films, one by one, narrowing down my options until I’ve found the exact version that most intrigues me. So far, the few I have settled on have been “out of print”, but luckily a robust second hand market appears to exist, keeping resell prices low. It’s slow going, but honestly I’m finding it quite fun. I’m also tracking my decisions and aim to release a full list, and break down of why I chose each film’s specific version, once I’m done.

For now, I figured it would be worth a quick round-up of the websites I’ve found most useful, so without further ado, and in no particular order, here are my top five film hunting locations:

1. DVD Double Dip
Not the prettiest site, nor the most complete in terms of information, but what it does have is extremely easy to read, compare and review. Probably the best starting point I’ve found but take the accuracy with a pinch of salt.

2. DVD Compare
Very accurate, particularly when it comes to extra features, and great for comparing regional differences in films. Take particular note of the “Cuts” and “Overall” sections at the bottom of a search page to see if the film is actively censored anywhere in the world. I wish you could compare films side-by-side, but still easily my favourite comparison site.

Probably the most complete database of film releases on this list but a bit of a pig to search accurately. There’s no way to easily compare film versions without opening multiple tabs, but you can filter by country directly on the search bar and the user reviews are solid, often clearing up any confusion over oddly phrased features.

4. Filmogs
Another very complete database without easy comparison methods. Easier to navigate than but the search is less intelligent (e.g. “Avengers” fails to pull back any collected sets). Again, useful for getting more information, plus acts as a competitively priced marketplace.

5. /r/DVDCollection
If all else fails, ask here and someone will probably either know the answer or own the film and be able to tell you. Really helpful bunch!

Of course, once you’ve narrowed down your options and decided which version is just right, you still need to buy the darn thing. Obviously if you’re looking at buying new then all the normal locations apply, but for second hand movies I’m having most success at the following:

1. Music Magpie – though be wary, several times I’ve spent a while looking at a film, come back later and found the price has shot up. Leave it a few days and it seems to drop back down again.
2. eBay
3. CEX
4. Amazon Marketplace

Happy hunting!

Capital Numbers

A List Apart has long been a fantastic source of knowledge and inspiration in terms of both website design and writing, but on top of these accolades every now an then it manage to completely floor me. Sometimes it’s because an explanation finally hits home after years of misunderstanding, but other times it’s simply by providing a piece of information which is simultaneously brand new and yet perfectly obvious. The type of fact which leaves you a little wide-eyed, questioning your very purpose of existence. A mind-blower, if you will.

That’s what happened today whilst reading the (brilliant) article/excerpt on Web Typography: Numerals. The article is well worth a read just for the thoughts on how the web finally makes footnotes genuinely useful (and, in doing so, guilt trips me about how these same ideas have been sat unacted upon in my head for years). However, it was a simple comment near the start that struck me like lightning:

We have at our disposal ‘uppercase’ numbers 0123456789 called lining or titling numerals, and ‘lowercase’ numerals 0123456789 called old-style or text numerals.

Wait… what?! Numbers can have cases? I read the sentence once, thought “that’s strange”, then read it again and noticed the different cases. These different glyphs are subtle but still instantly recognisable, yet I have never realised they existed before. Such a simple little thing which makes complete sense. Much like the recent viral expose of Papyrus or the infamous Fight Club burn marks, I get the feeling that text numerals are now going to be one of those things I just can’t not notice. Which is equal parts infuriating and awesome.

Mister Vimes’d Go Spare & Assorted Odds ‘n’ Ends [#23]

Well, back from trip number two, which was a little more relaxing (though a lot more tiring… I do not understand how bodies work). As a result, I’ve actually been reading a bunch of stuff, including some fascinating finds in my Pocket archive, which I just want to get off my chest.

First up is a pretty recent post from Brynn Metheney, a fantastic artist whose work I’ve followed for years. The post details a recent contribution to an interesting project, the Endangered Species Book. That’s an impressive list of artists to be working on a single project and it seems like a very worthy cause. Definitely one I’ll be keeping my eye on.

Next, are a combination of quite old posts that have taken me far too long to catch up on. Both are written by Richard Thornton, a friend of mine who is currently living/working out in Japan (I say currently, but he’s been out there for years now). The first is a brilliant look at sake culture, which was utterly alien to me but now has leap-frogged up my bucket list for the land of the rising sun. The second is a rather more personal account of shaving-procrastination (I can seriously relate) and snowboarding (I have zero life experience to understand this utter madness). Like everything Richard writes, they are funny, inciteful and make me equal parts jealous of his life and incredibly grateful for my own. Perhaps Japan should be the aim for 2018…

Finally, the oldest of the lot, is a short story I saved to my Pocket account so long ago I have zero recollection where it is from or how I found it. Mister Vimes’d Go Spare is an utterly fantastic piece of Discworld fan fiction; in fact, it’s so good that I was almost convinced it had been written by Pratchett himself. The script, phrasing and language is very witty and the overarching concept is so incredibly correct to the voice of the series that it is definitely part of my head-canon now. I almost added it to this month’s MiM, but I don’t feel fan-fic is something I need to keep track of in that way. If you’re a fan of the main series, you should definitely read this – it provides some clever closure on several key themes and characters.

That suggestion does come with a slight word of warning, however: it may get to you a little bit. Personally, reading Mister Vimes’d Go Spare made me realise I have been avoiding reading Pratchett since he passed away. It hasn’t been an intentional, conscious choice but it is clearly one I’ve stuck to. Reading a story that even mentions, and briefly touches on, several of these characters I love and hold so dearly was, at times, surprisingly hard. Not only that, but the core idea at work was, and remains, incredibly powerful. Vimes has always been one of my favourite characters and, I think, the one that has been most influential on my own personality and life. Part of that reason is the character’s understanding of and relationship with the concept of justice. It’s a very nuanced one, yet contains absolutes which have always appealed to me. Vimes and the Watch storylines shaped my own concepts of morality a great deal.

As a result, Mister Vimes’d Go Spare cut close to the bone. The central concept is that, in the wake of Vimes’ death, his ideals and belief in justice take on a life of their own. That shouldn’t be confused with ‘good’ or ‘right’; Vimes never lived in a ‘good’ world, never had much time for something just because it was ‘right’. But there are standards. Some things have to be done, and they have to be done in a certain way. That’s justice. Not making sure the good guys win and the bad guys lose, but making sure that the result is fair and that everything is equal. It’s a very powerful idea. Talking about why I enjoyed the short so much to my partner, even writing this now, and truly contemplating that idea gets to me. It gets to me because I believe it; because, to me at least, it is true. It also gets to me because it is one of those wonderful Pratchett ideologies that feels important and correct; something that is both worth remembering and striving to obtain in our world. And that gets to me because we won’t be getting any more of those. So be warned: it might get to you, too.

April Foolery 2017 [#15]


So another year, another April 1st. As is usually the case, the internet was awash with odd concepts, fake news and terrible puns. On top of which, we also had the annual Aprils Fools “pranks” (badum-dish).

Honestly, there weren’t many good jokes this year. The National Trust and Google Home Gnome teams put out some clever and well made efforts (I’m definitely a Duck man, too), but otherwise little even made me chuckle.

On the other hand, two April Fools events had me completely enthralled, not just throughout the 1st, but for several days afterwards. Reddit’s /r/place was incredibly fun to be a part of, regardless of your personal “style”. At first it just seemed a little gimmicky and, I won’t lie, I lost interest as the Colour Wars broke out, figuring that the internet hive mind had gone full blown /b/tard and wouldn’t be coming back. But then He-Man and /r/ainbowroad happened, ushering in a new era of creativity and collaboration on the Place which turned it into something both incredibly addictive and fascinating. The initial break out of the Void made me chuckle, and I must admit to even helping out initially (though it should be noted that I mistook their efforts for a Doctor Who homage). As their true nature became known, however, I forsake the black tiles and took up a roll as a caretaker for the next 48 hours. I was hooked, checking back in whenever I had a chance and found the ever-morphing canvas utterly enthralling. The final product is brilliant, a little slice of internet history that I genuinely feel somehow honoured to have been a part of. Even more interesting though are the various GIFs floating around in the days since, showing the more ridiculous sequences of time on the Place. Everything from Skeletor’s swift joint to the attempted overthrow of the US are just fantastic to watch, especially when sped up.

But the Place wasn’t all Reddit had to offer. As I watched (and helped) the logo for one of my favourite subreddits appear on the canvas, I realised I hadn’t checked out what they were doing. Despite ostensibly being a community for discussing Pokémon Go, /r/theSilphRoad has become a fantastic source of ideas. For past cultural events they’ve hosted Pokémon hunts (hidden within Reddit’s voting system, of all places), riddles and other fun surprises so I wasn’t too shocked when I saw that they had their own April Fools in full swing: Team Rocket had taken over the Silph Road! Obviously, the website had been reskinned with Rocket colours, logos etc., along with all the community moderators having their privileges receded and replaced with Rocket Grunts, but the ‘joke’ ran much, much deeper.

A post appeared early in the day saying that the Nest Atlas (a web tool the Silph team has built over the last few months for PoGo players) was the reason for Team Rocket’s intrusion and, sure enough, once people knew where to look they began to find clues. It turned out that the team behind the Road had been kidnapped and were being held hostage; better yet, one of the largest PoGo Youtuber’s (and a personal subscription), Nick from TrainerTips, had also been captured. The crossover of communities helped make the event extremely successful, as thousands of people around the world starting prising apart various websites, map elements and other clues in a fantastically well put-together hunt to track down the Silph team. Special locations and interactions had been built into the Nest Atlas, certain web files had been added to the stack with clues and there was even a Team Rocket ‘intranet’ that became accessible for a few hours.

On top of all that, the entire event (for calling it anything less would be insulting, frankly) led to a fully functional* mini-game hosted by theSilphRoad. Paying homage to the original Pokémon Go teaser trailer, the hunt eventually led to down-town Manhattan, where Team Rocket had captured “Silph HQ”. Entering the location on the Nest Atlas led to a selection screen to pick your squad and then an interactive battle with non-other than Mewtwo itself. The execution was lightweight (understandable, given the circumstances) but really fun and gave the community a single goal to work towards. Overall, an amazing experience that deserves serious applause.

*Until the TrainerTips’ fans descended and overwhelmed the servers; #BlameNick!

Writing with Pollution

Poster on a run down wall with a black circle approximately the size of a tennis ball and the text

The idea of Air-Ink is a very clever one, taking an approach to reducing pollution that I really like. Rather than using guilt trips or absolution, instead the team at Graviky Inc. have devised a way to make the carbon in car exhausts have a value. It may not be as ideal as simply removing car exhaust pollution, but I think this type of “green” thinking has a much higher change of succeeding. Give a waste product a value and you’ll find that scarcity will naturally increase. Definitely a project I’m glad to have discovered.

I am a little disappointed though with the execution. When I first saw the Kickstarter I misunderstood and thought they’d come up with a device you could attach to your car and use the output to syphon off ink. In reality they’re just selling the ink, having gathered it themselves and refined it at industrial scale. Don’t get me wrong, that’s still a great idea, but if they could somehow make the refining process integrated to the capture it would become an amazing idea. Can you imagine driving to work each day and refilling your ink well in the process? Perhaps some people would end up with excess ink, so the next step would be to make printers that could utilise it. Now that’s a truly win-win scenario that I would fund in an instant!

Martian Mirrors [#6]

Photograph of a wooden framed mirror with weather and calendar events projected onto the glass
Do I look Smart in this?

I absolutely love the execution and concept behind this “smart” mirror by Alex Repty (from MartianCraft). Sure, it’s clearly a labour of love rather than a commercial concept, but it’s also the kind of DIY tech project I aspire towards. Everything within the execution is custom made (even if the occasional part, such as the mirror, were purchased) so the end result can truly be called one-of-a-kind. Most impressive of all, to me at least, is that the software is also custom code whipped up to suit Alex’s particular needs. Whilst a “smart” mirror isn’t exactly top of my list of IoT devices, and the cost of the two-way mirror itself is a little off putting, I’d definitely like to try my hand at something similar in the future. Plus, with the whole process neatly written up for everyone to follow, it shouldn’t be too hard to reassemble the concept for my own needs. Top marks all round!

CMYK and the Magical Illusion of Colour

View post on

There’s isn’t much to add to the above GIF, other than that I think it’s a really cleverly designed and demonstrated concept. The idea of CMYK printing is nothing new to me; I’ve been interested in the physics used in forming imagery since I first noticed the tiny circles of colour (read: pixels) on a Coca-Cola vending machine image at school. Heck, I’ve worked for a newspaper company right next to the printing warehouse and currently program for a company that specialises in bulk printing!

That said, I still find the concept of combining varying levels of cyan, magenta, yellow and black to form a seemingly full-colour image a little magical. I’m clearly not the only one, too, and this was a little too well executed not to share.

PS: This is also the first time I’ve ever posted using a card. Having finally seen the process in the wild, this is also a little bit magical. I’m just a little concerned its magical towards the Dark Arts end of the spectrum.

It’s Been A While; Plus Thoughts on Pluses

So… it’s been a while.

It’s been a while since I last posted an article. Part of that has been due to a month of incredible busyness where even the planned “down time” became frantic research time for car insurance, holiday planning, present purchasing etc. The rest has been Pokémon Go!, which has eaten free time like nothing else in recent history.

It’s been a while since I reviewed a movie. Sort of. The reality is, I’ve actually managed to keep on top of those over the last week or so, but I have a couple annoyingly outstanding from a time BPg (Before Pokémon Go!) and I’m not 100% happy with the current state of some of the others. July’s MiM is coming, with some interesting new ‘features’, but it may be a week or so late.

It’s been a while since I spent any time working on this website. Again, in a time BPg (historians will catch on, I’m sure of it) I was getting close to making some pretty big, radical changes to the backend here. They hit a slight snag which morphed into a major roadblock simply because I still haven’t really sat down to mull it over. Still, plans are slowly creeping forward!

It’s been a while since I did any photo editing. Despite weekend after weekend of major events in the past month, which have produced hundreds of photos I’m genuinely proud of, I haven’t posted to Flickr in coughmumblemumble… I’m not proud of that, but again, plans may finally be moving forward.

It’s been a while since I saw something truly exemplary online. There have been some great videos and some new passions, but nothing that’s made me sit up and go: yes! I agree! Let’s think about/act on that right now! Luckily, this waiting period has actually ended thanks to the ever inspiring Vlogbrothers, John and Hank Green. I don’t want to go too deep into my thoughts right now because, well, this post has ballooned into something else which I quite like, but lets just say they’ve struck a nerve.

In a handful of recent Youtube videos, they both touched upon a worrying trend online, specifically that angry voices, ranting and outrage are becoming increasingly prevalent. In some ways, that’s totally okay, but the ubiquity and degree of rage is getting out of control. Between the two videos they discussed why anger leads to poor conversations, why it builds so much traction online, why that may provoke certain elements to create and foster this emotion above others and, most importantly, presented one method for potentially combating this trend. In terms of the “whys”, rather than regurgitate their words I’d urge you to just watch the videos. If the virality of anger is something that interests you, I’d also thoroughly recommend this analysis by CGPGrey.

What really made me sit up and take note, though, was their suggestion to combat saltiness and flame wars on Youtube: “+” comments. Because Youtube ranks comments based on the number of Likes and Comments they get, but comments are weighted higher than likes, angry/flamebait comments tend to rise to the top, causing a circle jerk of ever increasing rage. Hank Green, instead, suggested that people should leave a comment with a simple “+” symbol on any comments they felt worthy of praise, discussion or both. In doing so, they’ve hacked their own Youtube comments section into one where bile and trolling isn’t rewarded and genuine discussion/ideas are. It’s by no means perfect, but I thoroughly agree with both brothers when they say that it is as much the community’s job to police themselves and maintain order as it is the platform’s.

Personally, I’m a big fan of “+” comments. It may be that Youtube eventually begins to remove them or negate their importance, but in the meantime they seem to be a power for good. In particular, I feel they may be much more effective than reporting negative/trolling/abusive comments. I’m a firm believer that a carrot will be more likely to provoke change than a stick (plus, over reliance on the stick reduces it’s power/thorniness).

Drawing the UI for the IoT? There’s an App for that!

Forget UI design, let the user decide instead.

I absolutely love this concept from Marc Exposito which combines the best of IFTTT and augmented reality to allow you to quickly, simply and visually control Internet-of-Things enabled devices. I’ve yet to find any IoT gadgets that actually warrant a purchase (personally) but as the field expands and becomes more ubiquitous, apps such as Drawit will in turn become more and more powerful. These are the types of problems and subtle interactions that I can see the IoT, tablets, wearables etc. really embracing in the near future.