Solving Ancient Riddles with Neural Networks

A page of the Voynich manuscript showing a number of plant like drawings and several paragraphs in an unknown language or encrypted text
Could the Voynich Manuscript just be early world building? Image in public domain.

I, like just about everyone who has ever heard of it, have been fascinated by the Voynich manuscript for years. The idea of an eldritch textbook, written in an encrypted script and with baffling, other worldly diagrams and drawings is ripe for all manner of conspiracy and conjecture. That it is over half a century old and has managed to survive relatively intact just fuels those fires.

Personally, though, I don’t subscribe to any of the extra-terrestrial, spiritual or religious interpretations for the book. Honestly, whilst any of those would be ground breaking and revolutionary to our understanding of the universe, they also dismiss something far more interesting. The Voynich manuscripts could be some of the earliest genre fiction on the planet! That idea is genuinely more exciting to me; to find definitive proof that people 600 years ago were just as happy inventing fictional worlds, in entirety, as I am today.

Of course, ancient fiction is well documented. We have plenty of examples  from much earlier in our history, but the Voynich manuscripts are somehow more interesting then the likes of Lucian of Samosata’s True History, at least in one particular aspect. They don’t tell the tale of some great mythology or legend, at least not one which is still known, and they appear to have been created by a single author, given how consistent the writing and art style is throughout the 200+ pages. To me, that would make this veritable tome less “just a story” and more a body of work akin to Tolkien’s Middle-Earth; early proof of actual world-building, for no reason other than fun. That we may have found a 12th Century Tolkien or Martin* is far more exciting to me, personally, then any of the more grandiose theories.

With that said, it may not be that long before the Voynich manuscript finally gives up some secrets. In one of the more interesting applications of neural-network AI I’ve seen, fellows at the University of Alberta have recently been targeting the text of the manuscript. The big issue with decoding the text isn’t just the encryption; with computers we should be able to make some headway on that front. However, you can only crack an encryption system if you have some idea of what the unencrypted message will look like. For that, we need to at least know the language which was initially encrypted. Of course, if the manuscript truly was created by a 12th Century Tolkien then it may have been written in a fictional tongue, making the whole exercise fruitless. Still, this is the very mystery that Greg Kondrak, with aforementioned AI in tow, may have managed to crack.

Having trained the AI on finding lingual patterns within text (not actually deriving meaning, but recognising the mathematical models that make up a language’s semantics and syntax) it was fed the Voynich manuscript. The result: Hebrew, with a high degree of certainty, appears to be the underlying language. That’s completely amazing to me. That a computer can teach itself enough about human linguistics that it can derive the language of a block of text based on glyph placement and frequency is astonishing, but that it can then use that same logic to decode gibberish into the underlying root language is mind blowing.

Of course, the result does come with some major caveats, the first being that this is still a best guess. The AI has found a pattern, that is certain; a pattern which closely matches archaic Hebrew. But until we can decode the text and find that it makes sense in Hebrew it doesn’t get us much closer. Here, Kondrak has also made some headway, believing that the text might be using a simple alphagram system. An alphagram takes a word and reorders the letters into alphabetical order, a fairly simple form of pseudo-encryption. For example, the word “example” would become “aeelmpx”; not instantly recognisable but not the hardest riddle to solve either.

Based on that hunch, the AI has run the text through a decryption algorithm and, again, seems to have hit pay dirt. A large amount of the output words are recognisably Hebrew, with more than 80% matching known Hebrew text. Unfortunately, the AI is no good at translating ancient Hebrew, and the few sentences they have tried don’t make much sense, but it’s a promising start. Hopefully, with the help of scholars more versed in the ancient language, some light may finally be shed on just what, exactly, the Voynich manuscript is or was. And that would be pretty darn awesome!

* Or, indeed, an ancient dungeon master!

Life Between the Worlds [#37]

I have recently fallen back into an old habit: League of Legends. The eponymous MOBA remains immensely addictive, fun and interesting, but above all else my return (after over a year!) has highlighted that Riot are finally managing to get their world building in order. The lore behind Runeterra was always a big draw for me, leading me to pore over every new champion’s bio pages to find out how they fit into the world and whose stories they might impact. Over time, the original plot of League became a little stale and boring; champions that could simply be summoned from any region of the multiverse understandably felt disconnected from each other.

As a result, Riot made the decision a few years ago to begin reworking the story of Runeterra. Rather than completely overhauling everything, at great expense to time and resources, they have instead slowly been chipping away at the established characters. That leaves some, like my personal favourite Rammus, in a state of unknown origin, whilst others like Urgot have really begun to shine. It also means the in-game lore is a little disjointed, with some bios referencing events or characters that don’t add up, such as recent champion Ornn referencing Volibear as a demi-god, rather than the mortal leader his own bio describes him as. Overall, the effect can be a little confusing, but when it works well it produces some absolutely fantastic fantasy.

For example, in the past I’ve been incredibly interested by the setup of the Harrowing, an event which has it’s routes in Halloween but, over time, has become something far more sinister and interesting. Most importantly from a world building angle it helps to explain a number of the more demonic champions, giving them a shared and interlinked history whilst explaining how creatures of utter darkness aren’t simply ruling this world by now. It adds to the mythos wonderfully and remains the centre of some of the best in-game events they’ve had to date.

So, upon my latest return, I was excited to find another area of lore which has been fleshed out in a genuinely fascinating way. In an attempt to simultaneously develop how magic works within the game and explain numerous “chimeric” characters, the world-builders behind Runeterra have come up with the Vastaya. The full logic behind the decisions has been written up in a brilliant dev blog article, which is well worth a read if you’re interested in world building at all, but the outcome is genius. I love seeing entirely novel takes on something so integral to the genre as magic and, with the concept of the Vastaya and their ancient brethren, I genuinely believe Riot have achieved that.

There are a huge number of explanations for how magic works, yet most fantasy franchises just wave their hands or come up with something that seems like an explanation until you realise they just changed the word (cough Midichlorians cough). The route League has gone down is certainly not completely fleshed out; magic itself remains something ethereal and just naturally occurring rather than having a (necessarily) distinct source. I like their incorporation of ley lines, not because it’s unique or original (it isn’t) but because they have thought through the implications. I love that intersections of ley lines become areas of wilder magic, and that magic even has different breeds or flavours to begin with. That’s a nice touch which, as they state themselves, allows a huge amount of complexity to develop within the system.

Above all else though, the concept of the vastayashai’rei is genius. It’s one of those concepts which I read and instantly wished I had thought of myself. It’s wonderfully simple yet also feels very original (to be clear, I’m not saying it’s genuinely unique, but I’ve never seen it before). In Runeterra, magic is an extra-dimensional energy, bleeding through via ley lines, creating border zones: areas of world which are part magical dimension, part Runeterra. But the dimension in which magic originates is not just the standard swirling, lifeless maelstrom. It’s a functioning universe with it’s own ecosystems and, crucially, life. Whilst improbable, our own planet is proof that life thrives on these biological edges, in the types of habitat that just shouldn’t work. Look at any geothermal pool and you’ll see this effect in full swing. Right where the water reaches boiling point the lifeforms are unique, often occurring no where else on the planet.

When extrapolated out to a mixing of two entirely different dimensions you end up with creatures that have evolved to survive in both. Magical animals that can take physical form. I love it. I love the idea that a creature learnt that it could hop through the ley lines and find sustenance, or escape predators, by doing so. Over time, that developed into a fully functioning race of sentient creatures which could transgress the boundaries between the two worlds. Taking it one step further, the team at Riot realised that such creatures wouldn’t need a fixed physical form, as it wasn’t inherent to their nature. In short, they became shape shifters, creatures capable of adapting the forms they found themselves requiring within the physical world. Throw in a little bit of interbreeding or evolutionary branches that chose to remain on the physical side permanently and you explain chimeras, creatures with evolutionarily impossible physical forms. Sheer, pure, brilliance.

It’s nothing less than incredible that the reason behind this level of ingenuity is a game which lacks any form of story mode at all; there’s no need for any of these musings beyond making the world more entertaining. That’s pretty awesome, too!