One of our favourite pieces from the past 12 months. Originally published 22 July, 2022.
It takes a whole lot of GPU power to run today’s most powerful AI models. Even the most extreme gaming PC build (opens in new tab) wouldn’t be able to handle the kind of oomph needed to get the best AI Dungeon (opens in new tab) models up and running. In order for Latitude to power what’s essentially this less numbers-heavy, more free and creative online D&D campaign, you’d need something truly beastly.
According to Nick Walton, the CEO of Latitude and the software engineer who helped design the initial code for the D&D-inspired story generator (opens in new tab), “If you wanted to run the largest AI for AI dungeon, you would need not just a high-powered GPU, but a cluster of eight or 16 massive high-powered GPUs.” Even those would need to be more powerful than the best graphics cards (opens in new tab) on the consumer market.
Language models such as AI Dungeon’s Dragon experience, based on the 178 billion parameter Jurrasic-1 Jumbo language model developed by AI21 (opens in new tab), need a huge level of computing power to recognise and recall important story elements, and utilise complex writing styles.
Nick pointed me toward the DGX Station A100 (opens in new tab) as reference for the kind of machines Latitude uses to power the game’s more intense AI models (opens in new tab). It’s a computing monster even with just four of Nvidia’s $10,000 A100 GPUs—the first and essentially the most powerful Ampere graphics cards to surface.
AI Dungeon … is the most demanding game in terms of graphics card requirements
Nick Walton, Latitude CEO
“AI Dungeon, especially at launch and probably still even today, is the most demanding game in terms of graphics card requirements,” says Nick, “and for a long time it had no graphics.”
For machines like the DGX Station with, say, eight top-of-the-line A100 GPUs jammed in, you’re looking at around $150,000 in up front tech costs, and that’s just to run the most advanced AI alone—the company has several other AI models it needs to run simultaneously for your dungeoning pleasure. That’s one reason the company uses AI service providers, and even those have had a hard time getting hold of GPUs recently.
Obviously there’s a huge energy cost associated with running an AI business like this one, too; Nick tells me at one point the company was spending almost as much on running the machines as they were on payroll for the 16 person team (that’s including the higher-ups).
Adding to those energy costs is the game’s AI image tool (opens in new tab), which at the start of this week Latitude made the decision to roll out for free to all players. Nick was able to clear up how it worked for us.
He says a catalogue of tens of thousands of images is pre-generated, each one is captioned and then matched with stories as they’re being played, rather than them being generated on the fly. He makes it clear that generating images during gameplay isn’t really viable for an operation like this, but that the AI models behind the process are smart enough to know it can get away with matching an image captioned “Mountain forest with castle,” to a story in which a Medieval kingdom is mentioned.
Nick also laments that working with AI can be super challenging. Controlling what the AI says in order to stop it from going rogue (opens in new tab) is difficult, since “AI can sometimes be like an unruly toddler.” You can kind of influence it in the right direction but sometimes if you “give it instructions to not do something, you’ve now primed its brain, and it’s probably going to do that thing.”
Thankfully the team is working on optimisations for not only making their AI more efficient, but also for keeping it in check. That’s all going to become more important as the game rolls out on Steam (opens in new tab) next week, July 28.
Our Fraser believes that videogames need more Dungeon Masters (opens in new tab), and Nick is inclined to agree. In fact, it was his fascination with the freedom, and non-linearity of Dungeons and Dragons that lead him to create AI Dungeon.
Nick really exudes a passion for AI storytelling, believing that this kind of tech will change the storytelling landscape in a two major ways. First, “Ai is going to enable experiences that have freedom and dynamic-ness that has never been possible before.” He speaks of just how many people have completed those same quests in World of Warcraft, for example, and how it’s boring knowing other people are doing the same thing you are.
“I want to play an experience where I go to this small town, and I go through all these unique events that no one’s gone through before, and end up the hero of town,” Nick says. “Maybe no other player in the whole game cares about that town but I DO! It’s where I’ve had all these cool experiences that were unique and where I had the freedom to decide where the story goes.”
And it’s not just a unique experience that AI storytelling offers. “The second thing I think is really powerful,” says Nick, “is AI’s ability to enable an entirely new, massive generation of creators … enabling every person to be a creative director of their own experience.” Nick is convinced creators will one day be able to orchestrate their creative visions, bringing them to life through the power of AI.
Honestly, that’s a much nicer take on the future of AI than most people have, so I appreciate it. It’s exciting to watch companies like Latitude expand and evolve along with something I’m so fiercely passionate about: emergent storytelling. And I’m sure the Steam launch is going to be a success, so get AI Dungeon on your wishlist because boy are you in for a ride.