The video above will be familiar to anyone who’s played the 3D world-building game Minecraft. But it’s not a human constructing these castles, trees, and caterpillars—it’s artificial intelligence.
The algorithm takes its cue from the “Game of Life,” a so-called cellular automaton. There, squares in a grid turn black or white over a series of timesteps based on how many of their neighbors are black or white. The program mimics biological development, in which cells in an embryo behave according to cues in their local environment.
Some researchers have replaced the simple rules (e.g., any white square with three black neighbors turns black) with more complex ones decided by neural networks, machine-learning algorithms that roughly mimic the brain’s wiring. These are called “neural cellular automata.” But the grid is still only in two dimensions, or in three with only one kind of building block.
In a paper posted on the preprint server arXiv this month, researchers presented a system that uses neural cellular automata in 3D, and with 50 kinds of blocks, including some that act like pistons. Then they unleashed their system in Minecraft.
The scientists taught neural networks to grow single cubes into complex designs containing thousands of bricks, like the castle or tree or furnished apartment building above, and even into functional machines, like the caterpillar. And when they sliced a creation in half, it regenerated. (Normally in Minecraft, a user would have to reconstruct the object by hand.)
Going forward, the researchers hope to train systems to grow not only predefined forms, but to invent designs that perform certain functions. This could include flying, allowing engineers to find solutions human designers would not have otherwise foreseen. People might then build these machines in the real world. Or tiny robots might use local interactions (if your neighbor is doing X, do Y) to assemble rescue robots or self-healing buildings. It’s, ahem, a growing field.