PROJECT MALMO : USING MINECRAFT TO BUILD MORE INTELLIGENT TECHNOLOGY

Project Malmo or Malmo Project? Just to clarify

By Pierre-Louis Xech, Microsoft in the Revue TELECOM n° 187

How can we develop artificial intelligence that learns to make sense of complex environments? That learns from others, including humans, how to interact with the world? That learns transferable skills throughout its existence, and applies them to solve new, challenging problems? Project Malmo sets out to address these core research challenges, addressing them by integrating (deep) reinforcement learning, cognitive science, 
 and many ideas from artificial intelligence.

Project Malmo, is a software platform developed by researcher Katja Hofmann and her colleagues in Microsoft’s Cambridge, UK, lab and released in 2016. Project Malmo allows computer scientists to use the world of Minecraft as a testing ground for conducting research designed to improve artificial intelligence. Microsoft researchers are using Project Malmo for their own research, and they have made it available via an open-source license on Github (https://github.com/Microsoft/malmo).

Katja Hofmann came up with the idea for Project Malmo about a year before, in part because she was frustrated by the limitations of other platforms that use simpler, less sophisticated games for artificial intelligence research. The Malmo platform is a sophisticated AI experimentation platform built on top of Minecraft, and designed to support fundamental research in artificial intelligence. As described in Wikipedia, Minecraft is a sandbox video game created and designed by Swedish game designer Markus "Notch" Persson, and later fully developed and published by Mojang, a company acquired by Microsoft in 2014. The creative and building aspects of Minecraft allow players to build with a variety of different cubes in a 3D procedurally generated world. 

Other activities in the game include exploration, resource gathering, crafting, and combat. Minecraft is ideal for artificial intelligence research for the same reason it is addictively appealing to the millions of fans who enter its virtual world every day. Unlike other computer games, Minecraft offers its users endless possibilities, ranging from simple tasks, like walking around looking for treasure, to complex ones, like building a structure with a group of teammates. “Minecraft is the perfect platform for this kind of research because it’s this very open world,” Hofmann said. “You can do survival mode, you can do ‘build battles’ with your friends, you can do courses, you can implement our own games. This is really exciting for artificial intelligence because it allows us to create games that stretch beyond current abilities.”

The Project Malmo platform consists of a mod for the Java version, and code that helps artificial intelligence agents sense and act within the Minecraft environment. The two components can run on Windows, Linux, or MacOS, and researchers can program their agents in any programming language they’re comfortable with.

From doing to learning

Over the past few years, artificial intelligence researchers have gotten very good at teaching computers to do specific, often complicated tasks. Hofmann said artificial intelligence researchers are able to take tiny slices of that total awareness and build tools that do one thing, like recognize words, but they haven’t been able to combine them in the way that humans do effortlessly. She said that’s partly because we don’t really know how humans are combining those senses. “We don’t understand ourselves well enough,” she said.

From theory to practice

There has been plenty of theoretical research into general artificial intelligence, but researchers have always been limited by practical ways to test their systems. Building a robot and trying to teach it to climb a real hill is costly and impractical; unlike in Minecraft, you’d have to repair or replace the robot with another costly machine each time it fell into a river. It’s also not that easy to test general artificial intelligence research on systems people are using in the real world. The Minecraft platform was especially appealing because it allows players to make really complex decisions that have consequences, and to add more and more difficult elements as they get better. It also lets users work together, which could help researchers experiment with how humans and artificially intelligent agents could cooperate together.

Advancing all artificial intelligence research

From the beginning, Hofmann said the goal with Project Malmo was to build a system that would be useful both for Microsoft’s own research and for the broader artificial intelligence research community. In the early 2017, Microsoft Research started the “Malmo Collaborative AI Challenge” and asked teams of PhD Students to solve a game using collaborative agents, a project designed to push the state of the art of collaborative AI. Out of this challenge they learned that there is no single approach to this problem that was clearly superior to the others. Interstingly, many teams achieved their success through a combination of both classic AI approaches and incorporating newer heuristics and models. And sometimes the biggest learning is what question to ask next. One of the biggest problems in collaborative AI is the ability for an agent to, on-the-fly, adjust to new information in a dynamic environment. By introducing agents of unknown character and goals that the agent had to collaborate with, the teams were required to solve for this level of dynamism in real time. As a result, this challenge helped identify promising directions for additional research in helping AI agents adapt to unknown conditions.

Related

Project Malmö, named after a town between Cambridge, UK where it was developed and Stockholm, Sweden where Minecraft was created, is an Artificial Intelligence (AI) platform that allows researchers to create challenging and interesting tasks for evaluating agents, and Minecraft enthusiasts to engage in the Modding community and help advance AI. The platform is built keeping in mind recent advances in Deep Reinforcement Learning for Video Game playing; however, the project is intended to be very open ended allowing for research into more general AI topics such as Multi-Agent systems, Transfer Learning, and Human-AI interaction. 

Biographie de l'auteur

Pierre-Louis Xech est Directeur adjoint du Centre de Recherche Commun Inria-Microsoft Research situé sur le campus de l'Université Paris-Saclay, et "Program Manager » au sein de la division "Commercial Software Engineering » (CSE) à Microsoft France. Pierre-Louis Xech a rejoint Microsoft en 1997 tout d'abord comme expert et consultant en Génie Logiciel. En 2003 il rejoignait le laboratoire de recherche de Microsoft à Cambridge (UK) pour entamer des projets de recherche collaborative en Génie Logiciel, Informatique mobile et répartie ainsi qu'Interaction homme-machine. Depuis 2007 il codirige le Centre de Recherche Commun Inria-Microsoft Research et supervise les relations de Microsoft France avec la communauté de recherche.