The Best Way to Play Breakout Is to Stop Playing

David Malan, speaking at a family lecture on artificial intelligence, put a familiar screenshot on the screen: Breakout, the old arcade game with a paddle at the bottom, a bouncing ball, and rows of colored bricks above.

He started with Pong, the tennislike game where two players move a paddle up and down to bounce a ball back and forth. Breakout is really the same idea flipped around for one player. You bounce the ball against the bricks, the bricks disappear when hit, with the aim of clearing them all.

Games are a good entry point for explaining AI since they are fun and because they have crisp rules and a clear aim, such as getting the highest total. Even from a still image, people have an instinct for what to do. When the ball had recently bounced off the paddle toward the left and toward a brick, everyone knows the paddle should move left to meet it when it comes back.

You might think the way to win is to play very deterministically, moving left and right and gradually working through the blue bricks at the bottom, then the green, then the yellow, and so on toward the top.

But the video Malan showed next was of a machine figuring out through trial and reward which actions to repeat. Somewhere, he said, a human was probably giving it feedback, rewarding good moves and discouraging bad ones, or the lesson was baked into the score itself, based on the color of the bricks. Awarding higher values to upper rows than to lower rows amounts to favoring the strongest approach.

Then the AI, Malan said, "a little creepily," discovered that the ideal approach was to stop steering and allow the round to continue by itself. In the video, play shifts into hands-off mode, and the system keeps collecting the top total by itself, with no steady paddle work.

Malan suggested it only would have arrived there if perhaps it tested a few moves randomly and stumbled on something better than mindlessly going back and forth. It was an example of reinforcement learning at work, where there has to be some point system, and maybe a human supervising the process, for the machine to learn from.