Teach an AI to Play Pong

Step 5: Training 25 minutes

Now for the fun part. We’re going to create a neural network and train it to know where to move the paddle, given the ball’s current position, direction and speed, a.k.a. its “state”.

Modules

Neural networks are not part of the core Kid.js framework, so we need to load some additional code. It’s common for JavaScript to be organized this way. Rather than loading everything, the framework is broken down into smaller pieces we can pick and choose from. These pieces are sometimes called modules or packages.

Add the following line at the top of your app. This will load the “neural network” module.

import 'neural-network'

Neural Network

We can now create a neural network using the neuralNetwork() function and store it in a variable. You can use any variable name you'd like, so give your AI a creative name!

let bob = neuralNetwork()

We’re also going to need to keep track of a few other things. Create a variable called “mode” to keep track of whether our app is in “training” or “autopilot” mode, and a variable called “state”, that will store the ball’s last seen position, speed and direction.

let mode = 'TRAINING'
let state

Let’s display the current mode in the center of the screen so it’s clear to the user.

display(width / 2, height / 2, mode)

Inputs

When the ball bounces off of something, we want to take note of its new position, speed and direction.

We can store all three values in the state variable by combining them into an array. An array is a special type of variable that contains a list of items. Create an array by listing multiple values separated by commas, inside of square parentheses.

The position, speed and direction will be the inputs to our neural network.

function onBallBounced() {
  wait(0.01)
  state = [ball.x, ball.y, ball.speed, ball.direction]
}

ball.on('collision', onBallBounced)

Notice we wait a fraction of a second before recording the state. This is to give the ball a chance to change direction after hitting a wall or paddle.

We also want to take note of the ball’s state just after it’s launched. Modify the launch() function to run onBallBounced()

function launch() {
  ball.x = width /2
  ball.y = 70
  ball.speed = random(5, 10)
  ball.direction = random(45, 135)
  onBallBounced()
}

Output

The output of our neural network will be the x coordinate to move the paddle to.

If the ball hits the paddle, that means the paddle was in the right spot. When this happens, we’ll add the state and the paddle’s x position to the training data.

function hit() {
  bob.train(state, paddle.x)
}

paddle.on('collision', hit)

Step 6: Autopilot 25 minutes

After warming up our AI, we’re ready to hand the reins over. Write a function to change the mode to AUTOPILOT, and run it on the “click” event.

function startAutopilot() {
  mode = 'AUTOPILOT'
}

on('click', startAutopilot)

Now that the AI is in charge, moving our mouse shouldn’t move the paddle. Modify the movePaddle() function to only move the paddle when in training mode. Use an if statement to do this.

function onMouseMove() {
  if (mode == 'TRAINING') {
    paddle.x = mouseX
  }
}

In the onBallBounced() function, we’ll ask the AI where to move the paddle given its current state. This is where neural networks are powerful. Even if the network has never seen the ball in that exact position, speed and direction, it is still able to make a guess as to where to move the paddle.

Of course we only do this when in "autopilot" mode.

function onBallBounced() {
  wait(0.01)
  state = [ball.x, ball.y, ball.speed, ball.direction]

  if (mode == 'AUTOPILOT') {
    let x = bob.run(state)
    paddle.moveTo(x, paddle.y)
  }
}

Try it out. Spend some time training the AI. After 20 hits or so, click to switch over to autopilot.

import 'neural-network' gravity = 0 friction = 0 floor = false ceiling = false let ball = circle(200, 70, 40) let paddle = rect(width / 2, height - 50, 200, 20) function onMouseMove() { if (mode == 'TRAINING') { paddle.x = mouseX } } on('mousemove', onMouseMove) let bob = neuralNetwork() let mode = 'TRAINING' let state display(width / 2, height / 2, mode) function launch() { ball.x = random(0, width) ball.y = 70 ball.speed = random(5, 10) ball.direction = random(45, 135) onBallBounced() } launch() let hits = 0 let misses = 0 display(80, 80, hits) display(width - 80, 80, misses) function onBallMoved() { if (ball.y > height) { misses = misses + 1 launch() } if (ball.y < 0) { hits = hits + 1 launch() } } ball.on('move', onBallMoved) function onBallBounced() { wait(0.01) state = [ball.x, ball.y, ball.speed, ball.direction] if (mode == 'AUTOPILOT') { let x = bob.run(state) paddle.moveTo(x, paddle.y) } } ball.on('collision', onBallBounced) function hit() { bob.train(state, paddle.x) } paddle.on('collision', hit) function startAutopilot() { mode = 'AUTOPILOT' } on('click', startAutopilot)

Step 7: Human vs Machine 10 minutes

Think you can take on the AI? Let’s add a second paddle, but keep it hidden for the time being.

let human = rect(width / 2, 40, 200, 20)
human.hide()

Update the onMouseMove() function to move this paddle instead when not in training mode.

function onMouseMove() {
  if (mode == 'TRAINING') {
    paddle.x = mouseX
  } else {
    human.x = mouseX
  }
}

When autopilot starts, show the paddle and reset the score to zero. Add this to the startAutopilot() function.

function startAutopilot() {
  mode = 'AUTOPILOT'
  human.show()
  hits = 0
  misses = 0
}

Game on! May the best intelligence win.

import 'neural-network' gravity = 0 friction = 0 floor = false ceiling = false let ball = circle(200, 70, 40) let paddle = rect(width / 2, height - 50, 200, 20) let human = rect(width / 2, 40, 200, 20) human.hide() function onMouseMove() { if (mode == 'TRAINING') { paddle.x = mouseX } else { human.x = mouseX } } on('mousemove', onMouseMove) let bob = neuralNetwork() let mode = 'TRAINING' let state display(width / 2, height / 2, mode) function launch() { ball.x = random(0, width) ball.y = 70 ball.speed = random(5, 10) ball.direction = random(45, 135) onBallBounced() } launch() let hits = 0 let misses = 0 function onBallMoved() { if (ball.y > height) { misses = misses + 1 launch() } if (ball.y < 0) { hits = hits + 1 launch() } } ball.on('move', onBallMoved) display(80, 80, hits) display(width - 80, 80, misses) function onBallBounced() { wait(0.01) state = [ball.x, ball.y, ball.speed, ball.direction] if (mode == 'AUTOPILOT') { let x = bob.run(state) paddle.moveTo(x, paddle.y) } } ball.on('collision', onBallBounced) function hit() { bob.train(state, paddle.x) } paddle.on('collision', hit) function startAutopilot() { mode = 'AUTOPILOT' human.show() hits = 0 misses = 0 } on('click', startAutopilot)

Taking It Further

Try training the AI for shorter and longer amounts of time. How does that affect how well it does?
Increase the speed of the ball each point. Can the AI keep up? Can you?