Controlling Asphalt 8 with hand gestures, using mpu6050 and raspberry pi pico

Okay so what is this?

Well if you saw the video then I made a way to control almost any game, in this case “Asphalt 8: Airbone” , with just my hand. So if I tilt my hand forward, left, right or back the car in the game with accelerate, turn left, turn right and deaccelerate respectively.

How did I do this?

I simply used a raspberry pi pico with circuit python connected to a mpu6050. The mpu6050 is a module the outputs acceleration and gyroscope. The pico then reads those outputs from the module and makes it usable for controlling a game.

How do you get started?

First here is a list of thing you need:

• Raspberry pi pico (4USD )
• Mpu6050 (~2 USD)
• Jumper Wires

Secondly you need to get circuit python on the Pico. To do that you simply need to download the uf2 provided from the link below, then press the BOOTSEL button and connect your pico to the pc (while pressing it) and copy the uf2 file to the pico and disconnect you pico.

Download here

Next you’ll need to connect the mpu6050 to the pico. To do that you will need 4 jumper wires. Connect:

GP15 and GP14 are the last pins on the left side of the raspberry pi pico.

After doing so, connect your pico to your pc (you must have thonny configured)

The code is provided below:

import time
from math import atan2, degrees
import board
import busio
import adafruit_mpu6050
from adafruit_hid.keyboard import Keyboard as kbd
from adafruit_hid.keycode import Keycode
i2c = busio.I2C(board.GP15, board.GP14)
sensor = adafruit_mpu6050.MPU6050(i2c)
import usb_hid
Keyboard=kbd(usb_hid.devices)

kw=Keycode.W
ka=Keycode.A
ks=Keycode.S
kd=Keycode.D

k_nitro=Keycode.SPACE
# Given a point (x, y) return the angle of that point relative to x axis.
# Returns: angle in degrees


def vector_2_degrees(x, y,l=False):
    if l:
        angle = degrees(atan2(y, x))
    else:
        angle = degrees(atan2(y, x))
    if angle < 0:
        angle += 360
    return angle


# Given an accelerometer sensor object return the inclination angles of X/Z and Y/Z
# Returns: tuple containing the two angles in degrees
t=""
f=""
def get_inclination(_sensor):
    x, y, z = _sensor.acceleration
    return vector_2_degrees(x, z,True), vector_2_degrees(y, z), int(x),int(y),int(z)


x,y,z=0,0,0
def press(key):
    Keyboard.press(key)
def release(key):
    Keyboard.release(key)
turn=None
ac=None
toA=None
#l=[None,None]
while True:
    try:
        turn, ac,x,y,z = get_inclination(sensor)
        turn=int(turn)
        ac=int(ac)
    except:
        pass
    #print(turn,ac,x,y,z,sep=" : ")
    
    if turn<50:
        if t=="r":
            Keyboard.release(kd) #leave r
        print("left")
        Keyboard.press(ka)
        #l[0]=ka
        t="l"
    elif turn>130:
        if t=="l":
            Keyboard.release(ka) #leave l
        print("right")
        Keyboard.press(kd)
        #l[0]=kd
        t="r"
    else:
        if t=="l":
            Keyboard.release(ka) #leave l
        elif t=="r":
            Keyboard.release(kd) #leave r
        l[0]=None
    
    if ac<50:
        if f=="f":
            Keyboard.release(kw) # leave f
        print("brake")
        Keyboard.press(ks)
        #toA=ks
        f="b"
    elif ac>120:
        if f=="b":
            Keyboard.release(ks) #leave b
        print("accel")
        Keyboard.press(kw)
        #toA=kw
        f="f"
    else:
        if f=="f":
            Keyboard.release(kw) # leave f
        elif f=="b":
            Keyboard.release(ks) #leave b
    
    
    if z >=11:
        print("nitro")
        Keyboard.send(k_nitro)
    
    
    
    
    time.sleep(0.000001)

What is basically happening in the code is I am getting raw values from the module mentioned above and converting them using atan2() into inclination. I am used those converted values to get the tilt(foward, left, right or back) and then sending keyboard keys respectively.

Lastly I am using intertia on the Z axis to get the gravity. So if the gravity value increases “nitro” is activated.

Conclution

What next? Theoretically, we can share the laptop screen to a vr like google cardboard or oculus rift and play asphlat 8 in vr which is controlled by hand. I’ll make an update on it.