Okay so what is this?
Well if you saw the video then I made a way to control almost any game, in this case “Asphalt 8: Airbone” , with just my hand. So if I tilt my hand forward, left, right or back the car in the game with accelerate, turn left, turn right and deaccelerate respectively.
How did I do this?
I simply used a raspberry pi pico with circuit python connected to a mpu6050. The mpu6050 is a module the outputs acceleration and gyroscope. The pico then reads those outputs from the module and makes it usable for controlling a game.
How do you get started?
First here is a list of thing you need:
- Raspberry pi pico (4USD )
- Mpu6050 (~2 USD)
- Jumper Wires
Secondly you need to get circuit python on the Pico. To do that you simply need to download the uf2 provided from the link below, then press the BOOTSEL button and connect your pico to the pc (while pressing it) and copy the uf2 file to the pico and disconnect you pico.
Next you’ll need to connect the mpu6050 to the pico. To do that you will need 4 jumper wires. Connect:
GP15 and GP14 are the last pins on the left side of the raspberry pi pico.
After doing so, connect your pico to your pc (you must have thonny configured)
The code is provided below:
import time from math import atan2, degrees import board import busio import adafruit_mpu6050 from adafruit_hid.keyboard import Keyboard as kbd from adafruit_hid.keycode import Keycode i2c = busio.I2C(board.GP15, board.GP14) sensor = adafruit_mpu6050.MPU6050(i2c) import usb_hid Keyboard=kbd(usb_hid.devices) kw=Keycode.W ka=Keycode.A ks=Keycode.S kd=Keycode.D k_nitro=Keycode.SPACE # Given a point (x, y) return the angle of that point relative to x axis. # Returns: angle in degrees def vector_2_degrees(x, y,l=False): if l: angle = degrees(atan2(y, x)) else: angle = degrees(atan2(y, x)) if angle < 0: angle += 360 return angle # Given an accelerometer sensor object return the inclination angles of X/Z and Y/Z # Returns: tuple containing the two angles in degrees t="" f="" def get_inclination(_sensor): x, y, z = _sensor.acceleration return vector_2_degrees(x, z,True), vector_2_degrees(y, z), int(x),int(y),int(z) x,y,z=0,0,0 def press(key): Keyboard.press(key) def release(key): Keyboard.release(key) turn=None ac=None toA=None #l=[None,None] while True: try: turn, ac,x,y,z = get_inclination(sensor) turn=int(turn) ac=int(ac) except: pass #print(turn,ac,x,y,z,sep=" : ") if turn<50: if t=="r": Keyboard.release(kd) #leave r print("left") Keyboard.press(ka) #l=ka t="l" elif turn>130: if t=="l": Keyboard.release(ka) #leave l print("right") Keyboard.press(kd) #l=kd t="r" else: if t=="l": Keyboard.release(ka) #leave l elif t=="r": Keyboard.release(kd) #leave r l=None if ac<50: if f=="f": Keyboard.release(kw) # leave f print("brake") Keyboard.press(ks) #toA=ks f="b" elif ac>120: if f=="b": Keyboard.release(ks) #leave b print("accel") Keyboard.press(kw) #toA=kw f="f" else: if f=="f": Keyboard.release(kw) # leave f elif f=="b": Keyboard.release(ks) #leave b if z >=11: print("nitro") Keyboard.send(k_nitro) time.sleep(0.000001)
What is basically happening in the code is I am getting raw values from the module mentioned above and converting them using atan2() into inclination. I am used those converted values to get the tilt(foward, left, right or back) and then sending keyboard keys respectively.
Lastly I am using intertia on the Z axis to get the gravity. So if the gravity value increases “nitro” is activated.
What next? Theoretically, we can share the laptop screen to a vr like google cardboard or oculus rift and play asphlat 8 in vr which is controlled by hand. I’ll make an update on it.
2 thoughts on “Controlling Asphalt 8 with hand gestures, using mpu6050 and raspberry pi pico”