I wanted to build something fun, not another CRUD app or chatbot.
So I built Adaptive Hand Cricket: a browser-based game where you play hand cricket through your webcam, your hand gestures are recognized in real time, and the computer gradually becomes harder by combining randomness, live gesture reading, and machine learning.
How it works
The game uses:
- React + Vite for the frontend
- MediaPipe for real-time hand landmark detection
- TensorFlow.js for learning player patterns
-
Webcam input through
getUserMedia - localStorage to remember previous batting behavior
The supported gestures are:
1 -> index
2 -> index + middle
3 -> index + middle + ring
4 -> index + middle + ring + pinky
5 -> all fingers
6 -> thumb
7 -> thumb + index
8 -> thumb + index + middle
9 -> thumb + index + middle + ring
Instead of simply counting fingers, I classify the exact finger pattern.
The game loop
Every delivery follows:
Computer commits to a move
↓
3
2
1
↓
SHOW
↓
Human gesture is locked
↓
Both moves are revealed
↓
Score / wicket is calculated
This keeps the reveal synchronized like actual hand cricket.
Making the opponent adaptive
A completely random computer got boring quickly, so I added three decision strategies:
RANDOM
ML
PEEK
Difficulty automatically increases with the player's score:
0-39 -> EASY
40-99 -> MEDIUM
100+ -> HARD
Each level changes how often the opponent uses each strategy.
For example:
EASY: {
peekChance: 0.10,
mlChance: 0.30,
randomChance: 0.60
}
Peek strategy
Humans often form their gesture before the countdown reaches zero.
On some deliveries, the computer is secretly allowed to inspect the gesture at a random point during:
3... 2... 1...
If it gets a valid reading, it can use that number.
Slightly evil. Very effective.
Learning player patterns
The more interesting strategy uses TensorFlow.js.
Every batting move is stored:
7, 4, 2, 7, 8, 4, 7...
The model learns from:
- the player's last 5 moves
- overall frequency of numbers 1-9
- recent frequency of numbers 1-9
This produces 63 input features.
The network is intentionally small:
63 inputs
↓
Dense 64
↓
Dense 32
↓
9-way Softmax
The output might look like:
4 -> 18%
7 -> 42%
8 -> 12%
...
So the model predicts 7 as the player's next move.
Training starts after enough moves have been collected and retrains periodically rather than after every ball.
What surprised me
The machine learning was not actually the hardest part.
The trickier problems were:
- thumb detection across different hand orientations
- synchronizing countdown and capture timing
- preventing invalid camera frames from affecting gameplay
- managing game transitions cleanly
- keeping browser ML from interfering with real-time video processing
A small game ended up touching computer vision, React state management, probability, feature engineering, browser ML, and real-time interaction design.
What I'd improve next
I'd like to add:
- a short multi-frame voting window instead of using one camera frame
- proper ML-vs-random prediction accuracy tracking
- player profiles
- a comparison between neural networks and simpler approaches like Markov chains
- better gesture classification for rotated hands
Tech Stack
React
Vite
MediaPipe Tasks Vision
TensorFlow.js
WebRTC / getUserMedia
localStorage
It started as a simple question:
Can I play hand cricket against my webcam?
It turned into:
Can the computer learn how predictable I am?
