HandScribe is an AI-powered, gesture-based virtual calculator that lets you write math equations in the air using simple hand gestures. No physical buttons, no expensive smartboards—just your laptop camera and intuitive gestures to solve arithmetic, algebra, calculus, and geometry problems in real time.
- Gesture‑Driven Input
- Draw with your index finger
- Erase with thumb + middle finger
- Clear canvas with thumb + pinky finger
- Real‑Time AI Analysis
- Powered by Google’s Gemini 2.0 Flash model
- Returns step‑by‑step solutions
- Low‑Cost & Accessible
- Runs on any commodity 720p+ webcam
- No extra hardware required
- Inclusive Design
- Especially helpful for learners with dyscalculia
- Makes math more engaging and accessible
- Streamlit UI
- Live video feed + drawing canvas + AI output
- Python
- OpenCV & NumPy for image capture & canvas management
- MediaPipe for hand‑landmark detection
- Streamlit for the front‑end interface
- google-generativeai (Gemini 2.0 Flash) for equation solving
- python-dotenv for secure API‑key management
- Clone the repository
git clone https://github.com/your-username/HandScribe.git cd HandScribe
Create & activate a virtual environment
bash Copy Edit python -m venv venv source venv/bin/activate # macOS/Linux venv\Scripts\activate # Windows Install dependencies
bash Copy Edit pip install -r requirements.txt Add your Gemini API key in a .env file:
ini Copy Edit GOOGLE_API_KEY=your_gemini_api_key_here
Run the Streamlit app:
streamlit run app.py
-
Draw: Hold index finger up
-
Erase: Thumb + middle finger
-
Clear Canvas: Thumb + pinky finger
-
Solve: Make a peace sign (index + middle finger) to trigger AI analysis
📈 Performance Response Time: ~1–2 seconds (network & model dependent) Accuracy: Gemini 2.0 Flash achieves ~93% on the Math 500 quantitative reasoning benchmark
🤝 Contributing Contributions, issues, and feature requests are welcome!
Fork the repo Create your feature branch (git checkout -b feature/YourFeature) Commit your changes (git commit -m 'Add YourFeature') Push to the branch (git push origin feature/YourFeature) Open a Pull Request Give HandScribe a ⭐ if you find it helpful!