Project 03 · Computer Vision · Research

HOLORACT

Wearable-free hand tracking framework for interactive holographic STEM education. 94% accuracy at 60 FPS. Research paper under review at Springer.

View on GitHub ↗

Overview

HOLORACT is a gesture-controlled holographic framework designed for interactive STEM education. It enables students to manipulate 3D educational content through natural hand gestures — without requiring any wearable devices, depth sensors, or specialized hardware.

The system uses a single standard RGB camera for hand landmark detection, a custom Python gesture recognition pipeline, and Three.js & React for WebGL holographic rendering — all communicating through low-latency socket connections.

This project resulted in a research paper submitted to the SCRS CIMA Conference (Springer), currently under review.

Architecture

Standard RGB Camera Input ↓ MediaPipe Hand Landmark Detection ↓ Custom Python Gesture Recognition Model ↓ Real-time Socket Data Transmission ↓ Three.js 3D Rendering Engine ↓ Holographic Object Manipulation ↓ User Visual Feedback

System Demonstration

Real-time hand tracking demonstration showing gesture-controlled interaction with 3D holographic content.

Extended demo showcasing the holographic projection and interactive STEM learning modules.

Key Features

Wearable-Free Setup: Requires only a standard RGB camera — no depth sensors, gloves, or specialized hardware.
94% Recognition Accuracy: Custom gesture recognition model achieving high accuracy across diverse hand shapes and lighting conditions.
Low-Latency Pipeline: End-to-end processing at approximately 68ms latency, enabling real-time interaction at 60 FPS.
Interactive 3D Rendering: Three.js and React-based WebGL holographic environment for manipulating educational 3D models.
Robust Tracking: MediaPipe hand landmark detection performs reliably across varied lighting conditions.
Research Validation: Full pipeline validated through published research methodology.

System Interface & Rendering

System Interface

Real-time hand tracking UI with camera feed and interactive modules.

Holographic Display

3D projection showing interactive educational content.

Results & Metrics

94%

Gesture Recognition Accuracy

60 FPS

Real-time Processing

~68ms

End-to-end Latency

Tech Stack

Frontend & 3D UI

Three.js React WebGL Holographic HUD

Backend & Networking

Python Socket Communication Real-time Pipeline

Database & Storage

SQLite Session Logs

AI & Computer Vision

MediaPipe OpenCV Gesture Model