Journal of Information and Communications Technology: Algorithms, Systems And Applications
A single-blind peer-reviewed, quarterly, open-access journal committed to advancing cutting-edge research across the full spectrum of ICT.
Magic Learn-DrawInAir: Redefining Creativity, Problem Solving, Building Worlds with AI-Powered Gesture Learning
J. Inf. Commun. Technol. Algorithms Syst. Appl., 2025, 1(2), 25310 https://doi.org/10.64189/ict.25310
Received: 18 July 2025 | Revised: 10 September 2025 | Accepted: 16 September 2025
Cite article
S. Sarkar, D. Saliya, H. Patel, D. Shrimal, Magic Learn-DrawInAir: redefining creativity, problem solving, building worlds with AI-powered gesture learning, Journal of Information and Communications Technology: Algorithms, Systems and Applications, 2025, 1(2), 25310, doi: . https://doi.org/10.64189/ict.25310
Rights and permissions
(c) The Author(s) 2025.
Open Access
This article is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License, which permits the non-commercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as appropriate credit is given and changes are indicated. https://creativecommons.org/licenses/by-nc/4.0/
Abstract
Magic Learn-DrawInAir is an AI-powered educational tool that enables users to draw, solve equations, control presentations, stream drawings via virtual camera, and interact through a real-time 3D avatar using only hand gestures and facial tracking, eliminating the need for physical input devices. The system integrates MediaPipe for real-time hand tracking, OpenCV for virtual canvas rendering, and Streamlit for a user-friendly web interface. A unique aspect is the use of Google Gemini API, which analyzes gesture-based drawings to solve mathematical expressions or describe creative visuals. The platform also supports gesture-based navigation of PowerPoint or PDF slides, making it highly suitable for virtual teaching and learning environments. The platform supports gesture-based navigation and annotation of PowerPoint or PDF slides, virtual camera output for drawing and erasing in OBS Studio, Google Meet, and Zoom, and a 3D avatar using MediaPipe FaceMesh for immersive interaction. Designed to be hardware-independent and cost-effective, the system enhances accessibility and creativity in education. It offers a futuristic learning experience through intuitive gesture control, facial tracking, and AI-enhanced understanding.
Graphical Abstract
Novelty Statement
The Magic Learn-DrawInAir system successfully demonstrates the potential of gesture-based, AI-powered educational tools that operate without specialized hardware.