May 2025 – Dec 2025
Multimodal Hand Tracking for Stroke Rehabilitation
iPhone and Apple Vision Pro working as one tracking surface to assess hand and eye movement after a stroke, built in collaboration with Apple at Interactive 3D Lab.
Stack
Swift, RealityKit, visionOS, ARKit, iOS
Links
Research source private
Notes
I built the visionOS app, the spatial interface and the real-time tracking, plus the iPhone companion. It was tested with real clinicians and real patients. The implementation link stays private while the research disclosure process continues.
The numbers
- Contribution
- Built the visionOS spatial UI, real-time hand and gaze tracking, and an iPhone companion app.
- Validation
- Reached 89% cross-device accuracy, 30 fps, and sub-millisecond latency; evaluated across three sessions with three patients per session.
- Research status
- Co-author on a paper in preparation and co-inventor on a patent application in progress.