Device Sensors Hub
Access device camera and learn about other sensor integrations.
Device-Specific Assessment Allocation & Instructions
ABI Promedical Rehab Assistant leverages a suite of hardware devices—each selected for their medical-grade data fidelity and compatibility with real-time AI analysis. These devices power a modular framework of assessments designed to capture clinically relevant signals across physical, neurological, and behavioral domains.
Device Camera Access
Enable your device camera. This feed can be used for visual data capture for various assessments.
Click "Enable Camera" to start video feed.
Microsoft Kinect v2
Full-body skeletal tracking and posture modeling.
Use: Full-body skeletal tracking and posture modeling.
Assessments Enabled:
- Gait symmetry and step cycle
- Sit-to-stand performance
- Joint range of motion (ROM) for hips, knees, shoulders
- Postural sway and center-of-mass stability
- Ergonomic RULA Scoring (Auto-calculates posture strain)
- Facial Expression Recognition (AI-based mood assessment, opt-in)
Relevance:
Enables orthopedic, neurological, and fall-risk analysis with precision motion tracking validated against lab-grade tools. Kinect v2 offers advanced infrared and skeletal data that closely mirrors in-clinic physical exams, helping detect muscular compensation, pain-avoidance behavior, and asymmetry. (Requires dedicated hardware and backend processing).
Infrared Sensors (Kinect IR + Smartphone IR)
Passive thermal scanning for physiological insights.
Use: Passive thermal scanning.
Assessments Enabled:
- Localized inflammation detection (e.g., joint heat mapping)
- Skin temperature differentials for wound surveillance
- Muscle Contraction Zones (Monitor active vs dormant zones via thermal load mapping - conceptual)
- Peripheral Cold Sensitivity (Useful for diabetic neuropathy or vascular disorders - conceptual)
Relevance:
IR insights reveal non-visible physiological changes, enhancing early detection and triage models. IR feedback can provide early indicators of joint inflammation, nerve dysfunction, or vascular insufficiency. (Requires specific IR sensors and backend processing).
User Instructions in App
Guidance for performing assessments.
- Each test includes an animated demonstration with audio guidance.
- Explanations provide the clinical context: e.g., “This task helps detect early balance impairment that may lead to falls.”
- Users receive real-time feedback, corrective tips, and AI-generated summaries after each test.
Together, this hardware ecosystem allows ABI Promedical to function as a comprehensive digital rehabilitation lab—capable of autonomously performing multimodal assessments with clinical-level accuracy, guided interpretation, and actionable output.
Smartphone Sensors
Gyroscope, Accelerometer, Camera, Microphone for mobile-based analysis.
Use: Mobile-based motion, audio, and image analysis.
Assessments Enabled:
- Hand tremor quantification (neurological screening)
- Arm swing balance (e.g., Parkinson’s indicators)
- Voice-based respiratory rate detection
- Camera-based upper limb ROM estimation
- Sit-to-Stand & Balance Tasks (Real-time motion tracking of trunk movement)
- Gait and Step Cadence (Gyro/accelerometer analysis of foot impact and stride timing)
Relevance:
Transforms everyday mobile phones into decentralized diagnostic tools that mirror clinical sensor workflows. Combined with AI, they help uncover fatigue patterns, instability, and deterioration before they lead to injury. (Camera, motion sensors, and microphone access require user permissions).
Eye Tracking (Front Camera-based + SDKs)
Visual engagement and oculomotor control testing.
Use: Visual engagement and oculomotor control testing.
Assessments Enabled:
- Fixation and pursuit tracking (cognitive fatigue index)
- Reaction time with visual target tests
- Saccades, Smooth Pursuit, Fixation Holding (Detects cranial nerve function, cognitive fatigue - conceptual)
- Visual Balance Control (Paired with posture tasks - conceptual)
Relevance:
Key tool for concussion, stroke, and cognitive workload evaluation. Eye movements are windows into both the brain and muscular systems. Abnormal eye behavior can indicate early-stage neurodegeneration or vestibular dysfunction. (Relies on camera feed and potentially advanced backend AI processing or third-party SDKs).
Wearable Sensors (Optional Integration)
Support for smartwatches, fitness bands, and EMG armbands.
ABI modules support connection to smartwatches, fitness bands, and EMG armbands.
Used for: Continuous HRV, sleep tracking, limb muscle activation, and step cadence.
Integration Note
Connecting to specific wearables often requires vendor-specific SDKs or APIs and may involve Bluetooth or cloud-to-cloud data synchronization. This section highlights the platform's readiness to integrate such data sources when available and consented by the user.
Advanced Control & Interaction
Enable alternative interaction methods. (Conceptual - requires further integration).
Enable voice commands for app navigation and control. This would typically require microphone access.
Voice Activation Note
Full voice activation uses the browser's Speech Recognition API (which involves a microphone permission prompt if not already granted for the site) or integrates with a backend voice processing service. Enabling this switch signifies intent to use such features; disabling it stops the intent.
Enable eye-tracking for attention monitoring or gaze-based controls. Relies on camera feed.
Eye-Tracking Note
Robust eye-tracking relies on the camera feed being active and processed (typically by a backend AI system). Enabling this switch signifies intent to use such features with the camera. Disabling this switch, or stopping the camera, deactivates this conceptual feature.
These features are illustrative. Full implementation requires specific APIs and/or backend processing. User consent for camera/microphone access is paramount.
Manual Data Log & AI Analysis
Log observations or notes related to sensor data or symptoms for backend AI processing.
Backend AI Processing
In a complete system, these manual logs, along with data from enabled sensors (like the camera feed or future Kinect integrations), would be sent to a backend (similar to your comprehensive Python script). The backend would perform AI analysis (e.g., movement classification, anomaly detection, correlating logs with sensor data) and store structured insights in Firestore. This app would then display those insights.