rPPG-Toolbox (TS-CAN)
Contactless Heart Rate (Deep rPPG)
RUN
This tool brings the rPPG-Toolbox
TS-CAN model into the EZ-MMLA Toolkit as a browser-based,
contactless heart-rate monitor. It recovers the blood-volume-pulse (BVP)
signal from subtle skin-color changes in a person's face captured by the
webcam, then estimates heart rate in beats per minute.
Inference runs entirely in the browser via
ONNX Runtime Web. A
lightweight face detector locates the face each half-second; every captured
frame is cropped, resized to 72 x 72, and normalized into the two
TS-CAN input branches (a DiffNormalized motion branch and a
Standardized appearance branch). Over a rolling analysis window the
model outputs a per-frame BVP value, and the dominant frequency in the
physiological band (0.7–3.0 Hz) is converted to a heart-rate estimate.
It follows the same run workflow as the other JS tools: webcam (or uploaded video) capture, recording controls, a live BVP waveform, downloadable CSV output, and a session overview.
Note: this is a research and teaching tool, not a medical device. Accuracy depends on stable lighting, limited head motion, and a clear view of the face.
The dominant-frequency search always returns some answer, even from
a noisy or motion-corrupted window — so each estimate ships with a
confidence score (0–1): the winning frequency bin's share of
total power across the scanned band. A clean pulse concentrates power in one
narrow peak; a noisy one spreads it out. Readings below roughly 0.15 are
flagged "(low confidence)" live rather than shown with the same certainty as
a clean one — filter on the confidence column before trusting
heart_rate_bpm for anything downstream.
| Output | Description |
|---|---|
heart_rate_bpm |
Estimated heart rate (beats per minute) for the current window. |
confidence |
Peak-to-band-power ratio (0–1); low values mean the estimate is closer to noise than a real pulse. |
sampling_fps |
Effective frame-capture rate measured over the analysis window. |
window_frames |
Number of frames in the window used for the estimate. |
timestamp_world |
Wall-clock time (ms) of the estimate. |
timestamp_video |
Playback time (s) when analyzing an uploaded video. |