Speech Emotion Recognition
Real-Time Vocal Emotion Recognition
RUN
Background
This tool listens to vocal tone rather than word content: it classifies short windows of audio into one of six emotions — angry, disgust, fear, happy, neutral, sad — based on how something is said, not what is said. It is a companion to the toolkit's text-based sentiment/discourse tools (like jsLDA and the Toxicity classifier), which work from transcripts instead.
Under the hood it uses Wav2Vec2, a speech representation model fine-tuned for emotion classification, exported to ONNX and run entirely in the browser via Transformers.js — the same in-browser inference library this toolkit uses for Whisper transcription. No audio ever leaves the page.
Process
Audio from the live microphone (or an uploaded video's audio track) is captured in short, non-overlapping windows (2–5 seconds, configurable in the settings panel). Each window is downmixed to mono and resampled to the 16kHz rate the model expects, then classified into scores for all six emotions. The dominant emotion and its confidence are shown live as overlaid bars; while recording, every window's full score set is saved for CSV export.
Because it analyzes whatever the microphone picks up as a whole, it does not separate individual speakers — it reports one emotion reading per window for the room, not per person. In a multi-speaker setting, treat it as a signal about the group's vocal tone in that moment, not any one individual's.
Results
The run page follows the toolkit's standard workflow: listen live from the microphone or analyze an uploaded video, record a session, tag moments, view a session overview, and download both the captured video and a prediction CSV. Each CSV row is one analyzed window with all six emotion scores, the dominant label, and its confidence.
References
- Model: onnx-community/wav2vec2-base-Speech_Emotion_Recognition-ONNX
- Wav2Vec2: https://arxiv.org/abs/2006.11477
- Transformers.js: https://github.com/huggingface/transformers.js