ollos-mcp

Eyes and ears for agents, running entirely on your machine.

Role: Author and maintainer
Status: Open source · npm
Stack: Node.js · ONNX Runtime · Whisper · OCR
Link: github.com/kelvinbiffi/ollos-mcp ↗

01The problem

Agents cannot watch a video or hear a call, and sending private media to a cloud API just so a model can read it is often not allowed.

02What I built

Whisper transcription, speaker segmentation, perceptual-hash keyframing and OCR, all running locally in Node through ONNX Runtime. No Python, no cloud, no API key, so the file never leaves the machine. The same core ships as an MCP server, a CLI and a Node library.

03The decision that mattered

It reports confidence per block and per segment, and filters the lines Whisper invents over silence. A transcript that does not say where it is unsure is worse than one that is visibly wrong.

04Receipts

On npm as ollos-mcp
Transcribe, diarize, keyframes, OCR, search and review
Confidence reported per segment, hallucinated silence filtered

05MCP

Transcribe, split speakers, pick keyframes and read on-screen text from local media, with confidence per segment. · Local · no key

ollos_probe
ollos_transcribe
ollos_diarize
ollos_keyframes
ollos_frames
ollos_read_screen
ollos_search
ollos_review
ollos_job
ollos_cancel

06Stack

Node.js
ONNX Runtime
Whisper
OCR
MCP

Visit github.com/kelvinbiffi/ollos-mcp

Next case MentalKare An accessible, multilingual mental health hub that connects professionals and patients.