Whosaid MCP Server

whosaid

Local, speaker-attributed transcription for Apple Silicon — who said what, on your Mac, nothing leaves the machine.

whosaid pairs Whisper speech-to-text with speaker diarization to turn a meeting, interview, call, or podcast recording into a transcript that says who spoke when — running fully offline and on-device on an Apple Silicon Mac, with no cloud service, no API keys, and no Hugging Face token. Think of it as whisper + speaker labels + voice-based speaker recognition, in one command.

Example output (meeting.speakers.txt):

[00:00:04] Alice: Thanks for jumping on, I know it's late for you.
[00:00:11] SPEAKER_01: No problem at all, happy to make it work.
[00:00:19] Alice: Let's start with the roadmap for next quarter.
[00:00:27] SPEAKER_01: Sounds good, I've got a few updates on that.

Why whosaid

• Meetings, interviews, calls, podcasts — get a transcript where every turn is attributed to a person, not just a wall of text.
• Privacy by construction — audio, text, and voice embeddings never leave your Mac. The only cloud step is one you opt into: [summarizer] engine = "claude" sends each transcript's text to Anthropic to draft action items. Leave it unset and nothing is sent.
• Your name on your own lines — a one-time ~45s voice enrollment teaches whosaid your voice, so your turns read as your name instead of SPEAKER_00.
• Tells you how many people spoke — the number of distinct speakers is auto-detected and reported up front, with per-speaker turn counts and talk time.
• Speaker cards to identify who's who — for every speaker, whosaid writes a card of that voice's most representative snippets, so you can read a few lines and know who was talking.
• Remembers people you name — identify a speaker once with whosaid relabel, and their voiceprint is saved to a private local registry so they're auto-named in every future transcript.
• Fast on long recordings — recordings over ~15 min are diarized in parallel windows and stitched back into consistent speakers by voiceprint, so an 80-minute meeting is minutes, not tens of minutes, recovering the same speakers as a single-pass run.
• No accounts, no API keys, no Hugging Face token — every model comes from an open, ungated source.
• Search everything you have recorded. whosaid index builds a local full-text (and, with Ollama, meaning) index over every transcript in a meeting workspace; whosaid search finds the turn, whosaid context shows the minute around it, and whosaid graph answers who committed to what, when. See Search your meetings.
• Hands-free from Voice Memos. whosaid watch install puts a launchd agent on the Voice Memos folder (or any folder): stop recording, and a few minutes later the meeting is transcribed, summarized, and searchable. See Hands-free ingest.
• Usable from an AI agent, too. whosaid mcp exposes transcribe, relabel, doctor, and the read-only workspace search tools as MCP tools for Claude Code, Claude Desktop, Kiro, and other MCP clients, with the same local-only guarantee as the CLI.

How it compares

Looking for a MacWhisper, whisperX, or aTrain alternative? If you want speaker-attributed transcription that runs fully offline on Apple Silicon and can put real names on voices — not just SPEAKER_00 labels — that's the gap whosaid fills. Here's how it stacks up:

Quickstart

git clone https://github.com/sblattj/whosaid && cd whosaid   # get the code
./bootstrap.sh                                                # check deps, download models, install ~/.local/bin/whosaid
whosaid enroll                                                # ~45s reading a printed passage — teaches whosaid your voice
whosaid path/to/meeting.m4a                                   # transcribe + diarize + label -> meeting.speakers.txt (and friends)

If ~/.local/bin is not on your shell's PATH, add it or invoke the installed command by its absolute path. WHOSAID_INSTALL_DIR=/another/bin ./whosaid install selects another install directory. The installed command is a symlink to the checkout, so updating the checkout updates the command without copying or duplicating the implementation.

whosaid's helper scripts need Python 3.11 or newer, because whosaid.toml is read with tomllib. The launcher picks one interpreter: $WHOSAID_PYTHON if it is set, otherwise the first of python3, python3.13, python3.12 and python3.11 on your PATH that is 3.11+, otherwise uv python find '>=3.11'. It skips the stock macOS python3 (3.9) automatically. If no interpreter qualifies, commands that run Python stop with an error; uv python install 3.13 fixes that. whosaid doctor shows which interpreter was picked.

Homebrew is optional when uv, ffmpeg, and ffprobe are already installed. The launcher and watcher search ~/.local/bin, ~/bin, and ~/homebrew/bin as well as the usual system prefixes. Setup can install a pinned, checksum-verified Apple Silicon uv archive into ~/.local/bin. Homebrew FFmpeg installation requires a bottle; a nonstandard prefix will not silently trigger a source build.

For a Mac without Homebrew, or one where no bottle is available, use:

./bootstrap.sh --build-ffmpeg

This builds pinned FFmpeg source into user-local ffmpeg and ffprobe executables. It needs Apple's Command Line Tools (xcode-select --install if missing), compilation time, and free disk space, but no administrator-owned install prefix or GPG installation. The source archive is checked with macOS shasum against a checksum verified from FFmpeg's signed release. Existing usable tools are retained. Setup then warms the Python environments and downloads the models as usual.

Commands

Record a browser meeting tab (no permissions)

whosaid record --app needs Screen & System Audio Recording permission for your terminal. A browser meeting can skip that. Start a Chromium-family browser (Chrome, Vivaldi, Edge) with a debugging port, join the meeting in it, then record that one tab:

# once, when you launch the browser
open -a "Google Chrome" --args --remote-debugging-port=9222

whosaid record --browser                      # port 9222, or $WHOSAID_CDP_PORT
whosaid record --browser --cdp-port 9333 --target teams.microsoft.com

• Only that tab is recorded. Left channel is your mic, right channel is the meeting audio. No system sounds, no notifications.
• --target is a targetId or a piece of the tab's URL. You need it when more than one tab could match. The error lists the candidates.
• It stops on its own when the call ends. It arms once the hang-up button has been seen, then stops after 2 missed polls. If the button is never seen, use Ctrl-C or --max-seconds (default 10800). --leave-selector CSS overrides the built-in Teams, Meet and Zoom web selectors. --leave clicks hang-up at --max-seconds.
• If the tab is closed mid-call, whatever was already captured is kept.
• --into DIR writes DIR/<timestamp>[-label].m4a and skips transcription. The file appears only when it is complete, so point it at a whosaid watch source folder and the watcher ingests it.
• From an agent, use the whosaid_record_browser MCP tool with action set to start, status or stop. It runs in the background and needs no terminal.

Record every call, unattended: whosaid autorecord

whosaid autorecord waits for a call, records it, saves it, and waits for the next one. Pair it with whosaid watch and every call ends up transcribed in your workspace with no clicks.

# desktop meeting app (here native Microsoft Teams); needs Screen & System Audio Recording
whosaid autorecord install --app com.microsoft.teams2 --label teams

# or meetings in the browser; needs no permission and no admin rights
whosaid autorecord install --browser --label teams

# transcribe each saved call
whosaid watch install --into ~/meetings --source ~/Recordings --no-fda

whosaid autorecord status        # installed? running? recent log lines (--json for scripts)
whosaid autorecord pause --minutes 60   # skip calls for an hour; `resume` undoes it
whosaid autorecord stop-now      # end the call being recorded now and save it
whosaid autorecord uninstall     # a call in progress is saved first

A new Mac in one command. After git clone, whosaid autorecord setup runs whosaid setup --yes (dependencies, models), autorecord install, then watch install, stops at the first failed step, and lists what is left to do by hand (the permission toggle, whosaid enroll). --owner starts the workspace's whosaid.toml with your name; an existing one is left as is. Try it with --dry-run first.

git clone https://github.com/sblattj/whosaid ~/code/whosaid && cd ~/code/whosaid
./whosaid autorecord setup --app com.microsoft.teams2 --label teams \
  --workspace ~/meetings --owner Alice_Example   # or --browser instead of --app

• When it records. --app watches whether the app has the microphone open: it starts when the app opens the mic and stops 30 s (--idle-seconds) after it closes it. --browser watches meeting tabs (Teams web, Meet, Zoom web) for their hang-up button. Neither reads the screen.
• Login. install starts it at every login. --app builds a small launcher, ~/Applications/Whosaid Call Recorder.app, and opens System Settings so you can turn it on under Screen & System Audio Recording. That permission belongs to the launcher, which is only rebuilt when whosaid changes it, so re-running install with other flags keeps it.
• No admin rights? If that toggle asks for an admin password, use --browser with the meeting on the web. Start the browser with --remote-debugging-port=9222 (autorecord never launches or changes your browser). An organization policy can turn remote debugging off.
• Robust by default. One copy per folder. An app that isn't running or a browser that isn't open yet is a waiting state; other failures back off to 5 minutes and keep trying. Log: ~/Recordings/.autorecord.log.
• Live with every call (opt-in). Add --live to install or setup and each capture also runs whosaid live, which starts when the capture starts and stops when it is saved, so the app's Live now and the whosaid_live tool follow every call with no extra step. Its log: ~/Recordings/.autorecord/live.log. If that transcriber dies mid-call, autorecord logs live transcript failed (exit N), restarts it up to 3 times with growing delays (5 s, 15 s, 45 s), and autorecord status (and --json: live_state, live_exit, live_restarts) and the menu bar show it, while the recording carries on. The app's Recorder page shows whether it is on and the command to turn it on.
• Environment for the agent. launchd starts the job with a bare environment, so install and setup copy the certificate variables from the shell you run them in (SSL_CERT_FILE, SSL_CERT_DIR, REQUESTS_CA_BUNDLE, CURL_CA_BUNDLE, UV_SYSTEM_CERTS, UV_NATIVE_TLS) plus WHOSAID_BIN and WHOSAID_ACTION_ITEMS_HOOK into the LaunchAgent, same as watch install. --env KEY=VALUE (repeatable) adds more and wins over the copied ones; values you set earlier survive a reinstall. --offline adds HF_HUB_OFFLINE=1 (plus TRANSFORMERS_OFFLINE, telemetry off) so nothing round-trips to huggingface.co. Check with whosaid autorecord install --dry-run.
• whosaid autorecord run is the same loop in the foreground, for a terminal tab.

Follow the call as it happens: whosaid live

While record --app or autorecord is capturing a call, whosaid live transcribes it as it goes, on this Mac, and keeps a rough transcript you or an agent can read before the call ends.

whosaid live                 # follow each call being recorded; Ctrl-C to quit
whosaid live show --minutes 5   # the last five minutes of the newest live transcript

• Me vs them, no diarization. The recording keeps your mic and the meeting app apart, so each line is me: or them: with the wall-clock time. Your mic also hears the other side through your speakers; mic audio that isn't clearly louder than the app is ignored, so their words aren't repeated as yours.
• Where it goes. ~/Recordings/.live/<start>.md and .jsonl, updated every ~20 s (--chunk-seconds). Silent stretches are skipped. Model: whisper-large-v3-turbo (or WHOSAID_LIVE_MODEL).
• Ask about the meeting. The whosaid_live MCP tool returns the live transcript (or its last N minutes) and whether the call is still going, so an agent can answer "what did they just ask me?" mid-call. whosaid_live_insights adds the call's topics, rolling summaries and agreed or suggested next steps (the Live page's data), and whosaid_live_ask finds the lines that match a question, with their times. whosaid itself sends nothing anywhere; the agent you ask sends the text it reads to its own model.
• Topics and next steps. whosaid live insights follow turns the newest live transcript into topic buckets, wordgraph nodes, 5-minute summaries and agreed / suggested next steps in <start>.insights.json, locally (live insights build --jsonl F for one file). whosaid live ask --question "..." answers from the live transcript: a search by default, Claude only when the workspace opts in with [summarizer] engine = "claude".
• AI labels and recap (opt-in, CLOUD). With [live] ai = "claude" in <ws>/whosaid.toml, whosaid live insights follow --ws <ws> (the app passes it) also asks Claude (claude -p, Sonnet) for topic labels, a short recap and next steps once per ai_every seconds of talk (default 180), at most ai_max_calls (40) per call, ai_timeout 90 s and ai_max_tokens 1500 per call. Each call sends the newest ~24k characters of the transcript plus the local topics to Anthropic, about 9k tokens. Answers land in <start>.ai.json and are laid over the local insights; every topic keeps its localLabel, and a failed call keeps the last good answer. Remove the line (or set ai = "") to turn it off.
• It only reads the recording in progress. The full transcript with names and action items still comes from whosaid watch after the call.

One status feed for apps and scripts: whosaid app-state

whosaid app-state --json --workspace ~/meetings   # one snapshot
whosaid app-state --watch 2 --workspace ~/meetings  # a JSON line now and on every change

It joins autorecord status --json, the watcher's state for the workspace (the same rules as the SwiftBar plugin), live status --json, the newest meeting and the count of unnamed voices from the last two weeks into one object, plus a single menubar.state (warn, agent-not-loaded, recording-live, recording, paused, ingesting, waiting, done, idle). It is read-only. whosaid tracking set-status <ws> CM-012 done --note "..." is the matching write path for marking an item done (see docs/tracking.md).

A native macOS app: whosaid app

A menu bar app plus a library window: meetings, live now, action items, your commitments, topics, chats, people, unnamed voices and the recorder, in one vertical sidebar. It is SwiftUI, built with the Command Line Tools only (no Xcode needed), and it reads whosaid the way an agent does: whosaid mcp for the library and one whosaid app-state --watch for the menu bar.

whosaid app sign-setup  # once per Mac: a local signing certificate, so updates keep Full Disk Access
whosaid app install     # swift build -c release, then ~/Applications/whosaid.app
whosaid app open
whosaid app test        # swift test for WhosaidKit
whosaid app uninstall

The app never records and never asks for Screen & System Audio Recording: Whosaid Call Recorder.app keeps that grant. Pick the workspace and the recordings folder in its Settings (or set WHOSAID_WORKSPACE / WHOSAID_RECORDINGS). With no workspace, or one that is not indexed yet, the window says what to do next. Settings also has "Open at login". Needs macOS 15 and Swift 6.

The menu bar item shows one glyph per state, from whosaid app-state: a waveform while idle, a red record dot with the elapsed time while a call is recorded (plus a pencil when whosaid live is transcribing it), gears while the last call is transcribed, a check just after, a pause sign while autorecord is paused and a warning triangle when something needs a look. Its panel has Stop and save now, Pause autorecord for 1 hour, the last three meetings, Name N unknown voices, search and Settings.

The library window lists meetings and Teams chats newest first (a call is named by what recorded it, a chat by its chat name). A meeting opens on its transcript, colored per speaker under a talk-time bar, with Summary, Action items and Speakers tabs. ⌘K searches every meeting (whosaid_search) and shows the minute around a hit (whosaid_context). People lists every named voice with what they own and asked for (whosaid_person); Unnamed voices lists the ones still called SPEAKER_NN. If the workspace sits in ~/Desktop or ~/Documents, macOS asks once whether whosaid may read that folder.

A meeting's Speakers tab has one card per voice: turns and talk time, a name field, role chips (self, boss, peer, report, external) and Save, which runs whosaid relabel and then whosaid index so the name shows up everywhere. An unnamed voice shows the closest enrolled voice that just missed the match ("Sounds like Carol (0.64)"), one click from the name field. Play clip cuts a few seconds of that voice with whosaid samples into ~/Library/Caches/whosaid/clips and plays it; the app plays nothing any other way.

My commitments is the ranked worklist for the workspace owner (whosaid_worklist): P1 to P3 chips, why each ranks where it does, and a link that opens the meeting at the moment it was said. Above it, "Looks done?" lists later statements that may show an item is finished (whosaid_tracking_suggestions), with the quote and why it matched. Nothing changes until you click Mark done, which records one status event with that quote as evidence (whosaid_set_status). Action items lists every item from every meeting, with the same Open / Done / All filter.

Live now follows the newest whosaid live session in five lanes: the me / them transcript, topic buckets, a wordgraph, 5-minute summaries and next steps (agreed vs suggested). Topics are buckets: when the talk comes back to a subject, its bucket is marked "back 2×" instead of starting a new one, and a fresh one is marked "new". The wordgraph is the balloon graph from jibbr (topic, idea, detail by color; dashed lines join words shared by two topics): drag and throw the balloons, click one to fold its branch, or switch it to full window. The lanes come from whosaid live insights follow, a small local engine the app starts only while a call is live; it writes <start>.insights.json next to the live transcript. ⌘L opens a narrow floating panel to keep next to the call, with the last lines, the topics and "Ask about this call" (whosaid live ask): it searches the live transcript, unless the workspace sets [summarizer] engine = "claude", which sends the transcript to Claude for the answer. With [live] ai = "claude" the topic labels, a "So far (AI)" recap and the next steps come from Claude instead (see Live above). The wordgraph page is bundled in the app and can load only its own files: no fetches, and any other navigation is cancelled.

Recorder shows what autorecord is doing (waiting, recording, paused), what it records and into which folder, whether the live transcript and the watcher are on, the last saved call or error and the recent log, with the same Stop and save now, Pause and Resume as the menu bar. It never installs anything itself: when the live transcript is off it shows the whosaid autorecord install ... --live command to copy.

Topics rolls up what each meeting was about, across meetings, from <workspace>/_topics.json. Topics are ranked by how many meetings they came up in, then by how recently. A topic shows its terms, its own wordgraph and the meetings it came up in, each opening its transcript where the topic starts. Rebuild runs whosaid topics build.

Topics across meetings: whosaid topics build

whosaid topics build ~/meetings          # writes ~/meetings/_topics.json
whosaid topics build ~/meetings --chats  # Teams chats and mail too

It runs the same local topic clustering as whosaid live insights over each meeting's speaker transcript, then merges topics across meetings: same label, or terms that match closely enough. A topic keeps its id as meetings are added, so links to it stay put. Each topic lists its meetings (date, the label it had there, talk time, where it starts), its top terms and a wordgraph. Speaker names are not written to the file. Stdlib only: no model, no network.

A drafts outbox for agent-written chat messages: whosaid drafts

When several AI agents draft chat messages for you, they fight over the one browser tab, you lose track of which session wrote which draft, and a draft goes stale as the work moves. whosaid drafts keeps them offline until you stage them, one folder per draft under ~/.drafts/<id>/ (WHOSAID_DRAFTS_DIR overrides): draft.md, claims.json and draft.json.

whosaid drafts new --to "Test Person" --topic "dns records" --page ~/pages/dns-plan.html
whosaid drafts stakeholders ~/meetings --topic "dns records" --repo ~/code/infra --path dns/
whosaid drafts claims <id> --file claims.json   # claims, stakeholders, docs
whosaid drafts text <id> --file draft.md
whosaid drafts verify <id>
whosaid drafts lint --all                       # the voice lint alone, read-only
whosaid drafts list                             # state and the session that wrote each one
whosaid drafts stale-check ~/meetings           # stakeholders said something new since verify?
whosaid drafts stage <id>                       # you run this; nothing is sent
whosaid drafts mark-sent --check ~/meetings     # staged drafts you have since sent
whosaid drafts html                             # review page of every open draft

• Who wrote it. new records the Claude Code session (CLAUDE_CODE_SESSION_ID, joined to the session registry for its name, cwd and pid) and a resume command. show prints it.

• Claims first. Each claim in claims.json has a kind (code, tech, people, status), a verdict (confirmed, refuted, unverified) and evidence. verify refuses a draft with an unchecked claim, a refuted claim not marked dropped, a confirmed claim without evidence, a confirmed tech claim without a doc URL, no stakeholders, a stakeholder with no searched topics, or an em dash. It stamps a hash of the message, claims and pages; any later edit reads as stale.

• Voice lint. verify also checks the message shape with lib/voice_lint.py (plain rules, no model). Shape rules (more than one paragraph, a greeting opener like Hi Sam,, an attached page or Thanks! closer) fail verify when the workspace's whosaid.toml says so; the rest (over 47 words, "want me to" asks, banned words and characters) only print warnings. The workspace is --ws, else WHOSAID_WORKSPACE; with none, the default warn applies.

  [drafts]
  voice_lint = "enforce"   # enforce | warn (default: every hit is a warning) | off
  [drafts.voice]           # optional: voice_lint.py profile keys over the default
  banned_words = ["synergy"]
  

  A real exception gets a per-draft override with a reason, kept in draft.json and its history: whosaid drafts verify <id> --voice warn --reason "..." (--voice clear drops it). lint [<id>|--all] prints what verify would say without stamping anything (exit 1 if one would fail).

• Who has opinions. stakeholders ranks speakers by search rank on the topics with a 90-day recency half-life (unnamed SPEAKER_NN voices skipped), plus who changed --path in --repo over the last six months. Read what the top people said before claiming what they want.

• Staging last. stage takes ~/.drafts/.outbox.lock, refuses stale drafts and a second open draft to the same person, then runs the stager (the built-in one, or WHOSAID_DRAFTS_STAGER, see below) with --chat NAME --md FILE [--attach PAGE ...], plus --replace --only-these-attachments on a restage, and TEAMS_OUTBOX_LOCK set. One draft per run with --all. mark-sent and drop close a draft (a soft delete: list --all still shows it).

• The built-in stager. When WHOSAID_DRAFTS_STAGER is unset, stage uses lib/drafts_stager.py. It needs a Chromium browser started with --remote-debugging-port (the port comes from --cdp-port / WHOSAID_CDP_PORT, default 9222) with Teams web signed in. It opens the chat by exact name (left rail, else search; --new-chat starts one via the To picker), pastes the draft as rich text, attaches the pages, and with --replace clears the box (--only-these-attachments also removes the attachments already there). Each --mention NAME becomes a real @mention at the start of the message, in order, before the body; a literal @NAME in the text is dropped so it is not doubled. It never sends (no Enter, no click on Send), prints a JSON line and leaves the chat focused. Set WHOSAID_DRAFTS_STAGER to your own command to stage into another chat app.

• One owner at a time. Changing a draft another live session owns is refused without --force (or force=true over MCP): ask that session instead. A draft whose session is gone is adopted by whoever changes it next.

• One draft per person. new to someone who already has an open draft joins it: same id, you become a contributor who may write it, and your text becomes a new revision. Merge your points in instead of starting a rival message. --new makes a separate draft anyway. --mention NAME records who to @mention; it is passed to the stager as --mention.

• Every revision kept. Each text change is saved under revisions/ with who wrote it. show <id> --rev N prints one, diff <id> [--from N --to M] compares two.

• Reads the chat first. With a workspace of ingested chats (whosaid teams ingest), stage prints the last messages with that person and skips a draft that already appears there as your message. mark-sent --check marks staged drafts sent the same way, so the outbox closes itself after you send.

Agents use the same outbox over MCP: whosaid_draft_new, whosaid_draft_write, whosaid_draft_verify, whosaid_draft_close, whosaid_drafts, whosaid_draft_stakeholders, whosaid_draft_stale, whosaid_draft_sent_check: create, read (with rev and diff), update and close. whosaid_draft_verify takes voice and reason for the override and returns voice_warnings; whosaid_drafts(action="lint") is the read-only lint. There is no stage or send tool.

Enroll from an existing recording

Already have a clip of the person speaking? Skip the mic and cut a reference straight from it:

whosaid enroll Alice --from meeting.m4a --ss 3:20 --t 20   # or --to 3:40

Meeting workspaces

One-off transcriptions are files; a recurring meeting series is a corpus. The meeting-workspace layer gives that corpus a home: every recording is transcribed into a dated folder, each meeting can carry generated action items and dev commitments, and one roll-up produces the index, the audit, a living action-item list, and the dev-commitments corpus across all of them — still entirely offline.

whosaid ingest — a batch into dated folders

whosaid ingest weekly/*.m4a --into ./meetings --folder-by created \
  --tz America/Los_Angeles --action-items

Every file is transcribed with the full set of whosaid <audio>… flags passed through, into meetings/YYYY-MM-DD-HHMM/. The timestamp comes from the recording's own container creation_time rendered in --tz (default UTC), falling back to the file's mtime — so folder order reflects when meetings actually happened, not when you got around to copying the files. Ingest is idempotent by source sha256: re-running a batch never re-transcribes or duplicates a recording.

Documentation truncated — see the full README on GitHub.