Direct answer — What is semantic search for transcripts? Semantic search finds passages in a transcript archive by meaning rather than by exact wording. It converts your query and every stored passage into embeddings — numerical vectors that place similar meanings close together — then ranks passages by proximity. Describe what was said, such as “the part about the funding delay”, and it surfaces the passage even when nobody used those words.
If you have ever tried to search transcription history on a Mac and failed to find an interview you are certain you recorded, the archive is rarely at fault. The query is. Keyword search matches strings; memory stores meaning, and the two only occasionally coincide.
That gap is why semantic search transcripts separates a folder of text files from an archive you can use. This guide covers what semantic search does, why it fails on its own, how hybrid ranking fixes it, and what it takes to find a transcript by meaning entirely on your own Mac.
Why does keyword search fail on a transcript archive?
Because a transcript records speech, and speech is the most paraphrase-heavy text you will ever store. You remember the point that was made; the speaker remembered nothing and simply talked.
Three properties of spoken material make lexical matching brittle:
- Nobody says it twice the same way. A budget overrun becomes “we went a bit over”, “the numbers slipped” or “it cost more than we said”, in one conversation.
- There are no headings, no title, no summary. An article gives a retrieval system structure to lean on. A ninety-minute interview gives it a wall of sentences.
- Proper nouns arrive unspelt. A company name heard once may be transcribed three ways across an archive, and each one fails an exact-match query.
Retrieval research quantifies the gap. On the development set of the MIRACL benchmark — human-annotated retrieval over Wikipedia passages, averaged across 16 languages — BM25, the classic lexical ranking function, scores 39.3 nDCG@10 against 60.8 for the multilingual-e5-small embedding model.
| MIRACL dev set, averaged over 16 languages | BM25 (lexical) | multilingual-e5-small (semantic) |
|---|---|---|
| nDCG@10 | 39.3 | 60.8 |
| Recall@100 | 78.7 | 92.4 |
Two caveats matter. MIRACL is Wikipedia prose, not spoken transcripts, so the absolute figures do not transfer to your interviews. And BM25’s Recall@100 of 78.7 shows lexical search still finds most relevant material — it simply ranks it badly. That is the argument for hybrid search rather than replacement.
What is semantic search, and how does it actually work?
Semantic search is retrieval based on vector similarity rather than string matching. A model reads a passage and outputs an embedding: a fixed-length list of numbers positioned so that texts with similar meanings land near each other in that space.
Your query goes through the same model, so ranking becomes a geometry problem — which stored vectors sit closest to the query vector — rather than a text problem.
These models are trained at scale. The multilingual E5 technical report describes contrastive pre-training on one billion multilingual text pairs, then supervised fine-tuning on labelled datasets, in three sizes: small, base and large. MIRACL, the multilingual retrieval benchmark used in the table above, covers 18 languages and carries over 700,000 human relevance judgements for roughly 77,000 queries, all assessed by native speakers.
Can a query in one language find a transcript in another?
Yes, provided the embedding model is multilingual and both languages sit inside it. The capability falls straight out of how the model was trained.
A multilingual embedding model places a sentence and its translation close together in the same vector space. The multilingual E5 report evaluates exactly this under the name bitext mining, defined as “a cross-lingual similarity search task that requires the matching of two sentences with little lexical overlap”, across more than 100 languages.
In practice, on a cross-lingual transcript search, a query typed in French can rank a passage spoken in Japanese or Russian. No translation step runs and no keyword ever matches — the two texts simply mean the same thing, and the model was built to notice. For anyone keeping an archive in several languages, that turns one search per language into a single query.
Why does hybrid search beat either method alone?
Because the two fail in opposite directions, and you cannot predict which kind of question you are about to ask. Exact quotes, surnames and reference numbers favour the lexical index; concepts and half-remembered arguments favour the vector index.
The established way to merge two ranked lists is Reciprocal Rank Fusion (RRF), published at SIGIR 2009. Each document scores 1 / (k + rank) in every list it appears in, the scores are summed, and the fused list is sorted. The authors fixed k = 60 during a pilot and never altered it.
Their results, on Text REtrieval Conference collections and on the LETOR 3 learning-to-rank corpus of 583,850 document-query pairs:
- RRF’s mean average precision exceeded Condorcet Fuse in all cases and CombMNZ in all but one.
- RRF outperformed Condorcet, CombMNZ and the best individual system by 4% to 5% on average.
- On LETOR 3, RRF reached a MAP of 0.6051 against 0.5846 for the best individual ranking, bettering all of them at p < .003.
What makes RRF attractive for a desktop app is what it does not need: no training examples, no tuning, and no comparable scores between the two systems. Ranks are enough.
| You are looking for | Keyword (FTS5) | Semantic (embeddings) | Hybrid (RRF) |
|---|---|---|---|
| An exact quote, word for word | ✅ Strongest | ⚠️ May rank paraphrases above it | ✅ |
| A surname, product name or acronym | ✅ Strongest | ⚠️ Weak on rare tokens | ✅ |
| A topic you can only describe | ❌ Misses the paraphrase | ✅ Strongest | ✅ |
| A passage spoken in another language | ❌ | ✅ Strongest | ✅ |
| A figure or date read aloud | ✅ | ⚠️ Weak | ✅ |
How does an AI transcript search run entirely on a Mac?
With three local components and no network call. This is what decides whether your interviews stay confidential, so each piece is worth naming.
Weesper Transcribe builds its searchable transcription history from:
- A keyword index — SQLite FTS5, which the official documentation defines as “an SQLite virtual table module that provides full-text search functionality to database applications”. It ships a built-in BM25 ranking function and supports phrase and prefix queries, so
"quarterly forecast"andforecast*both work. - A local embedding model — multilingual-e5-small, run through Candle, described in its official repository as “a minimalist ML framework for Rust with a focus on performance (including GPU support) and ease of use”. The small variant fits a laptop’s memory and battery budget while keeping multilingual coverage.
- Reciprocal Rank Fusion to merge the two ranked lists into the single list you see.
Everything above runs on your Mac. Transcription is already on-device, and the search layer adds no exception: embeddings are computed, stored and queried locally. An interview under embargo never becomes a request to a third party.
Where this sits in the current landscape
General-purpose on-device semantic search tools for macOS do exist — they index your files, notes or clipboard history, several with local models. What they do not do is own the transcription step.
That matters more than it sounds. A transcript-aware search knows about timestamps and per-file boundaries, so a hit takes you to the moment in the audio rather than to a paragraph in a text file — and the same archive feeds the local automation API when you script around it.
How do you search a transcript archive well?
Match the query style to the index you want to win. Four habits cover most cases.
- Describe the idea, not the sentence. “The bit where they refused to give a date” beats guessing the exact verb — that is what the vector index is for.
- Quote when you are certain. For a phrase you remember word for word, or a surname, type it exactly: FTS5 phrase matching does the work and the fusion keeps the hit near the top.
- Search in your own working language. Cross-lingual retrieval happens at search time, so the archive need not be in the language you think in, and translating at transcription time buys nothing.
- Keep the archive whole. Retrieval quality scales with what is indexed. Export a copy in SRT, VTT or one of the other formats, but leave the transcript in the history.
Search across the history is part of the Pro upgrade, alongside batch processing, the export formats and inline editing. The free download transcribes files up to 15 minutes at full quality, enough to check accuracy on your own audio first. Because Pro is a one-time purchase rather than a subscription, an archive you build this year stays searchable without a recurring fee.
Weighing apps rather than techniques? Our comparison with TranscribeNext covers the meeting-capture side of the same decision.
Frequently asked questions
How is semantic search different from keyword search in a transcription app?
Keyword search matches the characters you typed against the characters in the transcript. If the speaker said “we pushed the launch back” and you search for “delay”, a pure keyword index returns nothing. Semantic search converts query and passages into vectors that place similar meanings close together, then ranks by proximity — so it finds the paraphrase. For rare strings such as a surname, the keyword index stays more reliable.
Can I search a transcript in one language with a query in another?
Yes, when the embedding model is multilingual. It maps a sentence and its translation to nearby vectors, so a query typed in French can rank a passage spoken in Japanese or Russian. The multilingual E5 technical report calls this bitext mining — “a cross-lingual similarity search task that requires the matching of two sentences with little lexical overlap” — and evaluates it in over 100 languages.
Does semantic search work offline, or does it send my transcripts to a server?
It depends on where the embedding model runs. Cloud services compute embeddings on their own infrastructure, so your transcripts and your questions about them sit on someone else’s servers. Weesper Transcribe runs the model on your Mac through Candle, next to a local SQLite index — no query leaves the machine, and the archive stays searchable with the Wi-Fi off.
Why combine keyword and semantic search instead of picking one?
Because each fails where the other succeeds. Reciprocal Rank Fusion, published at SIGIR 2009, merges two ranked lists by scoring each result 1/(k + rank) and summing, with k fixed at 60. Its authors report that RRF outperformed Condorcet, CombMNZ and the best individual system by 4% to 5% on average. It needs no training data and no comparable scores, which suits a desktop app.
Do I need Weesper Transcribe Pro to search my transcript history?
Yes. The app is free to download from the Mac App Store and transcribes files up to 15 minutes at full quality. Full-text and semantic search across the history come with the Pro upgrade, together with the removal of the length limit, batch processing, the nine export formats and inline editing. Pro is a one-time App Store purchase, not a subscription.
Does semantic search find exact quotes reliably?
Not on its own. Vector similarity rewards passages that mean the same thing, so a close paraphrase can outrank the literal sentence. Exact quotes belong to the keyword index: FTS5 supports phrase queries and prefix queries and ranks with BM25. Hybrid ranking means you never have to choose — both hits are fused into one list.
Conclusion
Keyword search asks you to remember the words. After a hundred hours of recordings, nobody does. Semantic search asks you to remember the meaning, which is what memory actually stores — and the published benchmarks agree that meaning-based retrieval ranks far better on multilingual material, while lexical matching stays unbeatable on exact strings.
So the answer is neither alone but both, fused by a method that has held up since 2009. Run locally, that turns a pile of transcripts into an archive you can interrogate in any language you speak, without a byte leaving your Mac.
Ready to make your recordings findable? See how the searchable transcript archive works, or get the app from the Mac App Store — free to download, macOS 13 or later, one-time Pro upgrade, no subscription. Setup questions are answered in the support documentation.