Skip to content

Model releases

Cohere Embed 5 splits indexing and search across two models

5 min read

Written by an AI model from the linked sources and checked against them by a second model. How we work

Image: thenewstack.io

Cohere’s own tests covered 40 datasets. Querying an Embed 5 Pro index with the cheaper Embed 5 Fast model scored 98.4 in them, against 100 when Pro handled both indexing and queries. That figure is the case Cohere makes for Embed 5, a two-model embedding family it released on September 30.

The setup, as The New Stack describes it: documents go into the index through Pro, and queries hit those same vectors through Fast, with no second index. Cohere recommends this split for RAG and agent workloads, where the same data gets searched over and over and latency adds up with each search. Running Fast for both indexing and queries scored 96.6 on the same suite. So the mixed setup gives up less than half of what an all-Fast pipeline loses.

One space, two models

Until now a team picked one embedding model and used it for everything, so index quality and query speed pulled against each other. Embed 5 splits that choice in two. Cohere’s changelog lists embed-v5.0-pro as optimized for the highest retrieval quality, particularly offline indexing and quality-critical retrieval. It lists embed-v5.0-fast for low latency and high throughput: interactive search, agent loops and high-volume query traffic.

It works because the two models share an embedding space. Both turn an input into a vector in the same coordinate system, so a query vector from Fast can be compared directly with document vectors from Pro. Both produce compatible vectors at the same dimensions. Cohere says the two can still be mixed with Matryoshka truncation, where a vector is cut down to its first few hundred numbers and stays usable, and with int8 quantization. The models can also be swapped in either direction without re-embedding the corpus, and re-embedding is normally the expensive part of changing embedders.

Pro costs $0.12 per million tokens and Fast $0.08. Image tokens cost $0.40 per million on both. In Cohere’s tests Fast delivered an average of 2.4 times Pro’s document throughput. In a RAG system that ingests documents less often than it searches them, the split puts Pro on documents entering the index and Fast on the much heavier query traffic.

What the 98.4 measures

The 40 datasets behind the cross-model figure cover text, images, fused text-image documents and parsed documents. Cohere says none of them showed a major drop when Pro and Fast were used together. These cross-model tests use standard nDCG@10, a score for how well the ten top-ranked results are ordered by relevance.

Other numbers in the release use a different method. Embed 5 is Cohere’s first model family evaluated with RCP-nDCG@10, which judges results against relevance criteria written for each query instead of fixed labels. Cohere says this catches relevant results the original benchmark labels missed. The New Stack points out that RCP-nDCG@10 measures reranking over a fixed set of candidates, not first-stage retrieval from the full corpus. Cohere evaluates first-stage retrieval separately with standard nDCG and Recall. The fused text-image, page-image and cross-model tests use standard nDCG@10, so scores from different evaluations are not directly comparable.

The 98.4 is also an average over Cohere’s own evaluation suite, a point The New Stack makes too. It does not come from customer corpora.

How small can the vectors get?

Both models output six dimensions: 256, 512, 768, 1,024, 1,536 and 2,048, each in float32, int8 or binary. Cohere’s worked example puts a 2,048-dimensional float32 vector at 8 KB, or roughly 819 GB for 100 million chunks. At 1,024 dimensions in int8, the same corpus takes about 102 GB, roughly an eighth of the full-precision footprint. At 256 dimensions in binary, it shrinks to about 3.2 GB.

For most deployments Cohere recommends 1,024-dimensional int8, which cuts memory and storage while keeping retrieval quality close to full precision. Binary vectors compress further but lose some accuracy. Cohere positions them for a first retrieval pass, with a higher-precision reranking step after it.

Beyond plain text

Embed 5 handles text, images and fused text-image inputs in more than 100 languages, with a 128K-token context window. It can embed page images directly or combine image and text into a single vector.

Here are Cohere’s comparisons, each on its own evaluation:

  • Fused text-image, five datasets: Pro 82.3, Fast 81.2, Google’s Gemini Embedding 2 61.3.
  • Parsed PDFs: Pro 84.8, Voyage 4 Large 83.6, Fast 83.4, Gemini Embedding 2 80.8.
  • ViDoRe V3, run on parsed text outputs curated by the benchmark’s authors instead of page images: Pro 85.8, Fast 84.5, Voyage 4 Large 83.7, Gemini Embedding 2 83.2, Cohere’s previous Embed 4 77.
  • Financial documents: Pro 80.1 on FinanceBench and 90.0 on FinQA.

Fast lands just behind Voyage 4 Large on parsed PDFs and just ahead of it on ViDoRe V3. Cohere’s changelog also claims major gains over Embed 4 on code, financial filings and multilingual retrieval.

The multilingual results are the least one-sided, even after Cohere’s recent push into machine translation. Pro leads the five-language European average with 77, against 76 for Voyage 4 Large and 73 for Gemini Embedding 2. It trails Gemini Embedding 2 on nine of the individual tests.

Where it runs

Both models are available through Cohere’s API, Microsoft Foundry and Amazon SageMaker, and through Model Vault, Cohere’s single-tenant option. Private VPC and on-premises deployment run through vLLM.

The New Stack says production RAG and agent systems still need to test Pro-to-Fast against Pro-to-Pro on their own corpus and query distribution. That matters most where a retrieval error can carry through several steps of an agent workflow. The comparison takes two configurations over one index: embed-v5.0-pro on both sides, then embed-v5.0-fast on the query side only.

No comments yet. Be the first.

Join the conversation

Sign in to leave a comment. It appears after moderation.