Cohere’s Rerank 3.5 release changed both retrieval ranking and the API contract
Cohere released Rerank v3.5 and a v2 Rerank API on December 2, 2024, adding a 4,096-token context limit and explicit model selection.
Historical reporting: Cohere released Rerank v3.5 and a v2 Rerank API on December 2, 2024. Its release notes state a 4,096-token context length, required model selection, and replacement of max_chunks_per_doc with max_tokens_per_doc, defaulting to 4096. A maintained models overview is later context only. Cohere’s performance positioning is not treated as an independent relevance measurement.
Key takeaways
- Rerank v3.5 scored a query against candidate documents after retrieval, with a stated 4,096-token context limit.
- The v2 API made the model explicit and replaced a chunk-count control with a token-per-document control.
- Reranking can improve ordering only within the candidate set a first-stage retriever supplied.
The December event paired a model release with an interface change[source 1]
Cohere announced its new Rerank model and v2 Rerank API on December 2, 2024. The release notes gave the model a 4,096-token context length and said that the v2 endpoint requires a model parameter. They also replaced max_chunks_per_doc with max_tokens_per_doc, whose default was 4096.
This was more than a new name in a model catalog. An explicit model field makes model selection part of a caller’s contract, while a token-based document limit describes the input boundary differently from a count of chunks created elsewhere in an application. That change makes truncation policy visible at the integration boundary.
Reranking acts after candidate retrieval, not before it[source 1][source 2]
A typical retrieval path first gathers candidate passages using a lexical, vector, or hybrid index, then asks a reranker to order that smaller set against the query. The reranker can make a more detailed comparison than a broad index lookup, but it cannot return a relevant document that the first stage never supplied. Candidate recall and reranking precision are therefore linked but different things to evaluate.
Cohere’s maintained overview later lists Rerank v3.5 as a model option and records its 4K context. That current catalog is useful context, not proof that a specific December integration used the same endpoint, candidate limit, or operating policy.
Token limits make document segmentation part of ranking behavior[source 1]
When a candidate is longer than the model’s effective input budget, an application must decide what portion reaches the ranker. Segmenting earlier can preserve several local passages, while sending a broad document can retain global context but consume more of the 4,096-token allowance. The v2 move to max_tokens_per_doc exposes that choice at the request boundary. An application migrating an old chunk-count setting cannot reuse the same integer as a token limit without changing how much of each document is examined.
Neither a token limit nor a YAML-formatted semi-structured document guarantees the right ordering. A useful evaluation fixes the candidate generator, corpus version, query set, relevance labels, truncation rule, and ranking metric so a changed model or API parameter has a visible effect.
A reranker release is not an end-to-end search result[source 1][source 2]
Cohere’s release records a dated model and API evolution. It does not disclose a customer’s corpus, access rules, first-stage retriever, latency target, fallback behavior, or relevance labels. Those surrounding choices can outweigh a model change for a real search product.
The narrow historical conclusion is that Cohere introduced Rerank version 3.5 and the stated v2 API changes on December 2. It does not prove a universal relevance gain, a lower cost, or a compliant retrieval workflow for another organization.
Sources
Continue reading
Related reading
News analysisVoyage’s 3.5 embedding release put dimensions and quantization beside retrieval scores3 min read
News analysisGoogle made Gemini Embedding stable while keeping vector size a retrieval choice3 min read
News analysisCohere released Rerank 4 with Pro and Fast variants for multilingual and JSON document ranking3 min read