NVIDIA AI Releases Nemotron 3 Embed: An Open Embedding Collection Whose 8B Checkpoint Ranks #1 on RTEB

Embedding models decide which routes an agent will look at. nvidia released Nemotron 3 embed model To work on that layer. It targets production-scale RAG, agentive recovery, code recovery, and agent memory.

What is Nemotron 3 Embed?

The model collection includes three open stools. Nemotron-3-Embed-8B-BF16 Accuracy-first option. Nemotron-3-Embed-1B-BF16 Carries the same design but in a smaller footprint. Nemotron-3-Embed-1B-NVFP4 Blackwell-optimized 4-bit path.

These three transformers are trained with encoder hide bidirectional attention. The final embedding comes from average pooling On token-level representations. The maximum sequence length at each checkpoint is 32,768 tokens.

Each model was evaluated in 34 languages. all three take OpenMDW License Agreement, Version 1.1 (OpenMDW-1.1). Specifically, there are base Mistral models. Made with 8B Ministral-3-8B-Instruct-2512. Both use the 1B variant Ministral-3-3B-Instruct-2512.

Display

Nemotron-3-Embed-8B-BF16 rank #1 overall on RTEB (as of July 17, 2026),Retrieval Embedding Benchmark. The assessment includes 16 of its public works. Each figure below is the average NDCG@10 on model sequence length 4096.

Two shortcomings are worth noting. 1B received 10.4 RTEB points llama-nemotron-embed-vl-1b-v2Baseline of the previous generation. Separately, NVFP4 costs 0.38 RTEB points or 99.5% retention versus its BF16 parent.

How the 1B model was created?

Those 1B scores come from a compression pipeline, not a short training run. were guardians nemotron-3-embed-3bTrimmed and distilled in two iterative rounds.

First, the 3B parent was truncated to 2B NVIDIA ModelOpt mcore_minitron Neural Architecture Search (NAS). The search includes hidden width, FFN size, attention head and depth. It then selects the best candidate from the top-10 Pareto front. A 50k in-domain calibration corpus scored those candidates.

Subsequently, the 2B model was distilled from the refined 8B embedding teacher. distillation joint Cosine distance loss (COS) And Mean Square Error (MSE) Loss. The data mix was multilingual and in-domain. Finally, the same process was repeated to create the 1.14b checkpoint.

NVFP4 serving tradeoff

Compression then continues into the serving format. Quantization only affects the weights and activations of the linear layers, targeting nvfp4 data type. The research team experimented nvidia-modelopt v0.45.0. Quantization-aware distillation (QAD) It was mainly followed to achieve accuracy on long inputs.

512 samples were used in calibration: 256 questions and 256 excerpts. abisee/cnn_dailymail. 20k samples were used in QAD training.

RSEARCH Team Reports NVFP4 on Blackwell Delivery Up to 2x higher throughput than BF16. it maintains BF16 retrieval accuracy of 99%+. The NVFP4 card also documents dynamic embedding sizes. You can crop a 2048-D vector from scratch to 1024 or 512 dimensions. Normalize again later.

Interactive Explainer: The Five-Step Recovery Path

Before touching the code, see the path running. It animates prefixing, bidirectional encoding, average pooling, L2 normalization, and dot-product scoring. Scores come from the published expected output of each card.

deployment matrix

As that walkthrough shows, checkpoints do not share a runtime path.

Along with Checkpoints, the NVIDIA research team released an optimized version NIM Microservices For 1B model. The Rust-based NIM matches or outperforms the vLLM checkpoint on the GB200 and RTX PRO 6000. NVIDIA tested input sequence lengths of 256 and 1024. separately, nvidia nemo automodel Recipes involve fine-tuning and distillation.

using it in code

Considering those paths, the prefixes come first. take questions query: and take documents passage: . The embeddings are L2-normalized, so the dot product is equivalent to cosine similarity.

# pip install --upgrade "transformers>=5.2.0" "sentence-transformers>=5.4.1"
import torch
from sentence_transformers import SentenceTransformer

QUERIES = ["How can someone reduce exposure to pollen during allergy season?"]
DOCUMENTS = ["People with pollen allergy can reduce exposure by staying indoors "
             "on dry, windy days, avoiding early-morning outdoor activity, and "
             "going outside after rain when pollen levels are lower."]

model = SentenceTransformer(
    "nvidia/Nemotron-3-Embed-8B-BF16",
    device="cuda",
    model_kwargs="dtype": torch.bfloat16,
                  # use "sdpa" if FlashAttention-2 is unavailable
                  "attn_implementation": "flash_attention_2",
    processor_kwargs="padding_side": "left",
)
model.max_seq_length = 32768

q = model.encode_query(QUERIES, batch_size=1, convert_to_tensor=True)
d = model.encode_document(DOCUMENTS, batch_size=1, convert_to_tensor=True)
print(model.similarity(q, d))  # card's published q[3]/d[3] score: 0.8008

encode_query And encode_document Read saved prompts. That’s why you never add prefixes by hand. for service, /v2/embed applies them from input_type instead:

vllm serve nvidia/Nemotron-3-Embed-1B-NVFP4 \
  --max-model-len 4096 \
  --max-num-batched-tokens 4096 \
  --max-cudagraph-capture-size 4096

import numpy as np, requests

def embed(input_type: str, texts: list[str]) -> np.ndarray:
    r = requests.post(
        "http://localhost:8000/v2/embed",
        json="model": "nvidia/Nemotron-3-Embed-1B-NVFP4",
              "input_type": input_type,          # "query" or "document"
              "texts": texts,
              "embedding_types": ["float"],
              "truncate": "END",
        timeout=120,
    )
    r.raise_for_status()
    return np.array(r.json()["embeddings"]["float"], dtype=np.float32)

scores = embed("query", QUERIES) @ embed("document", DOCUMENTS).T

use cases with examples

• multilingual enterprise search: A support team indexes Hindi, Japanese and English tickets simultaneously. Because retrieval occurs in different languages, a German query may bring up a Japanese resolution note.
• code recovery: includes training coir_apps, coir_cosqa, synthetic_text2sqland SWE-Bench. Therefore natural-language-to-code lookup is close to in-distribution.
• agent memory: The 32,768-token limit lets an agent embed summaries of longer conversations without aggressive rebuttals.
• cost-level RAG: Serve 1B-NVFP4 for high volume recalls, and route difficult queries to 8B. Since the widths vary, this requires two indexes.

key takeaways

• Nemotron-3-Embed-8B-BF16 is ranked #1 on RTEB with a 78.46 average NDCG@10.
• The three open outposts include 8B BF16, 1B BF16 and 1B NVFP4.
• NVFP4 maintains 99%+ BF16 accuracy at up to 2x Blackwell throughput.
• 1B came from ModelOpt NAS pruning plus COS+MSE distillation from 8B.
• All checkpoints use OpenMDW-1.1 and support 32,768-token input.

check it out NVIDIA launch post on hugging face, Nemotron 3 Embed Collection, 8B-BF16 card, 1B-BF16 card And 1B-NVFP4 Card. Also, feel free to follow us Twitter And don’t forget to join us 150k+ml subreddit and subscribe our newsletter. wait! Are you on Telegram? Now you can also connect with us on Telegram.

Do you need to partner with us to promote your GitHub repo or Hugging Face page or product release or webinar, etc? join us

Asif Razzaq Marktechpost Media Inc. Is the CEO of. As a visionary entrepreneur and engineer, Asif is committed to harnessing the potential of Artificial Intelligence for social good. Their most recent endeavor is the launch of MarketTechPost, an Artificial Intelligence media platform, known for its in-depth coverage of Machine Learning and Deep Learning news that is technically robust and easily understood by a wide audience. The platform boasts of over 2 million monthly views, which shows its popularity among the audience.