Domain Knowledge with Pinecone

Cookbook #1 gave you a fluent agent. Fluency is not knowledge: ask it about your own product or internal docs and it will guess. Classic RAG patches this by handing the model raw document chunks at inference time, which burns tokens, adds latency, and still invites hallucination. This recipe takes a different route: a knowledge engine.

What you will buildWhat you will build

A FastAPI service that uses Pinecone as the agent’s knowledge layer, with records embedded on Nebius and answers grounded by typed, per-field citations. The design follows the Pinecone Nexus model, which moves reasoning upstream into knowledge compilation so the agent queries curated knowledge instead of sifting raw chunks.

As of this writing, Pinecone Nexus is not yet generally available. That matters for the implementation. The recipe is framed around the Nexus mental model and architecture, but the runnable code is standard Pinecone-backed RAG knowledge: embed your records on Nebius, store them in Pinecone, look up the most relevant context at request time, and pass that context to the answer model.

Read this cookbook in two layers:

1. Target architecture: Nexus as the long-term knowledge engine.
2. Practical implementation today: classic Pinecone knowledge as the production-available substitute.

The three Nexus pieces this recipe leans on:

• Context Compiler, which turns a raw data estate into task-specific knowledge artifacts (one estate, many agents, distinct artifacts per task).
• Composable Retriever, which serves those artifacts with low latency, typed fields, per-field citations, and confidence scores.
• KnowQL, a declarative query language. The agent expresses what it needs through six primitives: intent, filter, provenance, output shape, confidence, and budget.

None of the three are implemented in the code below. The service queries Pinecone directly with a vector similarity search and a metadata filter, without a compiler, a query language, or field-level citations. Treat the code as the fallback shape the README describes, not as a Nexus integration.

PrerequisitesPrerequisites

• Python 3.12+
• uv
• A Nebius API key from the Nebius console
• A Pinecone API key with Nexus access from pinecone.io

Planned architecturePlanned architecture

documents ──► /ingest ──► Nexus Context Compiler ──► knowledge artifacts
                                                            │
question ──► KnowQL query ──► Composable Retriever ──► typed, cited context
                                                            │
                                                            ▼
                                          chat model (Nebius) ──► SSE answer

• Answer model: Qwen/Qwen3.5-397B-A17B on Nebius, set through NEBIUS_MODEL.
• Embedding model: Qwen/Qwen3-Embedding-8B on Nebius, the vector representation behind compilation and KnowQL knowledge. Served from the OpenAI-compatible embeddings endpoint, POST {NEBIUS_BASE_URL}/embeddings.
• Knowledge engine: Pinecone Nexus. Compilation, knowledge, and governance (PII tagging, versioning, RBAC) happen here, not in app code, once Nexus is available.

Until Nexus reaches general availability, the actual recipe implementation is the compatible fallback shape:

flowchart LR
    A[Documents] --> B[Normalize and vectorize]
    B --> C[Pinecone index]
    D[Question] --> E[Vector knowledge and filters]
    C --> E
    E --> F[Cited context]
    F --> G[Nebius answer model]
    G --> H[SSE answer]

The ingest path and the query path stay separate. Compilation is expensive and write-time; querying is cheap and read-time. Splitting them lets you recompile on a schedule, or on a data-change webhook, without coupling it to request latency.

Data sources and vectorizationData sources and vectorization

The knowledge layer starts with a structured data estate, not an ad hoc pile of Markdown files. The current vectorization notes use Goodreads exports as the concrete example, because they give enough relational structure to show how raw records turn into compiled knowledge.

The expected source files are:

• goodreads_books.json or goodreads_books.json.gz
• goodreads_book_authors.json or goodreads_book_authors.json.gz
• goodreads_book_genres_initial.json or goodreads_book_genres_initial.json.gz

Those files are not treated as three independent corpora. The pipeline first inspects what is available, then joins books with author and genre data so each vectorizable unit carries richer domain context than a single raw row. Books are prioritized, then enriched with author names and genre labels when those sources are present.

The vectorization flow:

1. Analyze the available JSON or gzipped JSON dumps in a data directory.
2. Build the text payload for each record from books, authors, and genres.
3. Generate embeddings with Nebius using Qwen/Qwen3-Embedding-8B.
4. Upsert vectors into Pinecone in fixed-size batches.

Embedding is the expensive semantic step. Pinecone upsert is the indexing step. The script exposes separate knobs for each, and the practical default is to batch embeddings in groups of 100 and Pinecone writes in groups of 200. That amortizes network overhead on the Nebius side without making Pinecone upserts unwieldy.

There is also an analysis-only mode that runs before anything gets written, so you can check the dataset footprint and confirm the estate is usable before paying for embeddings or mutating an index. For partial runs, the pipeline can skip individual source families (books, authors, genres), keep empty-text records if needed, and emit periodic progress updates while vectors are being written.

To run the vectorization script with the cookbook’s uv environment, sync dependencies first, then load the required API keys:

cd cookbooks/02-domain-knowledge-pinecone-nexus
uv sync
cp .env.example .env.local

uv creates the local environment for you; there is no separate .venv_vectorize workflow to maintain or commit. The script loads .env.local or .env automatically if present.

For a normal ingestion run, use batched embeddings and batched Pinecone upserts:

uv run python scripts/vectorize_goodreads_to_pinecone.py \
  --data-dir ../../data \
  --embed-batch-size 100 \
  --embed-concurrency 6 \
  --pinecone-batch-size 200 \
  --progress-interval 1000

The script pipelines those phases instead of running them in strict lockstep: while one batch is being upserted into Pinecone, the next embedding requests can already be in flight to Nebius. For bigger runs, --embed-concurrency is the main throughput lever, and --max-pending-embed-batches caps memory and backpressures the pipeline.

For a slower, more granular debug mode, you can force one embedding request per record with --embed-batch-size 1. That is useful for inspection and troubleshooting, but it is the wrong setting for high-throughput ingestion, since it pays the full API round trip cost for every single vector.

The pattern is meant to transfer to your own domain data too: product catalogs, support tickets, internal documentation, CRM exports, policy libraries, or any other structured source you can normalize into records, compose into text payloads, embed on Nebius, and upsert into Pinecone. Goodreads is just the example estate used to make the ingestion and vectorization flow concrete.

Querying the book RAGQuerying the book RAG

Once the Goodreads book vectors are stored in Pinecone, the query script runs a RAG loop:

1. Embed the user’s book request with Nebius.
2. Look up matching book vectors from Pinecone.
3. Expand around the first matches with related knowledge for the same author, same theme, and same publication year.
4. Pass the book knowledge context to the Nebius chat model.
5. Return grounded recommendations with citation markers.

That supports two common reader workflows: asking for books on a topic, such as books about Cold War espionage or climate fiction, and asking what to read after a book, such as recommendations after Dune or The Left Hand of Darkness.

For example:

uv run python scripts/query_goodreads_pinecone.py \
  "Recommend books about political intrigue and empire-building for someone who liked Dune" \
  --top-k 10 \
  --related-top-k 4 \
  --show-matches

By default, knowledge lookup is filtered to record_type=book, so author and genre helper vectors do not leak into the answer set. The related pass uses metadata from the first retrieved books. Same-author and same-year knowledge lookup work with the existing book metadata. Same-theme knowledge lookup uses the genres metadata written by the current vectorizer, so older vectors created before that field was added need to be upserted again before explicit theme filtering can work.

The main knowledge lesson of this cookbook is that vectorization is not just “chunk and embed.” The quality of the final knowledge engine depends on source selection, record joining, text construction, and batching strategy long before the serving path ever sees a query.

Run the cookbookRun the cookbook

After vectorizing books into Pinecone, run the cookbook backend:

cd cookbooks/02-domain-knowledge-pinecone-nexus
uv sync
cp .env.example .env
make dev

Then fill in NEBIUS_API_KEY and PINECONE_API_KEY in .env, and set PINECONE_INDEX_NAME to the index you vectorized into.

Set NEBIUS_BASE_URL=https://api.tokenfactory.nebius.com/v1/ and NEBIUS_MODEL=Qwen/Qwen3.5-397B-A17B in your .env. The repo’s .env.example and config default still carry the older api.studio.nebius.ai hostname and a chat model that Token Factory no longer serves, so override both when you copy the file.

Then call the SSE endpoint:

curl -N http://localhost:8000/agent/run \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "Recommend books about political intrigue and empire-building for someone who liked Dune",
    "top_k": 10,
    "related_top_k": 4,
    "include_related": true
  }'

The backend emits named SSE events:

event: status
data: {"phase": "sending_to_nebius","message":"Sending to Nebius Token Factory"}

event: status
data: {"phase": "knowledge","message":"Requesting Pinecone Knowledge"}

event: context
data: {"books": [...]}

event: status
data: {"phase": "synthesizing","message":"Synthesizing"}

event: answer
data: {"text": "..."}

event: status
data: {"phase":"done","message":"Done (742 token in | 219 token out | Cost: 0.000000 USD)","usage":{...}}

event: done
data: {"embeddingTokens":31,"inputTokens":742,"outputTokens":219,"totalTokens":992,"costUsd":0.0}

The model sees more retrieved candidates than it returns, but the answer is limited to at most 5 recommended books.

Cost is estimated from Nebius GET /v1/models?verbose=true when the model catalog exposes pricing fields. If that lookup fails, the app falls back to optional pricing env vars in .env: NEBIUS_INPUT_PRICE_PER_MILLION_TOKENS, NEBIUS_OUTPUT_PRICE_PER_MILLION_TOKENS, and NEBIUS_EMBEDDING_PRICE_PER_MILLION_TOKENS.

EndpointsEndpoints

Design decisionsDesign decisions

Why a knowledge engine instead of classic RAG? RAG retrieves, then reasons, so the model pays in tokens, latency, and hallucination risk to re-derive structure from raw chunks on every call. Nexus compiles that structure once, at write time, into typed artifacts. The agent reads a fact with a confidence score instead of inferring a fact from prose.

Why KnowQL over a raw vector query? A top_k similarity search returns things that look like the question: you get nearest neighbors back and have to re-derive everything else yourself. KnowQL turns knowledge into a contract. The agent states its intent, applies a filter, sets provenance requirements, specifies an output shape and confidence threshold, and caps a budget. A raw vector query cannot express provenance requirements or a confidence gate, which is also why the KnowQL result is auditable: the query is a declarative object you can log, diff, and replay.

Where governance lives. PII tagging, versioning, and RBAC are Nexus concerns, not app concerns. Keeping them in the knowledge layer means every consumer of an artifact inherits the same policy, so an agent cannot bypass redaction by querying differently.

Trade-off to accept. A knowledge engine adds a compilation step and a freshness window: an artifact is only as current as its last compile. For slowly changing domain knowledge that is the right trade. For fast-changing facts it is the wrong tool, which is why Cookbook #3 in this arc reaches for live web search instead.

Failure modesFailure modes

These are the failure modes the target Nexus architecture is designed to handle. The shipped fallback code does not implement compilation, a confidence primitive, or conflict resolution, so treat this table as design guidance for when Nexus ships rather than a description of current behavior.

Test itTest it

The repository ships a pytest suite under tests/ that monkeypatches the Nebius and Pinecone clients, so it runs with no network access and no live credentials:

cd cookbooks/02-domain-knowledge-pinecone-nexus
uv sync
uv run pytest -q

That covers the SSE agent route, the health and readiness probes, and the daily rate limiter. There is no test target in the Makefile, so run uv run pytest directly rather than make test.

Ship itShip it

There is no Dockerfile in this recipe yet. To run it as a long-lived service, deploy the FastAPI app with uv run uvicorn app.main:app --host 0.0.0.0 --port 8000 behind your own process manager and reverse proxy, set the required environment variables from .env.example, and point PINECONE_INDEX_NAME at an index that has already been vectorized with scripts/vectorize_goodreads_to_pinecone.py.

Going furtherGoing further

Next in the Agent Blueprint Recipes arc: Grounding, Real-Time Data with Tavily. Once your agent knows your compiled domain knowledge, teach it to reach for fresh facts on the open web.

Reference: Pinecone, Knowledge infrastructure for agents.