Real-Time Data with Tavily

A Pinecone-backed book recommender built in an earlier cookbook (see cookbook #2 in the partner repo) can recommend from a Goodreads-style corpus, but that corpus stops around 2017. It cannot answer questions about pricing, availability, current editions, or newly released books because those facts change constantly and a static vector index goes stale the moment it is built.

This cookbook keeps the book memory from cookbook #2 and adds a live grounding layer with Tavily, a Nebius partner. Pinecone answers “what in my curated corpus is semantically relevant?” Tavily answers “what changed on the web since this corpus was built?” A Nebius model then synthesizes both into one streamed recommendation.

What you will buildWhat you will build

A FastAPI service that answers book recommendation questions with this pipeline:

flowchart LR
    A[User book request] --> B[Nebius embedding]
    B --> C[Pinecone book knowledge]
    C --> D[Related books by author, theme, year]
    D --> E[Tavily fresh web search]
    E --> F[Nebius answer model]
    F --> G[SSE recommendation]

The route streams each phase to the client as server-sent events:

• agent_message events for human-readable progress
• status events for machine-readable phase changes
• context with the Pinecone book candidates
• sources with the Tavily web sources
• token events for the final answer
• done with elapsed time, token usage, and estimated cost

Why Tavily hereWhy Tavily here

The vector index is curated and stable, which makes it good for semantic recommendations and for expanding by author, theme, or year. It is not good for facts that move every week. Tavily is used for freshness signals only: newer books adjacent to the reader’s request, current editions or formats, availability and pricing context, and current discussion, reviews, awards, or bestseller context.

The answer model receives both contexts and is instructed to keep them separate: Goodreads and Pinecone citations use [1], [2], [3]; Tavily web citations use [W1], [W2], [W3].

PrerequisitesPrerequisites

• Python 3.12+
• uv
• A Nebius API key
• A Pinecone API key, plus the Goodreads book vectors from cookbook #2 already upserted into Pinecone
• A Tavily API key (get one at tavily.com)
• Docker, optionally

Run the cookbookRun the cookbook

cd cookbooks/03-real-time-data-tavily
uv sync
cp .env.example .env

Fill in:

NEBIUS_API_KEY=...
NEBIUS_BASE_URL=https://api.tokenfactory.nebius.com/v1/
NEBIUS_MODEL=Qwen/Qwen3.5-397B-A17B
PINECONE_API_KEY=...
PINECONE_INDEX_NAME=books-demo
TAVILY_API_KEY=...

The two NEBIUS_ overrides matter: the repo’s .env.example still ships the older api.studio.nebius.ai hostname and a chat model that Token Factory no longer serves, so the first request fails without them.

Then start the backend:

make dev

Send a request:

curl -N -X POST http://localhost:8000/agent/run \
  -H 'content-type: application/json' \
  -d '{
    "prompt": "Find cozy fantasy books launched after 2021 with recent review context",
    "top_k": 10,
    "related_top_k": 4,
    "include_related": true
  }'

Walk-throughWalk-through

Sample SSE flowSample SSE flow

event: agent_message
data: {"text":"I am mapping your Dune request into the book index."}

event: status
data: {"phase":"embedding","message":"Preparing the semantic query"}

event: status
data: {"phase":"knowledge","message":"Requesting Pinecone Knowledge"}

event: context
data: {"books":[...]}

event: status
data: {"phase":"searching","message":"Requesting Tavily Results"}

event: sources
data: {"items":[...]}

event: status
data: {"phase":"synthesizing","message":"Synthesizing"}

event: token
data: {"text":"If you liked Dune..."}

event: token
data: {"text":"\n\n---\nTime: 4.31s | Tokens: 36 embed, 1420 in, 390 out | Cost: $0.000312"}

event: done
data: {"embeddingTokens":36,"inputTokens":1420,"outputTokens":390,"totalTokens":1846,"costUsd":0.000312,"elapsedSeconds":4.31}

The service also emits two phases and one event that this sample flow does not show: a sending_to_nebius status ahead of the embedding call, a planning_web_search status, and a web_search_plan event that carries the query and rationale the model built for Tavily before the search runs.

How this differs from the Pinecone-only recipeHow this differs from the Pinecone-only recipe

Cookbook #2 stops after Pinecone knowledge, which is enough when the answer should stay inside the static corpus. This cookbook adds one more step before synthesis:

fresh_sources = rag.search_fresh_context(plan)
stream = rag.stream_synthesis(prompt, books, fresh_sources)

The Tavily query is built from the original user request plus the strongest retrieved book titles, so Tavily gets enough context to search for information around the reader’s intent instead of running a generic web search.

Data and vectorizationData and vectorization

This recipe reuses the Pinecone index created in cookbook #2. If you have not built it yet, run the vectorization flow there first:

cd cookbooks/02-domain-knowledge-pinecone-nexus
uv sync
uv run python scripts/vectorize_goodreads_to_pinecone.py \
  --data-dir ../../data \
  --embed-batch-size 100 \
  --embed-concurrency 6 \
  --pinecone-batch-size 200 \
  --progress-interval 1000

You can use your own data instead of Goodreads. The only requirement is that your vectors carry enough metadata for the serving path to render useful context: title, authors, themes or genres, ratings or quality signals, and publication year when available.

Design decisionsDesign decisions

NEBIUS_BASE_URL and NEBIUS_MODEL are the two values the quickstart tells you to override, since the repo’s .env.example and config default still point at the retired api.studio.nebius.ai host and model.

Failure modesFailure modes

Test itTest it

uv run pytest
uv run ruff check
uv run ruff format --check

The tests monkeypatch Nebius, Pinecone, and Tavily, so they do not call the network by default.

Ship itShip it

A multi-stage Dockerfile is included: it builds the dependencies with uv sync --frozen, copies the app, and runs it as a non-root user with a container healthcheck against /healthz. Build and run it with:

docker build -t real-time-data-tavily .
docker run --env-file .env -p 8000:8000 real-time-data-tavily

Going furtherGoing further

• Add a dedicated small-model query planner before Tavily if you want multiple live searches per request.
• Cache Tavily responses for a few minutes to avoid repeat searches during demos.
• Add a critic pass that rejects uncited fresh claims before streaming done.
• The next cookbook in this arc rewrites the hand-wired flow as a LangGraph so planning, retrieval, writing, and memory have explicit state boundaries: cookbook #4 in the partner repo.