Inference

Using open models from Nebius Token Factory with Claude Code

Point Claude Code at Kimi K2.7 Code on Nebius Token Factory through a local relay, run a headless coding task with real tool use, and see what it cost, without touching your Claude Code config.

Build it with an agent

Paste into Claude Code, Cursor, or any coding agent — it builds the recipe for you.

Build a small, production-minded Python application that relays the Anthropic Messages API to Nebius Token Factory, so an Anthropic-only client such as Claude Code can run on an open model. Use only the Python standard library at runtime. Deliver `app.py`, `requirements.txt`, a `Dockerfile`, a `.dockerignore`, pytest tests under `tests/`, and a `README.md`.

Read the credential from `NEBIUS_API_KEY`, the API base URL from `NEBIUS_BASE_URL` (default `https://api.tokenfactory.nebius.com/v1`), and the model ID from `NEBIUS_MODEL` (default `moonshotai/Kimi-K2.7-Code`); exit with a clear error if the credential is missing, and never log or commit it.

Serve `POST /v1/messages` on `127.0.0.1`, matching the route on the request path with the query string stripped, because clients call `/v1/messages?beta=true`. Translate each request into a `POST /chat/completions` call: fold `system` into a leading system message, flatten Anthropic content blocks to text, pass `max_tokens` and `temperature` through, convert `tools` entries into OpenAI function tools using `input_schema` as `parameters`, turn assistant `tool_use` blocks into `tool_calls`, and turn user `tool_result` blocks into `role: "tool"` messages carrying `tool_use_id` as `tool_call_id`. Translate the reply back: emit `text` and `tool_use` content blocks, map `finish_reason` `stop`/`length`/`tool_calls` to `stop_reason` `end_turn`/`max_tokens`/`tool_use`, and report `usage` as `input_tokens` and `output_tokens`.

Support streaming when the request sets `stream: true`. Consume the upstream SSE chunks and re-emit them as the Anthropic event sequence (`message_start`, `content_block_start`, `content_block_delta`, `content_block_stop`, `message_delta`, `message_stop`), using `text_delta` for text and, for each streamed tool call, a `tool_use` content block whose argument fragments are re-emitted as `input_json_delta` frames with `partial_json`. Return Anthropic-shaped JSON errors and never leak upstream response bodies to the caller.

The container command `python app.py` must start the relay on an ephemeral port, send one streaming Anthropic-shaped request through it to the configured model, print the collected SSE event names, print the assistant text prefixed exactly with `Relay verified:`, shut the server down, and exit zero.

Keep the tests deterministic and offline: unit-test the request translation, the response translation, the `finish_reason` mapping, the query-string route match, and the streamed `tool_use` event sequence against stubbed upstream chunks, so no test needs a live API call. Document local `.venv` setup, running the tests, running the application once, and pointing a real Anthropic client at the relay with `ANTHROPIC_BASE_URL`. Include verification, troubleshooting, Docker, and cleanup instructions.

Recipe

Claude Code only speaks the Anthropic Messages API, and Token Factory serves its open coding models through an OpenAI-compatible API. Claude Code reads its endpoint from the environment, so a local translation proxy is enough to run Kimi, Qwen, MiniMax, or DeepSeek behind the CLI you already use, with no change to the agent itself.

This cookbook does that two ways: with nebiusrelay, the proxy Nebius links for Claude Code, and, through the one-shot prompt, with a relay you build yourself.

What you will build

A working Claude Code session whose every token is served by an open model on Token Factory:

  • moonshotai/Kimi-K2.7-Code as the coding model: 262K context, text-only, built for agentic coding.
  • nebiusrelay as the local Anthropic-to-Token-Factory translator, injected for one run at a time.
  • Claude Code itself, unmodified. Your login, subscription, settings, and CLAUDE.md files stay exactly as they are.

A measured headless task that wrote one file took 22 seconds and cost $0.032 for 30,873 input and 609 output tokens.

Why Claude Code needs a proxy at all

Cursor, Cline, and OpenCode can point straight at Token Factory because they speak the OpenAI chat-completions format Token Factory serves. Claude Code speaks Anthropic Messages: a different request shape, a different response shape, and a different server-sent-event vocabulary for streaming. The relay accepts Anthropic Messages on localhost, rewrites each request into chat-completions, and rewrites the reply, including tool calls, back into Anthropic blocks. The one-shot prompt reproduces that translation in about 280 lines of standard-library Python.

Prerequisites

  • macOS or Linux, and curl
  • Claude Code already installed. The relay routes it but does not install it.
  • Nebius Token Factory account and API key
  • The key in an environment variable, never pasted into a tracked file

nebiusrelay runs on Bun and installs it for you if it is missing.

Run the cookbook

  1. Install the relay. It writes to ~/.nebiusrelay/, links the wrappers into a writable directory already on your PATH, and adds a PATH line to your shell profile:

    curl -fsSL https://nebius-tf-relay.vercel.app/install.sh | sh
    

    Read the script before piping it to a shell if you have not seen it before. Then restart your shell, or run the export PATH=... line it prints, and confirm the install:

    nebiusrelay --version
    
  2. Give it your Token Factory key. Either store it once, interactively:

    nebiusrelay configure
    

    or export it and skip the prompts, which is the form to use in CI or from another agent:

    export NEBIUS_API_KEY="your-token-factory-key"
    
  3. Launch Claude Code on an open model. Put --main before the claude subcommand. The troubleshooting section explains why the order matters:

    nebiusrelay --main moonshotai/Kimi-K2.7-Code claude
    

    The banner confirms where your tokens are going before the session starts:

    Nebius TF Relay ▸ Routing Claude Code → Nebius Token Factory (Kimi K2.7 Code). Not Anthropic.
    

    Omit --main and you get the relay’s built-in default, which is Kimi K2.7 Code in v0.14.3. The short alias nclaude is the same thing with no model flag.

  4. Run one headless task so that success is a file on disk. Close stdin with < /dev/null; without it a non-interactive run blocks waiting for input:

    nebiusrelay --main moonshotai/Kimi-K2.7-Code claude \
      -p "Use the Write tool to create hello.py whose only line is: print('relay ok')" \
      --dangerously-skip-permissions --output-format json < /dev/null
    

    --dangerously-skip-permissions stops a headless run from stalling on a permission prompt. Use it only in a throwaway directory.

Verify the result

The file exists, with the right contents. This shows that tool calls survived the translation in both directions:

cat hello.py

The tokens came from Token Factory. The relay prints a cost line when the session ends, priced against the model’s real per-token rates:

[nebiusrelay cost] session total: $0.0318 (30,873 in, 609 out)

For history across sessions, all of it stored locally under ~/.nebiusrelay and never uploaded:

nebiusrelay usage --last 7d

Switching models works. Run the same task again against a different one and watch the banner change:

nebiusrelay --main moonshotai/Kimi-K2.6 claude -p "reply with OK only" \
  --dangerously-skip-permissions < /dev/null

Every model your account can see is available here, including moonshotai/Kimi-K3, Qwen/Qwen3.5-397B-A17B, MiniMaxAI/MiniMax-M3, deepseek-ai/DeepSeek-V4-Pro, and zai-org/GLM-5.2, because the relay pulls the catalog from Token Factory at startup.

Troubleshooting

  • --main seems to be ignored. nclaude --main X and nebiusrelay claude --main X both put the flag after the harness name, where it is passed through to Claude Code and dropped, so the run silently uses the default model. The banner names the model actually in use. Always write nebiusrelay --main <model> claude ....

  • The agent says it did the work, but nothing changed on disk. Open coding models will sometimes narrate a file write without emitting the tool call. Naming the tool fixes it: “Use the Write tool to create hello.py…” succeeded where “Create a file hello.py…” only replied DONE. Verify with git status, not with the transcript.

  • A headless run hangs and produces nothing. Add < /dev/null. Note that the line Reading additional input from stdin... also prints on healthy runs; a real hang shows as output that stops growing.

  • No Nebius API key found. Run nebiusrelay configure, or export NEBIUS_API_KEY in the same shell.

  • claude: command not found. The relay does not install agent CLIs. Install Claude Code, then retry.

  • Web search fails inside the session. Search is emulated through a separate provider key that nebiusrelay configure collects; without it, searches return an explicit “not set” error rather than failing quietly. In v0.14.3 the docs name this key inconsistently, so set it via configure rather than guessing an environment variable.

  • A model is missing from the catalog. Nebius adds and removes models; zai-org/GLM-5.2 and zai-org/GLM-5.1 have traded places before. Check what your account can see:

    curl -s https://api.tokenfactory.nebius.com/v1/models \
      -H "Authorization: Bearer $NEBIUS_API_KEY" | grep '"id"'
    

Build the relay yourself

prompt.md is a one-shot prompt for the underlying exercise: build the Anthropic-Messages-to-Token-Factory relay from scratch, in standard-library Python, with tests. Translating the streaming protocol by hand makes the failure modes above easier to understand, and the result is usable: real Claude Code will drive a Token Factory model through it, tool calls included.

Two details are easy to miss. Claude Code posts to /v1/messages?beta=true, so a route match on the raw path rejects every request before it reaches your translator; match on the path with the query string stripped. Streamed tool calls arrive as OpenAI tool_calls argument fragments and have to be re-emitted as Anthropic input_json_delta events inside a tool_use content block; if that is wrong, the agent can talk but cannot act.

Clean up

Nothing was written to your Claude Code configuration, so there is nothing to restore. Stop using the wrappers and your setup is unchanged. To stop the shared background proxy:

nebiusrelay daemon stop

To remove the relay completely, delete its directory and the PATH line the installer added to your shell profile:

rm -rf ~/.nebiusrelay
unset NEBIUS_API_KEY

If you pasted your Token Factory key into a shell history file, a notebook, or a commit, rotate it in the Token Factory console rather than deleting the file and assuming it is gone.