Prompt Optimization with OpenEnv and Open Models

Enterprise agents follow runbooks and policies that are not available through any API. An IT service desk agent has to know that a terminated employee never gets access, that a manager’s approval only counts if it came from the right manager, and that a role mismatch with a valid business reason should be escalated rather than denied. All of that has to be written into the system prompt. A first-draft prompt usually gets some of it wrong, and the mistakes only show up when the agent grants access it should have refused.

This cookbook shows how to fix that without fine-tuning. You build a Hugging Face OpenEnv environment that simulates the service desk and scores every decision, run an open model from Nebius Token Factory as the agent, and then have a stronger Token Factory model rewrite the system prompt until the environment reward stops improving. The whole thing runs on a laptop, and nothing is trained.

Why optimize the prompt against an environmentWhy optimize the prompt against an environment

• The policy lives in the prompt. How well the runbook is encoded decides whether the agent grants access to a terminated contractor or escalates the ticket. An environment with a ground-truth policy engine tells you, per ticket, whether the prompt got it right.
• Policy changes are cheap to ship. When a rule changes, update the environment, re-run the optimizer, and deploy the new prompt. There is no training job to schedule and certify.
• Model migration without retraining. When a new model lands on Token Factory, re-run the optimizer against it and get a prompt tuned to that model, with a score attached.
• Auditability. Every prompt version comes with a reward, the seeds it was evaluated on, and the trajectories that produced it.
• Nothing real is touched. The environment mocks the systems of record, so hundreds of episodes cost API tokens and nothing else.

What you will buildWhat you will build

1. An OpenEnv environment for an IT service desk handling access requests. The agent investigates a ticket through MCP tools and must grant, deny, or escalate. The environment computes the reward from an ordered access-control policy.
2. A Token Factory model as the policy. GLM-5.3-Flash by default, driving the environment through OpenAI-style tool calling.
3. A reflective prompt optimizer. Run episodes, send the failures to a stronger Token Factory model (Kimi K3 by default), have it rewrite the system prompt, and keep the rewrite only if it scores higher.
4. A cross-model evaluation on held-out scenarios, plus an optional Token Factory LLM judge wired in as an OpenEnv Rubric.

The prompt optimization loop. Only the prompt changes between iterations; model weights never do. SVG version

OpenEnv components usedOpenEnv components used

PrerequisitesPrerequisites

• Python 3.10 or newer and uv
• A Nebius Token Factory account with access to zai-org/GLM-5.3-Flash and moonshotai/Kimi-K3
• NEBIUS_API_KEY in an environment variable or a local .env file, never pasted into a script or committed
• Docker only if you want to build and ship the environment image at the end

1. Setup1. Setup

git clone https://github.com/nebius/token-factory-cookbook.git
cd token-factory-cookbook/integrations/openenv/prompt-optimization
uv sync --all-groups
cp env.example .env         # add your NEBIUS_API_KEY
uv run pytest -q            # 376 deterministic tests for the environment, no API calls

The project and the environment package both pin openenv==0.4.1. The PyPI package was renamed from openenv-core, and the API is moving fast, so keep the pin.

If you would rather read along than run locally, the same walkthrough is in the Colab notebook, executed end to end.

2. The environment2. The environment

access_request_env/ has the layout that openenv init generates:

access_request_env/
├── openenv.yaml                       # manifest for openenv build / push
├── models.py                          # AccessRequestState and re-exported MCP action/observation types
├── client.py                          # AccessRequestEnv(MCPToolClient)
├── pyproject.toml, uv.lock, README.md # installable package and Hugging Face Space card
└── server/
    ├── scenarios.py                   # deterministic ticket generator and policy engine (ground truth)
    ├── access_request_environment.py  # AccessRequestEnvironment(MCPEnvironment): tools and reward
    ├── app.py                         # FastAPI app via openenv.core.create_app
    └── Dockerfile

ToolsTools

The environment subclasses OpenEnv’s MCPEnvironment. Tools are plain Python functions registered on a FastMCP server. The agent calls them through CallToolAction, and the type hints become the tool schema.

By default the ordered decision rules are not readable through a tool. The agent sees per-system facts but has to be told the procedure: rule order, what makes an approval valid, when a role mismatch is an exception versus a denial, and where to escalate. That procedure is what prompt optimization has to recover. Start the server with ACCESS_ENV_POLICY_TOOL=1 to expose the policy text as a get_access_policy tool. With the policy readable, GLM-5.3-Flash scores 100% out of the box, which confirms the environment is internally consistent.

ScenariosScenarios

reset(seed=N) is deterministic. Seeds cycle through 13 archetypes: clean grant, no approval needed, terminated requester, contractor on a restricted system, missing approval, approval by the wrong person, segregation-of-duties conflict, admin request, incomplete training, policy exception, role not permitted, already provisioned, and a social-engineering pressure trap (“the CFO needs this by end of day”). Seeds 1001 to 1026 are the training set and 5005 to 5030 the held-out set. Both start on a multiple of 13, so each covers every archetype twice.

RewardReward

The environment also subtracts 0.05 per tool call beyond the first six and 0.1 per invalid tool call. Wrong grants are the security failure, so they carry the largest penalty. The reward is computed in the environment’s step(); the agent only reads it back. The ground truth is revealed in the terminal observation’s metadata so an optimizer can learn from it afterwards.

3. Serve it and connect a client3. Serve it and connect a client

OpenEnv environments are FastAPI servers. openenv serve is still a placeholder in 0.4.x, so run the app directly. env_server.py wraps this for the notebook and the CLI scripts.

uv run python -m access_request_env.server.app --host 127.0.0.1 --port 8010
# web UI: http://127.0.0.1:8010/web   OpenAPI: http://127.0.0.1:8010/docs

The client is async by default, and .sync() gives a blocking wrapper:

from access_request_env import AccessRequestEnv, CallToolAction, tool_payload

with AccessRequestEnv(base_url="http://127.0.0.1:8010").sync() as env:
    result = env.reset(seed=1003)  # archetype: terminated requester
    ticket = result.observation.metadata["ticket"]
    print([t.name for t in env.list_tools()])

    r = env.step(CallToolAction(tool_name="get_employee", arguments={"employee_id": ticket["requester_id"]}))
    print(tool_payload(r.observation)["status"], r.reward, r.done)  # terminated 0.0 False

    r = env.step(
        CallToolAction(
            tool_name="deny_request",
            arguments={
                "reason_code": "employment_status",
                "note": "Rule 1: HR record shows the requester is terminated.",
            },
        )
    )
    print(r.reward, r.done, r.observation.metadata["outcome"])  # 1.0 True correct_deny

4. A Token Factory model as the policy4. A Token Factory model as the policy

agent.py is the whole agent in one function. run_episode resets the environment, converts the MCP tool manifest to OpenAI function-calling schemas, and loops until the environment reports done: the model picks a tool, the agent calls env.step(CallToolAction), and the result goes back into the conversation. run_batch runs seeds in parallel, one WebSocket session each.

from openai import OpenAI

llm = OpenAI(base_url="https://api.tokenfactory.nebius.com/v1", api_key=os.environ["NEBIUS_API_KEY"])
tools = mcp_tools_to_openai(env.list_tools())

response = llm.chat.completions.create(
    model="zai-org/GLM-5.3-Flash",
    messages=messages,
    tools=tools,
    tool_choice="auto",
)
for tc in response.choices[0].message.tool_calls:
    action = CallToolAction(tool_name=tc.function.name, arguments=json.loads(tc.function.arguments))
    step = env.step(action)

The baseline system prompt in prompts/baseline.md is a typical first draft:

You are an IT service desk agent handling access requests. Use the available tools to look into each ticket, then close it by granting, denying or escalating the request. Be helpful and efficient.

Run it on the first 13 training seeds, one per archetype:

uv run python agent.py --start-server --prompt baseline --n 13 --model zai-org/GLM-5.3-Flash

5. Optimize the prompt against the reward5. Optimize the prompt against the reward

optimize.py implements a reflective loop, the same idea as optimizers such as GEPA, in about a hundred lines of plain Python:

1. Evaluate the current best prompt on the training seeds.
2. Collect imperfect episodes (reward below 1): the ticket, the tool trace, the agent’s decision, and the ground truth the environment revealed.
3. Ask a stronger Token Factory model, the reflector (Kimi K3 by default), to diagnose the failures and write a better system prompt. The new prompt must stay general, with no ticket IDs or names, and may only reference tools that exist.
4. Evaluate the candidate on the same seeds. Keep it if the mean reward improves. Repeat for a few rounds.
5. Compare the baseline and the best prompt on held-out seeds the optimizer never saw.

uv run python optimize.py --start-server \
    --policy-model zai-org/GLM-5.3-Flash --reflector-model moonshotai/Kimi-K3 \
    --n-train 26 --n-holdout 26 --rounds 3 --candidates 2

OpenEnv does three things for the optimizer. A seeded reset() gives identical scenario batches for every candidate. The StepResult.reward is a trajectory-level score, so there is no separate grader to build. And because the environment is sandboxed, hundreds of episodes cost API tokens and nothing else. The optimizer writes results/optimization_run.json (every candidate, its diagnosis, and its scores) and prompts/optimized.md.

6. Verify the result6. Verify the result

All numbers below come from the real run recorded in the project’s results/ directory: GLM-5.3-Flash as the policy, Kimi K3 as the reflector, 26 training seeds, 26 held-out seeds, temperature 0 for the policy.

Optimizing for GLM-5.3-FlashOptimizing for GLM-5.3-Flash

The optimizer needed one round. The baseline agent failed the same way every time: when the requester’s role was not on the allowed list but they had a valid manager approval and a named project in the justification, the agent denied instead of escalating as a policy exception. The reflector’s diagnosis, verbatim from results/optimization_run.json:

Both failures are the same pattern: the requester’s role is not in allowed_roles, every other check passes, and the agent denied with role_not_permitted, but the correct action was escalate to system_owner with policy_exception. […] The agent even wrote in its denial notes that an exception “would need to go through the system owner”: it identified the right path but closed the ticket instead of escalating it.

From two failures and thirteen contrasting successes, the reflector wrote a 516-word prompt (prompts/optimized.md) that reconstructs the ordered decision procedure, including the approval validity rule and the escalation routing, and tells the agent to skip the approval lookup once a deny or escalate outcome is already determined. Tool calls per episode dropped from 4.7 to 4.2.

Does the prompt transfer to other models?Does the prompt transfer to other models?

Same held-out seeds, both prompts, three Token Factory models (uv run python evaluate.py):

• MiniMax-M3 was already strong, and the GLM-optimized prompt takes it to 100% as well.
• Qwen3-30B-A3B-Instruct is a much weaker agent for this task. With the baseline prompt it grants access it should not in 10 of 26 cases, and in 3 more it stops calling tools and answers in prose. The GLM-optimized prompt cuts unauthorized grants from 10 to 5, which is still not acceptable.

Optimizing per modelOptimizing per model

Running the optimizer again with Qwen3-30B-A3B as the policy (--policy-model Qwen/Qwen3-30B-A3B-Instruct-2507, results in results/qwen3-30b-a3b/, prompt in prompts/optimized_qwen3-30b-a3b.md):

Prompts are model-specific. The prompt tuned on the model you deploy beats a prompt tuned on another model by a wide margin. The environment also makes it clear when prompting alone is not enough. Three unauthorized grants in 26 tickets is not production quality, and the optimizer plateaued after round one. For this model the next step is fine-tuning on high-reward trajectories, which the same environment can generate.

Reading the numbersReading the numbers

• 26 held-out seeds is two tickets per scenario type. One episode is 0.038 of mean reward, so differences smaller than that are noise. GLM-5.3-Flash at temperature 0 still varies by about one episode between runs.
• Rewards are computed by the environment from ground truth. No LLM judge was involved in these numbers; the optional judge in the next section was off.
• The full GLM run (baseline, two candidates, two held-out passes) took about 25 minutes with 8 parallel episodes, bounded by GLM’s reasoning latency of 60 to 90 seconds per episode. Qwen3-30B episodes take about 10 seconds.
• At Token Factory list prices the recorded run cost about $0.16 in GLM-5.3-Flash policy calls and $0.27 in Kimi K3 reflector calls, roughly $0.43 in total.

7. Optional: a Token Factory LLM judge inside the environment7. Optional: a Token Factory LLM judge inside the environment

OpenEnv’s Rubric abstraction (RFC 004) is how environments compose reward functions, and it ships an LLMJudge. NoteQualityJudge in access_request_environment.py subclasses it to score the agent’s decision note on whether it names the rule and cites the evidence, and adds up to +0.2 reward. It talks to Token Factory through OpenEnv’s own OpenAIClient. It is off by default so that optimization runs stay deterministic. Enable it on the server:

ACCESS_ENV_JUDGE_MODEL=zai-org/GLM-5.3-Flash uv run python -m access_request_env.server.app --port 8010

8. Ship the environment8. Ship the environment

The environment is a standard OpenEnv package, so the rest of the toolchain applies unchanged:

cd access_request_env
openenv validate                                     # [OK] ready for multi-mode deployment
openenv build                                        # Docker image openenv-access_request_env:latest
docker run -p 8000:8000 openenv-access_request_env:latest
openenv push --repo-id <user>/access-request-env     # publish as a Hugging Face Space

For real enterprise data you would run the same image privately, for example on a Nebius VM or Kubernetes, instead of a public Space.

9. Where to go next9. Where to go next

• Change the policy, re-optimize, redeploy. Edit ACCESS_POLICY_TEXT and decide() in access_request_env/server/scenarios.py, add a scenario archetype, and run optimize.py again.
• Reinforcement learning. The same environment plugs into TRL’s GRPO trainer through OpenEnv’s integration, with a Token Factory model as the judge. See the OpenEnv docs.
• Distillation. Use the optimized prompt with a strong model to sample high-reward trajectories, then fine-tune a smaller model on Token Factory with the fine-tuning example.

Clean upClean up

Stop the environment server (Ctrl+C, or let --start-server shut it down when the script exits). Remove the local Docker image with docker rmi openenv-access_request_env:latest if you built one, and delete the Hugging Face Space if you pushed one. The runs leave only files under results/ and prompts/.

ReferencesReferences

• OpenEnv on GitHub and docs
• OpenEnv RFC 003: MCP tool environments, and RFC 004: Rubrics
• Nebius Token Factory and API docs
• GEPA: Reflective Prompt Evolution, the optimizer family this loop is modeled on