Ornith-1.0: self-improving open-source models for agentic coding
Aloha! 🌺 Ornith-1.0 is a self-improving open-source models for agentic coding.
Each model is evaluated against its size-appropriate baselines. All three use the same harnesses and decoding setup (see the notes under the tables).
* Terminal-Bench 2.1 (Terminus-2): evaluated with the Harbor/Terminus-2 framework, parser=json, temperature=1.0, top_p=1.0, 128K context window. Each run uses a 4-hour timeout with 32 CPU cores and 48GB RAM, averaged over 5 runs. We adjust the Qwen chat template to keep training and inference consistent and modify Harbor to align with vLLM's reasoning_content key. * Terminal-Bench 2.1 (Claude Code): evaluated with Claude Code 2.1.126, parser=json, temperature=1.0, top_p=1.0, max_new_tokens=131072, averaged over 5 runs (Qwen chat template likewise modified). * SWE-bench Verified / Pro / Multilingual: OpenHands harness, temp=1.0, top_p=0.95, 256K context window. * SWE Atlas QnA / RF / TW: mini-SWE-agent harness, temp=1.0, top_p=0.95, 128K context window, averaged over 5 runs. * NL2Repo: temperature=1.0, top_p=1.0, 400K context, 48K output, anti-hacking filters. * ClawEval: an agentic code benchmark over real-user task distributions; temp=0.6, 256K context.
Ornith-1.0 is a reasoning model : by default the assistant turn opens with a <think> … </think> block before the final answer. The serving recipes below enable a reasoning parser so the chain-of-thought is returned in a separate reasoning_content field, and a tool-call parser so the model's <tool_call> blocks are surfaced as OpenAI-style tool_calls .
Serving Ornith-1.0 requires recent runtimes:
Recommended sampling parameters: temperature=0.6 , top_p=0.95 , top_k=20 (use temperature=1.0 to reproduce the reported benchmark setup).