Microsoft & Hugging Face Release ThinkingBox-Training for AI Agents

Disclosure: this post contains affiliate links. If you buy through them we may earn a commission at no extra cost to you. Learn more.

TL;DR: Microsoft and Hugging Face released ThinkingBox-Training, an open-source system that turns database state verification into reinforcement learning rewards for training AI agents. When applied to Qwen3.8-27B for 50 training steps, the system improved the model's reliability on all-attempts task completion from 7.50% to 21.10%, while raising average success from 51.70% to 60.78%.

What's New

Microsoft and Hugging Face have released ThinkingBox-Training, now available on GitHub. The system evaluates AI agents by comparing final database states with expected outcomes, then uses those executable checks as rewards for reinforcement learning training.

The approach addresses a critical problem: AI agents that claim tasks are complete when database records tell a different story. ThinkingBox-Bench, the companion benchmark dataset, contains 507 tasks that test agents against real backend systems. Each task provides the agent with tools and a simulated user, then verifies what actually changed in the database.

When applied to Qwen3.8-27B using GRPO (Group Relative Policy Optimization) for 50 steps, the results showed significant reliability improvements. Tasks completed correctly on all twenty attempts rose from 38 to 107—nearly three times the baseline. Average success across all attempts increased from 51.70% to 60.78%.

The training methodology introduces "offline policy-relative difficulty," which estimates each task's success probability once using the base model before training begins. This ensures the training pool contains tasks that are neither too easy nor impossible for the current model—both of which would fail to improve learning.

A concrete example: in a corporate hotel-booking task requiring manager approval for room downgrades, the base model incorrectly changed bookings without approval in 15 of 20 attempts. After training, the fine-tuned model preserved booking and payment state in all 20 attempts.

Pricing and Availability

ThinkingBox-Training is available as open source on GitHub. The system is a collaboration between Microsoft and Hugging Face, with contributions from researchers at the University of Pittsburgh.

Estimated evaluation costs for the fine-tuned Qwen3.8-27B model decreased by 7.6% compared to the base model, from $925.80 to $855.58 for a full 20-run campaign on 507 tasks. Cost per dependable task—tasks passing all 20 attempts—dropped from $24.36 to $8.00, a 67.2% reduction. These figures represent comparative serving costs, not including training or policy-probe costs.

How to Try It

The training stack is available on GitHub under the ThinkingBox-Training repository. Implementation details are provided in the release documentation, including instructions for installation, environment setup, training data preparation, configuration, training execution, and checkpoint evaluation through OpenEnv.

The system evaluated the trained checkpoint against ThinkingBox-Bench's 507 tasks, which represent out-of-distribution tests: their domains, databases, and most MCP tools were not encountered during reinforcement learning training. Tasks span five domains: retail, auto insurance, travel, neobank, and consulting.

Who It's For

ThinkingBox-Training targets researchers and organizations developing AI agents that interact with production database systems. The system is particularly relevant for teams building agents that need consistent, repeatable performance rather than occasional success.

The training approach proved effective across multiple business domains. Pass@1 and observed 20/20 metrics rose in all five tested domains, though pass@20 (tasks solved at least once) showed mixed results, declining in retail, travel, and auto insurance while increasing in neobank and consulting.

Alternatives

The release includes benchmark comparisons with several frontier models. Claude Opus 5.5 leads the benchmark with 241 tasks passing all 20 attempts (47.53%), followed by Claude Opus 5 with the same count. GPT-6 Astra achieved 231 tasks (45.56%), while GPT-5.4 reached 128 tasks (25.25%). The fine-tuned Qwen3.8-27B at 107 tasks (21.10%) outperformed Claude Sonnet 4.6's 102 tasks and significantly exceeded its own base model performance of 38 tasks.

For teams working with AI agent development and training workflows, Make provides automation capabilities that could complement agent training pipelines.