AWS Fine-Tunes GPT-OSS-20B for Multilingual Reasoning on SageMaker

Disclosure: this post contains affiliate links. If you buy through them we may earn a commission at no extra cost to you. Learn more.

TL;DR: AWS has published a technical guide showing how they fine-tuned GPT-OSS-20B, an open-weight reasoning model, to reason in multiple languages instead of defaulting to English. Using supervised fine-tuning (SFT) on Amazon SageMaker HyperPod, they increased reasoning-language accuracy from 22% to 100%, with reinforcement learning (GRPO) adding measurable gains where additional improvement was possible. The work demonstrates when each technique is most effective for customizing open-weight models.

What's New

Amazon Web Services has released a detailed technical case study on fine-tuning GPT-OSS-20B, an open-weight reasoning model, to perform chain-of-thought reasoning in multiple languages on Amazon SageMaker HyperPod. The blog post, authored by Mijanur Palash and Mark Vinciguerra and published on October 20, 2026, addresses a critical limitation: the base GPT-OSS-20B model thinks exclusively in English, even when asked to reason in other languages.

In testing on a 50-example multilingual reasoning dataset, the base model reasoned in the correct target language only 22% of the time. Performance by language was stark: English 100%, French 0%, German 10%, Spanish 0%, and Italian 0%. The model would acknowledge instructions to reason in French or Spanish, then produce its chain-of-thought in English anyway.

AWS demonstrated two fine-tuning approaches: supervised fine-tuning (SFT) and reinforcement learning with Group Relative Policy Optimization (GRPO). Using SFT with Low-Rank Adaptation (LoRA), they increased reasoning-language accuracy from 22% to 100%. GRPO added measurable gains on top of SFT where headroom existed for further refinement.

The training used the HuggingFaceH4/Multilingual-Thinking dataset, which contains approximately 949 training examples and 50 validation examples. The dataset covers chain-of-thought reasoning translated into French, Spanish, German, and Italian alongside English. Evaluation measured two metrics: whether the chain-of-thought reasoning was in the requested language, and whether the final answer was in the requested language.

AWS's key finding: SFT teaches behaviors the model doesn't have, while GRPO optimizes behaviors the model can already sometimes produce. GRPO works by sampling multiple responses per prompt and pushing the model toward above-average ones. If the desired behavior has zero probability in the base model, there's little for GRPO to reinforce.

The training stack ran on Amazon SageMaker HyperPod with Amazon Elastic Kubernetes Service (Amazon EKS) orchestration. For GPT-OSS-20B fine-tuning, AWS used the PyTorch TRL library. For a control experiment with Qwen2.5-7B, they switched to veRL as the trainer, with Ray and KubeRay for distributed orchestration and vLLM for fast generation. Checkpoints and datasets lived on Amazon FSx for Lustre, and inter-node communication used Elastic Fabric Adapter (EFA).

Pricing and Availability

The blog post does not specify pricing for the fine-tuning process. The work was conducted using Amazon SageMaker HyperPod clusters. For GPT-OSS SFT, AWS used 4× ml.g6e.12xlarge instances (16× L40S 48GB GPUs). The Qwen control experiment used 2× g6e.48xlarge instances (16× L40S GPUs). Pricing would depend on the specific Amazon SageMaker instance types and duration of use.

The code, manifests, and scripts are available in the awsome-distributed-ai GitHub repository under 3.test_cases/pytorch/trl/gpt-oss-lora-grpo/. GPT-OSS-20B is an open-weight model available through HuggingFace as openai/gpt-oss-20b.

How to Try It

AWS provides detailed prerequisites and setup instructions in the blog post. Users need an Amazon SageMaker HyperPod cluster with Amazon EKS orchestration, an Amazon FSx for Lustre file system mounted on the cluster, EFA-enabled instances, and a HuggingFace access token for the model and dataset.

The complete workflow involves cloning the awsome-distributed-ai GitHub repository, configuring environment variables including ECR registry and HuggingFace token, building and pushing the training image to a container registry, and applying Kubernetes manifests to launch training jobs. The training image bundles PyTorch, TRL, HuggingFace Transformers, PEFT, and the necessary training and inference scripts.

Who It's For

This release targets ML engineers and data scientists deploying reasoning models for non-English users. Organizations serving multilingual customer bases will find particular value, as users who ask questions in French, German, Spanish, or Italian expect models to reason and answer in those languages rather than defaulting to English mid-thought.

The technical guide is also valuable for teams deciding between supervised fine-tuning and reinforcement learning approaches when customizing open-weight models. AWS's controlled experiments across different tasks and capability gaps provide practical decision criteria.

Alternatives

While this blog post focuses on fine-tuning GPT-OSS-20B on Amazon SageMaker HyperPod, teams have other options for multilingual reasoning. They could use natively multilingual models that reason across languages without fine-tuning, though these may not match the reasoning capabilities of specialized models. Alternatively, organizations could use other cloud platforms' ML training infrastructure, or build their own distributed training systems using open-source tools like Ray, vLLM, and veRL independently.

For broader AI development and experimentation, platforms like Replit offer environments for building and testing AI applications, though at a different scale than enterprise fine-tuning infrastructure.