Developers Adapt Qwen3-0.6B for Complex Tool Calling

A newly detailed supervised fine-tuning workflow outlines how to train the Qwen3-0.6B language model for advanced tool-calling capabilities. Utilizing the XYZ-Aquila-SFT dataset, the end-to-end pipeline leverages PyTorch, Hugging Face Transformers, and Parameter-Efficient Fine-Tuning to parse multi-turn trajectories and extract structured tool calls. This approach allows developers to preserve embedded reasoning and observation patterns that are often lost during standard training procedures.

To prevent the loss of critical training data, the pipeline avoids the standard Hugging Face chat template. The default Qwen3 template typically deletes reasoning blocks, represented by think tags, from all assistant turns except the final one. To bypass this issue, the developers manually rendered the trajectories in ChatML format. This manual formatting ensures that token-exact masking is applied, restricting loss calculations exclusively to assistant-generated tokens while retaining all reasoning supervision.

The training process utilizes Low-Rank Adaptation to update Qwen3-0.6B efficiently on consumer-grade hardware. The pipeline requires specific library versions, including datasets 3.0.0 or higher, transformers 4.51.0 or higher, peft 0.13.0 or higher, and accelerate 1.0.0 or higher. During a baseline smoke test, the model underwent 30 training steps on approximately 350 trajectories using a batch size of one, gradient accumulation, and a cosine learning-rate schedule.

Before and after the training run, the pipeline evaluates tool-calling performance using teacher-forced probes. These probes cut conversational trajectories right before an assistant turn to measure tool-name accuracy and argument-key F1 scores. Finally, the system exports the fine-tuned LoRA adapter, the tokenizer, and a structured JSONL dataset containing the parsed trajectories and corpus statistics for further experimentation.