Fine-Tune SmolLM2 on the SupraLabs Reasoning Corpus

SupraLabs has detailed a complete Google Colab pipeline designed to transform its massive multi-model reasoning corpus into a compact, reasoning-focused language model. The workflow demonstrates how developers can stream a representative subset of the SupraLabs/reasoning-corpus-4K-5M-v1 dataset directly from the Hugging Face Hub, avoiding the need to download the entire dataset. This streamed data is then used to adapt the HuggingFaceTB/SmolLM2-135M-Instruct model using Low-Rank Adaptation (LoRA) through the TRL library's SFTTrainer.

Before training begins, the pipeline applies a series of quality-filtering heuristics to ensure high-quality training examples. The workflow filters out samples with unsuitable token lengths, keeping only those between 200 and 3,000 tokens. It also discards empty or near-empty responses by requiring thought traces to exceed 100 characters and assistant answers to exceed 20 characters. To prevent looping behaviors, a repetition filter drops traces where a single line repeats more than 30% of the time. Additionally, a reasoning ratio filter ensures that the reasoning content comprises between 15% and 97% of the total response.

The retained samples are formatted into structured chat templates containing a system prompt, user query, and an assistant response enclosed in explicit think tags. The training process runs on a single T4 GPU, taking approximately 10 to 20 minutes to complete. Once fine-tuned, the SmolLM2 model can generate step-by-step reasoning before outputting its final answer. The pipeline concludes by testing the model on logic and arithmetic problems, separating the generated thinking process from the final answer, and exporting the curated training and evaluation subsets as Parquet files.