The modern AI developer often encounters a frustrating paradox during the fine-tuning process. You spend weeks curating a specialized dataset to make a model an expert in a specific domain, such as legal analysis or medical coding, and the initial results are exhilarating. The model nails the domain-specific jargon and follows the required format perfectly. However, the moment you test that same model on a basic logic puzzle or a middle-school math problem, the system collapses. It is as if the act of learning a new professional skill has caused the model to suffer a sudden, catastrophic loss of basic cognitive function. This intelligence trade-off has become a silent bottleneck in the deployment of specialized LLMs, leaving teams to choose between a narrow expert that cannot think and a generalist that cannot specialize.

The Mechanics of Reasoning Suppression

This phenomenon, known as reasoning suppression, is vividly illustrated in the performance of the Amazon Nova 2 Lite model during vanilla Supervised Fine-Tuning (SFT). In controlled experiments, the base model starts with a mathematical reasoning proficiency of 70%. Yet, after undergoing standard SFT to improve performance on a target task, that score plummets to an average of 6%. This is not a marginal dip; it is a total erasure of the model's internal reasoning capabilities. The danger here is that a practitioner might see the target task accuracy increase and assume the fine-tuning is a success, unaware that they have effectively lobotomized the model's general intelligence.

Even attempting to maintain a reasoning mode during training does not solve the problem if the data is flawed. When the training dataset consists solely of input-output pairs—lacking any intermediate steps or Chain-of-Thought (CoT) traces—the model loses its ability to reason regardless of the settings. The contradiction is stark: while the target task performance improves, the model's capacity for independent thought vanishes. This proves that the content of the data, rather than the consistency of the training configuration, is the primary driver of reasoning preservation.

The root cause lies in the loss function used during training. In a standard SFT setup, the loss function calculates penalties for both reasoning tokens and final output tokens. If the training data contains no intermediate reasoning steps, the model receives no supervisory signal to generate a logical path toward the answer. Consequently, the loss function treats any attempt by the model to reason as a deviation from the target output, effectively penalizing the reasoning tokens. To minimize loss, the model learns to bypass its internal reasoning mechanisms entirely, jumping straight from the prompt to the answer. This is a classic case of shortcut learning, where the model relies on superficial correlations in the data rather than robust logical patterns. Once this reasoning engine is suppressed, the damage extends beyond math, degrading the model's ability to handle complex coding tasks and intricate logical structures.

SDR and the End of the Intelligence Trade-off

For most organizations, the obvious solution—creating a massive dataset of high-quality, human-verified Chain-of-Thought traces—is a practical impossibility. The cost and time required for experts to manually write out every logical step for thousands of samples create a scaling wall that few teams can climb. This is where Self-Distilled Reasoning (SDR) changes the equation. Instead of relying on external human labels or an expensive teacher model, SDR leverages the latent reasoning capabilities already present within the base Amazon Nova 2 Lite model.

SDR operates by using the model's own generated reasoning traces as a form of in-training regularization. When faced with a simple input-output pair that lacks a reasoning path, the model generates its own internal thought process and then uses that trace to augment the training data. This process ensures that the model does not fall into the trap of simple memorization. By forcing the model to re-learn the paths it used to arrive at the correct answer, SDR physically prevents the destruction of the reasoning mechanism during the SFT process. Unlike traditional knowledge distillation, which requires a separate, more powerful teacher model to provide the ground truth, SDR is a self-contained loop.

One of the most significant practical advantages of SDR is that it eliminates the need for post-hoc interpolation. There is no need to blend weights or perform complex averaging after training is complete. Because the regularization happens during the learning phase, the model preserves its base intelligence while simultaneously absorbing new domain expertise. This makes SDR an immediately applicable strategy for any SFT pipeline, regardless of the domain, providing a path to specialization that does not require sacrificing the model's foundational cognitive abilities.

Outperforming Model Merging in the Real World

Before the advent of SDR, the industry standard for recovering lost general performance was model merging. This involves taking the SFT checkpoint and mathematically blending its weights back with the original base model. While this can revive some of the lost reasoning capabilities, it introduces a structural trade-off: the more you lean toward the base model to recover general intelligence, the more you dilute the specialized expertise gained during SFT. You are essentially searching for a middle ground where the model is mediocre at both general reasoning and the target task.

This struggle is rooted in catastrophic forgetting, where the intense focus on a narrow dataset wipes out the broad knowledge base of the model. Model merging is a palliative measure, not a cure. In tests using Amazon Nova 2 Lite across five different benchmarks, SDR proved to be fundamentally more efficient than merging. In terms of general math performance, both SDR and model merging managed to keep scores around the 68% to 70% range. However, the divergence appeared in target task performance. SDR achieved an average increase of 6.5% over model merging in target accuracy.

This 6.5% gap is critical because it proves that general intelligence and domain specialization are not a zero-sum game. While model merging is a simple arithmetic average of two states, SDR is a dynamic regularization of the learning process. By preventing the reasoning engine from being turned off in the first place, SDR allows the model to reach a higher peak of specialization without the cognitive collapse associated with vanilla SFT. This aligns with recent research suggesting that self-distillation is one of the most effective mechanisms for mitigating catastrophic forgetting in large-scale models.

Validating SDR Across Multimodal Benchmarks

To prove the versatility of SDR, it was tested across five diverse benchmarks that simulate real-world production environments. The ToolACE benchmark, utilizing 10,000 samples, focused on tool-calling accuracy and argument inference. Using a strict success rate where the function name and all arguments must match the ground truth exactly, SDR demonstrated that the model could maintain precise structured output capabilities even in short-context environments of approximately 2,000 tokens. This confirms that the model can handle the rigid requirements of API specifications without losing its logical grip.

For long-context processing, the CoCoHD benchmark used 730 samples consisting of US Congressional hearing records ranging from 35,000 to 85,000 characters. The task required extracting 10 specific metadata fields, such as committee names and attendees, into a JSON format. SDR's ability to parse these massive documents and map them to a strict schema without losing information highlights its utility in automated data pipelines where precision is non-negotiable.

Similarly, the GovReport benchmark tested the model's ability to summarize US government reports (GAO, CRS) of similar lengths (35,000 to 85,000 characters). Using the ROUGE-L metric to measure the overlap between the generated summary and the ground truth, SDR showed that the model could synthesize dense policy documents into concise 250-to-450-character summaries without omitting core facts. This demonstrates that the preservation of reasoning directly translates to better abstractive summarization in high-stakes domains.

Finally, the effectiveness of SDR extended into multimodal inputs. The Invoice-OCR benchmark used 500 samples to extract structured fields from scanned invoices, measured by Average Normalized Levenshtein Similarity (ANLS). Meanwhile, CaptionGen used 1,800 samples to evaluate the model's ability to describe video clips in detail. In both cases, SDR enhanced the target task performance while maintaining the underlying intelligence of the Amazon Nova 2 Lite base. Across text, images, and video, the result was consistent: SDR allows for specialization without degradation.

For practitioners, the strategy for maximizing SFT performance is now clear. The pursuit of Golden CoT data—perfectly curated reasoning chains—is often a waste of resources. Instead, the focus should shift to a workflow of data preparation, SDR-augmented SFT, and final verification in reasoning mode. By using the base model's own intelligence as a scaffold, developers can finally stop choosing between a model that knows the answer and a model that knows how to think.