Evaluation Architecture
A strict, three-layer evaluation framework with immutable frozen suites to ensure target capabilities are acquired without catastrophic forgetting of the baseline model's strengths.
Layer 1: Canonical Baseline Benchmark
FrozenImmutable research anchor. Provides the primary comparison point against unmodified google/gemma-4-E4B-it.
Layer 2: Capability Preservation Suite
FrozenMonitors regression across baseline Gemma 4 strengths like teacher assistance, safety, and complex scientific explanation.
Layer 3: Held-Out Behavioral Suite
FrozenMeasures whether fine-tuning acquired the 4 target behaviors: Socratic tutoring, concise calculation, clean translation, and Indic pedagogical tone.
Strict Isolation Protocol
Under no circumstances may any case from these suites be included in, paraphrased for, or referenced during SFT or DPO dataset generation. Every proposed training batch must pass scripts/check_leakage.py with zero exact matches, zero parameter collisions, and < 35% n-gram overlap before inclusion.