Training Pipeline

PAUL Open utilizes a highly curated, two-stage parameter-efficient fine-tuning (PEFT) pipeline optimized for reproducibility on consumer hardware.

The SFT → DPO Pipeline

Base Model

google/gemma-4-E4B-it

SFT Stage

PAUL SFT

180 Examples

DPO Stage

PAUL DPO

65 Preference Pairs

Stage 1: Supervised Fine-Tuning

The SFT stage injects formatting compliance and baseline scientific accuracy. The dataset consists of exactly 180 high-quality, human-verified examples across 10 Indian languages.

configs/training/sft_qlora.yaml

model_name: "google/gemma-4-E4B-it"
dataset_path: "data/train/sft_train.jsonl"
per_device_train_batch_size: 4
gradient_accumulation_steps: 4
learning_rate: 2e-4
max_seq_length: 2048
lora_r: 16
lora_alpha: 32
lora_dropout: 0.05
target_modules: ["q_proj", "k_proj", "v_proj", "o_proj"]
optim: "paged_adamw_8bit"
lr_scheduler_type: "cosine"

Stage 2: Direct Preference Optimization

The DPO stage aligns the model's behavior to specific pedagogical targets without introducing verbosity bias. The dataset consists of 65 preference pairs, including 14 specifically designed corrective pairs.

configs/training/dpo.yaml

model_name: "google/gemma-4-E4B-it"
adapter_path: "results/sft_final"
dataset_path: "data/train/dpo_train.jsonl"
beta: 0.1
per_device_train_batch_size: 2
gradient_accumulation_steps: 8
learning_rate: 5e-5
max_seq_length: 2048
max_prompt_length: 512
optim: "paged_adamw_8bit"