OpenAI fine-tuning end to end: JSONL prep, validation, job, checkpoints, deploy

TL;DR

Set a suffix so the resulting model name is recognizable in your project. Build the dataset in JSONL: one complete JSON object per line, each a messages array showing the behavior you want. Upload with purpose set to fine-tune. If the problem is missing knowledge, fix retrieval first.

When to use

You are seeing this: Fifty great examples beat five hundred mediocre ones. Use this skill when you run into "OpenAI fine-tuning end to end: JSONL prep, validation, job, checkpoints, deploy".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.

Fine-tuning, phase by phase

The steps

1. Build the dataset in JSONL: one complete JSON object per line, each a messages array showing the behavior you want. Fifty great examples beat five hundred mediocre ones. Strip anything the model should not memorize: secrets, personal data, internal URLs.
2. Validate the JSONL before upload: every line parses, every example has the message roles the model expects, no empty strings, no duplicated examples. Most failed jobs trace back to malformed lines the uploader never checked.
3. Upload with purpose set to fine-tune. The purpose matters; files uploaded for other purposes cannot be used for training.
4. Create the job with the supervised method. Leave hyperparameters at defaults for the first run; tune only after you see the baseline learning curves. Set a suffix so the resulting model name is recognizable in your project.
5. Watch the training curves. If loss drops fast then the validation metrics stall, the model is memorizing your dataset. That is what checkpoints are for: OpenAI saves a full model checkpoint at the end of each epoch.
6. Evaluate every checkpoint on your eval set, not just the last one. Pick the checkpoint with the best eval score, which is often an earlier epoch. Deploying the final checkpoint by default is how overfit models reach production.

The trap

Fine-tuning before trying a better prompt or RAG. Fine-tuning teaches style and format reliably; it teaches new facts unreliably. If the problem is missing knowledge, fix retrieval first. If the problem is tone, format, or consistent behavior, fine-tune.

Checklist

• JSONL validated line by line before upload.
• Upload purpose is fine-tune.
• First run uses default hyperparameters.
• Every checkpoint evaluated; best checkpoint deployed, not just the last.
• The eval that justified the fine-tune still passes on the deployed model.

OpenAI's supervised fine-tuning guide documents the JSONL format, the fine-tune upload purpose, the method parameter, and per-epoch checkpoints for overfitting. Verified live at https://developers.openai.com/api/docs/guides/supervised-fine-tuning.