Fine-tuning is a dataset with a training run attached
Fine-tuning an open source language model without a PhD means taking a model you can download and continuing its training on examples you prepared, so it follows a format or a vocabulary your base model keeps missing. It is a reasonable project when you have a few hundred real examples and a failure you can describe. It is a bad project when you have twenty examples, a vibe, and a hope that the model will become an expert in your company.
You do not need a doctorate. You do need to be honest about data, and you need to be able to run the base model first. If you have not done that, stop and follow the beginner local guide or the Mac guide. Fine-tuning a model you cannot run is a tutorial you are copying.
Decide whether you should fine-tune at all
Try these cheaper fixes first, in order.
- A clearer prompt with two examples pasted in. Many "we should fine-tune" problems die here.
- A smaller task. Ask for a label, not an essay.
- A retrieval note: put the policy text in the context instead of baking it into weights. Policies change. Weights linger.
- A different base model in the same size class.
Fine-tune when the model already sees the right context and still will not hold a format, a tone you have examples of, or a set of labels, and you can show the before and after on a set you did not train on.
Do not fine-tune to memorize private facts you could have looked up. That is how a model leaks a customer list in a demo. Retrieval keeps the list in a store you can delete. Weights do not have a tidy delete button for one fact.
What "without a PhD" still requires
You need examples that look like the real task, a way to score them, and a machine or a rented GPU you understand the bill for. The budget setup guide is about running models, not training them. Training a large model does not fit that ceiling. Training a small adapter on a modest file sometimes fits a single rented session that you turn off. Read the rental price before you start the clock.
You need a base model whose license allows fine-tuning and whose weights you can actually obtain. "Open" is used loosely. Read the license. If you cannot use the result commercially, do not spend the weekend.
You need logs. A run you cannot repeat because you forgot the learning rate is a hobby, not a model you will ship.
Build a small, boring dataset
Write pairs: the input the model will see, and the output you wish it had produced. Take them from real work with names removed, or write them by hand. Do not ask a larger model to invent the whole set and then train on the invention. You will teach your small model to imitate the large model's habits, including the bad ones, and you will think you created data.
Aim for a set you can read in an afternoon. If you cannot read it, you cannot trust it. Split off a chunk you will not train on. That chunk is the only opinion that counts. If you score the training examples and celebrate, you have measured memory.
Remove duplicates. A example that appears twenty times will look like a behavior. It is a photocopy.
Label the data with its source and date in a README next to the file. Future you, or a teammate, will need to know whether a customer consented to this use. If you cannot write that sentence, you do not have permission. Stop.
A training loop that stays small
Use a maintained tool for adapters (people say LoRA and mean a small set of extra weights) rather than a from-scratch training script you found in a gist. Follow that tool's current quickstart for the exact base model. When the quickstart and a blog disagree, believe the quickstart you can run.
Change one setting at a time if the first run fails. Save the command line. Train for a short run first so you can see whether the loss number is even moving, then decide whether a longer run is worth the rental.
Watch for a model that scores well on your training lines and worse than the base model on the held-out lines. That is overfitting. It means you taught it your examples and damaged everything else. Keep the base model. A worse general model with a cute party trick is not a successful fine-tune.
Evaluate like a user
Make a sheet of twenty held-out inputs. Columns: base model output, fine-tuned output, which one you would actually send, and why. If you would not send the fine-tuned output more often, you are not done, and you may be finished anyway. Shipping the base model with a better prompt is a valid result. Write it down so the next person does not repeat the run.
Include inputs that should be refused or handed to a human. A fine-tune that becomes more willing to answer medical or legal questions is a regression, even if your tone examples look friendly. The legal research guide is a reminder that format is not advice.
Serving the result
A fine-tuned adapter still has to be loaded by something. Document the base model version and the adapter file as a pair. If you update the base and keep the adapter, you may get nonsense. Pin both.
Do not delete the base model's behavior you still need. If the fine-tune only helps one workflow, call it only for that workflow. Routing every chat through it is how a format win becomes a general loss.
The glossary defines the words the quickstart will throw at you. Read it before you change three numbers you cannot define.
Frequently asked questions
Q: Can I fine-tune an LLM without a machine learning degree?
A: Yes, if you can run the base model, prepare a few hundred real examples, and follow a maintained adapter quickstart. The degree is not the gate. The held-out test is the gate.
Q: How much data do I need?
A: Enough real examples to cover the behavior, and a separate set you did not train on. A handful of prompts is a prompt. It is not a training set. If you cannot read the whole set in an afternoon, it is probably too messy to trust yet.
Q: Is fine-tuning better than a long prompt?
A: Only when the long prompt already fails on examples you can score. Try the prompt and a retrieval note first. They are reversible. Weights are harder to debug.
Q: Can I do this on a laptop?
A: Sometimes, for a small adapter and a small base model, slowly. Often you want a short rental and then you download the adapter. Turn the rental off. The cost guide applies to GPU clocks too.
Q: What license issues matter?
A: Whether you may fine-tune, whether you may use the result for your purpose, and whether your training text had its own license or a privacy promise. Read all three. A tutorial will not.
Q: Can I fine-tune on my company's documents?
A: Only with permission, with secrets removed, and usually you should retrieve those documents instead of training on them. Policies change. A baked-in policy is a bug with a ship date.
Q: Why did the model get worse?
A: It memorized the training lines or you trained too long. Compare to the base model on held-out lines. If the base wins, keep the base.
Q: Do I need a PhD to know it worked?
A: No. You need twenty examples you did not train on and a column that says which output you would send. If you cannot fill the column, it did not work yet.
Keep the base model
Fine-tuning is optional even after you understand it. Run the base model, try a better prompt, then build a dataset you can read. If the held-out sheet prefers the adapter, keep the pair and document it. If it does not, you learned that for the cost of a careful weekend. The blog and the directory are for the hosted tools that remain the right answer when the data is allowed to leave your disk.
