Cover image by Ji Yume
In this article, we use a case that everyone is familiar with to help you thoroughly understand the “tall” concepts of large language models:
- Pre-training
- Fine tune
- Prompt Word Project
- RAG (Retrieval-Augmented Generation).
This case isThe study exam process that each of us has experienced。
After reading this article, you will have a new understanding of how AI works, and you will be more comfortable using AI in the future.
Tips: This article was edited by DeepSeek based on my recordings, and I only did “fine-tuning”.
Pre-training
At the beginning of each semester, the teacher will give us a bunch of teaching materials and then explain the knowledge points in class.
In this process, we gradually internalize, understand and absorb the knowledge of a semester –This is actually the pre-training process of large language models。
Here’s a key difference: rote vs. genuine understanding.
rote: You can only remember the questions and standard answers, and you will do it when you encounter the original questions in the exam, but you will be helpless if the questions change slightly
Truly understand: You can integrate and deal with different question types and scenarios – this is the core strategy of modern large language models
AI answers = exam
When you take the exam, you will see the questions you will be:
1. Understand the requirements of the questions
2. Call on relevant knowledge in the brain
3. After a series of reasoning and knowledge integration
4. Finally, write the answer word by word
This is exactly the process of AI answering questions！
When you ask the AI a question, it also:
1. Understand your question (prompt)
2. Call on the knowledge learned in pre-training
3. Generate answers verbatim through internal reasoning and integration
Prompt words VS exam questions
Sometimes, you learn too much and too much, and you may not know which knowledge point to associate with a very simple problem.
ThenThe clarity of the question is crucial。
If the teacher clearly states when asking questions:
- Examine the knowledge points of which chapter and section
- What specific knowledge needs to be invoked
- Format requirements for responses
So no matter how messy your knowledge is, as long as the questions are clear enough, you can accurately call the relevant knowledge to answer.
The essence of prompt word engineering
That’s the essence of prompt word engineering！
When you ask the AI:
- The simpler and more vague the question, the more likely the AI is to “answer randomly” (in fact, it is a random call of relevant knowledge)
- The more detailed the question, the clearer the direction, and the more standardized the format requirements, the higher the quality of the AI answers
Tips: AI is not “random answers”, but its knowledge is too complex, and when the question is not clear, it can only randomly select relevant knowledge to answer.
Fine-tuning = doing the real question
Many people mistakenly think that “this knowledge AI can’t do it, just fine-tune it”.This is a misunderstanding of fine-tuning！
Fine-tuning is more like the teacher will take you to explain the real questions before the exam:
You don’t know what questions will be tested in the college entrance examination
But the teacher will explain the correct answers and problem-solving ideas of previous years’ real questions
In this way, you learn “how to answer better”
The essence of fine-tuning is to teach AI to answer better, instead of teaching it new knowledge.
If a certain knowledge point AI has not learned at all, it is useless to do more real questions (fine-tuning)!
Continuation of pre-training = re-reading
The only way to let AI master knowledge points that it would not otherwise haveContinue pre-training(equivalent to student repetition):
- Prepare corpus containing proprietary knowledge (e.g., internal company materials)
- These corpora contain explanations of knowledge and various correlations
- AI acquires new knowledge by continuing to learn these materials
In the same way, if you get a concept that you have never learned in the college entrance examination, even if you are given a reference book, you will not be able to master and answer correctly in a short period of time.
– This is the limitation of RAG (Retrieval Augmented Generation).
RAG = Open Book Exam
RAG is like an open-book exam, and its key is:How to prepare and organize a cheat sheet：
- Bad practice: Bring the whole book directly into the examination room
- The right thing to do: Organize the knowledge points and keyword index in advance, and quickly locate the relevant content during the exam
So do RAG:
- It must be finely cleaned and disassembled to ensure that the fragment is intact
- On the basis of ensuring that the paragraph is complete, try to keep it as short as possible and easy to copy
The current popular “personal knowledge base” product simply splits the document (such as a paragraph every 2,000 words), which may lead to shredding of knowledge points and ultimately low quality AI responses.
Good students Bad students
A large language model is like a “good student with good academic performance”, butGood students don’t always get high scores。
Because it does not have the discernment and judgment of a “good student”:
1. It requires a “good question” (clear prompt), otherwise the knowledge points will be randomly selected to answer the question
2. If the information provided in the open-book exam is wrong or incomplete, it will not be corrected or repaired, but will be copied
3. When encountering knowledge that has not been learned at all, no matter how many “real questions” you do, it will not help
For example, I asked the Claude model “What is MCP” (a new feature officially released by Claude) and it was completely nonsense! Because:
When MCP is released, the model has “graduated” (trained complete)
It doesn’t understand, but it can make it up
summary
Human and large language models inKnowledge learning and outputlogicallySurprisingly consistent。 Understanding this, you can:
- Use AI tools more effectively
- Reasonable expectations of the boundaries of AI’s capabilities
- Choose the appropriate method for different needs (pre-training, fine-tuning or RAG)
Now, do you have a new understanding of how large language models work?
If this article helps you clarify your thoughts, help me brush 666 in the comment area
