Behind the vigorous development of artificial intelligence, there is a group of “invisible” workers – AI data annotators. They help AI models learn and grow by labeling massive data, but they face career difficulties and future confusion. This paper deeply analyzes the current situation of data annotators: from the influx of high salary temptation to the asymmetry between work intensity and income; From repetitive labor without technical barriers to potential crises of being replaced by AI.
In a café near Beijing 798, AI data annotator Liao Zai repeatedly mentioned the coffee robot in the store during the conversation.
In this nearly 3,000-square-meter café, many baristas work around the central circular island, but the most eye-catching of them is a coffee robot with a humanoid robotic arm. It is said that the robot’s face is modeled after the coffee shop manager.
If the time goes back three or four years, Liao Zai would not have imagined that robots could make coffee, nor would he have imagined that he would enter the AI track.
Born in 99, he had a college degree and worked in a unit within the system in Shenzhen, because he didn’t want his life to be like this, so Liao Zai left his job to study an architectural design-related course. Later, he entered the AI industry from a designer and eventually became an outsourced data annotator for a large factory. Behind the career change, Liao Zai’s income has also risen, and his monthly salary has risen from 3K at the beginning to 13K now.
Soda, which is in the gap period, has also tried to enter the industry.
After graduating with a master’s degree in 985, she had been working smoothly before, but after leaving her job last year due to conflicts with her boss, she entered a long career window. In the past six months, Soda has also thought about changing tracks. The current hot AI industry has made her excited, and data annotators have been regarded as one of the directions of career transformation by her.
To achieve these three challenges, product managers will only continue to appreciate
Good product managers are very scarce, and product managers who understand users, business, and data are still in demand when they go out of the Internet. On the contrary, if you only do simple communication, inefficient execution, and shallow thinking, I am afraid that you will not be able to go through the torrent of the next 3-5 years.
View details >
But after a part-time job, Soda gave up the idea. “This is pure brain-burning physical labor, and there is no room for improvement.” She said to “Fixed Focus One”.
As a job type of artificial intelligence trainer, data annotators were officially included in the National Occupational Classification Catalog in 2020, but the discussion around this career prospect is ice and fire.
On the one hand, during the period of rapid expansion of basic large models, tens of thousands of job seekers attracted by the high salaries and “AI dividends” of large factories, and many training courses under the banner of AI trainers have even emerged across the country; On the other hand, there is uneasiness and anxiety among practitioners, many people feel that they are doing odd jobs for AI, or just become a consumable for large model optimization, which is difficult to form technology accumulation and may be replaced by AI at any time.
Nowadays, as the development of large models shifts from “fighting for the underlying parameters” to “fighting for the landing of the scene”, the demand for this type of work is also changing. The marked positions are no longer as “batch volume” as in the past, but are replaced by more vertical needs and stronger professional thresholds. Liao Zai, who successfully transformed, and Soda, who withdrew, are two typical footnotes under this AI wave.
01 Three postures of “screwing”: the secret layering of data annotators
If you want to enter the AI industry, data annotation may be the position with the least threshold – you can easily find a part-time job on the Internet.
“Fixed Focus One” experienced a part-time video review project on a crowdsourcing platform, tasked with annotating data for vending machines. Before officially taking up the post, job seekers must first enter the group for a round of training – annotating 500 videos, and the accuracy rate is more than 90% to pass the assessment. After the official order is received, it is charged in the form of piecework, and the cost of each order fluctuates between 0.04 yuan and 0.1 yuan, and the wrong label will also deduct money.
Each piece counting video is about ten seconds long, and it is necessary to identify the type and quantity of goods that customers take from the vending machine. The task may seem simple, but it is not easy to do. The packaging of many drinks and snacks is very close, and it is easy to misjudge at night. “Prime One” tried to mark 20 videos, which took 25 minutes, and only 14 were completely correct.
The teacher in charge of training in the group repeatedly encouraged everyone: it is normal for the error rate to be high at the beginning, but later you will become more and more proficient and have a higher and higher accuracy rate, and you can do up to 3,000 videos a day after becoming proficient.
But people who have done similar part-time jobs complain on social media: I really can’t do it for too long, and my eyes can’t stand it. In the group of nearly 200 people marked as 11 groups, people are constantly withdrawing and joining, like a never-ending virtual assembly line.
Soda is also in a similar WeChat group.
Not long ago, she saw the data released by a major domestic manufacturer on the recruitment platform marking part-time positions. There is no limit to major, no limit to experience, the only threshold is academic qualifications – must be 985/211 master’s degree or above.
This part-time job is to score the thinking process and output results of large models. Whether the output is correct or not, whether it takes into account the user’s emotions and feelings, and whether the thinking process is logical and efficient all need to be taken into account.
After Soda passed the screening, he was also pulled to a WeChat group. Similarly, training and testing are required before officially accepting orders.
Soda received a document of dozens of pages detailing the scoring dimensions and judging criteria. According to this scoring system, she needs to conduct two or three rounds of trial bidding first, and then she can take orders after meeting the standard. After passing the test, the accuracy rate must also be ensured in the formal labeling process. If the accuracy rate is below average, you will lose your labeling qualification and will need to be retested.
According to Soda’s observation, the pass rate of the test in her group is not high.
“The difficulty of this job is that the cost of memorization and understanding is particularly high. Before labeling, you must first understand and remember their evaluation system and scoring criteria. “What makes Soda even more uncomfortable is that these standards are not fixed. Sometimes, when faced with similar questions and answers, she scores with the same way of thinking, but the result is diametrically opposed.
It’s like writing test papers with no standard answers, unable to improve the accuracy rate through self-effort or learning, you can only keep spinning in place, consuming your brain and physical strength, and the final reward is minimal. Soda told “Fixed-Focus One” that this part-time job is also charged on a piece-rate basis, and the cost of marking one piece is only 3-7 yuan.
Lucky than Soda, Liao Zai does not have these strict KPIs and assessment standards.
Liao Zai participated in the outsourcing project of another major Internet company in China. He leads a group of 10 annotators. In the project, there are several such groups to evaluate, appraise, and specify annotation rules for the large models of the large factory. Liao Zai will assign tasks that need to be marked every day, and then tell the team members specific rules and judging criteria to ensure objectivity. In addition to data annotation, he also needs to communicate with the algorithm team and product R&D team to adjust the evaluation and appraisal of the model based on upstream and downstream feedback.
Liao Zai still uses coffee robots as an example, if you want AI to make coffee, then you need to tell it the entire link, including how to plant coffee trees, what types of coffee beans are, what is the molecular structure, how to grind, etc. Through the data annotation of each step, it is adjusted, and then returned to the model to train autonomously.
Three types of data annotation work can roughly outline the invisible layering behind this profession: vending machine annotation, testing “physical strength + attention”, relying on repetition and proficiency to improve efficiency; Scoring the thinking process and output results of large models requires strong understanding and memory, such as answering test papers without standard answers; Large model evaluation undertakes process management and communication work in addition to annotation, and has a certain degree of autonomy.
Some people often compare data annotation to “screws” on AI assembly lines. In Liao Zai’s view, even if he screws screws, at least he knows what tools to use and how to screw them more efficiently.
02 Awkward position: important, but cheap
Standing higher in the industrial chain, Jackson can examine the significance of data annotation from a more complete assembly line.
Jackson graduated from a prestigious overseas university and is now engaged in basic model training at a technology company in Shanghai. He told “Fixed-Focus One” that model training mainly consists of three parts: pre-training, supervised fine-tuning and reinforcement learning.
The amount of data required for pre-training is often more than ten terabytes, mainly from public crawler data, model synthesis data, third-party procurement data or enterprise-owned data. This stage is less dependent on manual annotation.
The data annotator is mainly involved in the last two stages.
The goal of the Supervised Fine-Tuning (SFT) stage is to adapt the pre-trained general-purpose language model to specific tasks or conversational scenarios, making its output more in line with human expectations. In short, it is how the church model “answers” after entering specific data.
At its core, Reinforcement Learning from Human Feedback (RLHF) uses human preference data to optimize model output quality.
In more layman’s terms, SFT is to write an answer for AI to learn and imitate; RLHF, on the other hand, helps the AI choose an answer that is more in line with human preferences after giving a few answers.
Most of Liao Zai’s work belongs to the former, which is difficult to quantify; The work of soda is the latter, which can be assessed on a piece-by-piece basis. Simpler data collection tasks such as vending machine annotation mentioned earlier will soon be replaced by AI.
Jackson said that some automated means can be used in the fine-tuning and enhancement stages, or data generated by other models, but the diversity, correctness and professionalism of its content may not be as good as manually annotated data. It’s like the content generated by DeepSeek is instantly visible.
“The best effect is definitely all manually annotated, but (AI company) bosses care more about cost than making a perfect model. It is also acceptable to use the model to synthesize a suboptimal version. ”
According to Jackson’s estimates, a complete fine-tuning and reinforcement training requires hundreds of thousands of pieces of data, and the model will be updated and iterated, and the demand for data will accumulate exponentially. According to his observation, at present, there are only a few top manufacturers in China that have the financial resources to do manual data annotation, and most of the other teams use other people’s models to generate data.
According to public information, ByteDance’s investment in AI will reach 80 billion in 2024 alone, and this number will double to 160 billion in 2025. In February this year, Alibaba Group CEO Wu Yongming announced that in the next three years, Alibaba will invest more than 380 billion yuan in building cloud and AI hardware infrastructure.
But even these top players must be careful in all aspects. As a part of cost controllability, data annotation has become the norm for large manufacturers to choose to outsource and crowdsource.
Soda’s part-time workload is about 3-4 hours a day, and she calculated the hourly wage, which is between 30-60 yuan. Soda said that these three or four hours must be fully focused, and no moisture can be squeezed out. Such a dedication and reward, if you are not interested in this industry, it is really difficult to persevere.
But the WeChat group where Soda is located continues to enter people every day. “If you don’t do it, some people are willing to do it, and the price will naturally not go up.”
The essence of the problem is not that data annotation is not important, but that there is a lack of technical barriers for this kind of work. The generation and optimization of large models is a very refined process. Each piece of data is like a stitch on a rag doll or a hair on a zebra, and it is difficult to discern its meaning to the whole. On this assembly line, it is difficult for annotators to accumulate “exclusive advantages” in personal ability, and it is very easy to be replaced.
Without barriers, it is difficult to have bargaining power.
Judging from the public information on the recruitment website, the daily salary of part-time data annotators is mostly between 120-500 yuan, and most of the monthly salaries of outsourcing positions are between 9-17K. The monthly salary for official positions in several large factories is between 15-25K. Compared with technical positions and algorithm positions, such salary levels are not high.
03 Replaced by self-trained AI: Who can break through the pyramid?
Because of the lack of growth, Soda finally gave up his part-time job and did not plan to invest in any positions related to data annotation. To this end, she also consulted a friend who has been engaged in AI data annotation for many years.
Before the big model became popular, this friend joined a large model team in China, and later changed jobs to another big factory. In the sunrise industry and high-paying positions, many people envy her for stepping on the outlet, but she advises Soda to deliver this position carefully. Because the career development space of data annotators is limited, it is difficult to jump into the real core link of the AI industry.
Jackson shares a similar view.
He used a pyramid to describe the stepped distribution of current AI practitioners: the bottom of the tower is the annotation, the waist is the application, and then the top is fine-tuning and post-training, and the top of the tower is the basic model design and pre-training. “Now it is basically the background that determines everything, and it is difficult to break through from the bottom of the tower layer by layer.”
The so-called background refers to academic qualifications and academic background. For example, for many positions, education is a hard threshold. Jackson analyzed that the application level requires a bachelor’s degree, a master’s degree in the fine-tuning and post-training stage, and the basic models are basically doctorates.
Take his algorithm position as an example, finding a job depends on several dimensions: academic qualifications, internships, competitions, and papers. The AI circle pays special attention to academic backgrounds. If you don’t have an excellent paper, even if you graduate from a school with a good ranking, it will be difficult to enter the AI team of a large factory.
“Most of the people standing on the top of the Golden Tower are doctors from top schools and need to publish a lot of papers.” He concluded.
At the same time, the model itself trained by the annotators is quietly competing with the annotators. Will it be replaced by AI and become the sword of Damocles hanging over the heads of the annotators?
Jackson pointed out that in some mature text models, the data synthesized by the model has replaced 80% of the manual annotation. The logic behind this is that when the model is not strong, the demand for annotation is great; If there are more annotations, the model will become stronger, and AI will replace the annotator in this task or field.
In some high-tech companies overseas, this situation has already happened.
According to Bloomberg, Apple closed a team related to Siri’s artificial intelligence business in January 2024. They were originally responsible for listening, analyzing, annotating, and understanding user needs based on the data generated by the user’s interaction with Siri. Also because of the significant improvement in automatic annotation capabilities, in June 2022, Tesla laid off 200 U.S. employees who annotated videos to improve assistance systems.
On the other hand, the changes in the strategy of large factories also affect the career prospects of data annotators.
At the beginning of 2023, the basic large model is the battlefield for all technology giants to compete in, and large manufacturers such as Baidu, Byte, Alibaba, and Tencent have bet on self-developed large models in a high-profile manner, and data annotation has once become an indispensable basic position.
But in 2024, the race has cooled significantly. A number of major manufacturers have successively adjusted their focus and begun to shift from “making models with larger parameters” to “making models truly land”.
This shift also directly affects the job supply and budget arrangement of the basic type of work marked by data. As a result, the data annotation requirements used to support the training of basic large models may be compressed. In the future, enterprises will no longer need thousands of “people who can standardize data”, but “people who understand business and models”.
Of course, the demand has not completely disappeared. On the one hand, Jackson explained that with the development of AI technology and the further implementation of large models, a large number of application scenarios will be generated. Whenever a new scene appears, you need to find someone to mark the data. The demand for data annotation will still exist for a long time and in large quantities. On the other hand, according to the “Intelligent Data Industry Development Observation Report” released by Tsinghua University, the number of enterprises with labor needs in the data annotation industry in 2024 will increase from 457 in 2023 to 1,195. According to IDC data, the market size of Chinese intelligent basic data services will exceed 12 billion yuan in 2025, with an average annual compound growth rate (CAGR) of about 47% from 2019 to 2025.
However, these growths are more of a “horizontal increment” – that is, the expansion of data annotation demand brought about by the new scenario, rather than the upward channel of “annotators” as the type of work itself. For the vast majority of practitioners, what they do is still work for the assembly line.
Liao Zai, who has already been “robbed” by AI once, is full of confidence in his professional future.
Before coming to Beijing, Liao Zai worked as a designer in a design company in Shanghai for two years. At that time, the impact of AI on the design industry had begun, and Liao Zai’s company had to transform to AI and decided to make a large customer service model. He took the initiative to invite Ying to participate, and this AI project opened the door to a new world for him.
Later, he left the company to learn more systematically about AI. After the Spring Festival this year, he joined his current company. No matter how late he gets off work every day, Liao Zai will learn AI-related content for two hours, and he also opened a Xiaohongshu account “Fried Mad Rabbit” to record his AI experience.
“Everything that happens is beneficial to me.” When communicating, Liao Zai repeatedly quoted this ancient saying.
The staff of the coffee shop will send some new products to taste from time to time, and the service is meticulous and appropriate. And the eye-catching coffee robot did not brew a cup of coffee in the afternoon. At least at this stage, the robot is more of a decoration for this café. Although the future is uncontrollable, human initiative is always key.
* Liao Zai, Soda, and Jackson in the article are all pseudonyms.
