With the rise of large model technology, many companies have invested resources to develop related application products in an attempt to occupy a place in this wave. However, with the gradual ebb of market popularity, products that can truly meet customer needs and create value have gradually emerged. Based on their practical experience in product development for large model applications, the author shares key decisions in product positioning, functional design and technology selection.
This year, almost all in products for large model applications.
Now the popularity of large models has shown signs of ebating, not as popular as last year, and everyone is discussing it. However, after this low tide, the opportunities that really belong to large model products emerge one by one, like shells on the beach.
The development of any thing will have a period of frenzy, and everyone will tout it and enter a stage of “god-making”.
At this stage, the product you make is a “toddler”, and the product in the eyes of the customer must be a “superhero” who can go to heaven and earth and do anything, so it is difficult for your product to meet the customer’s expectations.
When the first batch of customers who spent a lot of money to try it out, the freshness passed and they calmed down, and they entered the stage of “de-charming”.
At this time, the customer is like a rational adult, no longer blindly following the trend when choosing products, and has a reasonable expectation for the product, and will not make those unrealistic requirements.
But we are in this wave, how you should position the product and what kind of product route you choose will determine how far you can swim after the wave recedes.
Based on my experience in making products in the past few months, I would like to share with you some key decisions I made to make large model application products.
1. Is it okay to rely entirely on large models?
Let’s talk about what this big model can do. It’s like a super expert, data processing, feature extraction, article generation, and other tasks. If you make a software application, then basically cover all the functions that the product needs to use.
Take the document generation system we are working on as an example, users only need to pass the materials up, and all kinds of documents will be automatically generated at once, as if by magic.
As soon as such a good product comes out, which customer can not be moved? This is simply the “accelerator” of work efficiency and the “liberator” of productivity.
How can product managers do a good job in B-end digitalization?
All walks of life have taken advantage of the ride-hail of digital transformation and achieved the rapid development of the industry. Since B-end products are products that provide services for enterprises, how should enterprises ride the digital ride?
View details >
Therefore, when we first designed the product, we thought in our hearts that we simply relied on large models to achieve various functions.
So how to achieve it?
In fact, the principle is very simple, just like playing the game of “you ask me and I answer” with a large model. Let’s feed it the data one by one, and then ask questions through the prompt words, after it gives the answer, we will directly display the answer on the system page, live together!
However, we soon rejected this plan.
Why is this?
First of all, the problem is the size limit of the data input to the large model.
Those big models on the market are like having a small belly, which can only hold a certain amount of things. Depending on the size of the model parameters, the length of the understandable context is about 10,000 to 200,000 words.
We chose a 7B small parameter model, which can only understand 10,000 words at most. If you encounter a document with hundreds of pages of documents, it will be directly “confused” and will not be able to give an answer at all.
So, why not use a model with larger parameters?
This idea is good, but the problem is that the larger the model parameters, the higher the requirements for graphics card resources. A small 7B model with a 4090 graphics card can make it run. If you change to the 70B model, it is estimated that you will need 4 to 8 4090 graphics cards to serve it.
And we need to consider the production cost and the purchasing power of customers when making products, if the cost of purchasing graphics cards alone is one or two hundred thousand, the product will not be affordable for too many customers.
Secondly, there is another problem that cannot be solved at present, that is, large models are particularly prone to “confusion” and “hallucinations”.
To put it simply, it often “answers the wrong question”, and if you ask the same question ten times, it can give you ten different answers.
The stability of this product cannot be guaranteed, we can’t let customers use the product by luck, today is easy to use, tomorrow is not easy to use, then we must not make the customer angry away?
In this case, we cannot completely rely on large models when making products.
This is like when the pure tram first came out, most people had “mileage anxiety”, and they were afraid that they would run out of power on the highway, which would be embarrassing. So, later there was a “range extender” electric vehicle, which has the advantage of electric power and does not have to worry about running out of power.
In the same way, now that the large model has not been able to output the answer stably, we have to consider using the “large model + engineering” method to implement the function, so as to make the product more reliable!
2. Everyone makes “big and complete” products, how to choose?
At the beginning, we used large models to make products, and we thought in our hearts that we would start with a business scenario, just like playing a puzzle, first figure out this small piece, summarize the experience, and then engage in more business scenarios, and piece by piece to piece together a complete big picture.
As a result, time waits for no one. We are still doing a subdivision business, and the customer side has reported that a company has made a full-scenario product.
Our product has not yet seen the shadow, and it has already been left behind, and it is out of the question.
At first, I also felt very anxious, and muttered in my heart: Others are making such big and complete products, is it necessary for us to continue on this subdivision scene? Isn’t this going into a dead end?
The customer obviously wants a big and complete product, after all, he pays attention to the overall situation, and wants to make the advertised things look like they have everything
Later, in the communication with a customer, I found that from his work perspective, he didn’t care about the complete business covered by our products at all, he only cared about the business he was responsible for, and whether the effect of our product was good or not, whether he could worry less about snacks.
Now it’s good, there are two ways to choose products in front of us: one is to take the “big and comprehensive” route that the leader likes; the other is to take the “small but fine” route that business personnel are satisfied with.
Returning to the value level of making large model products, I found the answer.
In my opinion, the core value of large model products is to improve work efficiency and change productivity, so it will be sought after by everyone.
This kind of product does not bring any innovation in business models, nor is it a “management-oriented” product forced by policy. It is a real tool of production, just like a hammer or pliers in the hands of a worker. Since it is a tool, the criterion for judging whether it is good or not is to see if it can greatly improve production efficiency.
How can the product improve work efficiency?
The key depends on the accuracy of the generated content. The higher the accuracy, the more worry-free you will be when you use it, and you don’t have to change it here and there.
With limited resources, we have two choices: either make a “big and complete” product that can achieve a 60-point effect; or make a “small but refined” product that can reach 90 servings in a small number of scenarios.
There is a prerequisite here, that is, a general large model that has not been tuned and trained can only achieve a level of 30~50 points in a certain vertical field.
With such an analysis, from the perspective that the product must have stronger vitality and bring higher use value to customers, we decided to focus on a small number of scenarios and focus on the accuracy.
Later, after the “big competition” between products, it really proved that the direction we chose was right.
Publicity and publicity are all temporary, just like setting off fireworks, watching the excitement, and then disappearing in a while. Whether the product is good or not is decisive, just like good wine is not afraid of deep alleys, as long as the product is easy to use, customers will naturally come.
3. The customer wants a “dialog box”, do you want to do it?
Once when we reported on a product, a friend gave us a “precise positioning” of our product: we obtained answers in an “element-based” way, and they used a “question-and-answer” method to achieve interaction.
What does it mean?
Translate: If we want to make a big dish with full color and flavor (that is, a document with higher accuracy), we have to prepare various ingredients (the elements required for the document) in advance in the early stage, and extract these elements like cutting vegetables and preparing ingredients. Then combine these ingredients in a certain order and method to fry the dish one by one.
So what about the “question and answer” interaction?
To put it simply, it is to put an AI Q&A robot in the system, and if the user has any questions, tell it directly, and it will give you the answer immediately
Now we use Chat GPT, Deepseek, Wen Xin Yiyan, Doubao, etc., don’t they all have a Q&A input box, just ask what you want, and it will give you the answer.
Just because everyone is used to using these AI Q&A applications, I asked customers if they needed to have such a Q&A robot in our products, and everyone said in unison: Yes!
Now it’s good, it’s the “crossroads” of product decision-making: competing products have this thing, and customers also want it, so should we make this Q&A robot?
Let’s talk about the answer first: don’t do it.
Why don’t you do such a function that everyone needs?
I asked myself the following three questions:
1. Do users really want a Q&A tool?
In fact, no, what customers really want is the final result. Q&A is just a process, and only when you give him inaccurate results can he take the initiative to ask questions.
2. How often can users use Q&A robots?
It definitely can’t be high.
On the one hand, many users don’t ask questions at all, why? Because they are too lazy to think and don’t want to use their brains.
On the other hand, if the answer is not what they want, they will most likely not continue to ask. Moreover, if the answers are contradictory, then users may lose trust in our product.
3. Let’s do this Q&A robot, can we do it well?
I think it’s choking. Users will definitely compare our robots with Deepseek and Chat GPT applications, if our robot is not easy to use, then why don’t customers directly use the open source Deepseek, just like there is a better choice, who will use the bad one?
Later, judging from the usage data of friends’ products, this was also proved. The vast majority of users will hardly ask about this so-called “AI assistant”.
4. Customers want to be “intelligent” about everything, how to do it?
The principle of our products is to pursue accuracy, and when designing product functions, some information must be manually entered by users, but we dare not expect large models.
Why? I am afraid that the large model will pull out a brain and output an error message, then the accuracy of the final result will be miserable.
For example, we need to identify the names and roles of various characters from the documents. This big model is sometimes like a confused egg, putting the role of A on the head of B and writing the name of C as D
In order to solve this problem, we came up with a simple and crude method, allowing users to manually enter the names and roles of various characters at the beginning to ensure that this subject information is not wrong at all.
However, even if it is just such an input operation, many customers are unwilling.
Because, in everyone’s cognition, since large models can directly generate documents, why do I have to fill in the names and roles of a small character by hand?
The above is just a small example, but it begs a big question: which functions should be made “intelligent”?
Before making this product decision, we have to make it clear that all intelligence is not 100% reliable, just like weather forecasts, there are times when it is inaccurate, but the probability of error is different.
My decision-making thinking is: if a function is implemented through a large model, the accuracy rate cannot reach more than 90%, then this function should not be fully “intelligent”. We can create a “semi-intelligent” model, partly let the large model be generated, plus manual addition, deletion and modification.
Some people may ask, the large model has 80% accuracy, which sounds pretty good.
But if you think about it, even if there are 20% of errors, users may have to increase their working time by 40% if they want to correct these errors, which indirectly offsets the 80% burden reduction effect brought by the large model.
How to judge this accuracy?
Simply put, it is to test and verify through a large amount of data.
However, in practice, you may not have as much data and as much time for you to do this verification.
There is also a simple way: let’s see if there are any commonalities in this business scenario, as long as there are rules, it can be made into “intelligent mobility”.
Just like everyone’s ID number, it must be composed of 18 digits plus letters, and the ID number can be identified, extracted and filled by a large model. This “18-digit number + letter” is a common feature, just make it a rule.
However, even if you find this standard, in practice, you may still have some conventions. In the short term, software products cannot be fully transformed, software can only be an auxiliary tool, but don’t think about subverting old habits all at once.
Therefore, the safe way is to make intelligent assistance, not “fully autonomous driving”, but assisted driving. It lowers customer expectations and improves the “safety” of the product!
Final words
The wave of large model application products is still surging, and only by constantly thinking and making prudent decisions can we still move forward steadily after the wave recedes, so that the product can truly create value for customers.
