Building a Large AI Recommendation System from 0: Why Do Businesses Need a Recommendation System?

In today’s era of information explosion, user attention has become a scarce resource, and the importance of recommendation systems has become increasingly prominent. This paper delves into the necessity of building a recommendation system for enterprises, from the fit of business needs, the readiness of data foundation, to the refinement of user needs, and analyzes how the recommendation system optimizes user decision-making links, manages user lifecycle value, and improves content distribution efficiency.

In an era where user attention is the most scarce resource, recommendation systems have evolved from auxiliary tools to auxiliary tools to strategic infrastructure that supports business growth and user experience upgrades. As a product manager, a key strategic question must arise in the face of the huge performance of recommendation systems in the industry (such as significantly increasing conversion rates, user stickiness, etc.): Is the recommendation system the core engine that drives our own business to the next level? To answer this question, we cannot rely solely on industry popularity or a single case, but need to carry out systematic thinking and rigorous evaluation.

For product managers, building a recommendation system is not a simple technical procurement, but a complex project involving business strategy, data foundation, algorithmic capabilities, and continuous operations. The core of decision-making lies in conducting rigorous business suitability assessment based on multi-dimensional models, deeply understanding its data-driven value creation logic, and quantifying input and output through a scientific ROI measurement framework to control the timing of start-up. In the implementation process, product managers need to shoulder the core responsibilities of defining goals, leading A/B testing, and accurately transforming business requirements. Only in this way can we effectively open up incremental space by building an intelligent connection bridge between “user needs” and “content/services” in the fierce competition for stock, so that the recommendation system can truly become a powerful engine driving sustainable business growth.

1. Judge the fit between the recommendation system and business needs

Before deciding to invest resources in building a recommendation system, product managers must first conduct a rigorous business fit assessment. This is not a simple question of whether to do it, but a question of what conditions is most valuable and whether it currently has a foundation. The assessment should be based on three core dimensions:

Analysis of the matching degree of core business scenarios

The value of recommendation systems is not one-size-fits-all, it is highly dependent on specific business forms and user pain points. The core contradiction of information-intensive products (such as integrated e-commerce and large-scale information platforms) lies in the conflict between massive supply and users’ limited cognitive ability, that is, choice overload. Users are faced with thousands of products or articles, and their decision-making efficiency will drop sharply. Through intelligent “guess what you like” or “you may be interested”, the recommendation system can effectively reduce users’ screening costs and convert information overload into accurate matching.

Content production platform(such as short videos, music, UGC communities) face the efficiency of content distribution. Massive user-generated content (UGC) or professionally produced content (PGC) inevitably leads to a long-tail effect in traffic distribution, and a large amount of high-quality content can be overwhelmed. The core value of personalized recommendations is to activate the distribution efficiency of these waist content, help users discover content that meets their interests but has not yet received widespread attention, and solve the problem of difficulty in discovering high-quality content.

Functional complex or instrumental products(e.g., enterprise-level software, multi-level applications), users may need to go through complex paths to complete the target operation. At this time, the recommendation system can play the role of an intelligent guide, predicting the functions or services it may need next based on the user’s current behavior and status, optimizing the conversion path and improving operational efficiency.

To achieve these three challenges, product managers will only continue to appreciate
Good product managers are very scarce, and product managers who understand users, business, and data are still in demand when they go out of the Internet. On the contrary, if you only do simple communication, inefficient execution, and shallow thinking, I am afraid that you will not be able to go through the torrent of the next 3-5 years.

View details >

Key judgment points for product managers: Looking at your product, is there a clear contradiction between information density and user decision-making efficiency? Specifically, the number of products (SKUs), content items, or functional options provided by the platform have exceeded the user’s natural cognition and processing capabilities. When users start to complain about not being able to find something, not knowing what to do next, or there are obvious problems of insufficient browsing depth and high bounce rate on the data, the introduction of recommendation systems has a solid value foundation.

Data base and user scale readiness assessment

• The algorithm effect of the recommendation system is significantly positively correlated with the scale and quality of the data it can obtain. Experience shows that to support the effective operation of a basic recommendation model, it is usually necessary to meet some basic thresholds: the platform must have a certain scale of active user base (e.g., hundreds of thousands of monthly active users) and have continued to accumulate user behavior data for a sufficient period of time (e.g., more than 3 months). This behavioral data needs to cover the key nodes of user interaction with core content/products, including but not limited to browsing, clicking, favoriting, add-on, purchasing, playing, reading time, likes, comments, searches, etc.
• Data quality is the core lifeblood.The evaluation focuses on: the integrity of the behavioral data (are all key interaction points documented?) ), consistency and accuracy of the label system (are user portrait labels and content/product feature tags clearly defined and effectively mapped?) ), the timeliness of data updates (can log collection and processing achieve T+1 or even more real-time?) ）。 The data quality is not high, and even the most advanced algorithms are difficult to be effective.
• Strategies for Dealing with Cold Starts:For businesses that are still in the early stages of user or data accumulation, it is not completely impossible to launch a recommendation system, but a proactive data replenishment strategy needs to be adopted. This includes: building an initial seed user profile through in-depth user interviews, questionnaires, etc.; Invite domain experts to participate in the design of the top-level framework of the content/product labeling system; Under the premise of compliance, prudently introduce third-party data sources to supplement the basic attributes of users (such as region, age group, potential interest direction, etc.). Through a well-designed cold start strategy, the basic functions of the recommendation system can be implemented and gradually iteratively optimized even in the early stage of insufficient data accumulation.

The degree of refinement of user demand layering

• The power of the recommendation system lies in “thousands of people and thousands of faces”, and the degree of value release is closely related to the heterogeneity of the needs of user groups. When there are significant and identifiable differences within the user group in terms of interests, preferences, spending power, usage scenarios, etc., the greater the room for the recommendation system to achieve user experience upgrades and business growth through accurate matching.
• Product Manager’s Evaluation Methodology:It is necessary to use user segmentation technology to quantify the degree of differentiation of requirements. An effective way is to construct a value-interest two-dimensional matrix. First, the RFM model (RecencyFrequencyMonetary – last purchase, frequency of consumption, amount of consumption) or similar models is used to stratify the value of users (high value, medium value, low value, churn risk, etc.). Secondly, it deeply analyzes the user’s behavior sequence data (browsing history, search keywords, favorites, interaction behavior, etc.) to construct their interest graph (such as preferences for categories, themes, styles, etc.). Combining value hierarchies with multi-dimensional interests and preferences can form a clear user segmentation matrix.
• Key quantitative indicators:When the number of user groups divided through this analysis reaches a certain scale (e.g., more than 5 main groups), and the difference rate between core demand characteristics (such as preferred content type, purchased product category, price sensitivity, etc.) is significant (e.g., the difference rate exceeds 30%), it indicates that the heterogeneity of user needs is high enough, and personalized recommendation will become a key leverage point to improve user experience and business efficiency.

2. Understand how recommendation systems drive business growth

After clarifying the suitability of the recommendation system to the business, product managers need to deeply understand the underlying logic of its value creation so that they can design goals and evaluate the effectiveness of the project. The value of recommendation systems is mainly reflected in three core aspects:

Optimize user decision-making links

The essence of recommendation systems to improve conversion rates lies in the deep intervention and reconstruction of user decision-making processes. It transforms the traditional “user active search-> filtering->decision-making” model into a “system active prediction->accurate recommendation->convenient decision-making” model, significantly shortening the path from generation of needs to completing actions (such as purchasing, learning, and use).

Core mechanism: Based on the user’s historical behavior data and current context (such as pages viewed, time, device, etc.), the system uses algorithmic models to predict their potential needs or points of interest, and proactively presents highly relevant content or options in appropriate scenarios (such as product detail pages, content list pages, function operation pages). This effectively replaces passive search and information screening that users would otherwise need to spend a lot of time and effort on.

Product Manager Design Focus: It is necessary to systematically plan and recommend intervention points in different transformation links:

• Flow Inlet:Design the recommendation modules in the focus of the homepage, channel page, and list page to efficiently undertake the initial interest of users.
• Key Conversion Nodes:Provide accurate “related recommendations”, “matching recommendations” or “next recommendations” where users are about to make a decision (such as shopping cart page, course selection page, or mid-use of a feature) to drive conversions or complete key steps.
• After-sales/post-conversion link:After the user completes a behavior (the order is successful, the learning is completed, and the user is terminated), the recommendation of “you may still like it” or “try next time” based on this behavior is provided to guide repurchase or deepen use.

Performance tracking: Establishing a clear “User Behavior-> Recommendation Strategy-> Conversion Results” attribution analysis model is crucial. By analyzing the data, you can pinpoint which part of the conversion funnel is the best or has bottlenecks in the recommendation, allowing for targeted optimization. For example, identifying user groups with high cart abandonment rates and targeting “similar product price reductions” or “making up suggestions” can effectively recover churn.

Manage user lifetime value

Personalized recommendations are the core means to maintain long-term user activity. It continuously provides users with content or services that match their current or even potential interests, maintaining product attractiveness and freshness, effectively extending the user’s active cycle and increasing their total lifetime value (LTV).

Core mechanism: The system dynamically tracks the evolution of user interests (such as deepening, transferring, and decaying points of interest) to adjust the weight and diversity of recommended content in real time. This avoids user churn due to monotonous content or unsatisfied interests. The key is to build a dynamic model of user interests, capturing changes in their preferences in the short and long term.

Product Manager’s Intervention Strategy: A phased retention intervention system should be constructed:

• Cold Start for New Users:The goal is to quickly build a foundational interest model in the first few visits of the user. Strategies include allowing users to actively select interest tags, recommending popular and popular content/products, or using registration/survey information for initial recommendations.
• Growth Period Users:The goal is to balance “satisfying known interests” and “exploring potential interests.” The recommendation strategy should focus on the core interests that users have clearly expressed (e.g., 70%), supplemented by extended content discovered by algorithms that may be of interest to users (e.g., 30%), so as to stimulate new needs while maintaining user satisfaction.
• Users at risk of recession/churn:The goal is to reactivate user interest. The strategy focuses on pushing new developments (such as updates, price reductions, related new products) of content/products that users have had deep interactions with (such as buying, reading, favoriting, high ratings) in history, or trying to push high-quality content that is highly relevant to their core interests to awaken users’ memories and interests.

Content/product distribution efficiency construction

The recommendation system subverts the traditional head-focused or editor-led distribution model, achieving fairer and more efficient inclusive distribution. It gives more high-quality but non-head-end long-tail content/products a chance to reach potential users who are genuinely interested in it.

Core mechanism: It mainly relies on two major technical paths:

• Collaborative Filtration:Discover other users with similar interests to the target user, or find other items similar to items that the target user has liked, and recommend them. Good at spotting potential interests of users.
• Content-Based Recommendations:Analyze the matching degree between the characteristics of the content/product itself (tags, text, images, audio, etc.) and the characteristics of the user persona. Specializes in recommending content similar to users’ historical preferences.

The combination of the two allows the system to accurately identify the connection point between “high-quality content/products” and “potential interested users”.

The Art of Balance for Product Managers: When designing a distribution strategy, the core challenge lies in balancing efficiency with ecological health:

• Header content/product:Ensuring that they get enough exposure is necessary for efficiency, but it is necessary to set a reasonable traffic cap (e.g., individual content/product exposures do not exceed a certain percentage of total traffic) to avoid over-concentration and the Matthew effect.
• Waist Contents/Products:This is where recommendation systems are most valuable. A cold-start traffic pooling mechanism (e.g., providing a fixed initial exposure for 24-48 hours for effect testing for new content/products) needs to be designed to give it a fair chance to get started.
• Long tail content/product:Primarily activated through interest recall strategies, this long-tail content is prioritized when users search for relevant keywords, browse related topics, or show specific niche interests. Ensure diversity and richness of the platform.

A good balance strategy can significantly improve the overall content diversity index and creator/supplier motivation of the platform.

3. ROI quantitative input and output

Building a recommendation system is an important resource investment, and product managers must be able to clearly quantify its potential benefits and compare costs for a rigorous input-output (ROI) analysis.

The whole cycle cost is disassembled

The cost investment of the recommendation system is multi-dimensional and needs to be comprehensively considered:

Technology costs (the largest share, about 40-50%):

• Algorithm development: basic recommendation algorithms (such as collaborative filtering, matrix decomposition, deep learning models such as DIN/DIEN/MMOE), real-time computing frameworks (such as Flink/SparkStreaming), model training and iteration platforms (feature engineering, model training, evaluation, deployment pipelines).
• Server resources: offline large-scale computing cluster (processing historical data training), online real-time inference server (processing user requests), and massive data storage nodes (user behavior logs, feature data, model parameters).
• Data infrastructure: log collection and transmission system, user portrait and tag management system, feature storage, data warehouse/data lake construction and maintenance.
• Cost optimization suggestion: In the early stage, you can use a combination of “public cloud services + mature open source framework (such as TensorFlowPyTorchSparkMLlibFlink)” to greatly reduce the initial technical investment and operation and maintenance complexity.

Labor costs (about 30-40%):

• Algorithm engineer: model design, development, training, tuning, iteration.
• Data Engineer: Data pipeline construction, feature engineering, label system design and management, data quality monitoring.
• Back-end engineer: Recommended service API development, system architecture, performance optimization, and online service stability guarantee.
• Front-end engineer: Recommend the UI/UX implementation and user interaction logic of the module.
• Product Manager: Requirements Definition, Strategy Design, Effect Evaluation, Cross-Team Coordination, Project Promotion.
• (may involve) operation and maintenance engineers, test engineers.
• Cost optimization suggestions: Adopt an agile development model, take 2-4 weeks as an iteration cycle, give priority to building MVP (minimum viable product) to quickly launch and verify core value, and reduce risks before large-scale investment.

Time cost (important hidden cost):

It usually takes 3-6 months from project approval to the launch of the first recommendation function. Time is mainly consumed in:

• Data preparation and cleaning (~20%): data mapping, missing value handling, outlier handling, label alignment, etc.
• Model development, training, and tuning (~30%): feature selection, model selection, parameter adjustment, and offline evaluation.
• Engineering deployment and integration (~30%): Service API development, online environment deployment, integration with existing business systems, stress testing.
• Testing and acceptance (~20%): A/B test design, effect evaluation, problem fixing, and business acceptance.

Quantitative model construction of returns

The benefits of recommendation systems also need to be measured in multiple dimensions:

Direct commercial value (most quantifiable):

• GMV growth due to increased conversion rates: core metrics. Typically referral traffic (e.g., a detail page where a referral clicks on) will have a significantly higher conversion rate than organic traffic (e.g., active searches or category browsing). The increase ratio can be used as the core basis for GMV increment.
• Improved ad monetization efficiency: The click-through rate (CTR) and conversion rate of ad placements (in-feed ads, related recommendation ads) in personalized recommendation scenarios are usually much higher than those of ordinary ad placements. This directly leads to an increase in advertising revenue.
• Calculation method: Through conversion tracking and traffic source analysis by module, the direct GMV or advertising revenue increment brought by the recommendation module can be calculated relatively accurately.

User value appreciation (medium and long-term value):

• LTV growth due to improved user retention: By improving user engagement and satisfaction, recommendation systems can effectively extend user lifecycles and potentially increase ARPU (average revenue per user) during user activity. Calculate LTV growth by considering the length of time the average user lifecycle is extended due to increased retention and the change in ARPU during this lifetime (recommendations may lead users to purchase higher-value items or more frequently).
• Increased user usage time: More accurate and engaging recommendations significantly increase users’ time spent in the app. Longer usage time not only means more monetization opportunities (ad exposure, potential conversions), but also enhances user dependence and brand awareness on products.
• Calculation method: Compare the key user retention indicators (next-day/7-day/30-day retention rate), average daily usage time, DAU/WAU/MAU, etc., before and after the recommendation system is launched (or A/B testing), and combine the user value model to estimate the LTV increase.

Ecological value construction (long-term barriers):

• Long-tail content/product exposure improvement: Promote the prosperity of the platform’s content/product ecosystem, attract and retain more creators/suppliers.
• Increased User Engagement: A better experience may spur users to generate more UGC content (comments, shares, creations) or interactive behaviors.
• Calculation difficulties: Such values (such as the increase in the number of creators and the improvement of community atmosphere) are difficult to quantify directly in currency, but they are crucial for the long-term healthy development of the platform and the formation of competitive barriers. Trend assessment can be carried out through relevant indicators (proportion of long-tail content exposure, creator retention rate, UGC growth rate, etc.).

Indicator monitoring system design

To ensure that the recommendation system continues to create value and guide optimization, it is necessary to establish a hierarchical indicator monitoring system:

Strategic layer (Polaris metrics – reflecting core business impact):

• Overall business goal correlation: GMV growth rate (weekly monitoring), average daily usage time of users (daily/weekly monitoring), paying user conversion rate (real-time/daily monitoring). These metrics directly reflect the contribution of the recommendation system to the core objectives of the product.

Tactical layer (process indicators – guide recommendation strategy optimization):

• Core indicators of recommendation performance: recommendation click-through rate (CTR – measures the matching of content/product with user interests), recommendation conversion rate (CVR – measures the actual conversion power of recommendations).
• User behavior metrics: recommendation scene bounce rate (measures the attractiveness and relevance of recommendation modules), recommendation impressions/clicks per capita (measures recommendation penetration depth).
• Data health indicators: User profile coverage (measures the completeness of user tags), content/product feature coverage (measures the completeness of content/product tags).

Execution layer (health indicator – ensure the long-term operation of the system):

• Ecological health metrics: proportion of long-tail content/product exposure (measuring distribution fairness), diversity metrics (e.g., content category coverage).
• Algorithm iteration indicators: model update frequency (measuring algorithm optimization speed), model offline evaluation indicators (such as AUCPrecision@KRecall@K).
• System performance indicators: recommended service response delay (P99/P95), system availability (SLA), and data update delay.

The core actions of product managers:Establish a weekly/bi-weekly review mechanism. Core process: Monitoring indicators -> Comparison targets/benchmarks -> Analyze the causes of deviation (such as the CTR of a certain category is continuously low) -> Locate the root cause of the problem (such as inaccurate labeling, the model has not learned the feature well) -> Formulate and implement optimization strategies (such as supplementing manual annotation, adjusting feature weights, and model iteration) -> Verify the effect -> Closed-loop iteration. This process is the lifeline for the continuous optimization of the recommendation system.

4. Responsibilities of product managers

Product managers are the core promoters and responsible subjects of the recommendation system from concept to reality, and need to perform the following key responsibilities throughout the entire project life cycle:

Define clear and measurable goals

A successful recommendation system starts with clear goal setting. Be sure to follow the SMART principles:

• Specific:Clarify the optimization target. For example: “Increase click-through rate (CTR) of homepage feed referral bits” instead of the vague “Improve recommendation effect”.
• Measurable:Set quantitative targets and baselines. For example: “Increase the CTR of homepage feed recommendations from the current 8.5% to 10.2% before the end of Q3”.
• Attainable:Goals should be based on historical data trends, industry benchmarks, and team capability assessments to ensure that they can be achieved by jumping ahead, and avoid being too ambitious or conservative.
• Relevant:The goal must be strongly related to the core business indicators of the product. For example, the goal of increasing CTR needs to be able to deduce the expected contribution to GMV growth or user duration (e.g., an X% increase in CTR is expected to drive a Y% increase in GMV).
• Time-bound:Set clear time nodes. For example: “Achieve the goal in the next 3 months”.

Example of target disassembly (e-commerce scenario):

• Business core goal: 20% increase in GMV in the quarter.
• Recommendation system contribution target: Undertake 30% of the overall growth target, that is, directly or indirectly drive GMV growth of 6% through the recommendation system.
• Recommended core process indicators: The overall conversion rate of the recommendation module needs to be increased from 12% to 15%; The proportion of recommended traffic to the total site traffic needs to be increased from 25% to 30%.
• Specific implementation indicators: The CTR of the core recommendation position on the homepage needs to be increased from 8% to 10%; The click-through rate of the “Watch and See”/”Similar Recommendation” module on the product detail page needs to be increased from 5% to 7%. This disassembles layer by layer to ensure that the team’s goals are clear and actions are consistent.

Leading scientific and rigorous A/B testing

A/B testing is the gold standard for optimizing recommendation strategies and verifying algorithm effectiveness. The product manager leads the process:

1. Test design:Clearly define test variables (e.g., new recommendation algorithm vs. old algorithm; different recall strategies; different sorting weight formulas), determine the duration of the test (usually 7-14 days to cover the full cycle of user behavior, such as weekdays and weekends), calculate the required sample size (ensure statistical significance, usually no less than 5% of total traffic).
2. Traffic Division:A scientific triage mechanism (such as hierarchical sampling) is used to ensure that the distribution of key attributes (such as age, geography, and user value level) of the test group (TreatmentGroup) and the control group (<5%) is maintained to avoid distortion of the experimental results due to sample bias.
3. Indicator Monitoring:In addition to monitoring core metrics (e.g., CTRCVRGMV), it’s also important to keep an eye on secondary metrics and experience metrics (e.g., test group’s page bounce rate, user dwell time change, negative feedback rate). Prevent the trap of “improving core metrics but deteriorating user experience”.
4. Analysis of results:Statistical methods (such as two-tailed t-tests) were used to determine whether the difference in indicators was statistically significant (usually p-value < 0.05 was considered significant). Conduct in-depth analysis of segmentation dimensions (such as performance differences under different user segments, different time periods, and different content types) to explore deeper insights and guide the direction of subsequent strategy optimization.

Accurate translation of business requirements is an algorithm rule

The core value of a product manager is to act as a bridge between the “language of business” and the “language of technology”. Vague business goals or operational strategies need to be translated into understandable, enforceable, and clear rules or characteristics that algorithm engineers can understand:

Example 1: High-net-worth user operation strategy

• Business needs: “For high-value users who have spent more than 5,000 yuan in the past 30 days, priority is given to recommending high-quality products with high customer unit prices.” ”
• Algorithmic conversion: Define the user tag user_value_tier=”high” (based on the consumption amount > 5,000 yuan in the last 30 days). In the recommendation ranking stage, add a weight factor weight_high_value=1.5 (or other adjustment factor) to the user of the tag, which acts on the product feature item_price_tier=”high” or item_quality_score>threshold products. Rule: final_score=base_score*weight_high_value(ifuser_value_tier==”high”anditem_price_tier==”high”).

Example 2: New product cold start strategy

• Business Needs: “Ensure that new products receive sufficient initial exposure within 7 days of listing.” ”
• Algorithm conversion: Define product characteristics item_age=current_date-item_launch_date. Add a item_age-based weight boost factor to the sort formula: new_item_boost=max(0(7-item_age)/7)*boost_factor (e.g., boost_factor=0.3). Rule: final_score=base_score*(1+new_item_boost). In this way, the weight of the new product is the highest when it is first put on the shelves, decays over time, and returns to normal after 7 days.

Example 3: Big promotion strategy

• Business Requirements: “During the 618 promotion period (June 1-June 18), the exposure priority of high-discount products was significantly increased. ”
• Algorithm transformation: Define the time feature is_promotion_period=True(ifdatebetweenJun1andJun18). Define product characteristics discount_rate=(original_price-current_price)/original_price. Add conditional weights to the sort formula: promo_weight=1.5if(is_promotion_periodanddiscount_rate>0.15)else1.0. Rule: final_score=base_score*promo_weight.

Keys to Ensuring Accuracy:

Establish a Business Rule Statement that clearly describes each business strategy:

• Triggers:Precise user tags (e.g., user_value_tier=”high”), content/product attributes (e.g., item_age<7), time/event conditions (e.g., is_promotion_period=True).
• Intervention modalities:How exactly does it affect the recommendation results? Is it boosting, guaranteeing minimum impressions (guaranteed_impressions), or filtering specific types?
• Prioritization and Conflict Resolution:How do you prioritize when multiple rules apply at the same time? (For example, “The promotion discount rule takes precedence over the regular personalization rule”). This manual is the cornerstone for business, product, and algorithm teams to align their cognition and ensure that the strategy is accurately implemented.