Whether it is supply chain management or financial management, the efficiency and flexibility of document engines directly affect the operational efficiency and data consistency of enterprises. This article will provide an in-depth analysis of the design principles, application scenarios, product architecture and processes of document engines such as Alibaba, SAP, and Kingdee, helping everyone quickly grasp the core technology and practical points of document engines, and providing reference for enterprise informatization upgrades.
01 Document engine concept
The document engine is the core component used to automate the processing of business documents in the enterprise information system, and realizes the generation, circulation, conversion and association management of documents through rule configuration. Its core goal is to reduce manual intervention and improve business process efficiency and data consistency through flexible policy configuration.
Common medium and large ERP systems on the market have this function, such as Kingdee Cloud Starry’s BOTP (document conversion platform, see for details: Brief analysis of the accounting engine BOTP (document conversion platform) and principles), UFIDA’s TOP and process engine are in the same place; SAP uses the SAP S/4 HANA API to generate and convert documents, as shown in the figure below (the figure comes from the official website of SAP):
02 Application scenarios of document engine
The application of document engine is mainly concentrated in the two modules of supply chain and finance, mainly including:
1. Supply chain module conversion
a. Purchase order → purchase order → purchase inbound order
Scenario: After the supplier confirms the purchase order, the system automatically generates a purchase order as the basis for receipt.
b. Quotation → Sales order → Shipping note→ Shipping order → Sales outbound order
Scenario: After the customer confirms the order, the system generates a shipping notice, and generates a shipping order to inform the logistics vehicle to ship in batches or directly generate an outbound order, triggering an inventory deduction.
c. Sales order → purchase order
Scenario: In the pre-sale (sales or procurement) scenario, when the inventory is insufficient, the purchase order is immediately placed to realize the flow of information through document conversion.
2. Financial module
Including the transfer of other modules to the finance module, such as:
a. Material outbound order → production cost certificate
Scenario: When producing picking, the system automatically converts the material outbound order data into production cost entries (debit: production cost/raw materials, credit: raw materials).
b. Purchase receipt → Provisional estimate payable
Scenario: After the material arrives, the warehousing order is generated, and the system temporarily estimates the accounts payable (debit: raw materials, credit: payable provisional estimate).
c. Sales invoice → Accounts receivable and cost carryforward
Scenario: Generate accounts receivable vouchers (debit: accounts receivable, credit: main business income) when invoicing, and carry forward sales costs (debit: main business costs, credit: issued goods).
It should be noted that some enterprises → receivables according to the sales outbound order, because the sales invoice will be issued regularly and aggregatedly, unlike the sales outbound order that must be confirmed in the month of receipt or consolidated, or transferred to the receivable form on a one-to-one basis.
03 Document engine design principle
The essence of the document engine is the conversion of documents to documents, which realizes the flexible configuration of conversion rules, the process is transparent and traceable, and the complex business logic is decoupled
What does a product manager need to do?
In the process of a product from scratch, it is not easy to do a good job in the role of product manager, in addition to the well-known writing requirements, writing requirements, writing requirements, there are many things to do. The product manager is not what you think, but will only ask you for trouble, make a request:
View details >
Core capabilities: Support operations such as join, merge, split, and group of documents to adapt to complex business scenarios.
Technical essence: Based on the master-slave structure (single head + detail) data processing system, the automatic process is driven through the rule engine to realize the mapping and conversion of business documents and business documents (see the dismantling accounting engine (core part) for the conversion of documents and vouchers, which is not within the scope of the document engine).
The design principle of the document engine is based on the core of dynamic rules, configuration visualization, and process automation, and realizes flexible business adaptation through hierarchical architecture and metadata-driven. Kingdee BOTP focuses more on document conversion (generating various documents as follows), while ZTE Xinyun FOL focuses on document template customization and compliance control, both of which embody the design concept of “low-code/no-code” to lower technical thresholds and quickly respond to business changes.
04 Document engine product architecture
The document engine usually adopts a hierarchical architecture, which is divided into input layer, rule engine layer, execution layer, and output layer:
1. Input layer
The input layer includes data source configuration, metadata management, parsing rules, etc., and obtains document fields, types, and associations through metadata parsing.
For example, the “Expense Reimbursement Form” (advertising expenses) of the cost control system is connected to the “Payable Statement” of ERP, and then the “Payment Voucher” is generated; At the same time, the “Procurement Warehousing Form” of the procurement platform is also connected to the “Payable Order” of ERP, which generates the “Provisional Estimate Payable Voucher”;
In this layer, metadata is the key point, because the business is always dynamically changing, and in order to adapt to this dynamic change, low-cost, dynamically configurable, and low-code metadata is the direction to solve the problem. For example, the following metadata management:
2. Rule engine layer
Core modules, including the configuration of conversion rules, verification rules, and calculation rules. For example:
Kingdee BOTP defines field mapping, grouping policies, menu policies, etc. through KScript scripts.
a. Mapping of source documents and target documents, such as the source document “Purchase Order”, the target order can be checked “Payable Order”, “Payment Request Form”, “Purchase Return Form”, “Receipt Notice” or even “Sales Order”, etc.
b. Field mapping policy, that is, the A field of the source order, which is mapped to the B field of the target document;
c. Grouping strategy, configure the document grouping basis and entry merging basis used when the source order is pushed down in batches to generate the target order, such as grouping (merging) multiple “purchase requisitions” according to the same “organization and supplier” to generate a target “purchase order”;
d. Selection strategy, also known as funnel strategy, filters out the source orders that meet certain conditions and generates target orders;
Field mapping example:
ZTE New Cloud FOL configures field types, verification conditions (such as sensitive word detection, amount compliance), and calculation logic (such as automatic calculation of travel subsidies) through the visual interface.
Architecture process (overview) diagram:
Thereinto:
Source document: that is, metadata, is the table and field corresponding to the (document engine) system after the upstream system is transmitted, which is exactly the same as the upstream data structure, such as what are the fields of the “expense reimbursement form” of the upstream cost control system, and there will also be an identical “expense reimbursement form” to the document engine, which plays the role of undertaking and recording through metadata configuration.
Document rules: Set the specific rules for document generation, such as how each field of the document comes from, whether it is from a certain field of the source document or has specific rules; And whether it should be written off, and what are the rules for writing off? Is it a one-to-one reversal by order number or by supplier & document type? Example:
For example, the rules of [document code]: “keyword (‘FYBX’) + year month + serial number (reset across months)”;
For example, [exchange rate]: check the exchange rate of the corresponding range and the same currency in the “Exchange Rate Table” according to [Date] & [Currency] in the source table;
[Expense Type]: According to the [Reimbursement Item] in the source table “Expense Reimbursement Form”, check the corresponding [Expense Type Code] in the “Expense Type Mapping” table
Split and merge: The generation rules of the source document and the target document are 1:1 or multiple merges to generate one, or split into multiple, but many-to-many is prohibited! If it is a merge generation, what are the merge rules? Take Kingdee as an example:
For example, multiple “Expense Reimbursement Forms” are combined to generate one “Payable Order” document header according to the same [payee or supplier] and [organization], and then [reimbursement item (i.e., converted expense type], [expense bearing department], and [currency] are used as summarization, and the summary sum field is [reimbursement amount (total price and tax)];
The output policy, that is, which is the target document, and whether the target document is directly saved to the database after generating it, or should the interface be called to push downstream?
Model construction: assemble the above rules into an agreement or set, that is, save a specific logo or name for a set of rules, and only one rule with the same conditions is effective at the same time, for example, as mentioned earlier, the “Expense Reimbursement Form” and the “Purchase Intake Form” are both generated “Payables”, but the follow-up processing rules of the two are different, the document type of the “Expense Reimbursement Form” to generate the “Payable Order” is “Expense Order”, and the next step is to generate the “Payment Order”; The document type of the “Purchase Warehousing Order” to generate the “Payable Order” is “Provisional Estimate Order”, which has no next step. For example:
“Procurement and warehousing list”→ “Payable note”;
“Expense Reimbursement Form→ Payable Slip→ Payment Slip
In addition, if you want to do a more perfect job, you can design the document relationship into a draggable canvas, and you can design the entire chain of documents in one picture, which is clear and comprehensive, such as the following:
3. Executive layer
Call the script parsing engine or rule engine to perform conversion/verification operations, such as Kingdee BOTP parsing scripts and generating target documents through the KScript engine. Include:
Preprocessing: Preprocessing may be called when receiving the source order or when generating the target document, with the aim of supplementing the information required for document conversion and generation to ensure that the subsequent document conversion step can be carried out smoothly. This includes data cleaning and standardization (for example, the “[expense type]” rule defined above), rule prevalidation (such as field non-null validation, data type matching (such as whether the amount is numeric)), and context environment loading.
Document generation: Generate target documents based on defined document rules, including dynamic template filling, automated assignment logic, grouping, etc. In some scenarios, there may be more than one target document, and there may be more than one. For example, after the “Logistics Cost Bill of Lading” is sent from BMS (warehousing and logistics cost platform), the document engine must not only generate the “Accrual Payable” but also generate the “Provisional Valuation (Write-off) Payable” according to the rules.
Verification: including automatic system checksum manual review and correction;
- Automated verification includes integrity verification (such as checking required fields), logical consistency verification (verifying business rules, such as the total amount of documents = sum (the sum of each line [full price and tax)]), and compliance inspection (matching tax regulations (such as invoice tax rate compliance) and enterprise internal control standards);
- Manual review, provide a visual interface to mark abnormal items (such as red highlighted fields), and support reviewers to add comments; Support the “Rejection-Amendment” process: If the unit price is detected to be inconsistent with the contract, it can be returned to the preprocessing stage to rematch the data.
4. Output layer
The functional modules are as follows:
- Document preservation: The output layer focuses on generating and saving the target documents (such as payables, outbound orders, and approval orders) to record the document association before and after conversion, and supports traceability.
- Document circulation: In some scenarios, the generated documents are distributed to downstream consumers through the call interface or RabbitMQ/Kafka.
- Operation audit (log): Record the operation time, operation user, data version and other relevant information of the document reception, conversion, generation, and pushdown process, and provide real-time early warning of abnormal information. Record the upstream and downstream relationship map in the process of document conversion to facilitate later traceability and troubleshooting.
In terms of structural design, the output layer and the execution layer can be independent or merged into one module, depending on the business scenario and the complexity of the rules.
In actual business scenarios, the implementation of the document engine can be simplified and then complex, with the docking of the financial system as a pilot, such as expense reimbursement forms, small-scale development and rapid iteration in the form of MVP to meet the needs and then promote and upgrade, and consolidate R&D results.
