How We Build Our Custom ERP Ai Chatbots

There are many times when your business development, sales, and customer service employees end up in tight spots with important clients. Some clients may demand detailed information or business commitments that are impossible to provide in a short time because your employees don't have the data readily at hand.

Until just a few years ago, this would have been a common problem that may have led to bigger problems like false promises, underdelivering, or breach of contracts.

However, modern advances in artificial intelligence techniques like natural language processing (NLP) and large language models (LLMs) equipped with reasoning and tool use are enabling businesses to answer complex client questions quickly and accurately. LLM-powered chatbots that integrate with business systems like enterprise resource planning can reason about complex questions and comprehensively answer them with accurate data. In this article, we look into such ERP AI chatbots.

An ERP Use Case

In this section, we explore a typical use case that involves obtaining real-time business information from an enterprise resource planning (ERP) system, either for a user who's directly interacting with the ERP or for a system that's programmatically talking to the ERP:

1. The company is a wholesaler of electronic equipment like oscilloscopes. Its clients include retailers, labs, vendors, and electronics corporations that purchase such equipment in bulk.
2. An account executive at the company receives an availability query from their contact at a client company.
3. The client wants hundreds of oscilloscopes with specific criteria by the end of the week. The contact wants to know immediately if the company can fulfill the order.

The complication here is that the company has dozens of warehouses that stock thousands of devices. The client's requirements are many and intricate. Manually matching the products against these complex requirements and then verifying their availability is cumbersome as it is.

But the client wants an immediate answer and expects the order to be fulfilled in a short time, which sounds almost impossible. If the executive asks for more time, there's a risk that this client may buy from a competitor.

AI Automation to the Rescue

Such complex business interactions and repetitive tasks are where AI-powered chatbots can help your employees make solid business decisions and create good impressions with your customers. The complicated requirement matching as well as real-time inventory checking are things that state-of-the-art LLMs can do by fetching real-time data from your ERP systems. They enable your employees to quickly and accurately answer complex questions using your business data.

In the sections below, we explain how our chatbots solve such problems and help you keep your customers happy.

Custom Framework for ERP AI Chatbots

In this section, we explain our field-tested framework for integrating custom LLM-powered chatbots with ERP platforms like SAP S/4HANA and Microsoft Dynamics 365.

Our chatbot framework and chatbots support the following capabilities out of the box:

• Using reasoning and dynamic data fetching, answer complex business queries from your customers as well as internal stakeholders like sales executives and customer service agents.
• Answer queries based on your customer-facing and internal static documents, such as knowledge bases and standard operating procedures (SOPs). This is achieved with retrieval augmented generation (RAG).
• Include real-time dynamic data like product prices and availability in the answers. Such data are fetched from your ERP and other business systems.
• Provide a workflow for adding or modifying the static content used for RAG.

The architecture that enables all these is shown below:

Instead of a custom architecture, we could have relied on the OpenAI Assistants API. However, it has some drawbacks that make it unsuitable as of January 2024:

• It doesn't support fine-tuned models.
• It doesn't support function calling which is essential for our document search and data retrieval pipelines.
• It suffers much higher latencies than regular GPT-4 API calls. This is a usability drawback for our customer-facing interactive digital assistants.

For these reasons, we use our own custom architecture. In the following sections, we go into more detail on the components shown above.

LLMs With Reasoning and Problem-Solving Capabilities

The conversational AI powering our chatbots must be able to:

• Reason about a query step by step
• Find relevant static information in documents like knowledge bases, SOPs, frequently asked questions (FAQs), manuals, discussions, and datasheets
• Fetch relevant real-time dynamic information from ERP and other business systems
• Combine all that static and dynamic information into a coherent, accurate answer

We equip the GPT-4 LLM to do all this using the ReAct strategy described below.

ReAct Overview

ReAct is a prompting strategy to make an LLM plan on how to answer a complex question by combining reasoning with any necessary actions. In the example above, the LLM reasons that in order to answer a complex question comprehensively and accurately, it must break down the question into distinct actions that require interfacing with external systems.

The full list of available actions is domain-specific and supplied via few-shot examples to the LLM. It picks the actions that are most helpful for answering the question and acts on them.

Some few-shot examples for a question-answering task are shown below:

Few-shot examples for ReAct (Source: Yao et al.)

ReAct helps use cases like complex product search with multiple criteria:

• It starts by executing a high-level product search.
• It then uses reasoning to examine each product against the user's criteria.
• It eliminates any products that don't match some of the criteria.
• It verifies if the remaining products satisfy each of the other criteria.
• It homes in on the product that best satisfies all the criteria.

Let's look at ReAct in the context of ERP chatbots.

Document Search as a ReAct Action

For finding information relevant to a query in static documents — like FAQs, product descriptions, customer emails, or discussion forum posts — we implement a ReAct action named  "search(query)." It does the following operations:

• Chunking: Documents are split into chunks of text so that information can be retrieved in chunks. The splitting is done on meaningful semantic boundaries like headers, sections, posts, or comments.
• Document embeddings: An embedding generator produces an embedding for each chunk. Our framework defaults to a SentenceTransformer model tuned for asymmetric search since we find that it works well for most use cases. However, the framework can also be configured to use OpenAI's Embeddings API or any alternative embedding API.
• Vector database storage: The embeddings are stored in the configured vector database like Qdrant, Weaviate, Pinecone, or others.
• Semantic search: When a customer or internal query is received, it looks for chunks that are relevant to the query. It first converts the query to a query embedding using the embedding generator. It then searches the vector database for chunk embeddings that are nearest neighbors of the query embedding. Such chunks are likely to be the most relevant to the query and provided to the framework for further processing.

The framework also supports adding new documents or updating existing ones. After such changes, the document embeddings are recalculated and updated in the vector database.

ERP Dynamic Data Retrieval as ReAct Tools

The fetching of different data from ERP systems is implemented using ReAct actions. Depending on the prompt and the detailed metadata for these actions, the LLM decides on the fly whether to invoke one or more of these actions.

Some typical ERP actions are:

• For a client query, get the real-time product availability from the inventory managed by the ERP inventory management module.
• For a client purchase request that's currently under negotiation or close to finalization, reserve the requested number of products in the inventory via the ERP's order management module so that fulfillment and delivery are expedited.
• Get or modify detailed product information from the ERP's product information module.
• Look up vendor contract details from the ERP's contract management module.

The exact module names and APIs depend on the ERP platform you're using. We explore such details for two popular ERP platforms further down under the chatbot integration section.

Implementing ReAct Tools With GPT-4

We can implement ReAct actions using GPT-4 function-calling. The advantage of the GPT-4 API is that it provides a much more structured way to specify actions compared to ReAct's original proposals.

OpenAI's API enables you to specify actions and their information in great detail as shown below:

OpenAI function calling API example (Source: OpenAI)

GPT-4 uses that semantic information and metadata — the function description, parameter property descriptions, parameter units, and so on — to make better decisions on when to invoke your actions based on your queries.

GPT-4 also enables the structured invocation of actions and transmission of action responses back to the LLM.

LLM Answers to Queries

The framework combines all the retrieved document chunks and the relevant dynamic data into a coherent answer by appropriately prompting the LLM.

All that remains now is to get the user query from the ERP's or some other application's front end. Then we display the LLM's answer to the user via that same front end. That's explained in the next section.

Integrate Your Chatbots With Your ERP Systems

Microsoft Dynamics 365 is a popular ERP software among small and medium enterprises. On the other hand, large corporations typically choose SAP S/4HANA because their size and business complexity require its flexibility.

In the following sections, we explain the approaches for integrating a custom chatbot with both platforms.

Such integration always involves fetching data from the ERP backend. However, the chatbot front end (user experience) is typically integrated with a separate business application like a CRM or a service helpdesk. But for some special use cases, it may be integrated directly with the ERP system's dashboards. We explore all these approaches.

Microsoft Dynamics 365 Integration

Dynamics 365 ("D365" from here on) includes several products to manage different business aspects:

• The finance and operations product manages aspects like general ledger, accounting, payables, and project operations.
• Supply chain management (SCM) and intelligent order management help with inventory, warehouse management, transportation, and fulfillment.
• Products for sales, human resource management, customer service, and other common business aspects are also available.

Each product consists of modules and module APIs to fetch and modify their respective data stored in the ERP's databases. Our chatbot framework integrates with these APIs as explained in the following sections.

The Dynamics 365 Inventory Querying and Reservation API

D365 SCM's inventory management module provides an inventory visibility service that reports real-time inventory levels and availability across all facilities, warehouses, and channels. It also supports stock reservations to prevent problems like overselling and delayed fulfillment.

Our chatbots integrate with the visibility APIs via ReAct actions that are invoked by the LLM on demand. These APIs enable the chatbot to:

• Query the on-hand inventory of one or more products across locations.
• Query the on-hand inventory at specific locations or warehouses.
• Reserve the requested number of products on receiving a new customer order, to ensure fulfillment by the promised date.

When GPT-4 invokes one of these actions, the chatbot's server-side logic calls these APIs. The chatbot's server-side logic is typically deployed as a standalone web application.

But when working with third-party systems, the chatbot may also be deployed as an integrated platform component rather than a standalone application. For example, if it's assisting salespersons via their Salesforce dashboards, its server-side logic runs on Salesforce-managed servers.

Dynamics 365 Front-End Integration

D365 also offers sales, CRM, and customer service capabilities. So it's possible that some clients want our custom chatbots shown directly on the D365 dashboard itself rather than some third-party application.

For such custom front-end integration, we develop the chatbot as a Power App and use the Power Apps component framework for a custom user interface.

SAP S/4HANA Integration

S/4HANA provides components like:

• Inventory management and physical inventory to query and update inventory data across all locations
• Extended warehouse management to query and update product inventories at specific warehouses

These components publish API endpoints that our chatbots talk to as explained below.

S/4HANA Inventory APIs

The inventory APIs include:

• The material stock read API to report real-time product inventory levels
• The reservation API to reserve a certain number of products when a new order is received to prevent overselling

Our chatbot ReAct actions invoke these APIs in response to instructions from GPT-4.

S/4HANA Front-End Integration

Our chatbots are typically deployed as part of other business systems like Salesforce or Zendesk rather than integrating directly into S/4HANA dashboards. The backend components of our chatbots talk to the S4/HANA inventory APIs to get ERP data, combine them with other static information stored outside the ERP system, and show a coherent answer on Salesforce or Zendesk dashboards.

However, in some special scenarios, some job roles and workflows may require direct integration with S4/HANA dashboards. For such cases, our chatbots integrate with the following technologies:

• They integrate with the SAP Fiori front-end platform.
• The chatbot user interfaces are built using the SAPUI5 software development toolkit and components.

Need a custom chatbot for your ERP?

This chatbot pipeline strategy enables our chatbots to instantly and comprehensively answer very complex queries that you get from users and employees. By combining the awesome question-answering power of modern LLMs with real-time dynamic information from ERP systems and static knowledge from other sources, our chatbots provide unprecedented insights into business operations in real time and unparalleled productivity in business interactions.

Contact us to see how we can set up a custom chatbot for your ERP.

‍