How We Build Our Custom ERP Ai Chatbots

Every day, your finance, HR, procurement, sales, and customer service teams need answers that are locked inside your ERP: an invoice's payment status, stock on hand across warehouses, a customer's order history, a remaining leave balance. The data exists, but getting to it means knowing the right module, running the right report, or pinging the one person who does. So when a client or an executive wants an answer now, employees are forced to choose between a slow, accurate response and a fast, risky guess. That friction adds up. McKinsey has long pegged it at roughly 1.8 hours a day, close to a full day each week, spent just hunting down and assembling information.

Until just a few years ago, that tradeoff was unavoidable, and it carried real costs: false promises, underdelivering, missed commitments, even breached contracts.

However, modern advances in artificial intelligence techniques like agent systems and planning ai systems equipped with reasoning and tool use are enabling businesses to answer complex client questions quickly and accurately. LLM-powered agentic systems that integrate with business systems like enterprise resource planning can reason about complex questions and comprehensively answer them with accurate data. The payoff is measurable: BCG estimates generative AI can cut ERP implementation effort by 20% to 40% and reduce the time new users need to get up to speed by 50% to 60%. In this article, we look into such ERP AI chatbots that no longer require significant investment.

‍

An ERP Use Case

In this section, we explore a typical use case that involves obtaining real-time business information from an enterprise resource planning (ERP) system, either for a user who's directly interacting with the ERP or for a system that's programmatically talking to the ERP:

1. The company is a wholesaler of electronic equipment like oscilloscopes. Its clients include retailers, labs, vendors, and electronics corporations that purchase such equipment in bulk.
2. An account executive at the company receives an availability query from their contact at a client company.
3. The client wants hundreds of oscilloscopes with specific criteria by the end of the week. The contact wants to know immediately if the company can fulfill the order.

The complication here is that the company has dozens of warehouses that stock thousands of devices. The client's requirements are many and intricate. Manually matching the products against these complex requirements and then verifying their availability is cumbersome as it is.

But the client wants an immediate answer and expects the order to be fulfilled in a short time, which sounds almost impossible. If the executive asks for more time, there's a risk that this client may buy from a competitor.

Agentic Systems to the Rescue

Such complex business interactions and repetitive tasks are where AI-powered chatbots can help your employees make solid business decisions and create good impressions with your customers. The complicated requirement matching as well as real-time inventory checking are things that state-of-the-art LLMs can do by fetching real-time data from your ERP systems. They enable your employees to quickly and accurately answer complex questions using your business data.

In the sections below, we explain how our chatbots solve such problems and help you keep your customers happy.

Custom Framework for ERP AI Chatbots

In this section, we explain our field-tested framework for integrating custom LLM-powered chatbots with ERP platforms like SAP S/4HANA and Microsoft Dynamics 365.

Our ai ERP chatbot framework and chatbots support the following capabilities out of the box:

• Using reasoning and dynamic data fetching, answer complex business queries from your customers as well as internal stakeholders like sales executives and customer service agents.
• Natural language processing workflow for query/question augmentation for deeper search and domain specific understanding.
• Answer queries based on your customer-facing and internal static documents, such as knowledge bases and standard operating procedures (SOPs). This is achieved with agentic systems using retrieval augmented generation (RAG).
• Include real-time dynamic data like product prices and availability in the answers. Such data are fetched from your ERP and other business systems.
• Provide a machine learning workflow for adding or modifying the static content used for RAG.

The architecture that enables all these is shown below:

We could have built this on top of an off-the-shelf ERP AI solution, or general-purpose agent builders and workflow tools like n8n or Microsoft Copilot Studio. For simpler internal automations those are a reasonable starting point, but for complex, customer-facing ERP assistants they run into real limits:

• Off-the-shelf chatbot platforms assume a standard setup. Pre-built ERP chatbots and templates are designed for the common configuration, but most enterprises run customized modules, non-standard workflows, and their own data model. A custom build is shaped fully around your actual environment, however complex, instead of forcing your processes to fit the tool's assumptions.
• Limited support for fine-tuned or self-hosted models. Domain accuracy, data residency, and cost control often depend on a model you can tune and host yourself, which these platforms rarely allow.
• You still write custom code anyway. Complex ERP environments, legacy APIs, and your specific ReAct actions all need bespoke work, so the no-code savings mostly disappear on a real ERP integration.
• Extra orchestration adds latency. The added layers and round-trips sit between the user and the model, which is a usability problem for interactive, customer-facing assistants where responses need to feel instant.

For these reasons we use our custom architecture, which keeps us in control of the agent framework, the integrations, and the response time.

‍

Agents With Reasoning and Problem-Solving Capabilities

The conversational Ai powering our chatbots must be able to:

• Reason about a query/question step by step
• Find relevant static information in documents like knowledge bases, SOPs, customer data, manuals, discussions, and datasheets
• Fetch relevant real-time dynamic information from ERP APIs and other business systems
• Connect to external tools like CRMs, email etc
• Combine all that static and dynamic information into a coherent, accurate answer

We equip the chatbot with ReAct based agents that can do tool calling, pull data, validate the data, and take recursion steps to find additional information. 

ReAct Overview

ReAct is a prompting framework to make an LLM plan on how to answer a complex question by combining reasoning with any necessary actions. In the example above, the LLM reasons that in order to answer a complex question comprehensively and accurately, it must break down the question into distinct actions that require interfacing with external systems.

The full list of available actions is domain-specific and supplied via few-shot examples to the LLM. It picks the actions that are most helpful for answering the question and acts on them.

Some few-shot examples for a question-answering task are shown below:

ReAct helps use cases like complex product search with multiple criteria:

• It starts by executing a high-level product search.
• It then uses reasoning to examine each product against the user's criteria.
• It eliminates any products that don't match some of the criteria.
• It verifies if the remaining products satisfy each of the other criteria.
• It homes in on the product that best satisfies all the criteria.

Let's look at ReAct in the context of ERP chatbots.

Document Search as a ReAct Agent

For finding information relevant to a query in static documents  like FAQs, product descriptions, customer emails, or discussion forum posts — we implement a ReAct action named  "search(query)." It does the following operations:

• Chunking: Documents are split into chunks of text so that information can be retrieved in chunks. The splitting is done on meaningful semantic boundaries like headers, sections, posts, or comments.
• Document embeddings: An embedding generator produces an embedding for each chunk. Our framework defaults to a SentenceTransformer model tuned for asymmetric search since we find that it works well for most use cases. However, the framework can also be configured to use OpenAI's Embeddings API or any alternative embedding API.
• Vector database storage: The embeddings are stored in the configured vector database like Qdrant, Weaviate, Pinecone, or others.
• Semantic search: When a customer or internal query is received, it looks for chunks that are relevant to the query. It first converts the query to a query embedding using the embedding generator. It then searches the vector database for chunk embeddings that are nearest neighbors of the query embedding. Such chunks are likely to be the most relevant to the query and provided to the framework for further processing.

The framework also supports adding new documents or updating existing ones. After such changes, the document embeddings are recalculated and updated in the vector database.

ERP Dynamic Data Retrieval as ReAct Tools

The fetching of different data from ERP systems is implemented using ReAct actions. Depending on the prompt and the detailed metadata for these actions, the LLM decides on the fly whether to invoke one or more of these actions.

Some typical ERP actions are:

• For a client query, get the real-time product availability from the inventory managed by the ERP inventory management module.
• For a client purchase request that's currently under negotiation or close to finalization, reserve the requested number of products in the inventory via the ERP's order management module so that fulfillment and delivery are expedited.
• Get or modify detailed product information from the ERP's product information module.
• Look up vendor contract details from the ERP's contract management module.
• Navigating complex ERP dashboards to generate reports

The exact module names and APIs depend on the ERP platform you're using. We explore such details for two popular ERP platforms further down under the chatbot integration section.

Implementing a ReAct Agent with Claude

We implement the ReAct strategy as an agent built on Claude's tool use. Instead of answering from a single prompt, the agent runs a loop: Claude reasons about the question, calls a tool to gather evidence, reads the result, and decides whether it has enough to answer or needs to keep going. That loop is what turns a chatbot into an agent, and it maps directly onto the reason, act, observe cycle that ReAct uses.

We expose the ERP and its surrounding data to Claude as a small set of tools, each defined with a name, a description, and a JSON input schema so Claude knows when and how to call it:

• MCP servers for live ERP data. We connect the agent to the ERP through the Model Context Protocol (MCP), so inventory, order, pricing, and contract lookups become standard, typed tool calls instead of one-off integrations. The same interface lets the agent reach other systems, like a CRM or helpdesk, without custom glue for each one.
• Semantic database search for documents. A vector search tool retrieves the most relevant chunks from knowledge bases, SOPs, datasheets, and FAQs. This is the RAG layer described earlier, exposed to the agent as a tool it can choose to call.
• grep and keyword search for exact lookups. When the agent needs an exact match, like a specific part number, an order ID, or a clause buried in a datasheet, a keyword or regex search is faster and more precise than semantic search, so we give it that as a separate tool. This is very helpful for legacy erp systems with basic document stores. 

The agent does not stop at the first result. After each tool call it validates what came back against the user's real criteria, and recurses when the data is incomplete or fails the check. Walking through the bulk oscilloscope availability request:

1. Reason. Claude breaks the request into a plan: find candidate products, check each against the spec, then confirm stock and delivery.
2. Act. It calls the MCP inventory tool to pull candidate oscilloscopes, and greps the datasheets to confirm specifications the ERP record does not store.
3. Validate. It checks each candidate against every requirement, such as bandwidth, channel count, quantity, and delivery date, and discards the ones that do not qualify.
4. Recurse if needed. If the first warehouse cannot cover the order, Claude reasons that it should widen the search, calls the inventory tool again for other locations, and re-validates. The loop continues until the criteria are satisfied or the agent can state with confidence that they cannot be.
5. Answer. Once validation passes, the agent hands the assembled, checked data to the answer step.

Because validation and the decision to recurse happen inside the loop, the agent only commits to an answer it can support with retrieved, checked data, rather than guessing in a single pass. It is the same mechanism behind the guardrails covered later: the model plans and orchestrates, the tools and the ERP supply ground truth, and any write is confirmed before it commits.

‍

LLM Answers to Queries

The framework combines all the retrieved document chunks and the relevant dynamic data into a coherent answer by appropriately prompting the LLM.

All that remains now is to get the user query from the ERP's or some other application's front end. Then we display the LLM's answer to the user via that same front end. That's explained in the next section.

Integrate Your Chatbots With Your ERP Systems

Microsoft Dynamics 365 is a popular ERP software among small and medium enterprises. On the other hand, large corporations typically choose SAP S/4HANA because their size and business complexity require its flexibility.

In the following sections, we explain the approaches for integrating a custom chatbot with both platforms.

Such integration always involves fetching data from the ERP backend. However, the chatbot front end (user experience) is typically integrated with a separate business application like a CRM or a service helpdesk. But for some special use cases, it may be integrated directly with the ERP system's dashboards. We explore all these approaches.

Microsoft Dynamics 365 Integration

Dynamics 365 ("D365" from here on) includes several products to manage different business aspects:

• The finance and operations product manages aspects like general ledger, accounting, payables, and project operations.
• Supply chain management (SCM) and intelligent order management help with inventory, warehouse management, transportation, and fulfillment.
• Products for sales, human resource management, customer service, and other common business aspects are also available.

Each product consists of modules and module APIs to fetch and modify their respective data stored in the ERP's databases. Our chatbot framework integrates with these APIs as explained in the following sections.

The Dynamics 365 Inventory Querying and Reservation API

D365 SCM's inventory management module provides an inventory visibility service that reports real-time inventory levels and availability across all facilities, warehouses, and channels. It also supports stock reservations to prevent problems like overselling and delayed fulfillment.

Our chatbots integrate with the visibility APIs via ReAct actions that are invoked by the LLM on demand. These APIs enable the chatbot to:

• Query the on-hand inventory of one or more products across locations.
• Query the on-hand inventory at specific locations or warehouses.
• Reserve the requested number of products on receiving a new customer order, to ensure fulfillment by the promised date.

When the LLM system invokes one of these actions, the chatbot's server-side logic calls these APIs. The chatbot's server-side logic is typically deployed as a standalone web application.

But when working with third-party systems, the chatbot may also be deployed as an integrated platform component rather than a standalone application. For example, if it's assisting salespersons via their Salesforce dashboards, its server-side logic runs on Salesforce-managed servers.

Dynamics 365 Front-End Integration

D365 also offers sales, CRM, and customer service capabilities. So it's possible that some clients want our custom chatbots shown directly on the D365 dashboard itself rather than some third-party application.

For such custom front-end integration, we develop the chatbot as a Power App and use the Power Apps component framework for a custom user interface.

SAP S/4HANA Integration

S/4HANA provides components like:

• Inventory management and physical inventory to query and update inventory data across all locations
• Extended warehouse management to query and update product inventories at specific warehouses

These components publish API endpoints that our chatbots talk to as explained below.

S/4HANA Inventory APIs

The inventory APIs include:

• The material stock read API to report real-time product inventory levels
• The reservation API to reserve a certain number of products when a new order is received to prevent overselling

Our chatbot ReAct actions invoke these APIs in response to instructions from the main Claude system.

S/4HANA Front-End Integration

Our chatbots are typically deployed as part of other business systems like Salesforce or Zendesk rather than integrating directly into S/4HANA dashboards. The backend components of our chatbots talk to the S4/HANA inventory APIs to get ERP data, combine them with other static information stored outside the ERP system, and show a coherent answer on Salesforce or Zendesk dashboards.

However, in some special scenarios, some job roles and workflows may require direct integration with S4/HANA dashboards. For such cases, our chatbots integrate with the following technologies:

• They integrate with the SAP Fiori front-end platform.
• The chatbot conversational interfaces are built using the SAPUI5 software development toolkit and components.

Applying the Same Pipeline Across your ERP Processes

The oscilloscope availability scenario above is deliberately detailed, but nothing about the pipeline is specific to inventory. Each additional use case is just another ReAct action that maps a natural-language question to the right ERP module and API, while the reasoning, document search, and answer-composition layers stay the same. Here are some of the highest-value questions teams ask, and the ERP action behind each.

Two kinds of actions show up here. Read actions, like checking an invoice status or a leave balance, only fetch data and are the safest place to start. Write actions, like reserving stock, raising a purchase request, submitting an expense, or completing routine data entry, change records in the ERP., so they run through the same role-based permissions and approval workflows the user already has. The chatbot never becomes a way around those controls: before any action runs, it confirms who the user is and what they are allowed to see or do, and any sensitive write is routed for approval rather than committed silently. That keeps the assistant aligned with your existing ERP authorization model and your data security and compliance requirements.

Challenges and Limitations of ERP AI Chatbots

No ERP chatbot is plug-and-play, and the failure modes are predictable. Below are the challenges we plan for on every build, and how our architecture & ai algorithms addresses each.

Data quality and hallucinations

An LLM is only as reliable as the data it can reach, and ERP data is often siloed, inconsistent, or out of date. A model left to its own knowledge will produce plausible but wrong answers. We avoid this by never letting the LLM guess at business data: every figure in an answer comes from a grounded ReAct action that queries the ERP or a retrieved document, not from the model's parameters. When a value cannot be retrieved, the chatbot says so rather than inventing one, and we surface the source behind each answer so users can verify it.

Security, access control, and compliance during ERP Interactions

ERP systems hold sensitive financial, customer, and HR data, and a chatbot widens the surface that data can leak through. As covered in the use cases above, every action runs under the user's own ERP role and permissions, so the chatbot can never return something the user could not already access on their own. Beyond that, we log every query and action for auditability, keep personally identifiable information out of third-party model prompts where possible, and use a fine-tuned or self-hosted model when data residency rules require it. The deployment is aligned with whatever compliance regime the business operates under rather than bolted on afterward.

Integration with legacy and low-API ERP systems

Modern platforms like S/4HANA and Dynamics 365 expose clean APIs, but many ERP landscapes also include older or heavily customized systems with no usable endpoints. For those, we build connectors against whatever interface exists, use a middleware layer to normalize data across modules, and fall back to other integration methods only when no API is available. Mapping a natural-language request to the right ERP action is configuration work that has to be maintained as business rules change, so we scope that ongoing maintenance into every engagement instead of treating the integration as one-and-done.

Integration with existing predictive analytics and process automation tools

Most enterprises already run demand forecasting, predictive analytics, and RPA or workflow automation alongside the ERP, and a chatbot that ignores them either duplicates their logic or works against it. We treat those systems as additional tools the agent can call, so it surfaces an existing forecast or triggers an established automation rather than reinventing it, and it respects the guardrails those tools already enforce.

Latency and scale

Interactive assistants have to feel instant, and real-time ERP lookups under heavy concurrency can strain both the ERP and the model. This is one reason we use a custom architecture rather than a heavier orchestration tool: it lets us choose fine-tuned or smaller models where they are fast enough, cache and reuse frequent lookups, and run independent ReAct actions asynchronously so a multi-part answer does not block on the slowest call.

Complex, multi-step transactions

LLMs are strong at single-step retrieval and reasoning but weaker at long, multi-leg transactions and exact calculations. We keep the model in the role it does well, planning and orchestrating, and defer precise computation to the ERP itself. Totals, tax, and availability math are returned by the ERP engine, not inferred by the model, and high-stakes writes are confirmed with the user before they commit.

User adoption and trust during 

A chatbot that is wrong once, or that feels like a black box, gets abandoned. We design for trust: answers cite their source, the assistant has clear escalation paths to a human or a power user when it cannot help, and it is upfront about what it did and did not do. The goal is a tool people rely on for routine questions, not one they have to double-check.

‍

Build the Best ERP Ai Chatbot for your custom setup

This chatbot pipeline strategy enables our chatbots to instantly and comprehensively answer very complex queries that you get from users and employees. By combining the awesome question-answering power of modern LLMs with real-time dynamic information from your existing ERP systems and static knowledge from other sources, our chatbots provide unprecedented insights into business operations in real time, greater operational efficiency, and unparalleled productivity in business interactions.

Contact us to see how we can set up a custom chatbot for your ERP.

‍

‍