Our Supplier Catalog Ingestion AI Agent | Automating SKU Onboarding

A supplier sends you three things that describe the same catalog and none of them agree. A 4,000 row Excel export from their ERP, a 60 page PDF line sheet their sales team made, and a dealer portal login. The Excel file has clean part numbers and two sentence descriptions. The PDF has the specifications and no machine-readable structure at all. The portal has the good copy, written by their marketing team, behind a session cookie.

Getting that into your catalog/PIM is the job a supplier catalog ingestion AI agent does, and the interesting engineering is not where most people assume. Extracting product text from a webpage is close to solved. Deciding whether the page you just extracted actually describes the SKU sitting in row 2,847 of the spreadsheet is not, and getting that wrong is worse than getting nothing at all, because a wrong specification propagates to every channel you publish.

The two failure modes look nothing alike from the outside. A pipeline that finds nothing leaves you with the file you started with, which is annoying and immediately obvious. A pipeline that finds the wrong thing leaves you with a catalog that looks finished, reads well, and is quietly wrong in exactly the fields buyers filter on. The second one is the expensive one, and it is the one that survives review, because nothing about a confidently written product description signals that its dimensions were invented.

Most of the engineering in this system exists to prevent the second failure, which means most of it happens before any text gets written. That ordering is the whole design.

This is how the ingestion agent underneath Pumice is built, what it does before it writes anything, and the failure modes that shaped it.

Key Takeaways

• Ingestion is not import. Import moves rows. Ingestion works out what the fields mean, finds what is missing, and confirms the record is the product it claims to be.
• The hard problem is identity, not always extraction. Matching extracted product data to the exact sku is more complex than you think, considering the risk factor. 
• Catalog agents split by destination. Procurement-bound ingestion validates against contracted pricing and lands in an ERP for procurement teams. Merchandising-bound ingestion enriches toward a listing a shopper reads.
• A general purpose model (ChatGPT) cannot enrich at scale properly, because they have no live access to the vendor source, no knowledge of your taxonomy, and no view of your catalog.
• Source conflict and source absence need explicit handling. Flagging a SKU and moving on beats inventing a specification for it.

‍

What Ingestion Has to Mean for a Merchandising Agent

Ingestion Is Not Import

An import system moves rows from one system into another. It assumes the incoming data is already the data you need and that the only problem is format/field naming. Every catalog tool has done imports for twenty years, and robotic process automation scripts that mapping to manage routine tasks and cut manual effort, but both only work when the sender and the receiver already agree on what a product record contains.

Supplier data does not meet that condition. The file arrives with the fields the supplier's own systems needed, which is almost never the set of fields a storefront/PIM needs. Ingestion is the work of reading that file, working out what each column actually means, retrieving what is missing from somewhere real, and confirming the result describes the right product before anything downstream touches it.

A practical example is what happens to an input column labeled “DESC1”. An import system maps it to a description field in your system and moves on. An Ingestion system has to work out whether it holds a real description, a truncated one, an internal note, or a repetition of the title, and then decide whether to keep it, extend it, or throw it away and go find the real one. Multiply that decision by every column in every vendor file and the difference between the two approaches stops being semantic. Supplier onboarding at any real scale lives on the second one.

Two Destinations Split the Category

Catalog ingestion agents look similar until you ask what they are ingesting toward, and what the goals are.

Procurement-bound ingestion checks incoming catalogs against contracted pricing and historical norms, flags the discrepancies, and lands validated records in an ERP. The procurement processes around it are mature: procurement platforms cover the procurement lifecycle from contract management and contract lifecycle management through supplier management, supplier performance tracking, and sourcing alternative suppliers across global supply chains, with procurement automation lifting manual procurement tasks out of approval chains. Risk management, supplier risk assessments, strategic sourcing, and the work to research suppliers against market trends all sit in that same stack. Its definition of done is a buyer placing a correct order. Whether the description reads well is irrelevant, because no shopper will ever see it.

Merchandising-bound ingestion is aiming at a page a customer reads and a search engine indexes. Its definition of done is a complete listing: a title with the attributes people search on, a description with real specifications, a filled attribute table, and a category assignment that puts the product where buyers look. The same primitives appear in both, but you cannot repoint one at the other, because a pipeline optimized to detect a price variance has no reason to care whether the finish is walnut or honey brown.

The Formats Suppliers Send, and What Each One Withholds

The constraint teams handle is that you have no control over the input format. A distributor carrying 400 vendors receives 400 different file structures, with different levels of completeness and provided fields. Supplier documents vary more in data quality between vendors than within one.

The instinct most teams follow is to rank the formats by quality and always reach for the best one. That is the wrong frame. Each vendor source is strong for some fields, and poor for others, and an agent that treats them as interchangeable inherits the weakest available answer from whichever one it happened to read first.

‍

Two key points from this breakdown:

• The file a vendor sends you is usually the worst version of their data, because it was generated by the system with the least reason to describe the product well. And the best version, the one their marketing team wrote, is almost always sitting on a public page you can retrieve. Most of the value in ingestion comes from noticing that and acting on it.

• Vendors change format without warning you. A supplier who sent CSVs for two years sends a portal invitation instead, or moves from a flat file to an EDI feed after an ERP migration, or starts attaching a PDF because someone in their sales org decided it looked better. If the pipeline is written against a specific format, each of those is an outage that lands on whoever owns the catalog. If it is written against a source abstraction, with format handling behind it, each one is a configuration change.

The Most Important Part Is Validation, Not Extraction

Ask an engineer to build a catalog ingestion agent and the first instinct is to make the extraction better. Better parsing, better prompts, better handling of ugly HTML. Extraction is worth doing well, but it stopped being the bottleneck when we built our Ai Smart Scraper. The bottleneck is that you retrieved a page and you do not actually know whether it describes the product in front of you.

Why This Is Harder Than It Sounds

Supplier catalogs are full of products that differ in exactly one dimension. The same lamp in four finishes. The same fastener in eleven lengths. The same cleaning solution in a 12 count case and a 24 count case, with different part/sku numbers for each. 

The validation engine has to match a sparse product record in a flat file to the record on a product page that is in a very different format, with multiple products looking nearly identical. The difference between two completely different skus could be a single digit in an attribute, and the record in your vendor flat file doesn’t even have an attribute. 

Most product matchers torpedo in accuracy as the variance between available data fields grows. 

‍

That same vendor record has to match to this product, with complete data, while not matching to the version that has one attribute change “1 battery charger” vs “tool only”. These product pages look exactly the same. 

What a False Match Costs

A missing attribute is a gap. A wrong attribute is a liability, and it does not stay in one place. It goes into the product page, into the faceted filters, into the structured data, into the channel feeds, and into whatever an answer engine decides to quote. By the time it surfaces it is a customer receiving just the tool when they assume it came with a battery, and nobody traces that back to an ingestion run from four months earlier.

This asymmetry is the whole design argument. A pipeline that fills 60% of fields and is right about all of them is more valuable than one that fills 95% and is wrong about 3%, because the second one costs you trust in the other 92.

The Identity Gate We Solved For with machine learning

In our Pumice system an agent trained specifically for this task runs validation steps between retrieval of product data and augmentation of the sku data, and it is a hard gate rather than a score to look at later. A validation agent compares the retrieved record against the incoming row: identifiers, brand, model, and the distinguishing attributes for that category, which for one vendor is pack count and for another is dimensions or voltage. If it cannot 100% confirm the match, the data does not flow into generation.

Getting the gate wrong in the strict direction costs you coverage, which is recoverable. Getting it wrong in the permissive direction costs you data accuracy, which is not, because you will not know which records to distrust. That is the tradeoff to tune, and it is worth tuning per category rather than globally, since a category of near-identical variants needs a stricter gate than a category of unique items.

Why Not Just Use ChatGPT or Copilot for Sku Ingestion/Enrichment?

‍

There’s two different ways we see people try to handle sku ingestion/enrichment:

• ChatGPT/single model approach
• Some sort of out of the box agent system (Claude Cowork/OpenClaw)

‍

Both have their own issues.

‍

Why off the shelf models don’t work

Some people try using ChatGPT (or another large language model) to enrich product data by just asking the model to write a new title or description for the product, but due to ChatGPT not having access to the actual manufacturer product data, the model will hallucinate the newly created product data. Remember, LLMs only have access to the data they were trained on, and ChatGPT might only have been trained on data from 2025 and before. This means the product data you get from ChatGPT could be entirely unrelated to the actual product you're trying to enrich. Let's look at this example to understand.

Here we've loaded up a product into ChatGPT from a vendor flat file for a mirror. We've got an image, a title, and a short ERP description. We're going to have ChatGPT create the new data.

‍

At a first glance the results don't look too bad. All the data was generated and it seems related.

Here's the problem, it's completely hallucinated. The category does not follow the vendor's taxonomy, the finish of the wood is not walnut, it's honey brown finish. The height and width are completely wrong. There is no mention of the wood species or the veneer material, and raw materials are exactly the kind of attribute buyers filter on and compliance teams check. All of this data lives on the vendor's website and is live right now.

‍

Based on what we see on the manufacturer site, we get inaccurate data from ChatGPT.

It's not really the language model's fault. As I mentioned before they do not have direct access to live data, they do not know the vendor's taxonomy or required specifications. So many customers we work with were previously trusting ChatGPT results on their product data that is completely hallucinated.

This builds onto a larger issue. ChatGPT, Claude and Perplexity don't know anything about YOUR catalog structure and rules. They don't know what specs you like to use, they don't know your format for values attached to specs, and they certainly cannot look at your full catalog to determine what makes the most sense.

Agentic Ai Framework

Agentic frameworks get you closer to complete as they have web access, MCP access etc, but have a few very specific shortcomings. 

‍

• These frameworks are generalists, not built specifically for onboarding products. They will use generic prompts, generic logic, and struggle with the dynamic nature of correctly enriching products based on validated records
• No built in validation of the scraped product data. From experience this is not a task an LLM can handle at the level of accuracy we need. We built a specific fine-tuned model on millions of records just for this task.  
• The amount of setup needed for generalist agentic frameworks to understand brand rules, compliance etc. Both lean on generative ai and natural language processing for what is really a matching decision. General ai agents read a page well and decide badly, and purpose-built ai systems put a trained matcher in that slot. That is the gap between agentic ai as a category and an agent built for one job. 

‍

This is the difference between a domain specific supplier catalog ingestion AI agent and a general purpose system, and it is a difference in order of operations rather than model quality. Our agent system researches first. It reads the actual vendor source, whether that is a live page, a PDF line sheet, or a feed, confirms the source describes that exact SKU, and only then generates against your taxonomy, your spec formats, and your catalog context, with rules and validation you define. No invented dimensions. No made up finishes. No taxonomy drift.

When Sources Disagree, and When They Do Not Exist

Grounding the agent in a real source raises two questions that a chatbot never has to answer, because it never had a source to begin with.

Source Precedence

Retrieve a SKU from three places and you will get three answers. The PDF line sheet the vendor emailed in January says one dimension. Their website says another, because the product was revised. A retailer carrying the same item says a third, because they copied an older spec sheet and never updated it.

Guessing is not an option, and neither is averaging. The agent needs a declared precedence order per run, and the right order changes by vendor and by field. A manufacturer's own site usually wins on specifications. The line sheet often wins on pack configuration and pricing, because that is what it was produced for. A retailer page never wins on anything, and is only useful when nothing else exists.

The part worth building deliberately is that precedence is per field rather than per source. Declaring that the vendor site wins for everything means accepting their website's pack quantity over the line sheet they sent you, which is usually wrong.

Conflicts are also worth recording rather than silently resolving. When two grounded sources disagree on a dimension, that disagreement is information: it usually means a revision, a regional variant, or a stale page somewhere. A pipeline that logs the conflict alongside the resolution gives a merchandiser something to check. One that just picks a winner throws that signal away.

When There Is No Source

Some percentage of any real catalog has nothing behind it. Discontinued items still being sold from inventory. Private label goods with no public page. Vendors small enough to have no website worth retrieving. Regional variants that exist only in a distributor's own system.

The correct behavior here is unglamorous: fill what the incoming file supports, leave the rest empty, flag the row, and move on. What matters is that the pipeline finishes the batch rather than halting, and that the flagged rows land in a queue where human oversight resolves them. Zero human intervention was never the goal. Human expertise is the scarce resource, and the queue is how you aim it.

The temptation is to let the model fill the gap, since it will produce something plausible and the completion rate looks better. That is precisely the failure the identity gate exists to prevent, reintroduced one step later. A flagged row is a known unknown. An invented specification is an unknown unknown, sitting in your catalog looking exactly like a real one.

What the Generation Phase Inherits From Ingestion

Generation is the part everyone pictures when they hear “ai agent for sku ingestion”, and it is the part that takes our previous high leverage work and turns it into real product data.

Three controls do the shaping. Rules are plain instructions about format, required fields, banned terms and category-specific requirements. Examples show the model what a correct record looks like in your categories, which does work no rule phrasing achieves. Validation checks the output against hard limits and sends failures back with the specific reason, so the pipeline can retry rather than drop the SKU.

What generation cannot do is repair anything upstream of it. It has no way to know that the page behind its inputs described a different pack size, because by the time it runs, those inputs are simply the facts. This is why teams debugging bad output usually look in the wrong place. Rewriting the prompt changes how the output reads and leaves the errors exactly where they were, because the errors were decided two steps earlier by a retrieval that matched the wrong page.

Those three controls decide if the product data you write actually becomes usable in a structured catalog. 

‍

Good Ingestion Data Accuracy Numbers

Ingestion gets measured wrong more often than it gets measured badly. Teams report how many records the pipeline produced, which tells you the batch finished. Analyzing data properly means four numbers, because operational efficiency is easy to claim and the business outcomes that matter sit downstream of all four.

• Identity gate accuracy, and the false positive rate underneath it. On a 120,000 SKU catalog, the matcher confirms correct matches at 97.57%. The number that matters more is the one underneath it: a 0.07% false positive rate. A miss costs coverage and announces itself. A false positive writes the wrong specification into a record that looks finished, and nobody traces it back. Tune the gate against the second number, not the first.
• Validation pass rate and retry distribution. What share passed on the first attempt, what needed retries, and what exhausted them and got flagged. A rising retry count usually means the rules and the category drifted apart, not that the model got worse.
• Validation pass rate and retry distribution. What share passed on the first attempt, what needed retries, and what exhausted them and got flagged. A rising retry count usually means the rules and the category drifted apart, not that the model got worse.
• Throughput and cost per SKU. Records per run and what each one costs to process. This is what decides whether the approach survives contact with a catalog several times the size of the 120,000 above. We’re able to process roughly 1000 records per hour with 14 fields per record. 

‍

For calibration on the categorization step that runs alongside ingestion, production systems we have built reach 97% on a five level deep taxonomy for an ecommerce solutions company, 92% at the lowest level of a 5,585 category multilingual taxonomy for a wholesale marketplace, and 97.62% at 50 million products per month for a growing marketplace. 

Conclusion

The instinct when building a catalog ingestion agent is to spend the effort on generation, because that is the part that looks like AI. The effort belongs earlier. Almost every quality problem that surfaces months later traces back to a moment when the pipeline accepted a source it should not have, and almost every quality problem you can live with traces back to a moment when it correctly refused one.

Build the identity gate first, decide what happens when sources disagree, decide what happens when there is no source at all, and the generation step becomes the easy part. Those three decisions are also the ones that are painful to retrofit, because changing them later means reprocessing everything that went through the old version and having no clean way to tell which records were affected.

A catalog you can trust is worth more than a catalog that is full, and the order you build in is what determines which one you end up with. The strategic value is not the automation. It is that a catalog nobody has to second-guess becomes a competitive advantage while everyone else is still cleaning theirs.

‍

Need a Catalog Ingestion Agent Built Around Your Vendors?

Width.ai builds custom ingestion and enrichment pipelines: per-vendor source precedence, category-specific

identity gates, PDF and EDI handling, and taxonomy fitting against your own tree.

Tell us your vendor count, your SKU volume, and what your worst supplier file looks like.

‍

‍

‍

‍

Common questions we get asked:

What is a supplier catalog ingestion AI agent?

A system that takes supplier product data in whatever form it arrives, retrieves what is missing from real sources, validates that each source describes the right SKU, and generates complete product records from the result. The validation step is what makes it an ingestion agent rather than a writing tool.

How is this different from a procurement catalog agent?

Destination. Procurement agents like Domo's validate supplier catalogs against contracted pricing and historical norms and land clean records in an ERP, which is what procurement managers and procurement professionals need for supplier relationship management. Procurement operations and merchandising are different jobs. This one is aiming at a listing a shopper reads and a search engine indexes, so completeness and category placement matter more than variance detection.

Why not just use ai systems like ChatGPT, Claude or Copilot?

They write from training data rather than from your vendor's live page, so the output is fluent and wrong in specific ways: invented dimensions, finishes that do not exist, categories that do not match your tree. The gap is order of operations rather than model quality. Retrieve and verify first, then write.

What formats can be ingested?

CSV and Excel exports, PDF line sheets and spec catalogs, manufacturer and dealer web pages via retrieval, EDI feeds, and vendor APIs. Most real catalogs need several at once, because a distributor carrying hundreds of vendors receives all of them.

How does the agent know the source is the right product?

It checks the page it retrieved against the row it started from. Does the identifier line up, the brand, the model, and the attributes that separate this SKU from its nearest neighbors. Without a confirmation nothing reaches the generator and the row gets flagged instead. Pack variants and superseded part numbers are the cases this exists for.

Can brand voice and formatting rules be enforced?

Yes, through rules and examples defined per field. Rules cover format, required fields, banned terms and category requirements. Examples teach the shape your categories actually use. Validation rejects anything that misses, and the retry loop fixes it rather than dropping the SKU.

Does the output feed into a PIM or ERP?

Yes. The enriched records export as CSV or through an API back into whatever system holds your master data, and the same output reformats per sales channel from one run.

‍

‍

‍