What is retrieval augmented generation for business? In plain terms, it is a way of pointing a language model at your company's own documents and records so it answers with facts from your business instead of guessing from the open internet. The tool reads what you already have (PDFs, contracts, invoices, supplier statements, SOPs), retrieves the relevant pieces when a question or task comes up, and drafts an answer or a record entry a person can approve. That is the whole idea. The interesting part is not the plumbing. The interesting part is which chore inside your business is worth pointing it at.

Key takeaways

  • RAG is not a chatbot; it is a system that reads documents you already receive and writes draft entries into software you already run.
  • The mid-market use case is boring on purpose: rebate reconciliation, invoice matching, contract lookups, supplier statement review.
  • Inside Kelsan, a multi-state distributor in our group running Epicor P21, our rebate-and-margin tool has recovered over $100,000 in margin.
  • Every write to a live record goes through a person on your team before it posts. No exceptions on money-touching workflows.
  • Cost is metered like any cloud service and quoted in writing before a build. Every engagement starts with a fixed-fee AI Capability Audit ($7,500).

The plain-English definition, without the vector-database detour

A language model on its own knows what it was trained on. It does not know your supplier agreements, your price files, your customer contracts, or last quarter's invoices. Retrieval augmented generation is the pattern that fixes that gap. You give the model a way to look things up in your documents and records at the moment it needs to answer, so its output is grounded in your facts rather than the internet's guesses.

Two moving parts. A retriever that finds the right passages in your source material. A generator that writes the answer or draft using those passages. Every business RAG system also needs a third piece the technical explainers skip: a place for a person to approve what the system proposes before anything hits a live record. That third piece is what separates a demo from a system that runs in production.

How this differs from "just using ChatGPT"

A chatbot answers questions in a browser window. Someone still has to read the answer, decide what to do, and go type it into the system that matters. That is a useful assistant. It is not a system.

What we build is different. The tool reads the documents your business already receives, matches them against records in your ERP or system of record, and drafts the entry a person on your team approves. Nothing happens in a chat window. The work happens where the work already lives. If you want the deeper comparison, our page on wire generative AI into your systems walks through how the reading and writing gets stitched into the software you already run.

What this looks like in a real distributor

Here is the first-hand proof. Kelsan is a multi-state distributor in our own group, running Epicor P21. Suppliers send rebate agreements as PDFs and spreadsheets, in the formats they choose, on the cadences they choose. Historically, someone had to read those documents, hold them in their head, and compare them against thousands of invoice lines in P21 to catch what was owed. Some money got recovered. A lot did not.

We built a rebate-and-margin recovery system inside P21 that reads the supplier rebate documents, matches them against invoices in the ERP, and surfaces entries for a person to approve before they post. A member of the team reviews each proposed entry. Nothing writes to a live record without that approval. To date, the tool has recovered over $100,000 in margin that would otherwise have been lost. You can read more on the rebate-and-margin tool we built inside Kelsan if you want the fuller picture.

That is RAG in a business. Not a chat window. A chore, read and drafted by a tool, approved by a person, written into the software the business already runs.

Where else it fits in a mid-market company

The pattern generalizes anywhere you have (a) unstructured documents coming in and (b) a system of record where structured entries need to go out. A few honest examples:

  • Supplier statement reconciliation against AP.
  • Contract lookups: "what does our MSA with this customer say about freight?"
  • Policy and SOP retrieval for service teams answering customer questions.
  • Purchase order acknowledgment matching against original POs.
  • Sales quote drafting from a customer RFQ and current price files.

The common thread is documents-in, records-out. That is the shape we look for. Our capability pages on intelligent document automation and ERP automation go deeper on the two ends of that pipe.

What it costs to find out

Model usage and retrieval infrastructure are metered like any cloud service. We quote the run cost in writing before we build, and the numbers are usually smaller than operators expect for the volumes a mid-market company actually processes. The bigger cost is the build itself, not the compute.

Every engagement starts the same way: a fixed-fee AI Capability Audit at $7,500. In the audit we look at your system of record, the documents that flow in, the chore you want handled, and whether a RAG-shaped system is the honest answer or not. If it is, we scope the build as a separate fixed fee, agreed in writing. If it is not, we say so. The audit still leaves you with a written map of what would actually move the needle.

When you should not build one

Plenty of times. If the documents you want read do not exist in a consistent-enough form (a person is retyping them from photos, faxes, or verbal calls), fix the intake first. If the "system of record" is a shared spreadsheet with no discipline about who edits what, a RAG tool will just launder chaos faster. If the chore only happens twice a year, hire a temp. If your team will not approve entries because nobody has time to look, a system that surfaces drafts nobody reviews is worse than no system.

The honest filter: is there a repeating chore, on documents you already receive, with a system of record where the answer needs to land? If yes, this pattern earns its keep. If no, do not build.

Next step

If you are trying to decide whether this pattern applies to a specific chore in your business, the fastest way to find out is a short conversation and, if it makes sense, an audit that produces a written answer either way.

Book a call with Throughline, or start with a fixed-fee AI Capability Audit ($7,500). The call is free. Call 865-417-3554.

About the author

Throughline is a small team of builders inside Keller Group. We build AI systems into our own operating companies first, then into yours.