How AI Chatbots Actually Work (Simple Guide for Local Businesses)

What is actually happening when a chatbot “answers”?

Underneath almost every modern chatbot is a large language model (LLM). It is first trained on a huge amount of text to do one thing: predict the next word, given the words before it. It is then trained further, including with human feedback, so that it follows instructions and behaves like an assistant.^[1]

So when a customer types “Do you deliver to Halol?”, the model is not checking a database. It is producing the most likely sensible reply to that text, given everything else it has been shown. If you have shown it your delivery policy, the most likely reply will match the policy. If you haven’t, it will produce something plausible that may be wrong.

Models don’t read words exactly as we do. They read tokens: pieces of words, whole words or characters. How many characters make a token depends on the model and the language. For English on earlier Claude models, Anthropic puts it at roughly 3.5 characters per token.^[1] Tokens matter because you pay per token and because each model can only hold so many at once.

What happens step by step between a question and a reply?

Here is the path a message takes in a typical business chatbot that answers from your own documents:

1. The customer writes on your website chat or WhatsApp.
2. Your software looks up relevant material from a knowledge base you control: the price list, delivery areas, warranty terms, the last few messages in the chat.
3. It builds a prompt: fixed instructions (“You are the assistant for …; answer only from the documents; if unsure, hand over to a person”), the retrieved extracts, and the customer’s message.
4. The model writes a reply by predicting tokens one after another.
5. Your software checks the reply against simple rules, for example no prices that aren’t in the price list, and no promises about dates.
6. The reply is sent, or handed to a person if it failed a check or the customer asked for a human.

Steps 2 and 3 have a name: retrieval augmented generation, or RAG. Information is retrieved from a knowledge base when the question arrives and passed to the model alongside it, which grounds the answer in your material rather than the model’s memory. Anthropic’s own glossary notes that RAG is only as effective as the quality and relevance of the knowledge base.^[1] In plain terms: the bot is as good as your documents.

Does the chatbot remember customers and learn from chats?

Not by itself. The model has a context window, which is the amount of text it can look at while writing one reply. Anthropic describes it as a “working memory”, separate from what the model learned in training.^[1] Each new reply starts fresh unless your software includes earlier messages in the prompt.

So when a bot “remembers” a customer’s last order, that is your system fetching it from your records and adding it to the prompt. And the model doesn’t learn from today’s chats unless someone deliberately trains or updates it. That is good news for control: if an answer is wrong, you fix the document or the instruction, and the next answer changes.

It also means chat transcripts are data you hold. If they contain names, phone numbers or addresses, India’s DPDP Act makes your business responsible for them, including when a vendor processes them for you.^[9]

Why do AI chatbots make things up?

Because they are built to produce fluent, likely text, not to know when they don’t know. The US National Institute of Standards and Technology calls this “confabulation”: the production of confidently stated but erroneous or false content, which people usually call hallucination.^[2] A bot asked about a product you don’t stock may describe it anyway, because a description is the likely shape of an answer.

Three things reduce this a lot, though none removes it completely:

• Give it the facts. Retrieve the right documents every time (RAG) and keep them current.
• Tell it what not to do. “If the answer is not in the documents, say you will check and pass the chat to a person.”
• Check the risky bits in code. Prices, stock, dates and refunds should come from your system, not from the model’s wording.

Settings such as “temperature” make replies more or less varied, but even at the lowest setting replies are not perfectly repeatable.^[1] Test the same question several times before you trust it.

Can a chatbot reply in Gujarati, and does it cost more?

Modern models can read and write Gujarati and Hindi, and many customers in Vadodara mix English, Gujarati and Hindi in one message. The catch is tokens. Researchers at Oxford found that the same text can need up to 15 times as many tokens in some languages as in others.^[3] In their published data for the tokenizer used by GPT-4-era models, a standard set of sentences took about 52,800 tokens in English, 253,000 in Hindi and 406,000 in Gujarati: roughly 4.8 and 7.7 times as many.^[4]

That data is from 2023 and from one family of models; newer tokenizers handle Indian scripts differently, and the gap may be much smaller. The practical lesson still stands. More tokens means higher cost per message and less room in the context window for your documents. Before you launch, measure token counts for real Gujarati and Hindi messages on the exact model you plan to use, and decide which language your documents should be in.

What does an AI chatbot cost to run?

Model fees are charged per million tokens (MTok), separately for what you send (input) and what the model writes (output). As of 29 October 2026, Anthropic lists its Claude Haiku 5.5 model at US$0.10 per million input tokens and US$0.50 per million output tokens, for prompts up to 100,000 tokens.^[5] Other providers and models cost more or less.

So the model is rarely the expensive part. The real costs are elsewhere: preparing and maintaining the documents, building the retrieval and checks, connecting to WhatsApp or your website, any messaging-platform charges, testing, and someone reviewing conversations every week. If the same conversations are in Gujarati, the token lines above could be several times larger.^[4]

What rules apply to business chatbots on WhatsApp and elsewhere?

You are responsible for what the bot says

In Moffatt v. Air Canada (2024), a Canadian tribunal ordered the airline to compensate a customer after its website chatbot wrongly described a bereavement-fare refund. Air Canada argued the chatbot was a separate legal entity responsible for its own actions. The tribunal rejected that and found the airline liable for negligent misrepresentation.^[6] It is a small-claims decision from another country, not Indian law, but the logic is the one any customer will use: your bot spoke for your business.

WhatsApp allows business bots, not general AI assistants

Meta’s terms for the WhatsApp Business Platform bar AI providers from using it to offer AI where the AI is the primary functionality, rather than incidental or ancillary.^[7] Press coverage reported this applying to existing users from 15 January 2026, while confirming that businesses using AI for customer service, bookings or order tracking are not affected.^[8] The same terms say WhatsApp data cannot be used to train or improve AI models, although fine-tuning a model for your business’s exclusive use is allowed.^[7] If a vendor offers a “ChatGPT on WhatsApp” product, ask how it fits these terms.

The WhatsApp order automation guide for Gujarat distributors covers which order steps are safe to automate.

How should a local business start with an AI chatbot?

1. Pick one narrow job. “Answer delivery-area and stock-availability questions” beats “handle all customer queries”.
2. Write the source documents first. A one-page delivery policy and a current price list will do more than any model choice.
3. Collect 50 real questions from your WhatsApp and phone logs, in the languages customers really use, and decide the right answer for each.
4. Test before launch against those 50, several times each, and record failures.
5. Always offer a person. Make “talk to someone” easy, and hand over automatically when the bot is unsure.
6. Review weekly for the first months, and fix documents, not just prompts.

When a bot needs to take actions such as creating an order or updating a record, it starts to become an “agent”, with more risk and more need for limits. Our guide on what AI agents can and can’t do for small businesses covers that step. If you plan to buy a chatbot as a subscription, the checklist in what SaaS means for your business applies too.

Frequently asked questions

Sources and further reading

1. Anthropic. Glossary: pretraining, tokens, context window, RAG, temperature. Vendor documentation; the concepts are general to large language models.
2. National Institute of Standards and Technology. NIST AI 600-1: Generative AI Profile (July 2024). Defines confabulation.
3. Petrov, La Malfa, Torr and Bibi (NeurIPS 2023). Language Model Tokenizers Introduce Unfairness Between Languages. Up to 15× differences in token length across languages.
4. Petrov et al.. tokenization-fairness dataset (tokenization_lengths_validated.csv). cl100k_base counts: English 52,835; Hindi 253,270; Gujarati 406,224. 2023 data.
5. Anthropic. Pricing. Claude Haiku 5.5 rates, checked 29 October 2026. Prices change.
6. Civil Resolution Tribunal of British Columbia. Moffatt v. Air Canada, 2024 BCCRT 149 (14 February 2024). Airline liable for its chatbot’s misrepresentation.
7. Meta. Meta Terms for WhatsApp Business Platform, section 4.7 (AI Providers). Last updated 23 September 2026.
8. respond.io. WhatsApp general-purpose chatbots ban. Secondary: effective dates and Meta’s statement on customer-service AI.
9. Digital Personal Data Protection Act, 2023. Section 8: obligations of Data Fiduciary (text). Unofficial reproduction of the Act.

Want a chatbot that answers only from your documents?

We build AI assistants that use your own price lists and policies, hand over to a person when unsure, and are tested in the languages your customers use. See our AI development work in Vadodara.