<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[ChatNory Blog]]></title><description><![CDATA[ChatNory Blog]]></description><link>https://chatnory.hashnode.dev</link><generator>RSS for Node</generator><lastBuildDate>Fri, 13 Nov 2026 19:47:43 GMT</lastBuildDate><atom:link href="https://chatnory.hashnode.dev/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[One bot, four channels: what actually differs between web chat, WhatsApp, Messenger and Telegram]]></title><description><![CDATA["Put the same bot on WhatsApp and Telegram too" sounds like a configuration change. It is not a hard project either — but the parts that differ are not the…]]></description><link>https://chatnory.hashnode.dev/one-bot-four-channels-what-actually-differs-between-web-chat-whatsapp-messenger-and-telegram</link><guid isPermaLink="true">https://chatnory.hashnode.dev/one-bot-four-channels-what-actually-differs-between-web-chat-whatsapp-messenger-and-telegram</guid><category><![CDATA[chatbot]]></category><category><![CDATA[whatsapp]]></category><category><![CDATA[telegram]]></category><category><![CDATA[api]]></category><dc:creator><![CDATA[D Raju]]></dc:creator><pubDate>Thu, 12 Nov 2026 11:41:50 GMT</pubDate><content:encoded><![CDATA[<p>"Put the same bot on WhatsApp and Telegram too" sounds like a configuration change. It is not a hard project either — but the parts that differ are not the parts people expect.</p>
<p>The messaging APIs are the easy half. Each one is a POST with a token. What actually costs you time is that the four channels disagree about who the user is, how long you may speak, what a message can contain, and whether anyone is listening. Build as if those are the same and you get a bot that works beautifully in the browser and behaves strangely everywhere else.</p>
<p>Here is the comparison I wish I had seen before starting, and the shape of code that survives it.</p>
<h2>The comparison that matters</h2>
<table>
<thead>
<tr>
<th></th>
<th>Web chat</th>
<th>WhatsApp</th>
<th>Messenger</th>
<th>Telegram</th>
</tr>
</thead>
<tbody><tr>
<td>Who the user is</td>
<td>Anonymous, per browser</td>
<td>Phone number</td>
<td>Page-scoped id</td>
<td>Telegram user id</td>
</tr>
<tr>
<td>Identity survives</td>
<td>Until storage is cleared</td>
<td>Forever</td>
<td>Per page</td>
<td>Forever</td>
</tr>
<tr>
<td>When you may send</td>
<td>Any time</td>
<td>24h window, then templates only</td>
<td>24h window, with tagged exceptions</td>
<td>Any time</td>
</tr>
<tr>
<td>Costs money per message</td>
<td>No</td>
<td>Yes</td>
<td>No</td>
<td>No</td>
</tr>
<tr>
<td>Rich formatting</td>
<td>Anything you render</td>
<td>Limited markdown-ish</td>
<td>Plain text + templates</td>
<td>HTML or MarkdownV2</td>
</tr>
<tr>
<td>Typing indicator</td>
<td>Yours to fake</td>
<td>Supported</td>
<td>Supported</td>
<td>Supported</td>
</tr>
<tr>
<td>Delivery visible to you</td>
<td>Trivially</td>
<td>Separate status events</td>
<td>Separate status events</td>
<td>Send response only</td>
</tr>
<tr>
<td>Message length</td>
<td>Your choice</td>
<td>~4,096 chars</td>
<td>2,000 chars</td>
<td>4,096 chars</td>
</tr>
</tbody></table>
<p>Four rows in that table cause nearly all the work.</p>
<h2>1. Identity: the hardest one, and it is not technical</h2>
<p>On the web the user is anonymous. On WhatsApp you have a verified phone number. This sounds like a straight upgrade, and it is also a trap: <strong>a phone number is personal data, and the same person on two channels is two different users to you.</strong></p>
<p>Decide early what a "contact" is in your system, because it determines whether you can join conversations later:</p>
<ul>
<li>A <strong>conversation</strong> belongs to one channel and one channel identity. Always.</li>
<li>A <strong>contact</strong> is a person, and may link several channel identities.</li>
<li>Linking happens only when the person tells you — they give an email in web chat that matches a contact, they message from a number you already hold. Never infer it.</li>
</ul>
<p>Do not skip the contact layer because you launched with one channel. Adding it later means backfilling every conversation you ever stored, and guessing at links you no longer have evidence for.</p>
<p>And store the phone number the way personal data should be stored: scoped to the tenant, out of logs, out of analytics events, deletable on request.</p>
<h2>2. The reply window is a design constraint, not an API detail</h2>
<p>Web chat has no concept of being "allowed" to reply. The other three do, and WhatsApp's is the strictest: a customer's message opens a 24-hour window in which you may send free-form text, and outside it you may only send a pre-approved template. Messenger has its own 24-hour rule with a set of tagged exceptions. Telegram, pleasantly, just lets you message.</p>
<p>Two consequences worth designing for rather than discovering:</p>
<p><strong>Human handover breaks across the window.</strong> Your bot escalates at 11pm, an agent replies at 10am — on WhatsApp that reply is outside the window and will be rejected. The whole escalation flow needs a template path, or your agents will send messages that silently never arrive.</p>
<p><strong>Anything proactive is a different feature per channel.</strong> An appointment reminder is free text on Telegram, a template with an approval queue on WhatsApp, and a tagged message on Messenger. Build it as one feature and you will build it three times anyway; better to know that up front.</p>
<p>So the send path has a gate, and the gate is per channel:</p>
<pre><code class="language-js">// Each adapter answers for itself. The core never reasons about windows.
const canSendFreeForm = {
  web:       () =&gt; true,
  telegram:  () =&gt; true,
  whatsapp:  c =&gt; Date.now() - c.lastInboundAt &lt; 24 * 60 * 60 * 1000,
  messenger: c =&gt; Date.now() - c.lastInboundAt &lt; 24 * 60 * 60 * 1000
};
</code></pre>
<p>Evaluate it immediately before sending, never when queueing — a reply that waited forty minutes in a queue may have been legal when it was created and illegal by the time it goes out.</p>
<h2>3. Formatting: write once in a neutral form, render per channel</h2>
<p>The answer your model produces is Markdown. Nobody accepts Markdown.</p>
<p>Telegram wants HTML or MarkdownV2 — and MarkdownV2 requires escaping a long list of characters, so an unescaped <code>.</code> or <code>-</code> in an answer returns a parse error and the message never sends. WhatsApp has its own asterisk-and-underscore convention. Messenger is plain text with structured attachments beside it. Web chat can render whatever you like.</p>
<p>The mistake is letting channel formatting leak into the generation prompt — "reply in Telegram HTML" — because then the answer is wrong for every other channel and you cannot reuse it.</p>
<p>Generate neutral, render at the edge:</p>
<pre><code class="language-js">// The model returns Markdown. Each adapter is the only thing that knows
// what its channel accepts.
const render = {
  web:       md =&gt; markdownToHtml(md),
  telegram:  md =&gt; markdownToTelegramHtml(md),   // and escape &lt; &gt; &amp;
  whatsapp:  md =&gt; markdownToWhatsAppText(md),   // *bold*, _italic_, the rest flattened
  messenger: md =&gt; stripToPlainText(md)
};
</code></pre>
<p>Two details that bite: <strong>links</strong>, which web chat renders inline but plain-text channels need spelled out in full, so a bare URL is safer than <code>[click here](...)</code>; and <strong>length</strong>, since a 3,000-character answer is fine on the web, over the limit on Messenger, and unreadable on a phone anywhere. Cap the answer in the prompt and split long ones in the adapter.</p>
<h2>4. The adapter boundary that keeps this sane</h2>
<p>Everything above points at the same structure. One core that knows nothing about channels, and a small adapter per channel that knows everything:</p>
<pre><code class="language-js">// Every adapter normalises inbound to the same shape...
// { channel, channelUserId, conversationId, text, attachments, receivedAt }
//
// ...and takes outbound in the same shape:
// { text (markdown), quickReplies?, handoff? }

async function handleInbound(msg) {
  const conversation = await findOrCreateConversation(msg);
  const answer = await generateAnswer(conversation, msg.text);
  const adapter = adapters[msg.channel];

  if (!adapter.canSendFreeForm(conversation)) {
    return adapter.sendTemplate(conversation, 'reopen', { topic: answer.topic });
  }
  return adapter.send(conversation, adapter.render(answer.text));
}
</code></pre>
<p>The test for whether you have the boundary right: <strong><code>generateAnswer</code> contains no <code>if (channel === ...)</code>.</strong> If it does, the channel has leaked into the brain, and every new channel will mean touching it.</p>
<p>Two more things that belong in the adapter and nowhere else: retries and deduplication. WhatsApp and Messenger both redeliver webhooks when you are slow to acknowledge, so every adapter needs to ack fast and drop message ids it has already seen. Telegram's long-polling has its own version of the same problem. The core should never see a duplicate.</p>
<h2>5. What each channel is actually good at</h2>
<p>Worth being honest about, because "be everywhere" is not a strategy:</p>
<ul>
<li><strong>Web chat</strong> is the only one where the visitor is already looking at the thing they want to ask about. The best context, the worst retention — close the tab and they are gone.</li>
<li><strong>WhatsApp</strong> is where the customer already is, and the conversation persists for years in a thread they will scroll back through. It costs money and it has the strictest rules. Worth it when the customer base lives there.</li>
<li><strong>Messenger</strong> matters if the business's audience arrived through a Facebook page. If it did not, it is effort for nothing.</li>
<li><strong>Telegram</strong> is the easiest to build on by a wide margin — generous API, no fees, no windows — and the smallest audience for most businesses. It is a great first non-web channel precisely because it will teach you the adapter boundary cheaply.</li>
</ul>
<p>If you are adding your first extra channel, add Telegram to learn the shape, then WhatsApp because that is where the customers are.</p>
<h2>The short version</h2>
<p>The APIs are not the work. The work is identity, the reply window, formatting, and keeping all three out of the part that writes the answer. Get the adapter boundary right and the fourth channel takes an afternoon. Get it wrong and the second one takes a week and the third never ships.</p>
<hr />
<p><em>I build <a href="https://chatnory.com">ChatNory</a>, which runs one chatbot — trained on a business's own content — across a website, WhatsApp, Messenger and Telegram. The adapter boundary above is the second version; the first had channel checks in the answer generator and it was exactly as pleasant as it sounds.</em></p>
]]></content:encoded></item><item><title><![CDATA[Multi-tenant from day one: keeping one customer's chatbot data out of another's]]></title><description><![CDATA[The first version of almost every B2B tool is single-tenant by accident. You build it for one customer, it works, a second customer arrives, and you add a…]]></description><link>https://chatnory.hashnode.dev/multi-tenant-from-day-one-keeping-one-customer-s-chatbot-data-out-of-another-s</link><guid isPermaLink="true">https://chatnory.hashnode.dev/multi-tenant-from-day-one-keeping-one-customer-s-chatbot-data-out-of-another-s</guid><category><![CDATA[SaaS]]></category><category><![CDATA[Security]]></category><category><![CDATA[PostgreSQL]]></category><category><![CDATA[chatbot]]></category><dc:creator><![CDATA[D Raju]]></dc:creator><pubDate>Thu, 12 Nov 2026 11:39:16 GMT</pubDate><content:encoded><![CDATA[<p>The first version of almost every B2B tool is single-tenant by accident. You build it for one customer, it works, a second customer arrives, and you add a <code>customer_id</code> column.</p>
<p>That column is now the only thing standing between your two customers' data. One forgotten <code>WHERE</code> clause and customer A's chatbot answers a question using customer B's internal pricing document. You will not get a second chance to explain that.</p>
<p>This is what I would set up on day one of a multi-tenant product, in the order the risk actually appears. It is written around a chatbot — vector search makes one of these problems unusually sharp — but most of it is the same for any SaaS.</p>
<h2>1. Never pass the tenant id as an argument</h2>
<p>The single highest-value change, and it is architectural rather than clever.</p>
<p>If <code>getDocuments(tenantId, query)</code> takes the tenant as a parameter, then every call site is a chance to pass the wrong one, or to forget it in a new function written at 1am. Humans do not reliably remember a rule 400 times.</p>
<p>Instead, resolve the tenant <strong>once</strong>, at the edge, from the authenticated session — never from a request body, a query string, or a header the client controls — and carry it in a context the data layer reads for itself:</p>
<pre><code class="language-js">// One place turns a session into a tenant. Nothing downstream takes it as an argument.
app.use(async (req, res, next) =&gt; {
  const session = await getSession(req);
  if (!session) return res.sendStatus(401);
  // The tenant comes from the session, never from req.body.tenantId.
  tenantContext.run({ tenantId: session.tenantId }, () =&gt; next());
});

// The data layer reads the context itself. There is no overload without a tenant.
function documents() {
  const { tenantId } = tenantContext.getStore() ?? {};
  if (!tenantId) throw new Error('No tenant in context — refusing to query');
  return db('documents').where({ tenant_id: tenantId });
}
</code></pre>
<p>That <code>throw</code> matters more than it looks. The failure mode you want for a missing tenant is a loud crash, not a query that quietly returns everything.</p>
<p>In Node this is <code>AsyncLocalStorage</code>. Most ecosystems have an equivalent. The point is not the mechanism — it is that <strong>the safe path is the only path</strong>, so a new developer cannot write the unsafe one without noticing.</p>
<h2>2. Put the same rule in the database</h2>
<p>Application-level filtering is one bug away from failing. A second, independent layer means a bug needs two mistakes to become a breach.</p>
<p>In Postgres that is row-level security:</p>
<pre><code class="language-sql">ALTER TABLE documents ENABLE ROW LEVEL SECURITY;
ALTER TABLE documents FORCE ROW LEVEL SECURITY;   -- applies to the table owner too

CREATE POLICY tenant_isolation ON documents
  USING (tenant_id = current_setting('app.tenant_id')::uuid);
</code></pre>
<p>with the setting applied per transaction from the same context as above. <code>FORCE</code> is the line people miss: without it, the owner role bypasses the policy, and your application often connects as the owner.</p>
<p>This is defence in depth, not belt-and-braces theatre. The two layers fail for different reasons — one to a forgotten clause in code, the other to a misconfigured role — and it is unlikely you make both mistakes in the same hour.</p>
<h2>3. Vector search is where isolation quietly breaks</h2>
<p>Here is the chatbot-specific trap, and it is the one I would check first in anyone else's system.</p>
<p>A vector index does not have a <code>WHERE</code> clause in the way a SQL table does. Approximate nearest-neighbour search walks a graph and returns the <em>k</em> closest vectors it finds. If you then filter those results by tenant, two bad things happen:</p>
<ul>
<li><strong>Correctness:</strong> you asked for 20 and got 20 from across all tenants; after filtering, you might have 3 for this tenant, or none. The bot answers badly for no visible reason.</li>
<li><strong>Safety:</strong> every piece of filtering logic between the index and the answer is a place to get it wrong.</li>
</ul>
<p>Post-filtering is the wrong shape. The two shapes that work:</p>
<p><strong>An index per tenant.</strong> Simplest to reason about, and the cross-tenant bug becomes structurally impossible — a query against tenant A's index cannot return tenant B's rows, whatever the code does. The cost is many small indexes, which is fine for hundreds of tenants and painful for a hundred thousand.</p>
<p><strong>Pre-filtered search</strong>, where the tenant is a condition the index applies <em>during</em> the search, not after. <code>pgvector</code> does this naturally, because it is a Postgres query and row-level security already applies:</p>
<pre><code class="language-sql">SELECT id, content, 1 - (embedding &lt;=&gt; $1) AS score
FROM   documents                      -- RLS already scopes this to the tenant
ORDER  BY embedding &lt;=&gt; $1
LIMIT  20;
</code></pre>
<p>Managed vector databases support metadata pre-filtering too — but read their documentation carefully and test it, because "filter" in some engines means post-filter, and the difference is exactly the one that matters here.</p>
<p><strong>Write a test for this and never delete it.</strong> Two tenants, two documents, one query that would match the other tenant's document far better than its own:</p>
<pre><code class="language-js">test('a tenant never retrieves another tenant\'s content', async () =&gt; {
  await seed(tenantA, 'Our refund window is 30 days.');
  await seed(tenantB, 'Our refund window is 90 days.');

  const hits = await asTenant(tenantA, () =&gt; retrieve('how long is the refund window'));

  expect(hits).not.toHaveLength(0);                     // it still works
  expect(hits.every(h =&gt; h.tenantId === tenantA)).toBe(true);
  expect(hits.map(h =&gt; h.content).join(' ')).not.toMatch(/90 days/);
});
</code></pre>
<p>That last assertion is the one that catches a post-filter regression where the leak reaches the prompt but not the response object.</p>
<h2>4. Secrets belong to one tenant each</h2>
<p>Customers will give you API keys — their own model provider key, a WhatsApp token, an SMTP password. These need handling that plain columns do not give you:</p>
<ul>
<li><strong>Encrypted at rest with authenticated encryption</strong>, AES-256-GCM or equivalent, so a database dump is not a credential dump.</li>
<li><strong>Decrypted only at the moment of use</strong>, never loaded into a general-purpose settings object that gets logged or serialised into an API response.</li>
<li><strong>Masked everywhere they are displayed.</strong> Show the last four characters. There is no screen that needs to show the whole key, including the owner's own settings page — they have it already.</li>
<li><strong>Scrubbed from errors.</strong> A provider's error object will happily include the request headers you sent.</li>
</ul>
<h2>5. Make the tenant visible in everything you operate</h2>
<p>Isolation that you cannot observe is isolation you are only assuming.</p>
<ul>
<li><strong>Logs</strong> carry the tenant id on every line. Debugging a cross-tenant issue without it is guesswork.</li>
<li><strong>Metrics and rate limits</strong> are per tenant. One customer's traffic spike should not degrade everyone, and a per-tenant limit is also your defence against a runaway loop in someone's integration costing you a fortune.</li>
<li><strong>Background jobs</strong> carry the tenant explicitly, because they run outside the request that had the context. This is the most common place the whole scheme leaks: a cron job with no session, written months later.</li>
<li><strong>Caches</strong> are keyed by tenant. A cache key of <code>answer:${hash(question)}</code> will serve tenant A's answer to tenant B, and it will do it intermittently, which makes it nearly impossible to reproduce.</li>
</ul>
<p>That cache bug deserves emphasis. It is the one that gets shipped by careful teams, because the cache is added later as an optimisation, by someone who is thinking about latency rather than tenancy.</p>
<h2>What it costs to do this on day one</h2>
<p>Perhaps two days of work: the context plumbing, the RLS policies, the per-tenant index decision, the isolation test, the encrypted-secrets helper.</p>
<p>Retrofitting it onto a live product with real customers is weeks, because every query, job and cache written in the meantime has to be found and re-checked, and you cannot be sure you found them all. The second system is also the one where you are reading your own code wondering whether an untested path ever leaked.</p>
<p>The work is not hard. It is just much cheaper before the second customer than after the twentieth.</p>
<hr />
<p><em>I build <a href="https://chatnory.com">ChatNory</a>, a multi-tenant AI chatbot that answers from each business's own content. The isolation test in section 3 exists because the post-filtering version looked completely fine until it did not. If you run vector search across tenants, go and check whether your engine pre-filters or post-filters — it is a five-minute check and it is worth doing today.</em></p>
]]></content:encoded></item><item><title><![CDATA[What an AI support chatbot actually costs per conversation]]></title><description><![CDATA["How much will this cost to run?" is the question I get before any technical one, and the honest first answer is annoying: less than you fear, in a shape…]]></description><link>https://chatnory.hashnode.dev/what-an-ai-support-chatbot-actually-costs-per-conversation</link><guid isPermaLink="true">https://chatnory.hashnode.dev/what-an-ai-support-chatbot-actually-costs-per-conversation</guid><category><![CDATA[AI]]></category><category><![CDATA[SaaS]]></category><category><![CDATA[chatbot]]></category><category><![CDATA[Startups]]></category><dc:creator><![CDATA[D Raju]]></dc:creator><pubDate>Thu, 12 Nov 2026 11:36:39 GMT</pubDate><content:encoded><![CDATA[<p>"How much will this cost to run?" is the question I get before any technical one, and the honest first answer is annoying: <em>less than you fear, in a shape you did not expect.</em></p>
<p>People budget for the indexing — the "training on your content" part — because that is the part that sounds expensive. It is the cheapest line on the bill. The money goes somewhere else entirely, and if you do not know where, you will optimise the wrong thing for a month.</p>
<p>Here is the actual shape of it, with a method you can run on your own numbers. I am deliberately not quoting per-token prices: they change every few months, and a blog post with 2026 prices in it will be wrong by the time you read it. Go to your provider's pricing page, take today's numbers, and substitute them into the formulas below.</p>
<h2>The four lines on the bill</h2>
<p>Every answer your bot gives costs you four things, in wildly different proportions:</p>
<ol>
<li><strong>Indexing</strong> — embedding your content. Paid once per change.</li>
<li><strong>Query embedding</strong> — turning the customer's question into a vector. Paid per question.</li>
<li><strong>Generation</strong> — the LLM writing the answer. Paid per question, by input <em>and</em> output tokens.</li>
<li><strong>Everything that is not AI</strong> — hosting, the vector store, bandwidth, your time.</li>
</ol>
<p>Let me take them in order of how much people over- and under-estimate them.</p>
<h2>1. Indexing is rounding error</h2>
<p>Embedding is priced per token and embedding models are the cheapest thing in the stack, typically <strong>one to two orders of magnitude cheaper per token than a generation model's input</strong>.</p>
<p>A mid-sized business site — say 200 pages averaging 800 words — is roughly 210,000 tokens. Embedding that whole site, from scratch, costs a few cents at any current embedding price. Even re-embedding it nightly for a year is less than a single month of generation on a modestly busy bot.</p>
<p>The practical conclusion is a design decision, not a saving: <strong>stop treating re-indexing as expensive.</strong> Re-crawl nightly. Re-index the moment a page changes. The reason people's bots quote last year's prices is almost never cost — it is that nobody built the refresh.</p>
<h2>2. Query embedding is also rounding error</h2>
<p>A customer's question is 10–30 tokens. Embedding it costs a fraction of a fraction of a cent. You can safely ignore this line. I mention it only because people sometimes try to cache query embeddings, which is effort spent saving nothing.</p>
<h2>3. Generation is the whole bill</h2>
<p>Here is where it all is, and the number that drives it is not what people expect. It is not the answer length. It is <strong>the context you send</strong>.</p>
<p>A typical RAG request looks like:</p>
<table>
<thead>
<tr>
<th>Part</th>
<th>Rough size</th>
</tr>
</thead>
<tbody><tr>
<td>System prompt</td>
<td>300–600 tokens</td>
</tr>
<tr>
<td>Retrieved chunks (5 × ~600)</td>
<td>~3,000 tokens</td>
</tr>
<tr>
<td>Conversation history (4 turns)</td>
<td>400–1,200 tokens</td>
</tr>
<tr>
<td>The question</td>
<td>20 tokens</td>
</tr>
<tr>
<td><strong>Input total</strong></td>
<td><strong>~4,000–5,000 tokens</strong></td>
</tr>
<tr>
<td>The answer</td>
<td>100–250 tokens</td>
</tr>
</tbody></table>
<p>So you are paying for roughly <strong>4,500 input tokens and 200 output tokens per message</strong>. Input tokens are cheaper than output per token, but there are twenty times more of them — on most models' pricing, input is still the majority of the cost of a RAG answer.</p>
<p>And a "conversation" is not one message. Real support conversations run 3–6 messages. So:</p>
<pre><code class="language-plaintext">cost per conversation
  = turns × (input_tokens × input_price + output_tokens × output_price)
</code></pre>
<p>Plug in your provider's current per-million-token prices and you have your number. Do this before you price your product, not after.</p>
<h3>Why history is the sneaky one</h3>
<p>Notice what happens as a conversation goes on. Turn 1 sends 4,500 tokens. Turn 2 sends those 4,500 <em>plus</em> turn 1's question and answer. Turn 5 can be sending 8,000. Naive conversation handling makes cost grow <strong>quadratically with conversation length</strong>.</p>
<p>Two fixes, both cheap:</p>
<pre><code class="language-js">// 1. Cap history by tokens, not by message count, and always keep the
//    first user message — it usually holds the actual intent.
function trimHistory(messages, maxTokens = 1200) {
  const first = messages[0];
  const rest = [];
  let used = estimateTokens(first);
  for (const m of messages.slice(1).reverse()) {
    const t = estimateTokens(m);
    if (used + t &gt; maxTokens) break;
    rest.unshift(m);
    used += t;
  }
  return [first, ...rest];
}
</code></pre>
<ol start="2">
<li><strong>Re-retrieve per turn instead of accumulating chunks.</strong> If turn 3 is about shipping, you do not need turn 1's pricing chunks still sitting in the prompt. Retrieve fresh for each question and the input size stays flat instead of climbing.</li>
</ol>
<h3>The three levers that actually move the number</h3>
<p><strong>Prompt caching.</strong> Most providers now let you mark a stable prefix — your system prompt, your few-shot examples — as cacheable, and charge much less for cached input tokens on subsequent calls. Structure your prompt so everything stable comes <em>first</em> and everything variable comes last, or the cache never hits. This is the single largest saving available and it costs you one afternoon.</p>
<p><strong>Fewer, better chunks.</strong> Going from 10 chunks to 5 halves the largest part of your input. <a href="https://chatnory.com">Re-ranking</a> makes this safe — the five you keep are better than the ten you had. Cheaper <em>and</em> more accurate is a rare combination; take it.</p>
<p><strong>A small model for small jobs.</strong> Not every call needs your best model. Classifying "is this a greeting, a question, or abuse?", rewriting a follow-up into a standalone query, deciding whether to escalate — these run fine on the cheapest model in the family. Route by job, not by habit.</p>
<p>What does <strong>not</strong> move the number much: shortening your system prompt by a few sentences, trimming the answer length, or switching embedding models. People spend days there.</p>
<h2>4. The costs that are not tokens</h2>
<p>For a small deployment these often exceed the AI bill, and they get forgotten in every estimate I have seen:</p>
<ul>
<li><strong>The vector store.</strong> Hosted vector databases charge for storage and for the pods or units that keep the index warm, and that is a monthly floor whether or not anyone chats. At the scale of a single business's website, <code>pgvector</code> on a Postgres you already run, or SQLite with a vector extension, is genuinely enough — a few hundred thousand vectors is not big data. A managed vector service is a decision to make at scale, not at launch.</li>
<li><strong>Egress and hosting</strong> for the widget, the admin, the webhooks.</li>
<li><strong>WhatsApp</strong>, if you use it, is billed by Meta on its own per-message schedule, entirely separately from your AI costs. It is quite capable of being the largest line on the bill.</li>
<li><strong>Your time.</strong> Thirty minutes a month reading the unanswered-questions log costs more than the tokens.</li>
</ul>
<h2>Two numbers worth writing down</h2>
<p><strong>Cost per conversation</strong> tells you whether the unit economics work. Compute it from the formula above and sanity-check it against a real week of traffic.</p>
<p><strong>Cost per <em>resolved</em> conversation</strong> tells you whether the thing is worth running at all. If 40% of conversations end in a human being involved anyway, your real cost per deflected ticket is 2.5× the headline — and you should compare <em>that</em> against what the ticket would have cost in staff time.</p>
<p>That second number is the one to put in front of whoever approves the spend. It is also the one that improves when you fix your content, which brings everything back to the same place: the bot's economics are mostly a property of how well your website answers questions.</p>
<h2>The useful summary</h2>
<ul>
<li>Indexing: ignore it, and re-index far more often than you do.</li>
<li>Generation: all of the cost, and most of it is <strong>input</strong> tokens, not output.</li>
<li>Conversation history: the thing that quietly makes turn 5 twice the price of turn 1.</li>
<li>Prompt caching, fewer chunks, cheaper model for small jobs: the three levers that matter.</li>
<li>The vector store's monthly floor can beat your token bill at small scale. Start with Postgres.</li>
</ul>
<p>Work out the per-conversation number before you quote a client a monthly price. It takes twenty minutes and it is the difference between a margin and a surprise.</p>
<hr />
<p><em>I build <a href="https://chatnory.com">ChatNory</a>, an AI chatbot that answers from a business's own content on its website, WhatsApp and Telegram. These numbers come from running it, including the month I spent optimising the wrong line. If your cost shape looks different, I would like to hear why — especially if you are at a scale where the managed vector store starts paying for itself.</em></p>
]]></content:encoded></item><item><title><![CDATA[Six guardrails that stopped our support chatbot inventing answers]]></title><description><![CDATA[A support chatbot that makes things up is worse than no chatbot. A missing answer costs you one email. A confidently wrong answer about a refund policy…]]></description><link>https://chatnory.hashnode.dev/six-guardrails-that-stopped-our-support-chatbot-inventing-answers</link><guid isPermaLink="true">https://chatnory.hashnode.dev/six-guardrails-that-stopped-our-support-chatbot-inventing-answers</guid><category><![CDATA[chatbot]]></category><category><![CDATA[customer support ]]></category><category><![CDATA[AI]]></category><category><![CDATA[SaaS]]></category><dc:creator><![CDATA[D Raju]]></dc:creator><pubDate>Thu, 12 Nov 2026 05:13:48 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/6af3646bb23a0def963c8618/5b37f0b9-1875-49d1-865d-4483f0db7a19.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>A support chatbot that makes things up is worse than no chatbot. A missing answer costs you one email. A confidently wrong answer about a refund policy costs you an argument, and the customer has a screenshot.</p>
<p>"Use a better model" is not the fix. Most of the invented answers I have investigated were not the model being creative — they were the model being handed bad input and doing its best with it. The guardrails below are ordered roughly by how much they bought per hour of work.</p>
<h2>1. A retrieval floor, and a real "I don't know" path</h2>
<p>This one fixed more than the other five combined.</p>
<p>If your pipeline retrieves the top <em>k</em> chunks and passes them to the model, the model is <strong>always</strong> given context, even when the question has nothing to do with your content. Ask a plumbing company's bot about visa rules and it gets the three least-irrelevant plumbing paragraphs. Then it writes something.</p>
<p>So: score the best match, and if it falls below a threshold, do not call the generator at all.</p>
<pre><code class="language-js">const MIN_SCORE = 0.35;   // tune on your own data; this is not universal

async function answer(question) {
  const hits = await retrieve(question, { k: 20 });
  const best = hits[0];

  if (!best || best.score &lt; MIN_SCORE) {
    return {
      text: "I don't have that in our content. I can pass this to the team — " +
            "want me to take your email?",
      sources: [],
      unanswered: true          // log it, see #6
    };
  }
  return generate(question, rerank(question, hits).slice(0, 5));
}
</code></pre>
<p>Two notes. The threshold is <strong>not</strong> transferable between embedding models or even between content sets — measure it. And the fallback has to be a real path, not a dead end: an email capture, a WhatsApp handoff, a phone number. A bot that says "I don't know" and stops is just a slower 404.</p>
<h2>2. Make citation structural, not polite</h2>
<p>Asking the model nicely to "cite your sources" gets you citations most of the time. Making the output format require them gets you citations you can verify.</p>
<p>Number the chunks in the prompt and require the number inline:</p>
<pre><code class="language-plaintext">Source [1] (Pricing &gt; Starter plan):
...

Source [2] (Refund policy):
...

Answer the question using only the sources above. After every factual
sentence, put the source number in square brackets. If the sources do not
contain the answer, say exactly: NOT_IN_SOURCES
</code></pre>
<p>The point is not prettiness in the UI, although showing the source link does visibly increase trust. The point is that <strong>you can now check the work automatically</strong>. For every answer, take each cited chunk and ask: does this chunk plausibly support this sentence? A cheap second model call, or even keyword overlap, catches a surprising share of fabrications — and an answer with <em>no</em> citations at all is immediately suspicious.</p>
<p><code>NOT_IN_SOURCES</code> as a literal sentinel is deliberate. A string you can match beats trying to detect hedging in prose.</p>
<h2>3. Strip your own knowledge out of the system prompt</h2>
<p>Early on our system prompt contained the pricing tiers, the support hours and the refund window. It made the bot look great in testing.</p>
<p>Then the prices changed. The pages were updated, the index was rebuilt — and the bot kept quoting the old numbers with total confidence, because the system prompt outranked the retrieved content. It took days to find, because every other answer was fine.</p>
<p>The rule I now hold to: <strong>the system prompt gets behaviour, never facts.</strong> Tone, language, format, what to do when unsure, when to escalate. Anything a customer could be told as a fact lives in the content, where there is exactly one copy of it and updating the page updates the bot.</p>
<h2>4. Make "no" the easy answer, with examples</h2>
<p>Models hedge toward helpfulness. Telling them "say you don't know if you don't know" is weaker than showing them.</p>
<p>Two or three examples in the prompt do more than a paragraph of instruction:</p>
<pre><code class="language-plaintext">Q: Do you ship to Dubai?
Sources: [1] Shipping covers India and Nepal.
A: Our shipping page covers India and Nepal [1]. Dubai isn't listed, so I
   can't confirm it — I'll check with the team if you leave your email.

Q: Can I get a refund after 60 days?
Sources: [1] Refunds within 30 days of purchase.
A: NOT_IN_SOURCES
</code></pre>
<p>The second example is doing real work. It teaches the model that "30 days" does <em>not</em> license a reasoned extrapolation about 60 days. Without it, models are very willing to reason past the evidence — and that reasoning reads exactly as authoritative as a correct answer.</p>
<h2>5. Keep the retrieved context small and ordered</h2>
<p>More context is not better. Two things degrade as you stuff the prompt:</p>
<p><strong>Dilution.</strong> Twenty chunks with one relevant paragraph is worse than four chunks with one relevant paragraph. The model has more plausible-looking material to blend.</p>
<p><strong>Position.</strong> Long-context models still attend unevenly — material in the middle of a long context gets used less reliably than material at the start or end. If you retrieve twenty and keep five, put the best match first.</p>
<p>Four to six chunks after re-ranking has been the sweet spot for us. It is also the cheaper option, which makes it an unusually easy decision.</p>
<h2>6. Log every unanswered question and read the log weekly</h2>
<p>This is the only guardrail that gets <em>better</em> over time, and it is the one most people skip.</p>
<p>Every time the retrieval floor fires, write down the question. Then read the list.</p>
<p>What comes back is three kinds of entry, and all three are useful:</p>
<ul>
<li><strong>Questions your content genuinely does not cover.</strong> This is a content backlog written by your customers, prioritised by frequency. It is better input than any keyword tool.</li>
<li><strong>Questions your content <em>does</em> cover but retrieval missed.</strong> These are the real bugs. Usually vocabulary: customers say "bill", your page says "invoice". Fix by adding the customer's word to the page, or with a small synonym map at query time.</li>
<li><strong>Questions that are not your business at all.</strong> Noise, and reassuring — it means the floor is doing its job.</li>
</ul>
<p>A monthly pass over that list moved our answer rate more than any prompt rewrite.</p>
<h2>What this adds up to</h2>
<p>Nothing here is clever, and that is roughly the point. Five of the six guardrails are about <em>refusing</em> to answer in the right circumstances, and one is about reading the log.</p>
<p>The uncomfortable truth underneath them: an AI support bot's quality is mostly a property of your documentation, not your model. If the answer is not written down anywhere, the honest output is "I don't know" — and the real work is writing the page.</p>
<hr />
<p><em>I build</em> <a href="https://chatnory.com"><em>ChatNory</em></a><em>, an AI chatbot that answers from a business's own content on its website, WhatsApp and Telegram. All six of these exist because the earlier version did not have them. If you have a guardrail that earned its keep, put it in the comments — I will try it.</em></p>
]]></content:encoded></item><item><title><![CDATA[Putting an AI chatbot on WhatsApp: the five parts of the Cloud API that cost me a week]]></title><description><![CDATA[The hard part of putting a chatbot on WhatsApp is not the chatbot. It is WhatsApp. The messaging itself is a plain HTTPS POST and you will have it working…]]></description><link>https://chatnory.hashnode.dev/putting-an-ai-chatbot-on-whatsapp-the-five-parts-of-the-cloud-api-that-cost-me-a-week</link><guid isPermaLink="true">https://chatnory.hashnode.dev/putting-an-ai-chatbot-on-whatsapp-the-five-parts-of-the-cloud-api-that-cost-me-a-week</guid><category><![CDATA[whatsapp]]></category><category><![CDATA[chatbot]]></category><category><![CDATA[customer support ]]></category><category><![CDATA[AI]]></category><dc:creator><![CDATA[D Raju]]></dc:creator><pubDate>Thu, 12 Nov 2026 05:10:41 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/6af3646bb23a0def963c8618/c46b4546-a234-4d0e-b5bb-398be781dcf4.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>The hard part of putting a chatbot on WhatsApp is not the chatbot. It is WhatsApp.</p>
<p>The messaging itself is a plain HTTPS POST and you will have it working in twenty minutes. Then you spend a week discovering the rules Meta's platform enforces that no other channel has: a reply window, a template approval queue, a billing model that keeps changing, and a webhook that is cheerfully lying to you about delivery.</p>
<p>This is the list I wish I had read first. It assumes WhatsApp Cloud API (Meta's own hosted version, no BSP), Node on the receiving end, and that you already have some kind of answer engine you want to put behind it.</p>
<h2>1. Webhook verification is a GET, and the field you forget is <code>hub.challenge</code></h2>
<p>Meta verifies your endpoint with a <code>GET</code> before it will ever send you a <code>POST</code>. The same URL has to answer both.</p>
<pre><code class="language-js">app.get('/webhook/whatsapp', (req, res) =&gt; {
  const mode = req.query['hub.mode'];
  const token = req.query['hub.verify_token'];
  const challenge = req.query['hub.challenge'];

  // The verify token is a string you invent and paste into the Meta dashboard.
  // It is not a secret Meta gives you.
  if (mode === 'subscribe' &amp;&amp; token === process.env.WA_VERIFY_TOKEN) {
    // Must be echoed back as plain text, not JSON, or verification fails.
    return res.status(200).send(challenge);
  }
  return res.sendStatus(403);
});
</code></pre>
<p>Two traps in six lines. The verify token is something <em>you</em> choose — people hunt the dashboard for it. And the challenge must come back as a bare string: <code>res.json(challenge)</code> wraps it in quotes and the check fails with no useful error.</p>
<p>While you are here, verify the signature on incoming POSTs too. Meta signs the raw body with your app secret in <code>X-Hub-Signature-256</code>, and you need the <strong>raw</strong> bytes, before any JSON parser touched them:</p>
<pre><code class="language-js">import crypto from 'node:crypto';

// Mount this BEFORE express.json() for the webhook route.
app.post('/webhook/whatsapp',
  express.raw({ type: 'application/json' }),
  (req, res) =&gt; {
    const expected = 'sha256=' + crypto
      .createHmac('sha256', process.env.WA_APP_SECRET)
      .update(req.body)            // Buffer, untouched
      .digest('hex');

    const given = req.get('X-Hub-Signature-256') ?? '';
    const ok = given.length === expected.length &amp;&amp;
      crypto.timingSafeEqual(Buffer.from(given), Buffer.from(expected));

    if (!ok) return res.sendStatus(401);

    res.sendStatus(200);          // ack first, work after
    handle(JSON.parse(req.body.toString('utf8')));
  });
</code></pre>
<p>Without this, anyone who learns your webhook URL can post fake inbound messages to it and make your bot answer — or, worse, make it answer at your expense.</p>
<h2>2. Acknowledge in under a few seconds, or you get the same message five times</h2>
<p>Meta expects a <code>200</code> quickly. If your handler waits for an LLM to finish, the request times out, Meta retries, and your bot answers the same question several times — each answer billable.</p>
<p>So: respond <code>200</code> immediately, then do the work. A real queue is better, but even <code>setImmediate</code> beats answering inline.</p>
<p>Retries also mean <strong>you will receive duplicates anyway</strong>, on timeouts, redeliveries and network hiccups. Every inbound payload carries a message id; store the ones you have processed and drop repeats.</p>
<pre><code class="language-js">const seen = new Set();   // use Redis or a table in production

function handle(payload) {
  const msg = payload.entry?.[0]?.changes?.[0]?.value?.messages?.[0];
  if (!msg) return;        // status callbacks arrive on the same webhook
  if (seen.has(msg.id)) return;
  seen.add(msg.id);
  answer(msg);
}
</code></pre>
<p>Note <code>if (!msg) return</code>. Delivery receipts — sent, delivered, read, failed — come to the <em>same</em> webhook under <code>value.statuses</code>, not <code>value.messages</code>. Early on, roughly two thirds of the events I logged were status callbacks, and the handler kept crashing on them.</p>
<h2>3. The 24-hour customer service window is the rule that shapes your whole design</h2>
<p>This is the one that surprises people coming from web chat.</p>
<p>When a customer messages you, a <strong>24-hour customer service window</strong> opens. Inside it you may reply with free-form text — anything your bot wants to say. The window restarts every time the customer sends another message.</p>
<p>Outside the window you cannot send free-form text at all. You may only send a <strong>pre-approved template</strong>. Not "it gets flagged" — the API rejects it.</p>
<p>Consequences worth designing around:</p>
<ul>
<li><strong>Check the window at send time, not at queue time.</strong> A reply that sat in a queue for 40 minutes may have been valid when queued and invalid when sent. Re-read the customer's last-message timestamp immediately before you call the API.</li>
<li><strong>A human handover that takes a day is a different message.</strong> If your bot escalates to a person and that person replies tomorrow morning, their reply is no longer free-form. It needs a template. Build that path deliberately or your agents will quietly fail to reach customers.</li>
<li><strong>You cannot start conversations casually.</strong> Any outbound-first message — an abandoned-cart nudge, an appointment reminder — is a template, with an approval queue in front of it.</li>
</ul>
<p>Store the window state with each conversation rather than computing it ad hoc:</p>
<pre><code class="language-js">// Call this immediately before every send.
function canSendFreeForm(conversation) {
  if (!conversation.lastInboundAt) return false;
  const ageMs = Date.now() - conversation.lastInboundAt;
  return ageMs &lt; 24 * 60 * 60 * 1000;
}
</code></pre>
<h2>4. Templates are a product decision, not a formatting detail</h2>
<p>A template is a message approved in advance by Meta, with numbered placeholders you fill at send time:</p>
<pre><code class="language-plaintext">Hi {{1}}, your order {{2}} has shipped. Track it here: {{3}}
</code></pre>
<p>Three things make them harder than they look.</p>
<p><strong>The category you pick changes the price.</strong> Marketing, utility, authentication and service are billed differently, and Meta has been adjusting that model repeatedly — most recently with changes effective 1 October 2026. Do not hardcode the rate card into your business case; read <a href="https://developers.facebook.com/docs/whatsapp/pricing">Meta's current pricing page</a> the week you launch, and again before you scale. Anything written about WhatsApp pricing more than a few months ago, including blog posts quoting confident numbers, is likely wrong.</p>
<p><strong>Approval takes time and can be rejected.</strong> Promotional language in a utility template gets declined. Build your templates early, not the day before launch.</p>
<p><strong>Placeholders cannot contain newlines or tabs</strong>, and a parameter that arrives empty fails the send. Validate before calling the API, because the error you get back is not specific.</p>
<p>The practical pattern: keep a <strong>small</strong> set of templates whose job is to re-open a conversation — "You asked about X, reply to continue" — and keep all the real intelligence in the free-form replies inside the window. Trying to express a whole AI conversation through approved templates is a dead end.</p>
<h2>5. Delivery is not confirmation, and failures arrive later</h2>
<p>The <code>200</code> from the send endpoint means Meta accepted the message. It says nothing about whether it arrived. Delivery, read and failure all arrive later, on the webhook, as status events.</p>
<p>A message can fail after acceptance because the number is not on WhatsApp, because the user blocked you, or because you hit a rate limit. If your UI shows "sent" based on the POST response, it is showing fiction. Carry the status events through to whatever the business actually looks at.</p>
<h2>The part that is genuinely easy</h2>
<p>After all of that, sending is anticlimactic:</p>
<pre><code class="language-js">await fetch(
  `https://graph.facebook.com/v21.0/${process.env.WA_PHONE_NUMBER_ID}/messages`,
  {
    method: 'POST',
    headers: {
      Authorization: `Bearer ${process.env.WA_TOKEN}`,
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({
      messaging_product: 'whatsapp',   // required, and easy to forget
      to: customerPhoneE164,           // digits only, country code, no '+'
      type: 'text',
      text: { body: answer, preview_url: false }
    })
  }
);
</code></pre>
<p><code>messaging_product</code> is mandatory on every call and its absence produces an error message that does not mention it.</p>
<h2>What I would tell myself at the start</h2>
<p>Build the window logic before the bot. The retry and duplicate handling before the prompt. The template set before the launch date. Those four things are where the week went, and none of them are about AI at all — they are the cost of being a guest on someone else's messaging network.</p>
<p>The upside is real, though: WhatsApp is where a lot of customers already are, and a bot that answers there at 11pm is doing work nobody was going to do anyway.</p>
<hr />
<p><em>I build</em> <a href="https://chatnory.com"><em>ChatNory</em></a><em>, which puts one chatbot — trained on a business's own content — on a website, WhatsApp, Messenger and Telegram at the same time. Every bruise above is first-hand. If you have found a cleaner way to handle the handover-after-24-hours problem, I would genuinely like to hear it.</em></p>
]]></content:encoded></item><item><title><![CDATA[What "train the chatbot on your website" actually does: chunking, embeddings and retrieval]]></title><description><![CDATA[Every chatbot product says the same sentence: "trained on your own content." It sounds like fine-tuning. It almost never is. What nearly all of them do is…]]></description><link>https://chatnory.hashnode.dev/what-train-the-chatbot-on-your-website-actually-does-chunking-embeddings-and-retrieval</link><guid isPermaLink="true">https://chatnory.hashnode.dev/what-train-the-chatbot-on-your-website-actually-does-chunking-embeddings-and-retrieval</guid><category><![CDATA[AI]]></category><category><![CDATA[webdev]]></category><category><![CDATA[chatbot]]></category><category><![CDATA[SaaS]]></category><category><![CDATA[customer support ]]></category><dc:creator><![CDATA[D Raju]]></dc:creator><pubDate>Thu, 12 Nov 2026 05:06:58 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/6af3646bb23a0def963c8618/cac80628-e428-48f6-b1b1-05ce5ff816cb.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Every chatbot product says the same sentence: <em>"trained on your own content."</em> It sounds like fine-tuning. It almost never is.</p>
<p>What nearly all of them do is <strong>retrieval-augmented generation</strong> — the model is never trained on anything. Your content is cut into pieces, each piece is turned into a vector, and at question time the pieces that look closest to the question get pasted into the prompt. The model then answers from text it was handed half a second ago.</p>
<p>That distinction matters, because it tells you where the bugs live. When the answer is wrong, the model is usually innocent. The retrieval step handed it the wrong paragraph.</p>
<p>Here is what each step actually does, and the knobs that change the outcome.</p>
<h2>Step 1: getting the text out</h2>
<p>Before anything clever happens, something has to read your pages. This is the least glamorous step and the one that quietly ruins the most deployments.</p>
<p>A few things that bite:</p>
<ul>
<li><strong>Navigation and footers.</strong> Keep them and every chunk on the site contains your menu. Every chunk then looks slightly similar to every other chunk, and similarity search gets measurably worse. Strip to the main content region.</li>
<li><strong>JavaScript-rendered pages.</strong> A plain <code>fetch</code> of a React site returns an empty <code>&lt;div id="root"&gt;&lt;/div&gt;</code>. You need a headless browser, or the site's API, or its sitemap of pre-rendered pages.</li>
<li><strong>PDFs with two columns.</strong> Naive extraction reads straight across, so you get the first line of column one followed by the first line of column two. The text is technically present and completely meaningless.</li>
<li><strong>Tables.</strong> A price table flattened to one line of numbers loses which number belonged to which row. Converting tables to Markdown before chunking is worth the effort, because a model reads a Markdown table perfectly well.</li>
</ul>
<p>A sanity check that costs nothing: print the extracted text for five pages and read it yourself. If you cannot answer a question from it, no model can either.</p>
<h2>Step 2: chunking</h2>
<p>You cannot put the whole site in the prompt. Not mainly because of the context limit — modern models have room — but because accuracy drops when the relevant sentence is buried in two hundred pages of noise, and because you would pay for those tokens on every single question.</p>
<p>So the text gets split, and the split size is a real trade-off:</p>
<ul>
<li><strong>Too small</strong> (a sentence or two) and the chunk loses its subject. "It costs 1,499 per month" is useless without the sentence before it naming the plan.</li>
<li><strong>Too large</strong> (a whole page) and one chunk covers five topics. It matches many questions weakly and few questions strongly, so it crowds out better matches.</li>
</ul>
<p>In practice something in the <strong>400–800 token</strong> range works for prose, with a <strong>10–15% overlap</strong> between consecutive chunks so a fact that straddles a boundary survives in at least one piece.</p>
<p>Better than tuning the number is <strong>splitting on structure</strong>. Headings exist because the author already decided where topics begin:</p>
<pre><code class="language-js">// Split on headings first; fall back to size-based splitting
// only when a section is genuinely too long.
function chunkByHeading(markdown, maxTokens = 700) {
  const sections = markdown.split(/\n(?=#{1,3}\s)/);
  const out = [];

  for (const section of sections) {
    if (estimateTokens(section) &lt;= maxTokens) {
      out.push(section);
      continue;
    }
    // Long section: accumulate paragraphs up to the limit.
    let buffer = '';
    for (const para of section.split(/\n{2,}/)) {
      if (buffer &amp;&amp; estimateTokens(buffer + para) &gt; maxTokens) {
        out.push(buffer.trim());
        buffer = '';
      }
      buffer += para + '\n\n';
    }
    if (buffer.trim()) out.push(buffer.trim());
  }
  return out;
}
</code></pre>
<p>One cheap trick that pays for itself: <strong>prefix each chunk with its page title and heading path</strong> before embedding it. A chunk that begins <code>Pricing &gt; Starter plan —</code> matches a question about the starter plan far better than a bare paragraph does, because the words the user typed are literally in the text.</p>
<h2>Step 3: embeddings</h2>
<p>An embedding model turns a piece of text into a list of numbers — 1536 of them for OpenAI's <code>text-embedding-3-small</code>, 3072 for the <code>large</code> variant, 768 or 1024 for most open models. Text with similar meaning lands near other similar text in that space.</p>
<p>Three things worth knowing.</p>
<p><strong>The same model must be used on both sides.</strong> Chunks embedded with model A and questions embedded with model B produce coordinates in two unrelated spaces. The similarity scores will look plausible and mean nothing. If you upgrade the embedding model you re-embed everything, so store the model name next to every vector and you will know what needs rebuilding.</p>
<p><strong>Embeddings are cheap; generation is not.</strong> Embedding a few hundred pages costs cents. The recurring cost of a chatbot is answer generation, not indexing. This is why re-indexing a whole site nightly is perfectly affordable, and why "the bot still quotes last month's price" is a choice rather than a constraint.</p>
<p><strong>Similarity is not relevance.</strong> Two chunks about two different plans sit <em>very</em> close together in embedding space, because they share almost all their vocabulary. Embeddings are good at topic and bad at distinguishing near-identical things. Which leads straight to the next step.</p>
<h2>Step 4: retrieval, and why pure vector search disappoints</h2>
<p>Ask a vector index "what is the SKU for the 40mm variant" and it will happily return the chunk about the 50mm variant. The two sentences are nearly identical in meaning-space, and the one number that matters contributes almost nothing to the vector.</p>
<p>The standard fix is <strong>hybrid search</strong>: run a keyword search (BM25, Postgres full-text, SQLite FTS5 — whatever you already have) alongside the vector search and combine the rankings. Keyword search is precisely the tool that cares about the literal token <code>40mm</code>. Reciprocal rank fusion is the usual combiner and is about six lines:</p>
<pre><code class="language-js">// Combine two ranked lists of ids. k = 60 comes from the original RRF paper
// and is a sane default: it flattens the influence of deep ranks.
function fuse(vectorHits, keywordHits, k = 60) {
  const scores = new Map();
  for (const list of [vectorHits, keywordHits]) {
    list.forEach((id, i) =&gt; {
      scores.set(id, (scores.get(id) ?? 0) + 1 / (k + i + 1));
    });
  }
  return [...scores.entries()]
    .sort((a, b) =&gt; b[1] - a[1])
    .map(([id]) =&gt; id);
}
</code></pre>
<p>Two more retrieval details that matter more than people expect.</p>
<p><strong>Retrieve more than you keep.</strong> Pull twenty candidates, re-rank them, keep the best four to six. A cross-encoder re-ranker reads the question and the chunk <em>together</em> instead of comparing two independent vectors, so it is much better at exactly the near-duplicate problem above.</p>
<p><strong>Have a floor.</strong> If the best match scores below a threshold, the honest answer is "I don't have that in my content." Without a floor your bot is always handed <em>something</em>, and a model handed irrelevant text will dutifully build an answer out of it. Most hallucinations I have chased down were really retrieval failures that nobody blocked.</p>
<h2>Step 5: the prompt</h2>
<p>By now this part is almost boring, which is the goal. The retrieved chunks go into the prompt with instructions that do three jobs:</p>
<ol>
<li><strong>Scope it.</strong> Answer from the provided content only, and say so when the content does not cover the question.</li>
<li><strong>Make it citable.</strong> Number the chunks and ask for the number alongside each claim. You can then show a source link in the UI, and — more usefully — <em>grade</em> answers later, because an answer whose cited chunk does not contain the claim is a detectable failure.</li>
<li><strong>Fix the register.</strong> Length, language, and what to do at the end of an answer. This is where "reply in the language the customer wrote in" lives.</li>
</ol>
<p>What does not belong in the prompt is your knowledge. Every fact hardcoded into a system prompt is a fact that will be wrong after the next price change and that nobody will remember to update.</p>
<h2>How to tell whether any of it works</h2>
<p>Write down thirty real questions — pull them from your inbox, not your imagination — and the page that answers each one. Then measure one number: <strong>how often the correct page appears among the retrieved chunks.</strong> That is retrieval recall, and it is the ceiling on everything else. If the right page is not retrieved 95% of the time or better, a smarter model will not save you.</p>
<p>Only once that number is healthy does it make sense to argue about answer quality.</p>
<p>And keep the log of questions where retrieval found nothing. That log is the most useful document the whole system produces: a list, written by your actual customers, of the pages your website is missing.</p>
<hr />
<p><em>I build</em> <a href="https://chatnory.com"><em>ChatNory</em></a><em>, an AI chatbot that answers from a business's own content on its website, WhatsApp and Telegram — so I have had plenty of chances to get all of the above wrong first. Curious where other people landed on chunk size and re-ranking; it is the part where everyone's numbers seem to differ.</em></p>
]]></content:encoded></item><item><title><![CDATA[How to Add an AI Chatbot Trained on Your Own Content to a WordPress Site (No Code)]]></title><description><![CDATA[Train a WordPress chatbot on your pages, PDFs and price lists — with multilingual replies, human handover, messaging apps, and a clear privacy check.]]></description><link>https://chatnory.hashnode.dev/how-to-add-an-ai-chatbot-trained-on-your-own-content-to-a-wordpress-site-no-code</link><guid isPermaLink="true">https://chatnory.hashnode.dev/how-to-add-an-ai-chatbot-trained-on-your-own-content-to-a-wordpress-site-no-code</guid><category><![CDATA[AI]]></category><category><![CDATA[WordPress]]></category><category><![CDATA[chatbot]]></category><category><![CDATA[Web Development]]></category><dc:creator><![CDATA[D Raju]]></dc:creator><pubDate>Tue, 10 Nov 2026 18:43:43 GMT</pubDate><content:encoded><![CDATA[<p>A scripted chatbot is a flowchart wearing a speech bubble. It works until a customer asks something the tree does not contain — a shipping exception, a comparison, a policy buried in a PDF. Then the bot loops, dumps "contact us," or invents an answer.</p>
<p>A RAG-style chatbot works the other way around. You give it your existing material. When a visitor asks a question, the system retrieves the relevant passages and writes a reply from them. It can answer questions you never wrote a flow for, and it stays current as you publish. If the delivery policy is in the knowledge, it can handle "do you deliver to Salt Lake on Sunday if I order after 6?" You update the source instead of redrawing a tree.</p>
<p>The trade-off is honesty. A scripted bot fails loudly, because it has no path. A generated bot can fail quietly if it is allowed to guess. Prefer a tool that says it does not know, and offers a person, over one that always sounds confident.</p>
<h2>What to feed it</h2>
<p>Answers are only as good as the sources. Feed the bot what a good support person would open:</p>
<ul>
<li><strong>Published pages and posts</strong> — about, shipping, returns, services, and booking instructions.</li>
<li><strong>Product data</strong> — names, prices, stock status, SKUs and categories.</li>
<li><strong>PDFs, office files and a clean price list</strong> — rate cards, menus, policy handbooks, a spreadsheet or catalogue.</li>
<li><strong>Past questions</strong> — old chats or a short Q&amp;A list, in the phrasing people actually use.</li>
</ul>
<p>Leave out drafts, internal cost sheets, unpublished discounts, staff records, and other people's personal data. If a document is half-public, split it first. A plugin that syncs published WordPress content is worth more than a one-off paste.</p>
<h2>Languages your visitors actually type</h2>
<p>People ask in the language they think in — Bangla, English, Hindi, Arabic — and they expect the reply in the same one. A bot that only answers in English will miss a large share of those questions.</p>
<p>Look for a bot that replies in the language the visitor typed, rather than a separate tree per locale. <a href="https://chatnory.com/">ChatNory</a>, used in the walkthrough below, does that. It supports 11 languages, including Bangla, English, Hindi and Arabic.</p>
<p>Feed it sources in the languages you care about, or at least a clear canonical policy. A bot cannot invent your Sunday delivery rule in Bangla if that rule exists nowhere in the knowledge.</p>
<h2>Human handover, WhatsApp, Messenger and Telegram</h2>
<p>Some questions should never be left to a model: a complaint, a payment dispute, or anything the bot flagged as unknown. The useful pattern is live handover. Your team can step in; when a person starts typing, the bot steps aside so the visitor is not answering two voices at once.</p>
<p>Many customers never open your site widget. They already message you on WhatsApp, Messenger or Telegram. One chatbot, trained once, should answer in those channels. The WordPress plugin keeps site content in sync; the apps are connected in the product dashboard, not in wp-admin.</p>
<p>Check the plan before you promise this. On ChatNory, live chat and human handoff are on the Starter plan ($10 per month) and above. WhatsApp, Messenger and Telegram are on the Pro plan ($35 per month). Connect only channels you actually monitor.</p>
<h2>Privacy, before you paste a site key</h2>
<p>A chatbot that "knows your business" stores your business, and your visitors' questions, on someone else's server. Read the plugin disclosure and the product's privacy policy before you connect a site.</p>
<p>The ChatNory plugin sends the title, URL and plain text of published pages, posts and WooCommerce products (including price, stock, SKU and categories), plus your site address. Draft, private and password-protected content is not sent. Visitor messages and the page they are on go to the host. Nothing moves until you paste a server URL and site key.</p>
<p>Do not upload customer lists, invoices with personal data, or internal documents as training material. If visitors chat on your site, you are the controller of that conversation data. Use a host that lets you export or delete it, and mention the chat in your own privacy policy. Developers can limit the sync with the <code>chatnory_post_types</code> and <code>chatnory_sync_limit</code> filters if a custom post type should stay off the record.</p>
<h2>Step by step: WordPress, no code</h2>
<p>This uses ChatNory because it has a free WordPress plugin, a free plan, and it is built around training on your own content. The same checklist works with another host; only the setting names change.</p>
<p><strong>1. Install the plugin.</strong> In wp-admin go to Plugins → Add Plugin, search for ChatNory, or install <a href="https://wordpress.org/plugins/chatnory/">ChatNory – AI Chatbot, Live Chat &amp; Customer Support</a> from WordPress.org. Activate it (WordPress 5.8+, PHP 7.4+). You also need a ChatNory account, because the bot runs on a hosted model, not on shared hosting. The free plan is enough to try this.</p>
<p><strong>2. Create the chatbot.</strong> Sign up at <a href="https://chatnory.com/">chatnory.com</a>, create a chatbot, and give it a role and tone. Add sources the plugin will not sync: PDFs, a price list, Word or Excel files, or a short Q&amp;A.</p>
<p><strong>3. Connect the site.</strong> In the ChatNory dashboard open Settings → General → WordPress plugin and copy the Server URL and Site key. In WordPress go to Settings → ChatNory, paste both values, and click "Save &amp; connect."</p>
<p><strong>4. Ask it something awkward.</strong> The plugin trains the bot on published pages, posts and, if you use it, WooCommerce products. In a private window, ask what a real customer would ask, including one question you know is not documented. The right behaviour is "I don't know" plus a path to a human, not a guess.</p>
<p><strong>5. Choose how it appears.</strong> Placement, devices and published state are set in the ChatNory dashboard, not by editing your theme. To embed the chat inside a page, use the shortcode [chatnory]. Optional attributes are <code>height</code> (default 560) and <code>width</code> (default 100%).</p>
<p>There is a free plan; paid plans start at Starter for $10 per month and Pro for $35 per month.</p>
<h2>If ChatNory is not the right fit</h2>
<p><strong>Tidio</strong> is a strong choice if you want a familiar live-chat widget first, with automation and an AI layer on top, and you are happy in a large, established product.</p>
<p><strong>Crisp</strong> is a shared inbox that also does chatbots. If your team already runs support from Crisp, adding a bot there is usually better than introducing a second inbox.</p>
<p>If the problem is "the bot should answer from our pages, PDFs and price list, in the language the customer typed," a content-trained tool with a WordPress sync is the shorter path. If the problem is "five agents need one shared inbox tomorrow," start with the helpdesk you will actually live in.</p>
<p>Before you publish, train the bot only on public content, test a question it cannot answer, and match handover and channels to your plan. Keep a human in the loop, and do not let the model bluff.</p>
<p><em>I'm D. Raju, founder of ChatNory, based in Kolkata.</em></p>
]]></content:encoded></item></channel></rss>