Polaris
Jiun

What to Measure to Know Your WhatsApp Chatbot Is Working

Six numbers tell you whether your WhatsApp chatbot is earning its place. If you are only looking at message volume, you are watching activity, not results.

This guide covers the six KPIs that matter for an SME owner: first response time, deflection rate, leads captured, appointments booked, after-hours coverage, and customer satisfaction. For each one, here is what it means, why it matters, and a rough sense of what healthy looks like.

What is first response time and why does it decide your conversion rate?

First response time is how long a customer waits between sending their first message and getting a reply. It is one of the highest-leverage numbers in your business.

Research published in Harvard Business Review found that leads contacted within 5 minutes are 21 times more likely to qualify than leads contacted after 30 minutes. That gap is enormous. A customer asking about your service at 10 pm on a Tuesday is not going to wait until 9 am the next morning.

A bot that replies in under 5 seconds holds the conversation open while the customer is still paying attention. Polaris replies in under 5 seconds, day or night, so the lead does not go cold while you are busy with something else.

Healthy sign: average first response time under 1 minute, including outside business hours. Warning sign: average above 5 minutes, or a sharp spike in response time on weekends and evenings.

See also: how response time affects WhatsApp sales in Malaysia.

What is deflection rate and how do you read it?

Deflection rate is the percentage of conversations the bot resolves on its own, without routing to a human. A high deflection rate means your team is spending time on queries that actually need a person, not on “what are your operating hours.”

To calculate it: take the number of conversations closed by the bot without human involvement, divide by total conversations, and multiply by 100.

A bot handling simple, high-volume questions (pricing, location, hours, basic service info) should be able to handle a meaningful share of incoming chats on its own. The right number depends on your business. A clinic booking bot has a different profile than a legal firm’s intake bot.

Healthy sign: 50 to 70 percent deflection for bots covering repetitive queries. If you are seeing this, your team is freer to handle the conversations that need judgment. Warning sign: deflection below 20 percent often means the knowledge base is thin, or the bot is passing everything to a human too quickly. Deflection above 90 percent may mean the bot is deflecting things it should not be handling alone.

How many leads is the bot actually capturing?

Leads captured is the count of conversations where the bot collected a customer name and contact detail from someone who expressed interest. This is a concrete output, not an input.

Track it weekly. If your bot handles 200 conversations a week and captures 40 names and numbers, your capture rate is 20 percent. Compare that to what your team was capturing manually before the bot existed. If the number is higher, the bot is doing its job. If it is lower, the conversation flow probably does not ask for contact details at the right moment.

Healthy sign: capture rate climbing over the first few weeks as you tune the bot’s questions. Warning sign: capture rate near zero means the bot is answering questions but not moving customers toward action.

For a deeper look at return on investment from a WhatsApp chatbot, read WhatsApp chatbot ROI for Malaysian businesses.

Is the bot booking appointments, or just talking about them?

Appointments booked is the number of actual confirmed slots the bot creates in your calendar. Not conversations about booking. Not clicks on a link. Actual bookings.

This number should be tracked separately from leads captured because a booked appointment is a stronger signal of intent. A Penang dental clinic getting 30 WhatsApp booking requests a day wants to know how many of those the bot confirmed automatically versus how many dropped off because the flow broke or the customer gave up.

Healthy sign: appointments booked per week is growing, and drop-off between “I want to book” and a confirmed slot is low. Warning sign: high conversation volume but near-zero bookings usually means the booking flow is too long, the available slots are not displaying correctly, or the bot is handing off too early and the human handoff is slow.

How well does the bot cover after-hours conversations?

After-hours coverage is the percentage of messages received outside your business hours that still got a reply (from the bot) within a short window. For most SMEs in Malaysia, a large share of WhatsApp messages arrive in the evening, between 8 pm and midnight.

To measure it, filter your conversation logs by time. Count inbound messages outside your operating hours and check whether the bot responded. If your after-hours reply rate is significantly lower than during hours, something is misconfigured.

Healthy sign: the bot replies to 95 percent or more of after-hours messages within 1 minute. Warning sign: after-hours messages going unanswered for hours, or the bot routing to “please contact us during business hours” without collecting the customer’s query or contact detail.

How do you measure whether customers are actually satisfied?

Customer satisfaction (CSAT) for a chatbot is usually collected with a single question sent after the conversation closes. A thumbs-up or a 1-to-5 star rating works well in WhatsApp. Long surveys see very low completion rates in a messaging context.

Your CSAT score tells you something the other metrics do not: whether the bot’s replies are useful, not just fast. A bot that answers in 3 seconds but gives the wrong information will score poorly on CSAT even if it deflects 80 percent of conversations.

Healthy sign: CSAT above 4 out of 5 on routine queries. Below 3.5 suggests the bot is answering incorrectly, being too generic, or not escalating at the right moment. Warning sign: CSAT scores consistently below 3 are a signal to review the knowledge base, not to add more features.

Putting the six KPIs together

You do not need to track everything at once. Start with first response time and deflection rate. Those two tell you whether the bot is covering volume and doing it fast. Add leads captured and appointments booked once the basics are stable. Layer in after-hours coverage and CSAT to catch edge cases and quality problems.

The goal is not a perfect score on any single metric. It is a consistent trend in the right direction over 4 to 8 weeks. If first response time is down, deflection is rising, and CSAT is holding steady, the bot is working.

For a fuller picture of how an AI chatbot fits into an SME’s workflow, see the AI chatbot guide for Malaysian businesses.

Frequently asked questions

What is a good deflection rate for a WhatsApp chatbot?

There is no universal number. A rough starting target for a bot handling simple, repetitive questions is 50 to 70 percent of conversations resolved without a human. If your bot handles complex or emotional queries, a lower rate is expected and acceptable.

How do I measure first response time on WhatsApp?

Most shared inbox tools, including Polaris, log the timestamp of the first inbound message and the first outbound reply. Subtract one from the other. Check your average and your median; the median is often more useful because a few very slow replies can skew the average upward.

What counts as a lead captured by a chatbot?

A lead is captured when the bot collects a name and contact detail (phone number or email) from a person who expressed interest. Track how many of these the bot gathers per week and compare it against your total incoming conversations.

How do I know if my chatbot is covering after-hours well?

Filter your conversation logs by time. Count messages that arrived outside business hours and check whether the bot replied. If the reply rate outside hours is significantly lower than during hours, the bot is not routing or responding correctly after hours.

Should I use a star rating or a longer survey for chatbot CSAT?

A one-question thumbs-up or 1-to-5 star prompt sent immediately after a conversation closes works well for WhatsApp. Longer surveys see much lower completion rates in a messaging app context. Keep it short so you get enough responses to spot trends.

Ready to stop missing customer inquiries?

Let our team set up a managed AI chatbot on your WhatsApp in 24 hours. No tech setup needed from you.