KPI dictionary
Define before you dashboard
An average without distribution can hide a long outage. Also request median, percentiles, denominator and outlier list. Segment by website, brand, language, channel, team, agent, tag, bot and period.
Automation needs its own view. Bot resolution, bot handoff, human first response and duplicate replies must not disappear into an apparently strong overall time.
| Metric | Defensible definition | Common distortion |
|---|---|---|
| New conversations | unique start inside the period | reopen or late reply becomes a new chat |
| First response | first customer message to first public human reply | bot, auto-reply, queue or closed hours differ |
| Average reply | each customer message to next public reply | internal notes or parallel agents interfere |
| Resolution | opening to an explicit resolved state | a Teams thread may have no resolution state |
| Missed/unanswered | no human reply within a defined threshold | Tidio documents a three-hour live-conversation rule |
| CSAT | positive valid responses divided by valid responses plus response rate | only resolved or self-selected cases answer |
| SLA | target, calendar, business hours and breach event | pauses, reopens and priorities vary |
Product evidence
Teams activity and service reporting answer different questions
| Model | Documented | Boundary or pilot question |
|---|---|---|
| WebChat in Teams | channel/thread workflow; Teams posts and replies usage | no evidenced service-KPI definitions or exports in reviewed WebChat sources |
| Chatwoot | realtime, conversation/agent/inbox/team/label, CSAT, SLA, bot and CSV | assignee at resolution, business hours, plan/edition and no native email scheduler |
| Tidio | project analytics, new/replied/missed chats, FRT, CSAT, duration, agent and export | three-hour rule, flow exclusions and project/plan scope |
| LiveChat | volume, satisfaction, goals, response, duration, staffing, agent and CSV/raw data | CSV from Team and raw data from Business/Enterprise per source; exclude legacy tickets |
Export and privacy
Raw data creates auditability and privacy obligations
A downloadable report is not automatically a complete data export. Check whether it contains charts only or conversation-level records, and whether deleted, merged, reopened and bot-handled cases remain explainable.
Averages alone are unsafe for workforce decisions. Complexity, language, shift, utilisation, handoffs and case mix must accompany the metric; reporting supports diagnosis and capacity planning, not blind ranking.
- Require a data dictionary with field, type, timezone, event source, agent/bot marker and deletion logic.
- Test CSV, API, webhook or BI path for pagination, limits, freshness, historical corrections and schema changes.
- Scope report access by team, channel, brand and role; employee rankings require purpose and context review.
- Minimise or pseudonymise names, email, IP, message content and free text in analytics exports.
- Align retention and deletion across widget vendor, Teams, reporting tool, data lake, BI and local files.
- Publish every figure with period, timezone, filter, definition, data timestamp and accountable owner.
Measurement pilot
A controlled dataset exposes definition differences
- 01
Fix clock and scope
Choose one timezone, business-hours calendar, website, channel and named test agents.
- 02
Send eight cases
Create instant reply, after-hours, bot handoff, missed, reopened, transferred, parallel-reply and deletion cases.
- 03
Record events
Externally log visitor start, bot event, public agent reply, internal note, handoff, resolution and reopen.
- 04
Recalculate dashboard
Check counts, FRT, reply, wait, resolution, missed and SLA against the control log.
- 05
Test filters
Apply website, channel, brand, language, agent, team, tag, bot, date and business hours separately.
- 06
Inspect export
Pull CSV/raw/API data and check timezone, IDs, pagination, personal fields and totals.
- 07
Test correction
Reopen, reassign and delete; observe when dashboard and export update.
- 08
Sign off
Approve formula, screenshot, raw sample, plan, rights, retention, owner and known deviations.