Work/ 01
AI for Customer Service
@RukitaAI answers tenants' checkout and deposit questions and hands everything else to a person.
- ROLE
- Product Manager
- SCOPE
- Prompt architecture, escalation rules, flow design, testing
- WORKED WITH
- Engineering, Customer Support and UI/UX
- YEAR
- 2025 to 2026
Problem
Tenants asked about checkout and deposit rules all day, and the same questions kept coming. A tenant messaged on WhatsApp, the message waited in a queue, and CS read it and replied.
The cost was in the waiting: queue time, agent time and customer satisfaction.
Why AI
A fixed menu or static Q&A could have answered most of these questions. I chose AI for what happens after launch. Policies change, and each change would mean rebuilding a menu.
With the AI reading one maintained source of truth, an SOP change becomes a document update. The goal was easier maintenance and less CS operations time.
What I did and the approach
The AI had to work for two audiences at once: tenants in the app, who want an answer now, and CS in the dashboard, who need context fast and must know when to step in. One prompt logic served both.
I designed the prompt architecture, the escalation rules and the conversation flow, and I ran the testing. Tenants pick a topic and the booking it's about before they chat, so the AI starts from the right order. It confirms the booking and date first, then explains what applies. When a tenant shares something hard, like a death in the family, it responds with care before any rule. When a question goes to a human, CS gets the full conversation plus a short AI summary of the main issue, and the AI stops replying the moment an agent takes over, so the tenant never hears two voices.
I mapped every real case type with the CS team, then set which ones the AI answers, which go straight to a person, and the conditions at each boundary.
Model: a small, low-cost model, chosen on engineering's recommendation for cost and reliability. I tested reasoning settings. A higher setting added about 48 to 50 seconds of latency and a new bug, so we locked the lowest.
Built and shipped in about a month, then rolled out to 100% over the next four weeks:
- 20%soft launch, one entry point
- 50 to 70%
- 100%of in-scope checkout chats
What I built
before
- Tenant messages on WhatsApp
- Waits in queue40 mins++!!!
- CS reads
- CS replies
after
- Tenant picks a topic and booking
- Asks in in-app chat
- AI confirms the booking, then answers from that policyreplies in <1 min
- Complaint or unclear question
- CS gets the ticket labelled with topic and priority, plus an AI summary, in the internal dashboard
- CS repliesfewer tickets reach CS
The call I made
I drew two lines.
First, every complaint goes to a human ("Why is this late?", "Why is my deposit different?"), and the AI never promises an exception: disputes, exceptions and data that doesn't match go to a person. These need judgment, and a wrong answer makes an upset tenant angrier.
Second, the prompt doesn't do the math. Checkout dates and what they mean for the deposit stayed in engineering's logic, where it can be tested like code.
What could break, and how I tested it
A confident wrong answer that turns a calm tenant angry.
Complaint detection routes to a human, and the AI escalates when unsure.
Outdated policy.
When a rule changed, we re-ran only the exact chat bubbles that had gone wrong, on a testing platform, until the answer came back right.
A missed handoff.
Real conversations after launch showed cases the AI kept instead of handing off. I fixed the escalation rule and kept tuning on live chats.
Not knowing whose answer worked.
Every ticket is labelled AI only, AI and agent, or agent only, and AI chats get their own survey, so the two can be compared fairly.
Result
fewer tickets for CS → replies in under a minute → happier tenants
CSAT on tickets where AI and CS worked together ran 4.64 to 4.83 from Feb to May 2026, against 4.52 to 4.61 before the AI.
About 7% of tickets answered the survey. Fully AI-resolved tickets had too few responses to read, so I don't report a score for them.
What I'd change
Test cases before launch, not after.
The clearest evaluation came in June, months into production. A small expected-answer test set at launch would have shown the biggest problem on day one.
Hard-gate money and policy answers behind confirmed order context.
Most of the wrong answers traced back to the AI not reliably knowing the tenant's real order status. Prompt polish sat on top of a weak foundation.
Measure resolution, not only coverage.
I had clean data on volume and speed and thin data on quality. A re-open rate panel and auto-resolve should have been launch requirements.