Optimism ChatBot

Issues

• High Volume of Inquiries: The Web team was overwhelmed by the sheer number of questions, consuming significant man-hours.
• UX Risk: Inability to respond quickly to user inquiries posed a risk of damaging the user experience.

Solution

Introduction of a ChatBot to reduce the number of inquiries and create breathing room for human resources.

• Period: Approx. 2 weeks (July 26 - Aug 4)
• Cost: Approx. 2 million JPY

a. Reducing Inquiry Man-hours

• Goal: Reduce inquiries from 123 (previous year) to under 61 (50% reduction).
• Result: Inquiries dropped to 41, a 67% reduction year-over-year.
• Contribution: Contributed to resolving user issues by standing in the user's shoes and comprehensively covering expected questions. Accuracy improved from ~15% at the first QA to 81% by the start of Optimism.

b. Improving Answer Success Rate

• Goal: Achieve over 80% accuracy to prevent user stress.
• Result: Achieved 81% accuracy, clearing the goal and responding quickly/accurately to user doubts.
• Contribution: Created AI-friendly training data by understanding the backend structure, resulting in a high-precision ChatBot.

c. Pre-event Launch

• Goal: Release the ChatBot 1 week before the event.
• Result: Released exactly 1 week prior.
• Contribution: Set clear deadlines, listed tasks, and collaborated with CXD to release in just ~3 weeks. Continued to refine accuracy with minor updates. (▶︎ Requirements Definition & Schedule)

d. Enhancing User Experience

• Goal: Provide a system for immediate resolution to improve pre-event UX.
• Result: 2,038 users asked 4,794 questions. Handled detailed event-specific questions (e.g., parking availability, strollers), significantly boosting UX. (▶︎ Effect Estimation Sheet)
• Contribution: Involved the entire OPTIMISM team to gather feedback on usability and pain points. Reflected this in training data to improve accuracy from multiple perspectives. (▶︎ Feedback Note)

Furthermore, the infrastructure built this year allows for reuse in future Optimism events, promising sustainable efficiency.

a. Viber Integration for Email Reminders

• Goal: Prevent overlooked emails and improve efficiency.
• Result: Auto-notifications to Viber allowed easy mobile checks, preventing missed inquiries.
• Contribution: Built a flow for confirmation and reply using Viber, enabling rapid responses. Handled simple replies personally.

b. Auto-drafting Replies with ChatGPT API

• Goal: Use AI to draft replies for efficiency and better UX.
• Result: Enabled quick creation of polite, error-free responses, improving quality.
• Contribution: Built a system using Power Automate, ChatGPT API, and Viber API where AI proposes reply drafts.

c. Archiving Inquiry Records

• Goal: Record all inquiries in Excel for future reference.
• Result: Auto-recording provided samples for next year and reference data for ChatBot training.
• Contribution: Added functionality to auto-save inquiry contents to Excel using Power Automate.