In 2021, a major bank approached Markswebb to evaluate the competitive position of its chatbot and receive recommendations for improvement. The business goal of the team was to enhance the level of process automation within the bank, reduce support costs, increase user activity, and take another step towards developing remote sales through the bot.
To help the bank achieve these goals, we analyzed all strengths and areas for growth and provided 67 recommendations. Fully implementing these recommendations, according to the projected benchmark, should double the client's chatbot rating, allowing it to reach the top 2 market leaders. More details in this case study.
Contents
In 2021, Markswebb conducted its first comprehensive study of the chatbot market in retail, telecom, and banking sectors. As part of the first wave of the Chatbot Rank initiative, we explored how beneficial chatbots are to customers and businesses at their current level of development. We examined the prevailing standards of customer experience with chatbots, systematized best practices, and identified areas for improvement for major companies.
The client's bank chatbot was not part of the initial study, so it was evaluated and integrated into the existing ranking from scratch. The application team had implemented user rating collection at the end of chat interactions, along with a more detailed survey system through WhatsApp, offering more fields for comments. Comparing with the market using Markswebb's insights provided additional value.
We were able to see features present in other chatbots that we hadn't utilized, and consider alternative approaches. The bank's chatbot is one of the tools for transitioning customers to more convenient modern digital channels. It is the first point of contact for customers, making its analysis, along with that of operators, extremely important.
During the UX audit, researchers interacted with the bot and evaluated the quality of handling the 25 most popular customer requests in the following categories:
A new insight for Markswebb researchers was that the client was the only fintech player whose bot successfully handled requests equally well in the app, on the website, and via WhatsApp.
The largest banks in the region have focused primarily on developing virtual assistants within mobile apps, with less emphasis on other channels. In this context, the client's bank stood out: its bot was equally developed across all channels. In the app, it handled requests at an average market level and ranked sixth out of 11.
Dariya, UX Team Lead at Markswebb
Next, we will provide key recommendations and examples of best practices. For convenience, we have divided the features into basic and advanced categories. Basic features are those that are sufficiently developed and present in most market participants. Advanced functionality refers to features that are currently only found among some market leaders. The first section will help you self-assess, while the second will guide you on where to move forward if all the basic features are already implemented.
The readiness to engage with a chatbot largely depends on the user's awareness and confidence that the bot can solve their problem. Asking a question randomly and waiting for an answer wastes time. It is important to introduce users to the available features and tools upon their first entry into the chat.
For example, you could implement widgets with frequent query categories. However, it’s crucial to leave the option for users to type their queries since buttons cannot cover all topics of interest to clients. The ideal user path includes personalized questions based on the client’s experience, products, and services. At the time of research, such an approach was underdeveloped in the market.
It can sometimes be challenging to immediately come up with a clear query. Users often send overly broad or overly detailed questions. These types of queries are the most challenging for the bot to process, which can lead to irrelevant responses or a switch to a human consultant.
Suggestions can help resolve this issue - clickable hints that appear while typing. For example, a user typing in a banking app sees several possible queries predicting their question. These suggestions evolve as the user types, becoming more relevant to the task at hand. Suggestions can be scrollable to avoid occupying too much screen space.
A bot’s ability to resolve client queries within the chat without operator assistance significantly shortens the user journey, saves time, and increases customer satisfaction.
For many services, it is already standard for bots to provide non-personalized information immediately - such as fees for transfers or exchange rates. The virtual assistant could initially only redirect users to sections of the app containing this information. The product team prioritized the recommendation, and the bot learned significantly.
A higher level of difficulty is providing personalized information. Although the average market level is low, leading digital bots can pull data such as credit card debt, grace period end dates, debit card balances, and individual tariff conditions upon request.
Following our recommendations, the product team trained the bot to provide current service conditions instead of a hyperlink to the app section. The development of features like providing debit card balances and answering credit card queries was underway.
A bot's ability to perform actions, not just provide information, reduces user time and bank staff workload. For example, the best implementations in the fintech market can independently generate and send statements in the chat, manage cards, and activate additional services.
By the audit time, active functions were already queued in the product backlog. After reviewing the report, the team reprioritized and rescheduled. By the end of the second quarter, the bot could independently block and unblock cards upon client request.
Emotional complaints - such as "I’m tired of paying for nothing" - are crucial to address as they can be the final communication attempt before severing ties with the brand. A bot's ability to recognize and appropriately handle these complaints fosters a sense of care and a commitment to creating a comfortable product experience.
An excellent example of handling dissatisfaction is found in a telecom operator’s app. In response to a complaint, the bot provides a link to a personalized offers page, where the client can also find standard ways to optimize their expenses. If this doesn’t satisfy the client, the bot smoothly transitions the conversation to a human support specialist.
The product team continues to work with the report’s recommendations - in addition to the features already mentioned, the backlog includes early repayment through the bot, quick replies, issuing statements, and automatically filling out forms with bank data. They also launched a Telegram bot for clients.
For some mechanics, the product team devised alternatives to the report’s references. For instance, proactive user notifications about significant bank changes and onboarding - educational materials for users - are planned to be personalized and displayed before the chat, with topics tailored to clients. New users will see basic information, while experienced users will see only the latest bot functionality updates.
Customer satisfaction is the primary goal of the bot development team. With the implemented changes and improvements, the customer satisfaction index (CSAT) has increased by 10%.
Finding the perfect balance between our product vision, client demand, implementation timeframes, and bank strategy guided task selection for the backlog. Our bot has a six-level classification of topics, each tracked by automation level, complaints, and operator call requests. Automation increased in topics where we provided specific information rather than redirecting clients. Initially, we started at 30-40% automation; now, we’ve reached 60-70%, with some narrow topics up to 85% automated, which is a good level.
This case study demonstrates a strategic approach to building a market-leading chatbot for mobile banking. By partnering with Markswebb, the client successfully increased process automation, reduced support costs, and improved customer satisfaction, positioning itself as a leader in digital banking innovation.
Every year we conduct up to 15 studies of digital services. These are industry benchmarks that reflect the state of the market and trends.