Conversational artificial intelligence and machine learning is more than a fantasy now. Among several cutting edge practical AI offerings, conversational AI technology is evolving at a rapid pace to support modern enterprise demands for intelligently managing consumer and employee communications. In the world of conversational AI, chatbots are more preferred than ever before, but the fact remains that acquiring cognitive chatbots is still cumbersome, due to several factors like limited AI- and machine learning-ready data, challenges in natural language understanding and vendor qualification, complexity in the platform selection process, limited engineering and development support, bottlenecks with the traditional operational model, and enforced ethics, among others.
To become successful, choosing a proper platform is pivotal for ensuring intelligent, scalable, and cost-effective chatbot technology. Decision-makers are looking for an easy buying process with hassle-free configuration, training, and deployment to harvest ROI as quickly as possible with minimal effort and expense, so they are compelled to explore multiple options. The common choices are one of two options, buying or building the chatbot.
To understand the real difference between the buy and build approaches, we need to start from:
- What is a chatbot?
- What is the confidence and accuracy of a chatbot?
A chatbot is an independent computer software program based on natural language understanding that is designed to simulate human-like conversations with users via text, voice, or image inputs. Machine learning (ML) and Natural language processing (NLP) are the most commonly used AI subsets for "AI-powered" chatbots, but in contrast, "rule-based" chatbots are also playing a significant role in the market.
Under conversational AI, a chatbot with preconfigured actions and pre-written keywords used to understand user queries and to perform expected actions is called a "rule-based" or "decision tree" chatbot. Such a chatbot platform has limited dialog flows but still delivers predictable results within its limits.
Rule-based chatbots can be developed and trained rapidly to work with limited enterprise data and can also be integrated seamlessly with enterprise systems. Their dialog flows can also be configured, streamlined, and managed with less cost and effort. There is a similar chatbot called "retrieval-based" chatbot in the market, instead of Decision-Tree, it uses a repository of predefined responses and a ranking model to pick an appropriate response for the user input.
Coming to the "AI-Powered" chatbot paradigm, with the help of machine learning and NLP, these chatbots can read, understand, and process user queries like a human without any pre-loaded instructions. It performs skills such as "utterances", "intents", "entities", and "variations". NLP and ML are used to bring this sophisticated cousin.
Utterance: The entire statement from the user
Intent: The user's intention (verb in the statement)
Entity: It modifies an intent (noun in the statement)
Variations: The different ways of framing the statement
AI-powered chatbots will consume a long time to build and train but can bring massive savings in the long run. Such chatbots can deliver a rich user experience by establishing human-like conversations. They can learn from experience and continuously improve over sample data training. Like a human, they can make some self-decisions through supervised, semi-supervised, or unsupervised learnings. Most importantly, they can detect false positives and false negatives in their conversations to detect potential misbehavior.
Now let's see a real-time example for a chatbot's confidence and accuracy. A user, who is a sports enthusiast, bought a personal chatbot to catch up on the sports trends and match forecasts. This chatbot was trained with extensive sample data and configured for a unique personality which is opted by the user. Also, this chatbot is connected with several real-time data sources like analyst reports, user blogs, and sports news websites to generate custom predictions.
In general, a chatbot's accuracy could be as high as 99%, but anything less than 40% is low confidence. In the above example, both accuracy and confidence levels are high. For the same example, the chatbot would have responded something like below, if it has a low confidence score but a high accuracy rate.
Likewise, in case of high confidence score and low accuracy rate:
The confidence score is a threshold indicator, where the threshold determines the highest and lowest matching scores acceptable to trigger the respective actions. If the matching score falls below the threshold, then the chatbot will either provide a low confidence response or it will trigger a fallback interaction that either asks the user to repeat the query or transfers the query to human response teams for support. In case of a high confidence score, the chatbot will trigger the actions to produce the desired results.
The confidence threshold can be dragged up and down based on the availability of AI-ready sample data. Importantly, it is not necessary to have a high threshold in all scenarios. A low confidence score value is recommended only in a very limited number of use case scenarios. An inappropriate confidence threshold will bring a negative impact and leads to chatbot misbehavior.
Chatbot accuracy can be misinterpreted easily. Even though accuracy is a key driver, this alone will not create any impact. Accuracy of a chatbot can be defined as the % of utterances that had the correct "intent" returned for the identified actions. There are several cross-validation methods to test and validate chatbot accuracy, some of which are K-Fold, Monte Carlo, and 80–20% split, among others. A higher accuracy rate will bring better end results which increases customer satisfaction.
Let’s gear up with basic questions
Let's find the "Silver Bullet"!
Decision-makers who are ready to take the risk with the combination of an in-house AI engineering team, a well-defined strategy/roadmap, and successful track records in AI deployments in the same enterprise, may get the subscription directly from platform providers to build their chatbots. Some of the top providers are:
- Google - "DialogFlow"
- IBM - "Watson Conversation Services"
- Amazon - "Lex"
- Microsoft - "LUIS"
There are other types of platform providers in the market who offer application-specific platforms. The downside with them is they may go well only with their native applications, so the decision-makers who decide to go with such platforms, even after large investments, they need to invest extra effort and money to scale up their chatbot as an enterprise-wide offering or for cross-domain culture. A few examples include:
- Salesforce - "Einstein"
- SAP - "SAP Conversational AI"
- ServiceNow - "Conversational bot platform"
Typically, in cases that involve platforms such as Einstein and SAP Conversational AI, the chatbot build process is a time-consuming activity to achieve success. In order to simplify the process, the decision-makers with build strategy can consider the following factors:
Apart from a chatbot’s confidence and accuracy, there are other factors to consider while building a chatbot:
- Performance load
- Response speed
- Error management
- User feedback for chatbot optimization
Last but not least, the decision-makers with a buy strategy can play a different game which is more facile and foolproof. There are some platform providers in the market that offer their productized chatbot solution as a holistic product with professional services. These companies took most of the complex tasks to come up with a viable chatbot platform for the buyers.
- Out of the Box Use Cases: Like the way, in which an employee's productivity is dependent on their knowledge and learning, the productivity of the chatbot is dependent on error-free configuration, which comes from ready-to-use and pre-trained and tested use cases. A user-friendly portal with adaptors will support hassle-free new use cases configurations as well as training.
- Natural Language Understanding: How effectively can the chatbot understand user queries? Basically, this skill comes from the underlying NLP engine for "intents", "entities", "utterances", and "variations". A famous feature is the FAQ parser, where the chatbot is simply connected with any knowledge source system to answer the user's questions.
- Deployment: How effectively can the chatbot be deployed in the enterprise? Ideal time to market (TTM) with meaningful milestones can thrive well. Apart from effective dialog flow configuration with optimal performance load, precision, response speed, error management, and user feedback collection, the deployment phase contains four primary tasks: Integration, testing, information security, and channel.
- Chatbot Integration: Integration with external and in-house enterprise applications should be done using off-the-shelf integration adaptors. This skill comes from the number of robust tools and adaptors for the end to end chatbot integrations.
- Testing: Extensive dialog flow testing with the right sample data and user acceptance testing in the production environment with a valid "service ID" for the chatbot. The service ID should have required read/write access like a real employee ID.
- Security: In general, security and data protection itself is a vast area. In terms of AI, where machines take control of human tasks to serve the end user, security policies must be validated completely in all segments such as integration, information transfer, data storage, and data extraction, among others. Extra care should be taken for cloud deployments, integrating chatbots from cloud to on-premise systems should be achieved after DMZ assessment to bring the right middle layer solutions, like middleware servers, VPN tunnels, and secure gateways, among others.
- Chatbot Channels: Nowadays, advanced chatbots support multiple channels. The enterprise needs to consider the end user's comfort zone for selecting the channel. Some popular channels are:
- Desktop applications (Example: web browser and email, among others)
- A dedicated mobile and desktop app
- Voice device (Example: Alexa, Google Home, and Siri, among others)
- IVR (Example: telephone and voice, among others)
- Chat Engine (Example: MS Team and Jabber, among others)
- Social Media (Example: Facebook messenger and WhatsApp, among others)
- Pilot: A pilot run for a short time period will help the enterprise to test and measure the real-time performance of the chatbot in a production environment.
- Pricing: The most common pricing models in the chatbot market-
- Monthly license fee: Monthly cost based on limited or unlimited usage
- Pay-per-use: Cost based on usage and number of application integration
- Pay-per-performance: Cost based on successful conversations
Finally, what is the big fish?
Obtaining an enterprise-wide successful chatbot is a rigorous task for most AI experts and decision-makers. So, the leaders who are not in a rush and are prepared for all unexpected surprises can build their own chatbot. The aim is to lay out a clear vision plan and stick to that with patience. Leaders with a buy strategy will experience low or no risk in this journey, but they need to evaluate the KPIs to qualify the right provider for their chatbot needs. Either with the build or buy strategy, leaders should focus on the factors discussed above before making the decision.