Building your first Qlik Sense ChatBot using the Natural Language API

Qlik Sense’s self-service visual analytics platform has been compelling in processing and analyzing complex datasets to derive hidden patterns from the data and helping end-users make faster decisions by presenting interactive visualizations. To add to its charisma, Qlik incorporated Conversational Analytics, the “Insight Advisor” — an AI-powered chatbot platform that provides a faster mechanism for users to ask questions and help them discover insights using Natural language Processing(NLP).

By blending the robustness of Qlik’s Associative and Cognitive Engine, the Insight Advisor assistant instantly generates relevant answers in the form of narrative texts, visualization charts and recommendations to help users with any level of expertise to maximize their potential when deriving insights. The flexibility to switch between visual and conversational analytics in Qlik Sense seamlessly & without losing context bridges the gap that is often seen with traditional Business Intelligence tools.

Ref: https://www.qlik.com/us/products/qlik-sense/ai

So, what are we doing new now?

We are bringing the potential of our Cognitive Engine that powers the ‘Insight Advisor’ to the hands of the Developer community with the launch of our new — Natural Language API.

Potential Benefits?

1. Orgs starting out with new automation tools: Develop Qlik Chatbots specific to your company(customized UI) & interact with our cognitive and associative engines.
2. Orgs already using Chatbots: Integrate the NL API to the existing company chatbots/communication platforms(Slack, Teams) and drive the analytical questions through Qlik Sense.

Before we delve into the steps and technology stack to build up our first Qlik Chatbot, let me give you a brief idea about a couple of things running behind the scenes of this API.

Various components of ChatBot

Generally speaking, Natural Language Processing(NLP) comprises of two essential sub-components — Natural Language Understanding(NLU) and Natural Language Generation(NLG) that help in interpreting and generating human language.

As the name suggests, NLU is responsible for comprehending and transforming any unstructured data into a structured form that the machine can understand. This is particularly important when it comes to ambiguous texts, for example, texts that are similar but have different meanings and changes with respect to the context. NLG, on the other hand, generates natural language in a human-understandable format based on the machine’s response.

So how do these components work together in a ChatBot?

Well, like I discussed, the intent of a sentence is first deciphered by the NLU and then the NLG analyzes the data and a response in plain-text is provided back. NLP basically takes the role of an engine for the chatbot that helps in this process of understanding and fetching a response for the user.

Alright, let’s visually understand these things from Qlik’s NL API perspective now.

Qlik’s Natural Language API components working mechanism

As we see, the NLU first reads the sentence — “give me the sales” and then tries to understand the intent and what entity it is. In this case, the entity type is a ‘master_measure’ which aligns with what I have in my Qlik Sense app. The NLG then generates the response in the form of a conversationalResponse property that comprises of the responses type, which in this case is a narrative and the corresponding text “Sales is 23.89M”. This text is returned to the user.

Now that we have an understanding of the things running under the hood, let’s understand a user scenario, explore the API endpoint, and start building our first Qlik chatbot.

User scenario: A customer wants to build a new Embedded analytics solution that brings in the capabilities of a Qlik Sense Mashup(visualizations from various apps) and would also like to develop and embed a Qlik ChatBot on that same portal. Ultimately what the company wants to achieve is a balance between visual analytics & conversational analytics to allow any level of user(in terms of data literacy) to take full advantage of Qlik’s analytical platform.

Prerequisites for developing:

1. Register for a subscription on Qlik Sense SaaS.
2. Create a new web integration from the Management console.
3. Create an API key.
4. Enable apps for Insight Advisor chat.

Step 1: Start a new Node project —Our aim is to embed the Qlik Chatbot as part of a Mashup and the easiest way to get started building a mashup using Nebula.js is by using the following command:

npx @nebula.js/cli create mashup hello-saas

The command scaffolds a web project into the hello-saas folder with the following structure:

/src

• configure.js - Initial configuration of nebula.js
• connect.js - Connection setup with enigma.js
• index.html - A minimal html page
• index.js - Connect and visualize

/src/index.js is where you need to make modifications later on to bring everything together.

Step 2: Define the UI for the ChatBot — For this specific tutorial, we will keep the chatbot’s interface simple since our focus is to be able to communicate with the cognitive engine and get responses. So we design the chatbot as a popup form and embed it at the bottom of the mashup page using the following HTML code.

The result is below:

QS Mashup page with the chatbot embedded

Step 3: API and connection — Our final step is to use the API to communicate with the cognitive engine. To do that, we will be leveraging the /src/index.js file. But first, let’s take a look at the exposed public endpoint and the nitty-gritty in the request-response process.

API endpoint: '/api/v1/questions/actions/ask'

REQUEST: We first need to read the text that is being typed in the chatbot to be passed as the body of the POST request. To do that, we do as shown below. Now, the variable message will store the input text.

To initiate the POST request, we need to pass a few important parameters as shown in the code snippet below.

Let us take a closer look at these parameters :

1. The data is basically the Body of the POST request, and the message variable(where we store the input text) is assigned to the “text” property to pass whatever is typed in our chatbot. Also, since in this case, we just deal with one particular Qlik Sense app(not cross-apps), we pass the “id” and “name” of the app.
2. The next important parameter is the options. Here we specify the communication arguments such as hostname, port, API path, method type, and Qlik-specific qlik-web-integration-id, and API Key(authorization) as part of the Headers.

RESPONSE: Finally, we parse the JSON response and use it in a way that we would want it to be rendered back to the chatbot. Typically, the responses are of three types and have some important properties that can be returned to the chatbot.

• narrative — “text”: contains the textual description generated by the NLG
• chart — “imageUrl”: gives the path of the associated visualization chart
• info — “infoValues”/ “recId”: gives the ID of the recommendations based on the input text

For this user scenario, we focus on ‘narrative’ text and ‘chart’ ; we will send these two kinds of responses to the chatbot interaction in the UI.

How do we do that?

The logic behind the above implementation is as simple as it gets. Based on the input query, I check whether the engine’s response is just a narrative text or has an associated visualization with it. Depending on that, I send the response back to the front-end of the chatbot.

And here’s the first Qlik chatbot in action:

Qlik ChatBot in a Mashup

This brings us to the end of this exciting tutorial on developing a customized chatbot using the new Qlik Natural Language API. The post aims to serve as a boilerplate for the future development of chatbots using the API. I have also discussed the potential benefits of leveraging the Natural Language API and a hypothetical user scenario to which this can be applied.

The source code for the chatbot can be found at: https://github.com/dipankarqlik/NLP_API_Qlik