Artificial Intelligence, Machine Learning – these buzzwords are now all over the press, the Internet and social media. In the process of outsourcing everything possible, we are now actively outsourcing human intelligence, replacing it with machines. Oh wait, that sounded wrong. Perhaps we are just trying to make our work easier and more productive, constantly chasing the “better, cheaper, faster” paradigm? Who knows…

So, whatever you call it – Qlik Sense has it! It’s called Insight Advisor, and it’s getting better and better with every new release! Let’s take a look inside.

Insight Advisor is a relatively new feature, and its purpose is to suggest insights automatically. Once it’s activated, it will analyze your data, find good candidates for Dimensions and Measures, and then offer some pretty awesome insights – completely auto-magically!

Getting Started

To activate Insight Advisor (IE), click on the button “Insights” in the upper right corner of your Qlik Sense (Desktop) screen. Notice that the button is located in the Analysis view rather than in the Edit view – the target audience of this feature is the User, not so much the Developer. However, developers are still allowed, and they also might benefit from this feature. Some of the generated insights could be quite interesting even for experienced chart builders.

Once you activate Insight Advisor, you will see a very simple Google-like screen – a list of Fields and Master Items on the left, a Search box on top, and a single “Generate Insights” button in the middle.

For starters, hit the “Generate Insights” button without doing anything else. After a brief “thinking” moment, you should see about a dozen insights offered to you, based on the data in your App.

You should get a Bar chart, a Pivot table, a couple of Maps, a couple of Treemap charts, a Distribution Plot, a Combo Chart, two Line charts and two Scatter Charts. A little bit of everything for you to choose from – a Noah’s Arc of sorts.

Most insights are fairly simple, but a couple caught my attention. For example, a Pivot table that shows the Top Product by Sales for Channel and Brand – that’s something that resembles some of the advanced analytics that I teach at my “Sales Analysis and AGGR” session at the Masters Summit for Qlik – definitely above the pay grade for most beginners!

Making Improvements in Charts

Truth be told, the original generated Insight wasn’t all that smart – it suggested “Top Product by sum(Unit Cost) for Style and Warehouse”:

This is where human intelligence can still celebrate a little victory – every rookie knows that Unit Cost shouldn’t be summarized, and Warehouse is perhaps not the most insightful Dimension to be used. However, these choices will get substantially better in the next version, February 2019. I describe what to expect soon at the end of this article.

For now, we have an opportunity to teach the Cognitive Engine (that’s the actual “brain” behind the Insight Advisor) a thing or two, with a useful feature called “Light Authoring”. We can click on the icon of the Pivot table and the chart will be presented in “full screen” mode, with its Dimensions and Measures listed on the right. With just a few clicks, we can replace any Dimensions with better ones and replace the field and the aggregation function with better choices. Voilà – now the Pivot Table shows the desired analysis. Once you get something you like, you can now add it to one of your Sheets.

Every time you make modifications in the process of Light Authoring, Insight Advisor learns from you. You can ask for certain fields not to be offered again, or ask that certain fields be treated as Dimensions and not as Measures, and vice versa. Experiment on your own and see how far you can take the Insight Advisor learning process!

What About That Search Box?

My first impression was that the Search Box should provide some Natural Language query capabilities, and I started asking questions like “Who is my best Salesperson” or “Annual Sales by Channel”. While the Cognitive Engine produced some results, I quickly learned that this might be a good idea, but Insight Advisor is not there yet. For the time being, we can select any combinations of Fields, Master Dimensions, and Master Measures from the panel on the left, and IA will try its best to offer corresponding visualizations. For example, selecting Master Items like Sales, Brand, and Channel results in the following 3 charts:

The first Table is “right on the money” – just what I’d asked for. If nothing else, this is the fastest way of creating a Table with two Dimensions and a Measure. The other two could be improved a bit – and I believe they should get improved in the next version of Qlik Sense. Hopefully soon we will see smarter Distribution Plots with two dimensions, and a few other smarter insights.

Notice that when you search for Fields and Master Items, you also get the list of Existing charts that use the same data. In earlier versions of Qlik Sense, existing charts were offered in the results from the Smart Search feature. Recently this functionality was moved over to Insight Advisor. So, if you need to search for existing charts that use a particular field, this is the place for it!

How Can I Make Insight Advisor Get Smarter?

We’ve seen above how we can make Insight Advisor smarter for our own purposes so that it doesn’t continually make the same mistake over and over again.  But how can we share that benefit with other users in our organization? 

Qlik developers have been working for a while on a technique called the “App Scraping” – which is available in the Enterprise version of Qlik Sense.  When you publish an app, the Cognitive Engine analyzes that app and derives “precedents” that will be used by Insight Advisor when presenting insights for other users.  You can think of the publishing action as governance of the improvements to insights.  While you may not want the changes you make using light authoring to immediately impact other people’s insights, once you publish an app it’s a pretty good clue that the decisions you made creating that app are correct and suitable for the Cognitive Engine to learn from. 

The App Scraping module will “learn” from your Apps in three main areas:

1. Field Aggregations. The engine will learn what aggregations are used with certain fields. After this, it should not offer to sum “Unit Price” values. In order to leverage this feature, you can create Master Measures with the fields that you anticipate your users to be interested in, and use preferred aggregation functions in each formula.
2. Field Usage. It’s not always obvious if a certain field should be treated as a Dimension or as a Measure. Insight Advisor is going to learn that from your charts and Master Items.
3. Field pairing – which fields are used together more often than others.

So, in order to help Insight Advisor get smarter and learn from you, create plenty of charts and Master Items with your most common fields and metrics, and watch the suggested insights become more and more relevant in the context of your data.

Conclusion

Insight Advisor is a nifty tool that certainly deserves a close look. It might offer some interesting unexpected insights, or at least speed up some of your visualization tasks.

While Machine Learning allows machines to get new knowledge automatically (or so they say…) we humans still have to get our knowledge upgraded “manually” – usually through advanced education. And this is the perfect time for me to remind you that the next session of the Masters Summit for Qlik comes to Stockholm on April 1^st (and this is not an April Fools’ Day joke!). Not only are we coming to Qlik’s Motherland for the first time, but we are also offering a second track for the “cool kids”, or for those who want to become “cool kids”. In addition to our traditional curriculum for advanced Qlik Sense and QlikView developers, this time we’re offering a second option – the Qlik API track, covering the use of Qlik APIs for Web Developers. Presented by two recognized experts in Qlik Sense integration, Nick Webster and Speros Kokenes, this is a unique learning opportunity for Web Developers to learn how to use Qlik APIs, Qlik Core, and all these cool things that old dinosaurs like myself can only dream of. Check out our new agenda and decide what’s best for you – our traditional Qlik Sense and QlikView Track (Monday-Wednesday), the new Qlik API track (Wednesday-Friday), or a combination of the two (Monday-Friday), which you can enjoy along with a special discount! See you in Stockholm!