In today’s society, people are encouraged to “think outside the box.” That is great for the discovery of new ideas, concepts, and technologies. The downside is that it makes it very difficult to develop or even discuss things that are still in the box, as it were. We are experiencing an age in technology where nomenclature and branding are getting in the way of full understanding. A classic example is the term “Business Intelligence.” Depending on which company you consult, you will find a different definition of the term. Advanced Analytics must be defined and mapped to a high level picture of Business Intelligence prior to any discussions pertaining to its use and potential.
For many years companies have been capturing data through transactional systems and storing the data in operational relational database management systems (DBMS) such as Microsoft’s SQL Server. Most definitions of Business Intelligence assume a DBMS is already in place. Data in the DBMS, combined with supporting data from the internet (including social media) and other external sources, is aggregated into a data warehouse via a process called ETL (extract, transform, load) in which the data is cleaned and integrated. Everything that needs to be done for the data to be analyzed is done here. The data warehouse comes in many forms depending on the best alignment with the organization. It is from these data warehouses where definitions get fuzzy in business.
From the data warehouse (including data marts) companies can produce dashboards, self-service BI, OLAP cubes, spreadsheets, applications, XML for websites, and the list continues to grow. Many of these practices are included in the broad definition of Business Intelligence. But where does Advanced Analytics fit in?
According to IBM, “Advanced Analytics is a grouping of analytic techniques and is used to predict future outcomes. Advanced analytics can include Predictive Analytics, Simulation, and Optimization1.”
TechTarget describes Advanced Analytics a little more in depth as follows2:
Advanced analytics refers to future-oriented analyses that can be used to help drive changes and improvements in business practices. While the traditional analytical tools that comprise basic business intelligence (BI) examine historical data, tools for advanced analytics focus on forecasting future events and behaviors, allowing businesses to conduct what-if analyses to predict the effects of potential changes in business strategies. Predictive analytics, data mining, big data analytics, and location intelligence are just some of the analytical categories that fall under the heading of advanced analytics.
Notice that both definitions make the distinction that Advanced Analytics refers to the future. However, you may have also noticed that simulation, optimization, data mining, big data analytics, and location intelligence are not necessarily future oriented practices. The only thing about them that is in the future is the actual business decision based on the analysis of the current or past conditions.
BI consultant Rick Sherman does not do us any favors in the clarification of Advanced Analytics. His list of Advanced Analytics includes Statistics, Predictive Modeling, Forecasting, Data Mining, Descriptive Modeling, Econometrics, Operations Research, Optimization, Simulation, and Textual Analytics3. Again we see techniques that are not future based; statistics and descriptive modeling stand out specifically. The need for a standard definition is apparent.
Advanced Analytics should not be defined by time orientation or by techniques used. Instead, the definition should be: any computer-based analytical technique that relies both on current software technology and expertise of the analyst. TED Fellow Sean Gourley demonstrated in his TED Talks lecture4 that neither computers nor grandmasters alone can produce the strongest performance in the game of chess. Computers are capable of playing a strong game, but they are limited. Even though they can analyze far deeper than humans, they must be guided by a knowledgeable user to augment their performance. The same concept applies to analytics. An expert analyst can take the application functions found in analytics software like SAS or IBM’s SPSS modeler and do valuable analysis that could not be performed by the software alone or even with a typical end user.
Therefore, any of the techniques listed previously as Advanced Analytics may or may not actually fall into that camp. It depends on the involvement of the analyst. For example, a monthly report can be programmed to include statistics like mean, variance, confidence intervals, etc. The same report could also include forecasts based on certain assumptions that can be derived from captured data alone. The fact that these reports can be generated without human involvement would disqualify them from being considered Advanced Analytics.
Evidence of the need for experts to intervene with analysis can be seen in the fact that many of the software programs that specialize in model building are now including underlying R interfaces. The cookie cutter output provided by drag and drop modeling programs is no longer sufficient. In order to gain a competitive advantage the models must be customized to their purpose.
So what does this mean for the future of Business Intelligence? It really does not matter whether Advanced Analytics is included in the whole BI process definition or not. Analytical techniques will continue to be enhanced and called various catchy names. Machine learning will continue to help analysts to spend less time detecting patterns, but will always be just a tool in the hands of an experienced analyst.
So what does this mean for businesses? Analytics are infiltrating every industry. Report-type analytics alone will not be enough for an organization to maintain a competitive advantage. There must be employees or consultants who are knowledgeable about business intelligence systems, are well trained in mathematics (including statistics), and are very familiar with the nuances of both the industry and the particular organization – enter the data scientist. The current joke in the analytics industry is that data scientists are like unicorns. No one person can be an expert programmer, statistician, and business function SME. There is a lot of truth in that. The demand for true data scientists well outweighs the current supply and it is going to get worse before it gets better.
The increasing demand for data scientists will usher in increasing salaries, a simple effect of supply and demand. Businesses can avoid having to hire or outsource expensive resources by investing within. Learning the nuances of a business can only come from exposure to that business with the help of experienced mentors. Likewise, knowledge of the company’s data architecture can be learned through exposure. The most difficult asset for the data scientist to acquire is expert training in math/stats. Therefore, it only makes sense for a company to hire analysts and show them the ropes from within. When the analytical talent is attained, it is important to expose them to the business and data architecture by building Advanced Analytics teams. Without team involvement from business SMEs and IT workers, the analyst is really no better than the premade algorithms that come with the modeling software packages.
References
1 What is Advanced Analytics? (n.d.). Retrieved April 29, 2015, from http://www-01.ibm.com/software/data/infosphere/what-is-advanced-analytics/
2 Rouse, M. (2013, October 1). What is advanced analytics? – Definition from WhatIs.com. Retrieved April 29, 2015, from http://searchbusinessanalytics.techtarget.com/definition/advanced-analytics
3 Sherman, R. (2015). Advanced Analytics. In Business intelligence guidebook: From data integration to analytics (p. 381). Amsterdam: Elsevier.
4 Gourley, S. (Director) (2012, December 5). Big Data and the Rise of Augmented Intelligence. TEDxAuckland. Lecture conducted from TED Talks, Auckland, New Zealand. https://www.youtube.com/watch?v=mKZCa_ejbfg
