Complex data analysis is a multi-billion dollar business. Major data analysis tool makers alone report revenues totaling over $4 billion per year: SAS Institute ($3.2 Billion), IBM SPSS ($0.3-1.0 Billion), MathWorks ($850 Million), Wolfram Research (at least $40 million), and a number of less well known smaller firms. Medical businesses, financial firms, and science and engineering organizations spend billions of dollars per year on these tools and the salaries of the analysts, scientists, and engineers performing the analyses.
Complex data analysis increasingly determines the approval of new drugs and medical treatments, medical treatment decisions for individual patients, investment decisions for banks, pensions, and individuals, important public policy decisions, and the design and development of products from airplanes and cars to smart watches and children’s toys.
State-of-the-art complex data analysis is labor intensive, time consuming, and error prone — requiring highly skilled analysts, often Ph.D.’s or other highly educated professionals, using tools with large libraries of built-in statistical and data analytical methods and tests: Excel, MATLAB, the R statistical programming language and similar tools. Results often take months or even years to produce, are often difficult to reproduce, difficult to present convincingly to non-specialists, difficult to audit for regulatory compliance and investor due diligence, and sometimes simply wrong, especially where the data involves human subjects or human society. Many important problems in business and society remain unsolved despite modern computer-intensive data analysis methods.
A widely cited report from the McKinsey management consulting firm suggests that the United States may face a shortage of 140,000 to 190,000 such human analysts by 2018: http://www.mckinsey.com/business-functions/digital-mckinsey/our-insights/big-data-the-next-frontier-for-innovation
We are developing tools and algorithms to automate complex data analysis.
Automating Complex Data Analysis Presentation