Database Management Trends — Machine Intelligence

Datavail just released a white paper on the Top 10 Trends in Database Administration for the coming decade. The white paper is based on global surveys of hundreds of IT managers and input from Datavail’s hundreds of DBAs. The paper predicts a new kind of IT arising from cloud computing, artificial intelligence, and mobile devices — an IT that is “instant, invisible, and intelligent.”

This post looks at some of the findings from the Datavail white paper related to machine intelligence. With the Internet of Things (IoT) creating billions of information trails as data is exchanged between devices, Big Data is going to get much, much bigger. At the same time, Moore’s Law is still driving down the cost of data storage to near zero. Something must sit between those giant data flows and giant data warehouses to parse the chatter for insights.

Business Intelligence Analytics

Business Intelligence software (BI) is enterprise management based on data. Oracle’s Business Intelligence Enterprise Edition (OBIEE) is an example of BI that is used in many large organizations to provide real-time data analysis and visualization. BI is behind many of the successful new ventures in the sharing economy, such as Uber, Lyft and Airbnb.

“BI makes it possible to have dynamic pricing for your products based on real-time analysis of market trends. BI software makes it possible to run auction markets for things such as securities, airline tickets and advertising space.” — Top 10 Trends in Database Management

In a recent episode of CBS News’ popular TV news magazine show, 60 Minutes, billionaire Michael Bloomberg showed us why traders pay $25,000 a year for access to a Bloomberg terminal. Bloomberg showed how he could analyze a company like auto maker GM in a zillion different ways, jumping from visualization to visualization through a series of screen showing GM’s suppliers and their relationships to other companies.

Bloomberg terminal is a proprietary form of BI software that parses publicly-available data sources and allows it to be analyzed and visualized in a variety of ways. Similar software is available today for even small and medium-sized businesses. However, an accurate visualization of bad data is a bad visualization. Bloomberg works due to an abundance of public data in a standard format. What enterprises find when using BI in their own operations is based on getting comprehensive, accurate data out of departmental silos and into a single container. And that’s not easy.

Process Documentation

Process documentation involves recording the steps taken in the performance of activities. It can be a completely mechanized process, such as tracking changes in a Microsoft Word document, or it can require completion of a form, such as checking off steps in a quality review. Some of the data is structured, such as time and attendance logs, and some is not, such as surveillance camera video or screen captures. One of the tricks of BI is to be able to look at both structured and unstructured data when documenting processes.

Datavail recently released a white paper, Process and Project Documentation Best Practices and Case Histories, which shows how process documentation is tied into compliance audits and quality control systems, including:

• Information Technology Infrastructure Library (ITIL), an industry-wide framework of best practices in IT delivery
• Capability Maturity Model (CMM), a set of standards for software engineers
• International Organization for Standards ISO 9000/9001, standards for best practices in software development

Process documentation leads to the development of these standards for data processing operations. As data is collected, it is mapped to workflows. These workflows are compared to established standards to insure best practices are being followed. Then these workflows become the basis of machine learning programs, which can map the workflows back to data sets.

Predictive APIs

When your BI system applies workflows to data sets containing in some cases billions or trillions of records, it’s able to make predictions. These predictions fuel Predictive APIs — Application Programming Interfaces — which guide end users through complex tasks. How complex? Auto-completing your typing or autochecking your spelling are two quick examples.

“Predictive APIs are behind machine learning initiatives that are rapidly improving processes in medicine, science, engineering and technology.” — Top 10 Trends in Database Management

Predictive APIs are powering large artificial intelligence systems used in banking and finance, including credit rating and bond rating; in trading software, including exchanges and market analysis; in manufacturing, to enable computerized process control; and in data security, to identify fraud or unauthorized access. Good process documentation often starts with an assessment as part of a BI migration, leading to a structure for data input that ensures the organization can benefit from the BI investment.

Conclusion

Machine learning is transforming modern organizations by enabling them to capture, store, and analyze the enormous data flows resulting from the Internet of Things (IoT). This data stream can be tapped to keep your databases safe no matter how many appliances you add. It can be used to generate the process documentation that makes predictive APIs possible. This data can be mapped to workflows and data visualization software to alert you to opportunities in real time.

At Datavail, we understand the business analytics and Art of BI, and we can show you how much BI can improve your business. If you want to get started with BI — or you aren’t seeing the benefits of BI you expected — we can held build a BI solution that is tailored to your needs with expert assessment, installation, and training. Contact Datavail!

Datavail is a specialized IT services company focused on Data Management with solutions in BI/DW, analytics, database administration, custom application development, and enterprise applications. We provide both professional and managed services delivered via our global delivery model, focused on Microsoft, Oracle and other leading technologies.