Finding Gold: Accessing Your Unstructured Data

This case study details Datavail’s innovative process to convert its Client’s large unstructured data lakes into usable, analyze-able information.

Lacking a simple ‘apples-to-oranges’ solution, the project required developing a strategy that collected and transformed existing transactional data into traditional relational formats that are accessible by analytics programs. Datavail’s cloud data warehouse experts developed a Proof of Concept (POC) transformation strategy that is doable, replicable, and scalable.

Datavail’s ‘Client’ is a global media company responsible for providing billions of hours of entertainment to its millions of subscribers and customers every year. The company’s challenge was figuring out how to maximize the value of its vast volumes of legacy data that were stored in equally vast data lakes and warehouses. A significant portion of that data was its library of video materials, the millions of movies, commercials, television series, and music videos generated over 60+ years of visual and audio media.

These vast information banks were beyond the reach of today’s analytics programming because they were composed of unstructured, transactional data, and analytics programs pull their source data from structured, relational databanks. Further, at that time, there were no available or existing ‘out-of-the-box’ solutions that could convert the legacy data into the analyzable formatting, and even if there were, the immensity of this project may well have dwarfed those capacities anyway.

Download our paper to learn how we helped our Client transform their unstructured data to structured data using Amazon Redshift data warehouse, and a list of other AWS tools and technologies, to glean insights they could use regarding their valuable media assets, for both them and their clients.

• The Challenge – The Client had no way of tracking consumer or industry activities related to their files. With no analytics capacity available, the C-Suite couldn’t use their valuable data assets to make decisions.
• Mastering Data – Structured and Unstructured – Understanding the difference between the two types of data – structured and unstructured – illustrates why the Client was so eager to overcome this hurdle. The Client’s challenge was that it had incredible volumes of business intelligence trapped within billions of unstructured data bits. Extracting this information was beyond their capability.
• Facing the Challenge: Finding a Strategy – The Client wanted to maximize the values of all this unstructured data for both its own purposes and those of its customers. Two key goals were determined, with decidedly distinct challenges in and of themselves.
• AWS Tools – Datavail’s experts and the Client elected to pursue their key goals by employing a suite of AWS tools and programs.
• Building the Plan – The Client was already an AWS customer, so its workforce was familiar with those tools and options. Building on that existing knowledge would reduce the time needed to bring the corporate users up to speed on the new processes.
• Replication – A process to transition the transformed unstructured MongoDB data into the structured Redshift warehouse stacks wasn’t yet available, so Datavail’s professionals began devising one specifically for this project.
• Looking Ahead – The Client still had several hundred thousand other collections it wanted to be moved so that it could replicate its entire MongoDB source base in its proprietary and private Redshift stack.
• Working with Datavail to solve data management and analytics challenges – This case study demonstrates only a small percentage of the creative, innovative, and comprehensive services Datavail provides for its customers every day.

Looking outside the box to find novel program combinations and permutations, the company’s technology professionals are able to craft for their clients’ new processes, services, and features to enhance and improve their corporate concerns and achieve their business goals.

If it’s your company’s goal to reach even higher heights in the future, contact the data migration and analytics experts at Datavail to help get you there.