Disruptive Digital Transformation: Headache, Opportunity, Or Both?

Over the past decade, much has been written on the topic of digital transformation and how it has become imperative to survival in the modern world of global business. A recent study from Accenture though had a provocative call to action for Chief Data Officers (CDOs) with the headline of the report: "The Post-Digital Era is Upon Us - ARE YOU READY FOR WHAT'S NEXT?" The implication is that the conversation is shifting from “Do we do digital transformation or not” to “What does the business landscape look like post digital transformation?”

The State of Digital Transformation

While talk about digital transformation has been plentiful, the actual “doing” aspect has been noticeably absent. While it is difficult to measure all the elements of a digital transformation, a clear enabler in the journey is to become a data- or analytics-driven organization. New Vantage Partners recently published a report called Big Data and AI Executive Survey 2019 on the attitudes and progress on the journey to becoming data-driven, and found some progress has been made as:

• 90 percent of those who completed the survey are “C-level” executives—chief data, analytics, or information officers. A decade ago, only one of these jobs even existed.
• 92 percent of the respondents are increasing their pace of investment in big data and artificial intelligence (AI).
• 62 percent have already seen measurable results from their investments in big data and AI.

However, major causes for concern remains as:

• 72 percent of survey participants report that they have yet to forge a data culture
• 69 percent report that they have not created a data-driven organization.
• 52 percent admit that they are not competing on data and analytics.

As the study shows, disruption is difficult. The growth in big data and, more recently, artificial intelligence is great news, but if we learned anything from the big data adoption over the past decade, it is that success is not guaranteed. According to Gartner, some 85 percent of big data initiatives fail.

A key challenge for the big data and Hadoop adoption was that we applied our traditional mindsets for technology – i.e., let IT become the sole custodians of knowledge and expertise on the new technology. The problem with this approach is that the business demand for data and insights has far outstripped the availability of technical skills and resources in IT.

CDOs are typically tasked with changing the organization to embrace a data-driven culture, requiring the bridging of this divide, and bringing data proficiency to everyone – much like we did with word processors and spreadsheets that are now the domain of everyone in the enterprise.

Bridging the Divide By Finding Common Ground

The good news is that many organizations are succeeding and achieving incredible business results. In most cases, it required them to reimagine how the data to insights value chain is constructed. One way to go about this process is to find common ways to embrace new technologies such as:

• Embracing Cloud and Serverless – Hadoop, interestingly, started in the cloud in Yahoo but made its broad market entrance as on-premises platforms through Cloudera and Hortonworks which are now combined. Amazon Web Services (AWS), Microsoft Azure, and Google all quickly jumped into the mix to ease the management of Hadoop clusters into easy to get started, pay-by-the-drink services. Many Hadoop workloads have moved to the cloud over the past few years and, with the forming of the new Cloudera, their vision is to be a cloud Hadoop provider. Recommendation: When you find a new technology you wish to try, find a way to move to the cloud and preferably in a serverless, configuration-less manner.
• Solving the Data Preparation Bottleneck Through Self-Service – For decades, analytical projects suffered from the 80/20 principle in that 80 percent of the effort is spent on finding, cleansing and shaping the data. This same ratio now applies to data science and AI. We will never have enough IT developers to provision all the data requests. Recommendation: Self-service data fabrics that empower business analysts to find, clean, shape, and publish their data is a critical enabler for both analytical scale and accuracy.
• Agility meets governance – Having spent many years designing and implementing analytical platforms, many projects fail after the prototype or pilot phase because the project cannot be production-ized at scale in the enterprise. In most cases, this is because it lacks governance, security, auditability, and collaboration across the stakeholders. Recommendation: While you might act with agility and grab a new technology, never lose sight of the fact it will eventually need to grow into an enterprisewide competency.

The CDO Re-Balancing Act

CDOs are tasked with driving this journey towards their respective businesses by becoming data-driven and competitive on analytics. And as usual, the solution is not about technology only, but also the people and the processes. What is very clear is that traditional approaches to solving the scale (such as how broadly the organization needs to be reskilled) and complexity (of managing and consuming new technologies) will not hit the mark. It is time to reimagine how we are doing analytics end-to-end with a view on delivering this at both the speed of the business and/or the market need while maintaining governance and security of your enterprise’s data treasures.

Piet Loubser, senior vice president, Paxata authored this article.

The views and opinions expressed in this article are those of the author and do not necessarily reflect those of CDOTrends.