- NoSQL Databases
- Machine Learning
- Internet of Things
- Game-Changing Technologies Fueling the Data-Driven Enterprise
It’s been long acknowledged that data is the most precious commodity of the 21st-century business, and that all efforts and resources need to be dedicated to the acquisition and care of this resource. Lately, however, executives have become enamored with the vision of transforming their organizations into “data-driven” enterprises, which move forward into the future on data-supported insights.
So, what, exactly, does the ideal “data-driven enterprise” look like? Industry observers state there are some key qualities these emerging breeds of organization all have in common. “Data-driven enterprises think differently,” said Kimberly Nevala, director of business strategies on the best practices team at SAS. “They not only ask ‘how are we doing,’ but ask ‘why, what if, and what else?’”
This stems from a deep appreciation for and understanding of corporate data, “and how it can be used with publicly available or subscription data to help enterprises make better decisions,” said Mike Flannagan, senior vice president of analytics for SAP. “Data-driven enterprises also realize that their data is an asset that must be managed carefully, much like their real estate portfolio or human resources.”
Delivery of business value matters in the end, as data “is raw material, not a final solution,” said Joe Pasqua, executive vice president of products for MarkLogic. “Ideally, enterprises should be insight-driven and those insights will be based on data, knowledge of the market and customers, and an ability to act on those insights.”
Ultimately, “a data-driven enterprise makes all its trusted data available to everyone within the enterprise to allow them to make decisions more easily and quickly,” said Dennis Duckworth, director of product marketing for VoltDB. “It also provides tools to suggest such decisions based on the data. Data flows into the enterprise as fast as it is produced and is immediately acted upon, both analytically and transactionally, to produce the best business outcome.”
The forces supporting the move to data-driven enterprises are a direct response to the pressures of growing a 21st-century business. “It’s not the actual technology or implementation that is forcing organizations to become data-driven,” observed Patrick McFadin, vice-president of developer relations for DataStax. “It’s good old-fashioned competition. There is a clear group of winning companies exploiting data to gain the advantage. This has forced incumbents to innovate or die.”
Companies pursuing a competitive edge through analytics-driven innovation “cannot rely on technology alone—culture is central to success,” said Bob Berkey, analytics transformation lead at Accenture Analytics. “Hiring new skills such as data scientists, for example, will be of no business value if they are not embedded across the organization, able to inform new perspectives across teams, and drive intelligent business operations.”
There is a natural convergence between evolving to data-driven and embracing the latest automation technologies. “We find that data-driven businesses are able to drive more automation over time, which is massively reducing manual processes,” said John Carione, product and corporate marketing leader at QuickBase. “IoT, sensors, and other technologies make it easier to get more data faster. That makes the ability to react to changes in information even more important and more valuable.”
There are a number of technology strains that promise to speed the advance into the digital enterprise. IoT is one key area, providing “companies with the ability to generate volumes of data about their business that were impossible a decade ago,” said Flannagan.
“We’ve moved well beyond the days of simple transactional data stored in relational databases,” said Matthew Renze, a content and course author at Pluralsight. “IoT-enabled devices are becoming ubiquitous in the enterprise. These devices provide us with fine-grained sensing and control of all aspects of our enterprise using a flood of near real-time telemetry data.”
Technologies such as those associated with IoT promise to open up vast new opportunities, but also pose challenges to corporate data shops. “Traditional relational databases were built to capture transactions,” said Ravi Mayuram, senior vice president of products and engineering for Couchbase. “Data-driven enterprises that aspire to embrace digital transformation and customer experience need to focus on growing trends of customer engagement and interaction, rather than on simple transactions.”
Another transformative technology, machine learning or deep learning, is providing companies with “the ability to learn things from that data in a way that could never be accomplished by humans,” Flannagan continued. For his part, Bill Schmarzo, CTO for the big data practice of Dell EMC Services, sees machine learning as the key to “helping organizations to uncover new customer, product, and operational insights buried in the growing body of transactional, web, social, mobile, wearables, and IoT datasets.”
Of all emerging technologies, “machine learning has, perhaps, the most transformative potential,” according to Nevala. “Machine learning algorithms are super-charging traditional analytic applications, including risk and fraud detection and hyper-personalized, real-time marketing. Machine learning also underpins both emerging cognitive computing and so-called artificial intelligence applications. Equally important, machine learning may play a pivotal role in the future of data management. It provides the means to interrogate non-traditional information sources, including text, audio, and video. Machine learning is also being applied to support the ongoing curation of data sources in, what is for now, a semi-automated fashion.”
Still, artificial intelligence (AI) itself is only in the earliest stages, Seth Dobrin, vice president and chief data officer for IBM Analytics, pointed out. “We’ve barely scraped the surface when it comes to the potential of artificial intelligence. How do we implement AI in a secure way? How do we maximize the use of AI leveraging the cloud? How do we connect remote locations, cloud, or on-premises, with the right telecom infrastructure globally? These are the questions business leaders must ask to realize the potential of AI.”
Mighael Botha, CTO of Software AG North America, pointed to modern architecture approaches such as microservices and event streaming having a profound impact on how we generate, consume, store, and view data, and enabling new approaches to data. With machine learning, it is now possible to put all this data to good use by continuously improving the types of questions the business side asks and the answers that are received by refining what we are learning every second of the day, he noted.
Blockchain is another technology initiative that is likely to make a difference, Schmarzo added. “It has the ability to fundamentally change the nature of transactions between sellers and buyers, and could unleash creative ways to leverage data and analytics to power new business and customer engagement models.” Combined with machine learning, the new technology “could provide the catalyst for disrupting business models and disrupting existing customer relationships.”
It doesn’t stop there. New modes of data storage also are enabling this advance. Alternative storage mechanisms—NoSQL databases, Hadoop, Casandra, and MongoDB—are “revolutionizing the way we store and process data,” said Botha.
IT automation is another important enabler of the data-driven business. IT automation solutions have pre-built and tested logic to deal with processes associated with structured and unstructured data, which saves time and reduces the number of errors that come from relying solely on custom scripts, said Mehul Amin, director of engineering for Advanced Systems Concepts, Inc. (ASCI). For example, IT automation can make it possible to handle and integrate Hadoop components such as Hive, Spark, and Sqoop into existing applications and technologies, as well as supplement scripts with pre-coded logic and actions. “This makes it possible to quickly get the right information in front of the right decision makers,” Amin noted.
There is also progress on the hardware side, as well. For example, “high-performance all-flash storage ensures that analytics tools can access data quickly,” related Michael Elliott, cloud evangelist at NetApp. “Enterprises should also evaluate converged and hyper-converged infrastructure solutions which build on the ability of IT to more simply deliver predictable performance, speed up time to market, share, and integrate hardware resources, and integrate solution vendor support.”
The ability to not only manage but move data will make the difference between a data-driven and a data-dragging enterprise. “One of the main issues that companies have in their effort to become data-driven is the fact that so much data analysis is done on separate systems using separate copies of the data,” said Tim Willging, distinguished engineer with Rocket Software. “This data copying or movement imposes a great cost and introduces latency of the data being analyzed. The bottom-line is that you can’t expect real-time analytics if you’re taking data off of your main system—no matter what you’re using—to crunch the numbers.”
Technology, of course, weighs heavily on data transformation efforts and gives rise to the question of whether enterprises need to overhaul or replace existing infrastructure, or can they build on what they have. Industry observers agree that wholesale replacement of existing infrastructure, tools, and techniques is not only impractical, but completely unnecessary. Recognize that “the best outcomes will be not from innovative technologies in a silo, but from innovative technologies working in concert with existing data and systems,” said Flannagan.
“Never advocate for a full rip and replace—it’s not feasible,” advised Abhi Yadav, co-founder and CEO of ZyloTech. “Organizations should look for technologies that integrate with what they already have and make their existing delivery or call to action tools more powerful.”
At the same time, industry observers suggest more solution innovation is needed and is on the way. But innovation doesn’t only apply to hardware and software. Dobrin, for one, suggests more innovation is needed in the data governance process, as data needs to be unified across an organization and acted upon in real-time. The challenge is the “tools to achieve this simply don’t exist yet.”
Organizations may even need to focus on the data they already have, versus worrying about acquiring new sources. “There are external forces like IoT that are supplying more raw data, but enterprises haven’t come close to exhausting their existing data assets,” said Pasqua. “No one needs to wait for new data. They can capitalize on what they have. Moreover, many companies don’t have systems that allow them to effectively translate their insights into actions. Having insights is great, but unless they can be operationalized, they aren’t of much value.”
Enterprises need to re-evaluate and redesign their operating models to be able to keep up in a data-driven world, said Berkey. “Technology is not the inhibitor of scale or speed—
operating models are,” he observed. The goal of an operating model should be delivery of data-driven insights when and where they are needed. “Organizations can take insight-powered action at speed and scale, driving disruption through analytics. While as-a-service capabilities make this possible for enterprises migrating away from legacy infrastructure, the technology shifts must be accompanied by cultural shifts.”
SKILLS FOR SUCCESS
Evolving into a data-driven enterprise requires more than just investments in infrastructures, new technologies, and new architectures, however. “Even with the best tools and technology, data are just ones and zeros—they do not magically generate wealth on their own,” said Renze. “Organizations need individuals with backgrounds in data science and data engineering in order to transform their data into actionable insights.”
The most prominent skill needed in today’s fast-changing data environment is adaptability. “The ability to be a lifelong learner who is open to new trends and technologies is priceless in this industry,” said Dobrin. “One must also be willing to fail and learn from mistakes made.” From data science teams to the C-suite, lifelong learners are the ones who will propel data-driven enterprises, he observed.
Still, the success of a data-driven enterprise will depend on data science skills. “Data scientists are essential to help train machine-learning models which can then be deployed to identify actions to be taken in real-time,” said Botha. “We’ll see enterprises requiring new skills in data analytics but also creating small agile teams to apply these insights,” added Alex Robbio, president and co-founder of Belatrix Software. “Many organizations are struggling to find employees with the experience and skills in new areas, such as machine learning; as a result, we expect to see organizations forming partnerships with companies that can help them with these technologies. It’s worth noting that machine learning and neural networks don’t only require new specific skills but also a different mindset, which is hard to train for.”
A renaissance type of individual, with a polymathic set of skills, is needed to move enterprises to the data-driven era. “Historically, data was analyzed in a reactive manner, and then recommendations were made based on incomplete data or data trends,” said Scott Gnau, CTO of Hortonworks. “Data engineers tasked with analyzing this data will have to shift how they view their job, and the value they bring to their company by being more proactive opposed to reactive. They need to be both scientists—think algorithms—as well as artists, finding patterns, expressing connections. They also need to be detectives, sleuthing for new relationships.”
Jason Andersen, vice president of business line management for Stratus Technologies, concurred, stating that he sees “a new type of role I like to call ‘hybrid OT.’” These professionals “will have the same job responsibilities as operational technologists do now, but will have a far more technical background, particularly, with digital technologies. They will be less dependent on IT and will take on a more futuristic role where they will be the ones driving innovation forward.” He sees a convergence between this role and that of data scientists, as well. “We can draw parallels to the role of application developers when the cloud disrupted their space. They evolved into a DevOps role where they would now code, build, design, and more—things that app developers were not doing previously. I think we can expect a similar reaction as data collection and analysis disrupts the enterprise.”
Still, other industry experts anticipate there will be continued demand for traditional data skills. “This is going to sound terribly unsexy, but I continue to believe the two most valuable skills for an analyst to have are SQL skills and a fluency in translating business problems into analytic questions,” said Daniel Mintz, chief data evangelist at Looker. “SQL—because it’s the lingua franca of data and it aligns so well with the way analysts think. And then bilingualism—in business and in data—because that’s the hardest thing to find. Without people who can speak both those languages, you end up chasing interesting problems that don’t deliver any value to the business. I’d much rather hire a SQL person who’s bilingual but doesn’t yet know how to train a neural network, than a machine-learning specialist who can’t clean their own data and has no idea what kinds of models would help the bottom line.”