When strategizing Big Data, enterprises find themselves at the corner of a huge opportunity and the hurdles they need to jump over to turn opportunity into business growth. We’re still in the infancy of Big Data, but IT and line of business (LOB) leaders know that significant business value will accrue from efficiently managing and analyzing large, complex data sets.
Since the Enterprise Tech Journal interview with Denny Yost (see page 48) discusses overcoming the challenges facing front-line Big Data managers, this column concentrates on other issues—those faced by IT leaders and managers of Big Data teams—and provides practical advice for dealing with those issues. These are mainly challenges to the business, such as managing multiple vendors and access to finite resources with Big Data expertise. Of course, the larger issues of cost and return on investment also weigh on these managers’ minds: Will the business get the value it needs from Big Data and does the value justify the major investment required?
But first, it’s essential to step back and think strategically. Organizations need to assemble leaders from every unit, identify business intelligence goals, determine the level of resources (budget and human capital) needed to hit the mark and identify the initiative’s owner. Do you need customer behavior analytics for Marketing, data on social media posts about company products so that Development can deliver new services or info on cost-saving efficiencies for Operations? Whatever you decide, make sure the info you seek is actionable.
After determining your needs, prioritization begins so that the initial solution can zero in on the area that will return the biggest bang for the buck. Once the initial solution is perfected, it can be scaled up by applying it incrementally to other needs.
Here’s the biggest hurdle of all, one that’s well known by anyone who has researched Big Data solutions: While some products efficiently address pieces of the puzzle such as analytics, moving data between the system of record and the analytics environment, or scheduling and automating tasks, they all require different management platforms, which cloud visibility and add inefficiencies to the mix. The products don’t communicate with one another effectively, and scheduling of processes is nonexistent. While some companies do an admirable job of Big Data management, given the meager tools they have to work with, they might be better off with a solution that makes their job easier by providing a centralized workspace and a dashboard that ties together the multiple management platforms.
According to The Forrester Wave: Big Data Hadoop Solutions, Q1/2014 report, published on February 27, 2014, “Most firms estimate that they are only analyzing 12 percent of the data they already have, leaving 88 percent of it on the cutting-room floor.” And this doesn’t include the numerous new channels of data that can also provide greater insight if it were actually harvested. Given those statistics, very few companies are behind the Big Data curve—yet. So it behooves smart companies to strategize and prioritize up-front, provide trusted software solutions companies with their requirements and get in on the ground floor of any enterprisewide solutions in development. By watching the market like a hawk, those companies will be ready to swoop in and capture the Big Data advantage when a comprehensive solution presents itself.
Simultaneously, you can research resource requirements, take an inventory of in-house talent and shore up resources—financial and human—so that you’re ready to rally them when it’s time to deploy. The typical company lacks appropriate skillsets, such as Hadoop (Big Data) administrators, who program using the MapReduce framework. Another group in short supply are data scientists, who take the enterprise’s industry and its business and marketing strategies into consideration when asking questions of the data to produce actionable analysis. Now’s the time to develop those skillsets from within—or seek them from outside your company’s firewalls.
Big Data will also transition security needs away from securing systems and databases to identity and access management and malfeasance prevention.
One last issue regarding governance and corporate culture to ponder: Mainframe DBAs most often regard themselves as custodians of data, leaving business owners to determine how data will be used. Admins of non-relational databases are often both the business owner and the custodian of data. Beware of mixing oil and water.
No one yet knows what the full impact of Big Data will be, but we do know that a better understanding of customers, behaviors and business practices is too promising to put on the back burner. Technology is just one part of the Big Data puzzle. Database and data analysis skills, better security, agreement on business drivers and cultural acceptance are also critical to Big Data success. IT leaders and business leaders need to collaborate on a business intelligence strategy now so they will be ahead of the curve when the market, which is young and evolving quickly, catches up with them.