Many big data projects do not have a tangible return on investment but are part of a larger funded initiative
Big Data investments continue to rise but are the growth is contracting, as fewer companies plan to invest in future, according to Gartner. Nearly half (48%) of companies have invested in big data in 2016, up 3% from 2015. However, those who plan to invest in big data within the next two years fell from 31 to 25% in 2016.
Organizations increasingly understand that big data is not just about a specific technology but a collection of different data management technologies and practices to support multiple analytic uses. Thus, they are avoiding thinking about big data as a separate effort.
This shows that organizations are moving from vague notions of data and analytics to specific business problems that data can address.
Getting Big Data Projects to Production Is a Challenge
Big data projects appear to be receiving less spending priority than competing IT initiatives. Only 11% of respondents from organisations that have already invested in big data reported that their big data investments were as important, or more important, than other IT initiatives, while 46% stated that they were less important.
While nearly three quarters of respondents said that their organisation has invested or is planning to invest in big data, many remain stuck at the pilot stage. Only 15% of businesses reported deploying their big data project to production, effectively unchanged from last year (14%).
This could be due to the fact that many big data projects do not have a tangible return on investment that can be determined upfront, as per Gartner. Another reason could be that the big data initiative is a part of a larger funded initiative. This will become more common as the term "big data" fades away, and dealing with larger datasets and multiple data types continues to be the norm.
A further factor to consider is the lack of effective business leadership or involvement in data initiatives. Too often, pilots and experiments are built with ad-hoc technologies and infrastructure that are not created with production-level reliability in mind.