ENTLEVIN

Icon-facebook Icon-twitter Icon-youtube-v Icon-instagram-1
Menu

Why most big data analytics projects fail

In recent years, skepticism surrounding big data and analytics investments has considerably waned, thanks to tangible success stories from tech giants like Alibaba, Google, and Amazon. These companies have reshaped their business models by leveraging the power of data. As a result, senior business executives now recognize that investments in big data technologies can be a game-changer for unlocking cost savings, generating revenue, expanding into new markets, and building customer loyalty. This shift has driven the big data and business analytics (BDA) market to grow by an impressive 12% year-on-year, reaching $189 billion in 2019.

However, amid this growth, it’s crucial to acknowledge that a significant number of BDA projects fail to deliver the expected return on investment. Astonishingly, 60% of these initiatives never progress beyond the pilot and experimentation stage and are sometimes abandoned altogether (as reported by Gartner in 2015). Only a small fraction of the remaining 40% that manage to reach the production stage generates measurable incremental value – value that is genuinely deemed worthwhile by the end-users.

Common pitfalls that derail big data and analytics projects

With 15 years of professional experience across diverse industries, including banking, insurance, wealth management, technology, and retail, we have observed recurring pitfalls that frequently lead to the failure of BDA projects. Organizations set themselves up for failure when they:

  1. Poorly integrate new big data solutions with legacy systems

    • Many initial BDA implementations fail because they don’t align with the organization’s daily processes and decision-making norms. A disconnect exists between new BDA solutions and existing operations.

  2. Neglect cultural and organizational barriers

    • A significant hurdle is the slow pace at which established firms transition to a data-driven culture. Cultural and organizational barriers hinder the adoption of BDA initiatives.

  3. Lack a comprehensive data strategy

    • Organizations often focus on tactical initiatives and specific use cases, missing broader opportunities. They lack a sense of urgency and fail to capitalize on initial successes.

  4. Overemphasize pretty dashboards without focus

    • The proliferation of data visualization tools sometimes leads to the creation of numerous reports without adequate data cleansing and validation. Misleading insights can result in financial loss and reputation damage.

  5. Underinvest in data literacy training and adoption

    • Many companies allocate insufficient budget for training and adoption, even though successful practices emphasize the importance of these activities in driving BDA success.

  6. Underinvest in “Data Translators”

    • Bridging the gap between business and analytics is essential. Experts who can effectively explain how BDA solutions enhance decision-making are often missing, hindering buy-in from business users.

Overcoming the hurdles

To illustrate how to avoid these pitfalls, consider the case of a retailer aiming to optimize prices. The company initiated a pricing and promotions analytics task force that collaborated closely with category managers to understand their decision-making processes. This collaboration led to the development of pricing tools that seamlessly integrated into existing workflows. Ongoing training and aligned KPIs from top management further ensured the success of the project, resulting in improved promotional effectiveness.

In conclusion, while the potential benefits of big data and analytics are immense, it’s essential to address these common pitfalls proactively to ensure that BDA projects reach their full potential and deliver tangible value. By learning from successful practices and fostering a data-driven culture, organizations can unlock the true power of data analytics.