Upgrading Your Business With Next-Generation Data Technologies
September 4, 2014 Leave a comment
By Les King
Director, Big Data, Analytics and Database Solutions – Information Management, Software Group
Out in the field I meet with businesses daily to help demonstrate how they can leverage big data solutions to increase their bottom line. As I’m talking with clients one of the topics that continually comes up is what this will mean for their existing enterprise-class analytic infrastructure.
As existing analytic infrastructures evolve and modernize, there continues to be a fit for purpose design point. Leveraging new data solutions should be seen as an enhancement, extension, evolution or modernization of an existing analytic environment and not be seen as a rip and replace approach.
Some enterprise architects, attracted to the scalability and cost effectiveness of hadoop based solutions, jumped the gun and thought this would be a panacea. A new, less expensive platform, which would replace what is perceived to be more expensive analytic solutions currently being used. This has not turned out to be the case.
Here are the key application categories which I have seen:
- New, high data volume solutions — these are usually best served by a hadoop based architecture. For example, a landing zone for data. From this landing zone you determine which subsets of that data you wish to work with further
- Existing, high data volume solutions — these are currently handled by traditional database solutions. There are cost reduction opportunities here with hadoop based architecture. An example would be a high data volume queryable archive
- Existing and new, critically performing solutions — these leverage traditional databases today and should stay where they are. Additional value can be achieved through technologies such as dynamic in-memory and columnar.
- Existing and new, deep analytic and/or modelling solutions — these can be quite intensive and are quite often best suited for traditional databases, again leveraging the ever improving technologies for BI solutions.
Don’t be afraid of the complexity of these categories. The reality is many of these components are already in place today. By leveraging your existing infrastructure, you also are simplifying the process of ensuring high availability, disaster recovery, security, data life cycle governance and resilience are all in place.
These are all key characteristics of an enterprise-class analytics infrastructure. From an application standpoint, technologies such as BigSQL solutions provides a way for SQL application developers and SQL generating BI tools to work with data regardless of where it sits in this modernized analytic ecosystem. A good example of this is BigSQL 3.0 capabilities which is included in IBM’s BigInsights,
Here is a recently published article by David Loshin, President of Knowledge Integrity Inc., who discusses this exact topic of evolving an enterprise-class analytics infrastructure: http://public.dhe.ibm.com/common/ssi/ecm/en/iml14440usen/IML14440USEN.PDF