Next-Generation Data Warehouse Platforms

Many new options for data warehouse platforms have appeared in the last decade. They support many more options than just a few years ago. It can be difficult to keep track of all the advances and understand what options to use and when. Here’s more about data warehouse platforms that can help businesses to prioritize their specific needs.

What is a data warehouse platform?

A data warehouse platform has five basic components: server, operating system, database, storage, and networking. The platform manages a data warehouse that’s designed for the purposes of reporting, analyzing information, and making decisions.

The data warehouse is separate from the data warehouse platform and contains all the electronically stored data coming from the operational systems of an organization.

What’s next for data warehouse platforms?

Data warehouse platforms can vary greatly, depending on the needs of an organization. For example, a next-generation data warehouse platform may use leading-edge features or address administrative issues. It may tap into cloud computing, appliances and open source or feature hardware upgrades, architectural changes and data migrations.

A next-generation DW platform is, therefore, a relative concept depending on where a company starts off, what new requirements it must address, and how many resources it has.

Next-generation operational data stores (ODS) deals with all the limitations of the traditional ODS and in the same way legacy data warehouse platforms need modernization.

Why businesses should care about data warehouse platforms

 Businesses are facing more challenges than ever before. They are increasingly relying on the data warehouse and related business intelligence to understand change and react to it in the right way. DW platforms have to be updated to support changing business requirements.

Real-time integration with operational applications, scalable architectures, and advanced analytics are some of the technologies associated with next-generation DW platforms.

A successful data warehouse usually matures through many changes and so changes in underlying platforms need to happen too.

Updating a current data warehouse platform

The need for real-time analytics, scalability, support for in-memory processing and support for mixed workloads are some of the reasons DW platforms are evolving.

Organizations can decide to update a current platform or replace it with a new one. Many users have not yet utilized all the capabilities of their current platforms, such as in-memory processing, real-time functions, and analytics.

It is possible to remodel a DW without replacing the platform. Incremental additions to hardware are common (such as adding more memory or storage) and these can satisfy the next generation demand for scalability, in-memory databases and speedy queries.

Ripping out and replacing a data warehouse platform is expensive and intrusive for business users so it is generally better to update a current platform.

Business drivers for new generation data warehouses

The increasing use of advanced analytics is driven by the need for businesses to understand constantly changing environments, as well as to discover opportunities for reducing costs and more. Real-time and related technologies help to enable operational excellence and with the enterprise data warehouse at the heart of business intelligence and operational excellence, warehouse scalability is critical.

Various types of in-memory databases are larger than before, and data operations in-memory are much faster than those that involve disks. Reporting and analysis is very fast and easy to embed in operational applications.

This enables fraud detection, automated recommendations in e-commerce, improved call center service, and much more. When moving to a next-generation DW platform, businesses must expect to make architectural changes to accommodate new technical functions and their related business requirements.

Pressing requirements for next-generation data warehouse platforms

With technology changing the world, what are the most essential requirements for next-generation DW platforms?

Real-time data warehousing: Many DWs are likely to be retrofitted or replaced in the next few years to enable real-time data warehousing (RTDW). Business managers today understand more about the analytics, intelligence etc. that operational processes and applications gain from tight integration with a data warehouse. Various business operations are improved when using the historical context, full informational view, and analytic power of the DW.

Data management practices: Many users need to update the data management tools that process data for use through the DW. The goal of this is to receive quality data to make quality decisions and produce accurate, credible reports.

Cloud computing: Cloud computing is a new and innovative platform choice. Software applications can scale dynamically as workloads increase and as the workload of an application decreases, resources are freed up for use by other systems. This is useful when the data volume of the warehouse varies unpredictably, as this makes capacity planning difficult. Besides offering scalability, cloud computing also offers elasticity and improved performance.

In-memory processing: A leading reason for upgrading to 64-bit systems that typically offer faster CPUs is to deploy an in-memory database for reporting and analytic applications that need very fast query response. Tools for extract, transform, and load (ETL) commonly support in-memory processing in a 64-bit environment. In-memory databases and the 64-bit hardware and software that enable them are high priorities for next-generation data warehouse platforms.

Advanced analytics: Some users choose advanced analytic methods based on data mining, statistics, predictive analytics, artificial intelligence etc. However, many users seem to choose SQL-based methods due to their familiarity with them. Support for advanced analytics has been an area of great activity for software vendors in recent years. For example, many new software firms offer DBMSs built specifically for data warehousing and analytics.

Data warehouse appliances: An alternative platform choice is to pre-integrate a data warehouse appliance as opposed to assembly of a data warehouse platform in-house.

A final word

Expect analytics and real-time data warehousing to be two priorities for a next-generation DW platform. Whether businesses choose to go the route of updating a current data warehouse platform and acquiring a new one, there is a common realization that next-generation technology drivers are really business drivers and are necessary to be competitive in today’s marketplace.