What is a Modern Data Warehouse and Why Is It Important for Publishers?
We’ve all heard the common management complaint that they have lots of data but no information. Business Intelligence is a tricky area to get right. One explanation for this is that companies have built massive databases of historical data that were good for only one thing – looking at history, often in a very simplistic, stylized way. And that is the best case. Some find themselves with heaps of data that is unorganized, unusable, and unclear as to its accuracy. This system of data management provides little opportunity for top management to make intuitive judgments on how to adapt to rapidly changing and challenging business conditions.
In response, the information industry adapted and created “business intelligence” solutions that were a step forward from the database concept. However, it gave us little more than better ways to view and slice the data and slightly better accuracy than what we already had.
More significant developments in data analysis came when increased computing bandwidth allowed us to collect, store, and manipulate masses of data from multiple sources and in different formats. Importantly, this included input from outside the enterprise such as social media. Now, we could give management much better insights into their business which allowed them to make more informed and intuitive business decisions as a result.
This evolution gave us the data warehouse and later, the so-called Modern Data Warehouse (MDW).
What Is a Data Warehouse, and What Is a Modern Data Warehouse?
The concept of a data warehouse has existed for decades. The intent was to change the function of our systems from that of simply providing operational data to driving decision support systems for “business intelligence” applications and beyond. The data in these cases came from multiple internal sources like sales and finance, and external data from third-party applications and partners. The data warehouse pulled data from those systems and reformatted the new data to blend with the existing internal data, ready for decision-makers to access. The frequency of the pull was determined by the unique circumstances in each enterprise.
A data warehouse is not a database. A data warehouse is required when a company needs to perform analytics on a very large set of data. A database is simply not designed to do that. Today’s data warehouses are much more capable than the single-purpose databases of the past. Databases were intended for well-defined analyses on (often historical) datasets, sourced mainly from within the enterprise. Modern data warehouse architecture is designed for storing and analyzing masses of data from many potential sources, in various formats, that can lead to making better business decisions
Advanced analytics have demanded this development of the modern data warehouse. Traditional data warehouses typically struggle to keep up with the challenges of huge volumes of mixed structured and unstructured data such as graphics and XML files. And then there was The Cloud! Many of the conventional attempts at data warehousing had been built on on-premise models, and it became clear that the future of the data warehouse was in the cloud.
Cloud-Based Modern Data Warehouses
As businesses made the move to cloud computing, so, too, did their data warehousing tools. The cloud offers many advantages including flexibility, collaboration, and accessibility from anywhere. Popular tools followed including those from Microsoft’s Azure. knk Software is a Gold-Certified Microsoft Partner and offers a cloud-based information system for publishers, built on Microsoft Dynamics 365 Business Central.
A cloud-based modern data warehouse lowers the barriers to entry, cost, complexity, and time-to-value, that have traditionally limited the adoption and successful use of data warehousing technology. Cloud-based warehouses allow an organization to scale up or scale down as data volumes and requirements change. Also, less technical expertise is needed in-house, and with a subscription model, businesses find it easier to get started, with less up-front investment or time-consuming development processes.
The cloud data warehouse eliminates many of the risks implicit in an on-premise implementation. You don’t have to budget for, lease, or maintain hardware and software. There are no unplanned costs.
How to Create an MDW: First Steps
Most B2B media publishers want to be data-driven and monetize their vast amounts of data. We’re also seeing the trend in trade and other D2C publishing houses. But before starting to use advanced technologies like AI and Machine Learning, you should start by asking some specific questions including “what is our ultimate business purpose?” There are other more technical questions like:
- What are the sources of data that we need, external and internal?
- Does the architecture provide for scalability and larger volumes of data imports?
- Does the system support real-time analysis of streaming data?
- Does the structure support a concurrent business intelligence model?
Once you have an idea of your data goals, you’re ready to move forward. Consult with experts about what custom steps your company needs to take to achieve your data goals.
Traditional database architecture still has its place when working with similarly structured data types for business intelligence applications. However, this type of on-premise solution begins to fail when there’s more variety to the stored data. In addition, on-premise architecture is expensive to implement and maintain and does not function at the speed and flexibility required for modern datasets in the current age of Big Data.
Cloud-based data warehouse architecture, by contrast, is designed for the scalability of today’s data integration and analytics needs. It delivers significant performance and integration benefits and is much more flexible and cost-efficient given the variety of data formats that businesses now need to import. Simply stated, cloud-based data warehouse architecture is the most efficient use of data warehousing resources.
With over 30 years experience in publishing software, knk Software has helped hundreds of publishing houses achieve their data goals. Our cloud-based solutions are built on future-proof Microsoft technology, meaning you’ll be supported for years to come. Contact a member of our team today to get your media company on the path to being data-smart!