By Use Cases
By BI Tool
Subscribe to our newsletter>
Get the latest products updates, community events and other news.
The dominant data architectures in today’s market differ greatly from preceding generations. Modern data stacks center around cloud data warehouses (Snowflake, Amazon Web Services‘ Amazon Redshift, Google BigQuery, etc.), Cloud Data Lakes (Databricks), and Data Lakehouses.
What’s driving all of this change? To put it simply, the ability of cloud data warehouses and cloud data lakes to store enormous volumes of data cost-effectively and the lack of expertise needed to run them, along with the ability to offer consumption-based (pay-as-you-go) pricing, ticks all of the right boxes for most corporations.
This article aims to demystify metrics stores and semantic layers to help you understand the similarities and differences between metrics stores and semantic layers.
The potential for gaps between the data platform and how businesses wish to use their data can impede analysts and decision-makers from fully leveraging the data to innovate.
Why does this happen?
First, many critical data assets end up isolated on local servers, data centers and cloud services. Unifying them poses a significant challenge. Often, there are also no standardized data and business definitions, and this adds to the difficulty for businesses to tap into the full value of their data. As companies embark on new data management projects, they need to address these concerns; however, many have chosen to avoid this issue for one reason or another. This results in new data silos across the business.
Second, as every data warehouse practitioner knows, it’s difficult for most business users to interpret the data in the warehouse. Because technical metadata like table names, column names, and data types are typically worthless to business users, data warehouses aren’t enough to allow users to analyze independently.
From a business user’s perspective, what can be done to solve this problem?
Two popular solutions are metrics stores and semantic layers.
In the simplest terms, a metrics store is a layer that sits between upstream data warehouses/data sources and downstream business applications. Metrics platform, Headless BI, metrics layer, and the metrics store all refer to the same idea.
Unlike typical BI reporting, metrics stores separate metrics definitions from BI reporting and visualizations. The teams managing the metrics can define them once inside the metrics store, creating a single source of truth. They can reuse these definitions consistently across BI, automation tools, business workflows, and advanced analytics operations.
A semantic layer is a data representation for business that allows end-users to access data independently using conventional business words. The semantic layer accomplishes this by translating complex data into standard business terms like the product, customer, and revenue, resulting in a uniform, consolidated view of data across the enterprise.
Semantic layers frequently contain data in the form of measures, such as sales, distances, duration, and weight, which can be totaled, averaged, or both. They can also include dimensions, such as sales rep, city, and product, which are categorical buckets used to segment, filter, or group data. Additionally, metrics and KPIs, which are quantitative measures used to track and assess performance, can be built on top of this.
Both semantic layers and metrics stores can accommodate many analytics roles, such as consumers, explorers, innovators, and experts.
Both semantic layers and metrics stores support the following business priorities.
Both align with the overall goals of the organization.
Both approaches benefit end users across the business. Data is accessible to a larger group of users, is more adaptable, enables more sophisticated analytics, and is more economical.
Reusability and Availability
Both can act as a single source of truth that is easily accessible, integrates into apps and workflows, and is reusable across different systems and users.
For both approaches, governance, as well as advanced identification, access, and security management, is a central component.
Cost and SLA Optimization
Semantic layers and metrics stores deliver performant, dependable platforms that provide high-quality data at the lowest cost.
Metrics stores and semantic layers differ in some key ways:
Semantic layers provide a business-friendly set of logical data models, measures, and metrics, whereas metrics stores only offer a business-friendly set of metrics. For metrics stores, the data model is usually controlled by the underlying data source, such as a data warehouse or data mart.
Ease of Use
Semantic layers may be too complex for end-users to utilize, customize and update in some circumstances. IT also needs to be involved in the maintenance and update of the semantic layer. As a result, business users can only ever really be a semantic layer’s consumers.
On the other hand, metrics stores offer easy-to-use metrics as code or even a simple interface for business users to generate and change metrics, allowing businesses to achieve a higher level of self-service and increase acceptance and utilization.
Virtual vs. Physical
Most metrics stores serve as a virtual abstraction tier containing business-oriented metric logic. Data is rarely physically stored in the metrics store itself. Typically, metrics stores translate metric logic into underlying data source queries, with the corresponding data source having responsibility for the data store.
Alternatively, the semantic layer can be a virtual or physical tier between the data source and the downstream applications. In addition, the semantic layer may offer performance optimization techniques, such as pushdown, intermediate servers, caching, and precomputation, to make the semantic layer more performant across various sources and analytics use cases.
Some semantic layers support MDX queries, whereas metrics stores, based on the modern data stack, are typically SQL-based.
Various generations of analytics and business intelligence (A&BI) tools, data marts, data warehouses, query accelerators, knowledge graph/data fabric, and stand-alone virtualization platforms are all possible locations for the semantic layer. Also, many semantic layer solutions provided by vendors can be deployed both on-premises and in the cloud.
Regarding metrics stores, as the concept itself arises from the modern data stack, they usually reside on top of a cloud data warehouse and cloud data lake.
For any organizations planning to adopt a metrics store or semantic layer, here’s some advice from those who have already made the move:
Getting the onboarding right is the key to success.
Even if teams agree that a universal layer is needed, the challenge is making it simple for people across the business to accept and incorporate it into their work. For those able to overcome this challenge, their business will have a significant advantage in making the metrics store or semantic layer a reality for their enterprise.
Blog - Kyligence – Understanding the Metrics Store
Video: What Is a Metrics Store?
How Metrics Standardization Enables Self-Service Analytics
BI Dashboards are Creating a Technical Debt Black Hole
Learn how one big fast-food brand leveraged Kyligence capabilities and implemented precision marketing to maximize profit opportunities.
Come to see the Next Generation of SQL Query Engine
Learn how to achieve alternatives to SSAS.
In this article, we’ll dive into the unified Metrics Platform at Beike, introduce Beike’s practice of building the Metrics Platform infrastructure using Apache Kylin and some real use cases at Beike.
Learn Kyligence Cloud model design principles and how to use Kyligence Cloud to build models.
99 Almaden Boulevard Suite #663
San Jose, CA 95113
+1 (669) 256-3378
Ⓒ 2023 Kyligence, Inc. All rights reserved.
沪 ICP 备 16026036 号 -1
沪公网安备 31011502006713 号
Already have an account? Click here to login