By Use Cases
By BI Tool
Subscribe to our newsletter>
Get the latest products updates, community events and other news.
This article will describe how to implement semantic layers with data marts, as well as their definitions, user cases, and benefits.
A semantic layer is a business abstraction derived from the technical implementation layer – a model layer that uniformly maintains business logic, hierarchies, calculations, etc. This frees business users from concerns about the technical complexity and implementation of the underlying data source. A data consumer (no matter his/her data literacy) needs to be able to easily discover, understand, and utilize the data. The semantic layer provides business users with an easy way to understand the data.
A semantic layer contains the core logic required for business analysis, transforming the underlying data model into familiar business definitions (dimensions, measures, hierarchies) and easy-to-understand terms. It can contain commonly used derived measures, such as year-over-year, month-over-month, month-to-date, etc. Users can directly consume the calculated measures and reuse the semantics in different downstream applications.
Through a variety of query interfaces, the unified semantic layer serves as the endpoint for data analysis. This endpoint may be a BI tool or a customized application. So the end-users of the semantic layer should be the ones who will use the data for analysis, and they may be data analysts, business analysts, decision-makers, and report designers, not engineers and developers.
In the same digital world, there are diverse users, for example, business analysts, data analysts, data engineers, and data scientists, who each tell their own data story and need a unified data definition to translate their insights into data. The semantic layer is one such platform, storing not data but metadata. It acts as a single version of the truth, serving a diverse set of users.
Traditionally, a data mart is a structure/access pattern used to retrieve client-facing data in data warehouse setups. A data mart is a subset of a data warehouse that is typically focused on a single business line or team.
Nowadays, it is also possible to build a data mart structure on top of a data lake, as some vendors are using the term "lakehouse" to refer to building a data mart on top of the data lake.
Whereas data warehouses or data lakes have enterprise-wide depth, the information in data marts pertains to a single department. Each department or business unit may be considered the owner of its data mart, which includes all hardware, software, and data in some installations.
Often data mart is the last layer in the traditional data warehouse structure. Its position locates before connecting with business intelligence tools.
The semantic layer can be located in many layers, for example, in BI, on top of the logical data warehouse (LDW) or data mart. Defining the semantic layer with the data mart is an option for the semantic layer on top of the logical data warehouse (LDW).
The traditional semantic layer, which is linked to traditional A&BI tools, functions as a data mart, providing a layer of logic as well as a store of analytics-ready data with the context to support self-service by unskilled users. Data collection, on the other hand, can only go so far. As a result, new approaches centered on connecting to data have gained popularity.
For example, Tableau 2020.2 introduced a logical (semantic layer) model layer to assist users in associating more data models. With the addition of this function, each Tableau data source can now support the analysis of multiple fact tables as well as complex analysis scenarios like many-to-many relationships (previously, it could only support a single fact table).
Tableau now has a new semantic layer that improves its capacity to execute complicated modeling and analysis, as you can see. Tableau's software ecosystem may take advantage of this freshly released data source. More business users can utilize the logical model in the shared data source through their browsers as a result of publishing this logical model layer to Tableau Server, and IT can monitor the published data source and control/authorize user rights to data.
Tableau's semantic layer is designed for IT-centered model management with self-service features. The modeling technique is simple and straightforward, with a short learning curve. Tableau's semantic layer appeals to me because of its transparent and seamless modeling style. Tableau's semantic layer, on the other hand, does not operate with other BI products. Large corporations and their many business segments frequently use different BI products. Therefore the Tableau semantic layer can be highly restricted for these organizations.
The traditional IT-built semantic layer failed for two fundamental reasons:
As a result, we advocate putting semantics on the edge of a logical data warehouse. Because the logical data warehouse (LDW) is built to meet 95% of analytics requirements. LDWs offer a wide range of analytic engines, allowing them to accommodate a diverse collection of users and applications. As a result, including the semantic layer in the LDW is frequently recommended.
In this approach, the semantic layer functions as a data mart sitting on top of an LDW. It can source data from other data stores, but the data warehouse is specifically modeled as a star schema or snowflake schema to support the semantic layer. In the semantic layer, modelers can enhance the data by adding hierarchies, calculated measures, etc.
Users can build a semantic layer with a data mart to ensure the data definition and consistency inside a department, unit, or set of users in an organization. E.g., Marketing, Sales, HR, or finance. Let's look at the user cases of data mart with semantic layers within different industries and what business value can be generated.
A data mart is defined as a subset of a data warehouse that is focused on a single functional area of an organization. The semantic layer is the layer that BI tools usually connect to. Building a semantic layer with a data mart can implement many key benefits below.
A semantic layer contains the core logic required for business analysis, transforming the underlying data model into familiar business definitions (dimensions, measures, hierarchies) and easy-to-understand terms. It can contain commonly used derived measures, such as year-over-year, month-over-month, month-to-date, etc. A semantic layer with a specific data mart can unify the data definitions in a specific area. Users in their specific areas can directly consume the calculated measures and reuse semantics in different downstream applications.
A semantic layer ensures users and data access controls are uniformly applied in all downstream analysis or business applications. Data mart makes sure different areas of data get isolated, so IT doesn't need to configure data access control for individual downstream systems.
The semantic layer must have a powerful built-in engine or be able to connect to a big data engine such as Spark. The unified semantic layer can bring businesses a more comprehensive view of their data so that businesses can conduct analysis on massive, detailed datasets. This cannot be done without a powerful backend engine.
A data mart can unify data from multiple sources. It provides an interface that BI tools use to enable ad-hoc analysis with drag and drop. Through a variety of query interfaces, the unified semantic layer with a data mart serves as the endpoint for data analysis. This endpoint may be a BI tool or a customized application.
The financial industry has many data analysts who find the best portfolio of investments and calculate the risk factors of different markets. Financial analysts can use semantic layers to calculate the aggregated returns of multiple investment products, as well as risk factor assessments for large portfolios.
The semantic layer enables retailers across channels to integrate all data from POS systems, e-commerce, customer service, and marketing programs into one source. This enables analysts to assist marketers in creating better campaigns and experiences that meet customer expectations.
One of the biggest pain points in manufacturing is finding the production processes that need the most optimization. With easy access to data, manufacturing companies can build process forecasting tools to calculate the "best process" and execute against it.
With access to all relevant data, analysts can use semantic layers to analyze patients who are deteriorating and then allocate medical resources to the right patients, improving the management of medical resources.
Come to see the Next Generation of SQL Query Engine
Learn how to achieve alternatives to SSAS.
In this article, we’ll dive into the unified Metrics Platform at Beike, introduce Beike’s practice of building the Metrics Platform infrastructure using Apache Kylin and some real use cases at Beike.
Learn Kyligence Cloud model design principles and how to use Kyligence Cloud to build models.
99 Almaden Boulevard Suite #663
San Jose, CA 95113
+1 (669) 256-3378
Ⓒ 2022 Kyligence, Inc. All rights reserved.
Already have an account? Click here to login