Semantic Layer: The BI Trend You Don’t Want to Miss

Data is the cornerstone of every business decision today, and an increasing number of enterprises are using technologies such as data lakes and cloud computing for their digital transformation. However, this unprecedented data volume and distribution has created many challenges when it comes to enterprise data management.

 

One major challenge is that many valuable enterprise data assets are isolated in local servers, data centers, and cloud services. There are no unified data and business definitions, which makes it difficult for enterprises to effectively take advantage of the full value of their data assets. If enterprises carry out their data management initiatives without first addressing these issues, they wrestle with data silos.

 

Every data warehouse practitioner understands how challenging it is for business users to understand the data in the warehouse. Technical metadata such as table names, column names, and data types are often meaningless to business users, so data warehouses alone cannot enable businesses to analyze data.

 

What is missing between the business user and the data warehouse?

 

What Is a Semantic Layer?

 

What exactly is a semantic layer?

 

A semantic layer is a business abstraction derived from the technical implementation layer – a model layer that uniformly maintains business logic, hierarchies, calculations, etc. This frees business users from concerns about the technical complexity and implementation of the underlying data source.

 

A data consumer (no matter his/her data literacy) needs to be able to easily discover, understand, and utilize the data. The semantic layer provides business users with an easy way to understand the data.

 

Gartner points out the importance of a semantic layer in their report “How to Use Semantics to Drive the Business Value of Your Data”:

 

“Unprecedented levels of data scale and distribution are making it almost impossible for organizations to effectively exploit their data assets. Data and analytics leaders must adopt a semantic approach to their enterprise data assets or face losing the battle for competitive advantage.” [1]

 

How Can Organizations Benefit From a Semantic Layer?

 

• Provides you with a comprehensive view of all correlating data assets for a given scenario
• Allows you to have a wide range of data interpretation within your organization without jeopardizing consistency

 

Key Competencies of a Semantic Layer

 

How should data and analytics leaders implement the semantic layer to take full advantage of its capabilities?

 

First of all, the fully realized semantic layer should achieve the following key competencies: 

 

Shared Business Logic in Metrics

 

A semantic layer contains the core logic required for business analysis, transforming the underlying data model into familiar business definitions (dimensions, metrics, hierarchies) and easy-to-understand terms. It can contain commonly used derived metrics, such as year-over-year, month-over-month, month-to-date, etc. Users can directly consume the metrics and reuse the semantics in downstream applications.

 

Through a variety of query interfaces, the unified semantic layer serves as the endpoint for data analysis. This endpoint may be a BI tool or a customized application.

 

Unified Security Policy

 

A semantic layer ensures users and data access controls are uniformly applied in all downstream analysis or business applications, so IT doesn’t need to configure data access control for individual downstream systems.

 

High-Performance Backend Engines 

 

The semantic layer must have a powerful built-in engine or be able to connect to a big data engine such as Spark. The unified semantic layer can give businesses a more comprehensive view of their data to conduct analysis on massive detailed datasets. This cannot be done without a powerful backend engine.

 

Several vendors have introduced their own semantic layers. Let’s take a look at a few BI leaders and their semantic layer offerings.

   

What Types of Semantic Layers Are BI Vendors Offering? 

 

Tableau's Semantic Layer Capability: Enhanced Complex Modeling Capability

 

In the release version of Tableau 2020.2, Tableau introduced a logical (semantic layer) model layer to help users associate more data models. The introduction of this function enables each Tableau data source to support the analysis of multiple fact tables and complex analysis scenarios such as many-to-many relationships (previously, it could only support a single fact table). 

 

 

As you can see, Tableau provides a new semantic layer that enhances its ability to perform complex modeling and analysis. This newly launched data source can be used in Tableau's software ecosystem. By publishing this logical model layer to Tableau Server, more business users can use the logical model in the shared data source through their browsers, and IT can monitor the published data source and control/authorize user rights to data.

   

The semantic layer provided by Tableau assumes IT-centered model management with self-service capabilities in mind. The modeling process is simple and easy to use with a low learning curve. This transparent and seamless modeling method makes Tableau's semantic layer very appealing. However, Tableau's semantic layer does not work with other BI tools. It is common for large enterprises and their different business units to use different BI tools, and so, for these organizations, the Tableau semantic layer can be very limiting.

 

Power BI's Semantic Layer Capability: Unified Semantics With Support for Multiple Applications

 

In March 2020, Power BI released a public preview of read-write XMLA endpoints in Power BI Premium. Power BI Premium provides open-platform connectivity for Power BI datasets, enabling customers to leverage a semantic layer compatible with a wide range of data-visualization tools from different vendors. This means any third party can consume (read) or synchronize (write) the Power BI semantic layer by reading and writing XMLA endpoints.

 

The Power BI semantic layer can be used with other BI tools. From the below diagram by Power BI, we can see that Power BI supports third-party tools to define, manage, and diagnose the semantic layer of Power BI on the write side, and other visualization tools (including Tableau and Excel, as shown in the diagram) on the read side.

 

 

We can see that the latest read-write XMLA endpoints capability of Power BI enhances the capability of Power BI Premium with reusable models. Combined with its ability to perform ultra-complex modeling, its semantic layer can be a very good choice for enterprise-wide BI deployment.

 

MicroStrategy: Federated Analytics Enables a Unified Analysis Platform With IT Governance

 

MicroStrategy has emphasized its Federated Analytics capability since 2019. The semantic layer can be reused with different BI tools through reusable objects and definitions, realizing the unification of underlying discrete data sources and providing a single source of truth.

 

 

BI tools are continuously evolving, so it’s important to take a long-term investment view; enterprises should avoid vendor lock-in of the semantic layer. Separating the semantic layer from a particular BI provider allows for higher flexibility and scalability. Ideally, the semantic layer should be BI tool agnostic.

 

Other Important Considerations:

 

Open Ecosystem

 

One of the main purposes of the semantic layer is to give users easy access to a set of unified business definitions. If the semantic layer can only be used with a particular vendor’s tools, it defeats a primary purpose of a semantic layer. The semantic layer should work with various tools for maximum value and investment payback.

 

Cloud Deployment

 

Data is now stored in various locations, including the cloud, which is increasingly becoming the preferred choice for enterprise IT. A semantic layer that can be deployed in the cloud ensures it will be able to meet both current and future changes in IT architecture.

   

Kyligence: Metric-driven approach to build a semantic layer

 

At Kyligence, we recognize the immense value provided by a semantic layer. Based on years of experience working with our valued customers, we have concluded that a platform that converts technical data into business metrics is the best way to enable business users to explore, understand, and gain insights.

   

A unified metrics layer or metric store, built through a graphical user interface (GUI), empowers business users to precisely define business terms and calculation logic. This creates a universal data language for all businesses to align and collaborate in one convenient location, ensuring that everyone in the organization is on the same page.

 

Let's look at some key capabilities that Kyligence Zen, our metrics store, can provide:

 

Low-code Metrics Catalog

 

The Metrics Catalog is a central location where users can define, compute, organize, and share metrics using GUI. It allows users to define both basic metrics like overall sales and derived/composite metrics like year-to-date (YTD) and year-over-year (YOY)sales.

   

Automated Metric Computation

 

Our product can automatically compute metrics, freeing users to focus on the business impact and outcomes. With Kyligence Zen’s automated metrics calculation by the built-in augmented OLAP engine, business users can spend less time calculating and more time taking action on the data that matters most.

   

Open APIs

 

Kyligence Zen offers Open APIs to connect your organization's data and metrics with any BI or SaaS tool. This agnostic metrics store enables businesses to align data consumers and decision-makers on one unified platform, offering various use cases such as connecting BI for in-house data analytics and decision-making, IM for pushing alerts and collaborating among teams, and SaaS tools for realizing different business use cases or streamlining business workflow.

 

Summary: Critical Capabilities of a Semantic Layer

 

The common features of a semantic layer are:

 

• Reusability
• Unify discrete data sources and provide a single source of truth
• Support for multiple query interfaces
• Unified IT security and control

 

A semantic layer is built for enterprises to connect their data silos in order to realize unified analytic capabilities. In the real world, an enterprise will typically use more than one BI tool and need to analyze huge volumes of data.

 

Enterprises should consider the following when selecting a semantic layer:

 

• Does it convert technical data into a business-friendly format?
• Does it provide a user-friendly interface for non-technical users to build their own business logic?
• Does it integrate with common BI and SaaS tools to simplify workflows?
• Does it support massive data processing, or can it integrate with big data computing engines?

 

Sources and Additional Resources:

 

[1] How to Use Semantic to Drive the Business Value of Your Data

[2] 10 Enterprise Analytics Trends to Watch in 2020

[3] Tableau for the Enterprise: An Overview for IT 

[4] The Tableau Data Model

[5] MicroStrategy 2019 Whitepaper

[6] Announcing Read/Write XMLA Endpoints in Power BI Premium Public Preview

 

---

START A FREE TRIAL TODAY

---