Semantic Layer: The BI Trend You Don’t Want to Miss

Data is at the cornerstone of every business decision today, and an increasing number of enterprises are using technologies such as data lake and cloud computing for their digital transformation. However, this unprecedented data volume and distribution has created many challenges when it comes to enterprise data management. 

 

One major challenge is that many valuable enterprise data assets are isolated in local servers, data centers, and cloud services. There are no unified data and business definitions, which makes it difficult for enterprises to effectively take advantage of the full value of their data assets. If enterprises carry out their data management initiatives without first addressing these issues, they end up wrestling with data silos.

 

Every data warehouse practitioner understands how challenging the data in the warehouse is for business users to understand. Technical metadata such as table name, column name, and data type are often meaningless to business users, and so data warehouses aren’t sufficient on their own to enable businesses to carry out data analysis. 

 

What is missing between the business user and the data warehouse?

   

What Is a Semantic Layer?

 

What exactly is a semantic layer?

 

A semantic layer is a business abstraction derived from the technical implementation layer – a model layer that uniformly maintains business logic, hierarchies, calculations, etc. This frees business users from concerns about the technical complexity and implementation of the underlying data source. 

 

A data consumer (no matter his/her data literacy) needs to be able to easily discover, understand, and utilize the data. The semantic layer provides business users with an easy way to understand the data.

 

Gartner points out the importance of a semantic layer in their report “How to Use Semantics to Drive the Business Value of Your Data”:

 

“Unprecedented levels of data scale and distribution are making it almost impossible for organizations to effectively exploit their data assets. Data and analytics leaders must adopt a semantic approach to their enterprise data assets or face losing the battle for competitive advantage.” [1]

 

How Can Organizations Benefit From a Semantic Layer?

 

• Provides you with a comprehensive view of all correlating data assets for a given scenario
• Allows you to have a wide range of data interpretation within your organization without jeopardizing consistency

 

Key Competencies of a Semantic Layer

 

How should data and analytics leaders implement the semantic layer to take full advantage of its capabilities? 

 

First of all, the fully realized semantic layer should achieve the following key competencies: 

 

Shared Business Logic

 

A semantic layer contains the core logic required for business analysis, transforming the underlying data model into familiar business definitions (dimensions, measures, hierarchies) and easy-to-understand terms. It can contain commonly used derived measures, such as year-over-year, month-over-month, month-to-date, etc. Users can directly consume the calculated measures and reuse the semantics in different downstream applications.

 

Through a variety of query interfaces, the unified semantic layer serves as the endpoint for data analysis. This endpoint may be a BI tool or a customized application.

 

Unified Security Policy

 

A semantic layer ensures users and data access controls are uniformly applied in all downstream analysis or business applications, so IT doesn’t need to configure data access control for individual downstream systems.

 

High-Performance Backend Engines 

 

The semantic layer must have a powerful built-in engine or be able to connect to a big data engine such as Spark. The unified semantic layer can bring businesses a more comprehensive view of their data so that businesses can conduct analysis on massive detailed datasets. This cannot be done without a powerful backend engine.  

 

Several vendors have introduced their own semantic layers. Let’s take a look at a few BI leaders and their semantic layer offerings. 

   

What Types of Semantic Layers Are BI Vendors Offering? 

 

Tableau's Semantic Layer Capability: Enhanced Complex Modeling Capability 

 

In the release version of Tableau 2020.2, Tableau introduced a logical (semantic layer) model layer to help users associate more data models. The introduction of this function enables each Tableau data source to support the analysis of multiple fact tables and complex analysis scenarios such as many-to-many relationships (previously, it could only support a single fact table). 

 

 

As you can see, Tableau provides a new semantic layer that enhances its ability to perform complex modeling and analysis. This newly launched data source can be used in Tableau's software ecosystem. By publishing this logical model layer to Tableau Server, more business users can use the logical model in the shared data source through their browsers, and IT can monitor the published data source and control/authorize user rights to data.

   

The semantic layer provided by Tableau assumes IT-centered model management with self-service capabilities in mind. The modeling process is simple and easy to use with a low learning curve. This transparent and seamless modeling method makes Tableau's semantic layer very appealing. However, Tableau's semantic layer does not work with other BI tools. It is common for large enterprises and their different business units to use different BI tools, and so, for these organizations, the Tableau semantic layer can be very limiting. 

 

Power BI's Semantic Layer Capability: Unified Semantics With Support for Multiple Applications

 

In March 2020, Power BI released a public preview of read-write XMLA endpoints in Power BI Premium. Power BI Premium provides open-platform connectivity for Power BI datasets, enabling customers to leverage a semantic layer compatible with a wide range of data-visualization tools from different vendors. This means any third party can consume (read) or synchronize (write) the Power BI semantic layer by reading and writing XMLA endpoints.

 

The Power BI semantic layer can be used with other BI tools. From the below diagram by Power BI, we can see that Power BI supports third-party tools to define, manage, and diagnose the semantic layer of Power BI on the write side, and other visualization tools (including Tableau and Excel, as shown in the diagram) on the read side.

 

 

We can see that the latest read-write XMLA endpoints capability of Power BI enhances the capability of Power BI Premium with reusable models. Combined with its ability to perform ultra-complex modeling, its semantic layer can be a very good choice for enterprise-wide BI deployment.

 

MicroStrategy: Federated Analytics Enables a Unified Analysis Platform With IT Governance

 

MicroStrategy has emphasized its Federated Analytics capability since 2019. Through reusable objects and definitions, the semantic layer can be reused with different BI tools, realizing the unification of underlying discrete data sources and providing a single source of truth. 

 

 

BI tools are continuously evolving, so it’s important to take a long-term investment view; enterprises should avoid vendor lock-in of the semantic layer. Separating the semantic layer from a particular BI provider allows for higher flexibility and scalability. Ideally, the semantic layer should be BI tool agnostic. 

 

Other Important Considerations:

 

Open Ecosystem

 

One of the main purposes of the semantic layer is to give users easy access to a set of unified business definitions. If the semantic layer can only be used with a particular vendor’s tools, it defeats a primary purpose of a semantic layer. The semantic layer should work with a rich variety of tools for maximum value and investment payback. 

 

Cloud Deployment

 

Data is now stored in various locations, including the cloud, which is increasingly becoming the preferred choice for enterprise IT. A semantic layer that can be deployed in the cloud ensures it will be able to meet both current and future changes in IT architecture. 

   

Kyligence: Unified Semantics-as-a-Service Capability

 

At Kyligence, we recognize the immense value a semantic layer can provide. We created a semantic layer meant for enterprise-wide deployment. Let’s take a look at some key capabilities: 

 

Multiple Query Languages Support

 

The semantic layer maps complex data into user-friendly business terms. It provides each user with unified data definitions through standard interfaces (standard SQL and MDX), at petabyte-level scale.  

 

Unified Security Policy

 

Centralized security is essential for enterprise data. Kyligence’s semantic layer reduces data security risks caused by data dispersion in different business systems. User control and data access can be uniformly configured and applied in a consistent way to all downstream business applications, which means that IT doesn’t need to configure additional data access control for downstream systems. 

 

AI-Augmented Engine

 

Kyligence uses the computing power of Spark to pre-calculate data, and our AI-augmented intelligent engine can identify users’ patterns to automatically make calculations in the background. Kyligence uses Spark's distributed architecture for data processing and commodity cluster resources for storage. These aggregations are automatically created and stored in big data clusters. Using machine learning, Kyligence can automatically discover and identify the most important data from SQL history, analysts' behaviors, data sampling, and runtime indicators.  

 

On-Premise or in the Cloud: Flexible Deployment

 

The cloud continues to become the dominant choice for many enterprises. In order to ensure that your technology investment will not become obsolete in the future, enterprises should consider whether the semantic layer can support cloud architecture. From data source access to transformation and semantic modeling to integration with downstream applications, Kyligence provides end-to-end cloud big data analysis with a unified cloud-based semantic layer. 

 

Summary: Critical Capabilities of a Semantic Layer

   

 The common features of a semantic layer are:

• Reusability  
• Unify discrete data sources and provide a single source of truth
• Support for multiple query interfaces  
• Unified IT security and control 

 

A semantic layer is built for enterprises to connect their data silos in order to realize unified analytic capabilities. In the real world, an enterprise will typically use more than one BI tool and need to analyze huge volumes of data.

 

Enterprises should take the following points into account when selecting a semantic layer:

• Does it integrate with popular BI tools and other interfaces? 
• Does it support massive data processing, or can it integrate with big data compute engines? 
• Does it support cloud deployment, since the cloud is becoming increasingly popular?

 

Sources and Additional Resources:

 

[1] How to Use Semantic to Drive the Business Value of Your Data

[2] 10 Enterprise Analytics Trends to Watch in 2020

[3] Tableau for the Enterprise: An Overview for IT 

[4] The Tableau Data Model

[5] MicroStrategy 2019 Whitepaper

[6] Announcing Read/Write XMLA Endpoints in Power BI Premium Public Preview

 

---

START A FREE TRIAL TODAY

---