Excel Your KPIs with AI Copilot Start for free today
Your AI Copilot for Data
Definitive Guide to Decision Intelligence
Subscribe to our newsletter>
Get the latest products updates, community events and other news.
October 16, 2023: When I first penned down this article on semantic layers in 2020, the concept was fairly fresh in the market landscape. Over the span of three years, the data industry has evolved, and I've updated this article to reflect the latest developments. Among the notable advancements is the emergence of the metrics store concept, which is swiftly carving a niche for itself in the modern data stack, adding a new dimension to how we perceive and interact with data. This updated version aims to provide you with a crisp understanding of semantic layers while introducing you to the exciting realm of metrics stores, highlighting their significance in today's data-centric world.
In the digital age, businesses are inundated with data, yet starved for insights. As enterprises embrace technologies like data lakes and cloud computing, they often find themselves lost in a sea of data silos. The key metadata within data warehouses remains an enigma to business users, making data analysis a herculean task. What's the missing link between business users and the treasure trove of insights locked away in data warehouses?
The 'Semantic Layer' emerges as a solution to navigate these challenges. Acting as a bridge between technical data infrastructure and business users, it provides a unified and comprehensible view of data assets.
This article delves into the concept, importance, and implementation of a semantic layer in today's business intelligence landscape.
What exactly is a semantic layer?
A semantic layer is a business abstraction derived from the technical implementation layer – a model layer that uniformly maintains business logic, hierarchies, calculations, and many more. This frees business users from concerns about the technical complexity and implementation of the underlying data source.
A data consumer (no matter his/her data literacy) needs to be able to easily discover, understand, and utilize the data. The semantic layer provides business users with an easy way to understand the data.
Gartner points out the importance of a semantic layer in their report “How to Use Semantics to Drive the Business Value of Your Data”:
“Unprecedented levels of data scale and distribution are making it almost impossible for organizations to effectively exploit their data assets. Data and analytics leaders must adopt a semantic approach to their enterprise data assets or face losing the battle for competitive advantage.” 
Retail Business: Amidst the COVID-19 pandemic, a major Food and Beverage (F&B) retail company leveraged a semantic layer to fast-track its digital transformation on Microsoft Azure. The setup facilitated seamless data integration from various sources, processed through Apache Spark. The data, once processed, was funneled into Kyligence's Semantic Layer for analysis. This architecture simplified data management by auto-collecting and categorizing data into cubes based on predefined metrics, enabling non-technical users to conduct analysis using familiar tools like Excel.
Manufacturing: In the realm of automobile manufacturing, a prominent manufacturer leveraged a semantic layer to amplify the potential of its data lake on AWS, especially focusing on data analytics concerning the Internet of Vehicles. Car owner data was systematically collected and analyzed through the semantic layer, enabling precise service marketing by grouping car owners based on their behaviors. Additionally, it facilitated a detailed analysis of services-related churn rates to optimize user data access processes, thereby reducing churn. Furthermore, the collection and analysis of vehicle trajectory data through the semantic layer ushered in derivative services like designated driving, thus enriching the data dimensions and enhancing service offerings. The semantic layer played a quintessential role in translating technical data into actionable insights, propelling service optimization, and fostering innovation in service delivery.
Financial Services: Consider the transformation at a top commercial bank with the roll-out of Pandora, a self-service metrics platform, acting as a semantic layer. Initially, a data dashboard with 50 metrics took 12 workdays to deliver. With Pandora, the delivery time shrunk to 5 workdays, as 30 metrics were readily available and 15 were easily derived, leaving only 5 new metrics to be created. This platform, akin to a semantic layer, democratized data by enabling non-technical users to assemble dashboards effortlessly, while IT experts focused on creating new metrics. This scenario vividly illustrates the efficiency and simplification a semantic layer brings to data management and business self-service.
How should data and analytics leaders implement the semantic layer to take full advantage of its capabilities?
Foremost, the fully realized semantic layer should achieve the following key competencies:
A semantic layer contains the core logic required for business analysis, transforming the underlying data model into familiar business definitions (dimensions, metrics, hierarchies) and easy-to-understand terms. It can contain commonly used derived metrics, such as year-over-year, month-over-month, month-to-date, etc. Users can directly consume the metrics and reuse the semantics in downstream applications.
Through a variety of query interfaces, the universal semantic layer serves as the endpoint for data analysis. This endpoint may be a BI tool or a customized application.
A semantic layer ensures users and data access controls are uniformly applied in all downstream analysis or business applications, so IT doesn’t need to configure data access control for individual downstream systems.
The semantic layer must have a powerful built-in engine or be able to connect to a big data engine such as Apache Spark. The universal semantic layer can give businesses a more comprehensive view of their data to conduct analysis on massive, detailed datasets. This cannot be done without a powerful backend engine.
Several vendors have introduced their own semantic layers. Let’s take a look at a few BI leaders and their semantic layer offerings.
This option involves integrating the semantic layer within the existing data warehouse or data layer. It allows for the creation of abstractions, definitions, metrics, and measures on top of different data stores and file formats. The semantic layer connects to various layers, making data more accessible and standardized.
Example tools and vendors: Oracle, SAP, Amazon Web Services, Microsoft Google, IBM, Snowflake, Teradata, Incorta
In this option, the semantic layer exists as its own architectural element within the data and analytics stack. It offers a dedicated semantic layer independent of specific data services or technologies. Users can build abstractions, definitions, metrics, and measures by connecting to various data layers.
Example Tools and vendors: Kyligence, Cloudera, Microsoft, AtScale, Cambridge Semantics, Kyvos Insights
What's happening in the market
This option involves utilizing the semantic layer provided by analytics and business intelligence (A&BI) tools. These tools often provide built-in capabilities for creating and managing the semantic layer, allowing users to define and organize data models, measures, and hierarchies.
Example tools and vendors: Info Birst, IBM, Microsoft Power BI, MicroStrategy, Oracle, Salesforce, SAP, Sisense, Tableau, Google Cloud Looker
At Kyligence, we recognize the immense value provided by a semantic layer. Based on years of experience working with our valued customers, we have concluded that a platform that converts technical data into business metrics is the best way to enable business users to explore, understand, and gain insights.
A unified metrics layer or metric store, built through a graphical user interface (GUI), empowers business users to define business terms and calculation logic precisely. This creates a universal data language for all businesses to align and collaborate in one convenient location, ensuring that everyone in the organization is on the same page.
Let's look at some key capabilities that Kyligence Zen, our metrics store, can provide:
The Metrics Catalog is a central location where users can define, compute, organize, and share metrics using GUI. It allows users to define both basic metrics like overall sales and derived/composite metrics like year-to-date (YTD) and year-over-year (YOY)sales.
Our product can automatically compute metrics, freeing users to focus on the business impact and outcomes. With Kyligence Zen’s automated metrics calculation by the built-in augmented OLAP engine, business users can spend less time calculating and more time taking action on the data that matters most.
Kyligence Zen offers Open APIs to connect your organization's data and metrics with any BI or SaaS tool. This agnostic metrics store enables businesses to align data consumers and decision-makers on one unified platform, offering various use cases such as connecting BI for in-house data analytics and decision-making, IM for pushing alerts and collaborating among teams, and SaaS tools for realizing different business use cases or streamlining business workflow.
In the release version of Tableau 2020.2, Tableau introduced a logical (semantic layer) model layer to help users associate more data models. The introduction of this function enables each Tableau data source to support the analysis of multiple fact tables and complex analysis scenarios such as many-to-many relationships (previously, it could only support a single fact table).
As you can see, Tableau provides a new semantic layer that enhances its ability to perform complex modeling and analysis. This newly launched data source can be used in Tableau's software ecosystem. By publishing this logical model layer to Tableau Server, more business users can use the logical model in the shared data source through their browsers, and IT can monitor the published data source and control/authorize user rights to data.
The semantic layer provided by Tableau assumes IT-centered model management with self-service capabilities in mind. The modeling process is simple and easy to use with a low learning curve. This transparent and seamless modeling method makes Tableau's semantic layer very appealing. However, Tableau's semantic layer does not work with other BI tools. It is common for large enterprises and their different business units to use different BI tools, and so, for these organizations, the Tableau semantic layer can be very limiting.
In March 2020, Power BI released a public preview of read-write XMLA endpoints in Power BI Premium. Power BI Premium provides open-platform connectivity for Power BI datasets, enabling customers to leverage a semantic layer compatible with a wide range of data-visualization tools from different vendors. This means any third party can consume (read) or synchronize (write) the Power BI semantic layer by reading and writing XMLA endpoints.
The Power BI semantic layer can be used with other BI tools. From the below diagram by Power BI, we can see that Power BI supports third-party tools to define, manage, and diagnose the semantic layer of Power BI on the write side, and other visualization tools (including Tableau and Excel, as shown in the diagram) on the read side.
We can see that the latest read-write XMLA endpoints capability of Power BI enhances the capability of Power BI Premium with reusable models. Combined with its ability to perform ultra-complex modeling, its semantic layer can be a very good choice for enterprise-wide BI deployment.
MicroStrategy has emphasized its Federated Analytics capability since 2019. The semantic layer can be reused with different BI tools through reusable objects and definitions, realizing the unification of underlying discrete data sources and providing a single source of truth.
BI tools are continuously evolving, so it’s important to take a long-term investment view; enterprises should avoid vendor lock-in of the semantic layer. Separating the semantic layer from a particular BI provider allows for higher flexibility and scalability. Ideally, the semantic layer should be BI tool agnostic.
Other Important Considerations:
One of the main purposes of the semantic layer is to give users easy access to a set of unified business definitions. If the semantic layer can only be used with a particular vendor’s tools, it defeats a primary purpose of a semantic layer. The semantic layer should work with various tools for maximum value and investment payback.
Data is now stored in various locations, including the cloud, which is increasingly becoming the preferred choice for enterprise IT. A semantic layer that can be deployed in the cloud ensures it will be able to meet both current and future changes in IT architecture.
Enterprises should consider the following aspects when selecting a semantic layer:
Sources for evaluating potential vendors:
Adopting a Semantic Layer is pivotal for enterprises aiming to bridge the gap between complex data and insightful analytics. It serves as a catalyst in democratizing data, simplifying data access, and fostering a unified understanding across various business units. By translating technical data into user-friendly terms, it accelerates decision-making processes, maximizes data asset utilization, and propels enterprises toward a data-driven culture. The flexibility it offers in integration with popular BI tools and big data engines, coupled with its cloud deployment capability, makes it a future-proof investment for evolving IT architectures.Explore the transformative potential of a Semantic Layer by booking a demo with Kyligence to delve deeper into our robust solution tailored for your enterprise’s analytical needs.
We cover some of the hitching questions on the Semantic data layer
A semantic layer constitutes a data interpretation that aligns with business perspectives and delivers an integrated, comprehensive data view within a business entity. Diverse data descriptions from various data repositories can be efficiently interconnected through a semantic layer, providing a cohesive, reliable, and unique data perspective for analytical tasks and other commercial objectives.
Both Semantic Layer and Metrics Store are pivotal in bridging the gap between raw data and actionable insights. A Semantic Layer acts as a translator, simplifying complex data into business-friendly terms, aiding in easier comprehension and utilization. Conversely, a Metrics Store centralizes and standardizes metrics definitions ensuring consistency across various tools and dashboards. While the Semantic Layer focuses on data translation and simplification, Metrics Store emphasizes on consistency and standardization of metrics. Both play crucial roles in enhancing data accessibility and understanding, albeit through different approaches, contributing to a more robust and effective data analytics infrastructure.
 How to Use Semantic to Drive the Business Value of Your Data
 10 Enterprise Analytics Trends to Watch in 2020
 Tableau for the Enterprise: An Overview for IT
 The Tableau Data Model
 MicroStrategy 2019 Whitepaper
 Announcing Read/Write XMLA Endpoints in Power BI Premium Public Preview
Discover the 7 top AI analytics tools! Learn about their pros, cons, and pricing, and choose the best one to transform your business.
Discover operational and executive SaaS metrics that matter for customers success, importance, and why you should track them with Kyligence Zen.
Unlock the future of augmented analytics with this must-read blog. Discover the top 5 tools that are reshaping the analytics landscape.
What website metrics matter in business? Learn about categories, vital website metrics, how to measure them, and how Kyligence simplifies it.
Unlock potentials of analytics query accelerators for swift data processing and insights from cloud data lakes. Explore advanced features of Kyligence Zen.
Unlock power of data storytelling in business. Learn how to convey insights using narrative and visual representations, examples, and benefits.
Explore these exceptional cloud analytics tools. Assess their pros, cons, and pricing to pinpoint the optimal one for your business.
Learn what natural language query is and how it transforms your data analytics. Explore examples of natural language queries in Kyligence Zen.
Discover how AI shapes banking, healthcare, and data analytics sectors. Get insights into the future of industry disruption to guide your decisions.
99 Almaden Boulevard Suite #663
San Jose, CA 95113
+1 (669) 256-3378
Ⓒ 2023 Kyligence, Inc. All rights reserved.
Already have an account? Click here to login
A complete product experience
A guided demo of the whole process, from data import, modeling to analysis, by our data experts.
Q&A session with industry experts
Our data experts will answer your questions about customized solutions.
Please fill in your contact information.We'll get back to you in 1-2 business days.