Distributed MDX Engine – An Enhanced Semantic Layer Architecture on the Cloud

Have you ever tried analyzing 10 billion rows of data in Excel? Or ad-hoc analysis over 100 dimensions on top of that data volume?

I bet that would not be a happy experience as you would have to waste a lot of time just waiting for Excel to crunch the numbers and crash and recover from it. The same process goes over and over and over again. You will have to wait several minutes or even hours before getting the results.

Traditional OLAP engines, such as SQL Server Analysis Service(SSAS), provide multidimensional analysis capability and can connect with Excel. However, SSAS falls short of computing billions of rows of data.

All that can be ended today! Introducing Distributed MDX Engine, the Kyligence Cloud's enhanced semantic layer architecture, with which data analysis in Excel or any BI tools would be faster than ever, even with over hundreds of concurrent users.

What is Distributed MDX Engine

Kyligence Cloud provides an enhanced semantic layer and MDX service that connects to BI tools with the model defined in Kyligence. Distributed MDX Engine is an advanced feature that can boost the performance of Kyligence MDX. It is shipped with a built-in Spark cluster, so the MDX service can execute queries in a distributed Spark cluster. That's where the name came from.

By providing Spark-based MDX services, Kyligence Cloud provides stable MDX services for high concurrency and compute-intensive scenarios. Distributed MDX Engine can handle data at petabyte scale, and by combining powerful abilities offered by Smart Tiered Storage, it effectively improves the performance of detailed analysis, Ad-hoc query, and other scenarios, in addition to aggregated analysis.

From here on, everything will be centered on the self-service analysis of big data.

What's more, If you haven't read our previous blog about Opportunities for SSAS in the Cloud, here is the quick recap of the benefits you can get from Kyligence's semantic layer on the Cloud:

• Scale-out architecture
• Remove limits on the number of dimensions
• Moving to the cloud
• Higher ROI

How Distributed MDX Engine works

Thanks to the high concurrency of the Distributed MDX Engine, the response time (from query trigger to report refresh) will be greatly shortened. What's more, because the computation is pushed down to the Spark cluster, memory in the MDX application node can be released, hence query stability is improved. While for SAAS or other OLAP products that rely on the memory engine, they are easy to hit the memory bottleneck.

This is the architecture diagram of Kyligence Cloud with Distributed MDX Engine enabled.

Each MDX application node corresponds to a Spark cluster, and it supports scaling. You can set multiple MDX application nodes according to business needs, or achieve high availability.

MDX queries from the BI side will be routed to the new engine according to certain rules. As the picture demonstrates, MDX queries will be answered by Distributed MDX Engine if the patterns and functions are matched and the Spark cluster is working.

Scenarios

Distributed MDX Engine can greatly improve efficiency in:

• Ad-hoc analysis of multidimensional data (such as user tagging).
• Drill down to the lowest level in tables with billions & trillions of rows.

Easy use of Distributed MDX Engine

You can easily enable the Distributed MDX Engine in Kyligence Cloud with simple clicks.

As the picture shows below, you can turn on the MDX application node and Distributed MDX cluster options on the cluster setting page.

Key takeaways

Distributed MDX Engine empowers your team to make better decisions based on vast amounts of data. Business users can continue to use Excel to analyze data, and they won't even be aware of the existence of such Distributed MDX Engine, while the performance in the Excel pivot table of the extra-large dataset can be better than ever.

With Kyligence's semantic layer powered by the distributed MDX engine, you can easily implement the capabilities of OLAP on the cloud while migrating off from your traditional OLAP such as SSAS or AAS, and then enjoy managed distributed scalable query performance.

Stay tuned for in-coming features!

You might be interested:

How to connect Excel to ClickHouse within Kyligence Tiered Storage

Why OLAP Persists in the Era of Big Data Analytics

SSAS Disadvantages: Opportunities for SSAS in the Cloud Era

TEST DRIVE TODAY with $300 worth of free usage