Build the Common Data Language with the Metrics Platform Start Now
Kyligence Zen Kyligence Zen
Kyligence Enterprise Kyligence Enterprise
Cloud
Services
By Use Cases
By BI Tools
Customers
Resources
Apache Kylin
About
Partners
This is the last feature you can leverage to boost query performance. Let’s make the final touches to a Kyligence Data Model! If you have not read the previous blogs of this series, please go to the following links — Part 1, Part 2, Part 3, Part 4.
As defined in Part 2, a segment represents a physical folder under the hood. To fully leverage the segment-level folder pruning capability, it is recommended to size a segment according to real usage patterns.
For example, if most business users fetch quarterly data in a query, three months’ data per segment will assure great query performance. If most data consumers query annual data each time, then load 12-month data into a segment. Bottom Line — The less scattered segments, the faster a query runs.
A performance test has been run to demonstrate how QPS(Query Per Second) is tripled from 4+ to 12+ by just changing from 1-month data per segment to 6-month data per segment. In this test, most queries are aggregating quarterly or half-year data per query.
Best Practice: Merging small segments into larger ones on a regular basis is a good practice to ensure a consistent query performance over time.
If you have been practising building models using Kyligence products or Apache Kylin, you may have realized that one of the trickiest parts of this whole process is to KNOW user query patterns in order to create the “perfect” indexes and “perfect” layout.
Good News — we have a Pythonic way of discovering the unknown knowns from your SQL/MDX query history. We can heat-map your query patterns using a Python script plus a visualisation tool like Tableau or PowerBI. If you are interested, hit the Clap & Share buttons, so I will be more motivated to share it with you.
Hope this step-by-step model design guide has helped you successfully “WHOA” your stakeholders with the lightning-fast query performance.
If you have developed your own best practices, please feel free to share them in the comments. I’d love to see your creative techniques.
If you have any questions, please leave your comments down below. Always happy to help!
Kyligence Zen intelligently manages data in the retail industry. Read to learn how to develop the "North Star Metric" system to track goals and progress.
Kyligence introduces the deployment of OLAP on top of Azure, including data sources, features, benefits, and prerequisites. Learn more about Kyligence for Azure.
What's OLAP on big data? What're its benefits? Here's everything you need to know about OLAP.
Learn how one big fast-food brand leveraged Kyligence capabilities and implemented precision marketing to maximize profit opportunities.
Already have an account? Click here to login
预约演示,您将获得
完整的产品体验
从数据导入、建模到分析的全流程操作演示。
行业专家解惑
与资深行业专家的交流机会,解答您的个性化问题。
您还可以在云平台中 部署 Kyligence
直接获得 30 天免费试用
请填写真实信息,我们会在 1-2 个工作日内电话与您联系。