Excel Your KPIs with AI Copilot Start for free today
Your AI Copilot for Data
Definitive Guide to Decision Intelligence
Subscribe to our newsletter>
Get the latest products updates, community events and other news.
The Apache Kylin community is pleased to announce the release of Apache Kylin v2.6.0.
Apache Kylin is an open source Distributed Analytics Engine designed to provide SQL interface and multi-dimensional analysis (extreme OLAP) on Big Data supporting extremely large datasets.
This is a major release after 2.5.0 and includes many new enhancements. All of these changes can be found in the release notes. Here is a highlight of the major ones:
Apache Kylin already supports several data sources like Amazon Redshift and SQL Server through JDBC.
To help developers handle SQL dialect differences and easily implement a new data source through JDBC, Kylin provides a new data source SDK with APIs for:
Check KYLIN-3552 for more.
In the past, query caches were not efficiently used in Kylin for two reasons: aggressive cache expiration strategy and local cache. Because of the aggressive cache expiration strategy, useful caches are often cleaned up unnecessarily.
Because query caches are stored in local servers, they cannot be shared between servers. And because of the size limitation of local cache, not all useful query results can be cached.
To deal with these shortcomings, we changed the query cache expiration strategy by signature checking and introduced the memcached as Kylin’s distributed cache so that Kylin servers are able to share cache between servers.
And it’s easy to add memcached servers to scale out distributed cache. With enough memcached servers, we can cache things as much as possible. Then we also introduce segment level query cache which can not only speed up queries but also reduce the rpcs to HBase.
The related tasks are KYLIN-2895, KYLIN-2894, KYLIN-2896, KYLIN-2897, KYLIN-2898, KYLIN-2899.
In the past, fast OLAP cubing used split threads, task threads, and main thread to do the OLAP cube building. There is complex join and error handling logic.
This new implementation leverages the ForkJoinPool from JDK, and the event split logic is handled in main thread. Cuboid task and sub-tasks are handled in fork join pool, cube results are collected async and can be written to output earlier.
Check KYLIN-2932 for more.
In the past, the way to create HLLCounter and to compute harmonic mean were not efficient.
The new implementation improves the HLLCounter creation by copying register from another HLLCounter instead of merge. To compute harmonic mean in the HLLCSnapshot, it does this enhancement by :
Check KYLIN-3656 for more.
In the past, to add cuboids which are not prebuilt, the cube planner turns to mandatory cuboids which are selected if its rollup row count is above some threshold.
There are two shortcomings:
The new implementation improves the way to estimate the row count of un-prebuilt cuboids by rollup ratio rather than exact rollup row count. With better estimated row counts for un-prebuilt cuboids, the cost-based cube planner algorithm will decide which cuboid to be built or not and the threshold for previous mandatory cuboids.
With this improvement, we don’t need the threshold for mandatory cuboids recommendation, and mandatory cuboids can only be manually set and will not be recommended.
Check KYLIN-3540 for more.
To download Apache Kylin v2.6.0 source code or binary package, visit the download page.
Follow the upgrade guide.
If you face issues or have any questions, please send mail to the Apache Kylin dev or user mailing list: email@example.com, firstname.lastname@example.org; Before sending, please make sure you have subscribed the mailing list by dropping an email to email@example.com or firstname.lastname@example.org.
Great thanks to everyone who contributed!
Ensure you understand everything Apache Kylin has to offer and how its extreme OLAP technology compares to Kyligence. Learn more about these two augmented OLAP analytics solutions on our Kylin vs. Kyligence comparison page.
The driving force behind Meituan’s success is not simply a robust analytics system, but the OLAP engine that system is built upon - Apache Kylin.
Cloud Analytics News will share the important news on Apache Kylin, Kyligence Cloud, and related technologies. In this edition, we cover Apache Kylin 4.X beta, the launch of Kyligence Cloud 4, Pivot to Snowflake, and more.
UnionPay was able to consolidate the 1,200 Cognos cubes into 2 Kyligence cubes and a single ETL process. Besides extending the life of the analytics executed against this data, there was a massive improvement in operational efficiency.
A peek behind the curtain of the world's leading open source big data analytics project, Apache Kylin.
An introduction to Apache Kylin's new storage and compute architecture, Apache Parquet. This article introduces Kylin's query principles, Parquet storage, and accurate duplicate removal
99 Almaden Boulevard Suite #663
San Jose, CA 95113
+1 (669) 256-3378
Ⓒ 2023 Kyligence, Inc. All rights reserved.
Already have an account? Click here to login
A complete product experience
A guided demo of the whole process, from data import, modeling to analysis, by our data experts.
Q&A session with industry experts
Our data experts will answer your questions about customized solutions.
Please fill in your contact information.We'll get back to you in 1-2 business days.