The Apache Kylin community is pleased to announce the release of Apache Kylin v3.0.0-alpha.
Apache Kylin is an open source Distributed Analytics Engine designed to provide SQL interface and multi-dimensional analysis (extreme OLAP) on Big Data supporting extremely large datasets.
This is the first release of the new generation v3.x, the main feature introduced is the Real-time OLAP. All of the changes can be found in the release notes. Here we just highlight the main features.
Important Apache Kylin features
KYLIN-3654 – Real-time Extreme OLAP
With the newly introduced Kylin real-time receiver and coordinator components, Kylin can implement a millisecond-level data preparation delay for streaming data from sources like Apache Kafka. This means since v3.0 on, Kylin can support sub-second level OLAP over historical batch data, near real-time streaming as well as real-time streaming.
The user can use one OLAP platform to serve different scenarios. This solution has been deployed and verified in early adopters like eBay since 2018. For how to enable it, please refer to this tutorial.
KYLIN-3795 – Submit Spark jobs via Apache Livy
This feature allows the administrator to configure Kylin to integrate with Apache Livy (incubating) for Spark job submissions. The Spark job is submitted to the Livy Server through Livy’s REST API, instead of starting the Spark Driver process in local, which facilitates the management and monitoring of the Spark resources, and also releases the pressure of the nodes where the Kylin job server is running.
KYLIN-3820 – A curator-based job scheduler
A new job scheduler is added to automatically discover the Kylin nodes and do an automatic leader selection among them (only the leader will submit jobs). With this feature, you can easily deploy and scale out Kylin nodes without manually update the node address in
kylin.properties and restart Kylin to take effective.
KYLIN-3716 – FastThreadLocal replaces ThreadLocal
Using FastThreadLocal instead of ThreadLocal can improve Kylin’s overall performance to some extent.
KYLIN-3867 – Enable JDBC to use key store & trust store for https connection
By using HTTPS, the authentication information used by JDBC is protected, making Kylin more secure.
KYLIN-3905 – Enable shrunken dictionary default
By default, the shrunken dictionary is enabled, and the precise counting scene for high cardinal dimensions can significantly reduce the build time.
KYLIN-3839 – Storage clean up after the refreshing and deleting a segment
Clear unnecessary data files in a timely manner
To download Apache Kylin v3.0.0-alpha source code or binary package, visit the download page.
Follow the upgrade guide.
If you face issue or question, please send mail to Apache Kylin dev or user mailing list: firstname.lastname@example.org , email@example.com; Before sending, please make sure you have subscribed the mailing list by dropping an email to firstname.lastname@example.org or email@example.com.
Great thanks to everyone who contributed!