Apache Kylin Through the Eyes of the Founders – Episode Four

This article is part of a series of conversations with the founding members of Apache Kylin and Kyligence on the origins of Apache Kylin. You can find the first three installments here: Episode One, Episode Two, Episode Three. 

Episode Four: Happy Memories  

Apache Kylin is an open source distributed analytical data warehouse built with big data in mind. Via a clever combination of multi-dimensional cubes, plug-in architecture and precomputation technology, Kylin can provide near-constant query speeds no matter the size of your dataset with sub-second latency – cutting costs for adopters of this technology in both time and manpower needed for effective analysis. 

This recipient of multiple industry awards has been adopted by over a thousand organizations worldwide seeking a solution to the problem of storing and analyzing big data fast enough for their insights to make an impact on their business. 

This is the origin story of the unexpected hero of modern big data analytics, Apache Kylin, as told by its inventors. 

In this part of our conversation on the origins of Apache Kylin with its founding members, I want to hear about the good times. Over the years, the six of you have spent a lot of time together working on Kylin, an open source, top-level project with the Apache Software Foundation.  

What has been your favorite moment or memory of the time you’ve spent working on Kylin? What stands out to you? 

J A S O N:  I have many, many favorite memories from working on Kylin. Most of the time, the main reason why I enjoyed the project was our team. They are so talented. In my mind, they can solve every technical issue they come across and nothing can block us. I joined the team because I thought they were great and because I believed that we could do something that would really help others. We have the technical understanding and we are in this together. I liked them, their character, so I enjoyed working with them.  

It’s not about one or two moments – most of the time I spent working on Apache Kylin was time I enjoyed. We all wanted to do something different and interesting, something that could change things and make a difference in the data industry. So, it was a very exciting thing to know what Kylin, and later Kyligence, is doing is really helping many people. I could see the influence of our work spreading, which inspired me to join Kyligence so I could help even more people and solve many more problems. 

H O N G B I N:  I feel the same way. The happiest memories I have of Apache Kylin were when I first joined this team. I am very passionate and, when this project started, I was very young; I remember I spent many weekends coming into the eBay office to fix Apache Kylin problems and answer community emails. That was a very happy time for me. I wanted to join Kyligence because I wanted to try something different and I wondered how this project would evolve. I had no idea, so I wanted to see if we could push this project to the next level by starting a business. I was curious to see what we could accomplish.   

Y A N G:  The opportunities that come with working on this project have allowed us all to accomplish a lot, both personally and technically. Before Apache Kylin, I never had a chance to stand in a spotlight with international influence. I didn’t go to data summits, or talk much, or introduce my work to the world. Apache Kylin gave me that chance. Luke and I attended events quite frequently, I would say, between the years of 2014-2016. That’s a very important period of personal growth for me – I was a technical geek who spent nearly 24 hours per day at home, and I became someone who could stand up for a piece of technical work I created and introduce it to the globe. That’s a very significant change about myself, and it helped both me and Kylin.   

L U K E:  Absolutely! The first time Yang and I co-presented on the Apache Kylin project was at the Strata Hadoop conference in London in 2015. This was the first time we presented to the whole community. We felt a lot of momentum from talking to different people face to face and got a lot of helpful feedback. That is one of my favorite times from Apache Kylin.  

My second favorite memory is the Apache Kylin Meetup in Beijing in 2017. Doug Cutting, the author and father of Hadoop, went to Beijing with us and a lot of our friends. We formed this Meetup and Doug came to our event and delivered a talk. He is very famous in the community and supports us very much, so this was a really great experience and opportunity for Kylin. 

S H A O F E N G:  Another high point for us was when we started working with Meituan. Meituan is the biggest takeaway business in China with more than 100 million active users. One day in 2015, they contacted us and let us know they were evaluating OLAP technologies and found that Kylin was a very good project that matched their requirements, so they wanted to join the project and use Kylin as their platform. We met with them immediately to discuss their needs and concerns. This was a highlight for us because we knew that someone outside of eBay wanted to heavily use Apache Kylin. Meituan is one of the biggest Kylin users to this day – they still maintain a very big Kylin cluster. 

D O N G:  My most favorite moments are when I was giving a speech at global conferences such as Strata Data Conference New York or Singapore. We got to meet a lot of players in the big data technology industry. We saw that there are a lot of huge companies, like Cloudera or Microsoft, that also have their own innovative startup companies, but Kylin still held the leading position in the industry. We were very proud of that and we could see that our contributions could change the world. More and more we were being recognized by the industry and we now have users from around the globe. In short, when I found out that Kylin was in the leading position in big data technology, I felt honored to be a member of Apache Kylin. 

Those are some wonderful stories. Next time we speak, I’d like to hear about what you’ve learned from working on this project, both technically, professionally, and personally.  

Stay tuned for our next episode! If you missed the beginning of this story, check out Episode One, Episode Two, and Episode Three. 

Additional Resources 

Q&A with Apache  Kylin  Committer, Kaige Liu – How Apache  Kylin  Is Rapidly Changing the Way We Approach Big Data  

Learn About Real-Time Streaming - What’s New with Apache Kylin 3.0?  

4-Part Series on Count Distinct – Making Distinct Counting Work for Big Data  

Getting Started with Apache Kylin  

Get the Most From Your Kylin Deployment with These Resources  

Further Reading Is Available on Our Apache Kylin Blog  

The Apache Software Foundation  

Apache Kylin  

Roaring Elephant Podcast with Dong Li – Episode 93 – Apache Kylin: Extreme OLAP Engine for Big Data  

About the Founders 

Luke Han is the Co-Founder and CEO of Kyligence and Co-Founder of Apache Kylin, the first Apache Software Foundation top-level project developed in China. He is responsible for Kylin's strategic planning, development roadmap, product design, and more, and is committed to developing the Apache Kylin global community and ecosystem. He has served as Head of Big Data Products in eBay's Global Analytics Infrastructure Division, Chief Advisor to Actuate China, and Technical Director of Power Excellence East China.

Yang Li is the Co-Founder and CTO of Kyligence, Co-Founder of Apache Kylin, and member of the Project Management Committee (PMC). Previously, he was the Senior Architect of Big Data in eBay's Global Analytics Infrastructure, Vice President at Morgan Stanley, and during his time with IBM, he received the Outstanding Technology Contribution Award. Yang has more than 10 years of hands-on experience in big data analytics; he has focused on parallel computing, data indexing, relational mathematics, approximation algorithms, compression algorithms, and other cutting-edge technologies. Over the past 15 years, Yang has directly driven the development of OLAP technology in the big data space. 

Dong Li is the Founding Member and Senior Director of Product and Innovation at Kyligence, an Apache Kylin Core Developer (Committer) and member of the Project Management Committee (PMC) where he focuses on big data technology development. Previously, he was a Senior Engineer in eBay's Global Analytics Infrastructure Department, a Software Development Engineer for Microsoft Cloud Computing and Enterprise Products, and a core member of the Microsoft Business Products Dynamics Asia Pacific team where he participated in the development of a new generation of cloud-based ERP solutions. 

Shaofeng Shi is a Partner and Chief Software Architect at Kyligence, Apache Kylin Core Developer (Committer), and Chairman of the Project Management Committee (PMC Chair) where he focuses on big data analytics and cloud computing technologies. Previously, he was a Senior Data Engineer in eBay's Global Analytics Infrastructure Department and a Cloud Computing Software Architect at IBM. 

Hongbin Ma is the Vice President of Research and Development at Kyligence, an Apache Kylin Core Developer (Committer) and member of the Project Management Committee (PMC) where he focuses on big data infrastructure and platforms. He joined eBay as Apache Kylin's Chief Committer. Previously, he was a core contributor to Trinity, Microsoft's Asian Research Institute's graph database. He has contributed to Apache Kylin's storage engine, query optimization, test coverage, and other areas and is currently the technical leader of Kyligence Enterprise data warehouse products. 

Jason Zhong is a Partner and Senior Director at Kyligence, an Apache Kylin Core Developer (Committer), and a member of the Project Management Committee (PMC). He has worked in eBay's Global Analytics Infrastructure Division and been involved in operational automation product development as well as Kylin's development. After joining Kyligence, he worked in both research and development before becoming responsible for business sales and business development transformation. He has won consecutive Kyligence sales titles and is currently the Head of the Kyligence South Division. 

About the Author

Samantha Berlant is the Marketing Communications Manager at Kyligence and a big fan of AI, machine learning, and science-fiction. She spent several years leading content analytics projects at Facebook and Instagram and has been a writer and editor for over a decade. Samantha believes in the power of accessible data and her favorite Star Trek character is, coincidently, Data.