Leveraging Disruptive Technology without Disrupting Your Decision Makers

As technology continues to evolve rapidly, many industries are being disrupted and opportunity abounds for forward-thinking organizations. Amazon, Uber, Airbnb, and Netflix are the most commonly cited examples of this disruption, but the benefits of innovating based on data can be seen in every industry.

However, there is a risk that organizations attempting to undergo digital transformation can disrupt the normal flow of their business operations significantly. I have heard numerous cases of companies that create “data blackouts” for their decision makers when attempting to deploy new technology.

The past few years have been very exciting for those of us in the data ecosystem. Digital transformation within all types of businesses has generated mountains of data in a variety of forms and formats. Rapid innovation within the field of data management has made it possible for organizations to keep pace by providing the power to process petabytes of data quickly and efficiently.

This combination means that businesses have the power to truly leverage their data to make better decisions. The insight from data is critical when responding to a rapidly changing global economy and competing in today’s market.

One of the biggest challenges that organizations face today is finding a balance between the technical skills required to manage these vast amounts of data and the business acumen to leverage that data. For many years now, Excel has been a critical tool for business users when working with data. The pivot table has given business analysts and decision makers the ability to interact with data on their terms and gain insight into what is really happening in their business.

Unfortunately, the technologies that made pivot tables possible simply cannot handle the volumes of data that modern systems are generating. A new, more scalable approach to storing and processing data for analysis was required in order to continue to provide the flexibility and ease-of-use of pivot table across billions of rows of data. Kyligence has developed a way to power pivot tables with sub-second response time across today’s growing data volumes.

One of the most powerful features of a pivot table the ability that it provides to users to slice and dice the data along a number of dimensions. In order to leverage this power, the data needs to refresh almost instantly when a user interacts with the controls.

Not only are slow-running queries frustrating to users, they actually reduce the effectiveness of the analysis. To optimize decision making, those making the decisions need to be able to analyze and compare multiple combinations and permutations within their data.

A New Approach for Handling Scale

The engineering team at Kyligence has been working with extreme data volumes for a number of years now. They began within the Hadoop ecosystem and figured out a way to build an OLAP data structure that provided ultra-fast query performance for large datasets.

That work ultimately led to the development of Apache Kylin, which has been adopted by organizations around the world that need sub-second query performance. Kyligence was created to provide an enterprise solution built on Kylin that adds high-availability, security and AI-augmented auto-modeling to the core solution.

As our team was analyzing the ways that users were accessing their data, it became clear that support for pivot tables was a key requirement for our customers. Enabling an expert Excel user to seamlessly move from analyzing thousands of rows to being able to analyze billions of rows is truly a game-changing development.

As we analyzed the market and met with numerous organizations, it became apparent that many of them had created extremely complex systems to power their users' pivot tables. Many data workers were forced to extract aggregates from their data warehouse or data lake and then import them into Excel to enable the analysis requested by data consumers within the business.

Kyligence’s advanced aggregation engine provides the ability to enable pivot table functionality across extremely large datasets without over complicated modeling or fragile data pipelines. Even better, the same semantic model can also be used to power Tableau, MicroStrategy, or any other BI tool with MDX capability.

On the other side, Kyligence can pull data from any modern data lake including both on-premises and Cloud. Defining a model from any of the sources is simple, and the application is able to learn from both historical queries and active queries using advanced machine learning to understand how your organization uses the data.

Winning with Data

The ability to create and tune the models that define the aggregation strategy provides benefits for both IT and business users. For IT teams, it eliminates the need to manage semantic models which can become an extremely time-consuming task. For business users, this functionality provides responsive updates as business users interact with the data.

This creates a situation where both groups can be more efficient and significantly reduces the need for business users to wait for limited IT resources to service their requests. Organizations where decision makers have access to the data they need have a significant edge in today’s economy.

While forcing business users to adapt to new tools presents a sizeable challenge, even more costly is the process of re-creating reports and dashboards during a migration. Even moving between data platforms with similar syntax can result in hundreds or even thousands of hours of tedious work migrating content.

The beauty of Kyligence’s MDX is that it operates exactly like other MDX platforms so that no migration is needed. And better yet, there are no drivers to worry about updating for both client and server application. A simplified and accelerated migration to your next generation, scalable platform can result in a true business advantage.

Disruptive innovation is helping organizations of all shapes and sizes transform the way they operate and leverage data to be more successful. However, many organizations create chaos in the process of trying to innovate, costing them significant time and resources.

Kyligence offers a way to leverage the benefits of data lakes without disrupting the users who rely on data every day to make important decisions. If you have a data lake and are ready to see how easy it is to enable your Excel users, you can get started with a free trial of Kyligence Cloud or Kyligence Enterprise and experience the power for yourself.