Building a Metrics Store for Snowflake

In the era of big data, organizations across industries are inundated with vast amounts of data generated from various sources - be it user interactions on digital platforms, operational metrics from software applications, or performance indicators from hardware devices. Amidst this data deluge, the ability to efficiently collect, store, and analyze metrics data becomes crucial for informed decision-making, performance monitoring, and strategic planning. This is where a metric store plays a pivotal role.

In this article, our primary focus will be on outlining the process of constructing a metric store for Snowflake, utilizing Kyligence Zen. However, before we delve into the detailed steps, it's essential to first understand the concept and significance of a metric store.

Introduction to Metric Stores

A metric store is a specialized data storage architecture designed to handle the ingestion, storage, aggregation, and analysis of metrics data. Metrics data, often structured as time series data, encompasses a wide array of measurements or counts that are collected over time. These can range from user activity metrics, such as clicks and page views, to business essential metrics, like monthly sales revenue, daily stock prices, hourly temperature, quarterly profit growth rate, and beyond.

A metric store empowers users to codify, govern, and utilize business metrics directly from data warehouses, serving a broad spectrum of downstream analytics, data science, and business applications. Its primary function is to centralize metric definitions, enabling their consistent application across various analytical use cases. Ideally, it allows business users to define and maintain these metrics autonomously, while IT functions as the guardian of the infrastructure. This innovation extends the capabilities of standalone semantic layers by facilitating business user participation in metric definition and management, and by making these metrics accessible on a self-service basis for a wider range of use cases beyond traditional business intelligence (BI) and enterprise reporting.

Why Use Snowflake and Kyligence Zen?

Snowflake offers a cloud-based data warehousing solution that's scalable, easy to use, and supports a wide range of data analytics operations. Its architecture separates compute and storage, allowing users to scale resources dynamically according to their needs.

Kyligence Zen, on the other hand, is an AI-augmented semantic layer for big data and analytics. It provides a unified, consolidated view of data across various sources, simplifying data access and analysis. By building a metric store with Kyligence Zen on top of Snowflake, organizations can leverage the strengths of both platforms to achieve faster query performance, simplified data management, and advanced analytics capabilities.

Building a metric store on top of Snowflake with Kyligence Zen provides a powerful combination for handling large volumes of metrics data with ease. By following the below steps, organizations can set up a scalable, high-performance metric store that supports AI-driven analytics and data-driven decision-making.

Step 1: Setting Up Accounts

Setting up accounts for Snowflake and Kyligence Zen is remarkably straightforward, given their SaaS-based nature. You can swiftly initiate a free trial for Kyligence Zen through a simple, few-click process available on their website. This ease of access ensures that users can quickly explore the capabilities and features of both platforms without any hassle.

Step 2: Integrating Kyligence Zen with Snowflake

Kyligence Zen enables swift and straightforward integration with your Snowflake data. For the purposes of demonstration, we will opt to export data into CSV files. You have the flexibility to directly upload CSV files through Zen's web interface, or alternatively, you can place the files in an Amazon S3 bucket.

Unloading and saving valuable analytical data from Snowflake to Amazon S3 as CSV files is a straightforward two-step process. Once your data is securely stored, simply grant access to Kyligence Zen and establish a connection by following the steps outlined here: Connecting Data with Kyligence Zen. For those new to the platform, Kyligence Zen offers sample datasets, allowing beginners to delve into exploration and learning without the need to use their own data.

Step 3: Building and Optimizing the Metric Store

Kyligence Zen features an intuitive interface for defining metrics, simplifying the process even for those with complex logic requirements. It offers a range of built-in templates and functions, enabling users to easily add metrics that meet their business needs. For example, to define the annual growth rate of total sales, users can select the built-in year-over-year metric template, eliminating the need for manual SQL or code writing. Additionally, Kyligence Zen supports the definition of composite metrics, allowing users to derive and define new metrics using those already established.

It's noteworthy that throughout the entire process, users are not required to engage in preliminary model design and optimization tasks. Once the data upload is complete, they can proceed to define metrics. The creation and optimization of the data model are automatically handled by the underlying AI engine.

Step 4: Connecting to your analytical tools

It's time to delve into some data analysis and visualization work. After defining the metrics, users can proceed to data analysis with three available options:

1. Utilize Kyligence Zen's Built-in Agile Reporting and Analysis Features: This includes a variety of common chart types, supports custom dashboard creation, and enables sharing and collaboration with the team.
2. Employ Existing BI or Analytical Tools Such as Excel or Tableau. Kyligence Zen provides connectors that allow the defined metrics to be automatically synchronized with the user's existing reporting and analytical tools.
3. Use Kyligence Copilot: Users can interact with AI Copilot using natural language for interactive analysis, without the need for any technical or data analysis background. This leverages the capabilities of Generative AI to independently explore the value behind the data.

Get Started with Metrics Store Today

This guide facilitates a smooth and efficient startup process, unlocking the full potential of cloud-based data analytics. Begin your journey towards more intelligent and effective data management today.