Data Assets Dashboard - A Much-Needed Data Product for Data Engineers

Author
Coco Li
Product Marketing Manager
Dec. 08, 2021

PMO & Data Engineer - John's Story

John is our PMO. He is dedicated to helping our PM, R&D, and QA team improve our product delivery efficiency. He also acts as a Data Engineer and Data Analyst, who builds the reports and dashboard with metrics that gauge the delivery efficiency and product quality.

However, reports and dashboards themselves do not effectively quantify John's direct value to the business stakeholders (R&D, QA lead). Improvement to the delivery efficiency and quality will be attributed to the business stakeholders, rather than John.

How can John justify the value he provides to the business team?

Our engineer coded this Data Assets Dashboard for John, initially as an internal "playground" - a side-project to help him identify what dimensions, metrics the business mostly cares about.

 
Kyligence Data Assets Dashboard_top 5 queires

Example of what data is queried the most by data consumers

The data won't lie.

John can now quickly identify the most valuable data, dimensions, and metrics that business cares about, so he can prove his value - these dimensions, metrics, and indexes are all developed by John.

Now we have released this "playground" to the public for free on our website so that you can try it too!

Simply upload your SQL to quickly identify the most frequently used dimension, measures, and queries.

 
Data Assets Dashboard_demo

A demo showing how the dashboard is generated
 

Why You Should Use It - Know Your SQLs, Improve the Data Model Performance

The primary value of this dashboard is to serve a lot more people like John, who wants to quickly demo their Data ER relations and to showcase the data's value.

 
Data is an asset when it is organically organized

Data is an asset when it is "organically organized"

This is not a rare need. We helped many data engineers, BI architects to build their Big Data Architecture. But there's one question our users have asked us hundreds of times by our users - "Is there one simple and quick way to demonstrate the value of my daily job?"

Now we can say "try this!" Here is a complete overview of your data assets.

 
Kyligence-sample-Data-Assets-Dashboard- top 5 queries

A sample Data Assets Dashboard generated automatically

By analyzing the SQL query histories, the tool can grasp all the data assets behind SQLs - the data models, dimensions, measures, indexes, and even the popularity of these data assets. All while being extremely simple and quick to use.

This Data Assets Dashboard is well-designed and downloadable. Whit our Data Assets Dashboard, you will no longer hesitate to showcase your work with your colleagues.

 

How to Use - How to Get SQL Files From Your Database, Upload Requirements

As this tool requires your daily SQL queries as raw material for "input", we'll guide you through how to export historical SQL queries from each major SQL database. Since the exported query statements do not contain the actual result data, you don't have to worry about data sensitivity issues.

If you are using Redshift, please see: Article

If you are using Snowflake, please see: Article

If you are using Google Bigquery, please see: Article

You can save the query log into a .txt or .sql file, then analyze it within the "playground".

Other Requirements for the SQL files to be uploaded and analyzed:

  • The total file size cannot exceed 5 MB
  • Please use the semicolon to separate multiple SQLs
  • We recommend uploading more than 20 SQL queries to get a better result