Query Usage Dashboard - A Much-Needed Data Product for Data Engineers

PMO & Data Engineer - John's Story

John is our PMO. He is dedicated to helping our PM, R&D, and QA team improve our product delivery efficiency. He also acts as a Data Engineer and Data Analyst, who builds the reports and dashboard with metrics that gauge the delivery efficiency and product quality.

However, reports and dashboards themselves do not effectively quantify John's direct value to the business stakeholders (R&D, QA lead). Improvement to the delivery efficiency and quality will be attributed to the business stakeholders, rather than John.

How can John justify the value he provides to the business team?

Our engineer coded this Query Usage Dashboard for John, initially as an internal "playground" - a side-project to help him identify what dimensions, metrics the business mostly cares about.

The data won't lie.

John can now quickly identify the most valuable data, dimensions, and metrics that business cares about, so he can prove his value - these dimensions, metrics, and indexes are all developed by John.

Now we have released this "playground" to the public for free on our website so that you can try it too!

Simply upload your SQL to quickly identify the most frequently used dimension, measures, and queries.

Why You Should Use It - Know Your SQLs, Improve the Data Model Performance

The primary value of this dashboard is to serve a lot more people like John, who wants to quickly demo their Data ER relations and to showcase the data's value.

This is not a rare need. We helped many data engineers, BI architects to build their Big Data Architecture. But there's one question our users have asked us hundreds of times by our users - "Is there one simple and quick way to demonstrate the value of my daily job?"

Now we can say "try this!" Here is a complete overview of your data assets.

By analyzing the SQL query histories, the tool can grasp all the data assets behind SQLs - the data models, dimensions, measures, indexes, and even the popularity of these data assets. All while being extremely simple and quick to use.

This Query Usage Dashboard is well-designed and downloadable. Whit our Query Usage Dashboard, you will no longer hesitate to showcase your work with your colleagues.

How to Use - How to Get SQL Files From Your Database, Upload Requirements

As this tool requires your daily SQL queries as raw material for "input", we'll guide you through how to export historical SQL queries from each major SQL database. Since the exported query statements do not contain the actual result data, you don't have to worry about data sensitivity issues.

If you are using Redshift, please see: Article

If you are using Snowflake, please see: Article

If you are using Google Bigquery, please see: Article

You can save the query log into a .txt or .sql file, then analyze it within the "playground".

Other Requirements for the SQL files to be uploaded and analyzed:

• The total file size cannot exceed 5 MB
• Please use the semicolon to separate multiple SQLs
• We recommend uploading more than 20 SQL queries to get a better result