Supported Browsers: Google Chrome (64.0.*)
1. To create your key pair using the Amazon EC2 console
You can create a key pair or import key pair. In this example, we create a new key and name it kcdemokey.
Note: This is the only chance for you to save the private key file. You’ll need to provide the name of your key pair when you launch an instance and the corresponding private key each time you connect to the instance.
2. Amazon Simple Storage Service (Amazon S3) Bucket
Kyligence Cloud will store your Kyligence Enterprise data and EMR logs in the S3 bucket you specify.
From the Amazon S3 console dashboard, choose Create Bucket.
1. Search for Kyligence Cloudin AWS Marketplace search bar, go to the Kyligence Cloud product introduction page, and click Continue to Subscribe to start to subscribe and configure the Kyligence Cloud service.
2. When subscribing for the first time, you must accept the terms and conditions by selecting the Accept Terms button.
3. The following Thank you for subscribing message will be displayed. Click Continue to Configurationto review, modify and launch your Kyligence Service.
4. Please select Product Deployment for the Fulfillment Option column. Software Version please try to select the current latest version. In this example, we use 1.1 (May 08, 2019), and the Region should be consistent with the Amazon VPC region you created. In this example, we use US East (N. Virginia) and click Continue to Launch to the next configuration page.
5. The Launch page is then displayed summering the Configuration Details selected from the previous screen. Choose Actionplease select Launch CloudFormation and click Launch to proceed to the next step.
6. On the Specify templatepage, we have provided the configured template, you can choose the Template is ready for the Prerequisite – Prepare template and you don’t need to change anything, then click Next to enter the Specify stack details page.
7. Please configure the following information in order:
8. Set your own settings depends on your needs on the Configure stack options page and click Next.
9. On the Review page, please configure options setting according to your needs. After confirming your information on the Review page, check the box of I acknowledge that AWS CloudFormation might create IAM resources. Click Create stack to start deploying the Kyligence Cloud service.
10. You can see the stack in the list on the Stacks page on the AWS CloudFormation console. It usually takes 5-10 minutes to deploy EC2 and start the Kyligence Cloud service. After the creation is complete, you can check the created Amazon EC2 instance on the Instances page on the Amazon EC2 console.
11. This instance is named as “kyligence-cloud-server” by default, it is recommended that you name the instance for management, in this case, we named demoec2.
12. Associate Elastics IP Address with your instance. You can choose to Allocate new address or use the existing address. In this example, we allocate a new address.
Step 1：Create an Elastic IP Address for Your NAT Gateway
Step 2: Associate address
13. Set security group. Select the instance demoec2 and click the Security groups below to enter the demoec2 security group page.
In the security group’s Inbound rules, click Edit. Edit Source in Port Range 8080 and 22 to let the following IP address can access Kyligence Cloud service as shown below. Then click Save.
13. To access the Kyligence Cloud service using an external network, please select instance demoec2 and check the IPv4 Public IP below. Please input [IPv4 Public IP]:8080 in the browser address bar to enter the Kyligence Cloud login page, using the username and password you set on the Specify stack details page.
Note: It takes 5-10 minutes to start Kyligence Cloud service.
This chapter will talk about how to create a new EMR by Kyligence Cloud. If you already have a running EMR cluster on AWS and want to create a new cluster based on it, please refer to this link: Using existing EMR.
1. In the cluster list page, click the + Cluster button. In the pop-up box, choose Deploy with a new Hadoop cluster.
2. Fill in the Cluster Name, select the EMR Version. Set a brand new name for the cluster that hasn’t been used before. For EMR Version, we choose emr-5.23.0 as an example.
3. You can choose Spot Instance and Auto Scaling to further optimize costs on AWS. You can also choose to enable HA. You can check the Kyligence Cloud manual for the introduction.
4. You can create a key to access the cluster by clicking +Credential on the right side of the AWS Key Pairs information bar. You can choose to create a brand-new key or import your existing key.
Note: If you choose to create a new key, remember to save the two key documents (both the private and public keys). This is the only chance for you to save the key file. If you can’t download two key documents, check the web browser to see if they have been blocked.
5. You can choose any of the presets in the drop-down list as the Primary Subnet or Secondary Subnet. Primary Subnet and the Secondary Subnet shouldn’t be the same.
If there is no additional need, please remember to choose public subnet for the Primary Subnet and choose primary subnet for the Secondary Subnet.
6. Select the primary network, secondary network, and S3 buckets that you created in the AWS Management Console.
Tag Editor: You can tag the cluster resources created by Kyligence Cloud with the Tag Editor. Check Tag Editor, then you can add tags or edit existing tags via the + Tag button on the right side of the tab bar.
Elastic Load Balance: Check Elastic Load Balance to improve service stability.
7. In Kyligence Enterprise Setting. Select the build-in Kyligence Enterprise. You can also choose to install the self-service analysis tool Kyligence Insight.
8. Finally, click the Submit button in the lower right corner of the page to create a cluster, it displayed in the cluster list.
9. Start the cluster by clicking the Start button. When the cluster state changes to RUNNING, click Launch to log in Kyligence Enterprise.
Note: It takes 25 to 40minutes to start a cluster, depending on the cluster scale.
You can stop the cluster by click the Start/Stop Cluster button.
Kyligence Cloud supports scaling cluster automatically or manually. You can also set Auto scaling. Please refer to Kyligence Cloud user manual for details.
After click Launch Kyligence Enterprise service, a window of Update License will pop up when you open the web interface of Kyligence Enterprise. Click Apply Evaluation License to apply for your Kyligence Enterprise’s license. After the information required has been submitted, a one-month trial license will be effective immediately. If you have already gotten the Kyligence Enterprise’s license, you can upload the file to activate it.
The initial username is ADMIN and the initial password is KYLIN, the same as Kyligence Insight.
1. Kyligence Enterprise support to use Hive as the default data source. Use the provided SSH and the Key you created/use in Kyligence to log in to the edge node.
a. Change access rights: chmod 700 ~[ your private keypair directory]
b. Log in to the edge node:ssh -i ~[your private keypair directory] ec2-user@[your Kyligence Enterprise Instance IP]
c. Then command yes to continue connecting.
2. Then you can import the Kyligence Enterprise built-in sample data into Hive using executable scripts. Search for the storage path of the script’s directory. Its default storage path is tmp/kyligence/ke/[the file under ke]/bin.
cd [the file under ke]
3. Run the script to load the sample data. The script is sample.sh
4. After sample.sh is executed, it is required to choose Reload Metadata under the System page.
1. After importing the sample data, you can access learn_kylin project and build kylin_sales_cube.
In the left navigation bar, click Studio -> learn_kylin -> Cube. Then, you will see a sample cube kylin_sales_cube in the learn_kylin project.
2. The cube is in DISABLED status. Click Build to build it before query.
3. Pick an End Time like 2014-01-01 and click Submit. Kyligence Enterprise will start a building job.
4. You can monitor the build progress by clicking Monitorin the left navigation bar. The build can take about 30 minutes, which depends on your cluster size. When the progress achieves 100%, the cube status will be changed to Ready.
Click Insight in the left navigation bar. Input the following SQL to query the OLAP cube:
FROM KYLIN_SALES as KYLIN_SALES
INNER JOIN KYLIN_CAL_DT as KYLIN_CAL_DT
ON KYLIN_SALES.PART_DT = KYLIN_CAL_DT.CAL_DT
INNER JOIN KYLIN_CATEGORY_GROUPINGS as KYLIN_CATEGORY_GROUPINGS
ON KYLIN_SALES.LEAF_CATEG_ID = KYLIN_CATEGORY_GROUPINGS.LEAF_CATEG_ID AND KYLIN_SALES.LSTG_SITE_ID = KYLIN_CATEGORY_GROUPINGS.SITE_ID
Kyligence Enterprise will return results in sub-seconds.
Kyligence Enterprise supports Direct Query of leading BI software, such as Tableau, Excel, Power BI, MicroStrategy, Qlik, and Cognos. In this section, you will learn to connect with Tableau Desktop using Kyligence self-developed ODBC driver for visual analysis.
Currently, Kyligence ODBC Driver has versions of Windows 64bit/32bit and Linux 64bit/32bit. In this section, we take the version of Windows 64bit as an example to introduce how to install Kyligence ODBC driver (Windows 64bit version): Download Kyligence ODBC Driver (Windows 64bit) in Download and install it. If you have previously installed Kyligence ODBC driver, please uninstall it first.
1. Open ODBC Data Source Administrator: select Control Panel-> Administrative Tools to open ODBC Data Source Administrator.
2. Switch to System DSNtab, click Add and select KyligenceODBCDriver in the pop-up driver selection box, then click Finish.
3. In the pop-up window, input the Kyligence Enterprise server information:
Where the parameters are described below:
4. Click Test: Once it connects to the data source successfully, the following dialog will appear, clickOK to save the settings.
Once you have the Kyligence ODBC Driver in the environment where you have your Tableau Desktop installed, you can follow the following steps to analyze data from Kyligence Enterprise in Tableau Desktop.
If you want to use other BI tools to connect Kyligence Enterprise, you may skip this session and check Read More part.
1. Export TDS Files from Kyligence Enterprise
In the left navigation bar, click Studio -> Cube to select a READY cube in learn_kylin project. Click Export TDS in More Actions to download a TDS file.
2. Analyze the Sample Cube
For more detailed information about how to use Kyligence Cloud and Kyligence Enterprise, please refer to http://docs.kyligence.io.