Sample Use Case
This page provides you a sample use case so you can see how Datameer basically works.
You will be guided through:
- Logging into Datameer
- Creating a new Project
- Adding tables/ data to a Project
- Performing a Join operation
- Performing an Aggregate operation
- Publishing the View/ Table to Snowflake
Logging into Datameer#
At the beginning we have to login to access our Datameer instance. We open a Browser, in our case Google Chrome, and enter our Datameer URL. We enter the "Username" and "Password" and confirm with "Log In".
Creating a New Project and Adding Data or Tables to the Project#
Since it is our first time using Datameer, the first step is to create a Project wherein we will add your Snowflake datasets and schemas and perform our transformations. After you log in, you are redirected to the Project Overview page and on there click the "+ NEW PROJECT" button to create a new Project.
Then enter a suitable Project name and confirm with "OK".
What we now see is the Workbench of the created Project page with the Data Browser with all available data on the left and the Flow Area (without any data at this moment) in the middle.
To add some data, we navigate through the Data Browser and add for our use case the 'NATION' table by clicking on the "+" next to the table name.
In general, you can add any data or table that is presented in the Data Browser. Once the data is added to the Project and therefore added to the Flow Area, you can remove it again via the context menu or by clicking "-" next to the data name in the Data Browser. The Inspector on the right side presents several information about the dataset, that is marked in the Flow Area.
Let's now start to perform our first transformation. For that, we add the 'SUPPLIER' dataset by clicking the "+" in the Data Browser.
Performing the Join Operation#
Our goal for the first transformation in this sample use case is to join both source datasets 'NATION' and 'SUPPLIER' by their 'NATIONKEY' columns.
First we click on the "+" icon of the 'NATION' source and select the light data preparation operation "Join". The 'Join Configuration' view opens. On the left side the data preview for the source is displayed. On the right side, we can configure the Join operation.
We have already selected 'NATION' as the first source. That's why it appears on the left 'Sources' side and we need to select the second source from the dropdown on the right side next. The dropdown provides the 'SUPPLIER' dataset and we select it.
The best matches for joining both datasets are calculated and after a short time displayed. Next, select the Join mode "inner join".
Now we have two options: We can click on the '+Use Suggested Columns' button to select the columns or we can select the columns manually from the 'COLUMNS' section on the right side. After that, we confirm with "Apply" and close the configuration.
Finally, we are guided back to the Project's Workbench and view and explore our 'NATION 2' view. Both Join sources are connected to the new view with arrow lines. Viewing the new Datameer view in a highlighted square means that an operation has been applied. The counter in the 'NATION 2' node indicates that only one operation has been applied. The Data Grid provides both the columns as well as the data preview. On the right side we can track the transformation process.
Performing the Aggregate Operation#
After joining the two sources we now want to perform another transformation. We want to aggregate the new Datameer view and group by the 'N_NAME' and afterwards use the account balance column 'S_ACCTBAL' as the measure.
There are two options to apply the operations. The way you perform the transformation might differ depending on how many operations you want to apply in total.
Option 1 - Creating a new view based on an existing view
We start from the new 'NATION 2' view and click on the "+" and select "Aggregate". The 'Aggregate' view opens and on the left side we can see our columns.
Next, we click on the "+" next to 'Group Bys' and select the column we want to sort after. In our example, we mark the 'N_NAME' entry and confirm with "Apply".
Then we click on "+" next to 'Measures', mark the 'S_ACCTBAL' and confirm with "Apply".
We can now finish the aggregate operation and confirm with "Apply".
What we see as our Aggregate result is the following: In our Flow Area we have the two original Snowflake sources and the joined Datameer view we performed first. The new highlighted square that is named 'NATION 2 2' is the new aggregated Datameer view. We can later on rename the new view in the view's details, but for now we leave it as it is. The counter in the 'NATION 2 2' node indicates, that only one operation has been applied to the view. The connected arrow line illustrates that this new Datameer view is based on the former Datameer view.
Option 2 - Creating a new view by adding another operation to the recipe
The second option to perform a transformation can be done by adding the operation directly to the recipe. This requires to already have at least one operation applied to a transformed Datameer view.
Now we mark the former 'NATION 2' view and simply click on "+ Add to Recipe". Now the operation overview opens and we select the "Aggregate" operation.
The 'Aggregate' view opens and we can execute the further steps analogous to the option 1. Confirm with "Apply".
What we now see as the result is that our former view 'Nation 2' has an increased indicator from '1' to '2'. Furthermore we can see all applied operations in the operation stack on the right side as part of the transformation recipe. The operations are listed in the order in which they were performed. The most recent operation is at the bottom.
Publishing the View/ Table to Snowflake#
We are almost done. Finally we want to publish our view/ table in Snowflake. Find more information about how to publish here.
To do so, we click on "Publish to Snowflake" on top of the Flow Area.
The 'Publish to Snowflake dialog' opens and provides the publishing configuration. In general, publishing is possible as a view or as a table. For our sample use case, we select table. We could now rename our asset first and then select our Snowflake destination by selecting the destination from the list below. Finally we confirm with "Publish Data".
After a few moments, the publishing process is finished and we can see our published view in the Flow Area highlighted in green. The arrow line connects the view we created in Datameer with our published Snowflake view/ table.
Congratulations! We made it.