ETL

The Extract, Transform, and Load (ETL) application converts raw data to a structured format. Structured data is helpful for analytical purposes, giving more meaningful insights.

Data feed is used to import data from sources and transform it into a structured format. After data import, you can use reporting sets to group and organize the data for reporting purposes.

The process can be divided into -

  • Extract - extract raw data from different sources using data import

  • Transform - transform raw data into a structured format to consume for analytics

  • Load - transformed structured data is stored in the data warehouse

You must have proper permissions to use various features in this application.

Data Feed

A data feed is a continuous stream of structured data. It helps users to get updated information from different sources. A data feed may stream continuously or as per request.

To create a new data feed:

  1. Enter a Feed Name and Source Name.

  2. From the Manage Fields section, click Add Field.

  3. Click Save.

You can view, edit, reactivate, and delete a data feed by clicking Options.

Import Data

The custom data import feature allows users to manually import data into the system by selecting a data feed and uploading a file in either .csv or .txt format. The import is successful when the selected file contains columns that match the data feed's definition.

To import data:

  1. Select Feed Name and upload a CSV or TXT file that contains actual data.

    The column headings and data type in the file must match with the field name and data type of the data feed.

  2. Click Submit.

You can view or delete the imported data from the list of imports.

Reporting Set

Reporting sets helps you to group and organize data for reporting purposes.

To create a reporting set:

  1. Enter a Reporting Set Name and Description for the reporting set.

  2. Select an existing Category or create new category and select it.

    A category is a logical group for grouping reporting sets of similar kind.

  3. Select Base Feed which is already created in the Data Feed.

  4. You can select simple raw fields to add as it is. Click Add Fields and then select fields from the Simple Fields page. Click Save to add selected fields to the reporting set.

  5. Click Next to define joined feeds and filters to exclude the data in the reporting set. If you want to create reporting set with fields from multiple data feeds, you need to join feeds.

    • Joined Feeds: Click + Add from the left pane to create a joined feed.

      1. Enter a Joined Feed Name and select Feed 1 and Feed 2.

      2. From Joined Fields, select Field 1 corresponding to Feed 1 and Field 2 corresponding to Feed 2 to make a join. Click + to add multiple fields.

      3. From Filters, select Field 1 corresponding to Feed 1 and Field 2 corresponding to Feed 2 to exclude based on added Comparison Value. Click Add to save filer criteria. You can add multiple fields to exclude data.

    • Dates: Click + Add from the left pane to add date fields in the reporting set.

    • Data Fields: Click + Add from the left pane to select fields for creating a reporting set.

      • If you want to create customized data field, click Advanced from Add Fields page and the select criteria to create a field. From the Configuration pane, you can create a new data field.

    • Employee Group: You can tag employee group to the reporting set. Select employee group from left pane and then select criteria from the right pane to tag it.

    • Contact Group: You can tag contact group to the reporting set. Select contact group from left pane and then select criteria from the right pane to tag it.

    • Exclude If: If you want some data to be excluded from the reporting set, click Add Exclude If from the left pane. Select the exclusion criteria from the right pane.

You can edit, reactivate, and delete a reporting set by clicking Options.

Data Recalculation

You can recalculate your reporting set data and metric values. For any import or change in the reporting set definition, you can recalculate data. This ensures your numbers always stay accurate and up to date. Also, if there are changes in the metric definition, you can recalculate the metric value.

To recalculate data:

  1. Select a Date Range to select a duration for which you want to recalculate metric values and reporting sets.

  2. Select metrics and reporting sets for recalculation.

    You can see a list view or a detailed view of the metrics and reporting sets.

  3. Click Recalculate to recalculate the selected metrics and reporting sets.

Once recalculation is completed, the status changed to Completed.