Skip to content

Data Transformation Actions

The actions you can perform with an ETL flow display in the ETL Flows table in the Actions column. See the Getting Started with Data Transformation topic for more information on user roles and access to ETL flow functionality.

ETL Flows Table

Available actions display when you click the three ellipses in the Actions column in the ETL Flows table, as shown here:

Note

Only the actions permitted by a user's Data Transformation role display in the actions drop-down list.

Actions

The actions available for an ETL flow display according to your user role. All available actions are:

  • View: see the ETL flows table
  • Run: initiate an ETL flow
  • Schedule: configure an ETL flow to run on a schedule
  • Edit: make changes to an existing ETL flow
  • Copy: copy an ETL flow
  • Delete: delete an ETL flow from the ETL Flows table
  • Latest Data: retrieve only the latest data when running an ETL flow
  • Cancel: cancel a currently running ETL flow

Viewing ETL Flows

When you open Data Transformation from the Launchpad, the ETL Flows table is the main feature of the Data Transformation page. All user roles can view the ETL Flows table.

ETL Flows Table Illustration

flowsactions

How to View ETL Flows

Click the Data Transformation icon on the launchpad. The Data Transformation page opens and displays the ETL Flows table.

Running an ETL Flow

Depending on the complexity and volume of the data involved and the time it takes to provision the resources, it can take from one minute to two hours to execute an ETL flow.

How to Run an ETL Flow

Follow these steps to initiate an ETL flow:

  1. Select the row in the ETL Flows table for the ETL flow you want to run.
  2. Click the three ellipses in the Actions column.
  3. Select Run from the Actions drop-down list. The running flow details display in the ETL Flow Details panel.

Scheduling an ETL Flow

You can schedule an ETL flow to run automatically on one of the following bases: - Hourly - Daily - Weekly - Monthly

Schedule Dialog Example

scheduling

How to Schedule an ETL Flow

Follow these steps to schedule an ETL flow:

  1. Select the row in the ETL Flows table for the ETL flow you want to schedule.
  2. Click the three ellipses in the Actions column.
  3. Select Schedule from the Actions drop-down list.
  4. Slide the 'Enable schedule' toggle to 'On'.
  5. Select an hourly, daily, weekly, or monthly frequency.
  6. Select a value for when to run the flow; i.e., hour, day, date, etc.
  7. Click Apply. A 'Success' message displays.

Editing an ETL Flow

Once a flow is created, you can access it in the ETL Flows table and use the Edit feature to access the script and make any necessary changes.

How to Edit an ETL Flow

Follow these steps to edit an ETL flow:

  1. Select the row in the ETL Flows table for the ETL flow you want to edit.
  2. Click the three ellipses in the Actions column for that row.
  3. Select Edit from the Actions drop-down list. The ETL flow opens for editing.
  4. Make any edits you need to the ETL Flow Payload script.
  5. Click Save. The system displays a 'Success' message.

Copying an ETL Flow

Follow these steps to copy an ETL flow:

  1. Select the row in the ETL Flows table for the ETL flow you want to copy.
  2. Click the three ellipses in the Actions column for that row.
  3. Select Copy from the Actions drop-down list. The ETL flow opens.
  4. Enter a name for the copied ETL Flow.
  5. Click Save. The system displays a 'Success' message and the newly-named flow appears in the ETL Flows table.

Deleting an ETL Flow

Follow these steps to delete an ETL flow:

  1. Select the row in the ETL Flows table for the ETL flow you want to delete.
  2. Click the three ellipses in the Actions column for that row.
  3. Select Delete from the Actions drop-down list. An 'Are you sure?' message displays.
  4. Click Okay. The system displays a 'Deleted' message.

Pulling Only the Latest Data for an ETL Flow

Instead of pulling all of the data from a data source each time you run an ETL flow, you can pull just the new data since the last run. Once the 'Latest' check box is checked each time the ETL flow runs, it continues to pull only the latests data until unchecked.

ETL Flow Run Illustration

This image shows the Details panel, Latest check box, Status column, and the progress spinner and Cancel option that only display for a currently-running ETL flow.

runningflow

How to Pull Only New Data (Since Last Run)

Follow these steps to pull only data that is new since the last ETL flow run:

  1. Select the row in the ETL Flows table for the ETL flow you want to only get the latest data for.
  2. In the right details panel, select the 'Latest' check box.
  3. Click the three ellipses in the Actions column for that row.
  4. Select Run from the Actions drop-down list. The date and time display in a new line in the Flow Runs list, along with a progress icon in the status column that spins until the ETL Flow finishes running.

Info

The Progress icon spins while a flow is running. Only currently running flows can be cancelled. See image below.

Canceling an ETL Flow

Click the Cancel link to cancel a currently-running ETL flow, as shown here:

Last update: January 25, 2024

Except where otherwise noted, content on this site is licensed under the Development License Agreement.