Skip to content

IDL and Data ctx nodes

Integrated Data Lake (IDL) and Data Contextualization (Data ctx) nodes provides a simple way to prepare data for establishing semantic correlations and data query processing. The following table describes the IDL and Data ctx nodes and their respective functions:

Node Name Image Description
list objects list objects This node allows you to list the objects from Integrated Data Lake
read object read object This node allows you to read the object content from Integrated Data Lake
write object write object This node allows you to write the object content to Integrated Data Lake
delete object delete object This node allows you to delete the object from Integrated Data Lake
subscribe object subscribe object This node allows you to subscribe the object in Integrated Data Lake
data ctx query results data ctx query results This node allows you to get results from Data Contextualization queries. If you select a query id, the node will return the latest result for both static and dynamic queries. If you select an execution job id, the node will return the result for the defined execution job. Either the query id or the execution job id has to be defined.
data ctx execution job data ctx execution job This node allows you to creates a Data Contextualization execution job for dynamic queries.
data ctx create query data ctx create query This node allows you to create a Data Contextualization query. You can either create dynamic or static queries.
data ctx ingest status data ctx ingest status This node allows you to get the status of Data Contextualization ingest jobs. You can query the list of all ingest jobs or a single one.

For more information about queries and execution jobs, refer to Data Contextualization.

Using Data ctx nodes

To use the Data ctx nodes with Integrated Data Lake, follow these steps:

Visual Flow Creator can now use data ctx nodes to execute the following tasks:

Create a query

To create a query using "data ctx create query" node, follow these steps:

1.In the node palette, select the "data ctx create query" node from the "IDL and data ctx" palette. 2.Select the "inject" node from the input palette and "Debug" node from the output palette. Connect these nodes with the "data ctx create query" node as shown in the image below:

Create query node

3.Double click the "data ctx create query" node to edit the properties:

  • Name: My new query
  • Description: Creating a query
  • Dynamic: Yes
  • SQL Statement: SELECT caseyear4 from sdi_test79 where caseyear4=:"caseyear4"
  • To save the properties, click "Done"

4.Save and Deploy.

The output is displayed in the debug window:

create query output

Execution jobs

To execute jobs using "data ctx execution job" node, follow these steps:

1.In the node palette, select the "data ctx execution job" node from the "IDL and data ctx" palette. 2.Select the "inject" node from the input palette and "Debug" node from the output palette. Connect these nodes with the "data ctx execution job" node as shown in the image below:

Execution job node

3.Double click the "data ctx execution job" node to edit the properties:

  • Name: execution job for WELLINGTON
  • Description: creating execution job
  • Query ID: 5ec3c9dc55e38921e78e1f66
  • Parameters: Name="caseyear4" and Value="WELLINGTON"
  • To save the properties, click "Done"

4.Save and deploy.

The output is displayed in the debug window:

Execution job output

Note

"data ctx execution job" node is only for dynamic queries.

Execute queries

To execute queries using "data ctx query results" node, follow these steps:

1.In the node palette, select the "data ctx query results" node from the "IDL and data ctx" palette. 2.Select the "inject" node from the input palette and "Debug" node from the output palette. Connect these nodes with the "data ctx query results" node as shown in the image below:

Result of query

3.Double click the "data ctx create query" node to edit the properties:

  • Name: data ctx query results
  • Query ID: 5ec3c9dc55e38921e78e1f66
  • Execution job ID: 5ec3ee5255e38921e78e1f79
  • To save the properties, click "Done"

4.Save and deploy.

The output is displayed in the debug window:

Query result node output

Ingest status

To get the status of ingest node using "data ctx ingest status" node, follow these steps:

1.In the node palette, select the "data ctx ingest status" node from the "IDL and data ctx" palette. 2.Select the "inject" node from the input palette and "Debug" node from the output palette. Connect these nodes with the "data ctx ingest status" node as shown in the image below:

Ingest node

3.Double click the "data ctx ingest status" node to edit the properties:

  • Name: data ctx ingest node
  • Job ID: 5ec3ee5255e38921e78e1f79
  • To save the properties, click "Done"

4.Save and deploy.

The output is displayed in the debug window:

ingest status output


Last update: June 5, 2024