Integrated Data Lake nodes¶

Usage of Integrated Data Lake nodes¶

The Integrated Data Lake is a repository that allows you to store structured and unstructured data or objects in its native format as needed. It handles large data pools for which the schema and data requirements are not defined until the data is queried.

Integrated Data Lake nodes allows you to list, read, write, delete, subscribe and query the files or objects from Integrated Data Lake.

List objects¶

The "list objects" node lists the files or objects from Integrated Data Lake and stores the data in the message payload. Additionally, you can filter the files or objects by "Sub-tenant" property. It is possible by the main environment.

Read object¶

The "read object" node reads the file or object content from the specified path and place(s) the content in the message payload. The mode parameter defines the type of content to be read from the file or object. The read object mode parameters are given below:

• Object: Reads only the content of the file or object.
• Object+Metadata: Reads both content and metadata of the file or object.
• Metadata: Reads only the metadata of the file or object.

Write object¶

The "Write object" node writes or updates the file or object to the specified path and place(s) the content in the message payload The mode parameter defines the type of content to be updated to the file or object. The write object mode parameters are given below:

• Object: Writes only the content to the file or object.
• Object+Metadata: Writes both content and metadata to the file or object.
• Metadata: Writes only the metadata to the file or object.

Subscribe object¶

The "subscribe object" node will notify the modifications that took place in the specified path. The path should be specified in the edit properties dialog box to receive the notification in the message payload. For example, the message object is shown below:

Delete object¶

The "delete object" node deletes the files or objects from the specified path. You must select the path from the "Select File" dialog box.

Using Integrated Data Lake nodes¶

Example scenario¶

You can read the files or objects of the turbo engine assets by using "read object" node from Integrated Data Lake. The extracted data can be used for further analysis as per customer's requirement.

Objective¶

To read the files or objects from Integrated Data Lake.

Requirements¶

• Inject node
• read object node
• Debug node

Procedure¶

1. Drag and drop the Inject, read object and debug nodes from the dashboard palette. Interconnect the nodes:

   

2. Double click the read object node to edit the properties:

   

3. Click to select the file.

4. Select the Mode.
5. Click "Done".
6. Save and deploy.

Result¶

You can view the results in the message payload.

In this example the "read object" node reads the file or object from the selected path and displays the data in the message payload.