Integrated Data Lake nodes¶

Usage of Integrated Data Lake nodes¶

The Integrated Data Lake is a repository that allows you to store structured and unstructured data/ objects in its native format as needed. It handles large data pools for which the schema and data requirements are not defined until the data is queried.

Integrated Data Lake nodes allows to list, read, write, delete, subscribe and query the files/objects from Integrated Data Lake.

List objects¶

The "list objects" node lists the files/objects from Integrated Data Lake and stores the data in the message payload. Additionally, you can filter the files/objects by "Sub-tenant" property. It is possible by the main environment.

Read object¶

The "read object" node reads the file/object content from the specified path and place(s) the content in the message payload. The mode parameter defines the type of content to be read from the file/object. The read object mode parameters are given below:

• Object: Reads only the content of the file/object.
• Object+Metadata: Reads both content and metadata of the file/object.
• Metadata: Reads only the metadata of the file/object.

Write object¶

The "Write object" node writes/updates the file/object to the specified path and place(s) the content in the message payload The mode parameter defines the type of content to be updated to the file/object. The wrtie object mode parameters are given below:

• Object: Writes only the content to the file/object.
• Object+Metadata: Writes both content and metadata to the file/object.
• Metadata: Writes only the metadata to the file/object.

Subscribe object¶

The "subscribe object" node will notify the modfiications that took place in the specified path. The path should be specified in the edit properties dialog box to recieve the notification in the message payload. For example, the message object is shown below:

Delete object¶

The "delete object" node deletes the files/objects from the specified path. You must select the path from the "Select File" dialog box.

Using Integrated Data Lake nodes¶

Example scenario¶

You can read the files/objects of the turbo engine assets by using "read object" node from Integrated Data Lake. The extracted data can be used for further analysis as per customer's requirement.

Objective¶

To read the files/objects from Integrated Data Lake.

Requirements¶

• Inject node
• read object node
• Debug node

Procedure¶

1. Drag and drop the Inject, read object and debug nodes from the dashboard palette. Interconnect the nodes:

   

2. Double click the read object node to edit the properties:

   

3. Click to select the file.

4. Select the Mode.
5. Click "Done".
6. Save and deploy.

Result¶

You can view the results in the message payload.

In this example the "read object" node reads the file/object from the selected path and displays the data in the message payload.