Skip to content

Integrated Data Lake nodes

Usage of Integrated Data Lake nodes

The Integrated Data Lake is an application within MindSphere. It is a repository that allows you to store structured and unstructured data/ objects in its native format as needed. It handles large data pools for which the schema and data requirements are not defined until the data is queried.

Integrated Data Lake nodes allows to list, read, write, delete, subscribe and query the files/objects from MindSphere Integrated Data Lake.

Note

Integrated Data Lake nodes are only available on AWS.

List objects

List object

The "list objects" node lists the files/objects from MindSphere Integrated Data Lake and stores the data in the message payload. Additionally, you can filter the files/objects by "Sub-tenant" property. It is possible by the main tenant.

Read object

Read object

The "read object" node reads the file/object content from the specified path and place(s) the content in the message payload. The mode parameter defines the type of content to be read from the file/object. The read object mode parameters are given below:

  • Object: Reads only the content of the file/object.
  • Object+Metadata: Reads both content and metadata of the file/object.
  • Metadata: Reads only the metadata of the file/object.

Write object

Write object

The "Write object" node writes/updates the file/object to the specified path and place(s) the content in the message payload The mode parameter defines the type of content to be updated to the file/object. The wrtie object mode parameters are given below:

  • Object: Writes only the content to the file/object.
  • Object+Metadata: Writes both content and metadata to the file/object.
  • Metadata: Writes only the metadata to the file/object.

Subscribe object

Subscribe object

The "subscribe object" node will notify the modfiications that took place in the specified path. The path should be specified in the edit properties dialog box to recieve the notification in the message payload. For example, the message object is shown below:

Message object

Note

Metadata changes cannot be subscribed.

Delete object

Delete object

The "delete object" node deletes the files/objects from the specified path. You must select the path from the "Select File" dialog box.

Note

If the files/objects is deleted then the metadata will also be deleted.

Using Integrated Data Lake nodes

Example scenario

You can read the files/objects of the turbo engine assets by using "read object" node from MindSphere Integrated Data Lake. The extracted data can be used for further analysis as per customer's requirement.

Objective

To read the files/objects from MindSphere Integrated Data Lake.

Requirements

  • Inject node
  • read object node
  • Debug node

Procedure

  1. Drag and drop the Inject, read object and debug nodes from the dashboard palette. Interconnect the nodes:

    Interconnect node

  2. Double click the read object node to edit the properties:

    Edit read object

  3. Click Select to select the file.

  4. Select the Mode.
  5. Click "Done".
  6. Save and deploy.

Note

  • Path should be specified for the "List object" and "Subscribe object" nodes.
  • Select the path from the "Select File" dialog box for the "Read object", "Write object" and "Delete object" nodes from the file browser dialog box.
  • Mode defining is applicable for "Read object" and "Write object" nodes.

Result

You can view the results in the message payload.

In this example the "read object" node reads the file/object from the selected path and displays the data in the message payload.

Read object

Any questions left?

Ask the community


Except where otherwise noted, content on this site is licensed under the MindSphere Development License Agreement.


Last update: April 29, 2022