Skip to content

Uploading the files to Data Registry

The files are uploaded to the Data Registry from the Data Explorer in Integrated Data Lake application. The files uploaded could be in CSV, JSON, PARQUET, or XML format.

When the file is uploaded in Integrated Data Lake application, a notification is triggered in Data Contextualization. Data Contextualization uses the folder metadata tag to determine the registry for the file and the application then proceeds to ingest the file into the designated registry and initiates the schema discovery process.

Data Registry flow chart

Uploading the files for Enterprise Data Registry

In Enterprise Data Registry, the user can initiate the schema discovery process by uploading files in Integrated Data Lake after creating the registry. The user needs to tag the files in the Integrated Data Lake to indicate association of the files to a specific Data Registry. The user can use following procedure to tag and upload the files in the Integrated Data Lake.

Note

For Enterprise Data Registry, the folder must be created in the "sdi" folder of the Integrated Data Lake. User can create different folders for different registries.

Procedure

To upload the file in Integrated Data Lake, refer to Uploading the objects. To add metadata tag to the folder, follow these steps:

  1. In Integrated Data Lake, click eye-icon to add the metadata tags for the created folder.

    Note

    Metadata tags can only be added to .csv files in Integrated Data Lake.

  2. Select "tags" and enter the metadata tag to the folder in the following format, registryId_ABCD where "ABCD" is the "registryId".

  3. Click "Update".

Note

  • If a user tags a folder with a registryId, all files copied in that folder will be ingested in the same registry.
  • A user can optionally add a metadata tag on a file. The metadata must be in the same format as the folder metadata. When both file metadata and (parent) folder metadata are present, Data Contextualization gives preference to the file metadata. So, file metadata can be used to override the registryId specified on the folder metadata.

Uploading the file in Integrated Data Lake is successfull and the upload status changes to successfully uploaded.

Uploaded file

Uploading a file for IoT Data Registry

After creating a registry in the IoT Data Registry, user needs to import the IoT data to Integrated Data Lake. To import the IoT data in Integrated Data Lake, refer to Importing Time Series Data.

Note

For IoT Data Registry, the folder must be created in the "sdi" folder of the Integrated Data Lake. User can create different folders for different registries.

Last update: August 19, 2024