Uploading the files to Data Registry¶
The files are uploaded to the Data Registry from the Data Explorer in Integrated Data Lake application. The files uploaded could be in CSV, JSON, PARQUET, or XML format.
When the file is uploaded in Integrated Data Lake application, a notification is triggered in Data Contextualization. Data Contextualization uses the folder metadata tag to determine the registry for the file and the application then proceeds to ingest the file into the designated registry and initiates the schema discovery process.
Uploading the files for Enterprise Data Registry¶
In Enterprise Data Registry, the user can initiate the schema discovery process by uploading files in Integrated Data Lake after creating the registry. The user needs to tag the files in the Integrated Data Lake to indicate association of the files to a specific Data Registry. The user can use following procedure to tag and upload the files in the Integrated Data Lake.
For Enterprise Data Registry, the folder must be created in the "sdi" folder of the Integrated Data Lake. User can create different folders for different registries.
Uploading a file in Integrated Data Lake¶
To upload the file into Integrated Data Lake, follow these steps:
- In the Integrated Data Lake, navigate to the 'Data Explorer' window and click "Create Folder".
- Create a folder inside the "sdi" folder.
- Enter the folder name and click to save the folder.
- Click to add the metadata tags for the created folder.
- Select "Metadata Tags", click "Add Tag".
- Enter metadata tag to the folder in the following format,
registryId_ABCDwhere "ABCD" is the "registryId".
In the Upload Objects, click "Add Files" and select the file to be uploaded.
- If a user tags a folder with a registryId, all files copied in that folder will be ingested in the same registry.
- A user can optionally add a metadata tag on a file. The metadata must be in the same format as the folder metadata. When both file metadata and (parent) folder metadata are present, Data Contextualization gives preference to the file metadata. So, file metadata can be used to override the registryId specified on the folder metadata.
Uploading the file is successful and the upload status changes to successfully uploaded.
In Data Explorer window, click "Refresh View" button to view the uploaded files in the table.
Uploading a file for IoT Data Registry¶
After creating a registry in the IoT Data Registry, user needs to import the IoT data to Integrated Data Lake. To import the IoT data into Integrated Data Lake, refer Importing data into Integrated Data Lake.
For IoT Data Registry, the folder must be created in the "sdi" folder of the Integrated Data Lake. User can create different folders for different registries.
Except where otherwise noted, content on this site is licensed under the Development License Agreement.