Predictive Learning (PrL) API - Developer Documentation
Skip to content

Predictive Learning (PrL) API

This storage API is provided for users who require the ability to upload and download large files and have the files available to work with Predictive Learning. The PrL API is a simple remote storage repository that allows all operations using a temporary token and session.

Access to the API

Access to the storage path and API is provided for PrL clusters, environments, and jobs. AWS and Azure CLI can interact with the PrL API using these commands:

Access

aws s3 ls s3://prl-storage-<digits>/<tenant>/data/mypath/

Uploading files

aws s3 cp /tmp/myfile.txt s3://prl-storage-<digits>/data/<tenant>/data/mypath/myfile.txt

Downloading files

aws s3 cp s3://prl-storage-<digits>/<tenant>/data/mypath/myfile.txt /tmp/myfile.txt

External Access to the API

To access the storage API from outside Predictive Learning, you must obtain a temporary token from the PrL API and use the provided credentials, through either a credentials file store, or through environment variables.

Use a POST request to obtain a token from the API:

curl --location --request POST $GATEWAY_ENDPOINT$'/prlstorage/v3/generateAccessToken' --header 'Content-Type: application/json'

Which results in:

    {
        "credentials": {
            "accessKeyId": "<access_key>",
            "secretAccessKey": "<secret>",
            "sessionToken": "<session>",
            "expirationTime": "2019-11-19T14:33:21.000"
        },
        "storageAccount": "prl-storage-<account>",
        "storagePath": "/<mytenant>/data/"
    }

Alternatively, you can extract the needed keys if you store the request result in a bash variable that can be parsed with the following commands:

    %%bash
    content=$(curl --location --request POST $GATEWAY_ENDPOINT$'/prlstorage/v3/generateAccessToken' --header 'Content-Type: application/json')
    secret=$(jq -r '.credentials.secretAccessKey' <<< "${content}")
    session=$(jq -r '.credentials.sessionToken' <<< "${content}")
    accesskey=$(jq -r '.credentials.accessKeyId' <<< "${content}")
    export AWS_ACCESS_KEY_ID=$(echo "${accesskey}")
    export AWS_SECRET_ACCESS_KEY=$(echo "${secret}")
    export AWS_SESSION_TOKEN=$(echo "${session}")

When PrL storage is used as an input or output source for a job, the root tenant path is available to the Docker container being executed. Both input and output is mounted in the container's /data/input and /data/output paths.


Last update: January 22, 2024