Secrets

HPE ML Data Management uses Kubernetes’ Secrets to store and manage sensitive data, such as passwords, OAuth tokens, or ssh keys. You can use any of Kubernetes’ types of Secrets that match your use case. Namely, generic (or Opaque), tls, or docker-registry.

About Secrets #

When you install or upgrade a cluster, you can provide values for the configuration fields in your Helm Chart values.yaml file. However, some of those values are sensitive and should not be stored in your values.yaml file.

HPE ML Data Management provides a way to inject those values at the time of the deployment or upgrade. We call those values platform secrets.

HPE ML Data Management Platform Secrets Map #

If no Secret KEY name is provided for the Helm Chart’s Secret NAME Attribute, HPE ML Data Management will use the Helm Chart’s RAW Attribute to populate its own platform secrets at the time of the installation/upgrade. Those that are not marked as required are automatically generated by the platform if not provided.

Create A Secret #

The creation of a Secret in HPE ML Data Management requires a JSON configuration file.

A good way to create this file is:

1. To generate it by calling a dry-run of the kubectl create secret ... --dry-run=client --output=json > myfirstsecret.json command.
2. Then call pachctl create secret -f myfirstsecret.json.

⚠️

Kubernetes Secrets are, by default, stored as unencrypted base64-encoded strings (i.e., the values for all keys in the data field have to be base64-encoded strings). When using the kubectl create secret command, the encoding is done for you. If you choose to manually create your JSON file, make sure to use your own base 64 encoder.

Generate Your Secret Config File #

Let’s first generate your secret configuration file using the kubectl command. For example:

• for a generic authentication secret:

  kubectl create secret generic mysecretname --from-literal=username=<myusername> --from-literal=password=<mypassword> --dry-run=client  --output=json > myfirstsecret.json

  
• for a tls secret:

  kubectl create secret tls mysecretname --cert=<Path to your certificate> --key=<Path to your SSH key> --dry-run=client  --output=json > myfirstsecret.json 

  
• for a docker registry secret:

  kubectl create secret docker-registry mysecretname --dry-run=client --docker-server=<DOCKER_REGISTRY_SERVER> --docker-username=<DOCKER_USER> --docker-password=<DOCKER_PASSWORD> --output=json > myfirstsecret.json

  

Generic Secret Example #

{
   "apiVersion": "v1",
   "kind": "Secret",
   "metadata": {
      "name": "clearml"
   },
   "type": "Opaque",
   "stringData": {
      "access": "<CLEARML_API_ACCESS_KEY>",
      "secret": "<CLEARML_API_SECRET_KEY>"
   }
}

Find more detailed information on the creation of Secrets in Kubernetes documentation.

Create your Secret in HPE ML Data Management #

Next, run the following to actually create the secret in the HPE ML Data Management Kubernetes cluster:

pachctl create secret -f myfirstsecret.json 

You can run pachctl list secret to verify that your secret has been properly created. You should see an output that looks like the following:

NAME     TYPE                           CREATED        
mysecret kubernetes.io/dockerconfigjson 11 seconds ago 

ℹ️

Use pachctl delete secret to delete a secret given its name, pachctl inspect secret to list a secret given its name.

You can now edit your pipeline specification file as follow.

Reference a Secret in a Pipeline Spec #

Now that your secret is created on HPE ML Data Management cluster, you will need to notify your pipeline by updating your pipeline specification file. In HPE ML Data Management, a Secret can be used in three different ways:

1. As a container environment variable:

   In this case, in HPE ML Data Management’s pipeline specification file, you need to reference Kubernetes’ Secret by its:

   • name
   • and specify an environment variable named envVar that the value of your key should be bound to.

   This makes for easy access to your Secret’s data in your pipeline’s code. For example, this is useful for passing the password to a third-party system to your pipeline’s code.

   "transform": {
      "image": "string",
      "cmd": [ string ],
      ...
      "secrets": [ {
         "name": "string",
         "envVar": "string",
         "key": string
      }]
   }

   

   Example #

   Example of a pipeline specification file assigning a Secret’s values to environment variables.

   Look at the pipeline specification in this example and see how we used the "envVar" to pass CLEARML API credentials to the pipeline code.

   {
      "pipeline": {
         "name": "mnist"
      },
      "description": "MNIST example logging to ClearML",
      "input": {
         "pfs": {
            "repo": "data",
            "branch": "master",
            "glob": "/*"
         }
      },
      "transform": {
         "cmd": [
            "/bin/sh"
         ],
         "stdin": [
            "python pytorch_mnist.py --lr 0.2 --save-location /pfs/out"
         ],
         "image": "pachyderm/clearml_mnist:dev0.11",
         "secrets": [
            {
            "name": "clearml",
            "envVar": "CLEARML_API_ACCESS_KEY",
            "key": "access"
            },
            {
            "name": "clearml",
            "envVar": "CLEARML_API_SECRET_KEY",
            "key": "secret"
            }
         ]
      }
   }

   

2. As a file in a volume mounted on a container:

   In this case, in HPE ML Data Management’s pipeline specification file, you need to reference Kubernetes’ Secret by its:

   • name
   • and specify the mount point (mount_path) to the secret (ex: "/var/my-app-secret").

   HPE ML Data Management mounts all of the keys in the secret with file names corresponding to the keys. This is useful for secure configuration files.

   "transform": {
      "image": "string",
      "cmd": [ string ],
      ...
      "secrets": [ {
         "name": "string",
         "mount_path": string
      }]
   }

   

3. When pulling images:

   Image pull Secrets are a different kind of secret used to store access credentials to your private image registry.

   You reference Image Pull Secrets (or Docker Registry Secrets) by setting the imagePullSecrets field of your pipeline specification file to the secret’s name you created (ex: "mysecretname").

   "transform": {
      "image": "string",
      "cmd": [ string ],
      ...
      "imagePullSecrets": [ string ]
   }