Ξ Configuration
The needed Helm Charts relies on environment variables read from your shell.
The following table details those environment variables by alphabetical order.
| Variable | Description |
|---|---|
| AWS_ACCESS_KEY_ID | Set the access key that gives access to the S3 bucket for the backups. |
| AWS_DEFAULT_PROFILE | Set the default profile that gives access to the S3 bucket for the backups. |
| AWS_DEFAULT_REGION | Set the defaul region that gives access to the S3 bucket for the backups. |
| AWS_REGION | Set the region of the S3 bucket for the backups. |
| AWS_SECRET_ACCESS_KEY | Set the secret key that gives access to the S3 bucket for the backups. |
| DATALAYER_AUTHZ_ENGINE | Type of Authorization engine to use - possible value: none | openfga - default is openfga. |
| DATALAYER_CLUSTER_TYPE | The Kubernetes cluster type (this is only needed for advanced cases). Possible values are eks or ovh. |
| DATALAYER_CREDITS_PROVIDER | A string that defines up the addon providing the credits method within IAM for billing and usage. |
| DATALAYER_DOCKER_REGISTRY_HOST | Hostname of the OCI Registry that contains the Docker Images. |
| DATALAYER_DOCKER_REGISTRY | Full name of the OCI Registry that contains the Docker Images, e.g $DATALAYER_DOCKER_REGISTRY_HOST/datalayer |
| DATALAYER_DOCKER_REGISTRY_USERNAME | Username of the OCI Registry that contains the Docker Images. |
| DATALAYER_DOCKER_REGISTRY_PASSWORD | Password of the OCI Registry that contains the Docker Images. |
| DATALAYER_GITHUB_CLIENT_ID | If you enable authentication via GitHub, define the client id of the GitHub Application. |
| DATALAYER_GITHUB_CLIENT_SECRET | If you enable authentication via GitHub, define the client secret of the GitHub Application |
| DATALAYER_GRAFANA_ADMIN_PWD | Define the password for the Grafana service. |
| DATALAYER_HELM_REGISTRY_HOST | Hostname of the OCI Registry that contains the Helm Charts. |
| DATALAYER_HELM_REGISTRY | Full name of the OCI Registry that contains the Helm Charts, e.g $DATALAYER_HELM_REGISTRY_HOST/datalayer-charts |
| DATALAYER_HELM_REGISTRY_USERNAME | Username of the OCI Registry that contains the Helm Charts. |
| DATALAYER_HELM_REGISTRY_PASSWORD | Password of the OCI Registry that contains the Helm Charts. |
| DATALAYER_JWT_ISSUER | If you enable authentication via JSON Web Token (JWT), define the JWT issuer. e.g. https://dev.datalayer.io |
| DATALAYER_JWT_SECRET | If you enable authentication via JSON Web Token (JWT), define the JWT secret. |
| DATALAYER_JWT_ALGORITHM | If you enable authentication via JSON Web Token (JWT), define the JWT issuer. e.g. HS256 |
| DATALAYER_JWT_ALLOWED_ISSUERS | If you enable authentication via JSON Web Token (JWT), define the JWT allowed issuers. e.g. https://id.your-company.cloud |
| DATALAYER_JWT_DEFAULT_KID_ISSUER | If you enable authentication via JSON Web Token (JWT), define the JWT default key ID issuer. e.g. https://your-company.cloud/api/iam/v1/api-keys |
| DATALAYER_JWT_SKIP_3RD_TOKEN_SIGNATURE_VERIFICATION | If you enable authentication via JSON Web Token (JWT), define the JWT signature should be verified. Set true or false. |
| DATALAYER_CERT_ISSUER | The issuer for the certificates, e.g. letsencrypt or letsencrypt-prod. |
| DATALAYER_INGRESS_CLASS_NAME | The Ingress class name for the Operator, e.g. datalayer-traefik or nginx. |
| DATALAYER_IAM_API_KEY | Key used to protect the internal Kubernetes cluster communications. |
| DATALAYER_IAM_HOST | IAM internal Kubernetes service hostname + port. |
| DATALAYER_OPENFGA_REST_URL | |
| DATALAYER_OPENFGA_STORE_ID | |
| DATALAYER_OPENFGA_AUTHZ_MODEL_ID | |
| DATALAYER_OPERATOR_API_KEY | Set a secret to secure the internal connexions between the services. |
| DATALAYER_PUB_SUB_ENGINE | Type of Pub/Sub engine to use - possible value: none | pulsar - default is pulsar. |
| DATALAYER_PULSAR_URL | Internal URL to pulsar broker e.g. pulsar://datalayer-pulsar-broker.datalayer-pulsar.svc.cluster.local:6650. |
| DATALAYER_USERS_PVC_NAME | Name of the permanent volume claim for user storage. |
| DATALAYER_VAULT_URL | Internal URL to the Vault service. |
| DATALAYER_VAULT_TOKEN | Token to access the Vault service. |
| DATALAYER_RUN_URL | The hostname of the public-facing services. You will need to update your DNS system to point the Load Balancer IP address to the hostname of the URL. The Load Balancer is created by the Ingress service. SSL certificates will be created by the cert-manager service for that hostname. |
| DATALAYER_RUNTIME_ENV | dev, qa or prod |
| DATALAYER_SMTP_HOST | Define the SMTP server hostname for email communications. |
| DATALAYER_SMTP_PORT | Define the SMTP server port number for email communications. |
| DATALAYER_SMTP_USERNAME | Define the SMTP server username for email communications. |
| DATALAYER_SMTP_PASSWORD | Define the SMTP server password for email communications. |
| DATALAYER_SOLR_USERNAME | Define the username to protect Solr. |
| DATALAYER_SOLR_PASSWORD | Define the password to protect Solr. |
| DATALAYER_SOLR_BACKUP_S3_BUCKET_NAME | Define the S3 bucket name for the Solr backups. |
| DATALAYER_SOLR_BACKUP_S3_BUCKET_REGION | Define the S3 bucket region for the Sorl backups. |
| DATALAYER_SUPPORT_EMAIL | Define the email for support communications. e.g. support@your-company.io |
| KUBECONFIG | The path to your Kubeconfig file. |
| OTEL_SDK_DISABLED | Disable OpenTelemetry instrumentation on service and use non-enriched logs (no metadata) [true or false]. |
| OTEL_EXPORTER_OTLP_METRICS_ENDPOINT | URL for exporting services metrics and traces to opentelemetry collector. |
| OTEL_EXPORTER_OTLP_TRACES_ENDPOINT | URL for exporting services metrics and traces to opentelemetry collector. |
tip
Use any of the following command to generate tokens.
TOKEN=$(openssl rand -hex 32)
TOKEN=$(python -c "import secrets; print(secrets.token_hex(32))")
Minimal Configuration
The minimal setup for IAM, Jupyter, Operator is the listed here with example values.
Create a ~/.datalayer/datalayerrc and source that file from ~/.bash_profile or similar based on your shell.
#!/usr/bin/env bash
# ~/.datalayer/datalayerrc
export DATALAYER_AUTHZ_ENGINE="none"
export DATALAYER_CDN_URL="https://datalayer.io"
export DATALAYER_CLUSTER_NAME="my-cluster"
export DATALAYER_CLUSTER_TYPE="eks"
export DATALAYER_CREDITS_PROVIDER="stripe"
export DATALAYER_DOCKER_REGISTRY="${DATALAYER_DOCKER_REGISTRY_HOST}/datalayer"
export DATALAYER_DOCKER_REGISTRY_HOST="my-registry.net"
export DATALAYER_DOCKER_REGISTRY_PASSWORD="..."
export DATALAYER_DOCKER_REGISTRY_USERNAME="..."
export DATALAYER_HELM_REGISTRY="${DATALAYER_HELM_REGISTRY_HOST}/datalayer-charts"
export DATALAYER_HELM_REGISTRY_HOST="my-registry.net"
export DATALAYER_HELM_REGISTRY_PASSWORD="..."
export DATALAYER_HELM_REGISTRY_USERNAME="..."
export DATALAYER_IAM_API_KEY="..."
export DATALAYER_INITIAL_USER_CREDITS=10
export DATALAYER_JWT_ALGORITHM="HS256"
export DATALAYER_JWT_ALLOWED_ISSUERS="https://my-id.datalayer.run"
export DATALAYER_JWT_ISSUER="https://my-id.datalayer.run"
export DATALAYER_JWT_SECRET="..."
export DATALAYER_JWT_SKIP_EXTERNAL_TOKEN_SIGNATURE_VERIFICATION=false
export DATALAYER_OPERATOR_API_KEY="..."
export DATALAYER_PUB_SUB_ENGINE="none"
export DATALAYER_RUNTIME_ENV="prod"
export DATALAYER_RUN_URL=https://my-cluster.datalayer.run
export DATALAYER_SOLR_PASSWORD="..."
export DATALAYER_SOLR_USERNAME="admin"
export KUBECONFIG="~/.datalayer/my-cluster.yaml"
export OTEL_SDK_DISABLED=true
source ~/.datalayer/datalayerrc
Stripe Addon
The configuration for the Stripe Addon is listed here.
| Variable | Description |
|---|---|
| DATALAYER_STRIPE_API_KEY | Private Stripe API key. |
| DATALAYER_STRIPE_CHECKOUT_ROUTE | React router route (relative to usage route) to redirect the user to the payment form (default is /usage/payment). |
| DATALAYER_STRIPE_JS_API_KEY | Public Stripe API key. |
| DATALAYER_STRIPE_PRODUCT_ID | Stripe Product ID. |
| DATALAYER_STRIPE_WEBHOOK_SECRET | Stripe webhook secret. |