Skip to main content

Jupyter Environments

Jupyter Environments Specification​

Jupyter Environments define:

  • The available libraries which are available in a Docker Image. The Docker Image can be compared to a Conda Environment.
  • The mounted Jupyter Contents.
  • Some Environment Variables.
  • The default number of CPU/GPU and memory resources.
  • A template for the Kernel given-name.
  • Snippets.
  • Links to examples.

The complete specification is available here and the following sections show simple examples.

Manage the Environments​

kubectl get jupyter-environments -A
kubectl get jupyter-environments -o yaml -A

Kernel Launcher​

The information provided in the specification is used when by the Kernel Launcher.

Platform and User Environments​

The Platform Administrator is responsible to provision the Platform Environments.

note

The Users will be able to create their own Jupyter Environments.

CVE based on Software Bill of Materials​

The environments are scanned for CVE (Cybersecurity Vulnerabilities) based on their Docker Image Software Bill of Materials.

As the scanning is done at the Docker level, it covers all Linux libraries as the various packages installed e.g. in a Conda Environment defined in that Docker Image.

Conda2Docker​

The Jupyter Environments can be seen as en extension in the Cloud-era of the well-known Conda Environments.

Actually, Conda Environments are more similar to a Docker Image and we are working on Conda2Docker, a solution to automatically create a Docker Image from a Conda Environment.

With Jupyter Environments, Datalayer allows to specify on top of Conda Environments some Contents, some Environment Variables, Snippets, Examples, the CPU/GPU and memory resources, the name of the Kernel as other more cloud-native features.

Simple Environment Example​

cat <<EOF | kubectl apply -f -
apiVersion: datalayer.io/v1
kind: JupyterEnvironment
metadata:
  name: python-simple-env
  namespace: datalayer-jupyter
  labels:
    project.datalayer.io/name: poc
spec:
  title: Python Simple Environment
  description: A Python environment for simple coding.
  language: python
  owner: datalayer
  visibility: public
  example: https://raw.githubusercontent.com/datalayer/examples/e8390ce9eb4aee36721eb6e7cdc31ef8455ff359/python-simple/python-simple-example.ipynb
  dockerImage: ${DATALAYER_DOCKER_REGISTRY}/datalayer/jupyter-python:0.0.8
  kernel:
    givenNameTemplate: A Python kernel for simple coding.
  resourcesRanges:
    default:
      requests:
        cpu: "250m"
        memory: "64Mi"
      limits:
        cpu: "500m"
        memory: "512Mi"
EOF

Xarray Environment Example​

cat <<EOF | kubectl apply -f -
apiVersion: datalayer.io/v1
kind: JupyterEnvironment
metadata:
  name: xarray-env
  namespace: datalayer-jupyter
  labels:
    project.datalayer.io/name: dev
spec:
  title: XArray Environment
  description: |
    A environment for analysis with Xarray. **[xarray](https://github.com/pydata/xarray)** (pronounced "ex-array", formerly known as **xray**) is an open source project and Python package that makes working with labelled multi-dimensional arrays simple, efficient, and fun!

    **Typical use cases**

    EO (Earth observation) analysis.

    **Mounted contents**

    Analysis-ready, cloud-optimized (ARCO) from Sentinel-2 mission:Lland monitoring constellation of two satellites that provide high resolution optical imagery and provide continuity for the current SPOT and Landsat missions. The mission provides a global coverage of the Earth's land surface every 5 days, making the data of great use in on-going studies. L1C data are available from June 2015 globally. L2A data are available from November 2016 over Europe region and globally since January 2017.

    ![](https://www.esa.int/var/esa/storage/images/esa_multimedia/images/2022/09/copernicus_sentinel_family/24451531-1-eng-GB/Copernicus_Sentinel_family_pillars.jpg)

    **Hardware recommandation for large scale analysis**

    - [x] GPU (CUDA > 11)
    - [x] GPU Memory: Minimum 8BG

  language: python
  owner: datalayer
  visibility: public
  example: https://raw.githubusercontent.com/datalayer/examples/e8390ce9eb4aee36721eb6e7cdc31ef8455ff359/python-simple/python-simple-example.ipynb
  dockerImage: 9ol9b008.c1.bhs5.container-registry.ovh.net/datalayer/jupyter-python:0.0.8
  kernel:
    givenNameTemplate: A kernel for analysis with XArray.
  snippets:
    - title: Useful imports
      code: |
        import matplotlib.pyplot as plt
        import numpy as np
        import pandas as pd
  resourcesRanges:
    default:
      requests:
        cpu: "250m"
        memory: "64Mi"
      limits:
        cpu: "500m"
        memory: "512Mi"
EOF