Modify our custom JupyterHub image#
The 2i2c hubs use a custom
hub image that is defined in the helm-charts directory of the infrastructure repository.
This custom hub image is built on top of the jupyterhub/k8s-hub Docker image and configured based on the needs of 2i2c hubs. This allows adding and configuring other packages like the jupyterhub-configurator or using specific versions of the spawner and authenticator. More information about this custom image can be found in the Dockerfile itself.
The experimental hub images#
In addition to the
hub image, there are also experimental hub images, that are defined alongside the main
hub image in the helm-charts directory of the infrastructure repository, through individual
requirements.txt files, named after the experiment that’s being performed.
The experimental hub images and the main
hub image are quite similar. They are based off the same
Dockerfile, but use different
requirements.txt files, because their primary goal is to be used to test changes like new packages, newer package versions, or even unreleased or unmerged versions of them. They are used to deploy such changes just to one or a few specific communities, without having to worry about the instabilities they can generate for other communities.
Both the 2i2c custom
hub and the
hub-experimental images live at Quay.io.
When to roll out the changes in the experimental images to
Our current policy states that a change in a hub experimental image must be rolled out into
hub, within 4 weeks, otherwise it should be removed. Any issues identified within these 4 weeks, must be fixed it until it’s good enough to deploy to all hubs.
The workflow is the following:
How to create a new hub experimental image#
You will first need to create a new
.txtfile with a name relevant for the experiment, let’s say
chartpressmust be instructed to create a new docker image using the
new-experiment-requirements.txtTo do this, edit the
chartpress.yamlfile located at the root of the
helm-chartes/imagesdirectory to add another image under the
Go to https://quay.io/new/ and create a new public repository using your
2i2corganizational account. Name it the same with the suffix of the name set under
chartpress.yaml. In this example is
How to install an unreleased version of a package in an experimental hub image#
To install an unreleased package, we will need to install directly from GitHub and not from PyPI:
Identify the package and commit you wish to install into the hub image
Specify a full commit hash after
@, not a branch name. This way, our builds are more reproducible!
.txtfile of this specific experiment, let’s say
new-experiment-requirements.txt, to add this package and commit, prefixed by a
git+on a new row
Commit the changes
git add helm-charts/images/hub/new-experiment-requirements.txt git commit
The commit SHA with be used to generate the image tag.
How to build and push a new version of the available hub images#
Rebuild the Docker image and push it to the Quay.io registry
@2i2caddress should give you access to push to the Quay.io registry where the hub image lives, but make sure you are logged into quay.io container registry with the right credentials and these creds are configured to have access to https://quay.io/repository/2i2c/pilot-hub. Please contact someone at 2i2c for access if this is not the case.
docker login quay.io
Checkout the Getting Started with Quay.io docs for more info.
Make sure you have jupyterhub/chartpress installed.
pip install chartpress
This package is also listed under
dev-requirements.txt, so it should be present if you’ve installed the dev dependencies.
Make sure you are in the
helm-chartsdirectory, where the
Run chartpress to build the image, push it to the registry and update the basehub helm chart to use the updated image tag
If building and pushing the
hubimage, then commit the changes made by
helm-charts/basehub/values.yaml, but discard the changes made to
helm-charts/basehub/Chart.yamlas the last may cause problems with the
git add helm-charts/basehub/values.yaml git commit
If building and pushing any of the experimental images, then discard the changes to both
helm-charts/basehub/Chart.yaml, because we only want to deploy this image to particular hubs
How to configure a hub to use an experimental new image#
You will need to put a config similar to the one below in your hub configuration file: