Automated migrations

Related components and artifacts

openDesk comes with two upgrade steps as part of the deployment; they can be found in the folder /helmfile/apps along with all other components:

• migrations-pre: Is the very first app that gets deployed.
• migrations-post: Is the last app that gets deployed.

Both migrations must be deployed exclusively at their first/last position and not parallel with other components.

The status of the upgrade migrations is tracked in the ConfigMap migrations-status, more details can be found in the README.md of the related container image.

With openDesk 1.17.0 the migrations that are triggered are defined in the shared migrations values file migrations.yaml.gotmpl, which is the basis for both the migrations-pre and the migrations-post.

The file declares, below the key migrations:

releaseVersion

The openDesk release currently being deployed, taken from global.systemInformation.releaseVersion.

upgradeFrom.min and upgradeFrom.max

The range of openDesk releases the declared migrations support upgrading from. They only run when the release that was migrated successfully last - recorded in the ConfigMap migrations-status, see Related components and artifacts - lies within this range. This version gating is what enforces the mandatory upgrade path.

actions.pre and actions.post

The actions to be executed before (migrations-pre) respectively after (migrations-post) the deployment of all other components, each in the order listed. An action is

id: <action-id>
tag: <run-once-key>
config: <parameters />

• id selects the action module shipped in the migrations image.
• config carries its non-secret parameters.
• tag (optional) makes an action run once: The pair (id, tag) is recorded in the history of the ConfigMap migrations-status, and an action whose pair is already recorded is skipped - on a redeployment of the same release as well as in any later release. It is used for work that must not be repeated, like a data migration or a one-time restart of a component, and is omitted for actions that are meant to run on every deployment that declares them. A later release that needs the same work again declares the action under a new tag, so tagging with the release the work belongs to (e.g. v1.17.0) keeps the pairs distinct.

Actions are only declared when the components they work on are actually installed, gated by the same conditions as the corresponding releases.

actionsSkip.pre and actionsSkip.post

The actions to opt out of. In contrast to everything above, this is not part of the migration’s definition but a per-deployment setting, configurable in migrations.yaml.gotmpl.

It mirrors actions: An entry names the stage, the id and the tag of the action it skips - the same footprint, just without its config - and has to match the declared action exactly. See Skip single actions of the automated migrations in updates.md.

secretFiles

The credentials the actions need, mounted from existing Kubernetes Secrets as files, so that no credential is passed via environment variables or duplicated into a new Secret.

Automated migrations overview

The following table lists the actions the openDesk releases declare. Declared with is the release whose migration definition added the call, Upgrades covered is the upgradeFrom range of that definition, so the range of installed releases the call is executed for.

Actions

The actions listed here are the ones shipped as modules in the openDesk Migrations image. They are the library the migrations actions section in shared migrations values file can draw from.

Whether an action runs once or on every deployment is not a property of the action itself but of its declaration: An action declared with a tag runs once, an action declared without one runs whenever a release declares it.

The permissions the actions need are granted in the role.yaml of the migrations Helm chart, where a comment names the action each rule belongs to.

ox_connector_restart

Restarts the StatefulSet named in statefulset.name so that it picks up the most recent configuration, especially required when e.g. the provisioning secrets are updated. It is restarted by scaling it down to zero and back up to statefulset.replicas instead of by a rollout restart, waiting statefulset.waitSeconds in between.

migrations:
  actions:
    post:
      - id: "ox_connector_restart"
        config:
          statefulset:
            name: "ox-connector"
            replicas: 1
            waitSeconds: 30

Development

When a new upgrade migration is required, ensure to address the following list:

• Update the generated release version file global.generated.yaml.gotmpl at least on the patch level to test the upgrade in your feature branch and trigger it in the develop branch after the feature branch was merged. During the release process, the value is overwritten by the release’s version number.
• You have to implement the migration logic as a runner script in the opendesk-migrations image. Please find more instructions in the linked repository.
• You most likely have to update the opendesk-migrations Helm chart within the rules section of the role.yaml to provide the permissions required for the execution of your migration’s logic.
• You must set the runner’s ID you want to execute in the migrations.yaml.gotmpl. See also the migrations.* section of the Helm chart’s README.md.
• Update the charts.yaml.gotmpl and images.yaml.gotmpl to reflect the newer releases of the opendesk-migrations Helm chart and container image.