1.14 to latest 2.x

This guide covers migrating from Emissary 1.14.X to Emissary $version$. If this is not your exact situation, see the migration matrix. This guide is written for upgrading an installation made without using Helm. If you originally installed with Helm, see the Helm-based upgrade instructions.

We’re pleased to introduce Emissary $version$! The 2.X family introduces a number of changes to allow Emissary to more gracefully handle larger installations (including multitenant or multiorganizational installations), reduce memory footprint, and improve performance. In keeping with SemVer, Emissary 2.X introduces some changes that aren’t backward-compatible with 1.X. These changes are detailed in Major Changes in Emissary 2.X.

Migration Overview

Read the migration instructions below before making any changes to your cluster!

The recommended strategy for migration is to run Emissary 1.14 and Emissary $version$ side-by-side in the same cluster. This gives Emissary $version$ and Emissary 1.14 access to all the same configuration resources, with some important caveats:

  1. Emissary 1.14 will not see any getambassador.io/v3alpha1 resources.

    This is intentional; it provides a way to apply configuration only to Emissary $version$, while not interfering with the operation of your Emissary 1.14 installation.

  2. If needed, you can use labels to further isolate configurations.

    If you need to prevent your Emissary $version$ installation from seeing a particular bit of Emissary 1.14 configuration, you can apply a Kubernetes label to the configuration resources that should be seen by your Emissary $version$ installation, then set its AMBASSADOR_LABEL_SELECTOR environment variable to restrict its configuration to only the labelled resources.

    For example, you could apply a version-two: true label to all resources that should be visible to Emissary $version$, then set AMBASSADOR_LABEL_SELECTOR=version-two=true in its Deployment.

  3. Be careful about label selectors on Kubernetes Services!

    If you have services in Emissary 1.14 that use selectors that will match Pods from Emissary $version$, traffic will be erroneously split between Emissary 1.14 and Emissary $version$. The labels used by Emissary $version$ include:

    app.kubernetes.io/name: emissary-ingress
    app.kubernetes.io/instance: emissary-ingress
    app.kubernetes.io/part-of: emissary-ingress
    app.kubernetes.io/managed-by: getambassador.io
    product: aes
    profile: main
    
  4. Be careful to only have one Emissary Agent running at a time.

    The Emissary Agent is responsible for communications between Emissary and Ambassador Cloud. If multiple versions of the Agent are running simultaneously, Ambassador Cloud could see conflicting information about your cluster.

    The migration YAML used below to install Emissary $version$ will not install a duplicate agent. If you are building your own YAML, make sure not to include a duplicate agent.

You can also migrate by installing Emissary $version$ in a separate cluster. This permits absolute certainty that your Emissary 1.14 configuration will not be affected by changes meant for Emissary $version$, and it eliminates concerns about ACME, but it is more effort.

Side-by-Side Migration Steps

Migration is a seven-step process:

  1. Make sure that older configuration resources are not present.

    Emissary 2.X does not support getambassador.io/v0 or getambassador.io/v1 resources, and Kubernetes will not permit removing support for CRD versions that are still in use for stored resources. To verify that no resources older than getambassador.io/v2 are active, run

    kubectl get crds -o 'go-template={{range .items}}{{.metadata.name}}={{.status.storedVersions}}{{"\n"}}{{end}}' | fgrep getambassador.io
    

    If v1 is present in the output, do not begin migration. The old resources must be converted to getambassador.io/v2 and the storedVersion information in the cluster must be updated. If necessary, contact Ambassador Labs on Slack for more information.

  2. Install new CRDs.

    Before installing Emissary $version$ itself, you must configure your Kubernetes cluster to support its new getambassador.io/v3alpha1 configuration resources. Note that getambassador.io/v2 resources are still supported, but you must install support for getambassador.io/v3alpha1 to run Emissary $version$, even if you intend to continue using only getambassador.io/v2 resources for some time.

    kubectl apply -f https://app.getambassador.io/yaml/emissary/$version$/emissary-crds.yaml
    kubectl wait --timeout=90s --for=condition=available deployment emissary-apiext -n emissary-system
    
    Emissary $version$ includes a Deployment in the `emissary-system` namespace called emissary-apiext. This is the APIserver extension that supports converting Emissary CRDs between getambassador.io/v2 and getambassador.io/v3alpha1. This Deployment needs to be running at all times. If the emissary-apiext Deployment's Pods all stop running, you will not be able to use getambassador.io/v3alpha1 CRDs until restarting the emissary-apiext Deployment. There is a known issue with the emissary-apiext service that impacts all Emissary 2.x and 3.x users. Specifically, the TLS certificate used by apiext expires one year after creation and does not auto-renew. All users who are running Emissary/Ambassador Edge Stack 2.x or 3.x with the apiext service should proactively renew their certificate as soon as practical by running kubectl delete --all secrets --namespace=emissary-system to delete the existing certificate, and then restart the emissary-apiext deployment with kubectl rollout restart deploy/emissary-apiext -n emissary-system. This will create a new certificate with a one year expiration. We will issue a software patch to address this issue well before the one year expiration. Note that certificate renewal will not cause any downtime.
  3. Install Emissary $version$.

    After installing the new CRDs, you need to install Emissary $version$ itself in the same namespace as your existing Emissary 1.14 installation. It’s important to use the same namespace so that the two installations can see the same secrets, etc.

    We publish two manifests for different namespaces. Use only the one that matches the namespace into which you installed Emissary 1.14:

    If you installed Emissary 1.14 into some other namespace, you’ll need to download one of the files and edit it to match your namespace.

    If you need to set AMBASSADOR_LABEL_SELECTOR, you’ll need to download your chosen file and and edit it to do so.

    Assuming that you’re using the default namespace:

    kubectl apply -f https://app.getambassador.io/yaml/emissary/$version$/emissary-defaultns.yaml && \
    kubectl rollout status -n default deployment/edge-stack -w
    
    Emissary $version$ includes a Deployment in the $productNamespace$ namespace called emissary-apiext. This is the APIserver extension that supports converting Emissary CRDs between getambassador.io/v2 and getambassador.io/v3alpha1. This Deployment needs to be running at all times. If the emissary-apiext Deployment's Pods all stop running, you will not be able to use getambassador.io/v3alpha1 CRDs until restarting the emissary-apiext Deployment.
  4. Install Listeners and Hosts as needed.

    An important difference between Emissary 1.14 and Emissary $version$ is the new mandatory Listener CRD. Also, when running both installations side by side, you will need to make sure that a Host is present for the new Emissary $version$ Service. For example:

    kubectl apply -f - <<EOF
    ---
    apiVersion: getambassador.io/v3alpha1
    kind: Listener
    metadata:
      name: ambassador-http-listener
    spec:
      port: 8080
      protocol: HTTPS
      securityModel: XFP
      hostBinding:
        namespace:
          from: ALL
    ---
    apiVersion: getambassador.io/v3alpha1
    kind: Listener
    metadata:
      name: ambassador-https-listener
    spec:
      port: 8443
      protocol: HTTPS
      securityModel: XFP
      hostBinding:
        namespace:
          from: ALL
    ---
    apiVersion: getambassador.io/v3alpha1
    kind: Host
    metadata:
      name: emissary-host
    spec:
      hostname: $EMISSARY_HOSTNAME
      tlsSecret:
        name: $EMISSARY_TLS_SECRET
    EOF
    

    This example requires that you know the hostname for the Emissary Service ($EMISSARY_HOSTNAME) and that you have created a TLS Secret for it in $EMISSARY_TLS_SECRET.

  5. Test!

    Your Emissary $version$ installation can support the getambassador.io/v2 configuration resources used by Emissary 1.14, but you may need to make some changes to the configuration, as detailed in the documentation on configuring Emissary Communications and updating CRDs to getambassador.io/v3alpha1.

    Kubernetes will not allow you to have a getambassador.io/v3alpha1 resource with the same name as a getambassador.io/v2 resource or vice versa: only one version can be stored at a time.

    If you find that your Emissary $version$ installation and your Emissary 1.14 installation absolutely must have resources that are only seen by one version or the other way, see overview section 2, "If needed, you can use labels to further isolate configurations".

    If you find that you need to roll back, just reinstall your 1.14 CRDs and delete your installation of Emissary $version$.

  6. When ready, switch over to Emissary $version$.

    You can run Emissary 1.14 and Emissary $version$ side-by-side as long as you care to. However, taking full advantage of Emissary 2.X’s capabilities requires updating your configuration to use getambassador.io/v3alpha1 configuration resources, since some useful features in Emissary $version$ are only available using getambassador.io/v3alpha1 resources.

    When you’re ready to have Emissary $version$ handle traffic on its own, switch your original Emissary 1.14 Service to point to Emissary $version$. Use kubectl edit service ambassador and change the selectors to:

    app.kubernetes.io/instance: emissary-ingress
    app.kubernetes.io/name: emissary-ingress
    profile: main
    

    Repeat using kubectl edit service ambassador-admin for the ambassador-admin Service.

  7. Finally, install the Emissary $version$ Ambassador Agent.

    First, scale the 1.14 agent to 0:

    kubectl scale -n default deployment/ambassador-agent --replicas=0
    

    Once that’s done, install the new Agent into the same namespace as your Emissary deployment. Again, we supply two files for two different namespaces: use only the one that matches the namespace into which you installed Emissary 1.14.

    If you installed Emissary 1.14 into some other namespace, you’ll need to download one of the files and edit it to match your namespace.

    Assuming that you’re using the default namespace:

    kubectl apply -f https://app.getambassador.io/yaml/emissary/$version$/emissary-defaultns-agent.yaml
    

Congratulations! At this point, Emissary $version$ is fully running and it’s safe to remove the ambassador and ambassador-agent Deployments:

kubectl delete deployment/ambassador deployment/ambassador-agent

Once Emissary 1.14 is no longer running, you may convert any remaining getambassador.io/v2 resources to getambassador.io/v3alpha1. You may also want to redirect DNS to the edge-stack Service and remove the ambassador Service.