Running on

Ambassador Module

Mon, 01 Jan 0001 00:00:00 +0000

If present, the ambassador Module defines system-wide configuration for Emissary. You won’t need it unless you need to change one of the system-wide configuration settings below.

To use the ambassador Module to configure Emissary, it MUST be named ambassador, otherwise it will be ignored. To create multiple ambassador Modules in the same Kubernetes namespace, you will need to apply them as annotations with separate ambassador_ids: you will not be able to use multiple CRDs.

AWS

Mon, 01 Jan 0001 00:00:00 +0000

Emissary with AWS

Emissary is a platform agnostic Kubernetes API gateway. It will run in any distribution of Kubernetes whether it is managed by a cloud provider or on homegrown bare-metal servers.

This document serves as a reference for different configuration options available when running Kubernetes in AWS. See Installing Emissary for the various installation methods available.

Recommended configuration

There are lot of configuration options available to you when running Emissary in AWS. While you should read this entire document to understand what is best for you, the following is the recommended configuration when running Emissary in AWS:

Debugging

Mon, 01 Jan 0001 00:00:00 +0000

If you’re experiencing issues with the Emissary and cannot diagnose the issue through the /ambassador/v0/diag/ diagnostics endpoint, this document covers various approaches and advanced use cases for debugging Emissary issues.

We assume that you already have a running Emissary installation in the following sections.

A Note on TLS

TLS can appear intractable if you haven’t set up certificates correctly. If you’re having trouble with TLS, always check the logs of your Emissary Pods and look for certificate errors.

Deployment architecture

Mon, 01 Jan 0001 00:00:00 +0000

Emissary can be deployed in a variety of configurations. The specific configuration depends on your data center.

Public cloud

If you’re using a public cloud provider such as Amazon, Azure, or Google, Emissary can be deployed directly to a Kubernetes cluster running in the data center. Traffic is routed to Emissary via a cloud-managed load balancer such as an Amazon Elastic Load Balancer or Google Cloud Load Balancer. Typically, this load balancer is transparently managed by Kubernetes in the form of the LoadBalancer service type. Emissary then routes traffic to your services running in Kubernetes.

Diagnostics

Mon, 01 Jan 0001 00:00:00 +0000

With Emissary Diagnostics and Ambassador Cloud, you get a summary of the current status and Mappings of your cluster and it’s services, which gets displayed in Diagnostics Overview.

Troubleshooting

Can’t access Emissary Diagnostics Overview?

Create an Ambassador Module if one does not already exist, and add the following config to enable diagnostics data.

apiVersion: getambassador.io/v3alpha1
kind: Module
metadata:
  name: ambassador
spec:
  config:
    diagnostics:
      enabled: true

Next, ensure that the AES_REPORT_DIAGNOSTICS_TO_CLOUD environment variable is set to "true" on the Agent deployment to allow diagnostics information to be reported to the cloud.

Environment Variables

Mon, 01 Jan 0001 00:00:00 +0000

Use the following variables for the environment of your Emissary container:

Variable	Default value	Value type
`AMBASSADOR_ID`	`[ "default" ]`	List of strings
`AES_LOG_LEVEL`	`warn`	Log level
`AGENT_CONFIG_RESOURCE_NAME`	`ambassador-agent-cloud-token`	String
`AMBASSADOR_AMBEX_NO_RATELIMIT`	`false`	Boolean: `true`=true, any other value=false
`AMBASSADOR_AMBEX_SNAPSHOT_COUNT`	`30`	Integer
`AMBASSADOR_CLUSTER_ID`	Empty	String
`AMBASSADOR_CONFIG_BASE_DIR`	`/ambassador`	String
`AMBASSADOR_DISABLE_FEATURES`	Empty	Any
`AMBASSADOR_DRAIN_TIME`	`600`	Integer
`AMBASSADOR_ENVOY_API_VERSION`	`V3`	String Enum; `V3` or `V2`
`AMBASSADOR_GRPC_METRICS_SINK`	Empty	String (address:port)
`AMBASSADOR_HEALTHCHECK_BIND_ADDRESS`	`0.0.0.0`	String
`AMBASSADOR_HEALTHCHECK_BIND_PORT`	`8877`	Integer
`AMBASSADOR_HEALTHCHECK_IP_FAMILY`	`ANY`	String Enum; `IPV4_ONLY` or `IPV6_ONLY`
`AMBASSADOR_ISTIO_SECRET_DIR`	`/etc/istio-certs`	String
`AMBASSADOR_JSON_LOGGING`	`false`	Boolean; non-empty=true, empty=false
`AMBASSADOR_READY_PORT`	`8006`	Integer
`AMBASSADOR_READY_LOG`	`false`	Boolean; Go `strconv.ParseBool`
`AMBASSADOR_LABEL_SELECTOR`	Empty	String (label=value)
`AMBASSADOR_NAMESPACE`	`default` (¹)	Kubernetes namespace
`AMBASSADOR_RECONFIG_MAX_DELAY`	`1`	Integer
`AMBASSADOR_SINGLE_NAMESPACE`	Empty	Boolean; non-empty=true, empty=false
`AMBASSADOR_SNAPSHOT_COUNT`	`4`	Integer
`AMBASSADOR_VERIFY_SSL_FALSE`	`false`	Boolean; `true`=true, any other value=false
`DD_ENTITY_ID`	Empty	String
`DOGSTATSD`	`false`	Boolean; Python `value.lower() == "true"`
`SCOUT_DISABLE`	`false`	Boolean; `false`=false, any other value=true
`STATSD_ENABLED`	`false`	Boolean; Python `value.lower() == "true"`
`STATSD_PORT`	`8125`	Integer
`STATSD_HOST`	`statsd-sink`	String
`STATSD_FLUSH_INTERVAL`	`1`	Integer
`_AMBASSADOR_ID`	Empty	String
`_AMBASSADOR_TLS_SECRET_NAME`	Empty	String
`_AMBASSADOR_TLS_SECRET_NAMESPACE`	Empty	String
`_CONSUL_HOST`	Empty	String
`_CONSUL_PORT`	Empty	Integer
`AMBASSADOR_DISABLE_SNAPSHOT_SERVER`	`false`	Boolean; non-empty=true, empty=false
`AMBASSADOR_ENVOY_BASE_ID`	`0`	Integer

Feature Flag Environment Variables

Variable	Default value	Value type
`AMBASSADOR_EDS_BYPASS`	`false`	Boolean; Python `value.lower() == "true"`
`AMBASSADOR_FORCE_SECRET_VALIDATION`	`false`	Boolean: `true`=true, any other value=false
`AMBASSADOR_KNATIVE_SUPPORT`	`false`	Boolean; non-empty=true, empty=false
`AMBASSADOR_UPDATE_MAPPING_STATUS`	`false`	Boolean; `true`=true, any other value=false
`ENVOY_CONCURRENCY`	Empty	Integer
`DISABLE_STRICT_LABEL_SELECTORS`	`false`	Boolean; `true`=true, any other value=false

`AMBASSADOR_ID`

Emissary supports running multiple installs in the same cluster without restricting a given instance of Emissary to a single namespace. The resources that are visible to an installation can be limited with the AMBASSADOR_ID environment variable.

GKE

Mon, 01 Jan 0001 00:00:00 +0000

Google offers a L7 load balancer to leverage network services such as managed SSL certificates, SSL offloading or the Google content delivery network. A L7 load balancer in front of Emissary can be configured by hand or by using the Ingress-GCE resource. Using the Ingress resource also allows you to create Google-managed SSL certificates through Kubernetes.

With this setup, HTTPS will be terminated at the Google load balancer. The load balancer will be created and configured by the Ingress-GCE resource. The load balancer consists of a set of forwarding rules and a set of backend services. In this setup, the ingress resource creates two forwarding rules, one for HTTP and one for HTTPS. The HTTPS forwarding rule has the SSL certificates attached. Also, one backend service will be created to point to a list of instance groups at a static port. This will be the NodePort of the Emissary service.

Gzip compression

Mon, 01 Jan 0001 00:00:00 +0000

Gzip enables Emissary to compress upstream data upon client request. Compression is useful in situations where large payloads need to be transmitted without compromising the response time. Compression can also save on bandwidth costs at the expense of increased computing costs.

How does it work?

When the gzip filter is enabled, request and response headers are inspected to determine whether or not the content should be compressed. If so, and the request and response headers allow, the content is compressed and then sent to the client with the appropriate headers. It also uses the zlib module, which provides Deflate compression and decompression code.

Host CRD

Mon, 01 Jan 0001 00:00:00 +0000

The custom Host resource defines how Emissary will be visible to the outside world. It collects all the following information in a single configuration resource:

The hostname by which Emissary will be reachable
How Emissary should handle TLS certificates
How Emissary should handle secure and insecure requests
Which Mappings should be associated with this Host

Remember that Listener resources are required for a functioning Emissary installation!
Learn more about Listener. Remember than Emissary does not make sure that a wildcard Host exists! If the wildcard behavior is needed, a Host with a hostname of "*" must be defined by the user.

A minimal Host resource, assuming no TLS configuration, would be:

HTTP/3 configuration

Mon, 01 Jan 0001 00:00:00 +0000

HTTP/3 in Emissary

HTTP/3 is the third version of the Hypertext Transfer Protocol (HTTP). It is built on the QUIC network protocol rather than Transmission Control Protocol (TCP) like previous versions.

The changes and challenges of HTTP/3

Since the QUIC network protocol is built on UDP, most clients will require Emissary to advertise its support for HTTP/3 using the alt-svc response header. This header is added to the response of the HTTP/2 and HTTP/1.1 connections. When the client sees the alt-svc it can choose to upgrade to HTTP/3 and connect to Emissary using the QUIC protocol.

Ingress controller

Mon, 01 Jan 0001 00:00:00 +0000

An Ingress resource is a popular way to expose Kubernetes services to the Internet. In order to use Ingress resources, you need to install an ingress controller. Emissary can function as a fully-fledged Ingress controller, making it easy to work with other Ingress-oriented tools within the Kubernetes ecosystem.

Listener CRD

Mon, 01 Jan 0001 00:00:00 +0000

The Listener CRD defines where, and how, Emissary should listen for requests from the network, and which Host definitions should be used to process those requests. For further examples of how to use Listener, see Configuring Emissary Communications.

Note that Listeners are never created by Emissary, and must be defined by the user. If you do not define any Listeners, Emissary will not listen anywhere for connections, and therefore won’t do anything useful. It will log a WARNING to this effect.

Load balancing

Mon, 01 Jan 0001 00:00:00 +0000

Load balancing configuration can be set for all Emissary mappings in the ambassador Module, or set per Mapping. If nothing is set, simple round robin balancing is used via Kubernetes services.

To use advanced load balancing, you must first configure a resolver that supports advanced load balancing (e.g., the Kubernetes Endpoint Resolver or Consul Resolver). Once a resolver is configured, you can use the load_balancer attribute. The following fields are supported:

load_balancer:
  policy: <load balancing policy to use>

Supported load balancer policies:

Performance and scaling

Mon, 01 Jan 0001 00:00:00 +0000

Scaling any cloud native application is inherently domain specific, however the content here reflects common issues, tips, and tricks that come up frequently.

Performance dimensions

The performance of Emissary’s control plane can be characterized along a number of different dimensions:

The number of TLSContext resources.
The number of Host resources.
The number of Mapping resources per Host resource.
The number of Mapping resources that will span all Host resources (either because they’re using host_regex, or because they’re using hostname: "*").

If your application involves a larger than average number of any of the above resources, you may find yourself in need of some of the content in this section.

Running and deployment

Mon, 01 Jan 0001 00:00:00 +0000

This section is intended for operators running Emissary, and covers various aspects of deploying and configuring the Emissary in production.

Emissary and Kubernetes

Emissary relies on Kubernetes for reliability, availability, and scalability. This means that features such as Kubernetes readiness and liveness probes, rolling updates, and the Horizontal Pod Autoscaling should be utilized to manage Emissary.

Default configuration

The default configuration of Emissary includes default resource limits, as well as readiness and liveness probes. These values should be adjusted for your specific environment.

Service discovery and resolvers

Mon, 01 Jan 0001 00:00:00 +0000

Service discovery is how cloud applications and their microservices are located on the network. In a cloud environment, services are ephemeral, existing only as long as they are needed and in use, so a real-time service discovery mechanism is required. Emissary uses information from service discovery to determine where to route incoming requests.

Emissary support for service discovery

Emissary supports different mechanisms for service discovery. These mechanisms are:

Kubernetes service-level discovery (default).
Kubernetes endpoint-level discovery.
Consul endpoint-level discovery.

Kubernetes service-level discovery

By default, Emissary uses Kubernetes DNS and service-level discovery. In a Mapping resource, specifying service: foo will prompt Emissary to look up the DNS address of the foo Kubernetes service. Traffic will be routed to the foo service. Kubernetes will then load balance that traffic between multiple pods. For more details on Kubernetes networking and how this works, see our blog post on Session affinity, load balancing controls, gRPC-Web, and Emissary.

Running on

Ambassador Module

Contents

AWS

Emissary with AWS

Recommended configuration

Debugging

A Note on TLS

Deployment architecture

Public cloud

Diagnostics

Troubleshooting

Can’t access Emissary Diagnostics Overview?

Environment Variables

Feature Flag Environment Variables

`AMBASSADOR_ID`

GKE

Gzip compression

How does it work?

Host CRD

HTTP/3 configuration

HTTP/3 in Emissary

The changes and challenges of HTTP/3

Ingress controller

Contents

Listener CRD

Load balancing

Performance and scaling

Performance dimensions

Running and deployment

Emissary and Kubernetes

Default configuration

Service discovery and resolvers

Emissary support for service discovery

Kubernetes service-level discovery