Ingress and TLS

Getting HTTPS traffic to a ClusterIP Service through an Ingress, and making the app see the original scheme and client IP.

Syntaxoptions.ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto;

A ClusterIP Service can only be reached from inside the cluster. To let the public in, you need one address on the outside that forwards to the right Service. That is what an Ingress describes: a set of rules matching a host name and a URL path to a Service and port.

The Ingress object does nothing on its own. A separate program in the cluster, the ingress controller, reads these objects and configures a real proxy. ingressClassName says which controller should take the rule.

Host and path routing

Rules match on host, then on path with a pathType of Prefix, Exact or ImplementationSpecific. So api.example.com/orders can reach one Service and /customers another, with both pods unchanged.

TLS stops at the ingress

Certificates live in a Secret of type kubernetes.io/tls, holding tls.crt and tls.key, named by the Ingress's tls section. The controller terminates TLS there and forwards plain HTTP to your pod. Your app therefore sees http, a proxy's IP as the client, and often a different host.

Getting the real values back

The proxy puts them in X-Forwarded-Proto and X-Forwarded-For. UseForwardedHeaders copies them onto the request, and must run before anything that reads the scheme or client IP. It only trusts loopback by default, so the ingress controller's pod network has to be added to KnownIPNetworks. That property is the .NET 10 name; the older KnownNetworks is now obsolete.

The Ingress API is stable but closed to new features. New clusters are increasingly configured with the Gateway API instead, which expresses the same routing in richer objects.

Example

using Microsoft.AspNetCore.HttpOverrides;

var builder = WebApplication.CreateBuilder(args);

builder.Services.Configure<ForwardedHeadersOptions>(options =>
{
    options.ForwardedHeaders =
        ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto;

    // Only loopback is trusted by default, and the ingress controller is
    // not on loopback. Clearing both lists trusts the headers from any
    // caller, which is only safe while the pod is unreachable except
    // through the ingress. The next example narrows it to one network.
    options.KnownIPNetworks.Clear();
    options.KnownProxies.Clear();
});

var app = builder.Build();

// Must run before anything that reads the scheme or the client address.
app.UseForwardedHeaders();

app.MapGet("/whoami", (HttpContext context) => new
{
    scheme = context.Request.Scheme,
    host = context.Request.Host.Value,
    clientIp = context.Connection.RemoteIpAddress?.ToString()
});

app.Run();

When to use it

Publish a .NET API on a public host name with HTTPS.
Put two services behind one domain on different URL prefixes.
Make request logs and rate limiting see the real client IP instead of the proxy's.

More examples

The Ingress

Two .NET services behind one host name and one certificate. The port number is the Service's port; the Service already knows to forward to 8080 in the container. Note that the path is not stripped: a request to /orders/1 arrives at the pod as /orders/1, so the routes in the app must include the prefix, or the controller must be told to rewrite it.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: api
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - api.example.com
      secretName: api-tls          # a kubernetes.io/tls Secret
  rules:
    - host: api.example.com
      http:
        paths:
          - path: /orders
            pathType: Prefix
            backend:
              service:
                name: orders       # the ClusterIP Service
                port:
                  number: 80       # the Service port, not the container port
          - path: /customers
            pathType: Prefix
            backend:
              service:
                name: customers
                port:
                  number: 80

Trusting only the ingress network

This is the version to ship when the client IP is used for anything that matters, such as rate limiting or an audit log. An untrusted caller can put any value in X-Forwarded-For, so a cleared trust list means any caller who can reach the pod directly can claim to be any address.

using System.Net;
using Microsoft.AspNetCore.HttpOverrides;

var builder = WebApplication.CreateBuilder(args);

builder.Services.Configure<ForwardedHeadersOptions>(options =>
{
    options.ForwardedHeaders =
        ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto;

    // Accept forwarded headers only from the cluster's pod network.
    // Check your cluster's pod CIDR; 10.42.0.0/16 is one common value.
    // System.Net and Microsoft.AspNetCore.HttpOverrides both declare an
    // IPNetwork, so the plain name is ambiguous here; name the System.Net
    // one, which is the type KnownIPNetworks holds.
    options.KnownIPNetworks.Add(
        new System.Net.IPNetwork(IPAddress.Parse("10.42.0.0"), 16));

    // One proxy only, so only the last entry in the chain is believed.
    options.ForwardLimit = 1;
});

var app = builder.Build();
app.UseForwardedHeaders();
app.MapGet("/orders/1", () => new { id = 1, total = 42 });
app.Run();

Certificate and a check through the proxy

Running both calls is the quickest way to prove the forwarded headers are being applied: the scheme should read https through the ingress and http on the port-forward. If it reads http in both, either UseForwardedHeaders is missing, it runs too late, or the caller is not in the trust list.

# A certificate you already hold, as a kubernetes.io/tls Secret.
kubectl create secret tls api-tls --cert=tls.crt --key=tls.key

# In a real cluster cert-manager issues and renews it instead, by
# annotating the Ingress with cert-manager.io/cluster-issuer.

# Through the ingress: TLS outside, plain HTTP inside.
curl https://api.example.com/whoami
# {"scheme":"https","host":"api.example.com","clientIp":"203.0.113.7"}

# Straight at the pod, with no proxy and no headers.
kubectl port-forward deployment/orders 8080:8080
curl http://localhost:8080/whoami
# {"scheme":"http","host":"localhost:8080","clientIp":"127.0.0.1"}

Discussion

Be the first to comment on this lesson.