Free Ebook cover Web Servers 101: How HTTP, DNS, TLS, and Reverse Proxies Work

Web Servers 101: How HTTP, DNS, TLS, and Reverse Proxies Work

New course

13 pages

TLS Handshake in Words: Key Exchange, Session Keys, and SNI

Capítulo 9

Estimated reading time: 13 minutes

+ Exercise

What the TLS handshake is trying to achieve

The TLS handshake is the short conversation that happens right after a TCP connection is established and before any HTTP request is sent over that connection. Its job is to let the client (usually a browser or app) and the server agree on security parameters and produce shared secret keys that will encrypt and authenticate all application data that follows.

In practical terms, the handshake answers these questions:

  • Which TLS version and cipher suite will we use?
  • Which server identity are we talking to (especially when multiple hostnames share one IP)?
  • How do we create fresh session keys so that each connection has unique encryption keys?
  • Can we resume a previous session to avoid repeating expensive steps?

Even though you may hear “TLS handshake” as one thing, the exact message flow differs between TLS 1.2 and TLS 1.3. Modern browsers prefer TLS 1.3, but you will still encounter TLS 1.2 in older clients, legacy systems, and some enterprise environments. The key ideas—key exchange, session keys, and SNI—apply to both.

Key exchange in words: agreeing on a shared secret without sending it

Key exchange is the part of the handshake that lets client and server derive the same secret value even though an attacker can observe the network. That shared secret is not used directly to encrypt your HTTP data; instead it is fed into a key derivation function to produce multiple session keys (for encryption and integrity).

In modern TLS, key exchange is almost always based on (EC)DHE: (Elliptic Curve) Diffie–Hellman Ephemeral. “Ephemeral” matters: it means each connection uses a fresh, one-time key exchange, which provides forward secrecy. Forward secrecy means that even if the server’s long-term private key is compromised later, past captured traffic cannot be decrypted because the session keys were derived from ephemeral secrets that were never stored.

Continue in our app.

You can listen to the audiobook with the screen off, receive a free certificate for this course, and also have access to 5,000 other free online courses.

Or continue reading below...
Download App

Download the app

Diffie–Hellman (DHE/ECDHE) in plain language

Think of Diffie–Hellman like this: both sides contribute a random ingredient, and there is a public mixing process that produces a shared result. Each side can compute the shared result using its own private ingredient plus the other side’s public ingredient. An eavesdropper sees only the public ingredients and cannot feasibly compute the shared result.

In ECDHE, the “public ingredient” is an ephemeral public key on a chosen elliptic curve, and the “private ingredient” is the corresponding ephemeral private key. The shared secret is computed via elliptic-curve scalar multiplication.

Where the server certificate fits into key exchange

The server certificate is not the key exchange itself. Instead, it is used to authenticate the server during the handshake. The server proves it controls the private key corresponding to the certificate by signing parts of the handshake. This prevents a man-in-the-middle from swapping in their own key exchange parameters.

So you can separate the roles:

  • Key exchange (ECDHE): produces shared secret material for session keys.
  • Certificate + signature: proves the server is the intended identity and binds the handshake to that identity.

Session keys: what gets created and what they are used for

After key exchange, both sides derive a set of symmetric keys. Symmetric cryptography is used for bulk data because it is fast. TLS uses authenticated encryption (AEAD) ciphers such as AES-GCM or ChaCha20-Poly1305, which provide both confidentiality (encryption) and integrity (tamper detection) in one construction.

Session keys are connection-specific and short-lived. They are derived from:

  • Random values exchanged in the handshake (to ensure uniqueness and prevent replay issues).
  • The shared secret from the key exchange.
  • Handshake transcript data (in TLS 1.3) to bind keys to the exact handshake that occurred.

Different keys for different directions

TLS does not use one key for everything. At minimum, there are separate keys for client-to-server and server-to-client traffic. This separation reduces the risk of certain cryptographic attacks and simplifies nonce/sequence management.

In AEAD modes, each record also uses a nonce (sometimes derived from a per-connection IV plus a record sequence number). The important operational point: both sides must stay in sync on record numbers; if packets are lost or reordered, TLS runs over TCP so the stream is reassembled in order before TLS records are processed.

Handshake keys vs application data keys (TLS 1.3)

TLS 1.3 derives multiple generations of keys:

  • Handshake traffic keys: used to encrypt most of the handshake after the initial messages.
  • Application traffic keys: used to encrypt HTTP data after the handshake completes.
  • Resumption master secret: used to create tickets/PSKs for future session resumption.

This staged approach means that sensitive handshake messages (like the certificate) are encrypted in TLS 1.3, unlike TLS 1.2 where the certificate is typically sent in cleartext.

SNI: how the client tells the server which hostname it wants

SNI (Server Name Indication) is a TLS extension that lets the client include the intended hostname in the ClientHello message. This matters because many servers host multiple domains on the same IP address (virtual hosting). The server needs to know which hostname the client is trying to reach so it can choose the correct certificate and configuration.

Without SNI, a server would have to guess which certificate to present. Historically, that meant either:

  • One certificate that covers many names (wildcards or multi-SAN certificates), or
  • One IP address per hostname, which does not scale well.

With SNI, the server can select the right certificate for api.example.com vs www.example.com even if they share the same IP and port.

Important nuance: SNI is visible on the network (unless ECH is used)

In typical deployments today, SNI is sent in cleartext because it is part of the ClientHello, which must be readable by the server before encryption is established. This means observers can often see which hostname you are connecting to, even though the HTTP content is encrypted.

There is a newer mechanism called ECH (Encrypted ClientHello) designed to encrypt SNI and other ClientHello details, but it requires coordinated support from clients, servers, and often CDNs. You may encounter it in modern browsers for some sites, but it is not universal.

TLS 1.3 handshake: step-by-step in words

Below is a practical, message-oriented walkthrough of a typical TLS 1.3 handshake using ECDHE, with SNI present. The exact set of extensions varies, but the structure is consistent.

Step 1: ClientHello (client → server)

The client starts by sending a ClientHello. It includes:

  • Supported TLS versions (TLS 1.3 and possibly TLS 1.2 for fallback).
  • Supported cipher suites (for TLS 1.3 these are AEAD + hash combinations).
  • Supported key exchange groups (elliptic curves like X25519, secp256r1).
  • Key share: the client’s ephemeral public key for one or more groups.
  • SNI: the hostname it wants (e.g., shop.example.com).
  • ALPN: which application protocol it wants over TLS (commonly h2 for HTTP/2 or http/1.1).
  • Optional PSK/resumption info: if the client is attempting session resumption or 0-RTT.

Operationally, SNI and ALPN are two of the most important “routing” hints. SNI helps pick the certificate; ALPN helps pick HTTP/2 vs HTTP/1.1 behavior after the handshake.

Step 2: ServerHello (server → client)

The server responds with ServerHello, choosing:

  • The TLS version (TLS 1.3).
  • The cipher suite from the client’s offered list.
  • The key exchange group and the server’s key share (server ephemeral public key).

At this point, both sides have enough information to compute the ECDHE shared secret (client uses its ephemeral private key + server public key; server uses its ephemeral private key + client public key). From that shared secret, they derive handshake traffic keys.

Step 3: Encrypted handshake messages (server → client)

After ServerHello in TLS 1.3, the rest of the handshake is encrypted with the handshake traffic keys. The server typically sends:

  • EncryptedExtensions: confirms negotiated extensions (for example ALPN selection).
  • Certificate: the server’s certificate chain (encrypted in TLS 1.3).
  • CertificateVerify: a signature over the handshake transcript proving possession of the certificate private key.
  • Finished: a MAC over the transcript proving the server derived the same handshake secrets.

From a troubleshooting perspective, this is why packet captures of TLS 1.3 show less detail unless you have keys for decryption: the certificate and many parameters are not visible.

Step 4: Client verifies and finishes (client → server)

The client validates the server’s identity (certificate checks were covered elsewhere) and then sends its own Finished message, proving it also derived the same handshake secrets.

Once both Finished messages are exchanged, the handshake is complete and both sides switch to application traffic keys. Now the client can send the HTTP request over the encrypted channel.

TLS 1.2 handshake: step-by-step in words (for comparison)

You will still see TLS 1.2 in logs and diagnostics. The flow is similar but with key differences: more handshake data is in cleartext, and the key schedule is different.

Step 1: ClientHello (client → server)

ClientHello includes the client random, supported cipher suites, extensions (including SNI and ALPN), and supported elliptic curves. In TLS 1.2, the client typically does not send a key share in the first message (though there are extensions that can).

Step 2: ServerHello + Certificate + ServerKeyExchange (server → client)

The server responds with:

  • ServerHello: selects version and cipher suite.
  • Certificate: usually sent in cleartext in TLS 1.2.
  • ServerKeyExchange: contains the server’s ephemeral ECDHE parameters and a signature binding them to the certificate.
  • ServerHelloDone: indicates the server is done with its hello/key exchange messages.

Step 3: ClientKeyExchange + Finished (client → server)

The client sends:

  • ClientKeyExchange: the client’s ephemeral ECDHE public key.
  • ChangeCipherSpec: signals that subsequent records will be encrypted (a TLS 1.2 mechanism).
  • Finished: proves key agreement and transcript integrity.

Step 4: Server ChangeCipherSpec + Finished (server → client)

The server switches to encrypted mode and sends its Finished. After that, application data can flow.

Practical implication: TLS 1.2 usually costs more round trips than TLS 1.3, which is one reason TLS 1.3 improves latency.

Session resumption: reusing trust without redoing the full handshake

Full handshakes require public-key operations and extra round trips. Session resumption reduces cost by letting the client and server establish new session keys based on a previously established shared secret, without repeating the full certificate and key exchange process.

There are two common resumption styles you will see in practice:

  • Session IDs (older): the server stores session state and the client presents an ID to resume it.
  • Session tickets / PSK (common today): the server gives the client a ticket that encodes resumption state; the client presents it later as a PSK identity.

In TLS 1.3, resumption is PSK-based. The server issues one or more tickets after the handshake. On a later connection, the client includes PSK identities in ClientHello. If the server accepts one, both sides can derive new keys quickly.

0-RTT early data (TLS 1.3) and when not to use it

TLS 1.3 optionally allows the client to send “early data” (0-RTT) immediately with the first flight, before the handshake fully completes, when resuming with a PSK. This can reduce latency, but it has a major caveat: early data can be replayed by an attacker in some network scenarios.

Practical guidance:

  • Only allow 0-RTT for requests that are safe to replay (typically idempotent reads like GET for cacheable resources).
  • Do not allow 0-RTT for operations that change state (payments, writes, account actions) unless you have robust replay protection at the application layer.

How SNI and reverse proxies/CDNs affect the handshake

In many real deployments, the TLS endpoint is not the same machine that runs your application. A reverse proxy, load balancer, or CDN edge terminates TLS and then forwards the request to an origin server. In that setup:

  • The client’s TLS handshake happens with the edge/proxy.
  • SNI is used by the edge to select the correct certificate and routing configuration.
  • The edge may open a separate connection to the origin, which may be plain HTTP or may be another TLS connection (often called “TLS to origin”).

This matters when debugging because the certificate the client sees is the edge’s certificate, not necessarily the origin’s. Also, the cipher suite and TLS version are negotiated between client and edge; the edge-to-origin connection can use different settings.

Practical walkthrough: reading a handshake from real tooling

You often need to confirm which TLS version, cipher, SNI, and ALPN were negotiated. The following are practical steps using common command-line tools. These examples focus on what to look for rather than repeating general HTTPS basics.

Use OpenSSL to inspect SNI, certificate selection, and negotiated parameters

OpenSSL can initiate a TLS connection and show handshake details. The key is to provide SNI explicitly with -servername, otherwise you might see the “default” certificate on a multi-tenant server.

openssl s_client -connect 203.0.113.10:443 -servername shop.example.com -alpn h2 -tls1_3

What to look for in the output:

  • Server certificate subject/SAN: confirms the server picked the certificate for the SNI name.
  • ALPN protocol: shows whether h2 or http/1.1 was selected.
  • TLS version and cipher: confirms TLS 1.3 vs 1.2 and the AEAD cipher.

If you omit -servername on a shared IP, you may see a certificate mismatch or a generic certificate. That is often the fastest way to demonstrate what SNI is doing.

Compare behavior with and without SNI

Try the same command without SNI:

openssl s_client -connect 203.0.113.10:443 -alpn h2

Possible outcomes:

  • You get a different certificate (the server’s default vhost).
  • The handshake fails if the server requires SNI to route properly.
  • You still get a valid certificate if the server uses a single certificate covering many names.

Use curl to see negotiated TLS and HTTP protocol

curl can show which TLS version and HTTP protocol were used. This is useful when ALPN negotiation changes behavior (HTTP/2 vs HTTP/1.1).

curl -v --http2 https://shop.example.com/

In verbose output, look for lines indicating:

  • TLS handshake completion and negotiated cipher.
  • ALPN selection (e.g., “server accepted h2”).
  • Whether the connection is reused for subsequent requests (separate from TLS resumption, but often observed together).

Common handshake failure modes tied to key exchange, session keys, and SNI

SNI-related certificate mismatch

Symptom: the client reports a hostname mismatch, or you see the wrong certificate being served.

Typical causes:

  • Client did not send SNI (older clients, misconfigured libraries, or direct IP connections).
  • Proxy/load balancer has incorrect SNI routing rules.
  • Multiple certificates installed but the default certificate is not appropriate.

Practical fix path: confirm with OpenSSL using -servername, then check the TLS termination layer’s virtual host configuration and certificate bindings.

Key exchange group mismatch

Symptom: handshake fails early with errors like “no shared cipher” or “handshake failure”.

Typical causes:

  • Server only supports a limited set of elliptic curves/groups.
  • Client is constrained (embedded devices, strict crypto policies).

Practical fix path: ensure overlap in supported groups (e.g., enable X25519 and secp256r1) and modern cipher suites.

Resumption issues (tickets/PSK not accepted)

Symptom: you expect fast resumption but see full handshakes repeatedly.

Typical causes:

  • Load balancing without shared ticket keys or without sticky sessions (server cannot decrypt/accept the ticket).
  • Tickets expiring quickly or being disabled.
  • Client privacy settings clearing session state frequently.

Practical fix path: configure consistent session ticket keys across TLS terminators, or use a resumption mechanism compatible with your load balancing strategy.

Decrypt errors after handshake (session key mismatch)

Symptom: handshake completes but then you see record decryption failures.

Typical causes:

  • Middleboxes interfering with TLS traffic.
  • Broken TLS interception or misconfigured proxies.
  • Implementation bugs or incorrect offload settings in some appliances.

Practical fix path: reproduce with a direct path (bypass proxy), compare TLS versions, and inspect whether any device is terminating and re-encrypting traffic unexpectedly.

Now answer the exercise about the content:

In a typical TLS 1.3 connection using ECDHE, what is the primary role of the server certificate during the handshake?

You are right! Congratulations, now go to the next page

You missed! Try again.

The certificate is for authentication: the server signs parts of the handshake to prove it controls the private key, preventing a man-in-the-middle from swapping parameters. The shared secret and session keys come from ECDHE and key derivation.

Next chapter

Reverse Proxies: Front Doors for Web Servers and Applications

Arrow Right Icon
Download the app to earn free Certification and listen to the courses in the background, even with the screen off.