Handshake Protocol

In short: The TLS sub-protocol that governs connection setup: negotiating the cipher suite, exchanging certificates and agreeing on the session key.

In more detail: A simplified sequence: the client sends a “ClientHello” message with supported cipher suites and TLS versions. The server answers with “ServerHello” (the chosen cipher suite), its certificate and possibly further parameters. A shared session key is then established via asymmetric cryptography. Only after that do both sides signal via the change cipher spec protocol that communication is encrypted from now on.

In Depth

The complete handshake sequence in TLS 1.3 (simplified; more modern and faster than TLS 1.2):

Client                                              Server
  |-- ClientHello (supported cipher suites,
  |    key exchange guessed in advance) ------------>|
  |<---- ServerHello (chosen cipher suite,
  |       own key share) ----------------------------|
  |<---- EncryptedExtensions, certificate -----------|
  |<---- CertificateVerify (server signs
  |       the handshake transcript so far) ----------|
  |<---- Finished -----------------------------------|
  |-- Finished ------------------------------------->|
  |== from here: encrypted application data ========|

The biggest difference from TLS 1.2: TLS 1.3 only needs a single round trip (1-RTT) instead of two, because client and server already send their key exchange parameters in ClientHello/ServerHello instead of negotiating them in separate messages afterwards — which saves a complete network round trip and makes connection setup noticeably faster. In addition, TLS 1.3 supports “0-RTT” for repeat connections to an already known server: the client can already send encrypted data with the very first message, with the drawback, however, that this first data is vulnerable to so-called replay attacks (an attacker could record the first message and play it back again).

CertificateVerify is the decisive proof of authenticity: with it, the server cryptographically signs the entire handshake transcript so far with its private key — the client can check this signature with the public key from the server certificate and can thus be sure that it’s really talking to the holder of the private key, not to a man in the middle.

The path from TLS 1.2 to TLS 1.3

TLS 1.2 (2008) still needed two complete network round trips for a full handshake: first client and server exchanged their supported cipher suites, and ONLY THEN did the actual key exchange follow in a second round trip. TLS 1.3 (2018) shortened this drastically by having the client already send its “best guess” for the key exchange in the very first ClientHello (typically for the parameters the server is known from experience to support) — if the guess is right, a single round trip is enough. If the guess is wrong, the protocol falls back to an additional HelloRetryRequest, which rarely happens in practice, however, since the common parameter combinations are now largely standardised.

Forward secrecy as a mandatory component

An important security advance in TLS 1.3: perfect forward secrecy (via ephemeral Diffie-Hellman keys, see cipher suite) is no longer optional but mandatory for EVERY connection. This means: even if an attacker at some later point obtains a server’s long-term private key, they can’t use it to decrypt encrypted connections recorded in the past — each session used its own temporary key, discarded afterwards. This property gained particular importance after the Snowden revelations in 2013, when it became known that intelligence agencies were systematically recording encrypted internet traffic in the hope of being able to decrypt it later with captured keys.

Session resumption: faster reconnection

For repeat connections to an already known server, TLS 1.3 supports “session resumption” via so-called pre-shared keys (PSK): instead of carrying out the complete handshake again, the client uses a key derived from the previous visit to restore the connection much faster (sometimes with just a single message, see 0-RTT in the application data protocol) — noticeably relevant for the loading time of websites that are visited frequently.

Common causes of handshake failures in practice

Typical causes of failed handshakes: an expired certificate or one issued for the wrong host name, no shared cipher suite (e.g. when an old client only supports outdated methods that the server deliberately no longer offers), or a misconfigured server clock (certificate validity is checked based on time — a strongly deviating system time can make valid certificates wrongly appear expired).

See also: Handshake, TLS Record Protocol, Change Cipher Spec Protocol