Handshake

In short: The connection setup between two systems, in which both sides agree on parameters for further communication before the actual data flows.

In more detail: The term is used at several levels: in the TCP three-way handshake, two hosts agree on a reliable connection (SYN/SYN-ACK/ACK). In the TLS handshake, client and server additionally agree on a cipher suite, exchange certificates and negotiate a session key. Only after a successful handshake does the actual data transfer begin.

In Depth

The handshake principle recurs at different network layers, each with its own goal:

TCP three-way handshake   - layer 4 (transport): establishes a reliable
                             connection (SYN, SYN-ACK, ACK)
TLS handshake             - the layer above: negotiates encryption,
                             exchanges certificates, negotiates session keys
SSH handshake             - similar to TLS, but for SSH connections

The order is important: before a TLS handshake can take place at all, the TCP three-way handshake must first have completed successfully — TLS builds on an already existing, reliable TCP connection and ONLY takes care of encryption and authenticity, not of the basic delivery of the data packets. In a typical HTTPS connection, the TCP handshake therefore happens first, then the TLS handshake, and only after that does the actual encrypted data transfer begin.

A failed handshake is one of the most common causes of connection errors in practice: if the TCP handshake doesn’t go through, the server isn’t reachable (e.g. because of a firewall rule or because the service isn’t running). If the TLS handshake doesn’t go through, there’s usually a certificate problem (expired, wrong host name) or there’s no cipher suite supported by both client and server.

Why an explicit negotiation phase at all?

The basic idea behind every handshake principle: two systems that know nothing about each other beforehand first have to agree on a shared “language” before meaningful communication is possible — similar to two people meeting for the first time who first clarify which language they both speak before a real conversation begins. Without this negotiation phase, both sides would have to use exactly the same, fixed configuration from the outset — every future protocol improvement would then immediately exclude all older, non-updated systems from communication. The handshake instead enables backwards compatibility: a modern server can talk to older clients just as well as to brand-new ones, because both sides find out in the handshake which capabilities they have in common.

Handshakes beyond TCP and TLS

The handshake principle isn’t limited to network protocols: many other forms of communication use the same basic pattern too. In the 1990s, modems agreed on the best mutually supported transmission speed when dialling in over a telephone line via a characteristic, audible “screeching”. Bluetooth devices go through a cryptographic handshake when “pairing”, which establishes a shared key for the further connection. Similar patterns can even be found at the application level, for example in the WebSocket protocol, which “upgrades” an existing HTTP connection via a special handshake into a permanent, bidirectional connection (Upgrade header).

Time cost of a handshake

Every handshake costs time — at least one, often several network round trips before the actual payload can flow. With a high-latency connection (e.g. over a satellite link with a round-trip time of several hundred milliseconds), this is clearly noticeable — one reason why modern protocols such as TCP Fast Open and TLS 1.3 specifically work on minimising the number of round trips needed in the handshake, especially for repeat connections to already known destinations.

See also: Handshake Protocol, Three-way handshake