When two computers need to exchange data, they have to agree on a long list of things: how data is packaged, who speaks first, what a response means, and what happens if something goes wrong. Those agreements are called network protocols, and the internet is essentially a large collection of them working together.
This guide is an overview: what a protocol is, why protocols are built in layers, how they depend on each other, and how they handle errors. Along the way, we point to the guides that go into depth on the individual protocols.
What is a protocol?
A protocol defines three things: the format (what a message looks like, and where each field goes), the order (who sends what when) and the meaning (what the recipient should do with the message). Take a browser fetching a page with HTTP. The format says the request starts with a method and an address. The order says the client asks and the server answers. And the meaning says the response 404 means the page doesn’t exist. Because the rules are written down and public, a browser from one company can talk to a server from a completely different one without them ever having met.
Layers and encapsulation
No single protocol solves the whole task. Instead they’re organized in layers, where each layer solves one problem and builds on the layer below. At the bottom, it’s about getting signals across a wire or through the air. Above that, it’s about finding the way between networks, above that about delivering data reliably between two programs, and at the top about what the programs actually want to say to each other.
The layers work together through encapsulation. When a program sends a message, each layer adds its own header with the information it needs and passes the result down. At the recipient, the reverse happens: each layer reads and removes its header and passes the rest up.
Application [ HTTP message ]
Transport [ TCP header | HTTP message ]
Internet [ IP header | TCP header | HTTP message ]
Link [ Ethernet | IP header | TCP header | HTTP message | Ethernet ]The beauty of it is that the layers don’t need to know each other. TCP doesn’t know whether it’s carrying HTTP or email, and IP doesn’t know whether the packet is sent over fiber or Wi-Fi. That’s why a layer can be replaced or improved without the rest having to change. It’s the reason the same web page can be fetched over a mobile network, a cable or a satellite. At the very bottom, it’s all just bits, which we explain in the guide to binary numbers.
There are two models for the layers. The OSI model has seven layers and is mostly used as a shared vocabulary and in teaching. The TCP/IP model has four and describes what the internet is actually built from. When someone talks about “layer 3” and “layer 4”, they mean OSI’s network and transport layers, which correspond to IP and TCP. We go through the practical model in the guide to TCP/IP.
With or without a connection
Protocols can roughly be split into two families. Connection-oriented protocols, like TCP, start by establishing a connection before any data is sent. Both parties then know they’re talking to each other, and the protocol can keep track of what has been received, resend what’s missing, and deliver everything in order. It’s like a phone call: you dial, talk and hang up.
Connectionless protocols, like UDP and IP itself, send each message on its own without any prior agreement, more like sending a postcard. There’s nothing to set up first and nothing to clean up afterwards, but also no guarantee the message arrives.
Neither is better. A file transfer or a website has to arrive whole, and then it’s worth paying for the connection. A DNS lookup is one question and one answer, and a video call would rather lose a frame than wait for it, so there the connectionless model is right. Modern protocols like QUIC mix the two: they build a connection on top of connectionless UDP to get the best of both.
Network architecture
Most of the internet follows the client-server model: a client, such as a browser, sends requests to a server, which answers. It’s simple to manage and secure, because data and logic are centralized. Peer-to-peer is the alternative, where every participant both asks and answers, as in file sharing and some video calls. It scales well, but is harder to manage.
Physically, networks are built from local networks (LANs), like your home or office, connected by larger networks (WANs) and finally by the internet’s big hubs. Local networks today are almost always built as a star, with every device connected to a central switch or Wi-Fi access point. How devices find each other across all of that is a question of addresses, which we cover in the guide to IP addresses.
How protocols depend on each other
Even something as everyday as opening a website on a laptop that has just been switched on relies on a whole chain of protocols, and each of them has to succeed before the next can start:
- DHCP gives the computer an IP address and says where the router and DNS server are.
- ARP finds the router’s MAC address on the local network.
- DNS translates the website’s name into an IP address.
- TCP (or QUIC) opens a connection to the server.
- TLS encrypts the connection and checks the server’s certificate.
- HTTP fetches the page itself.
The chain shows both the strength and the weakness. Each protocol can be kept simple because it can rely on the others. But if one link fails, everything stops, and the error often shows up somewhere completely different from where it occurs. A DNS server that’s down looks to the user like “the internet isn’t working”, even though everything else is fine. That’s why the protocols everything else depends on, like DNS, are also the ones built with the most redundancy.
Errors and robustness
Networks are unreliable by nature: packets get lost, bits get corrupted, and servers go down. Protocols are designed on the assumption that this will happen. Checksums in the packets reveal data that has been corrupted along the way. Acknowledgments and retransmission make sure lost data is sent again. Timeouts make sure a party doesn’t wait forever for an answer that never comes, and many protocols have built-in fallbacks, so a client can try another server or fall back to an older version.
There’s also a design philosophy behind it, known as the robustness principle: be strict in what you send, and tolerant in what you receive. It has made it possible for thousands of different implementations to work together, but experience has shown that too much tolerance can hide errors and create security holes, so newer protocols are more precise about what’s allowed.
Who decides the protocols?
The internet’s protocols are open standards. Most are developed in the IETF (Internet Engineering Task Force), where anyone can take part, and described in documents called RFCs. Other organizations are responsible for other parts: the IEEE for Ethernet and Wi-Fi, and the W3C for many web standards. The fact that no single company owns the protocols is a big part of why the internet has been able to grow the way it has. You can read how that happened in our guide to the history of network protocols.