IP/Network Layer

Basic Understanding of IP#

The main function of the network layer is to enable communication between hosts, also known as end-to-end communication.

What is the relationship between the network layer and the data link layer?#

The role of MAC is to facilitate communication between two directly connected devices, while IP is responsible for communication transmission between two networks that are not directly connected.

In the transmission of data packets over the network, the source IP address and destination IP address do not change during transmission (provided that NAT is not used), only the source MAC address and destination MAC address keep changing.

Basic Knowledge of IP Addresses#

In TCP/IP network communication, to ensure normal communication, each device needs to be configured with the correct IP address; otherwise, normal communication cannot be achieved.
An IP address (IPv4 address) is represented by a 32-bit positive integer, and IP addresses are processed in binary by computers.
For convenience, humans use a dotted decimal notation, which divides the 32-bit IP address into 4 groups of 8 bits each, separated by a "." and converts each group into decimal.

Classification of IP Addresses#

IP addresses are classified into 5 types: Class A, Class B, Class C, Class D, and Class E.

What are Class A, B, and C addresses?

For Class A, B, and C, they are mainly divided into two parts: the network number and the host number.

How is the maximum number of hosts for Class A, B, and C addresses calculated?

The maximum number of hosts depends on the number of bits in the host number. For example, for Class C addresses, the host number occupies 8 bits, so the maximum number of hosts for Class C addresses is: $2^8 - 2 = 254$
Why subtract 2?
Because there are two special IP addresses in IP addressing: one where the host number is all 1s and one where it is all 0s.

The host number all being 1s designates all hosts within a certain network for broadcasting.
The host number all being 0s designates a specific network.

Therefore, during allocation, these two cases should be excluded.

What is the broadcast address used for?

The broadcast address is used to send data packets between hosts that are interconnected within the same link.
When the host number is all 1s, it indicates the broadcast address for that network. For example, the binary representation of 172.20.0.0/16 is:
10101100.00010100.00000000.00000000
Changing all the host part of this address to 1s forms the broadcast address:
10101100.00010100.11111111.11111111
In decimal, this address is represented as 172.20.255.255.
Broadcast addresses can be divided into local broadcast and direct broadcast.

A local broadcast occurs within the same network. For example, if the network address is 192.168.0.0/24, the broadcast address is 192.168.0.255. Since this broadcast address's IP packets will be filtered by routers, they will not reach other links outside of 192.168.0.0/24.
A direct broadcast occurs between different networks. For example, if a host with the network address 192.168.0.0/24 sends an IP packet to the target address 192.168.1.255/24. The router receiving this packet will forward the data to 192.168.1.0/24, allowing all hosts from 192.168.1.1 to 192.168.1.254 to receive this packet (due to certain security issues with direct broadcasts, they are usually set not to forward on routers).

What are Class D and E addresses?

Class D and E addresses do not have host numbers, so they cannot be used for host IPs. Class D is commonly used for multicast, while Class E is a reserved classification that is not currently in use.

What is the multicast address used for?

Multicast is used to send packets to all hosts within a specific group.

The multicast addresses used are Class D addresses, where the first four bits are 1110, indicating it is a multicast address, and the remaining 28 bits represent the multicast group number.
The range for multicast is from 224.0.0.0 to 239.255.255.255, which is divided into three categories:

224.0.0.0 to 224.0.0.255 are reserved multicast addresses, which can only be used within a local area network, and routers will not forward them.
224.0.1.0 to 238.255.255.255 are user-available multicast addresses that can be used on the Internet.
239.0.0.0 to 239.255.255.255 are locally scoped multicast addresses, which can be used internally within a private network and are only valid within a specific local range.

Advantages of IP classification

Whether it is a router or a host resolving an IP address, we can quickly determine if the first bit of the IP address is 0. If it is 0, it indicates a Class A address, allowing us to quickly identify the network address and host address.

Disadvantages of IP classification

Disadvantage 1:
There is no address hierarchy within the same network. For example, a company may use a Class B address but may need to classify addresses based on production, testing, and development environments. However, this IP classification lacks the functionality to classify address hierarchies, leading to a lack of address flexibility.
Disadvantage 2:
Classes A, B, and C face an awkward situation where they do not match well with real-world networks.

The maximum number of hosts that Class C addresses can accommodate is too few, only 254, which may not even be enough for an internet café.
Conversely, the maximum number of hosts that Class B addresses can accommodate is too many, with over 60,000 machines under one network, which is generally beyond the scale of most enterprises, leading to wasted addresses.

Both of these disadvantages can be resolved with CIDR (Classless Inter-Domain Routing).

Classless Addressing CIDR#

Due to the many drawbacks of IP classification, a classless addressing scheme was proposed later, known as CIDR.
This method no longer has the concept of class addresses; the 32-bit IP address is divided into two parts: the network number and the host number.

How is the network number and host number divided?

The representation is a.b.c.d/x, where /x indicates that the first x bits belong to the network number, and x can range from 0 to 32, making IP addresses more flexible.

Another way to divide the network number and host number is through a subnet mask, which means masking the host number, leaving the network number.
By performing a bitwise AND operation between the subnet mask and the IP address, the network number can be obtained.

Why separate the network number and host number?

Because for two computers to communicate, it is first necessary to determine whether they are in the same broadcast domain, i.e., whether their network addresses are the same. If the network addresses are the same, it indicates that the recipient is on the same network, allowing the data packet to be sent directly to the target host.
In the routing addressing process, this is how the corresponding network number is found, allowing the data packet to be forwarded to the corresponding network.

How to perform subnetting?

As we know, we can use the subnet mask to divide the network number and host number. In fact, the subnet mask also serves to divide subnets.
Subnetting essentially divides the host address into two parts: the subnet network address and the subnet host address. The format is as follows:

IP address without subnetting: network address + host address
IP address after subnetting: network address + (subnet network address + subnet host address)

Assuming we subnet a Class C address with a network address of 192.168.1.0, using a subnet mask of 255.255.255.192.
In a Class C address, the first 24 bits are the network number, and the last 8 bits are the host number. According to the subnet mask, we can borrow 2 bits from the 8-bit host number to serve as the subnet number.

Public IP Addresses and Private IP Addresses#

In Class A, B, and C addresses, there is a distinction between public IP addresses and private IP addresses.

Who manages public IP addresses?
Private IP addresses are usually managed by internal IT personnel, while public IP addresses are managed by the ICANN organization, known in Chinese as "互联网名称与数字地址分配机构" (Internet Corporation for Assigned Names and Numbers).

IP Address and Routing Control#

The network address part of an IP address is used for routing control.
The routing control table records the network address and the address to which the next hop should be sent. Both hosts and routers have their own routing control tables.
When sending an IP packet, the target address in the IP packet header must first be determined, and then a record with the same network address is found in the routing control table. Based on this record, the IP packet is forwarded to the corresponding next router. If there are multiple records with the same network address in the routing control table, the one with the longest matching prefix is selected.

Host A wants to send an IP packet with a source address of 10.1.1.30 and a destination address of 10.1.2.10. Since no matching network address for the destination address 10.1.2.10 is found in Host A's routing table, the packet is forwarded to the default route (Router 1).
Router 1 receives the IP packet and matches it against its routing table for a record with the same network address as the destination address. Upon finding a match, it forwards the IP data packet to Router 2 at 10.1.0.2.
Router 2 receives the packet and similarly compares it against its routing table, finds a match, and forwards the IP packet out through its interface at 10.1.2.1, ultimately forwarding the IP data packet to the target host via a switch.

The loopback address does not flow to the network.

The loopback address is a default address used for network communication between programs on the same computer.
Computers use a special IP address, 127.0.0.1, as the loopback address. The hostname "localhost" has the same meaning as this address. When using this IP or hostname, the data packet does not flow to the network.

IP Fragmentation and Reassembly#

When the size of an IP packet exceeds the MTU, the IP packet will be fragmented.
After fragmentation, the reassembly of the IP datagram can only be done by the destination host; routers do not perform reassembly.

Basic Understanding of IPv6#

IPv6 addresses are 128 bits long, allowing for an astonishing number of assignable addresses. To put it humorously, IPv6 can ensure that every grain of sand on Earth can be assigned an IP address.
However, beyond having more addresses, IPv6 also offers better security and scalability, which means that IPv6 provides a better network experience compared to IPv4.
But because IPv4 and IPv6 are not compatible with each other, not only do our devices like computers and phones need to support it, but network operators also need to upgrade existing equipment, which may be one reason for the slow adoption of IPv6.

Highlights of IPv6

IPv6 not only increases the number of assignable addresses but also has many other highlights.

IPv6 supports automatic configuration, allowing for automatic IP address assignment even without a DHCP server, making it truly plug-and-play.
The IPv6 header has a fixed length of 40 bytes, eliminating the header checksum, simplifying the header structure, reducing the load on routers, and significantly improving transmission performance.
IPv6 includes network security features to combat IP address spoofing and prevent line eavesdropping, greatly enhancing security.

Identification method for IPv6 addresses

IPv4 addresses are 32 bits long, represented in groups of 8 bits using dotted decimal notation.
IPv6 addresses are 128 bits long, represented in groups of 16 bits, separated by colons ":".
When there are consecutive zeros, these can be omitted and replaced with two colons "::". However, an IP address can only contain two consecutive colons once.

Structure of IPv6 addresses

IPv6 addresses mainly consist of the following types:

Unicast address, used for one-to-one communication
Multicast address, used for one-to-many communication
Anycast address, used for communication with the nearest node, where the nearest node is determined by the routing protocol
No broadcast address

Types of IPv6 Unicast Addresses

For one-to-one communication IPv6 addresses, there are three main types of unicast addresses, each with different valid ranges.

Link-local unicast addresses are used for communication within the same link without going through a router; this type does not exist in IPv4.
Unique local addresses can be used for unicast communication within an internal network, equivalent to IPv4 private IPs.
Global unicast addresses can be used for communication over the Internet, equivalent to IPv4 public IPs.

IPv4 Header vs. IPv6 Header#

Improvements in the IPv6 header compared to the IPv4 header:

The header checksum field has been removed. Since checks are performed at both the data link layer and transport layer, IPv6 directly eliminates the IP checksum.
Fragmentation/reassembly-related fields have been removed. Fragmentation and reassembly are time-consuming processes, and IPv6 does not allow fragmentation and reassembly at intermediate routers; this can only occur at the source and destination hosts, significantly increasing the forwarding speed of routers.
The options field has been removed. The options field is no longer part of the standard IP header but may appear in the "next header" field of the IPv6 header. Removing this options field makes the IPv6 header a fixed length of 40 bytes.

DNS Domain Name Resolution#

DNS domain name resolution allows DNS to automatically convert domain names into specific IP addresses.

Hierarchical relationship of domain names

In DNS, domain names are separated by periods, such as www.server.com, where the periods represent boundaries between different levels.
In a domain name, the position furthest to the right indicates the highest level.
The root domain is at the top level, with the next level being the top-level domain com, followed by server.com.
Thus, the hierarchical relationship of domain names resembles a tree structure:

Root DNS server
Top-level domain DNS server (com)
Authoritative DNS server (server.com)

Workflow of domain name resolution

ARP and RARP Protocols#

When transmitting an IP datagram, after determining the source IP address and destination IP address, the next hop for the IP packet is determined through the host's "routing table." However, since the next layer of the network layer is the data link layer, we also need to know the MAC address of the "next hop."
Since the routing table of the host can find the next hop's IP address, the MAC address of the next hop can be obtained through the ARP protocol.

How does ARP know the other party's MAC address?

ARP determines the MAC address using two types of packets: ARP request and ARP response.

The host sends an ARP request via broadcast, which contains the IP address of the host whose MAC address is being sought.
When all devices on the same link receive the ARP request, they will unpack the contents of the ARP request packet. If the target IP address in the ARP request matches their own IP address, that device will return its MAC address in an ARP response packet to the host.

Operating systems typically cache the MAC addresses obtained through ARP for future reference, allowing them to directly find the corresponding MAC address from the cache. However, the cache for MAC addresses has a certain expiration period, after which the cached content will be cleared.

Do you know what the RARP protocol is?

The RARP protocol is used to obtain an IP address when the MAC address is known. This is often used when connecting small embedded devices like printers to the network.
Typically, a RARP server needs to be set up, registering the device's MAC address and its corresponding IP address. Then, when the device connects to the network:

The device sends a request message saying, "My MAC address is XXXX, please tell me what my IP address should be."
The RARP server receives this message and returns information stating, "The device with MAC address XXXX has the IP address XXXX."

Finally, the device sets its IP address based on the response received from the RARP server.

DHCP Dynamic IP Address Acquisition#

Our computers usually obtain IP addresses dynamically through DHCP, greatly simplifying the tedious process of configuring IP information.

First, it should be noted that the DHCP client process listens on port 68, while the DHCP server process listens on port 67.
The four steps are as follows:

The client first initiates a DHCP discovery message (DHCP DISCOVER) as an IP datagram. Since the client does not have an IP address and does not know the DHCP server's address, it uses UDP broadcast communication, with the broadcast destination address set to 255.255.255.255 (port 67) and using 0.0.0.0 (port 68) as the source IP address. The DHCP client passes this IP datagram to the link layer, which then broadcasts the frame to all devices on the network.
When the DHCP server receives the DHCP discovery message, it responds to the client with a DHCP offer message (DHCP OFFER). This message still uses the IP broadcast address 255.255.255.255 and carries information about the IP address, subnet mask, default gateway, DNS server, and IP address lease duration provided by the server.
After the client receives one or more DHCP offer messages from servers, it selects one server and sends a DHCP request message (DHCP REQUEST) to the chosen server, echoing the configured parameters.
Finally, the server responds to the DHCP request message with a DHCP ACK message, confirming the requested parameters.

Once the client receives the DHCP ACK, the interaction is complete, and the client can use the IP address assigned by the DHCP server for the lease duration.
If the lease for the DHCP IP address is nearing expiration, the client will send a DHCP request message to the server:
If the server agrees to continue the lease, it will respond with a DHCP ACK message, and the client will extend the lease.
If the server does not agree to continue the lease, it will respond with a DHCP NACK message, and the client must stop using the leased IP address.
It can be observed that throughout the DHCP interaction, UDP broadcast communication is used.

Since it uses broadcast, what if the DHCP server and client are not on the same local area network, and routers do not forward broadcast packets? Does that mean each network needs to configure a DHCP server?

To solve this problem, DHCP relay agents were introduced. With DHCP relay agents, IP address allocation across different subnets can be managed by a single DHCP server.

The DHCP client sends a DHCP request packet to the DHCP relay agent, which, upon receiving this broadcast packet, forwards it to the DHCP server in a unicast manner.
The server then responds to the DHCP relay agent, which broadcasts this response back to the DHCP client.

Thus, even if the DHCP server is not on the same link, it can still achieve unified allocation and management of IP addresses.

NAT Network Address Translation#

IPv4 addresses are in short supply, leading to the proposal of a method called Network Address Translation (NAT), which alleviates the exhaustion of IPv4 addresses.
This involves converting both the IP address and port number.
This way, only one global IP address is needed, and this conversion technology is called Network Address and Port Translation (NAPT).

In the diagram, there are two clients, 192.168.1.10 and 192.168.1.11, communicating with the server 183.232.231.172, both using local port 1025.
At this point, both private IP addresses are converted to the public address 120.229.175.121, but distinguished by different port numbers.
Thus, a NAPT router's translation table is generated, allowing the correct conversion of address and port combinations, enabling clients A and B to communicate simultaneously with the server.
This translation table is automatically generated on the NAT router. For example, in the case of TCP, when the SYN packet is sent during the initial handshake of establishing a TCP connection, this table is created. It is then deleted from the table when the FIN packet confirming the closure of the connection is received.

What are the disadvantages of NAT?

Since NAT/NAPT relies on its own translation table, the following issues arise:

External entities cannot actively connect to NAT internal servers because there are no translation records in the NAPT table.
The generation of the translation table and the translation operations incur performance overhead.
During communication, if the NAT router restarts, all TCP connections will be reset.

How to solve the potential problems of NAT?

Switch to IPv6.
NAT traversal technology.
This allows network applications to actively discover that they are behind a NAT device, obtain the public IP of the NAT device, and establish port mapping entries for themselves, all of which are done automatically by the applications behind the NAT device.

ICMP Internet Control Message Protocol#

ICMP stands for Internet Control Message Protocol.
Network packets often encounter various issues in complex network transmission environments. Therefore, it is necessary to send messages reporting the encountered problems to adjust transmission strategies and control the overall situation.

What functions does ICMP have?

ICMP's main functions include confirming whether IP packets successfully reach the target address, reporting the reasons for IP packets being discarded during transmission, and improving network settings.
If an IP packet fails to reach the target address for some reason during IP communication, ICMP is responsible for notifying the specific reason.

In the example above, Host A sends a data packet to Host B, but due to some reason, Router 2 along the way fails to detect the existence of Host B. In this case, Router 2 will send an ICMP destination unreachable packet to Host A, indicating that the packet sent to Host B was not successful.
ICMP's notification messages are sent using IP.
Thus, the ICMP packet returned from Router 2 will follow the usual routing control, first passing through Router 1 before being forwarded to Host A. Upon receiving this ICMP packet, Host A will analyze the ICMP header and data field to determine the specific reason for the problem.

Types of ICMP

ICMP can be roughly divided into two main categories:

One category is diagnostic query messages, known as "query message types."
The other category is error messages that notify of problems, known as "error message types."

Query Message Types#

Echo Message - Type 0 and 8

Echo messages are used between communicating hosts or routers to determine whether the sent data packets have successfully reached the other end; the ping command utilizes this message.

Ping#

Let's look at the sending and receiving process of ping.

When the ping command is executed, the source host first constructs an ICMP echo request message data packet.
Then, the ICMP protocol hands this packet along with the address 192.168.1.2 to the IP layer. The IP layer will use 192.168.1.2 as the destination address, the local IP address as the source address, set the protocol field to 1 to indicate it is ICMP, and add some other control information to construct an IP packet.
Next, a MAC header needs to be added. - ARP
When Host B receives this data frame, it first checks its destination MAC address and compares it with its own MAC address. If they match, it receives the packet; otherwise, it discards it.
After receiving, it checks the data frame, extracts the IP packet from the frame, and hands it to its own IP layer. Similarly, the IP layer checks and extracts the useful information to pass to the ICMP protocol.
Host B will construct an ICMP echo response message data packet, with the type field of the response data packet set to 0, the sequence number matching that of the received request data packet, and then sends it back to Host A.
If the source host does not receive the ICMP response packet within the specified time, it indicates that the target host is unreachable; if it receives the ICMP echo response message, it indicates that the target host is reachable.
At this point, the source host will check the current time minus the time the packet was initially sent from the source host to determine the ICMP packet's round-trip time.

Error Message Types#

Destination Unreachable Message - Type 3
When an IP router cannot send an IP packet to the target address, it will return a destination unreachable ICMP message to the sending host, indicating the specific reason for the unreachability, which is recorded in the code field of the ICMP packet header.
Source Quench Message - Type 4
In the case of low-speed wide area lines, routers connecting to WANs may encounter network congestion.
The purpose of the ICMP source quench message is to alleviate this congestion.
When a router sends data to a low-speed line and its sending queue becomes zero and cannot send, it can send an ICMP source quench message to the source address of the IP packet.
The host receiving this message understands that there is congestion at some point along the entire line, thus increasing the transmission interval of IP packets to reduce network congestion.
However, since this ICMP message may cause unfair network communication, it is generally not used.
Redirect Message - Type 5
If a router discovers that the sending host is using a "non-optimal" path to send data, it will return an ICMP redirect message to that host.
This message contains the most suitable routing information and source data. This mainly occurs when the router has better routing information. The router will inform the sender through this ICMP message to send to another router next time.
Time Exceeded Message - Type 11
An IP packet has a field called TTL (Time To Live), which decreases by 1 each time it passes through a router. When it reaches 0, the IP packet is discarded.
At this point, the router will send an ICMP time exceeded message to the sending host, notifying that the packet has been discarded.
The main purpose of setting the IP packet's lifetime is to avoid endless forwarding of IP packets in the network when routing control encounters problems.

Traceroute#

Function 1: Intentionally set a special TTL to trace the routers passed on the way to the destination.

How does this function work?

Its principle is to use the IP packet's lifetime, starting from 1 and incrementing sequentially while sending UDP packets, forcing the reception of ICMP time exceeded messages.
For example, setting the TTL to 1 will cause the first router to discard the packet, returning an ICMP error message of type time exceeded.
Next, setting the TTL to 2 will allow the first router to pass but cause the second router to discard the packet, also returning an ICMP error message. This process continues until reaching the destination host.
Through this process, traceroute can obtain the IP addresses of all routers.
Of course, some routers may not return this ICMP message, so for some public addresses, the intermediate routers may not be visible.

How does the sender know if the sent UDP packet has reached the destination host?

When sending UDP packets, traceroute fills in an impossible port number as the UDP destination port: 33434. For each subsequent probe, it increments this number, and these ports are generally considered unused. However, it is unknown what happens when certain applications listen on such ports.
When the destination host receives the UDP packet, it will return an ICMP error message, but the type of this error message will be "port unreachable."
Thus, when the error message type is "port unreachable," it indicates that the UDP packet sent by the sender has reached the destination host.
Function 2: Intentionally set no fragmentation to determine the path's MTU.

Why do this?

The purpose is to discover the path MTU.
Sometimes we do not know the MTU size of the routers; the MTU on Ethernet data links is usually 1500 bytes, but non-Ethernet MTU values can vary. Therefore, we need to know the MTU size to control the size of the packets sent.

Its working principle is as follows:
First, when the sending host sends an IP datagram, it sets the fragmentation prohibited flag in the IP packet header to 1. Based on this flag, the routers along the way will not fragment large packets but will discard them instead.
Subsequently, an ICMP unreachable message will be sent back to the sending host, indicating the MTU value of the data link, with the unreachable message type being "fragmentation needed but the don't fragment bit was set."
Each time the sending host receives an ICMP error message, it reduces the packet size to locate a suitable MTU value to ensure it can reach the target host.

IGMP Internet Group Management Protocol#

Earlier, we learned about multicast addresses, which are Class D addresses. Since it is multicast, it indicates that only a group of hosts can receive the data packets, while hosts not in the group cannot receive them. So, how is membership in a group managed? This is where the IGMP protocol comes in.

IGMP is the Internet Group Management Protocol, operating between hosts (multicast members) and the last-hop router, as shown in the blue section of the diagram.

IGMP messages request routers to join and leave multicast groups. By default, routers do not forward multicast packets to connected hosts unless the hosts join the multicast group through IGMP. When a host requests to join a multicast group, the router records this in the IGMP router table, and subsequently, the router will forward multicast packets to the corresponding hosts.
IGMP messages are encapsulated in IP, with the protocol number in the IP header set to 2, and the TTL field value is usually set to 1, as IGMP operates between the host and the connected router.

IGMP Working Mechanism

IGMP is divided into three versions: IGMPv1, IGMPv2, and IGMPv3.
Next, taking IGMPv2 as an example, we will discuss the mechanisms of general queries and responses and leaving multicast groups.

General Query and Response Mechanism
Leaving a Multicast Group Mechanism
Situation 1: The multicast group still exists in the subnet:

Situation 2: The multicast group does not exist in the subnet: