How does the PCIE auto-negotiation protocol work

In the networking and computing field, every innovation aims at solving problems that were left over in earlier phases. The emergence of the PCIe standard (PCIe = peripheral component interconnect express) is intended to meet the need for higher bandwidth, greater flexibility and better performance of network devices. Since its debut in 2001, PCIe has developed rapidly and is widely used in many network devices, especially for the PCIe card (aka PCI-Express card, PCIe-based card). The definition, principle of operation, advantages, types and selection of PCI Express cards are included in this comprehensive introduction.

What is a PCI Express Card?

A PCIe card is a type of network adapter with a PCIe interface that is used as an expansion card interface in motherboard-level connections. In particular, PCIe-based expansion cards are designed to fit into PCIe-based slots on the motherboard of devices such as the host, server and network switch. Most current PC motherboards have PCIe slots only for PCIe cards. The PCI Express card can be installed in the corresponding PCIe slot. Usually the slot is as wide as or wider than the card.

How does the PCI Express card work?

Instead of acting like a bus handling data from multiple sources, a PCI Express card can use switches to create a series of point-to-point connections to control where the data needs to go. After inserting a PCIe network card, a logical connection is established between the slot and this card in order to communicate with one another. This connection, known as an interconnect or link, enables a point-to-point communication channel between two PCIe ports and allows both to send and receive normal PCI requests and interrupts. As the following diagram shows, the PCIe slot contains one or more lanes. With the x2 connection, each lane consists of two different data transmission pairs, one pair for sending and one pair for receiving data. Therefore, each lane consists of four wires or signal lanes.

Figure 1: How does the PCIe card work?

Why a PCI Express Card?

Before the introduction of the PCIe card, there were two main network adapters on the market: PCI and PCI-X cards. With a completely different connector and electrical design, the newly introduced PCI-E card was an improvement on the first two and became the most widely used card among them. The article "PCI vs. PCI-X vs. PCI-E" explains why we use PCI-E the most.

How many types of PCIe cards are there?

A variety of formats have been developed for PCI Express cards. The following section introduces the PCI Express card types in terms of sizes and versions.

Based on PCIe card sizes

The size of a PCI Express card is usually indicated by the number of lanes. There are generally five physical sizes of PCIe cards: x1, x4, x8, x16 and x32 (FYI, PCIe x32 are available with a maximum of 32 lanes, but they are ultra-rare and not a mainstream card). The number after the "x" refers to the number of lanes in the PCIe slot. For example, a PCIe x4 card means that the card has four lanes.

Figure 2: Comparison of different PCIe card sizes

In applications, you must insert the PCIe card into a PCIe slot on a host or server of the same size and configuration as this card. However, in cases such as insufficient slots, the PCIe card can also be used in a wider slot. For example, you can insert a PCIe x8 card into a PCIe x16 slot if the PCIe x8 slot is occupied, but this card always runs in PCIe x8 mode. Refer to the following table for more details on PCIe card sizes.

PCIe slot width Number of pins length
PCIe x1 18 25 mm
PCIe x4 21 39 mm
PCIe x8 49 56 mm
PCIe x16 82 89 mm

Based on PCIe card versions

PCI Express is a high-speed serial computer expansion bus standard for connecting high-speed components that has replaced the older AGP (Accelerated Graphics Port), PCI, and PCI-X bus standards. The PCIe standard has been revised several times to improve performance and other features. PCIe 1.0 was originally introduced in 2002. To meet the growing demand for higher bandwidth, successive versions have been developed and brought to market. There are currently five different generations of PCIe standards: PCIe 1.0, PCIe 2.0, PCIe 3.0, PCIe 4.0, and PCIe 5.0. PCIe 6.0 will come in the not too distant future. The transmission speed doubles with each generation.

Accordingly, five versions of PCIe cards are used: PCIe 1.x, PCIe 2.x, PCIe 3.x, PCIe 4.x and PCIe 5.x are all currently available. The higher performance PCIe 5.x is the latest version that was only released in 2019. And PCIe 6.x should appear around 2021.

It is worth mentioning that all PCIe card versions are backwards compatible. This means that any version of the PCIe card and motherboard can work together in the lowest version mode. For a better understanding, the following table shows a comparison of the transfer rates between five typical PCIe versions (e.g. the original version n.0).

version introduction Transfer rate
(Throughput, x1)
Transfer rate
(Throughput, x16)
Line code
PCIe 1.0 2003 2.5 GT / s (250 MB / s) 40 GT / s (4.0 GB / s) 8b / 10b
PCIe 2.0 2007 5.0 GT / s (500 MB / s) 80 GT / s (8.0 GB / s) 8b / 10b
PCIe 3.0 2010 8.0 GT / s (984.6 MB / s) 128 GT / s (15.75 GB / s) 128b / 130b
PCIe 4.0 2017 16.0 GT / s (1969 MB / s) 256 GT / s (31.51 GB / s) 128b / 130b
PPCIe 5.0 2019 32.0 GT / s (3938 MB / s) 512 GT / s (63.02 GB / s) 128b / 130b

How do you choose the right PCIe card?

To use a PCI Express card, your computer must have at least one free PCI Express slot. If you are using a wired network without buying a developed system, you should start looking for the correct PCIe cards. However, it can be very confusing to select a suitable PCIe card. There are a few factors to consider when choosing:

  • PCIe card version and slot width: Make sure the PCI Express card type is compatible with your current equipment and network environment

  • Protocol standard: Make sure that the card supports the standards you need such as RDMA, RoCE, iSCSI and FCoE

  • Controller: Chips from Intel, Broadcom, Mellanox and Realtek are all the rage

After properly considering the above three factors, there are still a few factors that will influence your decisions, such as transfer speed, port number, connector type, operating system, brand and price, etc. A complete guide to buying network adapters can help you with this: How did you choose the right network card?

Conclusion

High-end software continues to evolve at a rapid pace to meet increasing demands, which is the impetus for further advances in PCIe performance. By complying with the de facto PCIe standard, the new generation of PCIe cards such as PCIe4.x, PCIe 5.x and the upcoming PCIe 6.x have proven that they reduce the development gap between PCIe-based cards and hosts has great potential.