PCI Express Gen3 Controller

Feature Set N-ANL9

Napatech SmartNIC
Feature Description

In this chapter

This chapter describes the PCI Express Gen3 controller on the Napatech NT accelerators, which handles the PCI Express interface to the host. The PCI Express configuration space as defined in Section 7 of PCIe3.0 is implemented within the PCI Express IP cores.

The PCIe block handles the data transfer between the host (server/motherboard) and the FPGA.

Note: If an N200A01 accelerator is inserted in a server that does not support bifurcation, the accelerator must be configured to use only one 8-lane PCI end point (see DN-0487).

PCIe performance

The performance of an NT accelerator will depend on the slot in which it is mounted: An accelerator mounted in a slot supporting PCIe Gen1 will have Gen1 functionality, an accelerator mounted in a slot supporting PCIe Gen2 will have Gen2 functionality, and an accelerator mounted in a slot supporting PCIe Gen3 will have Gen3 functionality.

DMA data transfers

The PCI Express controllers implement a number of DMA channels for data exchange between the host memory and the FPGA. All DMA data transfers are initiated by the FPGA.

The FPGA can initiate a DMA read from data in host memory or a DMA write of data to the host memory.

Number of lanes

The PCI Express controllers support ×1, ×4, ×8 and ×16 lane operation depending on the number of available lanes in the physical connector in the host. The number of active lanes is determined during the PCI Express enumeration process at boot time.

Maximum number of lanes

This table shows the maximum number of lanes supported by the PCI Express controller on an NT accelerator for different physical connectors.

Routed Lines Physical Connector
×1 ×4 ×8 ×16
1-lane N/A N/A ×1 ×1
4-lane N/A N/A ×4 ×4
8-lane N/A N/A ×8 ×8
16-lane N/A N/A ×8 ×16

PCI Express throughput

The achievable PCI Express throughput from the FPGA to the host memory and from the host memory to the FPGA depends on a number of different factors, such as:

  • Number of active PCI Express lanes
  • PCI Express maximum transaction layer packet (TLP) size (MAX_PAYLOAD_SIZE)
  • Number of flow control credits
  • Number of completion resources

Maximum PCI Express throughput

This table shows the maximum theoretical PCI Express throughput as a function of the number of PCI Express lanes and MAX_PAYLOAD_SIZE.

MAX_PAYLOAD_SIZE [in bytes] PCI Express Throughput [in Gbit/s]
×4 ×8 ×16
128 27.3 54.5 109.0
256 29.2 58.5 116.9
512 30.3 60.6 121.3
Note: The actual PCI Express throughput also depends on the server chipset and server implementation.
Note: The available PCI Express bandwidth must be used for both data and frame overhead, such as packet descriptors.

Required PCI Express bandwidth for NT100E3-1-PTP

This table shows the required PCI Express bandwidth as a function of frame size for frames with only a standard packet descriptor, for frames with also an extended packet descriptor, for frames with a dynamic descriptor 1 and for frames with a dynamic descriptor 2 or 3 for an NT100E3-1-PTP accelerator running at line rate.

Frame Size [in bytes] PPS Gbit/s to be Transferred over the PCIe Bus
Standard Descriptor Extended Descriptor 9 Dynamic Descriptor 1 Dynamic Descriptor 2 or 3
64 148,809,523 95.2 133.3 97.6 102.4
128 84,459,459 97.3 118.9 98.6 101.4
256 45,289,855 98.6 110.1 99.3 100.7
512 23,496,240 99.2 105.2 99.6 100.4
1024 11,973,180 99.6 102.7 99.8 100.2
1518 8,127,438 99.7 101.8 99.9 100.1