Receive Side Scaling (RSS)

A maximum of 16 different RSS configurations are supported. This implies that 16 different combinations of header fields can be configured, including one RSS configuration on ports (physical and virtual ports). By default, the 5-tuple hash mode on the outer layer is configured on all ports. The contents of these header fields are used:

• 32-bit IPv4 / 128-bit IPv6 source address
• 32-bit IPv4 / 128-bit IPv6 destination address
• 16-bit UDP, TCP or SCTP source port number
• 16-bit UDP, TCP or SCTP destination port number
• 8-bit IPv4/IPv6 protocol number / next header

The following table shows the supported header fields.

Selected header fields for hash calculation are stored in a 320-bit memory (plus an 8-bit memory for the IP protocol) in the SmartNIC. This consists of 2 blocks with four-words each (128 bits) and 2 blocks with one-word each (32 bits). If the SmartNIC cannot store the selected fields in its four blocks, the hash configuration fails and it triggers to generate an error message.

It is possible to configure the RSS feature on all physical and virtual ports using the DPDK API function rte_eth_dev_rss_hash_update with the rte_eth_rss_conf structure. The rte_eth_rss_conf structure is defined as follows.

struct rte_eth_rss_conf {
    uint8_t                         *rss_key;
    uint8_t                         rss_key_len;
    uint64_t                        rss_hf;
    enum rte_eth_hash_function      algorithm;
};

where:

• rss_key: Set to either NULL or a pointer to a 4-byte key for NTH10 or a 40-byte key for Toeplitz. A key with a length shorter than 40 bytes is right-padded with zeros.
• rss_key_len: Specifies the length of rss_key in bytes.
• rss_hf: Combination of RTE_ETH_RSS_* flags to select header fields for hash calculation. See Supported header fields.
• algorithm: Set to either RTE_ETH_HASH_FUNCTION_DEFAULT for NTH10 or RTE_ETH_HASH_FUNCTION_TOEPLITZ for Toeplitz

The rte_flow_action_rss structure is defined as follows.

struct rte_flow_action_rss {
    enum rte_eth_hash_function      func; 
    uint32_t                        level;
    uint64_t                        types; 
    uint32_t                        key_len; 
    uint32_t                        queue_num; 
    const uint8_t                   *key; 
    const uint16_t                  *queue; 
};

where:

• func: Set to either RTE_ETH_HASH_FUNCTION_DEFAULT for NTH10 or RTE_ETH_HASH_FUNCTION_TOEPLITZ for Toeplitz.
• level: Specifies the inner layer or the outer layer. Set to 1 for the outermost layer or 2 for the innermost layer. It overrides RTE_ETH_RSS_LEVEL_INNERMOST and RTE_ETH_RSS_LEVEL_OUTERMOST flags from types.
• types: Combination of RTE_ETH_RSS_* flags to select header fields for hash calculation. See Supported header fields.
• key_len: Specifies the length of key in bytes
• key: Set to either NULL or a pointer to a 4-byte key for NTH10 or a 40-byte key for Toeplitz. A key with a length shorter than 40 bytes is right-padded with zeros.
• queue: Specifies queues for traffic distribution.

Even though the SmartNIC supports a maximum of 256 queues (128 RX queues + 128 TX queues), the number of supported queues can be limited by the maximum number of memory maps in DPDK, which is set to 256 by default. Out of 256 memory maps available, 2 (or more) are used by internal processes. This implies that DPDK can create 254 or fewer RX and TX queues using the default configuration. If an application attempts to create more memory maps than allowed, it may generate an error. It is possible to modify the maximum number of memory maps in the lib/eal/linux/eal_vfio.c file of the DPDK package. The following line in the file defines the value:

#define VFIO_MAX_USER_MEM_MAPS 256

If RX queues are configured only for physical ports, the same number of queues are assigned to each port. For the SmartNIC with two physical ports, a maximum of 64 RX queues per port can be used. The number of queues for physical ports can be configured using the rxqs parameter in the DPDK application. See the following DPDK testpmd command example.

./dpdk-testpmd --iova-mode=pa --vfio-vf-token=14d63f20-8445-11ea-8900-1f9ce7d5650d \
-a 0000:65:00.0,rxqs=64 -- -i

Using this command, 64 RX queues are created per port for the SmartNIC with two physical ports..

In addition, the maximum number of queues for virtual functions are limited by the maximum number of virtual queue pairs, which is defined in the DPDK drivers/net/virtio/virtio.h file as follows.

#define VIRTIO_MAX_VIRTQUEUE_PAIRS 8

By default, it is set to 8, meaning that DPDK supports a maximum of 8 virtual queue pairs for each virtio device. If you need to change this limit, you can modify the value in the header file to suit your specific requirements. When the limit is adjusted, it must be evaluated and tested as it can impact system performance and resource utilization.