Napatech Main-App Container

Link-Inline™ Software User Guide
Platform
Napatech SmartNIC
Content Type
User Guide
Capture Software Version
Link-Inline™ Software 3.2

In the Napatech main-app container, the ntmain application uses the Napatech DPDK PMD, which is connected to the physical function of the SmartNIC.

Enabling/disabling the Napatech main-app container

The Napatech device-plugin pod can operate in two modes as the presence of the Napatech main-app container is optional.
  • Running with the Napatech main-app container: To enable this mode, the MainApp parameter in the ntdevplugin.yaml file must be set to "enabled". In this mode, the Napatech main-app container is active in the Napatech device-plugin pod. Consequently, the Napatech device-plugin pod is connected to the Napatech DPDK PMD (ntnic), which is required for configuring flows in the SmartNIC.
  • Running with a custom main-app container: To configure this mode, the MainApp parameter must be set to "disabled". In this configuration, the Napatech main-app container is inactive in the Napatech device-plugin pod. This means that a custom main-app container is responsible for establishing a connection to the Napatech DPDK PMD (ntnic), which is essential for flow configuration in the SmartNIC.

Napatech DPDK PMD ntnic

The ntmain application uses the Napatech DPDK PMD (ntnic). The ntnic driver automatically detects and configures the SmartNIC by performing the following tasks.
  • SmartNIC initialization: Detects the SmartNIC and configures the PCIe interface, onboard memory, and port speed.
  • Detection of link up/down events on the physical ports.
  • Reading statistics from physical ports, virtual ports (virtual functions), allocated queues and flows. These statistics can be accessed using either the DPDK API or the NTConnect API functions.
  • Setting up vHost-vDPA (virtual host - virtual data path acceleration): If any virtual function is activated, the vhost-vDPA setup is executed. This involves allocating necessary queues and establishing connections to the SmartNIC.

Configuring flows in a user-application container

Using the default setup, the Napatech main-app container is deployed in the same pod as the Napatech device-plugin container. The ntmain application runs in the main-app container, attached to the physical function of the SmartNIC. In the ntmain application, the VLAN ID for each virtual function is passed to the Napatech DPDK PMD, which then creates a table of VLAN IDs with associated virtual functions as illustrated in the following figure.


rte_flow API function calls in the user-application pod are forwarded to the Napatech main-app container via the NTConnect socket connection. The ntnic PMD then adds the VLAN ID of the associated virtual function to the rte_flow configuration commands before forwarding them to the SmartNIC.

When rte_flow API functions are called in the user application, they are passed to the Napatech DPDK PMD (ntnic) via the NTConnect socket connection. The Napatech DPDK PMD then adds the VLAN ID of the associated virtual function to the rte_flow configuration commands before forwarding them to the SmartNIC (see the red path in the figure). This ensures that the SmartNIC delivers traffic flows to each virtual function based on the VLAN ID of each frame (see the blue path in the figure).

The initial VLAN ID is determined by the vfVlanStart parameter in the ntdevplugin.yaml file. The VLAN ID increments for subsequent virtual functions. The total number of the virtual functions to be created is determined by the vfCount parameter in the ntdevplugin.yaml file.

For information on the ntdevplugin.yaml file, see ConfigMap. For information on how to configure VLAN IDs for virtual functions, see Associating a VLAN ID to a virtual function in the ntmain application.

Mandatory DPDK EAL command-line parameters

In the ntmain application, DPDK EAL command-line parameters can be used to control various aspects of your DPDK application's behavior, memory usage or core affinity as required. The following options are mandatory.
  • --vfio-vf-token=<UUID>: Specifies a VFIO (Virtual Function I/O) VF (Virtual Function) token file. This is used to assign a specific VF to a DPDK application. The token file helps to ensure proper security and access control. Set --vfio-vf-token to the UUID, which is generated using the uuidgen command. For example:
    uuidgen
14d63f20-8445-11ea-8900-1f9ce7d5650d
  • --file-prefix: Specifies a prefix for all files created by DPDK, such as shared memory files, log files, and statistics files. This option is required to ensure uniqueness when multiple DPDK applications are running on the same system as the main application and all other applications are DPDK applications.
  • -a <PCI_ID>: Specifies the PCI bus ID of the SmartNIC (physical function). For example: 
    -a 0000:b1:00.0
    This option is also used to configure additional supported parameters for the physical function such as portqueues and exception_path.
    Note: If not all virtual functions are needed, it is also possible to select specific virtual functions using the -a option. The following command example specifies the physical function and selects 2 virtual functions:
    -a 0000:b1:00.0 -a 0000:b1:00.4 -a 0000:b1:00.5
    This option loads the Napatech DPDK PMD (ntnic) on physical function 0000:b1:00.0, and performs the necessary vHost-vDPA setup for virtual function 4 and virtual function 5.

For a list of available EAL command-line parameters, see https://doc.dpdk.org/guides-22.11/linux_gsg/linux_eal_parameters.html.

Setting up multiple queues

To set up multiple queues in the ntmain application, add the portqueues parameter to the -a option for the physical function as follows.
-a <PF_PCI_ID>,portqueues=[<VF_number>:<Number_of_queues>,<VF_number>:<Number_of_queues>,…]
where:
  • PF_PCI_ID: Specifies the PCI bus ID of the physical function.
  • VF_number: Specifies the virtual function number, which starts from 4.
  • Number_of_queues: Specifies the number of queues to be allocated for the specified virtual function.
For example:
-a 0000:b1:00.0,portqueues=[4:3,5:3,6:3,7:3,8:3,9:3]
This allocates 3 queues per virtual function for virtual function 4 to 9. This configuration does not enable receive side scaling (RSS), but only allocates the specified number of queues. RSS must be enabled using the rte_flow API in the user application running on the virtual function. See API: Setting up Receive Side Scaling (RSS). The specified number of queues must be passed to the user application via a command line parameter, as the user application needs to initialize the queues. See Receive side scaling (RSS).
Note: The portqueues parameter is valid only to the physical function.
By default, the number of queues per virtual function is set to 1 using the vfNumberOfQueues parameter in the ntdevplugin.yaml file.

For more information on the ntdevplugin.yaml file, see ConfigMap.

Associating a VLAN ID to a virtual function in the ntmain application

To associate a VLAN ID to a specific virtual function, add the vlan parameter to the -a option for the target virtual function.
-a <VF_PCI_ID>,vlan=<VLAN_ID>
where:
  • VF_PCI_ID: Specifies the PCI bus ID of the target virtual function.
  • VLAN_ID: Specifies the VLAN ID to be associated to the specified virtual function.
For example:
-a 0000:b1:00.4,vlan=44 -a 0000:b1:00.5,vlan=45
This creates a VLAN ID table in the Napatech DPDK PMD (ntnic). When the application configures flows using the rte_flow API, the VLAN ID of the associated virtual function is appended to the rte_flow commands in the Napatech DPDK PMD, before they are delivered to the SmartNIC. VLAN ID 44 is added to the rte_flow commands for virtual function 0000:b1:00.4, and VLAN ID 45 is added for virtual function 0000:b1:00.5. There is no VLAN ID and virtual function binding by default. See Configuring flows in a user-application container.

Shared/separated socket directory

By default, files (used to connect applications and virtual functions) are placed in the /usr/local/var/run directory on the host. This directory is mounted on user-application containers running on virtual functions, enabling user applications to access the sockets of virtual functions. This setup can potentially pose a security risk, as all pods gain access to all sockets.

Using the sep=1 option, each socket file is placed in its individual directory, and each directory is then mounted on a specific container. Consequently, each container can only access its associated socket.

Configure the separated socket directory as follows.
-a <VF_PCI_ID>,sep=1
where:
  • VF_PCI_ID: Specifies the PCI bus ID of the target virtual function.
  • sep=1: Configures a separated directory for the vHost user socket file of the specified virtual function.
A socket file of a virtual function is placed in the /usr/local/var/run/stdvio<VF_number> directory, where VF_number specifies the virtual function number.
For instance: 
-a 0000:b1:00.4,vlan=44,sep=1
The socket file directory of the 0000:b1:00.4 virtual function is /usr/local/var/run/stdvio4/stdvio4. To use the shared directory, set sep to 0 as follows. 
-a 0000:b1:00.4,sep=0
The directory of the stdvio4 socket is the /usr/local/var/run/ directory using the sep=0 option.

Handling non-matched frames

The SmartNIC with Napatech Link-Inline™ Software decodes every received frame and looks up the flow based on the learned flow information. If the flow of a frame is found (hit) in the flow table of the SmartNIC, the frame is processed according to user-defined rules and actions. If the flow of a frame is not found (miss), the frame can be delivered to the ntmain application or custom main application using the exception_path=1 option. The frames can then be processed by the application.
Note: By default, the ntmain application does not takes any further actions, but discards the received frames.
The exception_path=1 option can be configured as follows.
a <PF_PCI_ID>,exception_path=1
where:
  • PF_PCI_ID: Specifies the PCI bus ID of the physical function.
  • exception_path=1: Enables the forwarding of non-matched frames to queue 0.
    Note: This option is valid only for the physical function.
For example:
-a 0000:b1:00.0,exception_path=1
The custom main application must actively poll queue 0 on all ports to handle these non-matched frames.

Configuration handling by the Napatech main-app container

The Napatech main-app container takes care of configuring EAL parameters for the ntmain application and also manages the separation of all vHost-user-socket directories. For user-application containers, refer to the start.sh script in the following directory of the Link-Inline™ Software package.
/ntinl_package-<version>-linux/kubernetes_v<version>/docker/start.sh
where version is the version identifier of the corresponding package.

Mounting host directories

The Napatech device plugin mounts the following host directories on the Napatech main-app container.
Directories on the host Mounted directories on the Napatech main-app container
/var/run/ntconnect /var/run/ntconnect
/usr/local/var/run /usr/local/var/run