Use the DPDK API function rte_eth_dev_callback_register to set up callback functions for retrieving load information for both the flow manager and ports, optimizing performance and managing resources effectively.
rte_pmd_ntnic.h
To access load information, the rte_pmd_ntnic.h header file must be included. This header file is available if DPDK has been installed using ninja install. See Installing Link-Inline™ Software.
Setting up callback functions
This example sets up callback functions to manage load data of the flow manager and ports using the DPDK API rte_eth_dev_callback_register function.
#include <rte_pmd_ntnic.h>
rte_eth_dev_callback_register(0, (enum rte_eth_event_type)RTE_NTNIC_FLM_LOAD_EVENT, flm_load_event_callback, NULL);
for (portid = 0; portid < 2; portid++) {
rte_eth_dev_callback_register(portid, (enum rte_eth_event_type)RTE_NTNIC_PORT_LOAD_EVENT, port_load_event_callback, NULL);
}
The enum types are specific to the Napatech SmartNIC extension, which are
defined in the rte_pmd_ntnic.h file as follows.
enum rte_ntnic_event_type {
RTE_NTNIC_FLM_LOAD_EVENT = RTE_ETH_EVENT_MAX,
RTE_NTNIC_PORT_LOAD_EVENT,
};
| Event type | Description |
|---|---|
| RTE_NTNIC_FLM_LOAD_EVENT | Receiving load data of the flow manager. |
| RTE_NTNIC_PORT_LOAD_EVENT | Receiving load data of a specified port. |
Since the flow manager is a shared resource for all ports, the returned information for port 0 and port 1 is the same. In this example, the flm_load_event_callback function is registered only for port 0 to read the load of the flow manager.
To read the load of ports, the port_load_event_callback function is registered for both port 0 and 1.
When these events occur, the respective callback functions (flm_load_event_callback and port_load_event_callback) will be called to handle the load data. See Callback functions.
Callback functions
A callback function must have the following
format.
typedef int (*rte_eth_dev_cb_fn)(uint16_t port_id, enum rte_eth_event_type event, void *cb_arg, void *ret_param);where:
- uint16_t port_id: Specifies the physical port.
- enum rte_eth_event_type event: Specifies the type of event.
- void *cb_arg: Represents additional context information that can be passed during callback registration.
- void *ret_param: Contains the returned load information.
The following example demonstrates how to handle events for reading the load of the flow
manager and ports in callback functions. Adjust the code as needed based on your setup and
requirements.
/*
* Callback for port load events.
* An event containing port load data are received here.
* A context cb_arg can be added by the register function.
*/
static int port_load_event_callback(uint16_t port_id,
enum rte_eth_event_type event __rte_unused,
void *cb_arg __rte_unused,
void *ret_param)
{
if (port_id > 1) return 0;
pthread_mutex_lock(&port_load_event_lck);
memcpy(&port_load_data[port_id], ret_param, sizeof(ntnic_port_load_t));
pthread_mutex_unlock(&port_load_event_lck);
return 0;
}
/*
* Callback for flm load events.
* An event containing flm load data are received here.
* A context cb_arg can be added by the register function.
*/
static int flm_load_event_callback(uint16_t port_id __rte_unused,
enum rte_eth_event_type event __rte_unused,
void *cb_arg __rte_unused,
void *ret_param)
{
pthread_mutex_lock(&flm_load_event_lck);
memcpy((void*)&flm_load_data, ret_param, sizeof(ntnic_flm_load_t));
pthread_mutex_unlock(&flm_load_event_lck);
return 0;
}
These callback functions use locks (pthread_mutex_lock) to ensure that load data is safely copied into shared data structures (port_load_data and flm_load_data), preventing concurrent access issues when multiple threads might be handling these events simultaneously.
Setting duration for stable load measurement
The load is measured using a moving average with a configurable window size. By default,
the window size is set to 2 seconds. The window size is defined in the
driver/modules/ntutil/include/nt_util.h file as
follows.
/* * The window size in seconds for measuring FLM load and Port load. * The windows size must not exceed 3 minutes to prevent overflow. */ #define FLM_LOAD_WINDOWS_SIZE 2ULL #define PORT_LOAD_WINDOWS_SIZE 2ULL
Load data for the flow manager
The ntnic_flm_load_s structure represents load data of the flow
manager and is defined as
follows.
When the lookup/access values reach the
lookup_maximum/access_maximum values, the SmartNIC is
fully loaded, and frames may be dropped.
typedef struct ntnic_flm_load_s {
uint64_t lookup;
uint64_t lookup_maximum;
uint64_t access;
uint64_t access_maximum;
} ntnic_flm_load_t;
| Variable | Description |
|---|---|
| lookup | The current number of lookups. A lookup may or may not hit in the flow manager cache. Since not all frames may require a lookup, the lookup load might be lower than the traffic load. However, sometimes multiple lookups are required for a frame, leading to a higher lookup load than the traffic load. |
| lookup_maximum | The maximum number of possible lookups. |
| access | The number of SDRAM accesses. This represents the actual load on the SDRAM. A stateless lookup requires two accesses while a stateful lookup requires three accesses. Due to cache hits, the access load is usually lower than the lookup load. |
| access_maximum | The maximum number of possible SDRAM accesses. |
Load data for a port
The ntnic_port_load_s structure, representing load data of a port, is
defined as
follows.
When the current values reach the maximum values, the SmartNIC is fully loaded, and
frames may be dropped.
typedef struct ntnic_port_load_s {
uint64_t rx_pps;
uint64_t rx_pps_maximum;
uint64_t tx_pps;
uint64_t tx_pps_maximum;
uint64_t rx_bps;
uint64_t rx_bps_maximum;
uint64_t tx_bps;
uint64_t tx_bps_maximum;
} ntnic_port_load_t;
| Variable | Description |
|---|---|
| rx_pps/tx_pps | RX/TX frames per second on a port |
| rx_pps_maximum/tx_pps_maximum | The maximum possible number of RX/TX frames on a port. |
| rx_bps/tx_bps | Bits per second for RX/TX frames on a port. The bandwidth is the Ethernet bandwidth, including preamble and inter-frame gap. For example, if a 100G port is fully loaded, the computed value will be 100e9. |
| rx_bps_maximum/tx_bps_maximum | The maximum possible bits per second or bandwidth for RX/TX frames on a port. |
Accessing flow manager and port load counters
The related counters can be retrieved via the DPDK API rte_eth_xstats_get using the rte_eth_xstat structure. For more information on supported counters, see Load counters of the Flow Manager and Ports in DN-1355.
Monitoring lookup and SDRAM access rates
The lookups per second and the SDRAM accesses per second are displayed in the
monitoring tool, presented as a percentage of their respective maximum
values as
follows.
FLM statistics ┌──────────────────────────────────────────────────────────────────────────────────┐ │ FlowMatcherId: 0 │ │ current : 153793 sta_done : 0 │ │ learn_done : 4325378 inf_done : 9339775 │ │ learn_ignore : 32932 inf_skip : 0 │ │ learn_fail : 0 pck_hit : 257786882804 │ │ unlearn_done : 4171585 pck_miss : 2956155672 │ │ unlearn_ignore : 32148 pck_unh : 2 │ │ auto_unlearn_done : 0 pck_dis : 313694884 │ │ auto_unlearn_ignore : 0 csh_hit : 258915706386 │ │ auto_unlearn_fail : 0 csh_miss : 260755437050 │ │ timeout_unlearn_done : 0 csh_unh : 2 │ │ rel_done : 0 cuc_start : 0 │ │ rel_ignore : 0 cuc_move : 0 │ │ prb_done : 0 load_lps : 32% │ │ prb_ignore : 0 load_aps : 51% │ └──────────────────────────────────────────────────────────────────────────────────┘ Reset Port TX/Rx Queue Color Flowstat
Using statistic tool
Lookup and SDRAM access rates of the flow manager can be displayed using the
statistic tool. For
example:
statistic -p 0 -g flm_statsAn output example:
Print flm stats: current : 1047049 learn_done : 1047044 learn_ignore : 55905 learn_fail : 0 unlearn_done : 0 unlearn_ignore : 0 auto_unlearn_done : 0 auto_unlearn_ignore : 0 auto_unlearn_fail : 0 timeout_unlearn_done : 0 rel_done : 0 rel_ignore : 0 prb_done : 0 prb_ignore : 0 sta_done : 0 inf_done : 13794 inf_skip : 0 pck_hit : 376812971 pck_miss : 758014247 pck_unh : 0 pck_dis : 386369152 csh_hit : 391293276 csh_miss : 1120349414 csh_unh : 0 cuc_start : 0 load_lps : 36% load_aps : 59%The lookups per second and the accesses per second of the flow manager are presented as a percentage of their respective maximum values.
The following command displays load information of port
0.
statistic -p 0 -g txrx_statsAn output example:
RX COUNTERS: Packets: 00000015803917544 Octets: 00002528626807020 Broadcast: 00000000000000000 Multicast: 00000000000000000 Unicast: 00000015803917544 Crc errors: 00000000000000000 64 octets: 00000000000000000 65-127 octets: 00000000000000000 128-255 octets: 00000015803917544 256-511 octets: 00000000000000000 512-1023 octets: 00000000000000000 1024-1518 octets: 00000000000000000 1519-2047 octets: 00000000000000000 2048-4095 octets: 00000000000000000 4096-8191 octets: 00000000000000000 8192-max octets: 00000000000000000 Undersize: 00000000000000000 Oversize: 00000000000000000 Fragments: 00000000000000000 Drop events: 00000013543314711 Packets per second: 48% Bits per second: 100% TX COUNTERS: Packets: 00000000152856782 Octets: 00000023217300824 Broadcast: 00000000000000000 Multicast: 00000000000000000 Unicast: 00000000152856782 Crc errors: 00000000000000000 64 octets: 00000000000000000 65-127 octets: 00000000000000000 128-255 octets: 00000000152856782 256-511 octets: 00000000000000000 512-1023 octets: 00000000000000000 1024-1518 octets: 00000000000000000 1519-2047 octets: 00000000000000000 2048-4095 octets: 00000000000000000 4096-8191 octets: 00000000000000000 8192-max octets: 00000000000000000 Undersize: 00000000000000000 Oversize: 00000000000000000 Fragments: 00000000000000000 Drop events: 00000000000000000 Packets per second: 46% Bits per second: 100%In the output, the Packets per second and the Bits per second of port 0 are presented as a percentage of their respective maximum values.