Create and start the OVS Database (OVSDB), configure ports, and start Open vSwitch in order to use the Napatech hardware offload capability.
About this task
Procedure
-
Create the OVSDB configuration database file.
mkdir -p /usr/local/etc/openvswitch ovsdb-tool create /usr/local/etc/openvswitch/conf.db \ /usr/local/share/openvswitch/vswitch.ovsschema mkdir -p /usr/local/var/run/openvswitch mkdir -p /usr/local/var/log/openvswitch
The OVSDB configuration database ordinarily need be created only one time. It provides a persistent data store which retains configuration information across system reboots.
Adding, removing and querying OVSDB configuration information is normally performed using the ovs-vsctl utility as described in the following steps of this procedure.
-
Start the OVSDB database service.
ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \ --remote=db:Open_vSwitch,Open_vSwitch,manager_options --pidfile --detach
-
Initialize OVSDB other_config settings, while OVSDB is running:
ovs-vsctl --no-wait init ovs-vsctl --no-wait clear Open_vSwitch . other_config ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true ovs-vsctl --no-wait set Open_vSwitch . other_config:hw-offload=true ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-socket-mem="4096,0" ovs-vsctl --no-wait get Open_vSwitch . other_config
where:- other_config:dpdk-init=true specifies to initialize OVS and support DPDK ports.
- other_config:hw-offload=true enables OVS with hardware offload. Specify other_config:hw-offload=false to run OVS without hardware offload.
- other_config:dpdk-socket-mem="4096,0" specifies an allocation of 4 GB on NUMA node 0, and 0 GB on NUMA node 1.
Note:In this example dpdk-socket-mem allocates on NUMA node 0 only. If the server has more than one NUMA node, allocate memory on the same NUMA node as the SmartNIC NUMA node location to achieve optimal performance. The following commands show how to determine the SmartNIC NUMA node location.
DEV=$(lspci -Dd 18f4: | cut -d' ' -f1 | head -1) cat /sys/bus/pci/devices/$DEV/numa_node
An output example when the SmartNIC NUMA node is 0:
cat /sys/bus/pci/devices/$DEV/numa_node 0
-
Allocate one CPU core for the OVS control plane.
ovs-vsctl --no-wait set Open_vSwitch . other_config:pmd-cpu-mask=0x4 ovs-vsctl --no-wait get Open_vSwitch . other_config
In this example, we use 0x4 as a CPU mask. 0x4 (binary 100) is the hex mask that represents CPU core # 2.
Note:Select a CPU core on the same NUMA node as the SmartNIC NUMA node location. See step 3 about how to determine the SmartNIC NUMA node location. Use the lscpu command to determine the mapping of CPUs to NUMA nodes.
lscpu | grep NUMA NUMA node(s): 2 NUMA node0 CPU(s): 0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38 NUMA node1 CPU(s): 1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31,33,35,37,39
-
Configure the Napatech-specific SmartNIC ports.
ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-extra="-n4 \ --iova-mode=pa --vfio-vf-token=<uuid>""
where:- uuid is a generated value using the
uuidgen command. For example:
# uuidgen e6b9d23c-c9bc-4105-ae0c-3957eeea77a0
- uuid is a generated value using the
uuidgen command. For example:
-
Verify the configuration so far.
ovs-vsctl list open_vswitch
An output example:_uuid : f285c729-111f-4138-8c50-543f10a59975 bridges : [] cur_cfg : 0 datapath_types : [] datapaths : {} db_version : [] dpdk_initialized : false dpdk_version : [] external_ids : {} iface_types : [] manager_options : [] next_cfg : 0 other_config : {dpdk-extra="-n4 --iova-mode=pa --vfio-vf-token=e6b9d23c-c9bc-4105-ae0c-3957eeea77a0", dpdk-init="true", dpdk-socket-mem="4096,0", hw-offload="true", pmd-cpu-mask="0x4"} ovs_version : [] ssl : [] statistics : {} system_type : [] system_version : []
-
Start OVS vSwitch:
touch /usr/local/var/log/openvswitch/ovs-vswitch.log ovs-vswitchd unix:/usr/local/var/run/openvswitch/db.sock --pidfile \ --log-file=/usr/local/var/log/openvswitch/ovs-vswitch.log --detach
Verify whether OVS services are running:ps aux | grep ovs
An output example:root 83899 0.0 0.0 52196 4384 ? Ss 10:31 0:00 ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock --remote=db:Open_vSwitch,Open_vSwitch,manager_options --pidfile --detach root 83915 18.1 0.0 269147652 31508 ? SLsl 10:37 0:04 ovs-vswitchd unix:/usr/local/var/run/openvswitch/db.sock --pidfile --log-file=/usr/local/var/log/openvswitch/ovs-vswitch.log --detach root 83937 0.0 0.0 12144 1092 pts/0 S+ 10:37 0:00 grep --color=auto ovs
Note: If ovs-vswitchd is not shown in the output, check the log messages in the /usr/local/var/log/openvswitch/ovs-vswitch.log file