OVS-DPDK Initialization

Getting Started with Napatech Link-Virtualization™ Software

Platform
Napatech SmartNIC
Content Type
Getting Started Guide
Capture Software Version
Link-Virtualization™ Software 4.4

Create and start the OVS Database (OVSDB), configure ports, and start Open vSwitch in order to use the Napatech hardware offload capability.

Procedure

  1. Create the OVSDB configuration database file.
    mkdir -p /usr/local/etc/openvswitch
    ovsdb-tool create /usr/local/etc/openvswitch/conf.db \ 
    /usr/local/share/openvswitch/vswitch.ovsschema
    mkdir -p /usr/local/var/run/openvswitch
    mkdir -p /usr/local/var/log/openvswitch

    The OVSDB configuration database ordinarily need be created only one time. It provides a persistent data store which retains configuration information across system reboots.

    Adding, removing and querying OVSDB configuration information is normally performed using the ovs-vsctl utility as described in the following steps of this procedure.

  2. Start the OVSDB database service.
    ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
      --remote=db:Open_vSwitch,Open_vSwitch,manager_options --pidfile --detach
  3. Initialize OVSDB other_config settings, while OVSDB is running:
    ovs-vsctl --no-wait init
    ovs-vsctl --no-wait clear Open_vSwitch . other_config
    ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true
    ovs-vsctl --no-wait set Open_vSwitch . other_config:hw-offload=true
    ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-socket-mem="4096,0"
    ovs-vsctl --no-wait get Open_vSwitch . other_config
    where:
    • other_config:dpdk-init=true specifies to initialize OVS and support DPDK ports.
    • other_config:hw-offload=true enables OVS with hardware offload. Specify other_config:hw-offload=false to run OVS without hardware offload.
    • other_config:dpdk-socket-mem="4096,0" specifies an allocation of 4 GB on NUMA node 0, and 0 GB on NUMA node 1.
    Note:

    In this example dpdk-socket-mem allocates on NUMA node 0 only. If the server has more than one NUMA node, allocate memory on the same NUMA node as the SmartNIC NUMA node location to achieve optimal performance. The following commands show how to determine the SmartNIC NUMA node location.

    DEV=$(lspci -Dd 18f4: | cut -d' ' -f1 | head -1)
    cat /sys/bus/pci/devices/$DEV/numa_node

    An output example when the SmartNIC NUMA node is 0:

    cat /sys/bus/pci/devices/$DEV/numa_node
    0
    
  4. Allocate one CPU core for the OVS control plane.
    ovs-vsctl --no-wait set Open_vSwitch . other_config:pmd-cpu-mask=0x4
    ovs-vsctl --no-wait get Open_vSwitch . other_config

    In this example, we use 0x4 as a CPU mask. 0x4 (binary 100) is the hex mask that represents CPU core # 2.

    Note:

    Select a CPU core on the same NUMA node as the SmartNIC NUMA node location. See step 3 about how to determine the SmartNIC NUMA node location. Use the lscpu command to determine the mapping of CPUs to NUMA nodes.

    lscpu | grep NUMA
    NUMA node(s):          2
    NUMA node0 CPU(s):     0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38
    NUMA node1 CPU(s):     1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31,33,35,37,39
    
  5. Configure the Napatech-specific SmartNIC ports.
    ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-extra="-n4 \
    --vdev=eth_ntnic0,port=0 --vdev=eth_ntnic1,port=1"
    where:
    • --vdev=eth_ntnic0,port=0 --vdev=eth_ntnic1,port=1 specifies the Napatech SmartNIC ports on physical ports 0 and 1.
  6. Verify the configuration so far.
    ovs-vsctl list open_vswitch
    An output example:
    _uuid               : b4c73e35-705a-421f-9207-f17eedc6505b
    bridges             : []
    cur_cfg             : 0
    datapath_types      : []
    db_version          : []
    dpdk_initialized    : false
    dpdk_version        : []
    external_ids        : {}
    iface_types         : []
    manager_options     : []
    next_cfg            : 0
    other_config        : {dpdk-extra="-n4 --vdev=eth_ntnic0,port=0 --vdev=eth_ntnic1,port=1", dpdk-init="true", dpdk-socket-mem="4096,0", hw-offload="true", pmd-cpu-mask="0x4"}
    ovs_version         : []
    ssl                 : []
    statistics          : {}
    system_type         : []
    system_version      : []
  7. Start ntservice then OVS vSwitch:
    /opt/napatech3/bin/ntstart.sh
    
    cat /dev/null > /usr/local/var/log/openvswitch/ovs-vswitch.log
    ovs-vswitchd unix:/usr/local/var/run/openvswitch/db.sock --pidfile \
      --log-file=/usr/local/var/log/openvswitch/ovs-vswitch.log --detach
    Verify whether OVS services are running:
    ps aux | grep ovs
    An output example:
    root 10799 0.0 0.0  50708  3912 ?     Ss   09:27 0:00 ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock --remote=db:Open_vS
    root 11990 8.6 0.0 67976012 20964 ?   SLsl 11:04 0:00 ovs-vswitchd unix:/usr/local/var/run/openvswitch/db.sock --pidfile --log-file=/usr/loc
    root 12003 0.0 0.0  12136  1140 pts/0 S+   11:04 0:00 grep --color=auto ovs
    Note: If ovs-vswitchd is not shown in the output, check the log messages in the /usr/local/var/log/openvswitch/ovs-vswitch.log file

Results

OVS is running and you are ready setup specific configurations with hardware offload capability.