OVS-DPDK Initialization

Getting Started with Napatech Link-Virtualization™ Software

Platform
Napatech SmartNIC
Content Type
Getting Started
Getting Started Guide
Capture Software Version
Link-Virtualization™ Software 4.5

Create and start the OVS Database (OVSDB), configure ports, and start Open vSwitch in order to use the Napatech hardware offload capability.

About this task

This shows command examples to initialize OVS and start the services.

Procedure

  1. Create the OVSDB configuration database file.
    mkdir -p /usr/local/etc/openvswitch
    ovsdb-tool create /usr/local/etc/openvswitch/conf.db \ 
    /usr/local/share/openvswitch/vswitch.ovsschema
    mkdir -p /usr/local/var/run/openvswitch
    mkdir -p /usr/local/var/log/openvswitch

    The OVSDB configuration database ordinarily need be created only one time. It provides a persistent data store which retains configuration information across system reboots.

    Adding, removing and querying OVSDB configuration information is normally performed using the ovs-vsctl utility as described in the following steps of this procedure.

  2. Start the OVSDB database service.
    ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
      --remote=db:Open_vSwitch,Open_vSwitch,manager_options --pidfile --detach
  3. Initialize OVSDB other_config settings, while OVSDB is running:
    ovs-vsctl --no-wait init
    ovs-vsctl --no-wait clear Open_vSwitch . other_config
    ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true
    ovs-vsctl --no-wait set Open_vSwitch . other_config:hw-offload=true
    ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-socket-mem="4096,0"
    ovs-vsctl --no-wait get Open_vSwitch . other_config
    where:
    • other_config:dpdk-init=true specifies to initialize OVS and support DPDK ports.
    • other_config:hw-offload=true enables OVS with hardware offload. Specify other_config:hw-offload=false to run OVS without hardware offload.
    • other_config:dpdk-socket-mem="4096,0" specifies an allocation of 4 GB on NUMA node 0, and 0 GB on NUMA node 1.
    Note:

    In this example dpdk-socket-mem allocates on NUMA node 0 only. If the server has more than one NUMA node, allocate memory on the same NUMA node as the SmartNIC NUMA node location to achieve optimal performance. The following commands show how to determine the SmartNIC NUMA node location.

    DEV=$(lspci -Dd 18f4: | cut -d' ' -f1 | head -1)
    cat /sys/bus/pci/devices/$DEV/numa_node

    An output example when the SmartNIC NUMA node is 0:

    cat /sys/bus/pci/devices/$DEV/numa_node
    0
    
  4. Allocate one CPU core for the OVS control plane.
    ovs-vsctl --no-wait set Open_vSwitch . other_config:pmd-cpu-mask=0x4
    ovs-vsctl --no-wait get Open_vSwitch . other_config

    In this example, we use 0x4 as a CPU mask. 0x4 (binary 100) is the hex mask that represents CPU core # 2.

    Note:

    Select a CPU core on the same NUMA node as the SmartNIC NUMA node location. See step 3 about how to determine the SmartNIC NUMA node location. Use the lscpu command to determine the mapping of CPUs to NUMA nodes.

    lscpu | grep NUMA
    NUMA node(s):          2
    NUMA node0 CPU(s):     0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38
    NUMA node1 CPU(s):     1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31,33,35,37,39
    
  5. Configure the Napatech-specific SmartNIC ports.
    ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-extra="-n4 \
    --iova-mode=pa --vfio-vf-token=<uuid>""
    where:
    • uuid is a generated value using the uuidgen command. For example:
      # uuidgen
      e6b9d23c-c9bc-4105-ae0c-3957eeea77a0
  6. Verify the configuration so far.
    ovs-vsctl list open_vswitch
    An output example:
    _uuid               : f285c729-111f-4138-8c50-543f10a59975
    bridges             : []
    cur_cfg             : 0
    datapath_types      : []
    datapaths           : {}
    db_version          : []
    dpdk_initialized    : false
    dpdk_version        : []
    external_ids        : {}
    iface_types         : []
    manager_options     : []
    next_cfg            : 0
    other_config        : {dpdk-extra="-n4 --iova-mode=pa --vfio-vf-token=e6b9d23c-c9bc-4105-ae0c-3957eeea77a0", dpdk-init="true", dpdk-socket-mem="4096,0", hw-offload="true", pmd-cpu-mask="0x4"}
    ovs_version         : []
    ssl                 : []
    statistics          : {}
    system_type         : []
    system_version      : []
  7. Start OVS vSwitch:
    touch /usr/local/var/log/openvswitch/ovs-vswitch.log
    ovs-vswitchd unix:/usr/local/var/run/openvswitch/db.sock --pidfile \
      --log-file=/usr/local/var/log/openvswitch/ovs-vswitch.log --detach
    Verify whether OVS services are running:
    ps aux | grep ovs
    An output example:
    root       83899  0.0  0.0  52196  4384 ?        Ss   10:31   0:00 ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock --remote=db:Open_vSwitch,Open_vSwitch,manager_options --pidfile --detach
    root       83915 18.1  0.0 269147652 31508 ?     SLsl 10:37   0:04 ovs-vswitchd unix:/usr/local/var/run/openvswitch/db.sock --pidfile --log-file=/usr/local/var/log/openvswitch/ovs-vswitch.log --detach
    root       83937  0.0  0.0  12144  1092 pts/0    S+   10:37   0:00 grep --color=auto ovs
    Note: If ovs-vswitchd is not shown in the output, check the log messages in the /usr/local/var/log/openvswitch/ovs-vswitch.log file

Results

OVS is running and you are ready setup specific configurations with hardware offload capability.