Virtio Packed Ring

You can setup virtio packed ring in a VM or in a VM using RSS.

1. Setup QEMU for the packed ring and 2 queues on vPort6 as in the following example:

   # taskset -c 4,6,8,10 /usr/libexec/qemu-kvm \
    -enable-kvm -cpu host -m 8192 -smp 4 \
    -object memory-backend-file,id=mem,size=8192M,mem-path=/mnt/huge,share=on \
    -chardev socket,id=char0,path=/usr/local/var/run/stdvio6,server=on \
    -netdev type=vhost-user,id=mynet1,chardev=char0,vhostforce=on,queues=2 \
    -device virtio-net-pci,packed=on,netdev=mynet1,mq=on,vectors=6,mac=52:55:00:08:d9:02,mrg_rxbuf=on \
    -numa node,memdev=mem -mem-prealloc \
    -net user,hostfwd=tcp::10022-:22 \
    -net nic,macaddr=52:54:00:12:34:70 \
    -display gtk \
    -hda ./fedora33.img

   Note:

   • Remember to add packed=on on virtio-net-pci device, otherwise it will continue using the split ring. Therefore, leave it out when the split ring is preferred.
   • On netdev added queues=2 for 2 queues in a multiqueue setup (RSS). Leave it out when running 1 queue only.
   • mq=on,vectors=<2+2*queues> is also added for multiqueue (RSS). Leave it out when running 1 queue only.
   • mrg_rxbuf=on is for segmenting jumbo packets when virtio ring buffer size is smaller than the packet size.
2. Set IP address and ping from another VM on the L2 segment to verify that the Linux kernel virtio-net is running when not using RSS.
3. Verify:

   # ethtool -S <device>

4. Verify that the Linux kernel virtio-net is running when using multiqueue (RSS):

   # ethtool -l <device> 

   If the queues are not combined enter the following: command:

   # ethtool -L <device> combine 2

   In this example we combine 2 queues. The maximum of supported queues is 3.

5. Set the IP address and receive traffic of any kind for example, ping.
6. Verify load distribution on queues:

   # ethtool -S <device>

   Note:

   Linux kernel virtio-net driver relies on interrupts to work and the SmartNIC needs the packed ring to communicate with the kernel driver. Since the VFs on the SmartNIC used for each queue has a limited amount of interrupt vectors, the maximum queues supported when running Linux kernel virtio-net driver is 3.

   If DPDK poll mode driver is used, interrupts are not used and there is no limitation. We can then have up to 126 queues on a single virtio port. Steps 7 and 8 is an example of how to test multiqueue in a DPDK application in a VM.

7. Using DPDK testpmd in VM with 2 RSS queues:

   # ./testpmd -n6 -- -i --portmask=1 --rxq=2 --txq=2 --nb-cores=2 \
   --eth-peer=0,<MAC address of traffic generator>

8. Using testpmd:

   testpmd> set fwd mac
   testpmd> start tx_first