Understanding PFs, VFs and PVFs

To find how physical functions and virtual functions are mapped to PVF numbers on the host or the IPU, run the following command on the IPU.

soc# source /opt/ipu_workload_ovs/src/init_ovs.sh
soc# cd /opt/ipu_workload_ovs/src/ipu_mngmt_ovs/software/csc_mgmt_tools/src
soc# ./csc_mngmt --show-pvf-map --<host|soc>

where:

• soc: Refers to the SoC on the IPU.
• host: Refers to the server where the IPU is installed.

To obtain PVF mapping on the IPU, execute the following command.

soc# ./csc_mngmt --show-pvf-map --soc

An output example:

SoC PVF to PF/VF mapping:

PVF: 000, PF: 07, VF: N/A, Type: NET_DEV
PVF: 001, PF: 06, VF: N/A, Type: NET_DEV
PVF: 002, PF: 05, VF: N/A, Type: NET_DEV
PVF: 003, PF: 04, VF: N/A, Type: NET_DEV
PVF: 004, PF: 03, VF: N/A, Type: NET_DEV
PVF: 005, PF: 02, VF: N/A, Type: NET_DEV
PVF: 006, PF: 01, VF: N/A, Type: NET_DEV
PVF: 007, PF: 00, VF: N/A, Type: NUL_DEV

In this output, PVF 0 to 7 are mapped to PF 7 to 0 respectively.

To identify PFs, run the lshw command as follows.

soc# dnf install -y lshw
soc# lshw -class network -businfo

The output displays information about network devices on the system, including their class and bus information.

An output example:

Bus info          Device          Class          Description
============================================================
pci@0000:15:00.0                  network        Intel Corporation
pci@0000:15:00.1                  network        Virtio network device
pci@0000:15:00.2                  network        Virtio network device
pci@0000:15:00.3                  network        Virtio network device
virtio@2          ens6f3          network        Ethernet interface
pci@0000:15:00.4                  network        Virtio network device
virtio@3          ens6f4          network        Ethernet interface
pci@0000:15:00.5                  network        Virtio network device
virtio@4          ens6f5          network        Ethernet interface
pci@0000:15:00.6                  network        Virtio network device
virtio@5          ens6f6          network        Ethernet interface
pci@0000:15:00.7                  network        Virtio network device
virtio@6          ens6f7          network        Ethernet interface
…
…

This output displays how each PF is mapped to each virtio driver instance and each network device name. The PFs are listed as 15:00.0 to 15:00.7, corresponding to PF 0 to 7. Virtio drivers 2 to 6 are mapped to PF 3 to 7 respectively.

The MAC address of each PVF number on the IPU can be checked as follows.

soc# csc_mngmt --show --soc

where:

• --soc: Refers to the SoC on the IPU.

An output example:

…
"net-cfg"    : [
{ "PVF": 000, "config": … MAC: 00:e8:ca:11:bf:79, MTU: 1500" },
{ "PVF": 001, "config": … MAC: 00:e8:ca:11:bf:7a, MTU: 1500" },
{ "PVF": 002, "config": … MAC: 00:e8:ca:11:bf:7b, MTU: 1500" },
{ "PVF": 003, "config": … MAC: 00:e8:ca:11:bf:7c, MTU: 1500" },
{ "PVF": 004, "config": … MAC: 00:e8:ca:11:bf:7d, MTU: 1500" },
{ "PVF": 005, "config": … MAC: 00:e8:ca:11:bf:7e, MTU: 1500" },
{ "PVF": 006, "config": … MAC: 00:e8:ca:11:bf:7f, MTU: 1500" }
],
…

Compare the MAC addresses to network devices. For example:

soc# ip -br link

If namespaces are configured, run the command within the namespace. For example:

soc# ip netns exec soc6 ip -br link
soc# ip netns exec soc7 ip -br link

The output displays a brief overview of the network interfaces on the system. An output example:

…
ens6f3 DOWN  00:e8:ca:11:bf:7d <BROADCAST,MULTICAST>
ens6f4 DOWN  00:e8:ca:11:bf:7c <BROADCAST,MULTICAST>
ens6f5 DOWN  00:e8:ca:11:bf:7b <BROADCAST,MULTICAST>
ens6f6 DOWN  00:e8:ca:11:bf:7a <BROADCAST,MULTICAST>
ens6f7 DOWN  00:e8:ca:11:bf:79 <BROADCAST,MULTICAST>

Combining all of this information, the network device names and the PVF numbers on the IPU are mapped as follows. By default, PVF 0 and 1 on the IPU are configured in the setup and start scripts. See Creating an OVS Bridge on the IPU.

PF/VF mapping to PVF numbers on the host can be found by running the following command on the IPU.

soc# source /opt/ipu_workload_ovs/src/init_ovs.sh
soc# ./csc_mngmt --show-pvf-map --host

where:

• --host: Refers to the server where the IPU is installed.

An output example:

Host PVF to PF/VF mapping:

PVF: 000, PF: 31, VF: N/A, Type: BLK_DEV
PVF: 001, PF: 30, VF: N/A, Type: BLK_DEV
PVF: 002, PF: 29, VF: N/A, Type: BLK_DEV…
…
PVF: 013, PF: 18, VF: N/A, Type: BLK_DEV
PVF: 014, PF: 17, VF: N/A, Type: BLK_DEV
PVF: 015, PF: 16, VF: N/A, Type: BLK_DEV
PVF: 016, PF: 15, VF: N/A, Type: NET_DEV
PVF: 017, PF: 14, VF: N/A, Type: NET_DEV
PVF: 018, PF: 13, VF: N/A, Type: NET_DEV
…
PVF: 029, PF: 02, VF: N/A, Type: NET_DEV
PVF: 030, PF: 01, VF: N/A, Type: NET_DEV
PVF: 031, PF: 00, VF: N/A, Type: NET_DEV

In this case, the PCIe addresses for each PF appear on different PCI buses due to the nature of the implementation.

host# lshw -class network -businfo

The output displays information about network devices on the system, including their class and bus information. An output example:

Bus info          Device     Class          Description
=======================================================
…
pci@0000:86:00.0             network        Virtio network device
virtio@0          enp134s0   network        Virtual I/O device
pci@0000:87:00.0             network        Virtio network device
virtio@1          enp135s0   network        Virtual I/O device
pci@0000:88:00.0             network        Virtio network device
virtio@2          enp136s0   network        Ethernet interface
pci@0000:89:00.0             network        Virtio network device
virtio@3          enp137s0   network        Ethernet interface
pci@0000:8a:00.0             network        Virtio network device
virtio@4          enp138s0   network        Ethernet interface
…
pci@0000:93:00.0             network        Virtio network device
virtio@13         enp147s0   network        Ethernet interface
pci@0000:94:00.0             network        Virtio network device
virtio@14         enp148s0   network        Ethernet interface
pci@0000:95:00.0             network        Virtio network device
virtio@15         enp149s0   network        Ethernet interface

Check the MAC address of each PVF for the host as follows.

soc# ./csc_mngmt --show --host

An output example:

…
"net-cfg"    : [
{ "PVF": 016, "config":  … MAC: 00:e8:ca:11:be:71, MTU: 1500" },
{ "PVF": 017, "config":  … MAC: 00:e8:ca:11:be:72, MTU: 1500" },
{ "PVF": 018, "config":  … MAC: 00:e8:ca:11:be:73, MTU: 1500" },
{ "PVF": 019, "config":  … MAC: 00:e8:ca:11:be:74, MTU: 1500" },
…
…
{ "PVF": 027, "config":  … MAC: 00:e8:ca:11:be:7c, MTU: 1500" },
{ "PVF": 028, "config":  … MAC: 00:e8:ca:11:be:7d, MTU: 1500" },
{ "PVF": 029, "config":  … MAC: 00:e8:ca:11:be:7e, MTU: 1500" },
{ "PVF": 030, "config":  … MAC: 00:e8:ca:11:be:7f, MTU: 1500" },
{ "PVF": 031, "config":  … MAC: 00:e8:ca:11:be:80, MTU: 1500" }

Compare the MAC addresses to network devices. For example:

host# ip -br link

If namespaces are configured, run the command within the namespace. For example:

soc# ip netns exec pf0 ip -br link
soc# ip netns exec pf1 ip -br link

The output displays a brief overview of the network interfaces on the system. An output example:

…
enp134s0 UP  00:e8:ca:11:be:80 <BROADCAST,MULTICAST,UP,LOWER_UP>
enp135s0 UP  00:e8:ca:11:be:7f <BROADCAST,MULTICAST,UP,LOWER_UP>
enp136s0 UP  00:e8:ca:11:be:7e <BROADCAST,MULTICAST,UP,LOWER_UP>
enp137s0 UP  00:e8:ca:11:be:7d <BROADCAST,MULTICAST,UP,LOWER_UP>
enp138s0 UP  00:e8:ca:11:be:7c <BROADCAST,MULTICAST,UP,LOWER_UP>
…
…
enp146s0 UP  00:e8:ca:11:be:74 <BROADCAST,MULTICAST,UP,LOWER_UP>
enp147s0 UP  00:e8:ca:11:be:73 <BROADCAST,MULTICAST,UP,LOWER_UP>
enp148s0 UP  00:e8:ca:11:be:72 <BROADCAST,MULTICAST,UP,LOWER_UP>
enp149s0 UP  00:e8:ca:11:be:71 <BROADCAST,MULTICAST,UP,LOWER_UP>

Combining all of this information, the network device names and the PVFs on the host are mapped as follows. By default, PVF 31 and 30 on the host are configured in the setup and start scripts. See Creating an OVS Bridge on the IPU.