Overcloud Deployment

Getting Started with Napatech Link-Virtualization™ Software

Platform
Napatech SmartNIC
Content Type
Getting Started Guide
Capture Software Version
Link-Virtualization™ Software 4.4

Initially, introspection is done to extract information from the nodes. The information is used when installing the overcloud.

About this task

Introspection is an initial part of the overcloud deployment. Introspection is a special process with a special image, ironic python agent image (IPA) that is PXE-loaded onto all nodes in the overcloud configuration. IPA images are designed to extract information from the nodes into a database, which will be consulted by TripleO when installing the overcloud. Properties of the hardware such as disk, NIC and CPU are extracted using standard Linux tools and scripts.

Before you begin

Procedure

  1. Login in to the undercloud server (director) and change to the non-root user (stack).
    su – stack
    Note: Overcloud deployment is performed in the undercloud server.
    You can access undercloud services after sourcing the stackrc file.
    source /home/stack/stackrc
    An output example:
    [stack@director ~]$ source /home/stack/stackrc
    (undercloud) [stack@director ~]$
  2. Create a new directory for overcloud images
    mkdir /home/stack/images
    cd /home/stack/images
  3. Download the CentOS IPA image.
    wget \
    https://images.rdoproject.org/centos8/victoria/rdo_trunk/current-tripleo/ironic-python-agent.tar
    Unpack the package.
    tar xf ironic-python-agent.tar
    Delete the tar file.
    rm ironic-python-agent.tar
  4. Place the prepared overcloud images, overcloud_full.qcow2 in the /home/stack/images directory. For example:
    cp overcloud-full.qcow2 /home/stack/images/
    cp overcloud-full.initrd /home/stack/images/
    cp overcloud-full.vmlinuz /home/stack/images/
    See Preparing the Overcloud Image for more information about how to build the overcloud image.
  5. Upload the images.
    cd /home/stack/images/
    openstack overcloud image upload --update-existing
    Note: This command must be executed in the /home/stack/images directory. All the images in the current directory will be uploaded. It takes about 30 minutes.
    An output example:
    /usr/lib/python3.6/site-packages/openstack/resource.py:351: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec()
      len(inspect.getargspec(type_).args) > 1)Image "overcloud-full-vmlinuz" was uploaded.
    +--------------------------------------+------------------------+-------------+----------+--------+
    |                  ID                  |          Name          | Disk Format |   Size   | Status |
    +--------------------------------------+------------------------+-------------+----------+--------+
    | 27b6a7c8-75a3-4018-ad1e-ab4692d2994c | overcloud-full-vmlinuz |     aki     | 10460296 | active |
    +--------------------------------------+------------------------+-------------+----------+--------+
    Image "overcloud-full-initrd" was uploaded.
    +--------------------------------------+-----------------------+-------------+----------+--------+
    |                  ID                  |          Name         | Disk Format |   Size   | Status |
    +--------------------------------------+-----------------------+-------------+----------+--------+
    | 5925e0c6-da52-482e-9f8d-b58922e8a659 | overcloud-full-initrd |     ari     | 70253801 | active |
    +--------------------------------------+-----------------------+-------------+----------+--------+
    Image "overcloud-full" was uploaded.
    +--------------------------------------+----------------+-------------+------------+--------+
    |                  ID                  |      Name      | Disk Format |    Size    | Status |
    +--------------------------------------+----------------+-------------+------------+--------+
    | 94518cc5-99f9-4fdf-9ed1-4ef3e21dfd0b | overcloud-full |     raw     | 5108203520 | active |
    +--------------------------------------+----------------+-------------+------------+--------+
    Note: The existing overcloud images must be deleted to upload new overcloud images. Run the following command to check the uploaded overcloud image:
    openstack image list
    An output example:
    ...
    +--------------------------------------+------------------------+--------+
    | ID                                   | Name                   | Status |
    +--------------------------------------+------------------------+--------+
    | db5e4e8e-cd2a-45a6-b39a-cbc4194294cf | overcloud-full         | active |
    | 9dd68205-404f-47ba-aab3-8fb4927e707f | overcloud-full-initrd  | active |
    | cc3336b7-a841-48b2-b768-a61282bf1547 | overcloud-full-vmlinuz | active |
    +--------------------------------------+------------------------+--------+
    Delete the overcloud images.
    openstack image delete <image_id>
    For example:
    openstack image delete adf7568a-33e2-40e1-9bbb-3ded500bf19e
    After the existing images are deleted, copy the new images to the /home/stack/images directory and upload them.
  6. Import the nodes.json file.

    The nodes.json file contains information about each node, such as the number of CPU cores, the CPU architecture type, the size of disk and available memory, the type of the remote control interface and related credentials.

    • address: The MAC address of the network interface for the provisioning network
    • pm_type: The type of the remote management interface
    • name: The node name
    • pm_user: The user name for IPMI or iDRAC
    • pm_password: The password for IPMI or iDRAC
    • pm_addr: The IP address of the iDRAC interface
    Confirm the configuration in the file and run the following command to import the nodes.json file.
    cd /home/stack/templates
    openstack overcloud node import nodes.json
    An output example:
    Successfully registered node UUID 87839a51-a554-4d2a-8c25-bf11fc180b13
    Successfully registered node UUID a780e535-b7a3-4a8f-9ac1-ef13d346ae59
    Successfully registered node UUID be7d21f0-8ed1-42b3-b9ac-47546b070c99
  7. Update firstboot.yaml with the UUID of each node.
    Run the following command to display the UUID of each node after the nodes.json file has been imported as described in step 6.
    openstack baremetal node list
    An output example:
    +--------------------------------------+-----------+---------------+-------------+----------------
    | UUID                                 | Name      | Instance UUID | Power State | Provisioning...
    +--------------------------------------+-----------+---------------+-------------+----------------
    | 87839a51-a554-4d2a-8c25-bf11fc180b13 | control   | None          | power on    | manageable  ...
    | a780e535-b7a3-4a8f-9ac1-ef13d346ae59 | compute-0 | None          | power on    | manageable  ...
    | be7d21f0-8ed1-42b3-b9ac-47546b070c99 | compute-1 | None          | power on    | manageable  ...
    +--------------------------------------+-----------+---------------+-------------+----------------
  8. Set the force_persistent_boot_device parameter in the driver-info to Never on all nodes.
    Note: This step applies to Dell servers only. An iDRAC error may occur if this is not done. See the Red Hat official document at Troubleshooting the bare metal service.
    openstack baremetal node set --driver-info \
    force_persistent_boot_device=Never <node_uuid>
    where node_uuid is the UUID of each node. For example:
    openstack baremetal node set --driver-info force_persistent_boot_device=Never \
    87839a51-a554-4d2a-8c25-bf11fc180b13
    openstack baremetal node set --driver-info force_persistent_boot_device=Never \
    a780e535-b7a3-4a8f-9ac1-ef13d346ae59
    openstack baremetal node set --driver-info force_persistent_boot_device=Never \
    be7d21f0-8ed1-42b3-b9ac-47546b070c99
  9. Configure all nodes in manageable state for the deployment.
    openstack overcloud node configure --all-manageable
    An output example:
    Successfully configured the nodes.
  10. Introspect all nodes in manageable state and make the nodes available for deployment once the introspection is finished. It takes about 45 minutes.
    openstack overcloud node introspect --provide --all-manageable
    An output example:
    ...
    PLAY RECAP *********************************************************************
    localhost                  : ok=7    changed=4    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0
    2022-04-25 14:37:20.754819 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Summary Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    2022-04-25 14:37:20.755615 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Total Tasks: 9          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    2022-04-25 14:37:20.756408 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Elapsed Time: 0:00:40.356479 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    2022-04-25 14:37:20.757233 |                                 UUID |       Info |       Host |   Task Name |   Run Time
    2022-04-25 14:37:20.758072 | a0b3cce9-60a2-7964-dd06-000000000013 |    SUMMARY |  localhost | Wait for nova resources | 17.81s
    2022-04-25 14:37:20.758858 | a0b3cce9-60a2-7964-dd06-000000000016 |    SUMMARY |  localhost | poll for completion | 6.39s
    2022-04-25 14:37:20.759697 | a0b3cce9-60a2-7964-dd06-000000000012 |    SUMMARY |  localhost | Run cell_v2 host discovery | 5.63s
    2022-04-25 14:37:20.760470 | a0b3cce9-60a2-7964-dd06-00000000000f |    SUMMARY |  localhost | Make nodes available | 4.24s
    2022-04-25 14:37:20.761323 | a0b3cce9-60a2-7964-dd06-000000000010 |    SUMMARY |  localhost | Detect nova service endpoint | 4.14s
    2022-04-25 14:37:20.762147 | a0b3cce9-60a2-7964-dd06-000000000015 |    SUMMARY |  localhost | Power off nodes | 1.82s
    2022-04-25 14:37:20.762927 | a0b3cce9-60a2-7964-dd06-000000000008 |    SUMMARY |  localhost | Check for required inputs | 0.06s
    2022-04-25 14:37:20.763737 | a0b3cce9-60a2-7964-dd06-00000000000c |    SUMMARY |  localhost | Notice | 0.05s
    2022-04-25 14:37:20.764619 | a0b3cce9-60a2-7964-dd06-00000000000a |    SUMMARY |  localhost | Set node_uuids_provide fact | 0.04s
    2022-04-25 14:37:20.765381 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ End Summary Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    Successfully provided nodes: ['87839a51-a554-4d2a-8c25-bf11fc180b13', 'a780e535-b7a3-4a8f-9ac1-ef13d346ae59', 'be7d21f0-8ed1-42b3-b9ac-47546b070c99']
  11. Set appropriate parameters for deployment.
    openstack baremetal node set control --property capabilities=profile:baremetal
    openstack baremetal node set compute-1 --property capabilities=profile:baremetal
    openstack baremetal node set compute-0 --property capabilities=profile:baremetal
    openstack baremetal node set control --resource-class baremetal
    openstack baremetal node set compute-1 --resource-class baremetal
    openstack baremetal node set compute-0 --resource-class baremetal
    openstack baremetal node set compute-1 --property capabilities=node:computedpdk
    openstack baremetal node set compute-0 --property capabilities=node:computedpdk
    openstack baremetal node set control --property capabilities=node:controller
    openstack subnet set --dns-nameserver 8.8.8.8 ctlplane-subnet
    Note: The node names are defined in the nodes.json file.
  12. Run overcloud deployment.
    cd /home/stack/templates
    openstack overcloud deploy --templates -r roles_data.yaml -n network_data.yaml \
    -e /usr/share/openstack-tripleo-heat-templates/ci/environments/network/multiple-nics/network-environment.yaml \
    -e /usr/share/openstack-tripleo-heat-templates/ci/environments/network/multiple-nics/network-isolation.yaml \
    -e /usr/share/openstack-tripleo-heat-templates/environments/services/neutron-ovs-dpdk.yaml \
    -e /usr/share/openstack-tripleo-heat-templates/environments/services/neutron-ovs.yaml \
    -e scheduler_hints_env.yaml -e firstboot.yaml -e custom-network-configuration.yaml --verbose --debug \
    -e containers-prepare-parameters.yaml
    An output example after successful deployment.
    ...
    PLAY RECAP *********************************************************************
    overcloud-computeovsdpdk-0 : ok=520  changed=211  unreachable=0    failed=0    skipped=196  rescued=0    ignored=0
    overcloud-computeovsdpdk-1 : ok=506  changed=211  unreachable=0    failed=0    skipped=196  rescued=0    ignored=0
    overcloud-controller-0     : ok=592  changed=267  unreachable=0    failed=0    skipped=218  rescued=0    ignored=0
    undercloud                 : ok=80   changed=29   unreachable=0    failed=0    skipped=9    rescued=0    ignored=0
    2022-04-26 12:40:13.892265 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Summary Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    2022-04-26 12:40:13.893272 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Total Tasks: 1736       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    2022-04-26 12:40:13.894124 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Elapsed Time: 0:32:07.502815 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    2022-04-26 12:40:13.895018 |                                 UUID |       Info |       Host |   Task Name |   Run Time
    2022-04-26 12:40:13.896034 | a0b3cce9-60a2-9b39-37ce-0000000029c2 |    SUMMARY | overcloud-controller-0 | Pre-fetch all the containers | 348.50s
    2022-04-26 12:40:13.896959 | a0b3cce9-60a2-9b39-37ce-0000000029f5 |    SUMMARY | overcloud-computeovsdpdk-1 | Pre-fetch all the containers | 303.65s
    2022-04-26 12:40:13.897839 | a0b3cce9-60a2-9b39-37ce-000000002a20 |    SUMMARY | overcloud-computeovsdpdk-0 | Pre-fetch all the containers | 297.23s
    ...
    Host 192.168.24.15 not found in /home/stack/.ssh/known_hosts
    Overcloud Endpoint: http://192.168.24.15:5000
    Overcloud Horizon Dashboard URL: http://192.168.24.15:80/dashboard
    Overcloud rc file: /home/stack/overcloudrc
    Overcloud Deployed without error
    2022-04-26 12:41:10.312 293470 INFO osc_lib.shell [-] END return value: None
    Run the following command to view the deployment result.
    openstack stack list
    An output example:
    +--------------------------------------+------------+----------------------------------+-----------------...
    | ID                                   | Stack Name | Project                          | Stack Status    ...
    +--------------------------------------+------------+----------------------------------+-----------------...
    | 7ffaea8a-41df-42d7-a939-6953a1e83b6d | overcloud  | 19c194354320469c8e3054d2dfe50849 | CREATE_COMPLETE ...
    +--------------------------------------+------------+----------------------------------+-----------------...
    Note: When you need to redeploy overcloud, run the following command to delete any existing overcloud.
    openstack stack delete overcloud
    It takes a few minutes. Check whether overcloud has been deleted before redeployment.
    openstack stack list
    If the Stack Status is DELETE_FAILED, run the openstack stack delete overcloud command again.
  13. List the overcloud servers.
    openstack server list
    An output example:
    ...
    +--------------------------------------+----------------------------+--------+------------------------...
    | ID                                   | Name                       | Status | Networks               ...
    +--------------------------------------+----------------------------+--------+------------------------...
    | 393058b5-8820-42d3-9265-c79b344a5280 | overcloud-controller-0     | ACTIVE | ctlplane=192.168.24.6  ...
    | 488ff29d-afcf-4dab-b232-1ffc89d44de0 | overcloud-computeovsdpdk-1 | ACTIVE | ctlplane=192.168.24.20 ...
    | 9e662061-7a6a-4003-80f6-1ed262a300ef | overcloud-computeovsdpdk-0 | ACTIVE | ctlplane=192.168.24.7  ...
    +--------------------------------------+----------------------------+--------+------------------------...
  14. Access the compute nodes for configuration after overcloud deployment. The changes must be made on each compute node.
    For example:
    ssh root@192.168.24.7
  15. Edit the nova.conf file.
    vi /var/lib/config-data/puppet-generated/nova_libvirt/etc/nova/nova.conf
    Add filters to the [filter_scheduler] section to set the filters on the compute nodes.
    [filter_scheduler] 
    available_filters = nova.scheduler.filters.all_filters
    
    enabled_filters = AvailabilityZoneFilter, ComputeFilter, ComputeCapabilitiesFilter, ImagePropertiesFilter, ServerGroupAntiAffinityFilter, ServerGroupAffinityFilter, SameHostFilter, DifferentHostFilter, PciPassthroughFilter
    Note: The values for the available_filters and enabled_filters parameters must be added as one line.
    Specify the PCI bus IDs of the Napatech SmartNIC virtual function interfaces in the [pci] section. For the VXLAN setup:
    [pci]
    passthrough_whitelist = { "address": "<xxxx:xx:xx.xx>"}
    passthrough_whitelist = { "address": "<xxxx:xx:xx.xx>"}

    where:

    xxxx:xx:xx.xx is a PCI ID of the Napatech SmartNIC virtual function interfaces. Run the following command to check PCI IDs:
    lspci -Dd 18f4:

    Use the devices xxxx:xx:xx.05 and higher.

    A VXLAN example:
    [pci] 
    passthrough_whitelist = { "address": "0000:65:00.05" } 
    passthrough_whitelist = { "address": "0000:65:00.06" }
    passthrough_whitelist = { "address": "0000:65:00.07" } 
    passthrough_whitelist = { "address": "0000:65:01.00" }
    passthrough_whitelist = { "address": "0000:65:01.01" } 
    passthrough_whitelist = { "address": "0000:65:01.02" }
    passthrough_whitelist = { "address": "0000:65:01.03" } 
    passthrough_whitelist = { "address": "0000:65:01.04" }
    passthrough_whitelist = { "address": "0000:65:01.05" }
    A VLAN example:
    [pci] 
    passthrough_whitelist = { "address": "0000:65:00.05" , "physical_network": "public"}
    passthrough_whitelist = { "address": "0000:65:00.06" , "physical_network": "public"}
    passthrough_whitelist = { "address": "0000:65:00.07" , "physical_network": "public"}
    passthrough_whitelist = { "address": "0000:65:01.00" , "physical_network": "public"}
    passthrough_whitelist = { "address": "0000:65:01.01" , "physical_network": "public"}
    passthrough_whitelist = { "address": "0000:65:01.02" , "physical_network": "public"}
    passthrough_whitelist = { "address": "0000:65:01.03" , "physical_network": "public"}
    passthrough_whitelist = { "address": "0000:65:01.04" , "physical_network": "public"}
    passthrough_whitelist = { "address": "0000:65:01.05" , "physical_network": "public"}
  16. Update QEMU access rights.
    vi /var/lib/config-data/puppet-generated/nova_libvirt/etc/libvirt/qemu.conf
    Add these lines to the file.
    user = "root" 
    group = "root"
    Restart nova services.
    systemctl restart tripleo_nova*
    Check the status.
    systemctl status tripleo_nova*
  17. Disable SELinux to be able create the Napatech OVS socket.
    setenforce 0
    Check the status of SELinux.
    sestatus
    An output example:
    SELinux status:                 enabled
    SELinuxfs mount:                /sys/fs/selinux
    SELinux root directory:         /etc/selinux
    Loaded policy name:             targeted
    Current mode:                   permissive
    Mode from config file:          enforcing
    Policy MLS status:              enabled
    Policy deny_unknown status:     allowed
    Memory protection checking:     actual (secure)
    Max kernel policy version:      33
    Current mode is set to permissive.
  18. Close the connection to the compute node.
    exit
    Note: Repeat from step 14 on each compute node.