Initially, introspection is done to extract information from the nodes. The information is used when installing the overcloud.
About this task
Introspection is an initial part of the overcloud deployment. Introspection is a special process with a special image, ironic python agent image (IPA) that is PXE-loaded onto all nodes in the overcloud configuration. IPA images are designed to extract information from the nodes into a database, which will be consulted by TripleO when installing the overcloud. Properties of the hardware such as disk, NIC and CPU are extracted using standard Linux tools and scripts.
Before you begin
- PXE boot is enabled on the overcloud servers: See Configuring PXE Boot in BIOS.
- The overcloud image is prepared: See Preparing the Overcloud Image.
- The modified TripleO templates are placed in the correct directories: See Preparing Templates
- The container images are prepared and uploaded: See Preparing the Container Images with Podman.
- The SmartNIC is updated with an appropriate FPGA image: See Updating the FPGA image.
Procedure
-
Login in to the undercloud server (director) and change to the non-root user
(stack).
su – stack
Note: Overcloud deployment is performed in the undercloud server.You can access undercloud services after sourcing the stackrc file.source /home/stack/stackrc
An output example:[stack@director ~]$ source /home/stack/stackrc (undercloud) [stack@director ~]$
-
Create a new directory for overcloud images
mkdir /home/stack/images cd /home/stack/images
-
Download the CentOS IPA image.
wget \ https://images.rdoproject.org/centos8/victoria/rdo_trunk/current-tripleo/ironic-python-agent.tar
Unpack the package.tar xf ironic-python-agent.tar
Delete the tar file.rm ironic-python-agent.tar
-
Place the prepared overcloud images, overcloud_full.qcow2 in the
/home/stack/images directory. For example:
cp overcloud-full.qcow2 /home/stack/images/ cp overcloud-full.initrd /home/stack/images/ cp overcloud-full.vmlinuz /home/stack/images/
See Preparing the Overcloud Image for more information about how to build the overcloud image. -
Upload the images.
cd /home/stack/images/ openstack overcloud image upload --update-existing
Note: This command must be executed in the /home/stack/images directory. All the images in the current directory will be uploaded. It takes about 30 minutes.An output example:/usr/lib/python3.6/site-packages/openstack/resource.py:351: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec() len(inspect.getargspec(type_).args) > 1)Image "overcloud-full-vmlinuz" was uploaded. +--------------------------------------+------------------------+-------------+----------+--------+ | ID | Name | Disk Format | Size | Status | +--------------------------------------+------------------------+-------------+----------+--------+ | 27b6a7c8-75a3-4018-ad1e-ab4692d2994c | overcloud-full-vmlinuz | aki | 10460296 | active | +--------------------------------------+------------------------+-------------+----------+--------+ Image "overcloud-full-initrd" was uploaded. +--------------------------------------+-----------------------+-------------+----------+--------+ | ID | Name | Disk Format | Size | Status | +--------------------------------------+-----------------------+-------------+----------+--------+ | 5925e0c6-da52-482e-9f8d-b58922e8a659 | overcloud-full-initrd | ari | 70253801 | active | +--------------------------------------+-----------------------+-------------+----------+--------+ Image "overcloud-full" was uploaded. +--------------------------------------+----------------+-------------+------------+--------+ | ID | Name | Disk Format | Size | Status | +--------------------------------------+----------------+-------------+------------+--------+ | 94518cc5-99f9-4fdf-9ed1-4ef3e21dfd0b | overcloud-full | raw | 5108203520 | active | +--------------------------------------+----------------+-------------+------------+--------+
Note: The existing overcloud images must be deleted to upload new overcloud images. Run the following command to check the uploaded overcloud image:openstack image list
An output example:... +--------------------------------------+------------------------+--------+ | ID | Name | Status | +--------------------------------------+------------------------+--------+ | db5e4e8e-cd2a-45a6-b39a-cbc4194294cf | overcloud-full | active | | 9dd68205-404f-47ba-aab3-8fb4927e707f | overcloud-full-initrd | active | | cc3336b7-a841-48b2-b768-a61282bf1547 | overcloud-full-vmlinuz | active | +--------------------------------------+------------------------+--------+
Delete the overcloud images.openstack image delete <image_id>
For example:openstack image delete adf7568a-33e2-40e1-9bbb-3ded500bf19e
After the existing images are deleted, copy the new images to the /home/stack/images directory and upload them. -
Import the nodes.json file.
The nodes.json file contains information about each node, such as the number of CPU cores, the CPU architecture type, the size of disk and available memory, the type of the remote control interface and related credentials.
- address: The MAC address of the network interface for the provisioning network
- pm_type: The type of the remote management interface
- name: The node name
- pm_user: The user name for IPMI or iDRAC
- pm_password: The password for IPMI or iDRAC
- pm_addr: The IP address of the iDRAC interface
cd /home/stack/templates openstack overcloud node import nodes.json
An output example:Successfully registered node UUID 87839a51-a554-4d2a-8c25-bf11fc180b13 Successfully registered node UUID a780e535-b7a3-4a8f-9ac1-ef13d346ae59 Successfully registered node UUID be7d21f0-8ed1-42b3-b9ac-47546b070c99
-
Update firstboot.yaml with the UUID of each node.
Run the following command to display the UUID of each node after the nodes.json file has been imported as described in step 6.
openstack baremetal node list
An output example:+--------------------------------------+-----------+---------------+-------------+---------------- | UUID | Name | Instance UUID | Power State | Provisioning... +--------------------------------------+-----------+---------------+-------------+---------------- | 87839a51-a554-4d2a-8c25-bf11fc180b13 | control | None | power on | manageable ... | a780e535-b7a3-4a8f-9ac1-ef13d346ae59 | compute-0 | None | power on | manageable ... | be7d21f0-8ed1-42b3-b9ac-47546b070c99 | compute-1 | None | power on | manageable ... +--------------------------------------+-----------+---------------+-------------+----------------
-
Set the force_persistent_boot_device parameter in the
driver-info to Never on all nodes.
Note: This step applies to Dell servers only. An iDRAC error may occur if this is not done. See the Red Hat official document at Troubleshooting the bare metal service.
openstack baremetal node set --driver-info \ force_persistent_boot_device=Never <node_uuid>
where node_uuid is the UUID of each node. For example:openstack baremetal node set --driver-info force_persistent_boot_device=Never \ 87839a51-a554-4d2a-8c25-bf11fc180b13 openstack baremetal node set --driver-info force_persistent_boot_device=Never \ a780e535-b7a3-4a8f-9ac1-ef13d346ae59 openstack baremetal node set --driver-info force_persistent_boot_device=Never \ be7d21f0-8ed1-42b3-b9ac-47546b070c99
-
Configure all nodes in manageable state for the deployment.
openstack overcloud node configure --all-manageable
An output example:Successfully configured the nodes.
-
Introspect all nodes in manageable state and make the nodes available for deployment
once the introspection is finished. It takes about 45 minutes.
openstack overcloud node introspect --provide --all-manageable
An output example:... PLAY RECAP ********************************************************************* localhost : ok=7 changed=4 unreachable=0 failed=0 skipped=2 rescued=0 ignored=0 2022-04-25 14:37:20.754819 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Summary Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2022-04-25 14:37:20.755615 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Total Tasks: 9 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2022-04-25 14:37:20.756408 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Elapsed Time: 0:00:40.356479 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2022-04-25 14:37:20.757233 | UUID | Info | Host | Task Name | Run Time 2022-04-25 14:37:20.758072 | a0b3cce9-60a2-7964-dd06-000000000013 | SUMMARY | localhost | Wait for nova resources | 17.81s 2022-04-25 14:37:20.758858 | a0b3cce9-60a2-7964-dd06-000000000016 | SUMMARY | localhost | poll for completion | 6.39s 2022-04-25 14:37:20.759697 | a0b3cce9-60a2-7964-dd06-000000000012 | SUMMARY | localhost | Run cell_v2 host discovery | 5.63s 2022-04-25 14:37:20.760470 | a0b3cce9-60a2-7964-dd06-00000000000f | SUMMARY | localhost | Make nodes available | 4.24s 2022-04-25 14:37:20.761323 | a0b3cce9-60a2-7964-dd06-000000000010 | SUMMARY | localhost | Detect nova service endpoint | 4.14s 2022-04-25 14:37:20.762147 | a0b3cce9-60a2-7964-dd06-000000000015 | SUMMARY | localhost | Power off nodes | 1.82s 2022-04-25 14:37:20.762927 | a0b3cce9-60a2-7964-dd06-000000000008 | SUMMARY | localhost | Check for required inputs | 0.06s 2022-04-25 14:37:20.763737 | a0b3cce9-60a2-7964-dd06-00000000000c | SUMMARY | localhost | Notice | 0.05s 2022-04-25 14:37:20.764619 | a0b3cce9-60a2-7964-dd06-00000000000a | SUMMARY | localhost | Set node_uuids_provide fact | 0.04s 2022-04-25 14:37:20.765381 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ End Summary Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Successfully provided nodes: ['87839a51-a554-4d2a-8c25-bf11fc180b13', 'a780e535-b7a3-4a8f-9ac1-ef13d346ae59', 'be7d21f0-8ed1-42b3-b9ac-47546b070c99']
-
Set appropriate parameters for deployment.
openstack baremetal node set control --property capabilities=profile:baremetal openstack baremetal node set compute-1 --property capabilities=profile:baremetal openstack baremetal node set compute-0 --property capabilities=profile:baremetal openstack baremetal node set control --resource-class baremetal openstack baremetal node set compute-1 --resource-class baremetal openstack baremetal node set compute-0 --resource-class baremetal openstack baremetal node set compute-1 --property capabilities=node:computedpdk openstack baremetal node set compute-0 --property capabilities=node:computedpdk openstack baremetal node set control --property capabilities=node:controller openstack subnet set --dns-nameserver 8.8.8.8 ctlplane-subnet
Note: The node names are defined in the nodes.json file. -
Run overcloud deployment.
cd /home/stack/templates openstack overcloud deploy --templates -r roles_data.yaml -n network_data.yaml \ -e /usr/share/openstack-tripleo-heat-templates/ci/environments/network/multiple-nics/network-environment.yaml \ -e /usr/share/openstack-tripleo-heat-templates/ci/environments/network/multiple-nics/network-isolation.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/services/neutron-ovs-dpdk.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/services/neutron-ovs.yaml \ -e scheduler_hints_env.yaml -e firstboot.yaml -e custom-network-configuration.yaml --verbose --debug \ -e containers-prepare-parameters.yaml
An output example after successful deployment.... PLAY RECAP ********************************************************************* overcloud-computeovsdpdk-0 : ok=520 changed=211 unreachable=0 failed=0 skipped=196 rescued=0 ignored=0 overcloud-computeovsdpdk-1 : ok=506 changed=211 unreachable=0 failed=0 skipped=196 rescued=0 ignored=0 overcloud-controller-0 : ok=592 changed=267 unreachable=0 failed=0 skipped=218 rescued=0 ignored=0 undercloud : ok=80 changed=29 unreachable=0 failed=0 skipped=9 rescued=0 ignored=0 2022-04-26 12:40:13.892265 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Summary Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2022-04-26 12:40:13.893272 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Total Tasks: 1736 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2022-04-26 12:40:13.894124 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Elapsed Time: 0:32:07.502815 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2022-04-26 12:40:13.895018 | UUID | Info | Host | Task Name | Run Time 2022-04-26 12:40:13.896034 | a0b3cce9-60a2-9b39-37ce-0000000029c2 | SUMMARY | overcloud-controller-0 | Pre-fetch all the containers | 348.50s 2022-04-26 12:40:13.896959 | a0b3cce9-60a2-9b39-37ce-0000000029f5 | SUMMARY | overcloud-computeovsdpdk-1 | Pre-fetch all the containers | 303.65s 2022-04-26 12:40:13.897839 | a0b3cce9-60a2-9b39-37ce-000000002a20 | SUMMARY | overcloud-computeovsdpdk-0 | Pre-fetch all the containers | 297.23s ... Host 192.168.24.15 not found in /home/stack/.ssh/known_hosts Overcloud Endpoint: http://192.168.24.15:5000 Overcloud Horizon Dashboard URL: http://192.168.24.15:80/dashboard Overcloud rc file: /home/stack/overcloudrc Overcloud Deployed without error 2022-04-26 12:41:10.312 293470 INFO osc_lib.shell [-] END return value: None
Run the following command to view the deployment result.openstack stack list
An output example:+--------------------------------------+------------+----------------------------------+-----------------... | ID | Stack Name | Project | Stack Status ... +--------------------------------------+------------+----------------------------------+-----------------... | 7ffaea8a-41df-42d7-a939-6953a1e83b6d | overcloud | 19c194354320469c8e3054d2dfe50849 | CREATE_COMPLETE ... +--------------------------------------+------------+----------------------------------+-----------------...
Note: When you need to redeploy overcloud, run the following command to delete any existing overcloud.openstack stack delete overcloud
It takes a few minutes. Check whether overcloud has been deleted before redeployment.openstack stack list
If the Stack Status is DELETE_FAILED, run the openstack stack delete overcloud command again. -
List the overcloud servers.
openstack server list
An output example:... +--------------------------------------+----------------------------+--------+------------------------... | ID | Name | Status | Networks ... +--------------------------------------+----------------------------+--------+------------------------... | 393058b5-8820-42d3-9265-c79b344a5280 | overcloud-controller-0 | ACTIVE | ctlplane=192.168.24.6 ... | 488ff29d-afcf-4dab-b232-1ffc89d44de0 | overcloud-computeovsdpdk-1 | ACTIVE | ctlplane=192.168.24.20 ... | 9e662061-7a6a-4003-80f6-1ed262a300ef | overcloud-computeovsdpdk-0 | ACTIVE | ctlplane=192.168.24.7 ... +--------------------------------------+----------------------------+--------+------------------------...
-
Access the compute nodes for configuration after overcloud deployment. The changes must
be made on each compute node.
For example:
ssh root@192.168.24.7
-
Edit the nova.conf file.
vi /var/lib/config-data/puppet-generated/nova_libvirt/etc/nova/nova.conf
Add filters to the [filter_scheduler] section to set the filters on the compute nodes.[filter_scheduler] available_filters = nova.scheduler.filters.all_filters enabled_filters = AvailabilityZoneFilter, ComputeFilter, ComputeCapabilitiesFilter, ImagePropertiesFilter, ServerGroupAntiAffinityFilter, ServerGroupAffinityFilter, SameHostFilter, DifferentHostFilter, PciPassthroughFilter
Note: The values for the available_filters and enabled_filters parameters must be added as one line.Specify the PCI bus IDs of the Napatech SmartNIC virtual function interfaces in the [pci] section. For the VXLAN setup:[pci] passthrough_whitelist = { "address": "<xxxx:xx:xx.xx>"} passthrough_whitelist = { "address": "<xxxx:xx:xx.xx>"}where:
xxxx:xx:xx.xx is a PCI ID of the Napatech SmartNIC virtual function interfaces. Run the following command to check PCI IDs:lspci -Dd 18f4:
Use the devices xxxx:xx:xx.05 and higher.
A VXLAN example:[pci] passthrough_whitelist = { "address": "0000:65:00.05" } passthrough_whitelist = { "address": "0000:65:00.06" } passthrough_whitelist = { "address": "0000:65:00.07" } passthrough_whitelist = { "address": "0000:65:01.00" } passthrough_whitelist = { "address": "0000:65:01.01" } passthrough_whitelist = { "address": "0000:65:01.02" } passthrough_whitelist = { "address": "0000:65:01.03" } passthrough_whitelist = { "address": "0000:65:01.04" } passthrough_whitelist = { "address": "0000:65:01.05" }A VLAN example:[pci] passthrough_whitelist = { "address": "0000:65:00.05" , "physical_network": "public"} passthrough_whitelist = { "address": "0000:65:00.06" , "physical_network": "public"} passthrough_whitelist = { "address": "0000:65:00.07" , "physical_network": "public"} passthrough_whitelist = { "address": "0000:65:01.00" , "physical_network": "public"} passthrough_whitelist = { "address": "0000:65:01.01" , "physical_network": "public"} passthrough_whitelist = { "address": "0000:65:01.02" , "physical_network": "public"} passthrough_whitelist = { "address": "0000:65:01.03" , "physical_network": "public"} passthrough_whitelist = { "address": "0000:65:01.04" , "physical_network": "public"} passthrough_whitelist = { "address": "0000:65:01.05" , "physical_network": "public"} -
Update QEMU access rights.
vi /var/lib/config-data/puppet-generated/nova_libvirt/etc/libvirt/qemu.conf
Add these lines to the file.user = "root" group = "root"
Restart nova services.systemctl restart tripleo_nova*
Check the status.systemctl status tripleo_nova*
-
Disable SELinux to be able create the Napatech OVS socket.
setenforce 0
Check the status of SELinux.sestatus
An output example:SELinux status: enabled SELinuxfs mount: /sys/fs/selinux SELinux root directory: /etc/selinux Loaded policy name: targeted Current mode: permissive Mode from config file: enforcing Policy MLS status: enabled Policy deny_unknown status: allowed Memory protection checking: actual (secure) Max kernel policy version: 33
Current mode is set to permissive. -
Close the connection to the compute node.
exit
Note: Repeat from step 14 on each compute node.