Context

For one of our customers, we had to work on an OpenStack cloud infrastructure with a certain amount of specifities. We are here talking about the Queens version, latest stable version to date.

The goal of this platform is to provide Compute instances with direct access to specific hardware (using PCI Passthrough). On the network side, creation and management of networks inside the cloud is not an identified need, therefore we made architecture choices to aim at the simplest, for example not deploying Neutron L3 agent and only using directly an external network connected to the compute node. All this with a number of hosts reduced to the minimum at the beginning, 1 controller and 1 compute node.

This apparent simplicity needs to be put in perspective with a not so minor constraint: the two hosts (controller and compute) are not connected on the same L2 network. Traffic is routed (and is actually transported through VPN).

The deployment tool we use here, OpenStack-Ansible, allows us to quite easily address these constraints: not deploying some components and connecting the two hosts through a routed network. There is still one issue to address though.

Question of providing IP addresses to instances

The usual mechanism used to provide IP addresses to instances is DHCP. The Neutron DHCP agent is in charge of spawning and configuring a DHCP server (dnsmasq by default) to do the job. Given the network constraints discussed previously, this solution is not easily usable as DHCP works inside an L2 network and cannot cross a router.

Thus we decided not to use DHCP and didn't deploy the Neutron DHCP agent. However, without connectivity, instances have no way to communicate with the metadata API (http://169.254.169.254), which raises a new question. That's why we also decided not to use this service: we disabled the metadata API exposed by Nova and didn't deploy the Neutron metadata agent which is usually in charge of proxying traffic between instances and this API.

So we had two issues to address and find solutions for:

• Provide IP addresses to instances
• Expose metadata to instances

Config-drive solution

The config-drive feature provided by Nova, the Compute service, addresses these two issues.

Config-drive replaces the metadata API in the sense that it embeds the informations usually exposed by this API in a disk ("config drive") that is itself connected to the instance. The cloud-init tool which is found in most of the cloud images is able to access this disk and use the data it contains.

Moreover, by enabling the appropriate option, Nova can provide network informations of the instance through metadata. These informations will also be used by cloud-init in order to configure the instance.

The force_config_drive option in nova.conf enables the config-drive feature by default without requiring the user to explicitly request it when creating an instance through the Compute API.

Then it's just about enabling the flat_injected option so that Nova generates and includes network informations.

Result

Inside an instance connected to a network without DHCP, we mount the config-drive and note that we can find the network configuration there, and we then note that this configuration has indeed been implemented by cloud-init in order to provide network connectivity:

root@myinstance:~# mkdir /mnt/config-drive
root@myinstance:~# mount -oro /dev/sr0 /mnt/config-drive/
root@myinstance:~# cd /mnt/config-drive/openstack/latest/
root@myinstance:/mnt/config-drive/openstack/latest# cat network_data.json
{"services": [], "networks": [{"network_id": "affc4c87-f1d1-4605-afda-3829c4a4cbc9", "type": "ipv4", "services": [], "netmask": "255.255.255.0", "link": "tap2498eebd-35", "routes": [{"netmask": "0.0.0.0", "network": "0.0.0.0", "gateway": "10.42.42.1"}], "ip_address": "10.42.42.6", "id": "network0"}], "links": [{"ethernet_mac_address": "fa:16:3e:62:7c:09", "mtu": 1500, "type": "bridge", "id": "tap2498eebd-35", "vif_id": "2498eebd-3593-49b9-938f-b0380930586b"}]}
root@myinstance:/mnt/config-drive/openstack/latest# cat /etc/network/interfaces.d/50-cloud-init.cfg
# This file is generated from information provided by
# the datasource.  Changes to it will not persist across an instance.
# To disable cloud-init's network configuration capabilities, write a file
# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
# network: {config: disabled}
auto lo
iface lo inet loopback

auto ens3
iface ens3 inet static
    address 10.42.42.6/24
    mtu 1500
    post-up route add default gw 10.42.42.1 || true
    pre-down route del default gw 10.42.42.1 || true

Conclusion

Not using DHCP inside the OpenStack Compute service is actually quite easy thanks to the config-drive feature.

This is only one of the solutions implemented in order to address the different specific constraints of this OpenStack deployment, and let's note again that all of this was technically feasible directly with OpenStack-Ansible, without requiring the use of additional external tools.