Table Of Contents

Previous topic

Virtualization

Next topic

Authentication and Authorization

This Page

Psst... hey. You're reading the latest content, but it might be out of sync with code. You can read Nova 2011.2 docs or all OpenStack docs too.

Networking

Todo

  • document hardware specific commands (maybe in admin guide?) (todd)
  • document a map between flags and managers/backends (todd)

The nova.network.manager Module

Network Hosts are responsible for allocating ips and setting up network.

There are multiple backend drivers that handle specific types of networking topologies. All of the network commands are issued to a subclass of NetworkManager.

Related Flags

network_driver:Driver to use for network creation
flat_network_bridge:
 Bridge device for simple network instances
flat_interface:FlatDhcp will bridge into this interface if set
flat_network_dns:
 Dns for simple network
vlan_start:First VLAN for private networks
vpn_ip:Public IP for the cloudpipe VPN servers
vpn_start:First Vpn port for private networks
cnt_vpn_clients:
 Number of addresses reserved for vpn clients
network_size:Number of addresses in each private subnet
floating_range:Floating IP address block
fixed_range:Fixed IP address block
fixed_ip_disassociate_timeout:
 Seconds after which a deallocated ip is disassociated
create_unique_mac_address_attempts:
 Number of times to attempt creating a unique mac address
exception AddressAlreadyAllocated(message=None, **kwargs)

Bases: nova.exception.NovaException

Address was already allocated.

class FlatDHCPManager(network_driver=None, *args, **kwargs)

Bases: nova.network.manager.RPCAllocateFixedIP, nova.network.manager.FloatingIP, nova.network.manager.NetworkManager

Flat networking with dhcp.

FlatDHCPManager will start up one dhcp server to give out addresses. It never injects network settings into the guest. It also manages bridges. Otherwise it behaves like FlatManager.

FlatDHCPManager.DHCP = True
FlatDHCPManager.SHOULD_CREATE_BRIDGE = True
FlatDHCPManager.init_host()

Do any initialization that needs to be run if this is a standalone service.

class FlatManager(network_driver=None, *args, **kwargs)

Bases: nova.network.manager.NetworkManager

Basic network where no vlans are used.

FlatManager does not do any bridge or vlan creation. The user is responsible for setting up whatever bridges are specified when creating networks through nova-manage. This bridge needs to be created on all compute hosts.

The idea is to create a single network for the host with a command like: nova-manage network create 192.168.0.0/24 1 256. Creating multiple networks for for one manager is currently not supported, but could be added by modifying allocate_fixed_ip and get_network to get the a network with new logic instead of network_get_by_bridge. Arbitrary lists of addresses in a single network can be accomplished with manual db editing.

If flat_injected is True, the compute host will attempt to inject network config into the guest. It attempts to modify /etc/network/interfaces and currently only works on debian based systems. To support a wider range of OSes, some other method may need to be devised to let the guest know which ip it should be using so that it can configure itself. Perhaps an attached disk or serial device with configuration info.

Metadata forwarding must be handled by the gateway, and since nova does not do any setup in this mode, it must be done manually. Requests to 169.254.169.254 port 80 will need to be forwarded to the api server.

FlatManager.deallocate_fixed_ip(context, address, **kwargs)

Returns a fixed ip to the pool.

FlatManager.get_floating_ip(context, *args, **kwargs)

Returns a floating IP as a dict

FlatManager.get_floating_ip_by_address(context, *args, **kwargs)

Returns a floating IP as a dict

FlatManager.get_floating_ips_by_fixed_address(context, *args, **kwargs)

Returns the floating IPs associated with a fixed_address

FlatManager.get_floating_ips_by_project(context, *args, **kwargs)

Returns the floating IPs allocated to a project

FlatManager.get_floating_pools(context, *args, **kwargs)

Returns list of floating pools

FlatManager.timeout_fixed_ips = False
class FloatingIP

Bases: object

Mixin class for adding floating IP functionality to a manager.

FloatingIP.add_dns_entry(context, *args, **kwargs)
FloatingIP.allocate_floating_ip(context, *args, **kwargs)

Gets a floating ip from the pool.

FloatingIP.allocate_for_instance(context, *args, **kwargs)

Handles allocating the floating IP resources for an instance.

calls super class allocate_for_instance() as well

rpc.called by network_api

FloatingIP.associate_floating_ip(context, *args, **kwargs)

Associates a floating ip with a fixed ip.

Makes sure everything makes sense then calls _associate_floating_ip, rpc’ing to correct host if i’m not it.

FloatingIP.create_private_dns_domain(context, *args, **kwargs)
FloatingIP.create_public_dns_domain(context, *args, **kwargs)
FloatingIP.deallocate_floating_ip(context, *args, **kwargs)

Returns an floating ip to the pool.

FloatingIP.deallocate_for_instance(context, *args, **kwargs)

Handles deallocating floating IP resources for an instance.

calls super class deallocate_for_instance() as well.

rpc.called by network_api

FloatingIP.delete_dns_domain(context, *args, **kwargs)
FloatingIP.delete_dns_entry(context, *args, **kwargs)
FloatingIP.disassociate_floating_ip(context, *args, **kwargs)

Disassociates a floating ip from its fixed ip.

Makes sure everything makes sense then calls _disassociate_floating_ip, rpc’ing to correct host if i’m not it.

FloatingIP.get_dns_domains(context, *args, **kwargs)
FloatingIP.get_dns_entries_by_address(context, *args, **kwargs)
FloatingIP.get_dns_entries_by_name(context, *args, **kwargs)
FloatingIP.get_floating_ip(context, *args, **kwargs)

Returns a floating IP as a dict

FloatingIP.get_floating_ip_by_address(context, *args, **kwargs)

Returns a floating IP as a dict

FloatingIP.get_floating_ips_by_fixed_address(context, *args, **kwargs)

Returns the floating IPs associated with a fixed_address

FloatingIP.get_floating_ips_by_project(context, *args, **kwargs)

Returns the floating IPs allocated to a project

FloatingIP.get_floating_pools(context, *args, **kwargs)

Returns list of floating pools

FloatingIP.init_host_floating_ips()

Configures floating ips owned by host.

FloatingIP.modify_dns_entry(context, *args, **kwargs)
class NetworkManager(network_driver=None, *args, **kwargs)

Bases: nova.manager.SchedulerDependentManager

Implements common network manager functionality.

This class must be subclassed to support specific topologies.

host management:
hosts configure themselves for networks they are assigned to in the table upon startup. If there are networks in the table which do not have hosts, those will be filled in and have hosts configured as the hosts pick them up one at time during their periodic task. The one at a time part is to flatten the layout to help scale
NetworkManager.DHCP = False
NetworkManager.SHOULD_CREATE_BRIDGE = False
NetworkManager.SHOULD_CREATE_VLAN = False
NetworkManager.add_fixed_ip_to_instance(context, *args, **kwargs)

Adds a fixed ip to an instance from specified network.

NetworkManager.add_virtual_interface(context, instance_id, network_id)
NetworkManager.allocate_fixed_ip(context, instance_id, network, **kwargs)

Gets a fixed ip from the pool.

NetworkManager.allocate_for_instance(context, *args, **kwargs)

Handles allocating the various network resources for an instance.

rpc.called by network_api

NetworkManager.build_network_info_model(context, vifs, networks, rxtx_factor, instance_host)

Builds a NetworkInfo object containing all network information for an instance

NetworkManager.create_networks(context, label, cidr, multi_host, num_networks, network_size, cidr_v6, gateway, gateway_v6, bridge, bridge_interface, dns1=None, dns2=None, fixed_cidr=None, **kwargs)

Create networks based on parameters.

NetworkManager.deallocate_fixed_ip(context, address, **kwargs)

Returns a fixed ip to the pool.

NetworkManager.deallocate_for_instance(context, *args, **kwargs)

Handles deallocating various network resources for an instance.

rpc.called by network_api kwargs can contain fixed_ips to circumvent another db lookup

NetworkManager.delete_network(context, *args, **kwargs)
NetworkManager.disassociate_network(context, *args, **kwargs)
NetworkManager.get_all_networks(context, *args, **kwargs)
NetworkManager.get_dhcp_leases(ctxt, network_ref)

Broker the request to the driver to fetch the dhcp leases

NetworkManager.get_fixed_ip(context, *args, **kwargs)

Return a fixed ip

NetworkManager.get_fixed_ip_by_address(context, *args, **kwargs)
NetworkManager.get_floating_ips_by_fixed_address(context, fixed_address)
NetworkManager.get_instance_id_by_floating_address(context, address)

Returns the instance id a floating ip’s fixed ip is allocated to

NetworkManager.get_instance_nw_info(context, *args, **kwargs)

Creates network info list for instance.

called by allocate_for_instance and network_api context needs to be elevated :returns: network info list [(network,info),(network,info)...] where network = dict containing pertinent data from a network db object and info = dict containing pertinent networking data

NetworkManager.get_instance_uuids_by_ip_filter(context, *args, **kwargs)
NetworkManager.get_network(context, *args, **kwargs)
NetworkManager.get_vif_by_mac_address(context, mac_address)

Returns the vifs record for the mac_address

NetworkManager.get_vifs_by_instance(context, *args, **kwargs)

Returns the vifs associated with an instance

NetworkManager.init_host()

Do any initialization that needs to be run if this is a standalone service.

NetworkManager.lease_fixed_ip(context, address)

Called by dhcp-bridge when ip is leased.

NetworkManager.release_fixed_ip(context, address)

Called by dhcp-bridge when ip is released.

NetworkManager.remove_fixed_ip_from_instance(context, *args, **kwargs)

Removes a fixed ip from an instance from specified network.

NetworkManager.rpc_setup_network_on_host(context, network_id, teardown)
NetworkManager.set_network_host(context, network_ref)

Safely sets the host of the network.

NetworkManager.setup_networks_on_host(context, instance_id, host, teardown=False)

calls setup/teardown on network hosts associated with an instance

NetworkManager.timeout_fixed_ips = True
NetworkManager.validate_networks(context, *args, **kwargs)

check if the networks exists and host is set to each network.

class RPCAllocateFixedIP

Bases: object

Mixin class originally for FlatDCHP and VLAN network managers.

used since they share code to RPC.call allocate_fixed_ip on the correct network host to configure dnsmasq

RPCAllocateFixedIP.deallocate_fixed_ip(context, address, host, **kwargs)

Call the superclass deallocate_fixed_ip if i’m the correct host otherwise call to the correct host

class VlanManager(network_driver=None, *args, **kwargs)

Bases: nova.network.manager.RPCAllocateFixedIP, nova.network.manager.FloatingIP, nova.network.manager.NetworkManager

Vlan network with dhcp.

VlanManager is the most complicated. It will create a host-managed vlan for each project. Each project gets its own subnet. The networks and associated subnets are created with nova-manage using a command like: nova-manage network create 10.0.0.0/8 3 16. This will create 3 networks of 16 addresses from the beginning of the 10.0.0.0 range.

A dhcp server is run for each subnet, so each project will have its own. For this mode to be useful, each project will need a vpn to access the instances in its subnet.

VlanManager.DHCP = True
VlanManager.SHOULD_CREATE_BRIDGE = True
VlanManager.SHOULD_CREATE_VLAN = True
VlanManager.add_network_to_project(context, *args, **kwargs)

Force adds another network to a project.

VlanManager.allocate_fixed_ip(context, instance_id, network, **kwargs)

Gets a fixed ip from the pool.

VlanManager.create_networks(context, **kwargs)

Create networks based on parameters.

VlanManager.init_host()

Do any initialization that needs to be run if this is a standalone service.

check_policy(context, action)
wrap_check_policy(func)

Check policy corresponding to the wrapped methods prior to execution

The nova.network.linux_net Driver

Implements vlans, bridges, and iptables rules using linux utilities.

class IptablesManager(execute=None)

Bases: object

Wrapper for iptables.

See IptablesTable for some usage docs

A number of chains are set up to begin with.

First, nova-filter-top. It’s added at the top of FORWARD and OUTPUT. Its name is not wrapped, so it’s shared between the various nova workers. It’s intended for rules that need to live at the top of the FORWARD and OUTPUT chains. It’s in both the ipv4 and ipv6 set of tables.

For ipv4 and ipv6, the built-in INPUT, OUTPUT, and FORWARD filter chains are wrapped, meaning that the “real” INPUT chain has a rule that jumps to the wrapped INPUT chain, etc. Additionally, there’s a wrapped chain named “local” which is jumped to from nova-filter-top.

For ipv4, the built-in PREROUTING, OUTPUT, and POSTROUTING nat chains are wrapped in the same was as the built-in filter chains. Additionally, there’s a snat chain that is applied after the POSTROUTING chain.

IptablesManager.apply(*args, **kwargs)

Apply the current in-memory set of iptables rules.

This will blow away any rules left over from previous runs of the same component of Nova, and replace them with our current set of rules. This happens atomically, thanks to iptables-restore.

class IptablesRule(chain, rule, wrap=True, top=False)

Bases: object

An iptables rule.

You shouldn’t need to use this class directly, it’s only used by IptablesManager.

class IptablesTable

Bases: object

An iptables table.

IptablesTable.add_chain(name, wrap=True)

Adds a named chain to the table.

The chain name is wrapped to be unique for the component creating it, so different components of Nova can safely create identically named chains without interfering with one another.

At the moment, its wrapped name is <binary name>-<chain name>, so if nova-compute creates a chain named ‘OUTPUT’, it’ll actually end up named ‘nova-compute-OUTPUT’.

IptablesTable.add_rule(chain, rule, wrap=True, top=False)

Add a rule to the table.

This is just like what you’d feed to iptables, just without the ‘-A <chain name>’ bit at the start.

However, if you need to jump to one of your wrapped chains, prepend its name with a ‘$’ which will ensure the wrapping is applied correctly.

IptablesTable.empty_chain(chain, wrap=True)

Remove all rules from a chain.

IptablesTable.remove_chain(name, wrap=True)

Remove named chain.

This removal “cascades”. All rule in the chain are removed, as are all rules in other chains that jump to it.

If the chain is not found, this is merely logged.

IptablesTable.remove_rule(chain, rule, wrap=True, top=False)

Remove a rule from a chain.

Note: The rule must be exactly identical to the one that was added. You cannot switch arguments around like you can with the iptables CLI tool.

class LinuxBridgeInterfaceDriver

Bases: nova.network.linux_net.LinuxNetInterfaceDriver

classmethod LinuxBridgeInterfaceDriver.ensure_bridge(*args, **kwargs)

Create a bridge unless it already exists.

Parameters:
  • interface – the interface to create the bridge on.
  • net_attrs – dictionary with attributes used to create bridge.

If net_attrs is set, it will add the net_attrs[‘gateway’] to the bridge using net_attrs[‘broadcast’] and net_attrs[‘cidr’]. It will also add the ip_v6 address specified in net_attrs[‘cidr_v6’] if use_ipv6 is set.

The code will attempt to move any ips that already exist on the interface onto the bridge and reset the default gateway if necessary.

classmethod LinuxBridgeInterfaceDriver.ensure_vlan(*args, **kwargs)

Create a vlan unless it already exists.

classmethod LinuxBridgeInterfaceDriver.ensure_vlan_bridge(_self, vlan_num, bridge, bridge_interface, net_attrs=None, mac_address=None)

Create a vlan and bridge unless they already exist.

LinuxBridgeInterfaceDriver.get_dev(network)
LinuxBridgeInterfaceDriver.plug(network, mac_address, gateway=True)
LinuxBridgeInterfaceDriver.unplug(network)
class LinuxNetInterfaceDriver

Bases: object

Abstract class that defines generic network host API

LinuxNetInterfaceDriver.get_dev(network)

Get device name

LinuxNetInterfaceDriver.plug(network, mac_address)

Create Linux device, return device name

LinuxNetInterfaceDriver.unplug(network)

Destory Linux device, return device name

class LinuxOVSInterfaceDriver

Bases: nova.network.linux_net.LinuxNetInterfaceDriver

LinuxOVSInterfaceDriver.get_dev(network)
LinuxOVSInterfaceDriver.plug(network, mac_address, gateway=True)
LinuxOVSInterfaceDriver.unplug(network)
class QuantumLinuxBridgeInterfaceDriver

Bases: nova.network.linux_net.LinuxNetInterfaceDriver

QuantumLinuxBridgeInterfaceDriver.BRIDGE_NAME_PREFIX = 'brq'
QuantumLinuxBridgeInterfaceDriver.GATEWAY_INTERFACE_PREFIX = 'gw-'
classmethod QuantumLinuxBridgeInterfaceDriver.create_tap_dev(_self, dev, mac_address=None)
QuantumLinuxBridgeInterfaceDriver.get_bridge(network)
QuantumLinuxBridgeInterfaceDriver.get_dev(network)
QuantumLinuxBridgeInterfaceDriver.plug(network, mac_address, gateway=True)
QuantumLinuxBridgeInterfaceDriver.unplug(network)
add_snat_rule(ip_range)
bind_floating_ip(floating_ip, device)

Bind ip to public interface.

ensure_floating_forward(floating_ip, fixed_ip)

Ensure floating ip forwarding rule.

ensure_metadata_ip()

Sets up local metadata ip.

ensure_path(path)
ensure_vpn_forward(public_ip, port, private_ip)

Sets up forwarding rules for vlan.

floating_forward_rules(floating_ip, fixed_ip)
get_binary_name()

Grab the name of the binary we’re running in.

get_dev(network)
get_dhcp_hosts(context, network_ref)

Get network’s hosts config in dhcp-host format.

get_dhcp_leases(context, network_ref)

Return a network’s hosts config in dnsmasq leasefile format.

get_dhcp_opts(context, network_ref)

Get network’s hosts config in dhcp-opts format.

init_host(ip_range=None)

Basic networking setup goes here.

initialize_gateway_device(dev, network_ref)
kill_dhcp(dev)
metadata_accept()

Create the filter accept rule for metadata.

metadata_forward()

Create forwarding rule for metadata.

plug(network, mac_address, gateway=True)
release_dhcp(dev, address, mac_address)
remove_floating_forward(floating_ip, fixed_ip)

Remove forwarding for floating ip.

restart_dhcp(*args, **kwargs)

(Re)starts a dnsmasq server for a given network.

If a dnsmasq instance is already running then send a HUP signal causing it to reload, otherwise spawn a new instance.

unbind_floating_ip(floating_ip, device)

Unbind a public ip from public interface.

unplug(network)
update_dhcp(context, dev, network_ref)
update_dhcp_hostfile_with_text(dev, hosts_text)
update_ra(*args, **kwargs)
write_to_file(file, data, mode='w')

Tests

The network_unittest Module

Legacy docs

The nova networking components manage private networks, public IP addressing, VPN connectivity, and firewall rules.

Components

There are several key components:

  • NetworkController (Manages address and vlan allocation)
  • RoutingNode (NATs public IPs to private IPs, and enforces firewall rules)
  • AddressingNode (runs DHCP services for private networks)
  • BridgingNode (a subclass of the basic nova ComputeNode)
  • TunnelingNode (provides VPN connectivity)

Component Diagram

Overview:

                               (PUBLIC INTERNET)
                                |              \
                               / \             / \
                 [RoutingNode] ... [RN]    [TunnelingNode] ... [TN]
                       |             \    /       |              |
                       |            < AMQP >      |              |
[AddressingNode]--  (VLAN) ...         |        (VLAN)...    (VLAN)      --- [AddressingNode]
                       \               |           \           /
                      / \             / \         / \         / \
                       [BridgingNode] ...          [BridgingNode]


                 [NetworkController]   ...    [NetworkController]
                                   \          /
                                     < AMQP >
                                        |
                                       / \
                      [CloudController]...[CloudController]

While this diagram may not make this entirely clear, nodes and controllers communicate exclusively across the message bus (AMQP, currently).

State Model

Network State consists of the following facts:

  • VLAN assignment (to a project)
  • Private Subnet assignment (to a security group) in a VLAN
  • Private IP assignments (to running instances)
  • Public IP allocations (to a project)
  • Public IP associations (to a private IP / running instance)

While copies of this state exist in many places (expressed in IPTables rule chains, DHCP hosts files, etc), the controllers rely only on the distributed “fact engine” for state, queried over RPC (currently AMQP). The NetworkController inserts most records into this datastore (allocating addresses, etc) - however, individual nodes update state e.g. when running instances crash.

The Public Traffic Path

Public Traffic:

               (PUBLIC INTERNET)
                      |
                    <NAT>  <-- [RoutingNode]
                      |
[AddressingNode] -->  |
                   ( VLAN )
                      |    <-- [BridgingNode]
                      |
               <RUNNING INSTANCE>

The RoutingNode is currently implemented using IPTables rules, which implement both NATing of public IP addresses, and the appropriate firewall chains. We are also looking at using Netomata / Clusto to manage NATting within a switch or router, and/or to manage firewall rules within a hardware firewall appliance.

Similarly, the AddressingNode currently manages running DNSMasq instances for DHCP services. However, we could run an internal DHCP server (using Scapy ala Clusto), or even switch to static addressing by inserting the private address into the disk image the same way we insert the SSH keys. (See compute for more details).