From sacadmin Thu Mar 15 12:10:50 2007 Received: from stickit.sfbay.sun.com (stickit.SFBay.Sun.COM [129.146.226.107]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id l2FJAoDf016762; Thu, 15 Mar 2007 12:10:50 -0700 (PDT) Received: from stickit.sfbay.sun.com (localhost [127.0.0.1]) by stickit.sfbay.sun.com (8.13.3+Sun/8.13.1) with ESMTP id l2FIxtca012816; Thu, 15 Mar 2007 10:59:55 -0800 (PST) Received: (from jhd@localhost) by stickit.sfbay.sun.com (8.13.3+Sun/8.13.1/Submit) id l2FIxtuZ012808; Thu, 15 Mar 2007 10:59:55 -0800 (PST) Date: Thu, 15 Mar 2007 10:59:55 -0800 (PST) From: John Danielson Message-Id: <200703151859.l2FIxtuZ012808@stickit.sfbay.sun.com> To: PSARC@sac.sfbay.sun.com Cc: matrix-eng@sun.com Subject: IPv4 Network Configuration Enhancements for Xen Guest Domains [PSARC/2007/155 Timeout: 03/22/2007] Status: RO Content-Length: 15363 Subject: PSARC FastTrack [03/22/2007]: IPv4 Network Configuration Enhancements for Xen Guest Domains Template Version: @(#)sac_nextcase %I% %G% SMI This information is Copyright 2007 Sun Microsystems, Inc. 1. Introduction 1.1. Project/Component Working Name: IPv4 Network Configuration Enhancements for Xen Guest Domains 1.2. Name of Document Author/Supplier: Author: David Edmondson 1.3 Date of This Document: 15 March, 2007 4. Technical Description I am sponsoring the following fast track for David Edmondson with a timeout set for March 22, 2007. The case is requesting a Patch release binding. A copy of the devprop man page has been deposited the case directory. - john ========= IPv4 Network Configuration Enhancements for Xen Guest Domains ============================================================= Author: David Edmondson Date: 2007/03/15 13:50:28 $Id: xen-ip-config.org,v 1.5 2007/03/14 17:55:48 dme Exp $ Table of Contents ================= 1 Introduction 2 Configuration Options 2.1 Specifying Configuration Options 2.2 Kernel Command-line Arguments 2.2.1 Examples 3 Accessing the Configuration Data in a Guest Domain 4 Non-Diskless Systems 4.1 Interaction with sysidtool(1M) 5 Diskless Systems 5.1 Ordering of Configuration Sources 6 Oustanding Issues 6.1 Multiple Network Interfaces 7 /sbin/devprop 8 Interaction with other projects 9 Interface Table 10 Further Work 1 Introduction ############## The Xen hypervisor and the port of Solaris to it are described in PSARC/2006/260. That case is considered required pre-reading for this. The IPv4 configuration attributes for a network interface in Solaris are traditionally managed in three ways: - Static configuration files in /etc used by the /network/physical SMF service (this includes DHCP for diskfull systems), - DHCP sourced information for diskless systems, typically obtained by firmware and provided to the bootloader and Solaris kernel during boot, - RARP sourced information for diskless systems, typically obtained by the Solaris kernel during boot. The first and last of these mechanisms work for guest domains today. The second mechanism is specific to systems booting on hardware and hence not currently relevant to Solaris on Xen. As part of the configuration details for a guest domain it is possible and common to specify a limited set of IPv4 attributes. These details are passed to the kernel booting in the guest domain as command line arguments. The set of attributes supported is defined by the pre-existing hypervisor configuration tools. This proposal describes how the current Solaris IPv4 configuration mechanisms will be enhanced to observe and, as appropriate, use the configuration details provided to the kernel. 2 Configuration Options ####################### 2.1 Specifying Configuration Options ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Guest domain configuration files are fragments of Python[1] code. Certain variable names are "well known" to the configuration infrastructure and can be set to control the creation of the guest domain. In this context the configuration variables of concern are: | Variable | Type | Meaning | Examples | |------------+--------+-----------------------------------+-----------------------------| | ip | string | IPv4 address of the domain | "10.6.39.40" | | netmask | string | netmask of the domain | "255.255.255.0" | | gateway | string | default router for the domain | "10.6.39.1" | | dhcp | string | is DHCP enabled? | "on" "off" | | vif | list | network interfaces specifications | see below | | nfs_root | string | pathname of NFS root filesystem | "pushrod:/export/root/xpv1" | | nfs_server | string | IPv4 address of the NFS server | "10.6.39.80" | This set of variables are defined by the implementation of the hypervisor control tools, which are External. The "vif" specification controls how many[2] inter-domain network interfaces are available in the guest domain and some configuration information for those network interfaces. The "vif" specification takes the form of a Python list with an element for each network interface available to the guest domain. For example: : vif = [ 'mac="aa:0:1:2:3:4"' ] indicates that a single network interface is provided to the domain (as the list contains a single element) and the MAC address of that interface is "aa:0:1:2:3:4". Similarly: : vif = [ '', '' ] indicates two network interfaces provided to the domain. Both of these interfaces have automatically generated MAC addresses. The "vif" specification allows other configuration items per-network interface that are not relevant to this case. The hypervisor control tools use the configuration variables during the creation of the guest domain and also construct two command line arguments that are passed to the booted kernel. 2.2 Kernel Command-line Arguments ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The configuration details specified during domain creation are used to construct two command line arguments for the kernel being booted in the guest domain. Each of the arguments takes the form of name, the equals sign ("=") and a colon separated list of settings. The two parameters of relevance are "ip" and "nfsroot". Each is followed by a fixed number of settings, seven for "ip" and three for "nfsroot". The meaning of each setting is defined by its' position. For "ip": | Setting | Meaning | |---------+---------------------------------------------------| | 1 | IPv4 address | | 2 | NFS server (ignored in favor of "nfsroot" below) | | 3 | default router | | 4 | netmask | | 5 | hostname | | 6 | network interface name | | 7 | host configuration protocol | For "nfsroot": | Setting | Meaning | |---------+------------------------------------| | 1 | NFS server IPv4 address | | 2 | NFS server name | | 3 | root filesystem path on NFS server | If no information is provided for a setting it is simply left blank. 2.2.1 Examples ~~~~~~~~~~~~~ : ip=10.6.39.40:10.6.39.80::255.255.255.0::eth0:off | Setting | Value | |-----------------------------+---------------| | IPv4 address | 10.6.39.40 | | NFS server | 10.6.39.80 | | default router | none | | netmask | 255.255.255.0 | | hostname | none | | network interface name | eth0 | | host configuration protocol | off | : nfsroot=10.6.39.80:pushrod:/export/root/xpv1 | Setting | Value | |-------------------------+-------------------| | NFS server IPv4 address | 10.6.39.80 | | NFS server name | pushrod | | root filesystem path | /export/root/xpv1 | 3 Accessing the Configuration Data in a Guest Domain #################################################### During early startup the Solaris kernel creates boot properties for all command line arguments provided by the hypervisor guest domain builder. Each command line argument has "xpv-" prefixed to generate a boot property name. Hence the following system properties will be created by the example above (output extracted from "prtconf -v"): : name='xpv-ip' type=string items=1 : value='10.6.39.40:10.6.39.80::255.255.255.0::eth0:off' : name='xpv-nfsroot' type=string items=1 : value='10.6.39.80:pushrod:/export/root/xpv1' The "xpv-ip" and "xpv-nfsroot" properties are further decomposed and used to set other system properties. Where possible system properties already used by Solaris are set to an appropriate value: | System property | Meaning | Existing property? | |-------------------+-----------------------------+--------------------| | host-ip | IPv4 address | yes | | router-ip | default router | yes | | subnet-mask | netmask | yes | | network-interface | network interface name | yes[3] | | xpv-hcp | host configuration protocol | no | | fstype | root filesystem type | yes | | server-ip | NFS server IPv4 address | yes | | server-name | NFS server name | yes | | server-path | root filesystem path | yes | All of these system properties can be accessed by the normal property lookup routines (see ddi_prop_lookup(9F)). A new utility is provided to allow the examination of system properties from within shell scripts (/sbin/devprop). 4 Non-Diskless Systems ###################### The configuration of network interfaces in non-diskless systems is typically the responsibility of the /network/physical service. This service has been enhanced such that in the absence of any of the existing configuration mechanisms (e.g. files in /etc) the system properties generated from the guest domain configuration are examined. If the "network-interface" property is set then the "xpv-hcp" property is examined. Three possible values for "xpv-hcp" are understood: "dhcp" - DHCPv4 based configuration, "off" - static IP address configuration, empty - static IP address configuration. A setting of "dhcp" results in the specified network interface being plumbed and, presuming that this is successful, the DHCP agent is started to manage the interface. For the "off" and empty settings three further system properties are examined - host-ip, subnet-mask and router-ip. If host-ip and subnet-mask are present then the specified network interface is plumbed, the IP address and netmask are set and the interface is marked "up". If router-ip is also set then a default route via that host is added to the system. 4.1 Interaction with sysidtool(1M) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ When a freshly installed guest domain is first booted the sysidtool(1M) framework will offer to store configuration data for any network interfaces available to the guest domain. Any IPv4 configuration attributes entered here will prevent the use of attributes specified in the guest domain configuration files during any subsequent boot of the guest domain (as the contents of files in /etc take precedence over the attributes specified in the guest domain configuration file). 5 Diskless Systems ################## Diskless systems rely on kernel-based configuration of a network interface to allow them to reach the NFS server which provides their root filesystem. As such it is not possible to rely on changes to the /network/physical service. The diskless root mechanism is enhanced to examine system properties in a manner similar to that described for non-diskless systems with the following differences: - the "xpv-hcp" property is ignored and no DHCP based mechanism is available - the address must be statically configured, - all of the following properties _must_ be specified: host-ip, subnet-mask, router-ip, server-path, server-name, server-ip Providing all of the required properties are present, they are used to configure the network interface and mount the root filesystem from the NFS server. 5.1 Ordering of Configuration Sources ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The current diskless boot mechanism prefers DHCP parameters over RARP. This project will cause the system to prefer system properties over DHCP parameters. 6 Oustanding Issues ################### 6.1 Multiple Network Interfaces ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The guest domain configuration file allows an administrator to indicate that multiple network interfaces should be provided to the guest domain but only allows the configuration details for the first interface to be declared. 7 /sbin/devprop ############### : Usage: devprop [-n device-path] [-vq] [-{b|i|l|s}] [property [...]] devprop provides easy access to properties from the device tree for administrators and scripting language developers (most commonly shell scripts). devprop will be delivered on all Solaris platforms. 8 Interaction with other projects ################################# The Network Auto-Magic (NWAM) project[4] aims to: "...simplify and automate network configuration on Solaris." The current description[5] of NWAM suggests that the information provided in the hypervisor guest domain configuration would be input to, or perhaps an automatically created, link-layer profile. The details of this interaction should be discussed further with the NWAM community. 9 Interface Table ################# | Interface | Stability | Comments | |-----------------------------------------------------------+-------------+-----------------------------| | /sbin/devprop | Committed | location | | devprop [-n device-path] [-vq] [-{bils}] [property [...]] | Volatile | CLI options, output | | kernel command line arguments | Uncommitted | Hypervisor tools may change | | system properties beginning "xpv-" | Uncommitted | Hypervisor tools may change | 10 Further Work ############### Future projects may include: - Enhancing the hypervisor control tools (specifically the domain builder) to perform DHCP based attribute discovery (see RFE 6528791). - Improving the interaction between the hypervisor control tools and other components that may use a DHCP server in the control domain to manage local virtual machines. - Enhancing the hypervisor control tools and associated mechanism to support the specification of IPv6 attributes. None of the possible future projects are included in this proposal, and it is believed that the proposal places no restriction on their possible implementation at a future date. Footnotes: [1] http://www.python.org [2] Earlier versions of hypervisor tools supported a "nics" variable which set the number of network interfaces available to the guest domain. This use of this parameter is discouraged and it will cease to work in a future revision of the tools. [3] The hypervisor tools have a default network interface name of "eth0". This string, if seen, is translated into "xnf0" as a usability improvement. A message is output reporting that the translation has taken place. [4] http://www.opensolaris.org/os/project/nwam/ [5] http://www.opensolaris.org/os/project/nwam/architecture/ 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: FastTrack From sacadmin Thu Mar 15 13:32:57 2007 Received: from phorcys.east.sun.com (phorcys.East.Sun.COM [129.148.174.143]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id l2FKWves018272 for ; Thu, 15 Mar 2007 13:32:57 -0700 (PDT) Received: from phorcys.east.sun.com (localhost [127.0.0.1]) by phorcys.east.sun.com (8.14.0+Sun/8.14.0) with ESMTP id l2FKWuCt028344; Thu, 15 Mar 2007 16:32:56 -0400 (EDT) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.0+Sun/8.14.0/Submit) id l2FKWuTR028341; Thu, 15 Mar 2007 16:32:56 -0400 (EDT) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <17913.44408.450869.94836@gargle.gargle.HOWL> Date: Thu, 15 Mar 2007 16:32:56 -0400 From: James Carlson To: John Danielson Cc: PSARC@sac.sfbay.sun.com, matrix-eng@sun.com Subject: Re: IPv4 Network Configuration Enhancements for Xen Guest Domains [PSARC/2007/155 Timeout: 03/22/2007] In-Reply-To: <200703151859.l2FIxtuZ012808@stickit.sfbay.sun.com> References: <200703151859.l2FIxtuZ012808@stickit.sfbay.sun.com> X-Mailer: VM 7.01 under Emacs 21.3.1 Status: RO Content-Length: 929 John Danielson writes: > The current description[5] of NWAM suggests that the information > provided in the hypervisor guest domain configuration would be input > to, or perhaps an automatically created, link-layer profile. > > The details of this interaction should be discussed further with the > NWAM community. Has this discussion started? > [3] The hypervisor tools have a default network interface name of "eth0". > This string, if seen, is translated into "xnf0" as a usability > improvement. A message is output reporting that the translation > has taken place. heh. Shouldn't this be an open case? Xen, the system kernel, and IP configuration mechanisms are all open bits. -- James Carlson, Solaris Networking Sun Microsystems / 1 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 From sacadmin Fri Mar 16 00:50:16 2007 Received: from cheesecake.hh.sledj.net (vpn-129-150-120-92.UK.Sun.COM [129.150.120.92]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id l2G7oEG4027788 for ; Fri, 16 Mar 2007 00:50:15 -0700 (PDT) Received: from cheesecake.hh.sledj.net (localhost [127.0.0.1]) by cheesecake.hh.sledj.net (8.13.6+Sun/8.13.6) with ESMTP id l2G7qbpw028321; Fri, 16 Mar 2007 07:52:37 GMT Received: (from dme@localhost) by cheesecake.hh.sledj.net (8.13.6+Sun/8.13.6/Submit) id l2G7qaMQ028320; Fri, 16 Mar 2007 07:52:36 GMT X-Authentication-Warning: cheesecake.hh.sledj.net: dme set sender to dme@sun.com using -f Date: Fri, 16 Mar 2007 07:52:36 +0000 From: David Edmondson To: James Carlson Cc: John Danielson , PSARC@sac.sfbay.sun.com, matrix-eng@sun.com Subject: Re: IPv4 Network Configuration Enhancements for Xen Guest Domains [PSARC/2007/155 Timeout: 03/22/2007] Message-ID: <20070316075236.GP26801@cheesecake.uk.sun.com> References: <200703151859.l2FIxtuZ012808@stickit.sfbay.sun.com> <17913.44408.450869.94836@gargle.gargle.HOWL> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <17913.44408.450869.94836@gargle.gargle.HOWL> User-Agent: Mutt/1.5.7i Status: RO Content-Length: 978 On Thu, Mar 15, 2007 at 04:32:56PM -0400, James Carlson wrote: > John Danielson writes: > > The current description[5] of NWAM suggests that the information > > provided in the hypervisor guest domain configuration would be input > > to, or perhaps an automatically created, link-layer profile. > > > > The details of this interaction should be discussed further with the > > NWAM community. > > Has this discussion started? An earlier version of the proposal (no substantive differences) was copied to the nwam-discuss alias. I saw no comments from that direction. I'll prod. > > [3] The hypervisor tools have a default network interface name of "eth0". > > This string, if seen, is translated into "xnf0" as a usability > > improvement. A message is output reporting that the translation > > has taken place. > > heh. Indeed. > Shouldn't this be an open case? Xen, the system kernel, and IP > configuration mechanisms are all open bits. Yes. dme. From sacadmin Fri Mar 16 06:28:59 2007 Received: from phorcys.east.sun.com (phorcys.East.Sun.COM [129.148.174.143]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id l2GDSwbp002717 for ; Fri, 16 Mar 2007 06:28:59 -0700 (PDT) Received: from phorcys.east.sun.com (localhost [127.0.0.1]) by phorcys.east.sun.com (8.14.0+Sun/8.14.0) with ESMTP id l2GDSwua002759; Fri, 16 Mar 2007 09:28:58 -0400 (EDT) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.0+Sun/8.14.0/Submit) id l2GDSwGF002756; Fri, 16 Mar 2007 09:28:58 -0400 (EDT) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <17914.39834.289128.459469@gargle.gargle.HOWL> Date: Fri, 16 Mar 2007 09:28:58 -0400 From: James Carlson To: David Edmondson Cc: John Danielson , PSARC@sac.sfbay.sun.com, matrix-eng@sun.com Subject: Re: IPv4 Network Configuration Enhancements for Xen Guest Domains [PSARC/2007/155 Timeout: 03/22/2007] In-Reply-To: <20070316075236.GP26801@cheesecake.uk.sun.com> References: <200703151859.l2FIxtuZ012808@stickit.sfbay.sun.com> <17913.44408.450869.94836@gargle.gargle.HOWL> <20070316075236.GP26801@cheesecake.uk.sun.com> X-Mailer: VM 7.01 under Emacs 21.3.1 Status: RO Content-Length: 1193 David Edmondson writes: > On Thu, Mar 15, 2007 at 04:32:56PM -0400, James Carlson wrote: > > Has this discussion started? > > An earlier version of the proposal (no substantive differences) was > copied to the nwam-discuss alias. I saw no comments from that > direction. I'll prod. OK; thanks. I don't have any serious concerns about it (the direction you're going makes some sense to me, though I might quibble with grot like "the default router"), and I'm just making sure the loop is closed. Note that NWAM is under terrific pressure to deliver in the next few days, so you may need the cattle option on that prod. > > Shouldn't this be an open case? Xen, the system kernel, and IP > > configuration mechanisms are all open bits. > > Yes. Could you have your case owner clean this up? (Mark it as open in the IAM* file, make sure there aren't any stray "confidential" marks on any of the files, and resend the announcement to psarc-ext instead of psarc.) -- James Carlson, Solaris Networking Sun Microsystems / 1 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 From jhd@stickit.sfbay.sun.com Fri Mar 16 10:00:06 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id l2GH05Ag007536 for ; Fri, 16 Mar 2007 10:00:05 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail5.uk.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l2GH04Jh010847; Fri, 16 Mar 2007 17:00:04 GMT Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JF00090B9W1WC00@nwk-avmta-1.sfbay.Sun.COM>; Fri, 16 Mar 2007 10:00:01 -0700 (PDT) Received: from sfbaymail1sca.SFBay.Sun.COM ([129.145.154.35]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JF0009KL9VZ6010@nwk-avmta-1.sfbay.Sun.COM>; Fri, 16 Mar 2007 09:59:59 -0700 (PDT) Received: from stickit.sfbay.sun.com (stickit.SFBay.Sun.COM [129.146.226.107]) by sfbaymail1sca.SFBay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l2GGxxf0012999; Fri, 16 Mar 2007 09:59:59 -0700 (PDT) Received: from stickit (stickit [129.146.226.107]) by stickit.sfbay.sun.com (8.13.3+Sun/8.13.1) with SMTP id l2GGn47R013651; Fri, 16 Mar 2007 08:49:04 -0800 (PST) Date: Fri, 16 Mar 2007 08:49:04 -0800 (PST) From: John Danielson Subject: IPv4 Network Configuration Enhancements for Xen Guest Domains [PSARC/2007/155 Timeout: 03/22/2007] To: psarc-ext@sun.com Cc: matrix-eng@sun.com Reply-to: John Danielson Message-id: <200703161649.l2GGn47R013651@stickit.sfbay.sun.com> MIME-version: 1.0 X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.6 SunOS 5.10 sun4u sparc Content-type: TEXT/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-MD5: MF9ubylQPAdRH/87clY3FA== X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 15404 I am resubmitting this fast track for David Edmondson as an open case. The timeout for the case is now March 23, 2007. The case requests a Patch release binding. A copy of the devprop.1M man page has been deposited the case directory. - john ========= Subject: PSARC FastTrack [03/22/2007]: IPv4 Network Configuration Enhancements for Xen Guest Domains Template Version: @(#)sac_nextcase %I% %G% SMI This information is Copyright 2007 Sun Microsystems, Inc. 1. Introduction 1.1. Project/Component Working Name: IPv4 Network Configuration Enhancements for Xen Guest Domains 1.2. Name of Document Author/Supplier: Author: David Edmondson 1.3 Date of This Document: 15 March, 2007 4. Technical Description IPv4 Network Configuration Enhancements for Xen Guest Domains ============================================================= Author: David Edmondson Date: 2007/03/15 13:50:28 $Id: xen-ip-config.org,v 1.5 2007/03/14 17:55:48 dme Exp $ Table of Contents ================= 1 Introduction 2 Configuration Options 2.1 Specifying Configuration Options 2.2 Kernel Command-line Arguments 2.2.1 Examples 3 Accessing the Configuration Data in a Guest Domain 4 Non-Diskless Systems 4.1 Interaction with sysidtool(1M) 5 Diskless Systems 5.1 Ordering of Configuration Sources 6 Oustanding Issues 6.1 Multiple Network Interfaces 7 /sbin/devprop 8 Interaction with other projects 9 Interface Table 10 Further Work 1 Introduction ############## The Xen hypervisor and the port of Solaris to it are described in PSARC/2006/260. That case is considered required pre-reading for this. The IPv4 configuration attributes for a network interface in Solaris are traditionally managed in three ways: - Static configuration files in /etc used by the /network/physical SMF service (this includes DHCP for diskfull systems), - DHCP sourced information for diskless systems, typically obtained by firmware and provided to the bootloader and Solaris kernel during boot, - RARP sourced information for diskless systems, typically obtained by the Solaris kernel during boot. The first and last of these mechanisms work for guest domains today. The second mechanism is specific to systems booting on hardware and hence not currently relevant to Solaris on Xen. As part of the configuration details for a guest domain it is possible and common to specify a limited set of IPv4 attributes. These details are passed to the kernel booting in the guest domain as command line arguments. The set of attributes supported is defined by the pre-existing hypervisor configuration tools. This proposal describes how the current Solaris IPv4 configuration mechanisms will be enhanced to observe and, as appropriate, use the configuration details provided to the kernel. 2 Configuration Options ####################### 2.1 Specifying Configuration Options ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Guest domain configuration files are fragments of Python[1] code. Certain variable names are "well known" to the configuration infrastructure and can be set to control the creation of the guest domain. In this context the configuration variables of concern are: | Variable | Type | Meaning | Examples | |------------+--------+-----------------------------------+--------------------- --------| | ip | string | IPv4 address of the domain | "10.6.39.40" | | netmask | string | netmask of the domain | "255.255.255.0" | | gateway | string | default router for the domain | "10.6.39.1" | | dhcp | string | is DHCP enabled? | "on" "off" | | vif | list | network interfaces specifications | see below | | nfs_root | string | pathname of NFS root filesystem | "pushrod:/export/root/xpv1" | | nfs_server | string | IPv4 address of the NFS server | "10.6.39.80" | This set of variables are defined by the implementation of the hypervisor control tools, which are External. The "vif" specification controls how many[2] inter-domain network interfaces are available in the guest domain and some configuration information for those network interfaces. The "vif" specification takes the form of a Python list with an element for each network interface available to the guest domain. For example: : vif = [ 'mac="aa:0:1:2:3:4"' ] indicates that a single network interface is provided to the domain (as the list contains a single element) and the MAC address of that interface is "aa:0:1:2:3:4". Similarly: : vif = [ '', '' ] indicates two network interfaces provided to the domain. Both of these interfaces have automatically generated MAC addresses. The "vif" specification allows other configuration items per-network interface that are not relevant to this case. The hypervisor control tools use the configuration variables during the creation of the guest domain and also construct two command line arguments that are passed to the booted kernel. 2.2 Kernel Command-line Arguments ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The configuration details specified during domain creation are used to construct two command line arguments for the kernel being booted in the guest domain. Each of the arguments takes the form of name, the equals sign ("=") and a colon separated list of settings. The two parameters of relevance are "ip" and "nfsroot". Each is followed by a fixed number of settings, seven for "ip" and three for "nfsroot". The meaning of each setting is defined by its' position. For "ip": | Setting | Meaning | |---------+---------------------------------------------------| | 1 | IPv4 address | | 2 | NFS server (ignored in favor of "nfsroot" below) | | 3 | default router | | 4 | netmask | | 5 | hostname | | 6 | network interface name | | 7 | host configuration protocol | For "nfsroot": | Setting | Meaning | |---------+------------------------------------| | 1 | NFS server IPv4 address | | 2 | NFS server name | | 3 | root filesystem path on NFS server | If no information is provided for a setting it is simply left blank. 2.2.1 Examples ~~~~~~~~~~~~~ : ip=10.6.39.40:10.6.39.80::255.255.255.0::eth0:off | Setting | Value | |-----------------------------+---------------| | IPv4 address | 10.6.39.40 | | NFS server | 10.6.39.80 | | default router | none | | netmask | 255.255.255.0 | | hostname | none | | network interface name | eth0 | | host configuration protocol | off | : nfsroot=10.6.39.80:pushrod:/export/root/xpv1 | Setting | Value | |-------------------------+-------------------| | NFS server IPv4 address | 10.6.39.80 | | NFS server name | pushrod | | root filesystem path | /export/root/xpv1 | 3 Accessing the Configuration Data in a Guest Domain #################################################### During early startup the Solaris kernel creates boot properties for all command line arguments provided by the hypervisor guest domain builder. Each command line argument has "xpv-" prefixed to generate a boot property name. Hence the following system properties will be created by the example above (output extracted from "prtconf -v"): : name='xpv-ip' type=string items=1 : value='10.6.39.40:10.6.39.80::255.255.255.0::eth0:off' : name='xpv-nfsroot' type=string items=1 : value='10.6.39.80:pushrod:/export/root/xpv1' The "xpv-ip" and "xpv-nfsroot" properties are further decomposed and used to set other system properties. Where possible system properties already used by Solaris are set to an appropriate value: | System property | Meaning | Existing property? | |-------------------+-----------------------------+--------------------| | host-ip | IPv4 address | yes | | router-ip | default router | yes | | subnet-mask | netmask | yes | | network-interface | network interface name | yes[3] | | xpv-hcp | host configuration protocol | no | | fstype | root filesystem type | yes | | server-ip | NFS server IPv4 address | yes | | server-name | NFS server name | yes | | server-path | root filesystem path | yes | All of these system properties can be accessed by the normal property lookup routines (see ddi_prop_lookup(9F)). A new utility is provided to allow the examination of system properties from within shell scripts (/sbin/devprop). 4 Non-Diskless Systems ###################### The configuration of network interfaces in non-diskless systems is typically the responsibility of the /network/physical service. This service has been enhanced such that in the absence of any of the existing configuration mechanisms (e.g. files in /etc) the system properties generated from the guest domain configuration are examined. If the "network-interface" property is set then the "xpv-hcp" property is examined. Three possible values for "xpv-hcp" are understood: "dhcp" - DHCPv4 based configuration, "off" - static IP address configuration, empty - static IP address configuration. A setting of "dhcp" results in the specified network interface being plumbed and, presuming that this is successful, the DHCP agent is started to manage the interface. For the "off" and empty settings three further system properties are examined - host-ip, subnet-mask and router-ip. If host-ip and subnet-mask are present then the specified network interface is plumbed, the IP address and netmask are set and the interface is marked "up". If router-ip is also set then a default route via that host is added to the system. 4.1 Interaction with sysidtool(1M) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ When a freshly installed guest domain is first booted the sysidtool(1M) framework will offer to store configuration data for any network interfaces available to the guest domain. Any IPv4 configuration attributes entered here will prevent the use of attributes specified in the guest domain configuration files during any subsequent boot of the guest domain (as the contents of files in /etc take precedence over the attributes specified in the guest domain configuration file). 5 Diskless Systems ################## Diskless systems rely on kernel-based configuration of a network interface to allow them to reach the NFS server which provides their root filesystem. As such it is not possible to rely on changes to the /network/physical service. The diskless root mechanism is enhanced to examine system properties in a manner similar to that described for non-diskless systems with the following differences: - the "xpv-hcp" property is ignored and no DHCP based mechanism is available - the address must be statically configured, - all of the following properties _must_ be specified: host-ip, subnet-mask, router-ip, server-path, server-name, server-ip Providing all of the required properties are present, they are used to configure the network interface and mount the root filesystem from the NFS server. 5.1 Ordering of Configuration Sources ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The current diskless boot mechanism prefers DHCP parameters over RARP. This project will cause the system to prefer system properties over DHCP parameters. 6 Oustanding Issues ################### 6.1 Multiple Network Interfaces ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The guest domain configuration file allows an administrator to indicate that multiple network interfaces should be provided to the guest domain but only allows the configuration details for the first interface to be declared. 7 /sbin/devprop ############### : Usage: devprop [-n device-path] [-vq] [-{b|i|l|s}] [property [...]] devprop provides easy access to properties from the device tree for administrators and scripting language developers (most commonly shell scripts). devprop will be delivered on all Solaris platforms. 8 Interaction with other projects ################################# The Network Auto-Magic (NWAM) project[4] aims to: "...simplify and automate network configuration on Solaris." The current description[5] of NWAM suggests that the information provided in the hypervisor guest domain configuration would be input to, or perhaps an automatically created, link-layer profile. The details of this interaction should be discussed further with the NWAM community. 9 Interface Table ################# | Interface | Stability | Comments | |-----------------------------------------------------------+-------------+----- ------------------------| | /sbin/devprop | Committed | location | | devprop [-n device-path] [-vq] [-{bils}] [property [...]] | Volatile | CLI options, output | | kernel command line arguments | Uncommitted | Hypervisor tools may change | | system properties beginning "xpv-" | Uncommitted | Hypervisor tools may change | 10 Further Work ############### Future projects may include: - Enhancing the hypervisor control tools (specifically the domain builder) to perform DHCP based attribute discovery (see RFE 6528791). - Improving the interaction between the hypervisor control tools and other components that may use a DHCP server in the control domain to manage local virtual machines. - Enhancing the hypervisor control tools and associated mechanism to support the specification of IPv6 attributes. None of the possible future projects are included in this proposal, and it is believed that the proposal places no restriction on their possible implementation at a future date. Footnotes: [1] http://www.python.org [2] Earlier versions of hypervisor tools supported a "nics" variable which set the number of network interfaces available to the guest domain. This use of this parameter is discouraged and it will cease to work in a future revision of the tools. [3] The hypervisor tools have a default network interface name of "eth0". This string, if seen, is translated into "xnf0" as a usability improvement. A message is output reporting that the translation has taken place. [4] http://www.opensolaris.org/os/project/nwam/ [5] http://www.opensolaris.org/os/project/nwam/architecture/ 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: FastTrack . From erik.nordmark@sun.com Fri Mar 16 13:27:37 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id l2GKRakH011296 for ; Fri, 16 Mar 2007 13:27:37 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l2GKRQIL029000; Sat, 17 Mar 2007 04:27:33 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JF000C05JHW2Z00@brm-avmta-1.central.sun.com>; Fri, 16 Mar 2007 14:27:32 -0600 (MDT) Received: from jurassic.eng.sun.com ([129.146.17.55]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JF00041OJHVIU70@brm-avmta-1.central.sun.com>; Fri, 16 Mar 2007 14:27:31 -0600 (MDT) Received: from [192.9.61.11] (punchin-nordmark.SFBay.Sun.COM [192.9.61.11]) by jurassic.eng.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l2GKRSjs609056 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 16 Mar 2007 13:27:29 -0700 (PDT) Date: Fri, 16 Mar 2007 13:27:28 -0700 From: Erik Nordmark Subject: Re: IPv4 Network Configuration Enhancements for Xen Guest Domains [PSARC/2007/155 Timeout: 03/22/2007] In-reply-to: <200703161649.l2GGn47R013651@stickit.sfbay.sun.com> To: John Danielson Cc: psarc-ext@sun.com, matrix-eng@sun.com Message-id: <45FAFDB0.9010600@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200703161649.l2GGn47R013651@stickit.sfbay.sun.com> User-Agent: Thunderbird 1.5.0.5 (X11/20060911) Status: RO Content-Length: 1106 John Danielson wrote: > 4.1 Interaction with sysidtool(1M) > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > When a freshly installed guest domain is first booted the > sysidtool(1M) framework will offer to store configuration data for any > network interfaces available to the guest domain. Any IPv4 > configuration attributes entered here will prevent the use of > attributes specified in the guest domain configuration files during > any subsequent boot of the guest domain (as the contents of files in > /etc take precedence over the attributes specified in the guest domain > configuration file). I don't understand whether the above is advise to the users (i.e. something which will go in the documentation), or an indication that sysidtool needs to be changed to not accidentally override hypervisor parameters. > 10 Further Work > ############### [...] > - Enhancing the hypervisor control tools and associated mechanism to > support the specification of IPv6 attributes. Regarding IPv6 support, what is the reason for not doing it now? Has Xen not defined any IPv6 attributes yet? Erik From dme@sun.com Sat Mar 17 02:25:09 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id l2H9P9SE009349 for ; Sat, 17 Mar 2007 02:25:09 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail5.uk.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l2H9P1Tc002115; Sat, 17 Mar 2007 09:25:06 GMT Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JF100A07JHS2I00@nwk-avmta-2.sfbay.sun.com>; Sat, 17 Mar 2007 02:25:04 -0700 (PDT) Received: from cheesecake.hh.sledj.net ([129.150.120.92]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JF100ECSJHQLQ50@nwk-avmta-2.sfbay.sun.com>; Sat, 17 Mar 2007 02:25:04 -0700 (PDT) Received: from cheesecake.hh.sledj.net (localhost [127.0.0.1]) by cheesecake.hh.sledj.net (8.13.6+Sun/8.13.6) with ESMTP id l2H9ROWc028753; Sat, 17 Mar 2007 09:27:24 +0000 (GMT) Received: (from dme@localhost) by cheesecake.hh.sledj.net (8.13.6+Sun/8.13.6/Submit) id l2H9RNUG028752; Sat, 17 Mar 2007 09:27:23 +0000 (GMT) Date: Sat, 17 Mar 2007 09:27:23 +0000 From: David Edmondson Subject: Re: IPv4 Network Configuration Enhancements for Xen Guest Domains [PSARC/2007/155 Timeout: 03/22/2007] In-reply-to: <45FAFDB0.9010600@sun.com> To: Erik Nordmark Cc: John Danielson , psarc-ext@sun.com, matrix-eng@sun.com Message-id: <20070317092723.GY26801@cheesecake.uk.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <200703161649.l2GGn47R013651@stickit.sfbay.sun.com> <45FAFDB0.9010600@sun.com> X-Authentication-warning: cheesecake.hh.sledj.net: dme set sender to dme@sun.com using -f User-Agent: Mutt/1.5.7i Status: RO Content-Length: 1444 On Fri, Mar 16, 2007 at 01:27:28PM -0700, Erik Nordmark wrote: > John Danielson wrote: > > >4.1 Interaction with sysidtool(1M) > >^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > > >When a freshly installed guest domain is first booted the > >sysidtool(1M) framework will offer to store configuration data for any > >network interfaces available to the guest domain. Any IPv4 > >configuration attributes entered here will prevent the use of > >attributes specified in the guest domain configuration files during > >any subsequent boot of the guest domain (as the contents of files in > >/etc take precedence over the attributes specified in the guest domain > >configuration file). > > I don't understand whether the above is advise to the users (i.e. > something which will go in the documentation), or an indication that > sysidtool needs to be changed to not accidentally override > hypervisor parameters. It should be advice to users. Both cases (administrator sets parameters "inside" the guest domain and administrator allows "outside" parameters) are useful in different situations, I think. > >10 Further Work > >############### > [...] > >- Enhancing the hypervisor control tools and associated mechanism to > > support the specification of IPv6 attributes. > > Regarding IPv6 support, what is the reason for not doing it now? Has Xen > not defined any IPv6 attributes yet? The current Xen tools have no IPv6 attributes defined. dme. From sacadmin Wed Mar 28 09:46:23 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id l2SGkMc7025746 for ; Wed, 28 Mar 2007 09:46:22 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail5.uk.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l2SGkECG029869; Wed, 28 Mar 2007 17:46:19 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JFM00305H96UJ00@nwk-avmta-1.sfbay.Sun.COM>; Wed, 28 Mar 2007 09:46:18 -0700 (PDT) Received: from nwk-ea-fw-1.sun.com ([10.4.134.5]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JFM00IA1H963660@nwk-avmta-1.sfbay.Sun.COM>; Wed, 28 Mar 2007 09:46:18 -0700 (PDT) Received: from d1-sfbay-10.sun.com ([192.18.39.120]) by nwk-ea-fw-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l2SGkICX008436; Wed, 28 Mar 2007 08:46:18 -0800 (PST) Received: from conversion-daemon.d1-sfbay-10.sun.com by d1-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JFM00I01H5RRS00@d1-sfbay-10.sun.com> (original mail from Sherri.Shieh@Sun.COM); Wed, 28 Mar 2007 09:46:17 -0700 (PDT) Received: from [129.145.154.54] by d1-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JFM00MZ2H8WDB8Z@d1-sfbay-10.sun.com>; Wed, 28 Mar 2007 09:46:17 -0700 (PDT) Date: Wed, 28 Mar 2007 09:46:08 -0700 From: Sherri Shieh Subject: PSARC Fast track: IPv4 Network Configuration Enhancements for Xen Guest Domains (2007/155) Sender: Sherri.Shieh@sun.com To: psarc@sun.com, John Danielson , David Edmondson Message-id: <460A9BD0.9030802@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=us-ascii Content-transfer-encoding: 7BIT X-Accept-Language: en-us, en X-PMX-Version: 5.2.0.264296 User-Agent: Mozilla/5.0 (X11; U; SunOS sun4v; en-US; rv:1.7) Gecko/20060629 Status: RO Content-Length: 341 This fast track was approved last week. I've marked it closed approved. - Sherri -- ========================================================= Sherri Shieh Sun Microsystems, Inc. Program Manager Email: sherri.shieh@sun.com Systems Architecture Phone: 650-786-5245/x85245 =========================================================== From jhd@stickit.eng.sun.com Mon Oct 8 09:11:47 2007 Received: from stickit.Eng.Sun.COM (stickit [129.146.226.107]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l98GBlSA007513 for ; Mon, 8 Oct 2007 09:11:47 -0700 (PDT) Received: from stickit (stickit [129.146.226.107]) by stickit.Eng.Sun.COM (8.13.3+Sun/8.13.1) with SMTP id l98FslLN015391; Mon, 8 Oct 2007 08:54:48 -0700 (PDT) Message-Id: <200710081554.l98FslLN015391@stickit.Eng.Sun.COM> Date: Mon, 8 Oct 2007 08:54:48 -0700 (PDT) From: John Danielson Reply-To: John Danielson Subject: ammendment to PSARC/2007/155 IPv4 Network Configuration Enhancements for Xen Guest Domains To: psarc-ext@sac.sfbay.sun.com Cc: matrix-eng@sun.com MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: bJhraUv3gERON3lQIVA8Lg== X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.6 SunOS 5.10 sun4u sparc Status: RO Content-Length: 596 I am sponsoring the following ammendment for David Edmondson. - john The original PSARC states that a 'subnet-mask' property is required to be present before boot properties are using to configure a network interface (in both diskless and non-diskless cases). David Comay pointed out that Solaris has traditionally not required a netmask to be specified - the stack guesses one based on the class of the interface. The amendment is to remove the requirement that the 'subnet-mask' property be present. If it's there it will be used, otherwise the stack is allowed to calculated a default. From sommerfeld@sun.com Mon Oct 8 09:47:59 2007 Received: from dm-east-01.east.sun.com (dm-east-01.East.Sun.COM [129.148.9.192]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l98Glx8I008214 for ; Mon, 8 Oct 2007 09:47:59 -0700 (PDT) Received: from thunk.east.sun.com (thunk.East.Sun.COM [129.148.174.66]) by dm-east-01.east.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id l98GiiXm001358; Mon, 8 Oct 2007 12:44:44 -0400 (EDT) Received: from [IPv6:::1] (localhost [IPv6:::1]) by thunk.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id l98Giibt023752; Mon, 8 Oct 2007 12:44:44 -0400 (EDT) Subject: Re: ammendment to PSARC/2007/155 IPv4 Network Configuration Enhancements for Xen Guest Domains From: Bill Sommerfeld To: John Danielson Cc: psarc-ext@sac.sfbay.sun.com, matrix-eng@sun.com In-Reply-To: <200710081554.l98FslLN015391@stickit.Eng.Sun.COM> References: <200710081554.l98FslLN015391@stickit.Eng.Sun.COM> Content-Type: text/plain Date: Mon, 08 Oct 2007 12:44:43 -0400 Message-Id: <1191861883.23113.12.camel@thunk> Mime-Version: 1.0 X-Mailer: Evolution 2.10.2 Content-Transfer-Encoding: 7bit Status: RO Content-Length: 1070 On Mon, 2007-10-08 at 08:54 -0700, John Danielson wrote: > The original PSARC states that a 'subnet-mask' property is required to be > present before boot properties are using to configure a network > interface (in both diskless and non-diskless cases). David Comay > pointed out that Solaris has traditionally not required a netmask to > be specified - the stack guesses one based on the class of the > interface. With all due respect to Mr. Comay, just because something is traditional doesn't make it right. It was an error to separate configuration of ip address from configuration of netmask. That error leads to operational disruption. (most recently when installzone.sfbay was given the wrong netmask and was unreachable from large parts of the swan). All other devices I'm aware of require that netmask be supplied at the same time as the ip address. Anyone in a position to correctly assign an ip address to a network node will know the netmask and should be able to supply it. If I were not taking a sabbatical, i'd derail this case. - Bill From carlsonj@phorcys.east.sun.com Mon Oct 8 09:54:04 2007 Received: from phorcys.east.sun.com (phorcys.East.Sun.COM [129.148.174.143]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l98Gs4TB008338 for ; Mon, 8 Oct 2007 09:54:04 -0700 (PDT) Received: from phorcys.east.sun.com (localhost [127.0.0.1]) by phorcys.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id l98GoYSp002721; Mon, 8 Oct 2007 12:50:34 -0400 (EDT) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.1+Sun/8.14.1/Submit) id l98GoVh8002718; Mon, 8 Oct 2007 12:50:31 -0400 (EDT) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <18186.24535.401439.487240@gargle.gargle.HOWL> Date: Mon, 8 Oct 2007 12:50:31 -0400 From: James Carlson To: Bill Sommerfeld Cc: John Danielson , psarc-ext@sac.sfbay.sun.com, matrix-eng@sun.com Subject: Re: ammendment to PSARC/2007/155 IPv4 Network Configuration Enhancements for Xen Guest Domains In-Reply-To: <1191861883.23113.12.camel@thunk> References: <200710081554.l98FslLN015391@stickit.Eng.Sun.COM> <1191861883.23113.12.camel@thunk> X-Mailer: VM 7.01 under Emacs 21.3.1 Status: RO Content-Length: 1877 Bill Sommerfeld writes: > On Mon, 2007-10-08 at 08:54 -0700, John Danielson wrote: > > The original PSARC states that a 'subnet-mask' property is required to be > > present before boot properties are using to configure a network > > interface (in both diskless and non-diskless cases). David Comay > > pointed out that Solaris has traditionally not required a netmask to > > be specified - the stack guesses one based on the class of the > > interface. > > With all due respect to Mr. Comay, just because something is traditional > doesn't make it right. > > It was an error to separate configuration of ip address from > configuration of netmask. That error leads to operational disruption. > (most recently when installzone.sfbay was given the wrong netmask and > was unreachable from large parts of the swan). > > All other devices I'm aware of require that netmask be supplied at the > same time as the ip address. Anyone in a position to correctly assign > an ip address to a network node will know the netmask and should be able > to supply it. I agree with that, particularly for boot properties. It makes no sense there. You should have either a complete interface configuration, or a deferral to DHCP/BOOTP to do the right thing. (There's some existing head injury related to looking up netmasks for some interfaces via NIS -- of all things! -- using another interface previously configured, and thus omitting explicit netmasks there. This is just daft, and certainly does not work well, but we need to rope the area off more effectively [i.e., deliberately remove support from the system] before forcing more people down that road.) -- James Carlson, Solaris Networking Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 From David.Comay@Sun.COM Mon Oct 8 10:13:15 2007 Received: from izimbra.SFBay.Sun.COM (izimbra [129.146.226.141]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l98HDFRf009189 for ; Mon, 8 Oct 2007 10:13:15 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by izimbra.SFBay.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id l98H9xsJ010160; Mon, 8 Oct 2007 10:09:59 -0700 (PDT) Date: Mon, 8 Oct 2007 10:09:59 -0700 (PDT) From: David.Comay@Sun.COM Sender: comay@izimbra.sfbay.sun.com To: Bill Sommerfeld cc: John Danielson , psarc-ext@sac.sfbay.sun.com, matrix-eng@Sun.COM Subject: Re: ammendment to PSARC/2007/155 IPv4 Network Configuration Enhancements for Xen Guest Domains In-Reply-To: <1191861883.23113.12.camel@thunk> Message-ID: References: <200710081554.l98FslLN015391@stickit.Eng.Sun.COM> <1191861883.23113.12.camel@thunk> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Status: RO Content-Length: 988 >> The original PSARC states that a 'subnet-mask' property is required to be >> present before boot properties are using to configure a network >> interface (in both diskless and non-diskless cases). David Comay >> pointed out that Solaris has traditionally not required a netmask to >> be specified - the stack guesses one based on the class of the >> interface. > > With all due respect to Mr. Comay, just because something is traditional > doesn't make it right. Agreed and as I mentioned to Dave at the time of code review (where I noticed this), I'm OK requiring the netmask too. However, it's my understanding that requiring the netmask will mean that configuring guest domains using our xVM implementation will differ from that of vendors using similar technology. Perhaps that's not a big deal given there are other options supported by xm(1) that are specific to the underlying dom0 OS but I do worry a bit about such operational differences from the upstream community. dsc From dme@sun.com Mon Oct 8 13:38:53 2007 Received: from enoexec.uk.sun.com (enoexec.UK.Sun.COM [129.156.173.21]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l98Kcqvg014991 for ; Mon, 8 Oct 2007 13:38:53 -0700 (PDT) Received: from enoexec.uk.sun.com (localhost [127.0.0.1]) by enoexec.uk.sun.com (8.14.1+Sun/8.14.1) with ESMTP id l98KZZ15008838; Mon, 8 Oct 2007 21:35:35 +0100 (BST) Received: (from dme@localhost) by enoexec.uk.sun.com (8.14.1+Sun/8.14.1/Submit) id l98KZZ85008837; Mon, 8 Oct 2007 21:35:35 +0100 (BST) X-Authentication-Warning: enoexec.uk.sun.com: dme set sender to dme@sun.com using -f Date: Mon, 8 Oct 2007 21:35:35 +0100 From: David Edmondson To: John Danielson Cc: psarc-ext@sac.sfbay.sun.com, matrix-eng@sun.com Subject: Re: ammendment to PSARC/2007/155 IPv4 Network Configuration Enhancements for Xen Guest Domains Message-ID: <20071008203534.GB1326@enoexec.uk.sun.com> References: <200710081554.l98FslLN015391@stickit.Eng.Sun.COM> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200710081554.l98FslLN015391@stickit.Eng.Sun.COM> User-Agent: Mutt/1.5.16 (2007-06-09) Status: RO Content-Length: 205 On Mon, Oct 08, 2007 at 08:54:48AM -0700, John Danielson wrote: > I am sponsoring the following ammendment for David Edmondson. I'm happy to withdraw the proposed amendment if it is considered unsound. From sommerfeld@sun.com Mon Oct 8 14:43:16 2007 Received: from dm-east-02.east.sun.com (dm-east-02.East.Sun.COM [129.148.13.5]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l98LhGGL016994 for ; Mon, 8 Oct 2007 14:43:16 -0700 (PDT) Received: from thunk.east.sun.com (thunk.East.Sun.COM [129.148.174.66]) by dm-east-02.east.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id l98Le091054519; Mon, 8 Oct 2007 17:40:00 -0400 (EDT) Received: from [IPv6:::1] (localhost [IPv6:::1]) by thunk.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id l98LdxZ9025319; Mon, 8 Oct 2007 17:39:59 -0400 (EDT) Subject: Re: ammendment to PSARC/2007/155 IPv4 Network Configuration Enhancements for Xen Guest Domains From: Bill Sommerfeld To: David.Comay@sun.com Cc: John Danielson , psarc-ext@sac.sfbay.sun.com, matrix-eng@sun.com In-Reply-To: References: <200710081554.l98FslLN015391@stickit.Eng.Sun.COM> <1191861883.23113.12.camel@thunk> Content-Type: text/plain Date: Mon, 08 Oct 2007 17:39:59 -0400 Message-Id: <1191879599.23113.114.camel@thunk> Mime-Version: 1.0 X-Mailer: Evolution 2.10.2 Content-Transfer-Encoding: 7bit Status: RO Content-Length: 670 On Mon, 2007-10-08 at 10:09 -0700, David.Comay@sun.com wrote: > However, it's my > understanding that requiring the netmask will mean that configuring > guest domains using our xVM implementation will differ from that of > vendors using similar technology. I'm a little surprised by this. can you expand on this or explain further? I'd imagine that one very common case is that the guest domain is given an address on the same ip subnet as the host, in which case it would be possible to for the host to unambiguously determine the correct subnet mask for the guest based on the host's configuration. But the guest can't figure this out on its own. - Bill From dme@sun.com Mon Oct 8 22:57:46 2007 Received: from enoexec.uk.sun.com (enoexec.UK.Sun.COM [129.156.173.21]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l995vjFO026317 for ; Mon, 8 Oct 2007 22:57:45 -0700 (PDT) Received: from enoexec.uk.sun.com (localhost [127.0.0.1]) by enoexec.uk.sun.com (8.14.1+Sun/8.14.1) with ESMTP id l995sUrS007266; Tue, 9 Oct 2007 06:54:30 +0100 (BST) Received: (from dme@localhost) by enoexec.uk.sun.com (8.14.1+Sun/8.14.1/Submit) id l995sUJV007265; Tue, 9 Oct 2007 06:54:30 +0100 (BST) X-Authentication-Warning: enoexec.uk.sun.com: dme set sender to dme@sun.com using -f Date: Tue, 9 Oct 2007 06:54:30 +0100 From: David Edmondson To: Bill Sommerfeld Cc: David.Comay@sun.com, John Danielson , psarc-ext@sac.sfbay.sun.com, matrix-eng@sun.com Subject: Re: ammendment to PSARC/2007/155 IPv4 Network Configuration Enhancements for Xen Guest Domains Message-ID: <20071009055430.GC1326@enoexec.uk.sun.com> References: <200710081554.l98FslLN015391@stickit.Eng.Sun.COM> <1191861883.23113.12.camel@thunk> <1191879599.23113.114.camel@thunk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1191879599.23113.114.camel@thunk> User-Agent: Mutt/1.5.16 (2007-06-09) Status: RO Content-Length: 1513 On Mon, Oct 08, 2007 at 05:39:59PM -0400, Bill Sommerfeld wrote: > On Mon, 2007-10-08 at 10:09 -0700, David.Comay@sun.com wrote: > > However, it's my > > understanding that requiring the netmask will mean that configuring > > guest domains using our xVM implementation will differ from that of > > vendors using similar technology. > > I'm a little surprised by this. can you expand on this or explain > further? > > I'd imagine that one very common case is that the guest domain is given > an address on the same ip subnet as the host, in which case it would be > possible to for the host to unambiguously determine the correct subnet > mask for the guest based on the host's configuration. > > But the guest can't figure this out on its own. A Linux kernel running in a guest domain is happy to guess a netmask based on the class of the IP address assigned, so an administrator is not required to specify a netmask if that derived from the class of the address is correct. The existing tools (xm/xend, virsh) and their associated APIs do not require a netmask to be specified alongside the IP address of a network interface. So, if Solaris requires that a netmask is specified it will be different to the other common guest domain implementation. If we believe that being prepared to guess a netmask results in it being wrong more than right then it is sensible to require it. I'm inclined to believe that (which is why it was originally required), but don't have good evidence to back up the belief. From sommerfeld@sun.com Tue Oct 9 07:57:14 2007 Received: from localhost.east.sun.com (punchin-sommerfeld.East.Sun.COM [129.148.19.3]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l99EvDqh005677 for ; Tue, 9 Oct 2007 07:57:14 -0700 (PDT) Received: from localhost.east.sun.com (localhost [127.0.0.1]) by localhost.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id l99Eojji010270; Tue, 9 Oct 2007 10:50:45 -0400 (EDT) Received: (from sommerfeld@localhost) by localhost.east.sun.com (8.14.1+Sun/8.14.1/Submit) id l99EoiVM010269; Tue, 9 Oct 2007 10:50:44 -0400 (EDT) X-Authentication-Warning: localhost.east.sun.com: sommerfeld set sender to sommerfeld@sun.com using -f Subject: Re: ammendment to PSARC/2007/155 IPv4 Network Configuration Enhancements for Xen Guest Domains From: Bill Sommerfeld To: David Edmondson Cc: David.Comay@sun.com, John Danielson , psarc-ext@sac.sfbay.sun.com, matrix-eng@sun.com In-Reply-To: <20071009055430.GC1326@enoexec.uk.sun.com> References: <200710081554.l98FslLN015391@stickit.Eng.Sun.COM> <1191861883.23113.12.camel@thunk> <1191879599.23113.114.camel@thunk> <20071009055430.GC1326@enoexec.uk.sun.com> Content-Type: text/plain Content-Transfer-Encoding: 7bit Date: Tue, 09 Oct 2007 10:50:44 -0400 Message-Id: <1191941444.10194.12.camel@localhost> Mime-Version: 1.0 X-Mailer: Evolution 2.10.2 Status: RO Content-Length: 1258 On Tue, 2007-10-09 at 06:54 +0100, David Edmondson wrote: > The existing tools (xm/xend, virsh) and their associated APIs do not > require a netmask to be specified alongside the IP address of a > network interface. > > So, if Solaris requires that a netmask is specified it will be > different to the other common guest domain implementation. Maybe I'm confused about the case boundary. Are we talking about host behavior, solaris guest behavior, or both? I'd want to apply the Postel principle here (be conservative in what you send and liberal in what you accept). When we're the host we should always pass both address and netmask so that the guest doesn't have to guess. When we're the guest we may have to guess but we should be clear to the admin that we're guessing because the host screwed up in not including a netmask with the ip address. It's an administrative error, but we recover from the error by guessing a netmask and stumbling onward rather than halting or failing to configure the interface. If it turns out that we need compatibility at the admin interface, the way to go would be to have the host guess the netmask (based on the greater amount of information available to it) rather than having the guest do it. - Bill From dme@sun.com Tue Oct 9 08:06:37 2007 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02 [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l99F6b5f005801 for ; Tue, 9 Oct 2007 08:06:37 -0700 (PDT) Received: from gmp-eb-mail-2.sun.com (gmp-eb-mail-2.EU.Sun.COM [192.18.6.24]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id l99F3LkZ037762 for ; Tue, 9 Oct 2007 08:03:21 -0700 (PDT) Received: from fe-emea-09.sun.com (gmp-eb-lb-2-fe3.eu.sun.com [192.18.6.12]) by gmp-eb-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id l99F3FS7008153 for ; Tue, 9 Oct 2007 15:03:15 GMT Received: from conversion-daemon.fe-emea-09.sun.com by fe-emea-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JPN00001GBCRC00@fe-emea-09.sun.com> (original mail from dme@sun.com) for psarc-ext@sac.sfbay.sun.com; Tue, 09 Oct 2007 16:03:15 +0100 (BST) Received: from apfelstrudel.hh.sledj.net ([83.217.165.81]) by fe-emea-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JPN00B5ZGGNVZ50@fe-emea-09.sun.com>; Tue, 09 Oct 2007 16:02:49 +0100 (BST) Date: Tue, 09 Oct 2007 16:06:28 +0100 From: David Edmondson Subject: Re: ammendment to PSARC/2007/155 IPv4 Network Configuration Enhancements for Xen Guest Domains In-reply-to: <1191941444.10194.12.camel@localhost> Sender: David.Edmondson@sun.com To: Bill Sommerfeld Cc: David.Comay@sun.com, John Danielson , psarc-ext@sac.sfbay.sun.com, matrix-eng@sun.com Message-id: <20071009150628.GC19381@apfelstrudel.hh.sledj.net> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline References: <200710081554.l98FslLN015391@stickit.Eng.Sun.COM> <1191861883.23113.12.camel@thunk> <1191879599.23113.114.camel@thunk> <20071009055430.GC1326@enoexec.uk.sun.com> <1191941444.10194.12.camel@localhost> User-Agent: Mutt/1.5.16 (2007-06-09) Status: RO Content-Length: 1069 On Tue, Oct 09, 2007 at 10:50:44AM -0400, Bill Sommerfeld wrote: > On Tue, 2007-10-09 at 06:54 +0100, David Edmondson wrote: > > The existing tools (xm/xend, virsh) and their associated APIs do not > > require a netmask to be specified alongside the IP address of a > > network interface. > > > > So, if Solaris requires that a netmask is specified it will be > > different to the other common guest domain implementation. > > Maybe I'm confused about the case boundary. Are we talking about host > behavior, solaris guest behavior, or both? The case concerns the behaviour of Solaris as a guest. > When we're the guest we may have to guess but we should be clear to > the admin that we're guessing because the host screwed up in not > including a netmask with the ip address. It's an administrative > error, but we recover from the error by guessing a netmask and > stumbling onward rather than halting or failing to configure the > interface. If the consensus is that this is the correct approach, I'm happy to re-word the amendment and implement the changes.