From sebastien.roy@oracle.com Wed May 12 12:57:55 2010 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4CJvtsc013832 for ; Wed, 12 May 2010 12:57:55 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o4CJvtGQ010904 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 12 May 2010 12:57:55 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2B00901OSJLU00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 12 May 2010 12:57:55 -0700 (PDT) Received: from sca-ea-mail-2.sun.com ([192.18.43.25]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2B0001HOSIZUB0@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 12 May 2010 12:57:54 -0700 (PDT) Received: from rcsinet15.oracle.com (rcsinet15.oracle.com [148.87.113.117]) by sca-ea-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o4CJvrID015409 for ; Wed, 12 May 2010 19:57:54 +0000 (GMT) Received: from acsmt353.oracle.com (acsmt353.oracle.com [141.146.40.153]) by rcsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o4CH5D45011198 for ; Wed, 12 May 2010 19:57:52 +0000 (GMT) Received: from abhmt019.oracle.com by acsmt355.oracle.com with ESMTP id 236206941273694173; Wed, 12 May 2010 12:56:13 -0700 Received: from [129.148.174.103] (/129.148.174.103) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 12 May 2010 12:56:13 -0700 Date: Wed, 12 May 2010 15:56:06 -0400 From: Sebastien Roy Subject: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] To: PSARC-ext Cc: sowmini.varadhan@oracle.com Message-id: <4BEB07D6.2040409@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt353.oracle.com [141.146.40.153] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090206.4BEB0841.00A0:SCFMA4539814,ss=1,fgs=0 User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.8) Gecko/20100412 Lightning/1.0b1 Thunderbird/3.0.3 Status: RO Content-Length: 12214 I'm submitting this fast-track for Sowmini Varadhan. The release binding is Minor and new zonecfg(1M) properties are Committed. Summary: ------- This case proposes a solution for 6944327 need to support address and defrouter resources for exclusive-IP zones Problem Description -------------------- Typical zone deployments that exist today use shared-IP zones to run applications and services like Apache or Weblogic in the contained environment provided by the shared-IP zone. In these use-cases, the Administrator in the global zone has full control over the networking resources used by the non-global zone. In the common case, networking is simply configured by specifying the IP interface, IP addresses and, optionally, the default routers from zonecfg(1m). The configuration resources thus supplied are then applied for the non-global zone when it is booted, and the non-global zone itself may not modify any of these configuration parameters. However, there is no such simple configuration mechanism for the simple networking use-case in place for exclusive IP zones, which must be configured through sysidcfg, ifconfig, and an assortment of other methods, all of which are not controllable from the global zone, and may not be managed through zonecfg(1m). The addition of many new virtualization and resource management features to Solaris (PSARC 2006/357, PSARC 2005/132 and associated cases) makes Exclusive-IP zones a cleaner and more powerful Zone model than shared-IP zones. Thus, bridging any gaps in the configuration methods for the simple use cases between shared- and exlusive-IP zones is important to ease the transition of shared-IP customer configurations to exclusive-IP. In addition, there are many ongoing projects in the "Zones Networking" effort [ZONES-NET] to facilitate the consolidation of existing Solaris 10 host installations as Solaris 10 Containers. These efforts would leverage from the ability to specify networking resource values for address and default router uniformly for zones using zonecfg(1m). This case proposes support in zonecfg(1m) for 'address' and 'defrouter' properties in the 'net' resource for exclusive-IP zones. The semantics of the values will be the same as with the shared-IP zone definitions that exist today ([PSARC/2002/174], [PSARC/2003/621], [PSARC/2008/057]) When the global-zone specifies the 'address' for an interface via zonecfg(1m), the non-global zone may not use any other addresses for the specified IP Version on that interface. The address information provided via zonecfg(1m) will be used to set up Layer-3 protection [PSARC/2009/436] for the non-global zone during zone-boot to filter out all other addresses for the selected interface. For instance, when zonecfg(1m) has been used in the global-zone to set one or more IPv4 addresses on an interface, an attempt to set an IPv4 address on the interface that is outside the globally defined set will encounter the EPERM failure. Thus ifconfig(1m), ipadm(1m), and associated ioctls will receive this error if they are used within the non-global zone to set addresses that are not in the set that is permitted from the global-zone, and attempts by the non-global zone to turn on forwarding on the interface will also encounter EPERM. Note that IPv4 and IPv6 are considered as independent resources, so that specification of an IPv4 address via zonecfg(1m) does not place any constraints on permissible IPv6 addresses, and vice-versa. Attempts to boot a zone that has already been configured for IP, or has previously customized values for Layer-3 protection [PSARC/2009/436] will fail. Implementation Overview: ------------------------ A brief overview of the implementation is provided here. Note that all interfaces between zoneadmd(1m) and the kernel/zonecfg(1m), are Private interfaces, subject to change in the future. When the non-global zone is booted, zoneadmd(1m) will store the information specified by the 'address' and 'defrouter' properties as nvlists in the kernel following a mechanism similar to that in use today for other zonecfg(1m) resources such as the "physical" datalink. At the same time, the zone boot process will also ensure 'ip-nospoof' protection for the datalink with the specified addresses used as input to the 'allowed-ips' property. The stored nvlist information for 'address' and 'defrouter' will be retrieved and re-applied in the non-global zone by the daemon associated with the ip-interface-management service before any other IP configuration is applied. The GLDv3 layer communicates the set of "allowed-ips" to the IP layer through DLPI notifications. The notification sent is a DL_NOTIFY_IND message of (Project Private) type DL_NOTE_ALLOWED_IPS, sent to the IP clients that have registered for this notification. The payload of the DL_NOTE_ALLOWED_IPS message is the mac_protect_t data-structure introduced by PSARC 2009/436. Receipt of this notification enables the IP layer to track the current set of permitted IP addresses per interface, so that the IP layer can return informative error diagnostics if an attempt is made to enable forwarding or set addresses on the interface that are outside the permitted set. All configuration state, including datalink properties like protection, allowed-ips and zone, and associated nvlist information, that is created during zone boot will be reset to the default values when the zone is halted. Relationship to other configuration mechanisms ---------------------------------------------- sysidcfg: The Solaris Install team is currently working in conjunction with the Network Configuration team [INSTALL] on the implementation of a framework to replace sysidcfg(4) that will provide configuration profiles to be processed by startup scripts when a system boots. However this configuration information is only applied on the first boot of the zone, and unlike zonecfg, may not be easily modified in the global zone across reboot. Moreover, there is no mechanism in place to prevent the non-global zone from modifying the configuration provided via profiles. As mentioned earlier, the global-zone is the authoritative source of IP configuration information for the "secure container" use-case. This implies that attempts to use boot profiles in the non-global zone to add addresses other than those mandated in the global-zone will fail with EINVAL erors. Note that the profiles defined in [INSTALL] may still be applied to interfaces for which 'net' resource properties have not been defined in the global-zone, i.e., this proposal is fully backward-compatible. svc:/network/physical:{default, nwam} Configuration specified via zonecfg(1m) in the global-zone overrides any information specified via /etc/hostname.intf, ipadm(1m) or nwam. Thus if the IP address information specified by these other methods is not a subset of the globally managed information, attempts to apply it within the non-global zone will encounter an error. DHCP: It is assumed that when IP resources for an IP interface have been set up in the global-zone using zonecfg(1m), then that interface has been marked for static IP configuration only, and attempts to send out outgoing DHCP DISCOVER packets will be dropped in GLDv3. Thus attempts to start DHCP on these interfaces in the non-global zone will time out. Examples -------- root@gz# zonecfg -z tz1 zonecfg:tz1> info zonename: tz1 zonepath: /rpool/zones/tz1 brand: ipkg autoboot: false bootargs: pool: limitpriv: scheduling-class: ip-type: exclusive hostid: net: address: 11.1.1.1/24 physical: vnic0 defrouter: 11.1.1.2 net: address: taddr2 physical: vnic0 defrouter not specified net: address not specified physical: vnic1 defrouter not specified In this example, the zone 'tz1' has been assigned 2 interfaces: vnic0 and vnic1. The vnic1 interface does not have any IP address or default router assigned to it, so that it may be configured in any desired way from within the zone using ifconfig(1m), ipadm(1m), NWAM etc. The vnic0 interface has 2 addresses assigned: 11.1.1.1/24 (with default router 11.1.1.2 on the 11.1.1.0/24 subnet) and the address 'taddr2' whose numeric value and netmask will be resolved when the zone boots. Thus, if, at the time of zone boot, the global-zone is also configured as: root@gz# egrep 'hosts|netmask' /etc/nsswitch.conf hosts: files nis dns netmasks: files nis root@gz# grep taddr2 /etc/hosts 12.2.3.4 taddr2 root@gz# grep 12 /etc/netmasks 12.2.0.0 255.255.0.0 tz1 will be booted as root@gz# zoneadm -z tz1 boot root@gz# zlogin tz1 ipadm show-addr ADDROBJ TYPE STATE ADDR vnic0/? static ok 11.1.1.1/24 vnic0/? static ok 12.2.3.4/16 lo0/v4 static ok 127.0.0.1/8 lo0/v6 static ok ::1/128 root@gz# zlogin tz1 netstat -rn Routing Table: IPv4 Destination Gateway Flags Ref Use Interface -------------------- -------------------- ----- ----- ---------- --------- default 11.1.1.2 UG 2 2 vnic0 11.1.1.0 11.1.1.1 U 3 2 vnic0 12.2.0.0 12.2.3.4 U 2 0 vnic0 127.0.0.1 127.0.0.1 UH 2 0 lo0 Routing Table: IPv6 Destination/Mask Gateway Flags Ref Use If --------------------------- ---------------------- ----- --- ------- ----- ::1 ::1 UH 2 0 lo0 Proposed man page changes: -------------------------- --- zonecfg.1m.orig Thu Apr 22 11:22:12 2010 +++ zonecfg.1m.new Thu Apr 22 11:27:48 2010 @@ -676,8 +676,13 @@ for the interface should be specified here. For an exclusive-IP zone, the physical property must be - set and the address and default router properties cannot - be set. + set and the address and default router properties may + be set. The interface specifed for the physical property + must not be in use in the global zone. If an address and + default router are specified via zonecfg(1m), these will be + applied to the interface when the non-global zone is booted, + and the non-global zone will not be able to apply any other + addresses to that interface. zhadum(866)% diff -wub dladm.1m.txt dladm.1m.txt.new --- dladm.1m.txt Mon May 10 15:24:49 2010 +++ dladm.1m.txt.new Mon May 10 15:28:09 2010 @@ -4176,7 +4176,17 @@ option's link layer address must match the link's MAC address. + When a datalink has been protected by setting allowed-ips + to a set of one or more IPv4 addresses, any attempts to + configure IPv4 addresses that are not in this set will + fail with an EPERM error being returned to the user. + Moreover, the interface may not be used for forwarding + IPv4 packets, and attempts to set the ipadm(1m) forwarding + property on the interface will encounter EPERM error. + Analogous constraints on address configuration and + forwarding settings also apply for IPv6 packets. + dhcp-nospoof DHCP client ID (DUID for DHCPv6) and hardware References: ---------- [PSARC/2002/174] Virtualization and Namespace Isolation in Solaris, aka "Zones" [PSARC/2003/621] Zone Administration Updates [PSARC/2008/057] Default Route For Zones [PSARC/2009/436] Anti-spoofing Link Protection [INSTALL] http://solaris-networking.sfbay/index.php/Network_Configuration:install [ZONES-NET] http://solaris-networking.sfbay/index.php/Zones_Networking From darren.reed@oracle.com Wed May 12 14:43:05 2010 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4CLh5se015628 for ; Wed, 12 May 2010 14:43:05 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o4CLh5Pp015924 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 12 May 2010 14:43:05 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2B00H0BTNT9H00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 12 May 2010 15:43:05 -0600 (MDT) Received: from brmea-mail-2.sun.com ([192.18.98.43]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2B00GEYTNSNO00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 12 May 2010 15:43:04 -0600 (MDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o4CLh4du000327 for ; Wed, 12 May 2010 21:43:04 +0000 (GMT) Received: from acsmt353.oracle.com (acsmt353.oracle.com [141.146.40.153]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o4CCLDKD023511 for ; Wed, 12 May 2010 21:43:03 +0000 (GMT) Received: from abhmt014.oracle.com by acsmt353.oracle.com with ESMTP id 236460891273700571; Wed, 12 May 2010 14:42:51 -0700 Received: from mintslice.sfbay.sun.com (/129.146.106.55) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 12 May 2010 14:42:51 -0700 Date: Wed, 12 May 2010 14:43:08 -0700 From: Darren Reed Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <4BEB07D6.2040409@oracle.com> To: sowmini.varadhan@oracle.com Cc: PSARC-ext Message-id: <4BEB20EC.70408@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt353.oracle.com [141.146.40.153] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090207.4BEB20E8.003D:SCFMA4539814,ss=1,fgs=0 References: <4BEB07D6.2040409@oracle.com> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-GB; rv:1.9.1.5) Gecko/20091206 Thunderbird/3.0 Status: RO Content-Length: 7302 On 12/05/10 12:56 PM, Sebastien Roy wrote: > ... > Typical zone deployments that exist today use shared-IP zones to run > applications and services like Apache or Weblogic in the contained > environment provided by the shared-IP zone. In these use-cases, the > Administrator in the global zone has full control over the networking > resources used by the non-global zone. In the common case, networking > is simply configured by specifying the IP interface, IP addresses and, > optionally, the default routers from zonecfg(1m). The configuration > resources thus supplied are then applied for the non-global zone when > it is booted, and the non-global zone itself may not modify any of > these configuration parameters. > > However, there is no such simple configuration mechanism for the > simple networking use-case in place for exclusive IP zones, which must > be configured through sysidcfg, ifconfig, and an assortment of other > methods, all of which are not controllable from the global zone, and > may not be managed through zonecfg(1m). > > The addition of many new virtualization and resource management > features to Solaris (PSARC 2006/357, PSARC 2005/132 and associated > cases) makes Exclusive-IP zones a cleaner and more powerful Zone model > than shared-IP zones. Thus, bridging any gaps in the configuration > methods for the simple use cases between shared- and exlusive-IP zones > is important to ease the transition of shared-IP customer > configurations to exclusive-IP. > ... > When the global-zone specifies the 'address' for an interface via > zonecfg(1m), the non-global zone may not use any other addresses for > the specified IP Version on that interface. The address information > provided via zonecfg(1m) will be used to set up Layer-3 protection > [PSARC/2009/436] for the non-global zone during zone-boot to filter > out all other addresses for the selected interface. For instance, > when zonecfg(1m) has been used in the global-zone to set one or more > IPv4 addresses on an interface, an attempt to set an IPv4 address on > the interface that is outside the globally defined set will encounter > the EPERM failure. Thus ifconfig(1m), ipadm(1m), and associated > ioctls will receive this error if they are used within the non-global > zone to set addresses that are not in the set that is permitted from > the global-zone, and attempts by the non-global zone to turn on > forwarding on the interface will also encounter EPERM. Note that IPv4 > and IPv6 are considered as independent resources, so that > specification of an IPv4 address via zonecfg(1m) does not place any > constraints on permissible IPv6 addresses, and vice-versa. > > Attempts to boot a zone that has already been configured for IP, or > has previously customized values for Layer-3 protection > [PSARC/2009/436] will fail. > > Implementation Overview: > ------------------------ > A brief overview of the implementation is provided here. Note that all > interfaces between zoneadmd(1m) and the kernel/zonecfg(1m), are > Private interfaces, subject to change in the future. > > When the non-global zone is booted, zoneadmd(1m) will store the > information specified by the 'address' and 'defrouter' properties as > nvlists in the kernel following a mechanism similar to that in use > today for other zonecfg(1m) resources such as the "physical" > datalink. At the same time, the zone boot process will also ensure > 'ip-nospoof' protection for the datalink with the specified addresses > used as input to the 'allowed-ips' property. The stored nvlist > information for 'address' and 'defrouter' will be retrieved and > re-applied in the non-global zone by the daemon associated with the > ip-interface-management service before any other IP configuration is > applied. What is done to ensure the alignment of "allowed-ips" and the addresses configured? If I change the "allowed-ips" directly with dladm after a zone has booted, will there be any warning? If I want to change the IP address of a network interface in a zone that is so restricted, what is the correct procedure? At the top, it states that if the global zone configures the address for a network interface then the local zone is not permitted to change it. Why should that have anything to do with forwarding? Is the right approach being taken here? This would appear to be a security issue (revoking the right of a zone to make a change to its network configuration) but yet the implementation of the support for this does not make use of any security fundamentals. Why not? At the bottom of this document, the man page update mentions that this feature is incompatible with enabling forwarding. Why? Shouldn't we be engineering Solaris to make features compatible with each other rather than incompatible? > The GLDv3 layer communicates the set of "allowed-ips" to the IP layer > through DLPI notifications. The notification sent is a DL_NOTIFY_IND > message of (Project Private) type DL_NOTE_ALLOWED_IPS, sent to the IP > clients that have registered for this notification. The payload of > the DL_NOTE_ALLOWED_IPS message is the mac_protect_t data-structure > introduced by PSARC 2009/436. Receipt of this notification enables > the IP layer to track the current set of permitted IP addresses per > interface, so that the IP layer can return informative error > diagnostics if an attempt is made to enable forwarding or set > addresses on the interface that are outside the permitted set. I'm uncomfortable with this because it increases the complexity of the interaction with GLD and IP for questionable gain. My view of this is that it is an attempt to make up for the lack of proper architecture whereby networking resources for a zone are supplied, set and applied. The title of this case is "layer-3 net properties for exclusive-IP zones" but there's nothing in the design being put forward that looks anything like an attempt to solve this problem in a generic fashion aside from the common repository for configuration data being the zone configuration file. Two different properties, the IP address and the default router, are applied to the zone in a very different way. The danger I see here is that there is no pattern to follow for future properties and that it is hard to recognise which one of the two being used here would be used for others. If, for example, I wanted to extend this work to supply other layer-3 network properties, how would I do that? Implement a new mechanism for each one? Does that really scale? My concern here is that given a problem that would have future benefit from real architecture being developed, the solution that we're being given is a few hacks to solve specific problems. > DHCP: > It is assumed that when IP resources for an IP interface have been > set up in the global-zone using zonecfg(1m), then that interface > has been marked for static IP configuration only, and attempts to > send out outgoing DHCP DISCOVER packets will be dropped in > GLDv3. Thus attempts to start DHCP on these interfaces in the > non-global zone will time out. If we can present error messages for ifconfig/ipadm when it attempts to change the IP address, why can't we present an error message when someone tries to use DHCP on the network interface? Darren From sowmini.varadhan@oracle.com Wed May 12 16:36:14 2010 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4CNaEcW018611 for ; Wed, 12 May 2010 16:36:14 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o4CNaDoG004235 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 12 May 2010 16:36:13 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2B00L01YWD4000@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 12 May 2010 16:36:13 -0700 (PDT) Received: from quasimodo.East.Sun.COM ([129.148.174.94]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2B00HKNYWCHX20@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 12 May 2010 16:36:13 -0700 (PDT) Received: from quasimodo.East.Sun.COM (localhost [127.0.0.1]) by quasimodo.East.Sun.COM (8.14.4+Sun/8.14.4) with ESMTP id o4CNTNfM025896; Wed, 12 May 2010 19:29:23 -0400 (EDT) Received: (from sowmini@localhost) by quasimodo.East.Sun.COM (8.14.4+Sun/8.14.4/Submit) id o4CNTNDP025895; Wed, 12 May 2010 19:29:23 -0400 (EDT) Date: Wed, 12 May 2010 19:29:23 -0400 From: sowmini.varadhan@oracle.com Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <4BEB20EC.70408@oracle.com> To: Darren Reed Cc: PSARC-ext Message-id: <20100512232923.GA25862@quasimodo.East.Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <4BEB07D6.2040409@oracle.com> <4BEB20EC.70408@oracle.com> X-Authentication-warning: quasimodo.East.Sun.COM: sowmini set sender to sowmini.varadhan@oracle.com using -f User-Agent: Mutt/1.5.14 (2007-03-31) Status: RO Content-Length: 3519 On (05/12/10 14:43), Darren Reed wrote: > > What is done to ensure the alignment of "allowed-ips" and the > addresses configured? We've already had extensive internal discussions about this, but let me reiterate.. I don't know what you mean by "alignment", but as Erik explained to you "Today if you set allowed-ips with dladm and then use ifconfig/ipadm to set the IP address to something which is not in allowed-ips, then the system silently drops all your sent packets on the floor. With this proposal instead the SIOCSLIFADDR ioctl will fail, so ifconfig and ipadm can report that failure to the admin." > If I change the "allowed-ips" directly with dladm after a zone > has booted, will there be any warning? No, you would only be able to do this from the Global Zone, and the assumption is that the GZ is the authoritative source of configuration, so if that's the choice made in the GZ, nothing will prevent this. > If I want to change the IP address of a network interface in a > zone that is so restricted, what is the correct procedure? you would have to do the same thing that you do for shared IP today- the GZ would have to change zonecfg and the change would be picked up on the next reboot. > At the top, it states that if the global zone configures the address > for a network interface then the local zone is not permitted to > change it. Why should that have anything to do with forwarding? > Is the right approach being taken here? This would appear to > be a security issue (revoking the right of a zone to make a change > to its network configuration) but yet the implementation of the > support for this does not make use of any security fundamentals. > Why not? In internal discussions, it was my understanding that *you* had expressed the objection that setting allowed-ips on the interface would, in effect, prevent the NGZ from forwarding out of that interface, and that this behavior would happen without warning. Also in internal discussions many of us have repeatedly explained to you that this is the case where we want to simplify the simple network configuration case using the same techniques that are already familiar to shared-IP users. Thus, as Michael Hunter observed, to look at this as a "pure configuration" or "pure security" solution is misleading. > At the bottom of this document, the man page update mentions > that this feature is incompatible with enabling forwarding. Why? > Shouldn't we be engineering Solaris to make features compatible > with each other rather than incompatible? This feature does not introduce any feature incompatibility. It merely makes the error semantics explicit. > I'm uncomfortable with this because it increases the complexity > of the interaction with GLD and IP for questionable gain. My > view of this is that it is an attempt to make up for the lack > of proper architecture whereby networking resources for a zone > are supplied, set and applied. : > If, for example, I wanted to extend this work to supply other layer-3 > network properties, how would I do that? Implement a new mechanism > for each one? Does that really scale? > > My concern here is that given a problem that would have future benefit > from real architecture being developed, the solution that we're being > given is a few hacks to solve specific problems. perhaps we should just agree to disagree, because from the numerous internal discussions, I don't think your opinion is shared by a majority. --Sowmini From darren.reed@oracle.com Wed May 12 17:08:53 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4D08rjf019270 for ; Wed, 12 May 2010 17:08:53 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o4D08qnT027849 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 12 May 2010 18:08:52 -0600 (MDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2C007050ESSD00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 12 May 2010 18:08:52 -0600 (MDT) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2C00G2W0ESNN90@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 12 May 2010 18:08:52 -0600 (MDT) Received: from rcsinet15.oracle.com (rcsinet15.oracle.com [148.87.113.117]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o4D08pfO005380 for ; Thu, 13 May 2010 00:08:52 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by rcsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o4CN8hoY010645 for ; Thu, 13 May 2010 00:08:51 +0000 (GMT) Received: from abhmt018.oracle.com by acsmt355.oracle.com with ESMTP id 236763431273709283; Wed, 12 May 2010 17:08:03 -0700 Received: from mintslice.sfbay.sun.com (/129.146.106.55) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 12 May 2010 17:08:03 -0700 Date: Wed, 12 May 2010 17:08:21 -0700 From: Darren Reed Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <20100512232923.GA25862@quasimodo.East.Sun.COM> To: sowmini.varadhan@oracle.com Cc: PSARC-ext Message-id: <4BEB42F5.4030305@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090202.4BEB4313.012A:SCFMA4539814,ss=1,fgs=0 References: <4BEB07D6.2040409@oracle.com> <4BEB20EC.70408@oracle.com> <20100512232923.GA25862@quasimodo.East.Sun.COM> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-GB; rv:1.9.1.5) Gecko/20091206 Thunderbird/3.0 Status: RO Content-Length: 5303 On 12/05/10 04:29 PM, sowmini.varadhan@oracle.com wrote: > On (05/12/10 14:43), Darren Reed wrote: >> >> What is done to ensure the alignment of "allowed-ips" and the >> addresses configured? > > We've already had extensive internal discussions about this, but > let me reiterate.. > > I don't know what you mean by "alignment", but as Erik explained to you > > "Today if you set allowed-ips with dladm and then use ifconfig/ipadm to > set the IP address to something which is not in allowed-ips, then the > system silently drops all your sent packets on the floor. > With this proposal instead the SIOCSLIFADDR ioctl will fail, so ifconfig > and ipadm can report that failure to the admin." The first problem cited here is independent of the one that this project is aiming to solve: making it easier and more secure to configure networking for a zone with an exclusive IP instance. I'm also at a loss to see why solving that problem of silently dropped packets is required to solve the problem of supporting a configuration transition for those running with shared IP instances that we want to get running on exclusive IP instances. If the link already has a non-empty set of IP addresses set in its "allowed-ips" property, will this feature add new addresses to it or remove all of the existing ones and insert only the new one? >> If I change the "allowed-ips" directly with dladm after a zone >> has booted, will there be any warning? > > No, you would only be able to do this from the Global Zone, and the > assumption is that the GZ is the authoritative source of configuration, > so if that's the choice made in the GZ, nothing will prevent this. Does this make sense? New bits have been added to force synchronisation of IP with what has been loaded into the MAC layer (it cannot set the interface IP to something not in the advertised set) but there is no further synchronisation if the details change in the MAC layer. For example, in that case, shouldn't further messages from GLD be sent up to IP? >> At the top, it states that if the global zone configures the address >> for a network interface then the local zone is not permitted to >> change it. Why should that have anything to do with forwarding? >> Is the right approach being taken here? This would appear to >> be a security issue (revoking the right of a zone to make a change >> to its network configuration) but yet the implementation of the >> support for this does not make use of any security fundamentals. >> Why not? > > In internal discussions, it was my understanding that *you* had expressed > the objection that setting allowed-ips on the interface would, in effect, > prevent the NGZ from forwarding out of that interface, and that this > behavior would happen without warning. Also in internal discussions > many of us have repeatedly explained to you that this is the > case where we want to simplify the simple network configuration case > using the same techniques that are already familiar to shared-IP users. I cannot see why a configuration requirement of only being able to set the IP address to that specified in the zone's configuration file needs to be tied to the list of IP addresses that are allowed in the source address field of a packet. Solving that problem should be independent of solving the problem that packets with a source address not in the list of allowed IP addresses disappear silently. >> At the bottom of this document, the man page update mentions >> that this feature is incompatible with enabling forwarding. Why? >> Shouldn't we be engineering Solaris to make features compatible >> with each other rather than incompatible? > > This feature does not introduce any feature incompatibility. It merely > makes the error semantics explicit. Yes, it does. The use of "allowed-ips", today, is not incompatible with a zone functioning as a router and nor is configuring a zone with an exclusive instance of IP incompatible. Nor does the operation of either feature introduce any specific errors or failure conditions that preclude forwarding from functioning. >> I'm uncomfortable with this because it increases the complexity >> of the interaction with GLD and IP for questionable gain. My >> view of this is that it is an attempt to make up for the lack >> of proper architecture whereby networking resources for a zone >> are supplied, set and applied. > : >> If, for example, I wanted to extend this work to supply other layer-3 >> network properties, how would I do that? Implement a new mechanism >> for each one? Does that really scale? >> >> My concern here is that given a problem that would have future benefit >> from real architecture being developed, the solution that we're being >> given is a few hacks to solve specific problems. > > perhaps we should just agree to disagree, because from the numerous > internal discussions, I don't think your opinion is shared by a majority. Whilst I have no issue with the changes and interfaces presented for zonecfg's configuration file, what goes on behind that smells bad to me and nothing that I've read in terms of explanations from anyone has done anything to change that. But if that's what you really want to do, I can't stop you. Darren From edward.pilatowicz@oracle.com Wed May 12 18:26:27 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4D1QQXm020067 for ; Wed, 12 May 2010 18:26:27 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o4D1QQDc063334 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 12 May 2010 19:26:26 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2C00N054022F00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 12 May 2010 18:26:26 -0700 (PDT) Received: from jurassic.Eng.Sun.COM ([129.146.17.63]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2C008YY401KNE0@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 12 May 2010 18:26:25 -0700 (PDT) Received: from jurassic.Eng.Sun.COM (localhost [127.0.0.1]) by jurassic.Eng.Sun.COM (8.14.4+Sun/8.14.4) with ESMTP id o4D1QPTh232707 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 12 May 2010 18:26:25 -0700 (PDT) Received: (from edp@localhost) by jurassic.Eng.Sun.COM (8.14.4+Sun/8.14.4/Submit) id o4D1QPG3232706; Wed, 12 May 2010 18:26:25 -0700 (PDT) Date: Wed, 12 May 2010 18:26:25 -0700 From: Edward Pilatowicz Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <4BEB07D6.2040409@oracle.com> To: Sebastien Roy Cc: PSARC-ext , sowmini.varadhan@oracle.com Message-id: <20100513012624.GJ176566@jurassic.Eng.Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <4BEB07D6.2040409@oracle.com> X-Authentication-warning: jurassic.Eng.Sun.COM: edp set sender to edward.pilatowicz@oracle.com using -f User-Agent: Mutt/1.5.20 (2009-06-14) Status: RO Content-Length: 14651 a few comments / questions. - this bit seems odd to me: ---8<--- ... Note that IPv4 and IPv6 are considered as independent resources, so that specification of an IPv4 address via zonecfg(1m) does not place any constraints on permissible IPv6 addresses, and vice-versa. ---8<--- ipv4 and ipv6 addresses are considered independent resources from a zonecfg(1m) perspective, but from a systems management perspective they don't seem that different to me. to me, it seems that once an admin decides to restrict what addresses can be used on a given interface, that restriction should apply to both ipv4 and ipv6 addresses. for example, with the current mechanism, if an admin want to prevent the zone from configuring any ipv6 address, how do they go about doing this? - does the existence of a zonecfg(1m) defrouter property on an interface prevent a zone from changing the default router for that given interface? if so, will that change persist across reboots? - does the existence of a zonecfg(1m) defrouter property on an interface prevent a zone from adding additional routes on a given interface? - can exclusive stack zones manipulate arp tables? if so, does use of the zonecfg(1m) defrouter property restrict arp table manipulation in any way? (i'm wondering if arp table manipulation could be used to step around any potential routing limitations placed on an interface.) - can exclusive stack zones manipulate mac addresses on network interfaces? if so, does the use of the zonecfg(1m) address property restrict this in any way? - the currently proposed DHCP time-out failure isn't that great. is there a way to improve this failure? say by returning EPERM for whatever operation it is that's used to initiate DHCP configuration on an interface. if not it isn't that big of an issue, but as long as we know we're going to fail it's usually nicer not to keep the user waiting... ;) ed On Wed, May 12, 2010 at 12:56:06PM -0700, Sebastien Roy wrote: > I'm submitting this fast-track for Sowmini Varadhan. The release > binding is Minor and new zonecfg(1M) properties are Committed. > > Summary: > ------- > This case proposes a solution for > > 6944327 need to support address and defrouter resources for > exclusive-IP zones > > Problem Description > -------------------- > > Typical zone deployments that exist today use shared-IP zones to run > applications and services like Apache or Weblogic in the contained > environment provided by the shared-IP zone. In these use-cases, the > Administrator in the global zone has full control over the networking > resources used by the non-global zone. In the common case, networking > is simply configured by specifying the IP interface, IP addresses and, > optionally, the default routers from zonecfg(1m). The configuration > resources thus supplied are then applied for the non-global zone when > it is booted, and the non-global zone itself may not modify any of > these configuration parameters. > > However, there is no such simple configuration mechanism for the > simple networking use-case in place for exclusive IP zones, which must > be configured through sysidcfg, ifconfig, and an assortment of other > methods, all of which are not controllable from the global zone, and > may not be managed through zonecfg(1m). > > The addition of many new virtualization and resource management > features to Solaris (PSARC 2006/357, PSARC 2005/132 and associated > cases) makes Exclusive-IP zones a cleaner and more powerful Zone model > than shared-IP zones. Thus, bridging any gaps in the configuration > methods for the simple use cases between shared- and exlusive-IP zones > is important to ease the transition of shared-IP customer > configurations to exclusive-IP. > > In addition, there are many ongoing projects in the "Zones Networking" > effort [ZONES-NET] to facilitate the consolidation of existing Solaris > 10 host installations as Solaris 10 Containers. These efforts would > leverage from the ability to specify networking resource values for > address and default router uniformly for zones using zonecfg(1m). > > This case proposes support in zonecfg(1m) for 'address' and > 'defrouter' properties in the 'net' resource for exclusive-IP zones. > The semantics of the values will be the same as with the shared-IP > zone definitions that exist today ([PSARC/2002/174], [PSARC/2003/621], > [PSARC/2008/057]) > > When the global-zone specifies the 'address' for an interface via > zonecfg(1m), the non-global zone may not use any other addresses for > the specified IP Version on that interface. The address information > provided via zonecfg(1m) will be used to set up Layer-3 protection > [PSARC/2009/436] for the non-global zone during zone-boot to filter > out all other addresses for the selected interface. For instance, > when zonecfg(1m) has been used in the global-zone to set one or more > IPv4 addresses on an interface, an attempt to set an IPv4 address on > the interface that is outside the globally defined set will encounter > the EPERM failure. Thus ifconfig(1m), ipadm(1m), and associated > ioctls will receive this error if they are used within the non-global > zone to set addresses that are not in the set that is permitted from > the global-zone, and attempts by the non-global zone to turn on > forwarding on the interface will also encounter EPERM. Note that IPv4 > and IPv6 are considered as independent resources, so that > specification of an IPv4 address via zonecfg(1m) does not place any > constraints on permissible IPv6 addresses, and vice-versa. > > Attempts to boot a zone that has already been configured for IP, or > has previously customized values for Layer-3 protection > [PSARC/2009/436] will fail. > > Implementation Overview: > ------------------------ > A brief overview of the implementation is provided here. Note that all > interfaces between zoneadmd(1m) and the kernel/zonecfg(1m), are > Private interfaces, subject to change in the future. > > When the non-global zone is booted, zoneadmd(1m) will store the > information specified by the 'address' and 'defrouter' properties as > nvlists in the kernel following a mechanism similar to that in use > today for other zonecfg(1m) resources such as the "physical" > datalink. At the same time, the zone boot process will also ensure > 'ip-nospoof' protection for the datalink with the specified addresses > used as input to the 'allowed-ips' property. The stored nvlist > information for 'address' and 'defrouter' will be retrieved and > re-applied in the non-global zone by the daemon associated with the > ip-interface-management service before any other IP configuration is > applied. > > The GLDv3 layer communicates the set of "allowed-ips" to the IP layer > through DLPI notifications. The notification sent is a DL_NOTIFY_IND > message of (Project Private) type DL_NOTE_ALLOWED_IPS, sent to the IP > clients that have registered for this notification. The payload of > the DL_NOTE_ALLOWED_IPS message is the mac_protect_t data-structure > introduced by PSARC 2009/436. Receipt of this notification enables > the IP layer to track the current set of permitted IP addresses per > interface, so that the IP layer can return informative error > diagnostics if an attempt is made to enable forwarding or set > addresses on the interface that are outside the permitted set. > > All configuration state, including datalink properties like > protection, allowed-ips and zone, and associated nvlist information, > that is created during zone boot will be reset to the default values > when the zone is halted. > > Relationship to other configuration mechanisms > ---------------------------------------------- > > sysidcfg: > The Solaris Install team is currently working in conjunction with > the Network Configuration team [INSTALL] on the implementation of a > framework to replace sysidcfg(4) that will provide configuration > profiles to be processed by startup scripts when a system > boots. However this configuration information is only applied on > the first boot of the zone, and unlike zonecfg, may not be easily > modified in the global zone across reboot. Moreover, there is no > mechanism in place to prevent the non-global zone from modifying > the configuration provided via profiles. > > As mentioned earlier, the global-zone is the authoritative source > of IP configuration information for the "secure container" > use-case. This implies that attempts to use boot profiles in the > non-global zone to add addresses other than those mandated in the > global-zone will fail with EINVAL erors. > > Note that the profiles defined in [INSTALL] may still be applied to > interfaces for which 'net' resource properties have not been > defined in the global-zone, i.e., this proposal is fully > backward-compatible. > > svc:/network/physical:{default, nwam} > Configuration specified via zonecfg(1m) in the global-zone > overrides any information specified via /etc/hostname.intf, > ipadm(1m) or nwam. Thus if the IP address information specified by > these other methods is not a subset of the globally managed > information, attempts to apply it within the non-global zone will > encounter an error. > > DHCP: > It is assumed that when IP resources for an IP interface have been > set up in the global-zone using zonecfg(1m), then that interface > has been marked for static IP configuration only, and attempts to > send out outgoing DHCP DISCOVER packets will be dropped in > GLDv3. Thus attempts to start DHCP on these interfaces in the > non-global zone will time out. > > Examples > -------- > root@gz# zonecfg -z tz1 > zonecfg:tz1> info > zonename: tz1 > zonepath: /rpool/zones/tz1 > brand: ipkg > autoboot: false > bootargs: > pool: > limitpriv: > scheduling-class: > ip-type: exclusive > hostid: > net: > address: 11.1.1.1/24 > physical: vnic0 > defrouter: 11.1.1.2 > net: > address: taddr2 > physical: vnic0 > defrouter not specified > net: > address not specified > physical: vnic1 > defrouter not specified > > In this example, the zone 'tz1' has been assigned 2 interfaces: vnic0 > and vnic1. > > The vnic1 interface does not have any IP address or default router > assigned to it, so that it may be configured in any desired way from > within the zone using ifconfig(1m), ipadm(1m), NWAM etc. > > The vnic0 interface has 2 addresses assigned: 11.1.1.1/24 (with > default router 11.1.1.2 on the 11.1.1.0/24 subnet) and the address > 'taddr2' whose numeric value and netmask will be resolved when the > zone boots. Thus, if, at the time of zone boot, the global-zone is > also configured as: > > root@gz# egrep 'hosts|netmask' /etc/nsswitch.conf > hosts: files nis dns > netmasks: files nis > > root@gz# grep taddr2 /etc/hosts > 12.2.3.4 taddr2 > > root@gz# grep 12 /etc/netmasks > 12.2.0.0 255.255.0.0 > > tz1 will be booted as > > root@gz# zoneadm -z tz1 boot > > root@gz# zlogin tz1 ipadm show-addr > ADDROBJ TYPE STATE ADDR > vnic0/? static ok 11.1.1.1/24 > vnic0/? static ok 12.2.3.4/16 > lo0/v4 static ok 127.0.0.1/8 > lo0/v6 static ok ::1/128 > > root@gz# zlogin tz1 netstat -rn > Routing Table: IPv4 > Destination Gateway Flags Ref Use > Interface > -------------------- -------------------- ----- ----- ---------- > --------- > default 11.1.1.2 UG 2 2 > vnic0 > 11.1.1.0 11.1.1.1 U 3 2 > vnic0 > 12.2.0.0 12.2.3.4 U 2 0 > vnic0 > 127.0.0.1 127.0.0.1 UH 2 0 > lo0 > > Routing Table: IPv6 > Destination/Mask Gateway Flags Ref > Use If > --------------------------- ---------------------- ----- --- > ------- ----- > ::1 ::1 UH 2 0 > lo0 > > > Proposed man page changes: > -------------------------- > --- zonecfg.1m.orig Thu Apr 22 11:22:12 2010 > +++ zonecfg.1m.new Thu Apr 22 11:27:48 2010 > @@ -676,8 +676,13 @@ > for the interface should be specified here. > > For an exclusive-IP zone, the physical property must be > - set and the address and default router properties cannot > - be set. > + set and the address and default router properties may > + be set. The interface specifed for the physical property > + must not be in use in the global zone. If an address and > + default router are specified via zonecfg(1m), these will be > + applied to the interface when the non-global zone is booted, > + and the non-global zone will not be able to apply any other > + addresses to that interface. > > zhadum(866)% diff -wub dladm.1m.txt dladm.1m.txt.new > --- dladm.1m.txt Mon May 10 15:24:49 2010 > +++ dladm.1m.txt.new Mon May 10 15:28:09 2010 > @@ -4176,7 +4176,17 @@ > option's link layer address must match the > link's MAC address. > > + When a datalink has been protected by setting allowed-ips > + to a set of one or more IPv4 addresses, any attempts to > + configure IPv4 addresses that are not in this set will > + fail with an EPERM error being returned to the user. > + Moreover, the interface may not be used for forwarding > + IPv4 packets, and attempts to set the ipadm(1m) forwarding > + property on the interface will encounter EPERM error. > + Analogous constraints on address configuration and > + forwarding settings also apply for IPv6 packets. > > + > dhcp-nospoof > > DHCP client ID (DUID for DHCPv6) and hardware > > > > References: > ---------- > [PSARC/2002/174] Virtualization and Namespace Isolation in Solaris, > aka "Zones" > [PSARC/2003/621] Zone Administration Updates > [PSARC/2008/057] Default Route For Zones > [PSARC/2009/436] Anti-spoofing Link Protection > [INSTALL] http://solaris-networking.sfbay/index.php/Network_Configuration:install > [ZONES-NET] http://solaris-networking.sfbay/index.php/Zones_Networking From sowmini.varadhan@oracle.com Wed May 12 19:30:03 2010 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4D2U30e021051 for ; Wed, 12 May 2010 19:30:03 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o4D2U3Af026248 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 12 May 2010 19:30:03 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2C00H0D6Y3PG00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 12 May 2010 19:30:03 -0700 (PDT) Received: from quasimodo.East.Sun.COM ([129.148.174.94]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2C0047B6Y28BF0@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 12 May 2010 19:30:03 -0700 (PDT) Received: from quasimodo.East.Sun.COM (localhost [127.0.0.1]) by quasimodo.East.Sun.COM (8.14.4+Sun/8.14.4) with ESMTP id o4D2NDKS026025; Wed, 12 May 2010 22:23:13 -0400 (EDT) Received: (from sowmini@localhost) by quasimodo.East.Sun.COM (8.14.4+Sun/8.14.4/Submit) id o4D2NDhc026024; Wed, 12 May 2010 22:23:13 -0400 (EDT) Date: Wed, 12 May 2010 22:23:13 -0400 From: sowmini.varadhan@oracle.com Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <20100513012624.GJ176566@jurassic.Eng.Sun.COM> To: Edward Pilatowicz Cc: Sebastien Roy , PSARC-ext Message-id: <20100513022313.GB25937@quasimodo.East.Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <4BEB07D6.2040409@oracle.com> <20100513012624.GJ176566@jurassic.Eng.Sun.COM> X-Authentication-warning: quasimodo.East.Sun.COM: sowmini set sender to sowmini.varadhan@oracle.com using -f User-Agent: Mutt/1.5.14 (2007-03-31) Status: RO Content-Length: 3524 > - this bit seems odd to me: > > ---8<--- > ... Note that IPv4 > and IPv6 are considered as independent resources, so that > specification of an IPv4 address via zonecfg(1m) does not place any > constraints on permissible IPv6 addresses, and vice-versa. > ---8<--- > > ipv4 and ipv6 addresses are considered independent resources from a > zonecfg(1m) perspective, but from a systems management perspective > they don't seem that different to me. to me, it seems that once an > admin decides to restrict what addresses can be used on a given > interface, that restriction should apply to both ipv4 and ipv6 I could go either way on this.. one interpretation of specifying only ipv4 addresses in zonecfg would be "ipv6-addrs are dont-cares". Another is that "ipv6-addrs are forbidden". I've somewhat arbitrarily selected the former interpretation, but could just as easily go with the latter. > addresses. for example, with the current mechanism, if an admin want > to prevent the zone from configuring any ipv6 address, how do they go > about doing this? Currently, none, though the "only ipv4 specified implies ipv6-addrs are forbidden" approach solves that. In retrospect, that choices seems simpler and cleaner. Is that preferable? > - does the existence of a zonecfg(1m) defrouter property on an interface > prevent a zone from changing the default router for that given > interface? No. > - does the existence of a zonecfg(1m) defrouter property on an interface > prevent a zone from adding additional routes on a given interface? No. Note that, even though a shared-IP zone cannot add routes today, there's really nothing that constrains it to using the routes specified in the routing table- it could easily circumvent the routing table by using IP source-routes, or by using IP*NEXTHOP or IP*PKTINFO socket options. So preventing the addition of routes doesn't actually achieve any limitations. > - can exclusive stack zones manipulate arp tables? yes. > if so, does use of > the zonecfg(1m) defrouter property restrict arp table manipulation in > any way? (i'm wondering if arp table manipulation could be used to > step around any potential routing limitations placed on an interface.) there are many ways to step around routing limitations even with shared-IP today, so not sure what limitations you have in mind. Basically, regardless of what the routing table has, a node can always send a packet to any onlink router. If what we want, is to prevent the NGZ from DOSing some router(s), then bandwidth controls are a more reliable way to achieve that. > - can exclusive stack zones manipulate mac addresses on network > interfaces? yes- they can use 'ifconfig .. ether <..>'. > if so, does the use of the zonecfg(1m) address property > restrict this in any way? no, it does not - the address property only clamps dow the IP address, and makes no promises about the mac address associated with the IP address. > - the currently proposed DHCP time-out failure isn't that great. is > there a way to improve this failure? say by returning EPERM for > whatever operation it is that's used to initiate DHCP configuration on > an interface. It would be hard to catch all of them because the udp DHCP DISCOVER could be sent in multiple ways (e.g. as a udp packet, via IP_HDRINCL etc.) > if not it isn't that big of an issue, but as long as we > know we're going to fail it's usually nicer not to keep the user > waiting... ;) --Sowmini From girish.moodalbail@oracle.com Wed May 12 20:54:35 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4D3sY4d023157 for ; Wed, 12 May 2010 20:54:34 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o4D3sXRV062441 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 12 May 2010 21:54:34 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2C00C0HAUXNI00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 12 May 2010 20:54:33 -0700 (PDT) Received: from sca-ea-mail-4.sun.com ([192.18.43.22]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2C00H3QAUXHTB0@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 12 May 2010 20:54:33 -0700 (PDT) Received: from rcsinet15.oracle.com (rcsinet15.oracle.com [148.87.113.117]) by sca-ea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o4D3sWBq021733 for ; Thu, 13 May 2010 03:54:32 +0000 (GMT) Received: from acsmt353.oracle.com (acsmt353.oracle.com [141.146.40.153]) by rcsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o4D3sTtd019243 for ; Thu, 13 May 2010 03:54:29 +0000 (GMT) Received: from abhmt005.oracle.com by acsmt355.oracle.com with ESMTP id 237082221273722765; Wed, 12 May 2010 20:52:45 -0700 Received: from [10.7.250.129] (/10.7.250.129) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 12 May 2010 20:52:44 -0700 Date: Wed, 12 May 2010 22:52:42 -0500 From: Girish Moodalbail Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <4BEB07D6.2040409@oracle.com> To: Sebastien Roy Cc: PSARC-ext , sowmini.varadhan@oracle.com Message-id: <4BEB778A.6010509@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt353.oracle.com [141.146.40.153] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090206.4BEB77F8.0081:SCFMA4539814,ss=1,fgs=0 References: <4BEB07D6.2040409@oracle.com> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.8) Gecko/20100412 Lightning/1.0b1 Thunderbird/3.0.3 Status: RO Content-Length: 1195 > root@gz# egrep 'hosts|netmask' /etc/nsswitch.conf > hosts: files nis dns > netmasks: files nis > > root@gz# grep taddr2 /etc/hosts > 12.2.3.4 taddr2 > > root@gz# grep 12 /etc/netmasks > 12.2.0.0 255.255.0.0 > > tz1 will be booted as > > root@gz# zoneadm -z tz1 boot > > root@gz# zlogin tz1 ipadm show-addr > ADDROBJ TYPE STATE ADDR > vnic0/? static ok 11.1.1.1/24 > vnic0/? static ok 12.2.3.4/16 > lo0/v4 static ok 127.0.0.1/8 > lo0/v6 static ok ::1/128 A nit: If the daemon associated with the ip-interface-management service is used to configure above addresses on zone boot and address object names were not explicitly provided then an address object name will be generated for that address by the daemon. Therefore in your examples above you will never see '?' and instead you will see address object names, starting with '_', such as vnic0/_a, vnic0/_b, et al. So the example above has to be corrected. There is a difference between vnic0/? and vnic0/_a. vnic0/? means that the address was created outside libipadm/ipmgmtd and therefore cannot be managed by ipadm(1M) (ipadm show-addrprop, set-addrprop and reset-addrprop). However vnic0/_a can be managed by ipadm. ~Girish From sowmini.varadhan@oracle.com Thu May 13 03:14:58 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4DAEwpv018811 for ; Thu, 13 May 2010 03:14:58 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o4DAEu0W049168 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Thu, 13 May 2010 04:14:58 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2C00901SGYNY00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 13 May 2010 03:14:58 -0700 (PDT) Received: from quasimodo.East.Sun.COM ([129.148.174.94]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2C002NPSGXZQ80@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 13 May 2010 03:14:57 -0700 (PDT) Received: from quasimodo.East.Sun.COM (localhost [127.0.0.1]) by quasimodo.East.Sun.COM (8.14.4+Sun/8.14.4) with ESMTP id o4DA87Yq026144; Thu, 13 May 2010 06:08:07 -0400 (EDT) Received: (from sowmini@localhost) by quasimodo.East.Sun.COM (8.14.4+Sun/8.14.4/Submit) id o4DA87Wr026143; Thu, 13 May 2010 06:08:07 -0400 (EDT) Date: Thu, 13 May 2010 06:08:07 -0400 From: sowmini.varadhan@oracle.com Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <4BEB778A.6010509@oracle.com> To: Girish Moodalbail Cc: Sebastien Roy , PSARC-ext Message-id: <20100513100807.GA26129@quasimodo.East.Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <4BEB07D6.2040409@oracle.com> <4BEB778A.6010509@oracle.com> X-Authentication-warning: quasimodo.East.Sun.COM: sowmini set sender to sowmini.varadhan@oracle.com using -f User-Agent: Mutt/1.5.14 (2007-03-31) Status: RO Content-Length: 1648 On (05/12/10 22:52), Girish Moodalbail wrote: > > If the daemon associated with the ip-interface-management service is used to > configure above addresses on zone boot and address object names were not > explicitly provided then an address object name will be generated for that > address by the daemon. > > Therefore in your examples above you will never see '?' and instead you will > see address object names, starting with '_', such as vnic0/_a, vnic0/_b, et > al. So the example above has to be corrected. Actually no. this is deep into implementation details, but since ipmgmtd is calling libipadm, and we don't want to recurse back into libipadm for name-space verification (this is done early on, before any doors are set up so that no other application can invoke ipmgmtd), whether an addrobj name is provided or not is an implementation choice. We could either 1. have ipmgmtd provide an addrobj name, in which case it would have to make function calls to set up the addrobj names 2. not provide any name at all. In either case the library has to be modified to not recurse back into libipadm, which is ok, since this particular configuration method is guaranteed to have no ipadm name-space collision. I've chosen #2 but we could revisit #1 when code review comes around. I don't think this impacts ARC review though. > There is a difference between vnic0/? and vnic0/_a. vnic0/? means that the > address was created outside libipadm/ipmgmtd and therefore cannot be managed > by ipadm(1M) (ipadm show-addrprop, set-addrprop and reset-addrprop). However > vnic0/_a can be managed by ipadm. > > ~Girish From girish.moodalbail@oracle.com Thu May 13 06:52:53 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4DDqq7o023338 for ; Thu, 13 May 2010 06:52:53 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o4DDqpuu035885 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Thu, 13 May 2010 07:52:52 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2D00B0X2K3KY00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 13 May 2010 06:52:51 -0700 (PDT) Received: from sca-ea-mail-3.sun.com ([192.18.43.21]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2D00FB42K0SE90@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 13 May 2010 06:52:48 -0700 (PDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by sca-ea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o4DDqmjj015355 for ; Thu, 13 May 2010 13:52:48 +0000 (GMT) Received: from acsmt355.oracle.com (acsmt355.oracle.com [141.146.40.155]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o4D9GRhI009190 for ; Thu, 13 May 2010 13:52:47 +0000 (GMT) Received: from abhmt013.oracle.com by acsmt353.oracle.com with ESMTP id 261862771273758764; Thu, 13 May 2010 06:52:44 -0700 Received: from [10.7.250.129] (/10.7.250.129) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 13 May 2010 06:52:44 -0700 Date: Thu, 13 May 2010 08:52:42 -0500 From: Girish Moodalbail Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <20100513100807.GA26129@quasimodo.East.Sun.COM> To: sowmini.varadhan@oracle.com Cc: Sebastien Roy , PSARC-ext Message-id: <4BEC042A.5050300@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt355.oracle.com [141.146.40.155] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090202.4BEC042F.00D0:SCFMA4539814,ss=1,fgs=0 References: <4BEB07D6.2040409@oracle.com> <4BEB778A.6010509@oracle.com> <20100513100807.GA26129@quasimodo.East.Sun.COM> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.8) Gecko/20100412 Lightning/1.0b1 Thunderbird/3.0.3 Status: RO Content-Length: 2311 On 05/13/10 05:08 AM, sowmini.varadhan@oracle.com wrote: > On (05/12/10 22:52), Girish Moodalbail wrote: >> >> If the daemon associated with the ip-interface-management service is used to >> configure above addresses on zone boot and address object names were not >> explicitly provided then an address object name will be generated for that >> address by the daemon. >> >> Therefore in your examples above you will never see '?' and instead you will >> see address object names, starting with '_', such as vnic0/_a, vnic0/_b, et >> al. So the example above has to be corrected. > > Actually no. > > this is deep into implementation details, > but since ipmgmtd is calling > libipadm, and we don't want to recurse back into libipadm for name-space > verification (this is done early on, before any doors are set up so > that no other application can invoke ipmgmtd), whether an addrobj name > is provided or not is an implementation choice. We could either > > 1. have ipmgmtd provide an addrobj name, in which case it would have to > make function calls to set up the addrobj names > > 2. not provide any name at all. > > In either case the library has to be modified to not recurse back into > libipadm, which is ok, since this particular configuration method > is guaranteed to have no ipadm name-space collision. > > I've chosen #2 but we could revisit #1 when code review comes around. > I don't think this impacts ARC review though. ipadm(1M) is going to be new ifconfig(1M) and should be used to manage addresses and IP interfaces, going forward. To manage addresses using ipadm(1M), they need to have clearly defined names (either user provided or daemon generated), if not one cannot perform * disable-addr * enable-addr * set-addrprop * reset-addrprop * show-addrprop * delete-addr With the choice made, the address added in the exclusive-ip zone cannot be managed at all. How do you propose to manage that address? Therefore I think it's not an implementation detail but is related to architecture. Further, the daemon is the one that manages namespace and generates unique names. The library libipadm is just a vehicle to retrieve the name. Since you will be creating addresses from within ipmgmtd, you can directly access those API's and create a name. ~Girish From sowmini.varadhan@oracle.com Thu May 13 07:24:13 2010 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4DEODrR024208 for ; Thu, 13 May 2010 07:24:13 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o4DEOD15027601 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Thu, 13 May 2010 07:24:13 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2D00G0N40DGN00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 13 May 2010 08:24:13 -0600 (MDT) Received: from quasimodo.East.Sun.COM ([129.148.174.94]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2D00CW140C8C20@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 13 May 2010 08:24:13 -0600 (MDT) Received: from quasimodo.East.Sun.COM (localhost [127.0.0.1]) by quasimodo.East.Sun.COM (8.14.4+Sun/8.14.4) with ESMTP id o4DEHMgA027284; Thu, 13 May 2010 10:17:22 -0400 (EDT) Received: (from sowmini@localhost) by quasimodo.East.Sun.COM (8.14.4+Sun/8.14.4/Submit) id o4DEHMn9027283; Thu, 13 May 2010 10:17:22 -0400 (EDT) Date: Thu, 13 May 2010 10:17:22 -0400 From: sowmini.varadhan@oracle.com Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <4BEC042A.5050300@oracle.com> To: Girish Moodalbail Cc: Sebastien Roy , PSARC-ext Message-id: <20100513141722.GB27083@quasimodo.East.Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <4BEB07D6.2040409@oracle.com> <4BEB778A.6010509@oracle.com> <20100513100807.GA26129@quasimodo.East.Sun.COM> <4BEC042A.5050300@oracle.com> X-Authentication-warning: quasimodo.East.Sun.COM: sowmini set sender to sowmini.varadhan@oracle.com using -f User-Agent: Mutt/1.5.14 (2007-03-31) Status: RO Content-Length: 1213 On (05/13/10 08:52), Girish Moodalbail wrote: > > ipadm(1M) is going to be new ifconfig(1M) and should be used to manage > addresses and IP interfaces, going forward. To manage addresses using > ipadm(1M), they need to have clearly defined names (either user provided or > daemon generated), if not one cannot perform > * disable-addr > * enable-addr > * set-addrprop > * reset-addrprop > * show-addrprop > * delete-addr > > With the choice made, the address added in the exclusive-ip zone cannot be > managed at all. How do you propose to manage that address? Therefore I think > it's not an implementation detail but is related to architecture. > > Further, the daemon is the one that manages namespace and generates unique > names. The library libipadm is just a vehicle to retrieve the name. Since > you will be creating addresses from within ipmgmtd, you can directly access > those API's and create a name. As I said, it's an implementation detail and I can change it so that ipadm sends in an addrobj. but the fact that addresses created by ifconfig (without an addrobj name) cannot be managed by ipadm is itself a bug that needs to be revisited separately. --Sowmini From girish.moodalbail@oracle.com Thu May 13 07:36:03 2010 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4DEa2FX024336 for ; Thu, 13 May 2010 07:36:03 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o4DEa1Ah014425 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Thu, 13 May 2010 07:36:02 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2D00H034K2GF00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 13 May 2010 08:36:02 -0600 (MDT) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2D00CAY4K18540@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 13 May 2010 08:36:01 -0600 (MDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o4DEa1wu004011 for ; Thu, 13 May 2010 14:36:01 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o4DEOs4f015523 for ; Thu, 13 May 2010 14:35:58 +0000 (GMT) Received: from abhmt005.oracle.com by acsmt354.oracle.com with ESMTP id 261992361273761238; Thu, 13 May 2010 07:33:58 -0700 Received: from [10.7.250.129] (/10.7.250.129) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 13 May 2010 07:33:58 -0700 Date: Thu, 13 May 2010 09:33:56 -0500 From: Girish Moodalbail Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <20100513141722.GB27083@quasimodo.East.Sun.COM> To: sowmini.varadhan@oracle.com Cc: Sebastien Roy , PSARC-ext Message-id: <4BEC0DD4.801@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090203.4BEC0E50.00D4:SCFMA4539814,ss=1,fgs=0 References: <4BEB07D6.2040409@oracle.com> <4BEB778A.6010509@oracle.com> <20100513100807.GA26129@quasimodo.East.Sun.COM> <4BEC042A.5050300@oracle.com> <20100513141722.GB27083@quasimodo.East.Sun.COM> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.8) Gecko/20100412 Lightning/1.0b1 Thunderbird/3.0.3 Status: RO Content-Length: 1751 On 05/13/10 09:17 AM, sowmini.varadhan@oracle.com wrote: > On (05/13/10 08:52), Girish Moodalbail wrote: >> >> ipadm(1M) is going to be new ifconfig(1M) and should be used to manage >> addresses and IP interfaces, going forward. To manage addresses using >> ipadm(1M), they need to have clearly defined names (either user provided or >> daemon generated), if not one cannot perform >> * disable-addr >> * enable-addr >> * set-addrprop >> * reset-addrprop >> * show-addrprop >> * delete-addr >> >> With the choice made, the address added in the exclusive-ip zone cannot be >> managed at all. How do you propose to manage that address? Therefore I think >> it's not an implementation detail but is related to architecture. >> >> Further, the daemon is the one that manages namespace and generates unique >> names. The library libipadm is just a vehicle to retrieve the name. Since >> you will be creating addresses from within ipmgmtd, you can directly access >> those API's and create a name. > > As I said, it's an implementation detail and I can change it so > that ipadm sends in an addrobj. > > but the fact that addresses created by ifconfig (without an addrobj name) > cannot be managed by ipadm is itself a bug that needs to be revisited > separately. That is not true. # ifconfig ce1 plumb # ifconfig ce1 192.168.17.1/24 up # ipadm show-addr ADDROBJ TYPE STATE ADDR lo0/v4 static ok 127.0.0.1/8 ce1/_a static ok 192.168.17.1/24 lo0/v6 static ok ::1/128 ce1/_a an ddress object created by daemon maps to '192.168.17.1/24' created by ifconfig(1M). Since it has a name ipadm(1m) can manage that address. ~Girish From sowmini.varadhan@oracle.com Thu May 13 07:45:09 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4DEj98J024442 for ; Thu, 13 May 2010 07:45:09 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o4DEj6DJ065093 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Thu, 13 May 2010 08:45:08 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2D00M0N4Z8DC00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 13 May 2010 07:45:08 -0700 (PDT) Received: from quasimodo.East.Sun.COM ([129.148.174.94]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2D00F004Z7CBB0@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 13 May 2010 07:45:08 -0700 (PDT) Received: from quasimodo.East.Sun.COM (localhost [127.0.0.1]) by quasimodo.East.Sun.COM (8.14.4+Sun/8.14.4) with ESMTP id o4DEcIsa027293; Thu, 13 May 2010 10:38:18 -0400 (EDT) Received: (from sowmini@localhost) by quasimodo.East.Sun.COM (8.14.4+Sun/8.14.4/Submit) id o4DEcHQU027292; Thu, 13 May 2010 10:38:18 -0400 (EDT) Date: Thu, 13 May 2010 10:38:17 -0400 From: sowmini.varadhan@oracle.com Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <4BEC0DD4.801@oracle.com> To: Girish Moodalbail Cc: Sebastien Roy , PSARC-ext Message-id: <20100513143817.GC27083@quasimodo.East.Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <4BEB07D6.2040409@oracle.com> <4BEB778A.6010509@oracle.com> <20100513100807.GA26129@quasimodo.East.Sun.COM> <4BEC042A.5050300@oracle.com> <20100513141722.GB27083@quasimodo.East.Sun.COM> <4BEC0DD4.801@oracle.com> X-Authentication-warning: quasimodo.East.Sun.COM: sowmini set sender to sowmini.varadhan@oracle.com using -f User-Agent: Mutt/1.5.14 (2007-03-31) Status: RO Content-Length: 556 On (05/13/10 09:33), Girish Moodalbail wrote: > > That is not true. we digress, but.. > > # ifconfig ce1 plumb > # ifconfig ce1 192.168.17.1/24 up > # ipadm show-addr > ADDROBJ TYPE STATE ADDR > lo0/v4 static ok 127.0.0.1/8 > ce1/_a static ok 192.168.17.1/24 > lo0/v6 static ok ::1/128 > > ce1/_a an ddress object created by daemon maps to '192.168.17.1/24' created > by ifconfig(1M). Since it has a name ipadm(1m) can manage that address. great! From edward.pilatowicz@oracle.com Thu May 13 13:25:45 2010 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4DKPjxk001580 for ; Thu, 13 May 2010 13:25:45 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o4DKPiX6023270 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Thu, 13 May 2010 13:25:45 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2D0000HKQXHT00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 13 May 2010 14:25:45 -0600 (MDT) Received: from jurassic.Eng.Sun.COM ([129.146.17.59]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2D007TMKQWB9B0@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 13 May 2010 14:25:44 -0600 (MDT) Received: from jurassic.Eng.Sun.COM (localhost [127.0.0.1]) by jurassic.Eng.Sun.COM (8.14.4+Sun/8.14.4) with ESMTP id o4DKPiwP385684 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 13 May 2010 13:25:44 -0700 (PDT) Received: (from edp@localhost) by jurassic.Eng.Sun.COM (8.14.4+Sun/8.14.4/Submit) id o4DKPi5Q385683; Thu, 13 May 2010 13:25:44 -0700 (PDT) Date: Thu, 13 May 2010 13:25:44 -0700 From: Edward Pilatowicz Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <20100513022313.GB25937@quasimodo.East.Sun.COM> To: sowmini.varadhan@oracle.com Cc: Sebastien Roy , PSARC-ext Message-id: <20100513202544.GB350787@jurassic.Eng.Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <4BEB07D6.2040409@oracle.com> <20100513012624.GJ176566@jurassic.Eng.Sun.COM> <20100513022313.GB25937@quasimodo.East.Sun.COM> X-Authentication-warning: jurassic.Eng.Sun.COM: edp set sender to edward.pilatowicz@oracle.com using -f User-Agent: Mutt/1.5.20 (2009-06-14) Status: RO Content-Length: 4038 On Wed, May 12, 2010 at 07:23:13PM -0700, sowmini.varadhan@oracle.com wrote: > > - this bit seems odd to me: > > > > ---8<--- > > ... Note that IPv4 > > and IPv6 are considered as independent resources, so that > > specification of an IPv4 address via zonecfg(1m) does not place any > > constraints on permissible IPv6 addresses, and vice-versa. > > ---8<--- > > > > ipv4 and ipv6 addresses are considered independent resources from a > > zonecfg(1m) perspective, but from a systems management perspective > > they don't seem that different to me. to me, it seems that once an > > admin decides to restrict what addresses can be used on a given > > interface, that restriction should apply to both ipv4 and ipv6 > > I could go either way on this.. one interpretation of specifying > only ipv4 addresses in zonecfg would be "ipv6-addrs are dont-cares". > Another is that "ipv6-addrs are forbidden". I've somewhat arbitrarily > selected the former interpretation, but could just as easily go with > the latter. > > > addresses. for example, with the current mechanism, if an admin want > > to prevent the zone from configuring any ipv6 address, how do they go > > about doing this? > > Currently, none, though the "only ipv4 specified implies ipv6-addrs > are forbidden" approach solves that. In retrospect, that choices > seems simpler and cleaner. Is that preferable? > i think so. > > - does the existence of a zonecfg(1m) defrouter property on an interface > > prevent a zone from changing the default router for that given > > interface? > > No. > > > - does the existence of a zonecfg(1m) defrouter property on an interface > > prevent a zone from adding additional routes on a given interface? > > No. > > Note that, even though a shared-IP zone cannot add routes today, there's > really nothing that constrains it to using the routes specified in the > routing table- it could easily circumvent the routing table by using > IP source-routes, or by using IP*NEXTHOP or IP*PKTINFO socket options. > So preventing the addition of routes doesn't actually achieve any > limitations. > ok. so this really isn't much of an issue. > > - can exclusive stack zones manipulate arp tables? > yes. > > > > if so, does use of > > the zonecfg(1m) defrouter property restrict arp table manipulation in > > any way? (i'm wondering if arp table manipulation could be used to > > step around any potential routing limitations placed on an interface.) > > there are many ways to step around routing limitations even with shared-IP > today, so not sure what limitations you have in mind. > Basically, regardless of what the routing table has, a node can always > send a packet to any onlink router. > > If what we want, is to prevent the NGZ from DOSing some router(s), then > bandwidth controls are a more reliable way to achieve that. > this last question only really made since in a context where we might try to restrict the routing configuration in a zone. given that shared stack zones have always had the ability to circumvent any predefined routing configuration i see no reason to need to limit arp table manipulation. > > - can exclusive stack zones manipulate mac addresses on network > > interfaces? > > yes- they can use 'ifconfig .. ether <..>'. > > > if so, does the use of the zonecfg(1m) address property > > restrict this in any way? > > no, it does not - the address property only clamps dow the IP address, > and makes no promises about the mac address associated with the IP address. > given that one of the motivation for this work is to prevent zones from using addresses they shouldn't (and there by being capable of DOS-ing hosts using those addresses) it seems like we should have a zonecfg mechanism that prevents mac address manipulation. i don't know if that should be bundled in with this proposed IP limiting mechanism (ie. if a user specifies an IP address the mac would automatically be locked down) or if there should be a seperate knob to control this. thoughts? ed From sowmini.varadhan@oracle.com Thu May 13 13:36:11 2010 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4DKaBPi001840 for ; Thu, 13 May 2010 13:36:11 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o4DKaBPW000551 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Thu, 13 May 2010 13:36:11 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2D00G0DL8BRP00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 13 May 2010 13:36:11 -0700 (PDT) Received: from quasimodo.East.Sun.COM ([129.148.174.94]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2D00EWBL8ASDA0@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 13 May 2010 13:36:10 -0700 (PDT) Received: from quasimodo.East.Sun.COM (localhost [127.0.0.1]) by quasimodo.East.Sun.COM (8.14.4+Sun/8.14.4) with ESMTP id o4DKTKNU027584; Thu, 13 May 2010 16:29:20 -0400 (EDT) Received: (from sowmini@localhost) by quasimodo.East.Sun.COM (8.14.4+Sun/8.14.4/Submit) id o4DKTKSm027583; Thu, 13 May 2010 16:29:20 -0400 (EDT) Date: Thu, 13 May 2010 16:29:20 -0400 From: sowmini.varadhan@oracle.com Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <20100513202544.GB350787@jurassic.Eng.Sun.COM> To: Edward Pilatowicz Cc: Sebastien Roy , PSARC-ext Message-id: <20100513202920.GG27083@quasimodo.East.Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <4BEB07D6.2040409@oracle.com> <20100513012624.GJ176566@jurassic.Eng.Sun.COM> <20100513022313.GB25937@quasimodo.East.Sun.COM> <20100513202544.GB350787@jurassic.Eng.Sun.COM> X-Authentication-warning: quasimodo.East.Sun.COM: sowmini set sender to sowmini.varadhan@oracle.com using -f User-Agent: Mutt/1.5.14 (2007-03-31) Status: RO Content-Length: 1409 On (05/13/10 13:25), Edward Pilatowicz wrote: > > Currently, none, though the "only ipv4 specified implies ipv6-addrs > > are forbidden" approach solves that. In retrospect, that choices > > seems simpler and cleaner. Is that preferable? > > > > i think so. Ok, I'll send out an updated spec (that also incorporates Girish's feedback) later this week. > > > - can exclusive stack zones manipulate mac addresses on network > > > interfaces? > > > > yes- they can use 'ifconfig .. ether <..>'. > > .. the address property only clamps dow the IP address, > > and makes no promises about the mac address associated with the IP address. > > > > given that one of the motivation for this work is to prevent zones from > using addresses they shouldn't (and there by being capable of DOS-ing > hosts using those addresses) it seems like we should have a zonecfg > mechanism that prevents mac address manipulation. i don't know if that > should be bundled in with this proposed IP limiting mechanism (ie. if a > user specifies an IP address the mac would automatically be locked down) > or if there should be a seperate knob to control this. thoughts? Rishi Srivatsavai is looking into the work entailed to have mac-nospoof enabled for NGZ by default.. just talked to Rishi, and I think it makes sense, as part of that work, to also ensure that the mac address cannot be changed by ifconfig. --Sowmini From edward.pilatowicz@oracle.com Thu May 13 14:14:36 2010 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4DLEZIp002617 for ; Thu, 13 May 2010 14:14:35 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o4DLEZaI011576 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Thu, 13 May 2010 14:14:35 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2D00405N0BT800@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 13 May 2010 15:14:35 -0600 (MDT) Received: from jurassic.Eng.Sun.COM ([129.146.17.63]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2D00779N0BB9E0@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 13 May 2010 15:14:35 -0600 (MDT) Received: from jurassic.Eng.Sun.COM (localhost [127.0.0.1]) by jurassic.Eng.Sun.COM (8.14.4+Sun/8.14.4) with ESMTP id o4DLEYpA392119 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 13 May 2010 14:14:34 -0700 (PDT) Received: (from edp@localhost) by jurassic.Eng.Sun.COM (8.14.4+Sun/8.14.4/Submit) id o4DLEYpr392118; Thu, 13 May 2010 14:14:34 -0700 (PDT) Date: Thu, 13 May 2010 14:14:34 -0700 From: Edward Pilatowicz Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <20100513202920.GG27083@quasimodo.East.Sun.COM> To: sowmini.varadhan@oracle.com Cc: Sebastien Roy , PSARC-ext Message-id: <20100513211434.GD350787@jurassic.Eng.Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <4BEB07D6.2040409@oracle.com> <20100513012624.GJ176566@jurassic.Eng.Sun.COM> <20100513022313.GB25937@quasimodo.East.Sun.COM> <20100513202544.GB350787@jurassic.Eng.Sun.COM> <20100513202920.GG27083@quasimodo.East.Sun.COM> X-Authentication-warning: jurassic.Eng.Sun.COM: edp set sender to edward.pilatowicz@oracle.com using -f User-Agent: Mutt/1.5.20 (2009-06-14) Status: RO Content-Length: 1190 On Thu, May 13, 2010 at 01:29:20PM -0700, sowmini.varadhan@oracle.com wrote: > > > > - can exclusive stack zones manipulate mac addresses on network > > > > interfaces? > > > > > > yes- they can use 'ifconfig .. ether <..>'. > > > .. the address property only clamps dow the IP address, > > > and makes no promises about the mac address associated with the IP address. > > > > > > > given that one of the motivation for this work is to prevent zones from > > using addresses they shouldn't (and there by being capable of DOS-ing > > hosts using those addresses) it seems like we should have a zonecfg > > mechanism that prevents mac address manipulation. i don't know if that > > should be bundled in with this proposed IP limiting mechanism (ie. if a > > user specifies an IP address the mac would automatically be locked down) > > or if there should be a seperate knob to control this. thoughts? > > Rishi Srivatsavai is looking into the work entailed to have mac-nospoof > enabled for NGZ by default.. just talked to Rishi, and I think it makes > sense, as part of that work, to also ensure that the mac address cannot > be changed by ifconfig. > sounds good to me. thanks. ed From darren.reed@oracle.com Thu May 13 16:10:05 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4DNA5vt005906 for ; Thu, 13 May 2010 16:10:05 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o4DNA2CQ002662 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Thu, 13 May 2010 18:10:04 -0500 (CDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2D00007SCRAH00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 13 May 2010 16:10:03 -0700 (PDT) Received: from sca-ea-mail-4.sun.com ([192.18.43.22]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2D00L8OSCRO010@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 13 May 2010 16:10:03 -0700 (PDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by sca-ea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o4DNA367008073 for ; Thu, 13 May 2010 23:10:03 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o4DHKfD0030505 for ; Thu, 13 May 2010 23:10:02 +0000 (GMT) Received: from abhmt007.oracle.com by acsmt355.oracle.com with ESMTP id 263583791273792187; Thu, 13 May 2010 16:09:47 -0700 Received: from mintslice.sfbay.sun.com (/129.146.106.55) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 13 May 2010 16:09:47 -0700 Date: Thu, 13 May 2010 16:10:05 -0700 From: Darren Reed Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <20100513202920.GG27083@quasimodo.East.Sun.COM> To: sowmini.varadhan@oracle.com Cc: Edward Pilatowicz , Sebastien Roy , PSARC-ext Message-id: <4BEC86CD.1030408@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090204.4BEC86CB.0046:SCFMA4539814,ss=1,fgs=0 References: <4BEB07D6.2040409@oracle.com> <20100513012624.GJ176566@jurassic.Eng.Sun.COM> <20100513022313.GB25937@quasimodo.East.Sun.COM> <20100513202544.GB350787@jurassic.Eng.Sun.COM> <20100513202920.GG27083@quasimodo.East.Sun.COM> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-GB; rv:1.9.1.5) Gecko/20091206 Thunderbird/3.0 Status: RO Content-Length: 1996 On 13/05/10 01:29 PM, sowmini.varadhan@oracle.com wrote: > On (05/13/10 13:25), Edward Pilatowicz wrote: >>> Currently, none, though the "only ipv4 specified implies ipv6-addrs >>> are forbidden" approach solves that. In retrospect, that choices >>> seems simpler and cleaner. Is that preferable? >>> >> >> i think so. > > Ok, I'll send out an updated spec (that also incorporates Girish's > feedback) later this week. > >>>> - can exclusive stack zones manipulate mac addresses on network >>>> interfaces? >>> >>> yes- they can use 'ifconfig .. ether<..>'. >>> .. the address property only clamps dow the IP address, >>> and makes no promises about the mac address associated with the IP address. >>> >> >> given that one of the motivation for this work is to prevent zones from >> using addresses they shouldn't (and there by being capable of DOS-ing >> hosts using those addresses) it seems like we should have a zonecfg >> mechanism that prevents mac address manipulation. i don't know if that >> should be bundled in with this proposed IP limiting mechanism (ie. if a >> user specifies an IP address the mac would automatically be locked down) >> or if there should be a seperate knob to control this. thoughts? > > Rishi Srivatsavai is looking into the work entailed to have mac-nospoof > enabled for NGZ by default.. just talked to Rishi, and I think it makes > sense, as part of that work, to also ensure that the mac address cannot > be changed by ifconfig. It really doesn't matter what controls you put on changing any address via ifconfig if hostile behaviour is your concern. As long as I can open a raw socket for a NIC, I can pump whatever I like down the wire. To that end, the "allowed-ips" and "mac-nospoof" filtering in mac are required to prevent hostile behaviour from the local zone because they both actively filter all packets transmitted out of the NIC. This is why I earlier asked about whether or not net-rawaccess could be revoked for such zones. Darren From edward.pilatowicz@oracle.com Thu May 13 16:32:49 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4DNWnE5006100 for ; Thu, 13 May 2010 16:32:49 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o4DNWnC6018965 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@Sun.COM>; Thu, 13 May 2010 17:32:49 -0600 (MDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2D00H05TEPOR00@brm-avmta-1.central.sun.com> for PSARC-ext@Sun.COM (ORCPT PSARC-ext@Sun.COM); Thu, 13 May 2010 17:32:49 -0600 (MDT) Received: from jurassic.Eng.Sun.COM ([129.146.17.63]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2D006M0TEOFO40@brm-avmta-1.central.sun.com> for PSARC-ext@Sun.COM (ORCPT PSARC-ext@Sun.COM); Thu, 13 May 2010 17:32:48 -0600 (MDT) Received: from jurassic.Eng.Sun.COM (localhost [127.0.0.1]) by jurassic.Eng.Sun.COM (8.14.4+Sun/8.14.4) with ESMTP id o4DNWmP3408850 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 13 May 2010 16:32:48 -0700 (PDT) Received: (from edp@localhost) by jurassic.Eng.Sun.COM (8.14.4+Sun/8.14.4/Submit) id o4DNWmvd408849; Thu, 13 May 2010 16:32:48 -0700 (PDT) Date: Thu, 13 May 2010 16:32:48 -0700 From: Edward Pilatowicz Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <4BEC86CD.1030408@oracle.com> To: Darren Reed Cc: sowmini.varadhan@oracle.com, Sebastien Roy , PSARC-ext Message-id: <20100513233247.GB399464@jurassic.Eng.Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <4BEB07D6.2040409@oracle.com> <20100513012624.GJ176566@jurassic.Eng.Sun.COM> <20100513022313.GB25937@quasimodo.East.Sun.COM> <20100513202544.GB350787@jurassic.Eng.Sun.COM> <20100513202920.GG27083@quasimodo.East.Sun.COM> <4BEC86CD.1030408@oracle.com> X-Authentication-warning: jurassic.Eng.Sun.COM: edp set sender to edward.pilatowicz@oracle.com using -f User-Agent: Mutt/1.5.20 (2009-06-14) Status: RO Content-Length: 2512 On Thu, May 13, 2010 at 04:10:05PM -0700, Darren Reed wrote: > On 13/05/10 01:29 PM, sowmini.varadhan@oracle.com wrote: > >On (05/13/10 13:25), Edward Pilatowicz wrote: > >>>Currently, none, though the "only ipv4 specified implies ipv6-addrs > >>>are forbidden" approach solves that. In retrospect, that choices > >>>seems simpler and cleaner. Is that preferable? > >>> > >> > >>i think so. > > > >Ok, I'll send out an updated spec (that also incorporates Girish's > >feedback) later this week. > > > >>>>- can exclusive stack zones manipulate mac addresses on network > >>>> interfaces? > >>> > >>> yes- they can use 'ifconfig .. ether<..>'. > >>>.. the address property only clamps dow the IP address, > >>>and makes no promises about the mac address associated with the IP address. > >>> > >> > >>given that one of the motivation for this work is to prevent zones from > >>using addresses they shouldn't (and there by being capable of DOS-ing > >>hosts using those addresses) it seems like we should have a zonecfg > >>mechanism that prevents mac address manipulation. i don't know if that > >>should be bundled in with this proposed IP limiting mechanism (ie. if a > >>user specifies an IP address the mac would automatically be locked down) > >>or if there should be a seperate knob to control this. thoughts? > > > >Rishi Srivatsavai is looking into the work entailed to have mac-nospoof > >enabled for NGZ by default.. just talked to Rishi, and I think it makes > >sense, as part of that work, to also ensure that the mac address cannot > >be changed by ifconfig. > > It really doesn't matter what controls you put on changing any > address via ifconfig if hostile behaviour is your concern. As long > as I can open a raw socket for a NIC, I can pump whatever I like > down the wire. To that end, the "allowed-ips" and "mac-nospoof" > filtering in mac are required to prevent hostile behaviour from > the local zone because they both actively filter all packets > transmitted out of the NIC. This is why I earlier asked about > whether or not net-rawaccess could be revoked for such zones. > as far as i can tell, if "allowed-ips" and "mac-nospoof" are used to restrict the ip and mac addresses that a zone can use then that's good enough. removing net-rawaccess would be unnecessary because it wouldn't buy us any more protection else. removing net-rawaccess would actually reduce the available functionality in the zone unnecessarily. (for example, the zone could no longer run snoop.) ed From sebastien.roy@oracle.com Thu May 13 17:17:53 2010 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4E0HrCT006841 for ; Thu, 13 May 2010 17:17:53 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o4E0HrIJ015984 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Thu, 13 May 2010 17:17:53 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2D00401VHT5P00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 13 May 2010 17:17:53 -0700 (PDT) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2D00LG3VHSO140@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 13 May 2010 17:17:53 -0700 (PDT) Received: from rcsinet15.oracle.com (rcsinet15.oracle.com [148.87.113.117]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o4E0HqKg004012 for ; Fri, 14 May 2010 00:17:52 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by rcsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o4D8Qgve002840 for ; Fri, 14 May 2010 00:17:52 +0000 (GMT) Received: from abhmt006.oracle.com by acsmt355.oracle.com with ESMTP id 240146561273796255; Thu, 13 May 2010 17:17:35 -0700 Received: from [129.148.19.4] (/129.148.19.4) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 13 May 2010 17:17:35 -0700 Date: Thu, 13 May 2010 20:17:33 -0400 From: Sebastien Roy Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <20100513233247.GB399464@jurassic.Eng.Sun.COM> To: Edward Pilatowicz Cc: Darren Reed , sowmini.varadhan@oracle.com, PSARC-ext Message-id: <4BEC969D.7020300@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090205.4BEC96B0.007E:SCFMA4539814,ss=1,fgs=0 References: <4BEB07D6.2040409@oracle.com> <20100513012624.GJ176566@jurassic.Eng.Sun.COM> <20100513022313.GB25937@quasimodo.East.Sun.COM> <20100513202544.GB350787@jurassic.Eng.Sun.COM> <20100513202920.GG27083@quasimodo.East.Sun.COM> <4BEC86CD.1030408@oracle.com> <20100513233247.GB399464@jurassic.Eng.Sun.COM> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.8) Gecko/20100412 Lightning/1.0b1 Thunderbird/3.0.3 Status: RO Content-Length: 2590 On 05/13/10 07:32 PM, Edward Pilatowicz wrote: > On Thu, May 13, 2010 at 04:10:05PM -0700, Darren Reed wrote: >> On 13/05/10 01:29 PM, sowmini.varadhan@oracle.com wrote: >>> Rishi Srivatsavai is looking into the work entailed to have mac-nospoof >>> enabled for NGZ by default.. just talked to Rishi, and I think it makes >>> sense, as part of that work, to also ensure that the mac address cannot >>> be changed by ifconfig. >> >> It really doesn't matter what controls you put on changing any >> address via ifconfig if hostile behaviour is your concern. As long >> as I can open a raw socket for a NIC, I can pump whatever I like >> down the wire. To that end, the "allowed-ips" and "mac-nospoof" >> filtering in mac are required to prevent hostile behaviour from >> the local zone because they both actively filter all packets >> transmitted out of the NIC. This is why I earlier asked about >> whether or not net-rawaccess could be revoked for such zones. >> > > as far as i can tell, if "allowed-ips" and "mac-nospoof" are used to > restrict the ip and mac addresses that a zone can use then that's good > enough. removing net-rawaccess would be unnecessary because it wouldn't > buy us any more protection else. removing net-rawaccess would actually > reduce the available functionality in the zone unnecessarily. (for > example, the zone could no longer run snoop.) Yes, and I also look at this from a resource management perspective. If I explicitly assign an IP address and a MAC address to a zone, then it implies that I don't want processes in that zone to steal (intentionally or not) the resources of another zone or system and "spoof" another zone or system by using a different address. At least then, if anything does happen on a network as a result of packets transmitted from the zone, it will be accountable. I don't see this as a "hostile behaviour" prevention mechanism, as that problem isn't solvable; it's not a bounded problem at all if you allow any bits to flow out of the confines of the zone's environment. Generally, the scope of the allowed-ips and mac-nospoof protection settings is indeed simply to prevent spoofing, and not to prevent all possible network-based malicious attacks, so those are appropriate mechanisms to accomplish the above goal. This case (and the future case that will propose to enable mac-nospoof for links assigned to non-global zones) is partly about automating the configuration of these settings in a way that they were originally intended (and providing appropriate administrative error semantics). -Seb From darren.reed@oracle.com Thu May 13 18:03:31 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4E13VoJ007348 for ; Thu, 13 May 2010 18:03:31 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o4E13U7o016768 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Thu, 13 May 2010 20:03:31 -0500 (CDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2D00605XLUUJ00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 13 May 2010 18:03:30 -0700 (PDT) Received: from brmea-mail-2.sun.com ([192.18.98.43]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2D00LFGXLTO050@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 13 May 2010 18:03:30 -0700 (PDT) Received: from rcsinet15.oracle.com (rcsinet15.oracle.com [148.87.113.117]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o4E13T9T011071 for ; Fri, 14 May 2010 01:03:29 +0000 (GMT) Received: from acsmt353.oracle.com (acsmt353.oracle.com [141.146.40.153]) by rcsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o4DMN4an001618 for ; Fri, 14 May 2010 01:03:28 +0000 (GMT) Received: from abhmt002.oracle.com by acsmt354.oracle.com with ESMTP id 240218691273798898; Thu, 13 May 2010 18:01:38 -0700 Received: from mintslice.sfbay.sun.com (/129.146.106.55) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 13 May 2010 18:01:37 -0700 Date: Thu, 13 May 2010 18:01:54 -0700 From: Darren Reed Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <20100513233247.GB399464@jurassic.Eng.Sun.COM> To: Edward Pilatowicz Cc: sowmini.varadhan@oracle.com, Sebastien Roy , PSARC-ext Message-id: <4BECA102.2020306@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt353.oracle.com [141.146.40.153] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090203.4BECA160.012D:SCFMA4539814,ss=1,fgs=0 References: <4BEB07D6.2040409@oracle.com> <20100513012624.GJ176566@jurassic.Eng.Sun.COM> <20100513022313.GB25937@quasimodo.East.Sun.COM> <20100513202544.GB350787@jurassic.Eng.Sun.COM> <20100513202920.GG27083@quasimodo.East.Sun.COM> <4BEC86CD.1030408@oracle.com> <20100513233247.GB399464@jurassic.Eng.Sun.COM> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-GB; rv:1.9.1.5) Gecko/20091206 Thunderbird/3.0 Status: RO Content-Length: 2744 On 13/05/10 04:32 PM, Edward Pilatowicz wrote: > On Thu, May 13, 2010 at 04:10:05PM -0700, Darren Reed wrote: >> On 13/05/10 01:29 PM, sowmini.varadhan@oracle.com wrote: >>> On (05/13/10 13:25), Edward Pilatowicz wrote: >>>>> Currently, none, though the "only ipv4 specified implies ipv6-addrs >>>>> are forbidden" approach solves that. In retrospect, that choices >>>>> seems simpler and cleaner. Is that preferable? >>>>> >>>> >>>> i think so. >>> >>> Ok, I'll send out an updated spec (that also incorporates Girish's >>> feedback) later this week. >>> >>>>>> - can exclusive stack zones manipulate mac addresses on network >>>>>> interfaces? >>>>> >>>>> yes- they can use 'ifconfig .. ether<..>'. >>>>> .. the address property only clamps dow the IP address, >>>>> and makes no promises about the mac address associated with the IP address. >>>>> >>>> >>>> given that one of the motivation for this work is to prevent zones from >>>> using addresses they shouldn't (and there by being capable of DOS-ing >>>> hosts using those addresses) it seems like we should have a zonecfg >>>> mechanism that prevents mac address manipulation. i don't know if that >>>> should be bundled in with this proposed IP limiting mechanism (ie. if a >>>> user specifies an IP address the mac would automatically be locked down) >>>> or if there should be a seperate knob to control this. thoughts? >>> >>> Rishi Srivatsavai is looking into the work entailed to have mac-nospoof >>> enabled for NGZ by default.. just talked to Rishi, and I think it makes >>> sense, as part of that work, to also ensure that the mac address cannot >>> be changed by ifconfig. >> >> It really doesn't matter what controls you put on changing any >> address via ifconfig if hostile behaviour is your concern. As long >> as I can open a raw socket for a NIC, I can pump whatever I like >> down the wire. To that end, the "allowed-ips" and "mac-nospoof" >> filtering in mac are required to prevent hostile behaviour from >> the local zone because they both actively filter all packets >> transmitted out of the NIC. This is why I earlier asked about >> whether or not net-rawaccess could be revoked for such zones. >> > > as far as i can tell, if "allowed-ips" and "mac-nospoof" are used to > restrict the ip and mac addresses that a zone can use then that's good > enough. removing net-rawaccess would be unnecessary because it wouldn't > buy us any more protection else. removing net-rawaccess would actually > reduce the available functionality in the zone unnecessarily. (for > example, the zone could no longer run snoop.) For snoop/tcpdump, the net_observabilty privilege exists (which allows read-only access to NICs.) The other problem is tracroute. Darren From sowmini.varadhan@oracle.com Fri May 14 07:27:50 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4EERn1Q010008 for ; Fri, 14 May 2010 07:27:50 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o4EERne4008333 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Fri, 14 May 2010 09:27:49 -0500 (CDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2E0020DYUDD100@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 14 May 2010 07:27:49 -0700 (PDT) Received: from quasimodo.East.Sun.COM ([129.148.174.94]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2E00H2GYUAUO80@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 14 May 2010 07:27:47 -0700 (PDT) Received: from quasimodo.East.Sun.COM (localhost [127.0.0.1]) by quasimodo.East.Sun.COM (8.14.4+Sun/8.14.4) with ESMTP id o4EEKuV6000516; Fri, 14 May 2010 10:20:56 -0400 (EDT) Received: (from sowmini@localhost) by quasimodo.East.Sun.COM (8.14.4+Sun/8.14.4/Submit) id o4EEKuD9000515; Fri, 14 May 2010 10:20:56 -0400 (EDT) Date: Fri, 14 May 2010 10:20:56 -0400 From: sowmini.varadhan@oracle.com Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <20100513202920.GG27083@quasimodo.East.Sun.COM> To: Edward Pilatowicz Cc: PSARC-ext Message-id: <20100514142056.GA249@quasimodo.East.Sun.COM> MIME-version: 1.0 Content-type: multipart/mixed; boundary="Boundary_(ID_Mddb/FC1sBhUeOx4+0q8LA)" Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <4BEB07D6.2040409@oracle.com> <20100513012624.GJ176566@jurassic.Eng.Sun.COM> <20100513022313.GB25937@quasimodo.East.Sun.COM> <20100513202544.GB350787@jurassic.Eng.Sun.COM> <20100513202920.GG27083@quasimodo.East.Sun.COM> X-Authentication-warning: quasimodo.East.Sun.COM: sowmini set sender to sowmini.varadhan@oracle.com using -f User-Agent: Mutt/1.5.14 (2007-03-31) Status: RO Content-Length: 17049 --Boundary_(ID_Mddb/FC1sBhUeOx4+0q8LA) Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline > On (05/13/10 13:25), Edward Pilatowicz wrote: > > > Currently, none, though the "only ipv4 specified implies ipv6-addrs > > > are forbidden" approach solves that. In retrospect, that choices > > > seems simpler and cleaner. Is that preferable? > > > > > > > i think so. On (05/13/10 16:29), sowmini.varadhan@oracle.com wrote: > Ok, I'll send out an updated spec (that also incorporates Girish's > feedback) later this week. As promised.. the new spec is attached. The diffs are --- zonecfg.txt 2010/05/14 13:31:01 1.1 +++ zonecfg.txt 2010/05/14 14:25:02 @@ -49,22 +49,19 @@ that exist today ([PSARC/2002/174], [PSARC/2003/621], [PSARC/2008/057]) When the global-zone specifies the 'address' for an interface via -zonecfg(1m), the non-global zone may not use any other addresses for -the specified IP Version on that interface. The address information -provided via zonecfg(1m) will be used to set up Layer-3 protection -[PSARC/2009/436] for the non-global zone during zone-boot to filter -out all other addresses for the selected interface. For instance, when -zonecfg(1m) has been used in the global-zone to set one or more IPv4 -addresses on an interface, an attempt to set an IPv4 address on the -interface that is outside the globally defined set will encounter the -EPERM failure. Thus ifconfig(1m), ipadm(1m), and associated ioctls -will receive this error if they are used within the non-global zone to -set addresses that are not in the set that is permitted from the -global-zone, and attempts by the non-global zone to turn on forwarding -on the interface will also encounter EPERM. Note that IPv4 and IPv6 -are considered as independant resources, so that specification of an -IPv4 address via zonecfg(1m) does not place any constraints on -permissible IPv6 addresses, and vice-versa. +zonecfg(1m), the non-global zone may not use any other IP addresses for +that interface. The address information provided via zonecfg(1m) will +be used to set up Layer-3 protection [PSARC/2009/436] for the +non-global zone during zone-boot to filter out all other addresses for +the selected interface. For instance, when zonecfg(1m) has been used +in the global-zone to set one or more IPv4 addresses on an interface, +an attempt to set an IPv4 or IPv6 address on the interface that is +outside the globally defined set will encounter the EPERM failure. +Thus ifconfig(1m), ipadm(1m), and associated ioctls will receive this +error if they are used within the non-global zone to set addresses that +are not in the set that is permitted from the global-zone, and attempts +by the non-global zone to turn on forwarding on the interface will also +encounter EPERM. Attempts to boot a zone that has already been configured for IP, or has previously customized values for Layer-3 protection [PSARC/2009/436] @@ -194,8 +191,8 @@ root@gz# zlogin tz1 ipadm show-addr ADDROBJ TYPE STATE ADDR - vnic0/? static ok 11.1.1.1/24 - vnic0/? static ok 12.2.3.4/16 + vnic0/_a static ok 11.1.1.1/24 + vnic0/_b static ok 12.2.3.4/16 lo0/v4 static ok 127.0.0.1/8 lo0/v6 static ok ::1/128 @@ -232,22 +229,18 @@ + and the non-global zone will not be able to apply any other + addresses to that interface. -zhadum(866)% diff -wub dladm.1m.txt dladm.1m.txt.new --- dladm.1m.txt Mon May 10 15:24:49 2010 -+++ dladm.1m.txt.new Mon May 10 15:28:09 2010 -@@ -4176,7 +4176,17 @@ ++++ dladm.1m.txt.new Fri May 14 10:24:02 2010 +@@ -4176,7 +4176,14 @@ option's link layer address must match the link's MAC address. + When a datalink has been protected by setting allowed-ips -+ to a set of one or more IPv4 addresses, any attempts to -+ configure IPv4 addresses that are not in this set will -+ fail with an EPERM error being returned to the user. -+ Moreover, the interface may not be used for forwarding -+ IPv4 packets, and attempts to set the ipadm(1m) forwarding -+ property on the interface will encounter EPERM error. -+ Analogous constraints on address configuration and -+ forwarding settings also apply for IPv6 packets. ++ to a set of one or more IP addresses, IP interfaces over ++ that datalink may not configure IP addresses outside the ++ allowed-ips set, or enable forwarding for the IP interface ++ and attempts to do so will fail with an EPERM error being ++ returned to the user. + dhcp-nospoof @@ -255,7 +248,6 @@ DHCP client ID (DUID for DHCPv6) and hardware --Boundary_(ID_Mddb/FC1sBhUeOx4+0q8LA) Content-type: text/plain; NAME=zonecfg.txt; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: attachment; filename=zonecfg.txt zonecfg(1m) enhancements for exclusive-IP zones Requested release binding: Minor Summary: ------- This case proposes a solution for 6944327 need to support address and defrouter resources for exclusive-IP zones Problem Description -------------------- Typical zone deployments that exist today use shared-IP zones to run applications and services like Apache or Weblogic in the contained environment provided by the shared-IP zone. In these use-cases, the Administrator in the global zone has full control over the networking resources used by the non-global zone. In the common case, networking is simply configured by specifying the IP interface, IP addresses and, optionally, the default routers from zonecfg(1m). The configuration resources thus supplied are then applied for the non-global zone when it is booted, and the non-global zone itself may not modify any of these configuration parameters. However, there is no such simple configuration mechanism for the simple networking use-case in place for exclusive IP zones, which must be configured through sysidcfg, ifconfig, and an assortment of other methods, all of which are not controllable from the global zone, and may not be managed through zonecfg(1m). The addition of many new virtualization and resource management features to Solaris (PSARC 2006/357, PSARC 2005/132 and associated cases) makes Exclusive-IP zones a cleaner and more powerful Zone model than shared-IP zones. Thus, bridging any gaps in the configuration methods for the simple use cases between shared- and exlusive-IP zones is important to ease the transition of shared-IP customer configurations to exlusive-IP. In addition, there are many ongoing projects in the "Zones Networking" effort [ZONES-NET] to facilitate the consolidation of existing Solaris 10 host installations as Solaris 10 Containers. These efforts would leverage from the ability to specify networking resource values for address and default router uniformly for zones using zonecfg(1m). This case proposes support in zonecfg(1m) for 'address' and 'defrouter' properties in the 'net' resource for exclusive-IP zones. The semantics of the values will be the same as with the shared-IP zone definitions that exist today ([PSARC/2002/174], [PSARC/2003/621], [PSARC/2008/057]) When the global-zone specifies the 'address' for an interface via zonecfg(1m), the non-global zone may not use any other IP addresses for that interface. The address information provided via zonecfg(1m) will be used to set up Layer-3 protection [PSARC/2009/436] for the non-global zone during zone-boot to filter out all other addresses for the selected interface. For instance, when zonecfg(1m) has been used in the global-zone to set one or more IPv4 addresses on an interface, an attempt to set an IPv4 or IPv6 address on the interface that is outside the globally defined set will encounter the EPERM failure. Thus ifconfig(1m), ipadm(1m), and associated ioctls will receive this error if they are used within the non-global zone to set addresses that are not in the set that is permitted from the global-zone, and attempts by the non-global zone to turn on forwarding on the interface will also encounter EPERM. Attempts to boot a zone that has already been configured for IP, or has previously customized values for Layer-3 protection [PSARC/2009/436] will fail. Implementation Overview: ------------------------ A brief overview of the implementation is provided here. Note that all interfaces between zoneadmd(1m) and the kernel/zonecfg(1m), are Private interfaces, subject to change in the future. When the non-global zone is booted, zoneadmd(1m) will store the information specified by the 'address' and 'defrouter' properties as nvlists in the kernel following a mechanism similar to that in use today for other zonecfg(1m) resources such as the "physical" datalink. At the same time, the zone boot process will also ensure 'ip-nospoof' protection for the datalink with the specified addresses used as input to the 'allowed-ips' property. The stored nvlist information for 'address' and 'defrouter' will be retrieved and re-applied in the non-global zone by the daemon associated with the ip-interface-management service before any other IP configuration is applied. The GLDv3 layer communicates the set of "allowed-ips" to the IP layer through DLPI notifications. The notification sent is a DL_NOTIFY_IND message of (Project Private) type DL_NOTE_ALLOWED_IPS, sent to the IP clients that have registered for this notification. The payload of the DL_NOTE_ALLOWED_IPS message is the mac_protect_t data-structure introduced by PSARC 2009/436. Receipt of this notification enables the IP layer to track the current set of permitted IP addresses per interface, so that the IP layer can return informative error diagnostics if an attempt is made to enable forwarding or set addresses on the interface that are outside the permitted set. All configuration state, including datalink properties like protection, allowed-ips and zone, and associated nvlist information, that is created during zone boot will be reset to the default values when the zone is halted. Relationship to other configuration mechanisms ---------------------------------------------- sysidcfg: The Solaris Install team is currently working in conjunction with the Network Configuration team [INSTALL] on the implementation of a framework to replace sysidcfg(4) that will provide configuration profiles to be processed by startup scripts when a system boots. However this configuration information is only applied on the first boot of the zone, and unlike zonecfg, may not be easily modified in the global zone across reboot. Moreover, there is no mechanism in place to prevent the non-global zone from modifying the configuration provided via profiles. As mentioned earlier, the global-zone is the authoritative source of IP configuration information for the "secure container" use-case. This implies that attempts to use boot profiles in the non-global zone to add addresses other than those mandated in the global-zone will fail with EINVAL erors. Note that the profiles defined in [INSTALL] may still be applied to interfaces for which 'net' resource properties have not been defined in the global-zone, i.e., this proposal is fully backward-compatible. svc:/network/physical:{default, nwam} Configuration specified via zonecfg(1m) in the global-zone overrides any information specified via /etc/hostname.intf, ipadm(1m) or nwam. Thus if the IP address information specified by these other methods is not a subset of the globally managed information, attempts to apply it within the non-global zone will encounter an error. DHCP: It is assumed that when IP resources for an IP interface have been set up in the global-zone using zonecfg(1m), then that interface has been marked for static IP configuration only, and attempts to send out outgoing DHCP DISCOVER packets will be dropped in GLDv3. Thus attempts to start DHCP on these interfaces in the non-global zone will time out. Examples -------- root@gz# zonecfg -z tz1 zonecfg:tz1> info zonename: tz1 zonepath: /rpool/zones/tz1 brand: ipkg autoboot: false bootargs: pool: limitpriv: scheduling-class: ip-type: exclusive hostid: net: address: 11.1.1.1/24 physical: vnic0 defrouter: 11.1.1.2 net: address: taddr2 physical: vnic0 defrouter not specified net: address not specified physical: vnic1 defrouter not specified In this example, the zone 'tz1' has been assigned 2 interfaces: vnic0 and vnic1. The vnic1 interface does not have any IP address or default router assigned to it, so that it may be configured in any desired way from within the zone using ifconfig(1m), ipadm(1m), NWAM etc. The vnic0 interface has 2 addresses assigned: 11.1.1.1/24 (with default router 11.1.1.2 on the 11.1.1.0/24 subnet) and the address 'taddr2' whose numeric value and netmask will be resolved when the zone boots. Thus, if, at the time of zone boot, the global-zone is also configured as: root@gz# egrep 'hosts|netmask' /etc/nsswitch.conf hosts: files nis dns netmasks: files nis root@gz# grep taddr2 /etc/hosts 12.2.3.4 taddr2 root@gz# grep 12 /etc/netmasks 12.2.0.0 255.255.0.0 tz1 will be booted as root@gz# zoneadm -z tz1 boot root@gz# zlogin tz1 ipadm show-addr ADDROBJ TYPE STATE ADDR vnic0/_a static ok 11.1.1.1/24 vnic0/_b static ok 12.2.3.4/16 lo0/v4 static ok 127.0.0.1/8 lo0/v6 static ok ::1/128 root@gz# zlogin tz1 netstat -rn Routing Table: IPv4 Destination Gateway Flags Ref Use Interface -------------------- -------------------- ----- ----- ---------- --------- default 11.1.1.2 UG 2 2 vnic0 11.1.1.0 11.1.1.1 U 3 2 vnic0 12.2.0.0 12.2.3.4 U 2 0 vnic0 127.0.0.1 127.0.0.1 UH 2 0 lo0 Routing Table: IPv6 Destination/Mask Gateway Flags Ref Use If --------------------------- ---------------------- ----- --- ------- ----- ::1 ::1 UH 2 0 lo0 Proposed man page changes: -------------------------- --- zonecfg.1m.orig Thu Apr 22 11:22:12 2010 +++ zonecfg.1m.new Thu Apr 22 11:27:48 2010 @@ -676,8 +676,13 @@ for the interface should be specified here. For an exclusive-IP zone, the physical property must be - set and the address and default router properties cannot - be set. + set and the address and default router properties may + be set. The interface specifed for the physical property + must not be in use in the global zone. If an address and + default router are specified via zonecfg(1m), these will be + applied to the interface when the non-global zone is booted, + and the non-global zone will not be able to apply any other + addresses to that interface. --- dladm.1m.txt Mon May 10 15:24:49 2010 +++ dladm.1m.txt.new Fri May 14 10:24:02 2010 @@ -4176,7 +4176,14 @@ option's link layer address must match the link's MAC address. + When a datalink has been protected by setting allowed-ips + to a set of one or more IP addresses, IP interfaces over + that datalink may not configure IP addresses outside the + allowed-ips set, or enable forwarding for the IP interface + and attempts to do so will fail with an EPERM error being + returned to the user. + dhcp-nospoof DHCP client ID (DUID for DHCPv6) and hardware References: ---------- [PSARC/2002/174] Virtualization and Namespace Isolation in Solaris, aka "Zones" [PSARC/2003/621] Zone Administration Updates [PSARC/2008/057] Default Route For Zones [PSARC/2009/436] Anti-spoofing Link Protection [INSTALL] http://solaris-networking.sfbay/index.php/Network_Configuration:install [ZONES-NET] http://solaris-networking.sfbay/index.php/Zones_Networking --Boundary_(ID_Mddb/FC1sBhUeOx4+0q8LA)-- From erik.nordmark@oracle.com Mon May 17 14:47:13 2010 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4HLlDQx010968 for ; Mon, 17 May 2010 14:47:13 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o4HLl6dj022177 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Mon, 17 May 2010 14:47:13 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2L0031336NYN00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 17 May 2010 14:47:11 -0700 (PDT) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2L0008B36NLZ50@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 17 May 2010 14:47:11 -0700 (PDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o4HLlAdO000618 for ; Mon, 17 May 2010 21:47:10 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o4HKMcLi031046 for ; Mon, 17 May 2010 21:47:09 +0000 (GMT) Received: from abhmt017.oracle.com by acsmt353.oracle.com with ESMTP id 272218101274132748; Mon, 17 May 2010 14:45:48 -0700 Received: from [10.7.251.248] (/10.7.251.248) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 17 May 2010 14:45:48 -0700 Date: Mon, 17 May 2010 14:45:39 -0700 From: Erik Nordmark Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <4BEB20EC.70408@oracle.com> To: Darren Reed Cc: sowmini.varadhan@oracle.com, PSARC-ext Message-id: <4BF1B903.30209@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090209.4BF1B95E.0048:SCFMA4539814,ss=1,fgs=0 References: <4BEB07D6.2040409@oracle.com> <4BEB20EC.70408@oracle.com> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.8) Gecko/20100412 Lightning/1.0b1 Thunderbird/3.0.3 Status: RO Content-Length: 1008 On 05/12/10 02:43 PM, Darren Reed wrote: > At the bottom of this document, the man page update mentions > that this feature is incompatible with enabling forwarding. Why? > Shouldn't we be engineering Solaris to make features compatible > with each other rather than incompatible? > Yes, it does. The use of "allowed-ips", today, is not incompatible > with a zone functioning as a router and nor is configuring a > zone with an exclusive instance of IP incompatible. Nor does > the operation of either feature introduce any specific errors > or failure conditions that preclude forwarding from functioning. allowed-ips today is incompatible with IP forwarding; this case doesn't propose changing that. allowed-ips is similar to Ethernet switch technology (such as Cisco source guard or the IETF work in the SAVI WG) in this respect; you have to explicitly configure the device/function which performs the filtering to not do any filtering for the packets that are coming from an IP router. Erik From sebastien.roy@oracle.com Wed May 19 08:12:30 2010 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4JFCU4f004087 for ; Wed, 19 May 2010 08:12:30 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o4JFCSqv003266 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 19 May 2010 08:12:29 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2O00KANA8S1B00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 19 May 2010 09:12:28 -0600 (MDT) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2O00H7XA8RHM20@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 19 May 2010 09:12:27 -0600 (MDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o4JFCRvv009991 for ; Wed, 19 May 2010 15:12:27 +0000 (GMT) Received: from acsmt355.oracle.com (acsmt355.oracle.com [141.146.40.155]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o4JBED6x020661 for ; Wed, 19 May 2010 15:12:26 +0000 (GMT) Received: from abhmt001.oracle.com by acsmt355.oracle.com with ESMTP id 250809931274281825; Wed, 19 May 2010 08:10:25 -0700 Received: from [129.148.19.4] (/129.148.19.4) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 19 May 2010 08:10:24 -0700 Date: Wed, 19 May 2010 11:10:23 -0400 From: Sebastien Roy Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <4BEB07D6.2040409@oracle.com> To: PSARC-ext Cc: sowmini.varadhan@oracle.com Message-id: <4BF3FF5F.60701@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt355.oracle.com [141.146.40.155] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090201.4BF3FFDA.016C:SCFMA4539814,ss=1,fgs=0 References: <4BEB07D6.2040409@oracle.com> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.8) Gecko/20100412 Lightning/1.0b1 Thunderbird/3.0.3 Status: RO Content-Length: 197 Note to PSARC members: This case times out today. It has received adequate review from members of both the networking and the zones teams, but has yet to receive a +1 from a PSARC member. -Seb From erik.nordmark@oracle.com Wed May 19 22:38:55 2010 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4K5ctHU022546 for ; Wed, 19 May 2010 22:38:55 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o4K5ctdn025175 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 19 May 2010 22:38:55 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2P00C03ECVI700@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 19 May 2010 22:38:55 -0700 (PDT) Received: from brmea-mail-2.sun.com ([192.18.98.43]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2P00MP6ECUHFB0@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 19 May 2010 22:38:54 -0700 (PDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o4K5csDP021267 for ; Thu, 20 May 2010 05:38:54 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o4JAQSTO016863 for ; Thu, 20 May 2010 05:38:52 +0000 (GMT) Received: from abhmt015.oracle.com by acsmt354.oracle.com with ESMTP id 252826571274333854; Wed, 19 May 2010 22:37:34 -0700 Received: from [10.7.251.248] (/10.7.251.248) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 19 May 2010 22:37:33 -0700 Date: Wed, 19 May 2010 22:37:29 -0700 From: Erik Nordmark Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <20100514142056.GA249@quasimodo.East.Sun.COM> To: sowmini.varadhan@oracle.com Cc: Edward Pilatowicz , PSARC-ext Message-id: <4BF4CA99.6070409@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090206.4BF4CAED.00D9:SCFMA4539814,ss=1,fgs=0 References: <4BEB07D6.2040409@oracle.com> <20100513012624.GJ176566@jurassic.Eng.Sun.COM> <20100513022313.GB25937@quasimodo.East.Sun.COM> <20100513202544.GB350787@jurassic.Eng.Sun.COM> <20100513202920.GG27083@quasimodo.East.Sun.COM> <20100514142056.GA249@quasimodo.East.Sun.COM> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.8) Gecko/20100412 Lightning/1.0b1 Thunderbird/3.0.3 Status: RO Content-Length: 2224 On 05/14/10 07:20 AM, sowmini.varadhan@oracle.com wrote: >> On (05/13/10 13:25), Edward Pilatowicz wrote: >>>> Currently, none, though the "only ipv4 specified implies ipv6-addrs >>>> are forbidden" approach solves that. In retrospect, that choices >>>> seems simpler and cleaner. Is that preferable? >>>> >>> >>> i think so. > > On (05/13/10 16:29), sowmini.varadhan@oracle.com wrote: >> Ok, I'll send out an updated spec (that also incorporates Girish's >> feedback) later this week. > > As promised.. the new spec is attached. The diffs are The behavior as seen from the GZ looks fine. But staring at the NGZ we have: > root@gz# zlogin tz1 ipadm show-addr > ADDROBJ TYPE STATE ADDR > vnic0/_a static ok 11.1.1.1/24 > vnic0/_b static ok 12.2.3.4/16 > lo0/v4 static ok 127.0.0.1/8 > lo0/v6 static ok ::1/128 This looks identical to the case when vnic0 was configured locally in the ngz (that is what "static" means). Hence the NGZ admin have no reason to believe it isn't possible to do a ipadm delete-addr vnic0/_a. But either such a delete fails, or after a reboot the address magically reappers; either of which would be a surprise to the ngz admin since the addresses are marked as "static". These addresses provided and enforced by the GZ are provided by an external entity (the GZ) the same way as the addresses in a dhcp or addrconf address object are provided by an external entity (the dhcp server and stateless addrconf config on the routers, respectively. Thus I think it makes sense to label them as a different type of address object to make this clear to the ngz admin I don't know what a good name would be - we used "from_gz" when we talked about this on the phone. That way the ngz admin will see something like vnic0/_a from_gz ok 11.1.1.1/24 vnic0/_b from_gz ok 12.2.3.4/16 hence have visibility into the origin of these addresses. (Perhaps all addresses can be part of the same vnic0/_a address object, just like addrconf address objects contain multiple addresses, but that is a detail.) Erik From darren.reed@oracle.com Thu May 20 02:45:47 2010 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4K9jlYP017116 for ; Thu, 20 May 2010 02:45:47 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o4K9jl16025774 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Thu, 20 May 2010 02:45:47 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2P00401PSBQL00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 20 May 2010 03:45:47 -0600 (MDT) Received: from sca-ea-mail-3.sun.com ([192.18.43.21]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2P008CRPSADNC0@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 20 May 2010 03:45:46 -0600 (MDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by sca-ea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o4K9jjRP016620 for ; Thu, 20 May 2010 09:45:46 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o4K9jUnn011279 for ; Thu, 20 May 2010 09:45:31 +0000 (GMT) Received: from abhmt009.oracle.com by acsmt355.oracle.com with ESMTP id 253368921274348643; Thu, 20 May 2010 02:44:03 -0700 Received: from [192.168.1.230] (/76.191.213.211) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 20 May 2010 02:44:03 -0700 Date: Thu, 20 May 2010 02:43:49 -0700 From: Darren Reed Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <4BF3FF5F.60701@oracle.com> To: Sebastien Roy Cc: PSARC-ext , sowmini.varadhan@oracle.com Message-id: <4BF50455.9030506@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090203.4BF504C9.0084:SCFMA4539814,ss=1,fgs=0 References: <4BEB07D6.2040409@oracle.com> <4BF3FF5F.60701@oracle.com> User-Agent: Thunderbird 2.0.0.23 (Windows/20090812) Status: RO Content-Length: 387 Sebastien Roy wrote: > Note to PSARC members: This case times out today. It has received > adequate review from members of both the networking and the zones > teams, but has yet to receive a +1 from a PSARC member. There's still parallel discussion going on internally (about issues that have not surfaced here and that will result in spec changes), how can anyone +1 it? Darren From sowmini.varadhan@oracle.com Thu May 20 05:35:17 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4KCZHqB019863 for ; Thu, 20 May 2010 05:35:17 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o4KCZG4h007377 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Thu, 20 May 2010 07:35:16 -0500 (CDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2P00J03XMS5500@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 20 May 2010 06:35:16 -0600 (MDT) Received: from quasimodo.East.Sun.COM ([129.148.174.94]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2P00AVVXMSE680@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 20 May 2010 06:35:16 -0600 (MDT) Received: from quasimodo.East.Sun.COM (localhost [127.0.0.1]) by quasimodo.East.Sun.COM (8.14.4+Sun/8.14.4) with ESMTP id o4KCSLaX010003; Thu, 20 May 2010 08:28:21 -0400 (EDT) Received: (from sowmini@localhost) by quasimodo.East.Sun.COM (8.14.4+Sun/8.14.4/Submit) id o4KCSLpk010002; Thu, 20 May 2010 08:28:21 -0400 (EDT) Date: Thu, 20 May 2010 08:28:21 -0400 From: sowmini.varadhan@oracle.com Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <4BF50455.9030506@oracle.com> To: Darren Reed Cc: Sebastien Roy , PSARC-ext Message-id: <20100520122821.GA9994@quasimodo.East.Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <4BEB07D6.2040409@oracle.com> <4BF3FF5F.60701@oracle.com> <4BF50455.9030506@oracle.com> X-Authentication-warning: quasimodo.East.Sun.COM: sowmini set sender to sowmini.varadhan@oracle.com using -f User-Agent: Mutt/1.5.14 (2007-03-31) Status: RO Content-Length: 615 On (05/20/10 02:43), Darren Reed wrote: > > Sebastien Roy wrote: > > Note to PSARC members: This case times out today. It has received adequate > > review from members of both the networking and the zones teams, but has yet > > to receive a +1 from a PSARC member. > > There's still parallel discussion going on internally (about issues that > have not surfaced here and that will result in spec changes), how can anyone > +1 it? yes, I attended the psarc meeting, and asked for an extension, in the light of the internal discussion. So no worries, we are all doing the Right Thing here. --Sowmini From sowmini.varadhan@oracle.com Thu May 20 06:14:40 2010 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4KDEe4O020346 for ; Thu, 20 May 2010 06:14:40 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o4KDEbIc026491 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Thu, 20 May 2010 06:14:39 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2P00M0XZGFF400@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 20 May 2010 07:14:39 -0600 (MDT) Received: from quasimodo.East.Sun.COM ([129.148.174.94]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2P00ANXZGEE6C0@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 20 May 2010 07:14:39 -0600 (MDT) Received: from quasimodo.East.Sun.COM (localhost [127.0.0.1]) by quasimodo.East.Sun.COM (8.14.4+Sun/8.14.4) with ESMTP id o4KD7iV8010109; Thu, 20 May 2010 09:07:44 -0400 (EDT) Received: (from sowmini@localhost) by quasimodo.East.Sun.COM (8.14.4+Sun/8.14.4/Submit) id o4KD7iNa010108; Thu, 20 May 2010 09:07:44 -0400 (EDT) Date: Thu, 20 May 2010 09:07:43 -0400 From: sowmini.varadhan@oracle.com Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <4BF4CA99.6070409@oracle.com> To: Erik Nordmark Cc: Edward Pilatowicz , PSARC-ext Message-id: <20100520130743.GD9994@quasimodo.East.Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <4BEB07D6.2040409@oracle.com> <20100513012624.GJ176566@jurassic.Eng.Sun.COM> <20100513022313.GB25937@quasimodo.East.Sun.COM> <20100513202544.GB350787@jurassic.Eng.Sun.COM> <20100513202920.GG27083@quasimodo.East.Sun.COM> <20100514142056.GA249@quasimodo.East.Sun.COM> <4BF4CA99.6070409@oracle.com> X-Authentication-warning: quasimodo.East.Sun.COM: sowmini set sender to sowmini.varadhan@oracle.com using -f User-Agent: Mutt/1.5.14 (2007-03-31) Status: RO Content-Length: 3724 On (05/19/10 22:37), Erik Nordmark wrote: > > This looks identical to the case when vnic0 was configured locally in the > ngz (that is what "static" means). Hence the NGZ admin have no reason to > believe it isn't possible to do a ipadm delete-addr vnic0/_a. > > But either such a delete fails, or after a reboot the address magically > reappers; either of which would be a surprise to the ngz admin since the > addresses are marked as "static". > > These addresses provided and enforced by the GZ are provided by an external > entity (the GZ) the same way as the addresses in a dhcp or addrconf address > object are provided by an external entity (the dhcp server and stateless > addrconf config on the routers, respectively. Thus I think it makes sense to > label them as a different type of address object to make this clear to the > ngz admin > > I don't know what a good name would be - we used "from_gz" when we talked > about this on the phone. > > That way the ngz admin will see something like > vnic0/_a from_gz ok 11.1.1.1/24 > vnic0/_b from_gz ok 12.2.3.4/16 > hence have visibility into the origin of these addresses. (Perhaps all > addresses can be part of the same vnic0/_a address object, just like > addrconf address objects contain multiple addresses, but that is a detail.) If all we want is to keep the origin clear, that can be done by simply setting an address flag (IFF_FROM_GZ) on addresses added by ipmgmtd, and using that to print output in show-addr. But we discussed some more complex issues on the phone, specifically the ability for the NGZ to persistently set interface/address properties on the from_gz interface/addresses, the ability to not have these things recreated on each reboot, and the ability to recover from_gz information at any time after boot using ipadm if, for example, vnic0 above got accidentally deleted. As we discussed on the phone, one solution for the anticipated life-cycles for the "from_gz" address objects in ipkg excl-IP zones would be - on the first boot, set up vnic0 and and the from_gz addresses persistently as far as ipadm is concerned, - on subsequent boots the numeric values of address and mask for from_gz objects are obtained from the nvlists set up by zoneadmd in the kernel, everything else (for the interface and address) is obtained from the persistent store - the administrator can delete the interface/addresses associated with the from_gz object at any time. The from_gz object, like dhcp or addrconf, would map to several addresses, so address operations would have to apply to all the addresses. That would necessarily mean that the addrobj name would have to be the same for all the addresses - to recreate the from_gz interface after its been deleted, we'd need a new # ipadm create-addr -T from_gz vnic0/gzaddrs - once an administrator has deleted the from_gz object that was handed down from the GZ, it will never be recreated. In contrast, the current proposal creates all the addrobjs (and the interface) in the NGZ temporarily, so individual addresses (and address objects) can be selectively modified/deleted temporarily at any time, and while the administrator can delete/recreate the same address (object) persistently, on the next reboot the GZ information will win. The latter model is possibly simpler, though it admittedly allows the NGZ less flexibility (esp in setting interface properties persistently), and, short of restarting ipmgmtd, doesn't allow the administrator to recover an accidentally deleted the "from_gz" interface in a graceful way. Thoughts? --Sowmini From sebastien.roy@oracle.com Thu May 20 06:44:26 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4KDiQlN020420 for ; Thu, 20 May 2010 06:44:26 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o4KDiP1f012896 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Thu, 20 May 2010 08:44:26 -0500 (CDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2Q0080P0U2KG00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 20 May 2010 06:44:26 -0700 (PDT) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2Q005BF0U0BG10@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 20 May 2010 06:44:24 -0700 (PDT) Received: from rcsinet15.oracle.com (rcsinet15.oracle.com [148.87.113.117]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o4KDiN17025392 for ; Thu, 20 May 2010 13:44:23 +0000 (GMT) Received: from acsmt355.oracle.com (acsmt355.oracle.com [141.146.40.155]) by rcsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o4KDiMCm020086 for ; Thu, 20 May 2010 13:44:23 +0000 (GMT) Received: from abhmt017.oracle.com by acsmt354.oracle.com with ESMTP id 284495701274363034; Thu, 20 May 2010 06:43:54 -0700 Received: from [129.148.19.4] (/129.148.19.4) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 20 May 2010 06:43:54 -0700 Date: Thu, 20 May 2010 09:43:52 -0400 From: Sebastien Roy Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <4BF50455.9030506@oracle.com> To: Darren Reed Cc: PSARC-ext , sowmini.varadhan@oracle.com Message-id: <4BF53C98.4090903@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt355.oracle.com [141.146.40.155] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090203.4BF53CB7.00D5:SCFMA4539814,ss=1,fgs=0 References: <4BEB07D6.2040409@oracle.com> <4BF3FF5F.60701@oracle.com> <4BF50455.9030506@oracle.com> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.8) Gecko/20100412 Lightning/1.0b1 Thunderbird/3.0.3 Status: RO Content-Length: 517 On 05/20/10 05:43 AM, Darren Reed wrote: > Sebastien Roy wrote: >> Note to PSARC members: This case times out today. It has received >> adequate review from members of both the networking and the zones >> teams, but has yet to receive a +1 from a PSARC member. > > There's still parallel discussion going on internally (about issues that > have not surfaced here and that will result in spec changes), how can > anyone +1 it? As we discussed during the meeting yesterday, this case is in waiting-need-spec... -Seb From erik.nordmark@oracle.com Fri May 21 06:57:07 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4LDv6Bd005509 for ; Fri, 21 May 2010 06:57:07 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o4LDv390017129 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Fri, 21 May 2010 08:57:06 -0500 (CDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2R00C11W36WM00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 21 May 2010 06:57:06 -0700 (PDT) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2R00KN7W35WRA0@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 21 May 2010 06:57:05 -0700 (PDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o4LDv5Gf020564 for ; Fri, 21 May 2010 13:57:05 +0000 (GMT) Received: from acsmt355.oracle.com (acsmt355.oracle.com [141.146.40.155]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o4K3fYmN027766 for ; Fri, 21 May 2010 13:57:04 +0000 (GMT) Received: from abhmt013.oracle.com by acsmt353.oracle.com with ESMTP id 257529881274450223; Fri, 21 May 2010 06:57:03 -0700 Received: from [10.7.251.248] (/10.7.251.248) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 21 May 2010 06:57:01 -0700 Date: Fri, 21 May 2010 06:56:35 -0700 From: Erik Nordmark Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <20100520130743.GD9994@quasimodo.East.Sun.COM> To: sowmini.varadhan@oracle.com Cc: Edward Pilatowicz , PSARC-ext Message-id: <4BF69113.60500@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt355.oracle.com [141.146.40.155] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090205.4BF69130.01B6:SCFMA4539814,ss=1,fgs=0 References: <4BEB07D6.2040409@oracle.com> <20100513012624.GJ176566@jurassic.Eng.Sun.COM> <20100513022313.GB25937@quasimodo.East.Sun.COM> <20100513202544.GB350787@jurassic.Eng.Sun.COM> <20100513202920.GG27083@quasimodo.East.Sun.COM> <20100514142056.GA249@quasimodo.East.Sun.COM> <4BF4CA99.6070409@oracle.com> <20100520130743.GD9994@quasimodo.East.Sun.COM> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.8) Gecko/20100412 Lightning/1.0b1 Thunderbird/3.0.3 Status: RO Content-Length: 3046 On 05/20/10 06:07 AM, sowmini.varadhan@oracle.com wrote: > If all we want is to keep the origin clear, that can be done by simply > setting an address flag (IFF_FROM_GZ) on addresses added by ipmgmtd, and > using that to print output in show-addr. But we discussed some more complex > issues on the phone, specifically the ability for the NGZ to persistently > set interface/address properties on the from_gz interface/addresses, > the ability to not have these things recreated on each reboot, and > the ability to recover from_gz information at any time after boot > using ipadm if, for example, vnic0 above got accidentally deleted. Yes, it would be very odd if the ngz admin was not able to set persistent interface properties, such as an mtu, just because the IP addresses were provided from the global zone. > As we discussed on the phone, one solution for the anticipated > life-cycles for the "from_gz" address objects in ipkg excl-IP zones > would be > > - on the first boot, set up vnic0 and and the from_gz addresses > persistently as far as ipadm is concerned, > - on subsequent boots the numeric values of address and mask for > from_gz objects are obtained from the nvlists set up by zoneadmd > in the kernel, everything else (for the interface and address) is > obtained from the persistent store > - the administrator can delete the interface/addresses associated > with the from_gz object at any time. The from_gz object, like dhcp > or addrconf, would map to several addresses, so address operations > would have to apply to all the addresses. That would necessarily > mean that the addrobj name would have to be the same for all the > addresses > - to recreate the from_gz interface after its been deleted, we'd need > a new > # ipadm create-addr -T from_gz vnic0/gzaddrs > - once an administrator has deleted the from_gz object that was handed > down from the GZ, it will never be recreated. > > In contrast, the current proposal creates all the addrobjs (and the interface) > in the NGZ temporarily, so individual addresses (and address objects) can > be selectively modified/deleted temporarily at any time, and while the > administrator can delete/recreate the same address (object) > persistently, on the next reboot the GZ information will win. > > The latter model is possibly simpler, though it admittedly allows the > NGZ less flexibility (esp in setting interface properties > persistently), and, short of restarting ipmgmtd, doesn't allow > the administrator to recover an accidentally deleted the "from_gz" > interface in a graceful way. Most of your comments above, while correct, are about implementation and not about the architectural and interface aspects of the proposal. I think the added architectural material is a "from_gz" (or whatever it makes sense to call it) type of address object, that has the same type of behavior as seen by the ngz administrator as the addrconf or dhcp type of address objects. Erik From sowmini.varadhan@oracle.com Mon May 24 12:41:03 2010 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4OJf3Rm007346 for ; Mon, 24 May 2010 12:41:03 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o4OJf1n6010209 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Mon, 24 May 2010 12:41:02 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2X00L03W0EQM00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 24 May 2010 13:41:02 -0600 (MDT) Received: from quasimodo.East.Sun.COM ([129.148.174.94]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2X00K2KW0D0I10@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 24 May 2010 13:41:01 -0600 (MDT) Received: from quasimodo.East.Sun.COM (localhost [127.0.0.1]) by quasimodo.East.Sun.COM (8.14.4+Sun/8.14.4) with ESMTP id o4OJXvQ6016215; Mon, 24 May 2010 15:33:57 -0400 (EDT) Received: (from sowmini@localhost) by quasimodo.East.Sun.COM (8.14.4+Sun/8.14.4/Submit) id o4OJXvcA016214; Mon, 24 May 2010 15:33:57 -0400 (EDT) Date: Mon, 24 May 2010 15:33:57 -0400 From: sowmini.varadhan@oracle.com Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <4BF69113.60500@oracle.com> To: Erik Nordmark Cc: Edward Pilatowicz , PSARC-ext Message-id: <20100524193357.GC15873@quasimodo.East.Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <4BEB07D6.2040409@oracle.com> <20100513012624.GJ176566@jurassic.Eng.Sun.COM> <20100513022313.GB25937@quasimodo.East.Sun.COM> <20100513202544.GB350787@jurassic.Eng.Sun.COM> <20100513202920.GG27083@quasimodo.East.Sun.COM> <20100514142056.GA249@quasimodo.East.Sun.COM> <4BF4CA99.6070409@oracle.com> <20100520130743.GD9994@quasimodo.East.Sun.COM> <4BF69113.60500@oracle.com> X-Authentication-warning: quasimodo.East.Sun.COM: sowmini set sender to sowmini.varadhan@oracle.com using -f User-Agent: Mutt/1.5.14 (2007-03-31) Status: RO Content-Length: 5223 On (05/21/10 06:56), Erik Nordmark wrote: > > Yes, it would be very odd if the ngz admin was not able to set persistent > interface properties, such as an mtu, just because the IP addresses were > provided from the global zone. : > Most of your comments above, while correct, are about implementation and not > about the architectural and interface aspects of the proposal. It's not that simple. Even architecturally, the "from_gz" address object type will quickly hit the incompleteness in the current ipadm model around address objects when they have more than one address. Here is why. The example that you listed itself highlights the problem. You had suggested > That way the ngz admin will see something like > vnic0/_a from_gz ok 11.1.1.1/24 > vnic0/_b from_gz ok 12.2.3.4/16 this has issues: 1. that's misleading, because it leads me to believe that I can do # ipadm delete-addr vnic0/_a or # ipadm set-addrprop -p foo=bar vnic0/_a while leaving _b untouched. If the whole address object is a set, I'm not supposed to be able to do that. We could prevent it by having some command-specific filters in libipadm, but that then leads to: 2. how should I co-relate the name of each address in the set to the one used with create-addr? e.g., if the command used to create from-gz was ipadm create-addr -T from_gz vnic0/one-name-for-all-of-it how do I co-relate "_a" with "one-name-for-all-of-it"? If, on the other hand, I have a single address object name for the "from_gz" addresses, then how should show-addrprop correctly present that 11.1.1.1 has a /24 prefixlen, and 12.2.3.4 has a /16 prefixlen? > I think the added architectural material is a "from_gz" (or whatever it > makes sense to call it) type of address object, that has the same type of > behavior as seen by the ngz administrator as the addrconf or dhcp type of > address objects. DHCPv4 address objects today only have a single address, and the behavior for ipadm/IPv6 has the naming problem I list above for "show-addrprop". Today, ipadm is actually unusable for setting/showing addrprops (thus avoiding confronting issues like showing prefixlen correctly). We just discussed the proposal below on the phone, and it solves all of the requirements that you mention, and can still be extended to plug into the ipadm "address object with multiple props" later, when that model is more complete To re-enumerate the requirements: > Yes, it would be very odd if the ngz admin was not able to set persistent > interface properties, such as an mtu, just because the IP addresses were > provided from the global zone. plus other things that have been mentioned, namely a. show-addr should clearly show the origin as "from_gz", b. the administrator should be able to persistently create/delete the interface after the first boot c. the administrator should not be able to persistently delete/modify a single address out of a "from_gz" group (i.e., persistent operations, if supported, apply to the whole group of addresses in the object) d. it would be preferable if ipadm could be used for everything, instead of ifconfig. e. the address values within an object should be based on the allowed-ips limitations from the GZ. the steps below even allow the NGZ to never have to fall back to ifconfig: - on the first boot (and only on the first reboot), ipmgmtd will create the interface persistently, so that those properties and interface itself can be accessed through ipadm. I'm also proposing that the addresses will be created temporarily. - interfaces with allowed_ips constraints from the GZ will be marked IFF_L3PROTECT (new, read-only flag) and persistent addresses may not be created on these (ipadm will return EPERM, and print the appropriate error message, something like "Cannot create persistent addresses on interfaces managed by the global zone"). The IFF_L3PROTECT will be listed in ifconfig output, as well as by a "Z" flag in show-if output. - In subsequent boots, any interface that has persistent information in ipadm.conf and IP address information from the GZ will be reconstructed using /sbin/ipadm enable-if. - the administrator can do a delete-if of the l3protect-ed interfaces at any time to stop /lib/svc/method/net-physical from recreating these on reboot. - you can temporarily do set-addrprop on individual addrs from GZ at any time. The show-addrprop will also work correctly. - show-addr of addresses on 'Z' interfaces will print origin as "static/from_gz". Attempts to do "ipadm create-addr -T from_gz" will print an error message like "Unsupported address type". There's one aspect here, that Seb quickly noticed- while all this is flexible when it comes to modifications in the allowed-ips set, there's no well-defined way here to recognize changes in the set of assigned datalinks itself (e.g., on this boot, I had vnic1, but then the GZ generously decides to give me (vnic1 and vnic2) for the next reboot). The behavior for modifications to the set of datalinks follows the model in exclusive IP zones, not shared-IP zones. --Sowmini From girish.moodalbail@oracle.com Mon May 24 15:39:01 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4OMd1jY010628 for ; Mon, 24 May 2010 15:39:01 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o4OMd05Y017069 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Mon, 24 May 2010 16:39:01 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2Y00E074908O00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 24 May 2010 15:39:00 -0700 (PDT) Received: from sca-ea-mail-2.sun.com ([192.18.43.25]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2Y005T2490XDE0@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 24 May 2010 15:39:00 -0700 (PDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by sca-ea-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o4OMd0Nx026732 for ; Mon, 24 May 2010 22:39:00 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o4OMctjt021640 for ; Mon, 24 May 2010 22:38:56 +0000 (GMT) Received: from abhmt013.oracle.com by acsmt355.oracle.com with ESMTP id 294414861274740645; Mon, 24 May 2010 15:37:25 -0700 Received: from girish.local (/76.185.160.225) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 24 May 2010 15:37:24 -0700 Date: Mon, 24 May 2010 17:37:24 -0500 From: Girish Moodalbail Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <20100524193357.GC15873@quasimodo.East.Sun.COM> To: sowmini.varadhan@oracle.com Cc: Erik Nordmark , Edward Pilatowicz , PSARC-ext Message-id: <4BFAFFA4.2050005@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090209.4BFB0003.014D:SCFMA4539814,ss=1,fgs=0 References: <4BEB07D6.2040409@oracle.com> <20100513012624.GJ176566@jurassic.Eng.Sun.COM> <20100513022313.GB25937@quasimodo.East.Sun.COM> <20100513202544.GB350787@jurassic.Eng.Sun.COM> <20100513202920.GG27083@quasimodo.East.Sun.COM> <20100514142056.GA249@quasimodo.East.Sun.COM> <4BF4CA99.6070409@oracle.com> <20100520130743.GD9994@quasimodo.East.Sun.COM> <4BF69113.60500@oracle.com> <20100524193357.GC15873@quasimodo.East.Sun.COM> User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.1.9) Gecko/20100317 Lightning/1.0b1 Thunderbird/3.0.4 Status: RO Content-Length: 1523 > > the steps below even allow the NGZ to never have to fall back to ifconfig: > > - on the first boot (and only on the first reboot), ipmgmtd will create > the interface persistently, so that those properties and interface > itself can be accessed through ipadm. I'm also proposing that the > addresses will be created temporarily. > > - interfaces with allowed_ips constraints from the GZ will be marked > IFF_L3PROTECT (new, read-only flag) and persistent addresses may not > be created on these (ipadm will return EPERM, and print the appropriate > error message, something like "Cannot create persistent addresses on > interfaces managed by the global zone"). The IFF_L3PROTECT will > be listed in ifconfig output, as well as by a "Z" flag in show-if > output. > > - In subsequent boots, any interface that has persistent information > in ipadm.conf and IP address information from the GZ will be > reconstructed using /sbin/ipadm enable-if. > > - the administrator can do a delete-if of the l3protect-ed interfaces > at any time to stop /lib/svc/method/net-physical from recreating > these on reboot. In this case, how does NGZ admin recover the deleted interface and the 'from_gz' addresses configured on that deleted interface. Using, 'ipadm enable-if' wouldn't work as the semantics of enable-if is to restore the configuration by reading the persistent store and evidently we have erased the persistent store in this case. Will there be a new sub-command then? ~GmG From sowmini.varadhan@oracle.com Mon May 24 18:17:36 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4P1HaK5012670 for ; Mon, 24 May 2010 18:17:36 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o4P1HXxp005305 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Mon, 24 May 2010 20:17:33 -0500 (CDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2Y00601BL9Y800@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 24 May 2010 19:17:33 -0600 (MDT) Received: from quasimodo.East.Sun.COM ([129.148.174.94]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2Y003HMBL8Z180@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 24 May 2010 19:17:33 -0600 (MDT) Received: from quasimodo.East.Sun.COM (localhost [127.0.0.1]) by quasimodo.East.Sun.COM (8.14.4+Sun/8.14.4) with ESMTP id o4P1AXG5016585; Mon, 24 May 2010 21:10:33 -0400 (EDT) Received: (from sowmini@localhost) by quasimodo.East.Sun.COM (8.14.4+Sun/8.14.4/Submit) id o4P1AXtB016584; Mon, 24 May 2010 21:10:33 -0400 (EDT) Date: Mon, 24 May 2010 21:10:33 -0400 From: sowmini.varadhan@oracle.com Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <4BFAFFA4.2050005@oracle.com> To: Girish Moodalbail Cc: Erik Nordmark , Edward Pilatowicz , PSARC-ext Message-id: <20100525011033.GA16576@quasimodo.East.Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <20100513012624.GJ176566@jurassic.Eng.Sun.COM> <20100513022313.GB25937@quasimodo.East.Sun.COM> <20100513202544.GB350787@jurassic.Eng.Sun.COM> <20100513202920.GG27083@quasimodo.East.Sun.COM> <20100514142056.GA249@quasimodo.East.Sun.COM> <4BF4CA99.6070409@oracle.com> <20100520130743.GD9994@quasimodo.East.Sun.COM> <4BF69113.60500@oracle.com> <20100524193357.GC15873@quasimodo.East.Sun.COM> <4BFAFFA4.2050005@oracle.com> X-Authentication-warning: quasimodo.East.Sun.COM: sowmini set sender to sowmini.varadhan@oracle.com using -f User-Agent: Mutt/1.5.14 (2007-03-31) Status: RO Content-Length: 694 On (05/24/10 17:37), Girish Moodalbail wrote: > > In this case, how does NGZ admin recover the deleted interface and the > 'from_gz' addresses configured on that deleted interface. Using, 'ipadm > enable-if' wouldn't work as the semantics of enable-if is to restore the > configuration by reading the persistent store and evidently we have erased > the persistent store in this case. Will there be a new sub-command then? In this case the administrator would have done a "ipadm create-if" to create persistent configuration for the interface, and ipadm_enable_if will be modified to first see if there is persistent configuration (the authoritatitve one) for addresses. --Sowmini From sowmini.varadhan@oracle.com Tue May 25 08:16:33 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4PFGWNe014850 for ; Tue, 25 May 2010 08:16:33 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o4PFGWRq036719 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Tue, 25 May 2010 09:16:32 -0600 (MDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2Z00C09EFKQG00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 25 May 2010 09:16:32 -0600 (MDT) Received: from quasimodo.East.Sun.COM ([129.148.174.94]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2Z00MVKEFIT2B0@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 25 May 2010 09:16:30 -0600 (MDT) Received: from quasimodo.East.Sun.COM (localhost [127.0.0.1]) by quasimodo.East.Sun.COM (8.14.4+Sun/8.14.4) with ESMTP id o4PF9Ul6018839; Tue, 25 May 2010 11:09:30 -0400 (EDT) Received: (from sowmini@localhost) by quasimodo.East.Sun.COM (8.14.4+Sun/8.14.4/Submit) id o4PF9Upe018838; Tue, 25 May 2010 11:09:30 -0400 (EDT) Date: Tue, 25 May 2010 11:09:30 -0400 From: sowmini.varadhan@oracle.com Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <20100524193357.GC15873@quasimodo.East.Sun.COM> To: Erik Nordmark Cc: Edward Pilatowicz , PSARC-ext Message-id: <20100525150930.GA18828@quasimodo.East.Sun.COM> MIME-version: 1.0 Content-type: multipart/mixed; boundary="Boundary_(ID_9vM5RRhkPSoZ+iqMa7+26Q)" Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <4BEB07D6.2040409@oracle.com> <20100513012624.GJ176566@jurassic.Eng.Sun.COM> <20100513022313.GB25937@quasimodo.East.Sun.COM> <20100513202544.GB350787@jurassic.Eng.Sun.COM> <20100513202920.GG27083@quasimodo.East.Sun.COM> <20100514142056.GA249@quasimodo.East.Sun.COM> <4BF4CA99.6070409@oracle.com> <20100520130743.GD9994@quasimodo.East.Sun.COM> <4BF69113.60500@oracle.com> <20100524193357.GC15873@quasimodo.East.Sun.COM> X-Authentication-warning: quasimodo.East.Sun.COM: sowmini set sender to sowmini.varadhan@oracle.com using -f User-Agent: Mutt/1.5.14 (2007-03-31) Status: RO Content-Length: 26789 --Boundary_(ID_9vM5RRhkPSoZ+iqMa7+26Q) Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline here are updates to the documentation that include the RTF_ZONE related changes, as well as those based on Erik's comments about allowing persistent interface configuration management in the non-global zone. The full text of the updated document is attached. The diffs are: --- zonecfg.txt 2010/05/19 19:28:37 1.2 +++ zonecfg.txt 2010/05/25 15:08:12 @@ -79,10 +79,20 @@ other zonecfg(1m) resources such as the "physical" datalink. At the same time, the zone boot process will also ensure 'ip-nospoof' protection for the datalink with the specified addresses used as input to the -'allowed-ips' property. The stored nvlist information for 'address' -and 'defrouter' will be retrieved and re-applied in the non-global zone -by the daemon associated with the ip-interface-management service -before any other IP configuration is applied. +'allowed-ips' property. On the first boot of an installed non-global +zone, the stored nvlist information for 'address' will be retrieved by +the ipmgmtd daemon which will create the interface persistently, and +apply the IP addresses non-persistently before any other IP +configuration is applied. On subsequent boots, any IP interface with +persistent configuration in the ipadm data-store will be recreated using +IP address information from the kernel's nvlists set up by zoneadmd. + +On every boot of a non-global zone, ipmgmtd will retrieve and attempt to +re-apply any 'defrouter' information set up by the global zone before +any other IP configuration is applied. Routes added by ipmgmtd will be +identifiable in the 'netstat -r' output by the 'Z' flag which will +indicate that the associated route was based on information configured +in the global zone. The GLDv3 layer communicates the set of "allowed-ips" to the IP layer through DLPI notifications. The notification sent is a DL_NOTIFY_IND @@ -93,13 +103,44 @@ the IP layer to track the current set of permitted IP addresses per interface, so that the IP layer can return informative error diagnostics if an attempt is made to enable forwarding or set addresses -on the interface that are outside the permitted set. +on the interface that are outside the permitted set. IP interfaces +with 'allowed-ips' constraints will be identifiable by the "L3PROTECT" +flag in the output of ifconfig, and by a 'Z' flag in the output of +'ipadm show-if'. All configuration state, including datalink properties like protection, allowed-ips and zone, and associated nvlist information, that is created during zone boot will be reset to the default values when the zone is halted. +Interfaces added by this case: +------------------------------ +This case adds the following: + +- a new RTF_ZONE flag which will be passed down in the routing socket + message sent by the ipmgmtd daemon in the non-global zone to add + default route information set up via zonecfg(1m). The output of + 'netstat -r' will display 'Z' when the route was added by ipmgmtd in + this manner. The output of the 'route get' command will also display + the "ZONE" string as part of the 'flags' field for the route. + +- a read-only interface flag IFF_L3PROTECT which will be set by the + kernel on the IP interface when the 'allowed-ips' link property + has been set for the interface. The IFF_L3PROTECT flag will be indicated + by a 'Z' in the CURRENT column of the output from 'ipadm show-if' + + + Interface Classification Comments + _______________________________________________________________________ + netstat(1m) -r output Uncommitted (unchanged) + RTF_ZONE Committed + route(1m) Uncommitted (unchanged) + IFF_L3PROTECT Committed , read-only + +The output of 'ipadm show-addr' will be modified to display a TYPE of +'from-gz' when the address has been configured based on IP address +information allocated to the non-global zone from the global zone. + Relationship to other configuration mechanisms ---------------------------------------------- @@ -191,16 +232,24 @@ root@gz# zlogin tz1 ipadm show-addr ADDROBJ TYPE STATE ADDR - vnic0/_a static ok 11.1.1.1/24 - vnic0/_b static ok 12.2.3.4/16 + vnic0/_a from-gz ok 11.1.1.1/24 + vnic0/_b from-gz ok 12.2.3.4/16 lo0/v4 static ok 127.0.0.1/8 lo0/v6 static ok ::1/128 + root@gz# zlogin tz1 ipadm show-if + IFNAME STATE CURRENT PERSISTENT + vnic0 ok bm-------Z4- -4- + lo0 ok -m-v------46 --- + + + + root@gz# zlogin tz1 netstat -rn Routing Table: IPv4 Destination Gateway Flags Ref Use Interface -------------------- -------------------- ----- ----- ---------- --------- - default 11.1.1.2 UG 2 2 vnic0 + default 11.1.1.2 UGZ 2 2 vnic0 11.1.1.0 11.1.1.1 U 3 2 vnic0 12.2.0.0 12.2.3.4 U 2 0 vnic0 127.0.0.1 127.0.0.1 UH 2 0 lo0 @@ -211,6 +260,16 @@ ::1 ::1 UH 2 0 lo0 + roott@gz# zlogin tz1 route get default + route to: default + destination: default + mask: default + gateway: 11.1.1.2 + interface: vnic0 + flags: + recvpipe sendpipe ssthresh rtt,ms rttvar,ms hopcount mtu expire + + Proposed man page changes: -------------------------- --- zonecfg.1m.orig Thu Apr 22 14:41:33 2010 @@ -264,6 +323,78 @@ DHCP client ID (DUID for DHCPv6) and hardware +--- netstat.orig Wed May 19 17:02:56 2010 ++++ netstat.new Wed May 19 17:05:17 2010 +@@ -740,6 +740,9 @@ + I Indirect routes (gateway not directly reachable) esta- + blished with the -indirect option. + ++ Z (non-global zone only) The route was statically added ++ on boot based on routing information configured using ++ zonecfg(1m) in the global zone. + + + If the -a option is specified, there will be routing entries +@@ -881,7 +884,7 @@ + iostat(1M), kstat(1M), mibiisa(1M), savecore(1M), + vmstat(1M), hosts(4), inet_type(4), networks(4), proto- + cols(4), services(4), attributes(5), dhcp(5), kstat(7D), +- inet(7P), inet6(7P) ++ inet(7P), inet6(7P), zonecfg(1m) + + + Droms, R., RFC 2131, Dynamic Host Configuration Protocol, + +--- ipadm.1m.orig Tue May 25 09:12:35 2010 ++++ ipadm.1m.new Tue May 25 10:34:55 2010 +@@ -321,10 +321,11 @@ + a VRRP interface is in accept mode + (~IFF_NOACCEPT) + ++ Z Layer-3 protection of IP addresses for the ++ interface has been administratively enforced. + + + +- + SunOS 5.11 Last change: 29 Mar 2010 5 + + +@@ -891,9 +892,13 @@ + TYPE + + Type of the address object. It will be one of: +- static, dhcp, or addrconf. It corresponds to the +- type of the address object specified by the -T +- option of create-addr. ++ from-gz, static, dhcp, or addrconf. The static, ++ dhcp and addrconf types correspond to the type of the ++ address object specified by the -T option of ++ create-addr. The "from-gz" type will only be displayed ++ in non-global zones, and indicates that the address ++ was configured based on IP address information ++ allocated to the non-global zone from the global zone. + + + STATE + +--- ifconfig.1m.orig Tue May 25 11:05:48 2010 ++++ ifconfig.1m.new Tue May 25 11:07:31 2010 +@@ -1599,7 +1599,13 @@ + used on multiple interfaces. (See also the usesrc + option.) + ++ L3PROTECT + ++ Indicates that Layer-3 protection has been enforced on ++ the physical interface using the 'allowed-ips' link ++ property in dladm(1m) ++ ++ + LOGICAL INTERFACES + Solaris TCP/IP allows multiple logical interfaces to be + associated with a physical network interface. This allows a + References: ---------- --Boundary_(ID_9vM5RRhkPSoZ+iqMa7+26Q) Content-type: text/plain; NAME=zonecfg.txt; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: attachment; filename=zonecfg.txt zonecfg(1m) enhancements for exclusive-IP zones Requested release binding: Minor Summary: ------- This case proposes a solution for 6944327 need to support address and defrouter resources for exclusive-IP zones Problem Description -------------------- Typical zone deployments that exist today use shared-IP zones to run applications and services like Apache or Weblogic in the contained environment provided by the shared-IP zone. In these use-cases, the Administrator in the global zone has full control over the networking resources used by the non-global zone. In the common case, networking is simply configured by specifying the IP interface, IP addresses and, optionally, the default routers from zonecfg(1m). The configuration resources thus supplied are then applied for the non-global zone when it is booted, and the non-global zone itself may not modify any of these configuration parameters. However, there is no such simple configuration mechanism for the simple networking use-case in place for exclusive IP zones, which must be configured through sysidcfg, ifconfig, and an assortment of other methods, all of which are not controllable from the global zone, and may not be managed through zonecfg(1m). The addition of many new virtualization and resource management features to Solaris (PSARC 2006/357, PSARC 2005/132 and associated cases) makes Exclusive-IP zones a cleaner and more powerful Zone model than shared-IP zones. Thus, bridging any gaps in the configuration methods for the simple use cases between shared- and exlusive-IP zones is important to ease the transition of shared-IP customer configurations to exlusive-IP. In addition, there are many ongoing projects in the "Zones Networking" effort [ZONES-NET] to facilitate the consolidation of existing Solaris 10 host installations as Solaris 10 Containers. These efforts would leverage from the ability to specify networking resource values for address and default router uniformly for zones using zonecfg(1m). This case proposes support in zonecfg(1m) for 'address' and 'defrouter' properties in the 'net' resource for exclusive-IP zones. The semantics of the values will be the same as with the shared-IP zone definitions that exist today ([PSARC/2002/174], [PSARC/2003/621], [PSARC/2008/057]) When the global-zone specifies the 'address' for an interface via zonecfg(1m), the non-global zone may not use any other IP addresses for that interface. The address information provided via zonecfg(1m) will be used to set up Layer-3 protection [PSARC/2009/436] for the non-global zone during zone-boot to filter out all other addresses for the selected interface. For instance, when zonecfg(1m) has been used in the global-zone to set one or more IPv4 addresses on an interface, an attempt to set an IPv4 or IPv6 address on the interface that is outside the globally defined set will encounter the EPERM failure. Thus ifconfig(1m), ipadm(1m), and associated ioctls will receive this error if they are used within the non-global zone to set addresses that are not in the set that is permitted from the global-zone, and attempts by the non-global zone to turn on forwarding on the interface will also encounter EPERM. Attempts to boot a zone that has already been configured for IP, or has previously customized values for Layer-3 protection [PSARC/2009/436] will fail. Implementation Overview: ------------------------ A brief overview of the implementation is provided here. Note that all interfaces between zoneadmd(1m) and the kernel/zonecfg(1m), are Private interfaces, subject to change in the future. When the non-global zone is booted, zoneadmd(1m) will store the information specified by the 'address' and 'defrouter' properties as nvlists in the kernel following a mechanism similar to that in use today for other zonecfg(1m) resources such as the "physical" datalink. At the same time, the zone boot process will also ensure 'ip-nospoof' protection for the datalink with the specified addresses used as input to the 'allowed-ips' property. On the first boot of an installed non-global zone, the stored nvlist information for 'address' will be retrieved by the ipmgmtd daemon which will create the interface persistently, and apply the IP addresses non-persistently before any other IP configuration is applied. On subsequent boots, any IP interface with persistent configuration in the ipadm data-store will be recreated using IP address information from the kernel's nvlists set up by zoneadmd. On every boot of a non-global zone, ipmgmtd will retrieve and attempt to re-apply any 'defrouter' information set up by the global zone before any other IP configuration is applied. Routes added by ipmgmtd will be identifiable in the 'netstat -r' output by the 'Z' flag which will indicate that the associated route was based on information configured in the global zone. The GLDv3 layer communicates the set of "allowed-ips" to the IP layer through DLPI notifications. The notification sent is a DL_NOTIFY_IND message of (Project Private) type DL_NOTE_ALLOWED_IPS, sent to the IP clients that have registered for this notification. The payload of the DL_NOTE_ALLOWED_IPS message is the mac_protect_t data-structure introduced by PSARC 2009/436. Receipt of this notification enables the IP layer to track the current set of permitted IP addresses per interface, so that the IP layer can return informative error diagnostics if an attempt is made to enable forwarding or set addresses on the interface that are outside the permitted set. IP interfaces with 'allowed-ips' constraints will be identifiable by the "L3PROTECT" flag in the output of ifconfig, and by a 'Z' flag in the output of 'ipadm show-if'. All configuration state, including datalink properties like protection, allowed-ips and zone, and associated nvlist information, that is created during zone boot will be reset to the default values when the zone is halted. Interfaces added by this case: ------------------------------ This case adds the following: - a new RTF_ZONE flag which will be passed down in the routing socket message sent by the ipmgmtd daemon in the non-global zone to add default route information set up via zonecfg(1m). The output of 'netstat -r' will display 'Z' when the route was added by ipmgmtd in this manner. The output of the 'route get' command will also display the "ZONE" string as part of the 'flags' field for the route. - a read-only interface flag IFF_L3PROTECT which will be set by the kernel on the IP interface when the 'allowed-ips' link property has been set for the interface. The IFF_L3PROTECT flag will be indicated by a 'Z' in the CURRENT column of the output from 'ipadm show-if' Interface Classification Comments _______________________________________________________________________ netstat(1m) -r output Uncommitted (unchanged) RTF_ZONE Committed route(1m) Uncommitted (unchanged) IFF_L3PROTECT Committed , read-only The output of 'ipadm show-addr' will be modified to display a TYPE of 'from-gz' when the address has been configured based on IP address information allocated to the non-global zone from the global zone. Relationship to other configuration mechanisms ---------------------------------------------- sysidcfg: The Solaris Install team is currently working in conjunction with the Network Configuration team [INSTALL] on the implementation of a framework to replace sysidcfg(4) that will provide configuration profiles to be processed by startup scripts when a system boots. However this configuration information is only applied on the first boot of the zone, and unlike zonecfg, may not be easily modified in the global zone across reboot. Moreover, there is no mechanism in place to prevent the non-global zone from modifying the configuration provided via profiles. As mentioned earlier, the global-zone is the authoritative source of IP configuration information for the "secure container" use-case. This implies that attempts to use boot profiles in the non-global zone to add addresses other than those mandated in the global-zone will fail with EINVAL erors. Note that the profiles defined in [INSTALL] may still be applied to interfaces for which 'net' resource properties have not been defined in the global-zone, i.e., this proposal is fully backward-compatible. svc:/network/physical:{default, nwam} Configuration specified via zonecfg(1m) in the global-zone overrides any information specified via /etc/hostname.intf, ipadm(1m) or nwam. Thus if the IP address information specified by these other methods is not a subset of the globally managed information, attempts to apply it within the non-global zone will encounter an error. DHCP: It is assumed that when IP resources for an IP interface have been set up in the global-zone using zonecfg(1m), then that interface has been marked for static IP configuration only, and attempts to send out outgoing DHCP DISCOVER packets will be dropped in GLDv3. Thus attempts to start DHCP on these interfaces in the non-global zone will time out. Examples -------- root@gz# zonecfg -z tz1 zonecfg:tz1> info zonename: tz1 zonepath: /rpool/zones/tz1 brand: ipkg autoboot: false bootargs: pool: limitpriv: scheduling-class: ip-type: exclusive hostid: net: address: 11.1.1.1/24 physical: vnic0 defrouter: 11.1.1.2 net: address: taddr2 physical: vnic0 defrouter not specified net: address not specified physical: vnic1 defrouter not specified In this example, the zone 'tz1' has been assigned 2 interfaces: vnic0 and vnic1. The vnic1 interface does not have any IP address or default router assigned to it, so that it may be configured in any desired way from within the zone using ifconfig(1m), ipadm(1m), NWAM etc. The vnic0 interface has 2 addresses assigned: 11.1.1.1/24 (with default router 11.1.1.2 on the 11.1.1.0/24 subnet) and the address 'taddr2' whose numeric value and netmask will be resolved when the zone boots. Thus, if, at the time of zone boot, the global-zone is also configured as: root@gz# egrep 'hosts|netmask' /etc/nsswitch.conf hosts: files nis dns netmasks: files nis root@gz# grep taddr2 /etc/hosts 12.2.3.4 taddr2 root@gz# grep 12 /etc/netmasks 12.2.0.0 255.255.0.0 tz1 will be booted as root@gz# zoneadm -z tz1 boot root@gz# zlogin tz1 ipadm show-addr ADDROBJ TYPE STATE ADDR vnic0/_a from-gz ok 11.1.1.1/24 vnic0/_b from-gz ok 12.2.3.4/16 lo0/v4 static ok 127.0.0.1/8 lo0/v6 static ok ::1/128 root@gz# zlogin tz1 ipadm show-if IFNAME STATE CURRENT PERSISTENT vnic0 ok bm-------Z4- -4- lo0 ok -m-v------46 --- root@gz# zlogin tz1 netstat -rn Routing Table: IPv4 Destination Gateway Flags Ref Use Interface -------------------- -------------------- ----- ----- ---------- --------- default 11.1.1.2 UGZ 2 2 vnic0 11.1.1.0 11.1.1.1 U 3 2 vnic0 12.2.0.0 12.2.3.4 U 2 0 vnic0 127.0.0.1 127.0.0.1 UH 2 0 lo0 Routing Table: IPv6 Destination/Mask Gateway Flags Ref Use If --------------------------- ---------------------- ----- --- ------- ----- ::1 ::1 UH 2 0 lo0 roott@gz# zlogin tz1 route get default route to: default destination: default mask: default gateway: 11.1.1.2 interface: vnic0 flags: recvpipe sendpipe ssthresh rtt,ms rttvar,ms hopcount mtu expire Proposed man page changes: -------------------------- --- zonecfg.1m.orig Thu Apr 22 14:41:33 2010 +++ zonecfg.1m.new Mon May 17 17:09:46 2010 @@ -676,10 +676,17 @@ for the interface should be specified here. For an exclusive-IP zone, the physical property must be - set and the address and default router properties cannot - be set. + set and the address and default router properties may + be set. The interface specifed for the physical property + must not be in use in the global zone. If an address and + default router are specified via zonecfg(1m), these will be + applied to the interface when the non-global zone is booted, + and the non-global zone will not be able to apply any other + addresses to that interface nor transmit packets with a different + source address for that IP Version. A default router set up via + zonecfg(1m) may not be persistently deleted from within the + non-global zone using the -p flag with route(1m). - device: match Device name to match. @@ -1817,7 +1824,7 @@ svcadm(1M), sysidtool(1M), zfs(1M), zoneadm(1M), priv_str_to_set(3C), kstat(3KSTAT), vfstab(4), attri- butes(5), brands(5), fnmatch(5), lx(5), privileges(5), - resource_controls(5), zones(5) + resource_controls(5), zones(5), route(1m) System Administration Guide: Solaris Containers-Resource --- dladm.1m.txt Mon May 10 15:24:49 2010 +++ dladm.1m.txt.new Fri May 14 10:24:02 2010 @@ -4176,7 +4176,14 @@ option's link layer address must match the link's MAC address. + When a datalink has been protected by setting allowed-ips + to a set of one or more IP addresses, IP interfaces over + that datalink may not configure IP addresses outside the + allowed-ips set, or enable forwarding for the IP interface + and attempts to do so will fail with an EPERM error being + returned to the user. + dhcp-nospoof DHCP client ID (DUID for DHCPv6) and hardware --- netstat.orig Wed May 19 17:02:56 2010 +++ netstat.new Wed May 19 17:05:17 2010 @@ -740,6 +740,9 @@ I Indirect routes (gateway not directly reachable) esta- blished with the -indirect option. + Z (non-global zone only) The route was statically added + on boot based on routing information configured using + zonecfg(1m) in the global zone. If the -a option is specified, there will be routing entries @@ -881,7 +884,7 @@ iostat(1M), kstat(1M), mibiisa(1M), savecore(1M), vmstat(1M), hosts(4), inet_type(4), networks(4), proto- cols(4), services(4), attributes(5), dhcp(5), kstat(7D), - inet(7P), inet6(7P) + inet(7P), inet6(7P), zonecfg(1m) Droms, R., RFC 2131, Dynamic Host Configuration Protocol, --- ipadm.1m.orig Tue May 25 09:12:35 2010 +++ ipadm.1m.new Tue May 25 10:34:55 2010 @@ -321,10 +321,11 @@ a VRRP interface is in accept mode (~IFF_NOACCEPT) + Z Layer-3 protection of IP addresses for the + interface has been administratively enforced. - SunOS 5.11 Last change: 29 Mar 2010 5 @@ -891,9 +892,13 @@ TYPE Type of the address object. It will be one of: - static, dhcp, or addrconf. It corresponds to the - type of the address object specified by the -T - option of create-addr. + from-gz, static, dhcp, or addrconf. The static, + dhcp and addrconf types correspond to the type of the + address object specified by the -T option of + create-addr. The "from-gz" type will only be displayed + in non-global zones, and indicates that the address + was configured based on IP address information + allocated to the non-global zone from the global zone. STATE --- ifconfig.1m.orig Tue May 25 11:05:48 2010 +++ ifconfig.1m.new Tue May 25 11:07:31 2010 @@ -1599,7 +1599,13 @@ used on multiple interfaces. (See also the usesrc option.) + L3PROTECT + Indicates that Layer-3 protection has been enforced on + the physical interface using the 'allowed-ips' link + property in dladm(1m) + + LOGICAL INTERFACES Solaris TCP/IP allows multiple logical interfaces to be associated with a physical network interface. This allows a References: ---------- [PSARC/2002/174] Virtualization and Namespace Isolation in Solaris, aka "Zones" [PSARC/2003/621] Zone Administration Updates [PSARC/2008/057] Default Route For Zones [PSARC/2009/436] Anti-spoofing Link Protection [INSTALL] http://solaris-networking.sfbay/index.php/Network_Configuration:install [ZONES-NET] http://solaris-networking.sfbay/index.php/Zones_Networking --Boundary_(ID_9vM5RRhkPSoZ+iqMa7+26Q)-- From sebastien.roy@oracle.com Wed May 26 11:37:22 2010 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4QIbLmE006746 for ; Wed, 26 May 2010 11:37:21 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o4QIbLiZ022289 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 26 May 2010 11:37:21 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L3100H0DIE9LR00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 26 May 2010 12:37:21 -0600 (MDT) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L31008JNIE9TD50@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 26 May 2010 12:37:21 -0600 (MDT) Received: from rcsinet13.oracle.com (rcsinet13.oracle.com [148.87.113.125]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o4QIbK6u017268 for ; Wed, 26 May 2010 18:37:20 +0000 (GMT) Received: from acsmt355.oracle.com (acsmt355.oracle.com [141.146.40.155]) by rcsinet13.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o4QIUvaS020763 for ; Wed, 26 May 2010 18:37:20 +0000 (GMT) Received: from abhmt015.oracle.com by acsmt355.oracle.com with ESMTP id 271040921274899007; Wed, 26 May 2010 11:36:47 -0700 Received: from [129.148.19.4] (/129.148.19.4) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 26 May 2010 11:36:46 -0700 Date: Wed, 26 May 2010 14:36:38 -0400 From: Sebastien Roy Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <20100525150930.GA18828@quasimodo.East.Sun.COM> To: sowmini.varadhan@oracle.com Cc: Erik Nordmark , PSARC-ext Message-id: <4BFD6A36.9020208@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt355.oracle.com [141.146.40.155] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090203.4BFD6A60.00A2:SCFMA4539814,ss=1,fgs=0 References: <4BEB07D6.2040409@oracle.com> <20100513012624.GJ176566@jurassic.Eng.Sun.COM> <20100513022313.GB25937@quasimodo.East.Sun.COM> <20100513202544.GB350787@jurassic.Eng.Sun.COM> <20100513202920.GG27083@quasimodo.East.Sun.COM> <20100514142056.GA249@quasimodo.East.Sun.COM> <4BF4CA99.6070409@oracle.com> <20100520130743.GD9994@quasimodo.East.Sun.COM> <4BF69113.60500@oracle.com> <20100524193357.GC15873@quasimodo.East.Sun.COM> <20100525150930.GA18828@quasimodo.East.Sun.COM> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.9) Gecko/20100510 Lightning/1.0b1 Thunderbird/3.0.4 Status: RO Content-Length: 1855 Sowmini, On 05/25/10 11:09 AM, sowmini.varadhan@oracle.com wrote: > here are updates to the documentation that include the RTF_ZONE related > changes, as well as those based on Erik's comments about allowing > persistent interface configuration management in the non-global zone. Thanks for the update, I've set a new timer on the case to expire on 06/02/2010. One nit below: > --- zonecfg.txt 2010/05/19 19:28:37 1.2 > +++ zonecfg.txt 2010/05/25 15:08:12 > @@ -79,10 +79,20 @@ > other zonecfg(1m) resources such as the "physical" datalink. At the same > time, the zone boot process will also ensure 'ip-nospoof' protection > for the datalink with the specified addresses used as input to the > -'allowed-ips' property. The stored nvlist information for 'address' > -and 'defrouter' will be retrieved and re-applied in the non-global zone > -by the daemon associated with the ip-interface-management service > -before any other IP configuration is applied. > +'allowed-ips' property. On the first boot of an installed non-global > +zone, the stored nvlist information for 'address' will be retrieved by > +the ipmgmtd daemon which will create the interface persistently, and > +apply the IP addresses non-persistently before any other IP > +configuration is applied. On subsequent boots, any IP interface with > +persistent configuration in the ipadm data-store will be recreated using > +IP address information from the kernel's nvlists set up by zoneadmd. The above semantics will be slightly different in solaris10 branded zones since there are no persistent interface objects in such zones. In such zones, I believe it would be sufficient for ipmgmtd to configure everything non-persistently and let the administrator's hostname. files customize interface parameters appropriately (I believe this should "just work"). -Seb From sowmini.varadhan@oracle.com Wed May 26 11:48:05 2010 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4QIm5s2006779 for ; Wed, 26 May 2010 11:48:05 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o4QIm2U0029031 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 26 May 2010 11:48:04 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L3100I0ZIW3H700@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 26 May 2010 12:48:03 -0600 (MDT) Received: from quasimodo.East.Sun.COM ([129.148.174.94]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L310089PIW2TH60@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 26 May 2010 12:48:03 -0600 (MDT) Received: from quasimodo.East.Sun.COM (localhost [127.0.0.1]) by quasimodo.East.Sun.COM (8.14.4+Sun/8.14.4) with ESMTP id o4QIf2HP020748; Wed, 26 May 2010 14:41:02 -0400 (EDT) Received: (from sowmini@localhost) by quasimodo.East.Sun.COM (8.14.4+Sun/8.14.4/Submit) id o4QIf2s0020747; Wed, 26 May 2010 14:41:02 -0400 (EDT) Date: Wed, 26 May 2010 14:41:02 -0400 From: sowmini.varadhan@oracle.com Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <4BFD6A36.9020208@oracle.com> To: Sebastien Roy Cc: Erik Nordmark , PSARC-ext Message-id: <20100526184102.GG20212@quasimodo.East.Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <20100513022313.GB25937@quasimodo.East.Sun.COM> <20100513202544.GB350787@jurassic.Eng.Sun.COM> <20100513202920.GG27083@quasimodo.East.Sun.COM> <20100514142056.GA249@quasimodo.East.Sun.COM> <4BF4CA99.6070409@oracle.com> <20100520130743.GD9994@quasimodo.East.Sun.COM> <4BF69113.60500@oracle.com> <20100524193357.GC15873@quasimodo.East.Sun.COM> <20100525150930.GA18828@quasimodo.East.Sun.COM> <4BFD6A36.9020208@oracle.com> X-Authentication-warning: quasimodo.East.Sun.COM: sowmini set sender to sowmini.varadhan@oracle.com using -f User-Agent: Mutt/1.5.14 (2007-03-31) Status: RO Content-Length: 560 On (05/26/10 14:36), Sebastien Roy wrote: > > The above semantics will be slightly different in solaris10 branded zones > since there are no persistent interface objects in such zones. In such > zones, I believe it would be sufficient for ipmgmtd to configure everything > non-persistently and let the administrator's hostname. files customize > interface parameters appropriately (I believe this should "just work"). Is there a PSARC case for exclusive-IP solaris10 branded zones that I could refer to, in order to clarify this? --Sowmini From sowmini.varadhan@oracle.com Wed May 26 12:11:36 2010 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4QJBZBW007249 for ; Wed, 26 May 2010 12:11:35 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o4QJBYlZ008928 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 26 May 2010 12:11:35 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L3100K0LJZAMI00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 26 May 2010 13:11:34 -0600 (MDT) Received: from quasimodo.East.Sun.COM ([129.148.174.94]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L31008OSJZ9TH70@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 26 May 2010 13:11:33 -0600 (MDT) Received: from quasimodo.East.Sun.COM (localhost [127.0.0.1]) by quasimodo.East.Sun.COM (8.14.4+Sun/8.14.4) with ESMTP id o4QJ4VGi020854; Wed, 26 May 2010 15:04:31 -0400 (EDT) Received: (from sowmini@localhost) by quasimodo.East.Sun.COM (8.14.4+Sun/8.14.4/Submit) id o4QJ4VGv020853; Wed, 26 May 2010 15:04:31 -0400 (EDT) Date: Wed, 26 May 2010 15:04:31 -0400 From: sowmini.varadhan@oracle.com Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <20100526184102.GG20212@quasimodo.East.Sun.COM> To: Sebastien Roy Cc: PSARC-ext , Erik Nordmark Message-id: <20100526190431.GH20212@quasimodo.East.Sun.COM> MIME-version: 1.0 Content-type: multipart/mixed; boundary="Boundary_(ID_GY+fWfN5BSYoHHppHebApw)" Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <20100513202544.GB350787@jurassic.Eng.Sun.COM> <20100513202920.GG27083@quasimodo.East.Sun.COM> <20100514142056.GA249@quasimodo.East.Sun.COM> <4BF4CA99.6070409@oracle.com> <20100520130743.GD9994@quasimodo.East.Sun.COM> <4BF69113.60500@oracle.com> <20100524193357.GC15873@quasimodo.East.Sun.COM> <20100525150930.GA18828@quasimodo.East.Sun.COM> <4BFD6A36.9020208@oracle.com> <20100526184102.GG20212@quasimodo.East.Sun.COM> X-Authentication-warning: quasimodo.East.Sun.COM: sowmini set sender to sowmini.varadhan@oracle.com using -f User-Agent: Mutt/1.5.14 (2007-03-31) Status: RO Content-Length: 21365 --Boundary_(ID_GY+fWfN5BSYoHHppHebApw) Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline > On (05/26/10 14:36), Sebastien Roy wrote: > > > > The above semantics will be slightly different in solaris10 branded zones > > since there are no persistent interface objects in such zones. In such > > zones, I believe it would be sufficient for ipmgmtd to configure everything > > non-persistently and let the administrator's hostname. files customize > > interface parameters appropriately (I believe this should "just work"). Good catch. Yes, that is correct- it would be sufficient for ipmgmtd to configure everything temporarily. Also, another point that I noticed was that the defrouter can only be configured after the interface address has been configured. Thus, to match DHCP (as Erik had suggested) the same mechanism that configures the address must also configure the defrouter. This results in the following diffs: --- zonecfg.txt 2010/05/25 13:24:16 1.4 +++ zonecfg.txt 2010/05/26 19:09:18 @@ -80,19 +80,21 @@ time, the zone boot process will also ensure 'ip-nospoof' protection for the datalink with the specified addresses used as input to the 'allowed-ips' property. On the first boot of an installed non-global -zone, the stored nvlist information for 'address' will be retrieved by -the ipmgmtd daemon which will create the interface persistently, and -apply the IP addresses non-persistently before any other IP -configuration is applied. On subsequent boots, any IP interface with -persistent configuration in the ipadm data-store will be recreated using -IP address information from the kernel's nvlists set up by zoneadmd. - -On every boot of a non-global zone, ipmgmtd will retrieve and re-apply -any 'defrouter' information set up by the global zone before any other -IP configuration is applied. Routes added by ipmgmtd will be -identifiable in the 'netstat -r' output by the 'Z' flag which will -indicate that the associated route was based on information configured -in the global zone. +native (ipkg) zone, and on every boot of a solaris10 branded +zone [PSARC 2010/111], the stored nvlist information for 'address' will +be retrieved by the ipmgmtd daemon which will create the interface +(persistently, for native ipkg zones), and apply the IP addresses +non-persistently before any other IP configuration is applied. On +subsequent boots of native (ipkg) zones, any IP interface with +persistent configuration in the ipadm data-store will be recreated +using IP address information from the kernel's nvlists set up by zoneadmd. + +The 'defrouter' information set up by the global zone will be applied +after the 'address' information has been configured, so that the behavior +of the IP configuration scheme matches that of other address acquisition +protocols like DHCP. Routes added by ipmgmtd will be identifiable in +the 'netstat -r' output by the 'Z' flag which will indicate that the +associated route was based on information configured in the global zone. The GLDv3 layer communicates the set of "allowed-ips" to the IP layer through DLPI notifications. The notification sent is a DL_NOTIFY_IND --Boundary_(ID_GY+fWfN5BSYoHHppHebApw) Content-type: text/plain; NAME=zonecfg.txt; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: attachment; filename=zonecfg.txt zonecfg(1m) enhancements for exclusive-IP zones Requested release binding: Minor Summary: ------- This case proposes a solution for 6944327 need to support address and defrouter resources for exclusive-IP zones Problem Description -------------------- Typical zone deployments that exist today use shared-IP zones to run applications and services like Apache or Weblogic in the contained environment provided by the shared-IP zone. In these use-cases, the Administrator in the global zone has full control over the networking resources used by the non-global zone. In the common case, networking is simply configured by specifying the IP interface, IP addresses and, optionally, the default routers from zonecfg(1m). The configuration resources thus supplied are then applied for the non-global zone when it is booted, and the non-global zone itself may not modify any of these configuration parameters. However, there is no such simple configuration mechanism for the simple networking use-case in place for exclusive IP zones, which must be configured through sysidcfg, ifconfig, and an assortment of other methods, all of which are not controllable from the global zone, and may not be managed through zonecfg(1m). The addition of many new virtualization and resource management features to Solaris (PSARC 2006/357, PSARC 2005/132 and associated cases) makes Exclusive-IP zones a cleaner and more powerful Zone model than shared-IP zones. Thus, bridging any gaps in the configuration methods for the simple use cases between shared- and exlusive-IP zones is important to ease the transition of shared-IP customer configurations to exlusive-IP. In addition, there are many ongoing projects in the "Zones Networking" effort [ZONES-NET] to facilitate the consolidation of existing Solaris 10 host installations as Solaris 10 Containers. These efforts would leverage from the ability to specify networking resource values for address and default router uniformly for zones using zonecfg(1m). This case proposes support in zonecfg(1m) for 'address' and 'defrouter' properties in the 'net' resource for exclusive-IP zones. The semantics of the values will be the same as with the shared-IP zone definitions that exist today ([PSARC/2002/174], [PSARC/2003/621], [PSARC/2008/057]) When the global-zone specifies the 'address' for an interface via zonecfg(1m), the non-global zone may not use any other IP addresses for that interface. The address information provided via zonecfg(1m) will be used to set up Layer-3 protection [PSARC/2009/436] for the non-global zone during zone-boot to filter out all other addresses for the selected interface. For instance, when zonecfg(1m) has been used in the global-zone to set one or more IPv4 addresses on an interface, an attempt to set an IPv4 or IPv6 address on the interface that is outside the globally defined set will encounter the EPERM failure. Thus ifconfig(1m), ipadm(1m), and associated ioctls will receive this error if they are used within the non-global zone to set addresses that are not in the set that is permitted from the global-zone, and attempts by the non-global zone to turn on forwarding on the interface will also encounter EPERM. Attempts to boot a zone that has already been configured for IP, or has previously customized values for Layer-3 protection [PSARC/2009/436] will fail. Implementation Overview: ------------------------ A brief overview of the implementation is provided here. Note that all interfaces between zoneadmd(1m) and the kernel/zonecfg(1m), are Private interfaces, subject to change in the future. When the non-global zone is booted, zoneadmd(1m) will store the information specified by the 'address' and 'defrouter' properties as nvlists in the kernel following a mechanism similar to that in use today for other zonecfg(1m) resources such as the "physical" datalink. At the same time, the zone boot process will also ensure 'ip-nospoof' protection for the datalink with the specified addresses used as input to the 'allowed-ips' property. On the first boot of an installed non-global native (ipkg) zone, and on every boot of a solaris10 branded zone [PSARC 2010/111], the stored nvlist information for 'address' will be retrieved by the ipmgmtd daemon which will create the interface (persistently, for native ipkg zones), and apply the IP addresses non-persistently before any other IP configuration is applied. On subsequent boots of native (ipkg) zones, any IP interface with persistent configuration in the ipadm data-store will be recreated using IP address information from the kernel's nvlists set up by zoneadmd. The 'defrouter' information set up by the global zone will be applied after the 'address' information has been configured, so that the behavior of the IP configuration scheme matches that of other address acquisition protocols like DHCP. Routes added by ipmgmtd will be identifiable in the 'netstat -r' output by the 'Z' flag which will indicate that the associated route was based on information configured in the global zone. The GLDv3 layer communicates the set of "allowed-ips" to the IP layer through DLPI notifications. The notification sent is a DL_NOTIFY_IND message of (Project Private) type DL_NOTE_ALLOWED_IPS, sent to the IP clients that have registered for this notification. The payload of the DL_NOTE_ALLOWED_IPS message is the mac_protect_t data-structure introduced by PSARC 2009/436. Receipt of this notification enables the IP layer to track the current set of permitted IP addresses per interface, so that the IP layer can return informative error diagnostics if an attempt is made to enable forwarding or set addresses on the interface that are outside the permitted set. IP interfaces with 'allowed-ips' constraints will be identifiable by the "L3PROTECT" flag in the output of ifconfig, and by a 'Z' flag in the output of 'ipadm show-if'. All configuration state, including datalink properties like protection, allowed-ips and zone, and associated nvlist information, that is created during zone boot will be reset to the default values when the zone is halted. Interfaces added by this case: ------------------------------ This case adds the following: - a new RTF_ZONE flag which will be passed down in the routing socket message sent by the ipmgmtd daemon in the non-global zone to add default route information set up via zonecfg(1m). The output of 'netstat -r' will display 'Z' when the route was added by ipmgmtd in this manner. The output of the 'route get' command will also display the "ZONE" string as part of the 'flags' field for the route. - a read-only interface flag IFF_L3PROTECT which will be set by the kernel on the IP interface when the 'allowed-ips' link property has been set for the interface. The IFF_L3PROTECT flag will be indicated by a 'Z' in the CURRENT column of the output from 'ipadm show-if' Interface Classification Comments _______________________________________________________________________ netstat(1m) -r output Uncommitted (unchanged) RTF_ZONE Committed route(1m) Uncommitted (unchanged) IFF_L3PROTECT Committed , read-only The output of 'ipadm show-addr' will be modified to display a TYPE of 'from-gz' when the address has been configured based on IP address information allocated to the non-global zone from the global zone. Relationship to other configuration mechanisms ---------------------------------------------- sysidcfg: The Solaris Install team is currently working in conjunction with the Network Configuration team [INSTALL] on the implementation of a framework to replace sysidcfg(4) that will provide configuration profiles to be processed by startup scripts when a system boots. However this configuration information is only applied on the first boot of the zone, and unlike zonecfg, may not be easily modified in the global zone across reboot. Moreover, there is no mechanism in place to prevent the non-global zone from modifying the configuration provided via profiles. As mentioned earlier, the global-zone is the authoritative source of IP configuration information for the "secure container" use-case. This implies that attempts to use boot profiles in the non-global zone to add addresses other than those mandated in the global-zone will fail with EINVAL erors. Note that the profiles defined in [INSTALL] may still be applied to interfaces for which 'net' resource properties have not been defined in the global-zone, i.e., this proposal is fully backward-compatible. svc:/network/physical:{default, nwam} Configuration specified via zonecfg(1m) in the global-zone overrides any information specified via /etc/hostname.intf, ipadm(1m) or nwam. Thus if the IP address information specified by these other methods is not a subset of the globally managed information, attempts to apply it within the non-global zone will encounter an error. DHCP: It is assumed that when IP resources for an IP interface have been set up in the global-zone using zonecfg(1m), then that interface has been marked for static IP configuration only, and attempts to send out outgoing DHCP DISCOVER packets will be dropped in GLDv3. Thus attempts to start DHCP on these interfaces in the non-global zone will time out. Examples -------- root@gz# zonecfg -z tz1 zonecfg:tz1> info zonename: tz1 zonepath: /rpool/zones/tz1 brand: ipkg autoboot: false bootargs: pool: limitpriv: scheduling-class: ip-type: exclusive hostid: net: address: 11.1.1.1/24 physical: vnic0 defrouter: 11.1.1.2 net: address: taddr2 physical: vnic0 defrouter not specified net: address not specified physical: vnic1 defrouter not specified In this example, the zone 'tz1' has been assigned 2 interfaces: vnic0 and vnic1. The vnic1 interface does not have any IP address or default router assigned to it, so that it may be configured in any desired way from within the zone using ifconfig(1m), ipadm(1m), NWAM etc. The vnic0 interface has 2 addresses assigned: 11.1.1.1/24 (with default router 11.1.1.2 on the 11.1.1.0/24 subnet) and the address 'taddr2' whose numeric value and netmask will be resolved when the zone boots. Thus, if, at the time of zone boot, the global-zone is also configured as: root@gz# egrep 'hosts|netmask' /etc/nsswitch.conf hosts: files nis dns netmasks: files nis root@gz# grep taddr2 /etc/hosts 12.2.3.4 taddr2 root@gz# grep 12 /etc/netmasks 12.2.0.0 255.255.0.0 tz1 will be booted as root@gz# zoneadm -z tz1 boot root@gz# zlogin tz1 ipadm show-addr ADDROBJ TYPE STATE ADDR vnic0/_a from-gz ok 11.1.1.1/24 vnic0/_b from-gz ok 12.2.3.4/16 lo0/v4 static ok 127.0.0.1/8 lo0/v6 static ok ::1/128 root@gz# zlogin tz1 ipadm show-if IFNAME STATE CURRENT PERSISTENT vnic0 ok bm-------Z4- -4- lo0 ok -m-v------46 --- root@gz# zlogin tz1 netstat -rn Routing Table: IPv4 Destination Gateway Flags Ref Use Interface -------------------- -------------------- ----- ----- ---------- --------- default 11.1.1.2 UGZ 2 2 vnic0 11.1.1.0 11.1.1.1 U 3 2 vnic0 12.2.0.0 12.2.3.4 U 2 0 vnic0 127.0.0.1 127.0.0.1 UH 2 0 lo0 Routing Table: IPv6 Destination/Mask Gateway Flags Ref Use If --------------------------- ---------------------- ----- --- ------- ----- ::1 ::1 UH 2 0 lo0 roott@gz# zlogin tz1 route get default route to: default destination: default mask: default gateway: 11.1.1.2 interface: vnic0 flags: recvpipe sendpipe ssthresh rtt,ms rttvar,ms hopcount mtu expire Proposed man page changes: -------------------------- --- zonecfg.1m.orig Thu Apr 22 14:41:33 2010 +++ zonecfg.1m.new Mon May 17 17:09:46 2010 @@ -676,10 +676,17 @@ for the interface should be specified here. For an exclusive-IP zone, the physical property must be - set and the address and default router properties cannot - be set. + set and the address and default router properties may + be set. The interface specifed for the physical property + must not be in use in the global zone. If an address and + default router are specified via zonecfg(1m), these will be + applied to the interface when the non-global zone is booted, + and the non-global zone will not be able to apply any other + addresses to that interface nor transmit packets with a different + source address for that IP Version. A default router set up via + zonecfg(1m) may not be persistently deleted from within the + non-global zone using the -p flag with route(1m). - device: match Device name to match. @@ -1817,7 +1824,7 @@ svcadm(1M), sysidtool(1M), zfs(1M), zoneadm(1M), priv_str_to_set(3C), kstat(3KSTAT), vfstab(4), attri- butes(5), brands(5), fnmatch(5), lx(5), privileges(5), - resource_controls(5), zones(5) + resource_controls(5), zones(5), route(1m) System Administration Guide: Solaris Containers-Resource --- dladm.1m.txt Mon May 10 15:24:49 2010 +++ dladm.1m.txt.new Fri May 14 10:24:02 2010 @@ -4176,7 +4176,14 @@ option's link layer address must match the link's MAC address. + When a datalink has been protected by setting allowed-ips + to a set of one or more IP addresses, IP interfaces over + that datalink may not configure IP addresses outside the + allowed-ips set, or enable forwarding for the IP interface + and attempts to do so will fail with an EPERM error being + returned to the user. + dhcp-nospoof DHCP client ID (DUID for DHCPv6) and hardware --- netstat.orig Wed May 19 17:02:56 2010 +++ netstat.new Wed May 19 17:05:17 2010 @@ -740,6 +740,9 @@ I Indirect routes (gateway not directly reachable) esta- blished with the -indirect option. + Z (non-global zone only) The route was statically added + on boot based on routing information configured using + zonecfg(1m) in the global zone. If the -a option is specified, there will be routing entries @@ -881,7 +884,7 @@ iostat(1M), kstat(1M), mibiisa(1M), savecore(1M), vmstat(1M), hosts(4), inet_type(4), networks(4), proto- cols(4), services(4), attributes(5), dhcp(5), kstat(7D), - inet(7P), inet6(7P) + inet(7P), inet6(7P), zonecfg(1m) Droms, R., RFC 2131, Dynamic Host Configuration Protocol, --- ipadm.1m.orig Tue May 25 09:12:35 2010 +++ ipadm.1m.new Tue May 25 10:34:55 2010 @@ -321,10 +321,11 @@ a VRRP interface is in accept mode (~IFF_NOACCEPT) + Z Layer-3 protection of IP addresses for the + interface has been administratively enforced. - SunOS 5.11 Last change: 29 Mar 2010 5 @@ -891,9 +892,13 @@ TYPE Type of the address object. It will be one of: - static, dhcp, or addrconf. It corresponds to the - type of the address object specified by the -T - option of create-addr. + from-gz, static, dhcp, or addrconf. The static, + dhcp and addrconf types correspond to the type of the + address object specified by the -T option of + create-addr. The "from-gz" type will only be displayed + in non-global zones, and indicates that the address + was configured based on IP address information + allocated to the non-global zone from the global zone. STATE --- ifconfig.1m.orig Tue May 25 11:05:48 2010 +++ ifconfig.1m.new Tue May 25 11:07:31 2010 @@ -1599,7 +1599,13 @@ used on multiple interfaces. (See also the usesrc option.) + L3PROTECT + Indicates that Layer-3 protection has been enforced on + the physical interface using the 'allowed-ips' link + property in dladm(1m) + + LOGICAL INTERFACES Solaris TCP/IP allows multiple logical interfaces to be associated with a physical network interface. This allows a References: ---------- [PSARC/2002/174] Virtualization and Namespace Isolation in Solaris, aka "Zones" [PSARC/2003/621] Zone Administration Updates [PSARC/2008/057] Default Route For Zones [PSARC/2009/436] Anti-spoofing Link Protection [PSARC/2010/111] exclusive IP for s10c [INSTALL] http://solaris-networking.sfbay/index.php/Network_Configuration:install [ZONES-NET] http://solaris-networking.sfbay/index.php/Zones_Networking --Boundary_(ID_GY+fWfN5BSYoHHppHebApw)-- From edward.pilatowicz@oracle.com Tue Jun 1 13:42:53 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o51KgrbC022661 for ; Tue, 1 Jun 2010 13:42:53 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o51KgpcO011009 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Tue, 1 Jun 2010 15:42:53 -0500 (CDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L3C00B0RS7G4M00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 01 Jun 2010 13:42:52 -0700 (PDT) Received: from jurassic.Eng.Sun.COM ([129.146.17.63]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L3C00MEDS7FFG30@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 01 Jun 2010 13:42:51 -0700 (PDT) Received: from jurassic.Eng.Sun.COM (localhost [127.0.0.1]) by jurassic.Eng.Sun.COM (8.14.4+Sun/8.14.4) with ESMTP id o51KgogR556112 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 01 Jun 2010 13:42:50 -0700 (PDT) Received: (from edp@localhost) by jurassic.Eng.Sun.COM (8.14.4+Sun/8.14.4/Submit) id o51KgoWT556111; Tue, 01 Jun 2010 13:42:50 -0700 (PDT) Date: Tue, 01 Jun 2010 13:42:50 -0700 From: Edward Pilatowicz Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <20100525150930.GA18828@quasimodo.East.Sun.COM> To: sowmini.varadhan@oracle.com Cc: Erik Nordmark , PSARC-ext Message-id: <20100601204250.GA555205@jurassic.Eng.Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <20100513012624.GJ176566@jurassic.Eng.Sun.COM> <20100513022313.GB25937@quasimodo.East.Sun.COM> <20100513202544.GB350787@jurassic.Eng.Sun.COM> <20100513202920.GG27083@quasimodo.East.Sun.COM> <20100514142056.GA249@quasimodo.East.Sun.COM> <4BF4CA99.6070409@oracle.com> <20100520130743.GD9994@quasimodo.East.Sun.COM> <4BF69113.60500@oracle.com> <20100524193357.GC15873@quasimodo.East.Sun.COM> <20100525150930.GA18828@quasimodo.East.Sun.COM> X-Authentication-warning: jurassic.Eng.Sun.COM: edp set sender to edward.pilatowicz@oracle.com using -f User-Agent: Mutt/1.5.20 (2009-06-14) Status: RO Content-Length: 1262 hey sowmini, so i had some concerns about one small bit below... On Tue, May 25, 2010 at 08:09:30AM -0700, sowmini.varadhan@oracle.com wrote: > +'allowed-ips' property. On the first boot of an installed non-global > +zone, the stored nvlist information for 'address' will be retrieved by > +the ipmgmtd daemon which will create the interface persistently, and > +apply the IP addresses non-persistently before any other IP > +configuration is applied. On subsequent boots, any IP interface with > +persistent configuration in the ipadm data-store will be recreated using > +IP address information from the kernel's nvlists set up by zoneadmd. > + i'm concerned about the "on the first boot" bit above. i think that condition may be hard to detect due to zones life cycle. some examples: - if a detach and attach a zone and then boot it, is that a first boot? - if a detach and attach a zone on another host and then boot it, is that a first boot? - if a install a zone via cloning of another zone and then boot it, is that a first boot? it seems that above you've already specified the default behavior for all subsequent boots. so is there any reason this same behavior can't be applied to all boots, without having to identify a "first boot"? ed From sowmini.varadhan@oracle.com Wed Jun 2 03:10:21 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o52AALwE027858 for ; Wed, 2 Jun 2010 03:10:21 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o52AAKI3010141 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 2 Jun 2010 04:10:20 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L3D00E3XTL8SH00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 02 Jun 2010 03:10:20 -0700 (PDT) Received: from quasimodo.East.Sun.COM ([129.148.174.94]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L3D000LYTL6GK20@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 02 Jun 2010 03:10:19 -0700 (PDT) Received: from quasimodo.East.Sun.COM (localhost [127.0.0.1]) by quasimodo.East.Sun.COM (8.14.4+Sun/8.14.4) with ESMTP id o52A3CfF000222; Wed, 02 Jun 2010 06:03:12 -0400 (EDT) Received: (from sowmini@localhost) by quasimodo.East.Sun.COM (8.14.4+Sun/8.14.4/Submit) id o52A3Co6000221; Wed, 02 Jun 2010 06:03:12 -0400 (EDT) Date: Wed, 02 Jun 2010 06:03:12 -0400 From: sowmini.varadhan@oracle.com Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <20100601204250.GA555205@jurassic.Eng.Sun.COM> To: Edward Pilatowicz Cc: Erik Nordmark , PSARC-ext Message-id: <20100602100312.GA203@quasimodo.East.Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <20100513022313.GB25937@quasimodo.East.Sun.COM> <20100513202544.GB350787@jurassic.Eng.Sun.COM> <20100513202920.GG27083@quasimodo.East.Sun.COM> <20100514142056.GA249@quasimodo.East.Sun.COM> <4BF4CA99.6070409@oracle.com> <20100520130743.GD9994@quasimodo.East.Sun.COM> <4BF69113.60500@oracle.com> <20100524193357.GC15873@quasimodo.East.Sun.COM> <20100525150930.GA18828@quasimodo.East.Sun.COM> <20100601204250.GA555205@jurassic.Eng.Sun.COM> X-Authentication-warning: quasimodo.East.Sun.COM: sowmini set sender to sowmini.varadhan@oracle.com using -f User-Agent: Mutt/1.5.14 (2007-03-31) Status: RO Content-Length: 1630 On (06/01/10 13:42), Edward Pilatowicz wrote: > > hey sowmini, > > so i had some concerns about one small bit below... : > i'm concerned about the "on the first boot" bit above. i think that > condition may be hard to detect due to zones life cycle. some examples: > - if a detach and attach a zone and then boot it, is that a first boot? > - if a detach and attach a zone on another host and then boot it, is > that a first boot? > - if a install a zone via cloning of another zone and then boot it, is > that a first boot? The "first boot" state is actually detected by an internal private property inside ipmgmtd/ipadm, and the property is initialized exactly once. So the latter 2 cases would constitute a "first boot" whereas the first one would not. > it seems that above you've already specified the default behavior for > all subsequent boots. so is there any reason this same behavior can't > be applied to all boots, without having to identify a "first boot"? The motivation for this (based on feedback from Erik) is primarily to allow the NGZ admin the flexibility to persistently modify interface properties on, or even completely disable, the datalinks handed down from the GZ to the NGZ. If we blindly set up the interface on every boot, the Admin would have to resort to /etc/rc/*.d scripts to do these things, which is less than optimal "Setting up persistent interface configuration on the first boot only" also allows us to make the zone boot procedure parallel the Install procedures so that, if/when we switch to using an Install profiles for zones down the road, the changes would be minor. From edward.pilatowicz@oracle.com Wed Jun 2 12:01:54 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o52J1sFt014874 for ; Wed, 2 Jun 2010 12:01:54 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o52J1r69020606 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@Sun.COM>; Wed, 2 Jun 2010 14:01:53 -0500 (CDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L3E00507I6NRM00@brm-avmta-1.central.sun.com> for PSARC-ext@Sun.COM (ORCPT PSARC-ext@Sun.COM); Wed, 02 Jun 2010 13:01:35 -0600 (MDT) Received: from jurassic.Eng.Sun.COM ([129.146.17.63]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L3E0013EI6MSM20@brm-avmta-1.central.sun.com> for PSARC-ext@Sun.COM (ORCPT PSARC-ext@Sun.COM); Wed, 02 Jun 2010 13:01:34 -0600 (MDT) Received: from jurassic.Eng.Sun.COM (localhost [127.0.0.1]) by jurassic.Eng.Sun.COM (8.14.4+Sun/8.14.4) with ESMTP id o52J1Ywi662055 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 02 Jun 2010 12:01:34 -0700 (PDT) Received: (from edp@localhost) by jurassic.Eng.Sun.COM (8.14.4+Sun/8.14.4/Submit) id o52J1YSK662054; Wed, 02 Jun 2010 12:01:34 -0700 (PDT) Date: Wed, 02 Jun 2010 12:01:34 -0700 From: Edward Pilatowicz Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <20100602100312.GA203@quasimodo.East.Sun.COM> To: sowmini.varadhan@oracle.com Cc: Erik Nordmark , PSARC-ext Message-id: <20100602190134.GC651532@jurassic.Eng.Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <20100513202544.GB350787@jurassic.Eng.Sun.COM> <20100513202920.GG27083@quasimodo.East.Sun.COM> <20100514142056.GA249@quasimodo.East.Sun.COM> <4BF4CA99.6070409@oracle.com> <20100520130743.GD9994@quasimodo.East.Sun.COM> <4BF69113.60500@oracle.com> <20100524193357.GC15873@quasimodo.East.Sun.COM> <20100525150930.GA18828@quasimodo.East.Sun.COM> <20100601204250.GA555205@jurassic.Eng.Sun.COM> <20100602100312.GA203@quasimodo.East.Sun.COM> X-Authentication-warning: jurassic.Eng.Sun.COM: edp set sender to edward.pilatowicz@oracle.com using -f User-Agent: Mutt/1.5.20 (2009-06-14) Status: RO Content-Length: 2928 On Wed, Jun 02, 2010 at 03:03:12AM -0700, sowmini.varadhan@oracle.com wrote: > On (06/01/10 13:42), Edward Pilatowicz wrote: > > > > hey sowmini, > > > > so i had some concerns about one small bit below... > : > > i'm concerned about the "on the first boot" bit above. i think that > > condition may be hard to detect due to zones life cycle. some examples: > > - if a detach and attach a zone and then boot it, is that a first boot? > > - if a detach and attach a zone on another host and then boot it, is > > that a first boot? > > - if a install a zone via cloning of another zone and then boot it, is > > that a first boot? > > The "first boot" state is actually detected by an internal private property > inside ipmgmtd/ipadm, and the property is initialized exactly once. > So the latter 2 cases would constitute a "first boot" whereas the first > one would not. > how is case 1 differentiated from case 2? last i check the zones framework doesn't differentiate the two cases. also, if the property is stored in ipmgmtd/ipadm, presumably it is stored in the ngz. (since each ngz has it's own ipmgmtd.) so how does this property get cleared during detach/attach and after a clone? > > it seems that above you've already specified the default behavior for > > all subsequent boots. so is there any reason this same behavior can't > > be applied to all boots, without having to identify a "first boot"? > > The motivation for this (based on feedback from Erik) is primarily > to allow the NGZ admin the flexibility to persistently modify > interface properties on, or even completely disable, the datalinks > handed down from the GZ to the NGZ. If we blindly set up the interface > on every boot, the Admin would have to resort to /etc/rc/*.d scripts > to do these things, which is less than optimal > > "Setting up persistent interface configuration on the first boot only" > also allows us to make the zone boot procedure parallel the Install > procedures so that, if/when we switch to using an Install profiles for > zones down the road, the changes would be minor. ok. so i understand the motivation, but i'm not sure from an implementation perspective it's possible. also, this behavior seems inconsistent with the behavior we have for default routes. for default routes specified via zonecfg, we already have the limitation that the ngz is free to delete a zonecfg specified default, but the route will re-appear after the zone is rebooted. i'm ok if we have the same behavior for network interfaces ip addresses specified via zonecfg. really, having the zone administrator change these within the zone persistently is just a workaround, if persistent change is needed it should be made via zonecfg. of course, this argument doesn't apply to other datalink properties not specified via zonecfg. for those, the user should be able to specify them and have them persist across zone reboots. ed From sowmini.varadhan@oracle.com Wed Jun 2 12:21:10 2010 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o52JL9hl015206 for ; Wed, 2 Jun 2010 12:21:09 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o52JL9xP013039 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@Sun.COM>; Wed, 2 Jun 2010 12:21:09 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L3E00701J39EB00@brm-avmta-1.central.sun.com> for PSARC-ext@Sun.COM (ORCPT PSARC-ext@Sun.COM); Wed, 02 Jun 2010 13:21:09 -0600 (MDT) Received: from quasimodo.East.Sun.COM ([129.148.174.94]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L3E001L8J38SH30@brm-avmta-1.central.sun.com> for PSARC-ext@Sun.COM (ORCPT PSARC-ext@Sun.COM); Wed, 02 Jun 2010 13:21:09 -0600 (MDT) Received: from quasimodo.East.Sun.COM (localhost [127.0.0.1]) by quasimodo.East.Sun.COM (8.14.4+Sun/8.14.4) with ESMTP id o52JE4Mh002574; Wed, 02 Jun 2010 15:14:04 -0400 (EDT) Received: (from sowmini@localhost) by quasimodo.East.Sun.COM (8.14.4+Sun/8.14.4/Submit) id o52JE4Yl002573; Wed, 02 Jun 2010 15:14:04 -0400 (EDT) Date: Wed, 02 Jun 2010 15:14:04 -0400 From: sowmini.varadhan@oracle.com Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <20100602190134.GC651532@jurassic.Eng.Sun.COM> To: Edward Pilatowicz Cc: Erik Nordmark , PSARC-ext Message-id: <20100602191404.GA2537@quasimodo.East.Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <20100513202920.GG27083@quasimodo.East.Sun.COM> <20100514142056.GA249@quasimodo.East.Sun.COM> <4BF4CA99.6070409@oracle.com> <20100520130743.GD9994@quasimodo.East.Sun.COM> <4BF69113.60500@oracle.com> <20100524193357.GC15873@quasimodo.East.Sun.COM> <20100525150930.GA18828@quasimodo.East.Sun.COM> <20100601204250.GA555205@jurassic.Eng.Sun.COM> <20100602100312.GA203@quasimodo.East.Sun.COM> <20100602190134.GC651532@jurassic.Eng.Sun.COM> X-Authentication-warning: quasimodo.East.Sun.COM: sowmini set sender to sowmini.varadhan@oracle.com using -f User-Agent: Mutt/1.5.14 (2007-03-31) Status: RO Content-Length: 4178 On (06/02/10 12:01), Edward Pilatowicz wrote: > > > condition may be hard to detect due to zones life cycle. some examples: > > > - if a detach and attach a zone and then boot it, is that a first boot? > > > - if a detach and attach a zone on another host and then boot it, is > > > that a first boot? > > > - if a install a zone via cloning of another zone and then boot it, is > > > that a first boot? > > > > The "first boot" state is actually detected by an internal private property > > inside ipmgmtd/ipadm, and the property is initialized exactly once. > > So the latter 2 cases would constitute a "first boot" whereas the first > > one would not. > > > > how is case 1 differentiated from case 2? last i check the zones > framework doesn't differentiate the two cases. > also, if the property is stored in ipmgmtd/ipadm, presumably it is > stored in the ngz. (since each ngz has it's own ipmgmtd.) so how does > this property get cleared during detach/attach and after a clone? yes, you are correct, there's no way to differentiate between the two, as I too later realized. And yes, the property is stored within the ngz. So unless we have some special hooks that clears the property before detach, it will not get cleared by itself. But that behavior would match the behavior of other properties (e.g., dladm/datalink.conf or ipadm.conf or other smf properties) across detach/attach of exclusive IP zones. > > The motivation for this (based on feedback from Erik) is primarily > > to allow the NGZ admin the flexibility to persistently modify > > interface properties on, or even completely disable, the datalinks > > handed down from the GZ to the NGZ. If we blindly set up the interface > > on every boot, the Admin would have to resort to /etc/rc/*.d scripts > > to do these things, which is less than optimal > > > > "Setting up persistent interface configuration on the first boot only" > > also allows us to make the zone boot procedure parallel the Install > > procedures so that, if/when we switch to using an Install profiles for > > zones down the road, the changes would be minor. > > > ok. so i understand the motivation, but i'm not sure from an > implementation perspective it's possible. also, this behavior seems not possible? But I already have a prototype for this (for both "do it on every boot" and "do it on first boot only") which correctly picks up the current zonecfg address/defrouter information at all times. What part do you think would not be possible? > inconsistent with the behavior we have for default routes. for default > routes specified via zonecfg, we already have the limitation that the > ngz is free to delete a zonecfg specified default, but the route will > re-appear after the zone is rebooted. i'm ok if we have the same The behavior matches the behavior for DHCP, where you always acquire both the IP address and default router from the "address acquisition protocol" (DHCP, or "from_gz", where the latter is set up as kernel nvlists). As with DHCP, if you acquire the from_gz address, you also get the default route. If, otoh, you persistently delete the from_gz interface using ipadm, you do not acquire either address or defrouter (which would be analogous to not starting DHCP). > behavior for network interfaces ip addresses specified via zonecfg. > really, having the zone administrator change these within the zone > persistently is just a workaround, if persistent change is needed it > should be made via zonecfg. that's perhaps a topic that we (you, Erik, I) should discuss offline this friday.. one thing to remember is that the changes to the value of the IP address or default router *must* always be made via zonecfg in the GZ, regardless of whether we configure the interfaces on every boot, or only on the first boot. It's only for persistent control of the IP interface properties that the "ipmgmtd is involved in setting up config on the first boot only" change was introduced. > of course, this argument doesn't apply to other datalink properties not > specified via zonecfg. for those, the user should be able to specify > them and have them persist across zone reboots. From edward.pilatowicz@oracle.com Wed Jun 2 14:04:16 2010 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o52L4G4E018198 for ; Wed, 2 Jun 2010 14:04:16 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o52L4FTW026025 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@Sun.COM>; Wed, 2 Jun 2010 14:04:16 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L3E00G03NV4A000@brm-avmta-1.central.sun.com> for PSARC-ext@Sun.COM (ORCPT PSARC-ext@Sun.COM); Wed, 02 Jun 2010 15:04:16 -0600 (MDT) Received: from jurassic.Eng.Sun.COM ([129.146.17.59]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L3E001GANV3SE70@brm-avmta-1.central.sun.com> for PSARC-ext@Sun.COM (ORCPT PSARC-ext@Sun.COM); Wed, 02 Jun 2010 15:04:15 -0600 (MDT) Received: from jurassic.Eng.Sun.COM (localhost [127.0.0.1]) by jurassic.Eng.Sun.COM (8.14.4+Sun/8.14.4) with ESMTP id o52L4F6p677509 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 02 Jun 2010 14:04:15 -0700 (PDT) Received: (from edp@localhost) by jurassic.Eng.Sun.COM (8.14.4+Sun/8.14.4/Submit) id o52L4FvH677508; Wed, 02 Jun 2010 14:04:15 -0700 (PDT) Date: Wed, 02 Jun 2010 14:04:14 -0700 From: Edward Pilatowicz Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <20100602191404.GA2537@quasimodo.East.Sun.COM> To: sowmini.varadhan@oracle.com Cc: Erik Nordmark , PSARC-ext Message-id: <20100602210414.GG651532@jurassic.Eng.Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <20100514142056.GA249@quasimodo.East.Sun.COM> <4BF4CA99.6070409@oracle.com> <20100520130743.GD9994@quasimodo.East.Sun.COM> <4BF69113.60500@oracle.com> <20100524193357.GC15873@quasimodo.East.Sun.COM> <20100525150930.GA18828@quasimodo.East.Sun.COM> <20100601204250.GA555205@jurassic.Eng.Sun.COM> <20100602100312.GA203@quasimodo.East.Sun.COM> <20100602190134.GC651532@jurassic.Eng.Sun.COM> <20100602191404.GA2537@quasimodo.East.Sun.COM> X-Authentication-warning: jurassic.Eng.Sun.COM: edp set sender to edward.pilatowicz@oracle.com using -f User-Agent: Mutt/1.5.20 (2009-06-14) Status: RO Content-Length: 5108 On Wed, Jun 02, 2010 at 12:14:04PM -0700, sowmini.varadhan@oracle.com wrote: > On (06/02/10 12:01), Edward Pilatowicz wrote: > > > > condition may be hard to detect due to zones life cycle. some examples: > > > > - if a detach and attach a zone and then boot it, is that a first boot? > > > > - if a detach and attach a zone on another host and then boot it, is > > > > that a first boot? > > > > - if a install a zone via cloning of another zone and then boot it, is > > > > that a first boot? > > > > > > The "first boot" state is actually detected by an internal private property > > > inside ipmgmtd/ipadm, and the property is initialized exactly once. > > > So the latter 2 cases would constitute a "first boot" whereas the first > > > one would not. > > > > > > > how is case 1 differentiated from case 2? last i check the zones > > framework doesn't differentiate the two cases. > > also, if the property is stored in ipmgmtd/ipadm, presumably it is > > stored in the ngz. (since each ngz has it's own ipmgmtd.) so how does > > this property get cleared during detach/attach and after a clone? > > yes, you are correct, there's no way to differentiate between the two, > as I too later realized. And yes, the property is stored within > the ngz. So unless we have some special hooks that clears the property > before detach, it will not get cleared by itself. > > But that behavior would match the behavior of other properties > (e.g., dladm/datalink.conf or ipadm.conf or other smf properties) across > detach/attach of exclusive IP zones. > > > > The motivation for this (based on feedback from Erik) is primarily > > > to allow the NGZ admin the flexibility to persistently modify > > > interface properties on, or even completely disable, the datalinks > > > handed down from the GZ to the NGZ. If we blindly set up the interface > > > on every boot, the Admin would have to resort to /etc/rc/*.d scripts > > > to do these things, which is less than optimal > > > > > > "Setting up persistent interface configuration on the first boot only" > > > also allows us to make the zone boot procedure parallel the Install > > > procedures so that, if/when we switch to using an Install profiles for > > > zones down the road, the changes would be minor. > > > > > > ok. so i understand the motivation, but i'm not sure from an > > implementation perspective it's possible. also, this behavior seems > > not possible? But I already have a prototype for this (for both > "do it on every boot" and "do it on first boot only") which correctly > picks up the current zonecfg address/defrouter information at all > times. What part do you think would not be possible? > i'm thinking that the "on first boot only" is very difficult to detect, even after you define "first boot" (which i think is also non-trivial). also, "first boot" wouldn't take into account changes made via zonecfg(1m). > > inconsistent with the behavior we have for default routes. for default > > routes specified via zonecfg, we already have the limitation that the > > ngz is free to delete a zonecfg specified default, but the route will > > re-appear after the zone is rebooted. i'm ok if we have the same > > The behavior matches the behavior for DHCP, where you always acquire > both the IP address and default router from the "address acquisition > protocol" (DHCP, or "from_gz", where the latter is set up as kernel nvlists). > As with DHCP, if you acquire the from_gz address, you also get the default > route. If, otoh, you persistently delete the from_gz interface > using ipadm, you do not acquire either address or defrouter (which > would be analogous to not starting DHCP). > last i check, dhcp and l3 protect are incompatible. if you specify an ip address in zonecfg, you get l3 protect and you can't use dhcp what i was trying to compare this to is the default route behavior of: if you have an interface configured with a default route in zonecfg, and you delete that default route from with the zone, and then reboot the zone, that default route will re-appear. ie, you can't persistently delete just a default route. i think that this same behavior would be fine wrt network interfaces. i mean, what is the use case for assigning an interface to a zone via zonecfg if the admin of that zone is going to persistently disable that interface? > > behavior for network interfaces ip addresses specified via zonecfg. > > really, having the zone administrator change these within the zone > > persistently is just a workaround, if persistent change is needed it > > should be made via zonecfg. > > that's perhaps a topic that we (you, Erik, I) should discuss offline > this friday.. > > one thing to remember is that the changes to the value of the IP > address or default router *must* always be made via zonecfg in the GZ, > regardless of whether we configure the interfaces on every boot, or > only on the first boot. It's only for persistent control of the IP > interface properties that the "ipmgmtd is involved in setting up > config on the first boot only" change was introduced. > sure. ed From sowmini.varadhan@oracle.com Wed Jun 2 16:31:43 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o52NVhgE023383 for ; Wed, 2 Jun 2010 16:31:43 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o52NVg1V034868 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@Sun.COM>; Wed, 2 Jun 2010 17:31:43 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L3E00A01UOVBA00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@Sun.COM (ORCPT PSARC-ext@Sun.COM); Wed, 02 Jun 2010 16:31:43 -0700 (PDT) Received: from quasimodo.East.Sun.COM ([129.148.174.94]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L3E0082KUOUUY60@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@Sun.COM (ORCPT PSARC-ext@Sun.COM); Wed, 02 Jun 2010 16:31:42 -0700 (PDT) Received: from quasimodo.East.Sun.COM (localhost [127.0.0.1]) by quasimodo.East.Sun.COM (8.14.4+Sun/8.14.4) with ESMTP id o52NOZSj002765; Wed, 02 Jun 2010 19:24:35 -0400 (EDT) Received: (from sowmini@localhost) by quasimodo.East.Sun.COM (8.14.4+Sun/8.14.4/Submit) id o52NOZrK002764; Wed, 02 Jun 2010 19:24:35 -0400 (EDT) Date: Wed, 02 Jun 2010 19:24:35 -0400 From: sowmini.varadhan@oracle.com Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <20100602210414.GG651532@jurassic.Eng.Sun.COM> To: Edward Pilatowicz Cc: Erik Nordmark , PSARC-ext Message-id: <20100602232435.GB2728@quasimodo.East.Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <4BF4CA99.6070409@oracle.com> <20100520130743.GD9994@quasimodo.East.Sun.COM> <4BF69113.60500@oracle.com> <20100524193357.GC15873@quasimodo.East.Sun.COM> <20100525150930.GA18828@quasimodo.East.Sun.COM> <20100601204250.GA555205@jurassic.Eng.Sun.COM> <20100602100312.GA203@quasimodo.East.Sun.COM> <20100602190134.GC651532@jurassic.Eng.Sun.COM> <20100602191404.GA2537@quasimodo.East.Sun.COM> <20100602210414.GG651532@jurassic.Eng.Sun.COM> X-Authentication-warning: quasimodo.East.Sun.COM: sowmini set sender to sowmini.varadhan@oracle.com using -f User-Agent: Mutt/1.5.14 (2007-03-31) Status: RO Content-Length: 2954 On (06/02/10 14:04), Edward Pilatowicz wrote: > i'm thinking that the "on first boot only" is very difficult to detect, > even after you define "first boot" (which i think is also non-trivial). > also, "first boot" wouldn't take into account changes made via > zonecfg(1m). The implementation detail is that ipmgmtd will set a persistent private property on the first boot. On each boot it checks if the property is set - if it is set, then it is not the first boot. Else it is the first boot. Thus it is implementable, and works. > > The behavior matches the behavior for DHCP, where you always acquire > > both the IP address and default router from the "address acquisition > > protocol" (DHCP, or "from_gz", where the latter is set up as kernel nvlists). > > As with DHCP, if you acquire the from_gz address, you also get the default > > route. If, otoh, you persistently delete the from_gz interface > > using ipadm, you do not acquire either address or defrouter (which > > would be analogous to not starting DHCP). > > > > last i check, dhcp and l3 protect are incompatible. if you specify an > ip address in zonecfg, you get l3 protect and you can't use dhcp I did not say this *was* DHCP. I said it was *like* dhcp in perceived (address acquisition) behavior.. FWIW, to quote Erik's mail > From erik.nordmark@oracle.com Wed May 19 22:38:55 2010 : > These addresses provided and enforced by the GZ are provided by an > external entity (the GZ) the same way as the addresses in a dhcp or > addrconf address object are provided by an external entity The keywords are "in the same way as", which is not the same as "is". > what i was trying to compare this to is the default route behavior of: > if you have an interface configured with a default route in zonecfg, and > you delete that default route from with the zone, and then reboot the > zone, that default route will re-appear. ie, you can't persistently > delete just a default route. If the interface is configured with an IP address set up by zonecfg, then the default router set up with that IP address will appear. If the interface has been deleted using 'ipadm delete-if', neither the from_gz IP address nor the default route will be configured. Though this is *not* DHCP, it is identical in behavior to: if I got my address from a dhcp server, then I would get any default route that dhcpserver handed with the address. If I delete the default route, the next time I refresh the dhcp address, the default route is added back. If I drop my dhcp address on the IP interface (or unplumb it), the default route will not get created. > i think that this same behavior would be > fine wrt network interfaces. i mean, what is the use case for assigning > an interface to a zone via zonecfg if the admin of that zone is going to > persistently disable that interface? I'll let Erik take that one, since he felt it was an important use case. --Sowmini From erik.nordmark@oracle.com Wed Jun 2 23:42:15 2010 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o536gF4b029636 for ; Wed, 2 Jun 2010 23:42:15 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o536gEID018687 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 2 Jun 2010 23:42:14 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L3F00505EMERD00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 02 Jun 2010 23:42:14 -0700 (PDT) Received: from sca-ea-mail-1.sun.com ([192.18.43.24]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L3F00EEVEMEP9E0@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 02 Jun 2010 23:42:14 -0700 (PDT) Received: from rcsinet13.oracle.com (rcsinet13.oracle.com [148.87.113.125]) by sca-ea-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o536gD0Q026888 for ; Thu, 03 Jun 2010 06:42:14 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by rcsinet13.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o533GU02022927 for ; Thu, 03 Jun 2010 06:42:13 +0000 (GMT) Received: from abhmt004.oracle.com by acsmt353.oracle.com with ESMTP id 318320521275547332; Wed, 02 Jun 2010 23:42:12 -0700 Received: from [10.7.251.248] (/10.7.251.248) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 02 Jun 2010 23:42:11 -0700 Date: Wed, 02 Jun 2010 23:42:05 -0700 From: Erik Nordmark Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <20100602100312.GA203@quasimodo.East.Sun.COM> To: sowmini.varadhan@oracle.com Cc: Edward Pilatowicz , PSARC-ext Message-id: <4C074EBD.4000909@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090208.4C074EC5.017B:SCFMA4539814,ss=1,fgs=0 References: <20100513022313.GB25937@quasimodo.East.Sun.COM> <20100513202544.GB350787@jurassic.Eng.Sun.COM> <20100513202920.GG27083@quasimodo.East.Sun.COM> <20100514142056.GA249@quasimodo.East.Sun.COM> <4BF4CA99.6070409@oracle.com> <20100520130743.GD9994@quasimodo.East.Sun.COM> <4BF69113.60500@oracle.com> <20100524193357.GC15873@quasimodo.East.Sun.COM> <20100525150930.GA18828@quasimodo.East.Sun.COM> <20100601204250.GA555205@jurassic.Eng.Sun.COM> <20100602100312.GA203@quasimodo.East.Sun.COM> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.9) Gecko/20100510 Lightning/1.0b1 Thunderbird/3.0.4 Status: RO Content-Length: 1660 On 06/ 2/10 03:03 AM, sowmini.varadhan@oracle.com wrote: > On (06/01/10 13:42), Edward Pilatowicz wrote: >> >> hey sowmini, >> >> so i had some concerns about one small bit below... > : >> i'm concerned about the "on the first boot" bit above. i think that >> condition may be hard to detect due to zones life cycle. some examples: >> - if a detach and attach a zone and then boot it, is that a first boot? >> - if a detach and attach a zone on another host and then boot it, is >> that a first boot? >> - if a install a zone via cloning of another zone and then boot it, is >> that a first boot? > > The "first boot" state is actually detected by an internal private property > inside ipmgmtd/ipadm, and the property is initialized exactly once. > So the latter 2 cases would constitute a "first boot" whereas the first > one would not. I don't understand hoe #2 and #3 can be viewed as "first boot" since the ipadm persistent properties are stored in the ngz hence are as preserved as the zone is moved. The desired behavior is that "first boot" applies in the same cases as where the ngz looks for sysidcfg, i.e., on the first boot after install of the zone (where cloning is merely a particular way to install a zone). Note that for the cloned zones case (3rd one above) we have a bug in that sys-unconfig doesn't know about persistent ipadm (and dladm) information, but does know about /etc/hostname files. If sys-unconfig is taught that, and the result of sys-unconfig is to remove all the persistent ipadm properties including this private "first boot" property, then we'd end up with behavior that is self-consistent. Erik From erik.nordmark@oracle.com Thu Jun 3 00:04:08 2010 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o53748d6000224 for ; Thu, 3 Jun 2010 00:04:08 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o53747AQ025964 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Thu, 3 Jun 2010 00:04:08 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L3F0021JFMWZN00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 03 Jun 2010 01:04:08 -0600 (MDT) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L3F00BA9FMVSID0@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 03 Jun 2010 01:04:07 -0600 (MDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o53747mW020908 for ; Thu, 03 Jun 2010 07:04:07 +0000 (GMT) Received: from acsmt353.oracle.com (acsmt353.oracle.com [141.146.40.153]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o537447r023685 for ; Thu, 03 Jun 2010 07:04:04 +0000 (GMT) Received: from abhmt016.oracle.com by acsmt354.oracle.com with ESMTP id 292673471275548586; Thu, 03 Jun 2010 00:03:06 -0700 Received: from [10.7.251.248] (/10.7.251.248) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 03 Jun 2010 00:03:05 -0700 Date: Thu, 03 Jun 2010 00:03:01 -0700 From: Erik Nordmark Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <20100602210414.GG651532@jurassic.Eng.Sun.COM> To: Edward Pilatowicz Cc: sowmini.varadhan@oracle.com, PSARC-ext Message-id: <4C0753A5.60600@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt353.oracle.com [141.146.40.153] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090206.4C0753E6.00D1:SCFMA4539814,ss=1,fgs=0 References: <20100514142056.GA249@quasimodo.East.Sun.COM> <4BF4CA99.6070409@oracle.com> <20100520130743.GD9994@quasimodo.East.Sun.COM> <4BF69113.60500@oracle.com> <20100524193357.GC15873@quasimodo.East.Sun.COM> <20100525150930.GA18828@quasimodo.East.Sun.COM> <20100601204250.GA555205@jurassic.Eng.Sun.COM> <20100602100312.GA203@quasimodo.East.Sun.COM> <20100602190134.GC651532@jurassic.Eng.Sun.COM> <20100602191404.GA2537@quasimodo.East.Sun.COM> <20100602210414.GG651532@jurassic.Eng.Sun.COM> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.9) Gecko/20100510 Lightning/1.0b1 Thunderbird/3.0.4 Status: RO Content-Length: 2360 On 06/ 2/10 02:04 PM, Edward Pilatowicz wrote: > what i was trying to compare this to is the default route behavior of: > if you have an interface configured with a default route in zonecfg, and > you delete that default route from with the zone, and then reboot the > zone, that default route will re-appear. ie, you can't persistently > delete just a default route. i think that this same behavior would be > fine wrt network interfaces. i mean, what is the use case for assigning > an interface to a zone via zonecfg if the admin of that zone is going to > persistently disable that interface? The model we are trying to follow is the same as DHCP - the other case of externally provided IP address configuration. Basically the gz statement about the container in which the ngz can run is of the form "on bge0 zoneA can not use any IP addresses other than X and Y". (The similar statement for a DHCP server is that when asked by clientA, it will provide IP address Z.) But those statements don't actually say that zoneA is required to use its interface on bge0 (and nor does the DHCP server configuration require the client to ask the DHCP server). Hence just like a DHCP client can choose to not use bge0 (and use its other datalinks instead), a ngz should be able to choose to not use its vnic on bge0 (and use its other datalinks instead). Basically the gz provides constraints on which IP addresses can be used on a datalink, but neither mandates that that datalink be used nor mandates that all the IP addresses on the datalink be used by the ngz. This is somewhat analogous to providing a mountpoint to a ngz (the gz constrains which mountpoints are available to the ngz, but can't force the applications to actually access files under those mountpoints.) The primary purpose of the zonecfg IP address configuration for exclusive-IP zones is security, and having the ngz choose to not use the datalink doesn't introduce a security issue. In terms of the secondary purpose of ease of configuration, we are trying to find a balance between the gz being helpful and the ngz having autonomy. That has lead us down the path of following the DHCP model where the set of datalinks is initialized on first boot after install/clone, but the set of IP addresses are more dynamic. Let's chat more about this on the phone. Erik From edward.pilatowicz@oracle.com Fri Jun 4 15:33:44 2010 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o54MXiOP024631 for ; Fri, 4 Jun 2010 15:33:44 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o54MXiDH021250 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@Sun.COM>; Fri, 4 Jun 2010 15:33:44 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L3I00C03HC88900@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@Sun.COM (ORCPT PSARC-ext@Sun.COM); Fri, 04 Jun 2010 15:33:44 -0700 (PDT) Received: from jurassic.Eng.Sun.COM ([129.146.17.63]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L3I004O5HC8D270@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@Sun.COM (ORCPT PSARC-ext@Sun.COM); Fri, 04 Jun 2010 15:33:44 -0700 (PDT) Received: from jurassic.Eng.Sun.COM (localhost [127.0.0.1]) by jurassic.Eng.Sun.COM (8.14.4+Sun/8.14.4) with ESMTP id o54MXhSQ917907 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 04 Jun 2010 15:33:43 -0700 (PDT) Received: (from edp@localhost) by jurassic.Eng.Sun.COM (8.14.4+Sun/8.14.4/Submit) id o54MXhjH917906; Fri, 04 Jun 2010 15:33:43 -0700 (PDT) Date: Fri, 04 Jun 2010 15:33:43 -0700 From: Edward Pilatowicz Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <4C0753A5.60600@oracle.com> To: Erik Nordmark Cc: sowmini.varadhan@oracle.com, PSARC-ext Message-id: <20100604223343.GA884889@jurassic.Eng.Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <20100520130743.GD9994@quasimodo.East.Sun.COM> <4BF69113.60500@oracle.com> <20100524193357.GC15873@quasimodo.East.Sun.COM> <20100525150930.GA18828@quasimodo.East.Sun.COM> <20100601204250.GA555205@jurassic.Eng.Sun.COM> <20100602100312.GA203@quasimodo.East.Sun.COM> <20100602190134.GC651532@jurassic.Eng.Sun.COM> <20100602191404.GA2537@quasimodo.East.Sun.COM> <20100602210414.GG651532@jurassic.Eng.Sun.COM> <4C0753A5.60600@oracle.com> X-Authentication-warning: jurassic.Eng.Sun.COM: edp set sender to edward.pilatowicz@oracle.com using -f User-Agent: Mutt/1.5.20 (2009-06-14) Status: RO Content-Length: 295 On Thu, Jun 03, 2010 at 12:03:01AM -0700, Erik Nordmark wrote: > > Let's chat more about this on the phone. just a heads up for the alias, i've been talking with erik and sowmini offline trying to converge. we hope to have things ironed out by this coming wed so that we can wrap this up. ed From sowmini.varadhan@oracle.com Sat Jun 12 07:03:00 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o5CE2wch012229 for ; Sat, 12 Jun 2010 07:02:58 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o5CE2v2C026221 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@Sun.COM>; Sat, 12 Jun 2010 09:02:58 -0500 (CDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L3W00B07N0XCC00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@Sun.COM (ORCPT PSARC-ext@Sun.COM); Sat, 12 Jun 2010 07:02:57 -0700 (PDT) Received: from quasimodo.East.Sun.COM ([129.148.174.94]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L3W008VDN0WXK10@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@Sun.COM (ORCPT PSARC-ext@Sun.COM); Sat, 12 Jun 2010 07:02:57 -0700 (PDT) Received: from quasimodo.East.Sun.COM (localhost [127.0.0.1]) by quasimodo.East.Sun.COM (8.14.4+Sun/8.14.4) with ESMTP id o5CDthpJ019636; Sat, 12 Jun 2010 09:55:43 -0400 (EDT) Received: (from sowmini@localhost) by quasimodo.East.Sun.COM (8.14.4+Sun/8.14.4/Submit) id o5CDthWA019635; Sat, 12 Jun 2010 09:55:43 -0400 (EDT) Date: Sat, 12 Jun 2010 09:55:43 -0400 From: sowmini.varadhan@oracle.com Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <20100602190134.GC651532@jurassic.Eng.Sun.COM> To: PSARC-ext@sun.com Cc: Erik Nordmark , Edward Pilatowicz , sowmini.varadhan@oracle.com Message-id: <20100612135543.GD19314@quasimodo.East.Sun.COM> MIME-version: 1.0 Content-type: multipart/mixed; boundary="Boundary_(ID_AxqLBYi029t6B6FNlWJHag)" Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <20100513202920.GG27083@quasimodo.East.Sun.COM> <20100514142056.GA249@quasimodo.East.Sun.COM> <4BF4CA99.6070409@oracle.com> <20100520130743.GD9994@quasimodo.East.Sun.COM> <4BF69113.60500@oracle.com> <20100524193357.GC15873@quasimodo.East.Sun.COM> <20100525150930.GA18828@quasimodo.East.Sun.COM> <20100601204250.GA555205@jurassic.Eng.Sun.COM> <20100602100312.GA203@quasimodo.East.Sun.COM> <20100602190134.GC651532@jurassic.Eng.Sun.COM> X-Authentication-warning: quasimodo.East.Sun.COM: sowmini set sender to sowmini.varadhan@oracle.com using -f User-Agent: Mutt/1.5.14 (2007-03-31) Status: RO Content-Length: 22509 --Boundary_(ID_AxqLBYi029t6B6FNlWJHag) Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline On (06/02/10 12:01), Edward Pilatowicz wrote: > > > so i had some concerns about one small bit below... following up on this... Ed, Erik and I have had several discussions about when the proposed IP address configuration of zonecfg IP address and default router parameters should be applied (first boot of zone vs every boot of zone). To summarize this in Ed's words: " we still have different opinions on how this functionality should work but we were able to come up with a compromise solution. that compromise would be to rename the proposed zonecfg net resource property from "address=" to "allowed-address=". this compromise would prevent user confusion resulting from the same zonecfg property having different semantics in shared vs exclusive ip stack zones." Updated material is enclosed. These updates reflect the rename from 'address' to 'allowed-address'. In addition, man pages have been modified to explicitly document how an exclusive IP NGZ may recover the current set of allowed-address values for a from-gz interfaces at any time. The primary differences between 'address' and 'allowed-address' are 1. "address" is always enforced for a shared-IP zone when it boots, and is always configured for the NGZ 2. "allowed-address" for an NGZ is configured via ipadm in the NGZ, and all interfaces in the NGZ are subsequently managed by ipadm. The NGZ may use 'ipadm delete-if' to persistently delete the assigned interface if desired, but it may not violate the resource controls applied from the GZ. For both "address" and "allowed-address", the NGZ may not use any IP addresses other than those specified by the [allowed-]address set. --Sowmini --Boundary_(ID_AxqLBYi029t6B6FNlWJHag) Content-type: text/plain; NAME=zonecfg.txt; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: attachment; filename=zonecfg.txt zonecfg(1m) enhancements for exclusive-IP zones Requested release binding: Minor Summary: ------- This case proposes a solution for 6944327 need to support address and defrouter resources for exclusive-IP zones Problem Description -------------------- Typical zone deployments that exist today use shared-IP zones to run applications and services like Apache or Weblogic in the contained environment provided by the shared-IP zone. In these use-cases, the Administrator in the global zone has full control over the networking resources used by the non-global zone. In the common case, networking is simply configured by specifying the IP interface, IP addresses and, optionally, the default routers from zonecfg(1m). The configuration resources thus supplied are then applied for the non-global zone when it is booted, and the non-global zone itself may not modify any of these configuration parameters. However, there is no such simple configuration mechanism for the simple networking use-case in place for exclusive IP zones, which must be configured through sysidcfg, ifconfig, and an assortment of other methods, all of which are not controllable from the global zone, and may not be managed through zonecfg(1m). The addition of many new virtualization and resource management features to Solaris (PSARC 2006/357, PSARC 2005/132 and associated cases) makes Exclusive-IP zones a cleaner and more powerful Zone model than shared-IP zones. Thus, bridging any gaps in the configuration methods for the simple use cases between shared- and exlusive-IP zones is important to ease the transition of shared-IP customer configurations to exlusive-IP. In addition, there are many ongoing projects in the "Zones Networking" effort [ZONES-NET] to facilitate the consolidation of existing Solaris 10 host installations as Solaris 10 Containers. These efforts would leverage from the ability to specify networking resource values for address and default router uniformly for zones using zonecfg(1m). This case proposes support in zonecfg(1m) for IP configuration through the support of 'allowed-address' and 'defrouter' properties in the 'net' resource for exclusive-IP zones. When the global-zone specifies the 'allowed-address' for an interface via zonecfg(1m), the non-global zone may not use any other IP addresses for that interface. The address information provided via zonecfg(1m) will be used to set up Layer-3 protection [PSARC/2009/436] for the non-global zone during zone-boot to filter out all other addresses for the selected interface. For instance, when zonecfg(1m) has been used in the global-zone to set one or more IPv4 addresses on an interface, an attempt to set an IPv4 or IPv6 address on the interface that is outside the globally defined set will encounter the EPERM failure. Thus ifconfig(1m), ipadm(1m), and associated ioctls will receive this error if they are used within the non-global zone to set addresses that are not in the set that is permitted from the global-zone, and attempts by the non-global zone to turn on forwarding on the interface will also encounter EPERM. Attempts to boot a zone that has already been configured for IP, or has previously customized values for Layer-3 protection [PSARC/2009/436] will fail. Implementation Overview: ------------------------ A brief overview of the implementation is provided here. Note that all interfaces between zoneadmd(1m) and the kernel/zonecfg(1m), are Private interfaces, subject to change in the future. When the non-global zone is booted, zoneadmd(1m) will store the information specified by the 'allowed-address' and 'defrouter' properties as nvlists in the kernel following a mechanism similar to that in use today for other zonecfg(1m) resources such as the "physical" datalink. At the same time, the zone boot process will also ensure 'ip-nospoof' protection for the datalink with the specified addresses used as input to the 'allowed-ips' property. On the first boot of an installed non-global native (ipkg) zone, and on every boot of a solaris10 branded zone [PSARC 2010/111], the stored nvlist information for 'allowed-address' will be retrieved by the ipmgmtd daemon which will create the interface (persistently, for native ipkg zones), and apply the IP addresses non-persistently before any other IP configuration is applied. On subsequent boots of native (ipkg) zones, any IP interface with persistent configuration in the ipadm data-store will be recreated using IP address information from the kernel's nvlists set up by zoneadmd. The 'defrouter' information set up by the global zone will be applied after the 'allowed-address' information has been configured, so that the behavior of the IP configuration scheme matches that of other address acquisition protocols like DHCP. Routes added by ipmgmtd will be identifiable in the 'netstat -r' output by the 'Z' flag which will indicate that the associated route was based on information configured in the global zone. The GLDv3 layer communicates the set of "allowed-ips" to the IP layer through DLPI notifications. The notification sent is a DL_NOTIFY_IND message of (Project Private) type DL_NOTE_ALLOWED_IPS, sent to the IP clients that have registered for this notification. The payload of the DL_NOTE_ALLOWED_IPS message is the mac_protect_t data-structure introduced by PSARC 2009/436. Receipt of this notification enables the IP layer to track the current set of permitted IP addresses per interface, so that the IP layer can return informative error diagnostics if an attempt is made to enable forwarding or set addresses on the interface that are outside the permitted set. IP interfaces with 'allowed-ips' constraints will be identifiable by the "L3PROTECT" flag in the output of ifconfig, and by a 'Z' flag in the output of 'ipadm show-if'. All configuration state, including datalink properties like protection, allowed-ips and zone, and associated nvlist information, that is created during zone boot will be reset to the default values when the zone is halted. Interfaces added by this case: ------------------------------ This case adds the following: - a new RTF_ZONE flag which will be passed down in the routing socket message sent by the ipmgmtd daemon in the non-global zone to add default route information set up via zonecfg(1m). The output of 'netstat -r' will display 'Z' when the route was added by ipmgmtd in this manner. The output of the 'route get' command will also display the "ZONE" string as part of the 'flags' field for the route. - a read-only interface flag IFF_L3PROTECT which will be set by the kernel on the IP interface when the 'allowed-ips' link property has been set for the interface. The IFF_L3PROTECT flag will be indicated by a 'Z' in the CURRENT column of the output from 'ipadm show-if' Interface Classification Comments _______________________________________________________________________ netstat(1m) -r output Uncommitted (unchanged) RTF_ZONE Committed route(1m) Uncommitted (unchanged) IFF_L3PROTECT Committed , read-only The output of 'ipadm show-addr' will be modified to display a TYPE of 'from-gz' when the address has been configured based on IP address information allocated to the non-global zone from the global zone. Relationship to other configuration mechanisms ---------------------------------------------- sysidcfg: The Solaris Install team is currently working in conjunction with the Network Configuration team [INSTALL] on the implementation of a framework to replace sysidcfg(4) that will provide configuration profiles to be processed by startup scripts when a system boots. However this configuration information is only applied on the first boot of the zone, and unlike zonecfg, may not be easily modified in the global zone across reboot. Moreover, there is no mechanism in place to prevent the non-global zone from modifying the configuration provided via profiles. As mentioned earlier, the global-zone is the authoritative source of IP configuration information for the "secure container" use-case. This implies that attempts to use boot profiles in the non-global zone to add addresses other than those mandated in the global-zone will fail with EINVAL erors. Note that the profiles defined in [INSTALL] may still be applied to interfaces for which 'net' resource properties have not been defined in the global-zone, i.e., this proposal is fully backward-compatible. svc:/network/physical:{default, nwam} Configuration specified via zonecfg(1m) in the global-zone overrides any information specified via /etc/hostname.intf, ipadm(1m) or nwam. Thus if the IP address information specified by these other methods is not a subset of the globally managed information, attempts to apply it within the non-global zone will encounter an error. DHCP: It is assumed that when IP resources for an IP interface have been set up in the global-zone using zonecfg(1m), then that interface has been marked for static IP configuration only, and attempts to send out outgoing DHCP DISCOVER packets will be dropped in GLDv3. Thus attempts to start DHCP on these interfaces in the non-global zone will time out. Examples -------- root@gz# zonecfg -z tz1 zonecfg:tz1> info zonename: tz1 zonepath: /rpool/zones/tz1 brand: ipkg autoboot: false bootargs: pool: limitpriv: scheduling-class: ip-type: exclusive hostid: net: allowed-address: 11.1.1.1/24 physical: vnic0 defrouter: 11.1.1.2 net: allowed-address: taddr2 physical: vnic0 defrouter not specified net: address not specified physical: vnic1 defrouter not specified In this example, the zone 'tz1' has been assigned 2 interfaces: vnic0 and vnic1. The vnic1 interface does not have any IP address or default router assigned to it, so that it may be configured in any desired way from within the zone using ifconfig(1m), ipadm(1m), NWAM etc. The vnic0 interface has 2 addresses assigned: 11.1.1.1/24 (with default router 11.1.1.2 on the 11.1.1.0/24 subnet) and the address 'taddr2' whose numeric value and netmask will be resolved when the zone boots. Thus, if, at the time of zone boot, the global-zone is also configured as: root@gz# egrep 'hosts|netmask' /etc/nsswitch.conf hosts: files nis dns netmasks: files nis root@gz# grep taddr2 /etc/hosts 12.2.3.4 taddr2 root@gz# grep 12 /etc/netmasks 12.2.0.0 255.255.0.0 tz1 will be booted as root@gz# zoneadm -z tz1 boot root@gz# zlogin tz1 ipadm show-addr ADDROBJ TYPE STATE ADDR vnic0/_a from-gz ok 11.1.1.1/24 vnic0/_b from-gz ok 12.2.3.4/16 lo0/v4 static ok 127.0.0.1/8 lo0/v6 static ok ::1/128 root@gz# zlogin tz1 ipadm show-if IFNAME STATE CURRENT PERSISTENT vnic0 ok bm-------Z4- -4- lo0 ok -m-v------46 --- root@gz# zlogin tz1 netstat -rn Routing Table: IPv4 Destination Gateway Flags Ref Use Interface -------------------- -------------------- ----- ----- ---------- --------- default 11.1.1.2 UGZ 2 2 vnic0 11.1.1.0 11.1.1.1 U 3 2 vnic0 12.2.0.0 12.2.3.4 U 2 0 vnic0 127.0.0.1 127.0.0.1 UH 2 0 lo0 Routing Table: IPv6 Destination/Mask Gateway Flags Ref Use If --------------------------- ---------------------- ----- --- ------- ----- ::1 ::1 UH 2 0 lo0 roott@gz# zlogin tz1 route get default route to: default destination: default mask: default gateway: 11.1.1.2 interface: vnic0 flags: recvpipe sendpipe ssthresh rtt,ms rttvar,ms hopcount mtu expire Proposed man page changes: -------------------------- --- zonecfg.1m.orig Thu Apr 22 14:41:33 2010 +++ zonecfg.1m.new Fri Jun 11 18:54:58 2010 @@ -625,7 +625,7 @@ The directory path. - net: address, physical, defrouter + net: address, allowed-address, physical, defrouter The network address and physical interface name of the network interface. The network address is one of: @@ -673,13 +673,33 @@ to booting the non-global zone. However, if the inter- face is not used by the global zone, it should be con- figured down in the global zone, and the default router - for the interface should be specified here. + for the interface should be specified here. The + 'allowed-address' property cannot be set for a shared-IP + zone For an exclusive-IP zone, the physical property must be - set and the address and default router properties cannot - be set. + set and the 'address' property may not be set. Optionally, + the set of IP addresses that the exclusive-IP zone can use + may be constrained by specifying the 'allowed-address' + property. If 'allowed-address' has not been specified, then the + exclusive-IP zone may use any IP address on the associated 'physical' + interface for the net resource. Otherwise, when 'allowed-address' + is specified, the exclusive-IP zone cannot use IP addresses that + are not in any of the 'allowed-address' list for the 'physical'. + In addition, when the 'allowed-address' has been specified, + the 'defrouter' property may also be optionally specified. The + interface specifed for the physical property must not be in + use in the global zone. If an allowed-address and default + router are specified via zonecfg(1m), these will be applied to + the interface when it is enabled via ipadm(1m) in the + non-global exclusive-IP zone, typically during zone boot. + The non-global exclusive-IP zone will not be able to apply any other + addresses to that interface nor transmit packets with a different + source address for the specified IP Version. A default router + set up via zonecfg(1m) may not be persistently deleted from + within the non-global exclusive-IP zone using the -p flag with + route(1m). - device: match Device name to match. @@ -1817,7 +1837,7 @@ svcadm(1M), sysidtool(1M), zfs(1M), zoneadm(1M), priv_str_to_set(3C), kstat(3KSTAT), vfstab(4), attri- butes(5), brands(5), fnmatch(5), lx(5), privileges(5), - resource_controls(5), zones(5) + resource_controls(5), zones(5), route(1m) System Administration Guide: Solaris Containers-Resource --- dladm.1m.orig Fri May 7 10:10:28 2010 +++ dladm.1m.new Fri Jun 11 08:20:35 2010 @@ -3746,17 +3746,25 @@ IP address anti-spoof. This protection type works in conjunction with the link property allowed-ips. - allowed-ips is a list containing IPv4 addresses. + allowed-ips is a list containing IP (IPv4 or IPv6) addresses. This list is empty by default. The unspecified address (all-zeros) is implicitly in this list. - An outbound IPv4 packet may pass if its source - address is in allowed-ips. + An outbound IP packet may pass if its source + address is in allowed-ips An outbound ARP packet may pass if its sender proto- col address is in allowed-ips. + When a datalink has been protected by setting allowed-ips + to a set of one or more IP addresses, any attempts to + configure IP addresses that are not in this set will + fail with an EPERM error being returned to the user. + Moreover, the interface may not be used for forwarding + IP packets, and attempts to set the ipadm(1m) forwarding + property on the interface will encounter EPERM error. + restricted This protection restricts outgoing packet types to @@ -4572,7 +4580,7 @@ SEE ALSO acctadm(1M), autopush(1M), ifconfig(1M), ipsecconf(1M), ndd(1M), vrrpadm(1M), psrset(1M), wpad(1M), zonecfg(1M), - attributes(5), ieee802.3(5), dlpi(7P) + attributes(5), ieee802.3(5), dlpi(7P), ipadm(1m) --- netstat.orig Wed May 19 17:02:56 2010 +++ netstat.new Fri Jun 11 06:21:08 2010 @@ -740,6 +740,9 @@ I Indirect routes (gateway not directly reachable) esta- blished with the -indirect option. + Z (non-global exclusive-IP zone only) The route was statically added + on boot based on routing information configured using + zonecfg(1m) in the global zone. If the -a option is specified, there will be routing entries @@ -881,7 +884,7 @@ iostat(1M), kstat(1M), mibiisa(1M), savecore(1M), vmstat(1M), hosts(4), inet_type(4), networks(4), proto- cols(4), services(4), attributes(5), dhcp(5), kstat(7D), - inet(7P), inet6(7P) + inet(7P), inet6(7P), zonecfg(1m) Droms, R., RFC 2131, Dynamic Host Configuration Protocol, --- ipadm.1m.orig Tue May 25 09:12:35 2010 +++ ipadm.1m.new Fri Jun 11 08:30:45 2010 @@ -321,10 +321,11 @@ a VRRP interface is in accept mode (~IFF_NOACCEPT) + Z Layer-3 protection of IP addresses for the + interface has been administratively enforced. - SunOS 5.11 Last change: 29 Mar 2010 5 @@ -891,9 +892,14 @@ TYPE Type of the address object. It will be one of: - static, dhcp, or addrconf. It corresponds to the - type of the address object specified by the -T - option of create-addr. + from-gz, static, dhcp, or addrconf. The static, + dhcp and addrconf types correspond to the type of the + address object specified by the -T option of + create-addr. The "from-gz" type will only be displayed + in non-global zones, and indicates that the address + was configured based on the 'allowed-address' property + configured for the non-global exclusive-IP zone from + the global zone. STATE @@ -2315,7 +2321,21 @@ address objects configured on that interface are enabled also. + The following command creates persistent configuration for the + net0 interface in a non-global exclusive-IP zone + so that the net0 interface will be configured with the set of + addresses made available through the 'allowed-address' resource + from the global zone on the next reboot. + # ipadm create-if net0 + + The net0 interface may also be configured with the available set + of 'allowed-address' values in the non-global exclusive-IP zone + without a reboot by executing the following commands: + + # ipadm disable-if -t net0 + # ipadm enable-if -t net0 + ATTRIBUTES See attributes(5) for descriptions of the following attri- butes: --- ifconfig.1m.orig Tue May 25 11:05:48 2010 +++ ifconfig.1m.new Tue May 25 11:07:31 2010 @@ -1599,7 +1599,13 @@ used on multiple interfaces. (See also the usesrc option.) + L3PROTECT + Indicates that Layer-3 protection has been enforced on + the physical interface using the 'allowed-ips' link + property in dladm(1m) + + LOGICAL INTERFACES Solaris TCP/IP allows multiple logical interfaces to be associated with a physical network interface. This allows a References: ---------- [PSARC/2002/174] Virtualization and Namespace Isolation in Solaris, aka "Zones" [PSARC/2003/621] Zone Administration Updates [PSARC/2008/057] Default Route For Zones [PSARC/2009/436] Anti-spoofing Link Protection [PSARC/2010/111] exclusive IP for s10c [INSTALL] http://solaris-networking.sfbay/index.php/Network_Configuration:install [ZONES-NET] http://solaris-networking.sfbay/index.php/Zones_Networking --Boundary_(ID_AxqLBYi029t6B6FNlWJHag)-- From sebastien.roy@oracle.com Tue Jun 22 12:47:25 2010 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o5MJlOgm003972 for ; Tue, 22 Jun 2010 12:47:24 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o5MJlOdk010026 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Tue, 22 Jun 2010 12:47:24 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L4F00103LN05E00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 22 Jun 2010 13:47:24 -0600 (MDT) Received: from sca-ea-mail-3.sun.com ([192.18.43.21]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L4F00JHYLMZJ340@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 22 Jun 2010 13:47:23 -0600 (MDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by sca-ea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o5MJlNpx008455 for ; Tue, 22 Jun 2010 19:47:23 +0000 (GMT) Received: from acsmt355.oracle.com (acsmt355.oracle.com [141.146.40.155]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o5MD515B000330 for ; Tue, 22 Jun 2010 19:47:22 +0000 (GMT) Received: from abhmt019.oracle.com by acsmt353.oracle.com with ESMTP id 364809031277236040; Tue, 22 Jun 2010 12:47:20 -0700 Received: from [129.148.19.4] (/129.148.19.4) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 22 Jun 2010 12:47:18 -0700 Date: Tue, 22 Jun 2010 15:47:16 -0400 From: Sebastien Roy Subject: Re: layer-3 net properties for exclusive-IP zones [PSARC/2010/166 FastTrack timeout 05/19/2010] In-reply-to: <20100612135543.GD19314@quasimodo.East.Sun.COM> To: sowmini.varadhan@oracle.com Cc: PSARC-ext@sun.com, Erik Nordmark Message-id: <4C211344.6040600@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt355.oracle.com [141.146.40.155] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090201.4C21134A.01D0:SCFMA4539814,ss=1,fgs=0 References: <20100513202920.GG27083@quasimodo.East.Sun.COM> <20100514142056.GA249@quasimodo.East.Sun.COM> <4BF4CA99.6070409@oracle.com> <20100520130743.GD9994@quasimodo.East.Sun.COM> <4BF69113.60500@oracle.com> <20100524193357.GC15873@quasimodo.East.Sun.COM> <20100525150930.GA18828@quasimodo.East.Sun.COM> <20100601204250.GA555205@jurassic.Eng.Sun.COM> <20100602100312.GA203@quasimodo.East.Sun.COM> <20100602190134.GC651532@jurassic.Eng.Sun.COM> <20100612135543.GD19314@quasimodo.East.Sun.COM> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.9) Gecko/20100524 Lightning/1.0b1 Thunderbird/3.0.4 Status: RO Content-Length: 124 This case was approved during last week's meeting. Note that the final spec is enclosed in the materials directory. -Seb