From sacadmin Wed May 3 14:56:20 2006 Received: from eastmail1bur.East.Sun.COM (eastmail1bur.East.Sun.COM [129.148.9.49]) by sac.sfbay.sun.com (8.12.9+Sun/8.12.9) with ESMTP id k43LuKIQ007577 for ; Wed, 3 May 2006 14:56:20 -0700 (PDT) Received: from thunk.east.sun.com (thunk.East.Sun.COM [129.148.174.66]) by eastmail1bur.East.Sun.COM (8.12.10+Sun/8.12.10/ENSMAIL,v2.2) with ESMTP id k43LuJLo017424; Wed, 3 May 2006 17:56:19 -0400 (EDT) Received: from 127.0.0.1 (localhost [127.0.0.1]) by thunk.east.sun.com (8.13.5+Sun/8.13.5) with ESMTP id k43LuIa0007518; Wed, 3 May 2006 17:56:19 -0400 (EDT) Subject: 2006/289 Rationalized IFF_NOFAILOVER for IPMP Singleton From: Bill Sommerfeld To: psarc@sac.eng.sun.com Cc: Peter Memishian Content-Type: text/plain Message-Id: <1146693378.5552.371.camel@thunk> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.6.333 Date: Wed, 03 May 2006 17:56:18 -0400 Content-Transfer-Encoding: 7bit Status: RO Content-Length: 4233 I'm sponsoring this case for meem. Timeout is 5/10/2006. Proposed release binding is Patch; stability level of the newly clarified behavior is Stable. This case removes a barely documented implementation artifact which was not ARC-reviewed and which is causing trouble for customers. Overview ======== This case proposes the removal of an inconsistency in the way IPMP test addresses are chosen for Singleton IPMP groups. We are requesting a patch binding as the change in behavior is not known to affect any applications, and the current behavior is preventing IPMP deployments with Solaris 10. However, if that is seen as inappropriate, we request minor binding since this mistake must be corrected in order to implement more extensive changes to IPMP targeting Nevada (which will be covered by a forthcoming case). Details ======= PSARC/2002/278 introduced the notion of a "Singleton IPMP group", which is an IPMP group that contains only a single IP interface. Singleton groups were introduced specifically for Sun Cluster, and enable it to provide higher-level failover should an interface become unusable. That said, it is also possible to configure Singleton IPMP groups without using Sun Cluster (e.g., to be notified when an interface has failed). Though never stated or approved by PSARC/2002/278, IPMP test addresses are also handled specially for IPMP Singleton. This special behavior is also not mentioned in any manpages, but is covered briefly in the IP services manual on docs.sun.com. In particular, for an IPMP Singleton group, if no address is marked IFF_NOFAILOVER, in.mpathd will choose a data address in the group as a test address. Since, by definition of a IPMP Singleton group, there is nowhere for an address to failover to, allowing IFF_NOFAILOVER to be omitted was seen as a "convenience" in minimizing configuration. However, subsequent experience has revealed that this was a misguided decision, since: * If additional interfaces are subsequently added to an IPMP group, failure detection unexpectedly degrades (since probe- based failure detection will be disabled). * Test addresses were made optional in Solaris 10 (4840370), allowing customers to selectively enable probe-based failure detection (note that link-based failure detection is always enabled). However, because of the current behavior, it is not possible to disable probe-based failure detection on IPMP Singleton configurations. This is a critical issue for IPMP deployments where probe-based failure detection cannot work (e.g., because routers are configured to ignore ICMP probes). As such, this case proposes to remove this wart, and make the IPMP Singleton behavior uniform with normal IPMP groups: * If an IP interface in an IPMP Singleton group has one or more addresses with IFF_NOFAILOVER set, then one will be chosen as a test address, and probe-based failure detection will be enabled. * If an IP interface in an IPMP Singleton group has no addresses with IFF_NOFAILOVER set, then probe-based failure detection will be disabled. However, as always, link-based failure detection will remain enabled. The Sun Cluster team has confirmed that the Singleton IPMP groups that are automatically configured by Sun Cluster include the IFF_NOFAILOVER flag. Thus, the proposed change only stands to impact the small subset of customers that have either explicitly changed the default Sun Cluster IPMP configuration, or are using IPMP Singleton without Sun Cluster. However, not surprisingly, these advanced customers are exactly the ones who are requesting that probe-based failure detection be made optional. Note that in.mpathd already sends a message to syslog if probe-based failure detection is (or becomes) disabled or enabled on an IPMP group, mitigating the risk that this issue will go undetected. This message makes it clear that the customer can revert to the previous behavior by simply set the IFF_NOFAILOVER flag on the appropriate addresses. From sacadmin Wed May 3 17:33:13 2006 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.12.9+Sun/8.12.9) with ESMTP id k440XCIQ010964 for ; Wed, 3 May 2006 17:33:12 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.149.247.22]) by sunmail5.uk.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id k440X8ib024259; Thu, 4 May 2006 01:33:10 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0IYP0070BTJ8LO00@nwk-avmta-2.sfbay.sun.com>; Wed, 03 May 2006 17:33:08 -0700 (PDT) Received: from sineb-mail-1.sun.com ([192.18.19.6]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0IYP00EEYTJ4PPD0@nwk-avmta-2.sfbay.sun.com>; Wed, 03 May 2006 17:33:05 -0700 (PDT) Received: from fe-apac-03.sun.com (fe-apac-03.sun.com [192.18.19.174] (may be forged)) by sineb-mail-1.sun.com (8.12.10+Sun/8.12.9) with ESMTP id k440X3cv013070; Thu, 04 May 2006 08:33:03 +0800 (SGT) Received: from conversion-daemon.mail-apac.sun.com by mail-apac.sun.com (Sun Java System Messaging Server 6.2-4.02 (built Sep 9 2005)) id <0IYP00C01THJG300@mail-apac.sun.com> (original mail from Darren.Reed@Sun.COM); Thu, 04 May 2006 08:33:02 +0800 (SGT) Received: from [129.146.106.55] by mail-apac.sun.com (Sun Java System Messaging Server 6.2-4.02 (built Sep 9 2005)) with ESMTPSA id <0IYP006NUTJ1QZY1@mail-apac.sun.com>; Thu, 04 May 2006 08:33:02 +0800 (SGT) Date: Wed, 03 May 2006 17:29:10 -0700 From: Darren Reed Subject: Re: 2006/289 Rationalized IFF_NOFAILOVER for IPMP Singleton In-reply-to: <1146693378.5552.371.camel@thunk> Sender: Darren.Reed@sun.com To: Bill Sommerfeld Cc: psarc@sun.com Message-id: <44594AD6.5080103@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=us-ascii Content-transfer-encoding: 7BIT X-Accept-Language: en-us, en X-PMX-Version: 5.1.2.240295 References: <1146693378.5552.371.camel@thunk> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.7) Gecko/20041221 Status: RO Content-Length: 610 Thinking about the zpool+history discussion, should fast track postings list this, in general, also be copied to the relevant opensolaris list, if there is one? Or are we saying that some fast track cases are so complete that the only relevant review is to come from PSARC? Where do we draw the line? Looking at this document and thinking about some of the recent questions to come from networking-discuss, whilst posting this fastrack it may not generate review, it would definately be a step to informing the community more about our direction and they seem to be generally appreciative of that. Darren From sacadmin Wed May 3 17:51:56 2006 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.12.9+Sun/8.12.9) with ESMTP id k440ptIQ011387 for ; Wed, 3 May 2006 17:51:55 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id k440pnir001133; Thu, 4 May 2006 01:51:51 +0100 (BST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0IYP0020BUEC3V00@brm-avmta-1.central.sun.com>; Wed, 03 May 2006 18:51:48 -0600 (MDT) Received: from triplex.East.Sun.COM ([129.148.174.104]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0IYP00MQAUECMI30@brm-avmta-1.central.sun.com>; Wed, 03 May 2006 18:51:48 -0600 (MDT) Received: from triplex.East.Sun.COM (localhost [127.0.0.1]) by triplex.East.Sun.COM (8.13.5+Sun/8.13.5) with ESMTP id k440pmhC018934; Wed, 03 May 2006 20:51:48 -0400 (EDT) Received: (from meem@localhost) by triplex.East.Sun.COM (8.13.5+Sun/8.13.5/Submit) id k440pmSS018931; Wed, 03 May 2006 20:51:48 -0400 (EDT) Date: Wed, 03 May 2006 20:51:48 -0400 From: Peter Memishian Subject: Re: 2006/289 Rationalized IFF_NOFAILOVER for IPMP Singleton In-reply-to: <44594AD6.5080103@sun.com> To: Darren Reed Cc: Bill Sommerfeld , psarc@sun.com Message-id: <17497.20516.58024.411582@triplex.East.Sun.COM> MIME-version: 1.0 X-Mailer: VM 7.17 under 21.4 (patch 18) "Social Property" XEmacs Lucid Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.1.2.240295 References: <1146693378.5552.371.camel@thunk> <44594AD6.5080103@sun.com> Status: RO Content-Length: 1031 > Thinking about the zpool+history discussion, should fast > track postings list this, in general, also be copied to the > relevant opensolaris list, if there is one? > > Or are we saying that some fast track cases are so complete > that the only relevant review is to come from PSARC? > > Where do we draw the line? I think there are two separate pieces here: 1. Getting relevant review prior to submitting to PSARC. 2. Allowing the PSARC process to be visible externally. As I understand it, (2) will happen real soon now. Regarding (1), I think OpenSolaris only impacts the forums one should solicit for feedback on. For instance, if I was making a change that had broad impact on networking in the past, I'd probably solicit input from snt-interest@sun.com; now I'd solicit input from networking-discuss@opensolaris.org. Both then and now, there are some things that simply do not have much impact on (or interest to) others, and thus I feel do not require much review prior to submission to PSARC. - meem From sacadmin Wed May 10 10:11:18 2006 Received: from eastmail2bur.East.Sun.COM (eastmail2bur.East.Sun.COM [129.148.13.40]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id k4AHBIoX023672 for ; Wed, 10 May 2006 10:11:18 -0700 (PDT) Received: from thunk.east.sun.com (thunk.East.Sun.COM [129.148.174.66]) by eastmail2bur.East.Sun.COM (8.12.10+Sun/8.12.10/ENSMAIL,v2.2) with ESMTP id k4AHBGSQ008617; Wed, 10 May 2006 13:11:16 -0400 (EDT) Received: from 127.0.0.1 (localhost [127.0.0.1]) by thunk.east.sun.com (8.13.6+Sun/8.13.6) with ESMTP id k4AHBGH5007855; Wed, 10 May 2006 13:11:16 -0400 (EDT) Subject: Re: 2006/289 Rationalized IFF_NOFAILOVER for IPMP Singleton From: Bill Sommerfeld To: psarc@sac.sfbay.sun.com Cc: Peter Memishian In-Reply-To: <1146693378.5552.371.camel@thunk> References: <1146693378.5552.371.camel@thunk> Content-Type: text/plain Message-Id: <1147281075.7434.70.camel@thunk> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.6.338 Date: Wed, 10 May 2006 13:11:16 -0400 Content-Transfer-Encoding: 7bit Status: RO Content-Length: 54 This case was approved during today's PSARC meeting.