From Sebastien.Roy@sun.com Wed Dec 3 11:23:08 2008 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mB3JN8pM018689 for ; Wed, 3 Dec 2008 11:23:08 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id mB3JMwSL003001 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Wed, 3 Dec 2008 11:23:08 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KBB0051HF6I9600@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 03 Dec 2008 11:23:06 -0800 (PST) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KBB003D8F6H1K30@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 03 Dec 2008 11:23:05 -0800 (PST) Received: from fe-amer-09.sun.com ([192.18.109.79]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id mB3JN5mm023173 for ; Wed, 03 Dec 2008 19:23:05 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0KBB00201CU2HM00@mail-amer.sun.com> (original mail from Sebastien.Roy@Sun.COM) for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 03 Dec 2008 12:23:05 -0700 (MST) Received: from [129.148.174.103] by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0KBB00FQLF66ETD0@mail-amer.sun.com>; Wed, 03 Dec 2008 12:22:54 -0700 (MST) Date: Wed, 03 Dec 2008 14:22:48 -0500 From: Sebastien Roy Subject: PSARC 2008/575 Inception review summary Sender: Sebastien.Roy@sun.com To: psarc-ext Cc: Sangeeta Misra , "Erik.Nordmark" Message-id: <1228332168.16428.109.camel@strat> Organization: Sun Microsystems MIME-version: 1.0 X-Mailer: Evolution 2.24.0 Content-type: text/plain Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 Status: RO Content-Length: 2472 In addition to the responses that Sangeeta already provided in the issues file, the relevant parts of today's Inception review were: * Regarding djr-00 and the general interaction with filtering hooks: There isn't very much ILB specific code in the ip kernel module itself. Only the hooks are part of the IP datapath code, and those don't comprise very much code. Erik Nordmark gave an overview of the reasons why filtering hooks aren't a good fit for load balancing. Specifically, there are requirements on the location of the hooks which the filtering hooks don't meet, as well as issues regarding the ordering of the hook callbacks. There are also semantic requirements of the load balancing hooks (the ability to change the next hop on output for example) that the filtering hooks don't provide in an efficient way. There were concerns raised that the lack of a general-purpose hook mechanism (despite the claims that there is already such a mechanism, it doesn't appear to be the case) is leading to exploding complexity in the IP implementation. Erik noted that the existence of a general-purpose hook mechanism would not necessarily address the complexity problem due to the sensitive nature of the interactions between the different hook consumers. Given this information, the committee asks the project team to clearly document the interactions between load balancing and other hooks such as filtering, observability, and Dtrace probes. * Regarding Jim Carlson's whiteboard level 0: requirements on VRRP Sangeeta Misra noted that there are two failover cases that will use VRRP, neither of which would require any feature beyond what is currently defined in the VRRP RFC. There are more details in appendix C of the design document. This project does not require "grouping" (a feature that was initially included in the VRRP case), nor the scripting interface that is being defined by VRRP. Sangeeta took an action-item to send mail to the VRRP case log to state the requirements that this case poses on VRRP. * Regarding Glenn Skinner's issue of "one more special-purpose uid": Do we really need another special-purpose user-id? The project team will investigate whether this is really needed, including the possibility of using "root" with dropped privileges. There should be advice in the opinion for this case to the appropriate management team on defining architecture suitable to allow projects to use user-id's other than "root". -Seb From gww@eng.sun.com Wed Dec 3 11:54:02 2008 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mB3Js2V8019148 for ; Wed, 3 Dec 2008 11:54:02 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id mB3Jrx8Y001975 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Wed, 3 Dec 2008 11:54:02 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KBB00A03GM18X00@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 03 Dec 2008 12:54:01 -0700 (MST) Received: from dm-eng-02.sfbay.sun.com ([129.146.11.32]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KBB002HLGM07T60@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 03 Dec 2008 12:54:01 -0700 (MST) Received: from marduk.eng.sun.com (marduk.SFBay.Sun.COM [129.146.108.224]) by dm-eng-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id mB3Js0K8033470; Wed, 03 Dec 2008 11:54:00 -0800 (PST) Received: from marduk.eng.sun.com (localhost [127.0.0.1]) by marduk.eng.sun.com (8.13.6+Sun/8.12.11) with ESMTP id mB3JrTb2004310; Wed, 03 Dec 2008 11:53:29 -0800 (PST) Received: (from gww@localhost) by marduk.eng.sun.com (8.13.6+Sun/8.12.11/Submit) id mB3JrTWQ004309; Wed, 03 Dec 2008 11:53:29 -0800 (PST) Date: Wed, 03 Dec 2008 11:53:29 -0800 (PST) From: Gary Winiger Subject: Re: PSARC 2008/575 Inception review summary To: psarc-ext@sun.com, Sebastien.Roy@sun.com Cc: Sangeeta.Misra@sun.com, Erik.Nordmark@sun.com Message-id: <200812031953.mB3JrTWQ004309@marduk.eng.sun.com> Content-transfer-encoding: 7BIT X-Sun-Charset: US-ASCII X-PMX-Version: 5.4.1.325704 Status: RO Content-Length: 803 > * Regarding Glenn Skinner's issue of "one more special-purpose uid": > > Do we really need another special-purpose user-id? The project team > will investigate whether this is really needed, including the > possibility of using "root" with dropped privileges. > > There should be advice in the opinion for this case to the appropriate > management team on defining architecture suitable to allow projects to > use user-id's other than "root". I mentioned there was a discussion of a similar request for new userID from a case of Nico's. That case seems to be PSARC/2008/507 OpenLDAP for OpenSolaris. I thought there was more discussion than seems to be in the case log. Possibly it occurred off line. Perhaps I misremembered. Perhaps someone else has the case I was thinking of. Gary.. From carlsonj@phorcys.east.sun.com Wed Dec 3 13:26:35 2008 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mB3LQYpj006552 for ; Wed, 3 Dec 2008 13:26:35 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id mB3LQNZS022029; Wed, 3 Dec 2008 21:26:31 GMT Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KBB00B1RKW6P400@nwk-avmta-1.sfbay.Sun.COM>; Wed, 03 Dec 2008 13:26:30 -0800 (PST) Received: from phorcys.east.sun.com ([129.148.174.143]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KBB001CKKW50LC0@nwk-avmta-1.sfbay.Sun.COM>; Wed, 03 Dec 2008 13:26:29 -0800 (PST) Received: from phorcys.east.sun.com (localhost [127.0.0.1]) by phorcys.east.sun.com (8.14.3+Sun/8.14.3) with ESMTP id mB3LQSJx014851; Wed, 03 Dec 2008 16:26:28 -0500 (EST) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.3+Sun/8.14.3/Submit) id mB3LQR6v014848; Wed, 03 Dec 2008 16:26:28 -0500 (EST) Date: Wed, 03 Dec 2008 16:26:27 -0500 From: James Carlson Subject: Re: PSARC 2008/575 Inception review summary In-reply-to: <200812031953.mB3JrTWQ004309@marduk.eng.sun.com> To: Gary Winiger Cc: psarc-ext@sun.com, Sebastien.Roy@sun.com, Sangeeta.Misra@sun.com, Erik.Nordmark@sun.com Message-id: <18742.63875.981238.204209@gargle.gargle.HOWL> MIME-version: 1.0 X-Mailer: VM 7.01 under Emacs 21.3.1 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200812031953.mB3JrTWQ004309@marduk.eng.sun.com> Status: RO Content-Length: 1251 Gary Winiger writes: > > * Regarding Glenn Skinner's issue of "one more special-purpose uid": > > > > Do we really need another special-purpose user-id? The project team > > will investigate whether this is really needed, including the > > possibility of using "root" with dropped privileges. > > > > There should be advice in the opinion for this case to the appropriate > > management team on defining architecture suitable to allow projects to > > use user-id's other than "root". > > I mentioned there was a discussion of a similar request for > new userID from a case of Nico's. That case seems to be > PSARC/2008/507 OpenLDAP for OpenSolaris. I thought there > was more discussion than seems to be in the case log. > Possibly it occurred off line. Perhaps I misremembered. > Perhaps someone else has the case I was thinking of. I think it was also discussed in 2008/260, but, yes, 2008/507 was the main thread I remember. It sure would be nice to have a solution before the 99 magic numbers are all gone. -- James Carlson, Solaris Networking Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 From Darren.Reed@sun.com Wed Dec 3 18:14:45 2008 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mB42EjLE023709 for ; Wed, 3 Dec 2008 18:14:45 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id mB42EiXo001654 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Wed, 3 Dec 2008 18:14:45 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KBB00305Y8J6Z00@nwk-avmta-1.sfbay.Sun.COM> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 03 Dec 2008 18:14:43 -0800 (PST) Received: from gmp-eb-inf-1.sun.com ([192.18.6.21]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KBB00GATY8IGM80@nwk-avmta-1.sfbay.Sun.COM> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 03 Dec 2008 18:14:43 -0800 (PST) Received: from fe-emea-10.sun.com (gmp-eb-lb-2-fe3.eu.sun.com [192.18.6.12]) by gmp-eb-inf-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id mB42Egmu010078 for ; Thu, 04 Dec 2008 02:14:42 +0000 (GMT) Received: from conversion-daemon.fe-emea-10.sun.com by fe-emea-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0KBB00101Y6BOK00@fe-emea-10.sun.com> (original mail from Darren.Reed@Sun.COM) for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 04 Dec 2008 02:14:42 +0000 (GMT) Received: from [129.158.90.153] by fe-emea-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0KBB007NSY8FQM60@fe-emea-10.sun.com>; Thu, 04 Dec 2008 02:14:42 +0000 (GMT) Date: Thu, 04 Dec 2008 13:14:35 +1100 From: Darren Reed Subject: Re: PSARC 2008/575 Inception review summary In-reply-to: <1228332168.16428.109.camel@strat> Sender: Darren.Reed@sun.com To: Sebastien Roy Cc: psarc-ext , Sangeeta Misra , "Erik.Nordmark" Message-id: <49373D0B.2000202@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <1228332168.16428.109.camel@strat> User-Agent: Thunderbird 2.0.0.18 (Windows/20081105) Status: RO Content-Length: 2243 Sebastien Roy wrote: > In addition to the responses that Sangeeta already provided in the > issues file, the relevant parts of today's Inception review were: > > * Regarding djr-00 and the general interaction with filtering hooks: > > There isn't very much ILB specific code in the ip kernel module itself. > Only the hooks are part of the IP datapath code, and those don't > comprise very much code. Thanks for the clairification. The design document didn't give that impression. > Erik Nordmark gave an overview of the reasons why filtering hooks aren't > a good fit for load balancing. Specifically, there are requirements on > the location of the hooks which the filtering hooks don't meet, as well > as issues regarding the ordering of the hook callbacks. There are also > semantic requirements of the load balancing hooks (the ability to change > the next hop on output for example) that the filtering hooks don't > provide in an efficient way. What I am looking for is a new packet event, not the reuse of an existing one, that has different semantics to those used for packet filtering (firewall.) A load balancr needs to interact with the packet in a different way to a firewall/NAT, so I see no reason why hooks designed to support those specific features should be reused for load balancing or as the template from which to create a load balancing packet event. > There were concerns raised that the lack of a general-purpose hook > mechanism (despite the claims that there is already such a mechanism, it > doesn't appear to be the case) is leading to exploding complexity in the > IP implementation. Erik noted that the existence of a general-purpose > hook mechanism would not necessarily address the complexity problem due > to the sensitive nature of the interactions between the different hook > consumers. If the general purpose mechanism isn't general purpose enough then it behooves us to document what its failings are and try to improve it. Additionally, we should make a better attempt to use existing interfaces and improve them as required (especially when the idea is to promote them for external use) rather than baking new, private, ones every time we have a slightly different programming task. Darren From carlsonj@phorcys.east.sun.com Wed Dec 3 21:21:11 2008 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mB45LANU027500 for ; Wed, 3 Dec 2008 21:21:11 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id mB45L5Mm004220; Thu, 4 Dec 2008 05:21:05 GMT Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KBC006076V4DF00@nwk-avmta-1.sfbay.Sun.COM>; Wed, 03 Dec 2008 21:21:04 -0800 (PST) Received: from phorcys.east.sun.com ([129.148.174.143]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KBC00ELK6V3MW80@nwk-avmta-1.sfbay.Sun.COM>; Wed, 03 Dec 2008 21:21:04 -0800 (PST) Received: from phorcys.east.sun.com (localhost [127.0.0.1]) by phorcys.east.sun.com (8.14.3+Sun/8.14.3) with ESMTP id mB45L35T016285; Thu, 04 Dec 2008 00:21:03 -0500 (EST) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.3+Sun/8.14.3/Submit) id mB45L3mH016282; Thu, 04 Dec 2008 00:21:03 -0500 (EST) Date: Thu, 04 Dec 2008 00:21:03 -0500 From: James Carlson Subject: Re: PSARC 2008/575 Inception review summary In-reply-to: <49373D0B.2000202@Sun.COM> To: Darren Reed Cc: Sebastien Roy , psarc-ext , "Erik.Nordmark" , Sangeeta Misra Message-id: <18743.26815.767939.902768@gargle.gargle.HOWL> MIME-version: 1.0 X-Mailer: VM 7.01 under Emacs 21.3.1 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <1228332168.16428.109.camel@strat> <49373D0B.2000202@Sun.COM> Status: RO Content-Length: 1536 Darren Reed writes: > If the general purpose mechanism isn't general > purpose enough then it behooves us to document > what its failings are and try to improve it. > Additionally, we should make a better attempt to > use existing interfaces and improve them as required > (especially when the idea is to promote them for > external use) rather than baking new, private, ones > every time we have a slightly different programming > task. As we tried to point out during the review of netinfo/pfhooks, we've had a large number of "generic packet event frameworks" come through the ARC, including Fireengine and IPQoS. Each has said that there was some flaw with the previous one, that the previous one couldn't be extended to do what was needed, and that thus either a new one would be built, or some project would bypass the hooks and just link directly into IP. This just follows that tradition. It's certainly true that we could update netinfo. It's also true that the ILB project team made some good points about why they can't use it. But at a higher level, the problem we have is that we have "framework" projects that (for whatever reason) promise more than they seem to deliver, likely because there are fundamental issues (such as conflicts among hooks and performance) that haven't been resolved. -- James Carlson, Solaris Networking Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 From erik.nordmark@sun.com Thu Dec 4 10:43:15 2008 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mB4IhFjU027829 for ; Thu, 4 Dec 2008 10:43:15 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id mB4Ih74r018865; Thu, 4 Dec 2008 10:43:12 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KBD00I258009V00@brm-avmta-1.central.sun.com>; Thu, 04 Dec 2008 11:43:12 -0700 (MST) Received: from jurassic.eng.sun.com ([129.146.56.36]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KBD00F527ZZX820@brm-avmta-1.central.sun.com>; Thu, 04 Dec 2008 11:43:11 -0700 (MST) Received: from [10.7.251.248] (punchin-nordmark.SFBay.Sun.COM [10.7.251.248]) by jurassic.eng.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mB4IhA9h664305 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 04 Dec 2008 10:43:11 -0800 (PST) Date: Thu, 04 Dec 2008 10:43:10 -0800 From: Erik Nordmark Subject: Re: PSARC 2008/575 Inception review summary In-reply-to: <49373D0B.2000202@Sun.COM> To: Darren Reed Cc: Sebastien Roy , psarc-ext , Sangeeta Misra Message-id: <493824BE.7000502@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <1228332168.16428.109.camel@strat> <49373D0B.2000202@Sun.COM> User-Agent: Thunderbird 2.0.0.17 (X11/20081023) Status: RO Content-Length: 5067 Darren Reed wrote: > What I am looking for is a new packet event, > not the reuse of an existing one, that has different > semantics to those used for packet filtering (firewall.) > A load balancr needs to interact with the packet in a > different way to a firewall/NAT, so I see no reason why > hooks designed to support those specific features should > be reused for load balancing or as the template from > which to create a load balancing packet event. My understanding is that the pfhooks project provided a rather general approach for packet *filter* hooks, and they seem to work quite well for that purpose. You seem to think of them as "generic IP packet hooks". I don't think they were designed for that purpose in mind; I certainly didn't review them as such. And such generality might not be useful. >> There were concerns raised that the lack of a general-purpose hook >> mechanism (despite the claims that there is already such a mechanism, it >> doesn't appear to be the case) is leading to exploding complexity in the >> IP implementation. Erik noted that the existence of a general-purpose >> hook mechanism would not necessarily address the complexity problem due >> to the sensitive nature of the interactions between the different hook >> consumers. That was only one of my points. We have several cases when we use hard-code code paths to plug in different functionality in IP. IPsec is a good example of that. One could in principle argue that IPsec should have used a "generic IP packet hooks" interface instead of hard-coding callouts from IP to IPsec. But I think we did the right thing there, since IPsec is quite an integral part of the IP layer and we need a tight and evolving coupling between the two. I also pointed out in the meeting that we might implement future things that would likewise benefit from hard-coded callouts. Mobile IP and shim6 are things that it wouldn't make sense to implement using some generic "IP packet hooks" framework, for the same reasons that IPsec doesn't fit. I think L3/L4 LB fits in the same category even though its interactions with IP is currently simpler than e.g., IPsec or MIP. If we look at the micro-architecture of ip_input() we see that one of the things it needs to do is to determine the next-hop of the packet. This is determined from a source route option or the IP destination field. An L3/L4 LB affects this. In particular, when doing DSR i.e. the packet is not modified, this is the only effect the LB has; the nexthop is determined from the LB decision. But that micro-architecture has to be such that we get predictable behavior when combined with a 1) source route option and/or 2) a pfhooks user which NATs the packet. Hence the order is critical. Jim pointed out that this could be done using pfhooks plus the net routeto/inject. But that would make L3/L4 LB be (yet another) bolt on which doesn't fit well which seems counter to our need to reduce and manage the implementation complexity in the IP stack. > If the general purpose mechanism isn't general > purpose enough then it behooves us to document > what its failings are and try to improve it. I do think we should document the issues of how to use pfhooks for packet filter and NAT. But I don't think we should try to incrementally turn it into something it wasn't intended to be ("IP hooks"). FWIW I've told you of some of the issues a while back. 1. It isn't clear which hook entrypoints the hook user can modify the packet and have IP take notice (e.g., to route it based on a changed destination.) I think we should just document that this can only be done for "physical_in_event". 2. It would be good of the result of the hook was different when the packet was modified as opposed to passed through unmodified, since that would help detect incorrect cases of #1 and also allow optimizations in ip_input. 3. Having multiple consumers on the same hook where more than one of them modify the packet seems problematic. I don't know if we have any cases where folks want to do this,. If we do at least the order is critical. But it might also be that user2 needs to be aware that user1 modified the packet. [My, perhaps incorrect, understanding of Linux is that they manage such complexity by having iptables be the only user of the hook and which allows them to extend and modify iptables to do that ever they need. Hence they can avoid ships-in-the-night modifications of the same packet.] > Additionally, we should make a better attempt to > use existing interfaces and improve them as required > (especially when the idea is to promote them for > external use) rather than baking new, private, ones > every time we have a slightly different programming > task. You make a very general statement above, which might read like motherhood and apply pie. In any particular case I think it requires some careful engineering judgment to determine what can be reused and the trade offs. Reusing the round hole for the square peg is known to be problematic. Erik From Darren.Reed@sun.com Thu Dec 4 18:21:07 2008 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mB52L7ee026296 for ; Thu, 4 Dec 2008 18:21:07 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id mB52L5ZI008901 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Thu, 4 Dec 2008 18:21:07 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KBD00915T75O000@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 04 Dec 2008 18:21:05 -0800 (PST) Received: from gmp-eb-inf-2.sun.com ([192.18.6.24]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KBD00JKNT730FD0@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 04 Dec 2008 18:21:04 -0800 (PST) Received: from fe-emea-10.sun.com (gmp-eb-lb-1-fe3.eu.sun.com [192.18.6.10]) by gmp-eb-inf-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id mB52L3gw021010 for ; Fri, 05 Dec 2008 02:21:03 +0000 (GMT) Received: from conversion-daemon.fe-emea-10.sun.com by fe-emea-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0KBD00F01T3S4X00@fe-emea-10.sun.com> (original mail from Darren.Reed@Sun.COM) for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Fri, 05 Dec 2008 02:21:03 +0000 (GMT) Received: from [129.158.90.153] by fe-emea-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0KBD00MRTT702U50@fe-emea-10.sun.com>; Fri, 05 Dec 2008 02:21:03 +0000 (GMT) Date: Fri, 05 Dec 2008 13:20:56 +1100 From: Darren Reed Subject: Re: PSARC 2008/575 Inception review summary In-reply-to: <493824BE.7000502@sun.com> Sender: Darren.Reed@sun.com To: Erik Nordmark Cc: Sebastien Roy , psarc-ext , Sangeeta Misra Message-id: <49389008.1060804@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <1228332168.16428.109.camel@strat> <49373D0B.2000202@Sun.COM> <493824BE.7000502@sun.com> User-Agent: Thunderbird 2.0.0.18 (Windows/20081105) Status: RO Content-Length: 5072 Erik Nordmark wrote: > Darren Reed wrote: ... >>> There were concerns raised that the lack of a general-purpose hook >>> mechanism (despite the claims that there is already such a >>> mechanism, it >>> doesn't appear to be the case) is leading to exploding complexity in >>> the >>> IP implementation. Erik noted that the existence of a general-purpose >>> hook mechanism would not necessarily address the complexity problem due >>> to the sensitive nature of the interactions between the different hook >>> consumers. > > That was only one of my points. > > We have several cases when we use hard-code code paths to plug in > different functionality in IP. IPsec is a good example of that. One > could in principle argue that IPsec should have used a "generic IP > packet hooks" interface instead of hard-coding callouts from IP to > IPsec. But I think we did the right thing there, since IPsec is quite > an integral part of the IP layer and we need a tight and evolving > coupling between the two. > > I also pointed out in the meeting that we might implement future > things that would likewise benefit from hard-coded callouts. Mobile IP > and shim6 are things that it wouldn't make sense to implement using > some generic "IP packet hooks" framework, for the same reasons that > IPsec doesn't fit. > > I think L3/L4 LB fits in the same category even though its > interactions with IP is currently simpler than e.g., IPsec or MIP. I disagree... IPsec and Mobile IP/shim6 are integral features of the IP protocol suite and the way in which they need to be handled should reflect this, which is to say that I agree with how they're treated today. Load balancing is not in the same category. Whilst load balancing does need to sit at a very particular point in the code path for IP packets, it's interaction with IP and its role are distinctly value-add rather than basic functionality needed to interoperate with other systems - as is the case with NAT. > If we look at the micro-architecture of ip_input() we see that one of the things it needs to > do is to determine the next-hop of the packet. This is determined from a source route > option or the IP destination field. > > An L3/L4 LB affects this. In particular, when doing DSR i.e. the > packet is not modified, this is the only effect the LB has; the > nexthop is determined from the LB decision. But that > micro-architecture has to be such that we get predictable behavior > when combined with a 1) source route option and/or 2) a pfhooks user > which NATs the packet. > Hence the order is critical. I agree with this. >> If the general purpose mechanism isn't general >> purpose enough then it behooves us to document >> what its failings are and try to improve it. > > I do think we should document the issues of how to use pfhooks for > packet filter and NAT. But I don't think we should try to > incrementally turn it into something it wasn't intended to be ("IP > hooks"). Hmm, I'll follow up with you on this out of band.. > FWIW I've told you of some of the issues a while back. > 1. It isn't clear which hook entrypoints the hook user can modify the > packet and have IP take notice (e.g., to route it based on a changed > destination.) I think we should just document that this can only be > done for "physical_in_event". > 2. It would be good of the result of the hook was different when the > packet was modified as opposed to passed through unmodified, since > that would help detect incorrect cases of #1 and also allow > optimizations in ip_input. (1) should be captured on a man page, but which one? (2) sounds like something meaningful that can, at the very least, be put into an RFE - see 6781115. > 3. Having multiple consumers on the same hook where more than one of > them modify the packet seems problematic. I don't know if we have any > cases where folks want to do this,. If we do at least the order is > critical. But it might also be that user2 needs to be aware that user1 > modified the packet. [My, perhaps incorrect, understanding of Linux is > that they manage such complexity by having iptables be the only user > of the hook and which allows them to extend and modify iptables to do > that ever they need. Hence they can avoid ships-in-the-night > modifications of the same packet.] Linux does allow more than just iptables to hook in, however the method they choose to determine the order in which a packet is received (assignment of priority numbers for the various "hooks") was rejected by PSARC. iptables provides the NAT/firewalling for Linux. You should also have a look at what they call "netfilter" as it is "netfilter" that provides the infrastructure in Linux for iptables to use. So the ships-in-the-night problem is definately present there, as well as everywhere else that I've seen, too. But as long as the relative ordering of multiple consumers is correct and stable, then it shouldn't be necessary for others to be aware of someone else having made a change (or that will make one.) Darren From erik.nordmark@sun.com Thu Dec 4 21:51:56 2008 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mB55ptEp015236 for ; Thu, 4 Dec 2008 21:51:55 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id mB55ppff015268; Thu, 4 Dec 2008 21:51:52 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KBE00J092YFJY00@brm-avmta-1.central.sun.com>; Thu, 04 Dec 2008 22:51:51 -0700 (MST) Received: from jurassic.eng.sun.com ([129.146.68.36]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KBE00BPE2YDYD30@brm-avmta-1.central.sun.com>; Thu, 04 Dec 2008 22:51:50 -0700 (MST) Received: from [10.7.251.248] (punchin-nordmark.SFBay.Sun.COM [10.7.251.248]) by jurassic.eng.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mB55pmU4675450 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 04 Dec 2008 21:51:49 -0800 (PST) Date: Thu, 04 Dec 2008 21:51:48 -0800 From: Erik Nordmark Subject: Re: PSARC 2008/575 Inception review summary In-reply-to: <49389008.1060804@Sun.COM> To: Darren Reed Cc: Sebastien Roy , psarc-ext , Sangeeta Misra Message-id: <4938C174.3030109@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <1228332168.16428.109.camel@strat> <49373D0B.2000202@Sun.COM> <493824BE.7000502@sun.com> <49389008.1060804@Sun.COM> User-Agent: Thunderbird 2.0.0.17 (X11/20081023) Status: RO Content-Length: 1611 Darren Reed wrote: > I disagree... IPsec and Mobile IP/shim6 are integral features of the IP > protocol suite > and the way in which they need to be handled should reflect this, which > is to say that > I agree with how they're treated today. First of all, the purpose of the ILB project is to increase what is considered being integral parts of an IP stack in the 21st century. But are you arguing that the IP protocol suite (as defined by the IETF) is the only thing we should integrate tightly in IP? We have tight integration of at least CGTP, zones, and TX, neither of which is part of the protocol suite defined by the IETF. > Load balancing is not in the same category. > > Whilst load balancing does need to sit at a very particular point in the > code path > for IP packets, it's interaction with IP and its role are distinctly > value-add rather > than basic functionality needed to interoperate with other systems - as > is the case > with NAT. Is this a business argument that people would be willing to pay extra for a L3/L4 load balancer. In any case, none of the above reads like architectural arguments. > Linux does allow more than just iptables to hook in, > however the method they choose to determine the order > in which a packet is received (assignment of priority > numbers for the various "hooks") was rejected by PSARC. The issue isn't whether it is allowed or not. The issue is how people in practice handle the complexity of multiple things using the same hook at the same time. I was postulating that they handle it by avoiding using that capability. Erik From carlsonj@phorcys.east.sun.com Fri Dec 5 06:55:02 2008 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mB5Et2Zf024862 for ; Fri, 5 Dec 2008 06:55:02 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id mB5EsrD0019210; Fri, 5 Dec 2008 06:54:59 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KBE0070JS3LXQ00@brm-avmta-1.central.sun.com>; Fri, 05 Dec 2008 07:54:57 -0700 (MST) Received: from phorcys.east.sun.com ([129.148.174.143]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KBE00H6XS3KC4B0@brm-avmta-1.central.sun.com>; Fri, 05 Dec 2008 07:54:56 -0700 (MST) Received: from phorcys.east.sun.com (localhost [127.0.0.1]) by phorcys.east.sun.com (8.14.3+Sun/8.14.3) with ESMTP id mB5Esufl014113; Fri, 05 Dec 2008 09:54:56 -0500 (EST) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.3+Sun/8.14.3/Submit) id mB5Esut0014110; Fri, 05 Dec 2008 09:54:56 -0500 (EST) Date: Fri, 05 Dec 2008 09:54:56 -0500 From: James Carlson Subject: Re: PSARC 2008/575 Inception review summary In-reply-to: <49389008.1060804@Sun.COM> To: Darren Reed Cc: Erik Nordmark , psarc-ext , Sebastien Roy , Sangeeta Misra Message-id: <18745.16576.124469.547218@gargle.gargle.HOWL> MIME-version: 1.0 X-Mailer: VM 7.01 under Emacs 21.3.1 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <1228332168.16428.109.camel@strat> <49373D0B.2000202@Sun.COM> <493824BE.7000502@sun.com> <49389008.1060804@Sun.COM> Status: RO Content-Length: 1357 Darren Reed writes: > Linux does allow more than just iptables to hook in, > however the method they choose to determine the order > in which a packet is received (assignment of priority > numbers for the various "hooks") was rejected by PSARC. The use of priority itself wasn't rejected. What was specifically questioned in that approach was giving such a fundamental issue to the system administrator to resolve for the general case. There may be cases where there is some flexibility in ordering, but in general, determining the order of operations among hook users is a system design issue requiring deep understanding of how the code itself works, and ought to be specified adequately such that end users don't have to dream up their own designs. > So the ships-in-the-night problem is definately present there, > as well as everywhere else that I've seen, too. But as long as > the relative ordering of multiple consumers is correct and > stable, then it shouldn't be necessary for others to be aware > of someone else having made a change (or that will make one.) Yes, it's the "correct and stable" part that's an issue. -- James Carlson, Solaris Networking Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 From Sebastien.Roy@Sun.COM Wed Jul 1 11:13:55 2009 Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01.SFBay.Sun.COM [129.145.155.118]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n61IDtBw014432 for ; Wed, 1 Jul 2009 11:13:55 -0700 (PDT) Received: from brmea-mail-1.sun.com (brmea-mail-1.Sun.COM [192.18.98.31]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n61IDtwP026743 for ; Wed, 1 Jul 2009 11:13:55 -0700 (PDT) Received: from fe-amer-09.sun.com ([192.18.109.79]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n61IDtWS014067 for ; Wed, 1 Jul 2009 18:13:55 GMT MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KM4006007QKC000@mail-amer.sun.com> for psarc-ext@sac.sfbay.sun.com; Wed, 01 Jul 2009 12:13:54 -0600 (MDT) Received: from [129.148.174.103] ([unknown] [129.148.174.103]) by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KM400K067YC2T50@mail-amer.sun.com>; Wed, 01 Jul 2009 12:13:25 -0600 (MDT) Date: Wed, 01 Jul 2009 14:12:07 -0400 From: Sebastien Roy Subject: Commitment materials for PSARC 2008/575 ILB: Integrated L3/L4 Load balancer Sender: Sebastien.Roy@Sun.COM To: psarc-ext Cc: Sangeeta Misra Message-id: <1246471927.8298.44.camel@strat> Organization: Sun Microsystems X-Mailer: Evolution 2.26.1.1 Status: RO Content-Length: 320 The project team has supplied materials for next week's Commitment review of PSARC 2008/575. I've placed them in the commitment.materials directory. http://arc.opensolaris.org/caselog/PSARC/2008/575/commitment.materials/ A README.txt file summarizing the changes since Inception is included in the materials. -Seb From gdamore@sun.com Wed Jul 8 08:55:42 2009 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n68Ftg98002943 for ; Wed, 8 Jul 2009 08:55:42 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n68Ftf99032088 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 8 Jul 2009 09:55:41 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMH0092108SMW00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 08 Jul 2009 08:55:40 -0700 (PDT) Received: from sca-es-mail-2.sun.com ([192.18.43.133]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMH00G6W08Q9690@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 08 Jul 2009 08:55:39 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n68Ftcw7021157 for ; Wed, 08 Jul 2009 08:55:38 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMH00L0004Y1S00@fe-sfbay-09.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 08 Jul 2009 08:55:38 -0700 (PDT) Received: from [192.168.251.11] ([unknown] [76.93.15.33]) by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMH009R708QPQ40@fe-sfbay-09.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 08 Jul 2009 08:55:38 -0700 (PDT) Date: Wed, 08 Jul 2009 08:55:38 -0700 From: "Garrett D'Amore" Subject: PSARC 2008/575 ILB: Integrated L3/L4 Load balancer Sender: Garrett.Damore@sun.com To: PSARC-ext Message-id: <4A54C17A.3010007@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 User-Agent: Thunderbird 2.0.0.18 (X11/20081201) Status: RO Content-Length: 144 I've added a few issues to the case issues file. The project team may want to look at them before today's commitment meeting. - Garrett