From sacadmin Fri Oct 17 14:14:31 2008 Received: from sac.sfbay.sun.com (localhost [127.0.0.1]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m9HLEVfQ002525; Fri, 17 Oct 2008 14:14:31 -0700 (PDT) Received: (from carlsonj@localhost) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id m9HLEVpP002517; Fri, 17 Oct 2008 14:14:31 -0700 (PDT) Date: Fri, 17 Oct 2008 14:14:31 -0700 (PDT) From: James Carlson Message-Id: <200810172114.m9HLEVpP002517@sac.sfbay.sun.com> To: PSARC-record@sac.sfbay.sun.com Subject: iSCSI With DHCP [PSARC/2008/640 FastTrack timeout 10/24/2008] Status: RO Content-Length: 542 Template Version: @(#)sac_nextcase %I% %G% SMI This information is Copyright 2008 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: iSCSI With DHCP 1.2. Name of Document Author/Supplier: Author: Jack Meng 1.3 Date of This Document: 17 October, 2008 4. Technical Description See the case directory for more detail 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: FastTrack 6.6. ARC Exposure: open From carlsonj@phorcys.east.sun.com Fri Oct 17 14:17:11 2008 Received: from phorcys.east.sun.com (phorcys.East.Sun.COM [129.148.174.143]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m9HLHBMJ002614 for ; Fri, 17 Oct 2008 14:17:11 -0700 (PDT) Received: from phorcys.east.sun.com (localhost [127.0.0.1]) by phorcys.east.sun.com (8.14.3+Sun/8.14.3) with ESMTP id m9HLHBpI002432; Fri, 17 Oct 2008 17:17:11 -0400 (EDT) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.3+Sun/8.14.3/Submit) id m9HLHBhp002429; Fri, 17 Oct 2008 17:17:11 -0400 (EDT) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <18681.215.37321.861488@gargle.gargle.HOWL> Date: Fri, 17 Oct 2008 17:17:11 -0400 From: James Carlson To: psarc-ext@sac.sfbay.sun.com cc: Jack.Meng@sun.com Subject: 2008/640 iSCSI With DHCP X-Mailer: VM 7.01 under Emacs 21.3.1 Status: RO Content-Length: 1908 I'm sponsoring this fast-track request for Jack Meng. The timer is set to 10/24/2008. Background By default, dhcpagent "canonizes" interfaces under its control on receipt of SIGTERM. This means that it will reset the IP address back to 0.0.0.0 during shutdown. This happens regardless of whether dhcpagent is configured to release or drop leases. Problem When DHCP canonizes as a normal part of system shut-down, iSCSI may lose contact with the server. If the system is diskless, and the root file system is mounted via iSCSI, this causes at least a lock-up and may cause data loss. Solution Release binding for this change is Patch/Micro. A new Consolidation Private ioctl (ISCSI_IS_ACTIVE) will be added to "/devices/iscsi:devctl". dhcpagent will invoke this ioctl to determine whether it can canonize interfaces on exit. Related Projects and Future Work When the root file system is mounted via NFS, a completely different mechanism is used. In this case, /sbin/netstrategy indicates a "dhcp" boot, and we start dhcpagent with the somewhat obscure "-a" (adopt) flag. In addition to pulling DHCPACK information from OBP, this causes dhcpagent to avoid canonizing on shutdown (per CR 4291141). This entire area is one that requires future study, in particular for the relationship between booting, interface configuration, system shutdown sequencing, and SMF. However, that is not this project. References 6751246 dhcp release the lease before sync is committed on iSCSI disk PSARC 2008/427 iSCSI Boot 4291141 Reboot after install w/DHCP hangs due to dhcpagent canonizing interface 6701045 iSCSI boot on x86 -- James Carlson, Solaris Networking Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 From Darren.Moffat@Sun.COM Mon Oct 20 01:36:34 2008 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m9K8aYcW002960 for ; Mon, 20 Oct 2008 01:36:34 -0700 (PDT) Received: from gmp-eb-inf-2.sun.com (gmp-eb-inf-2.EU.Sun.COM [192.18.6.24]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m9K8aXMH001574 for ; Mon, 20 Oct 2008 01:36:33 -0700 (PDT) Received: from fe-emea-10.sun.com (gmp-eb-lb-2-fe2.eu.sun.com [192.18.6.11]) by gmp-eb-inf-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m9K8aRsx029759 for ; Mon, 20 Oct 2008 08:36:28 GMT Received: from conversion-daemon.fe-emea-10.sun.com by fe-emea-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K9100K013ARHL00@fe-emea-10.sun.com> (original mail from Darren.Moffat@Sun.COM) for psarc-ext@sac.sfbay.sun.com; Mon, 20 Oct 2008 09:36:27 +0100 (BST) Received: from [129.156.173.199] by fe-emea-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K910038R3W1IO90@fe-emea-10.sun.com> for psarc-ext@sac.sfbay.sun.com; Mon, 20 Oct 2008 09:36:01 +0100 (BST) Date: Mon, 20 Oct 2008 09:36:01 +0100 From: Darren J Moffat Subject: Re: 2008/640 iSCSI With DHCP In-reply-to: <18681.215.37321.861488@gargle.gargle.HOWL> Sender: Darren.Moffat@Sun.COM To: James Carlson Cc: psarc-ext@sac.sfbay.sun.com, Jack.Meng@Sun.COM Message-id: <48FC42F1.3000105@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <18681.215.37321.861488@gargle.gargle.HOWL> User-Agent: Thunderbird 2.0.0.16 (X11/20080825) Status: RO Content-Length: 1371 James Carlson wrote: > I'm sponsoring this fast-track request for Jack Meng. The timer is > set to 10/24/2008. > > Background > > By default, dhcpagent "canonizes" interfaces under its control on > receipt of SIGTERM. This means that it will reset the IP address > back to 0.0.0.0 during shutdown. This happens regardless of whether > dhcpagent is configured to release or drop leases. I'm sure this was considered but why bother with the reset to 0.0.0.0 on shutdown at all ? What does it achieve ? If that wasn't done would there be any need for this special knowlege of iSCSI (and NFS) both of which seem very "icky" but if needs must. What breaks if dhcpagent does not reset back to 0.0.0.0 on shutdown ? Could it still send the DHCP release to the DHCP server but not reset the interface ? When is dhcpagent going to call the new ioctl ? What are the responses it gets back and what circumstances ? What if we have a "mixed" boot environment where one size of the mirror in the ZFS root pool is local disk and the other is iSCSI ? Does that count as being ISCSI_IS_ACTIVE? What if there are no "boot" pools/filesystems under iSCSI but there are other active iSCSI devices ? Is this only if there is an an iSCSI initiator active, what about targets ? [ Though I wouldn't expect those machines to be iSCSI boot ]. -- Darren J Moffat From Jack.Meng@Sun.COM Mon Oct 20 04:01:04 2008 Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01.SFBay.Sun.COM [129.145.155.118]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m9KB14UZ005920 for ; Mon, 20 Oct 2008 04:01:04 -0700 (PDT) Received: from sineb-mail-2.sun.com ([192.18.19.7]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m9KB12Rx031098 for ; Mon, 20 Oct 2008 04:01:03 -0700 (PDT) Received: from fe-apac-05.sun.com (fe-apac-05.sun.com [192.18.19.176] (may be forged)) by sineb-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id m9KB0vo7002652 for ; Mon, 20 Oct 2008 11:00:57 GMT Received: from conversion-daemon.mail-apac.sun.com by mail-apac.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0K9100301AJLJ000@mail-apac.sun.com> (original mail from Jack.Meng@Sun.COM) for psarc-ext@sac.sfbay.sun.com; Mon, 20 Oct 2008 19:00:57 +0800 (SGT) Received: from [129.158.144.121] by mail-apac.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0K9100ALEALJR7YD@mail-apac.sun.com> for psarc-ext@sac.sfbay.sun.com; Mon, 20 Oct 2008 19:00:56 +0800 (SGT) Date: Mon, 20 Oct 2008 19:00:38 +0800 From: Jack Subject: Re: 2008/640 iSCSI With DHCP In-reply-to: <48FC42F1.3000105@Sun.COM> Sender: Jack.Meng@Sun.COM To: Darren J Moffat Cc: James Carlson , psarc-ext@sac.sfbay.sun.com Message-id: <48FC64D6.9030104@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <18681.215.37321.861488@gargle.gargle.HOWL> <48FC42F1.3000105@Sun.COM> User-Agent: Thunderbird 2.0.0.17 (Windows/20080914) Status: RO Content-Length: 2626 Hi Darren, Comments inline. Darren J Moffat wrote: > James Carlson wrote: >> I'm sponsoring this fast-track request for Jack Meng. The timer is >> set to 10/24/2008. >> >> Background >> >> By default, dhcpagent "canonizes" interfaces under its control on >> receipt of SIGTERM. This means that it will reset the IP address >> back to 0.0.0.0 during shutdown. This happens regardless of whether >> dhcpagent is configured to release or drop leases. > > I'm sure this was considered but why bother with the reset to 0.0.0.0 > on shutdown at all ? What does it achieve ? dhcpagent canonizes all interfaces upon receiving SIGTERM, which usually happends during shutdown/reboot/halt (kill(-1, SIGTERM)) to stop all processes. It makes sense in the view of DHCP protocol and harmless for most (if not all) userland processes as they usually have SMF dependencies. But this behavior hurts kernel elements (iSCSI, NFS..) which are based on TCP/IP stack and supposed to still work after 'kill(-1, SIGTERM)', e.g., to synchronize file system. > If that wasn't done would there be any need for this special knowlege > of iSCSI (and NFS) both of which seem very "icky" but if needs must. I think NO, no special knowledge needed if the configuration of the if doesn't change at all during shutdown. > What breaks if dhcpagent does not reset back to 0.0.0.0 on shutdown > ? Could it still send the DHCP release to the DHCP server but not > reset the interface ? It is a solution, but violating the protocol more severely. > > > When is dhcpagent going to call the new ioctl ? What are the > responses it gets back and what circumstances ? The proposal is to let dhcpagent to call the ioctl in its handler of SIGTERM. The response is either True or False indicating if there're active iSCSI sessions configured. > What if we have a "mixed" boot environment where one size of the > mirror in the ZFS root pool is local disk and the other is iSCSI ? > Does that count as being ISCSI_IS_ACTIVE? > What if there are no "boot" pools/filesystems under iSCSI but there > are other active iSCSI devices ? Yes, these circumstances stand for 'iSCSI active' as there're active iSCSI session and the system may have iSCSI disks in use (root partition or zpool or..). > > Is this only if there is an an iSCSI initiator active, what about > targets ? [ Though I wouldn't expect those machines to be iSCSI boot ]. > Current iSCSI target on Solaris in a service in userland, it addressed this issue well with SMF dependency, and also it is not supposed to still work after 'kill(-1, SIGTERM)'. Best regards, Jack From carlsonj@phorcys.east.sun.com Mon Oct 20 04:26:14 2008 Received: from phorcys.east.sun.com (phorcys.East.Sun.COM [129.148.174.143]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m9KBQEnZ006100 for ; Mon, 20 Oct 2008 04:26:14 -0700 (PDT) Received: from phorcys.east.sun.com (localhost [127.0.0.1]) by phorcys.east.sun.com (8.14.3+Sun/8.14.3) with ESMTP id m9KBQEFL004838; Mon, 20 Oct 2008 07:26:14 -0400 (EDT) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.3+Sun/8.14.3/Submit) id m9KBQED0004835; Mon, 20 Oct 2008 07:26:14 -0400 (EDT) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <18684.27350.29615.777878@gargle.gargle.HOWL> Date: Mon, 20 Oct 2008 07:26:14 -0400 From: James Carlson To: Darren J Moffat Cc: Jack.Meng@sun.com, psarc-ext@sac.sfbay.sun.com Subject: Re: 2008/640 iSCSI With DHCP In-Reply-To: <48FC42F1.3000105@Sun.COM> References: <18681.215.37321.861488@gargle.gargle.HOWL> <48FC42F1.3000105@Sun.COM> X-Mailer: VM 7.01 under Emacs 21.3.1 Status: RO Content-Length: 2705 Darren J Moffat writes: > > By default, dhcpagent "canonizes" interfaces under its control on > > receipt of SIGTERM. This means that it will reset the IP address > > back to 0.0.0.0 during shutdown. This happens regardless of whether > > dhcpagent is configured to release or drop leases. > > I'm sure this was considered but why bother with the reset to 0.0.0.0 on > shutdown at all ? What does it achieve ? [For what it's worth, someone noted by private email that the word "canonize" is used quite incorrectly here. I know. The code has been like that for a long time.] Once we lose the lease, we're required to stop using the address right away, because it may be assigned to someone else. When the dhcpagent daemon exits, though, we lose the ability to take down the address when required. We thus tear it down when we can. Part of the problem is that we have no way of knowing why dhcpagent is being sent SIGTERM. It could be that the user just wants that one service to stop and will wait hours or months before doing anything else, and we can't leave our address in place. It could also be that the system is being shut down, and we can safely just exit, leaving the address working until the OS halts. This is what the text refers to in the final section. This project is just a short-term fix for the operation of iSCSI with DHCP, and slightly cleaner and simpler than the one already in place for NFS. As a separate project, and possibly not a fast-track, someone needs to look into how DHCP, networking, and SMF all interact. I think a better answer in that direction is to make dhcpagent hang around as long as the lease is in place. That'll take a fair bit of rework, though, as we wouldn't want this one service "failing" to exit to block the shutdown of others on which the service claims dependency for start-up purposes. > If that wasn't done would > there be any need for this special knowlege of iSCSI (and NFS) both of > which seem very "icky" but if needs must. What breaks if dhcpagent does > not reset back to 0.0.0.0 on shutdown ? DHCP itself breaks. The server may reallocate the address, discover that it's still in use, and end up marking it as permanently "unusable" due to a conflict. > Could it still send the DHCP > release to the DHCP server but not reset the interface ? That's not legal, for the reason given above. DHCP servers check for address in use (by ARP and ICMP) before assigning them to clients. -- James Carlson, Solaris Networking Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 From casper@holland.sun.com Mon Oct 20 06:03:24 2008 Received: from dm-holland-01.uk.sun.com (dm-holland-01.UK.Sun.COM [129.156.101.192]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m9KD3NaJ008648 for ; Mon, 20 Oct 2008 06:03:24 -0700 (PDT) Received: from holland (room101.Holland.Sun.COM [10.16.117.40]) by dm-holland-01.uk.sun.com (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id m9KD3Ku0015557; Mon, 20 Oct 2008 14:03:20 +0100 (BST) Message-Id: <200810201303.m9KD3Ku0015557@dm-holland-01.uk.sun.com> From: Casper.Dik@sun.com To: James Carlson cc: Darren J Moffat , Jack.Meng@sun.com, psarc-ext@sac.sfbay.sun.com Subject: Re: 2008/640 iSCSI With DHCP In-Reply-To: <18684.27350.29615.777878@gargle.gargle.HOWL> References: <18681.215.37321.861488@gargle.gargle.HOWL> <48FC42F1.3000105@Sun.COM> <18684.27350.29615.777878@gargle.gargle.HOWL> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 20 Oct 2008 15:03:20 +0200 Sender: casper@holland.sun.com Status: RO Content-Length: 336 >That's not legal, for the reason given above. DHCP servers check for >address in use (by ARP and ICMP) before assigning them to clients. > And what is the rest doing? What happens if you don't give up the address? (My systems have a a "pkill -9 dhcpagent" when it sees a "drop/release" message). The system then works. Casper From carlsonj@phorcys.east.sun.com Mon Oct 20 06:36:25 2008 Received: from phorcys.east.sun.com (phorcys.East.Sun.COM [129.148.174.143]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m9KDaODK009059 for ; Mon, 20 Oct 2008 06:36:25 -0700 (PDT) Received: from phorcys.east.sun.com (localhost [127.0.0.1]) by phorcys.east.sun.com (8.14.3+Sun/8.14.3) with ESMTP id m9KDaOQa005466; Mon, 20 Oct 2008 09:36:24 -0400 (EDT) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.3+Sun/8.14.3/Submit) id m9KDaOMl005463; Mon, 20 Oct 2008 09:36:24 -0400 (EDT) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <18684.35160.346432.555279@gargle.gargle.HOWL> Date: Mon, 20 Oct 2008 09:36:24 -0400 From: James Carlson To: Casper.Dik@sun.com Cc: Jack.Meng@sun.com, psarc-ext@sac.sfbay.sun.com Subject: Re: 2008/640 iSCSI With DHCP In-Reply-To: <200810201303.m9KD3Ku0015557@dm-holland-01.uk.sun.com> References: <18681.215.37321.861488@gargle.gargle.HOWL> <48FC42F1.3000105@Sun.COM> <18684.27350.29615.777878@gargle.gargle.HOWL> <200810201303.m9KD3Ku0015557@dm-holland-01.uk.sun.com> X-Mailer: VM 7.01 under Emacs 21.3.1 Status: RO Content-Length: 1726 Casper.Dik@Sun.COM writes: > > > >That's not legal, for the reason given above. DHCP servers check for > >address in use (by ARP and ICMP) before assigning them to clients. > > > > And what is the rest doing? What happens if you don't give up the > address? By "the rest," I assume you're referring to DHCP servers that somehow manage to ignore the text in section 3.1(2) of RFC 2131, and thus fail to check whether the address is in use before handing it out again. In that case, the server thinks that (since the lease expired), the address is free to be reused. When the address is eventually reused, the other client that gets your still-in-use address is _required_ to check whether that address is errantly in use (section 3.1(5)), and send DHCPDECLINE if it is. When it gets your address, it'll see that you're still using it, and refuse to configure it. That DHCPDECLINE message generally causes the server to mark the address as "unusable" so that it's taken out of the address pool. As in the previous case (where the server detects the duplicate), the results are the same: addresses leak out of the pool, because nobody knows who is supposed to be using them. > (My systems have a a "pkill -9 dhcpagent" when it sees a "drop/release" > message). The system then works. Yes; it's possible to damage the daemon in order to obtain the behavior you need for some special situation, but I think it'd be far better if we redesigned the interaction to eliminate the "special" cases. -- James Carlson, Solaris Networking Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 From carlsonj@phorcys.east.sun.com Wed Oct 22 10:12:33 2008 Received: from phorcys.east.sun.com (phorcys.East.Sun.COM [129.148.174.143]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m9MHCWQH025774 for ; Wed, 22 Oct 2008 10:12:32 -0700 (PDT) Received: from phorcys.east.sun.com (localhost [127.0.0.1]) by phorcys.east.sun.com (8.14.3+Sun/8.14.3) with ESMTP id m9MHCWgD011325; Wed, 22 Oct 2008 13:12:32 -0400 (EDT) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.3+Sun/8.14.3/Submit) id m9MHCWGP011322; Wed, 22 Oct 2008 13:12:32 -0400 (EDT) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <18687.24320.276124.821426@gargle.gargle.HOWL> Date: Wed, 22 Oct 2008 13:12:32 -0400 From: James Carlson To: psarc-ext@sac.sfbay.sun.com cc: Jack.Meng@sun.com Subject: Re: 2008/640 iSCSI With DHCP In-Reply-To: <18681.215.37321.861488@gargle.gargle.HOWL> References: <18681.215.37321.861488@gargle.gargle.HOWL> X-Mailer: VM 7.01 under Emacs 21.3.1 Status: RO Content-Length: 404 James Carlson writes: > I'm sponsoring this fast-track request for Jack Meng. The timer is > set to 10/24/2008. This fast-track request was approved during PSARC business today. -- James Carlson, Solaris Networking Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 From Nicolas.Williams@sun.com Wed Oct 22 10:23:11 2008 Received: from binky.Central.Sun.COM (binky.Central.Sun.COM [129.153.128.104]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m9MHNAH3026064 for ; Wed, 22 Oct 2008 10:23:11 -0700 (PDT) Received: from binky.Central.Sun.COM (localhost [127.0.0.1]) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3) with ESMTP id m9MHMjpU020599; Wed, 22 Oct 2008 12:22:45 -0500 (CDT) Received: (from nw141292@localhost) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3/Submit) id m9MHMjPx020598; Wed, 22 Oct 2008 12:22:45 -0500 (CDT) X-Authentication-Warning: binky.Central.Sun.COM: nw141292 set sender to Nicolas.Williams@sun.com using -f Date: Wed, 22 Oct 2008 12:22:45 -0500 From: Nicolas Williams To: Jack Cc: Darren J Moffat , James Carlson , psarc-ext@sac.sfbay.sun.com Subject: Re: 2008/640 iSCSI With DHCP Message-ID: <20081022172245.GT8906@Sun.COM> References: <18681.215.37321.861488@gargle.gargle.HOWL> <48FC42F1.3000105@Sun.COM> <48FC64D6.9030104@sun.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <48FC64D6.9030104@sun.com> User-Agent: Mutt/1.5.7i Status: RO Content-Length: 1151 On Mon, Oct 20, 2008 at 07:00:38PM +0800, Jack wrote: > >If that wasn't done would there be any need for this special knowlege > >of iSCSI (and NFS) both of which seem very "icky" but if needs must. > I think NO, no special knowledge needed if the configuration of the if > doesn't change at all during shutdown. > > What breaks if dhcpagent does not reset back to 0.0.0.0 on shutdown > >? Could it still send the DHCP release to the DHCP server but not > >reset the interface ? > It is a solution, but violating the protocol more severely. Well, yes, but the real issue is that the lease should not be released until after kernel networking modules, like the iSCSI initiator and NFS client, have stopped all activity. Here we have a chicken-and-egg problem. The obvious solution is to move DHCP releasing into the kernel and have a way for the iSCSI initiator and NFS client to get holds on interfaces such that they will not release their leases until the holds are released. So when dhcpagent gets SIGTERM it should put the interface into "release DHCP lease and canonize as soon as all holds go" mode. Would that be ETOOHARD? Nico -- From carlsonj@phorcys.east.sun.com Wed Oct 22 10:27:33 2008 Received: from phorcys.east.sun.com (phorcys.East.Sun.COM [129.148.174.143]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m9MHRXAr026173 for ; Wed, 22 Oct 2008 10:27:33 -0700 (PDT) Received: from phorcys.east.sun.com (localhost [127.0.0.1]) by phorcys.east.sun.com (8.14.3+Sun/8.14.3) with ESMTP id m9MHRXcp011425; Wed, 22 Oct 2008 13:27:33 -0400 (EDT) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.3+Sun/8.14.3/Submit) id m9MHRXOW011422; Wed, 22 Oct 2008 13:27:33 -0400 (EDT) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <18687.25221.97827.667338@gargle.gargle.HOWL> Date: Wed, 22 Oct 2008 13:27:33 -0400 From: James Carlson To: Nicolas Williams Cc: Jack , Darren J Moffat , psarc-ext@sac.sfbay.sun.com Subject: Re: 2008/640 iSCSI With DHCP In-Reply-To: <20081022172245.GT8906@Sun.COM> References: <18681.215.37321.861488@gargle.gargle.HOWL> <48FC42F1.3000105@Sun.COM> <48FC64D6.9030104@sun.com> <20081022172245.GT8906@Sun.COM> X-Mailer: VM 7.01 under Emacs 21.3.1 Status: RO Content-Length: 1060 Nicolas Williams writes: > The obvious solution is to move DHCP releasing into the kernel and have > a way for the iSCSI initiator and NFS client to get holds on interfaces > such that they will not release their leases until the holds are > released. So when dhcpagent gets SIGTERM it should put the interface > into "release DHCP lease and canonize as soon as all holds go" mode. > > Would that be ETOOHARD? Yikes. That'd involve moving a lot of control path machinery into the kernel, particularly for the v6 side where "release" isn't just a shot-in-the-dark datagram. Before designing some detailed solution like that, I'd like to see a project that investigates how this stuff *should* work and looks at the range of possible solutions. I don't know whether what you're suggesting is necessarily part of the answer. Maybe. -- James Carlson, Solaris Networking Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 From Nicolas.Williams@sun.com Wed Oct 22 10:57:07 2008 Received: from binky.Central.Sun.COM (binky.Central.Sun.COM [129.153.128.104]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m9MHv6hE027403 for ; Wed, 22 Oct 2008 10:57:06 -0700 (PDT) Received: from binky.Central.Sun.COM (localhost [127.0.0.1]) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3) with ESMTP id m9MHnBSC020616; Wed, 22 Oct 2008 12:49:11 -0500 (CDT) Received: (from nw141292@localhost) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3/Submit) id m9MHnBHR020615; Wed, 22 Oct 2008 12:49:11 -0500 (CDT) X-Authentication-Warning: binky.Central.Sun.COM: nw141292 set sender to Nicolas.Williams@sun.com using -f Date: Wed, 22 Oct 2008 12:49:11 -0500 From: Nicolas Williams To: James Carlson Cc: Jack , Darren J Moffat , networking-discuss@opensolaris.org Subject: Re: 2008/640 iSCSI With DHCP Message-ID: <20081022174911.GV8906@Sun.COM> References: <18681.215.37321.861488@gargle.gargle.HOWL> <48FC42F1.3000105@Sun.COM> <48FC64D6.9030104@sun.com> <20081022172245.GT8906@Sun.COM> <18687.25221.97827.667338@gargle.gargle.HOWL> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <18687.25221.97827.667338@gargle.gargle.HOWL> User-Agent: Mutt/1.5.7i Status: RO Content-Length: 2028 On Wed, Oct 22, 2008 at 01:27:33PM -0400, James Carlson wrote: > Nicolas Williams writes: > > The obvious solution is to move DHCP releasing into the kernel and have > > a way for the iSCSI initiator and NFS client to get holds on interfaces > > such that they will not release their leases until the holds are > > released. So when dhcpagent gets SIGTERM it should put the interface > > into "release DHCP lease and canonize as soon as all holds go" mode. > > > > Would that be ETOOHARD? > > Yikes. That'd involve moving a lot of control path machinery into the > kernel, particularly for the v6 side where "release" isn't just a > shot-in-the-dark datagram. That's what I meant by ETOOHARD. > Before designing some detailed solution like that, I'd like to see a > project that investigates how this stuff *should* work and looks at > the range of possible solutions. I don't know whether what you're > suggesting is necessarily part of the answer. Maybe. [I'm switching to networking-discuss. Bcc'ing psarc-ext so it knows.] Well, the problem is that we're releasing leases too soon because other parts of the system are still using them, but that use is implied, and because that use may be crucial to system operation during shutdown (for / itself) the release operation has to be almost dead last on the list of things to do (like, just before halt/poweroff/reboot). But "almost dead last" means "long after all user-land processes have been killed." So either we make it so that dhcpagent can be left alive after all the other user-land processes have been killed, or we move the lease release code path into kernel-land. There is another not-a-band-aid option: don't release the lease on shutdown if proper shutdown past the killall step depends on networking. Not releasing a lease is anti-social, but arguably less so than continuing to use the IP address past releasing it. Then again, I think most DHCP servers and networks using DHCP will tolerate brief uses of IP addresses past release. Nico --