From markcarl@sac.sfbay.sun.com Fri Aug 3 14:41:13 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l73LfCGK026135 for ; Fri, 3 Aug 2007 14:41:12 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l73LctCQ026384; Sat, 4 Aug 2007 05:38:55 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM70050BW4UHS00@brm-avmta-1.central.sun.com>; Fri, 03 Aug 2007 15:38:54 -0600 (MDT) Received: from sfbaymail1sca.SFBay.Sun.COM ([129.145.154.35]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM7004ANW4TJA40@brm-avmta-1.central.sun.com>; Fri, 03 Aug 2007 15:38:53 -0600 (MDT) Received: from sac.sfbay.sun.com (sac.SFBay.Sun.COM [129.146.226.132]) by sfbaymail1sca.SFBay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l73LcrKL010258; Fri, 03 Aug 2007 14:38:53 -0700 (PDT) Received: from sac.sfbay.sun.com (localhost [127.0.0.1]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l73Lf8ZU026129; Fri, 03 Aug 2007 14:41:08 -0700 (PDT) Received: (from markcarl@localhost) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id l73Lf8Fa026125; Fri, 03 Aug 2007 14:41:08 -0700 (PDT) Date: Fri, 03 Aug 2007 14:41:08 -0700 (PDT) From: Mark Carlson Subject: iSCSI Software boot [PSARC/2007/450 FastTrack timeout 08/10/2007] To: psarc-ext@Sun.Com Cc: Davis@sac.sfbay.sun.com, Ken@sac.sfbay.sun.com Message-id: <200708032141.l73Lf8Fa026125@sac.sfbay.sun.com> Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 6611 Template Version: @(#)sac_nextcase 1.64 07/13/07 SMI This information is Copyright 2007 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: iSCSI Software boot 1.2. Name of Document Author/Supplier: Author: Sajid Zia 1.3 Date of This Document: 03 August, 2007 4. Technical Description 1. Background: -------------- PSARC 2003/126 introduced the Solaris iSCSI Initiator to Solaris. Solaris Initiator allows access to scsi storage using Sun's TCP/IP stack through a general NIC. The initiator provided a software only solution with no boot support. A traditional boot requires firmware support on adapters which a general NIC does not provide. 2. Problem Statement: --------------------- Storage Area Networks (SANs) have been rapidly growing and gaining popularity over the traditional client-server model due to their ability to efficiently handle data. Typically, Storage Area Networks require Fibre Channel, however, iSCSI for SANs, which encapsulates standard SCSI over existing IP networks, offers a lower cost alternative to expensive Fibre Channel. Fibre Channel SANs support booting from an FC device over a SAN thereby eliminating the need for a local hard disk on a system. Boot is considered a basic OS functionality, and currently no good iSCSI boot solution exists on the market due to the fact that a general NIC does not provide any iSCSI support in its BIOS. 3. Proposal: ------------ PSARC 2004/454 introduced the Solaris New Boot Architecture which greatly simplified a boot process on x86 platforms. iSCSI boot proposal leverages Solaris New Boot to offer a solution which requires no dependency on iSCSI support in bios. This basic idea here is to combine net and disk boot process. It uses pxegrub to download boot archives from the install server in the early stages of the boot and then loads iscsi initiator in client's RAM to access the boot device using regular block disk IOs. This proposal is offering a new iscsi boot functionality only, and does not impact in any way the general solaris install/boot process for non iscsi root devices. 4. Proposal Details: -------------------- When a client system powers up or rebooted, these are the steps which are followed for booting. 1. BIOS running on the machine makes a DHCP request. 2. DHCP server responds with boot server and boot file, i.e., pxegrub 3. BIOS downloads pxegrub using tftp 4. pxegrub displays the grub menu supplied by the boot server 5. pxegrub then gets the boot-archive (mini-kernel) from the boot server and executes the mini kernel. 6. mini-kernel plumbs the network interface and enumerates the iSCSI boot device. 7. root disk is seen through regular NIC port 8. rest of the OS image comes from the root disk using regular disk IOs The steps 1-5 are exactly the same as followed by a normal Solaris Installation process. The only difference is that the boot-archive in step 5 comes from the root disk and not an Installation boot-archive Installation ------------ The installation process changed a bit in the sense that we need to provide iscsi boot target IP address and client's Initiator Node Name during the installation to discover the boot device. This is accomplished by using bootparams. The following name=val entry is required in /etc/bootparams on the install server. iscsi-boot=Target_IP:Initiator_Node_Name An example: iscsi-1 iscsi-boot=192.29.74.155:iqn.1986-03.com.sun:01:iscsi-1 Once the installation is complete, the following extra steps an administrator needs to do to prepare the client for booting: 1. newly created boot archives need to be copied to the Install server for subsequent booting 2. menu list on the server need to reference the new boot archives for booting. The code changes for the project are mostly confined to iscsi driver and require no interface/architectural changes. These are some additional changes. 1. During the installation, an rc script is needed to discover and enumerate the boot lun. 2. filelist.ramdisk file needs to be updated with iscsi persistent database files which are needed to boot 3. SUNWsibi pkgmap needs to include iscsi database file entry. 4. A minor sockfs change is required, addressed by the CR 6422549 5. A minor update is needed in smf_include.sh script. 5. Limitations -------------- This solution offers an innovative way of accomplishing iscsi boot on a general NIC. The first phase of the project has few limitations which will be addressed in subsequent phases. 1. It is supported only on x86/x64 platforms due to a dependency on Solairs NewBoot 2. The customers will be encouraged to use the first phase of this solution over physically secured networks. The next phase of the project will add CHAP authentication. 3. Currently require targets to have TPGS disabled so that iqn based names appear in the root device name. The next phase will add multi-pathing support along with IPMP. 4. Only manual network installation is supported. 5. After the installation, the boot archives must be moved to the installed server to be used for booting. This needs to be done by the administrator. 6. Install server is required at the time of boot/reboots 7. The boot archives on the root disk and on the installed server must be in sync. This needs to be ensured by the administrator. 8. dump/sync is only supported on a local partition due to a networking bug 6205961, which will be addressed in the next phase. 6. Benefits ----------- Despite these limitations, this solution offers many benefits which should outweigh the limitations for phase I release. 1. Offers an innovative solution to combine net/disk boot 2. No hardware dependencies 3. Works on any modern general NIC which has pxe support. 4. Superior to "diskless boot" which requires boot server to be available all the time to service boot clients. 5. A generic Solaris iscsi boot solution will offer a huge marketing advantage over our competitors. Currently there is no good generic iSCSI boot solution available in the market. 8. References: -------------- 1. PSARC 2003/126 iSCSI Project 2. PSARC 2004/454 Solaris Boot Architecture 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: FastTrack 6.6. ARC Exposure: open From Nicolas.Williams@sun.com Fri Aug 3 14:58:10 2007 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l73LwApW026689 for ; Fri, 3 Aug 2007 14:58:10 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l73LtrTR002458 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Fri, 3 Aug 2007 14:55:53 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM700703WX5Q200@nwk-avmta-1.sfbay.Sun.COM> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Fri, 03 Aug 2007 14:55:53 -0700 (PDT) Received: from binky.Central.Sun.COM ([129.153.128.104]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM700F11WX4M170@nwk-avmta-1.sfbay.Sun.COM> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Fri, 03 Aug 2007 14:55:52 -0700 (PDT) Received: from binky.Central.Sun.COM (localhost [127.0.0.1]) by binky.Central.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id l73LtpIv003758; Fri, 03 Aug 2007 16:55:51 -0500 (CDT) Received: (from nw141292@localhost) by binky.Central.Sun.COM (8.14.1+Sun/8.14.1/Submit) id l73Ltp6Z003757; Fri, 03 Aug 2007 16:55:51 -0500 (CDT) Date: Fri, 03 Aug 2007 16:55:51 -0500 From: Nicolas Williams Subject: Re: iSCSI Software boot [PSARC/2007/450 FastTrack timeout 08/10/2007] In-reply-to: <200708032141.l73Lf8Fa026125@sac.sfbay.sun.com> To: Mark Carlson Cc: psarc-ext@sun.com, Davis@sac.sfbay.sun.com, Ken@sac.sfbay.sun.com Message-id: <20070803215551.GW1490@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <200708032141.l73Lf8Fa026125@sac.sfbay.sun.com> X-Authentication-warning: binky.Central.Sun.COM: nw141292 set sender to Nicolas.Williams@sun.com using -f User-Agent: Mutt/1.5.7i Status: RO Content-Length: 1596 On Fri, Aug 03, 2007 at 02:41:08PM -0700, Mark Carlson wrote: > 2. The customers will be encouraged to use the first phase of this > solution over physically secured networks. The next phase of the > project will add CHAP authentication. CHAP will not be sufficient. iSCSI relies on IPsec for integrity and confidentiality protection of data on the wire. I imagine that getting IKE up and running from a boot archive prior to mounting / simply does not fit the current architecture, so I'll not suggest that. But manually keying an SA that can be used until the boot process can get IKE up seems like a reasonable approach. To do better than manual IPsec SA keying will require a more general approach to security in the boot architecture as secure NFS w/ DH or Kerberos V, and iSCSI with IPsec and PSK or PKI for IKE currently require running quite a bit of code that currently only runs in user-land. I suppose that's not-this-case... OC (off case): Implementing those things in kernel-land would be an option, but it sounds like a lot of work. An alternative would be to support running a minimal set of user-land processes (including daemons) from the boot archive/miniroot (and with the archive/miniroot as /) and restart them when the real / is available. Either way we could support booting securely with / on NFS w/ RPCSEC_GSS or iSCSI w/ IPsec and non-manual SA keying. In any case, the point is: iSCSI w/ CHAP is not enough to get beyond the "physically secured networks" requirement. Nico -- From Nicolas.Williams@sun.com Fri Aug 3 15:04:41 2007 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l73M4foG026827 for ; Fri, 3 Aug 2007 15:04:41 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l73M2PjR005943 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Fri, 3 Aug 2007 15:02:25 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM700707X7Z0L00@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Fri, 03 Aug 2007 16:02:24 -0600 (MDT) Received: from binky.Central.Sun.COM ([129.153.128.104]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM700598X7Z5E10@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Fri, 03 Aug 2007 16:02:23 -0600 (MDT) Received: from binky.Central.Sun.COM (localhost [127.0.0.1]) by binky.Central.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id l73M2MiV003776; Fri, 03 Aug 2007 17:02:22 -0500 (CDT) Received: (from nw141292@localhost) by binky.Central.Sun.COM (8.14.1+Sun/8.14.1/Submit) id l73M2MBu003775; Fri, 03 Aug 2007 17:02:22 -0500 (CDT) Date: Fri, 03 Aug 2007 17:02:22 -0500 From: Nicolas Williams Subject: Re: iSCSI Software boot [PSARC/2007/450 FastTrack timeout 08/10/2007] In-reply-to: <20070803215551.GW1490@Sun.COM> To: Mark Carlson Cc: psarc-ext@sun.com, Ken@sac.sfbay.sun.com, Davis@sac.sfbay.sun.com Message-id: <20070803220222.GY1490@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <200708032141.l73Lf8Fa026125@sac.sfbay.sun.com> <20070803215551.GW1490@Sun.COM> X-Authentication-warning: binky.Central.Sun.COM: nw141292 set sender to Nicolas.Williams@sun.com using -f User-Agent: Mutt/1.5.7i Status: RO Content-Length: 46 Also, presumably this will work with WANboot? From David.Kahn@sun.com Fri Aug 3 15:06:56 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l73M6ton026847 for ; Fri, 3 Aug 2007 15:06:56 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l73M4VRb005676 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Sat, 4 Aug 2007 06:04:39 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM700C07XBOER00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 03 Aug 2007 15:04:36 -0700 (PDT) Received: from sfbaymail2sca.sfbay.sun.com ([129.145.155.42]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM70043KXBNMA70@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 03 Aug 2007 15:04:35 -0700 (PDT) Received: from dtmail.sfbay.sun.com (pkg.SFBay.Sun.COM [129.146.90.56]) by sfbaymail2sca.sfbay.sun.com (8.13.6+Sun/8.12.10/ENSMAIL,v2.2) with ESMTP id l73M4YCI020836; Fri, 03 Aug 2007 15:04:34 -0700 (PDT) Received: from [10.6.101.8] (dtsrs3 [10.6.101.8]) by dtmail.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l73M4XZP094326; Fri, 03 Aug 2007 15:04:34 -0700 (PDT) Date: Fri, 03 Aug 2007 15:04:33 -0700 From: David Kahn Subject: Re: iSCSI Software boot [PSARC/2007/450 FastTrack timeout 08/10/2007] To: Mark Carlson Cc: PSARC-ext@sun.com, John Johnson Message-id: <46B3A671.5030001@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 User-Agent: Thunderbird 2.0.0.4 (X11/20070622) Status: RO Content-Length: 699 Mark Carlson wrote: > 5. Limitations > -------------- > > This solution offers an innovative way of accomplishing iscsi boot > on a general NIC. The first phase of the project has few limitations > which will be addressed in subsequent phases. > > 1. It is supported only on x86/x64 platforms due to a dependency on > Solairs NewBoot See PSARC/2006/501 (sparc newboot support, fast track timeout Aug 7, 2007) Given 2006/501, why would we want to allow this case to be approved with the restriction I quoted above? Am I missing something? Please remove the restriction from the materials, including updating the section 4 example data flow which is currently x86 specific. -David From jek3@sun.com Fri Aug 3 15:10:43 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l73MAgYj026894 for ; Fri, 3 Aug 2007 15:10:43 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l73M8AcQ007173 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Sat, 4 Aug 2007 06:08:25 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM700C03XHZLN00@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Fri, 03 Aug 2007 15:08:23 -0700 (PDT) Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.59]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM7004W1XHYMK60@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Fri, 03 Aug 2007 15:08:23 -0700 (PDT) Received: from [129.150.12.124] (vpn-129-150-12-124.SFBay.Sun.COM [129.150.12.124]) by jurassic-x4600.sfbay.sun.com (8.14.1+Sun/8.14.1) with ESMTP id l73M8LFI655023; Fri, 03 Aug 2007 15:08:22 -0700 (PDT) Date: Fri, 03 Aug 2007 12:05:48 -1000 From: Joseph Kowalski Subject: Re: iSCSI Software boot [PSARC/2007/450 FastTrack timeout 08/10/2007] In-reply-to: <200708032141.l73Lf8Fa026125@sac.sfbay.sun.com> To: Mark Carlson Cc: PSARC-ext@sun.com, Davis@sac.sfbay.sun.com, Ken@sac.sfbay.sun.com Message-id: <46B3A6BC.5010209@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708032141.l73Lf8Fa026125@sac.sfbay.sun.com> User-Agent: Thunderbird 1.5.0.5 (X11/20060925) Status: RO Content-Length: 798 Mark Carlson wrote: > 3. Currently require targets to have TPGS disabled so that iqn based > names appear in the root device name. The next phase will add > multi-pathing support along with IPMP. > As the Geico Caveman says, "Huh?". (I even look a little like him.) Could you explain this just for my understanding (actually just the first sentence)? ------- I suspect the file-layout for the server is defined in an existing case. That would be? ------- > 8. dump/sync is only supported on a local partition due to a > networking bug 6205961, which will be addressed in the next > phase. This *seems* like a bit of a serious limitation (the "S" in RAS). Would it be wise to get this "bug" fixed before this case integrates (ie: a dependency)? - thanks, - jek3 From Sajid.Zia@sun.com Fri Aug 3 16:14:29 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l73NESI7028566 for ; Fri, 3 Aug 2007 16:14:28 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l73NC7cg029858 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Sat, 4 Aug 2007 07:12:11 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM800F050GA7S00@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Fri, 03 Aug 2007 16:12:10 -0700 (PDT) Received: from sca-es-mail-2.sun.com ([192.18.43.133]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM8004210G9MR90@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Fri, 03 Aug 2007 16:12:09 -0700 (PDT) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id l73NC9s7024746 for ; Fri, 03 Aug 2007 16:12:09 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JM8006010F3XR00@fe-sfbay-10.sun.com> (original mail from Sajid.Zia@Sun.COM) for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Fri, 03 Aug 2007 16:12:09 -0700 (PDT) Received: from [129.146.56.56] by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JM80022V0G81020@fe-sfbay-10.sun.com>; Fri, 03 Aug 2007 16:12:09 -0700 (PDT) Date: Fri, 03 Aug 2007 16:09:10 -0700 From: Sajid Zia Subject: Re: [Fwd: Re: iSCSI Software boot [PSARC/2007/450 FastTrack timeout 08/10/2007]] In-reply-to: <46B3A8C9.8070003@sun.com> Sender: Sajid.Zia@sun.com To: Nicolas.Williams@sun.com Cc: Kenneth Davis , psarc-ext@sun.com, markcarl@sac.sfbay.sun.com Message-id: <46B3B596.5050902@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <46B3A8C9.8070003@sun.com> User-Agent: Thunderbird 2.0.0.0 (X11/20070411) Status: RO Content-Length: 2479 Mark A. Carlson wrote: > > > ------------------------------------------------------------------------ > > Subject: > Re: iSCSI Software boot [PSARC/2007/450 FastTrack timeout 08/10/2007] > From: > Nicolas Williams > Date: > Fri, 03 Aug 2007 16:55:51 -0500 > To: > Mark Carlson > > To: > Mark Carlson > CC: > psarc-ext@sun.com, Davis@sac.sfbay.sun.com, Ken@sac.sfbay.sun.com > > > On Fri, Aug 03, 2007 at 02:41:08PM -0700, Mark Carlson wrote: > >> 2. The customers will be encouraged to use the first phase of this >> solution over physically secured networks. The next phase of the >> project will add CHAP authentication. >> > > CHAP will not be sufficient. iSCSI relies on IPsec for integrity and > confidentiality protection of data on the wire. > > I imagine that getting IKE up and running from a boot archive prior to > mounting / simply does not fit the current architecture, so I'll not > suggest that. But manually keying an SA that can be used until the boot > process can get IKE up seems like a reasonable approach. > > To do better than manual IPsec SA keying will require a more general > approach to security in the boot architecture as secure NFS w/ DH or > Kerberos V, and iSCSI with IPsec and PSK or PKI for IKE currently > require running quite a bit of code that currently only runs in > user-land. I suppose that's not-this-case... > > OC (off case): Implementing those things in kernel-land would be an > option, but it sounds like a lot of work. > > An alternative would be to support running a minimal set > of user-land processes (including daemons) from the boot > archive/miniroot (and with the archive/miniroot as /) and > restart them when the real / is available. > > Either way we could support booting securely with / on > NFS w/ RPCSEC_GSS or iSCSI w/ IPsec and non-manual SA > keying. > > In any case, the point is: iSCSI w/ CHAP is not enough to get beyond the > "physically secured networks" requirement. > > Nico > Nicolas, I totally agree with you, I didn't mean to imply that CHAP authentication is going to solve security issue when we remove the restriction of physically secured networks. We have to devise a strategy around IPsec support for iSCSI boot in the next Phase of the project. Any input from you will certainly be helpful. Thanks, Sajid From Sajid.Zia@sun.com Fri Aug 3 16:40:20 2007 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l73NeKeu028837 for ; Fri, 3 Aug 2007 16:40:20 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l73Nc24M029186 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Fri, 3 Aug 2007 16:38:03 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM800I051NFKP00@nwk-avmta-1.sfbay.Sun.COM> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Fri, 03 Aug 2007 16:38:03 -0700 (PDT) Received: from sca-es-mail-2.sun.com ([192.18.43.133]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM800F2W1NFM2C0@nwk-avmta-1.sfbay.Sun.COM> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Fri, 03 Aug 2007 16:38:03 -0700 (PDT) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id l73Nc2rf026416 for ; Fri, 03 Aug 2007 16:38:02 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JM8001011N4YG00@fe-sfbay-10.sun.com> (original mail from Sajid.Zia@Sun.COM) for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Fri, 03 Aug 2007 16:38:02 -0700 (PDT) Received: from [129.146.56.56] by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JM8002X11NE1080@fe-sfbay-10.sun.com>; Fri, 03 Aug 2007 16:38:02 -0700 (PDT) Date: Fri, 03 Aug 2007 16:35:04 -0700 From: Sajid Zia Subject: Re: [Fwd: Re: iSCSI Software boot [PSARC/2007/450 FastTrack timeout 08/10/2007]] In-reply-to: <46B3AA0C.6040003@sun.com> Sender: Sajid.Zia@sun.com To: jek3@sun.com Cc: psarc-ext@sun.com, Kenneth Davis , "Mark A. Carlson" Message-id: <46B3BBA8.10107@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <46B3AA0C.6040003@sun.com> User-Agent: Thunderbird 2.0.0.0 (X11/20070411) Status: RO Content-Length: 1372 > >> 3. Currently require targets to have TPGS disabled so that iqn based >> names appear in the root device name. The next phase will add >> multi-pathing support along with IPMP. >> > As the Geico Caveman says, "Huh?". (I even look a little like him.) > > Could you explain this just for my understanding (actually just the > first sentence)? This basically means that iscsi boot lun can not be under mpxio for this phase. mpxio based names do not include "iscsi" in device path, and currently this solution relies on device path to load the drivers. we will address it in the next phase. > > ------- > > I suspect the file-layout for the server is defined in an existing > case. That would be? file-layout of the server is what is defined by Solaris NewBoot project, which is PSARC 2004/454 > > ------- > >> 8. dump/sync is only supported on a local partition due to a >> networking bug 6205961, which will be addressed in the next >> phase. > This *seems* like a bit of a serious limitation (the "S" in RAS). > Would it be wise > to get this "bug" fixed before this case integrates (ie: a dependency)? We will be working with networking folks to resolve it before the putback. We don't want to make it a gating factor for the first release since there is a workaround. Thanks, Sajid > > - thanks, > > - jek3 > From sommerfeld@sun.com Fri Aug 3 17:50:55 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l740os2A029686 for ; Fri, 3 Aug 2007 17:50:55 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail5.uk.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l740maxp013585; Sat, 4 Aug 2007 01:48:37 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM80010F4WZFO00@nwk-avmta-1.sfbay.Sun.COM>; Fri, 03 Aug 2007 17:48:35 -0700 (PDT) Received: from localhost.east.sun.com ([129.148.19.3]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM800FPX4WPM9C0@nwk-avmta-1.sfbay.Sun.COM>; Fri, 03 Aug 2007 17:48:26 -0700 (PDT) Received: from localhost.east.sun.com (localhost [127.0.0.1]) by localhost.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id l740mMxZ001824; Fri, 03 Aug 2007 20:48:22 -0400 (EDT) Received: (from sommerfeld@localhost) by localhost.east.sun.com (8.14.1+Sun/8.14.1/Submit) id l740mKSQ001823; Fri, 03 Aug 2007 20:48:20 -0400 (EDT) Date: Fri, 03 Aug 2007 20:48:20 -0400 From: Bill Sommerfeld Subject: Re: iSCSI Software boot [PSARC/2007/450 FastTrack timeout 08/10/2007] In-reply-to: <200708032141.l73Lf8Fa026125@sac.sfbay.sun.com> To: Mark Carlson Cc: psarc-ext@sun.com, Ken@sac.sfbay.sun.com, Davis@sac.sfbay.sun.com Message-id: <1186188500.1808.13.camel@localhost> MIME-version: 1.0 X-Mailer: Evolution 2.10.2 Content-type: text/plain Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708032141.l73Lf8Fa026125@sac.sfbay.sun.com> X-Authentication-warning: localhost.east.sun.com: sommerfeld set sender to sommerfeld@sun.com using -f Status: RO Content-Length: 1587 On Fri, 2007-08-03 at 14:41 -0700, Mark Carlson wrote: > 7. root disk is seen through regular NIC port There's a lot of detail left out here. last I checked, solaris's iSCSI runs over the solaris IP stack, not over a raw NIC. A significant fraction of IP functionality (DHCP, IKE, routing, ipmp, etc.,) involve significant amounts of user space code (commands and daemons). Those daemons also make filesystem-related system calls and do other things which may need disk i/o in normal operation. If one of those daemons accesses part of the filesystem backed by an iSCSI disk, I'd expect some sort of deadlock will be possible (syscall made by routing daemon can't make forward progress without block X from the root volume; iscsi can't fetch that block without being able to transmit IP packets to the server; IP packets can't reach that server until the daemon reestablishes a working route to the server; daemon can't reestablish a working route until the aforementioned syscall can make forward progress...). I expect that running with an iSCSI root will have interactions with IP which closely resemble operating with NFS diskless -- a capability that is no longer actively used and which is not routinely tested. while it may be possible to boot off an iSCSI disk, I suspect that there are a bunch of dragons lurking here with respect to running reliably off of iSCSI in non-trivial networks. I'd feel a lot better about our ability to enumerate all these subtle dependencies, gotchas, and limitations if this case got a full (non-fasttrack) review. - Bill From sommerfeld@sun.com Fri Aug 3 18:01:37 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l7411a8n000257 for ; Fri, 3 Aug 2007 18:01:36 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l740xFKB015649; Sat, 4 Aug 2007 01:59:16 +0100 (BST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM800I055ER5P00@brm-avmta-1.central.sun.com>; Fri, 03 Aug 2007 18:59:15 -0600 (MDT) Received: from localhost.east.sun.com ([129.148.19.3]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM8004OU5EPJAB0@brm-avmta-1.central.sun.com>; Fri, 03 Aug 2007 18:59:14 -0600 (MDT) Received: from localhost.east.sun.com (localhost [127.0.0.1]) by localhost.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id l740xCBI001875; Fri, 03 Aug 2007 20:59:12 -0400 (EDT) Received: (from sommerfeld@localhost) by localhost.east.sun.com (8.14.1+Sun/8.14.1/Submit) id l740xCHL001874; Fri, 03 Aug 2007 20:59:12 -0400 (EDT) Date: Fri, 03 Aug 2007 20:59:11 -0400 From: Bill Sommerfeld Subject: Re: iSCSI Software boot [PSARC/2007/450 FastTrack timeout 08/10/2007] In-reply-to: <200708032141.l73Lf8Fa026125@sac.sfbay.sun.com> To: Mark Carlson Cc: psarc-ext@sun.com, Kenneth.Davis@sun.com, david.miner@sun.com, Sajid.Zia@sun.com Message-id: <1186189151.1808.19.camel@localhost> MIME-version: 1.0 X-Mailer: Evolution 2.10.2 Content-type: text/plain Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708032141.l73Lf8Fa026125@sac.sfbay.sun.com> X-Authentication-warning: localhost.east.sun.com: sommerfeld set sender to sommerfeld@sun.com using -f Status: RO Content-Length: 688 On Fri, 2007-08-03 at 14:41 -0700, Mark Carlson wrote: > The installation process changed a bit in the sense that we need to > provide iscsi boot target IP address and client's Initiator Node > Name during the installation to discover the boot device. This is > accomplished by using bootparams. The following name=val entry is > required in /etc/bootparams on the install server. > > iscsi-boot=Target_IP:Initiator_Node_Name whoa! you need *both* DHCP *and* bootparams configured for each client? Until now, you'd only ever need one or the other, and never both. Since early stages boot from DHCP, this should be in DHCP too, not bootparams... - Bill From Sajid.Zia@sun.com Fri Aug 3 20:55:38 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l743tbcK001817 for ; Fri, 3 Aug 2007 20:55:38 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l743rH5I006349; Sat, 4 Aug 2007 11:53:20 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM800603DGVEC00@brm-avmta-1.central.sun.com>; Fri, 03 Aug 2007 21:53:19 -0600 (MDT) Received: from brmea-mail-2.sun.com ([192.18.98.43]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM8005P9DGU5BB0@brm-avmta-1.central.sun.com>; Fri, 03 Aug 2007 21:53:18 -0600 (MDT) Received: from fe-amer-04.sun.com ([192.18.108.178]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l743rIug011824; Sat, 04 Aug 2007 03:53:18 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JM800D01C62UL00@mail-amer.sun.com> (original mail from Sajid.Zia@Sun.COM) ; Fri, 03 Aug 2007 21:53:18 -0600 (MDT) Received: from [76.200.183.153] by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JM800CBLDGNGS2B@mail-amer.sun.com>; Fri, 03 Aug 2007 21:53:18 -0600 (MDT) Date: Fri, 03 Aug 2007 20:55:29 -0700 From: Sajid Zia Subject: Re: iSCSI Software boot [PSARC/2007/450 FastTrack timeout 08/10/2007] In-reply-to: <1186189151.1808.19.camel@localhost> Sender: Sajid.Zia@sun.com To: Bill Sommerfeld Cc: Mark Carlson , psarc-ext@sun.com, Kenneth.Davis@sun.com, david.miner@sun.com Message-id: <46B3F8B1.1080904@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708032141.l73Lf8Fa026125@sac.sfbay.sun.com> <1186189151.1808.19.camel@localhost> User-Agent: Thunderbird 1.5.0.12 (Windows/20070509) Status: RO Content-Length: 1190 Bill Sommerfeld wrote: > On Fri, 2007-08-03 at 14:41 -0700, Mark Carlson wrote: > > >> The installation process changed a bit in the sense that we need to >> provide iscsi boot target IP address and client's Initiator Node >> Name during the installation to discover the boot device. This is >> accomplished by using bootparams. The following name=val entry is >> required in /etc/bootparams on the install server. >> >> iscsi-boot=Target_IP:Initiator_Node_Name >> > > whoa! you need *both* DHCP *and* bootparams configured for each client? > Until now, you'd only ever need one or the other, and never both. Since > early stages boot from DHCP, this should be in DHCP too, not > bootparams... > A bootparams parameter is a better approach rather than inventing a new dhcp macro or overloading an exisiting one to provide this information. That is the feedback I had gotten from Solaris NewBoot folks. Please note that we are not using any special dhcp macro to provide client specific information. We use the standard solaris net installation process to do the boot as well. Only thing new is the bootparams parameter. Sajid > - Bill > > From Sajid.Zia@sun.com Fri Aug 3 21:12:44 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l744Ch7F001955 for ; Fri, 3 Aug 2007 21:12:43 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l744AH6N016515; Sat, 4 Aug 2007 12:10:25 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM800405E9CEA00@nwk-avmta-2.sfbay.sun.com>; Fri, 03 Aug 2007 21:10:24 -0700 (PDT) Received: from brmea-mail-3.sun.com ([192.18.98.34]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM80004AE9CDJ70@nwk-avmta-2.sfbay.sun.com>; Fri, 03 Aug 2007 21:10:24 -0700 (PDT) Received: from fe-amer-02.sun.com ([192.18.108.176]) by brmea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l744AO1Y011234; Sat, 04 Aug 2007 04:10:24 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JM800101DPS8M00@mail-amer.sun.com> (original mail from Sajid.Zia@Sun.COM) ; Fri, 03 Aug 2007 22:10:23 -0600 (MDT) Received: from [76.200.183.153] by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JM8002FTE9ACJ72@mail-amer.sun.com>; Fri, 03 Aug 2007 22:10:23 -0600 (MDT) Date: Fri, 03 Aug 2007 21:12:40 -0700 From: Sajid Zia Subject: Re: [Fwd: Re: iSCSI Software boot [PSARC/2007/450 FastTrack timeout 08/10/2007]] In-reply-to: <46B3E8C9.2000208@sun.com> Sender: Sajid.Zia@sun.com To: Bill Sommerfeld Cc: psarc-ext@sun.com, Mark Carlson , Ken Davis Message-id: <46B3FCB8.2080701@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <46B3E8C9.2000208@sun.com> User-Agent: Thunderbird 1.5.0.12 (Windows/20070509) Status: RO Content-Length: 2203 Mark A. Carlson wrote: > > > > Hi Bill, the user space code works just fine and comes into the picture later when the root is already mounted. iscsi driver plumbs the interface and brings the network up during the mountroot operation itself. It uses ldi_vp_from_name() to create a vnode prior to modrootloaded. I have a system booted up and running with all the networking user/kernel code working fine. All the code changes are confined to iscsi driver and there are no interface/architectural changes. Thanks, Sajid > On Fri, 2007-08-03 at 14:41 -0700, Mark Carlson wrote: > >> 7. root disk is seen through regular NIC port >> > > There's a lot of detail left out here. > > last I checked, solaris's iSCSI runs over the solaris IP stack, not over > a raw NIC. > > A significant fraction of IP functionality (DHCP, IKE, routing, ipmp, > etc.,) involve significant amounts of user space code (commands and > daemons). Those daemons also make filesystem-related system calls and > do other things which may need disk i/o in normal operation. > > If one of those daemons accesses part of the filesystem backed by an > iSCSI disk, I'd expect some sort of deadlock will be possible (syscall > made by routing daemon can't make forward progress without block X from > the root volume; iscsi can't fetch that block without being able to > transmit IP packets to the server; IP packets can't reach that server > until the daemon reestablishes a working route to the server; daemon > can't reestablish a working route until the aforementioned syscall can > make forward progress...). > > I expect that running with an iSCSI root will have interactions with IP > which closely resemble operating with NFS diskless -- a capability that > is no longer actively used and which is not routinely tested. > > while it may be possible to boot off an iSCSI disk, I suspect that there > are a bunch of dragons lurking here with respect to running reliably off > of iSCSI in non-trivial networks. > > I'd feel a lot better about our ability to enumerate all these subtle > dependencies, gotchas, and limitations if this case got a full > (non-fasttrack) review. > > - Bill > > > > > > > > > From Nicolas.Williams@sun.com Fri Aug 3 22:03:44 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l7453hPw002482 for ; Fri, 3 Aug 2007 22:03:44 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail5.uk.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l7451IbY011481; Sat, 4 Aug 2007 06:01:23 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM800101GM9PD00@nwk-avmta-1.sfbay.Sun.COM>; Fri, 03 Aug 2007 22:01:21 -0700 (PDT) Received: from binky.Central.Sun.COM ([129.153.128.104]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM8000VKGM9E400@nwk-avmta-1.sfbay.Sun.COM>; Fri, 03 Aug 2007 22:01:21 -0700 (PDT) Received: from binky.Central.Sun.COM (localhost [127.0.0.1]) by binky.Central.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id l7451Ksj004127; Sat, 04 Aug 2007 00:01:20 -0500 (CDT) Received: (from nw141292@localhost) by binky.Central.Sun.COM (8.14.1+Sun/8.14.1/Submit) id l7451J2l004126; Sat, 04 Aug 2007 00:01:19 -0500 (CDT) Date: Sat, 04 Aug 2007 00:01:19 -0500 From: Nicolas Williams Subject: Re: iSCSI Software boot [PSARC/2007/450 FastTrack timeout 08/10/2007] In-reply-to: <46B3F8B1.1080904@sun.com> To: Sajid Zia Cc: Bill Sommerfeld , Kenneth.Davis@sun.com, psarc-ext@sun.com, Mark Carlson , david.miner@sun.com Message-id: <20070804050119.GH1490@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <200708032141.l73Lf8Fa026125@sac.sfbay.sun.com> <1186189151.1808.19.camel@localhost> <46B3F8B1.1080904@sun.com> X-Authentication-warning: binky.Central.Sun.COM: nw141292 set sender to Nicolas.Williams@sun.com using -f User-Agent: Mutt/1.5.7i Status: RO Content-Length: 749 On Fri, Aug 03, 2007 at 08:55:29PM -0700, Sajid Zia wrote: > Bill Sommerfeld wrote: > > whoa! you need *both* DHCP *and* bootparams configured for each client? > > Until now, you'd only ever need one or the other, and never both. Since > > early stages boot from DHCP, this should be in DHCP too, not > > bootparams... > > > A bootparams parameter is a better approach rather than inventing a > new dhcp macro or overloading an exisiting one to provide this > information. [...] But not everyone has a bootparams server. DHCP servers are available pretty much on every OS, but bootparams servers, not so much (and even where they are there's still the sticking point of integrated administration facilities, which are likely to be missing). From casper@holland.sun.com Sat Aug 4 03:10:21 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l74AAKWl006260 for ; Sat, 4 Aug 2007 03:10:21 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l74A7wLC029949; Sat, 4 Aug 2007 18:08:03 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM80070DUTCSN00@brm-avmta-1.central.sun.com>; Sat, 04 Aug 2007 04:08:00 -0600 (MDT) Received: from dm-holland-01.uk.sun.com ([129.156.101.192]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM80059HUTBWF10@brm-avmta-1.central.sun.com>; Sat, 04 Aug 2007 04:07:59 -0600 (MDT) Received: from holland (room101.Holland.Sun.COM [129.159.130.93]) by dm-holland-01.uk.sun.com (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l74A7u8H022759; Sat, 04 Aug 2007 11:07:56 +0100 (BST) Date: Sat, 04 Aug 2007 12:07:56 +0200 From: Casper.Dik@sun.com Subject: Re: iSCSI Software boot [PSARC/2007/450 FastTrack timeout 08/10/2007] In-reply-to: <1186188500.1808.13.camel@localhost> Sender: casper@holland.sun.com To: Bill Sommerfeld Cc: Mark Carlson , psarc-ext@sun.com, Ken@sac.sfbay.sun.com, Davis@sac.sfbay.sun.com Message-id: <200708041007.l74A7u8H022759@dm-holland-01.uk.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708032141.l73Lf8Fa026125@sac.sfbay.sun.com> <1186188500.1808.13.camel@localhost> Status: RO Content-Length: 532 >I expect that running with an iSCSI root will have interactions with IP >which closely resemble operating with NFS diskless -- a capability that >is no longer actively used and which is not routinely tested. Isn't there at least some form of regular testing of diskless boot when using network based installs? On x86 they run from a RAM disk, true, but on SPARC they still run from an NFS root. Not that you were able to use the more esoteric functions of IP: routing and such needs to be simple for it to bootstrap. Casper From casper@holland.sun.com Sat Aug 4 03:11:33 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l74ABWCM006281 for ; Sat, 4 Aug 2007 03:11:32 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail5.uk.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l74A9D4R015191; Sat, 4 Aug 2007 11:09:13 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM800L05UVCDL00@nwk-avmta-2.sfbay.sun.com>; Sat, 04 Aug 2007 03:09:12 -0700 (PDT) Received: from dm-holland-01.uk.sun.com ([129.156.101.192]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM8008FYUVBQL60@nwk-avmta-2.sfbay.sun.com>; Sat, 04 Aug 2007 03:09:12 -0700 (PDT) Received: from holland (room101.Holland.Sun.COM [129.159.130.93]) by dm-holland-01.uk.sun.com (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l74A97Mf022804; Sat, 04 Aug 2007 11:09:07 +0100 (BST) Date: Sat, 04 Aug 2007 12:09:07 +0200 From: Casper.Dik@sun.com Subject: Re: iSCSI Software boot [PSARC/2007/450 FastTrack timeout 08/10/2007] In-reply-to: <1186189151.1808.19.camel@localhost> Sender: casper@holland.sun.com To: Bill Sommerfeld Cc: Mark Carlson , psarc-ext@sun.com, Kenneth.Davis@sun.com, david.miner@sun.com, Sajid.Zia@sun.com Message-id: <200708041009.l74A97Mf022804@dm-holland-01.uk.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708032141.l73Lf8Fa026125@sac.sfbay.sun.com> <1186189151.1808.19.camel@localhost> Status: RO Content-Length: 813 >On Fri, 2007-08-03 at 14:41 -0700, Mark Carlson wrote: > >> The installation process changed a bit in the sense that we need to >> provide iscsi boot target IP address and client's Initiator Node >> Name during the installation to discover the boot device. This is >> accomplished by using bootparams. The following name=val entry is >> required in /etc/bootparams on the install server. >> >> iscsi-boot=Target_IP:Initiator_Node_Name > >whoa! you need *both* DHCP *and* bootparams configured for each client? >Until now, you'd only ever need one or the other, and never both. Since >early stages boot from DHCP, this should be in DHCP too, not >bootparams... Not to mention that what works for DHCP (relaying through local agents) may not necessarily work for bootparams. Casper From casper@holland.sun.com Sat Aug 4 03:37:57 2007 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l74Abvt4006367 for ; Sat, 4 Aug 2007 03:37:57 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l74AZc8E008526; Sat, 4 Aug 2007 03:35:38 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM800N01W3E2T00@nwk-avmta-2.sfbay.sun.com>; Sat, 04 Aug 2007 03:35:38 -0700 (PDT) Received: from dm-holland-01.uk.sun.com ([129.156.101.192]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM8009MVW3D9F50@nwk-avmta-2.sfbay.sun.com>; Sat, 04 Aug 2007 03:35:38 -0700 (PDT) Received: from holland (room101.Holland.Sun.COM [129.159.130.93]) by dm-holland-01.uk.sun.com (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l74AZXDJ026093; Sat, 04 Aug 2007 11:35:33 +0100 (BST) Date: Sat, 04 Aug 2007 12:35:33 +0200 From: Casper.Dik@Sun.COM Subject: Re: iSCSI Software boot [PSARC/2007/450 FastTrack timeout 08/10/2007] In-reply-to: <46B3F8B1.1080904@sun.com> Sender: casper@holland.sun.com To: Sajid Zia Cc: Bill Sommerfeld , Mark Carlson , psarc-ext@Sun.COM, Kenneth.Davis@Sun.COM, david.miner@Sun.COM Message-id: <200708041035.l74AZXDJ026093@dm-holland-01.uk.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708032141.l73Lf8Fa026125@sac.sfbay.sun.com> <1186189151.1808.19.camel@localhost> <46B3F8B1.1080904@sun.com> Status: RO Content-Length: 2840 >A bootparams parameter is a better approach rather than inventing a new >dhcp macro or overloading an exisiting one to provide this information. >That is the feedback I had gotten from Solaris NewBoot folks. Please note >that we are not using any special dhcp >macro to provide client specific information. We use the standard >solaris net installation >process to do the boot as well. Only thing new is the bootparams parameter. I'm very much with Bill on this one: you cannot require DHCP and bootparams both; if anything, bootparams should die as it is a non-standard protocol with many shortcomings: - Bootparams doesn't work on all infrastructures - Bootparams is limited to a broadcast domain (no relaying has been defined for it); i.e., while your client gets the DHCP data from your server it may not be able to get the bootparams data from the same server. - Bootparam values need to be of the form hostname:value (which doesn't make sense for things like terminal types) - Bootparams requires rpcbind to be configured in "open" mode (broadcast RPC) Requiring bootparams in addition to DHCP is a deal breaker by itself. I'd expect to use DHCP and defined a new key (not macro) for this along the lines of: # # SunOS vendor space -- see the Solaris System Administrator # documentation for more information on these options. # SrootOpt VENDOR, 1, ASCII, 1, 0, smi SrootIP4 VENDOR, 2, IP, 1, 1, smi SrootNM VENDOR, 3, ASCII, 1, 0, smi SrootPTH VENDOR, 4, ASCII, 1, 0, smi SswapIP4 VENDOR, 5, IP, 1, 1, smi SswapPTH VENDOR, 6, ASCII, 1, 0, smi SbootFIL VENDOR, 7, ASCII, 1, 0, smi Stz VENDOR, 8, ASCII, 1, 0, smi SbootRS VENDOR, 9, UNUMBER16, 1, 1, smi SinstIP4 VENDOR, 10, IP, 1, 1, smi SinstNM VENDOR, 11, ASCII, 1, 0, smi SinstPTH VENDOR, 12, ASCII, 1, 0, smi SsysidCF VENDOR, 13, ASCII, 1, 0, smi SjumpsCF VENDOR, 14, ASCII, 1, 0, smi Sterm VENDOR, 15, ASCII, 1, 0, smi SbootURI VENDOR, 16, ASCII, 1, 0, smi SHTTPproxy VENDOR, 17, ASCII, 1, 0, smi I was under the impression bootparams was pretty much dead (except for legacy use) and I have not seen new bootparams defined for a while. Of course, if you also want to support bootparams boot then that is fine, but that should only be in addition to DHCP boot. And I don't think it's a requirement to support bootparams. Casper From David.Kahn@sun.com Sat Aug 4 14:42:08 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l74Lg7e7012322 for ; Sat, 4 Aug 2007 14:42:08 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l74Ldmsw005432 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Sun, 5 Aug 2007 05:39:49 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM900703QUBMK00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Sat, 04 Aug 2007 15:39:47 -0600 (MDT) Received: from sfbaymail2sca.sfbay.sun.com ([129.145.155.42]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM900I0TQUBYKC0@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Sat, 04 Aug 2007 15:39:47 -0600 (MDT) Received: from dtmail.sfbay.sun.com (pkg.SFBay.Sun.COM [129.146.90.56]) by sfbaymail2sca.sfbay.sun.com (8.13.6+Sun/8.12.10/ENSMAIL,v2.2) with ESMTP id l74LdkoK018559; Sat, 04 Aug 2007 14:39:46 -0700 (PDT) Received: from [127.0.0.1] (noho [10.6.92.101]) by dtmail.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l74Ldiin032723; Sat, 04 Aug 2007 14:39:45 -0700 (PDT) Date: Sat, 04 Aug 2007 14:39:44 -0700 From: David Kahn Subject: Re: iSCSI Software boot [PSARC/2007/450 FastTrack timeout 08/10/2007] In-reply-to: <46B3A671.5030001@sun.com> To: Mark Carlson Cc: PSARC-ext@sun.com, John Johnson Message-id: <46B4F220.5060906@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <46B3A671.5030001@sun.com> User-Agent: Thunderbird 2.0.0.6 (Windows/20070728) Status: RO Content-Length: 886 I cited the wrong case number in my email. The correct case number for newboot sparc is 2006/525. -David David Kahn wrote: > > > Mark Carlson wrote: > >> 5. Limitations -------------- >> >> This solution offers an innovative way of accomplishing iscsi boot >> on a general NIC. The first phase of the project has few limitations >> which will be addressed in subsequent phases. >> >> 1. It is supported only on x86/x64 platforms due to a dependency on >> Solairs NewBoot > > > See PSARC/2006/501 (sparc newboot support, fast track > timeout Aug 7, 2007) > > Given 2006/501, why would we want to allow this case > to be approved with the restriction I quoted above? > Am I missing something? > > Please remove the restriction from the materials, > including updating the section 4 example data flow which > is currently x86 specific. > > -David > > From sommerfeld@sun.com Sun Aug 5 14:11:56 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l75LBtln026166 for ; Sun, 5 Aug 2007 14:11:55 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l75L9VDY007982; Mon, 6 Aug 2007 05:09:31 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JMB00703K3UPH00@brm-avmta-1.central.sun.com>; Sun, 05 Aug 2007 15:09:30 -0600 (MDT) Received: from localhost.east.sun.com ([129.148.19.3]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JMB007HUK3SJN00@brm-avmta-1.central.sun.com>; Sun, 05 Aug 2007 15:09:29 -0600 (MDT) Received: from localhost.east.sun.com (localhost [127.0.0.1]) by localhost.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id l75L9Qvm002098; Sun, 05 Aug 2007 17:09:26 -0400 (EDT) Received: (from sommerfeld@localhost) by localhost.east.sun.com (8.14.1+Sun/8.14.1/Submit) id l75L9P8H002097; Sun, 05 Aug 2007 17:09:25 -0400 (EDT) Date: Sun, 05 Aug 2007 17:09:25 -0400 From: Bill Sommerfeld Subject: Re: iSCSI Software boot [PSARC/2007/450 FastTrack timeout 08/10/2007] In-reply-to: <46B3F8B1.1080904@sun.com> To: Sajid Zia Cc: Mark Carlson , psarc-ext@sun.com, Kenneth.Davis@sun.com, david.miner@sun.com Message-id: <1186348165.1984.2.camel@localhost> MIME-version: 1.0 X-Mailer: Evolution 2.10.2 Content-type: text/plain Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708032141.l73Lf8Fa026125@sac.sfbay.sun.com> <1186189151.1808.19.camel@localhost> <46B3F8B1.1080904@sun.com> X-Authentication-warning: localhost.east.sun.com: sommerfeld set sender to sommerfeld@sun.com using -f Status: RO Content-Length: 615 On Fri, 2007-08-03 at 20:55 -0700, Sajid Zia wrote: > > whoa! you need *both* DHCP *and* bootparams configured for each client? > > Until now, you'd only ever need one or the other, and never both. Since > > early stages boot from DHCP, this should be in DHCP too, not > > bootparams... > A bootparams parameter is a better approach rather than inventing a > new dhcp macro or overloading an exisiting one to provide this > information. That is the feedback I had gotten from Solaris NewBoot > folks. As far as I can tell, that feedback is incorrect; currently, dhcp boot doesn't use bootparams. - Bill From sommerfeld@sun.com Sun Aug 5 14:27:49 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l75LRmqP026249 for ; Sun, 5 Aug 2007 14:27:48 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l75LPRPg003243; Sun, 5 Aug 2007 22:25:27 +0100 (BST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JMB00801KUERI00@brm-avmta-1.central.sun.com>; Sun, 05 Aug 2007 15:25:26 -0600 (MDT) Received: from localhost.east.sun.com ([129.148.19.3]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JMB0077VKUDJN30@brm-avmta-1.central.sun.com>; Sun, 05 Aug 2007 15:25:25 -0600 (MDT) Received: from localhost.east.sun.com (localhost [127.0.0.1]) by localhost.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id l75LPM08002104; Sun, 05 Aug 2007 17:25:22 -0400 (EDT) Received: (from sommerfeld@localhost) by localhost.east.sun.com (8.14.1+Sun/8.14.1/Submit) id l75LPMYq002103; Sun, 05 Aug 2007 17:25:22 -0400 (EDT) Date: Sun, 05 Aug 2007 17:25:22 -0400 From: Bill Sommerfeld Subject: Re: [Fwd: Re: iSCSI Software boot [PSARC/2007/450 FastTrack timeout 08/10/2007]] In-reply-to: <46B3FCB8.2080701@sun.com> To: Sajid Zia Cc: psarc-ext@sun.com, Mark Carlson , Ken Davis Message-id: <1186349122.1984.19.camel@localhost> MIME-version: 1.0 X-Mailer: Evolution 2.10.2 Content-type: text/plain Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <46B3E8C9.2000208@sun.com> <46B3FCB8.2080701@sun.com> X-Authentication-warning: localhost.east.sun.com: sommerfeld set sender to sommerfeld@sun.com using -f Status: RO Content-Length: 1316 On Fri, 2007-08-03 at 21:12 -0700, Sajid Zia wrote: > I have a system booted up and running with all the networking > user/kernel code working fine. That just means you haven't found the deadlock scenarios in testing yet. User-space components of IP are mostly involved in failover and recovery and I'd expect things to appear to work most of the time, but working most of the time doesn't mean that iSCSI root is done. > All the code changes are confined to iscsi driver and there are no > interface/architectural changes. Have you tried the following? 1) IPMP enable ipmp on the client. test an ipmp failover. test it again while the system paging heavily and some of in.mpathd is swapped out to an iSCSI lun. 2) Put a router between client and server. (iSCSI is scsi over IP, not scsi-over-ethernet...) Can you recover from a "route flush" ? 2.A) Router discovery Configure a client to use router discovery to find its default route. put a router between client and server. boot the client. crash the router. bring up a different router. does the client fail over to the alternate path all the time when the system is under heavy memory pressure?) 3) VNI-based dynamic routing with OSPF. (note: I'm being nice. I'm not asking about IPsec/IKE between client and server). - Bill From sommerfeld@sun.com Sun Aug 5 14:57:02 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l75Lv1Is026660 for ; Sun, 5 Aug 2007 14:57:02 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail5.uk.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l75LsfqV011051; Sun, 5 Aug 2007 22:54:43 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JMB00805M759H00@nwk-avmta-2.sfbay.sun.com>; Sun, 05 Aug 2007 14:54:41 -0700 (PDT) Received: from localhost.east.sun.com ([129.148.19.3]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JMB003CAM74X410@nwk-avmta-2.sfbay.sun.com>; Sun, 05 Aug 2007 14:54:41 -0700 (PDT) Received: from localhost.east.sun.com (localhost [127.0.0.1]) by localhost.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id l75Lsbvj002886; Sun, 05 Aug 2007 17:54:37 -0400 (EDT) Received: (from sommerfeld@localhost) by localhost.east.sun.com (8.14.1+Sun/8.14.1/Submit) id l75LsbUx002885; Sun, 05 Aug 2007 17:54:37 -0400 (EDT) Date: Sun, 05 Aug 2007 17:54:37 -0400 From: Bill Sommerfeld Subject: Re: iSCSI Software boot [PSARC/2007/450 FastTrack timeout 08/10/2007] In-reply-to: <200708041007.l74A7u8H022759@dm-holland-01.uk.sun.com> To: Casper.Dik@sun.com Cc: Mark Carlson , psarc-ext@sun.com, Ken@sac.sfbay.sun.com, Davis@sac.sfbay.sun.com Message-id: <1186350877.1984.35.camel@localhost> MIME-version: 1.0 X-Mailer: Evolution 2.10.2 Content-type: text/plain Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708032141.l73Lf8Fa026125@sac.sfbay.sun.com> <1186188500.1808.13.camel@localhost> <200708041007.l74A7u8H022759@dm-holland-01.uk.sun.com> X-Authentication-warning: localhost.east.sun.com: sommerfeld set sender to sommerfeld@sun.com using -f Status: RO Content-Length: 734 On Sat, 2007-08-04 at 12:07 +0200, Casper.Dik@sun.com wrote: > Isn't there at least some form of regular testing of diskless boot > when using network based installs? It's not at all the same -- install is a very different animal from being able to run for months at a time with an NFS-diskless root; what's more, it's IMHO likely that failures during a jumpstart may go undiagnosed and unreported if a second or third jumpstart attempt works. see: 6575901 libc`sharefs() and ld.so have conspired to kill smdiskless smdiskless (the administrative tool used to administer diskless systems) has been broken in nevada for 4 months. Moreover, the ONNV C-team does not view this breakage as a release stopper for SXDE. - Bill From Darren.Moffat@sun.com Mon Aug 6 05:34:24 2007 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l76CYOjf008152 for ; Mon, 6 Aug 2007 05:34:24 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l76CW3d0000100 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Mon, 6 Aug 2007 05:32:06 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JMC00I05QTHB700@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@Sun.COM); Mon, 06 Aug 2007 06:32:06 -0600 (MDT) Received: from gmp-ea-fw-1.sun.com ([129.156.42.5]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JMC00BKNQTG7250@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@Sun.COM); Mon, 06 Aug 2007 06:32:05 -0600 (MDT) Received: from d1-emea-10.sun.com (d1-emea-10.sun.com [192.18.2.120]) by gmp-ea-fw-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l76CW3M1006111 for ; Mon, 06 Aug 2007 12:32:03 +0000 (GMT) Received: from conversion-daemon.d1-emea-10.sun.com by d1-emea-10.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JMC00601QOXM900@d1-emea-10.sun.com> (original mail from Darren.Moffat@Sun.COM) for psarc-ext@Sun.COM (ORCPT psarc-ext@Sun.COM); Mon, 06 Aug 2007 13:32:03 +0100 (BST) Received: from [129.156.173.21] by d1-emea-10.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JMC00DZBQTF1L00@d1-emea-10.sun.com>; Mon, 06 Aug 2007 13:32:03 +0100 (BST) Date: Mon, 06 Aug 2007 13:32:02 +0100 From: Darren J Moffat Subject: Re: iSCSI Software boot [PSARC/2007/450 FastTrack timeout 08/10/2007] In-reply-to: <200708032141.l73Lf8Fa026125@sac.sfbay.sun.com> Sender: Darren.Moffat@sun.com To: Mark Carlson Cc: psarc-ext@sun.com, Davis@sac.sfbay.sun.com, Ken@sac.sfbay.sun.com Message-id: <46B714C2.6040403@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708032141.l73Lf8Fa026125@sac.sfbay.sun.com> User-Agent: Thunderbird 2.0.0.0 (X11/20070605) Status: RO Content-Length: 1423 * Why x86 only when there is a SPARC newboot case in progress now, this alone is a possible reason to derail. for me. * SPARC already has wanboot that has a lot of similarities in the problem space - and is actually a secure solution. * Why bootparams rather than DHCP - mixing them is ugly especially on x86 which almost never uses bootparams these days. WAN Boot uses DHCP and it did have to add new options - some might be reusable by this case. * Mention of "rc" script - this must be an SMF service * CHAP authentication is not secure enough, you really need IPsec. However getting IPsec (even manually keyed) running early in boot will be challenging - partly because of the userland need and partly because of crypto not being available until kcfd is up (which is currently after /usr) If we have reason we can fix that though. * Why can't at least CHAP be done in the first phase ? Security isn't optional and many of the use cases for iSCSI boot would be similar to wanboot - so assuming physical security is not acceptable. Personally I don't feel as if this case is anywhere near complete architecturally since it seems clear to me that it hasn't considered existing things like WANboot. This case just doesn't see obvious to me and I would have expected some thing as significant as this to be fully coordinated with SPARC newboot and be a full case. -- Darren J Moffat From Dave.Miner@Sun.COM Mon Aug 6 08:07:24 2007 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l76F7OvS010901 for ; Mon, 6 Aug 2007 08:07:24 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l76F56HC018619; Mon, 6 Aug 2007 08:05:06 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JMC00N2XXWG2K00@nwk-avmta-1.sfbay.Sun.COM>; Mon, 06 Aug 2007 08:05:04 -0700 (PDT) Received: from brmea-mail-3.sun.com ([192.18.98.34]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JMC00B7NXWEQP90@nwk-avmta-1.sfbay.Sun.COM>; Mon, 06 Aug 2007 08:05:02 -0700 (PDT) Received: from fe-amer-06.sun.com ([192.18.108.180]) by brmea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l76F52fp023846; Mon, 06 Aug 2007 15:05:02 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JMC00M01XQIJV00@mail-amer.sun.com> (original mail from Dave.Miner@Sun.COM); Mon, 06 Aug 2007 09:05:02 -0600 (MDT) Received: from [129.148.174.151] by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JMC00F3IXWDVX53@mail-amer.sun.com>; Mon, 06 Aug 2007 09:05:02 -0600 (MDT) Date: Mon, 06 Aug 2007 11:05:01 -0400 From: Dave Miner Subject: Re: iSCSI Software boot [PSARC/2007/450 FastTrack timeout 08/10/2007] In-reply-to: <46B3F8B1.1080904@sun.com> Sender: Dave.Miner@Sun.COM To: Sajid Zia Cc: Bill Sommerfeld , Mark Carlson , psarc-ext@Sun.COM, Kenneth.Davis@Sun.COM, david.miner@Sun.COM Message-id: <46B7389D.1020603@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708032141.l73Lf8Fa026125@sac.sfbay.sun.com> <1186189151.1808.19.camel@localhost> <46B3F8B1.1080904@sun.com> User-Agent: Thunderbird 2.0.0.4 (X11/20070703) Status: RO Content-Length: 1789 Sajid Zia wrote: > Bill Sommerfeld wrote: >> On Fri, 2007-08-03 at 14:41 -0700, Mark Carlson wrote: >> >> >>> The installation process changed a bit in the sense that we need to >>> provide iscsi boot target IP address and client's Initiator Node >>> Name during the installation to discover the boot device. This is >>> accomplished by using bootparams. The following name=val entry is >>> required in /etc/bootparams on the install server. >>> >>> iscsi-boot=Target_IP:Initiator_Node_Name >>> >> whoa! you need *both* DHCP *and* bootparams configured for each client? >> Until now, you'd only ever need one or the other, and never both. Since >> early stages boot from DHCP, this should be in DHCP too, not >> bootparams... >> > A bootparams parameter is a better approach rather than inventing a new > dhcp macro > or overloading an exisiting one to provide this information. That is the > feedback I had > gotten from Solaris NewBoot folks. Please note that we are not using any > special dhcp > macro to provide client specific information. We use the standard > solaris net installation > process to do the boot as well. Only thing new is the bootparams parameter. > I must respectfully, but vehemently, disagree with this advice (unfortunately, the Install team has not been consulted on this proposal prior to its presentation, so this is my first opportunity to comment). We have been moving network installation away from bootparams for a number of years due to its non-standard nature and lack of a reasonable ability to support any network topology beyond simple LANs. Requiring its use here is not defensible when ample methods for solving the boot-device location problem exist using DHCP or other standard protocols. Dave From sommerfeld@sun.com Mon Aug 6 08:20:48 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l76FKlI5011132 for ; Mon, 6 Aug 2007 08:20:48 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail5.uk.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l76FHnvh000471; Mon, 6 Aug 2007 16:18:27 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JMC0010BYIQ2T00@nwk-avmta-1.sfbay.Sun.COM>; Mon, 06 Aug 2007 08:18:26 -0700 (PDT) Received: from eastmail2bur.East.Sun.COM ([129.148.13.40]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JMC00BYNYIPQP90@nwk-avmta-1.sfbay.Sun.COM>; Mon, 06 Aug 2007 08:18:25 -0700 (PDT) Received: from thunk.east.sun.com (thunk.East.Sun.COM [129.148.174.66]) by eastmail2bur.East.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l76FIMLj026520; Mon, 06 Aug 2007 11:18:22 -0400 (EDT) Received: from [IPv6:::1] (localhost [IPv6:::1]) by thunk.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id l76FILKK014323; Mon, 06 Aug 2007 11:18:22 -0400 (EDT) Date: Mon, 06 Aug 2007 11:18:21 -0400 From: Bill Sommerfeld Subject: Re: iSCSI Software boot [PSARC/2007/450 FastTrack timeout 08/10/2007] In-reply-to: <46B7389D.1020603@sun.com> To: Dave Miner Cc: Sajid Zia , Mark Carlson , psarc-ext@sun.com, Kenneth.Davis@sun.com, david.miner@sun.com Message-id: <1186413501.14296.3.camel@thunk> MIME-version: 1.0 X-Mailer: Evolution 2.10.2 Content-type: text/plain Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708032141.l73Lf8Fa026125@sac.sfbay.sun.com> <1186189151.1808.19.camel@localhost> <46B3F8B1.1080904@sun.com> <46B7389D.1020603@sun.com> Status: RO Content-Length: 635 On Mon, 2007-08-06 at 11:05 -0400, Dave Miner wrote: > We have been moving network installation away from bootparams for a > number of years due to its non-standard nature and lack of a reasonable > ability to support any network topology beyond simple LANs. Requiring > its use here is not defensible when ample methods for solving the > boot-device location problem exist using DHCP or other standard protocols. Sounds like we could reduce the likelihood of missteps like this if we were to approve a case (not this case) formally reclassifying bootparams as Obsolete, with DHCP as the designated successor. - Bill From Dave.Miner@sun.com Mon Aug 6 08:23:30 2007 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l76FNUR5011171 for ; Mon, 6 Aug 2007 08:23:30 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l76FLBvw003599 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Mon, 6 Aug 2007 08:21:11 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JMC00C2TYNB0200@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Mon, 06 Aug 2007 08:21:11 -0700 (PDT) Received: from brmea-mail-2.sun.com ([192.18.98.43]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JMC008JIYN88650@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Mon, 06 Aug 2007 08:21:08 -0700 (PDT) Received: from fe-amer-01.sun.com ([192.18.108.175]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l76FL7AB005458 for ; Mon, 06 Aug 2007 15:21:07 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JMC00E01YHYX400@mail-amer.sun.com> (original mail from Dave.Miner@Sun.COM) for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Mon, 06 Aug 2007 09:21:07 -0600 (MDT) Received: from [129.148.174.151] by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JMC00FJFYN6C5E6@mail-amer.sun.com>; Mon, 06 Aug 2007 09:21:07 -0600 (MDT) Date: Mon, 06 Aug 2007 11:21:06 -0400 From: Dave Miner Subject: Re: iSCSI Software boot [PSARC/2007/450 FastTrack timeout 08/10/2007] In-reply-to: <46B714C2.6040403@Sun.COM> Sender: Dave.Miner@sun.com To: Darren J Moffat Cc: Mark Carlson , psarc-ext@sun.com, Davis@sac.sfbay.sun.com, Ken@sac.sfbay.sun.com Message-id: <46B73C62.4010501@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708032141.l73Lf8Fa026125@sac.sfbay.sun.com> <46B714C2.6040403@Sun.COM> User-Agent: Thunderbird 2.0.0.4 (X11/20070703) Status: RO Content-Length: 1166 Darren J Moffat wrote: ... > > Personally I don't feel as if this case is anywhere near complete > architecturally since it seems clear to me that it hasn't considered > existing things like WANboot. > I am very much in support of providing iSCSI boot support, but I concur with Darren here. The case makes a number of references to solving limitations in later phases, but I'm unaware of any of the plans for those later phases; there has been no coordination with my team in regard to how the various network installation options will be rationalized and managed. Our experience with WANboot/install should make us very leery of a first phase that relies on a great deal of manual labor on the administrator's part (and I note that the materials here are fairly vague on what those steps are) to set up complex network boot environments; the implied lack of Jumpstart support is especially troubling to me, since the profiles for iSCSI customers and Jumpstart customers seems to me to match nearly perfectly. It's 5 years on since the WANboot cases, after all, and we're still some time away from working on resolving the problems there. Dave From Dave.Miner@sun.com Mon Aug 6 08:24:59 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l76FOw9n011222 for ; Mon, 6 Aug 2007 08:24:59 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l76FMc4q028918; Mon, 6 Aug 2007 23:22:39 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JMC0042DYPPUI00@brm-avmta-1.central.sun.com>; Mon, 06 Aug 2007 09:22:37 -0600 (MDT) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JMC00BFAYPO6QE0@brm-avmta-1.central.sun.com>; Mon, 06 Aug 2007 09:22:36 -0600 (MDT) Received: from fe-amer-04.sun.com ([192.18.108.178]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l76FMaN2011652; Mon, 06 Aug 2007 15:22:36 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JMC00K01YI3ZV00@mail-amer.sun.com> (original mail from Dave.Miner@Sun.COM); Mon, 06 Aug 2007 09:22:36 -0600 (MDT) Received: from [129.148.174.151] by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JMC00CXTYPLGTF6@mail-amer.sun.com>; Mon, 06 Aug 2007 09:22:34 -0600 (MDT) Date: Mon, 06 Aug 2007 11:22:33 -0400 From: Dave Miner Subject: Re: iSCSI Software boot [PSARC/2007/450 FastTrack timeout 08/10/2007] In-reply-to: <1186413501.14296.3.camel@thunk> Sender: Dave.Miner@sun.com To: Bill Sommerfeld Cc: Sajid Zia , Mark Carlson , psarc-ext@sun.com, Kenneth.Davis@sun.com, david.miner@sun.com Message-id: <46B73CB9.709@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708032141.l73Lf8Fa026125@sac.sfbay.sun.com> <1186189151.1808.19.camel@localhost> <46B3F8B1.1080904@sun.com> <46B7389D.1020603@sun.com> <1186413501.14296.3.camel@thunk> User-Agent: Thunderbird 2.0.0.4 (X11/20070703) Status: RO Content-Length: 811 Bill Sommerfeld wrote: > On Mon, 2007-08-06 at 11:05 -0400, Dave Miner wrote: >> We have been moving network installation away from bootparams for a >> number of years due to its non-standard nature and lack of a reasonable >> ability to support any network topology beyond simple LANs. Requiring >> its use here is not defensible when ample methods for solving the >> boot-device location problem exist using DHCP or other standard protocols. > > Sounds like we could reduce the likelihood of missteps like this if we > were to approve a case (not this case) formally reclassifying bootparams > as Obsolete, with DHCP as the designated successor. > I'd figured that would be something I'd include when we came with the eventual update to Jumpstart, but yeah, perhaps we should just do it now. Dave From Darren.Moffat@sun.com Mon Aug 6 08:42:40 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l76FgdVl011554 for ; Mon, 6 Aug 2007 08:42:40 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l76FeFE8006830 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Mon, 6 Aug 2007 23:40:20 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JMC00203ZJ7ZS00@nwk-avmta-1.sfbay.Sun.COM> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Mon, 06 Aug 2007 08:40:19 -0700 (PDT) Received: from gmp-ea-fw-1.sun.com ([129.156.42.6]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JMC00BSAZJ6QWC0@nwk-avmta-1.sfbay.Sun.COM> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Mon, 06 Aug 2007 08:40:19 -0700 (PDT) Received: from d1-emea-10.sun.com ([192.18.2.120]) by gmp-ea-fw-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l76FeHup016051 for ; Mon, 06 Aug 2007 15:40:17 +0000 (GMT) Received: from conversion-daemon.d1-emea-10.sun.com by d1-emea-10.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JMC00C01ZF2KG00@d1-emea-10.sun.com> (original mail from Darren.Moffat@Sun.COM) for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Mon, 06 Aug 2007 16:40:17 +0100 (BST) Received: from [129.156.173.21] by d1-emea-10.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JMC0064OZJ56R00@d1-emea-10.sun.com>; Mon, 06 Aug 2007 16:40:17 +0100 (BST) Date: Mon, 06 Aug 2007 16:40:17 +0100 From: Darren J Moffat Subject: DERAIL Re: iSCSI Software boot [PSARC/2007/450 FastTrack timeout 08/10/2007] In-reply-to: <46B73C62.4010501@sun.com> Sender: Darren.Moffat@sun.com To: Mark Carlson , psarc-ext@sun.com, ken.davis@sun.com Cc: Dave Miner Message-id: <46B740E1.3010604@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708032141.l73Lf8Fa026125@sac.sfbay.sun.com> <46B714C2.6040403@Sun.COM> <46B73C62.4010501@sun.com> User-Agent: Thunderbird 2.0.0.0 (X11/20070605) Status: RO Content-Length: 903 Given the comments from several people, myself included, I do not believe this case meets fast-track criteria and is now derailed. The main critera for derailing was: * Apparent lack of coordination with Install * Not addressing SPARC * Overlap with existing shipping WANboot solution * Even basic security issues not addressed (CHAP). * Assumption of physical security conflicts experience with WANboot * Indication that this is multi phase - that alone often implies a full case or umbrella case to outline the phases of delivery. Project team please schedule a full open inception review. I'm happy to be the case sponsor for this but would like a PSARC intern assigned as well. NOTE of process order. Given that this case is now derailed as a fast-track there should be no need for further email discussion until the project team presents materials for full review. -- Darren J Moffat From Mark.Carlson@sun.com Mon Aug 6 08:45:22 2007 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l76FjMjP011605 for ; Mon, 6 Aug 2007 08:45:22 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l76Fh32T012487 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Mon, 6 Aug 2007 08:43:03 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JMC00C0DZNRP800@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Mon, 06 Aug 2007 08:43:03 -0700 (PDT) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JMC008NFZNQ8860@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Mon, 06 Aug 2007 08:43:02 -0700 (PDT) Received: from fe-amer-03.sun.com ([192.18.108.177]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l76Fh2K6020904 for ; Mon, 06 Aug 2007 15:43:02 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JMC00301ZC1ZA00@mail-amer.sun.com> (original mail from Mark.Carlson@Sun.COM) for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Mon, 06 Aug 2007 09:43:02 -0600 (MDT) Received: from MACsMAC.local ([129.150.34.204]) by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JMC0071DZNPNEN0@mail-amer.sun.com>; Mon, 06 Aug 2007 09:43:02 -0600 (MDT) Date: Mon, 06 Aug 2007 09:43:09 -0600 From: "Mark A. Carlson" Subject: Re: DERAIL Re: iSCSI Software boot [PSARC/2007/450 FastTrack timeout 08/10/2007] In-reply-to: <46B740E1.3010604@Sun.COM> Sender: Mark.Carlson@sun.com To: Darren J Moffat Cc: Mark Carlson , psarc-ext@sun.com, ken.davis@sun.com, Dave Miner Message-id: <46B7418D.5020607@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708032141.l73Lf8Fa026125@sac.sfbay.sun.com> <46B714C2.6040403@Sun.COM> <46B73C62.4010501@sun.com> <46B740E1.3010604@Sun.COM> User-Agent: Thunderbird 2.0.0.6 (Macintosh/20070728) Status: RO Content-Length: 1079 Sounds good. I will work with the project team to submit a full case that will address these issues. -- mark Darren J Moffat wrote: > Given the comments from several people, myself included, I do not > believe this case meets fast-track criteria and is now derailed. > > The main critera for derailing was: > > * Apparent lack of coordination with Install > * Not addressing SPARC > * Overlap with existing shipping WANboot solution > * Even basic security issues not addressed (CHAP). > * Assumption of physical security conflicts experience with WANboot > * Indication that this is multi phase - that alone often implies > a full case or umbrella case to outline the phases of delivery. > > Project team please schedule a full open inception review. I'm happy > to be the case sponsor for this but would like a PSARC intern assigned > as well. > > NOTE of process order. Given that this case is now derailed as a > fast-track there should be no need for further email discussion until > the project team presents materials for full review. > > -- > Darren J Moffat From dmk@noho.sfbay.sun.com Mon Nov 26 19:46:35 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lAR3kYWZ026451 for ; Mon, 26 Nov 2007 19:46:34 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id lAR3kHjW007559; Tue, 27 Nov 2007 03:46:29 GMT Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JS500H03BTFXO00@nwk-avmta-1.sfbay.Sun.COM>; Mon, 26 Nov 2007 19:46:27 -0800 (PST) Received: from noho.sfbay.sun.com ([10.6.92.101]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JS5009VUBTFEJ20@nwk-avmta-1.sfbay.Sun.COM>; Mon, 26 Nov 2007 19:46:27 -0800 (PST) Received: from noho.sfbay.sun.com (localhost [127.0.0.1]) by noho.sfbay.sun.com (8.13.7+Sun/8.13.7) with ESMTP id lAR3jY2q002410; Mon, 26 Nov 2007 19:45:34 -0800 (PST) Received: (from dmk@localhost) by noho.sfbay.sun.com (8.13.7+Sun/8.13.7/Submit) id lAR3jX3r002409; Mon, 26 Nov 2007 19:45:33 -0800 (PST) Date: Mon, 26 Nov 2007 19:45:33 -0800 (PST) From: David Kahn Subject: Re: iSCSI Software boot [PSARC/2007/450 FastTrack timeout 08/10/2007] To: Casper.Dik@Sun.COM, Sajid.Zia@Sun.COM Cc: Kenneth.Davis@Sun.COM, david.miner@Sun.COM, markcarl@sac.sfbay.sun.com, psarc-ext@Sun.COM, sommerfeld@Sun.COM Message-id: <200711270345.lAR3jX3r002409@noho.sfbay.sun.com> Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 1684 >I'd expect to use DHCP and defined a new key (not macro) for this along >the lines of: ... Sorry to dredge this up again, but there is an RFC for iscsi boot. RFC 4173. It describes a form of the root-path dhcp option for iscsi boot: A BOOTP or DHCP server (v4 or v6) MAY instruct an iSCSI client how to reach its boot device. This is done using the variable-length option named Root Path [Alexander93, Reynolds93]. The use of the option field is reserved for iSCSI boot use by prefacing the string with "iscsi:". The Root Path option is not currently defined for DHCPv6; if the option is defined for DHCPv6 in the future, the use of the option as defined for iSCSI boot will apply. The option field consists of an UTF-8 [Yergeau98] string. The string has the following composition: "iscsi:"":"":"":"":" Now, clearly that only works if the device firmware supports iscsi, because i believe we probably use root-path to define where the boot program (newboot) lives on the server (is that correct?) So we would need a secondary key in dhcp that describes the iscsi boot device path and options if the first level boot is to boot the ramdisk (newboot) from the network and add the iscsi and security stuff to that. It would be a good thing if we worked to standardize that second dhcp key, so it isn't Sun-specific. All vendors have the same issue for supporting iscsi boot on existing network devices that don't support iscsi boot natively in the firmware. We could look for the standard secondary key name and then the sun-specific key name until the secondary key name is standardized. -David From Lenin.Salvador@sun.com Wed Jan 9 19:31:50 2008 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m0A3VmAc023634 for ; Wed, 9 Jan 2008 19:31:49 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id m0A3Vbow001005; Thu, 10 Jan 2008 11:31:46 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JUE00205SGXY900@nwk-avmta-2.sfbay.sun.com>; Wed, 09 Jan 2008 19:31:45 -0800 (PST) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JUE0084PSGWZCF0@nwk-avmta-2.sfbay.sun.com>; Wed, 09 Jan 2008 19:31:44 -0800 (PST) Received: from fe-amer-09.sun.com ([192.18.109.79]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id m0A3ViiV025275; Thu, 10 Jan 2008 03:31:44 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JUE00201SE7GE00@mail-amer.sun.com> (original mail from Lenin.Salvador@Sun.COM); Wed, 09 Jan 2008 20:31:44 -0700 (MST) Received: from [129.150.64.230] by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JUE00KVLSGT57A0@mail-amer.sun.com>; Wed, 09 Jan 2008 20:31:43 -0700 (MST) Date: Wed, 09 Jan 2008 22:31:39 -0500 From: "Lenin Salvador " Subject: INQUIRY: iSCSI Boot Support ---- iSCSI Software boot [PSARC/2007/450 FastTrack timeout 08/10/2007] Sender: Lenin.Salvador@sun.com To: psarc-ext@sun.com, boot-interest@sun.com, amb.os@sun.com Cc: "Lenin Salvador " Reply-to: Lenin.Salvador@sun.com Message-id: <4785919B.7020600@Sun.COM> Organization: Sun Microsystems, Inc. MIME-version: 1.0 Content-type: multipart/alternative; boundary="Boundary_(ID_Ed5CO31CVzblGWwcrP/tMQ)" X-PMX-Version: 5.2.0.264296 User-Agent: Thunderbird 2.0.0.9 (Windows/20071031) Status: RO Content-Length: 22385 This is a multi-part message in MIME format. --Boundary_(ID_Ed5CO31CVzblGWwcrP/tMQ) Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT Hi iSCSI boot folks, Sorry for the wide distribution. I wasn't sure if I'm hitting the right folks for this question of mine either. My account Lehman Brothers is asking the following: [1] Is booting over iscsi supported yet in Solaris 10? Any supporting documentation on this? [2] Does BIOS on x86 Sun boxes support iSCSI boot? Any help/pointers highly appreciated. Thanks and kindest regards, -Lenin Salvador- Lead System Support Engineer Sun Services - Americas Lehman Brothers Account RE: Template Version: @(#)sac_nextcase 1.64 07/13/07 SMI This information is Copyright 2007 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: iSCSI Software boot 1.2. Name of Document Author/Supplier: Author: Sajid Zia 1.3 Date of This Document: 03 August, 2007 4. Technical Description 1. Background: -------------- PSARC 2003/126 introduced the Solaris iSCSI Initiator to Solaris. Solaris Initiator allows access to scsi storage using Sun's TCP/IP stack through a general NIC. The initiator provided a software only solution with no boot support. A traditional boot requires firmware support on adapters which a general NIC does not provide. 2. Problem Statement: --------------------- Storage Area Networks (SANs) have been rapidly growing and gaining popularity over the traditional client-server model due to their ability to efficiently handle data. Typically, Storage Area Networks require Fibre Channel, however, iSCSI for SANs, which encapsulates standard SCSI over existing IP networks, offers a lower cost alternative to expensive Fibre Channel. Fibre Channel SANs support booting from an FC device over a SAN thereby eliminating the need for a local hard disk on a system. Boot is considered a basic OS functionality, and currently no good iSCSI boot solution exists on the market due to the fact that a general NIC does not provide any iSCSI support in its BIOS. 3. Proposal: ------------ PSARC 2004/454 introduced the Solaris New Boot Architecture which greatly simplified a boot process on x86 platforms. iSCSI boot proposal leverages Solaris New Boot to offer a solution which requires no dependency on iSCSI support in bios. This basic idea here is to combine net and disk boot process. It uses pxegrub to download boot archives from the install server in the early stages of the boot and then loads iscsi initiator in client's RAM to access the boot device using regular block disk IOs. This proposal is offering a new iscsi boot functionality only, and does not impact in any way the general solaris install/boot process for non iscsi root devices. 4. Proposal Details: -------------------- When a client system powers up or rebooted, these are the steps which are followed for booting. 1. BIOS running on the machine makes a DHCP request. 2. DHCP server responds with boot server and boot file, i.e., pxegrub 3. BIOS downloads pxegrub using tftp 4. pxegrub displays the grub menu supplied by the boot server 5. pxegrub then gets the boot-archive (mini-kernel) from the boot server and executes the mini kernel. 6. mini-kernel plumbs the network interface and enumerates the iSCSI boot device. 7. root disk is seen through regular NIC port 8. rest of the OS image comes from the root disk using regular disk IOs The steps 1-5 are exactly the same as followed by a normal Solaris Installation process. The only difference is that the boot-archive in step 5 comes from the root disk and not an Installation boot-archive Installation ------------ The installation process changed a bit in the sense that we need to provide iscsi boot target IP address and client's Initiator Node Name during the installation to discover the boot device. This is accomplished by using bootparams. The following name=val entry is required in /etc/bootparams on the install server. iscsi-boot=Target_IP:Initiator_Node_Name An example: iscsi-1 iscsi-boot=192.29.74.155:iqn.1986-03.com.sun:01:iscsi-1 Once the installation is complete, the following extra steps an administrator needs to do to prepare the client for booting: 1. newly created boot archives need to be copied to the Install server for subsequent booting 2. menu list on the server need to reference the new boot archives for booting. The code changes for the project are mostly confined to iscsi driver and require no interface/architectural changes. These are some additional changes. 1. During the installation, an rc script is needed to discover and enumerate the boot lun. 2. filelist.ramdisk file needs to be updated with iscsi persistent database files which are needed to boot 3. SUNWsibi pkgmap needs to include iscsi database file entry. 4. A minor sockfs change is required, addressed by the CR 6422549 5. A minor update is needed in smf_include.sh script. 5. Limitations -------------- This solution offers an innovative way of accomplishing iscsi boot on a general NIC. The first phase of the project has few limitations which will be addressed in subsequent phases. 1. It is supported only on x86/x64 platforms due to a dependency on Solairs NewBoot 2. The customers will be encouraged to use the first phase of this solution over physically secured networks. The next phase of the project will add CHAP authentication. 3. Currently require targets to have TPGS disabled so that iqn based names appear in the root device name. The next phase will add multi-pathing support along with IPMP. 4. Only manual network installation is supported. 5. After the installation, the boot archives must be moved to the installed server to be used for booting. This needs to be done by the administrator. 6. Install server is required at the time of boot/reboots 7. The boot archives on the root disk and on the installed server must be in sync. This needs to be ensured by the administrator. 8. dump/sync is only supported on a local partition due to a networking bug 6205961, which will be addressed in the next phase. 6. Benefits ----------- Despite these limitations, this solution offers many benefits which should outweigh the limitations for phase I release. 1. Offers an innovative solution to combine net/disk boot 2. No hardware dependencies 3. Works on any modern general NIC which has pxe support. 4. Superior to "diskless boot" which requires boot server to be available all the time to service boot clients. 5. A generic Solaris iscsi boot solution will offer a huge marketing advantage over our competitors. Currently there is no good generic iSCSI boot solution available in the market. 8. References: -------------- 1. PSARC 2003/126 iSCSI Project 2. PSARC 2004/454 Solaris Boot Architecture 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: FastTrack 6.6. ARC Exposure: open -- * Lenin Mercado Salvador * System Support Engineer *Sun Microsystems, Inc.* 400 Atrium Drive, Suite 2 Somerset, NJ 08873 U.S.A. Mobile 917-560-4793 Email Lenin.Salvador@Sun.COM --Boundary_(ID_Ed5CO31CVzblGWwcrP/tMQ) Content-type: text/html; charset=ISO-8859-1 Content-transfer-encoding: 7BIT
Hi iSCSI boot folks,

Sorry for the wide distribution. I wasn't sure if I'm hitting the right folks for
this question of mine either.

My account Lehman Brothers is asking the following:

[1] Is booting over iscsi supported yet in Solaris 10? Any supporting documentation on this?
[2] Does BIOS on x86 Sun boxes support iSCSI boot?

Any help/pointers highly appreciated.

Thanks and kindest regards,

-Lenin Salvador-
Lead System Support Engineer
Sun Services - Americas
Lehman Brothers Account


RE:

Template Version: @(#)sac_nextcase 1.64 07/13/07 SMI
This information is Copyright 2007 Sun Microsystems
1. Introduction
    1.1. Project/Component Working Name:
         iSCSI Software boot
    1.2. Name of Document Author/Supplier:
         Author:  Sajid Zia
    1.3  Date of This Document:
        03 August, 2007
4. Technical Description

1. Background:
--------------

        PSARC 2003/126 introduced the Solaris iSCSI Initiator to Solaris.
        Solaris Initiator allows access to scsi storage using Sun's
        TCP/IP stack through a general NIC. The initiator provided a
        software only solution with no boot support. A traditional
        boot requires firmware support on adapters which a general
        NIC does not provide.

2. Problem Statement:
---------------------

        Storage Area Networks (SANs) have been rapidly growing and
        gaining popularity over the traditional client-server model due
        to their ability to efficiently handle data. Typically, Storage
        Area Networks require Fibre Channel, however, iSCSI for SANs,
        which encapsulates standard SCSI over existing IP networks,
        offers a lower cost alternative to expensive Fibre Channel.

        Fibre Channel SANs support booting from an FC device over
        a SAN thereby eliminating the need for a local hard disk on
        a system. Boot is considered a basic OS functionality, and
        currently no good iSCSI boot solution exists on the market
        due to the fact that a general NIC does not provide any iSCSI
        support in its BIOS.


3. Proposal:
------------
        PSARC 2004/454 introduced the Solaris New Boot Architecture which
        greatly simplified a boot process on x86 platforms. iSCSI boot
        proposal leverages Solaris New Boot to offer a solution which
        requires no dependency on iSCSI support in bios. This basic idea
        here is to combine net and disk boot process. It uses pxegrub
        to download boot archives from the install server in the early
        stages of the boot and then loads iscsi initiator in client's
        RAM to access the boot device using regular block disk IOs.

        This proposal is offering a new iscsi boot functionality only,
        and does not impact in any way the general solaris install/boot
        process for non iscsi root devices.


4. Proposal Details:
--------------------

        When a client system powers up or rebooted,  these are the steps
        which are followed for booting.

        1. BIOS running on the machine makes a DHCP request.
        2. DHCP server responds with boot server and boot file, i.e., pxegrub
        3. BIOS downloads pxegrub using tftp
        4. pxegrub displays the grub menu supplied by the boot server
        5. pxegrub then gets the boot-archive (mini-kernel) from the boot server
           and executes the mini kernel.
        6. mini-kernel plumbs the network interface and enumerates the iSCSI
           boot device.
        7. root disk is seen through regular NIC port
        8. rest of the OS image comes from the root disk using regular disk IOs

        The steps 1-5 are exactly the same as followed by a normal Solaris
        Installation process. The only difference is that the boot-archive in
        step 5 comes from the root disk and not an Installation boot-archive

        Installation
        ------------

        The installation process changed a bit in the sense that we need to
        provide iscsi boot target IP address and client's Initiator Node
        Name during the installation to discover the boot device. This is
        accomplished by using bootparams. The following name=val entry is
        required in /etc/bootparams on the install server.

        <Client's Name>  iscsi-boot=Target_IP:Initiator_Node_Name

        An example:
        iscsi-1 iscsi-boot=192.29.74.155:iqn.1986-03.com.sun:01:iscsi-1

        Once the installation is complete, the following extra steps an
        administrator needs to do to prepare the client for booting:

        1. newly created boot archives need to be copied to the Install
           server for subsequent booting

        2. menu list on the server need to reference the new boot archives
           for booting.

        The code changes for the project are mostly confined to iscsi
        driver and require no interface/architectural changes. These are
        some additional changes.

        1. During the installation, an rc script is needed to discover and
           enumerate the boot lun.

        2. filelist.ramdisk file needs to be updated with iscsi persistent
           database files which are needed to boot

        3. SUNWsibi pkgmap needs to include iscsi database file entry.

        4. A minor sockfs change is required, addressed by the CR 6422549

        5. A minor update is needed in smf_include.sh script.
   

5. Limitations
--------------

        This solution offers an innovative way of accomplishing iscsi boot
        on a general NIC. The first phase of the project has few limitations
        which will be addressed in subsequent phases.

        1. It is supported only on x86/x64 platforms due to a dependency on
           Solairs NewBoot

        2. The customers will be encouraged to use the first phase of this
           solution over physically secured networks. The next phase of the
           project will add CHAP authentication.

        3. Currently require targets to have TPGS disabled so that iqn based
           names appear in the root device name. The next phase will add
           multi-pathing support along with IPMP.

        4. Only manual network installation is supported.

        5. After the installation, the boot archives must be moved to the
           installed server to be used for booting. This needs to be done
           by the administrator.

        6. Install server is required at the time of boot/reboots

        7. The boot archives on the root disk and on the installed server
           must be in sync. This needs to be ensured by the administrator.
       
        8. dump/sync is only supported on a local partition due to a
           networking bug 6205961, which will be addressed in the next
           phase.


6. Benefits
-----------

        Despite these limitations, this solution offers many benefits
        which should outweigh the limitations for phase I release.

        1. Offers an innovative solution to combine net/disk boot

        2. No hardware dependencies

        3. Works on any modern general NIC which has pxe support.

        4. Superior to "diskless boot" which requires boot server to be
           available all the time to service boot clients.

        5. A generic Solaris iscsi boot solution will offer a huge
           marketing advantage over our competitors. Currently there is
           no good generic iSCSI boot solution available in the market.

8. References:
--------------

        1. PSARC 2003/126 iSCSI Project

        2. PSARC 2004/454 Solaris Boot Architecture

6. Resources and Schedule
    6.4. Steering Committee requested information
        6.4.1. Consolidation C-team Name:
                ON
    6.5. ARC review type: FastTrack
    6.6. ARC Exposure: open

--
Lenin Mercado Salvador
System Support Engineer

Sun Microsystems, Inc.
400 Atrium Drive, Suite 2
Somerset, NJ 08873 U.S.A.
Mobile 917-560-4793
Email Lenin.Salvador@Sun.COM
--Boundary_(ID_Ed5CO31CVzblGWwcrP/tMQ)-- From carlsonj@phorcys.east.sun.com Thu Jan 10 04:39:59 2008 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m0ACdxwU002215 for ; Thu, 10 Jan 2008 04:39:59 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m0ACdwUZ003721 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Thu, 10 Jan 2008 04:39:58 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JUF00E05HULWA00@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 10 Jan 2008 05:39:57 -0700 (MST) Received: from phorcys.east.sun.com ([129.148.174.143]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JUF0055HHUKFRA0@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 10 Jan 2008 05:39:57 -0700 (MST) Received: from phorcys.east.sun.com (localhost [127.0.0.1]) by phorcys.east.sun.com (8.14.2+Sun/8.14.2) with ESMTP id m0ACcaF4009916 for ; Thu, 10 Jan 2008 07:38:36 -0500 (EST) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.2+Sun/8.14.2/Submit) id m0ACcadV009913; Thu, 10 Jan 2008 07:38:36 -0500 (EST) Date: Thu, 10 Jan 2008 07:38:36 -0500 From: James Carlson Subject: Re: INQUIRY: iSCSI Boot Support ---- iSCSI Software boot [PSARC/2007/450 FastTrack timeout 08/10/2007] In-reply-to: <4785919B.7020600@Sun.COM> To: psarc-ext@sun.com Message-id: <18310.4556.442509.277792@gargle.gargle.HOWL> MIME-version: 1.0 X-Mailer: VM 7.01 under Emacs 21.3.1 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <4785919B.7020600@Sun.COM> Status: RO Content-Length: 311 The original poster has been notified of his mistake; no need for others to follow up. -- James Carlson, Solaris Networking Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677