From markcarl@sac.sfbay.sun.com Thu Sep 10 14:01:24 2009
Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n8AL1Nfl029321
	for <psarc-ext@sac.sfbay.sun.com>; Thu, 10 Sep 2009 14:01:23 -0700 (PDT)
Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11])
	by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n8AL1L9q009903;
	Thu, 10 Sep 2009 22:01:22 +0100 (BST)
Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by
 brm-avmta-1.central.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KPR0071BX2A3F00@brm-avmta-1.central.sun.com>; Thu,
 10 Sep 2009 15:01:22 -0600 (MDT)
Received: from dm-sfbay-01.sfbay.sun.com ([129.145.155.118])
 by brm-avmta-1.central.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KPR00KYNX29DYA0@brm-avmta-1.central.sun.com>; Thu,
 10 Sep 2009 15:01:21 -0600 (MDT)
Received: from sac.sfbay.sun.com (sac.SFBay.Sun.COM [129.146.226.132])
	by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2)
 with ESMTP id n8AL1IlR048757; Thu, 10 Sep 2009 14:01:18 -0700 (PDT)
Received: from sac.sfbay.sun.com (localhost [127.0.0.1])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n8AL1G2Y029316; Thu,
 10 Sep 2009 14:01:16 -0700 (PDT)
Received: (from markcarl@localhost)
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id n8AL1Gam029312; Thu,
 10 Sep 2009 14:01:16 -0700 (PDT)
Date: Thu, 10 Sep 2009 14:01:16 -0700 (PDT)
From: Mark Carlson <markcarl@sac.sfbay.sun.com>
Subject: Add bootpath into Solaris Sparc BootArchive for iSCSI boot
 [PSARC/2009/480 Self Review]
To: PSARC-ext@sun.com
Cc: Grant.Zhang@sun.com, Jack.Meng@sun.com
Message-id: <200909102101.n8AL1Gam029312@sac.sfbay.sun.com>
Content-transfer-encoding: 7BIT
X-PMX-Version: 5.4.1.325704
Status: RO
Content-Length: 4013

I am sponsoring this case for Jack Meng. It requests a minor binding and he ONNV consolidation. I have marked it automatic, but am happy to start a timer if an ARC member requests.

-- mark

Template Version: @(#)sac_nextcase 1.68 02/23/09 SMI
This information is Copyright 2009 Sun Microsystems
1. Introduction
    1.1. Project/Component Working Name:
	 Add bootpath into Solaris Sparc BootArchive for iSCSI boot
    1.2. Name of Document Author/Supplier:
	 Author:  Jack Meng
    1.3  Date of This Document:
	10 September, 2009
	
2. Project Summary
   2.1. Project Description
        This project is to introduce a private property, 'bootpath', along
        with the file 'bootenv.rc' into Solaris Sparc BootArchive to
        facilitate the iSCSI boot process in Solaris Sparc.
    
4. Technical Description
    4.1. Details:
	Currently Solaris Sparc kernel takes the hardware description of boot
	device from OBP, which is in compliance with IEEE Std. 1275, as the
	physical device path in OS. For example, the path,
		"/pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w21000004cf68e801,0"
	works fine for both OBP and Solaris as OBP considers this a
	platform-independent description while Solaris takes it as the physical
	device path.

	Things become different with the introduction of iSCSI Boot support, for
		1) iSCSI is not a part of IEEE Std. 1275, and also there is no
		other specification to precisely describe an iSCSI dis
		k/lun path, especially when there are different methods 
		to access the same iSCSI disk and make different physical device
		path name in Solaris OS
		2) the bootpath FWARC 2008/466 provided, '/iscsi-hba/disk', is
		not recognizable by Solaris Kernel

	OBP doesn't and shouldn't maintain any OS specific information so the
	Solaris	sparc kernel needs to get the OS boot device path from
	somewhere else and then	to mount the root onto it.

	To overcome this, a file, 'boot/solaris/bootenv.rc' is to be added into 
	the Solaris bootarchive	with a property 'bootpath' to record the OS boot 
	device path on sparc, e.g,
	
	setprop bootpath '/iscsi/ssd@0000iqn.1986-03.com.sun:02:43644186-e5bb-41f2-8b8e-f34be1afaebc0001,0:a'
	
	Such a bootpath is not available from OBP, and also Solaris is unable to
	make up one for,
		1) Different disk drivers, 'sd' or 'ssd' can be attached to an
		iSCSI disk
		2) Different TPGT can be used to access the boot disk which
		causes different device path
	Therefore the property 'bootpath' is needed to record the device path
	which is used to boot/install in last time. The same mechanism is in use
	to support Solaris UFS boot on x86.
	
	This property is proposed to be private, and is not exposed by 'eeprom'
	which is intended to interact with OBP rather than a file. The Solaris
	Installer will be responsible to update this file during installation
	and upgrade.
	
	This property is now designed to be used for iSCSI Boot only, however
	future boot support with software HBAs can also take benefit from it.
	
    4.5 Interfaces:

    Exported Interface                    | Stability    |    Description
    --------------------------------------|--------------|-------------------
    bootenv.rc                            | Consolidation| boot environment
                                          | Private      | resource file
    bootpath                              | Consolidation|
                                          | Private      |
    
    Imported Interface                    | Stability    |    Description
    --------------------------------------|--------------|-------------------
    usr/share/man/man1m/boot.1m           | Committed    |
        
    Other documents:
    FWARC 2008/466  iSCSI SPARC Openboot support http://sac.sfbay/FWARC/2008/466
    PSARC 2008/427  iSCSI Boot                   http://sac.sfbay/PSARC/2008/427
	 
6. Resources and Schedule
    6.4. Steering Committee requested information
   	6.4.1. Consolidation C-team Name:
		ON
    6.5. ARC review type: Automatic
    6.6. ARC Exposure: open


From bart.smaalders@sun.com Thu Sep 10 14:28:13 2009
Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n8ALSDw1029945
	for <psarc-ext@sac.sfbay.sun.com>; Thu, 10 Sep 2009 14:28:13 -0700 (PDT)
Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6])
	by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n8ALS8sD028342;
	Thu, 10 Sep 2009 22:28:11 +0100 (BST)
Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by
 nwk-avmta-2.sfbay.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KPR00G01YAWK200@nwk-avmta-2.sfbay.sun.com>; Thu,
 10 Sep 2009 14:28:08 -0700 (PDT)
Received: from zion.sfbay.sun.com ([129.146.17.75])
 by nwk-avmta-2.sfbay.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KPR00FWAYAWBS10@nwk-avmta-2.sfbay.sun.com>; Thu,
 10 Sep 2009 14:28:08 -0700 (PDT)
Received: from [129.146.228.109] (cyber.SFBay.Sun.COM [129.146.228.109])
	by zion.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n8ALRFNr025321; Thu,
 10 Sep 2009 21:27:15 +0000 (GMT)
Date: Thu, 10 Sep 2009 14:26:05 -0700
From: Bart Smaalders <bart.smaalders@sun.com>
Subject: Re: Add bootpath into Solaris Sparc BootArchive for iSCSI boot
 [PSARC/2009/480 Self Review]
In-reply-to: <200909102101.n8AL1Gam029312@sac.sfbay.sun.com>
To: Mark Carlson <markcarl@sac.sfbay.sun.com>
Cc: PSARC-ext@sun.com, Grant.Zhang@sun.com, Jack.Meng@sun.com,
        Jan Setje-Eilers <setje@eng.sun.com>
Message-id: <4AA96EED.7030206@Sun.COM>
Organization: Sun Microsystems
MIME-version: 1.0
Content-type: text/plain; charset=ISO-8859-1; format=flowed
Content-transfer-encoding: 7BIT
X-PMX-Version: 5.4.1.325704
References: <200909102101.n8AL1Gam029312@sac.sfbay.sun.com>
User-Agent: Thunderbird 2.0.0.22 (X11/20090804)
Status: RO
Content-Length: 1172

Mark Carlson wrote:

> 	Such a bootpath is not available from OBP, and also Solaris is unable to
> 	make up one for,
> 		1) Different disk drivers, 'sd' or 'ssd' can be attached to an
> 		iSCSI disk
> 		2) Different TPGT can be used to access the boot disk which
> 		causes different device path
> 	Therefore the property 'bootpath' is needed to record the device path
> 	which is used to boot/install in last time. The same mechanism is in use
> 	to support Solaris UFS boot on x86.

Is this really the right model to emulate, given that UFS boot is 
obsolete?  ZFS boot doesn't require this on x86.

Encoding the real boot path in a file that's part of the OS image you're 
trying to boot seems problematic.

> The Solaris
> 	Installer will be responsible to update this file during installation
> 	and upgrade.

There is no deus ex machina available during OpenSolaris upgrade to
rummage around, updating secret private config files needed to boot.

Is there really no better way to fix this?

- Bart



-- 
Bart Smaalders			Solaris Kernel Performance
barts@cyber.eng.sun.com		http://blogs.sun.com/barts
"You will contribute more with mercurial than with thunderbird."

From setje@sun.com Thu Sep 10 14:36:34 2009
Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n8ALaYfU000197
	for <psarc-ext@sac.sfbay.sun.com>; Thu, 10 Sep 2009 14:36:34 -0700 (PDT)
Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74])
	by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n8ALaSxX061736;
	Thu, 10 Sep 2009 15:36:31 -0600 (MDT)
Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by
 nwk-avmta-1.sfbay.Sun.COM
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KPR00J0VYOV1B00@nwk-avmta-1.sfbay.Sun.COM>; Thu,
 10 Sep 2009 14:36:31 -0700 (PDT)
Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.59])
 by nwk-avmta-1.sfbay.Sun.COM
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KPR007HCYOUA980@nwk-avmta-1.sfbay.Sun.COM>; Thu,
 10 Sep 2009 14:36:30 -0700 (PDT)
Received: from [129.146.226.114] (smack.SFBay.Sun.COM [129.146.226.114])
	by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3)
 with ESMTP id n8ALaU8v309019; Thu, 10 Sep 2009 14:36:30 -0700 (PDT)
Date: Thu, 10 Sep 2009 14:36:30 -0700
From: Jan Setje-Eilers <setje@sun.com>
Subject: Re: Add bootpath into Solaris Sparc BootArchive for iSCSI boot
 [PSARC/2009/480 Self Review]
In-reply-to: <4AA96EED.7030206@Sun.COM>
To: Bart Smaalders <bart.smaalders@sun.com>
Cc: Mark Carlson <markcarl@sac.sfbay.sun.com>, PSARC-ext@sun.com,
        Grant.Zhang@sun.com, Jack.Meng@sun.com,
        Jan Setje-Eilers <setje@eng.sun.com>
Message-id: <4AA9715E.2020805@sun.com>
MIME-version: 1.0
Content-type: text/plain; charset=ISO-8859-1; format=flowed
Content-transfer-encoding: 7BIT
X-PMX-Version: 5.4.1.325704
References: <200909102101.n8AL1Gam029312@sac.sfbay.sun.com>
 <4AA96EED.7030206@Sun.COM>
User-Agent: Thunderbird 2.0.0.22 (X11/20090818)
Status: RO
Content-Length: 1692

Bart Smaalders wrote:
> Mark Carlson wrote:
> 
>>     Such a bootpath is not available from OBP, and also Solaris is 
>> unable to
>>     make up one for,
>>         1) Different disk drivers, 'sd' or 'ssd' can be attached to an
>>         iSCSI disk
>>         2) Different TPGT can be used to access the boot disk which
>>         causes different device path
>>     Therefore the property 'bootpath' is needed to record the device path
>>     which is used to boot/install in last time. The same mechanism is 
>> in use
>>     to support Solaris UFS boot on x86.
> 
> Is this really the right model to emulate, given that UFS boot is 
> obsolete?  ZFS boot doesn't require this on x86.
> 
> Encoding the real boot path in a file that's part of the OS image you're 
> trying to boot seems problematic.

  Correct. My previous comment on this was to ask the proposing team to 
find a better approach.

  Storing paths such as this breaks systems as soon as a minor 
configuration change impacts the path. This is a major issue on x86 that 
has kept OS images from being as portable as they should, and is really 
something we want to fix as opposed to broadening the impact as you're 
proposing.

>> The Solaris
>>     Installer will be responsible to update this file during installation
>>     and upgrade.
> 
> There is no deus ex machina available during OpenSolaris upgrade to
> rummage around, updating secret private config files needed to boot.
> 
> Is there really no better way to fix this?

  I'd like to see some more investigation here.

  Why can't this path be determined on-the-fly? At worst this should 
require tasting a couple of possible devices at mountroot time.

-jan

From Darren.Moffat@sun.com Fri Sep 11 01:37:26 2009
Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n8B8bPbr000871
	for <psarc-ext@sac.sfbay.sun.com>; Fri, 11 Sep 2009 01:37:26 -0700 (PDT)
Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74])
	by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n8B8bAut004684
	for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Fri, 11 Sep 2009 16:37:24 +0800 (SGT)
Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by
 nwk-avmta-1.sfbay.Sun.COM
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KPS00H05TABEI00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com
 (ORCPT PSARC-ext@sun.com); Fri, 11 Sep 2009 01:37:23 -0700 (PDT)
Received: from gmp-eb-inf-2.sun.com ([192.18.6.24])
 by nwk-avmta-1.sfbay.Sun.COM
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KPS00A06TAAU7F0@nwk-avmta-1.sfbay.Sun.COM> for
 PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri,
 11 Sep 2009 01:37:23 -0700 (PDT)
Received: from fe-emea-10.sun.com
 (gmp-eb-lb-1-fe1.eu.sun.com [192.18.6.7] (may be forged))
	by gmp-eb-inf-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n8B8bM6v024356	for
 <PSARC-ext@sun.com>; Fri, 11 Sep 2009 08:37:22 +0000 (GMT)
Received: from conversion-daemon.fe-emea-10.sun.com by fe-emea-10.sun.com
 (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul  2 2009))
 id <0KPS00300T48Z900@fe-emea-10.sun.com> for PSARC-ext@sun.com
 (ORCPT PSARC-ext@sun.com); Fri, 11 Sep 2009 09:37:06 +0100 (BST)
Received: from [192.168.1.105]
 (cpc2-rdng20-2-0-cust917.15-3.cable.virginmedia.com [86.28.167.150])
 by fe-emea-10.sun.com
 (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul  2 2009))
 with ESMTPSA id <0KPS000TRT9MET10@fe-emea-10.sun.com>; Fri,
 11 Sep 2009 09:36:59 +0100 (BST)
Date: Fri, 11 Sep 2009 09:36:58 +0100
From: Darren J Moffat <Darren.Moffat@sun.com>
Subject: Re: Add bootpath into Solaris Sparc BootArchive for iSCSI boot
 [PSARC/2009/480 Self Review]
In-reply-to: <4AA9715E.2020805@sun.com>
Sender: Darren.Moffat@sun.com
To: Jan Setje-Eilers <setje@sun.com>
Cc: Bart Smaalders <Bart.Smaalders@sun.com>,
        Mark Carlson <markcarl@sac.sfbay.sun.com>, PSARC-ext@sun.com,
        Grant.Zhang@sun.com, Jack.Meng@sun.com,
        Jan Setje-Eilers <setje@eng.sun.com>
Message-id: <4AAA0C2A.1040707@Sun.COM>
MIME-version: 1.0
Content-type: text/plain; CHARSET=US-ASCII; format=flowed
Content-transfer-encoding: 7BIT
X-PMX-Version: 5.4.1.325704
References: <200909102101.n8AL1Gam029312@sac.sfbay.sun.com>
 <4AA96EED.7030206@Sun.COM> <4AA9715E.2020805@sun.com>
User-Agent: Thunderbird 2.0.0.22 (X11/20090818)
Status: RO
Content-Length: 2690

Jan Setje-Eilers wrote:
> Bart Smaalders wrote:
>> Mark Carlson wrote:
>>
>>>     Such a bootpath is not available from OBP, and also Solaris is 
>>> unable to
>>>     make up one for,
>>>         1) Different disk drivers, 'sd' or 'ssd' can be attached to an
>>>         iSCSI disk
>>>         2) Different TPGT can be used to access the boot disk which
>>>         causes different device path
>>>     Therefore the property 'bootpath' is needed to record the device 
>>> path
>>>     which is used to boot/install in last time. The same mechanism is 
>>> in use
>>>     to support Solaris UFS boot on x86.
>>
>> Is this really the right model to emulate, given that UFS boot is 
>> obsolete?  ZFS boot doesn't require this on x86.
>>
>> Encoding the real boot path in a file that's part of the OS image 
>> you're trying to boot seems problematic.
> 
>  Correct. My previous comment on this was to ask the proposing team to 
> find a better approach.
> 
>  Storing paths such as this breaks systems as soon as a minor 
> configuration change impacts the path. This is a major issue on x86 that 
> has kept OS images from being as portable as they should, and is really 
> something we want to fix as opposed to broadening the impact as you're 
> proposing.

That was my thought as well as soon as I saw this.

>>> The Solaris
>>>     Installer will be responsible to update this file during 
>>> installation
>>>     and upgrade.

Which installer ?  Has the project team got a committment from the team 
working on the Caiman installer to do this for LiveCD and AI installs ?
What kind of upgrade is being discussed here ?

What happens it the property isn't present ?

>> There is no deus ex machina available during OpenSolaris upgrade to
>> rummage around, updating secret private config files needed to boot.
>>
>> Is there really no better way to fix this?
> 
>  I'd like to see some more investigation here.
> 
>  Why can't this path be determined on-the-fly? At worst this should 
> require tasting a couple of possible devices at mountroot time.

Similar to that to even be able to read the file out of the boot archive 
means we must have sort of iSCSI device available already right ?

For me one of the major benefits of iSCSI boot over NFS diskless boot is 
that it should be highly portable and there should be much less "baked" 
into the image.


Given the comments from Bart, Jan and myself I'm very close to derailing 
this case but for the moment I'll strongly suggest it be put in "waiting 
need spec" and that the project team work with the installer and boot 
teams (in particular I strongly suggest taking advice from Jan for the 
boot issues).

-- 
Darren J Moffat

From Mark.Carlson@sun.com Fri Sep 11 04:50:44 2009
Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n8BBohvu012821
	for <psarc-ext@sac.sfbay.sun.com>; Fri, 11 Sep 2009 04:50:44 -0700 (PDT)
Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74])
	by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n8BBocVV010708
	for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Fri, 11 Sep 2009 12:50:42 +0100 (BST)
Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by
 nwk-avmta-1.sfbay.Sun.COM
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KPT00J0928FTK00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com
 (ORCPT PSARC-ext@Sun.COM); Fri, 11 Sep 2009 04:50:39 -0700 (PDT)
Received: from brmea-mail-4.sun.com ([192.18.98.36])
 by nwk-avmta-1.sfbay.Sun.COM
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KPT00LDK28FXM90@nwk-avmta-1.sfbay.Sun.COM> for
 PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.COM); Fri,
 11 Sep 2009 04:50:39 -0700 (PDT)
Received: from fe-amer-10.sun.com ([192.18.109.80])
	by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n8BBodct016778	for
 <PSARC-ext@Sun.COM>; Fri, 11 Sep 2009 11:50:39 +0000 (GMT)
Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com
 (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul  2 2009))
 id <0KPT00500221FQ00@mail-amer.sun.com> for PSARC-ext@Sun.COM
 (ORCPT PSARC-ext@Sun.COM); Fri, 11 Sep 2009 05:50:39 -0600 (MDT)
Received: from Macintosh-335.local ([unknown] [129.150.32.91])
 by mail-amer.sun.com
 (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul  2 2009))
 with ESMTPSA id <0KPT006PC28E7T80@mail-amer.sun.com>; Fri,
 11 Sep 2009 05:50:39 -0600 (MDT)
Date: Fri, 11 Sep 2009 05:50:38 -0600
From: "Mark A. Carlson" <Mark.Carlson@sun.com>
Subject: Re: Add bootpath into Solaris Sparc BootArchive for iSCSI boot
	[PSARC/2009/480 Self Review]
In-reply-to: <4AAA0C2A.1040707@Sun.COM>
Sender: Mark.Carlson@sun.com
To: Darren J Moffat <Darren.Moffat@sun.com>
Cc: PSARC-ext@sun.com, Bart Smaalders <Bart.Smaalders@sun.com>,
        Jack.Meng@sun.com, Grant.Zhang@sun.com
Message-id: <4AAA398E.4010800@sun.com>
MIME-version: 1.0
Content-type: text/plain; CHARSET=US-ASCII; format=flowed
Content-transfer-encoding: 7BIT
X-PMX-Version: 5.4.1.325704
References: <200909102101.n8AL1Gam029312@sac.sfbay.sun.com>
 <4AA96EED.7030206@Sun.COM> <4AA9715E.2020805@sun.com>
 <4AAA0C2A.1040707@Sun.COM>
User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812)
Status: RO
Content-Length: 235

Darren J Moffat wrote:
> ... suggest it be put in "waiting need spec" and that the project team 
> work with the installer and boot teams (in particular I strongly 
> suggest taking advice from Jan for the boot issues).
Done.

-- mark

From Jack.Meng@sun.com Fri Sep 11 08:31:20 2009
Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n8BFVJ9u029460
	for <psarc-ext@sac.sfbay.sun.com>; Fri, 11 Sep 2009 08:31:20 -0700 (PDT)
Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6])
	by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n8BFVBB1028116
	for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Fri, 11 Sep 2009 16:31:18 +0100 (BST)
Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by
 nwk-avmta-2.sfbay.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KPT00M01CG59Q00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com
 (ORCPT PSARC-ext@Sun.COM); Fri, 11 Sep 2009 08:31:17 -0700 (PDT)
Received: from sineb-mail-2.sun.com ([192.18.19.7])
 by nwk-avmta-2.sfbay.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KPT00DOSCG3NMB0@nwk-avmta-2.sfbay.sun.com> for
 PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.COM); Fri,
 11 Sep 2009 08:31:16 -0700 (PDT)
Received: from fe-apac-06.sun.com
 (fe-apac-06.sun.com [192.18.19.177] (may be forged))
	by sineb-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n8BFVFSq021205	for
 <PSARC-ext@Sun.COM>; Fri, 11 Sep 2009 15:31:15 +0000 (GMT)
Received: from conversion-daemon.mail-apac.sun.com by mail-apac.sun.com
 (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul  2 2009))
 id <0KPT00D00CAIJR00@mail-apac.sun.com> for PSARC-ext@Sun.COM
 (ORCPT PSARC-ext@Sun.COM); Fri, 11 Sep 2009 23:31:15 +0800 (SGT)
Received: from [115.171.238.209] ([unknown] [115.171.238.209])
 by mail-apac.sun.com
 (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul  2 2009))
 with ESMTPSA id <0KPT0060ICFZTV40@mail-apac.sun.com>; Fri,
 11 Sep 2009 23:31:15 +0800 (SGT)
Date: Fri, 11 Sep 2009 23:31:10 +0800
From: Jack Meng <Jack.Meng@sun.com>
Subject: Re: Add bootpath into Solaris Sparc BootArchive for iSCSI boot
	[PSARC/2009/480 Self Review]
In-reply-to: <4AAA398E.4010800@sun.com>
Sender: Jack.Meng@sun.com
To: "Mark A. Carlson" <Mark.Carlson@sun.com>
Cc: Darren J Moffat <Darren.Moffat@sun.com>, PSARC-ext@sun.com,
        Bart Smaalders <Bart.Smaalders@sun.com>, Grant.Zhang@sun.com,
        Jan Setje-Eilers <Jan.Setje-Eilers@sun.com>,
        Tarl Neustaedter <Tarl.Neustaedter@sun.com>,
        David Kahn <David.Kahn@sun.com>
Message-id: <4AAA6D3E.5040506@sun.com>
MIME-version: 1.0
Content-type: text/plain; CHARSET=US-ASCII; format=flowed
Content-transfer-encoding: 7BIT
X-PMX-Version: 5.4.1.325704
References: <200909102101.n8AL1Gam029312@sac.sfbay.sun.com>
 <4AA96EED.7030206@Sun.COM> <4AA9715E.2020805@sun.com>
 <4AAA0C2A.1040707@Sun.COM> <4AAA398E.4010800@sun.com>
User-Agent: Thunderbird 2.0.0.23 (Windows/20090812)
Status: RO
Content-Length: 4042

Mark A. Carlson wrote:
> Darren J Moffat wrote:
>> ... suggest it be put in "waiting need spec" and that the project 
>> team work with the installer and boot teams (in particular I strongly 
>> suggest taking advice from Jan for the boot issues).
> Done.
>
> -- mark
Adding materials about investigations made by the project team so far to 
solve the bootpath issue, and reflect the purpose of this case.


The project team has been trying different approaches to solve the 
'bootpath' issue,

1. Store the bootpath in firmware (OBP).

Details:
This approach needs to create a dedicated option value in OBP space to 
store the bootpath,
and then let Solaris kernel read it during booting.

ISSUES:
Negative opinion is received from firmware team for the bootpath of an 
iSCSI disk is a Solaris
specific property (not per IEEE 1275), and firmware shouldn't maintain 
OS-dependent
properties.

2. Store the bootpath in boot_archive as the existing mechanism to 
support UFS booting in x86.

Details:
set the bootpath in file boot/solaris/bootenv.rc with an entry to 
contain the bootpath, e.g.,
"setprop bootpath 
'/iscsi/ssd@0000iqn.1986-03.com.sun:02:43644186-e5bb-41f2-8b8e-f34be1afaebc0001,0:a'",
and add this file into the Solaris sparc boot archive.

The Solaris kernel reads the property from archive during boot, and then 
mount rootfs on the
bootpath. This is the existing method to support UFS boot in Solaris x86.
For ZFS boot in Solaris x86, the zfs loader reads the bootpath from 
rpool and implicitly
passes it to Solaris kernel.

ISSUES:
Storing paths such as this breaks systems as soon as a minor 
configuration change impacts the
path. This is a major issue on x86 that has kept OS images from being as 
portable. If adopted
it will broaden this back impact to Solaris sparc booting from iSCSI 
disk. This part has been
widely pointed with previous emails in this thread.

3. Dynamically build the bootpath before mounting the root disk in 
Solaris kernel.

Details:
Solaris build the bootpath with properties from firmware and then tries 
to mount the rootfs on it.

ISSUES:
The generated bootpath is not guaranteed to be same as the one used to 
contain the rootfs. A typical
iscsi boot path is in the form of,
"/iscsi/ssd@0000iqn.1986-03.com.sun:2510.600a0b800049c94d00000000493c920b0001,0:a"

with following components,

  1. iscsi                    - the iscsi driver name
  2. ssd                    - the disk driver name/alias
  3. 0000                    - the initiator ISID
  4. iqn.1986-03.com.sun:02:43644186-e5bb-41f2-8b8e-f34be1afaebc
                        - the target name
  5. 0001                    - the TargetPortalGroupTag
  6. 0                        - lun number
  7. :a                        - slice

Here component 2 and 5 can be different with different targets/settings.

For component 5,
It is the TargetPortalGroupTag which can be set in target side. However 
a possible solution for
this part could be limiting the TPGT value to a specific number for 
booting in delivery documentation,
that will make the problem much simpler.

For component 2,

iSCSI disk can be attached to either 'ssd' and 'sd' (disk), mainly but 
not totally decided by the MPxIO support of
the device. For example, an user installed Solaris with root disk 
attached to 'ssd', and then the user
changed the MPxIO setting in Solaris then the correct bootpah would be 
changed to "../disk@...".
The above makes that during boot time kernel has to try the possible 2 
paths to find the root.

Overall here the best situation would be 2 candidate paths. So to 
overcome this a new mechanism
to try to mount the root with possible paths needs to be introduced into 
Solaris kernel.
One issue with the above trial-and-error approach is how to 
differentiate the bootpath error with the
network configuration/traffic error.

The project has verified all above solutions are valid, and is trying to 
select a best approach to solve
the bootpath issue stated, and so is open to comments, and surely other 
better approaches.

From Darren.Moffat@Sun.COM Mon Sep 14 01:37:56 2009
Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n8E8btRK020902
	for <psarc-ext@sac.sfbay.sun.com>; Mon, 14 Sep 2009 01:37:55 -0700 (PDT)
Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74])
	by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n8E8brVR017318
	for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Mon, 14 Sep 2009 09:37:54 +0100 (BST)
Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by
 nwk-avmta-1.sfbay.Sun.COM
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KPY0070HDB51Q00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com
 (ORCPT PSARC-ext@sun.com); Mon, 14 Sep 2009 01:37:53 -0700 (PDT)
Received: from gmp-eb-inf-1.sun.com ([192.18.6.21])
 by nwk-avmta-1.sfbay.Sun.COM
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KPY0059MDB4D110@nwk-avmta-1.sfbay.Sun.COM> for
 PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon,
 14 Sep 2009 01:37:52 -0700 (PDT)
Received: from fe-emea-10.sun.com
 (gmp-eb-lb-1-fe1.eu.sun.com [192.18.6.7] (may be forged))
	by gmp-eb-inf-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n8E8bpNN010075	for
 <PSARC-ext@sun.com>; Mon, 14 Sep 2009 08:37:51 +0000 (GMT)
Received: from conversion-daemon.fe-emea-10.sun.com by fe-emea-10.sun.com
 (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul  2 2009))
 id <0KPY00D00CI3D500@fe-emea-10.sun.com> for PSARC-ext@sun.com
 (ORCPT PSARC-ext@sun.com); Mon, 14 Sep 2009 09:37:34 +0100 (BST)
Received: from [192.168.1.105]
 (cpc2-rdng20-2-0-cust917.15-3.cable.virginmedia.com [86.28.167.150])
 by fe-emea-10.sun.com
 (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul  2 2009))
 with ESMTPSA id <0KPY00FQJDAHVR90@fe-emea-10.sun.com>; Mon,
 14 Sep 2009 09:37:30 +0100 (BST)
Date: Mon, 14 Sep 2009 09:37:29 +0100
From: Darren J Moffat <Darren.Moffat@Sun.COM>
Subject: Re: Add bootpath into Solaris Sparc BootArchive for iSCSI boot
	[PSARC/2009/480 Self Review]
In-reply-to: <4AAA6D3E.5040506@sun.com>
Sender: Darren.Moffat@Sun.COM
To: Jack Meng <Jack.Meng@Sun.COM>
Cc: "Mark A. Carlson" <Mark.Carlson@Sun.COM>, PSARC-ext@Sun.COM,
        Bart Smaalders <Bart.Smaalders@Sun.COM>, Grant.Zhang@Sun.COM,
        Jan Setje-Eilers <Jan.Setje-Eilers@Sun.COM>,
        Tarl Neustaedter <Tarl.Neustaedter@Sun.COM>,
        David Kahn <David.Kahn@Sun.COM>
Message-id: <4AAE00C9.8090303@Sun.COM>
MIME-version: 1.0
Content-type: text/plain; CHARSET=US-ASCII; format=flowed
Content-transfer-encoding: 7BIT
X-PMX-Version: 5.4.1.325704
References: <200909102101.n8AL1Gam029312@sac.sfbay.sun.com>
 <4AA96EED.7030206@Sun.COM> <4AA9715E.2020805@sun.com>
 <4AAA0C2A.1040707@Sun.COM> <4AAA398E.4010800@sun.com>
 <4AAA6D3E.5040506@sun.com>
User-Agent: Thunderbird 2.0.0.22 (X11/20090818)
Status: RO
Content-Length: 888

Jack Meng wrote:
> 1. Store the bootpath in firmware (OBP).
> 
> Details:
> This approach needs to create a dedicated option value in OBP space to 
> store the bootpath,
> and then let Solaris kernel read it during booting.
> 
> ISSUES:
> Negative opinion is received from firmware team for the bootpath of an 
> iSCSI disk is a Solaris
> specific property (not per IEEE 1275), and firmware shouldn't maintain 
> OS-dependent
> properties.

OBP already stores Solaris specific properties and has Sun system and 
Solaris specific code and properties [at the very least for WAN boot but 
there was E10k specific stuff there too in the past and I'm sure there 
is more].  So I really don't see why this is an issue. It reads to me 
like it can be done but someone doesn't want to do it that way, is that 
correct ?  Or is there a technical reason why it can't be done ?

-- 
Darren J Moffat

From Jack.Meng@Sun.COM Mon Sep 14 09:08:23 2009
Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n8EG8NNt005693
	for <psarc-ext@sac.sfbay.sun.com>; Mon, 14 Sep 2009 09:08:23 -0700 (PDT)
Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6])
	by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n8EG7v9u063888
	for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Mon, 14 Sep 2009 10:08:22 -0600 (MDT)
Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by
 nwk-avmta-2.sfbay.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KPY00J2BY5X0100@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com
 (ORCPT PSARC-ext@sun.com); Mon, 14 Sep 2009 09:08:21 -0700 (PDT)
Received: from sineb-mail-2.sun.com ([192.18.19.7])
 by nwk-avmta-2.sfbay.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KPY00DV7Y5W1490@nwk-avmta-2.sfbay.sun.com> for
 PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon,
 14 Sep 2009 09:08:21 -0700 (PDT)
Received: from fe-apac-05.sun.com
 (fe-apac-05.sun.com [192.18.19.176] (may be forged))
	by sineb-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n8EG8JG0022676	for
 <PSARC-ext@sun.com>; Mon, 14 Sep 2009 16:08:19 +0000 (GMT)
Received: from conversion-daemon.mail-apac.sun.com by mail-apac.sun.com
 (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul  2 2009))
 id <0KPY00400XOB5E00@mail-apac.sun.com> for PSARC-ext@sun.com
 (ORCPT PSARC-ext@sun.com); Tue, 15 Sep 2009 00:08:19 +0800 (SGT)
Received: from [115.170.238.167] ([unknown] [115.170.238.167])
 by mail-apac.sun.com
 (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul  2 2009))
 with ESMTPSA id <0KPY00DJBY5RJXC0@mail-apac.sun.com>; Tue,
 15 Sep 2009 00:08:19 +0800 (SGT)
Date: Tue, 15 Sep 2009 00:08:12 +0800
From: Jack Meng <Jack.Meng@Sun.COM>
Subject: Re: Add bootpath into Solaris Sparc BootArchive for iSCSI boot
	[PSARC/2009/480 Self Review]
In-reply-to: <4AAE00C9.8090303@Sun.COM>
Sender: Jack.Meng@Sun.COM
To: Darren J Moffat <Darren.Moffat@Sun.COM>
Cc: "Mark A. Carlson" <Mark.Carlson@Sun.COM>, PSARC-ext@Sun.COM,
        Bart Smaalders <Bart.Smaalders@Sun.COM>, Grant.Zhang@Sun.COM,
        Jan Setje-Eilers <Jan.Setje-Eilers@Sun.COM>,
        Tarl Neustaedter <Tarl.Neustaedter@Sun.COM>,
        David Kahn <David.Kahn@Sun.COM>
Message-id: <4AAE6A6C.4000105@sun.com>
MIME-version: 1.0
Content-type: text/plain; CHARSET=US-ASCII; format=flowed
Content-transfer-encoding: 7BIT
X-PMX-Version: 5.4.1.325704
References: <200909102101.n8AL1Gam029312@sac.sfbay.sun.com>
 <4AA96EED.7030206@Sun.COM> <4AA9715E.2020805@sun.com>
 <4AAA0C2A.1040707@Sun.COM> <4AAA398E.4010800@sun.com>
 <4AAA6D3E.5040506@sun.com> <4AAE00C9.8090303@Sun.COM>
User-Agent: Thunderbird 2.0.0.23 (Windows/20090812)
Status: RO
Content-Length: 1063

Darren J Moffat wrote:
> Jack Meng wrote:
>> 1. Store the bootpath in firmware (OBP).
>>
>> Details:
>> This approach needs to create a dedicated option value in OBP space 
>> to store the bootpath,
>> and then let Solaris kernel read it during booting.
>>
>> ISSUES:
>> Negative opinion is received from firmware team for the bootpath of 
>> an iSCSI disk is a Solaris
>> specific property (not per IEEE 1275), and firmware shouldn't 
>> maintain OS-dependent
>> properties.
>
> OBP already stores Solaris specific properties and has Sun system and 
> Solaris specific code and properties [at the very least for WAN boot 
> but there was E10k specific stuff there too in the past and I'm sure 
> there is more].  So I really don't see why this is an issue. It reads 
> to me like it can be done but someone doesn't want to do it that way, 
> is that correct ?  Or is there a technical reason why it can't be done ?
>
Hi Darren,

The firmware team, cc'ed in the loop, should be able to provide their 
considerations regarding to your concern.

Best regards,
Jack

From David.Kahn@sun.com Mon Sep 14 09:13:05 2009
Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n8EGD4ig005837
	for <psarc-ext@sac.sfbay.sun.com>; Mon, 14 Sep 2009 09:13:04 -0700 (PDT)
Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74])
	by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n8EGD229001447
	for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Mon, 14 Sep 2009 10:13:04 -0600 (MDT)
Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by
 nwk-avmta-1.sfbay.Sun.COM
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KPY00I0FYDQLI00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com
 (ORCPT PSARC-ext@sun.com); Mon, 14 Sep 2009 09:13:02 -0700 (PDT)
Received: from dm-sfbay-02.sfbay.sun.com ([129.146.11.31])
 by nwk-avmta-1.sfbay.Sun.COM
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KPY0048JYDQ5TF0@nwk-avmta-1.sfbay.Sun.COM> for
 PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon,
 14 Sep 2009 09:13:02 -0700 (PDT)
Received: from dtmail.sfbay.sun.com (dt101-150.SFBay.Sun.COM [10.6.101.150])
	by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2)
 with ESMTP id n8EGCwVE063354; Mon, 14 Sep 2009 09:12:58 -0700 (PDT)
Received: from [127.0.0.1] (noho.SFBay.Sun.COM [10.6.92.101])
	by dtmail.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n8EGCqLM005262; Mon,
 14 Sep 2009 09:12:54 -0700 (PDT)
Date: Mon, 14 Sep 2009 09:12:54 -0700
From: David Kahn <David.Kahn@sun.com>
Subject: Re: Add bootpath into Solaris Sparc BootArchive for iSCSI boot
	[PSARC/2009/480 Self Review]
In-reply-to: <4AAE00C9.8090303@Sun.COM>
To: Darren J Moffat <Darren.Moffat@sun.com>
Cc: Jack Meng <Jack.Meng@sun.com>, "Mark A. Carlson" <Mark.Carlson@sun.com>,
        PSARC-ext@sun.com, Bart Smaalders <Bart.Smaalders@sun.com>,
        Grant.Zhang@sun.com, Jan Setje-Eilers <Jan.Setje-Eilers@sun.com>,
        Tarl Neustaedter <Tarl.Neustaedter@sun.com>
Message-id: <4AAE6B86.4030501@sun.com>
MIME-version: 1.0
Content-type: text/plain; charset=ISO-8859-1; format=flowed
Content-transfer-encoding: 7BIT
X-PMX-Version: 5.4.1.325704
References: <200909102101.n8AL1Gam029312@sac.sfbay.sun.com>
 <4AA96EED.7030206@Sun.COM> <4AA9715E.2020805@sun.com>
 <4AAA0C2A.1040707@Sun.COM> <4AAA398E.4010800@sun.com>
 <4AAA6D3E.5040506@sun.com> <4AAE00C9.8090303@Sun.COM>
User-Agent: Thunderbird 2.0.0.23 (Windows/20090812)
Status: RO
Content-Length: 1222


I'm not really sure what the history of this is, but OBP
supports native iscsi boot with a FWARC case. In that caae,
all the arguments are provided for and we can boot directly
from an iscsi target.

What else does the firmware need to support besides what it
already does support, and why?

-David


Darren J Moffat wrote:
> Jack Meng wrote:
>> 1. Store the bootpath in firmware (OBP).
>>
>> Details:
>> This approach needs to create a dedicated option value in OBP space to 
>> store the bootpath,
>> and then let Solaris kernel read it during booting.
>>
>> ISSUES:
>> Negative opinion is received from firmware team for the bootpath of an 
>> iSCSI disk is a Solaris
>> specific property (not per IEEE 1275), and firmware shouldn't maintain 
>> OS-dependent
>> properties.
> 
> OBP already stores Solaris specific properties and has Sun system and 
> Solaris specific code and properties [at the very least for WAN boot but 
> there was E10k specific stuff there too in the past and I'm sure there 
> is more].  So I really don't see why this is an issue. It reads to me 
> like it can be done but someone doesn't want to do it that way, is that 
> correct ?  Or is there a technical reason why it can't be done ?
> 

From Tarl.Neustaedter@sun.com Mon Sep 14 11:20:36 2009
Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n8EIKaXh019334
	for <psarc-ext@sac.sfbay.sun.com>; Mon, 14 Sep 2009 11:20:36 -0700 (PDT)
Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6])
	by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n8EIKOTW023470
	for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Mon, 14 Sep 2009 12:20:35 -0600 (MDT)
Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by
 nwk-avmta-2.sfbay.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KPZ004034AB1300@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com
 (ORCPT PSARC-ext@sun.com); Mon, 14 Sep 2009 11:20:35 -0700 (PDT)
Received: from brmea-mail-1.sun.com ([192.18.98.31])
 by nwk-avmta-2.sfbay.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KPZ001AJ4AAGK60@nwk-avmta-2.sfbay.sun.com> for
 PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon,
 14 Sep 2009 11:20:35 -0700 (PDT)
Received: from fe-amer-10.sun.com ([192.18.109.80])
	by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n8EIKYQ6005779	for
 <PSARC-ext@sun.com>; Mon, 14 Sep 2009 18:20:34 +0000 (GMT)
Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com
 (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul  2 2009))
 id <0KPZ00600425EP00@mail-amer.sun.com> for PSARC-ext@sun.com
 (ORCPT PSARC-ext@sun.com); Mon, 14 Sep 2009 12:20:34 -0600 (MDT)
Received: from dhcp-ubur-189-199.East.Sun.COM ([unknown] [129.148.189.199])
 by mail-amer.sun.com
 (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul  2 2009))
 with ESMTPSA id <0KPZ00MU649W0QG0@mail-amer.sun.com>; Mon,
 14 Sep 2009 12:20:21 -0600 (MDT)
Date: Mon, 14 Sep 2009 14:20:19 -0400
From: Tarl Neustaedter <Tarl.Neustaedter@sun.com>
Subject: Re: Add bootpath into Solaris Sparc BootArchive for iSCSI boot
	[PSARC/2009/480 Self Review]
In-reply-to: <4AAE00C9.8090303@Sun.COM>
Sender: Tarl.Neustaedter@sun.com
To: Darren J Moffat <Darren.Moffat@sun.com>
Cc: Jack Meng <Jack.Meng@sun.com>, "Mark A. Carlson" <Mark.Carlson@sun.com>,
        PSARC-ext@sun.com, Bart Smaalders <Bart.Smaalders@sun.com>,
        Grant.Zhang@sun.com, Jan Setje-Eilers <Jan.Setje-Eilers@sun.com>,
        David Kahn <David.Kahn@sun.com>
Message-id: <4AAE8963.7050200@sun.com>
MIME-version: 1.0
Content-type: multipart/alternative;
 boundary="Boundary_(ID_MkJpZb4LqsCEF4m7p9jubA)"
X-PMX-Version: 5.4.1.325704
References: <200909102101.n8AL1Gam029312@sac.sfbay.sun.com>
 <4AA96EED.7030206@Sun.COM> <4AA9715E.2020805@sun.com>
 <4AAA0C2A.1040707@Sun.COM> <4AAA398E.4010800@sun.com>
 <4AAA6D3E.5040506@sun.com> <4AAE00C9.8090303@Sun.COM>
User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812)
Status: RO
Content-Length: 7827

This is a multi-part message in MIME format.

--Boundary_(ID_MkJpZb4LqsCEF4m7p9jubA)
Content-type: text/plain; CHARSET=US-ASCII; format=flowed
Content-transfer-encoding: 7BIT

Darren J Moffat wrote:
> [...]
> OBP already stores Solaris specific properties and has Sun system and 
> Solaris specific code and properties [at the very least for WAN boot 
> but there was E10k specific stuff there too in the past and I'm sure 
> there is more].  So I really don't see why this is an issue. It reads 
> to me like it can be done but someone doesn't want to do it that way, 
> is that correct ?  Or is there a technical reason why it can't be done ?

What the Solaris iSCSI team is asking for is for us to make the Solaris 
internal path part of the boot arguments. Currently (per FWARC 
2008/466), the boot string on iSCSI looks like:

/pci@500/pci@0/pci@8/network@0:iscsi-target-ip=129.148.67.188,host-ip=129.148.67.111,iscsi-target-name=iqn.1986-03.com.sun:02:43644186-e5bb-41f2-8b8e-f34be1afaebc  


What the Solaris team is asking for is to add the Solaris internal root 
disk path to the above arguments. As I recall, they want to add an 
additional argument:

iscsi-phy-path=/iscsi/ssd@0000iqn.1986-03.com.sun:02:43644186-e5bb-41f2-8b8e-f34be1afaebc0001:a 
.

Aside from the architectural concerns (OBP has no business even knowing 
that it is Solaris being booted, let alone how Solaris was configured 
internally at the time Solaris was installed), most of the above is 
redundant information. There is also a concern about path lengths - if 
the boot string exceeds 256 bytes, life gets more painful (we have ways 
of handling it, but they aren't pleasant). The Solaris iSCSI path can be 
disassembled as follows:

/iscsi/ssd@0000iqn.1986-03.com.sun:02:43644186-e5bb-41f2-8b8e-f34be1afaebc0001,0:a

   1. /iscsi
   2. /ssd@
   3. 0000
   4. iqn.1986-03.com.sun:02:43644186-e5bb-41f2-8b8e-f34be1afaebc
   5. 0001
   6. 0
   7. :a

Item (1) is a constant. Item (2) can be either "ssd" or "disk", although 
I have yet to determine what conditions can result in the second case 
being configured or how to force it. Item three (the ISID) is 
effectively a constant. Item 4 is that iSCSI target name, available 
through /chosen properties. Item (5) is the Target Portal Group Tag, 
which at the moment is not provided by OBP, but can be determined during 
the iSCSI login process. Item 6 is the Lun, which is provided by OBP, 
and item 7 is the partition, which is also provided by OBP.

In other words, except item (2), all of the above information is either 
provided by OBP, is constant, or can be derived at runtime. I have yet 
to figure out how item (2) is variable, and whether we need to support 
bootability from the configuration which is different. Over the last 
week or so, Jan has been discussing this with the iSCSI boot iteam their 
options on this. At the moment, the iSCSI boot iteam is setting the 
bootpath string in the boot_archive, similar to what is done in x86, and 
Jan has objections to that.

An additional concern is that an alternate (I would say preferred) boot 
command form is:

boot net:dhcp

In this case, the iscsi information is provided by the DHCP server per 
RFC 4173, with the ROOTPATH option. This option does not include the 
possibility of any Solaris internal configuration information, so there 
is nowhere to add the information desired.


--Boundary_(ID_MkJpZb4LqsCEF4m7p9jubA)
Content-type: text/html; CHARSET=US-ASCII
Content-transfer-encoding: 7BIT

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
  <meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Darren J Moffat wrote:
<blockquote cite="mid:4AAE00C9.8090303@Sun.COM" type="cite">[...]<br>
OBP already stores Solaris specific properties and has Sun system and
Solaris specific code and properties [at the very least for WAN boot
but there was E10k specific stuff there too in the past and I'm sure
there is more].&nbsp; So I really don't see why this is an issue. It reads
to me like it can be done but someone doesn't want to do it that way,
is that correct ?&nbsp; Or is there a technical reason why it can't be done
?
  <br>
</blockquote>
<br>
What the Solaris iSCSI team is asking for is for us to make the Solaris
internal path part of the boot arguments. Currently (per FWARC
2008/466), the boot string on iSCSI looks like:<br>
<br>
<font color="#3333ff">/pci@500/pci@0/pci@8/network@0:iscsi-target-ip=129.148.67.188,host-ip=129.148.67.111,iscsi-target-name=iqn.1986-03.com.sun:02:43644186-e5bb-41f2-8b8e-f34be1afaebc&nbsp;</font>
<br>
<br>
What the Solaris team is asking for is to add the Solaris internal root
disk path to the above arguments. As I recall, they want to add an
additional argument:<br>
<br>
<font color="#6633ff"><a class="moz-txt-link-abbreviated" href="mailto:iscsi-phy-path=/iscsi/ssd@0000iqn.1986-03.com.sun:02:43644186-e5bb-41f2-8b8e-f34be1afaebc0001:a">iscsi-phy-path=/iscsi/ssd@0000iqn.1986-03.com.sun:02:43644186-e5bb-41f2-8b8e-f34be1afaebc0001:a</a></font>
.<br>
<br>
Aside from the architectural concerns (OBP has no business even knowing
that it is Solaris being booted, let alone how Solaris was configured
internally at the time Solaris was installed), most of the above is
redundant information. There is also a concern about path lengths - if
the boot string exceeds 256 bytes, life gets more painful (we have ways
of handling it, but they aren't pleasant). The Solaris iSCSI path can
be disassembled as follows:<br>
<br>
<font color="#cc66cc">/iscsi<font color="#6633ff">/</font></font><font
 color="#6633ff">ssd</font>@<font color="#33ff33">0000</font><font
 color="#cc9933">iqn.1986-03.com.sun:02:43644186-e5bb-41f2-8b8e-f34be1afaebc</font><font
 color="#339999">0001</font>,<font color="#ff0000">0</font>:a<br>
<ol>
  <li><font color="#cc66cc">/iscsi</font></li>
  <li><font color="#6633ff">/ssd@</font></li>
  <li><font color="#33ff33">0000</font></li>
  <li><font color="#cc9933">iqn.1986-03.com.sun:02:43644186-e5bb-41f2-8b8e-f34be1afaebc</font></li>
  <li><font color="#339999">0001</font></li>
  <li><font color="#ff0000">0</font></li>
  <li><font color="#ff0000"><font color="#000000">:a</font></font></li>
</ol>
Item (1) is a constant. Item (2) can be either "ssd" or "disk",
although I have yet to determine what conditions can result in the
second case being configured or how to force it. Item three (the ISID)
is effectively a constant. Item 4 is that iSCSI target name, available
through /chosen properties. Item (5) is the Target Portal Group Tag,
which at the moment is not provided by OBP, but can be determined
during the iSCSI login process. Item 6 is the Lun, which is provided by
OBP, and item 7 is the partition, which is also provided by OBP.<br>
<br>
In other words, except item (2), all of the above information is either
provided by OBP, is constant, or can be derived at runtime. I have yet
to figure out how item (2) is variable, and whether we need to support
bootability from the configuration which is different. Over the last
week or so, Jan has been discussing this with the iSCSI boot iteam
their options on this. At the moment, the iSCSI boot iteam is setting
the bootpath string in the boot_archive, similar to what is done in
x86, and Jan has objections to that.<br>
<br>
An additional concern is that an alternate (I would say preferred) boot
command form is:<br>
<br>
<font color="#3333ff">boot net:dhcp</font><br>
<br>
In this case, the iscsi information is provided by the DHCP server per
RFC 4173, with the ROOTPATH option. This option does not include the
possibility of any Solaris internal configuration information, so there
is nowhere to add the information desired.<br>
<br>
</body>
</html>

--Boundary_(ID_MkJpZb4LqsCEF4m7p9jubA)--

From Nicolas.Williams@sun.com Mon Sep 14 11:45:03 2009
Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n8EIj2AL026397
	for <psarc-ext@sac.sfbay.sun.com>; Mon, 14 Sep 2009 11:45:03 -0700 (PDT)
Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74])
	by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n8EIilgU020013;
	Mon, 14 Sep 2009 19:44:55 +0100 (BST)
Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by
 nwk-avmta-1.sfbay.Sun.COM
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KPZ00D035ESZ600@nwk-avmta-1.sfbay.Sun.COM>; Mon,
 14 Sep 2009 11:44:52 -0700 (PDT)
Received: from binky.Central.Sun.COM ([129.153.128.104])
 by nwk-avmta-1.sfbay.Sun.COM
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KPZ00DFD5ERB010@nwk-avmta-1.sfbay.Sun.COM>; Mon,
 14 Sep 2009 11:44:52 -0700 (PDT)
Received: from binky.Central.Sun.COM (localhost [127.0.0.1])
	by binky.Central.Sun.COM (8.14.3+Sun/8.14.3) with ESMTP id n8EIfGLb016522;
 Mon, 14 Sep 2009 13:41:16 -0500 (CDT)
Received: (from nw141292@localhost)
	by binky.Central.Sun.COM (8.14.3+Sun/8.14.3/Submit) id n8EIfGGt016521; Mon,
 14 Sep 2009 13:41:16 -0500 (CDT)
Date: Mon, 14 Sep 2009 13:41:16 -0500
From: Nicolas Williams <Nicolas.Williams@sun.com>
Subject: Re: Add bootpath into Solaris Sparc BootArchive for iSCSI boot
 [PSARC/2009/480 Self Review]
In-reply-to: <4AAA6D3E.5040506@sun.com>
To: Jack Meng <Jack.Meng@sun.com>
Cc: "Mark A. Carlson" <Mark.Carlson@sun.com>,
        Darren J Moffat <Darren.Moffat@sun.com>, PSARC-ext@sun.com,
        Bart Smaalders <Bart.Smaalders@sun.com>, Grant.Zhang@sun.com,
        Jan Setje-Eilers <Jan.Setje-Eilers@sun.com>,
        Tarl Neustaedter <Tarl.Neustaedter@sun.com>,
        David Kahn <David.Kahn@sun.com>
Message-id: <20090914184116.GE1033@Sun.COM>
MIME-version: 1.0
Content-type: text/plain; charset=us-ascii
Content-transfer-encoding: 7BIT
Content-disposition: inline
X-PMX-Version: 5.4.1.325704
References: <200909102101.n8AL1Gam029312@sac.sfbay.sun.com>
 <4AA96EED.7030206@Sun.COM> <4AA9715E.2020805@sun.com>
 <4AAA0C2A.1040707@Sun.COM> <4AAA398E.4010800@sun.com>
 <4AAA6D3E.5040506@sun.com>
X-Authentication-warning: binky.Central.Sun.COM: nw141292 set sender to
 Nicolas.Williams@sun.com using -f
User-Agent: Mutt/1.5.7i
Status: RO
Content-Length: 2660

On Fri, Sep 11, 2009 at 11:31:10PM +0800, Jack Meng wrote:
> The project team has been trying different approaches to solve the 
> 'bootpath' issue,
> 
> 2. Store the bootpath in boot_archive as the existing mechanism to 
> support UFS booting in x86.
> 
> 3. Dynamically build the bootpath before mounting the root disk in 
> Solaris kernel.

How about a combination?  Since the issue is really about "which disk
driver to layer above iSCSI" why not: a) pick a default driver, b) allow
one to be specified in the boot archive or as a boot argument.

See below.

> ISSUES:
> The generated bootpath is not guaranteed to be same as the one used to 
> contain the rootfs. A typical
> iscsi boot path is in the form of,
> "/iscsi/ssd@0000iqn.1986-03.com.sun:2510.600a0b800049c94d00000000493c920b0001,0:a"
> 
> with following components,
> 
>  1. iscsi                    - the iscsi driver name
>  2. ssd                    - the disk driver name/alias
>  3. 0000                    - the initiator ISID
>  4. iqn.1986-03.com.sun:02:43644186-e5bb-41f2-8b8e-f34be1afaebc
>                        - the target name
>  5. 0001                    - the TargetPortalGroupTag
>  6. 0                        - lun number
>  7. :a                        - slice
> 
> Here component 2 and 5 can be different with different targets/settings.
> 
> For component 5, It is the TargetPortalGroupTag which can be set in
> target side. However a possible solution for this part could be
> limiting the TPGT value to a specific number for booting in delivery
> documentation, that will make the problem much simpler.

Er, how can the OBP not know about TPGT?  If the OS needs to know that,
and since it's not OS-specific, then the firmware should provide that
information.  If by oversight the OBP does not provide TPGT, then
defaulting its value plus allowing for a boot archive/option override
seems like a reasonable approach.

> For component 2,
> 
> iSCSI disk can be attached to either 'ssd' and 'sd' (disk), mainly but
> not totally decided by the MPxIO support of the device. For example,
> an user installed Solaris with root disk attached to 'ssd', and then
> the user changed the MPxIO setting in Solaris then the correct bootpah
> would be changed to "../disk@...".  The above makes that during boot
> time kernel has to try the possible 2 paths to find the root.

Just because of the driver selection you'd force the admin to put the
entire bootpath in the boot archive?  That seems to me like throwing the
baby out with the bathwater.

Why would choosing a default driver selection, and allowing boot
archive/option override, not be a good enough solution?

Nico
-- 

From Tarl.Neustaedter@sun.com Mon Sep 14 11:51:39 2009
Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n8EIpdtY021942
	for <psarc-ext@sac.sfbay.sun.com>; Mon, 14 Sep 2009 11:51:39 -0700 (PDT)
Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11])
	by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id n8EIpcNK007813
	for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Mon, 14 Sep 2009 11:51:39 -0700 (PDT)
Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by
 brm-avmta-1.central.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KPZ00D035Q1KF00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com
 (ORCPT PSARC-ext@sun.com); Mon, 14 Sep 2009 12:51:37 -0600 (MDT)
Received: from brmea-mail-2.sun.com ([192.18.98.43])
 by brm-avmta-1.central.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KPZ00JX45Q15KD0@brm-avmta-1.central.sun.com> for
 PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon,
 14 Sep 2009 12:51:37 -0600 (MDT)
Received: from fe-amer-10.sun.com ([192.18.109.80])
	by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n8EIpbKV009101	for
 <PSARC-ext@sun.com>; Mon, 14 Sep 2009 18:51:37 +0000 (GMT)
Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com
 (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul  2 2009))
 id <0KPZ00L00546NV00@mail-amer.sun.com> for PSARC-ext@sun.com
 (ORCPT PSARC-ext@sun.com); Mon, 14 Sep 2009 12:51:37 -0600 (MDT)
Received: from dhcp-ubur-189-199.East.Sun.COM ([unknown] [129.148.189.199])
 by mail-amer.sun.com
 (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul  2 2009))
 with ESMTPSA id <0KPZ00EAP5Q0O5F0@mail-amer.sun.com>; Mon,
 14 Sep 2009 12:51:37 -0600 (MDT)
Date: Mon, 14 Sep 2009 14:51:35 -0400
From: Tarl Neustaedter <Tarl.Neustaedter@sun.com>
Subject: Re: Add bootpath into Solaris Sparc BootArchive for iSCSI boot
 [PSARC/2009/480 Self Review]
In-reply-to: <20090914184116.GE1033@Sun.COM>
Sender: Tarl.Neustaedter@sun.com
To: Nicolas Williams <Nicolas.Williams@sun.com>
Cc: Jack Meng <Jack.Meng@sun.com>, "Mark A. Carlson" <Mark.Carlson@sun.com>,
        Darren J Moffat <Darren.Moffat@sun.com>, PSARC-ext@sun.com,
        Bart Smaalders <Bart.Smaalders@sun.com>, Grant.Zhang@sun.com,
        Jan Setje-Eilers <Jan.Setje-Eilers@sun.com>,
        David Kahn <David.Kahn@sun.com>
Message-id: <4AAE90B7.7020704@sun.com>
MIME-version: 1.0
Content-type: text/plain; CHARSET=US-ASCII; format=flowed
Content-transfer-encoding: 7BIT
X-PMX-Version: 5.4.1.325704
References: <200909102101.n8AL1Gam029312@sac.sfbay.sun.com>
 <4AA96EED.7030206@Sun.COM> <4AA9715E.2020805@sun.com>
 <4AAA0C2A.1040707@Sun.COM> <4AAA398E.4010800@sun.com>
 <4AAA6D3E.5040506@sun.com> <20090914184116.GE1033@Sun.COM>
User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812)
Status: RO
Content-Length: 275

Nicolas Williams wrote:
> [...]
> Er, how can the OBP not know about TPGT?

It's provided by the target during the login sequence. Potentially we 
should have an RFE for OBP to specify TPGT rather than accepting what 
the target tells us, but that's how it currently works.


From Nicolas.Williams@sun.com Mon Sep 14 12:12:58 2009
Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n8EJCuib016772
	for <psarc-ext@sac.sfbay.sun.com>; Mon, 14 Sep 2009 12:12:57 -0700 (PDT)
Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11])
	by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n8EJCaBu010356;
	Tue, 15 Sep 2009 03:12:46 +0800 (SGT)
Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by
 brm-avmta-1.central.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KPZ00F036P9N400@brm-avmta-1.central.sun.com>; Mon,
 14 Sep 2009 13:12:45 -0600 (MDT)
Received: from binky.Central.Sun.COM ([129.153.128.104])
 by brm-avmta-1.central.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KPZ00JKZ6P85GE0@brm-avmta-1.central.sun.com>; Mon,
 14 Sep 2009 13:12:44 -0600 (MDT)
Received: from binky.Central.Sun.COM (localhost [127.0.0.1])
	by binky.Central.Sun.COM (8.14.3+Sun/8.14.3) with ESMTP id n8EJ9CgJ016557;
 Mon, 14 Sep 2009 14:09:12 -0500 (CDT)
Received: (from nw141292@localhost)
	by binky.Central.Sun.COM (8.14.3+Sun/8.14.3/Submit) id n8EJ9CKx016556; Mon,
 14 Sep 2009 14:09:12 -0500 (CDT)
Date: Mon, 14 Sep 2009 14:09:12 -0500
From: Nicolas Williams <Nicolas.Williams@sun.com>
Subject: Re: Add bootpath into Solaris Sparc BootArchive for iSCSI boot
 [PSARC/2009/480 Self Review]
In-reply-to: <4AAE90B7.7020704@sun.com>
To: Tarl Neustaedter <Tarl.Neustaedter@sun.com>
Cc: Jack Meng <Jack.Meng@sun.com>, "Mark A. Carlson" <Mark.Carlson@sun.com>,
        Darren J Moffat <Darren.Moffat@sun.com>, PSARC-ext@sun.com,
        Bart Smaalders <Bart.Smaalders@sun.com>, Grant.Zhang@sun.com,
        Jan Setje-Eilers <Jan.Setje-Eilers@sun.com>,
        David Kahn <David.Kahn@sun.com>
Message-id: <20090914190912.GH1033@Sun.COM>
MIME-version: 1.0
Content-type: text/plain; charset=us-ascii
Content-transfer-encoding: 7BIT
Content-disposition: inline
X-PMX-Version: 5.4.1.325704
References: <200909102101.n8AL1Gam029312@sac.sfbay.sun.com>
 <4AA96EED.7030206@Sun.COM> <4AA9715E.2020805@sun.com>
 <4AAA0C2A.1040707@Sun.COM> <4AAA398E.4010800@sun.com>
 <4AAA6D3E.5040506@sun.com> <20090914184116.GE1033@Sun.COM>
 <4AAE90B7.7020704@sun.com>
X-Authentication-warning: binky.Central.Sun.COM: nw141292 set sender to
 Nicolas.Williams@sun.com using -f
User-Agent: Mutt/1.5.7i
Status: RO
Content-Length: 493

On Mon, Sep 14, 2009 at 02:51:35PM -0400, Tarl Neustaedter wrote:
> Nicolas Williams wrote:
> >[...]
> >Er, how can the OBP not know about TPGT?
> 
> It's provided by the target during the login sequence. Potentially we 
> should have an RFE for OBP to specify TPGT rather than accepting what 
> the target tells us, but that's how it currently works.

If it's provided by the target then the OBP should pass it to the OS.

That leaves the driver choice.  Anything wrong with what I proposed?

From Tarl.Neustaedter@sun.com Mon Sep 14 12:27:54 2009
Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n8EJRseo026551
	for <psarc-ext@sac.sfbay.sun.com>; Mon, 14 Sep 2009 12:27:54 -0700 (PDT)
Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11])
	by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n8EJRm6v061705
	for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Mon, 14 Sep 2009 13:27:53 -0600 (MDT)
Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by
 brm-avmta-1.central.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KPZ00H0L7EG5900@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com
 (ORCPT PSARC-ext@sun.com); Mon, 14 Sep 2009 13:27:52 -0600 (MDT)
Received: from brmea-mail-4.sun.com ([192.18.98.36])
 by brm-avmta-1.central.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KPZ00FJ77EFYU00@brm-avmta-1.central.sun.com> for
 PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon,
 14 Sep 2009 13:27:51 -0600 (MDT)
Received: from fe-amer-09.sun.com ([192.18.109.79])
	by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n8EJRpBA013542	for
 <PSARC-ext@sun.com>; Mon, 14 Sep 2009 19:27:51 +0000 (GMT)
Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com
 (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul  2 2009))
 id <0KPZ00G006UVP500@mail-amer.sun.com> for PSARC-ext@sun.com
 (ORCPT PSARC-ext@sun.com); Mon, 14 Sep 2009 13:27:51 -0600 (MDT)
Received: from Plata.local ([unknown] [12.172.66.11])
 by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit
 (built Jul  2 2009)) with ESMTPSA id <0KPZ00E6I7EADW60@mail-amer.sun.com>; Mon,
 14 Sep 2009 13:27:48 -0600 (MDT)
Date: Mon, 14 Sep 2009 15:27:45 -0400
From: Tarl Neustaedter <Tarl.Neustaedter@sun.com>
Subject: Re: Add bootpath into Solaris Sparc BootArchive for iSCSI boot
 [PSARC/2009/480 Self Review]
In-reply-to: <20090914190912.GH1033@Sun.COM>
Sender: Tarl.Neustaedter@sun.com
To: Nicolas Williams <Nicolas.Williams@sun.com>
Cc: Jack Meng <Jack.Meng@sun.com>, "Mark A. Carlson" <Mark.Carlson@sun.com>,
        Darren J Moffat <Darren.Moffat@sun.com>, PSARC-ext@sun.com,
        Bart Smaalders <Bart.Smaalders@sun.com>, Grant.Zhang@sun.com,
        Jan Setje-Eilers <Jan.Setje-Eilers@sun.com>,
        David Kahn <David.Kahn@sun.com>
Message-id: <4AAE9931.8020201@sun.com>
MIME-version: 1.0
Content-type: text/plain; CHARSET=US-ASCII; format=flowed
Content-transfer-encoding: 7BIT
X-PMX-Version: 5.4.1.325704
References: <200909102101.n8AL1Gam029312@sac.sfbay.sun.com>
 <4AA96EED.7030206@Sun.COM> <4AA9715E.2020805@sun.com>
 <4AAA0C2A.1040707@Sun.COM> <4AAA398E.4010800@sun.com>
 <4AAA6D3E.5040506@sun.com> <20090914184116.GE1033@Sun.COM>
 <4AAE90B7.7020704@sun.com> <20090914190912.GH1033@Sun.COM>
User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812)
Status: RO
Content-Length: 1213

Nicolas Williams wrote:
> [...]
> If it's provided by the target then the OBP should pass it to the OS.
>   

If that's the consensus, we'll modify FWARC 2008/466 to accommodate - 
probably create a new chosen property to export it. Or Solaris could 
fetch the information by itself.

> That leaves the driver choice.  Anything wrong with what I proposed?
>   

I have no opinion about putting it in the boot_archive. I disagree with 
requiring OBP having to know which driver Solaris is going to use - the 
boot architecture (til now) has been that OBP gets enough information to 
read the secondary booter (ufsboot or zfsboot in this case), and 
transfers control, providing the information it has received. Currently 
the secondary booter reads in the boot_archive, and then transfers 
control to that. The knowledge of which driver to use within Solaris 
isn't needed until after this stage - so I don't think it belongs in the 
boot command in the first place.

And I don't see how to fit it in with the DHCP options. The string 
defined by RFC 4173 does not allow for any expansion of arguments, 
nowhere to fit more information unless we're going to have SPARC Solaris 
requiring special options in DHCP.


From bart.smaalders@sun.com Mon Sep 14 12:56:35 2009
Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n8EJuYH1024329
	for <psarc-ext@sac.sfbay.sun.com>; Mon, 14 Sep 2009 12:56:34 -0700 (PDT)
Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74])
	by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n8EJuHO7004649;
	Mon, 14 Sep 2009 20:56:27 +0100 (BST)
Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by
 nwk-avmta-1.sfbay.Sun.COM
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KPZ00N0B8Q10A00@nwk-avmta-1.sfbay.Sun.COM>; Mon,
 14 Sep 2009 12:56:25 -0700 (PDT)
Received: from zion.sfbay.sun.com ([129.146.17.75])
 by nwk-avmta-1.sfbay.Sun.COM
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KPZ00KU28Q19A50@nwk-avmta-1.sfbay.Sun.COM>; Mon,
 14 Sep 2009 12:56:25 -0700 (PDT)
Received: from [129.146.228.109] (cyber.SFBay.Sun.COM [129.146.228.109])
	by zion.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n8EJtUff008581; Mon,
 14 Sep 2009 19:55:30 +0000 (GMT)
Date: Mon, 14 Sep 2009 12:54:22 -0700
From: Bart Smaalders <bart.smaalders@sun.com>
Subject: Re: Add bootpath into Solaris Sparc BootArchive for iSCSI boot
 [PSARC/2009/480 Self Review]
In-reply-to: <4AAE9931.8020201@sun.com>
To: Tarl Neustaedter <Tarl.Neustaedter@sun.com>
Cc: Nicolas Williams <Nicolas.Williams@sun.com>, Jack Meng <Jack.Meng@sun.com>,
        "Mark A. Carlson" <Mark.Carlson@sun.com>,
        Darren J Moffat <Darren.Moffat@sun.com>, PSARC-ext@sun.com,
        Grant.Zhang@sun.com, Jan Setje-Eilers <Jan.Setje-Eilers@sun.com>,
        David Kahn <David.Kahn@sun.com>
Message-id: <4AAE9F6E.3010902@Sun.COM>
Organization: Sun Microsystems
MIME-version: 1.0
Content-type: text/plain; charset=ISO-8859-1; format=flowed
Content-transfer-encoding: 7BIT
X-PMX-Version: 5.4.1.325704
References: <200909102101.n8AL1Gam029312@sac.sfbay.sun.com>
 <4AA96EED.7030206@Sun.COM> <4AA9715E.2020805@sun.com>
 <4AAA0C2A.1040707@Sun.COM> <4AAA398E.4010800@sun.com>
 <4AAA6D3E.5040506@sun.com> <20090914184116.GE1033@Sun.COM>
 <4AAE90B7.7020704@sun.com> <20090914190912.GH1033@Sun.COM>
 <4AAE9931.8020201@sun.com>
User-Agent: Thunderbird 2.0.0.22 (X11/20090804)
Status: RO
Content-Length: 714

Tarl Neustaedter wrote:
> The knowledge of which driver to use within Solaris
> isn't needed until after this stage - so I don't think it belongs in the 
> boot command in the first place.

This is correct... and storing such driver info somewhere where a
special program needs to update it is just plain broken.

Solaris should be passed the _external_ info about its boot drive -
and then should dynamically figure out how to talk to that external
device.  Any other approach will break when drivers change,
paths to devices change, etc.

- Bart

-- 
Bart Smaalders			Solaris Kernel Performance
barts@cyber.eng.sun.com		http://blogs.sun.com/barts
"You will contribute more with mercurial than with thunderbird."

From Nicolas.Williams@sun.com Mon Sep 14 12:58:06 2009
Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n8EJw6Mx027808
	for <psarc-ext@sac.sfbay.sun.com>; Mon, 14 Sep 2009 12:58:06 -0700 (PDT)
Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11])
	by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id n8EJw0oU006345;
	Mon, 14 Sep 2009 12:58:01 -0700 (PDT)
Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by
 brm-avmta-1.central.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KPZ00K058SN6T00@brm-avmta-1.central.sun.com>; Mon,
 14 Sep 2009 13:57:59 -0600 (MDT)
Received: from binky.Central.Sun.COM ([129.153.128.104])
 by brm-avmta-1.central.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KPZ00FM58SMYI30@brm-avmta-1.central.sun.com>; Mon,
 14 Sep 2009 13:57:58 -0600 (MDT)
Received: from binky.Central.Sun.COM (localhost [127.0.0.1])
	by binky.Central.Sun.COM (8.14.3+Sun/8.14.3) with ESMTP id n8EJsPJV016586;
 Mon, 14 Sep 2009 14:54:25 -0500 (CDT)
Received: (from nw141292@localhost)
	by binky.Central.Sun.COM (8.14.3+Sun/8.14.3/Submit) id n8EJsPT6016585; Mon,
 14 Sep 2009 14:54:25 -0500 (CDT)
Date: Mon, 14 Sep 2009 14:54:25 -0500
From: Nicolas Williams <Nicolas.Williams@sun.com>
Subject: Re: Add bootpath into Solaris Sparc BootArchive for iSCSI boot
 [PSARC/2009/480 Self Review]
In-reply-to: <4AAE9931.8020201@sun.com>
To: Tarl Neustaedter <Tarl.Neustaedter@sun.com>
Cc: Jack Meng <Jack.Meng@sun.com>, "Mark A. Carlson" <Mark.Carlson@sun.com>,
        Darren J Moffat <Darren.Moffat@sun.com>, PSARC-ext@sun.com,
        Bart Smaalders <Bart.Smaalders@sun.com>, Grant.Zhang@sun.com,
        Jan Setje-Eilers <Jan.Setje-Eilers@sun.com>,
        David Kahn <David.Kahn@sun.com>
Message-id: <20090914195425.GI1033@Sun.COM>
MIME-version: 1.0
Content-type: text/plain; charset=us-ascii
Content-transfer-encoding: 7BIT
Content-disposition: inline
X-PMX-Version: 5.4.1.325704
References: <200909102101.n8AL1Gam029312@sac.sfbay.sun.com>
 <4AA96EED.7030206@Sun.COM> <4AA9715E.2020805@sun.com>
 <4AAA0C2A.1040707@Sun.COM> <4AAA398E.4010800@sun.com>
 <4AAA6D3E.5040506@sun.com> <20090914184116.GE1033@Sun.COM>
 <4AAE90B7.7020704@sun.com> <20090914190912.GH1033@Sun.COM>
 <4AAE9931.8020201@sun.com>
X-Authentication-warning: binky.Central.Sun.COM: nw141292 set sender to
 Nicolas.Williams@sun.com using -f
User-Agent: Mutt/1.5.7i
Status: RO
Content-Length: 1355

On Mon, Sep 14, 2009 at 03:27:45PM -0400, Tarl Neustaedter wrote:
> Nicolas Williams wrote:
> >[...]
> >If it's provided by the target then the OBP should pass it to the OS.
> >  
> 
> If that's the consensus, we'll modify FWARC 2008/466 to accommodate - 
> probably create a new chosen property to export it. Or Solaris could 
> fetch the information by itself.
> 
> >That leaves the driver choice.  Anything wrong with what I proposed?
> 
> I have no opinion about putting it in the boot_archive. I disagree with 
> requiring OBP having to know which driver Solaris is going to use - the 
> ...

I never said that the OBP should know which driver Solaris should use.

I said that the choice of driver should: a) be defaulted, b) be settable
via boot archive and/or boot option.  Administering a choice of driver
via boot archive seems reasonable to me.  Storing the full boot path in
the boot archive is not (since it is redundant, causing yet one more
place to update when a system's root migrates).

> And I don't see how to fit it in with the DHCP options. The string 
> defined by RFC 4173 does not allow for any expansion of arguments, 
> nowhere to fit more information unless we're going to have SPARC Solaris 
> requiring special options in DHCP.

I didn't mention DHCP, but it's certainly an option: just add yet
another DHCP option.

Nico
-- 

From Jack.Meng@sun.com Tue Sep 15 02:23:48 2009
Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n8F9Nk04019789
	for <psarc-ext@sac.sfbay.sun.com>; Tue, 15 Sep 2009 02:23:47 -0700 (PDT)
Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6])
	by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n8F9NdOo011881
	for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Tue, 15 Sep 2009 10:23:46 +0100 (BST)
Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by
 nwk-avmta-2.sfbay.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KQ000L0VA3KT800@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com
 (ORCPT PSARC-ext@sun.com); Tue, 15 Sep 2009 02:23:44 -0700 (PDT)
Received: from sineb-mail-2.sun.com ([192.18.19.7])
 by nwk-avmta-2.sfbay.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KQ0004E0A3JT5E0@nwk-avmta-2.sfbay.sun.com> for
 PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue,
 15 Sep 2009 02:23:43 -0700 (PDT)
Received: from fe-apac-06.sun.com
 (fe-apac-06.sun.com [192.18.19.177] (may be forged))
	by sineb-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n8F9NguL002048	for
 <PSARC-ext@sun.com>; Tue, 15 Sep 2009 09:23:42 +0000 (GMT)
Received: from conversion-daemon.mail-apac.sun.com by mail-apac.sun.com
 (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul  2 2009))
 id <0KQ000I009RACA00@mail-apac.sun.com> for PSARC-ext@sun.com
 (ORCPT PSARC-ext@sun.com); Tue, 15 Sep 2009 17:23:42 +0800 (SGT)
Received: from [129.158.144.121] ([unknown] [129.158.144.121])
 by mail-apac.sun.com
 (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul  2 2009))
 with ESMTPSA id <0KQ000D9SA3HJHG0@mail-apac.sun.com>; Tue,
 15 Sep 2009 17:23:42 +0800 (SGT)
Date: Tue, 15 Sep 2009 17:21:32 +0800
From: Jack <Jack.Meng@sun.com>
Subject: Re: Add bootpath into Solaris Sparc BootArchive for iSCSI boot
 [PSARC/2009/480 Self Review]
In-reply-to: <20090914195425.GI1033@Sun.COM>
Sender: Jack.Meng@sun.com
To: Nicolas Williams <Nicolas.Williams@sun.com>
Cc: Tarl Neustaedter <Tarl.Neustaedter@sun.com>,
        "Mark A. Carlson" <Mark.Carlson@sun.com>,
        Darren J Moffat <Darren.Moffat@sun.com>, PSARC-ext@sun.com,
        Bart Smaalders <Bart.Smaalders@sun.com>, Grant.Zhang@sun.com,
        Jan Setje-Eilers <Jan.Setje-Eilers@sun.com>,
        David Kahn <David.Kahn@sun.com>
Message-id: <4AAF5C9C.7030702@sun.com>
MIME-version: 1.0
Content-type: multipart/alternative;
 boundary="Boundary_(ID_UHHI8I66GND+XhAoDzKmjg)"
X-PMX-Version: 5.4.1.325704
References: <200909102101.n8AL1Gam029312@sac.sfbay.sun.com>
 <4AA96EED.7030206@Sun.COM> <4AA9715E.2020805@sun.com>
 <4AAA0C2A.1040707@Sun.COM> <4AAA398E.4010800@sun.com>
 <4AAA6D3E.5040506@sun.com> <20090914184116.GE1033@Sun.COM>
 <4AAE90B7.7020704@sun.com> <20090914190912.GH1033@Sun.COM>
 <4AAE9931.8020201@sun.com> <20090914195425.GI1033@Sun.COM>
User-Agent: Thunderbird 2.0.0.23 (Windows/20090812)
Status: RO
Content-Length: 5826

This is a multi-part message in MIME format.

--Boundary_(ID_UHHI8I66GND+XhAoDzKmjg)
Content-type: text/plain; CHARSET=US-ASCII; format=flowed
Content-transfer-encoding: 7BIT

Nicolas Williams wrote:
> On Mon, Sep 14, 2009 at 03:27:45PM -0400, Tarl Neustaedter wrote:
>   
>> Nicolas Williams wrote:
>>     
>>> [...]
>>> If it's provided by the target then the OBP should pass it to the OS.
>>>  
>>>       
>> If that's the consensus, we'll modify FWARC 2008/466 to accommodate - 
>> probably create a new chosen property to export it. Or Solaris could 
>> fetch the information by itself.
>>     
Solaris setup the bootpath before triggering discovery - logging to the 
iSCSI target
for this case, therefore a correct value is very helpful here. As it is 
already available during the
logging process made by OBP, to pass it to OS would be a good practice.
>>     
>>> That leaves the driver choice.  Anything wrong with what I proposed?
>>>       
>> I have no opinion about putting it in the boot_archive. I disagree with 
>> requiring OBP having to know which driver Solaris is going to use - the 
>> ...
>>     
>
> I never said that the OBP should know which driver Solaris should use.
>
> I said that the choice of driver should: a) be defaulted, b) be settable
> via boot archive and/or boot option.  Administering a choice of driver
> via boot archive seems reasonable to me.  Storing the full boot path in
> the boot archive is not (since it is redundant, causing yet one more
> place to update when a system's root migrates).
>   
This should be an working approach, I just feel it might be a bit 
complex as probably needs to figure
out the administrative method, which may involve the installer, and 
other utilities.
Since there are 2 options only I think that it is a practical way to do 
the selection based on trail-and-error
with the default 'ssd' driver. If the TPGT value is available, the worst 
case would be one false attempt to
match the path, which is proven with trivial impact to the boot process 
as the relogin and rediscovery will
not take place.

Best regards,
Jack

>   
>> And I don't see how to fit it in with the DHCP options. The string 
>> defined by RFC 4173 does not allow for any expansion of arguments, 
>> nowhere to fit more information unless we're going to have SPARC Solaris 
>> requiring special options in DHCP.
>>     
>
> I didn't mention DHCP, but it's certainly an option: just add yet
> another DHCP option.
>
> Nico
>   


--Boundary_(ID_UHHI8I66GND+XhAoDzKmjg)
Content-type: text/html; CHARSET=US-ASCII
Content-transfer-encoding: 7BIT

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
  <meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Nicolas Williams wrote:
<blockquote cite="mid:20090914195425.GI1033@Sun.COM" type="cite">
  <pre wrap="">On Mon, Sep 14, 2009 at 03:27:45PM -0400, Tarl Neustaedter wrote:
  </pre>
  <blockquote type="cite">
    <pre wrap="">Nicolas Williams wrote:
    </pre>
    <blockquote type="cite">
      <pre wrap="">[...]
If it's provided by the target then the OBP should pass it to the OS.
 
      </pre>
    </blockquote>
    <pre wrap="">If that's the consensus, we'll modify FWARC 2008/466 to accommodate - 
probably create a new chosen property to export it. Or Solaris could 
fetch the information by itself.
    </pre>
  </blockquote>
</blockquote>
Solaris setup the bootpath before triggering discovery - logging to the
iSCSI target<br>
for this case, therefore a correct value is very helpful here. As it is
already available during the<br>
logging process made by OBP, to pass it to OS would be a good practice.<br>
<blockquote cite="mid:20090914195425.GI1033@Sun.COM" type="cite">
  <blockquote type="cite">
    <pre wrap="">
    </pre>
    <blockquote type="cite">
      <pre wrap="">That leaves the driver choice.  Anything wrong with what I proposed?
      </pre>
    </blockquote>
    <pre wrap="">I have no opinion about putting it in the boot_archive. I disagree with 
requiring OBP having to know which driver Solaris is going to use - the 
...
    </pre>
  </blockquote>
  <pre wrap=""><!---->
I never said that the OBP should know which driver Solaris should use.

I said that the choice of driver should: a) be defaulted, b) be settable
via boot archive and/or boot option.  Administering a choice of driver
via boot archive seems reasonable to me.  Storing the full boot path in
the boot archive is not (since it is redundant, causing yet one more
place to update when a system's root migrates).
  </pre>
</blockquote>
This should be an working approach, I just feel it might be a bit
complex as probably needs to figure<br>
out the administrative method, which may involve the installer, and
other utilities.<br>
Since there are 2 options only I think that it is a practical way to do
the selection based on trail-and-error<br>
with the default 'ssd' driver. If the TPGT value is available, the
worst case would be one false attempt to<br>
match the path, which is proven with trivial impact to the boot process
as the relogin and rediscovery will<br>
not take place.<br>
<br>
Best regards,<br>
Jack<br>
<br>
<blockquote cite="mid:20090914195425.GI1033@Sun.COM" type="cite">
  <pre wrap="">
  </pre>
  <blockquote type="cite">
    <pre wrap="">And I don't see how to fit it in with the DHCP options. The string 
defined by RFC 4173 does not allow for any expansion of arguments, 
nowhere to fit more information unless we're going to have SPARC Solaris 
requiring special options in DHCP.
    </pre>
  </blockquote>
  <pre wrap=""><!---->
I didn't mention DHCP, but it's certainly an option: just add yet
another DHCP option.

Nico
  </pre>
</blockquote>
<br>
</body>
</html>

--Boundary_(ID_UHHI8I66GND+XhAoDzKmjg)--

From Nicolas.Williams@sun.com Tue Sep 15 08:07:43 2009
Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n8FF7gLM008922
	for <psarc-ext@sac.sfbay.sun.com>; Tue, 15 Sep 2009 08:07:43 -0700 (PDT)
Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11])
	by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n8FF7Uav025772;
	Tue, 15 Sep 2009 23:07:32 +0800 (SGT)
Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by
 brm-avmta-1.central.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KQ000315Q0J8F00@brm-avmta-1.central.sun.com>; Tue,
 15 Sep 2009 09:07:31 -0600 (MDT)
Received: from binky.Central.Sun.COM ([129.153.128.104])
 by brm-avmta-1.central.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KQ0002YMQ0IBZ00@brm-avmta-1.central.sun.com>; Tue,
 15 Sep 2009 09:07:31 -0600 (MDT)
Received: from binky.Central.Sun.COM (localhost [127.0.0.1])
	by binky.Central.Sun.COM (8.14.3+Sun/8.14.3) with ESMTP id n8FF3xsK017451;
 Tue, 15 Sep 2009 10:03:59 -0500 (CDT)
Received: (from nw141292@localhost)
	by binky.Central.Sun.COM (8.14.3+Sun/8.14.3/Submit) id n8FF3xV1017450; Tue,
 15 Sep 2009 10:03:59 -0500 (CDT)
Date: Tue, 15 Sep 2009 10:03:59 -0500
From: Nicolas Williams <Nicolas.Williams@sun.com>
Subject: Re: Add bootpath into Solaris Sparc BootArchive for iSCSI boot
 [PSARC/2009/480 Self Review]
In-reply-to: <4AAF5C9C.7030702@sun.com>
To: Jack <Jack.Meng@sun.com>
Cc: Tarl Neustaedter <Tarl.Neustaedter@sun.com>,
        "Mark A. Carlson" <Mark.Carlson@sun.com>,
        Darren J Moffat <Darren.Moffat@sun.com>, PSARC-ext@sun.com,
        Bart Smaalders <Bart.Smaalders@sun.com>, Grant.Zhang@sun.com,
        Jan Setje-Eilers <Jan.Setje-Eilers@sun.com>,
        David Kahn <David.Kahn@sun.com>
Message-id: <20090915150358.GR1033@Sun.COM>
MIME-version: 1.0
Content-type: text/plain; charset=us-ascii
Content-transfer-encoding: 7BIT
Content-disposition: inline
X-PMX-Version: 5.4.1.325704
References: <4AA9715E.2020805@sun.com> <4AAA0C2A.1040707@Sun.COM>
 <4AAA398E.4010800@sun.com> <4AAA6D3E.5040506@sun.com>
 <20090914184116.GE1033@Sun.COM> <4AAE90B7.7020704@sun.com>
 <20090914190912.GH1033@Sun.COM> <4AAE9931.8020201@sun.com>
 <20090914195425.GI1033@Sun.COM> <4AAF5C9C.7030702@sun.com>
X-Authentication-warning: binky.Central.Sun.COM: nw141292 set sender to
 Nicolas.Williams@sun.com using -f
User-Agent: Mutt/1.5.7i
Status: RO
Content-Length: 2342

On Tue, Sep 15, 2009 at 05:21:32PM +0800, Jack wrote:
> Nicolas Williams wrote:
> >On Mon, Sep 14, 2009 at 03:27:45PM -0400, Tarl Neustaedter wrote:

> Solaris setup the bootpath before triggering discovery - logging to
> the iSCSI target for this case, therefore a correct value is very
> helpful here. As it is already available during the logging process
> made by OBP, to pass it to OS would be a good practice.

Discovery needs to be done very early in boot.  So if you want to add a
DHCP option for TPGT discovery, then it has to be either implemented by
the OBP or implemented in the kernel and modules in the boot archive.

I think you can default TPGT for now and file the RFE for additional OBP
functionality or additional kernel DHCP functionality.

> >I never said that the OBP should know which driver Solaris should use.
> >
> >I said that the choice of driver should: a) be defaulted, b) be settable
> >via boot archive and/or boot option.  Administering a choice of driver
> >via boot archive seems reasonable to me.  Storing the full boot path in
> >the boot archive is not (since it is redundant, causing yet one more
> >place to update when a system's root migrates).
> >  
> This should be an working approach, I just feel it might be a bit
> complex as probably needs to figure out the administrative method,
> which may involve the installer, and other utilities.

Your approach has that problem too, but worse: in your scheme the boot
archive needs an update every time the system's boot path migrates from
one target to another, whereas that is not so in my scheme.  If all you
store in the boot archive is the choice of driver, and the default is a
reasonable one, then very few users will need to change it, and those
that do will need to change it just once.

> Since there are 2 options only I think that it is a practical way to
> do the selection based on trail-and-error with the default 'ssd'
> driver. If the TPGT value is available, the worst case would be one
> false attempt to match the path, which is proven with trivial impact
> to the boot process as the relogin and rediscovery will not take
> place.

How does the trial-and-error approach work?  Can the kernel detect that
it used the wrong driver?  If so, great!

As for TPGT, that's a 16-bit field.  You can't really try all TPGT
values.

Nico
-- 

From Grant.Zhang@SUN.COM Tue Sep 15 08:47:30 2009
Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n8FFlTGZ019368
	for <psarc-ext@sac.sfbay.sun.com>; Tue, 15 Sep 2009 08:47:29 -0700 (PDT)
Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11])
	by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n8FFlTAd047218
	for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Tue, 15 Sep 2009 09:47:29 -0600 (MDT)
Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by
 brm-avmta-1.central.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KQ000709RV44E00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com
 (ORCPT PSARC-ext@sun.com); Tue, 15 Sep 2009 09:47:28 -0600 (MDT)
Received: from sineb-mail-1.sun.com ([192.18.19.6])
 by brm-avmta-1.central.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KQ0002HDRV3BP40@brm-avmta-1.central.sun.com> for
 PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue,
 15 Sep 2009 09:47:28 -0600 (MDT)
Received: from fe-apac-05.sun.com
 (fe-apac-05.sun.com [192.18.19.176] (may be forged))
	by sineb-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n8FFlQ5B004573	for
 <PSARC-ext@sun.com>; Tue, 15 Sep 2009 15:47:26 +0000 (GMT)
Received: from conversion-daemon.mail-apac.sun.com by mail-apac.sun.com
 (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul  2 2009))
 id <0KQ000I00RO24N00@mail-apac.sun.com> for PSARC-ext@sun.com
 (ORCPT PSARC-ext@sun.com); Tue, 15 Sep 2009 23:47:26 +0800 (SGT)
Received: from grant.local ([unknown] [129.150.225.172])
 by mail-apac.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit
 (built Jul  2 2009)) with ESMTPSA id <0KQ000JJ4RUYZS40@mail-apac.sun.com>; Tue,
 15 Sep 2009 23:47:26 +0800 (SGT)
Date: Tue, 15 Sep 2009 08:47:21 -0700
From: Grant Zhang <Grant.Zhang@SUN.COM>
Subject: Re: Add bootpath into Solaris Sparc BootArchive for iSCSI boot
 [PSARC/2009/480 Self Review]
In-reply-to: <20090915150358.GR1033@Sun.COM>
Sender: Grant.Zhang@SUN.COM
To: Nicolas Williams <Nicolas.Williams@SUN.COM>
Cc: Jack <Jack.Meng@SUN.COM>, Tarl Neustaedter <Tarl.Neustaedter@SUN.COM>,
        "Mark A. Carlson" <Mark.Carlson@SUN.COM>,
        Darren J Moffat <Darren.Moffat@SUN.COM>, PSARC-ext@SUN.COM,
        Bart Smaalders <Bart.Smaalders@SUN.COM>,
        Jan Setje-Eilers <Jan.Setje-Eilers@SUN.COM>,
        David Kahn <David.Kahn@SUN.COM>
Message-id: <4AAFB709.5010209@sun.com>
MIME-version: 1.0
Content-type: text/plain; format=flowed; charset=UTF-8
Content-transfer-encoding: 8BIT
X-PMX-Version: 5.4.1.325704
References: <4AA9715E.2020805@sun.com> <4AAA0C2A.1040707@Sun.COM>
 <4AAA398E.4010800@sun.com> <4AAA6D3E.5040506@sun.com>
 <20090914184116.GE1033@Sun.COM> <4AAE90B7.7020704@sun.com>
 <20090914190912.GH1033@Sun.COM> <4AAE9931.8020201@sun.com>
 <20090914195425.GI1033@Sun.COM> <4AAF5C9C.7030702@sun.com>
 <20090915150358.GR1033@Sun.COM>
User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; zh-CN; rv:1.9.1.1)
 Gecko/20090715 Thunderbird/3.0b3
Status: RO
Content-Length: 3189

于 09-9-15 上午8:03, Nicolas Williams 写道:
> On Tue, Sep 15, 2009 at 05:21:32PM +0800, Jack wrote:
>    
>> Nicolas Williams wrote:
>>      
>>> On Mon, Sep 14, 2009 at 03:27:45PM -0400, Tarl Neustaedter wrote:
>>>        
>    
>> Solaris setup the bootpath before triggering discovery - logging to
>> the iSCSI target for this case, therefore a correct value is very
>> helpful here. As it is already available during the logging process
>> made by OBP, to pass it to OS would be a good practice.
>>      
> Discovery needs to be done very early in boot.  So if you want to add a
> DHCP option for TPGT discovery, then it has to be either implemented by
> the OBP or implemented in the kernel and modules in the boot archive.
>
> I think you can default TPGT for now and file the RFE for additional OBP
> functionality or additional kernel DHCP functionality.
>    
Agreed. The project team is in discussion with the OBP team for a RFE on 
passing the TPGT value.
>    
>>> I never said that the OBP should know which driver Solaris should use.
>>>
>>> I said that the choice of driver should: a) be defaulted, b) be settable
>>> via boot archive and/or boot option.  Administering a choice of driver
>>> via boot archive seems reasonable to me.  Storing the full boot path in
>>> the boot archive is not (since it is redundant, causing yet one more
>>> place to update when a system's root migrates).
>>>
>>>        
>> This should be an working approach, I just feel it might be a bit
>> complex as probably needs to figure out the administrative method,
>> which may involve the installer, and other utilities.
>>      
> Your approach has that problem too, but worse: in your scheme the boot
> archive needs an update every time the system's boot path migrates from
> one target to another, whereas that is not so in my scheme.  If all you
> store in the boot archive is the choice of driver, and the default is a
> reasonable one, then very few users will need to change it, and those
> that do will need to change it just once.
>    
I believe Jack was referring his proposed approach #3, dynamically 
generating bootpath based on the information provided by OBP. As a few 
people pointed out that storing information in boot archive could be 
problematic, the project team opt to generating bootpath on the fly as 
opposed to storing the whole bootpath in boot archive.
>    
>> Since there are 2 options only I think that it is a practical way to
>> do the selection based on trail-and-error with the default 'ssd'
>> driver. If the TPGT value is available, the worst case would be one
>> false attempt to match the path, which is proven with trivial impact
>> to the boot process as the relogin and rediscovery will not take
>> place.
>>      
> How does the trial-and-error approach work?  Can the kernel detect that
> it used the wrong driver?  If so, great!
>    
Jack can provide a webrev here but basically if the disk driver is wrong 
the kernel can detect it and in the second attempt use the right driver.
> As for TPGT, that's a 16-bit field.  You can't really try all TPGT
> values.
>    
Agreed. Retrying is not an option on TPGT.

Thanks,

Grant
> Nico
>    


From David.Kahn@sun.com Tue Sep 15 19:24:05 2009
Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n8G2O3qT021130
	for <psarc-ext@sac.sfbay.sun.com>; Tue, 15 Sep 2009 19:24:03 -0700 (PDT)
Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6])
	by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n8G2NugX014302
	for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 16 Sep 2009 10:24:02 +0800 (SGT)
Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by
 nwk-avmta-2.sfbay.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KQ100J07LC02500@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com
 (ORCPT PSARC-ext@sun.com); Tue, 15 Sep 2009 19:24:00 -0700 (PDT)
Received: from dm-sfbay-01.sfbay.sun.com ([129.145.155.118])
 by nwk-avmta-2.sfbay.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KQ10099KLBZG470@nwk-avmta-2.sfbay.sun.com> for
 PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue,
 15 Sep 2009 19:23:59 -0700 (PDT)
Received: from dtmail.sfbay.sun.com (dt101-150.SFBay.Sun.COM [10.6.101.150])
	by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2)
 with ESMTP id n8G2NtUo045916; Tue, 15 Sep 2009 19:23:55 -0700 (PDT)
Received: from [192.168.0.39] (noho.SFBay.Sun.COM [10.6.92.101])
	by dtmail.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n8G2NsDX017859; Tue,
 15 Sep 2009 19:23:54 -0700 (PDT)
Date: Tue, 15 Sep 2009 19:24:13 -0700
From: David Kahn <David.Kahn@sun.com>
Subject: Re: Add bootpath into Solaris Sparc BootArchive for iSCSI boot
 [PSARC/2009/480 Self Review]
In-reply-to: <4AAE9F6E.3010902@Sun.COM>
To: Bart Smaalders <bart.smaalders@sun.com>
Cc: Tarl Neustaedter <Tarl.Neustaedter@sun.com>,
        Nicolas Williams <Nicolas.Williams@sun.com>,
        Jack Meng <Jack.Meng@sun.com>,
        "Mark A. Carlson" <Mark.Carlson@sun.com>,
        Darren J Moffat <Darren.Moffat@sun.com>, PSARC-ext@sun.com,
        Grant.Zhang@sun.com, Jan Setje-Eilers <Jan.Setje-Eilers@sun.com>
Message-id: <4AB04C4D.4050609@sun.com>
MIME-version: 1.0
Content-type: text/plain; charset=ISO-8859-1; format=flowed
Content-transfer-encoding: 7BIT
X-PMX-Version: 5.4.1.325704
References: <200909102101.n8AL1Gam029312@sac.sfbay.sun.com>
 <4AA96EED.7030206@Sun.COM> <4AA9715E.2020805@sun.com>
 <4AAA0C2A.1040707@Sun.COM> <4AAA398E.4010800@sun.com>
 <4AAA6D3E.5040506@sun.com> <20090914184116.GE1033@Sun.COM>
 <4AAE90B7.7020704@sun.com> <20090914190912.GH1033@Sun.COM>
 <4AAE9931.8020201@sun.com> <4AAE9F6E.3010902@Sun.COM>
User-Agent: Thunderbird 2.0.0.23 (Windows/20090812)
Status: RO
Content-Length: 1099



Bart Smaalders wrote:
> Tarl Neustaedter wrote:
>> The knowledge of which driver to use within Solaris
>> isn't needed until after this stage - so I don't think it belongs in 
>> the boot command in the first place.
> 
> This is correct... and storing such driver info somewhere where a
> special program needs to update it is just plain broken.
> 
> Solaris should be passed the _external_ info about its boot drive -
> and then should dynamically figure out how to talk to that external
> device.  Any other approach will break when drivers change,
> paths to devices change, etc.

That's what the FWARC case does today, so if you are agreeing
with that, then I agree with you. Is that what you mean, Bart?

We don't want the OS method of the day to get encoded in the
firmware. It's likely that stuff will change. The iscsi stuff is
kind of a mess, as this case shows. The FWARC case is all standards
based and should provide whatever info any OS needs to continue
with an iscsi boot. If we missed something, we can certainly add it,
but I haven't seen that they did miss anything yet.

-David

From gdamore@sun.com Wed Sep 16 10:23:48 2009
Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n8GHNl7t007934
	for <psarc-ext@sac.sfbay.sun.com>; Wed, 16 Sep 2009 10:23:47 -0700 (PDT)
Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6])
	by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n8GHNfHJ028292
	for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 16 Sep 2009 18:23:46 +0100 (BST)
Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by
 nwk-avmta-2.sfbay.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KQ200A09QZK3800@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com
 (ORCPT PSARC-ext@sun.com); Wed, 16 Sep 2009 10:23:44 -0700 (PDT)
Received: from sca-es-mail-2.sun.com ([192.18.43.133])
 by nwk-avmta-2.sfbay.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KQ2001O9QZKYWC0@nwk-avmta-2.sfbay.sun.com> for
 PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed,
 16 Sep 2009 10:23:44 -0700 (PDT)
Received: from fe-sfbay-09.sun.com ([192.18.43.129])
	by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n8GHNiqW000312	for
 <PSARC-ext@sun.com>; Wed, 16 Sep 2009 10:23:44 -0700 (PDT)
Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com
 (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul  2 2009))
 id <0KQ200100QMSTS00@fe-sfbay-09.sun.com> for PSARC-ext@sun.com
 (ORCPT PSARC-ext@sun.com); Wed, 16 Sep 2009 10:23:44 -0700 (PDT)
Received: from [192.168.251.11] ([unknown] [76.93.15.33])
 by fe-sfbay-09.sun.com
 (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul  2 2009))
 with ESMTPSA id <0KQ2009IBQZJHK10@fe-sfbay-09.sun.com> for PSARC-ext@sun.com
 (ORCPT PSARC-ext@sun.com); Wed, 16 Sep 2009 10:23:44 -0700 (PDT)
Date: Wed, 16 Sep 2009 10:23:43 -0700
From: "Garrett D'Amore" <gdamore@sun.com>
Subject: PSARC 2009/480 Add bootpath into Solaris Sparc BootArchive for iSCSI
 boot
Sender: Garrett.Damore@sun.com
To: PSARC-ext <PSARC-ext@sun.com>
Message-id: <4AB11F1F.6080003@sun.com>
MIME-version: 1.0
Content-type: text/plain; CHARSET=US-ASCII; format=flowed
Content-transfer-encoding: 7BIT
X-PMX-Version: 5.4.1.325704
User-Agent: Thunderbird 2.0.0.18 (X11/20081201)
Status: RO
Content-Length: 235

This case has failed to meet the "obviousness" requirements for 
self-review.  At PSARC today, the agreement was to turn this into a fast 
track with a timer set for Sept 23.  I've updated the IAM file to 
reflect this.

    - Garrett

From bart.smaalders@sun.com Wed Sep 16 10:31:08 2009
Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n8GHV8MO008626
	for <psarc-ext@sac.sfbay.sun.com>; Wed, 16 Sep 2009 10:31:08 -0700 (PDT)
Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6])
	by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n8GHUw4V001208;
	Wed, 16 Sep 2009 11:31:04 -0600 (MDT)
Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by
 nwk-avmta-2.sfbay.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KQ200A3ZRBLIR00@nwk-avmta-2.sfbay.sun.com>; Wed,
 16 Sep 2009 10:30:58 -0700 (PDT)
Received: from zion.sfbay.sun.com ([129.146.17.75])
 by nwk-avmta-2.sfbay.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KQ2001G5RBLZ2C0@nwk-avmta-2.sfbay.sun.com>; Wed,
 16 Sep 2009 10:30:57 -0700 (PDT)
Received: from [129.146.228.109] (cyber.SFBay.Sun.COM [129.146.228.109])
	by zion.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n8GHU1TK022849; Wed,
 16 Sep 2009 17:30:01 +0000 (GMT)
Date: Wed, 16 Sep 2009 10:28:51 -0700
From: Bart Smaalders <bart.smaalders@sun.com>
Subject: Re: Add bootpath into Solaris Sparc BootArchive for iSCSI boot
	[PSARC/2009/480 Self Review]
In-reply-to: <4AB04C4D.4050609@sun.com>
To: David Kahn <David.Kahn@sun.com>
Cc: PSARC-ext@sun.com, Jan Setje-Eilers <Jan.Setje-Eilers@sun.com>,
        Tarl Neustaedter <Tarl.Neustaedter@sun.com>,
        Jack Meng <Jack.Meng@sun.com>, Grant.Zhang@sun.com
Message-id: <4AB12053.5050809@Sun.COM>
Organization: Sun Microsystems
MIME-version: 1.0
Content-type: text/plain; charset=ISO-8859-1; format=flowed
Content-transfer-encoding: 7BIT
X-PMX-Version: 5.4.1.325704
References: <200909102101.n8AL1Gam029312@sac.sfbay.sun.com>
 <4AA96EED.7030206@Sun.COM> <4AA9715E.2020805@sun.com>
 <4AAA0C2A.1040707@Sun.COM> <4AAA398E.4010800@sun.com>
 <4AAA6D3E.5040506@sun.com> <20090914184116.GE1033@Sun.COM>
 <4AAE90B7.7020704@sun.com> <20090914190912.GH1033@Sun.COM>
 <4AAE9931.8020201@sun.com> <4AAE9F6E.3010902@Sun.COM>
 <4AB04C4D.4050609@sun.com>
User-Agent: Thunderbird 2.0.0.22 (X11/20090804)
Status: RO
Content-Length: 1916

David Kahn wrote:
> 
> 
> Bart Smaalders wrote:
>> Tarl Neustaedter wrote:
>>> The knowledge of which driver to use within Solaris
>>> isn't needed until after this stage - so I don't think it belongs in 
>>> the boot command in the first place.
>>
>> This is correct... and storing such driver info somewhere where a
>> special program needs to update it is just plain broken.
>>
>> Solaris should be passed the _external_ info about its boot drive -
>> and then should dynamically figure out how to talk to that external
>> device.  Any other approach will break when drivers change,
>> paths to devices change, etc.
> 
> That's what the FWARC case does today, so if you are agreeing
> with that, then I agree with you. Is that what you mean, Bart?
> 

Yes.... I'm working on the new packaging system for Solaris next,
and we don't use an "installer" or "upgrade" program; instead,
changes are made to the delivered files as specified in the packages.
There is no release-specific code that gets run to perform arbitrary
fixups to config files, such as determining kernel-internal boot paths, 
etc.  Today, w/ ZFS root (mandatory in Solaris next for a variety of
reasons) this works well on both SPARC and x86 although USB connected
storage devices still need some help.

The first iscsi proposal required an installer program (running on
the previous release being upgraded via zfs clone) to compute the
new kernel's internal boot paths so that the new kernel could boot.
Needless to say, this would create significant technical and
management issues; since we manage to boot a ubiquitous CD image,
dynamically discovering paths to that device after initial image load,
I believe that we can and should do the same for iscsi boot devices.

- Bart

-- 
Bart Smaalders			Solaris Kernel Performance
barts@cyber.eng.sun.com		http://blogs.sun.com/barts
"You will contribute more with mercurial than with thunderbird."

From David.Kahn@sun.com Wed Sep 16 18:52:19 2009
Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n8H1qITu019599
	for <psarc-ext@sac.sfbay.sun.com>; Wed, 16 Sep 2009 18:52:19 -0700 (PDT)
Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6])
	by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n8H1qBYS007093
	for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Thu, 17 Sep 2009 09:52:18 +0800 (SGT)
Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by
 nwk-avmta-2.sfbay.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KQ300L01EJ43E00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com
 (ORCPT PSARC-ext@sun.com); Wed, 16 Sep 2009 18:52:16 -0700 (PDT)
Received: from dm-sfbay-02.sfbay.sun.com ([129.146.11.31])
 by nwk-avmta-2.sfbay.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KQ300GKDEJ4UW30@nwk-avmta-2.sfbay.sun.com> for
 PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed,
 16 Sep 2009 18:52:16 -0700 (PDT)
Received: from dtmail.sfbay.sun.com (dt101-150.SFBay.Sun.COM [10.6.101.150])
	by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2)
 with ESMTP id n8H1qC6I005636; Wed, 16 Sep 2009 18:52:12 -0700 (PDT)
Received: from [192.168.0.39] (noho.SFBay.Sun.COM [10.6.92.101])
	by dtmail.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n8H1qBiM025034; Wed,
 16 Sep 2009 18:52:11 -0700 (PDT)
Date: Wed, 16 Sep 2009 18:52:30 -0700
From: David Kahn <David.Kahn@sun.com>
Subject: Re: Add bootpath into Solaris Sparc BootArchive for iSCSI boot
	[PSARC/2009/480 Self Review]
In-reply-to: <4AB12053.5050809@Sun.COM>
To: Bart Smaalders <bart.smaalders@sun.com>
Cc: PSARC-ext@sun.com, Jan Setje-Eilers <Jan.Setje-Eilers@sun.com>,
        Tarl Neustaedter <Tarl.Neustaedter@sun.com>,
        Jack Meng <Jack.Meng@sun.com>, Grant.Zhang@sun.com
Message-id: <4AB1965E.6080700@sun.com>
MIME-version: 1.0
Content-type: text/plain; charset=ISO-8859-1; format=flowed
Content-transfer-encoding: 7BIT
X-PMX-Version: 5.4.1.325704
References: <200909102101.n8AL1Gam029312@sac.sfbay.sun.com>
 <4AA96EED.7030206@Sun.COM> <4AA9715E.2020805@sun.com>
 <4AAA0C2A.1040707@Sun.COM> <4AAA398E.4010800@sun.com>
 <4AAA6D3E.5040506@sun.com> <20090914184116.GE1033@Sun.COM>
 <4AAE90B7.7020704@sun.com> <20090914190912.GH1033@Sun.COM>
 <4AAE9931.8020201@sun.com> <4AAE9F6E.3010902@Sun.COM>
 <4AB04C4D.4050609@sun.com> <4AB12053.5050809@Sun.COM>
User-Agent: Thunderbird 2.0.0.23 (Windows/20090812)
Status: RO
Content-Length: 563


I just submitted FWARC/2009/498 for Tarl, which adds
a new /chosen property, iscsi-tpgt.

Apparently, the information was already available to
OF, so it was easy enough to publish it, even though
we don't need it, and in theory anybody could do the
same thing that OF does to get it. Nonetheless, OF
can easily publish it for the OS if they need it.
It doesn't affect the OF boot path or encode any OS
implementation specific info in the property, so we're
fine with it.

Does that solve the issues that were in this case?
Do they still need this case?

-David


From David.Kahn@sun.com Wed Sep 16 19:49:43 2009
Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n8H2nhOR025327
	for <psarc-ext@sac.sfbay.sun.com>; Wed, 16 Sep 2009 19:49:43 -0700 (PDT)
Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74])
	by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id n8H2ngH3003267
	for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 16 Sep 2009 19:49:42 -0700 (PDT)
Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by
 nwk-avmta-1.sfbay.Sun.COM
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KQ300J05H6T7600@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com
 (ORCPT PSARC-ext@sun.com); Wed, 16 Sep 2009 19:49:41 -0700 (PDT)
Received: from dm-sfbay-02.sfbay.sun.com ([129.146.11.31])
 by nwk-avmta-1.sfbay.Sun.COM
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KQ300B1LH6TJK50@nwk-avmta-1.sfbay.Sun.COM> for
 PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed,
 16 Sep 2009 19:49:41 -0700 (PDT)
Received: from dtmail.sfbay.sun.com (dt101-150.SFBay.Sun.COM [10.6.101.150])
	by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2)
 with ESMTP id n8H2naKG038661; Wed, 16 Sep 2009 19:49:36 -0700 (PDT)
Received: from [192.168.0.39] (noho.SFBay.Sun.COM [10.6.92.101])
	by dtmail.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n8H2nZST009877; Wed,
 16 Sep 2009 19:49:35 -0700 (PDT)
Date: Wed, 16 Sep 2009 19:49:54 -0700
From: David Kahn <David.Kahn@sun.com>
Subject: Re: Add bootpath into Solaris Sparc BootArchive for iSCSI boot
 [PSARC/2009/480 Self Review]
In-reply-to: <20090914195425.GI1033@Sun.COM>
To: Nicolas Williams <Nicolas.Williams@sun.com>
Cc: Tarl Neustaedter <Tarl.Neustaedter@sun.com>, Jack Meng <Jack.Meng@sun.com>,
        "Mark A. Carlson" <Mark.Carlson@sun.com>,
        Darren J Moffat <Darren.Moffat@sun.com>, PSARC-ext@sun.com,
        Bart Smaalders <Bart.Smaalders@sun.com>, Grant.Zhang@sun.com,
        Jan Setje-Eilers <Jan.Setje-Eilers@sun.com>
Message-id: <4AB1A3D2.6070505@sun.com>
MIME-version: 1.0
Content-type: text/plain; charset=ISO-8859-1; format=flowed
Content-transfer-encoding: 7BIT
X-PMX-Version: 5.4.1.325704
References: <200909102101.n8AL1Gam029312@sac.sfbay.sun.com>
 <4AA96EED.7030206@Sun.COM> <4AA9715E.2020805@sun.com>
 <4AAA0C2A.1040707@Sun.COM> <4AAA398E.4010800@sun.com>
 <4AAA6D3E.5040506@sun.com> <20090914184116.GE1033@Sun.COM>
 <4AAE90B7.7020704@sun.com> <20090914190912.GH1033@Sun.COM>
 <4AAE9931.8020201@sun.com> <20090914195425.GI1033@Sun.COM>
User-Agent: Thunderbird 2.0.0.23 (Windows/20090812)
Status: RO
Content-Length: 1911


> I never said that the OBP should know which driver Solaris should use.
> 
> I said that the choice of driver should: a) be defaulted, b) be settable
> via boot archive and/or boot option.  Administering a choice of driver
> via boot archive seems reasonable to me.  Storing the full boot path in
> the boot archive is not (since it is redundant, causing yet one more
> place to update when a system's root migrates).

I don't know if you solved the last piece of this or not.

We (the firmware) simply export a pseudo path for iscsi boot,
which is /iscsi-hba/disk

the disk node has a "compatible" property, which I'm told currently
contains the value "sd".

We could probably change that to something like "SUNW,iscsi-disk",
and the OS can bind that string to whatever driver you want to in
/etc/driver_aliases. (either bundled or via add_drv). If you want
us to change that, we can ask Tarl to do it, and we'll submit another
fast-track case for that change. But that is not something that is
or should be settable in the firmware at least to the normal used.
I can't imagine a user friendly system where the user has to configure
the name of the OS driver in the firmware that they want to use.
How would they know that? Nonetheless, it seems like the wrong thing
to want to do if you ask me.

We don't specify OS driver bindings in the firmware. We do provide
"compatible" property value definitions with default values as
specified in specific bus bindings, based on standards. The driver
binding is always left to the OS. That's worked for us for a long
time, and it's hard to envision a case now where it doesn't work
for us, as long as devices provide some unique string that the OS
can use to bind to a device driver name.

If you want something besides "sd" in the /iscsi-hba/disk compatible
propval, we can do that. Other than that, I'm not sure we can help
with the driver binding issue.

-David

From Jack.Meng@Sun.COM Wed Sep 16 20:25:02 2009
Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n8H3P2PC005423
	for <psarc-ext@sac.sfbay.sun.com>; Wed, 16 Sep 2009 20:25:02 -0700 (PDT)
Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74])
	by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n8H3P1pf008802
	for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 16 Sep 2009 21:25:01 -0600 (MDT)
Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by
 nwk-avmta-1.sfbay.Sun.COM
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KQ300103ITP3300@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com
 (ORCPT PSARC-ext@sun.com); Wed, 16 Sep 2009 20:25:01 -0700 (PDT)
Received: from sineb-mail-2.sun.com ([192.18.19.7])
 by nwk-avmta-1.sfbay.Sun.COM
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KQ300B3NITOJC80@nwk-avmta-1.sfbay.Sun.COM> for
 PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed,
 16 Sep 2009 20:25:01 -0700 (PDT)
Received: from fe-apac-05.sun.com
 (fe-apac-05.sun.com [192.18.19.176] (may be forged))
	by sineb-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n8H3P0BJ022495	for
 <PSARC-ext@sun.com>; Thu, 17 Sep 2009 03:25:00 +0000 (GMT)
Received: from conversion-daemon.mail-apac.sun.com by mail-apac.sun.com
 (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul  2 2009))
 id <0KQ300700IJQG100@mail-apac.sun.com> for PSARC-ext@sun.com
 (ORCPT PSARC-ext@sun.com); Thu, 17 Sep 2009 11:24:59 +0800 (SGT)
Received: from [129.158.144.121] ([unknown] [129.158.144.121])
 by mail-apac.sun.com
 (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul  2 2009))
 with ESMTPSA id <0KQ300M6PITMLN20@mail-apac.sun.com>; Thu,
 17 Sep 2009 11:24:59 +0800 (SGT)
Date: Thu, 17 Sep 2009 11:22:49 +0800
From: Jack <Jack.Meng@Sun.COM>
Subject: Re: Add bootpath into Solaris Sparc BootArchive for iSCSI boot
 [PSARC/2009/480 Self Review]
In-reply-to: <4AB1A3D2.6070505@sun.com>
Sender: Jack.Meng@Sun.COM
To: David Kahn <David.Kahn@Sun.COM>
Cc: Nicolas Williams <Nicolas.Williams@Sun.COM>,
        Tarl Neustaedter <Tarl.Neustaedter@Sun.COM>,
        "Mark A. Carlson" <Mark.Carlson@Sun.COM>,
        Darren J Moffat <Darren.Moffat@Sun.COM>, PSARC-ext@Sun.COM,
        Bart Smaalders <Bart.Smaalders@Sun.COM>, Grant.Zhang@Sun.COM,
        Jan Setje-Eilers <Jan.Setje-Eilers@Sun.COM>
Message-id: <4AB1AB89.2010905@sun.com>
MIME-version: 1.0
Content-type: text/plain; CHARSET=US-ASCII; format=flowed
Content-transfer-encoding: 7BIT
X-PMX-Version: 5.4.1.325704
References: <200909102101.n8AL1Gam029312@sac.sfbay.sun.com>
 <4AA96EED.7030206@Sun.COM> <4AA9715E.2020805@sun.com>
 <4AAA0C2A.1040707@Sun.COM> <4AAA398E.4010800@sun.com>
 <4AAA6D3E.5040506@sun.com> <20090914184116.GE1033@Sun.COM>
 <4AAE90B7.7020704@sun.com> <20090914190912.GH1033@Sun.COM>
 <4AAE9931.8020201@sun.com> <20090914195425.GI1033@Sun.COM>
 <4AB1A3D2.6070505@sun.com>
User-Agent: Thunderbird 2.0.0.23 (Windows/20090812)
Status: RO
Content-Length: 2676

David Kahn wrote:
>
>> I never said that the OBP should know which driver Solaris should use.
>>
>> I said that the choice of driver should: a) be defaulted, b) be settable
>> via boot archive and/or boot option.  Administering a choice of driver
>> via boot archive seems reasonable to me.  Storing the full boot path in
>> the boot archive is not (since it is redundant, causing yet one more
>> place to update when a system's root migrates).
>
> I don't know if you solved the last piece of this or not.
>
> We (the firmware) simply export a pseudo path for iscsi boot,
> which is /iscsi-hba/disk
>
> the disk node has a "compatible" property, which I'm told currently
> contains the value "sd".
>
> We could probably change that to something like "SUNW,iscsi-disk",
> and the OS can bind that string to whatever driver you want to in
> /etc/driver_aliases. (either bundled or via add_drv). If you want
> us to change that, we can ask Tarl to do it, and we'll submit another
> fast-track case for that change. But that is not something that is
> or should be settable in the firmware at least to the normal used.
> I can't imagine a user friendly system where the user has to configure
> the name of the OS driver in the firmware that they want to use.
> How would they know that? Nonetheless, it seems like the wrong thing
> to want to do if you ask me.
>
> We don't specify OS driver bindings in the firmware. We do provide
> "compatible" property value definitions with default values as
> specified in specific bus bindings, based on standards. The driver
> binding is always left to the OS. That's worked for us for a long
> time, and it's hard to envision a case now where it doesn't work
> for us, as long as devices provide some unique string that the OS
> can use to bind to a device driver name.
>
> If you want something besides "sd" in the /iscsi-hba/disk compatible
> propval, we can do that. Other than that, I'm not sure we can help
> with the driver binding issue.
>
> -David
Hi David,

There was no request to ask firmware to specify the driver name for 
binding in this case.
Solaris will figure out right drivers to bind by dynamically assembling 
the bootpath, that is
described as solution #3 in a previous email. That approach is with the 
help from
FWARC/2009/498, and is opposed to the original spec of this case.
I'll write up an updated spec for this case to detail the new approach 
(roughly described as solution #3
in previous email) based on comments received and progresses already 
made to solve the issue.

Thank you for the help on the FWARC case, and I think that's enough to 
solve the issue
here from firmware side.

Best regards,
Jack

From Mark.Carlson@sun.com Wed Sep 23 18:48:14 2009
Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n8O1mEVa001234
	for <psarc-ext@sac.sfbay.sun.com>; Wed, 23 Sep 2009 18:48:14 -0700 (PDT)
Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74])
	by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id n8O1mDak013854
	for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 23 Sep 2009 18:48:13 -0700 (PDT)
Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by
 nwk-avmta-1.sfbay.Sun.COM
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KQG00J05D0BPQ00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com
 (ORCPT PSARC-ext@sun.com); Wed, 23 Sep 2009 18:48:11 -0700 (PDT)
Received: from brmea-mail-2.sun.com ([192.18.98.43])
 by nwk-avmta-1.sfbay.Sun.COM
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KQG00LL9D0A1870@nwk-avmta-1.sfbay.Sun.COM> for
 PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed,
 23 Sep 2009 18:48:10 -0700 (PDT)
Received: from fe-amer-09.sun.com ([192.18.109.79])
	by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n8O1m98W002871	for
 <PSARC-ext@sun.com>; Thu, 24 Sep 2009 01:48:09 +0000 (GMT)
Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com
 (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul  2 2009))
 id <0KQG00E00CU98M00@mail-amer.sun.com> for PSARC-ext@sun.com
 (ORCPT PSARC-ext@sun.com); Wed, 23 Sep 2009 19:48:09 -0600 (MDT)
Received: from dhcp-cbjs05-219-172.prc.sun.com ([unknown] [129.158.219.172])
 by mail-amer.sun.com
 (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul  2 2009))
 with ESMTPSA id <0KQG00F9KD07YUF0@mail-amer.sun.com> for PSARC-ext@sun.com
 (ORCPT PSARC-ext@sun.com); Wed, 23 Sep 2009 19:48:09 -0600 (MDT)
Date: Wed, 23 Sep 2009 19:48:04 -0600
From: "Mark A. Carlson" <Mark.Carlson@sun.com>
Subject: Re: Dynamical assembling the bootpath for iSCSI boot on sparc [PSARC
 2009/480 FastTrack timeout 09/23/2009]
In-reply-to: <4AB11F1F.6080003@sun.com>
Sender: Mark.Carlson@sun.com
To: PSARC-ext <PSARC-ext@sun.com>
Cc: "Garrett D'Amore" <gdamore@sun.com>
Message-id: <4ABACFD4.8050109@sun.com>
MIME-version: 1.0
Content-type: multipart/mixed; boundary="Boundary_(ID_dbqFq6CVnZyDAv7MHYNPvw)"
X-PMX-Version: 5.4.1.325704
References: <4AB11F1F.6080003@sun.com>
User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812)
Status: RO
Content-Length: 6802

This is a multi-part message in MIME format.

--Boundary_(ID_dbqFq6CVnZyDAv7MHYNPvw)
Content-type: text/plain; CHARSET=US-ASCII; format=flowed
Content-transfer-encoding: 7BIT

The enclosed updated proposal has been placed in the case directory
documenting the discussions and decisions in this thread.

Changes made to the spec include:
1. The name of the project has been changed to
    "Dynamical assembling the bootpath for iSCSI boot on sparc"

2. The proposed solution in the original case is withdrawn.

3. 3 options are listed with a detailed explanation/justification
   of each one, including  the prposed option 3 of dynamical
   assembling the bootpath for iSCSI boot.

4. The Interfaces section has been updated to remove the boot_archive 
interface
   and an additional import of the property from OBP used in assembling the
   bootpath has been added.

Anybody need more time?

-- mark

Garrett D'Amore wrote:
> This case has failed to meet the "obviousness" requirements for 
> self-review.  At PSARC today, the agreement was to turn this into a 
> fast track with a timer set for Sept 23.  I've updated the IAM file to 
> reflect this.
>
>    - Garrett
> _______________________________________________
> opensolaris-arc mailing list
> opensolaris-arc@opensolaris.org


--Boundary_(ID_dbqFq6CVnZyDAv7MHYNPvw)
Content-type: text/plain; name=iscsibootpath.txt
Content-transfer-encoding: 7BIT
Content-disposition: inline; filename=iscsibootpath.txt

1. Introduction
    1.1. Project/Component Working Name:
     Dynamical assembling the bootpath for iSCSI boot on sparc
    1.2. Name of Document Author/Supplier:
     Author:  Jack Meng
    1.3  Date of This Document:
     September 18 2009
    
2. Project Summary
   2.1. Project Description
        This project is to introduce a new mechanism to get the bootpath
        by dynamical assembling it with information provided by firmware,
        rather than read it statically from somewhere else.

4. Technical Description
    4.1. Details:
    Currently Solaris Sparc kernel takes the hardware description of boot
    device from OBP, which is in compliance with IEEE Std. 1275, as the
    physical device path in OS. For example, the path,
        "/pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w21000004cf68e801,0"
    works fine for both OBP and Solaris as OBP considers this a
    platform-independent description while Solaris takes it as the physical
    device path.

    With the introduction of iSCSI Boot support in OBP things become
    different for the bootpath FWARC 2008/466 provided, '/iscsi-hba/disk',
    is not recognizable by Solaris Kernel. That makes Solaris kernel
    have to read the bootpath from somewhere else, or dynamically assemble
    the bootpath of an iscsi device path, e.g.,

    /iscsi/ssd@0000iqn.1986-03.com.sun:02:43644186-e5bb-41f2-8b8e-f34be1afaebc0001,0:a
    or
    /iscsi/disk@0000iqn.1986-03.com.sun:02:43644186-e5bb-41f2-8b8e-f34be1afaebc0001,0:a

    So far 3 options were discussed,

    Option 1

    To store and retrieve the bootpath from OBP. It is considered not a good
    approach for both it is an OS-specific property and most information adhering
    to the path has been given to the OS.
    
    Option 2
    To store and retrive the bootpath from boot_archive, as the same machnism in use
    to support Solaris UFS boot in x86, will cause a lot of issues to impact the
    flexibility in OS configuration and migration. For example, this approach will
    simply breaks the system as soon as a minor configuration change that impacts
    the bootpath. This is also a major issue on x86 that has kept OS images from
    being as portable as they should, and is needed to be fixed rather than
    broadening the adoption.
    
    Option 3
    FWARC 2008/466 and FWARC 2009/498 has provide enough information to compose
    the physical path of the iSCSI boot disk except the binding disk driver. For details,
    the path of an iSCSI disk has following components,

    1. iscsi             -    iscsi driver name, fixed
    2. ssd               -    name/alias of bound disk driver, is one of 'ssd' or 'disk'
    3. 0000              -    part of the iscsi initiator ISID as a fixed number in Solaris
    4. iqn.1986-03.com.sun:02:43644186-e5bb-41f2-8b8e-f34be1afaebc
                         -    value of the 'iscsi-target-name' property in OBP '/chosen' node
    5. 0001              -    value of the 'iscsi-tpgt' property in OBP '/chosen' node
    6. 0                 -    value of the 'iscsi-lun' property in OBP '/chosen' node
    7. :a                -    value of the 'iscsi-partition' property in OBP '/chosen' node
    
    Therefore all components can be determined except the disk driver, which has 2 options
    in Solaris, namely 'ssd' and 'sd'(disk). This means that dynamical assembling will generate
    2 options in this case. Since the binding driver is determined by both the MPxIO capability
    and user configuration so the kernel is unable to determine which driver to bind until
    the boot device is available in the device tree. Therefore an trial-and-error is introduced
    for these 2 options by trying to mount the root fs on the most possible one, 'ssd' in this case,
    and then the another. The impact of this approach is proved trivial as no relogin/rediscovery
    will be made to the target in case of failure, and so it is considered the best approach to
    solve the bootpath issue for iSCSI boot.

    4.5 Interfaces:

    Exported Interface                    | Stability    |    Description
    --------------------------------------|--------------|-------------------

    Imported Interface                    | Stability    |    Description
    --------------------------------------|--------------|-------------------
    iscsi-target-name                     | committed    |  iscsi target name in OBP '/chosen' node
    iscsi-tpgt                            | committed    |  iscsi target portl group tag in '/chosen'
    iscsi-lun                             | committed    |  boot lun in '/chosen'
    iscsi-partition                       | committed    |  boot partition in '/chosen'

    Other documents:
    FWARC 2008/466  iSCSI SPARC Openboot support    http://sac.sfbay/FWARC/2008/466
    FWARC 2009/498  iscsi boot update               http://sac.sfbay/FWARC/2009/498
    PSARC 2008/427  iSCSI Boot                      http://sac.sfbay/PSARC/2008/427
    
6. Resources and Schedule
    6.4. Steering Committee requested information
       6.4.1. Consolidation C-team Name:
        ONNV, ON10
        6.4.2. Target Release/Binding
                Request patch binding as this feature may get backported to S10.
    6.5. ARC review type: FastTrack
    6.6. ARC Exposure: open

--Boundary_(ID_dbqFq6CVnZyDAv7MHYNPvw)--

From gdamore@sun.com Wed Sep 23 20:54:14 2009
Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n8O3sDK5003375
	for <psarc-ext@sac.sfbay.sun.com>; Wed, 23 Sep 2009 20:54:14 -0700 (PDT)
Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11])
	by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n8O3sAFt007425
	for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Thu, 24 Sep 2009 11:54:12 +0800 (SGT)
Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by
 brm-avmta-1.central.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KQG00G09IUAGD00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com
 (ORCPT PSARC-ext@sun.com); Wed, 23 Sep 2009 21:54:10 -0600 (MDT)
Received: from sca-es-mail-2.sun.com ([192.18.43.133])
 by brm-avmta-1.central.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KQG0068BIU9EZF0@brm-avmta-1.central.sun.com> for
 PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed,
 23 Sep 2009 21:54:09 -0600 (MDT)
Received: from fe-sfbay-09.sun.com ([192.18.43.129])
	by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n8O3s9O6001576	for
 <PSARC-ext@sun.com>; Wed, 23 Sep 2009 20:54:09 -0700 (PDT)
Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com
 (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul  2 2009))
 id <0KQG00G00IKMJ600@fe-sfbay-09.sun.com> for PSARC-ext@sun.com
 (ORCPT PSARC-ext@sun.com); Wed, 23 Sep 2009 20:54:09 -0700 (PDT)
Received: from [192.168.251.11] ([unknown] [76.93.15.33])
 by fe-sfbay-09.sun.com
 (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul  2 2009))
 with ESMTPSA id <0KQG00BUZIU8CF20@fe-sfbay-09.sun.com> for PSARC-ext@sun.com
 (ORCPT PSARC-ext@sun.com); Wed, 23 Sep 2009 20:54:08 -0700 (PDT)
Date: Wed, 23 Sep 2009 20:54:08 -0700
From: "Garrett D'Amore" <gdamore@sun.com>
Subject: Re: Dynamical assembling the bootpath for iSCSI boot on sparc [PSARC
 2009/480 FastTrack timeout 09/23/2009]
In-reply-to: <4ABACFD4.8050109@sun.com>
Sender: Garrett.Damore@sun.com
To: "Mark A. Carlson" <Mark.Carlson@sun.com>
Cc: PSARC-ext <PSARC-ext@sun.com>
Message-id: <4ABAED60.7040502@sun.com>
MIME-version: 1.0
Content-type: text/plain; CHARSET=US-ASCII; format=flowed
Content-transfer-encoding: 7BIT
X-PMX-Version: 5.4.1.325704
References: <4AB11F1F.6080003@sun.com> <4ABACFD4.8050109@sun.com>
User-Agent: Thunderbird 2.0.0.18 (X11/20081201)
Status: RO
Content-Length: 1484

Mark A. Carlson wrote:
> The enclosed updated proposal has been placed in the case directory
> documenting the discussions and decisions in this thread.
>
> Changes made to the spec include:
> 1. The name of the project has been changed to
>    "Dynamical assembling the bootpath for iSCSI boot on sparc"
>
> 2. The proposed solution in the original case is withdrawn.
>
> 3. 3 options are listed with a detailed explanation/justification
>   of each one, including  the prposed option 3 of dynamical
>   assembling the bootpath for iSCSI boot.
>
> 4. The Interfaces section has been updated to remove the boot_archive 
> interface
>   and an additional import of the property from OBP used in assembling 
> the
>   bootpath has been added.
>
> Anybody need more time?

The approach chosen seems like a hack to me, but if it works, I'm ok 
with it.   (I still feel like the objections raised by the OBP group 
here are a bit... ahem... arbitrary.   But I also think the fact that we 
still have split sd and ssd drivers is a very unfortunate circumstance.)

    - Garrett

>
> -- mark
>
> Garrett D'Amore wrote:
>> This case has failed to meet the "obviousness" requirements for 
>> self-review.  At PSARC today, the agreement was to turn this into a 
>> fast track with a timer set for Sept 23.  I've updated the IAM file 
>> to reflect this.
>>
>>    - Garrett
>> _______________________________________________
>> opensolaris-arc mailing list
>> opensolaris-arc@opensolaris.org
>


From Mark.Carlson@sun.com Sat Sep 26 16:50:56 2009
Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n8QNouY5025367
	for <psarc-ext@sac.sfbay.sun.com>; Sat, 26 Sep 2009 16:50:56 -0700 (PDT)
Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74])
	by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n8QNor4L012678
	for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Sun, 27 Sep 2009 00:50:55 +0100 (BST)
Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by
 nwk-avmta-1.sfbay.Sun.COM
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KQL00C03RKU5Z00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com
 (ORCPT PSARC-ext@sun.com); Sat, 26 Sep 2009 16:50:54 -0700 (PDT)
Received: from brmea-mail-1.sun.com ([192.18.98.31])
 by nwk-avmta-1.sfbay.Sun.COM
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KQL00374RKU2DE0@nwk-avmta-1.sfbay.Sun.COM> for
 PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Sat,
 26 Sep 2009 16:50:54 -0700 (PDT)
Received: from fe-amer-09.sun.com ([192.18.109.79])
	by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n8QNorfl000301	for
 <PSARC-ext@sun.com>; Sat, 26 Sep 2009 23:50:53 +0000 (GMT)
Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com
 (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul  2 2009))
 id <0KQL00200RGUHO00@mail-amer.sun.com> for PSARC-ext@sun.com
 (ORCPT PSARC-ext@sun.com); Sat, 26 Sep 2009 17:50:53 -0600 (MDT)
Received: from vpn-129-150-64-152.east.sun.com ([unknown] [129.150.64.152])
 by mail-amer.sun.com
 (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul  2 2009))
 with ESMTPSA id <0KQL00LSURKRLU80@mail-amer.sun.com> for PSARC-ext@sun.com
 (ORCPT PSARC-ext@sun.com); Sat, 26 Sep 2009 17:50:53 -0600 (MDT)
Date: Sat, 26 Sep 2009 17:50:49 -0600
From: "Mark A. Carlson" <Mark.Carlson@sun.com>
Subject: Re: Dynamical assembling the bootpath for iSCSI boot on sparc [PSARC
 2009/480 FastTrack timeout 09/23/2009]
In-reply-to: <4ABAED60.7040502@sun.com>
Sender: Mark.Carlson@sun.com
To: PSARC-ext <PSARC-ext@sun.com>
Message-id: <4ABEA8D9.6020902@sun.com>
MIME-version: 1.0
Content-type: multipart/alternative;
 boundary="Boundary_(ID_+ZvIUMQAzHdHU+QCaMDasw)"
X-PMX-Version: 5.4.1.325704
References: <4AB11F1F.6080003@sun.com> <4ABACFD4.8050109@sun.com>
 <4ABAED60.7040502@sun.com>
User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812)
Status: RO
Content-Length: 5944

This is a multi-part message in MIME format.

--Boundary_(ID_+ZvIUMQAzHdHU+QCaMDasw)
Content-type: text/plain; CHARSET=US-ASCII; format=flowed
Content-transfer-encoding: 7BIT

OK, as the timer has expired, the issues addressed and acknowledged,
I am marking this case closed approved.

Thanks,

-- mark

Garrett D'Amore wrote:
> Mark A. Carlson wrote:
>> The enclosed updated proposal has been placed in the case directory
>> documenting the discussions and decisions in this thread.
>>
>> Changes made to the spec include:
>> 1. The name of the project has been changed to
>>    "Dynamical assembling the bootpath for iSCSI boot on sparc"
>>
>> 2. The proposed solution in the original case is withdrawn.
>>
>> 3. 3 options are listed with a detailed explanation/justification
>>   of each one, including  the prposed option 3 of dynamical
>>   assembling the bootpath for iSCSI boot.
>>
>> 4. The Interfaces section has been updated to remove the boot_archive 
>> interface
>>   and an additional import of the property from OBP used in 
>> assembling the
>>   bootpath has been added.
>>
>> Anybody need more time?
>
> The approach chosen seems like a hack to me, but if it works, I'm ok 
> with it.   (I still feel like the objections raised by the OBP group 
> here are a bit... ahem... arbitrary.   But I also think the fact that 
> we still have split sd and ssd drivers is a very unfortunate 
> circumstance.)
>
>    - Garrett
>
>>
>> -- mark
>>
>> Garrett D'Amore wrote:
>>> This case has failed to meet the "obviousness" requirements for 
>>> self-review.  At PSARC today, the agreement was to turn this into a 
>>> fast track with a timer set for Sept 23.  I've updated the IAM file 
>>> to reflect this.
>>>
>>>    - Garrett
>>> _______________________________________________
>>> opensolaris-arc mailing list
>>> opensolaris-arc@opensolaris.org
>>
>
> _______________________________________________
> opensolaris-arc mailing list
> opensolaris-arc@opensolaris.org

-- 
<http://www.sun.com> 	* Mark A. Carlson *
Sr. Architect

*Systems Group*
Phone x69559 / 303-223-6139
Email Mark.Carlson@Sun.COM
	



--Boundary_(ID_+ZvIUMQAzHdHU+QCaMDasw)
Content-type: text/html; CHARSET=US-ASCII
Content-transfer-encoding: 7BIT

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
  <meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
<font size="+1">OK, as the timer has expired, the issues addressed and
acknowledged,<br>
I am marking this case closed approved.<br>
<br>
Thanks,<br>
<br>
-- mark<br>
</font><br>
Garrett D'Amore wrote:
<blockquote cite="mid:4ABAED60.7040502@sun.com" type="cite">Mark A.
Carlson wrote:
  <br>
  <blockquote type="cite">The enclosed updated proposal has been placed
in the case directory
    <br>
documenting the discussions and decisions in this thread.
    <br>
    <br>
Changes made to the spec include:
    <br>
1. The name of the project has been changed to
    <br>
&nbsp;&nbsp; "Dynamical assembling the bootpath for iSCSI boot on sparc"
    <br>
    <br>
2. The proposed solution in the original case is withdrawn.
    <br>
    <br>
3. 3 options are listed with a detailed explanation/justification
    <br>
&nbsp; of each one, including&nbsp; the prposed option 3 of dynamical
    <br>
&nbsp; assembling the bootpath for iSCSI boot.
    <br>
    <br>
4. The Interfaces section has been updated to remove the boot_archive
interface
    <br>
&nbsp; and an additional import of the property from OBP used in assembling
the
    <br>
&nbsp; bootpath has been added.
    <br>
    <br>
Anybody need more time?
    <br>
  </blockquote>
  <br>
The approach chosen seems like a hack to me, but if it works, I'm ok
with it.&nbsp;&nbsp; (I still feel like the objections raised by the OBP group
here are a bit... ahem... arbitrary.&nbsp;&nbsp; But I also think the fact that
we still have split sd and ssd drivers is a very unfortunate
circumstance.)
  <br>
  <br>
&nbsp;&nbsp; - Garrett
  <br>
  <br>
  <blockquote type="cite"><br>
-- mark
    <br>
    <br>
Garrett D'Amore wrote:
    <br>
    <blockquote type="cite">This case has failed to meet the
"obviousness" requirements for self-review.&nbsp; At PSARC today, the
agreement was to turn this into a fast track with a timer set for Sept
23.&nbsp; I've updated the IAM file to reflect this.
      <br>
      <br>
&nbsp;&nbsp; - Garrett
      <br>
_______________________________________________
      <br>
opensolaris-arc mailing list
      <br>
<a class="moz-txt-link-abbreviated" href="mailto:opensolaris-arc@opensolaris.org">opensolaris-arc@opensolaris.org</a>
      <br>
    </blockquote>
    <br>
  </blockquote>
  <br>
_______________________________________________
  <br>
opensolaris-arc mailing list
  <br>
<a class="moz-txt-link-abbreviated" href="mailto:opensolaris-arc@opensolaris.org">opensolaris-arc@opensolaris.org</a>
  <br>
</blockquote>
<br>
<div class="moz-signature">-- <br>
<title></title>
<table border="0" cellpadding="0" cellspacing="0" width="519">
  <tbody>
    <tr valign="top">
      <td height="121" width="98"><a moz-do-not-send="true"
 href="http://www.sun.com"><img moz-do-not-send="true"
 src="http://www.sun.com/emrkt/sigs/6h_top.gif" border="0" height="92"
 width="98"></a></td>
      <td style="font-family: Arial; font-size: 10px;" height="121"
 width="249"><b> Mark A. Carlson </b><br>
Sr. Architect<br>
      <br>
      <b>Systems Group</b><br>
Phone x69559 / 303-223-6139<br>
Email <a moz-do-not-send="true" class="moz-txt-link-abbreviated"
 href="mailto:Mark.Carlson@Sun.COM">Mark.Carlson@Sun.COM</a><br>
      </td>
      <td style="font-family: Arial; font-size: 10px;" width="172"><img
 moz-do-not-send="true" src="http://www.sun.com/emrkt/sigs/q12.gif"
 height="118" width="172"></td>
    </tr>
  </tbody>
</table>
<div class="moz-signature"><br>
</div>
</div>
</body>
</html>

--Boundary_(ID_+ZvIUMQAzHdHU+QCaMDasw)--

