From markcarl@sac.sfbay.sun.com Tue Sep 4 07:57:01 2007 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l84Ev1aK009120 for ; Tue, 4 Sep 2007 07:57:01 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l84EsHEY008462; Tue, 4 Sep 2007 07:54:17 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JNU00N01MQH4J00@nwk-avmta-2.sfbay.sun.com>; Tue, 04 Sep 2007 07:54:17 -0700 (PDT) Received: from sfbaymail1sca.SFBay.Sun.COM ([129.145.154.35]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JNU00HKIMQGGW80@nwk-avmta-2.sfbay.sun.com>; Tue, 04 Sep 2007 07:54:16 -0700 (PDT) Received: from sac.sfbay.sun.com (new-sac.SFBay.Sun.COM [129.146.175.65]) by sfbaymail1sca.SFBay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l84EsFj6012336; Tue, 04 Sep 2007 07:54:15 -0700 (PDT) Received: from sac.sfbay.sun.com (localhost [127.0.0.1]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l84Euvro009088; Tue, 04 Sep 2007 07:56:57 -0700 (PDT) Received: (from markcarl@localhost) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id l84Euvqm009084; Tue, 04 Sep 2007 07:56:57 -0700 (PDT) Date: Tue, 04 Sep 2007 07:56:57 -0700 (PDT) From: Mark Carlson Subject: [s]sd-config-list version 2 and retry count tuning [PSARC/2007/505 FastTrack timeout 09/11/2007] To: PSARC-ext@Sun.COM Cc: Grant.Zhang@Sun.COM, Larry.Liu@Sun.COM Message-id: <200709041456.l84Euvqm009084@sac.sfbay.sun.com> Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 10457 Template Version: @(#)sac_nextcase 1.64 07/13/07 SMI This information is Copyright 2007 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: [s]sd-config-list version 2 and retry count tuning 1.2. Name of Document Author/Supplier: Author: Larry Liu 1.3 Date of This Document: 04 September, 2007 4. Technical Description Background ---------- Solaris 8/9 has a tunable [s]sd_retry_count which specifies the number of times a disk operation should be retried before failure is returned. [s]sd_retry_count can be set by adding the following line to the /etc/system file set sd:sd_retry_count=5 [s]sd is a system wide driver tunable in Solaris 8/9: once set, all [s]sd device nodes in the system use the same value of retry count. A few third party multipathing software(e.g Veritas DMP, EMC Powerpath) depend on [s]sd driver's error handling ability for path failover operations. With ssd default setting of IO timeout (60 seconds) and retry counts(3), it often takes more than 3 minutes for the third party multipathing software to detect the path error and failover. This prevents some large customers from migrating to s10. [s]sd_retry_count needs to be tuned to achieve a quicker failover. [s]sd-config-list is a [s]sd (7D) driver property which can be used to configure a set of disk behaviors on a per disk type basis. This property is matched to the vendor ID and product ID strings in the device's SCSI INQUIRY data. It was first introduced in PSARC 1999/015 (delayed retries), with an interface level of Partner Private. PSARC 2001/692 added minimum throttle setting support to ssd-config-list property: the minimum throttle can be set on per device type level via ssd-config-list property in ssd.conf. Similarly PSARC 2001/693 approved the use of ssd-config-list to enable/disable disk sorting. Both of the above cases were approved as Project Private interfaces. PSARC 2002/294 (SCSI LOGICAL UNIT RESET) extended [s]sd-config-list property to support scsi lun reset, with an interface level of Project Private. An example that enables LOGICAL UNIT RESET use is: [s]sd-config-list= "SUN T4", "t4-data"; t4-data=1,0x20000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1; The first number of t4-data is the version of the [s]sd-config-list. Currently only version 1 is supported. The second number of t4-data is a bit mask of values to set. Bits 0-17 are already defined and each bit identifies a particular disk behavior. For example, bit 17 is defined for LOGICAL UNIT RESET support. Problem ------- There are two problems this fasttrack would like to address: 1. Solaris 10 and above currently do not support tuning [s]sd_retry_count. This has been a problem for customers who are willing to migrate to S10 but still use these third party multipathing software. 2. Due to historical reasons, the definitions of bit positions in [s]sd-config-list are different across platforms: for example, fabricate device id is defined as bit 3 on Sparc but as bit 2 on x86/x64. The inconsistency in bit position definition is confusing and error-prone to [s]sd-config-list users. Please refer to Appendix A for details on the bit fields currently defined. Proposed Solution ----------------- The proposal is to introduce a new version (2) of [s]sd-config-list property which has a unified definition of bit fields across platforms. The syntax of [s]sd-config-list version 2 is the same as that of [s]sd-config-list version 1. Another part of the proposal is to add a new bit in the [s]sd-config-list version 2 proposed above. This new bit is defined for setting retry count of [s]sd driver for the disk devices with a matching combination of VID/PID. Please refer to Technical Details section for sepcific bit field definitions for [s]sd-config-list version 2. [s]sd driver will be enhanced to support both version 1 and 2 of [s]sd-config-list property. Version 1 is kept unchanged for backward compatibility. For a given vender ID and product ID string, if both version 1 and version 2 are found, version 2 will take effect. Interface name Commitment Comments --------------------------------------------------------------------- IO timeout Committed DDI property on sd device node retries bit in that specifies IO timeout retry sd-config-list count for the attached disk. version 2 Other bits in Project Private DDI property on sd device node sd-config-list that specifies various behaviors version 2 for the attached disk. IO timeout Committed DDI property on ssd device node retries bit in that specifies IO timeout retry ssd-config-list count for the attached disk. version 2 Other bits in Project Private DDI property on ssd device node ssd-config-list that specifies various behaviors version 2 for the attached disk. Technical Details ----------------- Bit field definition for [s]sd-config-list version 2: _______________________________________ |Bit|sd(sparc) ssd(Sparc) sd(x86/x64) | |___|_________________________________| | 0 | max throttle | |___|_________________________________| | 1 | min throttle | |___|_________________________________| | 2 | controller type | |___|_________________________________| | 3 | reservation release time | |___|_________________________________| | 4 | disable disksort | |___|_________________________________| | 5 | enable LUN reset | |___|_________________________________| | 6 | not ready retries | |___|_________________________________| | 7 | busy retries | |___|_________________________________| | 8 | reset retries | |___|_________________________________| | 9 | IO timeout retries | |___|_________________________________| | 10| cache is non-volatile | |___|_________________________________| Those bits that only exist in version 1 are obsolete in version 2. Release Binding --------------- Micro release/patch binding is requested References ---------- PSARC 1999/015 delayed retries PSARC 2002/294 SCSI LOGICAL UNIT RESET PSARC 2001/692 Per-Disk-Device Minimum Throttle Setting PSARC 2001/693 Per-Disk-Device Disabling of disksort http://software.emc.com/images/software/products/software_az/expanded_images/emc_powerpath.gif 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: FastTrack Appendix A: Bit field definitions for [s]sd-config-list version 1. ________________________________________________________ |Bit|sd (sparc) |ssd (Sparc) |sd/ssd (x86/x64) | |___|________________|________________|_________________| | | max | max | max | | 0 | throttle | throttle | throttle | |___|________________|________________|_________________| | | controller | not ready | controller | | 1 | type | retries | type | |___|________________|________________|_________________| | | not ready | busy | fabricate | | 2 | retries | retries | device id | |___|________________|________________|_________________| | | fabricate | fabricate | disable | | 3 | device id | device id | caching | |___|________________|________________|_________________| | | disable | disable | play | | 4 | caching | caching | BCD | |___|________________|________________|_________________| | | busy | controller | read sub- | | 5 | retries | type | channel BCD | |___|________________|________________|_________________| | | play | play | read TOC | | 6 | BCD | BCD | TRK BCD | |___|________________|________________|_________________| | | read sub- | read sub- | read TOC | | 7 | channel BCD | channel BCD | ADDR BCD | |___|________________|________________|_________________| | | read TOC | read TOC | no READ_HDR | | 8 | TRK BCD | TRK BCD | | |___|________________|________________|_________________| | | read TOC | read TOC | read CD XD4 | | 9 | ADDR BCD | ADDR BCD | | |___|________________|________________|_________________| | | no READ_HDR | no READ_HDR | not ready | | 10| | | retries | |___|________________|________________|_________________| | | read CD XD4 | read CD XD4 | busy retries | | 11| | | | |___|________________|________________|_________________| | | reset retries | reset retries | reset retries | | 12| | | | |___|________________|________________|_________________| | | reservation | reservation | reservation | | 13| release time | release time | release time | |___|________________|________________|_________________| | | TUR check | TUR check | TUR check | | 14| | | | |___|________________|________________|_________________| | | min throttle | min throttle | min throttle | | 15| | | | |___|________________|________________|_________________| | | disable | disable | disable | | 16| disksort | disksort | disksort | |___|________________|________________|_________________| | | enable LUN | enable LUN | enable LUN | | 17| reset | reset | reset | |___|________________|________________|_________________| | | cache is non- | cache is non- | cache is non- | | 18| volatile | volatile | volatile | |___|________________|________________|_________________| 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: FastTrack 6.6. ARC Exposure: open From sommerfeld@sun.com Tue Sep 4 09:29:35 2007 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l84GTZwt012316 for ; Tue, 4 Sep 2007 09:29:35 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l84GQn9a010391; Tue, 4 Sep 2007 09:26:51 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JNU0050ZR0RHF00@brm-avmta-1.central.sun.com>; Tue, 04 Sep 2007 10:26:51 -0600 (MDT) Received: from dm-east-01.east.sun.com ([129.148.9.192]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JNU00HT6R0Q55C0@brm-avmta-1.central.sun.com>; Tue, 04 Sep 2007 10:26:50 -0600 (MDT) Received: from thunk.east.sun.com (thunk.East.Sun.COM [129.148.174.66]) by dm-east-01.east.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id l84GQlID037224; Tue, 04 Sep 2007 12:26:47 -0400 (EDT) Received: from [IPv6:::1] (localhost [IPv6:::1]) by thunk.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id l84GQlAm007709; Tue, 04 Sep 2007 12:26:47 -0400 (EDT) Date: Tue, 04 Sep 2007 12:26:46 -0400 From: Bill Sommerfeld Subject: Re: [s]sd-config-list version 2 and retry count tuning [PSARC/2007/505 FastTrack timeout 09/11/2007] In-reply-to: <200709041456.l84Euvqm009084@sac.sfbay.sun.com> To: Mark Carlson Cc: PSARC-ext@sun.com, Larry.Liu@sun.com, Grant.Zhang@sun.com Message-id: <1188923206.7578.17.camel@thunk> MIME-version: 1.0 X-Mailer: Evolution 2.10.2 Content-type: text/plain Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200709041456.l84Euvqm009084@sac.sfbay.sun.com> Status: RO Content-Length: 1117 The property encoding being revised seems particularly error prone -- you need to define a bitmask and then an sequence of values where bit N of the bitmask tells the driver to look at value N in the sequence. If you're going to revise the encoding to be consistent across platforms, it also seems like it might be a good opportunity to make it less administrator-hostile in other ways. On Tue, 2007-09-04 at 07:56 -0700, Mark Carlson wrote: > Those bits that only exist in version 1 are obsolete in version 2. we generally closely review proposals which remove functionality or declare it obsolete; can you enumerate the set of removed bits and explain why it's appropriate to remove each tunable? Will the removed bits be turned into no-ops, or will they continue to be settable via a version 1 encoding? > For a given vender ID and > product ID string, if both version 1 and version 2 are found, > version 2 will take effect. I'm missing how that's possible given the encoding described in this case and in 2001/692 -- can you show an excerpt of [s]sd.conf syntax showing this ambiguity? - Bill From Larry.Liu@sun.com Wed Sep 5 04:21:13 2007 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l85BLCgf012439 for ; Wed, 5 Sep 2007 04:21:12 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l85BIQXh002915; Wed, 5 Sep 2007 04:18:27 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JNW003097EQHQ00@brm-avmta-1.central.sun.com>; Wed, 05 Sep 2007 05:18:26 -0600 (MDT) Received: from sineb-mail-1.sun.com ([192.18.19.6]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JNW00C8Z7EKP8B0@brm-avmta-1.central.sun.com>; Wed, 05 Sep 2007 05:18:21 -0600 (MDT) Received: from fe-apac-06.sun.com (fe-apac-06.sun.com [192.18.19.177] (may be forged)) by sineb-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l85BIK7q003834; Wed, 05 Sep 2007 11:18:20 +0000 (GMT) Received: from conversion-daemon.mail-apac.sun.com by mail-apac.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JNW004017CLCL00@mail-apac.sun.com> (original mail from Larry.Liu@Sun.COM) ; Wed, 05 Sep 2007 19:18:19 +0800 (SGT) Received: from [129.158.144.81] by mail-apac.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JNW00BW97EIX732@mail-apac.sun.com>; Wed, 05 Sep 2007 19:18:19 +0800 (SGT) Date: Wed, 05 Sep 2007 19:18:59 +0800 From: yu larry liu Subject: Re: [s]sd-config-list version 2 and retry count tuning [PSARC/2007/505 FastTrack timeout 09/11/2007] In-reply-to: <1188923206.7578.17.camel@thunk> Sender: Larry.Liu@sun.com To: Bill Sommerfeld Cc: Mark Carlson , PSARC-ext@sun.com, Grant.Zhang@sun.com Message-id: <46DE90A3.7050106@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200709041456.l84Euvqm009084@sac.sfbay.sun.com> <1188923206.7578.17.camel@thunk> User-Agent: Thunderbird 2.0b2 (X11/20070227) Status: RO Content-Length: 2500 > The property encoding being revised seems particularly error prone -- > you need to define a bitmask and then an sequence of values where bit N > of the bitmask tells the driver to look at value N in the sequence. > > We didn't change the syntax for property encoding from version 1. I agree with you it is error prone. But it is simple and efficient. And it leaves door open to future version development. If splitting the properties, it will be complex for the combination of VID, PID and single property. Any suggestion? > If you're going to revise the encoding to be consistent across > platforms, it also seems like it might be a good opportunity to make it > less administrator-hostile in other ways. > > On Tue, 2007-09-04 at 07:56 -0700, Mark Carlson wrote: > >> Those bits that only exist in version 1 are obsolete in version 2. >> > > we generally closely review proposals which remove functionality or > declare it obsolete; can you enumerate the set of removed bits and > explain why it's appropriate to remove each tunable? > > fabricate device id: This is the case of antiquated Sun disk drives. disable caching: This is a historic behavior. play BCD: read sub-channel BCD: read TOC TRK BCD: These three are for binary-coded decimal CD device. no READ_HDR: This is not used at all. Even in version 1 read CD XD4: This is to use command supported by 1st generation atapi drives TUR check: This is specific to some Seagate elite1 TQ devices get hung with disconnect/reconnect timeouts when they receive a TUR and the queue is not empty. All the bits above are Project Private. And they are for some specific old devices. In version 2, we plan to only support those properties defined in sd_tunables table which is commonly used. > Will the removed bits be turned into no-ops, or will they continue to be > settable via a version 1 encoding? > > They are continuously supported via a version 1 encoding. >> For a given vender ID and >> product ID string, if both version 1 and version 2 are found, >> version 2 will take effect. >> > > I'm missing how that's possible given the encoding described in this > case and in 2001/692 -- can you show an excerpt of [s]sd.conf syntax > showing this ambiguity? > [s]sd-config-list = "SUN T4", "t4-data"; "SUN T4", "t4-data1"; t4-data=1,0x20000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1; t4-data1=2,0x20000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1; Thanks, Larry From gww@eng.sun.com Wed Sep 5 09:55:32 2007 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l85GtWfl022271 for ; Wed, 5 Sep 2007 09:55:32 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l85Gqkmh027852 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@Sun.COM>; Wed, 5 Sep 2007 09:52:46 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JNW00A0HMVYWX00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@Sun.COM (ORCPT PSARC-ext@Sun.COM); Wed, 05 Sep 2007 09:52:46 -0700 (PDT) Received: from engmail3mpk.sfbay.Sun.COM ([129.146.11.26]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JNW008UZMVXAA30@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@Sun.COM (ORCPT PSARC-ext@Sun.COM); Wed, 05 Sep 2007 09:52:45 -0700 (PDT) Received: from marduk.eng.sun.com (marduk.SFBay.Sun.COM [129.146.108.224]) by engmail3mpk.sfbay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l85GqhG8016575; Wed, 05 Sep 2007 09:52:43 -0700 (PDT) Received: from marduk.eng.sun.com (localhost [127.0.0.1]) by marduk.eng.sun.com (8.13.6+Sun/8.12.11) with ESMTP id l85GrluP013955; Wed, 05 Sep 2007 09:53:47 -0700 (PDT) Received: (from gww@localhost) by marduk.eng.sun.com (8.13.6+Sun/8.12.11/Submit) id l85GrlO7013954; Wed, 05 Sep 2007 09:53:47 -0700 (PDT) Date: Wed, 05 Sep 2007 09:53:47 -0700 (PDT) From: Gary Winiger Subject: Re: [s]sd-config-list version 2 and retry count tuning [PSARC/2007/505 FastTrack timeout 09/11/2007] To: PSARC-ext@sun.com, markcarl@sac.sfbay.sun.com Cc: Grant.Zhang@sun.com, Larry.Liu@sun.com Message-id: <200709051653.l85GrlO7013954@marduk.eng.sun.com> Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 444 > Problem > ------- > There are two problems this fasttrack would like to address: > 1. Solaris 10 and above currently do not support tuning > [s]sd_retry_count. This has been a problem for customers who > are willing to migrate to S10 but still use these third > party multipathing software. Do I understand correctly that this case proposes to replace /etc/system:[s]sd_retry_count with a proper interface? (viz. [s]sd.conf ;-) Gary.. From Grant.Zhang@Sun.COM Wed Sep 5 16:00:14 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l85N0DJO006386 for ; Wed, 5 Sep 2007 16:00:14 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l85MvLT4000393 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Thu, 6 Sep 2007 06:57:27 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JNX0050B3RO7000@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.COM); Wed, 05 Sep 2007 15:57:24 -0700 (PDT) Received: from sineb-mail-1.sun.com ([192.18.19.6]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JNX007TN3RNSEC0@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.COM); Wed, 05 Sep 2007 15:57:24 -0700 (PDT) Received: from fe-apac-05.sun.com (fe-apac-05.sun.com [192.18.19.176] (may be forged)) by sineb-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l85MvN0Y015413 for ; Wed, 05 Sep 2007 22:57:23 +0000 (GMT) Received: from conversion-daemon.mail-apac.sun.com by mail-apac.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JNX001013LG5D00@mail-apac.sun.com> (original mail from Grant.Zhang@Sun.COM) for PSARC-ext@Sun.COM (ORCPT PSARC-ext@Sun.COM); Thu, 06 Sep 2007 06:57:23 +0800 (SGT) Received: from [192.168.0.144] ([125.33.50.214]) by mail-apac.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JNX00D0A3RI4ZRG@mail-apac.sun.com>; Thu, 06 Sep 2007 06:57:22 +0800 (SGT) Date: Thu, 06 Sep 2007 06:57:21 +0800 From: Grant Zhang Subject: Re: [s]sd-config-list version 2 and retry count tuning [PSARC/2007/505 FastTrack timeout 09/11/2007] In-reply-to: <200709051653.l85GrlO7013954@marduk.eng.sun.com> Sender: Grant.Zhang@Sun.COM To: Gary Winiger Cc: PSARC-ext@Sun.COM, markcarl@sac.sfbay.sun.com, Larry.Liu@Sun.COM Message-id: <46DF3451.1070908@sun.com> Organization: Sun Microsystems MIME-version: 1.0 Content-type: text/plain; charset=GB2312 Content-transfer-encoding: 8BIT X-PMX-Version: 5.2.0.264296 References: <200709051653.l85GrlO7013954@marduk.eng.sun.com> User-Agent: Thunderbird 2.0.0.6 (Windows/20070728) Status: RO Content-Length: 780 Gary Winiger дµÀ: >> Problem >> ------- >> There are two problems this fasttrack would like to address: >> 1. Solaris 10 and above currently do not support tuning >> [s]sd_retry_count. This has been a problem for customers who >> are willing to migrate to S10 but still use these third >> party multipathing software. >> > > Do I understand correctly that this case proposes > to replace /etc/system:[s]sd_retry_count with a proper > interface? (viz. [s]sd.conf ;-) > > Gary.. > Yes. In fact, /etc/system:[s]sd_retry_count no longer works in S10 and above. IIRC, system-wide [s]sd_retry_count was dropped during the merge of sd and ssd code. One objective of this case is to provide a mechansim to tune retry count on per device basis(identified by VID/PID). Grant From sommerfeld@sun.com Wed Sep 5 16:32:41 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l85NWe1U007040 for ; Wed, 5 Sep 2007 16:32:40 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l85NTj2S012694; Thu, 6 Sep 2007 07:29:55 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JNX0060159UO600@nwk-avmta-2.sfbay.sun.com>; Wed, 05 Sep 2007 16:29:54 -0700 (PDT) Received: from dm-east-02.east.sun.com ([129.148.13.5]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JNX00L1F59TK180@nwk-avmta-2.sfbay.sun.com>; Wed, 05 Sep 2007 16:29:53 -0700 (PDT) Received: from thunk.east.sun.com (thunk.East.Sun.COM [129.148.174.66]) by dm-east-02.east.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id l85NTogL063471; Wed, 05 Sep 2007 19:29:50 -0400 (EDT) Received: from [IPv6:::1] (localhost [IPv6:::1]) by thunk.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id l85NToFV014728; Wed, 05 Sep 2007 19:29:50 -0400 (EDT) Date: Wed, 05 Sep 2007 19:29:49 -0400 From: Bill Sommerfeld Subject: Re: [s]sd-config-list version 2 and retry count tuning [PSARC/2007/505 FastTrack timeout 09/11/2007] In-reply-to: <46DE90A3.7050106@sun.com> To: yu larry liu Cc: Mark Carlson , PSARC-ext@sun.com, Grant.Zhang@sun.com Message-id: <1189034989.12483.46.camel@thunk> MIME-version: 1.0 X-Mailer: Evolution 2.10.2 Content-type: text/plain Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200709041456.l84Euvqm009084@sac.sfbay.sun.com> <1188923206.7578.17.camel@thunk> <46DE90A3.7050106@sun.com> Status: RO Content-Length: 1194 On Wed, 2007-09-05 at 19:18 +0800, yu larry liu wrote: > We didn't change the syntax for property encoding from version 1. I > agree with you it is error prone. But it is simple and efficient. And it > leaves door open to future version development. If splitting the > properties, it will be complex for the combination of VID, PID and > single property. Any suggestion? I'm not 100% familiar with the constraints on entries in driver.conf files but an attribute=value encoding like: t4-data="nonvolatile-cache=1,timeout-retry-count=42,min-throttle=2,max-throttle=17"; would be way better than t4-data=2,0x303,2,17,0,0,0,0,0,0,0,42,1 (I *think* I got that right). (Maybe the answer here is to provide tools or scripts to construct and decode these entries, which wouldn't be this project but a worthy followon). > In version 2, we plan to only support those properties defined in > sd_tunables table which is commonly used. The word "support" is ambiguous and should generally be avoided in specifications. I think you mean "allow". BTW, it looks like the spec is missing a reference to 2007/053 (for the "non-volatile cache" bit) and perhaps a few others. - Bill From Larry.Liu@sun.com Wed Sep 5 23:13:30 2007 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l866DU6f012885 for ; Wed, 5 Sep 2007 23:13:30 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l8667LTu027161; Wed, 5 Sep 2007 23:10:45 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JNX00007NTUL000@brm-avmta-1.central.sun.com>; Thu, 06 Sep 2007 00:10:42 -0600 (MDT) Received: from sineb-mail-1.sun.com ([192.18.19.6]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JNX002PSNTSG8B0@brm-avmta-1.central.sun.com>; Thu, 06 Sep 2007 00:10:41 -0600 (MDT) Received: from fe-apac-06.sun.com (fe-apac-06.sun.com [192.18.19.177] (may be forged)) by sineb-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l866Ad2b011245; Thu, 06 Sep 2007 06:10:39 +0000 (GMT) Received: from conversion-daemon.mail-apac.sun.com by mail-apac.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JNX00D01NPTM300@mail-apac.sun.com> (original mail from Larry.Liu@Sun.COM) ; Thu, 06 Sep 2007 14:10:39 +0800 (SGT) Received: from [129.158.144.81] by mail-apac.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JNX00BB2NTBX767@mail-apac.sun.com>; Thu, 06 Sep 2007 14:10:24 +0800 (SGT) Date: Thu, 06 Sep 2007 14:11:04 +0800 From: yu larry liu Subject: Re: [s]sd-config-list version 2 and retry count tuning [PSARC/2007/505 FastTrack timeout 09/11/2007] In-reply-to: <1189034989.12483.46.camel@thunk> Sender: Larry.Liu@sun.com To: Bill Sommerfeld Cc: Mark Carlson , PSARC-ext@sun.com, Grant.Zhang@sun.com Message-id: <46DF99F8.2000400@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200709041456.l84Euvqm009084@sac.sfbay.sun.com> <1188923206.7578.17.camel@thunk> <46DE90A3.7050106@sun.com> <1189034989.12483.46.camel@thunk> User-Agent: Thunderbird 2.0b2 (X11/20070227) Status: RO Content-Length: 1989 >> We didn't change the syntax for property encoding from version 1. I >> agree with you it is error prone. But it is simple and efficient. And it >> leaves door open to future version development. If splitting the >> properties, it will be complex for the combination of VID, PID and >> single property. Any suggestion? >> > > I'm not 100% familiar with the constraints on entries in driver.conf > files but an attribute=value encoding like: > > t4-data="nonvolatile-cache=1,timeout-retry-count=42,min-throttle=2,max-throttle=17"; > > would be way better than > > t4-data=2,0x303,2,17,0,0,0,0,0,0,0,42,1 > > (I *think* I got that right). > > (Maybe the answer here is to provide tools or scripts to construct and > decode these entries, which wouldn't be this project but a worthy > followon). > > Yes, we could do this in sd.conf. And this seems much more friendly though not quite easy for implementation. However, both sd.conf and st.conf, (I believe there might be some other driver.conf) have the same situation. Similar syntax is used in st.conf when adding support to new tape device. So, actually, we want a generic friendly way in driver.conf, not specific in sd.conf, for property definition: for a given device type, identified by a 'VID + PID' combination, we would like the ability to define multiple properties associated with such device type; a version number is also desirable here for future upgrade. It is probably out of the scope of this project but worthy to dig further. >> In version 2, we plan to only support those properties defined in >> sd_tunables table which is commonly used. >> > > The word "support" is ambiguous and should generally be avoided in > specifications. I think you mean "allow". > > Yes. Thanks for point it out. I will correct it. > BTW, it looks like the spec is missing a reference to > 2007/053 (for the "non-volatile cache" bit) and perhaps a few others. > Will check and add. Thanks Larry