From cth@sac.sfbay.sun.com Mon Jul 21 07:00:30 2008 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m6LE0UTR015006 for ; Mon, 21 Jul 2008 07:00:30 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m6LE0UUt047297; Mon, 21 Jul 2008 08:00:30 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0K4D0060R08TJD00@nwk-avmta-2.sfbay.sun.com>; Mon, 21 Jul 2008 07:00:29 -0700 (PDT) Received: from dm-sfbay-02.sfbay.sun.com ([129.146.11.31]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0K4D005KX08SBV10@nwk-avmta-2.sfbay.sun.com>; Mon, 21 Jul 2008 07:00:28 -0700 (PDT) Received: from sac.sfbay.sun.com (new-sac.SFBay.Sun.COM [129.146.175.65]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m6LE0SQ1020383; Mon, 21 Jul 2008 07:00:28 -0700 (PDT) Received: from sac.sfbay.sun.com (localhost [127.0.0.1]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m6LE0RFP015000; Mon, 21 Jul 2008 07:00:27 -0700 (PDT) Received: (from cth@localhost) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id m6LE0QZD014996; Mon, 21 Jul 2008 07:00:26 -0700 (PDT) Date: Mon, 21 Jul 2008 07:00:26 -0700 (PDT) From: Christopher Horne Subject: Improved [s]sd-config-list support [PSARC/2008/465 FastTrack timeout 07/30/2008] To: PSARC-ext@sun.com Message-id: <200807211400.m6LE0QZD014996@sac.sfbay.sun.com> Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 Status: RO Content-Length: 17247 I am sponsoring the following fasttrack for Li He (Nikko), with timeout set to 07/30/2008. Micro/patch binding is requested -Chris 1. Introduction 1.1. Project/Component Working Name: Improved [s]sd-config-list support 1.2. Name of Document Author/Supplier: Author: Nikko He 1.3. Date of This Document: 15, July 2008 4. Technical Description 4.1 Background Solaris 8/9 supported a system(4) [s]sd driver tunable called [s]sd_retry_count to specify the number of times a disk operation should be retried before returning failure (EIO). This tunable was adjusted by adding a line to /etc/system: set sd:sd_retry_count=5 Some third-party multipathing implementations (e.g Veritas DMP, and EMC Powerpath) developed dependencies on being able to tune error handling parameters: the default settings (60 second IO timeout, 3 retries) caused problems for their products. Without full understanding of the third-party dependencies, this tunable was removed in s10 by PSARC 2003/556 [7]. Now, the inability to tune driver error handling parameters is causing problems for third-party multipathing solutions and is preventing some large customers from migrating to s10. The current SD_BSY_TIMEOUT define is used to specify the delay before retry when busy status received. This is currently hard coded in the [s]sd driver. A customer is requesting to tune this variable to reduce delay in their environment. Based on the scsi_get_device_type_scsi_options(9F) interface defined by PSARC 1995/445 [1], 'Partner Private' [s]sd-config-list support was added to [s]sd(7D) as part of PSARC 1999/015 "delayed retries" [2]. This property is matched to the vendor ID and product ID strings in the device's SCSI INQUIRY data. This mechanism was subsequently used (Project Private) for PSARC 2001/692 [4] minimum throttle setting support PSARC 2001/693 [5] enable/disable disk sorting support PSARC 2002/294 [6] SCSI LOGICAL UNIT RESET support An example that enables LOGICAL UNIT RESET use is: ssd-config-list= "SUN T4", "t4-data"; t4-data=1,0x20000,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1; The first number in the t4-data array above is the version of the ssd-config-list. Currently only version 1 is supported. The second number of t4-data is a bit mask of values to set. Bits 0-17 are already defined and each bit identifies a particular disk behavior. When a particular bit is set in the mask, the value associated with the behavior is found using "bit_number + 2" as an index into the t4-data. In the example above, bit 17 is set indicating support for the LOGICAL UNIT RESET tunable, and the tunable value is 1. 4.2 Problem There are two problems this fasttrack would like to address: 1. The current bit-mask/index approach and the use of indirect properties are hard to use. In addition, due to historical reasons, the definitions of bit positions in [s]sd-config-list are different across platforms: for example, fabricate device id is defined as bit 3 on Sparc but as bit 2 on x86/x64. This inconsistency in bit position definition is confusing and error-prone. 2. Solaris 10 and above currently do not support tuning retry counts. This has been a problem for customers who are willing to migrate to S10, but want to continue using third party multipathing software. The proposed public tunable 'retries-timeout' meets this requirement. Similarly, current SD_BSY_TIMEOUT functionality needs to be tunable to meet customer requirements. The proposed public tunable 'delay-busy' meets this requirement. We need to improve the way [s]sd-config-list works. 4.3 Proposed Solution The proposal is to introduce a new backwards compatible format for [s]sd-config-list property values which uses a name-value pair string, in JSON text format [10], to provide device-specific tuning. The existing [s]sd-conf-list format is '"VIDPID-string", "property-name"'. The new [s]sd-config-list format is '"VIDPID-string", "JSON-text"'. The format of the tuple can be detected by looking for ':' in the second string. In addition to supporting existing private behaviors, support for tuning various forms of retry behavior will be added. Using the proposed [s]sd-config-list improvements, tuning device specific behavior in [s]sd.conf will look something like: sd-config-list = "SUN T4", "delay-busy:600, retries-timeout:6"; Please refer to section 4.4 for specific tunables supported by the improved [s]sd-config-list, and also refer to section 4.5 for string syntax definition. The [s]sd driver will be enhanced to support both the current and the improved [s]sd-config-list formats. The current format is kept unchanged for backward compatibility. Interface name Commitment Comments --------------------------------------------------------------------- sd-config-list Committed New JSON-text format, but only ssd-config-list the specific tunables mentioned below are committed. delay-busy: Committed tunable name in [s]sd-config-list JSON-text that specifies the delay before retry when busy: nsecs retries-timeout: Committed tunable name in [s]sd-config-list JSON-text that specifies the number of retries to perform on an io timeout: count. : ProjPrivate tunable name in [s]sd-config-list JSON-text string. This case supersedes PSARC 2007/505 [9], which was never delivered. 4.4 Supported 'JSON-text' Tunable Names Tunables names supported by improved [s]sd-config-list: _______________________________________________________ |Tunable_Name |Commitment |Data_Type| |_________________________________|___________|_________| | cache-nonvolatile | Private | BOOLEAN | |_________________________________|___________|_________| | controller-type | Private | UINT32 | |_________________________________|___________|_________| | delay-busy | Committed | UINT32 | |_________________________________|___________|_________| | disksort | Private | BOOLEAN | |_________________________________|___________|_________| | timeout-releasereservation | Private | UINT32 | |_________________________________|___________|_________| | reset-lun | Private | BOOLEAN | |_________________________________|___________|_________| | retries-busy | Private | UINT32 | |_________________________________|___________|_________| | retries-timeout | Committed | UINT32 | |_________________________________|___________|_________| | retries-notready | Private | UINT32 | |_________________________________|___________|_________| | retries-reset | Private | UINT32 | |_________________________________|___________|_________| | throttle-max | Private | UINT32 | |_________________________________|___________|_________| | throttle-min | Private | UINT32 | |_________________________________|___________|_________| The general format of a tunable name is '-'. When choosing a name for a new tunables, consideration should be given to wether the tunable fits into any of the existing catefories. timeout- maximum time something should take. delay- delay time before issuing a retry. retries- number of retries before failure. throttle- activity control. reset- reset control. Some of the private tunables supported by the current v1 mechanism (all private) are not being carried forward (same ones dropped by [9]). See Appendix B for private tunables which are now considered obsolete. 4.5 Future We define the format of a compatible JSON-text with reference to the grammar of JSON [10]. In the future, we plan on providing generic interfaces in libnvpair [11] to convert between nvlist and JSON text, and will also adjust the sd implementation to use those generic interfaces at that time. The JSON RFC [10] states that "A JSON parser MAY accept non-JSON forms or extensions". This proposal accepts the following non-JSON forms: o In addition to supporting member = string name-separator value defined in section 2.2 of the RFC, we support member = name name-separator value where name is alpha = %x41-5A / %x61-7A ; A-Z a-z minus = 0x2D ; - ubar = 0x5F ; _ digit1-9 = 0x31-39 ; 1-9 zero = 0x30 ;0 alphap = alpha / zero / digit1-9 / ubar / minus name = alpha *alphap The result is that we will accept member names without quotes. o If JSON text does not start/end with 'begin-object/end-object' or 'begin-array/end-array' as specified in section 2.2/2.3 of the the RFC, then 'begin-object/end-object' bracketing is assumed. These are common extensions. The future nvlist JSON text decode implementation will obtain its type information from a prototype nvlist supplied by the caller that describes the name-type information of all supported objects. 4.6 Release Binding Micro/patch binding is requested. 4.7 References [1] PSARC 1995/445 scsi-options per device type http://sac.sfbay/PSARC/1995/445 http://www.opensolaris.org/os/community/arc/caselog/1995/445 [2] PSARC 1999/015 delayed retries http://sac.sfbay/PSARC/1999/015 http://www.opensolaris.org/os/community/arc/caselog/1999/015 [3] PSARC 2000/016 Merged Disk Driver (s81_37(s9)) http://sac.sfbay/PSARC/2000/016 http://www.opensolaris.org/os/community/arc/caselog/2000/016 [4] PSARC 2001/692 Per-Disk-Device Minimum Throttle Setting http://sac.sfbay/PSARC/2001/692 http://www.opensolaris.org/os/community/arc/caselog/2001/692 [5] PSARC 2001/693 Per-Disk-Device Disabling of disksort http://sac.sfbay/PSARC/2001/693 http://www.opensolaris.org/os/community/arc/caselog/2001/693 [6] PSARC 2002/294 SCSI LOGICAL UNIT RESET http://sac.sfbay/PSARC/2002/294 http://www.opensolaris.org/os/community/arc/caselog/2002/294 [7] PSARC 2003/556 Common Solaris Target Disk Driver (4961447 s10_55) http://sac.sfbay/PSARC/2003/556 http://www.opensolaris.org/os/community/arc/caselog/2003/556 [8] PSARC 2006/710 scsi_get_device_type_string http://sac.sfbay/PSARC/2006/710 http://www.opensolaris.org/os/community/arc/caselog/2006/710 [9] PSARC 2007/505 [s]sd-config-list version 2 and retry count tuning http://sac.sfbay/PSARC/2007/505 http://www.opensolaris.org/os/community/arc/caselog/2007/505 [10]JavaScript Object Notation http://tools.ietf.org/html/rfc4627 http://www.json.org [11]PSARC 2000/212 libnvpair - A Name Value Pairs Library http://sac.sfbay/PSARC/2000/212 http://www.opensolaris.org/os/community/arc/caselog/2000/212 [12]EMC Powerpath information 4.8 Manpage changes See below. Appendix A: Bit definitions for current [s]sd-config-list (v1) Bit sd (sparc) ssd (Sparc) sd/ssd (x86/x64) 0 max_throttle max_throttle max_throttle 1 controller_type not_ready_retries controller_type 2 not_ready_retries busy_retries fabricate_device_id 3 fabricate_device_id fabricate_device_id disable_caching 4 disable_caching disable_caching play_BCD 5 busy_retries controller_type read_subchannel_BCD 6 play_BCD play_BCD read_TOC_TRK_BCD 7 read_subchannel_BCD read_subchannel_BCD read_TOC_ADDR_BCD 8 read_TOC_TRK_BCD read_TOC_TRK_BCD no_READ_HDR 9 read_TOC_ADDR_BCD read_TOC_ADDR_BCD read_CD_XD4 10 no_READ_HDR no_READ_HDR not_ready_retries 11 read_CD_XD4 read_CD_XD4 busy retries 12 reset_retries reset_retries reset_retries 13 reserv_release_time reserv_release_time reserv_release_time 14 TUR_check TUR_check TUR_check 15 min_throttle min_throttle min_throttle 16 disable_disksort disable_disksort disable_disksort 17 enable_LUN_reset enable_LUN_reset enable_LUN_reset 18 cache_is_nonvolatile cache_is_nonvolatile cache_is_nonvolatile Appendix B: Mapping of new tunable 'names' to current pseudo-names above NEW: CURRENT: cache-nonvolatile cache_is_nonvolatile controller-type controller_type delay-busy disksort disable_disksort timeout-releasereservation reservation_release_time reset-lun enable_LUN_reset retries-busy busy_retries retries-timeout retries-notready not_ready_retries retries-reset reset_retries throttle-max max_throttle throttle-min min_throttle TUR_check disable_caching fabricate_device_id no_READ_HDR play_BCD read_CD_XD4 read_TOC_ADDR_BCD read_TOC_TRK_BCD read_subchannel_BCD Appendix C: Changes to sd(7D) --- sd.orig Fri Jul 18 10:46:12 2008 +++ sd.new Fri Jul 18 11:21:16 2008 @@ -216,10 +216,54 @@ (Note: the default behavior for the SPARC-based sd driver prior to Solaris 9 was not to bind to optical devices.) + In addition to the above properties, some device-specific + tunables can be configured in sd.conf using the 'sd-config-list' + global property. The value of this property is a list of + duplets. The formal syntax is: + + sd-config-list = [, ]* ; + + where + + := "" , "" + + and + + := [, ]*; + = : + + The is the string that is returned by the target device + on a SCSI inquiry command. + + The contains one or more tunables to apply to + for the target devices with the specified . + + Each is a " : " pair. Supported + tunable names are: + + delay-busy: when busy, nsecs of delay before retry. + + retries-timeout: retries to perform on an IO timeout. + +EXAMPLES + + The following is an example of a global sd-config-list property: + + sd-config-list = + "SUN T4", "delay-busy:600, retries-timeout:6", + "SUN StorEdge_3510", "retries-timeout:3"; + FILES /kernel/drv/sd.conf driver configuration file /dev/dsk/cntndnsn block files Appendix D: Changes to ssd(7D) --- ssd.orig Fri Jul 18 10:46:37 2008 +++ ssd.new Fri Jul 18 11:22:05 2008 @@ -97,10 +97,54 @@ iostat(1M) even though the -p/-P option is specified. Regardless of this setting, disk IO statis- tics are always maintained. + In addition to the above properties, some device-specific + tunable can be configured in ssd.conf using the 'ssd-config-list' + global property. The value of this property is a list of + duplets. The formal syntax is: + + ssd-config-list = [, ]* ; + + where + + := "" , "" + + and + + := [, ]*; + = : + + The is the string that is returned by the target device + on a SCSI inquiry command. + + The contains one or more tunables to apply to + for the target devices with the specified . + + Each is a " : " pair. Supported + tunable names are: + + delay-busy: when busy, nsecs of delay before retry. + + retries-timeout: retries to perform on an IO timeout. + +EXAMPLES + + The following is an example of a global ssd-config-list property: + + sd-config-list = + "SUN T4", "delay-busy:600, retries-timeout:6", + "SUN StorEdge_3510", "retries-timeout:3"; + FILES ssd.conf Driver configuration file /dev/dsk/cntndnsn block files 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: FastTrack 6.6. ARC Exposure: open 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: FastTrack 6.6. ARC Exposure: open From Chris.Horne@sun.com Fri Jul 25 10:23:45 2008 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m6PHNjCU027223 for ; Fri, 25 Jul 2008 10:23:45 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m6PHNhC4016378 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Fri, 25 Jul 2008 10:23:45 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0K4K00E2JOBKBM00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.Com); Fri, 25 Jul 2008 10:23:44 -0700 (PDT) Received: from brmea-mail-3.sun.com ([192.18.98.34]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0K4K00EHFOBH2F00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.Com); Fri, 25 Jul 2008 10:23:41 -0700 (PDT) Received: from fe-amer-10.sun.com ([192.18.109.80]) by brmea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id m6PHNfIO016021 for ; Fri, 25 Jul 2008 17:23:41 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K4K00201L55YI00@mail-amer.sun.com> (original mail from Chris.Horne@Sun.COM) for PSARC-ext@Sun.Com (ORCPT PSARC-ext@Sun.Com); Fri, 25 Jul 2008 11:23:41 -0600 (MDT) Received: from sun.com ([129.150.33.79]) by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K4K00AF2OB00T30@mail-amer.sun.com>; Fri, 25 Jul 2008 11:23:30 -0600 (MDT) Date: Fri, 25 Jul 2008 11:23:24 -0600 From: Chris Horne Subject: Re: Improved [s]sd-config-list support [PSARC/2008/465 FastTrack timeout 07/30/2008] In-reply-to: <200807211400.m6LE0QZD014996@sac.sfbay.sun.com> Sender: Chris.Horne@sun.com To: Christopher Horne Cc: PSARC-ext@sun.com Message-id: <488A0C0C.40609@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-Accept-Language: en-us, en X-PMX-Version: 5.4.1.325704 References: <200807211400.m6LE0QZD014996@sac.sfbay.sun.com> User-Agent: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.4) Gecko/20040414 Status: RO Content-Length: 52 This case was approved at PSARC this week. -Chris From Chris.Horne@sun.com Fri Jul 25 10:26:14 2008 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m6PHQDen027365 for ; Fri, 25 Jul 2008 10:26:14 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m6PHQAnT011064 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Fri, 25 Jul 2008 18:26:13 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0K4K00I23OFNS900@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.Com); Fri, 25 Jul 2008 10:26:11 -0700 (PDT) Received: from brmea-mail-2.sun.com ([192.18.98.43]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0K4K00EGHOFNBN60@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.Com); Fri, 25 Jul 2008 10:26:11 -0700 (PDT) Received: from fe-amer-10.sun.com ([192.18.109.80]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id m6PHQAwi015035 for ; Fri, 25 Jul 2008 17:26:10 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K4K00201L55YI00@mail-amer.sun.com> (original mail from Chris.Horne@Sun.COM) for PSARC-ext@Sun.Com (ORCPT PSARC-ext@Sun.Com); Fri, 25 Jul 2008 11:26:10 -0600 (MDT) Received: from sun.com ([129.150.33.79]) by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K4K00AK1OFE0T30@mail-amer.sun.com>; Fri, 25 Jul 2008 11:26:04 -0600 (MDT) Date: Fri, 25 Jul 2008 11:26:02 -0600 From: Chris Horne Subject: Re: Improved [s]sd-config-list support [PSARC/2008/465 FastTrack timeout 07/30/2008] In-reply-to: <488A0C0C.40609@sun.com> Sender: Chris.Horne@sun.com To: Chris Horne Cc: Christopher Horne , PSARC-ext@sun.com Message-id: <488A0CAA.5040406@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-Accept-Language: en-us, en X-PMX-Version: 5.4.1.325704 References: <200807211400.m6LE0QZD014996@sac.sfbay.sun.com> <488A0C0C.40609@sun.com> User-Agent: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.4) Gecko/20040414 Status: RO Content-Length: 130 ...I take that back, the timer is still running. Chris Horne wrote: > This case was approved at PSARC this week. > > -Chris > From Chris.Horne@Sun.COM Wed Jul 30 10:03:53 2008 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m6UH3rUX019622 for ; Wed, 30 Jul 2008 10:03:53 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m6UH3rHG005835 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 30 Jul 2008 10:03:53 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0K4T00801WQGHP00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.Com); Wed, 30 Jul 2008 10:03:52 -0700 (PDT) Received: from brmea-mail-2.sun.com ([192.18.98.43]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0K4T003EIWQEFG30@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.Com); Wed, 30 Jul 2008 10:03:50 -0700 (PDT) Received: from fe-amer-09.sun.com ([192.18.109.79]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id m6UH3nKh017395 for ; Wed, 30 Jul 2008 17:03:49 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K4T00J01WKEMU00@mail-amer.sun.com> (original mail from Chris.Horne@Sun.COM) for PSARC-ext@Sun.Com (ORCPT PSARC-ext@Sun.Com); Wed, 30 Jul 2008 11:03:49 -0600 (MDT) Received: from [172.20.25.50] by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K4T00MKCWQ49TC0@mail-amer.sun.com> for PSARC-ext@Sun.Com (ORCPT PSARC-ext@Sun.Com); Wed, 30 Jul 2008 11:03:40 -0600 (MDT) Date: Wed, 30 Jul 2008 11:02:27 -0600 From: Chris Horne Subject: Re: Improved [s]sd-config-list support [PSARC/2008/465 FastTrack timeout 07/30/2008] In-reply-to: <200807211400.m6LE0QZD014996@sac.sfbay.sun.com> Sender: Chris.Horne@Sun.COM To: PSARC-ext@Sun.COM Message-id: <48909EA3.6090207@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200807211400.m6LE0QZD014996@sac.sfbay.sun.com> User-Agent: Thunderbird 1.5.0.8 (X11/20061204) Status: RO Content-Length: 48 This case was approved at PSARC today. -Chris From sarito@cpubringup.sfbay.sun.com Tue May 19 10:45:17 2009 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n4JHjHTX020093 for ; Tue, 19 May 2009 10:45:17 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n4JHj9pm021332; Tue, 19 May 2009 10:45:16 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KJW00703JZGZ300@brm-avmta-1.central.sun.com>; Tue, 19 May 2009 11:45:16 -0600 (MDT) Received: from dm-sfbay-01.sfbay.sun.com ([129.145.155.118]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KJW005RXJZFST20@brm-avmta-1.central.sun.com>; Tue, 19 May 2009 11:45:15 -0600 (MDT) Received: from cpubringup.sfbay.sun.com (cpubringup.SFBay.Sun.COM [10.6.101.18]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n4JHjFMc039464; Tue, 19 May 2009 10:45:15 -0700 (PDT) Received: from cpubringup.sfbay.sun.com (localhost [127.0.0.1]) by cpubringup.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n4JHbj23003158; Tue, 19 May 2009 10:37:45 -0700 (PDT) Received: (from sarito@localhost) by cpubringup.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id n4JHbjgg003157; Tue, 19 May 2009 10:37:45 -0700 (PDT) Date: Tue, 19 May 2009 10:37:45 -0700 (PDT) From: Terry Whatley Subject: 2009/310 Disk IO PM Enhancement 1999/388 2005/250 2008/465 To: PSARC-ext@sun.com, terry.whatley@sun.com, zfs-pm-core@sun.com Message-id: <200905191737.n4JHbjgg003157@cpubringup.sfbay.sun.com> Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 Status: RO Content-Length: 11124 I'm sponsoring this fast-track for Jane Chu. A micro/patch binding is requested. The timer is set for 5/27/09. Template Version: @(#)onepager.txt 1.35 07/11/07 SMI Copyright 2007 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: Disk IO Stack PM Enhancement 1.2. Name of Document Author/Supplier: Author: Jane.Chu@sun.com 1.3. Date of This Document: 04/24/2009 1.4. Name of Major Document Customer(s)/Consumer(s): 1.4.1 The PAC or CPT you expect to review your project Solaris PAC 1.4.2 The ARC(s) you expect to review your project PSARC 1.4.3 The Director/VP who is "Sponsoring" this project michael.sanfratello@sun.com 1.4.4 The name of your business unit Common Software Feature, Solaris 1.5. Email Aliases: 1.5.1. Responsible Managers: Alan.Slivensky@sun.com Grant.Zhang@sun.com Kevin.X.Song@sun.com 1.5.2. Responsible Engineers: Jane.Chu@sun.com Ada.Feng@sun.com Larry.Liu@sun.com Li.He@sun.com 4. Technical Description 4.1 Background The Solaris power management framework provides for a mechanism (pm-components(9P)) for device drivers to export the set of power states that they support, and some interfaces (pm(7d), power.conf(4), pmconfig(1M)) for controlling the transitions of devices into those power states. The power states of devices are controlled by autopm if enabled (see power.conf(4)), or can be controlled by an application (currently only X, which controls frame buffer power states if autopm is enabled). Currently, sd(7D), the common disk driver, supports only On and Off states for disk drives. 4.2 The Problem Modern disks provide intermediate power states between On and Off, which provide for more fine grained control of power consumption and latency. The Solaris disk driver (sd) does not support these intermediate power states. Sd does not monitor and enforce SATA disk start-stop cycle limits, as it does for SCSI 3 disks, and has no mechanism for providing information about SATA disk supported power states or the attributes of those states, such as power consumption, latency to change states, etc. These features are required for more intelligent disk PM. 4.3 Proposal The IO framework and driver stack will be enhanced to provide generic support of modern disks' power management capabilities. The new features include support of SPC-4/SBC-3 compliant disk power conditions. The SATA framework will be enhanced to provide comparable software translation between ATA power conditions and SBC-3 power conditions. Disk health monitoring will be improved in that, if the driver firmware provides the necessary information, a disk will be protected from exceeding its maximum spindle start/stop cycles or head load/unload cycles. An interface will be provided (see 4.3.5.4 USCSI_PMFAILFAST flag support) that will allow the fma disk module to know not to spin up a disk to collect health data. There is no interesting data accumulating while the disk is spun down. There is some risk of deterioration if a disk stays down for too long. A self-test will be arranged on a weekly basis to prevent/predict such deterioration. Details follow. 4.3.1 pm_trans_check(9F) pm_trans_check(9F) will be updated to accept and process SMART start-stop count attribute data. pm_trans_check(9F) will also now (conditionally) expect an additional argument, the dev_info pointer of the device, to indicate the target device for internal bookkeeping. Whether the new or old version of pm_trans_check() is being called by the driver is detected by a field in the input data structure which is required to be 0 in the call to the existing version (and the existing code test for it being so and returns an error if not) and required to be non-zero in the proposed version. An updated man page is in the materials directory. 4.3.2 update to "pm-capable" property (PSARC/2005/250, consolidation private) The "pm-capable" property was originally proposed as a boolean property indicating that a target disk does not support the Log Sense page, but is nonetheless capable of power management. This property will be extended to a 32 bit integer bitmask. The lower 16 bits (bits under PM_CAPABLE_MODE_MASK) contains a value equivalent to one of the published ANSI-Approved versions to which the target's power conditions is compliant, (one of PM_CAPABLE_MODE_CCS, PM_CAPABLE_MODE_SCSI2, PM_CAPABLE_MODE_SPC, PM_CAPABLE_MODE_SPC2, PM_CAPABLE_MODE_SPC3, or PM_CAPABLE_MODE_SPC4). Refer to the SPC-4 specification for the published versions. If the lower 16 bits of "pm-capable" indicates that the target disk supports power management, but PM_CAPABLE_LOG_SUPPORTED (bit 16) is not set, it indicates that the target driver should not try to access Log Sense, but should honor all requests to power off the drive. If PM_CAPABLE_LOG_SUPPORTED (bit 16) is set in the "pm-capable" it indicates that the target may support Log Sense, and that the target driver should access the Log Sense page to get the information needed to determine whether the power off will be allowed. If PM_CAPABLE_SMART_LOG (bit 17) is set, it indicates that the target's Start-Stop Counter log page contains the SMART Start-stop count attribute tranlated into SCSI format. The remaining bits are reserved, and must be set to 0. The materials directory contains an updated usr/src/uts/common/sys/scsi/generic/inquiry.h file with symbolic definitions of the bit masks. 4.3.3 add power-condition tunable (PSARC 2008/465, committed) A new tunable "power-condition" is introduced into [s]sd-config-list. "power-condition" can be configured in sd.conf or added in the internal sd(7D) configuration table per device type (identified by Product ID and Vender ID) which can be set to false to indicate that the disk so identified reports PM_CAPABLE_SPC4 but does not support the power condition field in the START STOP UNIT command. If "power-condition" appears in both sd's internal configuration table and in sd.conf, the value in sd.conf takes precedence. This tunable defaults to true, indicating that the power condition field is supported. The tunable is defined as power-condition Committed BOOLEAN See "PSARC 2008/465 Improved [s]sd-config-list support" for the syntax. 4.3.4 disk power attribute driver properties sd(7D) will export a set of driver properties to indicate a disk's power attributes. See Table-2. Table-1 Disk Power Attribute Properties (array properties are indexed by power state in order of ascending power levels) ------------------------------------------------------------------------ Prop Name Prop Type | Prop Description ------------------------------------------------------------------------ "pm-resource-type" String | "resource-spindle-disk" for the | spindle disks ------------------------------------------------------------------------ "pm-perf" Integer array | array of average R/W | performance percentages ------------------------------------------------------------------------ "pm-pwr-saving" Integer array | array of average power saving in | units of 0.1watt ------------------------------------------------------------------------ "pm-latency" Integer array | array of time to first data in units | of 100ms ------------------------------------------------------------------------ 4.3.5 USCSI_PMFAILFAST flag A new USCSI flag, USCSI_PMFAILFAST, will be added in uscsi.h file To effectively use the USCSI_PMFAILFAST flag, a uscsi(7I) caller must open the device with flag O_NONBLOCK or O_NDELAY to prevent a powered down disk from being spun up as a result of the open(2) call. When handling a uscsi(7I) command, sd(7D) will fail the command and return ECANCELED if the USCSI_PMFAILFAST flag is set and disk is in low power state. 4.4 Interface Tables ------------------------------------------------------------------------ Interface Commitment Comments Name Level ------------------------------------------------------------------------ pm_trans_check(9F) Committed (dev_info_t *)dip is added as the last argument, rely on pm_scsi_cycles.flag to determine new versus old interface call (modifies: PSARC 1999/388) power-condition Committed tunable in [s]sd-config-list overrides disk reported power-condition field of PM_CAPABLE_MODE_SPC4 (modifies: PSARC/2008/465) USCSI_PMFAILFAST Consolidation uscsi(7I) flag indicates not Private to power up a disk to process the associated command, but fail pm-resource-type, Consolidation Describe pm features of device pm-perf, pm-pwr-saving, Private details in Table-1. pm-latency properties "pm-capable" property Consolidation Indicates type of pm support Private (modifies: PSARC 2005/250) ------------------------------------------------------------------------ 4.5 References [1] PSARC/1999/388 Advisory Check for Device Power Cyclespm_trans_check http://sac.sfbay/PSARC/1999/388 [2] PSARC/2005/250 SCSI target "pm-capable" property http://sac.sfbay/PSARC/2005/250 [3] PSARC/2008/465 Improved [s]sd-config-list support http://sac.sfbay/PSARC/2008/465 [4] PSARC/2006/672 USCSI LUN reset flag (general USCSI info) http://sac.sfbay/PSARC/2006/672 [5] T10 Specifications from http://t10.org/ spc4r17 - SCSI Primary Commands - 4(SPC-4), 13 November, 2008 sat2r6 - SCSI to ATA Translation - 2(SAT-2), 22 June 2008 sat-standby-09-113r1 - SCSI to ATA Translation-2(SAT-2), 13 March 2009 sbc3r17 - SCSI Block Commands - 3(SBC-3), 17 November, 2008 4.6 Man page changes See materials directory for the following new/changed man pages. Deliver (Evolving): pm_trans_check.9F.diff 4.7 Bug/RFE Numbers: 6827942 pm_trans_check enhancement 6832227 sd to export disk power attributes 6832232 enhanced sd(7D) to support power condition field in Start-Stop Unit command 6832245 "pm-capable" property updates 6832247 sata(7D) enhancement for SBC-3 power conditions and ATA power conditions translation 6781588 sata(7D) to support additional Mode Sense and Log Sense for PM 4.8 Doc Impact: 4.9 Admin/Config Impact: 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5 ARC review type: FaskTrack 6.6 ARC Exposure: Open From edward.pilatowicz@sun.com Tue May 19 18:23:30 2009 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n4K1NSHa028491 for ; Tue, 19 May 2009 18:23:28 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n4K1NNWV028829; Wed, 20 May 2009 02:23:27 +0100 (BST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KJX0080N570YC00@brm-avmta-1.central.sun.com>; Tue, 19 May 2009 19:23:24 -0600 (MDT) Received: from dm-sfbay-02.sfbay.sun.com ([129.146.11.31]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KJX00J1C56ZRG50@brm-avmta-1.central.sun.com>; Tue, 19 May 2009 19:23:23 -0600 (MDT) Received: from eng.sun.com (vpn-129-150-150-94.Singapore.Sun.COM [129.150.150.94]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n4K1NKAA013155; Tue, 19 May 2009 18:23:21 -0700 (PDT) Date: Wed, 20 May 2009 09:23:19 +0800 From: Edward Pilatowicz Subject: Re: 2009/310 Disk IO PM Enhancement 1999/388 2005/250 2008/465 In-reply-to: <200905191737.n4JHbjgg003157@cpubringup.sfbay.sun.com> To: Terry Whatley Cc: PSARC-ext@sun.com, terry.whatley@sun.com, zfs-pm-core@sun.com Message-id: <20090520012319.GB105166@eng.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <200905191737.n4JHbjgg003157@cpubringup.sfbay.sun.com> User-Agent: Mutt/1.5.19 (2009-01-05) x_sac_archived: PSARC/2009/310 x_sac_archived: PSARC/1999/388 Status: RO Content-Length: 1643 On Tue, May 19, 2009 at 10:37:45AM -0700, Terry Whatley wrote: > 4.3.4 disk power attribute driver properties > > sd(7D) will export a set of driver properties to indicate a disk's > power attributes. See Table-2. > > Table-1 Disk Power Attribute Properties (array properties are indexed > by power state in order of ascending power levels) > ------------------------------------------------------------------------ > Prop Name Prop Type | Prop Description > ------------------------------------------------------------------------ > "pm-resource-type" String | "resource-spindle-disk" for the > | spindle disks > ------------------------------------------------------------------------ > "pm-perf" Integer array | array of average R/W > | performance percentages > ------------------------------------------------------------------------ > "pm-pwr-saving" Integer array | array of average power saving in > | units of 0.1watt > ------------------------------------------------------------------------ > "pm-latency" Integer array | array of time to first data in units > | of 100ms > ------------------------------------------------------------------------ > exporting performance statistics via device properties seems weird to me. is there a precedent for this? why isn't this information being exported via kstats? do we really want to train users to start using prtconf -v to get performance data? ed From Pawel.Wojcik@sun.com Tue May 19 19:11:15 2009 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n4K2BEpk029169 for ; Tue, 19 May 2009 19:11:14 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n4K2BD7J006604; Tue, 19 May 2009 19:11:14 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KJX00M037EQCM00@nwk-avmta-1.sfbay.Sun.COM>; Tue, 19 May 2009 19:11:14 -0700 (PDT) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KJX003UB7EPDBC0@nwk-avmta-1.sfbay.Sun.COM>; Tue, 19 May 2009 19:11:13 -0700 (PDT) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n4K2BDNI019915; Tue, 19 May 2009 19:11:13 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KJX00F007B9IS00@fe-sfbay-10.sun.com>; Tue, 19 May 2009 19:11:13 -0700 (PDT) Received: from [129.153.3.91] ([unknown] [129.153.3.91]) by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KJX00IOE7EPKP70@fe-sfbay-10.sun.com>; Tue, 19 May 2009 19:11:13 -0700 (PDT) Date: Tue, 19 May 2009 19:09:04 -0700 From: Pawel Wojcik Subject: Re: 2009/310 Disk IO PM Enhancement 1999/388 2005/250 2008/465 In-reply-to: <20090520012319.GB105166@eng.sun.com> Sender: Pawel.Wojcik@sun.com To: Edward Pilatowicz Cc: Terry Whatley , PSARC-ext@sun.com, Terry.Whatley@sun.com, zfs-pm-core@sun.com Message-id: <4A136640.3020202@Sun.COM> MIME-version: 1.0 Content-type: multipart/alternative; boundary="Boundary_(ID_TQ1/Vw28VzoV9Nje2/q6jw)" X-PMX-Version: 5.4.1.325704 References: <200905191737.n4JHbjgg003157@cpubringup.sfbay.sun.com> <20090520012319.GB105166@eng.sun.com> User-Agent: Thunderbird 2.0.0.14 (X11/20080602) x_sac_archived: PSARC/2009/310 x_sac_archived: PSARC/1999/388 Status: RO Content-Length: 4961 This is a multi-part message in MIME format. --Boundary_(ID_TQ1/Vw28VzoV9Nje2/q6jw) Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT On 05/19/09 06:23 PM, Edward Pilatowicz wrote: > On Tue, May 19, 2009 at 10:37:45AM -0700, Terry Whatley wrote: > >> 4.3.4 disk power attribute driver properties >> >> sd(7D) will export a set of driver properties to indicate a disk's >> power attributes. See Table-2. >> >> Table-1 Disk Power Attribute Properties (array properties are indexed >> by power state in order of ascending power levels) >> ------------------------------------------------------------------------ >> Prop Name Prop Type | Prop Description >> ------------------------------------------------------------------------ >> "pm-resource-type" String | "resource-spindle-disk" for the >> | spindle disks >> ------------------------------------------------------------------------ >> "pm-perf" Integer array | array of average R/W >> | performance percentages >> ------------------------------------------------------------------------ >> "pm-pwr-saving" Integer array | array of average power saving in >> | units of 0.1watt >> ------------------------------------------------------------------------ >> "pm-latency" Integer array | array of time to first data in units >> | of 100ms >> ------------------------------------------------------------------------ >> >> > > exporting performance statistics via device properties seems weird to > me. is there a precedent for this? why isn't this information being > exported via kstats? do we really want to train users to start using > prtconf -v to get performance data? > > ed > I think these are not performance statistics. I believe that these are static arrays that are specific for a device type (most likely Sun disks only), that correlate specific power level with performance and power savings. These properties, I believe, are to be used by a storage power manager to decide at what power level disk should run at given time. Jane Chu may correct me here... -Pawel --Boundary_(ID_TQ1/Vw28VzoV9Nje2/q6jw) Content-type: text/html; CHARSET=US-ASCII Content-transfer-encoding: 7BIT On 05/19/09 06:23 PM, Edward Pilatowicz wrote: 
On Tue, May 19, 2009 at 10:37:45AM -0700, Terry Whatley wrote:
  
     4.3.4 disk power attribute driver properties

	sd(7D) will export a set of driver properties to indicate a disk's
	power attributes. See Table-2.

	Table-1 Disk Power Attribute Properties (array properties are indexed
		by power state in order of ascending power levels)
	------------------------------------------------------------------------
	Prop Name            Prop Type    | Prop Description
	------------------------------------------------------------------------
	"pm-resource-type"      String    | "resource-spindle-disk" for the
                                          | spindle disks
	------------------------------------------------------------------------
	"pm-perf" Integer array           | array of average R/W
                                          | performance percentages
	------------------------------------------------------------------------
	"pm-pwr-saving"  Integer array    | array of average power saving in
                                          | units of 0.1watt
	------------------------------------------------------------------------
	"pm-latency"     Integer array    | array of time to first data in units
                                          | of 100ms
	------------------------------------------------------------------------

    

exporting performance statistics via device properties seems weird to
me.  is there a precedent for this?  why isn't this information being
exported via kstats?  do we really want to train users to start using
prtconf -v to get performance data?

ed
I think these are not performance statistics. I believe that these are static arrays that are specific for a device type (most likely Sun disks only), that correlate specific power level with performance and power savings. These properties, I believe, are to be used by a storage power manager to decide at what power level disk should run at given time.
Jane Chu may correct me here...
-Pawel

--Boundary_(ID_TQ1/Vw28VzoV9Nje2/q6jw)-- From edward.pilatowicz@sun.com Tue May 19 19:29:27 2009 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n4K2TRtD029300 for ; Tue, 19 May 2009 19:29:27 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n4K2TPPY033001; Tue, 19 May 2009 20:29:27 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KJX00L05892GA00@nwk-avmta-2.sfbay.sun.com>; Tue, 19 May 2009 19:29:26 -0700 (PDT) Received: from dm-sfbay-02.sfbay.sun.com ([129.146.11.31]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KJX00B76892PA60@nwk-avmta-2.sfbay.sun.com>; Tue, 19 May 2009 19:29:26 -0700 (PDT) Received: from eng.sun.com (vpn-129-150-150-94.Singapore.Sun.COM [129.150.150.94]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n4K2TMlw040444; Tue, 19 May 2009 19:29:23 -0700 (PDT) Date: Wed, 20 May 2009 10:29:21 +0800 From: Edward Pilatowicz Subject: Re: 2009/310 Disk IO PM Enhancement 1999/388 2005/250 2008/465 In-reply-to: <4A136640.3020202@Sun.COM> To: Pawel Wojcik Cc: Terry Whatley , PSARC-ext@sun.com, Terry.Whatley@sun.com, zfs-pm-core@sun.com Message-id: <20090520022921.GF105166@eng.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <200905191737.n4JHbjgg003157@cpubringup.sfbay.sun.com> <20090520012319.GB105166@eng.sun.com> <4A136640.3020202@Sun.COM> User-Agent: Mutt/1.5.19 (2009-01-05) Status: RO Content-Length: 2428 On Tue, May 19, 2009 at 07:09:04PM -0700, Pawel Wojcik wrote: > On 05/19/09 06:23 PM, Edward Pilatowicz wrote: >> On Tue, May 19, 2009 at 10:37:45AM -0700, Terry Whatley wrote: >> >>> 4.3.4 disk power attribute driver properties >>> >>> sd(7D) will export a set of driver properties to indicate a disk's >>> power attributes. See Table-2. >>> >>> Table-1 Disk Power Attribute Properties (array properties are indexed >>> by power state in order of ascending power levels) >>> ------------------------------------------------------------------------ >>> Prop Name Prop Type | Prop Description >>> ------------------------------------------------------------------------ >>> "pm-resource-type" String | "resource-spindle-disk" for the >>> | spindle disks >>> ------------------------------------------------------------------------ >>> "pm-perf" Integer array | array of average R/W >>> | performance percentages >>> ------------------------------------------------------------------------ >>> "pm-pwr-saving" Integer array | array of average power saving in >>> | units of 0.1watt >>> ------------------------------------------------------------------------ >>> "pm-latency" Integer array | array of time to first data in units >>> | of 100ms >>> ------------------------------------------------------------------------ >>> >>> >> >> exporting performance statistics via device properties seems weird to >> me. is there a precedent for this? why isn't this information being >> exported via kstats? do we really want to train users to start using >> prtconf -v to get performance data? >> >> ed >> > I think these are not performance statistics. I believe that these are > static arrays that are specific for a device type (most likely Sun disks > only), that correlate specific power level with performance and power > savings. These properties, I believe, are to be used by a storage power > manager to decide at what power level disk should run at given time. > Jane Chu may correct me here... > -Pawel > if that's the case then having them as device properties seems ok, but documentation for these properties should make it clear that these are not actual system performance numbers. ed From carlsonj@phorcys.east.sun.com Thu May 21 05:07:13 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n4LC7Csu015776 for ; Thu, 21 May 2009 05:07:13 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n4LC783L021395; Thu, 21 May 2009 20:07:11 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KJZ00803TNX7600@nwk-avmta-2.sfbay.sun.com>; Thu, 21 May 2009 05:07:09 -0700 (PDT) Received: from dm-east-02.east.sun.com ([129.148.13.5]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KJZ00J3RTNWH1C0@nwk-avmta-2.sfbay.sun.com>; Thu, 21 May 2009 05:07:09 -0700 (PDT) Received: from phorcys.east.sun.com (phorcys.East.Sun.COM [129.148.174.143]) by dm-east-02.east.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n4LC77uE008463; Thu, 21 May 2009 08:07:07 -0400 (EDT) Received: from phorcys.east.sun.com (phorcys.local [127.0.0.1]) by phorcys.east.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n4LC68DT005875; Thu, 21 May 2009 08:06:08 -0400 (EDT) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.3+Sun/8.14.3/Submit) id n4LC68w5005872; Thu, 21 May 2009 08:06:08 -0400 (EDT) Date: Thu, 21 May 2009 08:06:08 -0400 From: James Carlson Subject: Re: 2009/310 Disk IO PM Enhancement 1999/388 2005/250 2008/465 In-reply-to: <200905191737.n4JHbjgg003157@cpubringup.sfbay.sun.com> To: Terry Whatley Cc: PSARC-ext@sun.com, Terry.Whatley@sun.com, zfs-pm-core@sun.com Message-id: <18965.17328.270448.274344@gargle.gargle.HOWL> MIME-version: 1.0 X-Mailer: VM 7.01 under Emacs 21.3.1 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200905191737.n4JHbjgg003157@cpubringup.sfbay.sun.com> x_sac_archived: PSARC/2009/310 x_sac_archived: PSARC/1999/388 Status: RO Content-Length: 1367 Terry Whatley writes: > The IO framework and driver stack will be enhanced to provide generic > support of modern disks' power management capabilities. The new > features include support of SPC-4/SBC-3 compliant disk power conditions. > The SATA framework will be enhanced to provide comparable software > translation between ATA power conditions and SBC-3 power conditions. Where's the rest of this project? I would expect that, in order to finish this project, there's some work necessary to make the power management subsystem consume this data. That doesn't appear to be a part of this project, though. Are the folks responsible for supporting autopm and the rest of the power management subsystem involved with this project? If not, are they aware of what this project is planning to deliver, and do they agree that the information will be usable to make power management decisions? If so, is there a future project ("autopm disk enhancements") that is dependent on this one? Help us to connect the dots from the set of low-level driver properties described here to the higher-level goal of providing better disk management. -- James Carlson, Solaris Networking Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 From djb@sun.com Thu May 21 16:06:59 2009 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n4LN6wdU024103 for ; Thu, 21 May 2009 16:06:58 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n4LN6v51027744; Thu, 21 May 2009 16:06:57 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KK000E01O7LMT00@brm-avmta-1.central.sun.com>; Thu, 21 May 2009 17:06:57 -0600 (MDT) Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.63]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KK00046FO7KGQ80@brm-avmta-1.central.sun.com>; Thu, 21 May 2009 17:06:56 -0600 (MDT) Received: from [129.146.226.67] (cantab.SFBay.Sun.COM [129.146.226.67]) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n4LN6ubc948429; Thu, 21 May 2009 16:06:56 -0700 (PDT) Date: Thu, 21 May 2009 16:07:09 -0700 From: "David J. Brown" Subject: Re: 2009/310 Disk IO PM Enhancement 1999/388 2005/250 2008/465 In-reply-to: <18965.17328.270448.274344@gargle.gargle.HOWL> To: "Terry (Sarito) Whatley" Cc: James Carlson , PSARC-ext@sun.com, zfs-pm-core@sun.com Reply-to: david.brown@sun.com Message-id: <4A15DE9D.2020303@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200905191737.n4JHbjgg003157@cpubringup.sfbay.sun.com> <18965.17328.270448.274344@gargle.gargle.HOWL> User-Agent: Thunderbird 2.0.0.18 (X11/20081201) x_sac_archived: PSARC/2009/310 Status: RO Content-Length: 1982 James Carlson wrote: > Terry Whatley writes: > >> The IO framework and driver stack will be enhanced to provide generic >> support of modern disks' power management capabilities. The new >> features include support of SPC-4/SBC-3 compliant disk power conditions. >> The SATA framework will be enhanced to provide comparable software >> translation between ATA power conditions and SBC-3 power conditions. >> > > Where's the rest of this project? I would expect that, in order to > finish this project, there's some work necessary to make the power > management subsystem consume this data. That doesn't appear to be a > part of this project, though. > > Are the folks responsible for supporting autopm and the rest of the > power management subsystem involved with this project? > > If not, are they aware of what this project is planning to deliver, > and do they agree that the information will be usable to make power > management decisions? > > If so, is there a future project ("autopm disk enhancements") that is > dependent on this one? > > Help us to connect the dots from the set of low-level driver > properties described here to the higher-level goal of providing better > disk management. > > Terry, Isn't ZFS the primary consumer of these new PM-related disk driver properties [interfaces] at present? Is there a design document that describes these new conceptions for storage-related power management? Seems like that would help provide the context whereby folks could understand this a bit better. It would also be very useful to have that around in the case's materials directory for anyone who came there later, wishing to learn about this. -db -- ; David J. Brown Ph.D. (cantab.) ; Solaris Engineering ; Sun Microsystems Inc. ; -- ; Postal Address: Telephone: (650) 786-5558 ; 4150 Network Circle, UMPK17-307 FAX: (650) 786-5734 ; Santa Clara, CA 95054 e-mail: djb@sun.com From Terry.Whatley@sun.com Wed Jun 24 10:56:25 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n5OHuO9O011751 for ; Wed, 24 Jun 2009 10:56:24 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n5OHuK0b014708; Thu, 25 Jun 2009 01:56:23 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KLR00F0D8HWAU00@nwk-avmta-2.sfbay.sun.com>; Wed, 24 Jun 2009 10:56:20 -0700 (PDT) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KLR00A4Z8HVLO70@nwk-avmta-2.sfbay.sun.com>; Wed, 24 Jun 2009 10:56:19 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n5OHuJJB028084; Wed, 24 Jun 2009 10:56:19 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KLR008007PM1W00@fe-sfbay-09.sun.com>; Wed, 24 Jun 2009 10:56:19 -0700 (PDT) Received: from [10.0.2.15] ([unknown] [192.18.41.196]) by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KLR002FZ8HH4V90@fe-sfbay-09.sun.com>; Wed, 24 Jun 2009 10:56:06 -0700 (PDT) Date: Wed, 24 Jun 2009 10:56:04 -0700 From: "Terry (Sarito) Whatley" Subject: 2009/310 Disk IO PM Enhancement approved, impact on 1999/388, 2008/465, 2005/250 In-reply-to: <4A16F230.4010600@sun.com> Sender: Terry.Whatley@sun.com To: PSARC-ext@sun.com, zfs-pm-core@sun.com Message-id: <4A4268B4.60003@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200905191737.n4JHbjgg003157@cpubringup.sfbay.sun.com> <18965.17328.270448.274344@gargle.gargle.HOWL> <4A163C63.1040609@sun.com> <18966.44077.385692.100707@gargle.gargle.HOWL> <4A16EED4.5020906@sun.com> <4A16F230.4010600@sun.com> User-Agent: Thunderbird 2.0.0.22 (X11/20090612) Status: RO Content-Length: 300 This case (2009/310) was approved at the PSARC meeting today. For impact this case has on the other cases listed, search for the case numbers in /shared/sac/PSARC/2009/310/materials/proposal . [the purpose of this mail is to get the above sentence into the mail files of the other cases] -sarito