From sarito@cpubringup.sfbay.sun.com Tue May 19 10:45:17 2009 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n4JHjHTX020093 for ; Tue, 19 May 2009 10:45:17 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n4JHj9pm021332; Tue, 19 May 2009 10:45:16 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KJW00703JZGZ300@brm-avmta-1.central.sun.com>; Tue, 19 May 2009 11:45:16 -0600 (MDT) Received: from dm-sfbay-01.sfbay.sun.com ([129.145.155.118]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KJW005RXJZFST20@brm-avmta-1.central.sun.com>; Tue, 19 May 2009 11:45:15 -0600 (MDT) Received: from cpubringup.sfbay.sun.com (cpubringup.SFBay.Sun.COM [10.6.101.18]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n4JHjFMc039464; Tue, 19 May 2009 10:45:15 -0700 (PDT) Received: from cpubringup.sfbay.sun.com (localhost [127.0.0.1]) by cpubringup.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n4JHbj23003158; Tue, 19 May 2009 10:37:45 -0700 (PDT) Received: (from sarito@localhost) by cpubringup.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id n4JHbjgg003157; Tue, 19 May 2009 10:37:45 -0700 (PDT) Date: Tue, 19 May 2009 10:37:45 -0700 (PDT) From: Terry Whatley Subject: 2009/310 Disk IO PM Enhancement 1999/388 2005/250 2008/465 To: PSARC-ext@sun.com, terry.whatley@sun.com, zfs-pm-core@sun.com Message-id: <200905191737.n4JHbjgg003157@cpubringup.sfbay.sun.com> Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 Status: RO Content-Length: 11124 I'm sponsoring this fast-track for Jane Chu. A micro/patch binding is requested. The timer is set for 5/27/09. Template Version: @(#)onepager.txt 1.35 07/11/07 SMI Copyright 2007 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: Disk IO Stack PM Enhancement 1.2. Name of Document Author/Supplier: Author: Jane.Chu@sun.com 1.3. Date of This Document: 04/24/2009 1.4. Name of Major Document Customer(s)/Consumer(s): 1.4.1 The PAC or CPT you expect to review your project Solaris PAC 1.4.2 The ARC(s) you expect to review your project PSARC 1.4.3 The Director/VP who is "Sponsoring" this project michael.sanfratello@sun.com 1.4.4 The name of your business unit Common Software Feature, Solaris 1.5. Email Aliases: 1.5.1. Responsible Managers: Alan.Slivensky@sun.com Grant.Zhang@sun.com Kevin.X.Song@sun.com 1.5.2. Responsible Engineers: Jane.Chu@sun.com Ada.Feng@sun.com Larry.Liu@sun.com Li.He@sun.com 4. Technical Description 4.1 Background The Solaris power management framework provides for a mechanism (pm-components(9P)) for device drivers to export the set of power states that they support, and some interfaces (pm(7d), power.conf(4), pmconfig(1M)) for controlling the transitions of devices into those power states. The power states of devices are controlled by autopm if enabled (see power.conf(4)), or can be controlled by an application (currently only X, which controls frame buffer power states if autopm is enabled). Currently, sd(7D), the common disk driver, supports only On and Off states for disk drives. 4.2 The Problem Modern disks provide intermediate power states between On and Off, which provide for more fine grained control of power consumption and latency. The Solaris disk driver (sd) does not support these intermediate power states. Sd does not monitor and enforce SATA disk start-stop cycle limits, as it does for SCSI 3 disks, and has no mechanism for providing information about SATA disk supported power states or the attributes of those states, such as power consumption, latency to change states, etc. These features are required for more intelligent disk PM. 4.3 Proposal The IO framework and driver stack will be enhanced to provide generic support of modern disks' power management capabilities. The new features include support of SPC-4/SBC-3 compliant disk power conditions. The SATA framework will be enhanced to provide comparable software translation between ATA power conditions and SBC-3 power conditions. Disk health monitoring will be improved in that, if the driver firmware provides the necessary information, a disk will be protected from exceeding its maximum spindle start/stop cycles or head load/unload cycles. An interface will be provided (see 4.3.5.4 USCSI_PMFAILFAST flag support) that will allow the fma disk module to know not to spin up a disk to collect health data. There is no interesting data accumulating while the disk is spun down. There is some risk of deterioration if a disk stays down for too long. A self-test will be arranged on a weekly basis to prevent/predict such deterioration. Details follow. 4.3.1 pm_trans_check(9F) pm_trans_check(9F) will be updated to accept and process SMART start-stop count attribute data. pm_trans_check(9F) will also now (conditionally) expect an additional argument, the dev_info pointer of the device, to indicate the target device for internal bookkeeping. Whether the new or old version of pm_trans_check() is being called by the driver is detected by a field in the input data structure which is required to be 0 in the call to the existing version (and the existing code test for it being so and returns an error if not) and required to be non-zero in the proposed version. An updated man page is in the materials directory. 4.3.2 update to "pm-capable" property (PSARC/2005/250, consolidation private) The "pm-capable" property was originally proposed as a boolean property indicating that a target disk does not support the Log Sense page, but is nonetheless capable of power management. This property will be extended to a 32 bit integer bitmask. The lower 16 bits (bits under PM_CAPABLE_MODE_MASK) contains a value equivalent to one of the published ANSI-Approved versions to which the target's power conditions is compliant, (one of PM_CAPABLE_MODE_CCS, PM_CAPABLE_MODE_SCSI2, PM_CAPABLE_MODE_SPC, PM_CAPABLE_MODE_SPC2, PM_CAPABLE_MODE_SPC3, or PM_CAPABLE_MODE_SPC4). Refer to the SPC-4 specification for the published versions. If the lower 16 bits of "pm-capable" indicates that the target disk supports power management, but PM_CAPABLE_LOG_SUPPORTED (bit 16) is not set, it indicates that the target driver should not try to access Log Sense, but should honor all requests to power off the drive. If PM_CAPABLE_LOG_SUPPORTED (bit 16) is set in the "pm-capable" it indicates that the target may support Log Sense, and that the target driver should access the Log Sense page to get the information needed to determine whether the power off will be allowed. If PM_CAPABLE_SMART_LOG (bit 17) is set, it indicates that the target's Start-Stop Counter log page contains the SMART Start-stop count attribute tranlated into SCSI format. The remaining bits are reserved, and must be set to 0. The materials directory contains an updated usr/src/uts/common/sys/scsi/generic/inquiry.h file with symbolic definitions of the bit masks. 4.3.3 add power-condition tunable (PSARC 2008/465, committed) A new tunable "power-condition" is introduced into [s]sd-config-list. "power-condition" can be configured in sd.conf or added in the internal sd(7D) configuration table per device type (identified by Product ID and Vender ID) which can be set to false to indicate that the disk so identified reports PM_CAPABLE_SPC4 but does not support the power condition field in the START STOP UNIT command. If "power-condition" appears in both sd's internal configuration table and in sd.conf, the value in sd.conf takes precedence. This tunable defaults to true, indicating that the power condition field is supported. The tunable is defined as power-condition Committed BOOLEAN See "PSARC 2008/465 Improved [s]sd-config-list support" for the syntax. 4.3.4 disk power attribute driver properties sd(7D) will export a set of driver properties to indicate a disk's power attributes. See Table-2. Table-1 Disk Power Attribute Properties (array properties are indexed by power state in order of ascending power levels) ------------------------------------------------------------------------ Prop Name Prop Type | Prop Description ------------------------------------------------------------------------ "pm-resource-type" String | "resource-spindle-disk" for the | spindle disks ------------------------------------------------------------------------ "pm-perf" Integer array | array of average R/W | performance percentages ------------------------------------------------------------------------ "pm-pwr-saving" Integer array | array of average power saving in | units of 0.1watt ------------------------------------------------------------------------ "pm-latency" Integer array | array of time to first data in units | of 100ms ------------------------------------------------------------------------ 4.3.5 USCSI_PMFAILFAST flag A new USCSI flag, USCSI_PMFAILFAST, will be added in uscsi.h file To effectively use the USCSI_PMFAILFAST flag, a uscsi(7I) caller must open the device with flag O_NONBLOCK or O_NDELAY to prevent a powered down disk from being spun up as a result of the open(2) call. When handling a uscsi(7I) command, sd(7D) will fail the command and return ECANCELED if the USCSI_PMFAILFAST flag is set and disk is in low power state. 4.4 Interface Tables ------------------------------------------------------------------------ Interface Commitment Comments Name Level ------------------------------------------------------------------------ pm_trans_check(9F) Committed (dev_info_t *)dip is added as the last argument, rely on pm_scsi_cycles.flag to determine new versus old interface call (modifies: PSARC 1999/388) power-condition Committed tunable in [s]sd-config-list overrides disk reported power-condition field of PM_CAPABLE_MODE_SPC4 (modifies: PSARC/2008/465) USCSI_PMFAILFAST Consolidation uscsi(7I) flag indicates not Private to power up a disk to process the associated command, but fail pm-resource-type, Consolidation Describe pm features of device pm-perf, pm-pwr-saving, Private details in Table-1. pm-latency properties "pm-capable" property Consolidation Indicates type of pm support Private (modifies: PSARC 2005/250) ------------------------------------------------------------------------ 4.5 References [1] PSARC/1999/388 Advisory Check for Device Power Cyclespm_trans_check http://sac.sfbay/PSARC/1999/388 [2] PSARC/2005/250 SCSI target "pm-capable" property http://sac.sfbay/PSARC/2005/250 [3] PSARC/2008/465 Improved [s]sd-config-list support http://sac.sfbay/PSARC/2008/465 [4] PSARC/2006/672 USCSI LUN reset flag (general USCSI info) http://sac.sfbay/PSARC/2006/672 [5] T10 Specifications from http://t10.org/ spc4r17 - SCSI Primary Commands - 4(SPC-4), 13 November, 2008 sat2r6 - SCSI to ATA Translation - 2(SAT-2), 22 June 2008 sat-standby-09-113r1 - SCSI to ATA Translation-2(SAT-2), 13 March 2009 sbc3r17 - SCSI Block Commands - 3(SBC-3), 17 November, 2008 4.6 Man page changes See materials directory for the following new/changed man pages. Deliver (Evolving): pm_trans_check.9F.diff 4.7 Bug/RFE Numbers: 6827942 pm_trans_check enhancement 6832227 sd to export disk power attributes 6832232 enhanced sd(7D) to support power condition field in Start-Stop Unit command 6832245 "pm-capable" property updates 6832247 sata(7D) enhancement for SBC-3 power conditions and ATA power conditions translation 6781588 sata(7D) to support additional Mode Sense and Log Sense for PM 4.8 Doc Impact: 4.9 Admin/Config Impact: 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5 ARC review type: FaskTrack 6.6 ARC Exposure: Open From edward.pilatowicz@sun.com Tue May 19 18:23:30 2009 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n4K1NSHa028491 for ; Tue, 19 May 2009 18:23:28 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n4K1NNWV028829; Wed, 20 May 2009 02:23:27 +0100 (BST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KJX0080N570YC00@brm-avmta-1.central.sun.com>; Tue, 19 May 2009 19:23:24 -0600 (MDT) Received: from dm-sfbay-02.sfbay.sun.com ([129.146.11.31]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KJX00J1C56ZRG50@brm-avmta-1.central.sun.com>; Tue, 19 May 2009 19:23:23 -0600 (MDT) Received: from eng.sun.com (vpn-129-150-150-94.Singapore.Sun.COM [129.150.150.94]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n4K1NKAA013155; Tue, 19 May 2009 18:23:21 -0700 (PDT) Date: Wed, 20 May 2009 09:23:19 +0800 From: Edward Pilatowicz Subject: Re: 2009/310 Disk IO PM Enhancement 1999/388 2005/250 2008/465 In-reply-to: <200905191737.n4JHbjgg003157@cpubringup.sfbay.sun.com> To: Terry Whatley Cc: PSARC-ext@sun.com, terry.whatley@sun.com, zfs-pm-core@sun.com Message-id: <20090520012319.GB105166@eng.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <200905191737.n4JHbjgg003157@cpubringup.sfbay.sun.com> User-Agent: Mutt/1.5.19 (2009-01-05) Status: RO Content-Length: 1643 On Tue, May 19, 2009 at 10:37:45AM -0700, Terry Whatley wrote: > 4.3.4 disk power attribute driver properties > > sd(7D) will export a set of driver properties to indicate a disk's > power attributes. See Table-2. > > Table-1 Disk Power Attribute Properties (array properties are indexed > by power state in order of ascending power levels) > ------------------------------------------------------------------------ > Prop Name Prop Type | Prop Description > ------------------------------------------------------------------------ > "pm-resource-type" String | "resource-spindle-disk" for the > | spindle disks > ------------------------------------------------------------------------ > "pm-perf" Integer array | array of average R/W > | performance percentages > ------------------------------------------------------------------------ > "pm-pwr-saving" Integer array | array of average power saving in > | units of 0.1watt > ------------------------------------------------------------------------ > "pm-latency" Integer array | array of time to first data in units > | of 100ms > ------------------------------------------------------------------------ > exporting performance statistics via device properties seems weird to me. is there a precedent for this? why isn't this information being exported via kstats? do we really want to train users to start using prtconf -v to get performance data? ed From Pawel.Wojcik@sun.com Tue May 19 19:11:15 2009 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n4K2BEpk029169 for ; Tue, 19 May 2009 19:11:14 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n4K2BD7J006604; Tue, 19 May 2009 19:11:14 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KJX00M037EQCM00@nwk-avmta-1.sfbay.Sun.COM>; Tue, 19 May 2009 19:11:14 -0700 (PDT) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KJX003UB7EPDBC0@nwk-avmta-1.sfbay.Sun.COM>; Tue, 19 May 2009 19:11:13 -0700 (PDT) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n4K2BDNI019915; Tue, 19 May 2009 19:11:13 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KJX00F007B9IS00@fe-sfbay-10.sun.com>; Tue, 19 May 2009 19:11:13 -0700 (PDT) Received: from [129.153.3.91] ([unknown] [129.153.3.91]) by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KJX00IOE7EPKP70@fe-sfbay-10.sun.com>; Tue, 19 May 2009 19:11:13 -0700 (PDT) Date: Tue, 19 May 2009 19:09:04 -0700 From: Pawel Wojcik Subject: Re: 2009/310 Disk IO PM Enhancement 1999/388 2005/250 2008/465 In-reply-to: <20090520012319.GB105166@eng.sun.com> Sender: Pawel.Wojcik@sun.com To: Edward Pilatowicz Cc: Terry Whatley , PSARC-ext@sun.com, Terry.Whatley@sun.com, zfs-pm-core@sun.com Message-id: <4A136640.3020202@Sun.COM> MIME-version: 1.0 Content-type: multipart/alternative; boundary="Boundary_(ID_TQ1/Vw28VzoV9Nje2/q6jw)" X-PMX-Version: 5.4.1.325704 References: <200905191737.n4JHbjgg003157@cpubringup.sfbay.sun.com> <20090520012319.GB105166@eng.sun.com> User-Agent: Thunderbird 2.0.0.14 (X11/20080602) Status: RO Content-Length: 4961 This is a multi-part message in MIME format. --Boundary_(ID_TQ1/Vw28VzoV9Nje2/q6jw) Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT On 05/19/09 06:23 PM, Edward Pilatowicz wrote: > On Tue, May 19, 2009 at 10:37:45AM -0700, Terry Whatley wrote: > >> 4.3.4 disk power attribute driver properties >> >> sd(7D) will export a set of driver properties to indicate a disk's >> power attributes. See Table-2. >> >> Table-1 Disk Power Attribute Properties (array properties are indexed >> by power state in order of ascending power levels) >> ------------------------------------------------------------------------ >> Prop Name Prop Type | Prop Description >> ------------------------------------------------------------------------ >> "pm-resource-type" String | "resource-spindle-disk" for the >> | spindle disks >> ------------------------------------------------------------------------ >> "pm-perf" Integer array | array of average R/W >> | performance percentages >> ------------------------------------------------------------------------ >> "pm-pwr-saving" Integer array | array of average power saving in >> | units of 0.1watt >> ------------------------------------------------------------------------ >> "pm-latency" Integer array | array of time to first data in units >> | of 100ms >> ------------------------------------------------------------------------ >> >> > > exporting performance statistics via device properties seems weird to > me. is there a precedent for this? why isn't this information being > exported via kstats? do we really want to train users to start using > prtconf -v to get performance data? > > ed > I think these are not performance statistics. I believe that these are static arrays that are specific for a device type (most likely Sun disks only), that correlate specific power level with performance and power savings. These properties, I believe, are to be used by a storage power manager to decide at what power level disk should run at given time. Jane Chu may correct me here... -Pawel --Boundary_(ID_TQ1/Vw28VzoV9Nje2/q6jw) Content-type: text/html; CHARSET=US-ASCII Content-transfer-encoding: 7BIT On 05/19/09 06:23 PM, Edward Pilatowicz wrote: 
On Tue, May 19, 2009 at 10:37:45AM -0700, Terry Whatley wrote:
  
     4.3.4 disk power attribute driver properties

	sd(7D) will export a set of driver properties to indicate a disk's
	power attributes. See Table-2.

	Table-1 Disk Power Attribute Properties (array properties are indexed
		by power state in order of ascending power levels)
	------------------------------------------------------------------------
	Prop Name            Prop Type    | Prop Description
	------------------------------------------------------------------------
	"pm-resource-type"      String    | "resource-spindle-disk" for the
                                          | spindle disks
	------------------------------------------------------------------------
	"pm-perf" Integer array           | array of average R/W
                                          | performance percentages
	------------------------------------------------------------------------
	"pm-pwr-saving"  Integer array    | array of average power saving in
                                          | units of 0.1watt
	------------------------------------------------------------------------
	"pm-latency"     Integer array    | array of time to first data in units
                                          | of 100ms
	------------------------------------------------------------------------

    

exporting performance statistics via device properties seems weird to
me.  is there a precedent for this?  why isn't this information being
exported via kstats?  do we really want to train users to start using
prtconf -v to get performance data?

ed
I think these are not performance statistics. I believe that these are static arrays that are specific for a device type (most likely Sun disks only), that correlate specific power level with performance and power savings. These properties, I believe, are to be used by a storage power manager to decide at what power level disk should run at given time.
Jane Chu may correct me here...
-Pawel

--Boundary_(ID_TQ1/Vw28VzoV9Nje2/q6jw)-- From edward.pilatowicz@sun.com Tue May 19 19:29:27 2009 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n4K2TRtD029300 for ; Tue, 19 May 2009 19:29:27 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n4K2TPPY033001; Tue, 19 May 2009 20:29:27 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KJX00L05892GA00@nwk-avmta-2.sfbay.sun.com>; Tue, 19 May 2009 19:29:26 -0700 (PDT) Received: from dm-sfbay-02.sfbay.sun.com ([129.146.11.31]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KJX00B76892PA60@nwk-avmta-2.sfbay.sun.com>; Tue, 19 May 2009 19:29:26 -0700 (PDT) Received: from eng.sun.com (vpn-129-150-150-94.Singapore.Sun.COM [129.150.150.94]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n4K2TMlw040444; Tue, 19 May 2009 19:29:23 -0700 (PDT) Date: Wed, 20 May 2009 10:29:21 +0800 From: Edward Pilatowicz Subject: Re: 2009/310 Disk IO PM Enhancement 1999/388 2005/250 2008/465 In-reply-to: <4A136640.3020202@Sun.COM> To: Pawel Wojcik Cc: Terry Whatley , PSARC-ext@sun.com, Terry.Whatley@sun.com, zfs-pm-core@sun.com Message-id: <20090520022921.GF105166@eng.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <200905191737.n4JHbjgg003157@cpubringup.sfbay.sun.com> <20090520012319.GB105166@eng.sun.com> <4A136640.3020202@Sun.COM> User-Agent: Mutt/1.5.19 (2009-01-05) Status: RO Content-Length: 2428 On Tue, May 19, 2009 at 07:09:04PM -0700, Pawel Wojcik wrote: > On 05/19/09 06:23 PM, Edward Pilatowicz wrote: >> On Tue, May 19, 2009 at 10:37:45AM -0700, Terry Whatley wrote: >> >>> 4.3.4 disk power attribute driver properties >>> >>> sd(7D) will export a set of driver properties to indicate a disk's >>> power attributes. See Table-2. >>> >>> Table-1 Disk Power Attribute Properties (array properties are indexed >>> by power state in order of ascending power levels) >>> ------------------------------------------------------------------------ >>> Prop Name Prop Type | Prop Description >>> ------------------------------------------------------------------------ >>> "pm-resource-type" String | "resource-spindle-disk" for the >>> | spindle disks >>> ------------------------------------------------------------------------ >>> "pm-perf" Integer array | array of average R/W >>> | performance percentages >>> ------------------------------------------------------------------------ >>> "pm-pwr-saving" Integer array | array of average power saving in >>> | units of 0.1watt >>> ------------------------------------------------------------------------ >>> "pm-latency" Integer array | array of time to first data in units >>> | of 100ms >>> ------------------------------------------------------------------------ >>> >>> >> >> exporting performance statistics via device properties seems weird to >> me. is there a precedent for this? why isn't this information being >> exported via kstats? do we really want to train users to start using >> prtconf -v to get performance data? >> >> ed >> > I think these are not performance statistics. I believe that these are > static arrays that are specific for a device type (most likely Sun disks > only), that correlate specific power level with performance and power > savings. These properties, I believe, are to be used by a storage power > manager to decide at what power level disk should run at given time. > Jane Chu may correct me here... > -Pawel > if that's the case then having them as device properties seems ok, but documentation for these properties should make it clear that these are not actual system performance numbers. ed From Terry.Whatley@sun.com Tue May 19 23:14:55 2009 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n4K6EtDb029568 for ; Tue, 19 May 2009 23:14:55 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n4K6EqLM059770; Wed, 20 May 2009 00:14:55 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KJX00305IOU9V00@nwk-avmta-1.sfbay.Sun.COM>; Tue, 19 May 2009 23:14:54 -0700 (PDT) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KJX00GH0IOT3J30@nwk-avmta-1.sfbay.Sun.COM>; Tue, 19 May 2009 23:14:53 -0700 (PDT) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n4K6Er5e028759; Tue, 19 May 2009 23:14:53 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KJX00100IBFG500@fe-sfbay-10.sun.com>; Tue, 19 May 2009 23:14:53 -0700 (PDT) Received: from [10.0.2.15] ([unknown] [67.123.136.98]) by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KJX00976IOS8E70@fe-sfbay-10.sun.com>; Tue, 19 May 2009 23:14:53 -0700 (PDT) Date: Tue, 19 May 2009 23:14:52 -0700 From: "Terry (Sarito) Whatley" Subject: Re: 2009/310 Disk IO PM Enhancement In-reply-to: <20090520022921.GF105166@eng.sun.com> Sender: Terry.Whatley@sun.com To: Edward Pilatowicz Cc: Pawel Wojcik , PSARC-ext@sun.com, zfs-pm-core@sun.com Message-id: <4A139FDC.2070007@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200905191737.n4JHbjgg003157@cpubringup.sfbay.sun.com> <20090520012319.GB105166@eng.sun.com> <4A136640.3020202@Sun.COM> <20090520022921.GF105166@eng.sun.com> User-Agent: Thunderbird 2.0.0.21 (X11/20090323) Status: RO Content-Length: 2683 Edward Pilatowicz wrote: > On Tue, May 19, 2009 at 07:09:04PM -0700, Pawel Wojcik wrote: > >> On 05/19/09 06:23 PM, Edward Pilatowicz wrote: >> >>> On Tue, May 19, 2009 at 10:37:45AM -0700, Terry Whatley wrote: >>> >>> >>>> 4.3.4 disk power attribute driver properties >>>> >>>> sd(7D) will export a set of driver properties to indicate a disk's >>>> power attributes. See Table-2. >>>> >>>> Table-1 Disk Power Attribute Properties (array properties are indexed >>>> by power state in order of ascending power levels) >>>> ------------------------------------------------------------------------ >>>> Prop Name Prop Type | Prop Description >>>> ------------------------------------------------------------------------ >>>> "pm-resource-type" String | "resource-spindle-disk" for the >>>> | spindle disks >>>> ------------------------------------------------------------------------ >>>> "pm-perf" Integer array | array of average R/W >>>> | performance percentages >>>> ------------------------------------------------------------------------ >>>> "pm-pwr-saving" Integer array | array of average power saving in >>>> | units of 0.1watt >>>> ------------------------------------------------------------------------ >>>> "pm-latency" Integer array | array of time to first data in units >>>> | of 100ms >>>> ------------------------------------------------------------------------ >>>> >>>> >>>> >>> exporting performance statistics via device properties seems weird to >>> me. is there a precedent for this? why isn't this information being >>> exported via kstats? do we really want to train users to start using >>> prtconf -v to get performance data? >>> >>> ed >>> >>> >> I think these are not performance statistics. I believe that these are >> static arrays that are specific for a device type (most likely Sun disks >> only), that correlate specific power level with performance and power >> savings. These properties, I believe, are to be used by a storage power >> manager to decide at what power level disk should run at given time. >> Jane Chu may correct me here... >> -Pawel >> >> That is correct. > > if that's the case then having them as device properties seems ok, but > documentation for these properties should make it clear that these are > not actual system performance numbers. > > ed > As consolidation private interfaces, these will not be documented, aside from this case. -sarito From carlsonj@phorcys.east.sun.com Thu May 21 05:07:13 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n4LC7Csu015776 for ; Thu, 21 May 2009 05:07:13 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n4LC783L021395; Thu, 21 May 2009 20:07:11 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KJZ00803TNX7600@nwk-avmta-2.sfbay.sun.com>; Thu, 21 May 2009 05:07:09 -0700 (PDT) Received: from dm-east-02.east.sun.com ([129.148.13.5]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KJZ00J3RTNWH1C0@nwk-avmta-2.sfbay.sun.com>; Thu, 21 May 2009 05:07:09 -0700 (PDT) Received: from phorcys.east.sun.com (phorcys.East.Sun.COM [129.148.174.143]) by dm-east-02.east.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n4LC77uE008463; Thu, 21 May 2009 08:07:07 -0400 (EDT) Received: from phorcys.east.sun.com (phorcys.local [127.0.0.1]) by phorcys.east.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n4LC68DT005875; Thu, 21 May 2009 08:06:08 -0400 (EDT) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.3+Sun/8.14.3/Submit) id n4LC68w5005872; Thu, 21 May 2009 08:06:08 -0400 (EDT) Date: Thu, 21 May 2009 08:06:08 -0400 From: James Carlson Subject: Re: 2009/310 Disk IO PM Enhancement 1999/388 2005/250 2008/465 In-reply-to: <200905191737.n4JHbjgg003157@cpubringup.sfbay.sun.com> To: Terry Whatley Cc: PSARC-ext@sun.com, Terry.Whatley@sun.com, zfs-pm-core@sun.com Message-id: <18965.17328.270448.274344@gargle.gargle.HOWL> MIME-version: 1.0 X-Mailer: VM 7.01 under Emacs 21.3.1 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200905191737.n4JHbjgg003157@cpubringup.sfbay.sun.com> Status: RO Content-Length: 1367 Terry Whatley writes: > The IO framework and driver stack will be enhanced to provide generic > support of modern disks' power management capabilities. The new > features include support of SPC-4/SBC-3 compliant disk power conditions. > The SATA framework will be enhanced to provide comparable software > translation between ATA power conditions and SBC-3 power conditions. Where's the rest of this project? I would expect that, in order to finish this project, there's some work necessary to make the power management subsystem consume this data. That doesn't appear to be a part of this project, though. Are the folks responsible for supporting autopm and the rest of the power management subsystem involved with this project? If not, are they aware of what this project is planning to deliver, and do they agree that the information will be usable to make power management decisions? If so, is there a future project ("autopm disk enhancements") that is dependent on this one? Help us to connect the dots from the set of low-level driver properties described here to the higher-level goal of providing better disk management. -- James Carlson, Solaris Networking Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 From djb@sun.com Thu May 21 16:06:59 2009 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n4LN6wdU024103 for ; Thu, 21 May 2009 16:06:58 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n4LN6v51027744; Thu, 21 May 2009 16:06:57 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KK000E01O7LMT00@brm-avmta-1.central.sun.com>; Thu, 21 May 2009 17:06:57 -0600 (MDT) Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.63]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KK00046FO7KGQ80@brm-avmta-1.central.sun.com>; Thu, 21 May 2009 17:06:56 -0600 (MDT) Received: from [129.146.226.67] (cantab.SFBay.Sun.COM [129.146.226.67]) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n4LN6ubc948429; Thu, 21 May 2009 16:06:56 -0700 (PDT) Date: Thu, 21 May 2009 16:07:09 -0700 From: "David J. Brown" Subject: Re: 2009/310 Disk IO PM Enhancement 1999/388 2005/250 2008/465 In-reply-to: <18965.17328.270448.274344@gargle.gargle.HOWL> To: "Terry (Sarito) Whatley" Cc: James Carlson , PSARC-ext@sun.com, zfs-pm-core@sun.com Reply-to: david.brown@sun.com Message-id: <4A15DE9D.2020303@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200905191737.n4JHbjgg003157@cpubringup.sfbay.sun.com> <18965.17328.270448.274344@gargle.gargle.HOWL> User-Agent: Thunderbird 2.0.0.18 (X11/20081201) Status: RO Content-Length: 1982 James Carlson wrote: > Terry Whatley writes: > >> The IO framework and driver stack will be enhanced to provide generic >> support of modern disks' power management capabilities. The new >> features include support of SPC-4/SBC-3 compliant disk power conditions. >> The SATA framework will be enhanced to provide comparable software >> translation between ATA power conditions and SBC-3 power conditions. >> > > Where's the rest of this project? I would expect that, in order to > finish this project, there's some work necessary to make the power > management subsystem consume this data. That doesn't appear to be a > part of this project, though. > > Are the folks responsible for supporting autopm and the rest of the > power management subsystem involved with this project? > > If not, are they aware of what this project is planning to deliver, > and do they agree that the information will be usable to make power > management decisions? > > If so, is there a future project ("autopm disk enhancements") that is > dependent on this one? > > Help us to connect the dots from the set of low-level driver > properties described here to the higher-level goal of providing better > disk management. > > Terry, Isn't ZFS the primary consumer of these new PM-related disk driver properties [interfaces] at present? Is there a design document that describes these new conceptions for storage-related power management? Seems like that would help provide the context whereby folks could understand this a bit better. It would also be very useful to have that around in the case's materials directory for anyone who came there later, wishing to learn about this. -db -- ; David J. Brown Ph.D. (cantab.) ; Solaris Engineering ; Sun Microsystems Inc. ; -- ; Postal Address: Telephone: (650) 786-5558 ; 4150 Network Circle, UMPK17-307 FAX: (650) 786-5734 ; Santa Clara, CA 95054 e-mail: djb@sun.com From Terry.Whatley@Sun.COM Thu May 21 22:47:18 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n4M5lHuq026478 for ; Thu, 21 May 2009 22:47:17 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n4M5lF5N014746; Fri, 22 May 2009 13:47:16 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KK100C036QQU500@brm-avmta-1.central.sun.com>; Thu, 21 May 2009 23:47:14 -0600 (MDT) Received: from sca-es-mail-2.sun.com ([192.18.43.133]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KK100EFK6QQ8G90@brm-avmta-1.central.sun.com>; Thu, 21 May 2009 23:47:14 -0600 (MDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n4M5lE0S009955; Thu, 21 May 2009 22:47:14 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KK100M006LHO800@fe-sfbay-09.sun.com>; Thu, 21 May 2009 22:47:13 -0700 (PDT) Received: from [10.0.2.15] ([unknown] [67.123.136.98]) by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KK1007X96QP1NC0@fe-sfbay-09.sun.com>; Thu, 21 May 2009 22:47:13 -0700 (PDT) Date: Thu, 21 May 2009 22:47:15 -0700 From: "Terry (Sarito) Whatley" Subject: Re: 2009/310 Disk IO PM Enhancement In-reply-to: <18965.17328.270448.274344@gargle.gargle.HOWL> Sender: Terry.Whatley@Sun.COM To: James Carlson Cc: Terry Whatley , PSARC-ext@Sun.COM, zfs-pm-core@Sun.COM Message-id: <4A163C63.1040609@sun.com> MIME-version: 1.0 Content-type: multipart/alternative; boundary="Boundary_(ID_jcnESg5q5Ykzz9KjVHajfA)" X-PMX-Version: 5.4.1.325704 References: <200905191737.n4JHbjgg003157@cpubringup.sfbay.sun.com> <18965.17328.270448.274344@gargle.gargle.HOWL> User-Agent: Thunderbird 2.0.0.21 (X11/20090323) Status: RO Content-Length: 10518 This is a multi-part message in MIME format. --Boundary_(ID_jcnESg5q5Ykzz9KjVHajfA) Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT James Carlson wrote: > Terry Whatley writes: > >> The IO framework and driver stack will be enhanced to provide generic >> support of modern disks' power management capabilities. The new >> features include support of SPC-4/SBC-3 compliant disk power conditions. >> The SATA framework will be enhanced to provide comparable software >> translation between ATA power conditions and SBC-3 power conditions. >> > > Where's the rest of this project? I would expect that, in order to > finish this project, there's some work necessary to make the power > management subsystem consume this data. That doesn't appear to be a > part of this project, though. > > No, the existing power management subsystem is not intended to consume this information. The existing pm framework will eventually be superseded* by a resource manager-based architecture where power management control of different devices will be taken by "resource managers" which have specific knowledge of the pm properties and usage of the devices. Some likely examples are the Solaris power aware dispatcher (CPU), file systems (disks), an advanced VM subsystem (memory), etc. In the cases where the autopm system already expects to manage the devices (cpu, disk), control would be taken away from the existing pm framework (and later, possibly, returned). (Autopm is already accustomed to having control taken away, as X takes control of the frame buffer). * Will still exist, and will, if autopm is enabled, still control any devices that have not been claimed by a resource manager. But that is not this case. > Are the folks responsible for supporting autopm and the rest of the > power management subsystem involved with this project? > > Yes, Randy Fishel has been involved in design discussion and code review (and the x86power-iteam alias has been copied on code review requests). > If not, are they aware of what this project is planning to deliver, > and do they agree that the information will be usable to make power > management decisions? > > The information being delivered is not intended for use by the existing power management (autopm) framework. The eventual plan is to supplement that functionality (which is based on a backward-looking idleness threshold) with the resource manager-based system mentioned above. But that is not this case. The only intersection between this specific case and the existing pm framework is the pm_trans_check(9F) function, which remains backwards compatible with the existing code while extending identical functionality to a class of devices (non-SCSI disks) not currently served, since this case would make the relevant information available. > If so, is there a future project ("autopm disk enhancements") that is > dependent on this one? > > No The directly dependent case on this one is likely to be approximately "kernel interfaces for resource power managers" which might also be used by a "power aware dispatcher as a cpu resource manager" case, and likely a "file system as a pm resource manager" case, etc. We intended to present all levels together, but have been asked to produce these low level interfaces in the mean time for intermediate use while we finish the design documentation necessary to present the "next level up" case. We are confident that the interfaces exposed here will not be changed by the next level case, as they are specific to the description of the power states supported by modern disk drives, which will be needed whatever the upper level interfaces are. > Help us to connect the dots from the set of low-level driver > properties described here to the higher-level goal of providing better > disk management. > Interfaces in the current pm framework already exist that could be used to provide better disk management given the information this current case provides. There are /dev/pm ioctls (see PSARC/1999/384) that could be used by an application (or kernel subsystem, via layered ioctls) to manage the power state of any given device(s). This current case provides only the information that such a controlling entity would need to know about the pm features of modern disks. This case does not require changes in the autopm framework except minor changes in pm_trans_check(9F), which is just an enhancement to improve current functionality if additional information (cycle count info from non-scsi devices) is available, and the addition of a hook for later use (the "dip" argument). thanks, sarito --Boundary_(ID_jcnESg5q5Ykzz9KjVHajfA) Content-type: text/html; CHARSET=US-ASCII Content-transfer-encoding: 7BIT
James Carlson wrote: 
Terry Whatley writes:
  
	The IO framework and driver stack will be enhanced to provide generic
	support of modern disks' power management capabilities.  The new
	features include support of SPC-4/SBC-3 compliant disk power conditions.
	The SATA framework will be enhanced to provide comparable software
	translation between ATA power conditions and SBC-3 power conditions.
    

Where's the rest of this project?  I would expect that, in order to
finish this project, there's some work necessary to make the power
management subsystem consume this data.  That doesn't appear to be a
part of this project, though.

  No, the existing power management subsystem is not intended to consume
this information.
  The existing pm framework will eventually be superseded* by a
resource manager-based architecture where power management control
of different devices will be taken by "resource managers" which have
specific knowledge of the pm properties and usage of the devices.
  Some likely examples are the Solaris power aware dispatcher (CPU),
file systems (disks), an advanced VM subsystem (memory), etc.
  In the cases where the autopm system already expects to manage the devices
(cpu, disk), control would be taken away from the existing pm framework
(and later, possibly, returned).  (Autopm is already accustomed to having control
taken away, as X takes control of the frame buffer).

* Will still exist, and will, if autopm is enabled, still control any devices
that have not been claimed by a resource manager.

But that is not this case.
Are the folks responsible for supporting autopm and the rest of the
power management subsystem involved with this project?

Yes, Randy Fishel has been involved in design discussion and code review
(and the x86power-iteam alias has been copied on code review requests).

If not, are they aware of what this project is planning to deliver,
and do they agree that the information will be usable to make power
management decisions?

The information being delivered is not intended for use by the existing
power management (autopm) framework.  The eventual plan is to
supplement that functionality (which is based on a backward-looking
idleness threshold) with the resource manager-based system mentioned above.

But that is not this case.

The only intersection between this specific case and the existing pm
framework is the pm_trans_check(9F) function, which remains backwards
compatible with the existing code while extending identical functionality
to a class of devices (non-SCSI disks) not currently served, since this case
would make the relevant information available.
If so, is there a future project ("autopm disk enhancements") that is
dependent on this one?

No

The directly dependent case on this one is likely to be approximately
"kernel interfaces for resource power managers" which might also be
used by a "power aware dispatcher as a cpu resource manager" case, and
likely a  "file system as a pm resource manager" case, etc.

We intended to present all levels together, but have been asked to produce
these low level interfaces in the mean time for intermediate use while
we finish the design documentation necessary to present the "next level
up" case.  We are confident that the interfaces exposed here will not
be changed by the next level case, as they are specific to the description
of the power states supported by modern disk drives, which will be needed
whatever the upper level interfaces are.
Help us to connect the dots from the set of low-level driver
properties described here to the higher-level goal of providing better
disk management.
Interfaces in the current pm framework already exist that could be used to
provide better disk management given the information this current
case provides.  There are /dev/pm ioctls (see PSARC/1999/384) that
could be used by an application (or kernel subsystem, via layered ioctls)
to manage the power state of any given device(s).  This current
case provides only the information that such a controlling entity would need
to know about the pm features of modern disks.

This case does not require changes in the autopm framework except
minor changes in pm_trans_check(9F), which is just an enhancement
to improve current functionality if additional information
(cycle count info from non-scsi devices) is available, and the addition
of a hook for later use (the "dip" argument).

thanks,
sarito

--Boundary_(ID_jcnESg5q5Ykzz9KjVHajfA)-- From carlsonj@phorcys.east.sun.com Fri May 22 06:45:16 2009 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n4MDjGUv029091 for ; Fri, 22 May 2009 06:45:16 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n4MDjB26027253; Fri, 22 May 2009 06:45:15 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KK100J05SVEXT00@nwk-avmta-1.sfbay.Sun.COM>; Fri, 22 May 2009 06:45:14 -0700 (PDT) Received: from dm-east-02.east.sun.com ([129.148.13.5]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KK1005T3SVDBV90@nwk-avmta-1.sfbay.Sun.COM>; Fri, 22 May 2009 06:45:14 -0700 (PDT) Received: from phorcys.east.sun.com (phorcys.East.Sun.COM [129.148.174.143]) by dm-east-02.east.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n4MDjCH7057427; Fri, 22 May 2009 09:45:12 -0400 (EDT) Received: from phorcys.east.sun.com (phorcys.local [127.0.0.1]) by phorcys.east.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n4MDiDo5009778; Fri, 22 May 2009 09:44:13 -0400 (EDT) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.3+Sun/8.14.3/Submit) id n4MDiDKW009775; Fri, 22 May 2009 09:44:13 -0400 (EDT) Date: Fri, 22 May 2009 09:44:13 -0400 From: James Carlson Subject: Re: 2009/310 Disk IO PM Enhancement In-reply-to: <4A163C63.1040609@sun.com> To: "Terry (Sarito) Whatley" Cc: Terry Whatley , PSARC-ext@sun.com, zfs-pm-core@sun.com Message-id: <18966.44077.385692.100707@gargle.gargle.HOWL> MIME-version: 1.0 X-Mailer: VM 7.01 under Emacs 21.3.1 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200905191737.n4JHbjgg003157@cpubringup.sfbay.sun.com> <18965.17328.270448.274344@gargle.gargle.HOWL> <4A163C63.1040609@sun.com> Status: RO Content-Length: 3329 Terry (Sarito) Whatley writes: > No, the existing power management subsystem is not intended to consume > this information. > The existing pm framework will eventually be superseded* by a > resource manager-based architecture where power management control > of different devices will be taken by "resource managers" which have > specific knowledge of the pm properties and usage of the devices. OK; that sounds like a better answer. So where is the documentation for that future project? Is this case dependent on some other case that provides the higher-level architectural details? > The only intersection between this specific case and the existing pm > framework is the pm_trans_check(9F) function, which remains backwards > compatible with the existing code while extending identical functionality > to a class of devices (non-SCSI disks) not currently served, since this case > would make the relevant information available. OK. > The directly dependent case on this one is likely to be approximately > "kernel interfaces for resource power managers" which might also be > used by a "power aware dispatcher as a cpu resource manager" case, and > likely a "file system as a pm resource manager" case, etc. That sounds a bit backwards to me, at least for the ARC. I understand that, as a consumer of these interfaces, the project you're describing is "dependent" on this one to export the interfaces, but from an architectural point of view, it's that *other* case that will be defining the things that this project needs to export and specifying how they'll be used. The dependency for us (in understanding the intended system architecture and then applying that understanding to the subsystem under review) runs the other direction. I don't think we can review this project without that information. Can it be made available? (In the meantime, it sounds like this case needs to be marked "waiting need spec.") > We intended to present all levels together, but have been asked to produce > these low level interfaces in the mean time for intermediate use while > we finish the design documentation necessary to present the "next level > up" case. We are confident that the interfaces exposed here will not > be changed by the next level case, as they are specific to the description > of the power states supported by modern disk drives, which will be needed > whatever the upper level interfaces are. You're asking us to approve the cart before the horse. On what basis should we review this? "Yes, those do indeed look like a bunch of interesting parameters, and maybe somebody could use them someday" doesn't seem like the right answer. > This case does not require changes in the autopm framework except > minor changes in pm_trans_check(9F), which is just an enhancement > to improve current functionality if additional information > (cycle count info from non-scsi devices) is available, and the addition > of a hook for later use (the "dip" argument). Except, of course, that most of the interfaces described here aren't needed at all by the existing autopm framework. -- James Carlson, Solaris Networking Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 From terry.whatley@sun.com Fri May 22 11:28:43 2009 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n4MISg6V002698 for ; Fri, 22 May 2009 11:28:43 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n4MISScv027926; Fri, 22 May 2009 19:28:41 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KK2003075ZRXW00@nwk-avmta-1.sfbay.Sun.COM>; Fri, 22 May 2009 11:28:39 -0700 (PDT) Received: from sca-es-mail-2.sun.com ([192.18.43.133]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KK2008L45ZR6FE0@nwk-avmta-1.sfbay.Sun.COM>; Fri, 22 May 2009 11:28:39 -0700 (PDT) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n4MISd4t024664; Fri, 22 May 2009 11:28:39 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KK200J005MD4100@fe-sfbay-10.sun.com>; Fri, 22 May 2009 11:28:39 -0700 (PDT) Received: from [10.0.2.15] ([unknown] [67.123.136.98]) by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KK200HDD5ZM94G0@fe-sfbay-10.sun.com>; Fri, 22 May 2009 11:28:34 -0700 (PDT) Date: Fri, 22 May 2009 11:28:36 -0700 From: "Terry (Sarito) Whatley" Subject: Re: 2009/310 Disk IO PM Enhancement In-reply-to: <18966.44077.385692.100707@gargle.gargle.HOWL> Sender: terry.whatley@sun.com To: James Carlson Cc: PSARC-ext@sun.com, zfs-pm-core@sun.com Message-id: <4A16EED4.5020906@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200905191737.n4JHbjgg003157@cpubringup.sfbay.sun.com> <18965.17328.270448.274344@gargle.gargle.HOWL> <4A163C63.1040609@sun.com> <18966.44077.385692.100707@gargle.gargle.HOWL> User-Agent: Thunderbird 2.0.0.21 (X11/20090323) Status: RO Content-Length: 278 I believe that I can make the case that this case is obvious and standalone, and that it is the presentation of the material so far that is the problem, but I'm on vacation next week. So I've marked the case "waiting need spec" and will take it up again when I return. -sarito From gdamore@sun.com Fri May 22 11:43:02 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n4MIh0xj003091 for ; Fri, 22 May 2009 11:43:01 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n4MIgxGM019815; Sat, 23 May 2009 02:42:59 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KK20060B6NMPL00@nwk-avmta-2.sfbay.sun.com>; Fri, 22 May 2009 11:42:58 -0700 (PDT) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KK2004UF6NM9S50@nwk-avmta-2.sfbay.sun.com>; Fri, 22 May 2009 11:42:58 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n4MIgwK5026509; Fri, 22 May 2009 11:42:58 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KK200A0057JA700@fe-sfbay-09.sun.com>; Fri, 22 May 2009 11:42:58 -0700 (PDT) Received: from [192.168.251.11] ([unknown] [76.93.15.33]) by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KK200A626NLB3H0@fe-sfbay-09.sun.com>; Fri, 22 May 2009 11:42:57 -0700 (PDT) Date: Fri, 22 May 2009 11:42:56 -0700 From: "Garrett D'Amore" Subject: Re: 2009/310 Disk IO PM Enhancement In-reply-to: <4A16EED4.5020906@sun.com> Sender: Garrett.Damore@sun.com To: "Terry (Sarito) Whatley" Cc: James Carlson , PSARC-ext@sun.com, zfs-pm-core@sun.com Message-id: <4A16F230.4010600@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200905191737.n4JHbjgg003157@cpubringup.sfbay.sun.com> <18965.17328.270448.274344@gargle.gargle.HOWL> <4A163C63.1040609@sun.com> <18966.44077.385692.100707@gargle.gargle.HOWL> <4A16EED4.5020906@sun.com> User-Agent: Thunderbird 2.0.0.18 (X11/20081201) Status: RO Content-Length: 1077 Terry (Sarito) Whatley wrote: > I believe that I can make the case that this case is obvious > and standalone, and that it is the presentation of the material > so far that is the problem, but I'm on vacation next week. > So I've marked the case "waiting need spec" and will take > it up again when I return. I'll look at it more carefully then; but I will tell you this much -- the meaty part of the entire case seemed like it was most definitely not "obvious". Part of that may be from my general experience with SCSA and sd -- the SCSA stack is among the more complex subsystems in Solaris, IMO, and I would like to be especially careful with any change that is going to increase that complexity. (In retrospect, I believe that my hack to make blk2scsa available for non-SCSI devices to use "sd" may have been a misguided attempt to avoid having to teach all the userland subsystems about a new disk driver, but that's a different rant entirely.... :-) In any event, I'll go over the materials carefully sometime in the next week. -- Garrett > > -sarito From Terry.Whatley@sun.com Mon Jun 22 17:40:58 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n5N0evLO006997 for ; Mon, 22 Jun 2009 17:40:57 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n5N0ehFk026245; Tue, 23 Jun 2009 08:40:56 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KLO00A031W6A400@nwk-avmta-1.sfbay.Sun.COM>; Mon, 22 Jun 2009 17:40:54 -0700 (PDT) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KLO00D821W6DSA0@nwk-avmta-1.sfbay.Sun.COM>; Mon, 22 Jun 2009 17:40:54 -0700 (PDT) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n5N0es0O010874; Mon, 22 Jun 2009 17:40:54 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KLO008001UXG600@fe-sfbay-10.sun.com>; Mon, 22 Jun 2009 17:40:54 -0700 (PDT) Received: from [10.0.2.15] ([unknown] [67.123.136.98]) by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KLO007S11W62500@fe-sfbay-10.sun.com>; Mon, 22 Jun 2009 17:40:54 -0700 (PDT) Date: Mon, 22 Jun 2009 17:40:53 -0700 From: "Terry (Sarito) Whatley" Subject: Restarting 2009/310 Disk IO PM Enhancement In-reply-to: <4A16F230.4010600@sun.com> Sender: Terry.Whatley@sun.com To: PSARC-ext@sun.com, zfs-pm-core@sun.com Message-id: <4A402495.6070402@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200905191737.n4JHbjgg003157@cpubringup.sfbay.sun.com> <18965.17328.270448.274344@gargle.gargle.HOWL> <4A163C63.1040609@sun.com> <18966.44077.385692.100707@gargle.gargle.HOWL> <4A16EED4.5020906@sun.com> <4A16F230.4010600@sun.com> User-Agent: Thunderbird 2.0.0.21 (X11/20090323) Status: RO Content-Length: 3811 I'm restarting this case as a fast-track, timing out 7/1/09. The project team has removed all of the interfaces which do not have current consumers from this case. In particular the pm-* properties which described the attributes of the different power states, and the new argument "dip" to pm_trans_check(9F) are removed. The case now just brings the disk IO stack power management up to date with new transport types (particularly SATA). Below is a summary and some explanation to help make the case more obvious. Summary This case is now composed of 4 parts. 1. a property used to communicate between the transport layer and the target driver code (sd) about what support the device has for power management features, eliminating the need for the target driver to know which transport is being used and which commands it supports. 2. extend pm_trans_check(9F) support to SATA devices 3. a tunable used to override information that disk/transport HW might present; 4. a USCSI flag to allow power management to stop FMA from bringing up a power managed disk to see if it is ok. Explanation 1. target-transport communication property (pm-capable) The sd/scsa system is designed to allow the target driver (sd) not to have to know about the details ofthe transport used to control the drive (one of SCSI, SATA, SAS, FCAL, IEEEE-1394, USB, etc.), and the transport driver not to have to know the details of the target device. Unfortunately, there are several different schemes for identifying and controlling the power states of a disk, depending on the transport and the set of power states supported by the disk, and to provide access to information about how many power cycles the disk has experienced (so that they can be meted out over the desired lifetime of the disk and not squandered immediately upon putting the disk in service). In particular, for control the current options are the Start field of the Start Stop Unit command, and the Power Condition field of the Start Stop Unit command. For getting the start-stop history information, the options are check or ignore log page 0Eh, and to expect (if not ignored) for log page 0Eh to contain SMART attribute data or SCSI native log page 0Eh information. The pm-capable property is extended to communicate this information to the target driver. This information is needed to allow the sd driver to support the existing autopm framework behavior on the new transports. This scheme was worked out by/in collaboration with the owners of all of the drivers involved. While the specific values in this property may not be obvious to those not familiar with the detailed specs of the various transports, I submit that using a property which already conveys related information to convey this info rather than perturbing the scsa/sd interface is obvious. 2. pm_trans_check change The modification of pm_trans_check() allows for the slightly different flavor of information (native SCSI vs. SMART) provided by different transports to be used in making power down decisions. 3. tunable Sometimes the firmware is buggy and lies about the capabilities of the device. The "power-condition" tunable allows a way to override the information presented by firmware for a specific device identified by Product ID and Vendor ID. 4. USCSI_PMFAILFAST flag There is a problem where an FMA disk health checking thread spins up disks periodically to test device status. This flag allows for the device driver to not spin up the device to get this information. The full text of the new proposal is in the materials directory as "proposal". thanks, sarito From gdamore@sun.com Wed Jun 24 09:23:50 2009 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n5OGNnb6004291 for ; Wed, 24 Jun 2009 09:23:49 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n5OGNlqB021766; Wed, 24 Jun 2009 10:23:49 -0600 (MDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KLR0011P47NTZ00@brm-avmta-1.central.sun.com>; Wed, 24 Jun 2009 10:23:47 -0600 (MDT) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KLR00FKR47IP970@brm-avmta-1.central.sun.com>; Wed, 24 Jun 2009 10:23:42 -0600 (MDT) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n5OGNgA8014955; Wed, 24 Jun 2009 09:23:42 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KLR00E003XN7M00@fe-sfbay-10.sun.com>; Wed, 24 Jun 2009 09:23:42 -0700 (PDT) Received: from [192.168.251.11] ([unknown] [76.93.15.33]) by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KLR00B9Q47GJ8F0@fe-sfbay-10.sun.com>; Wed, 24 Jun 2009 09:23:41 -0700 (PDT) Date: Wed, 24 Jun 2009 09:23:40 -0700 From: "Garrett D'Amore" Subject: Re: Restarting 2009/310 Disk IO PM Enhancement In-reply-to: <4A402495.6070402@sun.com> Sender: Garrett.Damore@sun.com To: "Terry (Sarito) Whatley" Cc: PSARC-ext@sun.com, zfs-pm-core@sun.com Message-id: <4A42530C.6030308@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200905191737.n4JHbjgg003157@cpubringup.sfbay.sun.com> <18965.17328.270448.274344@gargle.gargle.HOWL> <4A163C63.1040609@sun.com> <18966.44077.385692.100707@gargle.gargle.HOWL> <4A16EED4.5020906@sun.com> <4A16F230.4010600@sun.com> <4A402495.6070402@sun.com> User-Agent: Thunderbird 2.0.0.18 (X11/20081201) Status: RO Content-Length: 4325 The case looks pretty good now, and I'm almost ready to issue a +1, but I have one question, which concerns the pm-capable attribute. I think other HBAs might be using this property. Has the project team verified that no other consumers exist? Or if the 16-th bit is not set in the new mask value, does it behave in the old fashion? - Garrett Terry (Sarito) Whatley wrote: > I'm restarting this case as a fast-track, timing out > 7/1/09. > > The project team has removed all of the interfaces which > do not have current consumers from this case. > In particular the pm-* properties which described the > attributes of the different power states, and the new > argument "dip" to pm_trans_check(9F) are removed. > > The case now just brings the disk IO stack power > management up to date with new transport types > (particularly SATA). > > Below is a summary and some explanation to help make > the case more obvious. > > Summary > > This case is now composed of 4 parts. > 1. a property used to communicate between the transport layer > and the target driver code (sd) about what support the > device has for power management features, > eliminating the need for the target driver to know which > transport is being used and which commands it supports. > 2. extend pm_trans_check(9F) support to SATA devices > 3. a tunable used to override information that disk/transport > HW might present; > 4. a USCSI flag to allow power management to stop FMA from > bringing up a power managed disk to see if it is ok. > > Explanation > > 1. target-transport communication property (pm-capable) > > The sd/scsa system is designed to allow the target driver (sd) > not to have to know about the details ofthe transport used to > control the drive (one of SCSI, SATA, SAS, FCAL, IEEEE-1394, > USB, etc.), and the transport driver not to have to know > the details of the target device. > > Unfortunately, there are several different schemes for > identifying and controlling the power states of a disk, > depending on the transport and the set of power states supported > by the disk, and to provide access to information about how many > power cycles the disk has experienced (so that they can be meted > out over the desired lifetime of the disk and not squandered > immediately upon putting the disk in service). > > In particular, for control the current options are the > Start field of the Start Stop Unit command, and the > Power Condition field of the Start Stop Unit command. > For getting the start-stop history information, the options > are check or ignore log page 0Eh, and to expect (if not > ignored) for log page 0Eh to contain SMART attribute data > or SCSI native log page 0Eh information. > > The pm-capable property is extended to communicate this > information to the target driver. > > This information is needed to allow the sd driver to > support the existing autopm framework behavior on the new > transports. > > This scheme was worked out by/in collaboration with the owners of > all of the drivers involved. While the specific values in > this property may not be obvious to those not familiar with the > detailed specs of the various transports, I submit that using > a property which already conveys related information to convey > this info rather than perturbing the scsa/sd interface is obvious. > > > 2. pm_trans_check change > > The modification of pm_trans_check() allows for the slightly > different flavor of information (native SCSI vs. SMART) > provided by different transports to be used in making power > down decisions. > > > 3. tunable > > Sometimes the firmware is buggy and lies about the capabilities > of the device. The "power-condition" tunable allows a way to > override the information presented by firmware for a specific > device identified by Product ID and Vendor ID. > > > 4. USCSI_PMFAILFAST flag > > There is a problem where an FMA disk health checking thread spins > up disks periodically to test device status. This flag allows > for the device driver to not spin up the device to get this > information. > > The full text of the new proposal is in the materials directory as > "proposal". > > thanks, > sarito > From Jane.Chu@sun.com Wed Jun 24 09:40:03 2009 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n5OGe3IG005034 for ; Wed, 24 Jun 2009 09:40:03 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n5OGe1F9023529; Wed, 24 Jun 2009 09:40:03 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KLR00A094YPW800@nwk-avmta-2.sfbay.sun.com>; Wed, 24 Jun 2009 09:40:01 -0700 (PDT) Received: from sca-es-mail-2.sun.com ([192.18.43.133]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KLR00AIY4YPLO00@nwk-avmta-2.sfbay.sun.com>; Wed, 24 Jun 2009 09:40:01 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n5OGe147024416; Wed, 24 Jun 2009 09:40:01 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KLR002004IF3A00@fe-sfbay-09.sun.com>; Wed, 24 Jun 2009 09:40:01 -0700 (PDT) Received: from [192.168.0.2] ([unknown] [76.200.190.88]) by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KLR00MK54YDF4B0@fe-sfbay-09.sun.com>; Wed, 24 Jun 2009 09:39:50 -0700 (PDT) Date: Wed, 24 Jun 2009 09:39:49 -0700 From: Jane Chu Subject: Re: Restarting 2009/310 Disk IO PM Enhancement In-reply-to: <4A42530C.6030308@sun.com> Sender: Jane.Chu@sun.com To: "Garrett D'Amore" Cc: "Terry (Sarito) Whatley" , PSARC-ext@sun.com, zfs-pm-core@sun.com Message-id: <4A4256D5.9040201@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200905191737.n4JHbjgg003157@cpubringup.sfbay.sun.com> <18965.17328.270448.274344@gargle.gargle.HOWL> <4A163C63.1040609@sun.com> <18966.44077.385692.100707@gargle.gargle.HOWL> <4A16EED4.5020906@sun.com> <4A16F230.4010600@sun.com> <4A402495.6070402@sun.com> <4A42530C.6030308@sun.com> User-Agent: Thunderbird 2.0.0.17 (X11/20081023) Status: RO Content-Length: 4845 Hi, Garrett, Garrett D'Amore wrote: > The case looks pretty good now, and I'm almost ready to issue a +1, > but I have one question, which concerns the pm-capable attribute. I > think other HBAs might be using this property. Has the project team > verified that no other consumers exist? Or if the 16-th bit is not > set in the new mask value, does it behave in the old fashion? The improvement of the "pm-capable" property is backwards compatible with all existing HBA drivers with-in and with-out ON. If the 16-th bit is not set, it would behave in the old fashion, that is, sd would skip checking the Log page 0Eh and honor the PM capability. There are existing HBAs that do not set bit 16, such as USB SCSI, 1394 SCSI. thanks, -jane > > - Garrett > > Terry (Sarito) Whatley wrote: >> I'm restarting this case as a fast-track, timing out >> 7/1/09. >> >> The project team has removed all of the interfaces which >> do not have current consumers from this case. >> In particular the pm-* properties which described the >> attributes of the different power states, and the new >> argument "dip" to pm_trans_check(9F) are removed. >> >> The case now just brings the disk IO stack power >> management up to date with new transport types >> (particularly SATA). >> >> Below is a summary and some explanation to help make >> the case more obvious. >> >> Summary >> >> This case is now composed of 4 parts. >> 1. a property used to communicate between the transport layer >> and the target driver code (sd) about what support the >> device has for power management features, >> eliminating the need for the target driver to know which >> transport is being used and which commands it supports. >> 2. extend pm_trans_check(9F) support to SATA devices >> 3. a tunable used to override information that disk/transport >> HW might present; >> 4. a USCSI flag to allow power management to stop FMA from >> bringing up a power managed disk to see if it is ok. >> >> Explanation >> >> 1. target-transport communication property (pm-capable) >> >> The sd/scsa system is designed to allow the target driver (sd) >> not to have to know about the details ofthe transport used to >> control the drive (one of SCSI, SATA, SAS, FCAL, IEEEE-1394, >> USB, etc.), and the transport driver not to have to know >> the details of the target device. >> >> Unfortunately, there are several different schemes for >> identifying and controlling the power states of a disk, >> depending on the transport and the set of power states supported >> by the disk, and to provide access to information about how many >> power cycles the disk has experienced (so that they can be meted >> out over the desired lifetime of the disk and not squandered >> immediately upon putting the disk in service). >> >> In particular, for control the current options are the >> Start field of the Start Stop Unit command, and the >> Power Condition field of the Start Stop Unit command. >> For getting the start-stop history information, the options >> are check or ignore log page 0Eh, and to expect (if not >> ignored) for log page 0Eh to contain SMART attribute data >> or SCSI native log page 0Eh information. >> >> The pm-capable property is extended to communicate this >> information to the target driver. >> >> This information is needed to allow the sd driver to >> support the existing autopm framework behavior on the new >> transports. >> >> This scheme was worked out by/in collaboration with the owners of >> all of the drivers involved. While the specific values in >> this property may not be obvious to those not familiar with the >> detailed specs of the various transports, I submit that using >> a property which already conveys related information to convey >> this info rather than perturbing the scsa/sd interface is obvious. >> >> >> 2. pm_trans_check change >> >> The modification of pm_trans_check() allows for the slightly >> different flavor of information (native SCSI vs. SMART) >> provided by different transports to be used in making power >> down decisions. >> >> >> 3. tunable >> >> Sometimes the firmware is buggy and lies about the capabilities >> of the device. The "power-condition" tunable allows a way to >> override the information presented by firmware for a specific >> device identified by Product ID and Vendor ID. >> >> >> 4. USCSI_PMFAILFAST flag >> >> There is a problem where an FMA disk health checking thread spins >> up disks periodically to test device status. This flag allows >> for the device driver to not spin up the device to get this >> information. >> >> The full text of the new proposal is in the materials directory as >> "proposal". >> >> thanks, >> sarito >> > From gdamore@Sun.COM Wed Jun 24 09:47:43 2009 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n5OGlhBX005384 for ; Wed, 24 Jun 2009 09:47:43 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n5OGlgLE027783; Wed, 24 Jun 2009 09:47:43 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KLR004015BJAX00@nwk-avmta-1.sfbay.Sun.COM>; Wed, 24 Jun 2009 09:47:43 -0700 (PDT) Received: from sca-es-mail-2.sun.com ([192.18.43.133]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KLR00M755BIAL30@nwk-avmta-1.sfbay.Sun.COM>; Wed, 24 Jun 2009 09:47:42 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n5OGlfUh025377; Wed, 24 Jun 2009 09:47:41 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KLR00J0051U0L00@fe-sfbay-09.sun.com>; Wed, 24 Jun 2009 09:47:41 -0700 (PDT) Received: from [192.168.251.11] ([unknown] [76.93.15.33]) by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KLR00MOY5BGF4E0@fe-sfbay-09.sun.com>; Wed, 24 Jun 2009 09:47:40 -0700 (PDT) Date: Wed, 24 Jun 2009 09:47:40 -0700 From: "Garrett D'Amore" Subject: Re: Restarting 2009/310 Disk IO PM Enhancement In-reply-to: <4A4256D5.9040201@sun.com> Sender: Garrett.Damore@Sun.COM To: Jane Chu Cc: "Terry (Sarito) Whatley" , PSARC-ext@Sun.COM, zfs-pm-core@Sun.COM Message-id: <4A4258AC.30501@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200905191737.n4JHbjgg003157@cpubringup.sfbay.sun.com> <18965.17328.270448.274344@gargle.gargle.HOWL> <4A163C63.1040609@sun.com> <18966.44077.385692.100707@gargle.gargle.HOWL> <4A16EED4.5020906@sun.com> <4A16F230.4010600@sun.com> <4A402495.6070402@sun.com> <4A42530C.6030308@sun.com> <4A4256D5.9040201@sun.com> User-Agent: Thunderbird 2.0.0.18 (X11/20081201) Status: RO Content-Length: 5065 Jane Chu wrote: > Hi, Garrett, > > Garrett D'Amore wrote: >> The case looks pretty good now, and I'm almost ready to issue a +1, >> but I have one question, which concerns the pm-capable attribute. I >> think other HBAs might be using this property. Has the project team >> verified that no other consumers exist? Or if the 16-th bit is not >> set in the new mask value, does it behave in the old fashion? > The improvement of the "pm-capable" property is backwards compatible > with all existing HBA drivers > with-in and with-out ON. If the 16-th bit is not set, it would behave > in the old fashion, that is, sd would > skip checking the Log page 0Eh and honor the PM capability. There are > existing HBAs that do not > set bit 16, such as USB SCSI, 1394 SCSI. (That was exactly my concern! :-) With that, +1. - Garrett > > thanks, > -jane > > >> >> - Garrett >> >> Terry (Sarito) Whatley wrote: >>> I'm restarting this case as a fast-track, timing out >>> 7/1/09. >>> >>> The project team has removed all of the interfaces which >>> do not have current consumers from this case. >>> In particular the pm-* properties which described the >>> attributes of the different power states, and the new >>> argument "dip" to pm_trans_check(9F) are removed. >>> >>> The case now just brings the disk IO stack power >>> management up to date with new transport types >>> (particularly SATA). >>> >>> Below is a summary and some explanation to help make >>> the case more obvious. >>> >>> Summary >>> >>> This case is now composed of 4 parts. >>> 1. a property used to communicate between the transport layer >>> and the target driver code (sd) about what support the >>> device has for power management features, >>> eliminating the need for the target driver to know which >>> transport is being used and which commands it supports. >>> 2. extend pm_trans_check(9F) support to SATA devices >>> 3. a tunable used to override information that disk/transport >>> HW might present; >>> 4. a USCSI flag to allow power management to stop FMA from >>> bringing up a power managed disk to see if it is ok. >>> >>> Explanation >>> >>> 1. target-transport communication property (pm-capable) >>> >>> The sd/scsa system is designed to allow the target driver (sd) >>> not to have to know about the details ofthe transport used to >>> control the drive (one of SCSI, SATA, SAS, FCAL, IEEEE-1394, >>> USB, etc.), and the transport driver not to have to know >>> the details of the target device. >>> >>> Unfortunately, there are several different schemes for >>> identifying and controlling the power states of a disk, >>> depending on the transport and the set of power states supported >>> by the disk, and to provide access to information about how many >>> power cycles the disk has experienced (so that they can be meted >>> out over the desired lifetime of the disk and not squandered >>> immediately upon putting the disk in service). >>> >>> In particular, for control the current options are the >>> Start field of the Start Stop Unit command, and the >>> Power Condition field of the Start Stop Unit command. >>> For getting the start-stop history information, the options >>> are check or ignore log page 0Eh, and to expect (if not >>> ignored) for log page 0Eh to contain SMART attribute data >>> or SCSI native log page 0Eh information. >>> >>> The pm-capable property is extended to communicate this >>> information to the target driver. >>> >>> This information is needed to allow the sd driver to >>> support the existing autopm framework behavior on the new >>> transports. >>> >>> This scheme was worked out by/in collaboration with the owners of >>> all of the drivers involved. While the specific values in >>> this property may not be obvious to those not familiar with the >>> detailed specs of the various transports, I submit that using >>> a property which already conveys related information to convey >>> this info rather than perturbing the scsa/sd interface is obvious. >>> >>> >>> 2. pm_trans_check change >>> >>> The modification of pm_trans_check() allows for the slightly >>> different flavor of information (native SCSI vs. SMART) >>> provided by different transports to be used in making power >>> down decisions. >>> >>> >>> 3. tunable >>> >>> Sometimes the firmware is buggy and lies about the capabilities >>> of the device. The "power-condition" tunable allows a way to >>> override the information presented by firmware for a specific >>> device identified by Product ID and Vendor ID. >>> >>> >>> 4. USCSI_PMFAILFAST flag >>> >>> There is a problem where an FMA disk health checking thread spins >>> up disks periodically to test device status. This flag allows >>> for the device driver to not spin up the device to get this >>> information. >>> >>> The full text of the new proposal is in the materials directory as >>> "proposal". >>> >>> thanks, >>> sarito >>> >> > From Terry.Whatley@sun.com Wed Jun 24 10:56:25 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n5OHuO9O011751 for ; Wed, 24 Jun 2009 10:56:24 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n5OHuK0b014708; Thu, 25 Jun 2009 01:56:23 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KLR00F0D8HWAU00@nwk-avmta-2.sfbay.sun.com>; Wed, 24 Jun 2009 10:56:20 -0700 (PDT) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KLR00A4Z8HVLO70@nwk-avmta-2.sfbay.sun.com>; Wed, 24 Jun 2009 10:56:19 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n5OHuJJB028084; Wed, 24 Jun 2009 10:56:19 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KLR008007PM1W00@fe-sfbay-09.sun.com>; Wed, 24 Jun 2009 10:56:19 -0700 (PDT) Received: from [10.0.2.15] ([unknown] [192.18.41.196]) by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KLR002FZ8HH4V90@fe-sfbay-09.sun.com>; Wed, 24 Jun 2009 10:56:06 -0700 (PDT) Date: Wed, 24 Jun 2009 10:56:04 -0700 From: "Terry (Sarito) Whatley" Subject: 2009/310 Disk IO PM Enhancement approved, impact on 1999/388, 2008/465, 2005/250 In-reply-to: <4A16F230.4010600@sun.com> Sender: Terry.Whatley@sun.com To: PSARC-ext@sun.com, zfs-pm-core@sun.com Message-id: <4A4268B4.60003@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200905191737.n4JHbjgg003157@cpubringup.sfbay.sun.com> <18965.17328.270448.274344@gargle.gargle.HOWL> <4A163C63.1040609@sun.com> <18966.44077.385692.100707@gargle.gargle.HOWL> <4A16EED4.5020906@sun.com> <4A16F230.4010600@sun.com> User-Agent: Thunderbird 2.0.0.22 (X11/20090612) Status: RO Content-Length: 300 This case (2009/310) was approved at the PSARC meeting today. For impact this case has on the other cases listed, search for the case numbers in /shared/sac/PSARC/2009/310/materials/proposal . [the purpose of this mail is to get the above sentence into the mail files of the other cases] -sarito