From sacadmin Tue May 13 00:36:02 2008 Received: from groomlake.sfbay.sun.com (groomlake.SFBay.Sun.COM [192.29.77.113]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m4D7a2Fb008594; Tue, 13 May 2008 00:36:02 -0700 (PDT) Received: from groomlake.sfbay.sun.com (localhost [127.0.0.1]) by groomlake.sfbay.sun.com (8.14.1+Sun/8.14.1) with ESMTP id m4D6WFUT004140; Mon, 12 May 2008 23:32:15 -0700 (PDT) Received: (from ap25164@localhost) by groomlake.sfbay.sun.com (8.14.1+Sun/8.14.1/Submit) id m4D6WFul004136; Mon, 12 May 2008 23:32:15 -0700 (PDT) Date: Mon, 12 May 2008 23:32:15 -0700 (PDT) From: Alan Perry Message-Id: <200805130632.m4D6WFul004136@groomlake.sfbay.sun.com> To: PSARC@sac.sfbay.sun.com Cc: sata-dev@sun.com Subject: IOCTL interface for drive LED control [PSARC/2008/317 FastTrack timeout 05/20/2008] Status: RO Content-Length: 10797 I am sponsoring this fast-track for Pawel. It times out on 20 May. Template Version: @(#)sac_nextcase 1.66 04/17/08 SMI This information is Copyright 2008 Sun Microsystems, Inc. 1. Introduction 1.1. Project/Component Working Name: IOCTL interface for drive LED control 1.2. Name of Document Author/Supplier: Author: Pawel Wojcik 1.3 Date of This Document: 12 May, 2008 2. References [1] SFF-8485 Specification for Serial GPIO (SGPIO) Bus [2] Sun_Microsystems_HDD_LED_Requirements_Rev_1_8 4. Technical Description 4.1 Overview Drive-related LED indicators may be controlled by various hardware entities. Generally, Serial General Purpose Input/Output (SGPIO) interface is used to control LED indicators on the backplane. SGPIO is a 4-signal (or 4-wire) bus used between a Host Bus Adapter (HBA) and a backplane. Out of the 4 signals, 3 are driven by the HBA or other hardware controller and 1 is driven by the backplane. The data bits exchanged are related to drives supported by the target and are typically related to specific lanes of the high-speed serial storage interface. SGPIO and the SGPIO spec is generally adopted and implemented in products from most major HBA and Storage Controller vendors such as LSI, Intel, Adaptec, Nvidia, Broadcom and PMC-Sierra. Most products shipping with support for SAS and SATA drives support this standard. The SGPIO specification is given the official name SFF-8485 and is maintained by the SFF Committee. 4.2 Problem Description On most Sun platforms FAULT (Amber) and OK2RM (Blue) LEDs are controlled by Service Processor (SP) via IPMI, while ACTIVITY (Green) LED is controlled by the disk HBA. On some other platforms that use hardware RAID HBA, the FAULT LED may be controlled by HBA along with the ACTIVITY LED. Dorado/Tucana platform employs SATA controller embedded in nVidia chipset, which provides SGPIO interface, that can be used to control all type of drive-related LEDs. There may be other platforms, employing either nVidia chipset or Intel's ICH hub that could use SGPIO to control drive-related LEDs. Currently there are no APIs that HBA drivers implement to provide control over the LEDs for the target disk drives. This project proposes the new APIs for HBA drivers to provide access to LEDs that it may control. 4.3 Proposal This fasttrack proposes a new ioctl to control FAIL and OK2RM LED indicators via SGPIO interface of a Host Bus Adapter. Controlling software (zfs, FMA or other entity with the adequate permission) may use this interface to turn-on and turn-off FAULT and OK2RM LED indicators using HBA SGPIO hardware interface. If HBA does not support SGPIO interface, the ioctl would fail with an appropriate error. The default LED state set by HBA (without an explicit command to turn LED ON) would be OFF state. If the Service Processor controls FAULT LED via IPMI interface, IPMI LED ON setting will override the HBA LED OFF setting. 4.3.1 Considerations The DKIO type ioctl was considered and rejected, because it requires a presence of a device target node. In case of a failed device the target node for the device may not exists. Furthermore, there is no path in scsi target driver and scsi module to pass ioctls that are not processed by these modules to the HBA drivers. The new DEVCTL type ioctl was selected because its use does not depend on the presence of a device target node. The new ioctl would have to be implemented by HBA drivers providing SGPIO interface. An application will have to identify (using /device path) and open HBA control node (using /device path), identify LED type and number for this HBA using platform topo info, and execute ioctl call. In the future, the task of deriving the control node path could be delegated to libdevice libraray, when this interface would be used on many platforms. The ioctl-specific structure is used to pass LED specification to the HBA. The alternative would be to use nvlist. 4.4 Requested Binding patch 4.5 Details Exported Interface Classification Comments --------------------------------------------------------------- DEVCTL_SETLED | consolidation-private | Turn ON/OFF drive | | indicator LED | | --------------------------------------------------------------- DEVCTL_GETLED | consolidation-private | Get status of drive | | indicator LED | | --------------------------------------------------------------- struct dc_led_ctl | consolidation-private | pointer to this | | structure passed | | as ioctl argument | | --------------------------------------------------------------- SYNOPSIS: int ioctl(int fildes, int request, arg); DESCRIPTION The fildes argument is an open file descriptor that refers to a HBA control node. The request is DEVCTL_SETLED or DEVCTL_GETLED #define DEVCTL_SET_LED (DEVCTL_IOC | 39) #define DEVCTL_GET_LEDSTATE (DEVCTL_IOC | 40) The argument is a pointer to dc_led_ctl structure: struct dc_led_ctl { uint32_t led_number : 16; /* LED/device number */ uint32_t led_ctl_active : 1; /* Control active */ uint32_t led_type : 3; /* LED type */ uint32_t led_state: 4; /* LED ON/OFF/Blink state */ } /* Control active field *' #define DCL_CNTRL_OFF 0 /* Control inactive */ #define DCL_CNTRL_ON 1 /* Control active */ /* LED type field */ #define DCL_TYPE_FAIL 1 /* FAIL LED type */ #define DCL_TYPE_OK2RM 2 /* OK2RM LED type */ /* Led state field */ #define DCL_STATE_OFF 0 /* LED state OFF */ #define DCL_STATE_ON 1 /* LED state ON */ #define DCL_STATE_SLOW_BLNK 2 /* LED Slow blink */ #define DCL_STATE_FAST_BLNK 3 /* LED Fast blink */ led_num should be a value corresponding to the device number to which the LED indicator is related. In practical terms it is the LED/drive data position in the SGPIO's SDataOut bit-stream where each drive is represented by 3 bits (3-touple). The value range is 0 to n-1, where n is the number of devices supported by HBA instance. The LED number should be available in platform topology info. Note: For nVidia's MCP55 and IO55 chips and system using nv_sata driver, LED number is the same as the disk number in physical device path /devices/../disk@Y. For DEVCTL_SET_LED request: Caller has to set all fields of the dc_led_ctl structure. If DCL_CNTRL_OFF is specified, all other parameters are ignored and HBA should make SPGIO inactive (put SGPIO in tri-state) for all devices controlled by the HBA. Slow blink and fast blink rates are defined by Sun document Sun_Microsystems_HDD_LED_Requirements_Rev_1_8. If DCL_CNTRL_ON is specified, caller has to set all fields of the dc_led_ctl structure. For DEVCTL_GET_LEDSTATE request: Caller has to specify led_type and led_number in dc_led_ctl structure. HBA will set led_ctl_active and led_state fields. The LED state is not persistent over reboots. If the SGPIO control interface was not explicitly enabled via DEVCTL_SET_LED with DCL_CNTRL_ON request, the LED state is DCL_STATE_OFF for every LED controlled by the HBA instance. IOCTL may specify one LED type and device number at a time. Initial state of the SGPIO lines is tri-state (not driven by the HBA). In such state the default LED setting as provided by a backplane SGPIO target hardware determines the actual LED state and such state is unknown to the HBA driver. The default state of a LED indicator driven by HBA is OFF, i.e. if the LED of any type is not explicitly specified to be ON, the HBA should specify it state as OFF. If the system BIOS (or BIOS-extension) does not enable SGPIO interface, an explicit ioctl has to be sent to eenable the SGPIO interface even when only ACTIVITY LEDs are to be under HBA control (normally not controlled via DEVCTL_SET_LED ioctl). Some HBAs (such as hardware RAID HBAs) may be actively control FAIL and/or OK2RM LEDs. For these HBAs the initial state of LEDs is HBA-specific and cntrl_active field of the dc_led_ctl structure may be ignored by the HBA (ie. SGPIO interface cannot be deactivated and/or tri-stated). Drivers for such HBAs return DCL_CNTLR_ON sate in the cntrl_active field even without an explicit request to activate the control. Implementation Note: The ioctl call should return immediately. In case of the DEVCTL_SET_LED request the HBA driver should store the request internally and execute it on a separate execution thread if it cannot be executed immediately. The SATA HBA driver should maintain the state of LEDs for the purpose of providing LEDs state in response to DEVCTL_GET_LEDSTATE request. The LED control should not interfere with normal I/O activity (if any) on the specified device. PRIV_ALL privilege is required to invoke DEVCTL_LED_CTL ioctl. RETURN VALUES Upon successful completion the return value is 0. Otherwise -1 is returned and errno is set to indicate an error. ERRORS The IOCTL would fail if: ENOSUP HBA does not support LED control ENXIO the specified drive LED number is not supported by the controller and/or HBA driver. EINVAL the LED type or state is not supported ENOPERM no permision to execute this ioctl (not owner) 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: Fasttrack 6.6. ARC Exposure: open 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: FastTrack 6.6. ARC Exposure: closed 6.6.1. Rationale: The Sun document describing LED behavior is marked XXXfidential and is under NDA From sacadmin Tue May 13 06:04:41 2008 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m4DD4fe3015872 for ; Tue, 13 May 2008 06:04:41 -0700 (PDT) Received: from gmp-eb-inf-1.sun.com (gmp-eb-inf-1.EU.Sun.COM [192.18.6.21]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m4DD4fQp026621 for ; Tue, 13 May 2008 06:04:41 -0700 (PDT) Received: from fe-emea-09.sun.com (gmp-eb-lb-2-fe3.eu.sun.com [192.18.6.12]) by gmp-eb-inf-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m4DD4Zqq023070 for ; Tue, 13 May 2008 13:04:35 GMT Received: from conversion-daemon.fe-emea-09.sun.com by fe-emea-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K0T001015GA7P00@fe-emea-09.sun.com> (original mail from Darren.Moffat@Sun.COM) for PSARC@sac.sfbay.sun.com; Tue, 13 May 2008 14:04:35 +0100 (BST) Received: from [129.156.173.21] by fe-emea-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K0T00BNF5NAC510@fe-emea-09.sun.com>; Tue, 13 May 2008 14:04:23 +0100 (BST) Date: Tue, 13 May 2008 14:04:22 +0100 From: Darren J Moffat Subject: Re: IOCTL interface for drive LED control [PSARC/2008/317 FastTrack timeout 05/20/2008] In-reply-to: <200805130632.m4D6WFul004136@groomlake.sfbay.sun.com> Sender: Darren.Moffat@Sun.COM To: Alan Perry Cc: PSARC@sac.sfbay.sun.com, sata-dev@Sun.COM Message-id: <482991D6.7020004@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200805130632.m4D6WFul004136@groomlake.sfbay.sun.com> User-Agent: Thunderbird 2.0.0.12 (X11/20080326) Status: RO Content-Length: 905 Alan Perry wrote: > PRIV_ALL privilege is required to invoke DEVCTL_LED_CTL ioctl. Is it really that much of a security risk ? Requiring this much privilege to invoke this ioctl means that system daemons/commands that could otherwise run with very little (or may no) privilege have to run with all privilege instead. If an existing privilege is to be used then I think "sys_devices" is the closest match though I'm starting to think we need a privilege like sys_ip_config for "disk". Is there a similar interface in Linux or BSD systems ? If so how does this new Solaris interface compare, or do they not have the ability to set these LED ? Given that this case is closed (because of the closed reference document I believe) where is the source going to end up ? If the intent is that HBA drivers are implementing this ioctl why is it Consolidation Private to ON ? -- Darren J Moffat From sacadmin Tue May 13 08:52:58 2008 Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01.SFBay.Sun.COM [129.145.155.118]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m4DFqwdL021089 for ; Tue, 13 May 2008 08:52:58 -0700 (PDT) Received: from sca-es-mail-1.sun.com (sca-es-mail-1.Sun.COM [192.18.43.132]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m4DFqvU2018388 for ; Tue, 13 May 2008 08:52:57 -0700 (PDT) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m4DFqq4F009102 for ; Tue, 13 May 2008 08:52:52 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K0T00601AIN3C00@fe-sfbay-10.sun.com> (original mail from gdamore@sun.com) for PSARC@sac.sfbay.sun.com; Tue, 13 May 2008 08:52:52 -0700 (PDT) Received: from [192.168.251.106] ([76.174.83.55]) by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K0T008KYDFXYIF0@fe-sfbay-10.sun.com>; Tue, 13 May 2008 08:52:45 -0700 (PDT) Date: Tue, 13 May 2008 08:46:39 -0700 From: "Garrett D'Amore" Subject: Re: IOCTL interface for drive LED control [PSARC/2008/317 FastTrack timeout 05/20/2008] In-reply-to: <200805130632.m4D6WFul004136@groomlake.sfbay.sun.com> Sender: Garrett.Damore@sun.com To: Alan Perry Cc: PSARC@sac.sfbay.sun.com, sata-dev@sun.com Message-id: <4829B7DF.7080201@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200805130632.m4D6WFul004136@groomlake.sfbay.sun.com> User-Agent: Thunderbird 2.0.0.6 (X11/20071023) Status: RO Content-Length: 13358 This looks pretty good, but I do wonder if there should be #define's for different LED numbers, to identify which is which? Furthermore, in my opinion, I think it would be useful to allow different LED numbers (and perhaps "types of LEDs" to be used by this ioctl interface, as I think it is likely that this control interface has utility beyond just disk drives. (A common LED API is potentially useful to other kinds of systems/devices.) In order to be "common", I think a way to inquire what kinds of LEDs are present/supported would be useful. If the project team doesn't want to worry about other kinds of uses outside of disk drives for now, that's OK, too. It would still be nice to "plan" for future extensions though -- perhaps by having a simple inquiry interface: DEVCTL_NUM_LEDs which simply returns the total number of LEDs supported by the devctl node, and then by extending DEVCTL_GET_LED a bit, perhaps by adding something like: uint32_t led_capab; #define DEVCTL_CAP_SLOW_BLINK (1<< 0) #define DEVCTL_CAP_FAST_BLINK (1<<1) and then changing the led_type #defines just a little bit: #define DC_TYPE_DEVICE_FAIL 1 #define DC_TYPE_DEVICE_OK2RM 2 #define DC_TYPE_BUS_FAIL 3 (this just makes it clear that these LEDs are related to disk drives, and not to the HBA itself. One can imagine an LED with a HBA failure indicator (DC_TYPE_BUS_FAIL) separate from a drive (DC_TYPE_DEVICE_FAIL) failure. Nit: I think you need to correct your #define's for the ioctl commands. I *think* you probably meant to define DEVCTL_SET_LED and DEVCTL_GET_LED, rather than DEVCTL_GET_LEDSTATE. :-) Privilege: does PRIV_ALL really make the most sense here? I'd think that typically whatever privilege is required to open the device node for devctl (either in r/w or r/o mode) should be sufficient. It doesn't seem that there is much utility in having a different privilege requirement for these operations than what would be required to perform other cfgadm related operations. -- Garrett Alan Perry wrote: > I am sponsoring this fast-track for Pawel. It times out on 20 May. > > Template Version: @(#)sac_nextcase 1.66 04/17/08 SMI > This information is Copyright 2008 Sun Microsystems, Inc. > 1. Introduction > 1.1. Project/Component Working Name: > IOCTL interface for drive LED control > 1.2. Name of Document Author/Supplier: > Author: Pawel Wojcik > 1.3 Date of This Document: > 12 May, 2008 > > 2. References > [1] SFF-8485 Specification for Serial GPIO (SGPIO) Bus > [2] Sun_Microsystems_HDD_LED_Requirements_Rev_1_8 > > > 4. Technical Description > > 4.1 Overview > > Drive-related LED indicators may be controlled by various > hardware entities. Generally, Serial General Purpose > Input/Output (SGPIO) interface is used to control LED indicators > on the backplane. > > SGPIO is a 4-signal (or 4-wire) bus used between a Host Bus > Adapter (HBA) and a backplane. Out of the 4 signals, 3 are > driven by the HBA or other hardware controller and 1 is driven > by the backplane. The data bits exchanged are related to drives > supported by the target and are typically related to specific > lanes of the high-speed serial storage interface. > > SGPIO and the SGPIO spec is generally adopted and implemented > in products from most major HBA and Storage Controller vendors > such as LSI, Intel, Adaptec, Nvidia, Broadcom and PMC-Sierra. > Most products shipping with support for SAS and SATA drives > support this standard. The SGPIO specification is given the > official name SFF-8485 and is maintained by the SFF Committee. > > 4.2 Problem Description > > On most Sun platforms FAULT (Amber) and OK2RM (Blue) LEDs are > controlled by Service Processor (SP) via IPMI, while ACTIVITY > (Green) LED is controlled by the disk HBA. > > On some other platforms that use hardware RAID HBA, the > FAULT LED may be controlled by HBA along with the ACTIVITY LED. > Dorado/Tucana platform employs SATA controller embedded in nVidia > chipset, which provides SGPIO interface, that can be used to > control all type of drive-related LEDs. > > There may be other platforms, employing either nVidia chipset > or Intel's ICH hub that could use SGPIO to control drive-related > LEDs. > > Currently there are no APIs that HBA drivers implement to > provide control over the LEDs for the target disk drives. > This project proposes the new APIs for HBA drivers to > provide access to LEDs that it may control. > > 4.3 Proposal > > This fasttrack proposes a new ioctl to control FAIL and OK2RM LED > indicators via SGPIO interface of a Host Bus Adapter. > Controlling software (zfs, FMA or other entity with the adequate > permission) may use this interface to turn-on and turn-off FAULT > and OK2RM LED indicators using HBA SGPIO hardware interface. > If HBA does not support SGPIO interface, the ioctl would fail > with an appropriate error. > > The default LED state set by HBA (without an explicit command > to turn LED ON) would be OFF state. > > If the Service Processor controls FAULT LED via IPMI interface, > IPMI LED ON setting will override the HBA LED OFF setting. > > 4.3.1 Considerations > > The DKIO type ioctl was considered and rejected, because it > requires a presence of a device target node. In case of > a failed device the target node for the device may not exists. > Furthermore, there is no path in scsi target driver and scsi > module to pass ioctls that are not processed by these modules > to the HBA drivers. > > The new DEVCTL type ioctl was selected because its use does > not depend on the presence of a device target node. The new > ioctl would have to be implemented by HBA drivers providing > SGPIO interface. > > An application will have to identify (using /device path) > and open HBA control node (using /device path), identify LED > type and number for this HBA using platform topo info, and > execute ioctl call. > > In the future, the task of deriving the control node path > could be delegated to libdevice libraray, when this interface > would be used on many platforms. > > The ioctl-specific structure is used to pass LED specification > to the HBA. The alternative would be to use nvlist. > > > 4.4 Requested Binding > > patch > > > 4.5 Details > > Exported Interface Classification Comments > --------------------------------------------------------------- > DEVCTL_SETLED | consolidation-private | Turn ON/OFF drive > | | indicator LED > | | > --------------------------------------------------------------- > DEVCTL_GETLED | consolidation-private | Get status of drive > | | indicator LED > | | > --------------------------------------------------------------- > struct dc_led_ctl | consolidation-private | pointer to this > | | structure passed > | | as ioctl argument > | | > --------------------------------------------------------------- > > SYNOPSIS: > > int ioctl(int fildes, int request, arg); > > > DESCRIPTION > > The fildes argument is an open file descriptor that refers > to a HBA control node. > > The request is DEVCTL_SETLED or DEVCTL_GETLED > > #define DEVCTL_SET_LED (DEVCTL_IOC | 39) > #define DEVCTL_GET_LEDSTATE (DEVCTL_IOC | 40) > > The argument is a pointer to dc_led_ctl structure: > > struct dc_led_ctl { > uint32_t led_number : 16; /* LED/device number */ > uint32_t led_ctl_active : 1; /* Control active */ > uint32_t led_type : 3; /* LED type */ > uint32_t led_state: 4; /* LED ON/OFF/Blink state */ > } > > /* Control active field *' > #define DCL_CNTRL_OFF 0 /* Control inactive */ > #define DCL_CNTRL_ON 1 /* Control active */ > > /* LED type field */ > #define DCL_TYPE_FAIL 1 /* FAIL LED type */ > #define DCL_TYPE_OK2RM 2 /* OK2RM LED type */ > > /* Led state field */ > #define DCL_STATE_OFF 0 /* LED state OFF */ > #define DCL_STATE_ON 1 /* LED state ON */ > #define DCL_STATE_SLOW_BLNK 2 /* LED Slow blink */ > #define DCL_STATE_FAST_BLNK 3 /* LED Fast blink */ > > led_num should be a value corresponding to the device number > to which the LED indicator is related. In practical terms > it is the LED/drive data position in the SGPIO's SDataOut > bit-stream where each drive is represented by 3 bits > (3-touple). > > The value range is 0 to n-1, where n is the number of > devices supported by HBA instance. > > The LED number should be available in platform topology > info. > > Note: For nVidia's MCP55 and IO55 chips and system using > nv_sata driver, LED number is the same as the disk number > in physical device path /devices/../disk@Y. > > For DEVCTL_SET_LED request: > > Caller has to set all fields of the dc_led_ctl structure. > If DCL_CNTRL_OFF is specified, all other parameters are > ignored and HBA should make SPGIO inactive (put SGPIO > in tri-state) for all devices controlled by the HBA. > Slow blink and fast blink rates are defined by Sun > document Sun_Microsystems_HDD_LED_Requirements_Rev_1_8. > If DCL_CNTRL_ON is specified, caller has to set all fields > of the dc_led_ctl structure. > > > For DEVCTL_GET_LEDSTATE request: > > Caller has to specify led_type and led_number in dc_led_ctl > structure. HBA will set led_ctl_active and led_state fields. > The LED state is not persistent over reboots. > If the SGPIO control interface was not explicitly enabled > via DEVCTL_SET_LED with DCL_CNTRL_ON request, the LED state > is DCL_STATE_OFF for every LED controlled by the HBA instance. > > > IOCTL may specify one LED type and device number at a time. > Initial state of the SGPIO lines is tri-state (not driven > by the HBA). In such state the default LED setting as provided > by a backplane SGPIO target hardware determines the actual LED > state and such state is unknown to the HBA driver. > The default state of a LED indicator driven by HBA is OFF, > i.e. if the LED of any type is not explicitly specified > to be ON, the HBA should specify it state as OFF. > > If the system BIOS (or BIOS-extension) does not enable SGPIO > interface, an explicit ioctl has to be sent to eenable the > SGPIO interface even when only ACTIVITY LEDs are to be under > HBA control (normally not controlled via DEVCTL_SET_LED ioctl). > > Some HBAs (such as hardware RAID HBAs) may be actively control > FAIL and/or OK2RM LEDs. For these HBAs the initial state of > LEDs is HBA-specific and cntrl_active field of the dc_led_ctl > structure may be ignored by the HBA (ie. SGPIO interface cannot > be deactivated and/or tri-stated). Drivers for such HBAs > return DCL_CNTLR_ON sate in the cntrl_active field even > without an explicit request to activate the control. > > Implementation Note: > > The ioctl call should return immediately. In case of the > DEVCTL_SET_LED request the HBA driver should store the request > internally and execute it on a separate execution thread if it > cannot be executed immediately. > > The SATA HBA driver should maintain the state of LEDs for the > purpose of providing LEDs state in response to DEVCTL_GET_LEDSTATE > request. > > The LED control should not interfere with normal I/O activity > (if any) on the specified device. > > PRIV_ALL privilege is required to invoke DEVCTL_LED_CTL ioctl. > > > RETURN VALUES > > Upon successful completion the return value is 0. > Otherwise -1 is returned and errno is set to indicate an error. > > ERRORS > > The IOCTL would fail if: > > ENOSUP HBA does not support LED control > > ENXIO the specified drive LED number is not supported by > the controller and/or HBA driver. > > EINVAL the LED type or state is not supported > > ENOPERM no permision to execute this ioctl (not owner) > > > > 6. Resources and Schedule > 6.4. Steering Committee requested information > 6.4.1. Consolidation C-team Name: > ON > 6.5. ARC review type: Fasttrack > 6.6. ARC Exposure: open > > 6. Resources and Schedule > 6.4. Steering Committee requested information > 6.4.1. Consolidation C-team Name: > ON > 6.5. ARC review type: FastTrack > 6.6. ARC Exposure: closed > 6.6.1. Rationale: The Sun document describing LED behavior is marked XXXfidential and is under NDA > From sacadmin Tue May 13 09:12:13 2008 Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01.SFBay.Sun.COM [129.145.155.118]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m4DGCDXs022896 for ; Tue, 13 May 2008 09:12:13 -0700 (PDT) Received: from sca-es-mail-1.sun.com (sca-es-mail-1.Sun.COM [192.18.43.132]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m4DGCD6V033347 for ; Tue, 13 May 2008 09:12:13 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m4DGC8ts011798 for ; Tue, 13 May 2008 09:12:08 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K0T00D01E3NO500@fe-sfbay-09.sun.com> (original mail from Alan.Perry@Sun.COM) for PSARC@sac.sfbay.sun.com; Tue, 13 May 2008 09:12:08 -0700 (PDT) Received: from [10.0.0.4] ([97.113.5.85]) by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K0T005S4EC4YV50@fe-sfbay-09.sun.com>; Tue, 13 May 2008 09:12:05 -0700 (PDT) Date: Tue, 13 May 2008 09:12:04 -0700 From: Alan Perry Subject: Re: IOCTL interface for drive LED control [PSARC/2008/317 FastTrack timeout 05/20/2008] In-reply-to: <4829B7DF.7080201@sun.com> Sender: Alan.Perry@Sun.COM To: "Garrett D'Amore" Cc: Alan Perry , PSARC@sac.sfbay.sun.com, sata-dev@Sun.COM Message-id: <4829BDD4.1070103@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200805130632.m4D6WFul004136@groomlake.sfbay.sun.com> <4829B7DF.7080201@sun.com> User-Agent: Thunderbird 2.0.0.9 (X11/20080213) Status: RO Content-Length: 13900 Just a heads up. The submitter of this case (Pawel) is in China at this time, so might not respond immediately. alan Garrett D'Amore wrote: > This looks pretty good, but I do wonder if there should be #define's > for different LED numbers, to identify which is which? > > Furthermore, in my opinion, I think it would be useful to allow > different LED numbers (and perhaps "types of LEDs" to be used by this > ioctl interface, as I think it is likely that this control interface > has utility beyond just disk drives. (A common LED API is potentially > useful to other kinds of systems/devices.) > > In order to be "common", I think a way to inquire what kinds of LEDs > are present/supported would be useful. > > If the project team doesn't want to worry about other kinds of uses > outside of disk drives for now, that's OK, too. It would still be > nice to "plan" for future extensions though -- perhaps by having a > simple inquiry interface: > > DEVCTL_NUM_LEDs > > which simply returns the total number of LEDs supported by the devctl > node, and then by extending DEVCTL_GET_LED a bit, perhaps by adding > something like: > > uint32_t led_capab; > #define DEVCTL_CAP_SLOW_BLINK (1<< 0) > #define DEVCTL_CAP_FAST_BLINK (1<<1) > > and then changing the led_type #defines just a little bit: > > #define DC_TYPE_DEVICE_FAIL 1 > #define DC_TYPE_DEVICE_OK2RM 2 > #define DC_TYPE_BUS_FAIL 3 > > (this just makes it clear that these LEDs are related to disk drives, > and not to the HBA itself. One can imagine an LED with a HBA failure > indicator (DC_TYPE_BUS_FAIL) separate from a drive > (DC_TYPE_DEVICE_FAIL) failure. > > Nit: I think you need to correct your #define's for the ioctl > commands. I *think* you probably meant to define DEVCTL_SET_LED and > DEVCTL_GET_LED, rather than DEVCTL_GET_LEDSTATE. :-) > > Privilege: does PRIV_ALL really make the most sense here? I'd think > that typically whatever privilege is required to open the device node > for devctl (either in r/w or r/o mode) should be sufficient. It > doesn't seem that there is much utility in having a different > privilege requirement for these operations than what would be required > to perform other cfgadm related operations. > > -- Garrett > > Alan Perry wrote: >> I am sponsoring this fast-track for Pawel. It times out on 20 May. >> >> Template Version: @(#)sac_nextcase 1.66 04/17/08 SMI >> This information is Copyright 2008 Sun Microsystems, Inc. >> 1. Introduction >> 1.1. Project/Component Working Name: >> IOCTL interface for drive LED control >> 1.2. Name of Document Author/Supplier: >> Author: Pawel Wojcik >> 1.3 Date of This Document: >> 12 May, 2008 >> >> 2. References >> [1] SFF-8485 Specification for Serial GPIO (SGPIO) Bus >> [2] Sun_Microsystems_HDD_LED_Requirements_Rev_1_8 >> >> 4. Technical Description >> >> 4.1 Overview >> Drive-related LED indicators may be controlled by various >> hardware entities. Generally, Serial General Purpose >> Input/Output (SGPIO) interface is used to control LED indicators >> on the backplane. >> >> SGPIO is a 4-signal (or 4-wire) bus used between a Host Bus >> Adapter (HBA) and a backplane. Out of the 4 signals, 3 are >> driven by the HBA or other hardware controller and 1 is driven >> by the backplane. The data bits exchanged are related to drives >> supported by the target and are typically related to specific >> lanes of the high-speed serial storage interface. >> >> SGPIO and the SGPIO spec is generally adopted and implemented >> in products from most major HBA and Storage Controller vendors >> such as LSI, Intel, Adaptec, Nvidia, Broadcom and PMC-Sierra. >> Most products shipping with support for SAS and SATA drives >> support this standard. The SGPIO specification is given the >> official name SFF-8485 and is maintained by the SFF Committee. >> >> 4.2 Problem Description >> >> On most Sun platforms FAULT (Amber) and OK2RM (Blue) LEDs are >> controlled by Service Processor (SP) via IPMI, while ACTIVITY >> (Green) LED is controlled by the disk HBA. >> >> On some other platforms that use hardware RAID HBA, the >> FAULT LED may be controlled by HBA along with the ACTIVITY LED. >> Dorado/Tucana platform employs SATA controller embedded in nVidia >> chipset, which provides SGPIO interface, that can be used to >> control all type of drive-related LEDs. >> >> There may be other platforms, employing either nVidia chipset >> or Intel's ICH hub that could use SGPIO to control drive-related >> LEDs. >> >> Currently there are no APIs that HBA drivers implement to >> provide control over the LEDs for the target disk drives. >> This project proposes the new APIs for HBA drivers to >> provide access to LEDs that it may control. >> >> 4.3 Proposal >> >> This fasttrack proposes a new ioctl to control FAIL and OK2RM LED >> indicators via SGPIO interface of a Host Bus Adapter. >> Controlling software (zfs, FMA or other entity with the adequate >> permission) may use this interface to turn-on and turn-off FAULT >> and OK2RM LED indicators using HBA SGPIO hardware interface. >> If HBA does not support SGPIO interface, the ioctl would fail >> with an appropriate error. >> >> The default LED state set by HBA (without an explicit command >> to turn LED ON) would be OFF state. >> >> If the Service Processor controls FAULT LED via IPMI interface, >> IPMI LED ON setting will override the HBA LED OFF setting. >> >> 4.3.1 Considerations >> >> The DKIO type ioctl was considered and rejected, because it >> requires a presence of a device target node. In case of a >> failed device the target node for the device may not exists. >> Furthermore, there is no path in scsi target driver and scsi >> module to pass ioctls that are not processed by these modules >> to the HBA drivers. >> >> The new DEVCTL type ioctl was selected because its use does >> not depend on the presence of a device target node. The new >> ioctl would have to be implemented by HBA drivers providing >> SGPIO interface. >> >> An application will have to identify (using /device path) >> and open HBA control node (using /device path), identify LED >> type and number for this HBA using platform topo info, and >> execute ioctl call. >> >> In the future, the task of deriving the control node path >> could be delegated to libdevice libraray, when this interface >> would be used on many platforms. >> >> The ioctl-specific structure is used to pass LED specification >> to the HBA. The alternative would be to use nvlist. >> >> 4.4 Requested Binding >> >> patch >> >> >> 4.5 Details >> >> Exported Interface Classification Comments >> --------------------------------------------------------------- >> DEVCTL_SETLED | consolidation-private | Turn ON/OFF drive >> | | indicator LED >> | | >> --------------------------------------------------------------- >> DEVCTL_GETLED | consolidation-private | Get status of drive >> | | indicator LED >> | | >> --------------------------------------------------------------- >> struct dc_led_ctl | consolidation-private | pointer to this >> | | structure passed >> | | as ioctl argument >> | | >> --------------------------------------------------------------- >> >> SYNOPSIS: >> >> int ioctl(int fildes, int request, arg); >> >> >> DESCRIPTION >> >> The fildes argument is an open file descriptor that refers >> to a HBA control node. >> >> The request is DEVCTL_SETLED or DEVCTL_GETLED >> >> #define DEVCTL_SET_LED (DEVCTL_IOC | 39) >> #define DEVCTL_GET_LEDSTATE (DEVCTL_IOC | 40) >> >> The argument is a pointer to dc_led_ctl structure: >> >> struct dc_led_ctl { >> uint32_t led_number : 16; /* LED/device number */ >> uint32_t led_ctl_active : 1; /* Control active */ >> uint32_t led_type : 3; /* LED type */ >> uint32_t led_state: 4; /* LED ON/OFF/Blink state */ >> } >> >> /* Control active field *' >> #define DCL_CNTRL_OFF 0 /* Control inactive */ >> #define DCL_CNTRL_ON 1 /* Control active */ >> >> /* LED type field */ >> #define DCL_TYPE_FAIL 1 /* FAIL LED type */ >> #define DCL_TYPE_OK2RM 2 /* OK2RM LED type */ >> >> /* Led state field */ >> #define DCL_STATE_OFF 0 /* LED state OFF */ >> #define DCL_STATE_ON 1 /* LED state ON */ >> #define DCL_STATE_SLOW_BLNK 2 /* LED Slow blink */ >> #define DCL_STATE_FAST_BLNK 3 /* LED Fast blink */ >> >> led_num should be a value corresponding to the device number >> to which the LED indicator is related. In practical terms >> it is the LED/drive data position in the SGPIO's SDataOut >> bit-stream where each drive is represented by 3 bits >> (3-touple). >> >> The value range is 0 to n-1, where n is the number of >> devices supported by HBA instance. >> >> The LED number should be available in platform topology >> info. >> >> Note: For nVidia's MCP55 and IO55 chips and system using >> nv_sata driver, LED number is the same as the disk number >> in physical device path /devices/../disk@Y. >> >> For DEVCTL_SET_LED request: >> >> Caller has to set all fields of the dc_led_ctl structure. >> If DCL_CNTRL_OFF is specified, all other parameters are >> ignored and HBA should make SPGIO inactive (put SGPIO >> in tri-state) for all devices controlled by the HBA. >> Slow blink and fast blink rates are defined by Sun >> document Sun_Microsystems_HDD_LED_Requirements_Rev_1_8. >> If DCL_CNTRL_ON is specified, caller has to set all fields >> of the dc_led_ctl structure. >> >> >> For DEVCTL_GET_LEDSTATE request: >> >> Caller has to specify led_type and led_number in dc_led_ctl >> structure. HBA will set led_ctl_active and led_state fields. >> The LED state is not persistent over reboots. >> If the SGPIO control interface was not explicitly enabled >> via DEVCTL_SET_LED with DCL_CNTRL_ON request, the LED state >> is DCL_STATE_OFF for every LED controlled by the HBA instance. >> >> IOCTL may specify one LED type and device number at a time. >> Initial state of the SGPIO lines is tri-state (not driven >> by the HBA). In such state the default LED setting as provided >> by a backplane SGPIO target hardware determines the actual LED >> state and such state is unknown to the HBA driver. >> The default state of a LED indicator driven by HBA is OFF, >> i.e. if the LED of any type is not explicitly specified to be ON, >> the HBA should specify it state as OFF. >> >> If the system BIOS (or BIOS-extension) does not enable SGPIO >> interface, an explicit ioctl has to be sent to eenable the >> SGPIO interface even when only ACTIVITY LEDs are to be under >> HBA control (normally not controlled via DEVCTL_SET_LED ioctl). >> >> Some HBAs (such as hardware RAID HBAs) may be actively control >> FAIL and/or OK2RM LEDs. For these HBAs the initial state of >> LEDs is HBA-specific and cntrl_active field of the dc_led_ctl >> structure may be ignored by the HBA (ie. SGPIO interface cannot >> be deactivated and/or tri-stated). Drivers for such HBAs >> return DCL_CNTLR_ON sate in the cntrl_active field even >> without an explicit request to activate the control. >> >> Implementation Note: >> >> The ioctl call should return immediately. In case of the >> DEVCTL_SET_LED request the HBA driver should store the request >> internally and execute it on a separate execution thread if it >> cannot be executed immediately. >> >> The SATA HBA driver should maintain the state of LEDs for the >> purpose of providing LEDs state in response to DEVCTL_GET_LEDSTATE >> request. >> The LED control should not interfere with normal I/O activity >> (if any) on the specified device. >> >> PRIV_ALL privilege is required to invoke DEVCTL_LED_CTL ioctl. >> >> >> RETURN VALUES >> >> Upon successful completion the return value is 0. >> Otherwise -1 is returned and errno is set to indicate an error. >> >> ERRORS >> >> The IOCTL would fail if: >> >> ENOSUP HBA does not support LED control >> >> ENXIO the specified drive LED number is not supported by >> the controller and/or HBA driver. >> >> EINVAL the LED type or state is not supported >> >> ENOPERM no permision to execute this ioctl (not owner) >> >> >> >> 6. Resources and Schedule >> 6.4. Steering Committee requested information >> 6.4.1. Consolidation C-team Name: >> ON >> 6.5. ARC review type: Fasttrack >> 6.6. ARC Exposure: open >> >> 6. Resources and Schedule >> 6.4. Steering Committee requested information >> 6.4.1. Consolidation C-team Name: >> ON >> 6.5. ARC review type: FastTrack >> 6.6. ARC Exposure: closed >> 6.6.1. Rationale: The Sun document describing LED behavior is >> marked XXXfidential and is under NDA >> > From sacadmin Tue May 13 09:23:55 2008 Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01.SFBay.Sun.COM [129.145.155.118]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m4DGNtfW023759 for ; Tue, 13 May 2008 09:23:55 -0700 (PDT) Received: from sca-es-mail-2.sun.com (sca-es-mail-2.Sun.COM [192.18.43.133]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m4DGNtro040517 for ; Tue, 13 May 2008 09:23:55 -0700 (PDT) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m4DGNnkG028835 for ; Tue, 13 May 2008 09:23:49 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K0T00M01DW4AH00@fe-sfbay-10.sun.com> (original mail from Pawel.Wojcik@Sun.COM) for PSARC@sac.sfbay.sun.com; Tue, 13 May 2008 09:23:49 -0700 (PDT) Received: from [10.128.0.91] ([218.30.180.194]) by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K0T00BD5EV9RMG0@fe-sfbay-10.sun.com>; Tue, 13 May 2008 09:23:40 -0700 (PDT) Date: Tue, 13 May 2008 09:49:57 -0700 From: Pawel Wojcik Subject: Re: IOCTL interface for drive LED control [PSARC/2008/317 FastTrack timeout 05/20/2008] In-reply-to: <482991D6.7020004@Sun.COM> Sender: Pawel.Wojcik@Sun.COM To: Darren J Moffat Cc: Alan Perry , PSARC@sac.sfbay.sun.com, sata-dev@Sun.COM Message-id: <4829C6B5.3060801@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200805130632.m4D6WFul004136@groomlake.sfbay.sun.com> <482991D6.7020004@Sun.COM> User-Agent: Thunderbird 2.0.0.14 (Windows/20080421) Status: RO Content-Length: 2332 Darren J Moffat wrote: > Alan Perry wrote: >> PRIV_ALL privilege is required to invoke DEVCTL_LED_CTL ioctl. > > Is it really that much of a security risk ? > > Requiring this much privilege to invoke this ioctl means that system > daemons/commands that could otherwise run with very little (or may no) > privilege have to run with all privilege instead. > > If an existing privilege is to be used then I think "sys_devices" is > the closest match though I'm starting to think we need a privilege > like sys_ip_config for "disk". Perhaps it does not require PRIV_ALL privilege. But one has to consider implications of setting FAIL or OK2RM LED that may results in an unintended device removal from the system. We may consider the privileges that are required to control LED using IPMI through the System Processor, which is the other entity currently controlling LEDs. > > Is there a similar interface in Linux or BSD systems ? If so how does > this new Solaris interface compare, or do they not have the ability to > set these LED ? I do not know what Linux or BSD do in respect to LED control via SGPIO. The interface is to be complimentary to the Sun SP IPMI interface and for entities like FMA, ZFS and Disk Monitor. > > Given that this case is closed (because of the closed reference > document I believe) where is the source going to end up ? This is interesting question. The code is definitely in the open drivers - ahci and nv_sata. There is no reason to close these drivers because of "XXXfidential" status of the Sun spec document. It is not clear how to solve this dilemma. Shall we ask to remove XXXfidential classification from the doc? Or is it possible that the implementation could be open despite the closed nature of the document? The only part referenced in the PSARC case is (and potentially implemented) is the LED blinking pattern. In fact, it does not need to be implement it because it these blinking patters are not used on existing platforms - it was included in the interface just for the completeness of the interface. > > If the intent is that HBA drivers are implementing this ioctl why is > it Consolidation Private to ON ? > The interface is not stable and is considered mostly for Sun-manufactured platforms. There is no external requests for this interface. -Pawel From sacadmin Tue May 13 09:57:42 2008 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m4DGvg7Z025870 for ; Tue, 13 May 2008 09:57:42 -0700 (PDT) Received: from sca-es-mail-1.sun.com (sca-es-mail-1.Sun.COM [192.18.43.132]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m4DGvg68040466 for ; Tue, 13 May 2008 09:57:42 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m4DGvabe018360 for ; Tue, 13 May 2008 09:57:36 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K0T00I01GB9OY00@fe-sfbay-09.sun.com> (original mail from Pawel.Wojcik@Sun.COM) for PSARC@sac.sfbay.sun.com; Tue, 13 May 2008 09:57:36 -0700 (PDT) Received: from [10.128.0.91] ([218.30.180.194]) by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K0T0074RGFPNSD0@fe-sfbay-09.sun.com>; Tue, 13 May 2008 09:57:29 -0700 (PDT) Date: Tue, 13 May 2008 10:23:47 -0700 From: Pawel Wojcik Subject: Re: IOCTL interface for drive LED control [PSARC/2008/317 FastTrack timeout 05/20/2008] In-reply-to: <4829B7DF.7080201@sun.com> Sender: Pawel.Wojcik@Sun.COM To: "Garrett D'Amore" Cc: Alan Perry , PSARC@sac.sfbay.sun.com, sata-dev@Sun.COM Message-id: <4829CEA3.7000906@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200805130632.m4D6WFul004136@groomlake.sfbay.sun.com> <4829B7DF.7080201@sun.com> User-Agent: Thunderbird 2.0.0.14 (Windows/20080421) Status: RO Content-Length: 15571 Garrett D'Amore wrote: > This looks pretty good, but I do wonder if there should be #define's > for different LED numbers, to identify which is which? This interface was considered as complimentary to IPMI LED control for cases where SP may not be able to control some of the specified LEDs. Also, it is limited only to SGPIO used according SFF8485 spec, clearly limited to disk-related LEDs only. The data passed on SGPIO always specifies 3 LED tuple, as when we are referring to LED number, it is the LED tuple that we are referring to. Defining the actual LED number does not make much sense in this context, because they are limited in range by the number of devices supported by HBA, directly related to the ports (1:1) supported by the hba, and the actual routing to particular bay is isolated from the specification passed via SGPIO by the LED controller chip on the backplane. > > Furthermore, in my opinion, I think it would be useful to allow > different LED numbers (and perhaps "types of LEDs" to be used by this > ioctl interface, as I think it is likely that this control interface > has utility beyond just disk drives. (A common LED API is potentially > useful to other kinds of systems/devices.) Please see comment above. The scope of this interface is rather limited. > > In order to be "common", I think a way to inquire what kinds of LEDs > are present/supported would be useful. > > If the project team doesn't want to worry about other kinds of uses > outside of disk drives for now, that's OK, too. It would still be > nice to "plan" for future extensions though -- perhaps by having a > simple inquiry interface: > > DEVCTL_NUM_LEDs This may be a misleading number, as not all possibly controlled LEDs may be exposed/used by the platform. The SFF8485 spec is rather flexible in this respect, but the hardware implementation in the considered controllers are quite rigid and limited. Platform topology file should be generally used to determine LED number. > > which simply returns the total number of LEDs supported by the devctl > node, and then by extending DEVCTL_GET_LED a bit, perhaps by adding > something like: > > uint32_t led_capab; > #define DEVCTL_CAP_SLOW_BLINK (1<< 0) > #define DEVCTL_CAP_FAST_BLINK (1<<1) > > and then changing the led_type #defines just a little bit: > > #define DC_TYPE_DEVICE_FAIL 1 > #define DC_TYPE_DEVICE_OK2RM 2 > #define DC_TYPE_BUS_FAIL 3 > > (this just makes it clear that these LEDs are related to disk drives, > and not to the HBA itself. One can imagine an LED with a HBA failure > indicator (DC_TYPE_BUS_FAIL) separate from a drive > (DC_TYPE_DEVICE_FAIL) failure. SFF8584 does not provide such option (i.e. controlling LED not related to disk). > > Nit: I think you need to correct your #define's for the ioctl > commands. I *think* you probably meant to define DEVCTL_SET_LED and > DEVCTL_GET_LED, rather than DEVCTL_GET_LEDSTATE. :-) Yes, thank you. For both IOCTLs... > Privilege: does PRIV_ALL really make the most sense here? I'd think > that typically whatever privilege is required to open the device node > for devctl (either in r/w or r/o mode) should be sufficient. It > doesn't seem that there is much utility in having a different > privilege requirement for these operations than what would be required > to perform other cfgadm related operations. There is no plan to "connect" LED control functionality with cfgadm functionality. Perhaps it does not require PRIV_ALL privilege. But one has to consider implications of setting FAIL or OK2RM LED that may result in an unintended device removal from the system. We may consider the privileges that are required to control LED using IPMI through the System Processor, which is the other entity currently controlling LEDs. I will check what priviledges are required for LED control via IPMI and specify the same for this interface. > -- Garrett > > Alan Perry wrote: >> I am sponsoring this fast-track for Pawel. It times out on 20 May. >> >> Template Version: @(#)sac_nextcase 1.66 04/17/08 SMI >> This information is Copyright 2008 Sun Microsystems, Inc. >> 1. Introduction >> 1.1. Project/Component Working Name: >> IOCTL interface for drive LED control >> 1.2. Name of Document Author/Supplier: >> Author: Pawel Wojcik >> 1.3 Date of This Document: >> 12 May, 2008 >> >> 2. References >> [1] SFF-8485 Specification for Serial GPIO (SGPIO) Bus >> [2] Sun_Microsystems_HDD_LED_Requirements_Rev_1_8 >> >> 4. Technical Description >> >> 4.1 Overview >> Drive-related LED indicators may be controlled by various >> hardware entities. Generally, Serial General Purpose >> Input/Output (SGPIO) interface is used to control LED indicators >> on the backplane. >> >> SGPIO is a 4-signal (or 4-wire) bus used between a Host Bus >> Adapter (HBA) and a backplane. Out of the 4 signals, 3 are >> driven by the HBA or other hardware controller and 1 is driven >> by the backplane. The data bits exchanged are related to drives >> supported by the target and are typically related to specific >> lanes of the high-speed serial storage interface. >> >> SGPIO and the SGPIO spec is generally adopted and implemented >> in products from most major HBA and Storage Controller vendors >> such as LSI, Intel, Adaptec, Nvidia, Broadcom and PMC-Sierra. >> Most products shipping with support for SAS and SATA drives >> support this standard. The SGPIO specification is given the >> official name SFF-8485 and is maintained by the SFF Committee. >> >> 4.2 Problem Description >> >> On most Sun platforms FAULT (Amber) and OK2RM (Blue) LEDs are >> controlled by Service Processor (SP) via IPMI, while ACTIVITY >> (Green) LED is controlled by the disk HBA. >> >> On some other platforms that use hardware RAID HBA, the >> FAULT LED may be controlled by HBA along with the ACTIVITY LED. >> Dorado/Tucana platform employs SATA controller embedded in nVidia >> chipset, which provides SGPIO interface, that can be used to >> control all type of drive-related LEDs. >> >> There may be other platforms, employing either nVidia chipset >> or Intel's ICH hub that could use SGPIO to control drive-related >> LEDs. >> >> Currently there are no APIs that HBA drivers implement to >> provide control over the LEDs for the target disk drives. >> This project proposes the new APIs for HBA drivers to >> provide access to LEDs that it may control. >> >> 4.3 Proposal >> >> This fasttrack proposes a new ioctl to control FAIL and OK2RM LED >> indicators via SGPIO interface of a Host Bus Adapter. >> Controlling software (zfs, FMA or other entity with the adequate >> permission) may use this interface to turn-on and turn-off FAULT >> and OK2RM LED indicators using HBA SGPIO hardware interface. >> If HBA does not support SGPIO interface, the ioctl would fail >> with an appropriate error. >> >> The default LED state set by HBA (without an explicit command >> to turn LED ON) would be OFF state. >> >> If the Service Processor controls FAULT LED via IPMI interface, >> IPMI LED ON setting will override the HBA LED OFF setting. >> >> 4.3.1 Considerations >> >> The DKIO type ioctl was considered and rejected, because it >> requires a presence of a device target node. In case of a >> failed device the target node for the device may not exists. >> Furthermore, there is no path in scsi target driver and scsi >> module to pass ioctls that are not processed by these modules >> to the HBA drivers. >> >> The new DEVCTL type ioctl was selected because its use does >> not depend on the presence of a device target node. The new >> ioctl would have to be implemented by HBA drivers providing >> SGPIO interface. >> >> An application will have to identify (using /device path) >> and open HBA control node (using /device path), identify LED >> type and number for this HBA using platform topo info, and >> execute ioctl call. >> >> In the future, the task of deriving the control node path >> could be delegated to libdevice libraray, when this interface >> would be used on many platforms. >> >> The ioctl-specific structure is used to pass LED specification >> to the HBA. The alternative would be to use nvlist. >> >> 4.4 Requested Binding >> >> patch >> >> >> 4.5 Details >> >> Exported Interface Classification Comments >> --------------------------------------------------------------- >> DEVCTL_SETLED | consolidation-private | Turn ON/OFF drive >> | | indicator LED >> | | >> --------------------------------------------------------------- >> DEVCTL_GETLED | consolidation-private | Get status of drive >> | | indicator LED >> | | >> --------------------------------------------------------------- >> struct dc_led_ctl | consolidation-private | pointer to this >> | | structure passed >> | | as ioctl argument >> | | >> --------------------------------------------------------------- >> >> SYNOPSIS: >> >> int ioctl(int fildes, int request, arg); >> >> >> DESCRIPTION >> >> The fildes argument is an open file descriptor that refers >> to a HBA control node. >> >> The request is DEVCTL_SETLED or DEVCTL_GETLED >> >> #define DEVCTL_SET_LED (DEVCTL_IOC | 39) >> #define DEVCTL_GET_LEDSTATE (DEVCTL_IOC | 40) >> >> The argument is a pointer to dc_led_ctl structure: >> >> struct dc_led_ctl { >> uint32_t led_number : 16; /* LED/device number */ >> uint32_t led_ctl_active : 1; /* Control active */ >> uint32_t led_type : 3; /* LED type */ >> uint32_t led_state: 4; /* LED ON/OFF/Blink state */ >> } >> >> /* Control active field *' >> #define DCL_CNTRL_OFF 0 /* Control inactive */ >> #define DCL_CNTRL_ON 1 /* Control active */ >> >> /* LED type field */ >> #define DCL_TYPE_FAIL 1 /* FAIL LED type */ >> #define DCL_TYPE_OK2RM 2 /* OK2RM LED type */ >> >> /* Led state field */ >> #define DCL_STATE_OFF 0 /* LED state OFF */ >> #define DCL_STATE_ON 1 /* LED state ON */ >> #define DCL_STATE_SLOW_BLNK 2 /* LED Slow blink */ >> #define DCL_STATE_FAST_BLNK 3 /* LED Fast blink */ >> >> led_num should be a value corresponding to the device number >> to which the LED indicator is related. In practical terms >> it is the LED/drive data position in the SGPIO's SDataOut >> bit-stream where each drive is represented by 3 bits >> (3-touple). >> >> The value range is 0 to n-1, where n is the number of >> devices supported by HBA instance. >> >> The LED number should be available in platform topology >> info. >> >> Note: For nVidia's MCP55 and IO55 chips and system using >> nv_sata driver, LED number is the same as the disk number >> in physical device path /devices/../disk@Y. >> >> For DEVCTL_SET_LED request: >> >> Caller has to set all fields of the dc_led_ctl structure. >> If DCL_CNTRL_OFF is specified, all other parameters are >> ignored and HBA should make SPGIO inactive (put SGPIO >> in tri-state) for all devices controlled by the HBA. >> Slow blink and fast blink rates are defined by Sun >> document Sun_Microsystems_HDD_LED_Requirements_Rev_1_8. >> If DCL_CNTRL_ON is specified, caller has to set all fields >> of the dc_led_ctl structure. >> >> >> For DEVCTL_GET_LEDSTATE request: >> >> Caller has to specify led_type and led_number in dc_led_ctl >> structure. HBA will set led_ctl_active and led_state fields. >> The LED state is not persistent over reboots. >> If the SGPIO control interface was not explicitly enabled >> via DEVCTL_SET_LED with DCL_CNTRL_ON request, the LED state >> is DCL_STATE_OFF for every LED controlled by the HBA instance. >> >> IOCTL may specify one LED type and device number at a time. >> Initial state of the SGPIO lines is tri-state (not driven >> by the HBA). In such state the default LED setting as provided >> by a backplane SGPIO target hardware determines the actual LED >> state and such state is unknown to the HBA driver. >> The default state of a LED indicator driven by HBA is OFF, >> i.e. if the LED of any type is not explicitly specified to be ON, >> the HBA should specify it state as OFF. >> >> If the system BIOS (or BIOS-extension) does not enable SGPIO >> interface, an explicit ioctl has to be sent to eenable the >> SGPIO interface even when only ACTIVITY LEDs are to be under >> HBA control (normally not controlled via DEVCTL_SET_LED ioctl). >> >> Some HBAs (such as hardware RAID HBAs) may be actively control >> FAIL and/or OK2RM LEDs. For these HBAs the initial state of >> LEDs is HBA-specific and cntrl_active field of the dc_led_ctl >> structure may be ignored by the HBA (ie. SGPIO interface cannot >> be deactivated and/or tri-stated). Drivers for such HBAs >> return DCL_CNTLR_ON sate in the cntrl_active field even >> without an explicit request to activate the control. >> >> Implementation Note: >> >> The ioctl call should return immediately. In case of the >> DEVCTL_SET_LED request the HBA driver should store the request >> internally and execute it on a separate execution thread if it >> cannot be executed immediately. >> >> The SATA HBA driver should maintain the state of LEDs for the >> purpose of providing LEDs state in response to DEVCTL_GET_LEDSTATE >> request. >> The LED control should not interfere with normal I/O activity >> (if any) on the specified device. >> >> PRIV_ALL privilege is required to invoke DEVCTL_LED_CTL ioctl. >> >> >> RETURN VALUES >> >> Upon successful completion the return value is 0. >> Otherwise -1 is returned and errno is set to indicate an error. >> >> ERRORS >> >> The IOCTL would fail if: >> >> ENOSUP HBA does not support LED control >> >> ENXIO the specified drive LED number is not supported by >> the controller and/or HBA driver. >> >> EINVAL the LED type or state is not supported >> >> ENOPERM no permision to execute this ioctl (not owner) >> >> >> >> 6. Resources and Schedule >> 6.4. Steering Committee requested information >> 6.4.1. Consolidation C-team Name: >> ON >> 6.5. ARC review type: Fasttrack >> 6.6. ARC Exposure: open >> >> 6. Resources and Schedule >> 6.4. Steering Committee requested information >> 6.4.1. Consolidation C-team Name: >> ON >> 6.5. ARC review type: FastTrack >> 6.6. ARC Exposure: closed >> 6.6.1. Rationale: The Sun document describing LED behavior is >> marked XXXfidential and is under NDA >> > From sacadmin Tue May 13 10:30:54 2008 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m4DHUsX6028499 for ; Tue, 13 May 2008 10:30:54 -0700 (PDT) Received: from sca-es-mail-2.sun.com (sca-es-mail-2.Sun.COM [192.18.43.133]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m4DHUsUQ065057 for ; Tue, 13 May 2008 10:30:54 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m4DHUm9b008673 for ; Tue, 13 May 2008 10:30:48 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K0T00601HQO7400@fe-sfbay-09.sun.com> (original mail from gdamore@sun.com) for PSARC@sac.sfbay.sun.com; Tue, 13 May 2008 10:30:48 -0700 (PDT) Received: from [192.168.251.106] ([76.174.83.55]) by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K0T00DVXHYSC4F0@fe-sfbay-09.sun.com>; Tue, 13 May 2008 10:30:28 -0700 (PDT) Date: Tue, 13 May 2008 10:24:22 -0700 From: "Garrett D'Amore" Subject: Re: IOCTL interface for drive LED control [PSARC/2008/317 FastTrack timeout 05/20/2008] In-reply-to: <4829CEA3.7000906@sun.com> Sender: Garrett.Damore@sun.com To: Pawel Wojcik Cc: Alan Perry , PSARC@sac.sfbay.sun.com, sata-dev@sun.com Message-id: <4829CEC6.9000708@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200805130632.m4D6WFul004136@groomlake.sfbay.sun.com> <4829B7DF.7080201@sun.com> <4829CEA3.7000906@sun.com> User-Agent: Thunderbird 2.0.0.6 (X11/20071023) Status: RO Content-Length: 17072 So my initial suggestions about the interface being able to support other kinds of devices were oriented towards making the ioctl numbers and structures flexible enough to support other LED-type uses. I think the changes I proposed are simple enough (one could argue "trivial" - one additional ioctl that can in your HBA driver always return a fixed number, and an additional structure member that can also be implemented as a constant in HBA drivers) that they are worth doing in order to support possible future uses. If nobody ever makes use of them, that's fine -- it didn't hurt much to add the unused member/ioctl, but if we can make life a lot easier for someone in the future by making very small changes to support future expansion now, why wouldn't we? (Again, I'm talking about the *interface* used here -- the ioctls. I am not suggesting that anyone needs to actually implement support for anything other than SFF8485 SGPIO. I just would prefer an interface that can adapt easily beyond such limited use to one that is strictly limited to SFF8485 SGPIO.) - Garrett Pawel Wojcik wrote: > Garrett D'Amore wrote: >> This looks pretty good, but I do wonder if there should be #define's >> for different LED numbers, to identify which is which? > This interface was considered as complimentary to IPMI LED control for > cases where SP may not be able to control some of the specified LEDs. > Also, it is limited only to SGPIO used according SFF8485 spec, clearly > limited to disk-related LEDs only. > The data passed on SGPIO always specifies 3 LED tuple, as when we are > referring to LED number, it is the LED tuple that we are referring to. > Defining the actual LED number does not make much sense in this > context, because they are limited in range by the number of devices > supported by HBA, directly related to the ports (1:1) supported by the > hba, and the actual routing to particular bay is isolated from the > specification passed via SGPIO by the LED controller chip on the > backplane. >> >> Furthermore, in my opinion, I think it would be useful to allow >> different LED numbers (and perhaps "types of LEDs" to be used by this >> ioctl interface, as I think it is likely that this control interface >> has utility beyond just disk drives. (A common LED API is >> potentially useful to other kinds of systems/devices.) > Please see comment above. The scope of this interface is rather limited. >> >> In order to be "common", I think a way to inquire what kinds of LEDs >> are present/supported would be useful. >> >> If the project team doesn't want to worry about other kinds of uses >> outside of disk drives for now, that's OK, too. It would still be >> nice to "plan" for future extensions though -- perhaps by having a >> simple inquiry interface: >> >> DEVCTL_NUM_LEDs > This may be a misleading number, as not all possibly controlled LEDs > may be exposed/used by the platform. The SFF8485 spec is rather > flexible in this respect, but the hardware implementation in the > considered controllers are quite rigid and limited. Platform topology > file should be generally used to determine LED number. >> >> which simply returns the total number of LEDs supported by the devctl >> node, and then by extending DEVCTL_GET_LED a bit, perhaps by adding >> something like: >> >> uint32_t led_capab; >> #define DEVCTL_CAP_SLOW_BLINK (1<< 0) >> #define DEVCTL_CAP_FAST_BLINK (1<<1) >> >> and then changing the led_type #defines just a little bit: >> >> #define DC_TYPE_DEVICE_FAIL 1 >> #define DC_TYPE_DEVICE_OK2RM 2 >> #define DC_TYPE_BUS_FAIL 3 >> >> (this just makes it clear that these LEDs are related to disk drives, >> and not to the HBA itself. One can imagine an LED with a HBA failure >> indicator (DC_TYPE_BUS_FAIL) separate from a drive >> (DC_TYPE_DEVICE_FAIL) failure. > SFF8584 does not provide such option (i.e. controlling LED not related > to disk). >> >> Nit: I think you need to correct your #define's for the ioctl >> commands. I *think* you probably meant to define DEVCTL_SET_LED and >> DEVCTL_GET_LED, rather than DEVCTL_GET_LEDSTATE. :-) > Yes, thank you. For both IOCTLs... >> Privilege: does PRIV_ALL really make the most sense here? I'd think >> that typically whatever privilege is required to open the device node >> for devctl (either in r/w or r/o mode) should be sufficient. It >> doesn't seem that there is much utility in having a different >> privilege requirement for these operations than what would be >> required to perform other cfgadm related operations. > There is no plan to "connect" LED control functionality with cfgadm > functionality. > Perhaps it does not require PRIV_ALL privilege. But one has to > consider implications of setting FAIL or OK2RM LED that may result in > an unintended device removal from the system. We may consider the > privileges that are required to control LED using IPMI through the > System Processor, which is the other entity currently controlling LEDs. > I will check what priviledges are required for LED control via IPMI > and specify the same for this interface. >> -- Garrett >> >> Alan Perry wrote: >>> I am sponsoring this fast-track for Pawel. It times out on 20 May. >>> >>> Template Version: @(#)sac_nextcase 1.66 04/17/08 SMI >>> This information is Copyright 2008 Sun Microsystems, Inc. >>> 1. Introduction >>> 1.1. Project/Component Working Name: >>> IOCTL interface for drive LED control >>> 1.2. Name of Document Author/Supplier: >>> Author: Pawel Wojcik >>> 1.3 Date of This Document: >>> 12 May, 2008 >>> >>> 2. References >>> [1] SFF-8485 Specification for Serial GPIO (SGPIO) Bus >>> [2] Sun_Microsystems_HDD_LED_Requirements_Rev_1_8 >>> >>> 4. Technical Description >>> >>> 4.1 Overview >>> Drive-related LED indicators may be controlled by various >>> hardware entities. Generally, Serial General Purpose >>> Input/Output (SGPIO) interface is used to control LED indicators >>> on the backplane. >>> >>> SGPIO is a 4-signal (or 4-wire) bus used between a Host Bus >>> Adapter (HBA) and a backplane. Out of the 4 signals, 3 are >>> driven by the HBA or other hardware controller and 1 is driven >>> by the backplane. The data bits exchanged are related to drives >>> supported by the target and are typically related to specific >>> lanes of the high-speed serial storage interface. >>> >>> SGPIO and the SGPIO spec is generally adopted and implemented >>> in products from most major HBA and Storage Controller vendors >>> such as LSI, Intel, Adaptec, Nvidia, Broadcom and PMC-Sierra. >>> Most products shipping with support for SAS and SATA drives >>> support this standard. The SGPIO specification is given the >>> official name SFF-8485 and is maintained by the SFF Committee. >>> >>> 4.2 Problem Description >>> >>> On most Sun platforms FAULT (Amber) and OK2RM (Blue) LEDs are >>> controlled by Service Processor (SP) via IPMI, while ACTIVITY >>> (Green) LED is controlled by the disk HBA. >>> >>> On some other platforms that use hardware RAID HBA, the >>> FAULT LED may be controlled by HBA along with the ACTIVITY LED. >>> Dorado/Tucana platform employs SATA controller embedded in nVidia >>> chipset, which provides SGPIO interface, that can be used to >>> control all type of drive-related LEDs. >>> >>> There may be other platforms, employing either nVidia chipset >>> or Intel's ICH hub that could use SGPIO to control drive-related >>> LEDs. >>> >>> Currently there are no APIs that HBA drivers implement to >>> provide control over the LEDs for the target disk drives. >>> This project proposes the new APIs for HBA drivers to >>> provide access to LEDs that it may control. >>> >>> 4.3 Proposal >>> >>> This fasttrack proposes a new ioctl to control FAIL and OK2RM LED >>> indicators via SGPIO interface of a Host Bus Adapter. >>> Controlling software (zfs, FMA or other entity with the adequate >>> permission) may use this interface to turn-on and turn-off FAULT >>> and OK2RM LED indicators using HBA SGPIO hardware interface. >>> If HBA does not support SGPIO interface, the ioctl would fail >>> with an appropriate error. >>> >>> The default LED state set by HBA (without an explicit command >>> to turn LED ON) would be OFF state. >>> >>> If the Service Processor controls FAULT LED via IPMI interface, >>> IPMI LED ON setting will override the HBA LED OFF setting. >>> >>> 4.3.1 Considerations >>> >>> The DKIO type ioctl was considered and rejected, because it >>> requires a presence of a device target node. In case of a >>> failed device the target node for the device may not exists. >>> Furthermore, there is no path in scsi target driver and scsi >>> module to pass ioctls that are not processed by these modules >>> to the HBA drivers. >>> >>> The new DEVCTL type ioctl was selected because its use does >>> not depend on the presence of a device target node. The new >>> ioctl would have to be implemented by HBA drivers providing >>> SGPIO interface. >>> >>> An application will have to identify (using /device path) >>> and open HBA control node (using /device path), identify LED >>> type and number for this HBA using platform topo info, and >>> execute ioctl call. >>> >>> In the future, the task of deriving the control node path >>> could be delegated to libdevice libraray, when this interface >>> would be used on many platforms. >>> >>> The ioctl-specific structure is used to pass LED specification >>> to the HBA. The alternative would be to use nvlist. >>> 4.4 Requested Binding >>> >>> patch >>> >>> >>> 4.5 Details >>> >>> Exported Interface Classification Comments >>> --------------------------------------------------------------- >>> DEVCTL_SETLED | consolidation-private | Turn ON/OFF drive >>> | | indicator LED >>> | | >>> --------------------------------------------------------------- >>> DEVCTL_GETLED | consolidation-private | Get status of drive >>> | | indicator LED >>> | | >>> --------------------------------------------------------------- >>> struct dc_led_ctl | consolidation-private | pointer to this >>> | | structure passed >>> | | as ioctl argument >>> | | >>> --------------------------------------------------------------- >>> >>> SYNOPSIS: >>> >>> int ioctl(int fildes, int request, arg); >>> >>> >>> DESCRIPTION >>> >>> The fildes argument is an open file descriptor that refers >>> to a HBA control node. >>> >>> The request is DEVCTL_SETLED or DEVCTL_GETLED >>> >>> #define DEVCTL_SET_LED (DEVCTL_IOC | 39) >>> #define DEVCTL_GET_LEDSTATE (DEVCTL_IOC | 40) >>> >>> The argument is a pointer to dc_led_ctl structure: >>> >>> struct dc_led_ctl { >>> uint32_t led_number : 16; /* LED/device number */ >>> uint32_t led_ctl_active : 1; /* Control active */ >>> uint32_t led_type : 3; /* LED type */ >>> uint32_t led_state: 4; /* LED ON/OFF/Blink state */ >>> } >>> >>> /* Control active field *' >>> #define DCL_CNTRL_OFF 0 /* Control inactive */ >>> #define DCL_CNTRL_ON 1 /* Control active */ >>> >>> /* LED type field */ >>> #define DCL_TYPE_FAIL 1 /* FAIL LED type */ >>> #define DCL_TYPE_OK2RM 2 /* OK2RM LED type */ >>> >>> /* Led state field */ >>> #define DCL_STATE_OFF 0 /* LED state OFF */ >>> #define DCL_STATE_ON 1 /* LED state ON */ >>> #define DCL_STATE_SLOW_BLNK 2 /* LED Slow blink */ >>> #define DCL_STATE_FAST_BLNK 3 /* LED Fast blink */ >>> >>> led_num should be a value corresponding to the device number >>> to which the LED indicator is related. In practical terms >>> it is the LED/drive data position in the SGPIO's SDataOut >>> bit-stream where each drive is represented by 3 bits >>> (3-touple). >>> >>> The value range is 0 to n-1, where n is the number of >>> devices supported by HBA instance. >>> >>> The LED number should be available in platform topology >>> info. >>> >>> Note: For nVidia's MCP55 and IO55 chips and system using >>> nv_sata driver, LED number is the same as the disk number >>> in physical device path /devices/../disk@Y. >>> >>> For DEVCTL_SET_LED request: >>> >>> Caller has to set all fields of the dc_led_ctl structure. >>> If DCL_CNTRL_OFF is specified, all other parameters are >>> ignored and HBA should make SPGIO inactive (put SGPIO >>> in tri-state) for all devices controlled by the HBA. >>> Slow blink and fast blink rates are defined by Sun >>> document Sun_Microsystems_HDD_LED_Requirements_Rev_1_8. >>> If DCL_CNTRL_ON is specified, caller has to set all fields >>> of the dc_led_ctl structure. >>> >>> >>> For DEVCTL_GET_LEDSTATE request: >>> >>> Caller has to specify led_type and led_number in dc_led_ctl >>> structure. HBA will set led_ctl_active and led_state fields. >>> The LED state is not persistent over reboots. >>> If the SGPIO control interface was not explicitly enabled >>> via DEVCTL_SET_LED with DCL_CNTRL_ON request, the LED state >>> is DCL_STATE_OFF for every LED controlled by the HBA instance. >>> IOCTL may specify one LED type and device number at a time. >>> Initial state of the SGPIO lines is tri-state (not driven >>> by the HBA). In such state the default LED setting as provided >>> by a backplane SGPIO target hardware determines the actual LED >>> state and such state is unknown to the HBA driver. >>> The default state of a LED indicator driven by HBA is OFF, >>> i.e. if the LED of any type is not explicitly specified to be >>> ON, the HBA should specify it state as OFF. >>> >>> If the system BIOS (or BIOS-extension) does not enable SGPIO >>> interface, an explicit ioctl has to be sent to eenable the >>> SGPIO interface even when only ACTIVITY LEDs are to be under >>> HBA control (normally not controlled via DEVCTL_SET_LED ioctl). >>> >>> Some HBAs (such as hardware RAID HBAs) may be actively control >>> FAIL and/or OK2RM LEDs. For these HBAs the initial state of >>> LEDs is HBA-specific and cntrl_active field of the dc_led_ctl >>> structure may be ignored by the HBA (ie. SGPIO interface cannot >>> be deactivated and/or tri-stated). Drivers for such HBAs >>> return DCL_CNTLR_ON sate in the cntrl_active field even >>> without an explicit request to activate the control. >>> >>> Implementation Note: >>> >>> The ioctl call should return immediately. In case of the >>> DEVCTL_SET_LED request the HBA driver should store the request >>> internally and execute it on a separate execution thread if it >>> cannot be executed immediately. >>> >>> The SATA HBA driver should maintain the state of LEDs for the >>> purpose of providing LEDs state in response to DEVCTL_GET_LEDSTATE >>> request. >>> The LED control should not interfere with normal I/O activity >>> (if any) on the specified device. >>> >>> PRIV_ALL privilege is required to invoke DEVCTL_LED_CTL ioctl. >>> >>> >>> RETURN VALUES >>> >>> Upon successful completion the return value is 0. >>> Otherwise -1 is returned and errno is set to indicate an error. >>> >>> ERRORS >>> >>> The IOCTL would fail if: >>> >>> ENOSUP HBA does not support LED control >>> >>> ENXIO the specified drive LED number is not supported by >>> the controller and/or HBA driver. >>> >>> EINVAL the LED type or state is not supported >>> >>> ENOPERM no permision to execute this ioctl (not owner) >>> >>> >>> >>> 6. Resources and Schedule >>> 6.4. Steering Committee requested information >>> 6.4.1. Consolidation C-team Name: >>> ON >>> 6.5. ARC review type: Fasttrack >>> 6.6. ARC Exposure: open >>> >>> 6. Resources and Schedule >>> 6.4. Steering Committee requested information >>> 6.4.1. Consolidation C-team Name: >>> ON >>> 6.5. ARC review type: FastTrack >>> 6.6. ARC Exposure: closed >>> 6.6.1. Rationale: The Sun document describing LED behavior >>> is marked XXXfidential and is under NDA >>> >> > From sacadmin Tue May 13 17:27:33 2008 Received: from jurassic-x4600.sfbay.sun.com (cretaceous.SFBay.Sun.COM [129.146.17.59]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m4E0RXVJ014778 for ; Tue, 13 May 2008 17:27:33 -0700 (PDT) Received: from [129.146.108.62] (vinifera.SFBay.Sun.COM [129.146.108.62]) by jurassic-x4600.sfbay.sun.com (8.14.2+Sun/8.14.2) with ESMTP id m4E0RW5c734070; Tue, 13 May 2008 17:27:32 -0700 (PDT) Message-ID: <482A31F4.1080005@sun.com> Date: Tue, 13 May 2008 17:27:32 -0700 From: Scott Rotondo User-Agent: Thunderbird 2.0.0.6 (X11/20071119) MIME-Version: 1.0 To: Pawel Wojcik CC: Darren J Moffat , Alan Perry , PSARC@sac.sfbay.sun.com, sata-dev@sun.com Subject: Re: IOCTL interface for drive LED control [PSARC/2008/317 FastTrack timeout 05/20/2008] References: <200805130632.m4D6WFul004136@groomlake.sfbay.sun.com> <482991D6.7020004@Sun.COM> <4829C6B5.3060801@sun.com> In-Reply-To: <4829C6B5.3060801@sun.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Status: RO Content-Length: 1491 Pawel Wojcik wrote: > Darren J Moffat wrote: >> Alan Perry wrote: >>> PRIV_ALL privilege is required to invoke DEVCTL_LED_CTL ioctl. >> >> Is it really that much of a security risk ? >> >> Requiring this much privilege to invoke this ioctl means that system >> daemons/commands that could otherwise run with very little (or may no) >> privilege have to run with all privilege instead. >> >> If an existing privilege is to be used then I think "sys_devices" is >> the closest match though I'm starting to think we need a privilege >> like sys_ip_config for "disk". > Perhaps it does not require PRIV_ALL privilege. But one has to consider > implications of setting FAIL or OK2RM LED that may results in an > unintended device removal from the system. We may consider the > privileges that are required to control LED using IPMI through the > System Processor, which is the other entity currently controlling LEDs. Requiring all privileges is only justified where the operation can lead to privilege escalation (i.e. where a process can gain additional privileges from the operation, as it could by writing to /dev/kmem, for example). Something like sys_devices or sys_config seems more appropriate here. As Garrett points out, an explicit privilege check may not make sense at all. If opening the device to issue the ioctl already requires privilege, it may be better to rely on that access control mechanism rather than adding a new hard-coded privilege check. Scott From sacadmin Tue May 13 22:05:20 2008 Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01.SFBay.Sun.COM [129.145.155.118]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m4E55KV4020022 for ; Tue, 13 May 2008 22:05:20 -0700 (PDT) Received: from sca-es-mail-2.sun.com (sca-es-mail-2.Sun.COM [192.18.43.133]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m4E55Krq014467 for ; Tue, 13 May 2008 22:05:20 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m4E55Ek6009283 for ; Tue, 13 May 2008 22:05:14 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K0U00801E1PGB00@fe-sfbay-09.sun.com> (original mail from Pawel.Wojcik@Sun.COM) for PSARC@sac.sfbay.sun.com; Tue, 13 May 2008 22:05:14 -0700 (PDT) Received: from [129.158.219.241] by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K0U00A12E4NRK90@fe-sfbay-09.sun.com>; Tue, 13 May 2008 22:05:14 -0700 (PDT) Date: Tue, 13 May 2008 22:35:05 -0700 From: pawelw Subject: Re: IOCTL interface for drive LED control [PSARC/2008/317 FastTrack timeout 05/20/2008] In-reply-to: <4829CEC6.9000708@sun.com> Sender: Pawel.Wojcik@Sun.COM To: "Garrett D'Amore" Cc: Alan Perry , PSARC@sac.sfbay.sun.com, sata-dev@Sun.COM, Pawel.Wojcik@Sun.COM Message-id: <482A7A09.4070903@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200805130632.m4D6WFul004136@groomlake.sfbay.sun.com> <4829B7DF.7080201@sun.com> <4829CEA3.7000906@sun.com> <4829CEC6.9000708@sun.com> User-Agent: Thunderbird 2.0.0.14 (X11/20080421) Status: RO Content-Length: 19632 Garrett D'Amore wrote: > So my initial suggestions about the interface being able to support > other kinds of devices were oriented towards making the ioctl numbers > and structures flexible enough to support other LED-type uses. > > I think the changes I proposed are simple enough (one could argue > "trivial" - one additional ioctl that can in your HBA driver always > return a fixed number, and an additional structure member that can > also be implemented as a constant in HBA drivers) that they are worth > doing in order to support possible future uses. The following change should satisfy your requirement and leave some room for expansion: Adding #define DEVCTL_NUM_LED (DEVCTL_IOC | 41) Changing dcl_led_clt to: struct dc_led_ctl { uint32_t led_device_num : 16; /* LED/device number */ uint32_t led_ctl_active : 1; /* Control active */ uint32_t led_device_type : 3; /* LED type */ uint32_t led_device_state: 4; /* LED ON/OFF/Blink state */ uint32_t led_hba_num: 2 /* HBA LED number */ uint32_t led_hba_type: 3; /* HBA LED type */ uint32_t led_hba_state: 3; /* HBA LED ON/OFF/blink */ } /* Device LED type field - two are specified, one more may be added later */ #define DCL_TYPE_DEVICE_FAIL 1 /* Device FAIL LED type */ #define DCL_TYPE_DEVICE_OK2RM 2 /* Device OK2RM LED type */ /* HBA LED type field - two are specified, one more may be added later */ #define DCL_TYPE_HBA_FAIL 1 /* HBA FAIL LED type */ #define DCL_TYPE_HBA_OK2RM 2 /* HBA OK2RM LED type */ /* Led state field */ #define DCL_STATE_OFF 0 /* LED state OFF */ #define DCL_STATE_ON 1 /* LED state ON */ #define DCL_STATE_SLOW_BLNK 2 /* LED Slow blink */ #define DCL_STATE_FAST_BLNK 4 /* LED Fast blink */ DVCTL_GET_LED and DEVCTL_SET_LED would operate as described before. DEVCTL_NUM_LEDS would return in the same dc_led_ctl structure: led_device_num --- number of supported device LED tuples; led_device_type --- indication of supporting device FAIL and OK2RM LEDs (0x3 constant); led_device_state --- indication of supporting device LED blink patterns, as defined by DCL_STATE_XXX_BLNK ; led_hba_num --- number of supported HBA specific LEDs; led_hba_type --- indicator of supporting HBA FAIL and HBA OK2RM LEDs; led_hba_state --- indication of supporting HBA LED blink patterns; If this is acceptable, I will change the proposal accordingly (with other comment-related changes) -Pawel > If nobody ever makes use of them, that's fine -- it didn't hurt much > to add the unused member/ioctl, but if we can make life a lot easier > for someone in the future by making very small changes to support > future expansion now, why wouldn't we? > > (Again, I'm talking about the *interface* used here -- the ioctls. I > am not suggesting that anyone needs to actually implement support for > anything other than SFF8485 SGPIO. I just would prefer an interface > that can adapt easily beyond such limited use to one that is strictly > limited to SFF8485 SGPIO.) > > - Garrett > > Pawel Wojcik wrote: >> Garrett D'Amore wrote: >>> This looks pretty good, but I do wonder if there should be #define's >>> for different LED numbers, to identify which is which? >> This interface was considered as complimentary to IPMI LED control >> for cases where SP may not be able to control some of the specified >> LEDs. Also, it is limited only to SGPIO used according SFF8485 spec, >> clearly limited to disk-related LEDs only. >> The data passed on SGPIO always specifies 3 LED tuple, as when we are >> referring to LED number, it is the LED tuple that we are referring to. >> Defining the actual LED number does not make much sense in this >> context, because they are limited in range by the number of devices >> supported by HBA, directly related to the ports (1:1) supported by >> the hba, and the actual routing to particular bay is isolated from >> the specification passed via SGPIO by the LED controller chip on the >> backplane. >>> >>> Furthermore, in my opinion, I think it would be useful to allow >>> different LED numbers (and perhaps "types of LEDs" to be used by >>> this ioctl interface, as I think it is likely that this control >>> interface has utility beyond just disk drives. (A common LED API is >>> potentially useful to other kinds of systems/devices.) >> Please see comment above. The scope of this interface is rather limited. >>> >>> In order to be "common", I think a way to inquire what kinds of LEDs >>> are present/supported would be useful. >>> >>> If the project team doesn't want to worry about other kinds of uses >>> outside of disk drives for now, that's OK, too. It would still be >>> nice to "plan" for future extensions though -- perhaps by having a >>> simple inquiry interface: >>> >>> DEVCTL_NUM_LEDs >> This may be a misleading number, as not all possibly controlled LEDs >> may be exposed/used by the platform. The SFF8485 spec is rather >> flexible in this respect, but the hardware implementation in the >> considered controllers are quite rigid and limited. Platform topology >> file should be generally used to determine LED number. >>> >>> which simply returns the total number of LEDs supported by the >>> devctl node, and then by extending DEVCTL_GET_LED a bit, perhaps by >>> adding something like: >>> >>> uint32_t led_capab; >>> #define DEVCTL_CAP_SLOW_BLINK (1<< 0) >>> #define DEVCTL_CAP_FAST_BLINK (1<<1) >>> >>> and then changing the led_type #defines just a little bit: >>> >>> #define DC_TYPE_DEVICE_FAIL 1 >>> #define DC_TYPE_DEVICE_OK2RM 2 >>> #define DC_TYPE_BUS_FAIL 3 >>> >>> (this just makes it clear that these LEDs are related to disk >>> drives, and not to the HBA itself. One can imagine an LED with a >>> HBA failure indicator (DC_TYPE_BUS_FAIL) separate from a drive >>> (DC_TYPE_DEVICE_FAIL) failure. >> SFF8584 does not provide such option (i.e. controlling LED not >> related to disk). >>> >>> Nit: I think you need to correct your #define's for the ioctl >>> commands. I *think* you probably meant to define DEVCTL_SET_LED and >>> DEVCTL_GET_LED, rather than DEVCTL_GET_LEDSTATE. :-) >> Yes, thank you. For both IOCTLs... >>> Privilege: does PRIV_ALL really make the most sense here? I'd think >>> that typically whatever privilege is required to open the device >>> node for devctl (either in r/w or r/o mode) should be sufficient. >>> It doesn't seem that there is much utility in having a different >>> privilege requirement for these operations than what would be >>> required to perform other cfgadm related operations. >> There is no plan to "connect" LED control functionality with cfgadm >> functionality. >> Perhaps it does not require PRIV_ALL privilege. But one has to >> consider implications of setting FAIL or OK2RM LED that may result in >> an unintended device removal from the system. We may consider the >> privileges that are required to control LED using IPMI through the >> System Processor, which is the other entity currently controlling LEDs. >> I will check what priviledges are required for LED control via IPMI >> and specify the same for this interface. >>> -- Garrett >>> >>> Alan Perry wrote: >>>> I am sponsoring this fast-track for Pawel. It times out on 20 May. >>>> >>>> Template Version: @(#)sac_nextcase 1.66 04/17/08 SMI >>>> This information is Copyright 2008 Sun Microsystems, Inc. >>>> 1. Introduction >>>> 1.1. Project/Component Working Name: >>>> IOCTL interface for drive LED control >>>> 1.2. Name of Document Author/Supplier: >>>> Author: Pawel Wojcik >>>> 1.3 Date of This Document: >>>> 12 May, 2008 >>>> >>>> 2. References >>>> [1] SFF-8485 Specification for Serial GPIO (SGPIO) Bus >>>> [2] Sun_Microsystems_HDD_LED_Requirements_Rev_1_8 >>>> >>>> 4. Technical Description >>>> >>>> 4.1 Overview >>>> Drive-related LED indicators may be controlled by various >>>> hardware entities. Generally, Serial General Purpose >>>> Input/Output (SGPIO) interface is used to control LED indicators >>>> on the backplane. >>>> >>>> SGPIO is a 4-signal (or 4-wire) bus used between a Host Bus >>>> Adapter (HBA) and a backplane. Out of the 4 signals, 3 are >>>> driven by the HBA or other hardware controller and 1 is driven >>>> by the backplane. The data bits exchanged are related to drives >>>> supported by the target and are typically related to specific >>>> lanes of the high-speed serial storage interface. >>>> >>>> SGPIO and the SGPIO spec is generally adopted and implemented >>>> in products from most major HBA and Storage Controller vendors >>>> such as LSI, Intel, Adaptec, Nvidia, Broadcom and PMC-Sierra. >>>> Most products shipping with support for SAS and SATA drives >>>> support this standard. The SGPIO specification is given the >>>> official name SFF-8485 and is maintained by the SFF Committee. >>>> >>>> 4.2 Problem Description >>>> >>>> On most Sun platforms FAULT (Amber) and OK2RM (Blue) LEDs are >>>> controlled by Service Processor (SP) via IPMI, while ACTIVITY >>>> (Green) LED is controlled by the disk HBA. >>>> >>>> On some other platforms that use hardware RAID HBA, the >>>> FAULT LED may be controlled by HBA along with the ACTIVITY LED. >>>> Dorado/Tucana platform employs SATA controller embedded in nVidia >>>> chipset, which provides SGPIO interface, that can be used >>>> to control all type of drive-related LEDs. >>>> >>>> There may be other platforms, employing either nVidia chipset >>>> or Intel's ICH hub that could use SGPIO to control drive-related >>>> LEDs. >>>> >>>> Currently there are no APIs that HBA drivers implement to >>>> provide control over the LEDs for the target disk drives. >>>> This project proposes the new APIs for HBA drivers to >>>> provide access to LEDs that it may control. >>>> >>>> 4.3 Proposal >>>> >>>> This fasttrack proposes a new ioctl to control FAIL and OK2RM LED >>>> indicators via SGPIO interface of a Host Bus Adapter. >>>> Controlling software (zfs, FMA or other entity with the adequate >>>> permission) may use this interface to turn-on and turn-off FAULT >>>> and OK2RM LED indicators using HBA SGPIO hardware interface. >>>> If HBA does not support SGPIO interface, the ioctl would fail >>>> with an appropriate error. >>>> >>>> The default LED state set by HBA (without an explicit command >>>> to turn LED ON) would be OFF state. >>>> >>>> If the Service Processor controls FAULT LED via IPMI interface, >>>> IPMI LED ON setting will override the HBA LED OFF setting. >>>> >>>> 4.3.1 Considerations >>>> >>>> The DKIO type ioctl was considered and rejected, because it >>>> requires a presence of a device target node. In case of a >>>> failed device the target node for the device may not exists. >>>> Furthermore, there is no path in scsi target driver and scsi >>>> module to pass ioctls that are not processed by these modules >>>> to the HBA drivers. >>>> >>>> The new DEVCTL type ioctl was selected because its use does >>>> not depend on the presence of a device target node. The new >>>> ioctl would have to be implemented by HBA drivers providing >>>> SGPIO interface. >>>> >>>> An application will have to identify (using /device path) >>>> and open HBA control node (using /device path), identify LED >>>> type and number for this HBA using platform topo info, and >>>> execute ioctl call. >>>> >>>> In the future, the task of deriving the control node path >>>> could be delegated to libdevice libraray, when this interface >>>> would be used on many platforms. >>>> >>>> The ioctl-specific structure is used to pass LED specification >>>> to the HBA. The alternative would be to use nvlist. >>>> 4.4 Requested Binding >>>> >>>> patch >>>> >>>> >>>> 4.5 Details >>>> >>>> Exported Interface Classification Comments >>>> --------------------------------------------------------------- >>>> DEVCTL_SETLED | consolidation-private | Turn ON/OFF drive >>>> | | indicator LED >>>> | | >>>> --------------------------------------------------------------- >>>> DEVCTL_GETLED | consolidation-private | Get status of drive >>>> | | indicator LED >>>> | | >>>> --------------------------------------------------------------- >>>> struct dc_led_ctl | consolidation-private | pointer to this >>>> | | structure passed >>>> | | as ioctl argument >>>> | | >>>> --------------------------------------------------------------- >>>> >>>> SYNOPSIS: >>>> >>>> int ioctl(int fildes, int request, arg); >>>> >>>> >>>> DESCRIPTION >>>> >>>> The fildes argument is an open file descriptor that refers >>>> to a HBA control node. >>>> >>>> The request is DEVCTL_SETLED or DEVCTL_GETLED >>>> >>>> #define DEVCTL_SET_LED (DEVCTL_IOC | 39) >>>> #define DEVCTL_GET_LEDSTATE (DEVCTL_IOC | 40) >>>> >>>> The argument is a pointer to dc_led_ctl structure: >>>> >>>> struct dc_led_ctl { >>>> uint32_t led_number : 16; /* LED/device number */ >>>> uint32_t led_ctl_active : 1; /* Control active */ >>>> uint32_t led_type : 3; /* LED type */ >>>> uint32_t led_state: 4; /* LED ON/OFF/Blink state */ >>>> } >>>> >>>> /* Control active field *' >>>> #define DCL_CNTRL_OFF 0 /* Control inactive */ >>>> #define DCL_CNTRL_ON 1 /* Control active */ >>>> >>>> /* LED type field */ >>>> #define DCL_TYPE_FAIL 1 /* FAIL LED type */ >>>> #define DCL_TYPE_OK2RM 2 /* OK2RM LED type */ >>>> >>>> /* Led state field */ >>>> #define DCL_STATE_OFF 0 /* LED state OFF */ >>>> #define DCL_STATE_ON 1 /* LED state ON */ >>>> #define DCL_STATE_SLOW_BLNK 2 /* LED Slow blink */ >>>> #define DCL_STATE_FAST_BLNK 3 /* LED Fast blink */ >>>> >>>> led_num should be a value corresponding to the device number >>>> to which the LED indicator is related. In practical terms >>>> it is the LED/drive data position in the SGPIO's SDataOut >>>> bit-stream where each drive is represented by 3 bits >>>> (3-touple). >>>> >>>> The value range is 0 to n-1, where n is the number of >>>> devices supported by HBA instance. >>>> >>>> The LED number should be available in platform topology >>>> info. >>>> >>>> Note: For nVidia's MCP55 and IO55 chips and system using >>>> nv_sata driver, LED number is the same as the disk number >>>> in physical device path /devices/../disk@Y. >>>> >>>> For DEVCTL_SET_LED request: >>>> >>>> Caller has to set all fields of the dc_led_ctl structure. >>>> If DCL_CNTRL_OFF is specified, all other parameters are >>>> ignored and HBA should make SPGIO inactive (put SGPIO >>>> in tri-state) for all devices controlled by the HBA. >>>> Slow blink and fast blink rates are defined by Sun >>>> document Sun_Microsystems_HDD_LED_Requirements_Rev_1_8. >>>> If DCL_CNTRL_ON is specified, caller has to set all fields >>>> of the dc_led_ctl structure. >>>> >>>> >>>> For DEVCTL_GET_LEDSTATE request: >>>> >>>> Caller has to specify led_type and led_number in dc_led_ctl >>>> structure. HBA will set led_ctl_active and led_state fields. >>>> The LED state is not persistent over reboots. >>>> If the SGPIO control interface was not explicitly enabled >>>> via DEVCTL_SET_LED with DCL_CNTRL_ON request, the LED state >>>> is DCL_STATE_OFF for every LED controlled by the HBA instance. >>>> IOCTL may specify one LED type and device number at a time. >>>> Initial state of the SGPIO lines is tri-state (not driven >>>> by the HBA). In such state the default LED setting as provided >>>> by a backplane SGPIO target hardware determines the actual LED >>>> state and such state is unknown to the HBA driver. >>>> The default state of a LED indicator driven by HBA is OFF, >>>> i.e. if the LED of any type is not explicitly specified to be >>>> ON, the HBA should specify it state as OFF. >>>> >>>> If the system BIOS (or BIOS-extension) does not enable SGPIO >>>> interface, an explicit ioctl has to be sent to eenable the >>>> SGPIO interface even when only ACTIVITY LEDs are to be under >>>> HBA control (normally not controlled via DEVCTL_SET_LED ioctl). >>>> >>>> Some HBAs (such as hardware RAID HBAs) may be actively control >>>> FAIL and/or OK2RM LEDs. For these HBAs the initial state of >>>> LEDs is HBA-specific and cntrl_active field of the dc_led_ctl >>>> structure may be ignored by the HBA (ie. SGPIO interface cannot >>>> be deactivated and/or tri-stated). Drivers for such HBAs >>>> return DCL_CNTLR_ON sate in the cntrl_active field even >>>> without an explicit request to activate the control. >>>> >>>> Implementation Note: >>>> >>>> The ioctl call should return immediately. In case of the >>>> DEVCTL_SET_LED request the HBA driver should store the request >>>> internally and execute it on a separate execution thread if it >>>> cannot be executed immediately. >>>> >>>> The SATA HBA driver should maintain the state of LEDs for the >>>> purpose of providing LEDs state in response to DEVCTL_GET_LEDSTATE >>>> request. >>>> The LED control should not interfere with normal I/O activity >>>> (if any) on the specified device. >>>> >>>> PRIV_ALL privilege is required to invoke DEVCTL_LED_CTL ioctl. >>>> >>>> >>>> RETURN VALUES >>>> >>>> Upon successful completion the return value is 0. >>>> Otherwise -1 is returned and errno is set to indicate an error. >>>> >>>> ERRORS >>>> >>>> The IOCTL would fail if: >>>> >>>> ENOSUP HBA does not support LED control >>>> >>>> ENXIO the specified drive LED number is not supported by >>>> the controller and/or HBA driver. >>>> >>>> EINVAL the LED type or state is not supported >>>> >>>> ENOPERM no permision to execute this ioctl (not owner) >>>> >>>> >>>> >>>> 6. Resources and Schedule >>>> 6.4. Steering Committee requested information >>>> 6.4.1. Consolidation C-team Name: >>>> ON >>>> 6.5. ARC review type: Fasttrack >>>> 6.6. ARC Exposure: open >>>> >>>> 6. Resources and Schedule >>>> 6.4. Steering Committee requested information >>>> 6.4.1. Consolidation C-team Name: >>>> ON >>>> 6.5. ARC review type: FastTrack >>>> 6.6. ARC Exposure: closed >>>> 6.6.1. Rationale: The Sun document describing LED behavior >>>> is marked XXXfidential and is under NDA >>>> >>> >> > From sacadmin Tue May 13 22:14:59 2008 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m4E5Exq2020165 for ; Tue, 13 May 2008 22:14:59 -0700 (PDT) Received: from sca-es-mail-1.sun.com (sca-es-mail-1.Sun.COM [192.18.43.132]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m4E5Ex62057979 for ; Tue, 13 May 2008 22:14:59 -0700 (PDT) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m4E5EsOC000210 for ; Tue, 13 May 2008 22:14:54 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K0U00A01E9EC000@fe-sfbay-10.sun.com> (original mail from Pawel.Wojcik@Sun.COM) for PSARC@sac.sfbay.sun.com; Tue, 13 May 2008 22:14:54 -0700 (PDT) Received: from [129.158.219.241] by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K0U002TZEKD3ND0@fe-sfbay-10.sun.com>; Tue, 13 May 2008 22:14:39 -0700 (PDT) Date: Tue, 13 May 2008 22:44:31 -0700 From: pawelw Subject: Re: IOCTL interface for drive LED control [PSARC/2008/317 FastTrack timeout 05/20/2008] In-reply-to: <482A31F4.1080005@sun.com> Sender: Pawel.Wojcik@Sun.COM To: Scott Rotondo Cc: Darren J Moffat , Alan Perry , PSARC@sac.sfbay.sun.com, sata-dev@Sun.COM Message-id: <482A7C3F.5080005@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200805130632.m4D6WFul004136@groomlake.sfbay.sun.com> <482991D6.7020004@Sun.COM> <4829C6B5.3060801@sun.com> <482A31F4.1080005@sun.com> User-Agent: Thunderbird 2.0.0.14 (X11/20080421) Status: RO Content-Length: 1768 Scott Rotondo wrote: > Pawel Wojcik wrote: >> Darren J Moffat wrote: >>> Alan Perry wrote: >>>> PRIV_ALL privilege is required to invoke DEVCTL_LED_CTL ioctl. >>> >>> Is it really that much of a security risk ? >>> >>> Requiring this much privilege to invoke this ioctl means that system >>> daemons/commands that could otherwise run with very little (or may >>> no) privilege have to run with all privilege instead. >>> >>> If an existing privilege is to be used then I think "sys_devices" is >>> the closest match though I'm starting to think we need a privilege >>> like sys_ip_config for "disk". >> Perhaps it does not require PRIV_ALL privilege. But one has to >> consider implications of setting FAIL or OK2RM LED that may results >> in an unintended device removal from the system. We may consider the >> privileges that are required to control LED using IPMI through the >> System Processor, which is the other entity currently controlling LEDs. > > Requiring all privileges is only justified where the operation can > lead to privilege escalation (i.e. where a process can gain additional > privileges from the operation, as it could by writing to /dev/kmem, > for example). Something like sys_devices or sys_config seems more > appropriate here. > > As Garrett points out, an explicit privilege check may not make sense > at all. If opening the device to issue the ioctl already requires > privilege, it may be better to rely on that access control mechanism > rather than adding a new hard-coded privilege check. > > Scott OK The wording "PRIV_ALL privilege is required to invoke DEVCTL_LED_CTL ioctl." will be removed. The privilege required to open the device is sufficient and not explicit privilege checking is needed. -Pawel From sacadmin Tue May 13 22:21:08 2008 Received: from jurassic-x4600.sfbay.sun.com (cretaceous.SFBay.Sun.COM [129.146.17.63]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m4E5L8Q3020325 for ; Tue, 13 May 2008 22:21:08 -0700 (PDT) Received: from [129.150.13.200] (vpn-129-150-13-200.SFBay.Sun.COM [129.150.13.200]) by jurassic-x4600.sfbay.sun.com (8.14.2+Sun/8.14.2) with ESMTP id m4E5L7Jr756949; Tue, 13 May 2008 22:21:07 -0700 (PDT) Message-ID: <482A7720.2040308@sun.com> Date: Tue, 13 May 2008 19:22:40 -1000 From: Joseph Kowalski User-Agent: Thunderbird 2.0.0.9 (X11/20080225) MIME-Version: 1.0 To: pawelw CC: Scott Rotondo , Darren J Moffat , Alan Perry , PSARC@sac.sfbay.sun.com, sata-dev@sun.com Subject: Re: IOCTL interface for drive LED control [PSARC/2008/317 FastTrack timeout 05/20/2008] References: <200805130632.m4D6WFul004136@groomlake.sfbay.sun.com> <482991D6.7020004@Sun.COM> <4829C6B5.3060801@sun.com> <482A31F4.1080005@sun.com> <482A7C3F.5080005@sun.com> In-Reply-To: <482A7C3F.5080005@sun.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Status: RO Content-Length: 300 There seems to be enough changes (maybe at the code review level, but whatever), that I'd like to request an updates specification. Thanks,... Gee, isn't it good to see a couple of cases this week with real meat rather than arguing about how much change should be done to some FOSS? :-) - jek3 From sacadmin Tue May 13 22:35:39 2008 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m4E5Zd0u020675 for ; Tue, 13 May 2008 22:35:39 -0700 (PDT) Received: from sca-es-mail-2.sun.com (sca-es-mail-2.Sun.COM [192.18.43.133]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m4E5ZdLE000437 for ; Tue, 13 May 2008 22:35:39 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m4E5ZYXu010079 for ; Tue, 13 May 2008 22:35:34 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K0U00101FDUYR00@fe-sfbay-09.sun.com> (original mail from Pawel.Wojcik@Sun.COM) for PSARC@sac.sfbay.sun.com; Tue, 13 May 2008 22:35:34 -0700 (PDT) Received: from [129.158.219.241] by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K0U00AMTFJ7RKE0@fe-sfbay-09.sun.com>; Tue, 13 May 2008 22:35:33 -0700 (PDT) Date: Tue, 13 May 2008 23:05:25 -0700 From: pawelw Subject: Re: IOCTL interface for drive LED control [PSARC/2008/317 FastTrack timeout 05/20/2008] In-reply-to: <4829C6B5.3060801@sun.com> Sender: Pawel.Wojcik@Sun.COM To: Darren J Moffat Cc: Alan Perry , PSARC@sac.sfbay.sun.com, sata-dev@Sun.COM, Pawel.Wojcik@Sun.COM Message-id: <482A8125.2000703@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200805130632.m4D6WFul004136@groomlake.sfbay.sun.com> <482991D6.7020004@Sun.COM> <4829C6B5.3060801@sun.com> User-Agent: Thunderbird 2.0.0.14 (X11/20080421) Status: RO Content-Length: 3263 Pawel Wojcik wrote: > Darren J Moffat wrote: >> Alan Perry wrote: >>> PRIV_ALL privilege is required to invoke DEVCTL_LED_CTL ioctl. >> >> Is it really that much of a security risk ? >> >> Requiring this much privilege to invoke this ioctl means that system >> daemons/commands that could otherwise run with very little (or may >> no) privilege have to run with all privilege instead. >> >> If an existing privilege is to be used then I think "sys_devices" is >> the closest match though I'm starting to think we need a privilege >> like sys_ip_config for "disk". > Perhaps it does not require PRIV_ALL privilege. But one has to > consider implications of setting FAIL or OK2RM LED that may results in > an unintended device removal from the system. We may consider the > privileges that are required to control LED using IPMI through the > System Processor, which is the other entity currently controlling LEDs. The wording "PRIV_ALL privilege is required to invoke DEVCTL_LED_CTL ioctl." will be removed. The privilege required to open the device is sufficient and not explicit privilege checking is needed. >> >> Is there a similar interface in Linux or BSD systems ? If so how >> does this new Solaris interface compare, or do they not have the >> ability to set these LED ? > I do not know what Linux or BSD do in respect to LED control via SGPIO. > The interface is to be complimentary to the Sun SP IPMI interface and > for entities like FMA, ZFS and Disk Monitor. >> >> Given that this case is closed (because of the closed reference >> document I believe) where is the source going to end up ? > This is interesting question. The code is definitely in the open > drivers - ahci and nv_sata. > There is no reason to close these drivers because of "XXXfidential" > status of the Sun spec document. > It is not clear how to solve this dilemma. Shall we ask to remove > XXXfidential classification from the doc? Or is it possible that the > implementation could be open despite the closed nature of the > document? The only part referenced in the PSARC case is (and > potentially implemented) is the LED blinking pattern. In fact, it does > not need to be implement it because it these blinking patters are not > used on existing platforms - it was included in the interface just for > the completeness of the interface. The simplest way to solve the problem is to remove reference to the document and define fast and slow blinking pattern in the proposal itself. This will remove obstacle to opening this case and to keeping open HBA drivers implementing this interface. Therefore, following wording will be added: "Fast-Blinking LED pattern is defined as 4Hz 50% blink. Slow-Blinking pattern is defined as 1HZ 50% blink." The reference to Sun_Microsystems_HDD_LED_Requirements_Rev_1_8 document will be removed. Is this solution acceptable? If so, I will change the proposal accordingly (along with other comments-related changes). -Pawel >> >> If the intent is that HBA drivers are implementing this ioctl why is >> it Consolidation Private to ON ? >> > The interface is not stable and is considered mostly for > Sun-manufactured platforms. There is no external requests for this > interface. > > -Pawel > From sacadmin Tue May 13 22:39:49 2008 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m4E5dnXv020690 for ; Tue, 13 May 2008 22:39:49 -0700 (PDT) Received: from sca-es-mail-1.sun.com (sca-es-mail-1.Sun.COM [192.18.43.132]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m4E5dnld001714 for ; Tue, 13 May 2008 22:39:49 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m4E5dirb001005 for ; Tue, 13 May 2008 22:39:44 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K0U00301FLDTB00@fe-sfbay-09.sun.com> (original mail from Pawel.Wojcik@Sun.COM) for PSARC@sac.sfbay.sun.com; Tue, 13 May 2008 22:39:44 -0700 (PDT) Received: from [129.158.219.241] by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K0U00AANFQ5RKF0@fe-sfbay-09.sun.com>; Tue, 13 May 2008 22:39:44 -0700 (PDT) Date: Tue, 13 May 2008 23:09:35 -0700 From: pawelw Subject: Re: IOCTL interface for drive LED control [PSARC/2008/317 FastTrack timeout 05/20/2008] In-reply-to: <482A7720.2040308@sun.com> Sender: Pawel.Wojcik@Sun.COM To: Joseph Kowalski Cc: Scott Rotondo , Darren J Moffat , Alan Perry , PSARC@sac.sfbay.sun.com, sata-dev@Sun.COM Message-id: <482A821F.9000209@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200805130632.m4D6WFul004136@groomlake.sfbay.sun.com> <482991D6.7020004@Sun.COM> <4829C6B5.3060801@sun.com> <482A31F4.1080005@sun.com> <482A7C3F.5080005@sun.com> <482A7720.2040308@sun.com> User-Agent: Thunderbird 2.0.0.14 (X11/20080421) Status: RO Content-Length: 449 Joseph Kowalski wrote: > > There seems to be enough changes (maybe at the code review level, but > whatever), that I'd like to request an updates specification. Thanks,... > > Gee, isn't it good to see a couple of cases this week with real meat > rather than arguing about how much change should be done to some > FOSS? :-) > > - jek3 I am awaiting opinions about my proposed changes before consolidating them into the updated proposal. -Pawel From sacadmin Wed May 14 04:16:59 2008 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m4EBGxl9026445 for ; Wed, 14 May 2008 04:16:59 -0700 (PDT) Received: from gmp-eb-inf-2.sun.com (gmp-eb-inf-2.EU.Sun.COM [192.18.6.24]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m4EBGwEJ000743 for ; Wed, 14 May 2008 04:16:59 -0700 (PDT) Received: from fe-emea-10.sun.com (gmp-eb-lb-2-fe3.eu.sun.com [192.18.6.12]) by gmp-eb-inf-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m4EBGrb9028972 for ; Wed, 14 May 2008 11:16:53 GMT Received: from conversion-daemon.fe-emea-10.sun.com by fe-emea-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K0U00001V0U4V00@fe-emea-10.sun.com> (original mail from Darren.Moffat@Sun.COM) for PSARC@sac.sfbay.sun.com; Wed, 14 May 2008 12:16:53 +0100 (BST) Received: from [129.156.173.21] by fe-emea-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K0U00F6LVBXU310@fe-emea-10.sun.com>; Wed, 14 May 2008 12:16:46 +0100 (BST) Date: Wed, 14 May 2008 12:16:45 +0100 From: Darren J Moffat Subject: Re: IOCTL interface for drive LED control [PSARC/2008/317 FastTrack timeout 05/20/2008] In-reply-to: <482A821F.9000209@sun.com> Sender: Darren.Moffat@Sun.COM To: pawelw Cc: Joseph Kowalski , Scott Rotondo , Alan Perry , PSARC@sac.sfbay.sun.com, sata-dev@Sun.COM Message-id: <482ACA1D.3040100@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200805130632.m4D6WFul004136@groomlake.sfbay.sun.com> <482991D6.7020004@Sun.COM> <4829C6B5.3060801@sun.com> <482A31F4.1080005@sun.com> <482A7C3F.5080005@sun.com> <482A7720.2040308@sun.com> <482A821F.9000209@sun.com> User-Agent: Thunderbird 2.0.0.12 (X11/20080326) Status: RO Content-Length: 536 pawelw wrote: > Joseph Kowalski wrote: >> >> There seems to be enough changes (maybe at the code review level, but >> whatever), that I'd like to request an updates specification. Thanks,... >> >> Gee, isn't it good to see a couple of cases this week with real meat >> rather than arguing about how much change should be done to some >> FOSS? :-) >> >> - jek3 > I am awaiting opinions about my proposed changes before consolidating > them into the updated proposal. I'm very happy with the privileges change. -- Darren J Moffat From sacadmin Wed May 14 07:51:42 2008 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m4EEpgek003566 for ; Wed, 14 May 2008 07:51:42 -0700 (PDT) Received: from sca-es-mail-1.sun.com (sca-es-mail-1.Sun.COM [192.18.43.132]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m4EEpfZ7038300 for ; Wed, 14 May 2008 07:51:41 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m4EEpa9O018109 for ; Wed, 14 May 2008 07:51:36 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K0V0040150UO300@fe-sfbay-09.sun.com> (original mail from gdamore@sun.com) for PSARC@sac.sfbay.sun.com; Wed, 14 May 2008 07:51:36 -0700 (PDT) Received: from [192.168.251.106] ([76.174.83.55]) by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K0V009ZT59ZZO00@fe-sfbay-09.sun.com>; Wed, 14 May 2008 07:51:36 -0700 (PDT) Date: Wed, 14 May 2008 07:45:25 -0700 From: "Garrett D'Amore" Subject: Re: IOCTL interface for drive LED control [PSARC/2008/317 FastTrack timeout 05/20/2008] In-reply-to: <482A7A09.4070903@sun.com> Sender: Garrett.Damore@sun.com To: pawelw Cc: Alan Perry , PSARC@sac.sfbay.sun.com, sata-dev@sun.com Message-id: <482AFB05.8020809@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200805130632.m4D6WFul004136@groomlake.sfbay.sun.com> <4829B7DF.7080201@sun.com> <4829CEA3.7000906@sun.com> <4829CEC6.9000708@sun.com> <482A7A09.4070903@sun.com> User-Agent: Thunderbird 2.0.0.6 (X11/20071023) Status: RO Content-Length: 21078 pawelw wrote: > Garrett D'Amore wrote: >> So my initial suggestions about the interface being able to support >> other kinds of devices were oriented towards making the ioctl numbers >> and structures flexible enough to support other LED-type uses. >> >> I think the changes I proposed are simple enough (one could argue >> "trivial" - one additional ioctl that can in your HBA driver always >> return a fixed number, and an additional structure member that can >> also be implemented as a constant in HBA drivers) that they are worth >> doing in order to support possible future uses. > The following change should satisfy your requirement and leave some > room for expansion: > > Adding > #define DEVCTL_NUM_LED (DEVCTL_IOC | 41) > > Changing dcl_led_clt to: > struct dc_led_ctl { > uint32_t led_device_num : 16; /* LED/device number */ > uint32_t led_ctl_active : 1; /* Control active */ > uint32_t led_device_type : 3; /* LED type */ > uint32_t led_device_state: 4; /* LED ON/OFF/Blink state */ > uint32_t led_hba_num: 2 /* HBA LED number */ > uint32_t led_hba_type: 3; /* HBA LED type */ > uint32_t led_hba_state: 3; /* HBA LED ON/OFF/blink */ > } > /* Device LED type field - two are specified, one more may be > added later */ > #define DCL_TYPE_DEVICE_FAIL 1 /* Device FAIL LED > type */ > #define DCL_TYPE_DEVICE_OK2RM 2 /* Device OK2RM LED > type */ > > /* HBA LED type field - two are specified, one more may be added > later */ #define DCL_TYPE_HBA_FAIL 1 /* HBA FAIL LED type */ > #define DCL_TYPE_HBA_OK2RM 2 /* HBA OK2RM LED type */ > > /* Led state field */ > #define DCL_STATE_OFF 0 /* LED state OFF */ > #define DCL_STATE_ON 1 /* LED state ON */ > #define DCL_STATE_SLOW_BLNK 2 /* LED Slow blink */ > #define DCL_STATE_FAST_BLNK 4 /* LED Fast blink */ > > DVCTL_GET_LED and DEVCTL_SET_LED would operate as described before. > > DEVCTL_NUM_LEDS would return in the same dc_led_ctl structure: > led_device_num --- number of supported device LED tuples; > led_device_type --- indication of supporting device FAIL and OK2RM > LEDs (0x3 constant); > led_device_state --- indication of supporting device LED blink > patterns, > as defined by DCL_STATE_XXX_BLNK ; > led_hba_num --- number of supported HBA specific LEDs; > led_hba_type --- indicator of supporting HBA FAIL and HBA OK2RM LEDs; > led_hba_state --- indication of supporting HBA LED blink patterns; > If this is acceptable, I will change the proposal accordingly (with > other comment-related changes) I'd recommend removing the _hba_xxx fields, and just widening the bit fields associated with the structure, remove the _device componeent of the led_device_xxx field names. E.g.: struct dc_led_ctl { uint32_t led_num : 16; /* LED/device number */ uint32_t led_ctl_active : 1; /* Control active */ uint32_t led_type : 9; /* LED type */ uint32_t led_state: 6; /* LED ON/OFF/Blink state */ } The reasons for this are: 1) Other devctl nodes, besides HBAs (e.g. imagine a USB HCD, or an SDcard driver, or .... :-) So lets avoid names that are disk/hba specific in the generic struct. :-) 2) If the fields are wider, then HBA leds can be added later by just adding a new led_type #define. (But in order for this to support future expansion, it needs to be a wider field. I specified 9 bits above, but anything 5 bits or more is probably adequate.) -- Garrett > > -Pawel > >> If nobody ever makes use of them, that's fine -- it didn't hurt much >> to add the unused member/ioctl, but if we can make life a lot easier >> for someone in the future by making very small changes to support >> future expansion now, why wouldn't we? >> >> (Again, I'm talking about the *interface* used here -- the ioctls. I >> am not suggesting that anyone needs to actually implement support for >> anything other than SFF8485 SGPIO. I just would prefer an interface >> that can adapt easily beyond such limited use to one that is strictly >> limited to SFF8485 SGPIO.) >> >> - Garrett >> >> Pawel Wojcik wrote: >>> Garrett D'Amore wrote: >>>> This looks pretty good, but I do wonder if there should be >>>> #define's for different LED numbers, to identify which is which? >>> This interface was considered as complimentary to IPMI LED control >>> for cases where SP may not be able to control some of the specified >>> LEDs. Also, it is limited only to SGPIO used according SFF8485 spec, >>> clearly limited to disk-related LEDs only. >>> The data passed on SGPIO always specifies 3 LED tuple, as when we >>> are referring to LED number, it is the LED tuple that we are >>> referring to. >>> Defining the actual LED number does not make much sense in this >>> context, because they are limited in range by the number of devices >>> supported by HBA, directly related to the ports (1:1) supported by >>> the hba, and the actual routing to particular bay is isolated from >>> the specification passed via SGPIO by the LED controller chip on the >>> backplane. >>>> >>>> Furthermore, in my opinion, I think it would be useful to allow >>>> different LED numbers (and perhaps "types of LEDs" to be used by >>>> this ioctl interface, as I think it is likely that this control >>>> interface has utility beyond just disk drives. (A common LED API >>>> is potentially useful to other kinds of systems/devices.) >>> Please see comment above. The scope of this interface is rather >>> limited. >>>> >>>> In order to be "common", I think a way to inquire what kinds of >>>> LEDs are present/supported would be useful. >>>> >>>> If the project team doesn't want to worry about other kinds of uses >>>> outside of disk drives for now, that's OK, too. It would still be >>>> nice to "plan" for future extensions though -- perhaps by having a >>>> simple inquiry interface: >>>> >>>> DEVCTL_NUM_LEDs >>> This may be a misleading number, as not all possibly controlled LEDs >>> may be exposed/used by the platform. The SFF8485 spec is rather >>> flexible in this respect, but the hardware implementation in the >>> considered controllers are quite rigid and limited. Platform >>> topology file should be generally used to determine LED number. >>>> >>>> which simply returns the total number of LEDs supported by the >>>> devctl node, and then by extending DEVCTL_GET_LED a bit, perhaps by >>>> adding something like: >>>> >>>> uint32_t led_capab; >>>> #define DEVCTL_CAP_SLOW_BLINK (1<< 0) >>>> #define DEVCTL_CAP_FAST_BLINK (1<<1) >>>> >>>> and then changing the led_type #defines just a little bit: >>>> >>>> #define DC_TYPE_DEVICE_FAIL 1 >>>> #define DC_TYPE_DEVICE_OK2RM 2 >>>> #define DC_TYPE_BUS_FAIL 3 >>>> >>>> (this just makes it clear that these LEDs are related to disk >>>> drives, and not to the HBA itself. One can imagine an LED with a >>>> HBA failure indicator (DC_TYPE_BUS_FAIL) separate from a drive >>>> (DC_TYPE_DEVICE_FAIL) failure. >>> SFF8584 does not provide such option (i.e. controlling LED not >>> related to disk). >>>> >>>> Nit: I think you need to correct your #define's for the ioctl >>>> commands. I *think* you probably meant to define DEVCTL_SET_LED >>>> and DEVCTL_GET_LED, rather than DEVCTL_GET_LEDSTATE. :-) >>> Yes, thank you. For both IOCTLs... >>>> Privilege: does PRIV_ALL really make the most sense here? I'd >>>> think that typically whatever privilege is required to open the >>>> device node for devctl (either in r/w or r/o mode) should be >>>> sufficient. It doesn't seem that there is much utility in having a >>>> different privilege requirement for these operations than what >>>> would be required to perform other cfgadm related operations. >>> There is no plan to "connect" LED control functionality with cfgadm >>> functionality. >>> Perhaps it does not require PRIV_ALL privilege. But one has to >>> consider implications of setting FAIL or OK2RM LED that may result >>> in an unintended device removal from the system. We may consider the >>> privileges that are required to control LED using IPMI through the >>> System Processor, which is the other entity currently controlling LEDs. >>> I will check what priviledges are required for LED control via IPMI >>> and specify the same for this interface. >>>> -- Garrett >>>> >>>> Alan Perry wrote: >>>>> I am sponsoring this fast-track for Pawel. It times out on 20 May. >>>>> >>>>> Template Version: @(#)sac_nextcase 1.66 04/17/08 SMI >>>>> This information is Copyright 2008 Sun Microsystems, Inc. >>>>> 1. Introduction >>>>> 1.1. Project/Component Working Name: >>>>> IOCTL interface for drive LED control >>>>> 1.2. Name of Document Author/Supplier: >>>>> Author: Pawel Wojcik >>>>> 1.3 Date of This Document: >>>>> 12 May, 2008 >>>>> >>>>> 2. References >>>>> [1] SFF-8485 Specification for Serial GPIO (SGPIO) Bus >>>>> [2] Sun_Microsystems_HDD_LED_Requirements_Rev_1_8 >>>>> >>>>> 4. Technical Description >>>>> >>>>> 4.1 Overview >>>>> Drive-related LED indicators may be controlled by various >>>>> hardware entities. Generally, Serial General Purpose >>>>> Input/Output (SGPIO) interface is used to control LED indicators >>>>> on the backplane. >>>>> >>>>> SGPIO is a 4-signal (or 4-wire) bus used between a Host Bus >>>>> Adapter (HBA) and a backplane. Out of the 4 signals, 3 are >>>>> driven by the HBA or other hardware controller and 1 is driven >>>>> by the backplane. The data bits exchanged are related to drives >>>>> supported by the target and are typically related to specific >>>>> lanes of the high-speed serial storage interface. >>>>> >>>>> SGPIO and the SGPIO spec is generally adopted and implemented >>>>> in products from most major HBA and Storage Controller vendors >>>>> such as LSI, Intel, Adaptec, Nvidia, Broadcom and PMC-Sierra. >>>>> Most products shipping with support for SAS and SATA drives >>>>> support this standard. The SGPIO specification is given the >>>>> official name SFF-8485 and is maintained by the SFF Committee. >>>>> >>>>> 4.2 Problem Description >>>>> >>>>> On most Sun platforms FAULT (Amber) and OK2RM (Blue) LEDs are >>>>> controlled by Service Processor (SP) via IPMI, while ACTIVITY >>>>> (Green) LED is controlled by the disk HBA. >>>>> >>>>> On some other platforms that use hardware RAID HBA, the >>>>> FAULT LED may be controlled by HBA along with the ACTIVITY LED. >>>>> Dorado/Tucana platform employs SATA controller embedded in nVidia >>>>> chipset, which provides SGPIO interface, that can be used >>>>> to control all type of drive-related LEDs. >>>>> >>>>> There may be other platforms, employing either nVidia chipset >>>>> or Intel's ICH hub that could use SGPIO to control drive-related >>>>> LEDs. >>>>> >>>>> Currently there are no APIs that HBA drivers implement to >>>>> provide control over the LEDs for the target disk drives. >>>>> This project proposes the new APIs for HBA drivers to >>>>> provide access to LEDs that it may control. >>>>> >>>>> 4.3 Proposal >>>>> >>>>> This fasttrack proposes a new ioctl to control FAIL and OK2RM LED >>>>> indicators via SGPIO interface of a Host Bus Adapter. >>>>> Controlling software (zfs, FMA or other entity with the adequate >>>>> permission) may use this interface to turn-on and turn-off FAULT >>>>> and OK2RM LED indicators using HBA SGPIO hardware interface. >>>>> If HBA does not support SGPIO interface, the ioctl would fail >>>>> with an appropriate error. >>>>> >>>>> The default LED state set by HBA (without an explicit command >>>>> to turn LED ON) would be OFF state. >>>>> >>>>> If the Service Processor controls FAULT LED via IPMI >>>>> interface, IPMI LED ON setting will override the HBA LED OFF >>>>> setting. >>>>> >>>>> 4.3.1 Considerations >>>>> >>>>> The DKIO type ioctl was considered and rejected, because it >>>>> requires a presence of a device target node. In case of a >>>>> failed device the target node for the device may not exists. >>>>> Furthermore, there is no path in scsi target driver and scsi >>>>> module to pass ioctls that are not processed by these modules >>>>> to the HBA drivers. >>>>> >>>>> The new DEVCTL type ioctl was selected because its use does >>>>> not depend on the presence of a device target node. The >>>>> new ioctl would have to be implemented by HBA drivers providing >>>>> SGPIO interface. >>>>> >>>>> An application will have to identify (using /device path) >>>>> and open HBA control node (using /device path), identify LED >>>>> type and number for this HBA using platform topo info, and >>>>> execute ioctl call. >>>>> >>>>> In the future, the task of deriving the control node path >>>>> could be delegated to libdevice libraray, when this interface >>>>> would be used on many platforms. >>>>> >>>>> The ioctl-specific structure is used to pass LED specification >>>>> to the HBA. The alternative would be to use nvlist. >>>>> 4.4 Requested Binding >>>>> >>>>> patch >>>>> >>>>> >>>>> 4.5 Details >>>>> >>>>> Exported Interface Classification Comments >>>>> --------------------------------------------------------------- >>>>> DEVCTL_SETLED | consolidation-private | Turn ON/OFF drive >>>>> | | indicator LED >>>>> | | >>>>> --------------------------------------------------------------- >>>>> DEVCTL_GETLED | consolidation-private | Get status of drive >>>>> | | indicator LED >>>>> | | >>>>> --------------------------------------------------------------- >>>>> struct dc_led_ctl | consolidation-private | pointer to this >>>>> | | structure passed >>>>> | | as ioctl argument >>>>> | | >>>>> --------------------------------------------------------------- >>>>> >>>>> SYNOPSIS: >>>>> >>>>> int ioctl(int fildes, int request, arg); >>>>> >>>>> >>>>> DESCRIPTION >>>>> >>>>> The fildes argument is an open file descriptor that refers >>>>> to a HBA control node. >>>>> >>>>> The request is DEVCTL_SETLED or DEVCTL_GETLED >>>>> >>>>> #define DEVCTL_SET_LED (DEVCTL_IOC | 39) >>>>> #define DEVCTL_GET_LEDSTATE (DEVCTL_IOC | 40) >>>>> >>>>> The argument is a pointer to dc_led_ctl structure: >>>>> >>>>> struct dc_led_ctl { >>>>> uint32_t led_number : 16; /* LED/device number */ >>>>> uint32_t led_ctl_active : 1; /* Control active */ >>>>> uint32_t led_type : 3; /* LED type */ >>>>> uint32_t led_state: 4; /* LED ON/OFF/Blink state */ >>>>> } >>>>> >>>>> /* Control active field *' >>>>> #define DCL_CNTRL_OFF 0 /* Control inactive */ >>>>> #define DCL_CNTRL_ON 1 /* Control active */ >>>>> >>>>> /* LED type field */ >>>>> #define DCL_TYPE_FAIL 1 /* FAIL LED type */ >>>>> #define DCL_TYPE_OK2RM 2 /* OK2RM LED type */ >>>>> >>>>> /* Led state field */ >>>>> #define DCL_STATE_OFF 0 /* LED state OFF */ >>>>> #define DCL_STATE_ON 1 /* LED state ON */ >>>>> #define DCL_STATE_SLOW_BLNK 2 /* LED Slow blink */ >>>>> #define DCL_STATE_FAST_BLNK 3 /* LED Fast blink */ >>>>> >>>>> led_num should be a value corresponding to the device number >>>>> to which the LED indicator is related. In practical terms >>>>> it is the LED/drive data position in the SGPIO's SDataOut >>>>> bit-stream where each drive is represented by 3 bits >>>>> (3-touple). >>>>> >>>>> The value range is 0 to n-1, where n is the number of >>>>> devices supported by HBA instance. >>>>> >>>>> The LED number should be available in platform topology >>>>> info. >>>>> >>>>> Note: For nVidia's MCP55 and IO55 chips and system using >>>>> nv_sata driver, LED number is the same as the disk number >>>>> in physical device path /devices/../disk@Y. >>>>> >>>>> For DEVCTL_SET_LED request: >>>>> >>>>> Caller has to set all fields of the dc_led_ctl structure. >>>>> If DCL_CNTRL_OFF is specified, all other parameters are >>>>> ignored and HBA should make SPGIO inactive (put SGPIO >>>>> in tri-state) for all devices controlled by the HBA. >>>>> Slow blink and fast blink rates are defined by Sun >>>>> document Sun_Microsystems_HDD_LED_Requirements_Rev_1_8. >>>>> If DCL_CNTRL_ON is specified, caller has to set all fields >>>>> of the dc_led_ctl structure. >>>>> >>>>> >>>>> For DEVCTL_GET_LEDSTATE request: >>>>> >>>>> Caller has to specify led_type and led_number in dc_led_ctl >>>>> structure. HBA will set led_ctl_active and led_state fields. >>>>> The LED state is not persistent over reboots. >>>>> If the SGPIO control interface was not explicitly enabled >>>>> via DEVCTL_SET_LED with DCL_CNTRL_ON request, the LED state >>>>> is DCL_STATE_OFF for every LED controlled by the HBA instance. >>>>> IOCTL may specify one LED type and device number at a >>>>> time. >>>>> Initial state of the SGPIO lines is tri-state (not driven >>>>> by the HBA). In such state the default LED setting as provided >>>>> by a backplane SGPIO target hardware determines the actual LED >>>>> state and such state is unknown to the HBA driver. >>>>> The default state of a LED indicator driven by HBA is OFF, >>>>> i.e. if the LED of any type is not explicitly specified to be >>>>> ON, the HBA should specify it state as OFF. >>>>> >>>>> If the system BIOS (or BIOS-extension) does not enable SGPIO >>>>> interface, an explicit ioctl has to be sent to eenable the >>>>> SGPIO interface even when only ACTIVITY LEDs are to be under >>>>> HBA control (normally not controlled via DEVCTL_SET_LED ioctl). >>>>> >>>>> Some HBAs (such as hardware RAID HBAs) may be actively control >>>>> FAIL and/or OK2RM LEDs. For these HBAs the initial state of >>>>> LEDs is HBA-specific and cntrl_active field of the dc_led_ctl >>>>> structure may be ignored by the HBA (ie. SGPIO interface cannot >>>>> be deactivated and/or tri-stated). Drivers for such HBAs >>>>> return DCL_CNTLR_ON sate in the cntrl_active field even >>>>> without an explicit request to activate the control. >>>>> >>>>> Implementation Note: >>>>> >>>>> The ioctl call should return immediately. In case of the >>>>> DEVCTL_SET_LED request the HBA driver should store the request >>>>> internally and execute it on a separate execution thread if it >>>>> cannot be executed immediately. >>>>> >>>>> The SATA HBA driver should maintain the state of LEDs for the >>>>> purpose of providing LEDs state in response to >>>>> DEVCTL_GET_LEDSTATE >>>>> request. >>>>> The LED control should not interfere with normal I/O activity >>>>> (if any) on the specified device. >>>>> >>>>> PRIV_ALL privilege is required to invoke DEVCTL_LED_CTL ioctl. >>>>> >>>>> >>>>> RETURN VALUES >>>>> >>>>> Upon successful completion the return value is 0. >>>>> Otherwise -1 is returned and errno is set to indicate an error. >>>>> >>>>> ERRORS >>>>> >>>>> The IOCTL would fail if: >>>>> >>>>> ENOSUP HBA does not support LED control >>>>> >>>>> ENXIO the specified drive LED number is not supported by >>>>> the controller and/or HBA driver. >>>>> >>>>> EINVAL the LED type or state is not supported >>>>> >>>>> ENOPERM no permision to execute this ioctl (not owner) >>>>> >>>>> >>>>> >>>>> 6. Resources and Schedule >>>>> 6.4. Steering Committee requested information >>>>> 6.4.1. Consolidation C-team Name: >>>>> ON >>>>> 6.5. ARC review type: Fasttrack >>>>> 6.6. ARC Exposure: open >>>>> >>>>> 6. Resources and Schedule >>>>> 6.4. Steering Committee requested information >>>>> 6.4.1. Consolidation C-team Name: >>>>> ON >>>>> 6.5. ARC review type: FastTrack >>>>> 6.6. ARC Exposure: closed >>>>> 6.6.1. Rationale: The Sun document describing LED behavior >>>>> is marked XXXfidential and is under NDA >>>>> >>>> >>> >> > From sacadmin Wed May 14 08:00:24 2008 Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01.SFBay.Sun.COM [129.145.155.118]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m4EF0OGV003930 for ; Wed, 14 May 2008 08:00:24 -0700 (PDT) Received: from sca-es-mail-2.sun.com (sca-es-mail-2.Sun.COM [192.18.43.133]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m4EF0OtE022979 for ; Wed, 14 May 2008 08:00:24 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m4EF0JP1024536 for ; Wed, 14 May 2008 08:00:19 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K0V0010137PMZ00@fe-sfbay-09.sun.com> (original mail from gdamore@sun.com) for PSARC@sac.sfbay.sun.com; Wed, 14 May 2008 08:00:19 -0700 (PDT) Received: from [192.168.251.106] ([76.174.83.55]) by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K0V009TR5O4ZO40@fe-sfbay-09.sun.com>; Wed, 14 May 2008 08:00:05 -0700 (PDT) Date: Wed, 14 May 2008 07:53:54 -0700 From: "Garrett D'Amore" Subject: Re: IOCTL interface for drive LED control [PSARC/2008/317 FastTrack timeout 05/20/2008] In-reply-to: <482ACA1D.3040100@Sun.COM> Sender: Garrett.Damore@sun.com To: Darren J Moffat Cc: pawelw , Joseph Kowalski , Scott Rotondo , Alan Perry , PSARC@sac.sfbay.sun.com, sata-dev@sun.com Message-id: <482AFD02.8090701@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200805130632.m4D6WFul004136@groomlake.sfbay.sun.com> <482991D6.7020004@Sun.COM> <4829C6B5.3060801@sun.com> <482A31F4.1080005@sun.com> <482A7C3F.5080005@sun.com> <482A7720.2040308@sun.com> <482A821F.9000209@sun.com> <482ACA1D.3040100@Sun.COM> User-Agent: Thunderbird 2.0.0.6 (X11/20071023) Status: RO Content-Length: 587 Darren J Moffat wrote: > pawelw wrote: >> Joseph Kowalski wrote: >>> >>> There seems to be enough changes (maybe at the code review level, >>> but whatever), that I'd like to request an updates specification. >>> Thanks,... >>> >>> Gee, isn't it good to see a couple of cases this week with real meat >>> rather than arguing about how much change should be done to some >>> FOSS? :-) >>> >>> - jek3 >> I am awaiting opinions about my proposed changes before consolidating >> them into the updated proposal. > > I'm very happy with the privileges change. > +1. :-) -- Garrett From sacadmin Thu May 15 03:36:46 2008 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m4FAakEa010905 for ; Thu, 15 May 2008 03:36:46 -0700 (PDT) Received: from sca-es-mail-1.sun.com (sca-es-mail-1.Sun.COM [192.18.43.132]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m4FAakK4035158 for ; Thu, 15 May 2008 03:36:46 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m4FAafHH025169 for ; Thu, 15 May 2008 03:36:41 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K0W00301O0NGW00@fe-sfbay-09.sun.com> (original mail from Pawel.Wojcik@Sun.COM) for PSARC@sac.sfbay.sun.com; Thu, 15 May 2008 03:36:41 -0700 (PDT) Received: from [129.158.219.241] by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K0W00CNOO51AV60@fe-sfbay-09.sun.com>; Thu, 15 May 2008 03:36:40 -0700 (PDT) Date: Thu, 15 May 2008 03:40:48 -0700 From: pawelw Subject: Re: IOCTL interface for drive LED control [PSARC/2008/317 FastTrack timeout 05/20/2008] In-reply-to: <482AFD02.8090701@sun.com> Sender: Pawel.Wojcik@Sun.COM To: PSARC@sac.sfbay.sun.com Cc: "Garrett D'Amore" , Darren J Moffat , Joseph Kowalski , Scott Rotondo , Alan Perry , sata-dev@Sun.COM Message-id: <482C1330.9070101@sun.com> MIME-version: 1.0 Content-type: multipart/mixed; boundary="Boundary_(ID_uxWpi417Ox+KBACjlG2bVQ)" References: <200805130632.m4D6WFul004136@groomlake.sfbay.sun.com> <482991D6.7020004@Sun.COM> <4829C6B5.3060801@sun.com> <482A31F4.1080005@sun.com> <482A7C3F.5080005@sun.com> <482A7720.2040308@sun.com> <482A821F.9000209@sun.com> <482ACA1D.3040100@Sun.COM> <482AFD02.8090701@sun.com> User-Agent: Thunderbird 2.0.0.14 (X11/20080421) Status: RO Content-Length: 19313 This is a multi-part message in MIME format. --Boundary_(ID_uxWpi417Ox+KBACjlG2bVQ) Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT Pleas find attached an updated proposal containing the reviewers suggestions. Per Garret, Darren and Scott requests, PRIV_ALL access privileges were removed. Per Darren concern, the case was converted to open by removing dependence on Sun XXXfidential doc. Per Garret request, the interface was modified to become more generic, so additional LED types may be added in the future for devctl nodes used for controlling LEDs other than drive LEDs. Note: because of concern about pre-defining LED types for unknown (at this time) LED types, only DCL_TYPE_DEVICE_FAIL and DCL_TYPE_DEVICE_OK2RM LED types are explicitly defined. The control structure fields have room to for additional type of LEDs when they would be needed. The diff from the previous version is also attached for reference. -Pawel --Boundary_(ID_uxWpi417Ox+KBACjlG2bVQ) Content-type: text/plain; name=led-control-proposal Content-transfer-encoding: 7BIT Content-disposition: inline; filename=led-control-proposal Template Version: @(#)sac_nextcase 1.64 07/13/07 SMI This information is Copyright 2008 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: IOCTL interface for drive LED control 1.2. Name of Document Author/Supplier: Author: Pawel Wojcik 1.3 Date of This Document: 15 May, 2008 2. References [1] SFF-8485 Specification for Serial GPIO (SGPIO) Bus 4. Technical Description 4.1 Overview Drive-related LED indicators may be controlled by various hardware entities. Generally, Serial General Purpose Input/Output (SGPIO) interface is used to control LED indicators on the backplane. SGPIO is a 4-signal (or 4-wire) bus used between a Host Bus Adapter (HBA) and a backplane. Out of the 4 signals, 3 are driven by the HBA or other hardware controller and 1 is driven by the backplane. The data bits exchanged are related to drives supported by the target and are typically related to specific lanes of the high-speed serial storage interface. SGPIO and the SGPIO spec is generally adopted and implemented in products from most major HBA and Storage Controller vendors such as LSI, Intel, Adaptec, Nvidia, Broadcom and PMC-Sierra. Most products shipping with support for SAS and SATA drives support this standard. The SGPIO specification is given the official name SFF-8485 and is maintained by the SFF Committee. 4.2 Problem Description On most Sun platforms FAULT (Amber) and OK2RM (Blue) LEDs are controlled by Service Processor (SP) via IPMI, while ACTIVITY (Green) LED is controlled by the disk HBA. On some other platforms that use hardware RAID HBA, the FAULT LED may be controlled by HBA along with the ACTIVITY LED. Dorado/Tucana platform employs SATA controller embedded in nVidia chipset, which provides SGPIO interface, that can be used to control all type of drive-related LEDs. There may be other platforms, employing either nVidia chipset or Intel's ICH hub that could use SGPIO to control drive-related LEDs. Currently there are no APIs that HBA drivers implement to provide control over the LEDs for the target disk drives. This project proposes the new APIs for HBA drivers to provide access to LEDs that it may control. 4.3 Proposal This fasttrack proposes a new ioctl to control FAIL and OK2RM LED indicators via SGPIO interface of a Host Bus Adapter. Controlling software (zfs, FMA or other entity with the adequate permission) may use this interface to turn-on and turn-off FAULT and OK2RM LED indicators using HBA SGPIO hardware interface. If HBA does not support SGPIO interface, the ioctl would fail with an appropriate error. The default LED state set by HBA (without an explicit command to turn LED ON) would be OFF state. If the Service Processor controls FAULT LED via IPMI interface, IPMI LED ON setting will override the HBA LED OFF setting. As the ioctl is generic in nature and directed to devctl nodes, additional LED types may be supported by the driver implementing this ioctl. 4.3.1 Considerations The DKIO type ioctl was considered and rejected, because it requires a presence of a device target node. In case of a failed device the target node for the device may not exists. Furthermore, there is no path in scsi target driver and scsi module to pass ioctls that are not processed by these modules to the HBA drivers. The new DEVCTL type ioctl was selected because its use does not depend on the presence of a device target node. The new ioctl would have to be implemented by HBA drivers providing SGPIO interface. An application will have to identify (using /device path) and open HBA control node (using /device path), identify LED type and number for this HBA using platform topo info, and execute ioctl call. In the future, the task of deriving the control node path could be delegated to libdevice libraray, when this interface would be used on many platforms. The ioctl-specific structure is used to pass LED specification to the HBA. The alternative would be to use nvlist. 4.4 Requested Binding patch 4.5 Details Exported Interface Classification Comments --------------------------------------------------------------- DEVCTL_SET_LED | consolidation-private | Turn ON/OFF | | indicator LED | | -------------------------------------------------------------- DEVCTL_GET_LED | consolidation-private | Get status of | | indicator LED | | --------------------------------------------------------------- DEVCTL_NUM_LEDS | consolidation-private | Get number of | | supported indicator | | LEDs --------------------------------------------------------------- struct dc_led_ctl | consolidation-private | pointer to this | | structure passed | | as ioctl argument | | --------------------------------------------------------------- SYNOPSIS: int ioctl(int fildes, int request, arg); DESCRIPTION The fildes argument is an open file descriptor that refers to a HBA control node. The request is DEVCTL_SET_LED, DEVCTL_GET_LED or DEVCTL_NUM_LEDS #define DEVCTL_SET_LED (DEVCTL_IOC | 39) #define DEVCTL_GET_LED (DEVCTL_IOC | 40) #define DEVCTL_NUM_LEDS (DEVCTL_IOC | 41) The argument is a pointer to dc_led_ctl structure: struct dc_led_ctl { uint32_t led_number : 16; /* LED/device number */ uint32_t led_ctl_active : 1; /* Control active */ uint32_t led_type : 9; /* LED type */ uint32_t led_state: 6; /* LED ON/OFF/Blink state */ } /* Control active field *' #define DCL_CNTRL_OFF 0 /* Control inactive */ #define DCL_CNTRL_ON 1 /* Control active */ /* LED type field */ #define DCL_TYPE_DEVICE_FAIL 1 /* Device FAIL LED type */ #define DCL_TYPE_DEVICE_OK2RM 2 /* Device OK2RM LED type */ /* Led state field */ #define DCL_STATE_OFF 0 /* LED state OFF */ #define DCL_STATE_ON 1 /* LED state ON */ #define DCL_STATE_SLOW_BLNK 2 /* LED Slow blink */ #define DCL_STATE_FAST_BLNK 4 /* LED Fast blink */ For DEVCTL_GET_LED and DEVCTL_SET_LED requests: led_num should be a value corresponding to the device number to which the LED indicator is related. For requesst is related to SGPIO-controlled LEDs it is the LED/drive data position in the SGPIO's SDataOut bit-stream where each drive is represented by 3 bits (3-touple). The value range is 0 to n-1, where n is the number of devices supported by HBA instance. The LED number should be available in platform topology info. Note: For nVidia's MCP55 and IO55 chips and system using nv_sata driver, LED number is the same as the disk number in physical device path /devices/../disk@Y. For requests related to other type of leds, the led_num value is specific for the devctl node supporting it. The request may specify only one LED type and number at a time. For DEVCTL_SET_LED request: Caller has to set all fields of the dc_led_ctl structure. If DCL_CNTRL_OFF is specified, all other parameters are ignored and HBA should make an LED interface inactive (in case of SGPIO, put SGPIO in tri-state) for all LEDs controlled by the HBA. If DCL_CNTRL_ON is specified, caller has to set all fields of the dc_led_ctl structure. Slow blink and fast blink rates are defined as follow: SLow Blinking - 1Hz 50% ON/OFF LED cycles Fast Blinking - 4Hz 50% ON/OFF LED cycles For DEVCTL_GET_LED request: Caller has to specify led_type and led_number in dc_led_ctl structure. HBA will set led_ctl_active and led_state fields. The LED state is not persistent over reboots. If the LED state was not explicitly changed via DEVCTL_SET_LED with DCL_CNTRL_ON request, the LED state is DCL_STATE_OFF for every LED controlled by the HBA instance. For DEVCTL_NUM_LEDS request: Following information is returned in dc_led_ctl structure: led_num --- number of supported LEDs For devctl nodes controlling device LEDs via SGPIO interface this is a number of LEDs tuples supported by the HBA instance. For other devctl nodes this number is devctl node specific. led_ctl_active --- non-zero value indicates that the LED control interface is active, ie. LEDs are controlled by this HBA instance. led_type --- indication of supporting device FAIL and OK2RM LEDs (0x3 constant for SGPIO-controlled device LEDS); led_state --- indication of supporting device LED lighting patterns, as defined by DCL_STATE_XXX ; For devctl nodes controlling SGPIO interface: Initial state of the SGPIO lines is tri-state (not driven by the HBA). In such state the default LED setting provided by a backplane SGPIO target hardware determines the actual LED state and such state is unknown to the HBA driver. The default state of a LED indicator driven by HBA is OFF, i.e. if the LED of any type is not explicitly specified to be ON, the HBA should specify it state as OFF. If the system BIOS (or BIOS-extension) does not enable SGPIO interface, an explicit ioctl has to be sent to enable the SGPIO interface even when only ACTIVITY LEDs are to be under HBA control (normally not controlled via DEVCTL_SET_LED ioctl). Some HBAs (such as hardware RAID HBAs) may be actively controlling FAIL and/or OK2RM LEDs. For these HBAs the initial state of LEDs is HBA-specific and led_ctl_active field of the dc_led_ctl structure may be ignored by the HBA (ie. SGPIO interface cannot be deactivated and/or tri-stated). Drivers for such HBAs return DCL_CNTLR_ON sate in the cntrl_active field even without an explicit prior request to activate the control. Implementation Note: The ioctl call should return immediately. In case of the DEVCTL_SET_LED request the HBA driver should store the request internally and execute it on a separate execution thread if it cannot be executed immediately. The driver implementing the LED control ioctl should maintain the state of LEDs for the life of the HBA instance. The LED control should not interfere with normal I/O activity (if any) on the specified device. RETURN VALUES Upon successful completion the return value is 0. Otherwise -1 is returned and errno is set to indicate an error. ERRORS The IOCTL would fail if: ENOSUP HBA does not support LED control ENXIO the specified drive LED number is not supported by the controller and/or HBA driver. EINVAL the LED type or state is not supported ENOPERM no permision to execute this ioctl (not owner) 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: Fasttrack 6.6. ARC Exposure: open --Boundary_(ID_uxWpi417Ox+KBACjlG2bVQ) Content-type: text/plain; name=proposal.diffs Content-transfer-encoding: 7BIT Content-disposition: inline; filename=proposal.diffs 11c11 < 5 May, 2008 --- > 15 May, 2008 15d14 < [2] Sun_Microsystems_HDD_LED_Requirements_Rev_1_8 71a71,74 > As the ioctl is generic in nature and directed to devctl nodes, > additional LED types may be supported by the driver implementing > this ioctl. > 106c109 < DEVCTL_SETLED | consolidation-private | Turn ON/OFF drive --- > DEVCTL_SET_LED | consolidation-private | Turn ON/OFF 109,110c112,113 < --------------------------------------------------------------- < DEVCTL_GETLED | consolidation-private | Get status of drive --- > -------------------------------------------------------------- > DEVCTL_GET_LED | consolidation-private | Get status of 113a117,120 > DEVCTL_NUM_LEDS | consolidation-private | Get number of > | | supported indicator > | | LEDs > --------------------------------------------------------------- 130c137 < The request is DEVCTL_SETLED or DEVCTL_GETLED --- > The request is DEVCTL_SET_LED, DEVCTL_GET_LED or DEVCTL_NUM_LEDS 133c140,141 < #define DEVCTL_GET_LEDSTATE (DEVCTL_IOC | 40) --- > #define DEVCTL_GET_LED (DEVCTL_IOC | 40) > #define DEVCTL_NUM_LEDS (DEVCTL_IOC | 41) 140,141c148,149 < uint32_t led_type : 3; /* LED type */ < uint32_t led_state: 4; /* LED ON/OFF/Blink state */ --- > uint32_t led_type : 9; /* LED type */ > uint32_t led_state: 6; /* LED ON/OFF/Blink state */ 149,150c157,158 < #define DCL_TYPE_FAIL 1 /* FAIL LED type */ < #define DCL_TYPE_OK2RM 2 /* OK2RM LED type */ --- > #define DCL_TYPE_DEVICE_FAIL 1 /* Device FAIL LED type */ > #define DCL_TYPE_DEVICE_OK2RM 2 /* Device OK2RM LED type */ 156c164,166 < #define DCL_STATE_FAST_BLNK 3 /* LED Fast blink */ --- > #define DCL_STATE_FAST_BLNK 4 /* LED Fast blink */ > > For DEVCTL_GET_LED and DEVCTL_SET_LED requests: 159c169,170 < to which the LED indicator is related. In practical terms --- > to which the LED indicator is related. > For requesst is related to SGPIO-controlled LEDs 170a182,187 > For requests related to other type of leds, the led_num > value is specific for the devctl node supporting it. > > The request may specify only one LED type and number at > a time. > 175,178c192,194 < ignored and HBA should make SPGIO inactive (put SGPIO < in tri-state) for all devices controlled by the HBA. < Slow blink and fast blink rates are defined by Sun < document Sun_Microsystems_HDD_LED_Requirements_Rev_1_8. --- > ignored and HBA should make an LED interface inactive > (in case of SGPIO, put SGPIO in tri-state) for all LEDs > controlled by the HBA. 181a198,200 > Slow blink and fast blink rates are defined as follow: > SLow Blinking - 1Hz 50% ON/OFF LED cycles > Fast Blinking - 4Hz 50% ON/OFF LED cycles 183c202 < For DEVCTL_GET_LEDSTATE request: --- > For DEVCTL_GET_LED request: 188,189c207,208 < If the SGPIO control interface was not explicitly enabled < via DEVCTL_SET_LED with DCL_CNTRL_ON request, the LED state --- > If the LED state was not explicitly changed via > DEVCTL_SET_LED with DCL_CNTRL_ON request, the LED state 191a211,229 > For DEVCTL_NUM_LEDS request: > > Following information is returned in dc_led_ctl structure: > led_num --- number of supported LEDs > For devctl nodes controlling device LEDs via > SGPIO interface this is a number of LEDs tuples > supported by the HBA instance. > For other devctl nodes this number is devctl node > specific. > led_ctl_active --- non-zero value indicates that the LED > control interface is active, ie. LEDs are > controlled by this HBA instance. > led_type --- indication of supporting device FAIL and OK2RM LEDs > (0x3 constant for SGPIO-controlled device LEDS); > led_state --- indication of supporting device LED lighting patterns, > as defined by DCL_STATE_XXX ; > > > For devctl nodes controlling SGPIO interface: 193d230 < IOCTL may specify one LED type and device number at a time. 195c232 < by the HBA). In such state the default LED setting as provided --- > by the HBA). In such state the default LED setting provided 203c240 < interface, an explicit ioctl has to be sent to eenable the --- > interface, an explicit ioctl has to be sent to enable the 207c244 < Some HBAs (such as hardware RAID HBAs) may be actively control --- > Some HBAs (such as hardware RAID HBAs) may be actively controlling 209c246 < LEDs is HBA-specific and cntrl_active field of the dc_led_ctl --- > LEDs is HBA-specific and led_ctl_active field of the dc_led_ctl 213c250 < without an explicit request to activate the control. --- > without an explicit prior request to activate the control. 221,223c258,259 < The SATA HBA driver should maintain the state of LEDs for the < purpose of providing LEDs state in response to DEVCTL_GET_LEDSTATE < request. --- > The driver implementing the LED control ioctl should maintain the state > of LEDs for the life of the HBA instance. 227,228d262 < PRIV_ALL privilege is required to invoke DEVCTL_LED_CTL ioctl. < 252,253c286 < 6.4.1. Consolidation C-team Name: < ON --- > 6.4.1. Consolidation C-team Name: ON --Boundary_(ID_uxWpi417Ox+KBACjlG2bVQ)-- From sacadmin Thu May 15 08:10:13 2008 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m4FFADD3015779 for ; Thu, 15 May 2008 08:10:13 -0700 (PDT) Received: from sca-es-mail-2.sun.com (sca-es-mail-2.Sun.COM [192.18.43.133]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m4FFADs3039865 for ; Thu, 15 May 2008 08:10:13 -0700 (PDT) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m4FFA85a003577 for ; Thu, 15 May 2008 08:10:08 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K0W00J01YIOCU00@fe-sfbay-10.sun.com> (original mail from gdamore@sun.com) for PSARC@sac.sfbay.sun.com; Thu, 15 May 2008 08:10:08 -0700 (PDT) Received: from [192.168.251.106] ([76.174.83.55]) by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K0X001ZM0SJATC0@fe-sfbay-10.sun.com>; Thu, 15 May 2008 08:09:55 -0700 (PDT) Date: Thu, 15 May 2008 08:03:42 -0700 From: "Garrett D'Amore" Subject: Re: IOCTL interface for drive LED control [PSARC/2008/317 FastTrack timeout 05/20/2008] In-reply-to: <482C1330.9070101@sun.com> Sender: Garrett.Damore@sun.com To: pawelw Cc: PSARC@sac.sfbay.sun.com, Darren J Moffat , Joseph Kowalski , Scott Rotondo , Alan Perry , sata-dev@sun.com Message-id: <482C50CE.6070907@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200805130632.m4D6WFul004136@groomlake.sfbay.sun.com> <482991D6.7020004@Sun.COM> <4829C6B5.3060801@sun.com> <482A31F4.1080005@sun.com> <482A7C3F.5080005@sun.com> <482A7720.2040308@sun.com> <482A821F.9000209@sun.com> <482ACA1D.3040100@Sun.COM> <482AFD02.8090701@sun.com> <482C1330.9070101@sun.com> User-Agent: Thunderbird 2.0.0.6 (X11/20071023) Status: RO Content-Length: 913 pawelw wrote: > Pleas find attached an updated proposal containing the reviewers > suggestions. > > Per Garret, Darren and Scott requests, PRIV_ALL access privileges were > removed. > Per Darren concern, the case was converted to open by removing > dependence on Sun XXXfidential doc. > Per Garret request, the interface was modified to become more generic, > so additional LED types may be added in the future for devctl nodes > used for controlling LEDs other than drive LEDs. > > Note: because of concern about pre-defining LED types for unknown (at > this time) LED types, only DCL_TYPE_DEVICE_FAIL and > DCL_TYPE_DEVICE_OK2RM LED types are explicitly defined. The control > structure fields have room to for additional type of LEDs when they > would be needed. > > The diff from the previous version is also attached for reference. > > -Pawel This version looks really good to me. -- Garrett From Pawel.Wojcik@Sun.COM Thu May 15 10:57:37 2008 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m4FHvakJ020460 for ; Thu, 15 May 2008 10:57:36 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m4FHvao6046692 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Thu, 15 May 2008 11:57:36 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0K0X008038JZYE00@nwk-avmta-1.sfbay.Sun.COM> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 15 May 2008 10:57:35 -0700 (PDT) Received: from sca-es-mail-2.sun.com ([192.18.43.133]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0K0X0046T8JYHS30@nwk-avmta-1.sfbay.Sun.COM> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 15 May 2008 10:57:34 -0700 (PDT) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m4FHvYXR025901 for ; Thu, 15 May 2008 10:57:34 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K0X00K018C2EH00@fe-sfbay-10.sun.com> (original mail from Pawel.Wojcik@Sun.COM) for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 15 May 2008 10:57:34 -0700 (PDT) Received: from [10.128.0.91] ([218.30.180.194]) by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K0X00JWT8JOUY20@fe-sfbay-10.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 15 May 2008 10:57:26 -0700 (PDT) Date: Thu, 15 May 2008 11:01:29 -0700 From: Pawel Wojcik Subject: : 2008/317 [IOCTL interface for drive LED control] In-reply-to: <200805151652.m4FGqemv008691@ivrel.sfbay.sun.com> Sender: Pawel.Wojcik@Sun.COM To: psarc-ext@Sun.COM Cc: Pawel.Wojcik@Sun.COM Message-id: <482C7A79.60704@sun.com> MIME-version: 1.0 Content-type: multipart/mixed; boundary="Boundary_(ID_o1YzqbDJNeSEG85HW4gFSQ)" X-PMX-Version: 5.4.1.325704 References: <200805151652.m4FGqemv008691@ivrel.sfbay.sun.com> User-Agent: Thunderbird 2.0.0.14 (Windows/20080421) Status: RO Content-Length: 12609 This is a multi-part message in MIME format. --Boundary_(ID_o1YzqbDJNeSEG85HW4gFSQ) Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT Proposal for IOCTL interface for drive LED control [PSARC/2008/317 FastTrack timeout 05/20/2008] -Pawel --Boundary_(ID_o1YzqbDJNeSEG85HW4gFSQ) Content-type: text/plain; name=led-control-proposal Content-transfer-encoding: 7BIT Content-disposition: inline; filename=led-control-proposal Template Version: @(#)sac_nextcase 1.64 07/13/07 SMI This information is Copyright 2008 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: IOCTL interface for drive LED control 1.2. Name of Document Author/Supplier: Author: Pawel Wojcik 1.3 Date of This Document: 15 May, 2008 2. References [1] SFF-8485 Specification for Serial GPIO (SGPIO) Bus 4. Technical Description 4.1 Overview Drive-related LED indicators may be controlled by various hardware entities. Generally, Serial General Purpose Input/Output (SGPIO) interface is used to control LED indicators on the backplane. SGPIO is a 4-signal (or 4-wire) bus used between a Host Bus Adapter (HBA) and a backplane. Out of the 4 signals, 3 are driven by the HBA or other hardware controller and 1 is driven by the backplane. The data bits exchanged are related to drives supported by the target and are typically related to specific lanes of the high-speed serial storage interface. SGPIO and the SGPIO spec is generally adopted and implemented in products from most major HBA and Storage Controller vendors such as LSI, Intel, Adaptec, Nvidia, Broadcom and PMC-Sierra. Most products shipping with support for SAS and SATA drives support this standard. The SGPIO specification is given the official name SFF-8485 and is maintained by the SFF Committee. 4.2 Problem Description On most Sun platforms FAULT (Amber) and OK2RM (Blue) LEDs are controlled by Service Processor (SP) via IPMI, while ACTIVITY (Green) LED is controlled by the disk HBA. On some other platforms that use hardware RAID HBA, the FAULT LED may be controlled by HBA along with the ACTIVITY LED. Dorado/Tucana platform employs SATA controller embedded in nVidia chipset, which provides SGPIO interface, that can be used to control all type of drive-related LEDs. There may be other platforms, employing either nVidia chipset or Intel's ICH hub that could use SGPIO to control drive-related LEDs. Currently there are no APIs that HBA drivers implement to provide control over the LEDs for the target disk drives. This project proposes the new APIs for HBA drivers to provide access to LEDs that it may control. 4.3 Proposal This fasttrack proposes a new ioctl to control FAIL and OK2RM LED indicators via SGPIO interface of a Host Bus Adapter. Controlling software (zfs, FMA or other entity with the adequate permission) may use this interface to turn-on and turn-off FAULT and OK2RM LED indicators using HBA SGPIO hardware interface. If HBA does not support SGPIO interface, the ioctl would fail with an appropriate error. The default LED state set by HBA (without an explicit command to turn LED ON) would be OFF state. If the Service Processor controls FAULT LED via IPMI interface, IPMI LED ON setting will override the HBA LED OFF setting. As the ioctl is generic in nature and directed to devctl nodes, additional LED types may be supported by the driver implementing this ioctl. 4.3.1 Considerations The DKIO type ioctl was considered and rejected, because it requires a presence of a device target node. In case of a failed device the target node for the device may not exists. Furthermore, there is no path in scsi target driver and scsi module to pass ioctls that are not processed by these modules to the HBA drivers. The new DEVCTL type ioctl was selected because its use does not depend on the presence of a device target node. The new ioctl would have to be implemented by HBA drivers providing SGPIO interface. An application will have to identify (using /device path) and open HBA control node (using /device path), identify LED type and number for this HBA using platform topo info, and execute ioctl call. In the future, the task of deriving the control node path could be delegated to libdevice libraray, when this interface would be used on many platforms. The ioctl-specific structure is used to pass LED specification to the HBA. The alternative would be to use nvlist. 4.4 Requested Binding patch 4.5 Details Exported Interface Classification Comments --------------------------------------------------------------- DEVCTL_SET_LED | consolidation-private | Turn ON/OFF | | indicator LED | | -------------------------------------------------------------- DEVCTL_GET_LED | consolidation-private | Get status of | | indicator LED | | --------------------------------------------------------------- DEVCTL_NUM_LEDS | consolidation-private | Get number of | | supported indicator | | LEDs --------------------------------------------------------------- struct dc_led_ctl | consolidation-private | pointer to this | | structure passed | | as ioctl argument | | --------------------------------------------------------------- SYNOPSIS: int ioctl(int fildes, int request, arg); DESCRIPTION The fildes argument is an open file descriptor that refers to a HBA control node. The request is DEVCTL_SET_LED, DEVCTL_GET_LED or DEVCTL_NUM_LEDS #define DEVCTL_SET_LED (DEVCTL_IOC | 39) #define DEVCTL_GET_LED (DEVCTL_IOC | 40) #define DEVCTL_NUM_LEDS (DEVCTL_IOC | 41) The argument is a pointer to dc_led_ctl structure: struct dc_led_ctl { uint32_t led_number : 16; /* LED/device number */ uint32_t led_ctl_active : 1; /* Control active */ uint32_t led_type : 9; /* LED type */ uint32_t led_state: 6; /* LED ON/OFF/Blink state */ } /* Control active field *' #define DCL_CNTRL_OFF 0 /* Control inactive */ #define DCL_CNTRL_ON 1 /* Control active */ /* LED type field */ #define DCL_TYPE_DEVICE_FAIL 1 /* Device FAIL LED type */ #define DCL_TYPE_DEVICE_OK2RM 2 /* Device OK2RM LED type */ /* Led state field */ #define DCL_STATE_OFF 0 /* LED state OFF */ #define DCL_STATE_ON 1 /* LED state ON */ #define DCL_STATE_SLOW_BLNK 2 /* LED Slow blink */ #define DCL_STATE_FAST_BLNK 4 /* LED Fast blink */ For DEVCTL_GET_LED and DEVCTL_SET_LED requests: led_num should be a value corresponding to the device number to which the LED indicator is related. For requesst is related to SGPIO-controlled LEDs it is the LED/drive data position in the SGPIO's SDataOut bit-stream where each drive is represented by 3 bits (3-touple). The value range is 0 to n-1, where n is the number of devices supported by HBA instance. The LED number should be available in platform topology info. Note: For nVidia's MCP55 and IO55 chips and system using nv_sata driver, LED number is the same as the disk number in physical device path /devices/../disk@Y. For requests related to other type of leds, the led_num value is specific for the devctl node supporting it. The request may specify only one LED type and number at a time. For DEVCTL_SET_LED request: Caller has to set all fields of the dc_led_ctl structure. If DCL_CNTRL_OFF is specified, all other parameters are ignored and HBA should make an LED interface inactive (in case of SGPIO, put SGPIO in tri-state) for all LEDs controlled by the HBA. If DCL_CNTRL_ON is specified, caller has to set all fields of the dc_led_ctl structure. Slow blink and fast blink rates are defined as follow: SLow Blinking - 1Hz 50% ON/OFF LED cycles Fast Blinking - 4Hz 50% ON/OFF LED cycles For DEVCTL_GET_LED request: Caller has to specify led_type and led_number in dc_led_ctl structure. HBA will set led_ctl_active and led_state fields. The LED state is not persistent over reboots. If the LED state was not explicitly changed via DEVCTL_SET_LED with DCL_CNTRL_ON request, the LED state is DCL_STATE_OFF for every LED controlled by the HBA instance. For DEVCTL_NUM_LEDS request: Following information is returned in dc_led_ctl structure: led_num --- number of supported LEDs For devctl nodes controlling device LEDs via SGPIO interface this is a number of LEDs tuples supported by the HBA instance. For other devctl nodes this number is devctl node specific. led_ctl_active --- non-zero value indicates that the LED control interface is active, ie. LEDs are controlled by this HBA instance. led_type --- indication of supporting device FAIL and OK2RM LEDs (0x3 constant for SGPIO-controlled device LEDS); led_state --- indication of supporting device LED lighting patterns, as defined by DCL_STATE_XXX ; For devctl nodes controlling SGPIO interface: Initial state of the SGPIO lines is tri-state (not driven by the HBA). In such state the default LED setting provided by a backplane SGPIO target hardware determines the actual LED state and such state is unknown to the HBA driver. The default state of a LED indicator driven by HBA is OFF, i.e. if the LED of any type is not explicitly specified to be ON, the HBA should specify it state as OFF. If the system BIOS (or BIOS-extension) does not enable SGPIO interface, an explicit ioctl has to be sent to enable the SGPIO interface even when only ACTIVITY LEDs are to be under HBA control (normally not controlled via DEVCTL_SET_LED ioctl). Some HBAs (such as hardware RAID HBAs) may be actively controlling FAIL and/or OK2RM LEDs. For these HBAs the initial state of LEDs is HBA-specific and led_ctl_active field of the dc_led_ctl structure may be ignored by the HBA (ie. SGPIO interface cannot be deactivated and/or tri-stated). Drivers for such HBAs return DCL_CNTLR_ON sate in the cntrl_active field even without an explicit prior request to activate the control. Implementation Note: The ioctl call should return immediately. In case of the DEVCTL_SET_LED request the HBA driver should store the request internally and execute it on a separate execution thread if it cannot be executed immediately. The driver implementing the LED control ioctl should maintain the state of LEDs for the life of the HBA instance. The LED control should not interfere with normal I/O activity (if any) on the specified device. RETURN VALUES Upon successful completion the return value is 0. Otherwise -1 is returned and errno is set to indicate an error. ERRORS The IOCTL would fail if: ENOSUP HBA does not support LED control ENXIO the specified drive LED number is not supported by the controller and/or HBA driver. EINVAL the LED type or state is not supported ENOPERM no permision to execute this ioctl (not owner) 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: Fasttrack 6.6. ARC Exposure: open --Boundary_(ID_o1YzqbDJNeSEG85HW4gFSQ)-- From edward.pilatowicz@sun.com Thu May 15 12:28:45 2008 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m4FJSiSE023322 for ; Thu, 15 May 2008 12:28:45 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id m4FJSb9w016133; Fri, 16 May 2008 03:28:40 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0K0X00I01CRSNC00@nwk-avmta-1.sfbay.Sun.COM>; Thu, 15 May 2008 12:28:40 -0700 (PDT) Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.59]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0K0X004RTCRRHS80@nwk-avmta-1.sfbay.Sun.COM>; Thu, 15 May 2008 12:28:39 -0700 (PDT) Received: from jurassic-x4600.sfbay.sun.com (localhost [127.0.0.1]) by jurassic-x4600.sfbay.sun.com (8.14.2+Sun/8.14.2) with ESMTP id m4FJSdSL120198 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 15 May 2008 12:28:39 -0700 (PDT) Received: (from edp@localhost) by jurassic-x4600.sfbay.sun.com (8.14.2+Sun/8.14.2/Submit) id m4FJSdDa120197; Thu, 15 May 2008 12:28:39 -0700 (PDT) Date: Thu, 15 May 2008 12:28:39 -0700 From: Edward Pilatowicz Subject: Re: : 2008/317 [IOCTL interface for drive LED control] In-reply-to: <482C7A79.60704@sun.com> To: Pawel Wojcik Cc: psarc-ext@sun.com Message-id: <20080515192839.GD984343@eng.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <200805151652.m4FGqemv008691@ivrel.sfbay.sun.com> <482C7A79.60704@sun.com> X-Authentication-warning: jurassic-x4600.sfbay.sun.com: edp set sender to edward.pilatowicz@sun.com using -f User-Agent: Mutt/1.4.2.1i Status: RO Content-Length: 343 given that these are new HBA interfaces, do any of these hbas support mpxio? would adding mpxio support to any of these hbas require changes to these interfaces? ed On Thu, May 15, 2008 at 11:01:29AM -0700, Pawel Wojcik wrote: > Proposal for IOCTL interface for drive LED control [PSARC/2008/317 > FastTrack timeout 05/20/2008] > > -Pawel From Pawel.Wojcik@Sun.COM Thu May 15 17:28:59 2008 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m4G0SwQm002946 for ; Thu, 15 May 2008 17:28:58 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id m4G0SuMU012140 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Fri, 16 May 2008 08:28:57 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0K0X00701QO8XJ00@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 15 May 2008 17:28:56 -0700 (PDT) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0K0X004DZQO7HS70@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 15 May 2008 17:28:56 -0700 (PDT) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m4G0StlN021185 for ; Thu, 15 May 2008 17:28:55 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K0X00D01QLNEL00@fe-sfbay-10.sun.com> (original mail from Pawel.Wojcik@Sun.COM) for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 15 May 2008 17:28:55 -0700 (PDT) Received: from [10.128.0.91] ([218.30.180.194]) by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K0X008I3QO529B0@fe-sfbay-10.sun.com>; Thu, 15 May 2008 17:28:55 -0700 (PDT) Date: Thu, 15 May 2008 17:33:02 -0700 From: Pawel Wojcik Subject: Re: : 2008/317 [IOCTL interface for drive LED control] In-reply-to: <20080515192839.GD984343@eng.sun.com> Sender: Pawel.Wojcik@Sun.COM To: Edward Pilatowicz Cc: PSARC-ext@Sun.COM Message-id: <482CD63E.8090008@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200805151652.m4FGqemv008691@ivrel.sfbay.sun.com> <482C7A79.60704@sun.com> <20080515192839.GD984343@eng.sun.com> User-Agent: Thunderbird 2.0.0.14 (Windows/20080421) Status: RO Content-Length: 552 Edward Pilatowicz wrote: > given that these are new HBA interfaces, do any of these hbas > support mpxio? would adding mpxio support to any of these hbas > require changes to these interfaces? > ed > > On Thu, May 15, 2008 at 11:01:29AM -0700, Pawel Wojcik wrote: > >> Proposal for IOCTL interface for drive LED control [PSARC/2008/317 >> FastTrack timeout 05/20/2008] >> >> -Pawel >> No, none of the HBA implemented mpxio and will include mpxio support any time soon. I do not believe that the interface would require any changes. -Pawel From Pawel.Wojcik@sun.com Thu May 15 17:36:12 2008 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m4G0aBhu003006 for ; Thu, 15 May 2008 17:36:11 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m4G0aAel026738 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Thu, 15 May 2008 18:36:11 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0K0X00801R099B00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 15 May 2008 17:36:09 -0700 (PDT) Received: from sca-es-mail-2.sun.com ([192.18.43.133]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0K0X0041UR08HS80@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 15 May 2008 17:36:08 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m4G0a8Bs011283 for ; Thu, 15 May 2008 17:36:08 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K0X00A01QYTB200@fe-sfbay-09.sun.com> (original mail from Pawel.Wojcik@Sun.COM) for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 15 May 2008 17:36:08 -0700 (PDT) Received: from [10.128.0.91] ([218.30.180.194]) by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K0X000YQR06BAC0@fe-sfbay-09.sun.com>; Thu, 15 May 2008 17:36:08 -0700 (PDT) Date: Thu, 15 May 2008 17:40:14 -0700 From: Pawel Wojcik Subject: Re: : 2008/317 [IOCTL interface for drive LED control] In-reply-to: <482CD63E.8090008@sun.com> Sender: Pawel.Wojcik@sun.com To: Edward Pilatowicz Cc: Pawel Wojcik , PSARC-ext@sun.com Message-id: <482CD7EE.5000801@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200805151652.m4FGqemv008691@ivrel.sfbay.sun.com> <482C7A79.60704@sun.com> <20080515192839.GD984343@eng.sun.com> <482CD63E.8090008@sun.com> User-Agent: Thunderbird 2.0.0.14 (Windows/20080421) Status: RO Content-Length: 687 Pawel Wojcik wrote: > Edward Pilatowicz wrote: >> given that these are new HBA interfaces, do any of these hbas >> support mpxio? would adding mpxio support to any of these hbas >> require changes to these interfaces? >> ed >> >> On Thu, May 15, 2008 at 11:01:29AM -0700, Pawel Wojcik wrote: >> >>> Proposal for IOCTL interface for drive LED control [PSARC/2008/317 >>> FastTrack timeout 05/20/2008] >>> >>> -Pawel >>> > No, none of the HBA implemented mpxio and will include mpxio support > any time soon. I do not believe that the interface would require any > changes. > -Pawel > To clarify, none of the HBA for which this interface is intended implement mpxio. -Pawel