Template Version: @(#)sac_nextcase 1.64 07/13/07 SMI This information is Copyright 2008 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: IOCTL interface for drive LED control 1.2. Name of Document Author/Supplier: Author: Pawel Wojcik 1.3 Date of This Document: 15 May, 2008 2. References [1] SFF-8485 Specification for Serial GPIO (SGPIO) Bus 4. Technical Description 4.1 Overview Drive-related LED indicators may be controlled by various hardware entities. Generally, Serial General Purpose Input/Output (SGPIO) interface is used to control LED indicators on the backplane. SGPIO is a 4-signal (or 4-wire) bus used between a Host Bus Adapter (HBA) and a backplane. Out of the 4 signals, 3 are driven by the HBA or other hardware controller and 1 is driven by the backplane. The data bits exchanged are related to drives supported by the target and are typically related to specific lanes of the high-speed serial storage interface. SGPIO and the SGPIO spec is generally adopted and implemented in products from most major HBA and Storage Controller vendors such as LSI, Intel, Adaptec, Nvidia, Broadcom and PMC-Sierra. Most products shipping with support for SAS and SATA drives support this standard. The SGPIO specification is given the official name SFF-8485 and is maintained by the SFF Committee. 4.2 Problem Description On most Sun platforms FAULT (Amber) and OK2RM (Blue) LEDs are controlled by Service Processor (SP) via IPMI, while ACTIVITY (Green) LED is controlled by the disk HBA. On some other platforms that use hardware RAID HBA, the FAULT LED may be controlled by HBA along with the ACTIVITY LED. Dorado/Tucana platform employs SATA controller embedded in nVidia chipset, which provides SGPIO interface, that can be used to control all type of drive-related LEDs. There may be other platforms, employing either nVidia chipset or Intel's ICH hub that could use SGPIO to control drive-related LEDs. Currently there are no APIs that HBA drivers implement to provide control over the LEDs for the target disk drives. This project proposes the new APIs for HBA drivers to provide access to LEDs that it may control. 4.3 Proposal This fasttrack proposes a new ioctl to control FAIL and OK2RM LED indicators via SGPIO interface of a Host Bus Adapter. Controlling software (zfs, FMA or other entity with the adequate permission) may use this interface to turn-on and turn-off FAULT and OK2RM LED indicators using HBA SGPIO hardware interface. If HBA does not support SGPIO interface, the ioctl would fail with an appropriate error. The default LED state set by HBA (without an explicit command to turn LED ON) would be OFF state. If the Service Processor controls FAULT LED via IPMI interface, IPMI LED ON setting will override the HBA LED OFF setting. As the ioctl is generic in nature and directed to devctl nodes, additional LED types may be supported by the driver implementing this ioctl. 4.3.1 Considerations The DKIO type ioctl was considered and rejected, because it requires a presence of a device target node. In case of a failed device the target node for the device may not exists. Furthermore, there is no path in scsi target driver and scsi module to pass ioctls that are not processed by these modules to the HBA drivers. The new DEVCTL type ioctl was selected because its use does not depend on the presence of a device target node. The new ioctl would have to be implemented by HBA drivers providing SGPIO interface. An application will have to identify (using /device path) and open HBA control node (using /device path), identify LED type and number for this HBA using platform topo info, and execute ioctl call. In the future, the task of deriving the control node path could be delegated to libdevice libraray, when this interface would be used on many platforms. The ioctl-specific structure is used to pass LED specification to the HBA. The alternative would be to use nvlist. 4.4 Requested Binding patch 4.5 Details Exported Interface Classification Comments --------------------------------------------------------------- DEVCTL_SET_LED | consolidation-private | Turn ON/OFF | | indicator LED | | -------------------------------------------------------------- DEVCTL_GET_LED | consolidation-private | Get status of | | indicator LED | | --------------------------------------------------------------- DEVCTL_NUM_LEDS | consolidation-private | Get number of | | supported indicator | | LEDs --------------------------------------------------------------- struct dc_led_ctl | consolidation-private | pointer to this | | structure passed | | as ioctl argument | | --------------------------------------------------------------- SYNOPSIS: int ioctl(int fildes, int request, arg); DESCRIPTION The fildes argument is an open file descriptor that refers to a HBA control node. The request is DEVCTL_SET_LED, DEVCTL_GET_LED or DEVCTL_NUM_LEDS #define DEVCTL_SET_LED (DEVCTL_IOC | 39) #define DEVCTL_GET_LED (DEVCTL_IOC | 40) #define DEVCTL_NUM_LEDS (DEVCTL_IOC | 41) The argument is a pointer to dc_led_ctl structure: struct dc_led_ctl { uint32_t led_number : 16; /* LED/device number */ uint32_t led_ctl_active : 1; /* Control active */ uint32_t led_type : 9; /* LED type */ uint32_t led_state: 6; /* LED ON/OFF/Blink state */ } /* Control active field */ #define DCL_CNTRL_OFF 0 /* Control inactive */ #define DCL_CNTRL_ON 1 /* Control active */ /* LED type field */ #define DCL_TYPE_DEVICE_FAIL 1 /* Device FAIL LED type */ #define DCL_TYPE_DEVICE_OK2RM 2 /* Device OK2RM LED type */ /* Led state field */ #define DCL_STATE_OFF 0 /* LED state OFF */ #define DCL_STATE_ON 1 /* LED state ON */ #define DCL_STATE_SLOW_BLNK 2 /* LED Slow blink */ #define DCL_STATE_FAST_BLNK 4 /* LED Fast blink */ For DEVCTL_GET_LED and DEVCTL_SET_LED requests: led_num should be a value corresponding to the device number to which the LED indicator is related. For requesst is related to SGPIO-controlled LEDs it is the LED/drive data position in the SGPIO's SDataOut bit-stream where each drive is represented by 3 bits (3-touple). The value range is 0 to n-1, where n is the number of devices supported by HBA instance. The LED number should be available in platform topology info. Note: For nVidia's MCP55 and IO55 chips and system using nv_sata driver, LED number is the same as the disk number in physical device path /devices/../disk@Y. For requests related to other type of leds, the led_num value is specific for the devctl node supporting it. The request may specify only one LED type and number at a time. For DEVCTL_SET_LED request: Caller has to set all fields of the dc_led_ctl structure. If DCL_CNTRL_OFF is specified, all other parameters are ignored and HBA should make an LED interface inactive (in case of SGPIO, put SGPIO in tri-state) for all LEDs controlled by the HBA. If DCL_CNTRL_ON is specified, caller has to set all fields of the dc_led_ctl structure. Slow blink and fast blink rates are defined as follow: SLow Blinking - 1Hz 50% ON/OFF LED cycles Fast Blinking - 4Hz 50% ON/OFF LED cycles For DEVCTL_GET_LED request: Caller has to specify led_type and led_number in dc_led_ctl structure. HBA will set led_ctl_active and led_state fields. The LED state is not persistent over reboots. If the LED state was not explicitly changed via DEVCTL_SET_LED with DCL_CNTRL_ON request, the LED state is DCL_STATE_OFF for every LED controlled by the HBA instance. For DEVCTL_NUM_LEDS request: Following information is returned in dc_led_ctl structure: led_num --- number of supported LEDs For devctl nodes controlling device LEDs via SGPIO interface this is a number of LEDs tuples supported by the HBA instance. For other devctl nodes this number is devctl node specific. led_ctl_active --- non-zero value indicates that the LED control interface is active, ie. LEDs are controlled by this HBA instance. led_type --- indication of supporting device FAIL and OK2RM LEDs (0x3 constant for SGPIO-controlled device LEDS); led_state --- indication of supporting device LED lighting patterns, as defined by DCL_STATE_XXX ; For devctl nodes controlling SGPIO interface: Initial state of the SGPIO lines is tri-state (not driven by the HBA). In such state the default LED setting provided by a backplane SGPIO target hardware determines the actual LED state and such state is unknown to the HBA driver. The default state of a LED indicator driven by HBA is OFF, i.e. if the LED of any type is not explicitly specified to be ON, the HBA should specify it state as OFF. If the system BIOS (or BIOS-extension) does not enable SGPIO interface, an explicit ioctl has to be sent to enable the SGPIO interface even when only ACTIVITY LEDs are to be under HBA control (normally not controlled via DEVCTL_SET_LED ioctl). Some HBAs (such as hardware RAID HBAs) may be actively controlling FAIL and/or OK2RM LEDs. For these HBAs the initial state of LEDs is HBA-specific and led_ctl_active field of the dc_led_ctl structure may be ignored by the HBA (ie. SGPIO interface cannot be deactivated and/or tri-stated). Drivers for such HBAs return DCL_CNTLR_ON sate in the cntrl_active field even without an explicit prior request to activate the control. Implementation Note: The ioctl call should return immediately. In case of the DEVCTL_SET_LED request the HBA driver should store the request internally and execute it on a separate execution thread if it cannot be executed immediately. The driver implementing the LED control ioctl should maintain the state of LEDs for the life of the HBA instance. The LED control should not interfere with normal I/O activity (if any) on the specified device. RETURN VALUES Upon successful completion the return value is 0. Otherwise -1 is returned and errno is set to indicate an error. ERRORS The IOCTL would fail if: ENOSUP HBA does not support LED control ENXIO the specified drive LED number is not supported by the controller and/or HBA driver. EINVAL the LED type or state is not supported ENOPERM no permision to execute this ioctl (not owner) 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: Fasttrack 6.6. ARC Exposure: open