SCSI SAS 1275 Binding Title: Version 1.7 1 Introduction 1.1 Overview and References This document describes the application of Open Firmware to the SCSI-3 protocol as implemented on serial attached SCSI (SAS). 2 References and Definitions 2.1 References [1] IEEE Standard 1275-1994 IEEE Standard for Boot (Initialization and Configuration) Firmware: Core Requirements and Practices [2] Device Support Extensions to IEEE 1274-1994, Revision 1.0 [3] FWARC 2005/751, SAS WWID determined from system MAC address [4] FWARC 2006/035, LSI SAS Controller Methods for Manufacturing and Service [5] PCI Bus Binding to: IEEE Std 1275-1994 Standard for Boot Firmware Rev. 2.1 http://playground.sun.com/1275/bindings/pci/pci2_1.pdf [6] FWARC 2003/637, PCI Express Bus Binding to IEEE 1275 [7] SCSI Architecture Model-4 http://t10.org/ftp/t10/drafts/sam4/sam4r13.pdf [8] FWARC 2008/070, OBP Deblocker Package Changes for Large Disks [9] Serial Attached SCSI - 1.1 (SAS-1.1) ANSI T10/1601-D http://www.t11.org/t10/drafts/sas1/sas1r10.pdf [10] Serial Attached SCSI - 2 (SAS-2) ANSI T10/1760-D http://www.t11.org/t10/drafts/sas2/sas2r14d.pdf [11] AT Attachment 8 - ATA/ATAPI Command Set (ATA8-ACS) ANSI T13-1699D http://www.t13.org/Documents/UploadedDocuments/docs2007/D1699r4a-ATA8-ACS.pdf [12] Serial ATA Revision 2.6 http://www.sata-io.org/download.asp?SerialATA_Revision_2_6_Gold.zip [13] T13/e05180r1 IEEE OUI tutorial for ATA http://www.t13.org/Documents/UploadedDocuments/docs2006/e05180r1-IEEE_OUI_tutorial_for_ATA.pdf 2.2 Definition of Terms bus node: an Open Firmware device node that represents a bus controller. In cases where a node represents the interface, or "bridge", between one bus and another, the node is both a bus node relative to the bus it controls and a child node of its parent bus. Note that an Open Firmware device node is not in itself a physical hardware device, rather it is a software abstraction that describes a hardware device. child node: A device node that represents a SAS "target" device. Device_Name: A 64-bit globally unique identifier in IEEE NAA form that describes a specific SAS device. This is distinct from a devices' SAS address. See IEEE OUI Tutorial [13]. logical unit: A target resident entity that implements a device model and executes SCSI commands sent by an application client. phy: The hardware that drives one end of a single SAS or SATA serial link. phy identifier: A numerical identifier in the range of 0 through 0x7f used by a SAS controller, expander or device to identify a specific phy. SAS address: 64-bit globally unique identifier. SAS address formats use the IEEE company identifier as the direct or indirect basis for their unique values. See IEEE OUI Tutorial [13] for format. 3 Bus Characteristics Serial Attached SCSI (SAS) is a single standard composed of many parts. It mandates support for both SAS devices and Serial AT Attached (SATA) devices. Basic addressing in a SAS fabric is through a SAS address. A SAS address is a 64-bit identifier unique within a SAS domain. Native SAS devices have at least one globally unique SAS address assigned to them, usually at the time of manufacture. Packets are routed through the SAS fabric based on either the full SAS address or a derived 16-bit hashed SAS address. Additionally, all SAS devices are required to provide a "Device_Name". The Device_Name is also a 64-bit globally unique identifier, but it is distinct from the device port's SAS address. The Device_Name accessible through SCSI VPD pages and is also provided as part of the device discovery for SAS 2. SATA devices are designed to operate in a point-to-point configuration. They may have no associated authoritative unique identification. If they are compliant to ACS-ATA8[11], they will have a 64-bit globally unique identifier that has the same format as the SAS globally unique identifier, When operating within a SAS domain, SATA devices are managed by a SATA Transport Protocol (STP) to SATA bridge which has a SAS address and forwards commands across the serial link to the SATA device. The STP/SATA bridge can reside within a SAS expander, SAS HBA, or the operating system driver. The STP/SATA bridge is responsible for providing a unique SAS address for the SATA device it is managing. In SAS-2[10] (but not SAS-1 [9]), the specification also mandates the assignment of the SATA WWID (as defined in ACS-ATA8[11]) as the Device_Name by the STP/SATA bridge (when in an expander) which retrieves it by issuing an ATA IDENTIFY command when the device first becomes known to the STP/SATA bridge. 3.1 Physical Address Formats and Representations This specification defines three different address formats, depending on the hardware being attached: SAS Address, Device_Name (WWN), and PHY Number. For any given combination of hardware there is one and only one canonical form of address, as described below. For any implementation compliant with this specification, the canonical form of the unit-address shall be as defined in this section, regardless of the input form of the unit address. The native address for a SAS domain is the SAS address, a 64-bit WWN with a 4-bit NAA field containing 5, a 24-bit vendor field, and a 36-bit serial number. All native SAS devices have a SAS address for each initiator and target port in a SAS fabric, based on intrinsic hardware-programmed bits. SATA devices do not have native SAS addresses. With SAS2.0 HBAs and STP/SATA bridges, coupled with disks compliant with ATA8-ACS[11], SATA devices export a Device_Name (WWN) in an identify packet in response to an ATA IDENTIFY command set by the STP/SATA bridge when the device is first inserted or powered on. For this combination, the Device_Name shall be the physical address. See ATA8-ACS [11] section 7.16.7.54, Identify Packet words 108-111. Under SAS1.1, the Device_Name (WWN) for SATA devices is not preserved from the initial Identify Packet behind an expander, so we shall use the SAS Address of the STP/SATA bridge itself. Lastly, SATA devices not compliant to ATA8-ACS [11] which are directly attached to the HBA (without any intervening STP/SATA bridge) do not have either a Device_Name or SAS Address. We shall identify them by the PHY number connecting to the disk. The following table summarizes the canonocal address form for any given device in a fabric for an implementation compliant with this specification. Note: "-" means any matching value in that column. / | Device Type (SAS vs. SATA) | | / | | SAS 2.0 compliant (all expanders and the HBA must be SAS 2.0 compliant | | to merit a "Y" here. Any expander or HBA not compliant follows a "N") | | | | | | / | | | Disk connected directly or through expander? | | | | | | / | | | | Disk compliant with ATA8-ACS [11]? | | | | / | | | | | | | | | | | SAS | Y/N | - | - | Use SAS Target Port address of device | | | | | | SATA | Y | - | Y | Use SATA Device_Name (WWN) | | | | | | SATA | Y/N | direct | N | Use PHY# as address | | | | | | SATA | Y | expander | N | Use SAS Address of STP/SATA Bridge | | | | | | SATA | N | expander | - | Use SAS Address of STP/SATA bridge | | | | | | SATA | N | direct | Y | Use SATA Device_Name (WWN) 3.1.1 Physical Address Formats 3.1.1.1 Numerical Representation The numerical representation of the physical address consists of four cells and has two possible encodings from three possible values. The first form describes a 64-bit SAS address, which can be used to describe any device within a SAS domain. This form is also used to describe a 64-bit Device_Name. The second form, identifying a PHY number, is only used for the case of SATA devices not compliant with ATA8-ACS [11] directly attached to an HBA. 3.1.1.1.1 SAS Address and SATA Device_Name Binary Representation This form describes device by its SAS address or SATA Device_Name (WWN). While different addresses, they both follow the same form of a 64-bit GUID with the high-order nibble being mandated to contain the value "5", followed by 24 bits of vendor ID and 36 bits of serial number. Bit#0 refers to the least significant bit. Bit # 33222222 22221111 11111100 00000000 10987654 32109876 54321098 76543210 sas.hi cell: 0101vvvv vvvvvvvv vvvvvvvv vvvvssss sas.lo cell: ssssssss ssssssss ssssssss ssssssss lun.hi cell: uuuuuuuu uuuuuuuu uuuuuuuu uuuuuuuu lun.lo cell: uuuuuuuu uuuuuuuu uuuuuuuu uuuuuuuu where: 0101 The value "5" in the high-order NAA nibble vv..vv 24-bit IEEE Organization ID ss..ss 36-bit unsigned device serial number uu..uu 64-bit unsigned number logical unit number Note: The concatenation of 0101 + vv..vv + ss..ss is either a SAS Address or a SATA Device_Name. The leading 5 is called out explicitly, as it is required, and this gives us the means of distinguishing this address from that of a PHY identifier. 3.1.1.1.2 Phy Identifier Binary Representation This representation identifies a phy on the SAS adapter. Bit # 33222222 22221111 11111100 00000000 10987654 32109876 54321098 76543210 sas.hi cell: 00000000 00000000 00000000 00000000 sas.lo cell: 00000000 00000000 00000000 iiiiiiii lun.hi cell: uuuuuuuu uuuuuuuu uuuuuuuu uuuuuuuu lun.lo cell: uuuuuuuu uuuuuuuu uuuuuuuu uuuuuuuu where: 00..00 Bits with the value zero ii..ii 8-bit unsigned number phy identifier in the range of 0..FE uu..uu 64-bit unsigned number logical unit number Note: The high order 56 bits sas.hi ... sas.lo are required to be zero in this form, allowing us to distinguish it from the earlier SAS Address or Device_Name representation. 3.1.1.2 Text Representation 3.1.1.2.1 SAS Address and SATA Device_Name Text Representation w5VVVVVVSSSSSSSS[,UUUUUUUUUUUUUUUU] where: w is the letter 'w' 5 is the digit '5' VVVVVV is an ASCII hexadecimal number in the range 0..FFFFFF (2^24-1). SSSSSSSSS is an ASCII hexadecimal number in the range 0...FFFFFFFFF (2^36-1) UUUUUUUUUUUUUUUU is an ASCII hexadecimal number in the range 0...FFFFFFFFFFFFFFFF (2^64-1). This portion of the address is optional and may be omitted if zero. The correspondence between the text representation and numerical representation is as follows: w5VVVVVVSSSSSSSS,UUUUUUUUUUUUUUUU corresponds to a Node name with numerical value: 0101b is a fixed value (NAA value of 5) vv..vv is a binary encoding of VVVVVV ss..ss is a binary encoding of SSSSSSSSS uu..uu is a binary encoding of UUUUUUUUUUUUUUUU Conversion of hexadecimal numbers from text representation to numerical representation shall be case-insensitive and leading zeros shall be permitted but not required. Conversion from numerical representation to text representation shall use the lower case forms of the hexadecimal digits in the range a...f, suppressing leading zeros. 3.1.1.2.2 PHY Number Text Representation PP[,UUUUUUUUUUUUUUUU] where: PP is an ASCII hexadecimal number in the range of 0...FE UUUUUUUUUUUUUUUU is an ASCII hexadecimal number in the range of 0...FFFFFFFFFFFFFFFF This portion of the address is optional and may be omitted if zero. The correspondence between the text representation and numerical representation is as follows: PP,UUUUUUUUUUUUUUUU corresponds to a Node name with numerical value: iiiiiiii is a binary encoding of PP uu..uu is a binary encoding of UUUUUUUUUUUUUUUU Conversion of hexadecimal numbers from text representation to numerical representation shall be case-insensitive and leading zeros shall be permitted but not required. Conversion from numerical representation to text representation shall use the lower case forms of the hexadecimal digits in the range a...f, suppressing leading zeros. 4 Bus Nodes 4.1 Properties Since a SAS controller is not a root nexus and can be attached to many different bus types, the controller node needs to provide any properties or methods defined by its parent bus node. This document defines properties and methods that are specific to SAS controller nodes and their children. 4.1.1 Open Firmware-defined Properties for Bus Nodes The following standard properties, as defined in the IEEE 1275[1], have special meaning or interpretation for SAS: "name" Type: Prop-encoded-string Value: "scsi" "device_type" Type: Prop-encoded-string Value: "scsi-sas". "#address-cells" Type: Prop-encoded-integer Contents: Standard property name to define package's address format Value: 4 "local-wwid" Type: Prop-encoded-array Value: 64-bit base WWID address assigned to the device represented by the package containing this property 4.2 Methods 4.2.1 Open Firmware-defined Methods for Bus Nodes A Standard Package implementing the "scsi-sas" device type shall implement the following standard methods as defined in IEEE 1275[1], with physical address representations as specified in 3.1 of this standard: open ( -- okay? ) Prepare this device for subsequent use. Typical behavior is to allocate any special resource requirements it needs, map the device into virtual address space, initialize the device and perform a brief "sanity test" to ensure that the device appears to be working correctly. Return true if this open method was successful, false if not. When a device's open method is called, that device's parent has already been opened (and so on, up to the root node, which has no parent), so this open method can call its parent's methods, for instance to create mappings within the parent's address space. close ( -- ) Close this previously opened device. Restore the device (which has been previously opened) to its "not-in-use" state. Typical behavior is to turn off the device, unmap it, and deallocate any resources that were allocated by open. Note: When closing an instance chain, a particular instance's close method is executed before its parents instances are closed, so the parent's methods can still be used during the execution of close. decode-unit ( addr len -- lun.lo lun.hi sas.low sas.hi ) Convert text unit-string to physical address,. Convert unit-string (addr, len), the text string representation, to lun.lo ... sas.hi, the numerical representation of a physical address within the address space defined by this device node. decode-unit is a static method. Implementation note: If an implementation chooses to accept alternate text representations for addresses (e.g., using PHY format for slot number or allowing Device_Name instead of SAS Address), this method must determine the correct canonical address for the device, and shall return that instead of the alternate form of address originally provided. Implementation note: If an illegal text address is specified, the numerical representation shall be the invalid value FFFFFFFF.FFFFFFFF.FFFFFFFF.FFFFFFFF. encode-unit ( lun.lo lun.hi sas.low sas.hi -- addr len ) Convert physical address to text unit-string. Convert lun.lo ... sas.hi, the numerical representation, to unit-string, the text string representation of a physical address within the address space defined by this device node. The number of cells in the list lun.lo ... sas.hi is defined by the value of the #address-cells property of this node, which is fixed to 4. encode-unit is a static method. Implementation note: If an illegal physical address is found (does not correspond to either of the two defined formats), the returned text string shall be "". Note additional requirements on encode-unit defined in section 3.1. dma-alloc ( size -- virt ) Allocate a memory region for later use. Allocate 'size' bytes of memory, contiguous within the direct-memory-access address space of the device bus, suitable for direct memory access by a "bus master" device. Return the virtual address 'virt'. That virtual address is suitable for CPU access to the allocated region, but, in general, dma-map-in must be used to convert it to an address suitable for direct memory access by the bus-master device. Allocate the memory according to the most stringent alignment requirements for the bus. See also: dma-map-in, dma-free If the requested operation cannot be performed, a throw shall be called with an appropriate error message, as with abort". NOTE: Out-of-memory conditions may be detected and handled properly in the code with ['] dma-alloc catch. Allocate a memory region for later use. dma-free ( virt size -- ) Free memory allocated with dma-alloc. Free 'size' bytes of memory at virtual address 'virt', previously allocated by the dma-alloc method. dma-map-in ( virt size cacheable? -- devaddr ) Convert virtual address to device bus DMA address. Convert the virtual address range , previously allocated by the dma-alloc method, into an address suitable for DMA on the device bus. Return this address 'devaddr'. dma-map-in can also be used to map application-supplied data buffers for DMA use, if possible on the bus. If the flag cacheable?, is nonzero, the caller wishes to make use of caches for the DMA buffer if they are available. Immediately after dma-map-in has been executed, the contents of the address range as seen by the processor (the processor's "view") is the same as the contents as seen by the device that performs the DMA (the device's "view"). After the DMA device has performed DMA or the processor has performed a write to the range in question, the contents of the address range as seen by the processor (the processor's "view") is not necessarily the same as the contents as seen by the device that performs the DMA (the device's "view"). The two views can be made consistent by executing dma-map-out. If the requested operation cannot be performed, a throw shall be called with an appropriate error message, as with abort". NOTE: Out-of-memory conditions may be detected and handled properly in the code with ['] dma-map-in catch. dma-map-out ( virt devaddr size -- ) Free DMA mapping set up with dma-map-in. Free the DMA mapping specified by , previously created with the dma-map-in method. This will also have the effect of flushing all caches associated with that mapping. 4.2.2 Bus-specific Methods for Bus Nodes A package implementing the "scsi-sas" device type may implement the following optional bus-specific methods: max-transfer ( -- max-len ) Return size of largest possible transfer. Return the size (max-len) in bytes of the largest single transfer that this device can perform, rounded down to a multiple of block-size. set-address ( lun.lo lun.hi sas.lo sas.hi -- ) Set the SAS target and logical unit to which subsequent commands apply. set-timeout ( msecs -- ) Sets the maximum length of time in milliseconds that the driver will wait for completion of a command. The default value of zero means to wait indefinitely. A hardware error result is reported for a command that times out. show-children ( -- ) Searches the SAS for attached targets and their associated logical units. Displays the information that the SCSI inquiry reports for those devices. diagnose ( -- error-code | 0 ) Performs a simple self-test for generic SCSI device. Perform an SCSI "test-unit-ready" command on the currently selected target and unit (see set-address). If that fails, display a message indicating the details of the failure and return a non-zero error code. Otherwise, perform a SCSI "send-diagnostic" command, returning zero if it succeeds or a non-zero error code if it fails. show-sas-wwid ( -- ) The word show-sas-wwid prints the 64-bit base SAS WWID of the LSI controller in human-readable format. The precise format of the display is left to the implementation[4]. 5 Child Nodes Properties and Methods Child nodes shall implement the standard Open Firmware properties corresponding to the device type. The child nodes of SAS controllers do not have any "reg" property. SAS controllers support the attachment of disks and CDROM/DVD drives. CDROM/DVD drives will be categorized as disk devices. 5.1 DISK DEVICES 5.1.1 PROPERTIES "name" Type: Prop-encoded-string Value: "disk" "device_type" Type: Prop-encoded-string Value: "block" "compatible" Type: Prop-encoded-string Value: "sd" The following property is defined in FWARC 2008/070, "OBP Deblocker Package Changes for Large Disks"[8]: "lba64" Type: Prop-encoded-array Value: Property contains no data. prop-encoded-array is NULL. 5.1.2 METHODS Child nodes shall implement the standard Open Firmware methods as modified by the "Open Firmware Recommended Practice, Device Support Extensions"[2]. Devices that can be used as boot devices share be of type "block" and shall define the following methods: The following methods are required by IEEE 1275 [1] to use the "disk-label" package: open ( -- okay? ) Prepare this device for subsequent use. Typical behavior is to allocate any special resource requirements it needs, map the device into virtual address space, initialize the device and perform a brief "sanity test" to ensure that the device appears to be working correctly. Return true if this open method was successful, false if not. When a device's open method is called, that device's parent has already been opened (and so on, up to the root node, which has no parent), so this open method can call its parent's methods, for instance to create mappings within the parent's address space. close ( -- ) Close this previously opened device Restore the device (which has been previously opened) to its "not-in-use" state. Typical behavior is to turn off the device, unmap it, and deallocate any resources that were allocated by open. Note: When closing an instance chain, a particular instance's close method is executed before its parents instances are closed, so the parent's methods can still be used during the execution of close. load ( addr -- len ) Load a client program from device to memory. Load a client program from the device into memory beginning at address addr, returning len, the size in bytes of the program that was loaded. If the device can contain several such programs, the instance-arguments (as returned by my-args) can be used in a device-dependent manner to select the particular program. Usage Restriction: The package containing the load method must be open before the load method is executed. offset ( d.rel -- d.abs ) Convert partition-relative disk position to absolute position. This is a method of the disk label support package. d.rel is a double- number disk position, expressed as the number of bytes from the beginning of the partition that was specified in the arguments when the support package was opened. d.abs is the corresponding double-number disk position, expressed as the number of bytes from the beginning of the disk. If no partition was specified when the support package was opened, a system-dependent default partition is used. If the disk label support package does not support disk partitioning, d.abs is equal to d.rel. The following methods are required by IEEE 1275 [1] to use the "deblocker" package: read ( addr len -- actual ) Read device into memory buffer; return actual byte count. Read at most len bytes from the device into the memory buffer beginning at addr. Return actual, the number of bytes actually read. If actual is zero or negative, the read operation did not succeed. Some standard device types impose additional requirements on their read methods; see the descriptions of various device types (e.g., "network" ) for more information. For some devices, the seek method sets the position for the next read. write ( addr len -- actual ) Write memory buffer to device; return actual byte count. Write len bytes to the device from the memory buffer beginning at addr. Return actual, the number of bytes actually written. If actual is less than len, the write did not succeed. For some devices, the seek method sets the position for the next write. seek ( pos.lo pos.hi -- status ) Set device position for next read or write. Set the device position at which the next read or write will take place. The position is specified by a pair of numbers pos.lo pos.hi, whose interpretation depends on the device type. Return -1 if the operation fails and either zero or one if it succeeds. NOTE- The return value one (1) is meant as a concession to existing practice. Programs that use the seek method should treat either of the status values 0 or 1 as an indication of success. block-size ( -- block-len ) Return "granularity" for accesses to this device. Return the "granularity" in bytes for accesses to this device. Perform all transfers to the device in multiples of this size. A returned value of 1 signifies that arbitrary transfer sizes are support (up to the maximum specified by max-transfer). max-transfer ( -- max-len ) Return size of largest possible transfer. Return the size in bytes of the largest single transfer that this device can perform, rounded down to a multiple of block-size. read-blocks ( addr block# #blocks -- #read ) Read #blocks, starting at block#, from device into memory. Read #blocks records of length block-size bytes from the device (starting at device block block#) into memory (starting at addr). Return #read, the number of blocks actually read. If the device is not capable of random access (e.g., a sequential access tape device), block# is ignored. write-blocks ( addr block# #blocks -- #written ) Write #blocks from memory into device, starting at block#. Write #blocks records of length block-size bytes from memory (starting at addr) to the device (starting at device block block#). Return #written, the number of blocks actually written. If the device is not capable of random access (e.g., a sequential access tape device), block# is ignored. The following methods were defined in FWARC 2008/070,"OBP Deblocker Package Changes for Large Disks"[8]: read-blocks64 ( addr d.block# #blocks -- #read ) Read #blocks, starting at d.block#, from device into memory. Read #blocks records of length block-size bytes from the device (starting at block d.block#) into memory (starting at addr.) Return #read, the number of blocks actually read. If the device is not capable of random access (e.g., a sequential access tape device) d.block# is ignored. write-blocks64 (addr d.block# #blocks -- #written ) Write #blocks from memory into device, starting at d.block#. Write #blocks records of length block-size bytes from memory (starting at addr) to the device (starting at block number d.block#.) Return #written, the number of blocks actually written. If the device is not capable of random access (e.g., a sequential access tape device), d.block# is ignored. The following methods were defined in a proposal titled: "Additional requirements for SCSI devices": The "disk-label" standard support package and packages of device type "block" and "byte" shall implement the following method: size ( -- d ) Return as a double number "d", the number of bytes of storage associated with the device or instance. If the size cannot be determined, return the double number -1. Packages of device type "block" and "byte" shall implement the following methods: #blocks ( -- u ) Return as an unsigned number "u", the number of blocks of storage associated with the device or instance, where "block" is a unit of storage consisting of the number of bytes returned by the package's "block-size" method. If the size cannot be determined, or if the number of blocks exceeds the range of an unsigned number, return the maximum unsigned integer (which because of the Open Firmware's assumption of two's complement arithmetic is equivalent to the signed number -1). The "disk-label" standard support package and packages of device type "block" shall implement the following methods: offset-low ( -- u ) Return the least significant cell of the double number denoting the beginning offset of the disk partition that was specified when the "disk-label" support package was opened. In general, the offset is obtained by executing the offset method of the "disk-label" support package with an argument of zero. It is permissible for the disk package to execute the "disk-label" support package's offset method once after opening that support package, storing for later use. offset-high ( -- u ) Return the most significant cell of the double number denoting the beginning offset of the disk partition that was specified when the "disk-label" support package was opened. In general, the offset is obtained by executing the offset method of the "disk-label" support package with an argument of zero. It is permissible for the disk package to execute the "disk-label" support package's offset method once after opening that support package, storing for later use. The following method was defined in FWARC 2008/070, "OBP Deblocker Package Changes for Large Disks"[8], and shall implemented in packages of device type "block": #blocks64 ( -- d.blocks ) Return the size of the device in blocks. Return, as a double number "d.blocks", the number of blocks of storage associated with the device or instance, where a "block" is a unit of storage consisting of the number of bytes returned by the package's "block-size" method. If the size cannot be determined, or if the number of blocks exceeds the range of a double number, return the double number -1.