On Solaris, userland applications wishing to interact with SAS Management Protocol (SMP) targets often use proprietary utilities that are particular to specific HBAs and/or targets. Alternately, they may use usmp(7I) directly[1]. Neither solution is optimal, as both entail significant duplication of effort, leading to fragmentation of code, nomenclature, and programming model.
As part of the Sun storage strategy, it is necessary to create a set of library interfaces that hide such implementation details and provide clean interfaces on which higher level applications can be developed. These interfaces will be similar to and employ the same model as those in libses and libscsi[2]. Initial consumers will include the unbundled CAM storage management utility, HBA test utilities, and software used in the bringup of new hardware, but the library is being delivered separately as part of this case to enable future work (both in management software and hardware development) to occur in parallel. These interfaces will initially be delivered as consolidation private, with the expectation that they will be made public after we have more experience using them in a wider variety of applications.
SMP is an industry standard protocol for managing the transport fabric in SAS-1, SAS-1.1, and SAS-2 storage subsystems. This protocol is described by the relevant SAS specification and forms an integral part of these standards. Refer to t10.org for background and copies of the specifications. SMP operates at the same level as, but separate from, the SSP and STP transports used to perform I/O operations on traditional SCSI or ATA targets. SMP frames are directed to specific target devices in the fabric, usually SAS expanders, identified by their SAS addresses. Most disk enclosures with SAS host connectivity being produced by Sun, and many by other vendors, as well as a growing number of servers, incorporate SAS expanders and thus provide one or more SMP initiators/targets.
SMP support was first introduced into Solaris in 2008 with the integration of the smp(7D) device driver and its associated userland interfaces[1]. These interfaces provide a simple mechanism for transmitting and receiving raw SMP frames to and from particular target devices. This case builds on but does not alter this work. The strategy for libsmp, described by this case, is to provide generic services that can sit on top of usmp and/or other alternate low-level transports. The project team requests that new consumers of SMP functionality build on this case, rather than using the usmp(7I) interfaces directly or embedding proprietary vendor-specific functionality. This will allow cross-platform use of common interfaces when libsmp is ported to other operating systems, and the pluggable backend interfaces (see below) will allow for the use of arbitrary transports under circumstances in which the generic Solaris transport is not available.
The SAS-2 specification provides for two types of SMP frames, requests and responses. Each has two fixed fields denoting the frame type and the function number, plus additional data that is dependent on the function and the specific version of the standard(s) that are supported by the initiator and/or target devices. The function number is analogous to page numbers defined by similar management specifications such as SES-2[3] and SPC-4[4] and defines the operation that is being requested or has been completed.
Frames are of variable length and range in size from 8 to 1028 bytes, beginning with a 4-byte header and ending with a 4-byte CRC checksum. The computation of the correct amount of valid data in a frame depends on many factors, including the frame type, function number, supported standard levels, and other data within the frame itself. One of the important services provided by libsmp is computation of these lengths and allocation and copying of data to construct and/or extract data from SMP frames.
libsmp also provides a set of interfaces to obtain a handle to a specific target, obtain basic information about that target, and execute an operation by transmitting a request frame and waiting for a response. This case covers operation only as an initiator; target-mode operation is outside its scope.
Finally, this case also incorporates a set of frame, function, and error type definitions for both SAS-2 and SFF-8485[5] SMP frame types and interfaces for extracting these values from SMP frames.
libsmp incorporates support for pluggable backends, responsible for transmitting and receiving raw SMP frames using arbitrary transports. This architecture makes it possible to port the library to legacy platforms such as Microsoft Windows and GNU/Linux that lack usmp(7I) functionality, as well as to use the library on Solaris with hardware or firmware versions that may not provide smp(7D) device enumeration and attachment or a fully generic SMP passthrough mechanism.
A target is specified by front-end consumers as a pair:
(backend, arbitrary data)
The backend is specified simply as the string name of a loadable module that will provide the transport functionality. libsmp does not interpret the data that defines the target; it is passed to the backend when a consumer requests a handle to a device.
The only backend included in this case uses usmp(7I) as a transport, and interprets this data as the name of an smp(7D) device node.
Like libses, libsmp provides a set of standards-based functionality for constructing and interpreting frames, but also allows for vendor- and product-specific overrides. This case does not cover any such nonstandard functionality. The standards-based modules support SAS-1.1 and SAS-2 functions only. All are supported completely and exactly as described by the relevant standard.
All the interfaces are Contracted Consolidation Private. While it would be possible to deliver these objects in a separate package, core FMA infrastructure such as libtopo is likely to depend on these libraries, so it seems of little use to deliver them separately. These devlierables will be part of the core Solaris packages (SUNWcsl, SUNWhea, SUNWarc). Future vendor specific plugins are expected to be delivered as part of a new SUNWsmpp package in order to facilitate creation of minimized systems (such as the miniroot) and easier patching.
/usr/lib/scsi
SCSI libraries and private commands
/usr/lib/scsi/libsmp.so.1
/usr/lib/scsi/$MACH64/libsmp.so.1
SMP core library
/usr/lib/scsi/plugins/smp
SMP plugin modules (backends and functions)
/usr/lib/scsi/plugins/smp
/usr/lib/scsi/plugins/smp/engines/usmp.so
/usr/lib/scsi/plugins/smp/engines/$MACH64/usmp.so
Default system backend engine (usmp).
/usr/lib/scsi/plugins/smp/framework
/usr/lib/scsi/plugins/smp/framework/sas2.so
/usr/lib/scsi/plugins/smp/framework/$MACH64/sas2.so
Standard function plugins (SAS-1.1, SAS-2)
/usr/lib/scsi/plugins/smp/vendor
Vendor-specific function plugins
/usr/include/scsi
/usr/include/scsi/libsmp.h
/usr/include/scsi/libsmp_plugin.h
libsmp headers
/usr/include/scsi/plugins/smp
/usr/include/scsi/plugins/ses/framework
/usr/include/scsi/plugins/ses/vendor
Plugin-specific headers. The primary purpose is to define any additional types or structures specific to the plugin's libsmp implementation; many standard definitions are shared with the kernel.