Title ----- Pluggable fwflash(1M). Problem ------- PSARC/2005/126 introduced the fwflash(1M) Solaris firmware flashing utility, and requested that fwflash(1M) become the preferred interface for firmware flashing tools within Solaris. The initial release of fwflash(1M) was targeted at InfiniBand (IB) adapters attached via the tavor(7D) driver. Sun's Systems group has requested that fwflash(1M) be extended to provide support for devices attached via ses(7D) - SCSI Enclosure Services. This fasttrack addresses that request and in the process extends fwflash(1M) to also provide a generic, pluggable interface for other devices for which a firmware update mechanism might be required in the future. Further, we separate out the firmware image verification process into a separate plugin. The reason for this is that we anticipate shipping devices which attach to the host using the same driver, but which have different identification and firmware verification requires. By keeping the identifier separate from the verifier we enable a decrease in the sustaining effort over time. Separation also allows us to make most of the new fwflash(1M) Open, and leave CDA- or NDA-covered knowledge required in the closed part of the OpenSolaris source tree. The existing InfiniBand functionality will be re-factored to work within the proposed plugin framework. Binding ------- We request a Patch binding for this RFE (6616085) since the business requirement is for integration into Solaris 10 Update 6. Exposure -------- This is an Open fasttrack. Interfaces ---------- We propose the following changes to the command line syntax of fwflash(1M): 1. Remove the "device number" option to specify a device to be flashed, instead requiring the user to specify a device path. The option of using a "device number" to specify a device implicitly assumes that the system configuration is static between invocations of fwflash(1M). This assumption is not necessarily valid in a system which supports hotplug operation. (This change in the command line options was suggested by Garrett D'Amore during pre-submission review and is one that I'm more than happy to accept). 2. Add the ability to push multiple firmware images onto a single device. If -f is the last option specified on the command line, then all following arguments will be deemed to be filenames for separate firmware images. While it is anticipated that the primary use case for multiple firmware images will be in manufacturing environments on a pre-release basis, it is also possible that devices such as WiFi devices might also wish to use these flags. For the plugin interface, we propose the following structures. Firstly, the verification plugin: struct vrfyplugin { /* * fully-qualified filename in /usr/lib/fwflash/verify, * made up of [drivername]-[vendorname].so * * eg /usr/lib/fwflash/verify/ses-SUN.so * is the verification plugin for ses-attached devices which * have a vendorname of "SUN". */ char *filename; /* * The vendor name, such as "SUN" or "MELLANOX" */ char *vendor; /* * An opaque handle for dlopen()/dlclose() to use. */ void *handle; /* * Firmware image size in bytes, as reported by * stat(). */ unsigned int imgsize; /* * Flashable devices frequently have different buffers * to use for different image types. We track the buffer * required for this particular image with this variable. * * Once the verifier has figured out what sort of image * it's been passed, it will know what value to use for * this variable. */ unsigned int flashbuf; /* * Points to the entire firmware image in memory. * We do this so we can avoid multiple open()/close() * operations, and to make it easier for checksum * calculations. */ int *fwimage; /* * The verification function entry point. The code * in fwflash.c calls this function to verify that * the nominated firmware image file is valid for * the selected devicenode. * * Note that if the verification fails, the image * does _not_ get force-flashed to the device. * * All verification plugins must support this operation. */ int (*vendorvrfy)(struct devicelist *devicenode); }; Verification plugins are delivered in /usr/lib/fwflash/verify. There is only one verification plugin loaded for any flash update operation. The verification plugin must be written so that it can correctly identify different levels of firmware for a device (eg primary, secondary, tertiary) and verify each individually. In the case of the user requesting a firmware update for multiple images, these are verified serially, and then flashed to the device. To provide the common code with a means of both verifying and flashing the firmware, we pass that information using struct fwfile { /* * The fully qualified filename. No default location for * for the firmware image file is mandated. */ char *filename; /* Pointer to the identification plugin required */ struct fw_plugin *plugin; /* pointer to the identification summary structure */ struct vpr *ident; }; The struct vpr allows us to encapsulate device information such as the Vendor ID, Product ID and Revision ID returned from a SCSI INQUIRY(6) command, and additional information such as might be required for a tavor- attached (IB) device. If any of the fields are space-padded at the end, then the common code will do the work to strip those spaces characters off the end of the string before making use of the data. struct vpr { /* vendor ID, eg "HITACHI " */ char *vid; /* product ID, eg "DK32EJ36NSUN36G " */ char *pid; /* revision, eg "PQ08" */ char *revid; /* * Additional, encapsulated identifying information. * This pointer allows us to add details such as the * IB hba sector size, which command set should be * used or a part number. */ void *encap_ident; }; For the device identification and firmware update plugin: struct fw_plugin { /* * An opaque handle for dlopen()/dlclose() to use. */ void *handle; /* * fully-qualified filename in /usr/lib/fwflash/identify * made up of [drivername].so * * eg /usr/lib/fwflash/identify/ses.so * is the identification plugin for devices attached to * the host using the ses(7D) driver. */ char *filename; /* * The driver name that this plugin will search for in * the device tree snapshot using di_drv_first_node(3DEVINFO) * and di_drv_next_node(3DEVINFO). */ char *drvname; /* "ses" or "tavor" or .... */ /* * Function entry point to support the command-line "-r" * option - read image from device to persistent storage. * * Not all plugins and devices will support this operation. */ int (*fw_readfw)(struct devicelist *device, char *filename); /* * Function entry point to support the command-line "-f" * option - writes from persistent storage to device * * All identification plugins must support this operation. */ int (*fw_writefw)(struct devicelist *device, char *filename); /* * Function entry point used to build the list of valid, flashable * devices attached to the system using the loadable module drvname. * (Not all devices attached using drvname will be valid for this * plugin to report. * * start allows us to display flashable devices attached with * different drivers and provide the user with a visual clue * that these devices are different to others that are detected. * * All identification plugins must support this operation. */ int (*fw_identify)(int start); /* * Function entry point to support the command-line "-l" * option - list/report flashable devices attached to the system. * * All identification plugins must support this operation. */ int (*fw_devinfo)(struct devicelist *thisdev); }; Device identification and firmware update plugins are delivered in /usr/lib/fwflash/identify. Once the identification plugin has found an acceptable device, it adds a record to the global list fw_devices: struct devicelist { /* * fully qualified pathname, with /devices/.... prefix */ char *access_devname; /* * Which drivername did we find this device attached with * in our device tree walk? Eg, ses or tavor or sgen... */ char *drvname; /* * What class of device is this? For tavor-attached devices, * we set this to "IB". For other devices, unless there is * a common name to use, just make this the same as the * drvname field. */ char *classname; /* pointer to the VPR structure */ struct vpr *ident; /* * In the original fwflash(1M), it was possible to select a * device for flashing by using an index number called a * dev_num. We retain that concept for pluggable fwflash, with * the following change - whenever our identification plugin has * finished and found at least one acceptable device, we bump the * index number by 100. This provides the user with another key * to distinguish the desired device from a potentially very large * list of similar-looking devices. */ unsigned int index; /* * Contains SAS or FC Port-WWNs, or IB GUIDS. Both SAS and FC only * need one entry in this array since they really only have one * address which we should track. IB devices can have 4 GUIDs * (System Image, Node Image, Port 1 and Port 2). */ char *addresses[4]; /* * Pointer to the plugin needed to flash this device, and * to use for printing appropriate device-specific information * as required by the "-l" option to fwflash(1M). */ struct fw_plugin *plugin; /* Next entry in the list */ TAILQ_ENTRY(devicelist) nextdev; }; As a way of assisting the common code to easily find an appropriate plugin, we also have a pluginlist structure: struct pluginlist { /* * fully qualified filename in /usr/lib/fwflash/identify * made up of fwflash-[drivername].so * * eg /usr/lib/fwflash/identify/ses.so * is the identification plugin for devices attached to * the host using the ses(7D) driver. */ char *filename; /* * The driver name that this plugin will search for in * the device tree snapshot using di_drv_first_node(3DEVINFO) * and di_drv_next_node(3DEVINFO). */ char *drvname; /* * pointer to the actual plugin, so we can access its * function entry points */ struct fw_plugin *plugin; /* pointer to the next element in the list */ TAILQ_ENTRY(pluginlist) nextplugin; }; We do not mandate a specific filesystem location for firmware images. In order for the system to add a valid flashable device to the list of devices, the TAILQ family of macros from are used. Each device is added to the end of the global list using TAILQ_INSERT_TAIL(). Once the global list of valid devices has been built, we use the TAILQ_FOREACH() macro to access a specific device record, check whether the device matches our criteria (eg, device class, or /devices path), and then invoke the desired function entry point. We initialise the global variables pluginlist and devicelist as follows: TAILQ_HEAD(PLUGINLIST, pluginlist); TAILQ_HEAD(DEVICELIST, devicelist); struct PLUGINLIST *fw_pluginlist; struct DEVICELIST *fw_devices; The TAILQ_HEAD() macros wrap our variable structure (pluginlist, devicelist) and transform them into a list. See for more details, or the OpenBSD manpage for QUEUE(3) at http://www.openbsd.org/cgi-bin/man.cgi?query=queue&manpath=OpenBSD+Current&format=html (CR 6659566 has been logged requesting a Solaris manpage be delivered). Each function entry point should use the defined success or failure return codes of FWFLASH_SUCCESS FWFLASH_FAILURE Identification mechanisms ------------------------- For InfiniBand devices, the identification method has not changed, it has merely been refactored from the existing code into the new plugin architecture. For ses(7D)-attached devices, we walk the devinfo tree (the common code holds the devinfo root node pointer as di_node_t rootnode) looking for devices attached using this driver. If we find a ses(7D) device, we query using a simple uscsi(7I) GET DIAGNOSTIC(6) command to see whether the devices supports the Download Microcode Diagnostic Control Page (0x0e). If the device supports that page, then we attach the device to the global devicelist and continue our search of the device tree. If the device does not support that control page then we resume our search through the device tree for nodes which are acceptable. Stability classifications ------------------------- /usr/lib/fwflash Committed /usr/lib/fwflash/identify Committed /usr/lib/fwflash/verify Committed struct fw_plugin Committed struct vrfyplugin Committed struct pluginlist Committed struct vpr Committed struct fwflash Committed struct devicelist Committed FWFLASH_SUCCESS Committed FWFLASH_FAILURE Committed Manpage changes --------------- An updated manpage reflecting the more generic nature of fwflash(1M) will be submitted to the case log. This will mention the as-delivered list of supported device identification plugins (presently IB and ses only). The manpage will also incorporate instructions on how to provide multiple firmware images as command line arguments, as well as the device specifier that can be passed with the -d option. The "-l" (list) option retains the same format as PSARC/2005/126, though naturally each identification plugin will have its own mandatory information. The plugin for ses(7D)-attached devices will report device information in the following fashion: +----------------------------------------------------------+ | # /usr/sbin/fwflash -l | | List of available devices: | | Device[0] /devices/pci@7c0/pci@0/pci@2/scsi@0/ses@4,0:0 | | Class [ses] | | Target port WWN : 508002000031ad7d | | Vendor : LSILOGIC | | Product : SASX36 A.0 | | Firmware revision: 2020 | | | | Device[1] /devices/pci@7c0/pci@0/pci@2/scsi@0/ses@6,0:0 | | Class [ses] | | Target port WWN : 50800200006f517d | | Vendor : LSILOGIC | | Product : SASX28 A.0 | | Firmware revision: 2020 | +----------------------------------------------------------+ The above example will be added to the manpage examples section. An example of multiple image flashing will also be added to the manpage. References ---------- PSARC/2005/126 InfiniBand Flash Update Tool (ibflash) PSARC/2002/539 Tavor IB-HCA Driver CR 6616085 fwflash should be generic and pluggable