From sacadmin Thu Dec 14 13:17:34 2006 Received: from sac.sfbay.sun.com (localhost [127.0.0.1]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id kBELHYB4021472; Thu, 14 Dec 2006 13:17:34 -0800 (PST) Received: (from cth@localhost) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6/Submit) id kBELHY2f021468; Thu, 14 Dec 2006 13:17:34 -0800 (PST) Date: Thu, 14 Dec 2006 13:17:34 -0800 (PST) From: Christopher Horne Message-Id: <200612142117.kBELHY2f021468@sac.sfbay.sun.com> To: PSARC@sac.sfbay.sun.com Subject: ddi-forceload [PSARC/2006/708 Timeout: 01/04/2007] Status: RO Content-Length: 8432 I am sponsoring this following fasttrack for myself, requesting patch binding and a timeout of 01/04/2007. Use of the proposed 'ddi-forceload' property is by contract only, and the need to use the property will be obsoleted in nevada by new-boot_sparc (PSARC/2006/525). Subject: PSARC FastTrack [01/04/2007]: ddi-forceload Template Version: @(#)sac_nextcase %I% %G% SMI This information is Copyright 2006 Sun Microsystems, Inc. 1. Introduction 1.1. Project/Component Working Name: ddi-forceload 1.2. Name of Document Author/Supplier: Author: Chris Horne 1.3 Date of This Document: 14 December, 2006 4. Technical Description 4.1 Summary When PSARC/2005/050 [1] introduced ddi_modopen(), it mentioned the need for follow-on work to address preload requirements for consumers involved with machine boot. The same issue was also raised during PSARC/2001/229 (see 'ddi-forceattach-prerootmount' [2]). This fasttack proposes a new 'ddi-forceload' string array driver.conf(4) global property. The value of this property names additional kernel modules that a driver may require during boot. The 'ddi-forceload' property will be contract private with patch binding. The initial consumer in ON will be the scsi_vhci(7D), which is being split apart into multiple modules, so the majority of the source can be moved to OpenSolaris. We are also providing a contracts for use by NetworkStorage drivers (qlc(7D), emlxs(7D)). The expectation is that this 'ddi-forceload' will be obsoleted by PSARC/2006/525 (new-boot_sparc [3]). 4.2 Proposal When loading a driver module early in boot, the framework checks to see if it's driver.conf file uses 'ddi-forceload' and, in addition to the driver module, will also load the modules specified by the 'ddi-forceload' property. This proposed 'ddi-forceload' property is like a weak form of the ld(1) '-N' option (_depends_on). In order for modules to stay in memory after boot a stronger form of dependency, like a ddi_modopen(), must occur. The new behavior is similar to the 'forceload' command in system(4), but is specified in the driver.conf(4) file - making it easier for the driver developer to control. Initially, there are two types of consumers envisioned: type1: Drivers which, for the most part, could be released to opensolaris, except for the fact that they contain small sections of encumbered code. Currently, these small sections of encumbered code require the entire driver to be /closed/. For type1 consumers, if the encumbered source can be abstracted into interfaces, then the encumbered source can be delivered in a separate kernel module, allowing the majority of the source to be delivered to opensolaris. This technique works particularly well for device-specific functions where only the source associated with specific devices is encumbered. We would like to release the scsi_vhci/mpxio source to opensolaris. The scsi_vhci code currently has the concept of device specific 'failover ops', but these ops are currently all compiled into the monolithic scsi_vhci module. Since some of these 'failover ops' modules contain encumbered source (from EMC and LSI), all the source for the scsi_vhci module is currently in /closed/. At this point we want to make as much of the scsi_vhci code as possible available on opensolaris, so that work to evolve the interfaces to better support 3rd party storage can be done in a public forum. For scsi_vhci, the property would be added to the scsi_vhci.conf file. On sparc machines, until newboot-sparc is delivered, the framework would look at this property when the scsi_vhci driver was loaded. ddi-forceload = "misc/scsi_vhci/scsi_vhci_f_asym_sun", "misc/scsi_vhci/scsi_vhci_f_asym_lsi", "misc/scsi_vhci/scsi_vhci_f_asym_emc", "misc/scsi_vhci/scsi_vhci_f_sym_emc", "misc/scsi_vhci/scsi_vhci_f_sym", "misc/scsi_vhci/scsi_vhci_f_tpgs"; Also, use of the 'ddi-forceload' property by the scsi_vhci driver meets the immediate requirement of pushing scsi_vhci to opensolaris, however as the plugin architecture of scsi_vhci evolves a more sophisticated directory-content mechanism that allows the driver to either list (readdir(3C)), walk (nftw(3C)) or 'watch' the contents of the misc/scsi_vhci directory will be needed. NOTE: Currently, trying to implement a directory-content interface is hindered (on sparc) by the fact that the 'BOP' interfaces used during early boot don't support readdir/getdents - you need to already know the name of a module to load it. type2: Drivers that currently embed FW (firmware) inside their driver module. We currently have some huge driver kernel modules. For example the qlc(7D) driver module is 0.8MB, and the emlxs(7D) driver is 2.5MB. The reason these driver modules are so large is that the module contains FW images for all versions of HW (hardware) that the driver supports. On a given machine, only the FW images for installed HW are actually needed, and for some drivers the FW image is only needed during attach(9E) processing. To reduce the driver memory footprint, each FW image can be placed in a separate kernel module and the driver attach(9E) code can ddi_modopen() just the images for the HW installed. Once the module is opened, the driver can use ddi_modsym() to find the address and size information of the the FW. For drivers involved in kernel boot, all FW modules must be identified via 'ddi-forceload' so they get loaded into memory prior to lights-out (sparc). If the FW image is only needed during attach(9E) processing, the the driver can call ddi_modclose() from attach(9E). If the FW image is needed at other times, then the driver should call ddi_modclose() from detach(9E). After boot, FW images that are not open will get unloaded. The 'ddi-forceload' property works well for the type2 consumer because one vendor delivers both the driver module, the FW modules, and the driver.conf file. In addition, the names of the FW modules for a specific board are known by the driver - the driver code does not need to look a the ddi-forceload property itself, it can just ddi_modopen() the FW modules. Using qlc(7D) as an example, the following property would be added to the qlc.conf file to identify the FW images: ddi-forceload = "misc/qlc/fw_2200", "misc/qlc/fw_2300", "misc/qlc/fw_2400", "misc/qlc/fw_6322"; In the qlc source, each FW image is contained in a separate .c file with a huge initialized array. In addition to Makefile changes, a small amount of additional C code needs to added to convert these files into separate kernel modules. #include static struct modlmisc modlmisc = { &mod_miscops, "qlc/fw_2200 " qlc_fw2200cs2_VERSION_STRING }; static struct modlinkage modlinkage = {MODREV_1, (void *)&modlmisc}; int _init() {return (mod_install(&modlinkage));} int _fini() {return (mod_remove(&modlinkage));} int _info(struct modinfo *mip) {return (mod_info(&modlinkage, mip));} Representing FW data in a kernel module means, given current krtld, that we waste some disk space on a multi-architecture distribution like x86/amd64 because we must have a separate file for each architecture. An advantage of kernel modules is that they can be protected by elfsign(1). 4.3 Interface Table ---------------------------------------------------------------------------- Interface Level Comments ---------------------------------------------------------------------------- ddi-forceload Contract Private global driver.conf property 4.4 References [1] ddi_modopen dynamically access a loadable kernel module http://sac.sfbay/PSARC/2005/050 [2] Loading and Unloading of Device Drivers ('ddi-forceattach') http://sac.sfbay/PSARC/2001/229/mail [3] new-boot sparc http://sac.sfbay/PSARC/2006/525 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: FastTrack