PSARC Questions Version 1.22 1. What is the proposal being presented for review? * Give an overview of the project and its phase(s). The main goal of this project is to deliver a generic common hotplug framework to Solaris users, a foundation which can support the hotplug functionality for any hotpluggable bus and also, to support device migration functionality in virtualization environment through virtual hotplug. This new Solaris hotplug framework is a state machine based, which includes generic hotplug states and provides various generic common hotplug interfaces, which includes: o Management Interfaces Generic or Bus specific hotplug admin applications including GUIs can be written using these interfaces. o Hotplug Event Interfaces Hotplug-aware users such as drivers, kernel modules and hotplug aware applications can receive both synchronous and asynchronous hotplug events. o Device Driver Interfaces Device drivers can work with the hotplug framework to participate in hotplug activities such as surprise removal, hot replacement, dynamic resource balancing, error handling. o Hotplug Controller Driver (HPC) Interfaces HPC drivers will communicate with the hotplug framework through these interfaces. New HPC driver can be written easily for any hotpluggable bus. This project will be delivered in multiple phases and these phases are: Phase-I: - Basic hotplug framework fratures and interfaces - Virtual hotplug support for PCIe devices - Physical hotplug support for PCIe (Native, ACPI) and PCI (SHPC based) devices. - New hotplug CLI (including daemon and library) and a new cfgadm plugin to support cfgadm (1M) to work with the new hotplug framework. Phase-II: - Hotplug events through Device Contract and LDI frameworks - New hotplug DDI interfaces for device drivers - New PCI resource allocator and a common PCI Configurator with support for Dynamic resource re-balancing - Support hotplug FMA for PCIe devices - RBAC support for new hotplug CLI Phase-III: - Support for Express Card - Support for GUI admin tool * Describe the exposure (OpenSolaris), scope and type of review desired (overview, full case, etc.) This is a full case, with the first phase of the project tracked by PSARC 2008/181. The subsequent components of the project will be raised in future cases with this case as a back drop. Further this is a full case with open exposure. * Indicate the release binding requested by the project team. See: http://www.opensolaris.org/os/community/arc/policies/release-taxonomy/ This case seeking Patch release binding and targeted for Solaris Nevada and Solaris 10 update releases. * What are the project's deliverables? This project's deliverables will be delivered through Solaris WOS. All the kernel related deliverables will be delivered through existing binaries such as genunix, pcie misc module and other PCIe nexus drivers. This project will also deliver the following new userland binaries. Interface Stability Comments ---------------------------------------------------------------------- /usr/sbin/hotplug Consolidation hotplug (1M) CLI Private /usr/sbin/hotplugd Consolidation hotplugd (1M) daemon Private /lib/libhotplug.so.1 Consolidation libhotplug (3LIB) Private /usr/lib/cfgadm/cfgadm_shp.so Consolidation cfgadm plugin Private * How does this project align with existing or proposed ARC cases? The existing cfgadm (1M) and its libcfgadm plugins for DR and hotplug features will remain unchanged. But to integrate support for the new hotplug connectors defined by the Solaris Hotplug Framework into the existing cfgadm framework, a new libcfgadm plugin is required. For more information, please see section 3.4 of shp-userland.txt. Other existing userland modules such as hal (5), prtdiag (PICL) will remain unchanged since the Solaris Hotplug Framework will continue to generate current sysevents for all hotplug connectors. 2. Describe user interactions. * Are new user interfaces being proposed, or existing interfaces being changed? New CLI usr/sbin/hotplug is added along with new library and daemon. Plus, existing cfgadm (1M) will be extended to work with new solaris hotplug framework through new plugin SHPC. * Explain the similarities in proposed interfaces with existing OS user interfaces (Solaris, Linux, Windows, etc.). TBD * Are there any install time changes? There are no install time changes. 3. What are the exported (defined by your project) and imported (defined by another project that your project then references) interfaces or protocols and their respective stability levels? See: http://www.opensolaris.org/os/community/arc/policies/interface-taxonomy/ 3.1 Exported Kernel Interfaces Interface Stability Comments ---------------------------------------------------------------------- MODHPOPS Project Modctl command for hotplug Private operations MODHPOPS_CHANGE_STATE Project Modctl sub-command for changing Private hotplug state MODHPOPS_CREATE_PORT Project Modctl sub-command for creating Private a port MODHPOPS_REMOVE_PORT Project Modctl sub-command for removing Private a port ndi_hp_register() Consolidation Register a connection to hotplug Private framework ndi_hp_unregister() Consolidation Unregister a connection from Private hotplug framework ndi_hp_state_change_req() Consolidation Submit a state change request Private to hotplug framework ddi_hp_cn_info_t Consolidation Structure for the information Private of a connection bus_hp_op() Consolidation bus_ops entry for hotplug Private operations BUSO_REV_10 Consolidation New busop revision Private ddi_hp_op_t Consolidation Hotplug operations 3.2 Exported Userland Interfaces (libhotplug(3LIB)): Interface Stability Comments ---------------------------------------------------------------------- HP_NODE_DEVICE Consolidation Node type for a device node Private HP_NODE_CONNECTOR Consolidation Node type for physical connector Private HP_NODE_PORT Consolidation Node type for virtual port Private HP_NODE_USAGE Consolidation Node type for usage record Private hp_node_t Consolidation Structure of a node in snapshot Private hp_init() Consolidation Initialize a hotplug snapshot Private hp_fini() Consolidation Cleanup/Remove hotplug snapshot Private hp_traverse() Consolidation Traverse nodes in a snapshot Private hp_name() Consolidation Get a node's name Private hp_is_physical() Consolidation Boolean test if node is physical Private hotplug connector hp_type() Consolidation Get a node's type Private hp_state() Consolidation Get a node's current state Private hp_usage() Consolidation Get a node's usage description Private hp_parent() Consolidation Get a node's parent Private hp_child() Consolidation Get a node's first child Private hp_sibling() Consolidation Get a node's next sibling. Private hp_set_state() Consolidation Command to initiate state change Private 3.3 Exported Userland Interfaces (libdevinfo (3LIB)): Interface Stability Comments ---------------------------------------------------------------------- DINFOHP Consolidation Flag for di_init() to include Private hotplug information in snapshot DI_HP_NIL Consolidation A NULL di_hp_t structure Private di_hp_t Consolidation Structure of a hotplug connector Private associated with a di_node_t di_walk_hp() Consolidation Traverse hotplug connectors Private associated with a di_node_t di_hp_next() Consolidation Get next di_hp_t in a list Private di_hp_name() Consolidation Get name of a di_hp_t connector Private di_hp_state() Consolidation Get state of a di_hp_t connector Private di_hp_physical Consolidation 1 if physical, 0 if virtual Private di_hp_type() Consolidation 0 if physical, 1 if virtual Private di_hp_child() Consolidation Return child device node of a Private virtual hotplug port di_hp_physnum() Consolidation Return physical connector number Private associated with a connection di_hp_virtnum() Consolidation Return virtual port number Private associated with a connection * Is there a versioning scheme in place? There is no versioning for any userland interfaces. On the bus op interface, we are adding new bus_hp_op interface and also, incrementing BUSO_REV version. Our approach is to fail earlier version nexus drivers. * Has the team secured interface contracts where necessary? All new user-land and new kernel-land interfaces will be Consolidation Private for now. No contracts are needed for ON consolidation users. * Use an ARC prescribed interface table format. See sections 3.1 through 3.3. 4. Describe any dependencies on hardware (e.g. SPARC exclusive), and on other projects within Solaris. No hardware dependencies. 5. Projects need to be aware of the overall security of the system and how their components affect it. Which parts of this project are critical to the security of the system to avoid such unintended consequences such as unauthorized system entry, unauthorized access to or modification of data, elevation of privilege, denial of service, violation of labeled security, ...? Does this project require elevated privilege? First phase of this project will introduce new hotplug (1M) CLI, but conversion to RBAC will be covered in next phase of this project. All requests for hotplug status information are granted. But to perform a state change operation, root privilege is required. 6. Describe means of observing project functionality and performance, by an end user or by a system administrator. Project functionality information is available via following hotplug subcommands hotplug list [-a] [-v] [ | ]] changestate [-f] [-q] -s [ | ] passthru [ | ] For more information on each subcommand, please refer hotplug (1M). An MDB debugging module will be added to display the available physical connectors and virtual ports. 7. How does the project deal with faults and interruptions? Initialization and restarting? In principle, on faults and interruptions, we should return the system back to what it was before the execution of the command. In the case of hotplug (1M), appropriate error message will be displayed to the end user. The project team will take all the efforts to make the error messages clear, precise and succinct. In the case of library API's appropriate error message will be returned to the caller. Hotplug FMA for PCIe devices (including surprise removal) will be supported in the next phase of this project. See section 1.4 of shp-overview.pdf 8. How does the project interact with Solaris virtualization technologies (xVM, LDOMs, zones, Branded zones, SunCluster, etc.)? In principle, this technology should work from within xVM and LDOMs. There are no known impediments to use within a SunCluster environment for this component. The project team will work with the SunCluster team to make sure the two technologies are sufficiently tested. This can be used on systems with non-global zones installed as well as in non-global zones themselves. 9. Does this project require administration (i.e., configuration or management)? If so, * How is the project administered, and what sort of review process has this user interface undergone? The new /usr/sbin/hotplug (1M) will be used. The subcommands provided by hotplug will be reviewed on opensolaris. In addition, the existing cfgadm (1M) will continue to be supported to work with the new Solaris hotplug framework. * Is there a means of aggregating management and/or configuration with other related projects? No. This project will introduce be a new stand alone CLI. * Does this project deliver its own administration along with the other components, or is this project an administration interface for other projects? Yes, it provides a new administrative interface to configure and unconfigure both physical and virtually hotplugged devices. * Are there any external (to Solaris) management interfaces to consider, or being consumed? No. 10. Have you reviewed the Policies and Best Practices? Are there any exceptions this project needs? See http://www.opensolaris.org/os/community/arc/policies/ http://www.opensolaris.org/os/community/arc/bestpractices/ After reviewing the Policies and Best Practices, no exceptions are required for this project. Reference: o Solaris Hotplug Project - Webpage http://pcie.sfbay/hotplug o Documents - One Pager http://sac.sfbay/arc/PSARC/2008/181/20080306_govinda.tatti - Overall Proposal http://sac.sfbay/arc/PSARC/2008/181/inception.materials/shp-overview.pdf - Userland Hotplug Software http://sac.sfbay/arc/PSARC/2008/181/inception.materials/shp-userland.pdf - Man pages http://sac.sfbay/arc/PSARC/2008/181/inception.materials/hotplug.1m.txt http://sac.sfbay/arc/PSARC/2008/181/inception.materials/libhotplug.3lib.txt - Slides http://sac.sfbay/arc/PSARC/2008/181/inception.materials/shp-slides.pdf o ARC cases PSARC/1993/687 Hot Plugging PSARC/1996/285 DR for CPU/Memory Boards PSARC/1998/327 PCI Hotplug Support PSARC/1998/460 RCM Framework PSARC/1999/122 PCI Bus Resource Allocator PSARC/1999/135 Solaris 8 RAS Enhancements PSARC/1999/287 Solaris Embedded Fcode Interpreter PSARC/1999/322 System Event Framework PSARC/1999/686 Promote PCI Hotplug Interfaces PSARC/2000/085 Extensions for PCI Hotplug Framework PSARC/2000/218 RCM Framework Enhancements PSARC/2000/295 PICL Event Interfaces PSARC/2000/298 PCI Hotplug slot registration policy PSARC/2000/532 State Model for Device Information Nodes PSARC/2002/315 cPCI Autoconfiguration Support PSARC/2003/193 Solaris Contracts and restart agreements FWARC/2005/048 1275 Bindings for PCI Express Interconnect PSARC/2005/375 PCI Hotplug Extensions for PCIe PSARC/2006/037 PCI Express Hotplug Framework Interrupt Interfaces FWARC/2006/198 PCI Hotplug Resource Preallocation PSARC/2007/197 ZFS Hotplug PSARC/2007/290 Retire Agent for IO devices o Aliases mailto:ddi-hp-iteam@sun.com - Solaris Hotplug Framework Dev Team