Brussels - enhanced network driver configuration via dladm 1. Introduction --------------- Project Brussels will provide a configuration mechanism for administering network drivers through the GLDv3 framework. Essentially, we will extract the best features of existing administrative methods like dladm(1M), ndd(1M) and driver.conf(4) to provide the following: - like ndd, the ability to configure properties on the fly, - like driver.conf the ability to configure properties like Jumbo Frame MTU, and have the settings persist across reboot, - leverage from dladm's flexible syntax, and its user-friendly show-* sub-commands. A full description of the overall architecture of the Brussels project is described in [2]. Note, however, that this PSARC case (2007/429) will only deliver a subset of the design in [2]. The relation between the components of the Brussels project, as well as the deliverables associated with this umbrella case, are described in this document. 2. Brussels Overview -------------------- Dynamic configuration of network drivers on Solaris is done primarily via ndd(1M). The ndd interface is not documented, causing drivers to cut/paste potentially irrelevant code from other drivers. Changes made by ndd(1M) are not persistent across reboot/modunload. Syntax and semantics across ndd invocations are not uniform across drivers (e.g., CR 6504688). Related methods available for network driver configuration also have drawbacks, which are outlined in Section 1 of [2]. Project Brussels aims to fix these problem for GLDv3 drivers by providing a dladm based standardized interface. Specifically, this project provides a cleaner solution by using the dld layer as the intermediary. GLDv3 drivers will be able to register callback functions to be invoked for setting/getting property values. Dladm/libdladm will be modified to issue system calls to set/get property values via the dld layer. Invocations to the new property management interfaces will be integrated into smf(5) to provide persistent network-driver settings across interface restart and driver unload/reload. The consistent, well documented call-interface provided by Brussels across network drivers can be leveraged by link configuration tools for layered services like NWAM (PSARC 2007/132). This will also allow cleaner driver implementation by removing the need for drivers to implement the undocumented ndd(1m) support. Framework delivery will allow the usage of dladm for GLDv3 driver configuration is conformant with trends followed by recent features like Wifi (PSARC 2006/406) and Link Aggregation. Support for dynamically adjusting settings of commonly modified properties such as Jumbo Frame MTU will be provided for each converted driver. 3. Brussels components ----------------------- The components of the Brussels project are * Framework delivery including the associated dladm/libdladm changes * persistent property management * ndd compatibility handling. * conversion of multiple drivers to the Brussels framework. Each of these components will be separately delivered and will be represented by separate PSARC cases under this umbrella. This PSARC case ((PSARC 2007/429) will deliver the changes pertaining to Framework delivery in the kernel and user-space. Until the delivery of the last component, project teams that propose new drivers should be aware of the changes being introduced by the Brussels project. Minimally, drivers should be written modularly so that properties can be modified dynamically. Inflexible assumptions about availability of property values (e.g., at the start of the driver's attach(9E), and invariance of the value until driver detach) should not be made. After the delivery of PSARC 2007/429, drivers should provide setprop/getprop callbacks where possible, instead of using the ndd(1m) interfaces. 3.1 Framework delivery ---------------------- Architectural details of the Brussels framework are available in Section 3 of [2]. A diagram illustrating the architecture is available in Figure 1 of [2]. This PSARC case will provide the framework-related changes needed at the Nemo/GLDv3 layer of Figure 1 in [2], as well as the corresponding changes needed in one driver as a demonstration of the framework usage. Deliverables will include * changes to the mac layer for invoking driver callbacks, * changes to the selected sample driver (bge) with corresponding callbacks. * a consistent call interface for setting/getting driver properties via dladm as described in Section 3.2 of [2]. The first driver that will be converted to the Brussels framework will be the bge driver, chosen because of its wide usage, and the fact that its implementation typifies network drivers, making it a good "example" driver to demonstrate the usability of the Framework. Section 4 of this document identifies the details of the interfaces delivered by this PSARC case. 3.2 Persistent management of property settings ---------------------------------------------- As described in CR 6485961, property settings for a network driver module will need to be retained across an unload/reload sequence for the driver. There are currently no automatic mechanisms to achieve this. This Brussels component will provide changes to make link property values persist across unplumb/unload. Invocations to the new property management interfaces will be integrated into smf(5) to provide persistent network-driver settings across interface restart and driver unload/reload. 3.3 ndd compatibility handling ------------------------------ Existing ndd invocations will continue to be supported as described in Section 5.0.2 of [2]. Specifically, after a driver that has been converted to use the Brussels framework, legacy support for ndd will be provided by the framework itself. Until the delivery of this component, ndd properties will continue to work as they do today: the ND_SET and ND_GET ioctls will be delivered to the driver, and values returned back using the same paths as today. However, delivery of 3.4 will result in the interception and dispatching of the ND_SET/ND_GET ioctls at the dld layer, so that drivers will be released of the burden of repetitive cut/paste code to undocumented ndd interfacs. At that point, the driver will register legacy ndd ioctls with the dld layer, and will only be contacted to set/get values that are private to the driver module. 3.4 conversion of multiple drivers to the Brussels framework -------------------------------------------------------------- After at least one driver has been successfully converted to use the Brussels framework, other network drivers will also be modified to use the framework. The project will be complete when all ON drivers, including WiFi, have been converted to the Brussels framework. Note that WiFi drivers currently support property management via dladm by issuing ioctls directly to the net80211 module for reasons described in Section 3.2.1 of [2]. Conversion of wifi drivers like ath and wpi will provide mc_setprop/mc_getprop entry points for these properties so that the search order described in Section 3.2.1 will allow the eradication of the net80211 paths when compatibility issues are resolved. 4. Interfaces delivered by PSARC 2007/429 ------------------------------------------ The umbrella case will deliver the changes described in this section for the support of DLDIOCSETPROP/DLDIOCGETPROP system calls in dladm/libdladm and the mac and bge kernel modules. 4.1 MAC Client Interface Changes -------------------------------- Currently, every GLDv3 driver registers itself by invoking mac_register() with a set of driver callbacks. The callbacks are provided in a mac_callbacks_t structure in the mac_register_t passed to mac_register() (see PSARC/2006/249). PSARC 2007/429 will add two new callbacks will be added to the mac_callbacks_t structure: mac_propf_t mc_setprop; mac_propf_t mc_getprop; The callback functions may _optionally_ be provided by the drivers, so that drivers that do not wish to support property management via mc_setprop/mc_getprop may provide null values for these at registration. The mc_callbacks field in the mac_callbacks_t structure is used to denote which optional callbacks have been provided by the driver. Two new flags, MC_SETPROP and MC_GETPROP will be added to the set of possible flags that may be found in mc_callbacks to denote the presence of valid mc_setprop/mc_getprop pointers.. 4.1.1 mc_setprop ---------------- typedef int (*mac_propf_t)(void *driver, void *prname, int propnum, int valsize, void *val); The mc_setprop is an optional entry point provided by the driver to set the property value of a property identified by the string "prname" to the value "val". The "driver" argument contains a pointer to the mi_driver field that was provided by the driver during mac_register. For Public properties (see Section 3.2.3 of [2], the prnum additionally contains the encoded numeric mapping corresponding to prname. The valid set of name to encoded property numbers is defined in . 4.1.2 mc_getprop ---------------- typedef int (*mac_propf_t)(void *driver, void *prname, int propnum, int valsize, void *val); The mc_getprop is an optional entry point provided by the driver to obtain the current value of the property identified by the string "prname". to the value "val". Valsize denote the available buffer space in the memory pointed to by val. The "driver" argument contains a pointer to the mi_driver field that was provided by the driver during mac_register. For Public properties (see Section 3.2.3 of [2], the prnum additionally contains the encoded numeric mapping corresponding to prname. The valid set of name to encoded property numbers is defined in . 4.2 DLDIOCSETPROP/DLDIOCGETPROP ------------------------------- The DLDIOCSETPROP and DLDIOCGETPROP ioctls are intercepted in the kernel dld module which extracts the ioctl data to be passed to the driver's mc_setprop/mc_getprop routine. The ioctl data is expected to be formatted as a dld_ioc_prop_val_t structure defined in typedef struct dld_ioc_prop_val_s { int pr_version; uint_t pr_flags; /* private to libdladm */ char pr_linkname[LIFNAMSIZ]; /* interface name */ uint_t pr_num; /* char pr_name[DLD_LINKPROP_NAME_MAX]; uint_t pr_valsize; /* sizeof pr_val */ void *pr_val; } dld_ioc_prop_val_t; Section 3.2.1 of [2] describes each of the above fields in further detail. 4.3 Driver changes ------------------ The bge driver will plug into the Brussels framework by providing callback entry points at registration. These callback points, bge_m_setprop and bge_m_getprop will provide support for the setting/getting the following MII properties ifspeed link_duplex link_up adv_autoneg_cap adv_asmpause_cap adv_pause_cap adv_1000fdx_cap adv_1000hdx_cap adv_100fdx_cap adv_100hdx_cap adv_10fdx_cap adv_10hdx_cap In addition, the bge driver will be modified to support modifications to the Jumbo Frame MTU through settings of the "default_mtu" property. IMPORTED INTERFACES: Interface Classification Comments ------------------------------------------------------------------------------ GLDv3 MAC interfaces Consolidation Private PSARC 2006/249 mac_maxsdu_update Consolidation Private Clearview IP Tunneling (to be ARC'ed) EXPORTED INTERFACES: Interface Classification Comments ----------------------------------------------------------------------------- mac_callbacks_t Consolidation Private (modified) mac_register Consolidation Private (modified) dld_ioc_prop_val_t Consolidation Private (new) DLDIOCSETPROP Consolidation Private DLDIOCGETPROP Consolidation Private MC_SETPROP Consolidation Private MC_GETPROP Consolidation Private DLD_PROP_* Consolidation Private Prefix for encoding Brussels properties default_mtu Committed ifspeed Committed link_duplex Committed link_up Committed adv_autoneg_cap Committed adv_asmpause_cap Committed adv_pause_cap Committed adv_1000fdx_cap Committed adv_1000hdx_cap Committed adv_100fdx_cap Committed adv_100hdx_cap Committed adv_10fdx_cap Committed adv_10hdx_cap Committed 4. Recap of Brussels components -------------------------------- (1) Framework delivery, including dladm changes (PSARC 2007/429) (2) persistent property management (3) ndd compatibility handling. (4) conversion of multiple drivers to the Brussels framework. (2), (3), (4) will depend on (1). References ---------- [1] one-pager (http://sac.sfbay/arc/PSARC/2007/429/20070725_sowmini.varadhan) [2] "Brussels- NIC configuration Design Specification"