Nemo MAC-Type Plugin Architecture ================================= release binding: patch 1 Introduction ============== This fast-track case is part of the Clearview umbrella case (PSARC/2005/132). It modifies and makes additions to the interfaces defined in PSARC/2004/571 (Nemo - a.k.a. GLD v3). The justification for this work and its relationship to the rest of the Clearview project is discussed in that case. To allow GLDv3 to support MAC layers other than Ethernet, a new plugin mechanism will be introduced. MAC-type plugins will be kernel modules that register with the GLDv3 mac module. MAC drivers will then be required to request the use of a particular plugin to handle the operations specific to its MAC type. The main benefits of this plugin architecture are: * To remove MAC type specific code from the GLDv3 framework. * To allow support for new MAC types to be added at any time without recompiling the GLDv3 framework. * To not burden driver developers with needing to implement boilerplate code that is MAC type specific but not specific to their particular driver. 2 MAC-type Plugin Registration =============================== MAC Type plugins will interact with the GLDv3 mac module using the following four functions: mactype_register_t *mactype_alloc(uint_t mactype_version); int mactype_register(mactype_register_t *); void mactype_free(mactype_register_t *); int mactype_unregister(const char *); When a plugin module loads, its _init() routine is invoked. At that moment, it is expected to register with the GLDv3 framework by calling mactype_register(). First, it will need to allocate a mactype_register_t data-structure to be used as the argument to mactype_register by calling mactype_alloc(). This allocation routine shields plugins from future additions to the mactype_register_t structure, thus preserving the binary compatibility of the interface. The sole argument to mactype_alloc() must be MACTYPE_VERSION (defined in ). This allows mactype_alloc() to verify if the plugin was compiled against an compatible version of the MAC-Type framework. A version mismatch will result in mactype_alloc() returning NULL, and the plugin will not be allowed to register. After having registered using mactype_register(), the mactype_register_t used to register must be freed using mactype_free(). When a plugin module is unloaded, its _fini() routine is invoked, and it is expected to unregister with the GLDv3 framework by calling mactype_unregister(). If mactype_unregister() fails by returning a non-zero error code, the plugin must not continue with mod_remove() as drivers may still be using the plugin. In that case, the plugin must return from _fini() using the error code returned by mactype_unregister(). mactype_register_t has the following structure: typedef struct mactype_register_s { uint_t mtr_version; const char *mtr_ident; mactype_ops_t *mtr_ops; uint_t mtr_mactype; uint_t mtr_addrlen; uint8_t *mtr_brdcst_addr; const char *mtr_statname; mac_stat_info_t *mtr_stats; size_t mtr_statcount; } mactype_register_t; The fields are: * mtr_version: Automatically set by mactype_alloc(). This allows mactype_register() to know which version of the MAC-Type framework the plugin was compiled against. * mtr_ident: A NULL terminated string identifying the plugin. MAC drivers will bind to a particular plugin by using this string upon registering with mac_register() (this mechanism is described in the dependent case PSARC/2006/249.) This is also the string that plugins must use to unregister when calling mactype_unregister(). * mtr_ops: A structure containing plugin callback pointers. The structure and list of defined callbacks are defined in section 3. These callbacks are the meat of the plugins. They are what GLDv3 uses to execute the MAC-Type specific functionality. * mtr_mactype: The DLPI MAC type implemented by the plugin as defined in (DL_ETHER for example). * mtr_addrlen: The length of MAC addresses for the given MAC type. * mtr_brdcst_addr: If non-NULL, this points to the broadcast address for the media. Its size must be mtr_addrlen. The address is copied by the API. * mtr_statname: The name associated with the kstats defined by the plugin. * mtr_stats: An array of mac_stat_info_t structures defining the set of statistics for the plugin. Drivers that use a given plugin are responsible for implementing the defined statistics. When MAC drivers register, the GLDv3 framework will define kstats for the MAC in accordance with the list of statistics defined by the plugin. When a request is made for a given kstat, the GLDv3 framework calls into the driver to obtain that statistic's value. The dependent case PSARC/2006/249 contains more details about how GLDv3 and drivers will handle statistics in the description of the mc_getstat driver callback. There is one strict requirement that the GLDv3 framework poses on MAC type plugins regarding statistics; All statistics defined by plugins must have a minimum value of MACTYPE_STAT_MIN. This ensures that statistics defined by plugins do not clash with generic statistics defined by the GLDv3 framework. For an example, see the Ethernet plugin described in section 5. * mtr_statcount: The number of mac_stat_info_t array elements in mtr_stats. 3 Plugin Operations =================== When registering with mactype_register(), plugins supply a set of callbacks as the mtr_ops field of the mactype_register_t. The mtr_ops field has the following structure: typedef struct mactype_ops_s { uint_t mtops_ops; mtops_addr_verify_t mtops_unicst_verify; mtops_addr_verify_t mtops_multicst_verify; mtops_sap_verify_t mtops_sap_verify; mtops_header_t mtops_header; mtops_header_info_t mtops_header_info; mtops_pdata_verify_t mtops_pdata_verify; mtops_header_modify_t mtops_header_cook; mtops_header_modify_t mtops_header_uncook; } mactype_ops_t; The mtops_ops field is a set of flags defining which optional operations are defined by the plugin. This allows the framework to define additional optional operations without having to recompile plugins, thus maintaining backward compatibility with older binary plugins. The only optional operations currently defined are mtops_pdata_verify, mtops_header_cook, and mtops_header_uncook. The flags currently defined are thus: #define MTOPS_PDATA_VERIFY 0x001 #define MTOPS_HEADER_COOK 0x002 #define MTOPS_HEADER_UNCOOK 0x004 Each plugin operation function has as one of its arguments a pointer to optional (may be NULL) MAC plugin data that the plugin may use. Plugins may require such data from drivers, and if so, must define the format of the data in its documentation and must also provide the mtops_pdata_verify operation. Each driver may then register such data in its mac_register_t when issuing mac_register(), and may update its data using mac_pdata_update(). This is discussed in the dependent case PSARC/2006/249. The following sections describe each plugin operation. 3.1 mtops_unicst_verify ----------------------- This operation verifies that a given address is a valid unicast address for the MAC type implemented by the plugin. The operation has the following type: typedef int (*mtops_addr_verify_t)(const void *addr, void *pdata); The operation must either return 0 if the given address is a valid unicast address, or a non-zero errno. 3.2 mtops_multicst_verify ------------------------- This operation verifies that a given address is a valid multicast address for the MAC type implemented by the plugin. The operation has the same type as mtops_unicst_verify. The operation must either return 0 if the given address is a valid multicast address, or a non-zero errno. For example, if multicast is supported by the MAC type but the address has an invalid format, EINVAL would be an acceptable return value. If multicast is not supported at all by the MAC type, ENOTSUP would be an acceptable return value. 3.3 mtops_sap_verify -------------------- This operation verifies that a given DLPI SAP is valid for the MAC type implemented by the plugin. The operation has the following type: typedef boolean_t (*mtops_sap_verify_t)(uint32_t sap, uint32_t *bind_sap, void *pdata); The sap argument is the one being verified. The function returns B_TRUE if the sap is valid, or B_FALSE if it is nor. The operation also optionally sets bind_sap (if non_NULL) to the value to which GLDv3 should bind DLPI consumers. This is used by the Ethernet plugin (described in section 5) to bind all LLC SAPs to bind to the same value, 0. 3.4 mtops_header ---------------- This operation allocates and constructs a MAC header. It has the following type: typedef mblk_t *(*mtops_header_t)(const void *saddr, const void *daddr, uint32_t sap, void *pdata, mblk_t *payload, size_t extra_len); The operation is called in two different contexts. One is to construct a MAC header when the Nemo framework is requested to transmit a packet using DL_UNITDATA_REQ. In this case, the payload argument points to the data that Nemo will be transmitting using the header that this operation constructs. The other context is when the Nemo framework is requested to construct a MAC header for use by IP fast-path, in which case the payload argument is NULL. In this case, the header is being constructed to be cached for later transmission to a specific destination, so there is no payload to reference at the time the mtops_header() operation is called. The operation allocates an mblk_t and fills in its contents with a MAC header using the source address "saddr", the destination address "daddr", the given sap, and optional MAC plugin data. The extra_len argument tells the operation how much extra space to allocate following the end of the header. For example, the dls module uses this extra length feature to pre-allocate space for VLAN tag information. The operation returns a valid pointer to the allocated mblk_t upon success, or NULL upon failure. The b_wptr field of the returned mblk_t points to the end of the header, leaving any potential extra space after the b_wptr. 3.5 mtops_header_info --------------------- This operation takes an mblk_t whose b_rptr points to the beginning of a MAC header, and returns information about that header. typedef int (*mtops_header_info_t)(mblk_t *mp, void *pdata, mac_header_info_t *mhip); The function returns 0 upon success, or a non-zero errno upon failure (if the mblk is too small to contain a valid header, for example). The operation returns the needed information by filling in the mac_header_info_t structure pointed to by the third argument. typedef struct mac_header_info_s { size_t mhi_hdrsize; size_t mhi_pktsize; const uint8_t *mhi_daddr; const uint8_t *mhi_saddr; uint32_t mhi_origsap; uint32_t mhi_bindsap; mac_addrtype_t mhi_dsttype; } mac_header_info_t; * mhi_hdrsize is set to the size of the header. * mhi_pktsize is set to the size of the entire packet contained in the mblk (including the header). The special value 0 means that the packet is of size MBLKL(mp). This special value exists because most headers don't have packet size information, and thus the only logical size if MBLKL(mp). * mhi_daddr and mhi_saddr point to the destination and source addresses. These are pointers to the actual addresses in the header, not copies. * mhi_origsap is the value of the SAP in the header. * mhi_bindsap is the value to which DLPI consumers interested in this SAP will be bound to. This is the same value that would be returned if mhi_origsap were passed into the mtops_sap_verify operation. * mhi_dsttype describes the type of the destination address, which can be set to one of: MAC_ADDRTYPE_UNICAST MAC_ADDRTYPE_MULTICAST MAC_ADDRTYPE_BROADCAST The mac_header_info_t structure replaces what was the dls_header_info_t in PSARC/2004/571. 3.6 mtops_pdata_verify ---------------------- This operation verifies that the MAC plugin data being registered by a driver is valid. Its type is: typedef boolean_t (*mtops_pdata_verify_t)(void *pdata, size_t pdata_size); This optional operation must be provided if the plugin supports MAC plugin data. The function returns B_TRUE if the data is valid, or B_FALSE if it isn't. The validity of MAC plugin data is a plugin specific property, and the format of valid MAC plugin data should be documented in each plugin's documentation. This is called when drivers register MAC plugin data via mac_register(), and also when drivers update MAC plugin data via mac_pdata_update(). Both of these functions are described in PSARC/2006/249. 3.7 mtops_header_cook --------------------- This operation modifies the given MAC header as transmitted by RAW DLPI consumers. Its type is: typedef mblk_t *(*mtops_header_modify_t)(mblk_t *raw_header, void *pdata); Solaris DLPI has a special raw mode enabled using the DLIOCRAW ioctl. In this mode, DLPI consumers can send packets that include the desired link-layer header, and can receive packets that include the link-layer header. One example of an application that uses this mode is snoop(1M), which requires the ability to receive link-layer headers in order to properly display them to the user. This operation takes a packet as sent down to the GLDv3 framework from a raw DLPI consumer (raw_header), and "cooks" the header for transmission to the underlying driver. The mblk's DB_REF() count is guaranteed to be 1 by the GLDv3 framework. Upon failure, NULL is returned and the mblk passed in remains unmodified. Upon success, the modified mblk is returned. This is used, for example, by plugins that wish to give raw DLPI consumers the illusion that the device implements a MAC type different than what is actually being implemented by the driver. Examples of this are WiFi technologies that have very complex MAC layer protocols that wish to be accessed and observed as regular Ethernet by raw consumers. In fact, this project has worked closely with the WiFi team, which has developed a WiFi plugin that implements this operation (along with the associated mtops_header_uncook described below.) The WiFi MAC-Type plugin itself is not part of this case, and will be submitted separately. 3.8 mtops_header_uncook ----------------------- This operation does the opposite of mtops_header_cook. Its type is also the same. It takes a packet as received by the driver and "uncooks" the header such that the packet can be passed up to raw DLPI consumers. This operation can make the same DB_REF() assumption as mtops_header_cook(), and also has the same success and error semantics. 4 MAC Client Functions to Access Plugin Functionality ===================================================== The MAC-Type plugins register with the GLDv3 mac module, and therefore the mac module has direct access to the plugin operations described in the previous section. The functionality implemented by these operations needs to be accessed by other pieces of the GLDv3 stack, namely the MAC clients dld, dls and aggr. As such, new MAC client interfaces are defined here to give access to the new plugin functionality. Note that there are no new interfaces defined to directly access the mtops_unicst_verify and mtops_multicst_verify operations, as these operations are called from the existing MAC client interfaces, mac_unicst_set() and mac_multicst_add(). 4.1 mac_sap_verify ------------------ boolean_t mac_sap_verify(mac_handle_t mh, uint32_t sap, uint32_t *bind_sap); This function is the MAC client interface for the mtops_sap_verify MAC-Type plugin callback described in section 3.3. 4.2 mac_header -------------- mblk_t *mac_header(mac_handle_t mh, const uint8_t *dst, uint32_t sap, mblk_t *payload, size_t extralen); This function is the MAC client interface for the mtops_header MAC-Type plugin callback described in section 3.4. The Consolidation Private dls_header() function defined in PSARC/2004/571 is modified to accomodate the extra payload argument as follows: mblk_t *dls_header(dls_channel_t dc, const uint8_t *addr, uint16_t sap, uint_t pri, mblk_t *payload); 4.3 mac_header_info ------------------- void mac_header_info(mac_handle_t mh, mblk_t *mp, mac_header_info_t *mhip); This is the MAC client interface for the mtops_header_info MAC-Type plugin callback described in section 3.5. 4.4 mac_header_cook ------------------- mblk_t *mac_header_cook(mac_handle_t mh, mblk_t *mp); This is the MAC client interface for the mtops_header_cook MAC-Type plugin callback described in section 3.7. The mp argument must point to the beginning of the header. If an error occurs, mac_header_cook() returns NULL, and mp is unmodified. If mac_header_cook() is successful, it will return a pointer to the modified mblk. This pointer may be different from the original, so it is imperative that callers no longer use the mp pointer they passed in as the second argument, as that mp may have been freed by the interface (if DB_REF(mp) was > 1 for example). 4.5 mac_header_uncook --------------------- mblk_t *mac_header_uncook(mac_handle_t mh, mblk_t *mp); This is the MAC client interface for the mtops_header_uncook MAC-Type plugin callback described in section 3.8. It has the same calling semantics as mac_header_cook described above, except that it does the opposite operation on the header. 5 Loading Plugin Kernel Modules =============================== When a driver requests the use of a given plugin, the mac module attempts to load the plugin from /kernel/mac (a new directory introduced by this case) using modload(). The modload() is what drives the plugin's _init() routine to be called, and the plugin to register via mactype_register(). As a result, all MAC-Type plugins present and future _must_ install in /kernel/mac. 6 Ethernet MAC Type Plugin ========================== One plugin that will be delivered as part of this case will be a DL_ETHER Ethernet plugin. It will implement all plugin operations except for mtops_header_cook and mtops_header_uncook, which it has no need to implement. The plugin kernel module will install as /kernel/mac/ether. A header file will contain the necessary information for drivers to use the plugin, namely a MAC_PLUGIN_IDENT_ETHER macro used to identify the plugin during mac_register(), and the list of statistics defined by the plugin. The statistics are: enum ether_stat { /* RFC 1643 stats */ ETHER_STAT_ALIGN_ERRORS = MACTYPE_STAT_MIN, ETHER_STAT_FCS_ERRORS, ETHER_STAT_FIRST_COLLISIONS, ETHER_STAT_MULTI_COLLISIONS, ETHER_STAT_SQE_ERRORS, ETHER_STAT_DEFER_XMTS, ETHER_STAT_TX_LATE_COLLISIONS, ETHER_STAT_EX_COLLISIONS, ETHER_STAT_MACXMT_ERRORS, ETHER_STAT_CARRIER_ERRORS, ETHER_STAT_TOOLONG_ERRORS, ETHER_STAT_MACRCV_ERRORS, /* MII/GMII stats */ ETHER_STAT_XCVR_ADDR, ETHER_STAT_XCVR_ID, ETHER_STAT_XCVR_INUSE, ETHER_STAT_CAP_1000FDX, ETHER_STAT_CAP_1000HDX, ETHER_STAT_CAP_100FDX, ETHER_STAT_CAP_100HDX, ETHER_STAT_CAP_100FDX, ETHER_STAT_CAP_100HDX, ETHER_STAT_CAP_10FDX, ETHER_STAT_CAP_10HDX, ETHER_STAT_CAP_ASMPAUSE, ETHER_STAT_CAP_PAUSE, ETHER_STAT_CAP_AUTONEG, ETHER_STAT_ADV_CAP_1000FDX, ETHER_STAT_ADV_CAP_1000HDX, ETHER_STAT_ADV_CAP_100FDX, ETHER_STAT_ADV_CAP_100HDX, ETHER_STAT_ADV_CAP_10FDX, ETHER_STAT_ADV_CAP_10HDX, ETHER_STAT_ADV_CAP_ASMPAUSE, ETHER_STAT_ADV_CAP_PAUSE, ETHER_STAT_ADV_CAP_AUTONEG, ETHER_STAT_LP_CAP_1000FDX, ETHER_STAT_LP_CAP_1000HDX, ETHER_STAT_LP_CAP_100FDX, ETHER_STAT_LP_CAP_100HDX, ETHER_STAT_LP_CAP_10FDX, ETHER_STAT_LP_CAP_10HDX, ETHER_STAT_LP_CAP_ASMPAUSE, ETHER_STAT_LP_CAP_10HDX, ETHER_STAT_LP_CAP_ASMPAUSE, ETHER_STAT_LP_CAP_PAUSE, ETHER_STAT_LP_CAP_AUTONEG, ETHER_STAT_LINK_ASMPAUSE, ETHER_STAT_LINK_PAUSE, ETHER_STAT_LINK_AUTONEG, ETHER_STAT_LINK_DUPLEX }; Drivers using the Ethernet plugin will receive requests for these statistics via the mc_getstat driver callback described in PSARC/2006/249. 7 Interface Table ================= _____________________________________________________________________________ | Interfaces Exported | |________________________|_______________________|__________________________| | Interface | Classification | Comments | |________________________|_______________________|__________________________| | mactype_alloc() | Consolidation Private | | | mactype_free() | Consolidation Private | | | mactype_register() | Consolidation Private | | | mactype_unregister() | Consolidation Private | | | | | | | MTOPS_HEADER_COOK | Consolidation Private | | | MTOPS_HEADER_UNCOOK | Consolidation Private | | | MAC_ADDRTYPE_UNICAST | Consolidation Private | | | MAC_ADDRTYPE_MULTICAST | Consolidation Private | | | MAC_ADDRTYPE_BROADCAST | Consolidation Private | | | MACTYPE_STAT_MIN | Consolidation Private | | | | | | | mactype_register_t | Consolidation Private | | | mactype_ops_t | Consolidation Private | | | mac_header_info_t | Consolidation Private | | | | | | | mac_sap_verify() | Consolidation Private | | | mac_header() | Consolidation Private | | | mac_header_info() | Consolidation Private | | | mac_header_cook() | Consolidation Private | | | mac_header_uncook() | Consolidation Private | | | | | | | MAC_PLUGIN_IDENT_ETHER | Consolidation Private | | | enum ether_stat | Consolidation Private | | |________________________|_______________________|__________________________|