MAC Rings and Share Capabilities: -------------------------------- Release binding: patch Interface Taxonmy: Project Private 0. Introduction: --------------- The Niagara 2 NIU (and the Neptune ASIC) provides physical mechanisms to enable the sharing of the NIU's physical resources (e.g. RX and TX dma resources). In particular, the Niagara II NIU and Neptune provide two capabilities -- virtualization regiions and receive dma channel groups (or ring groups). Ring groups provide a separation of receive DMA channel resources and enable independent L2 addressing of the ring group. Additionally, ring groups can be mapped onto virtualization regions (or shares) to enable independent access by another virtual machine (e.g. LDom Guest Domain or xVM DomU). The mapping of transmit and receive ring groups to virtualization regions is handled by the hyeprvisor of a given virtualization region. Crossbow will offer a Multiple Ring Capability to enable a device to express it's ring group capabilities. The Hybrid NIU project slated for S10U6 needs this capability and the a share capability to enable clients of the NIU driver to group rings and share ring resources accordingly. This proposal provides two new GLDv3 capabilities that will initially be project private and the when Crossbow is avaialble will be reclassified appropriately. 1 Overview: ----------- MAC clients wanting to use the MAC_CAPAB_RINGS and MAC_CAPAB_SHARES capabilites would first use mac_getcap_get() to determine if the underlying driver (e.g. nxge) implements the capabilities. The call to the driver for MAC_CAPAB_RINGS would determine how many ring groups, how many rings, and provide the interface functions for manipulating ring groups, if available. Similarly, a call to mac_getcap_get() for MAC_CAPAB_SHARES would determine how many shares and the interfaces functions for managing shares the underlying driver has. 2.0 MAC Rings and Shares Capabilities: ------------------------------------- The addition to the MAC Layer capabilties is MAC_CAPAB_RINGS and MAC_CAPAB_SHARES. /* * MAC layer capabilities. These capabilities are handled by the drivers' * mc_capab_get() callbacks. Some capabilities require the driver to fill * in a given data structure, and others are simply boolean capabilities. * Note that capability values must be powers of 2 so that consumers and * providers of this interface can keep track of which capabilities they * care about by keeping a bitfield of these things around somewhere. * * 6648684: To obsolete RX_CLASSIFY capability. * 6648685: To obsolete MULTIADDRESS capability. * */ typedef enum { MAC_CAPAB_HCKSUM = 0x01, /* data is a uint32_t for the txflags */ MAC_CAPAB_POLL = 0x02, /* boolean only, no data */ MAC_CAPAB_MULTIADDRESS = 0x04, /* data is multiaddress_capab_t */ MAC_CAPAB_LSO = 0x08, /* data is mac_capab_lso_t */ MAC_CAPAB_RINGS = 0x100, /* data is mac_capab_rings_t */ MAC_CAPAB_SHARES = 0x200 /* data is mac_capab_share_t */ /* add new capabilities here */ } mac_capab_t 3.0 MAC_CAPAB_RINGS: ------------------- The addition of MAC_CAPAB_RINGS requires of a definition of MAC rings. This definition of MAC rings comes directly from the Crossbow documentation and code. /* * Multiple Rings capability */ typedef enum { MAC_RING_TYPE_RX = 1, /* Receive ring */ MAC_RING_TYPE_TX = 2 /* Transmit ring */ } mac_ring_type_t; /* * Grouping type of a ring group * * MAC_GROUP_TYPE_STATIC: The ring group can not be re-grouped. * MAC_GROUP_TYPE_DYNAMIC: The ring group support dynamic re-grouping */ typedef enum { MAC_GROUP_TYPE_STATIC = 1, /* Static ring group */ MAC_GROUP_TYPE_DYNAMIC = 2 /* Dynamic ring group */ } mac_group_type_t; typedef struct __mac_ring_driver *mac_ring_driver_t; typedef struct __mac_ring_handle *mac_ring_handle_t; typedef struct __mac_group_driver *mac_group_driver_t; typedef struct __mac_group_handle *mac_group_handle_t; typedef struct __mac_intr_handle *mac_intr_handle_t; typedef struct mac_ring_info_s mac_ring_info_t; typedef struct mac_group_info_s mac_group_info_t; typedef int (*mac_intr_enable_t)(mac_intr_handle_t); typedef int (*mac_intr_disable_t)(mac_intr_handle_t); typedef struct mac_intr_s { mac_intr_handle_t mi_handle; mac_intr_enable_t mi_enable; mac_intr_disable_t mi_disable; } mac_intr_t; 3.1 MAC_CAPAB_RINGS driver entry points: --------------------------------------- MAC Rings Interface functions are defined here. These are the driver entry points to manage and manipulate rings. typedef void (*mac_get_ring_t)(void *, mac_ring_type_t, const int, const int, mac_ring_info_t *, mac_ring_handle_t); /* * Return a group to the caller. */ typedef void (*mac_get_group_t)(void *, mac_ring_type_t, const int, mac_group_info_t *, mac_group_handle_t); /* * Add a ring to the group. */ typedef void (*mac_group_add_ring_t)(mac_group_driver_t, mac_ring_driver_t, mac_ring_type_t); /* * Remove a ring from the group. */ typedef void (*mac_group_rem_ring_t)(mac_group_driver_t, mac_ring_driver_t, mac_ring_type_t); 3.2 MAC_CAPAB_RINGS capability: ------------------------------ /* * Multiple Rings Capability */ typedef struct mac_capab_rings_s { mac_ring_type_t mr_type; /* Ring type */ mac_group_type_t mr_group_type; /* Grouping type */ void *mr_handle; /* Group Driver Handle. */ uint_t mr_rnum; /* Number of rings */ uint_t mr_gnum; /* Number of ring groups */ mac_get_ring_t mr_rget; /* Get ring from driver */ mac_get_group_t mr_gget; /* Get ring group from driver */ mac_group_add_ring_t mr_gadd_ring; /* Add ring into a group */ mac_group_rem_ring_t mr_grem_ring; /* Remove ring from a group */ } mac_capab_rings_t; typedef int (*mac_group_start_t)(mac_group_driver_t); typedef void (*mac_group_stop_t)(mac_group_driver_t); typedef int (*mac_add_mac_addr_t)(void *, const uint8_t *); typedef int (*mac_rem_mac_addr_t)(void *, const uint8_t *); struct mac_group_info_s { mac_group_driver_t mrg_driver; /* Driver reference */ mac_group_start_t mrg_start; /* Start the group */ mac_group_stop_t mrg_stop; /* Stop the group */ uint_t mrg_count; /* Count of rings */ mac_intr_t mrg_intr; /* Optional per-group intr */ /* Only used for rx groups */ mac_add_mac_addr_t mrg_addmac; /* Add a MAC address */ mac_rem_mac_addr_t mrg_remmac; /* Remove a MAC address */ }; 4.0 MAC_CAPAB_SHARES: -------------------- This capability manages shares (e.g., virtualization regions of the NIU). It provides the number of shares available and interface management functions to grab (allocate) a share, free a share and add and remove ring resources from the share. 4.1 MAC_CAPAB_SHARE driver entry points: --------------------------------------- /* * Share management functions. */ typedef uint64_t mac_share_handle_t; /* * Returns a Share Handle to the client calling from above. */ typedef int (*mac_alloc_share_t)(void *drvhandle, uint64_t cookie, uint64_t *rcookie, mac_share_handle_t *); /* * Destroys the share previously allocated and unallocates * all share resources (e.g. DMA's assigned to the share). */ typedef void (*mac_free_share_t)(mac_share_handle_t); !/* ! * Query interface for a share. ! * A share may have some restrictions that need to conveyed ! * to the upper layer. ! * ! * rmin: The minimum number of rings for a share. ! * rmax: The maximum number of rings for a share. ! * rmap: A bit-vector indicating the rings that be mapped to a share ! * gnum: The group number that can be used with the share. ! */ !typedef void (*mac_share_query_t)(mac_share_handle_t shdl, ! mac_ring_type_t type, uint32_t *rmin, uint32_t *rmax, ! uint64_t *rmap, uint64_t *gnum); /* * Basic idea, bind previously created ring groups to shares * for them to be exported (or shared) by another domain. * These interfaces bind/unbind the ring group to a share. The * of doing such causes the resources to be shared with the guest. */ typedef int (*mac_share_add_group_t)(mac_share_handle_t, mac_group_handle_t); typedef int (*mac_share_rem_group_t)(mac_share_handle_t, mac_group_handle_t); 4.2 MAC_CAPAB_SHARE capability: ------------------------------ typedef struct mac_capab_share_s { uint_t ms_snum; /* Number of shares (vr's) */ void *ms_handle; /* Handle to driver. */ mac_alloc_share_t ms_alloc; /* Get a share from driver. */ mac_free_share_t ms_sfree; /* Return a share to driver. */ mac_share_add_group_t ms_sadd; /* Add a group to the share. */ mac_share_rem_group_t ms_sremove; /* Remove group from share. */ ! mac_share_query_t ms_squery; /* Queury share */ } mac_capab_share_t; 5.0 Transition Plan: ------------------- It is noted that Crossbow will later subsume these two capabilities when Crossbow becomes available. Crossbow will offer a MAC client interface that gives access to shares for other consumers such as xVM and LDoms. LDoms will then migrate to use the of the MAC client interfaces. This interface will also be extended to support ring grouping and share capabilities found in other devices that offer similar partioning. 6.0 References: -------------- 1. Crossbow MAC Ring Architecture Specification http://www.opensolaris.org/os/project/crossbow/Docs/virtual_resources.pdf 2. Niagara II NIU PRM 3. Hybrid I/O NIU Architecture Specification http://www.opensolaris.org/os/project/crossbow/Docs/virtual_resources.pdf 4. PSARC/2006/248 Nemo MAC-Type Plugin Architecture http://sac.sfbay/Archives/CaseLog/arc/PSARC/2006/248 5. PSARC/2006/249 Nemo Changes for Binary Compatibility http://sac.sfbay/Archives/CaseLog/arc/PSARC/2006/249 6. PSARC/2006/357 Crossbow http://sac.sfbay/Archives/CaseLog/arc/PSARC/2006/357