1. Introduction 1.1. Project/Component Working Name: ndi flavors 1.2. Name of Document Author/Supplier: Author: Matthew Jacob 1.3. Date of This Document: Tue Feb 17 15:12:04 MST 2009 4. Technical Description 4.1 Background: Associated with each device node is a private data pointer managed via ddi_[gs]et_driver_private(9F). Typically, the meaning of the data that this pointer refers to is known to the driver that sets the private data pointer. However, the meaning of that data may also be know by other parts of the system where contracts (formal or otherwise) have been established. 4.2 Problem: One example of this is the use of a SCSA HBA nexus driver's private data pointer by common SCSA framework code. The private data pointer is a pointer to a public scsi_hba_tran(9S) structure. This is well known amongst multiple parts of the system. This becomes a problem when a SCSA HBA nexus driver needs to support more than just one kind (flavor) of child (i.e. more that just scsi_device(9S) children). How does the nexus maintain private data that is relevant to just one a specific kind (flavor) of child? A previous attempt has been made to address this problem via extensions to the scsi_hba_tran(9S) structure [1]. However, this only provided an out for one additional flavor of child, and only for SCSA. Additionally, code complexity was added by requiring indirection through an existing, unrelated, data structure with various ad hoc mechanisms for determining when such dereference was needed. 4.3 Proposal: The proposal is to allow a nexus driver to declare, at attach time, how many distinct flavors of children it supports. With this proposal, the DDI framework can maintain flavored private data pointers for supported child flavors beyond the default 'vanilla' flavor. The default vanilla flavor maps to the same private data pointer storage used by the existing ddi_[gs]et_driver_private(9F) interfaces. This allow a nexus to evolve to support more flavors of children without breaking existing code that is not flavor aware. The number of flavors used by a nexus driver is up to that nexus driver or, possibly, to common nexus code shared by multiple nexus drivers (like SCSA). This proposal provides an interface for the parent to set a flavor-specific private data pointer (in the parent). This flavor-specific private data pointer provides common parent information that applies to all children of the same flavor. The format of this flavor-specific private data is known by the parent, but may also be be shared with the child. Sharing with a child is typically indirect, via a flavor-specific API used by the child, but this is not always the case. By default, when devinfo child nodes are created, the node's flavor is set to the default vanilla flavor. This proposal provides an interface so a nexus driver can set (during bus config) the flavor of a child. After this, both the parent and the child can use the 'get flavor' interface to determine flavor of the node. The flavor can then be used to to access the flavor-specific private data pointer established in the parent. A child's access to flavor information may be indirect (or implied) by use of a flavor-specific API. The initial putback of the new interfaces will convert the SMP support delivered by [1] to use the new ndi_flavor* interfaces as well as extend the SMP attach framework to be more of a peer to the SCSI attach framework. This putback will also establish that, for SCSA's, default/legacy vanilla flavor is associated with scsi_device(9S) oriented children In addition, SCSA work in progress will be using the ndi_flavor* interfaces to identify device nodes associated with the dynamic enumeration of initiator ports (ala mscsi/iport). 4.4 Proposed Interfaces: ------------------------------------------------------------------ Interface Name Comm.Lev. Comments ------------------------------------------------------------------ ndi_flavorv_alloc() Cons.Private Allocate extra flavor private data pointer storage in the parent. ndi_flavorv_set() " Given a specific flavor of child, set the private data pointer value stored in the parent for children of that flavor. ndi_flavorv_get() " Given a specific flavor of child, get the private data pointer value stored in the parent for children of that flavor. ndi_flavor_set() " Set the flavor of a child. ndi_flavor_get() " Get the flavor of a child. sas_hba_tran_t " SMP specific transport structure smp_tran_alloc() " Allocate an SMP specific transport structure smp_attach_setup() " During attach(9E), set up an allocated SMP transport structure. smp_tran_free() " Free an SMP transport structure. 4.5 Interface Prototypes: Defined in sunndi.h: void ndi_flavorv_alloc(dev_info_t *, int); void ndi_flavorv_set(dev_info_t *, ndi_flavor_t, void *); void *ndi_flavorv_get(dev_info_t *, ndi_flavor_t); void ndi_flavor_set(dev_info_t *, ndi_flavor_t); ndi_flavor_t ndi_flavor_get(dev_info_t *); NOTE: It is not an error to call ndi_flavorv_alloc() multiple times as long as the number of defined flavors stays the same. NOTE: there is no ndi_flavorv_free() function as the underlying framework will free any allocations when the dev_info detaches. Defined in sas_transport.h: sas_hba_tran_t *sas_hba_tran_alloc(dev_info_t *dip, int flags); int sas_hba_attach_setup(dev_info_t *dip, sas_hba_tran_t *smp); void sas_hba_tran_free(sas_hba_tran_t *smp); 4.4 SCSA Example (fragments): All HBA drivers, from their attach(9E) implementation, call scsi_hba_tran_alloc(9F) (or sas_hba_tran_alloc()). In the SCSA framework code implementing *_tran_alloc() two things happen. First, the *_tran_alloc() implementation may call ndi_flavorv_alloc() to reserve space for all SCSA supported flavors. Second, ndi_flavor_set() will be called to plumb *_tran_alloc()'s flavor of tran. Storage allocated by the ndi_flavorv_alloc() call will be freed by the IO framework when the node detaches. When a child of a non-SCSAv3 HBA driver is created via bus_config operations, the flavor of the child may be set: if (child_is_smp) { ndi_flavor_set(dip, SCSA_FLAVOR_SMP); } In the future, with SCSAv3, establishing the flavor of a child will be done by SCSA itself. In the implementation of flavor-specific tran functions called by target drivers, flavors can be used to select the correct transport operation vector. sas_hba_tran_t *tran = ndi_flavorv_get(self, SCSA_FLAVOR_SMP); When common SCSA scsi_hba.c code needs to perform different operations based on flavor of an enumerated child: switch (ndi_flavor_get(child) { case XXX: _xxx_(); /* xxx flavor (i.e. SMP_device) */ break; case YYY: _yyy_(); /* yyy flavor (i.e. scsi_device) */ break; ... /* future: iport, SATA */ } With SCSA, the 'vanilla' flavor is associated with scsi_device(9S) child using scsi_hba_tran(9S) transport services. NOTE: For SCSAv3, the ndi_flavor* interfaces will only be used within the common SCSA framework code. There is no intent to make the ndi_flavor* interfaces public as part of the SCSAv3. 4.5 Alternatives: No reasonable alternatives to solve the problem have been found. Since all proposed interfaces are Private, this case is not preventing a future project from implementing more sophisticated ways of plumbing flavor-specific transport services. 4.6 Man Pages These are private interfaces that, like all other ndi_* functions and interfaces defined by [1], have no man pages. 4.7 Release Binding Micro/patch binding is requested. 4.8 References [1] PSARC case that introduced smp transport extensions Serial Attached SCSI (SAS) Management Protocol (SMP) support for Solaris http://sac.sfbay/PSARC/2007/595 http://www.opensolaris.org/os/community/arc/caselog/2007/595 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: Automatic 6.6. ARC Exposure: open