From cth@sac.sfbay.sun.com Fri Dec 5 16:02:36 2008 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mB602Zi1022774 for ; Fri, 5 Dec 2008 16:02:35 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id mB602GIG007104; Sat, 6 Dec 2008 00:02:34 GMT Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KBF00H19HG9WX00@brm-avmta-1.central.sun.com>; Fri, 05 Dec 2008 17:02:33 -0700 (MST) Received: from dm-sfbay-02.sfbay.sun.com ([129.146.11.31]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KBF00G03HG8DB10@brm-avmta-1.central.sun.com>; Fri, 05 Dec 2008 17:02:32 -0700 (MST) Received: from sac.sfbay.sun.com (new-sac.SFBay.Sun.COM [129.146.175.65]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id mB602VZP051687; Fri, 05 Dec 2008 16:02:31 -0800 (PST) Received: from sac.sfbay.sun.com (localhost [127.0.0.1]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mB602OkI021412; Fri, 05 Dec 2008 16:02:24 -0800 (PST) Received: (from cth@localhost) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id mB602OEX021391; Fri, 05 Dec 2008 16:02:24 -0800 (PST) Date: Fri, 05 Dec 2008 16:02:24 -0800 (PST) From: Christopher Horne Subject: ddi_ssoft_state(9F) and ddi_isoft_state(9F) [PSARC/2008/755 FastTrack timeout 12/17/2008] To: PSARC-ext@sun.com Message-id: <200812060002.mB602OEX021391@sac.sfbay.sun.com> Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 Status: RO Content-Length: 22903 I am sponsoring the following fasttrack for myself. Micro/patch binding is requested. The timer is set to expire on Dec. 17 2008. -Chris 1. Introduction 1.1. Project/Component Working Name: ddi_ssoft_state(9F) and ddi_isoft_state(9F) 1.2. Name of Document Author/Supplier: Author: Chris Horne 1.3. Date of This Document: Tue Nov 11 18:10:01 MST 2008 4. Technical Description 4.1 Background: ddi_soft_state(9F) provides utility interfaces to help device drivers manage different context. When small integer values, like ddi_get_instance(9F) return values, are directly related to context, the current "indexed" soft_state interfaces work well. 4.2 Problem: Two IEEE-1275 '@unit-address' issues can make the current ddi_soft_state(9F) support ineffective: o Compression: Some IEEE-1275 bindings, like the SCSI '@,' notation, compress multiple addressing levels into a single 'unit-address' string. If a driver needs to maintain soft state at intermediate levels, like the level, there is no instance number to use with ddi_soft_state(9F), and no dev_info node to allow use of ddi_[gs]et_driver_private(9F) or ddi_[gs]et_parent_data). o Complexity: An addressing level may be complex: both non-numeric, sparse, and large. This situation has forced drivers to maintain soft state context in ways that no longer fit the ddi_soft_state(9F) model. An example is SCSA HBA unit-address context. For the initial SCSI Parallel Interconnect transport the was as an integer limited to the [0-15] range, and ddi_soft_state(9F) worked well. On modern transports the complexity of the space has mushroomed, as shown below, and the space is now 64-bits and sparse. SPI: disk@0,0 WWN: disk@w216000c0ff8047dd,0 GUID: disk@g600c0ff0000000000047dd270fdf0901 (lun identity: MPxIO) ISCSI: disk@0000iqn.1986-03.com.sun%3A02%3Ac8a82272-b354-c913-80f9-db9cb378a6f60001,0 4.3 Proposal: The proposal is to extend the ddi_soft_state(9F) programming model by providing "string" based peers to todays "indexed" soft_state interfaces. In addition to "string" support, "indexed" interfaces with stronger typing are proposed. Implementation of proposed ddi_ssoft_state(9F) interfaces is built on top of the 'modhash' interfaces introduced by [1]. 4.4 Proposed Interfaces: ------------------------------------------------------------------ Interface Name Comm.Lev. Comments ------------------------------------------------------------------ ddi_isoft_state Private "indexed" soft_state ddi_ssoft_state " "string" soft_state ddi_isoft_state_init " typed equivalent of ddi_soft_state_init(9F) ddi_ssoft_state_init " string peer of ddi_isoft_state_init(9F) ddi_isoft_state_zalloc " typed equivalent of ddi_soft_state_zalloc(9F) ddi_ssoft_state_zalloc " string peer of ddi_isoft_state_zalloc(9F) ddi_isoft_state_get " typed equivalent of ddi_soft_state_get(9F) ddi_ssoft_state_get " string peer of ddi_isoft_state_get(9F) ddi_isoft_state_free " typed equivalent of ddi_soft_state_free(9F) ddi_ssoft_state_free " string peer of ddi_isoft_state_free(9F) ddi_isoft_state_fini " typed equivalent of ddi_soft_state_fini(9F) ddi_ssoft_state_fini " string peer of ddi_isoft_state_fini(9F) ddi_isoft_state_unassigned " return first unassigned item of an ddi_isoft_state. 4.5 Interface Prototypes: (defined in sunddi.h): typedef struct __ddi_isoft_state ddi_isoft_state; typedef struct __ddi_ssoft_state ddi_ssoft_state; int ddi_isoft_state_init(ddi_isoft_state **state_p, size_t size, size_t n_items); int ddi_ssoft_state_init(ddi_ssoft_state **state_p, size_t size, int hash_sz); int ddi_isoft_state_zalloc(ddi_isoft_state *state, int index); int ddi_ssoft_state_zalloc(ddi_ssoft_state *state, const char *stringkey); void * ddi_isoft_state_get(ddi_isoft_state *state, int index); void * ddi_ssoft_state_get(ddi_ssoft_state *state, const char *stringkey); void ddi_isoft_state_free(ddi_isoft_state *state, int index); void ddi_ssoft_state_free(ddi_ssoft_state *state, const char *stringkey); void ddi_isoft_state_fini(ddi_isoft_state **state_p); void ddi_ssoft_state_fini(ddi_ssoft_state **state_p); int ddi_isoft_state_unassigned(ddi_isoft_state **state_p); 4.4 Example: Assuming that 'tpaddr' is the portion SCSI 'unit-address' string, A SCSA HBA driver can use the following to access its private specific context. ss = ddi_isoft_state_get(xx_state, ddi_get_instance(devi)); tpss = ddi_ssoft_state_get(ss->ss_tpstate, tpaddr); Where xx_state would be setup in _init(9E) via ddi_isoft_state_init(9F), and ss_tpstate would be setup in xxattach(9E) via ddi_ssoft_state_init(9F). 4.5 Alternatives: An alternative 'compound' approach was considered and prototyped. The 'compound' approach implemented a set of 'ddi_strid_*(9E)' string-to-id mapping interfaces, and then used the existing "indexed" ddi_soft_state(9F) to get from an 'id' to the soft state. The 'ddi_strid_(9E)' interfaces, while a good fit for some applications, was found to increase SCSI host adapter driver complexity compared to the proposed ddi_ssoft_state(9F) interfaces. The proposed ddi_ssoft_state(9F) interfaces also present a familiar API: they are based on the well-established ddi_soft_state(9F) API structure. 4.6 Man Pages See Appendix A. Some of the examples are TBS at this point, more meaningful examples should be provided before these interfaces are promoted. 4.7 Release Binding Micro/patch binding is requested. 4.8 References [1] PSARC case that introduced modhash interfaces PSARC/1998/212 Extensions to Device Autoconfiguration [John Danielson] http://sac.sfbay/PSARC/1998/212 http://www.opensolaris.org/os/community/arc/caselog/PSARC/1998/212 Appendix A: A.1: New ddi_ssoft_state(9F) ========================================= :r ddi_ssoft_state.9f Kernel Functions for Drivers ddi_ssoft_state(9F) NAME ddi_ssoft_state, ddi_ssoft_state_get, ddi_ssoft_state_fini, ddi_ssoft_state_free, ddi_ssoft_state_init, ddi_ssoft_state_zalloc - driver string soft state utility routines SYNOPSIS #include #include typedef struct __ddi_ssoft_state ddi_ssoft_state; int ddi_ssoft_state_init(ddi_ssoft_state **state_p, size_t size, size_t hash_sz); int ddi_ssoft_state_zalloc(ddi_ssoft_state *state, int stringkey); void *ddi_ssoft_state_get(ddi_ssoft_state *state, int stringkey); void ddi_ssoft_state_free(ddi_ssoft_state *state, int stringkey); void ddi_ssoft_state_fini(ddi_ssoft_state **state_p); INTERFACE LEVEL Solaris DDI specific (Solaris DDI). PARAMETERS state_p Address of the opaque state pointer which will be initialized by ddi_ssoft_state_init() to point to implementation dependent data. state An opaque pointer to implementation-dependent data that describes the soft state. size Size of the soft state which will be allocated by subsequent calls to ddi_ssoft_state_zalloc(); zero is illegal. hash_sz The number of hashed lists which will be allocated; zero is not allowed. There is a performance .vs. space tradeoff in the selection of a hash_sz value: for 'n' soft state structures, on average each ddi_ssoft_state_get will need to traverse 'n'/hash_sz/2 entries to locate the requested soft state structure. stringkey The string key for the soft state structure; usually associated with some sub-portion of the device's 'unit-address'. DESCRIPTION Some drivers must maintain state information at a granularity unrelated to ddi_get_instance(9F) instance numbers or dev_info node ddi_get_driver_private(9F) data. When the needed soft state granularity can be associated with a string, such as a sub-portion of a 'unit-address' string, the ddi_ssoft_state(9F) utility routines can be used to help device drivers manage the space used by the driver to hold such state information. For example, if a SCSI HBA driver maintains state for each target-port address in a separate state structure, for transports with complex target-port address representations these routines can be used to dynamically allocate and deallocate a separate structure for each target-port as scsi_device(9S) associated with the target-port are initialized by tran_tgt_init(9E) and and freed by tran_tgt_free(9E). To use the routines, the driver writer needs to declare a state pointer, state_p, which the implementation uses as a place to hang a set of per-driver structures; everything else is managed by these routines. The routine ddi_ssoft_state_init() is usually called in the driver's attach(9E) routine to initialize the state pointer, and set the size of the soft state structure. The routine ddi_ssoft_state_zalloc() is called to allocate a soft state. The routine is passed a string key, which is used to refer to the structure in subsequent calls to ddi_ssoft_state_get() and ddi_ssoft_state_free(). The string key is usually some sub-portion of a 'unit-address'. The routine attempts to allocate space for the new structure, and if the space allocation was successful, DDI_SUCCESS is returned to the caller. Returned memory is zeroed. A pointer to the space previously allocated for a soft state structure can be obtained by calling ddi_ssoft_state_get() with the appropriate string key. The space used by a given soft state structure can be returned to the system using ddi_ssoft_state_free(). The space used by all the soft state structures allocated on a given state pointer, together with the housekeeping information used by the implementation can be returned to the system using ddi_ssoft_state_fini(). This routine is typically called from the driver's detach(9E) routine. The ddi_ssoft_state_zalloc(), ddi_ssoft_state_free() and ddi_ssoft_state_get() routines coordinate access to the underlying data structures in an MT-safe fashion, thus no additional locks should be necessary. RETURN VALUES ddi_ssoft_state_get() NULL The requested soft state structure was not allocated at the time of the call. pointer The pointer to the soft state structure. ddi_ssoft_state_init() 0 The allocation was successful. EINVAL Either the size parameter was zero, or the state_p parameter was invalid. ddi_ssoft_state_zalloc() DDI_SUCCESS The allocation was successful. DDI_FAILURE The routine failed to allocate the storage required; either the state parameter was invalid, the string key was NULL, or an attempt was made to allocate using a string key that was already allocated. CONTEXT The ddi_ssoft_state_init() and ddi_ssoft_state_alloc() functions can be called from user or kernel context only, since they may internally call kmem_zalloc(9F) with the KM_SLEEP flag. The ddi_ssoft_state_fini(), ddi_ssoft_state_free() and ddi_ssoft_state_get() routines can be called from any driver context. EXAMPLES <> SEE ALSO _fini(9E), _init(9E), attach(9E), detach(9E), ddi_get_instance(9F), ddi_soft_state(9E), ddi_isoft_state(9E), getminor(9F), kmem_zalloc(9F) NOTES If necessary, a hierarchy of state structures can be constructed by embedding state pointers in higher order state structures. DIAGNOSTICS <> All of the messages described below usually indicate bugs in the driver and should not appear in normal operation of the system. WARNING: ddi_ssoft_state_zalloc: bad handle WARNING: ddi_ssoft_state_free: bad handle WARNING: ddi_ssoft_state_fini: bad handle The implementation-dependent information kept in the state variable is corrupt. WARNING: ddi_ssoft_state_free: null handle WARNING: ddi_ssoft_state_fini: null handle The routine has been passed a null or corrupt state pointer. Check that ddi_ssoft_state_init() has been called. WARNING: ddi_ssoft_state_free: string key '%s' not allocated The routine has been asked to free a string key which is not allocated. The message prints out the invalid string key. A.2: New ddi_isoft_state(9F) ========================================= :r ddi_isoft_state.9f Kernel Functions for Drivers ddi_isoft_state(9F) NAME ddi_isoft_state, ddi_isoft_state_get, ddi_isoft_state_fini, ddi_isoft_state_free, ddi_isoft_state_init, ddi_isoft_state_zalloc - driver indexed soft state utility routines SYNOPSIS #include #include typedef struct __ddi_isoft_state ddi_isoft_state; int ddi_isoft_state_init(ddi_isoft_state **state_p, size_t size, size_t n_items); int ddi_isoft_state_zalloc(ddi_isoft_state *state, int item); void *ddi_isoft_state_get(ddi_isoft_state *state, int item); void ddi_isoft_state_free(ddi_isoft_state *state, int item); void ddi_isoft_state_fini(ddi_isoft_state **state_p); INTERFACE LEVEL Solaris DDI specific (Solaris DDI). PARAMETERS state_p Address of the opaque state pointer which will be initialized by ddi_isoft_state_init() to point to implementation dependent data. state An opaque pointer to implementation-dependent data that describes the soft state. size Size of the item which will be allocated by subsequent calls to ddi_isoft_state_zalloc(). n_items A hint of the number of items which will be preallocated; zero is allowed. item The item number for the state structure; usually the instance number of the associated devinfo node. DESCRIPTION Most device drivers maintain state information with each instance of the device they control; for example, a soft copy of a device control register, a mutex that must be held while accessing a piece of hardware, a partition table, or a unit structure. These utility routines are intended to help device drivers manage the space used by the driver to hold such state information. For example, if the driver holds the state of each instance in a single state structure, these routines can be used to dynamically allocate and deallocate a separate structure for each instance of the driver as the instance is attached and detached. To use the routines, the driver writer needs to declare a state pointer, state_p, which the implementation uses as a place to hang a set of per-driver structures; everything else is managed by these routines. The routine ddi_isoft_state_init() is usually called in the driver's _init(9E) routine to initialize the state pointer, set the size of the soft state structure, and to allow the driver to pre-allocate a given number of such structures if required. The routine ddi_isoft_state_zalloc() is usually called in the driver's attach(9E) routine. The routine is passed an item number which is used to refer to the structure in subsequent calls to ddi_isoft_state_get() and ddi_isoft_state_free(). The item number is usually just the instance number of the devinfo node, obtained with ddi_get_instance(9F). The routine attempts to allocate space for the new structure, and if the space allocation was successful, DDI_SUCCESS is returned to the caller. Returned memory is zeroed. A pointer to the space previously allocated for a soft state structure can be obtained by calling ddi_isoft_state_get() with the appropriate item number. The space used by a given soft state structure can be returned to the system using ddi_isoft_state_free(). This routine is usually called from the driver's detach(9E) entry point. The space used by all the soft state structures allocated on a given state pointer, together with the housekeeping information used by the implementation can be returned to the system using ddi_isoft_state_fini(). This routine can be called from the driver's _fini(9E) routine. The ddi_isoft_state_zalloc(), ddi_isoft_state_free() and ddi_isoft_state_get() routines coordinate access to the underlying data structures in an MT-safe fashion, thus no additional locks should be necessary. RETURN VALUES ddi_isoft_state_get() NULL The requested state structure was not allocated at the time of the call. pointer The pointer to the state structure. ddi_isoft_state_init() 0 The allocation was successful. EINVAL Either the size parameter was zero, or the state_p parameter was invalid. ddi_isoft_state_zalloc() DDI_SUCCESS The allocation was successful. DDI_FAILURE The routine failed to allocate the storage required; either the state parameter was invalid, the item number was negative, or an attempt was made to allocate an item number that was already allocated. CONTEXT The ddi_isoft_state_init() and ddi_isoft_state_alloc() functions can be called from user or kernel context only, since they may internally call kmem_zalloc(9F) with the KM_SLEEP flag. The ddi_isoft_state_fini(), ddi_isoft_state_free() and ddi_isoft_state_get() routines can be called from any driver context. EXAMPLES Example 1 Creating and Removing Data Structures The following example shows how the routines described above can be used in terms of the driver entry points of a character-only driver. The example concentrates on the portions of the code that deal with creating and removing the driver's data structures. typedef struct { volatile caddr_t *csr; /* device registers */ kmutex_t csr_mutex; /* protects 'csr' field */ unsigned int state; dev_info_t *dip; /* back pointer to devinfo */ } devstate_t; static ddi_isoft_state *statep; int _init(void) { int error; error = ddi_isoft_state_init(&statep, sizeof (devstate_t), 0); if (error != 0) return (error); if ((error = mod_install(&modlinkage)) != 0) ddi_isoft_state_fini(&statep); return (error); } int _fini(void) { int error; if ((error = mod_remove(&modlinkage)) != 0) return (error); ddi_isoft_state_fini(&statep); return (0); } static int xxattach(dev_info_t *dip, ddi_attach_cmd_t cmd) { int instance; devstate_t *softc; switch (cmd) { case DDI_ATTACH: instance = ddi_get_instance(dip); if (ddi_isoft_state_zalloc(statep, instance) != DDI_SUCCESS) return (DDI_FAILURE); softc = ddi_isoft_state_get(statep, instance); softc->dip = dip; ... return (DDI_SUCCESS); default: return (DDI_FAILURE); } } static int xxdetach(dev_info_t *dip, ddi_detach_cmd_t cmd) { int instance; switch (cmd) { case DDI_DETACH: instance = ddi_get_instance(dip); ... ddi_isoft_state_free(statep, instance); return (DDI_SUCCESS); default: return (DDI_FAILURE); } } static int xxopen(dev_t *devp, int flag, int otyp, cred_t *cred_p) { devstate_t *softc; int instance; instance = getminor(*devp); if ((softc = ddi_isoft_state_get(statep, instance)) == NULL) return (ENXIO); ... softc->state |= XX_IN_USE; ... return (0); } SEE ALSO _fini(9E), _init(9E), attach(9E), detach(9E), ddi_get_instance(9F), getminor(9F), kmem_zalloc(9F) WARNINGS There is no attempt to validate the item parameter given to ddi_isoft_state_zalloc() other than it must be a positive signed integer. Therefore very large item numbers may cause the driver to hang forever waiting for virtual memory resources that can never be satisfied. NOTES If necessary, a hierarchy of state structures can be constructed by embedding state pointers in higher order state structures. DIAGNOSTICS All of the messages described below usually indicate bugs in the driver and should not appear in normal operation of the system. WARNING: ddi_isoft_state_zalloc: bad handle WARNING: ddi_isoft_state_free: bad handle WARNING: ddi_isoft_state_fini: bad handle The implementation-dependent information kept in the state variable is corrupt. WARNING: ddi_isoft_state_free: null handle WARNING: ddi_isoft_state_fini: null handle The routine has been passed a null or corrupt state pointer. Check that ddi_isoft_state_init() has been called. WARNING: ddi_isoft_state_free: item %d not in range [0..%d] The routine has been asked to free an item which was never allocated. The message prints out the invalid item number and the acceptable range. A.3: Changes to existing ddi_soft_state(9F) ========================================= :r!diff -U5 ddi_soft_state.9f.orig ddi_soft_state.9f --- ddi_soft_state.9f.orig Mon Nov 10 11:45:36 2008 +++ ddi_soft_state.9f Tue Nov 11 18:08:26 2008 @@ -224,22 +224,29 @@ return (0); } SEE ALSO _fini(9E), _init(9E), attach(9E), detach(9E), - ddi_get_instance(9F), getminor(9F), kmem_zalloc(9F) + ddi_get_instance(9F), ddi_isoft_state(9E), ddi_ssoft_state(9E), + getminor(9F), kmem_zalloc(9F) WARNINGS There is no attempt to validate the item parameter given to ddi_soft_state_zalloc() other than it must be a positive signed integer. Therefore very large item numbers may cause the driver to hang forever waiting for virtual memory resources that can never be satisfied. + Use of the ddi_isoft_state(9E) interfaces is encouraged, it + provides stronger typing. + NOTES If necessary, a hierarchy of state structures can be constructed by embedding state pointers in higher order state structures. + + Equivalent functionality with stronger typing is provided + by the newer ddi_isoft_state(9F) interfaces. DIAGNOSTICS All of the messages described below usually indicate bugs in the driver and should not appear in normal operation of the system. From gdamore@sun.com Fri Dec 5 16:23:20 2008 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mB60NJgC018435 for ; Fri, 5 Dec 2008 16:23:20 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id mB60NF4N021239 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Sat, 6 Dec 2008 00:23:19 GMT Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KBF00501IETEW00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.Com); Fri, 05 Dec 2008 16:23:17 -0800 (PST) Received: from sca-es-mail-2.sun.com ([192.18.43.133]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KBF00JWEIES7S70@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.Com); Fri, 05 Dec 2008 16:23:16 -0800 (PST) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id mB60NFFu012710 for ; Fri, 05 Dec 2008 16:23:15 -0800 (PST) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0KBF00301I19D800@fe-sfbay-10.sun.com> (original mail from gdamore@sun.com) for PSARC-ext@Sun.Com (ORCPT PSARC-ext@Sun.Com); Fri, 05 Dec 2008 16:23:15 -0800 (PST) Received: from [10.7.251.172] by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0KBF00A2JIER4D80@fe-sfbay-10.sun.com>; Fri, 05 Dec 2008 16:23:15 -0800 (PST) Date: Fri, 05 Dec 2008 16:16:12 -0800 From: "Garrett D'Amore" Subject: Re: ddi_ssoft_state(9F) and ddi_isoft_state(9F) [PSARC/2008/755 FastTrack timeout 12/17/2008] In-reply-to: <200812060002.mB602OEX021391@sac.sfbay.sun.com> Sender: Garrett.Damore@sun.com To: Christopher Horne Cc: PSARC-ext@sun.com Message-id: <4939C44C.2090604@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200812060002.mB602OEX021391@sac.sfbay.sun.com> User-Agent: Thunderbird 2.0.0.14 (X11/20080616) Status: RO Content-Length: 25437 I have the strong feeling that this case is trying to conflate a general purpose hash table with soft state storage. While drivers could potentially use a hash table for soft state storage, it seems that creating a limited purpose set of routines which are really just accessors into a hash table structure might not be the best approach. Most previous usages of "soft state" consider one soft state structure per device-instance (dip), whereas it appears that this project considers the needs of a nexus driver to store state for each of potentially multiple child devices. This seems more like a general data structure problem to me. What is wrong with the modhash routines as they are today? Can't drivers just use those directly? Put another way, how does this case improve life for drivers beyond just using modhash directly? I'm also not sure I understand the point of ddi_get_isoft_state. Is the "typed" adjective here a reference to the fact that the first argument (the master state pointer) is typed to something other than void *? (Do we really believe that there significant cases here where the type safety this would afford justifies the cost of introducing another interface? I confess that I'm very skeptical of that.) Color me interested but as yet unconvinced. -- Garrett Christopher Horne wrote: > I am sponsoring the following fasttrack for myself. Micro/patch > binding is requested. The timer is set to expire on Dec. 17 2008. > > -Chris > > > 1. Introduction > 1.1. Project/Component Working Name: > ddi_ssoft_state(9F) and ddi_isoft_state(9F) > 1.2. Name of Document Author/Supplier: > Author: Chris Horne > 1.3. Date of This Document: > Tue Nov 11 18:10:01 MST 2008 > > 4. Technical Description > > 4.1 Background: > > ddi_soft_state(9F) provides utility interfaces to help device > drivers manage different context. When small integer values, like > ddi_get_instance(9F) return values, are directly related to > context, the current "indexed" soft_state interfaces work well. > > 4.2 Problem: > > Two IEEE-1275 '@unit-address' issues can make the current > ddi_soft_state(9F) support ineffective: > > o Compression: Some IEEE-1275 bindings, like the SCSI > '@,' notation, compress multiple addressing > levels into a single 'unit-address' string. If a driver needs > to maintain soft state at intermediate levels, like the > level, there is no instance number to use with > ddi_soft_state(9F), and no dev_info node to allow use of > ddi_[gs]et_driver_private(9F) or ddi_[gs]et_parent_data). > > o Complexity: An addressing level may be complex: both > non-numeric, sparse, and large. > > This situation has forced drivers to maintain soft state context in > ways that no longer fit the ddi_soft_state(9F) model. > > An example is SCSA HBA unit-address context. For the > initial SCSI Parallel Interconnect transport the was > as an integer limited to the [0-15] range, and ddi_soft_state(9F) > worked well. On modern transports the complexity of the > space has mushroomed, as shown below, and the > space is now 64-bits and sparse. > > SPI: disk@0,0 > WWN: disk@w216000c0ff8047dd,0 > GUID: disk@g600c0ff0000000000047dd270fdf0901 (lun identity: MPxIO) > ISCSI: disk@0000iqn.1986-03.com.sun%3A02%3Ac8a82272-b354-c913-80f9-db9cb378a6f60001,0 > > 4.3 Proposal: > > The proposal is to extend the ddi_soft_state(9F) programming model > by providing "string" based peers to todays "indexed" soft_state > interfaces. In addition to "string" support, "indexed" interfaces > with stronger typing are proposed. > > Implementation of proposed ddi_ssoft_state(9F) interfaces is built > on top of the 'modhash' interfaces introduced by [1]. > > 4.4 Proposed Interfaces: > > ------------------------------------------------------------------ > Interface Name Comm.Lev. Comments > ------------------------------------------------------------------ > > ddi_isoft_state Private "indexed" soft_state > > ddi_ssoft_state " "string" soft_state > > > ddi_isoft_state_init " typed equivalent of > ddi_soft_state_init(9F) > ddi_ssoft_state_init " string peer of > ddi_isoft_state_init(9F) > > > ddi_isoft_state_zalloc " typed equivalent of > ddi_soft_state_zalloc(9F) > ddi_ssoft_state_zalloc " string peer of > ddi_isoft_state_zalloc(9F) > > ddi_isoft_state_get " typed equivalent of > ddi_soft_state_get(9F) > ddi_ssoft_state_get " string peer of > ddi_isoft_state_get(9F) > > ddi_isoft_state_free " typed equivalent of > ddi_soft_state_free(9F) > ddi_ssoft_state_free " string peer of > ddi_isoft_state_free(9F) > > ddi_isoft_state_fini " typed equivalent of > ddi_soft_state_fini(9F) > ddi_ssoft_state_fini " string peer of > ddi_isoft_state_fini(9F) > > ddi_isoft_state_unassigned " return first unassigned > item of an ddi_isoft_state. > > 4.5 Interface Prototypes: (defined in sunddi.h): > > typedef struct __ddi_isoft_state ddi_isoft_state; > typedef struct __ddi_ssoft_state ddi_ssoft_state; > > int > ddi_isoft_state_init(ddi_isoft_state **state_p, > size_t size, size_t n_items); > int > ddi_ssoft_state_init(ddi_ssoft_state **state_p, > size_t size, int hash_sz); > > int > ddi_isoft_state_zalloc(ddi_isoft_state *state, int index); > int > ddi_ssoft_state_zalloc(ddi_ssoft_state *state, const char *stringkey); > > void * > ddi_isoft_state_get(ddi_isoft_state *state, int index); > void * > ddi_ssoft_state_get(ddi_ssoft_state *state, const char *stringkey); > > void > ddi_isoft_state_free(ddi_isoft_state *state, int index); > void > ddi_ssoft_state_free(ddi_ssoft_state *state, const char *stringkey); > > void > ddi_isoft_state_fini(ddi_isoft_state **state_p); > void > ddi_ssoft_state_fini(ddi_ssoft_state **state_p); > > int > ddi_isoft_state_unassigned(ddi_isoft_state **state_p); > > 4.4 Example: > > Assuming that 'tpaddr' is the portion SCSI > 'unit-address' string, A SCSA HBA driver can use the following to > access its private specific context. > > ss = ddi_isoft_state_get(xx_state, ddi_get_instance(devi)); > tpss = ddi_ssoft_state_get(ss->ss_tpstate, tpaddr); > > Where xx_state would be setup in _init(9E) via > ddi_isoft_state_init(9F), and ss_tpstate would be setup in > xxattach(9E) via ddi_ssoft_state_init(9F). > > 4.5 Alternatives: > > An alternative 'compound' approach was considered and prototyped. > The 'compound' approach implemented a set of 'ddi_strid_*(9E)' > string-to-id mapping interfaces, and then used the existing > "indexed" ddi_soft_state(9F) to get from an 'id' to the soft state. > The 'ddi_strid_(9E)' interfaces, while a good fit for some > applications, was found to increase SCSI host adapter driver > complexity compared to the proposed ddi_ssoft_state(9F) interfaces. > The proposed ddi_ssoft_state(9F) interfaces also present a familiar > API: they are based on the well-established ddi_soft_state(9F) API > structure. > > 4.6 Man Pages > > See Appendix A. Some of the examples are TBS at this point, more > meaningful examples should be provided before these interfaces are > promoted. > > 4.7 Release Binding > > Micro/patch binding is requested. > > 4.8 References > > [1] PSARC case that introduced modhash interfaces > PSARC/1998/212 Extensions to Device Autoconfiguration [John Danielson] > http://sac.sfbay/PSARC/1998/212 > http://www.opensolaris.org/os/community/arc/caselog/PSARC/1998/212 > > > Appendix A: > > A.1: New ddi_ssoft_state(9F) > ========================================= > :r ddi_ssoft_state.9f > > Kernel Functions for Drivers ddi_ssoft_state(9F) > > NAME > ddi_ssoft_state, ddi_ssoft_state_get, ddi_ssoft_state_fini, > ddi_ssoft_state_free, ddi_ssoft_state_init, ddi_ssoft_state_zalloc - > driver string soft state utility routines > > SYNOPSIS > #include > #include > > typedef struct __ddi_ssoft_state ddi_ssoft_state; > > int ddi_ssoft_state_init(ddi_ssoft_state **state_p, > size_t size, size_t hash_sz); > > int ddi_ssoft_state_zalloc(ddi_ssoft_state *state, int stringkey); > > void *ddi_ssoft_state_get(ddi_ssoft_state *state, int stringkey); > > void ddi_ssoft_state_free(ddi_ssoft_state *state, int stringkey); > > void ddi_ssoft_state_fini(ddi_ssoft_state **state_p); > > INTERFACE LEVEL > Solaris DDI specific (Solaris DDI). > > PARAMETERS > state_p Address of the opaque state pointer which will be > initialized by ddi_ssoft_state_init() to point to > implementation dependent data. > > state An opaque pointer to implementation-dependent data that > describes the soft state. > > size Size of the soft state which will be allocated by subsequent > calls to ddi_ssoft_state_zalloc(); zero is illegal. > > hash_sz The number of hashed lists which will be allocated; > zero is not allowed. There is a performance > .vs. space tradeoff in the selection of a hash_sz > value: for 'n' soft state structures, on average each > ddi_ssoft_state_get will need to traverse 'n'/hash_sz/2 > entries to locate the requested soft state structure. > > stringkey The string key for the soft state structure; usually > associated with some sub-portion of the device's > 'unit-address'. > > DESCRIPTION > > Some drivers must maintain state information at a granularity > unrelated to ddi_get_instance(9F) instance numbers or dev_info > node ddi_get_driver_private(9F) data. > > When the needed soft state granularity can be associated with a > string, such as a sub-portion of a 'unit-address' string, the > ddi_ssoft_state(9F) utility routines can be used to help device > drivers manage the space used by the driver to hold such state > information. > > For example, if a SCSI HBA driver maintains state for each > target-port address in a separate state structure, for transports > with complex target-port address representations these routines > can be used to dynamically allocate and deallocate a separate > structure for each target-port as scsi_device(9S) associated with > the target-port are initialized by tran_tgt_init(9E) and and freed > by tran_tgt_free(9E). > > To use the routines, the driver writer needs to declare a state > pointer, state_p, which the implementation uses as a place to hang > a set of per-driver structures; everything else is managed by > these routines. > > The routine ddi_ssoft_state_init() is usually called in the > driver's attach(9E) routine to initialize the state pointer, and set > the size of the soft state structure. > > The routine ddi_ssoft_state_zalloc() is called to allocate a soft > state. The routine is passed a string key, which is used to refer > to the structure in subsequent calls to ddi_ssoft_state_get() and > ddi_ssoft_state_free(). The string key is usually some sub-portion > of a 'unit-address'. The routine attempts to allocate space for > the new structure, and if the space allocation was successful, > DDI_SUCCESS is returned to the caller. Returned memory is zeroed. > > A pointer to the space previously allocated for a soft state > structure can be obtained by calling ddi_ssoft_state_get() with the > appropriate string key. > > The space used by a given soft state structure can be returned to > the system using ddi_ssoft_state_free(). > > The space used by all the soft state structures allocated on a > given state pointer, together with the housekeeping information > used by the implementation can be returned to the system using > ddi_ssoft_state_fini(). This routine is typically called from the > driver's detach(9E) routine. > > The ddi_ssoft_state_zalloc(), ddi_ssoft_state_free() and > ddi_ssoft_state_get() routines coordinate access to the underlying > data structures in an MT-safe fashion, thus no additional locks > should be necessary. > > RETURN VALUES > > ddi_ssoft_state_get() > > NULL The requested soft state structure was not > allocated at the time of the call. > > pointer The pointer to the soft state structure. > > > > ddi_ssoft_state_init() > > 0 The allocation was successful. > > EINVAL Either the size parameter was zero, or the > state_p parameter was invalid. > > > ddi_ssoft_state_zalloc() > > DDI_SUCCESS The allocation was successful. > > DDI_FAILURE The routine failed to allocate the storage > required; either the state parameter was > invalid, the string key was NULL, or an attempt > was made to allocate using a string key that > was already allocated. > > > CONTEXT > The ddi_ssoft_state_init() and ddi_ssoft_state_alloc() functions > can be called from user or kernel context only, since they may > internally call kmem_zalloc(9F) with the KM_SLEEP flag. > > The ddi_ssoft_state_fini(), ddi_ssoft_state_free() and > ddi_ssoft_state_get() routines can be called from any driver > context. > > EXAMPLES > <> > > SEE ALSO > _fini(9E), _init(9E), attach(9E), detach(9E), > ddi_get_instance(9F), ddi_soft_state(9E), ddi_isoft_state(9E), > getminor(9F), kmem_zalloc(9F) > > NOTES > If necessary, a hierarchy of state structures can be constructed > by embedding state pointers in higher order state structures. > > DIAGNOSTICS > <> > > All of the messages described below usually indicate bugs in the > driver and should not appear in normal operation of the system. > > WARNING: ddi_ssoft_state_zalloc: bad handle > WARNING: ddi_ssoft_state_free: bad handle > WARNING: ddi_ssoft_state_fini: bad handle > > The implementation-dependent information kept in the state > variable is corrupt. > > WARNING: ddi_ssoft_state_free: null handle > WARNING: ddi_ssoft_state_fini: null handle > > The routine has been passed a null or corrupt state pointer. > Check that ddi_ssoft_state_init() has been called. > > WARNING: ddi_ssoft_state_free: string key '%s' not allocated > > The routine has been asked to free a string key which is not > allocated. The message prints out the invalid string key. > > A.2: New ddi_isoft_state(9F) > ========================================= > :r ddi_isoft_state.9f > > Kernel Functions for Drivers ddi_isoft_state(9F) > > NAME > ddi_isoft_state, ddi_isoft_state_get, ddi_isoft_state_fini, > ddi_isoft_state_free, ddi_isoft_state_init, ddi_isoft_state_zalloc - > driver indexed soft state utility routines > > SYNOPSIS > #include > #include > > typedef struct __ddi_isoft_state ddi_isoft_state; > > int ddi_isoft_state_init(ddi_isoft_state **state_p, > size_t size, size_t n_items); > > int ddi_isoft_state_zalloc(ddi_isoft_state *state, int item); > > void *ddi_isoft_state_get(ddi_isoft_state *state, int item); > > void ddi_isoft_state_free(ddi_isoft_state *state, int item); > > void ddi_isoft_state_fini(ddi_isoft_state **state_p); > > INTERFACE LEVEL > Solaris DDI specific (Solaris DDI). > > PARAMETERS > state_p Address of the opaque state pointer which will be > initialized by ddi_isoft_state_init() to point to > implementation dependent data. > > state An opaque pointer to implementation-dependent data that > describes the soft state. > > size Size of the item which will be allocated by subsequent > calls to ddi_isoft_state_zalloc(). > > n_items A hint of the number of items which will be > preallocated; zero is allowed. > > item The item number for the state structure; usually the > instance number of the associated devinfo node. > > DESCRIPTION > Most device drivers maintain state information with each instance > of the device they control; for example, a soft copy of a device > control register, a mutex that must be held while accessing a > piece of hardware, a partition table, or a unit structure. These > utility routines are intended to help device drivers manage the > space used by the driver to hold such state information. > > For example, if the driver holds the state of each instance in a > single state structure, these routines can be used to dynamically > allocate and deallocate a separate structure for each instance of > the driver as the instance is attached and detached. > > To use the routines, the driver writer needs to declare a state > pointer, state_p, which the implementation uses as a place to hang > a set of per-driver structures; everything else is managed by > these routines. > > The routine ddi_isoft_state_init() is usually called in the > driver's _init(9E) routine to initialize the state pointer, set > the size of the soft state structure, and to allow the driver to > pre-allocate a given number of such structures if required. > > The routine ddi_isoft_state_zalloc() is usually called in the > driver's attach(9E) routine. The routine is passed an item number > which is used to refer to the structure in subsequent calls to > ddi_isoft_state_get() and ddi_isoft_state_free(). The item number is > usually just the instance number of the devinfo node, obtained > with ddi_get_instance(9F). The routine attempts to allocate space > for the new structure, and if the space allocation was successful, > DDI_SUCCESS is returned to the caller. Returned memory is zeroed. > > A pointer to the space previously allocated for a soft state > structure can be obtained by calling ddi_isoft_state_get() with the > appropriate item number. > > The space used by a given soft state structure can be returned to > the system using ddi_isoft_state_free(). This routine is usually > called from the driver's detach(9E) entry point. > > The space used by all the soft state structures allocated on a > given state pointer, together with the housekeeping information > used by the implementation can be returned to the system using > ddi_isoft_state_fini(). This routine can be called from the > driver's _fini(9E) routine. > > The ddi_isoft_state_zalloc(), ddi_isoft_state_free() and > ddi_isoft_state_get() routines coordinate access to the underlying > data structures in an MT-safe fashion, thus no additional locks > should be necessary. > > RETURN VALUES > > ddi_isoft_state_get() > > NULL The requested state structure was not allocated > at the time of the call. > > pointer The pointer to the state structure. > > > > ddi_isoft_state_init() > > 0 The allocation was successful. > > EINVAL Either the size parameter was zero, or the > state_p parameter was invalid. > > > ddi_isoft_state_zalloc() > > DDI_SUCCESS The allocation was successful. > > DDI_FAILURE The routine failed to allocate the storage > required; either the state parameter was > invalid, the item number was negative, or an > attempt was made to allocate an item number > that was already allocated. > > > CONTEXT > The ddi_isoft_state_init() and ddi_isoft_state_alloc() functions > can be called from user or kernel context only, since they may > internally call kmem_zalloc(9F) with the KM_SLEEP flag. > > The ddi_isoft_state_fini(), ddi_isoft_state_free() and > ddi_isoft_state_get() routines can be called from any driver > context. > > EXAMPLES > Example 1 Creating and Removing Data Structures > > The following example shows how the routines described above can > be used in terms of the driver entry points of a character-only > driver. The example concentrates on the portions of the code that > deal with creating and removing the driver's data structures. > > > typedef struct { > volatile caddr_t *csr; /* device registers */ > kmutex_t csr_mutex; /* protects 'csr' field */ > unsigned int state; > dev_info_t *dip; /* back pointer to devinfo */ > } devstate_t; > static ddi_isoft_state *statep; > > int > _init(void) > { > int error; > > error = ddi_isoft_state_init(&statep, sizeof (devstate_t), 0); > if (error != 0) > return (error); > if ((error = mod_install(&modlinkage)) != 0) > ddi_isoft_state_fini(&statep); > return (error); > } > > int > _fini(void) > { > int error; > > if ((error = mod_remove(&modlinkage)) != 0) > return (error); > ddi_isoft_state_fini(&statep); > return (0); > } > > static int > xxattach(dev_info_t *dip, ddi_attach_cmd_t cmd) > { > int instance; > devstate_t *softc; > > switch (cmd) { > case DDI_ATTACH: > instance = ddi_get_instance(dip); > if (ddi_isoft_state_zalloc(statep, instance) != > DDI_SUCCESS) > return (DDI_FAILURE); > softc = ddi_isoft_state_get(statep, instance); > softc->dip = dip; > ... > return (DDI_SUCCESS); > default: > return (DDI_FAILURE); > } > } > > static int > xxdetach(dev_info_t *dip, ddi_detach_cmd_t cmd) > { > int instance; > > switch (cmd) { > > case DDI_DETACH: > instance = ddi_get_instance(dip); > ... > ddi_isoft_state_free(statep, instance); > return (DDI_SUCCESS); > > default: > return (DDI_FAILURE); > } > } > > static int > xxopen(dev_t *devp, int flag, int otyp, cred_t *cred_p) > { > devstate_t *softc; > int instance; > > instance = getminor(*devp); > if ((softc = ddi_isoft_state_get(statep, instance)) == NULL) > return (ENXIO); > ... > softc->state |= XX_IN_USE; > ... > return (0); > } > > SEE ALSO > _fini(9E), _init(9E), attach(9E), detach(9E), > ddi_get_instance(9F), getminor(9F), kmem_zalloc(9F) > > WARNINGS > There is no attempt to validate the item parameter given to > ddi_isoft_state_zalloc() other than it must be a positive signed > integer. Therefore very large item numbers may cause the driver to > hang forever waiting for virtual memory resources that can never > be satisfied. > > NOTES > If necessary, a hierarchy of state structures can be constructed > by embedding state pointers in higher order state structures. > > DIAGNOSTICS > All of the messages described below usually indicate bugs in the > driver and should not appear in normal operation of the system. > > WARNING: ddi_isoft_state_zalloc: bad handle > WARNING: ddi_isoft_state_free: bad handle > WARNING: ddi_isoft_state_fini: bad handle > > The implementation-dependent information kept in the state > variable is corrupt. > > WARNING: ddi_isoft_state_free: null handle > WARNING: ddi_isoft_state_fini: null handle > > The routine has been passed a null or corrupt state pointer. > Check that ddi_isoft_state_init() has been called. > > WARNING: ddi_isoft_state_free: item %d not in range [0..%d] > > The routine has been asked to free an item which was never > allocated. The message prints out the invalid item number and the > acceptable range. > > > A.3: Changes to existing ddi_soft_state(9F) > ========================================= > :r!diff -U5 ddi_soft_state.9f.orig ddi_soft_state.9f > > --- ddi_soft_state.9f.orig Mon Nov 10 11:45:36 2008 > +++ ddi_soft_state.9f Tue Nov 11 18:08:26 2008 > @@ -224,22 +224,29 @@ > return (0); > } > > SEE ALSO > _fini(9E), _init(9E), attach(9E), detach(9E), > - ddi_get_instance(9F), getminor(9F), kmem_zalloc(9F) > + ddi_get_instance(9F), ddi_isoft_state(9E), ddi_ssoft_state(9E), > + getminor(9F), kmem_zalloc(9F) > > WARNINGS > There is no attempt to validate the item parameter given to > ddi_soft_state_zalloc() other than it must be a positive signed > integer. Therefore very large item numbers may cause the driver to > hang forever waiting for virtual memory resources that can never > be satisfied. > > + Use of the ddi_isoft_state(9E) interfaces is encouraged, it > + provides stronger typing. > + > NOTES > If necessary, a hierarchy of state structures can be constructed > by embedding state pointers in higher order state structures. > + > + Equivalent functionality with stronger typing is provided > + by the newer ddi_isoft_state(9F) interfaces. > > DIAGNOSTICS > All of the messages described below usually indicate bugs in the > driver and should not appear in normal operation of the system. > From Chris.Horne@Sun.COM Fri Dec 5 17:17:41 2008 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mB61HfLR020411 for ; Fri, 5 Dec 2008 17:17:41 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id mB61HeUf006120 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Fri, 5 Dec 2008 17:17:41 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KBF00209KXGW300@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 05 Dec 2008 18:17:40 -0700 (MST) Received: from brmea-mail-2.sun.com ([192.18.98.43]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KBF00GS9KXFD250@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 05 Dec 2008 18:17:39 -0700 (MST) Received: from fe-amer-10.sun.com ([192.18.109.80]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id mB61HdMO020447 for ; Sat, 06 Dec 2008 01:17:39 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0KBF00F01KQHG000@mail-amer.sun.com> (original mail from Chris.Horne@Sun.COM) for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 05 Dec 2008 18:17:39 -0700 (MST) Received: from sun.com ([129.150.35.140]) by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0KBF00GD9KWRGT10@mail-amer.sun.com>; Fri, 05 Dec 2008 18:17:16 -0700 (MST) Date: Fri, 05 Dec 2008 18:17:15 -0700 From: Chris Horne Subject: Re: ddi_ssoft_state(9F) and ddi_isoft_state(9F) [PSARC/2008/755 FastTrack timeout 12/17/2008] In-reply-to: <4939C44C.2090604@sun.com> Sender: Chris.Horne@Sun.COM To: "Garrett D'Amore" Cc: Christopher Horne , PSARC-ext@Sun.COM Message-id: <4939D29B.2060601@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-Accept-Language: en-us, en X-PMX-Version: 5.4.1.325704 References: <200812060002.mB602OEX021391@sac.sfbay.sun.com> <4939C44C.2090604@sun.com> User-Agent: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.4) Gecko/20040414 Status: RO Content-Length: 2351 Hi Garrett > I have the strong feeling that this case is trying to conflate a general > purpose hash table with soft state storage. While drivers could > potentially use a hash table for soft state storage, it seems that > creating a limited purpose set of routines which are really just > accessors into a hash table structure might not be the best approach. > > Most previous usages of "soft state" consider one soft state structure > per device-instance (dip), whereas it appears that this project > considers the needs of a nexus driver to store state for each of > potentially multiple child devices. This seems more like a general data > structure problem to me. > > What is wrong with the modhash routines as they are today? Can't > drivers just use those directly? The 'modhash.h' interfaces are not ARCed, they are not documented (AFAIK), they do not present an API familiar to driver writers, and their generality brings with it complexity. The ddi_ssoft_state* implementation is build on top of 'modhash.h'. The eventual promotion of the ddi_ssoft_state* interfaces is tied to SCSAv3 enhancements, where a DDI compliant mechanism to support soft_state for complex target port unit addresses is needed. The proposed ddi_ssoft_state* interfaces have the same API profile as the current ddi_soft_state* API, making them (in my opinion) more appropriate for inclusion as part of the DDI: they minimize the volume of solaris-specific API that a driver writer needs to understand by building on a well-established profile. > Put another way, how does this case improve life for drivers beyond just > using modhash directly? > > I'm also not sure I understand the point of ddi_get_isoft_state. Is the > "typed" adjective here a reference to the fact that the first argument > (the master state pointer) is typed to something other than void *? (Do > we really believe that there significant cases here where the type > safety this would afford justifies the cost of introducing another > interface? I confess that I'm very skeptical of that.) I added the ddi_isoft_state* interfaces so that the same level of type checking was available for both ddi_ssoft_state* and the indexed case. If a consensus objection persists, I can drop ddi_isoft_state* from the proposal. -Chris RE: http://sac.eng/PSARC/2008/755/ From gdamore@sun.com Fri Dec 5 17:23:36 2008 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mB61NaDJ020464 for ; Fri, 5 Dec 2008 17:23:36 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id mB61NZa8011810 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Fri, 5 Dec 2008 17:23:36 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KBF0030FL7AIA00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 05 Dec 2008 18:23:34 -0700 (MST) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KBF00G0ZL79D050@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 05 Dec 2008 18:23:33 -0700 (MST) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id mB61NXt2027976 for ; Fri, 05 Dec 2008 17:23:33 -0800 (PST) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0KBF00301L3SQT00@fe-sfbay-10.sun.com> (original mail from gdamore@sun.com) for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 05 Dec 2008 17:23:33 -0800 (PST) Received: from [10.7.251.172] by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0KBF00GQYL78IQB0@fe-sfbay-10.sun.com>; Fri, 05 Dec 2008 17:23:32 -0800 (PST) Date: Fri, 05 Dec 2008 17:16:30 -0800 From: "Garrett D'Amore" Subject: Re: ddi_ssoft_state(9F) and ddi_isoft_state(9F) [PSARC/2008/755 FastTrack timeout 12/17/2008] In-reply-to: <4939D29B.2060601@sun.com> Sender: Garrett.Damore@sun.com To: Chris Horne Cc: Christopher Horne , PSARC-ext@sun.com Message-id: <4939D26E.8010308@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200812060002.mB602OEX021391@sac.sfbay.sun.com> <4939C44C.2090604@sun.com> <4939D29B.2060601@sun.com> User-Agent: Thunderbird 2.0.0.14 (X11/20080616) Status: RO Content-Length: 3363 Chris Horne wrote: > Hi Garrett > > >> I have the strong feeling that this case is trying to conflate a general >> purpose hash table with soft state storage. While drivers could >> potentially use a hash table for soft state storage, it seems that >> creating a limited purpose set of routines which are really just >> accessors into a hash table structure might not be the best approach. >> >> Most previous usages of "soft state" consider one soft state structure >> per device-instance (dip), whereas it appears that this project >> considers the needs of a nexus driver to store state for each of >> potentially multiple child devices. This seems more like a general data >> structure problem to me. >> >> What is wrong with the modhash routines as they are today? Can't >> drivers just use those directly? >> > > > The 'modhash.h' interfaces are not ARCed, they are not documented > (AFAIK), they do not present an API familiar to driver writers, and > their generality brings with it complexity. > > The ddi_ssoft_state* implementation is build on top of 'modhash.h'. > > The eventual promotion of the ddi_ssoft_state* interfaces is tied to > SCSAv3 enhancements, where a DDI compliant mechanism to support > soft_state for complex target port unit addresses is needed. > > The proposed ddi_ssoft_state* interfaces have the same API profile as > the current ddi_soft_state* API, making them (in my opinion) more > appropriate for inclusion as part of the DDI: they minimize the volume > of solaris-specific API that a driver writer needs to understand by > building on a well-established profile. > But if what is needed is generic hash table facilities, then wouldn't it be more appropriate to uplevel the modhash routines or create more general purpose wrappers. I'm a big fan of reducing duplication in the DDI. While your goals seem to be inline with that, I'm not sure that this project achieves it as well as just providing the generic hash facility would. FWIW, a lot of drivers these days don't even bother with the ddi_soft_state stuff. I find them personally rather clumsy, and opt to use either my own stuff, or more commonly, if I can just use the private state pointer in the dev_info, I just use that. > > >> Put another way, how does this case improve life for drivers beyond just >> using modhash directly? >> >> I'm also not sure I understand the point of ddi_get_isoft_state. Is the >> "typed" adjective here a reference to the fact that the first argument >> (the master state pointer) is typed to something other than void *? (Do >> we really believe that there significant cases here where the type >> safety this would afford justifies the cost of introducing another >> interface? I confess that I'm very skeptical of that.) >> > > > I added the ddi_isoft_state* interfaces so that the same level of type > checking was available for both ddi_ssoft_state* and the indexed case. > > If a consensus objection persists, I can drop ddi_isoft_state* from the > proposal. > It seems like a small benefit, and comes at the cost of "seemingly arbitrary" difference between the legacy API and new API. (Possibly creating additional burden for folks working to support drivers on multiple releases of Solaris... of course, we *never* backport drivers, right? :-) -- Garrett From Chris.Horne@sun.com Fri Dec 5 18:29:20 2008 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mB62TKXO021353 for ; Fri, 5 Dec 2008 18:29:20 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id mB62TJL9016249 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Fri, 5 Dec 2008 18:29:20 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KBF00A0HO8UD700@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 05 Dec 2008 19:29:18 -0700 (MST) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KBF00GEFO8UD890@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 05 Dec 2008 19:29:18 -0700 (MST) Received: from fe-amer-10.sun.com ([192.18.109.80]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id mB62TIMj003457 for ; Sat, 06 Dec 2008 02:29:18 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0KBF00901NZVT500@mail-amer.sun.com> (original mail from Chris.Horne@Sun.COM) for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 05 Dec 2008 19:28:43 -0700 (MST) Received: from sun.com ([129.150.35.140]) by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0KBF00G1BO7QGT20@mail-amer.sun.com>; Fri, 05 Dec 2008 19:28:43 -0700 (MST) Date: Fri, 05 Dec 2008 19:28:38 -0700 From: Chris Horne Subject: Re: ddi_ssoft_state(9F) and ddi_isoft_state(9F) [PSARC/2008/755 FastTrack timeout 12/17/2008] In-reply-to: <4939C44C.2090604@sun.com> Sender: Chris.Horne@sun.com To: "Garrett D'Amore" Cc: Christopher Horne , PSARC-ext@sun.com Message-id: <4939E356.40200@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-Accept-Language: en-us, en X-PMX-Version: 5.4.1.325704 References: <200812060002.mB602OEX021391@sac.sfbay.sun.com> <4939C44C.2090604@sun.com> User-Agent: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.4) Gecko/20040414 Status: RO Content-Length: 1395 --%<-- > Most previous usages of "soft state" consider one soft state structure > per device-instance (dip), whereas it appears that this project > considers the needs of a nexus driver to store state for each of > potentially multiple child devices. To clarify - that is not where the need comes from, the ddi_[gs]et_parent_data interfaces solve that problem. When multiple levels of addressing are compressed into a single child unit-addresses "@,," the parent nexus driver may need to maintain separate state that relates to just one component of the child's unit-address: multiple children that share the same "" string value resolve to the same soft_state. This is a nexus driver problem. SCSA HBA drivers are the only public form of nexus driver. At last count we had about 30 SCSA HBA drivers in ON (compared to about 40 NIC drivers). Most new SCSA HBA drivers will face the "" string soft-state problem due to complexity of target-port addresses on most modern SCSI transports. I think the need for ddi_ssoft_state* exists, just like the need for the current indexed ddi_soft_state(9F) exists. The SCSA HBA driver writer does not need (or want) a full-blown generic hash table facility: the need is for simple, easy to understand, easy to document interfaces for "" string soft_state. -Chris From gdamore@sun.com Fri Dec 5 19:15:16 2008 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mB63FG7F021908 for ; Fri, 5 Dec 2008 19:15:16 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id mB63FFEH021039 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Fri, 5 Dec 2008 19:15:16 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KBF00F0FQDFCT00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 05 Dec 2008 20:15:15 -0700 (MST) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KBF00GWZQDED0A0@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 05 Dec 2008 20:15:14 -0700 (MST) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id mB63FDUW002555 for ; Fri, 05 Dec 2008 19:15:14 -0800 (PST) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0KBF00701Q5K3B00@fe-sfbay-10.sun.com> (original mail from gdamore@sun.com) for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 05 Dec 2008 19:15:13 -0800 (PST) Received: from [10.7.251.172] by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0KBF00EIBQDDEW60@fe-sfbay-10.sun.com>; Fri, 05 Dec 2008 19:15:13 -0800 (PST) Date: Fri, 05 Dec 2008 19:08:11 -0800 From: "Garrett D'Amore" Subject: Re: ddi_ssoft_state(9F) and ddi_isoft_state(9F) [PSARC/2008/755 FastTrack timeout 12/17/2008] In-reply-to: <4939E356.40200@sun.com> Sender: Garrett.Damore@sun.com To: Chris Horne Cc: Christopher Horne , PSARC-ext@sun.com Message-id: <4939EC9B.10105@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200812060002.mB602OEX021391@sac.sfbay.sun.com> <4939C44C.2090604@sun.com> <4939E356.40200@sun.com> User-Agent: Thunderbird 2.0.0.14 (X11/20080616) Status: RO Content-Length: 2956 Chris Horne wrote: > --%<-- > >> Most previous usages of "soft state" consider one soft state structure >> per device-instance (dip), whereas it appears that this project >> considers the needs of a nexus driver to store state for each of >> potentially multiple child devices. >> > > To clarify - that is not where the need comes from, the > ddi_[gs]et_parent_data interfaces solve that problem. > > When multiple levels of addressing are compressed into a single child > unit-addresses > > "@,," > > the parent nexus driver may need to maintain separate state that > relates to just one component of the child's unit-address: multiple > children that share the same "" string value resolve to > the same soft_state. > > This is a nexus driver problem. SCSA HBA drivers are the only public > form of nexus driver. At last count we had about 30 SCSA HBA drivers > in ON (compared to about 40 NIC drivers). Most new SCSA HBA drivers > will face the "" string soft-state problem due to > complexity of target-port addresses on most modern SCSI transports. > > I think the need for ddi_ssoft_state* exists, just like the need for > the current indexed ddi_soft_state(9F) exists. The SCSA HBA driver > writer does not need (or want) a full-blown generic hash table facility: > the need is for simple, easy to understand, easy to document interfaces > for "" string soft_state. > You make it sound like the "full blown hash table" would necessarily have a complex API. It isn't clear to me that a simple hash facility (mapping objects to strings) has to be that complex. No matter though. My suggestions at this point would be for name changes (ssoft_state seems unclear to me -- what's the leading "s" for? "string"?), and elimination of the "indexed" variant. So instead you'd have: ddi_strhash_init() ddi_strhash_fini() ddi_strhash_zalloc() ddi_strhash_free() ddi_strhash_lookup() (I'm not dead set on those names either-- I just think they are little better than the original proposal. I'm sure someone could come up with still better options.) Really, the only thing this has that is different from a "standard" hash table is that these versions the hash table implementation manages the memory internally, and so you can clobber the whole table with a single _fini. In my mind, the penalties for this generic interface often aren't worth the benefits. For example, in ddi_soft_state, you can't walk the entire set of objects, which makes the logic less than useful for a lot of tasks where it would other wise be a bit handier. If you change the code so that modules have to perform their own memory management, you could replace the ddi_strhash_zalloc() and _free() with ddi_strhash_set(), and then add generic walkers, and suddenly you've got a much, much more flexible API, with little additional complexity. -- Garrett From edward.pilatowicz@sun.com Mon Dec 8 20:51:27 2008 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mB94pQTn002811 for ; Mon, 8 Dec 2008 20:51:26 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id mB94pIjA019774; Tue, 9 Dec 2008 04:51:23 GMT Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KBL00107ETM5Z00@nwk-avmta-2.sfbay.sun.com>; Mon, 08 Dec 2008 20:51:22 -0800 (PST) Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.63]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KBL00C21ETLJXA0@nwk-avmta-2.sfbay.sun.com>; Mon, 08 Dec 2008 20:51:21 -0800 (PST) Received: from jurassic-x4600.sfbay.sun.com (localhost [127.0.0.1]) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id mB94pLMG872719 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 08 Dec 2008 20:51:21 -0800 (PST) Received: (from edp@localhost) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3/Submit) id mB94pLqw872716; Mon, 08 Dec 2008 20:51:21 -0800 (PST) Date: Mon, 08 Dec 2008 20:51:21 -0800 From: Edward Pilatowicz Subject: Re: ddi_ssoft_state(9F) and ddi_isoft_state(9F) [PSARC/2008/755 FastTrack timeout 12/17/2008] In-reply-to: <200812060002.mB602OEX021391@sac.sfbay.sun.com> To: Christopher Horne Cc: PSARC-ext@sun.com Message-id: <20081209045121.GD852061@eng.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <200812060002.mB602OEX021391@sac.sfbay.sun.com> X-Authentication-warning: jurassic-x4600.sfbay.sun.com: edp set sender to edward.pilatowicz@sun.com using -f User-Agent: Mutt/1.5.17 (2007-11-01) Status: RO Content-Length: 24925 hey chris, the current ddi soft state interfaces are "special" in my mind because i thought they were designed to be "fast". (note that i wasn't around for the original design, so my opinion is based off the implmentation.) their implementation uses direct array indexing, avoids locking, and avoids free memory, all seemingly in the name of soft state lookup speed. i guess i'm worried that other people may be making the same assumptions about the performance of the ddi soft state interfaces, and these assumptions don't really carry over to the new interfaces since they are really a thin wrapper around modhash. hence, i'd rather see something more straitforward, like some version/subset of the modhash interfaces promoted to public, or some new ddi*hash* interfaces. ed On Fri, Dec 05, 2008 at 04:02:24PM -0800, Christopher Horne wrote: > I am sponsoring the following fasttrack for myself. Micro/patch > binding is requested. The timer is set to expire on Dec. 17 2008. > > -Chris > > > 1. Introduction > 1.1. Project/Component Working Name: > ddi_ssoft_state(9F) and ddi_isoft_state(9F) > 1.2. Name of Document Author/Supplier: > Author: Chris Horne > 1.3. Date of This Document: > Tue Nov 11 18:10:01 MST 2008 > > 4. Technical Description > > 4.1 Background: > > ddi_soft_state(9F) provides utility interfaces to help device > drivers manage different context. When small integer values, like > ddi_get_instance(9F) return values, are directly related to > context, the current "indexed" soft_state interfaces work well. > > 4.2 Problem: > > Two IEEE-1275 '@unit-address' issues can make the current > ddi_soft_state(9F) support ineffective: > > o Compression: Some IEEE-1275 bindings, like the SCSI > '@,' notation, compress multiple addressing > levels into a single 'unit-address' string. If a driver needs > to maintain soft state at intermediate levels, like the > level, there is no instance number to use with > ddi_soft_state(9F), and no dev_info node to allow use of > ddi_[gs]et_driver_private(9F) or ddi_[gs]et_parent_data). > > o Complexity: An addressing level may be complex: both > non-numeric, sparse, and large. > > This situation has forced drivers to maintain soft state context in > ways that no longer fit the ddi_soft_state(9F) model. > > An example is SCSA HBA unit-address context. For the > initial SCSI Parallel Interconnect transport the was > as an integer limited to the [0-15] range, and ddi_soft_state(9F) > worked well. On modern transports the complexity of the > space has mushroomed, as shown below, and the > space is now 64-bits and sparse. > > SPI: disk@0,0 > WWN: disk@w216000c0ff8047dd,0 > GUID: disk@g600c0ff0000000000047dd270fdf0901 (lun identity: MPxIO) > ISCSI: disk@0000iqn.1986-03.com.sun%3A02%3Ac8a82272-b354-c913-80f9-db9cb378a6f60001,0 > > 4.3 Proposal: > > The proposal is to extend the ddi_soft_state(9F) programming model > by providing "string" based peers to todays "indexed" soft_state > interfaces. In addition to "string" support, "indexed" interfaces > with stronger typing are proposed. > > Implementation of proposed ddi_ssoft_state(9F) interfaces is built > on top of the 'modhash' interfaces introduced by [1]. > > 4.4 Proposed Interfaces: > > ------------------------------------------------------------------ > Interface Name Comm.Lev. Comments > ------------------------------------------------------------------ > > ddi_isoft_state Private "indexed" soft_state > > ddi_ssoft_state " "string" soft_state > > > ddi_isoft_state_init " typed equivalent of > ddi_soft_state_init(9F) > ddi_ssoft_state_init " string peer of > ddi_isoft_state_init(9F) > > > ddi_isoft_state_zalloc " typed equivalent of > ddi_soft_state_zalloc(9F) > ddi_ssoft_state_zalloc " string peer of > ddi_isoft_state_zalloc(9F) > > ddi_isoft_state_get " typed equivalent of > ddi_soft_state_get(9F) > ddi_ssoft_state_get " string peer of > ddi_isoft_state_get(9F) > > ddi_isoft_state_free " typed equivalent of > ddi_soft_state_free(9F) > ddi_ssoft_state_free " string peer of > ddi_isoft_state_free(9F) > > ddi_isoft_state_fini " typed equivalent of > ddi_soft_state_fini(9F) > ddi_ssoft_state_fini " string peer of > ddi_isoft_state_fini(9F) > > ddi_isoft_state_unassigned " return first unassigned > item of an ddi_isoft_state. > > 4.5 Interface Prototypes: (defined in sunddi.h): > > typedef struct __ddi_isoft_state ddi_isoft_state; > typedef struct __ddi_ssoft_state ddi_ssoft_state; > > int > ddi_isoft_state_init(ddi_isoft_state **state_p, > size_t size, size_t n_items); > int > ddi_ssoft_state_init(ddi_ssoft_state **state_p, > size_t size, int hash_sz); > > int > ddi_isoft_state_zalloc(ddi_isoft_state *state, int index); > int > ddi_ssoft_state_zalloc(ddi_ssoft_state *state, const char *stringkey); > > void * > ddi_isoft_state_get(ddi_isoft_state *state, int index); > void * > ddi_ssoft_state_get(ddi_ssoft_state *state, const char *stringkey); > > void > ddi_isoft_state_free(ddi_isoft_state *state, int index); > void > ddi_ssoft_state_free(ddi_ssoft_state *state, const char *stringkey); > > void > ddi_isoft_state_fini(ddi_isoft_state **state_p); > void > ddi_ssoft_state_fini(ddi_ssoft_state **state_p); > > int > ddi_isoft_state_unassigned(ddi_isoft_state **state_p); > > 4.4 Example: > > Assuming that 'tpaddr' is the portion SCSI > 'unit-address' string, A SCSA HBA driver can use the following to > access its private specific context. > > ss = ddi_isoft_state_get(xx_state, ddi_get_instance(devi)); > tpss = ddi_ssoft_state_get(ss->ss_tpstate, tpaddr); > > Where xx_state would be setup in _init(9E) via > ddi_isoft_state_init(9F), and ss_tpstate would be setup in > xxattach(9E) via ddi_ssoft_state_init(9F). > > 4.5 Alternatives: > > An alternative 'compound' approach was considered and prototyped. > The 'compound' approach implemented a set of 'ddi_strid_*(9E)' > string-to-id mapping interfaces, and then used the existing > "indexed" ddi_soft_state(9F) to get from an 'id' to the soft state. > The 'ddi_strid_(9E)' interfaces, while a good fit for some > applications, was found to increase SCSI host adapter driver > complexity compared to the proposed ddi_ssoft_state(9F) interfaces. > The proposed ddi_ssoft_state(9F) interfaces also present a familiar > API: they are based on the well-established ddi_soft_state(9F) API > structure. > > 4.6 Man Pages > > See Appendix A. Some of the examples are TBS at this point, more > meaningful examples should be provided before these interfaces are > promoted. > > 4.7 Release Binding > > Micro/patch binding is requested. > > 4.8 References > > [1] PSARC case that introduced modhash interfaces > PSARC/1998/212 Extensions to Device Autoconfiguration [John Danielson] > http://sac.sfbay/PSARC/1998/212 > http://www.opensolaris.org/os/community/arc/caselog/PSARC/1998/212 > > > Appendix A: > > A.1: New ddi_ssoft_state(9F) > ========================================= > :r ddi_ssoft_state.9f > > Kernel Functions for Drivers ddi_ssoft_state(9F) > > NAME > ddi_ssoft_state, ddi_ssoft_state_get, ddi_ssoft_state_fini, > ddi_ssoft_state_free, ddi_ssoft_state_init, ddi_ssoft_state_zalloc - > driver string soft state utility routines > > SYNOPSIS > #include > #include > > typedef struct __ddi_ssoft_state ddi_ssoft_state; > > int ddi_ssoft_state_init(ddi_ssoft_state **state_p, > size_t size, size_t hash_sz); > > int ddi_ssoft_state_zalloc(ddi_ssoft_state *state, int stringkey); > > void *ddi_ssoft_state_get(ddi_ssoft_state *state, int stringkey); > > void ddi_ssoft_state_free(ddi_ssoft_state *state, int stringkey); > > void ddi_ssoft_state_fini(ddi_ssoft_state **state_p); > > INTERFACE LEVEL > Solaris DDI specific (Solaris DDI). > > PARAMETERS > state_p Address of the opaque state pointer which will be > initialized by ddi_ssoft_state_init() to point to > implementation dependent data. > > state An opaque pointer to implementation-dependent data that > describes the soft state. > > size Size of the soft state which will be allocated by subsequent > calls to ddi_ssoft_state_zalloc(); zero is illegal. > > hash_sz The number of hashed lists which will be allocated; > zero is not allowed. There is a performance > .vs. space tradeoff in the selection of a hash_sz > value: for 'n' soft state structures, on average each > ddi_ssoft_state_get will need to traverse 'n'/hash_sz/2 > entries to locate the requested soft state structure. > > stringkey The string key for the soft state structure; usually > associated with some sub-portion of the device's > 'unit-address'. > > DESCRIPTION > > Some drivers must maintain state information at a granularity > unrelated to ddi_get_instance(9F) instance numbers or dev_info > node ddi_get_driver_private(9F) data. > > When the needed soft state granularity can be associated with a > string, such as a sub-portion of a 'unit-address' string, the > ddi_ssoft_state(9F) utility routines can be used to help device > drivers manage the space used by the driver to hold such state > information. > > For example, if a SCSI HBA driver maintains state for each > target-port address in a separate state structure, for transports > with complex target-port address representations these routines > can be used to dynamically allocate and deallocate a separate > structure for each target-port as scsi_device(9S) associated with > the target-port are initialized by tran_tgt_init(9E) and and freed > by tran_tgt_free(9E). > > To use the routines, the driver writer needs to declare a state > pointer, state_p, which the implementation uses as a place to hang > a set of per-driver structures; everything else is managed by > these routines. > > The routine ddi_ssoft_state_init() is usually called in the > driver's attach(9E) routine to initialize the state pointer, and set > the size of the soft state structure. > > The routine ddi_ssoft_state_zalloc() is called to allocate a soft > state. The routine is passed a string key, which is used to refer > to the structure in subsequent calls to ddi_ssoft_state_get() and > ddi_ssoft_state_free(). The string key is usually some sub-portion > of a 'unit-address'. The routine attempts to allocate space for > the new structure, and if the space allocation was successful, > DDI_SUCCESS is returned to the caller. Returned memory is zeroed. > > A pointer to the space previously allocated for a soft state > structure can be obtained by calling ddi_ssoft_state_get() with the > appropriate string key. > > The space used by a given soft state structure can be returned to > the system using ddi_ssoft_state_free(). > > The space used by all the soft state structures allocated on a > given state pointer, together with the housekeeping information > used by the implementation can be returned to the system using > ddi_ssoft_state_fini(). This routine is typically called from the > driver's detach(9E) routine. > > The ddi_ssoft_state_zalloc(), ddi_ssoft_state_free() and > ddi_ssoft_state_get() routines coordinate access to the underlying > data structures in an MT-safe fashion, thus no additional locks > should be necessary. > > RETURN VALUES > > ddi_ssoft_state_get() > > NULL The requested soft state structure was not > allocated at the time of the call. > > pointer The pointer to the soft state structure. > > > > ddi_ssoft_state_init() > > 0 The allocation was successful. > > EINVAL Either the size parameter was zero, or the > state_p parameter was invalid. > > > ddi_ssoft_state_zalloc() > > DDI_SUCCESS The allocation was successful. > > DDI_FAILURE The routine failed to allocate the storage > required; either the state parameter was > invalid, the string key was NULL, or an attempt > was made to allocate using a string key that > was already allocated. > > > CONTEXT > The ddi_ssoft_state_init() and ddi_ssoft_state_alloc() functions > can be called from user or kernel context only, since they may > internally call kmem_zalloc(9F) with the KM_SLEEP flag. > > The ddi_ssoft_state_fini(), ddi_ssoft_state_free() and > ddi_ssoft_state_get() routines can be called from any driver > context. > > EXAMPLES > <> > > SEE ALSO > _fini(9E), _init(9E), attach(9E), detach(9E), > ddi_get_instance(9F), ddi_soft_state(9E), ddi_isoft_state(9E), > getminor(9F), kmem_zalloc(9F) > > NOTES > If necessary, a hierarchy of state structures can be constructed > by embedding state pointers in higher order state structures. > > DIAGNOSTICS > <> > > All of the messages described below usually indicate bugs in the > driver and should not appear in normal operation of the system. > > WARNING: ddi_ssoft_state_zalloc: bad handle > WARNING: ddi_ssoft_state_free: bad handle > WARNING: ddi_ssoft_state_fini: bad handle > > The implementation-dependent information kept in the state > variable is corrupt. > > WARNING: ddi_ssoft_state_free: null handle > WARNING: ddi_ssoft_state_fini: null handle > > The routine has been passed a null or corrupt state pointer. > Check that ddi_ssoft_state_init() has been called. > > WARNING: ddi_ssoft_state_free: string key '%s' not allocated > > The routine has been asked to free a string key which is not > allocated. The message prints out the invalid string key. > > A.2: New ddi_isoft_state(9F) > ========================================= > :r ddi_isoft_state.9f > > Kernel Functions for Drivers ddi_isoft_state(9F) > > NAME > ddi_isoft_state, ddi_isoft_state_get, ddi_isoft_state_fini, > ddi_isoft_state_free, ddi_isoft_state_init, ddi_isoft_state_zalloc - > driver indexed soft state utility routines > > SYNOPSIS > #include > #include > > typedef struct __ddi_isoft_state ddi_isoft_state; > > int ddi_isoft_state_init(ddi_isoft_state **state_p, > size_t size, size_t n_items); > > int ddi_isoft_state_zalloc(ddi_isoft_state *state, int item); > > void *ddi_isoft_state_get(ddi_isoft_state *state, int item); > > void ddi_isoft_state_free(ddi_isoft_state *state, int item); > > void ddi_isoft_state_fini(ddi_isoft_state **state_p); > > INTERFACE LEVEL > Solaris DDI specific (Solaris DDI). > > PARAMETERS > state_p Address of the opaque state pointer which will be > initialized by ddi_isoft_state_init() to point to > implementation dependent data. > > state An opaque pointer to implementation-dependent data that > describes the soft state. > > size Size of the item which will be allocated by subsequent > calls to ddi_isoft_state_zalloc(). > > n_items A hint of the number of items which will be > preallocated; zero is allowed. > > item The item number for the state structure; usually the > instance number of the associated devinfo node. > > DESCRIPTION > Most device drivers maintain state information with each instance > of the device they control; for example, a soft copy of a device > control register, a mutex that must be held while accessing a > piece of hardware, a partition table, or a unit structure. These > utility routines are intended to help device drivers manage the > space used by the driver to hold such state information. > > For example, if the driver holds the state of each instance in a > single state structure, these routines can be used to dynamically > allocate and deallocate a separate structure for each instance of > the driver as the instance is attached and detached. > > To use the routines, the driver writer needs to declare a state > pointer, state_p, which the implementation uses as a place to hang > a set of per-driver structures; everything else is managed by > these routines. > > The routine ddi_isoft_state_init() is usually called in the > driver's _init(9E) routine to initialize the state pointer, set > the size of the soft state structure, and to allow the driver to > pre-allocate a given number of such structures if required. > > The routine ddi_isoft_state_zalloc() is usually called in the > driver's attach(9E) routine. The routine is passed an item number > which is used to refer to the structure in subsequent calls to > ddi_isoft_state_get() and ddi_isoft_state_free(). The item number is > usually just the instance number of the devinfo node, obtained > with ddi_get_instance(9F). The routine attempts to allocate space > for the new structure, and if the space allocation was successful, > DDI_SUCCESS is returned to the caller. Returned memory is zeroed. > > A pointer to the space previously allocated for a soft state > structure can be obtained by calling ddi_isoft_state_get() with the > appropriate item number. > > The space used by a given soft state structure can be returned to > the system using ddi_isoft_state_free(). This routine is usually > called from the driver's detach(9E) entry point. > > The space used by all the soft state structures allocated on a > given state pointer, together with the housekeeping information > used by the implementation can be returned to the system using > ddi_isoft_state_fini(). This routine can be called from the > driver's _fini(9E) routine. > > The ddi_isoft_state_zalloc(), ddi_isoft_state_free() and > ddi_isoft_state_get() routines coordinate access to the underlying > data structures in an MT-safe fashion, thus no additional locks > should be necessary. > > RETURN VALUES > > ddi_isoft_state_get() > > NULL The requested state structure was not allocated > at the time of the call. > > pointer The pointer to the state structure. > > > > ddi_isoft_state_init() > > 0 The allocation was successful. > > EINVAL Either the size parameter was zero, or the > state_p parameter was invalid. > > > ddi_isoft_state_zalloc() > > DDI_SUCCESS The allocation was successful. > > DDI_FAILURE The routine failed to allocate the storage > required; either the state parameter was > invalid, the item number was negative, or an > attempt was made to allocate an item number > that was already allocated. > > > CONTEXT > The ddi_isoft_state_init() and ddi_isoft_state_alloc() functions > can be called from user or kernel context only, since they may > internally call kmem_zalloc(9F) with the KM_SLEEP flag. > > The ddi_isoft_state_fini(), ddi_isoft_state_free() and > ddi_isoft_state_get() routines can be called from any driver > context. > > EXAMPLES > Example 1 Creating and Removing Data Structures > > The following example shows how the routines described above can > be used in terms of the driver entry points of a character-only > driver. The example concentrates on the portions of the code that > deal with creating and removing the driver's data structures. > > > typedef struct { > volatile caddr_t *csr; /* device registers */ > kmutex_t csr_mutex; /* protects 'csr' field */ > unsigned int state; > dev_info_t *dip; /* back pointer to devinfo */ > } devstate_t; > static ddi_isoft_state *statep; > > int > _init(void) > { > int error; > > error = ddi_isoft_state_init(&statep, sizeof (devstate_t), 0); > if (error != 0) > return (error); > if ((error = mod_install(&modlinkage)) != 0) > ddi_isoft_state_fini(&statep); > return (error); > } > > int > _fini(void) > { > int error; > > if ((error = mod_remove(&modlinkage)) != 0) > return (error); > ddi_isoft_state_fini(&statep); > return (0); > } > > static int > xxattach(dev_info_t *dip, ddi_attach_cmd_t cmd) > { > int instance; > devstate_t *softc; > > switch (cmd) { > case DDI_ATTACH: > instance = ddi_get_instance(dip); > if (ddi_isoft_state_zalloc(statep, instance) != > DDI_SUCCESS) > return (DDI_FAILURE); > softc = ddi_isoft_state_get(statep, instance); > softc->dip = dip; > ... > return (DDI_SUCCESS); > default: > return (DDI_FAILURE); > } > } > > static int > xxdetach(dev_info_t *dip, ddi_detach_cmd_t cmd) > { > int instance; > > switch (cmd) { > > case DDI_DETACH: > instance = ddi_get_instance(dip); > ... > ddi_isoft_state_free(statep, instance); > return (DDI_SUCCESS); > > default: > return (DDI_FAILURE); > } > } > > static int > xxopen(dev_t *devp, int flag, int otyp, cred_t *cred_p) > { > devstate_t *softc; > int instance; > > instance = getminor(*devp); > if ((softc = ddi_isoft_state_get(statep, instance)) == NULL) > return (ENXIO); > ... > softc->state |= XX_IN_USE; > ... > return (0); > } > > SEE ALSO > _fini(9E), _init(9E), attach(9E), detach(9E), > ddi_get_instance(9F), getminor(9F), kmem_zalloc(9F) > > WARNINGS > There is no attempt to validate the item parameter given to > ddi_isoft_state_zalloc() other than it must be a positive signed > integer. Therefore very large item numbers may cause the driver to > hang forever waiting for virtual memory resources that can never > be satisfied. > > NOTES > If necessary, a hierarchy of state structures can be constructed > by embedding state pointers in higher order state structures. > > DIAGNOSTICS > All of the messages described below usually indicate bugs in the > driver and should not appear in normal operation of the system. > > WARNING: ddi_isoft_state_zalloc: bad handle > WARNING: ddi_isoft_state_free: bad handle > WARNING: ddi_isoft_state_fini: bad handle > > The implementation-dependent information kept in the state > variable is corrupt. > > WARNING: ddi_isoft_state_free: null handle > WARNING: ddi_isoft_state_fini: null handle > > The routine has been passed a null or corrupt state pointer. > Check that ddi_isoft_state_init() has been called. > > WARNING: ddi_isoft_state_free: item %d not in range [0..%d] > > The routine has been asked to free an item which was never > allocated. The message prints out the invalid item number and the > acceptable range. > > > A.3: Changes to existing ddi_soft_state(9F) > ========================================= > :r!diff -U5 ddi_soft_state.9f.orig ddi_soft_state.9f > > --- ddi_soft_state.9f.orig Mon Nov 10 11:45:36 2008 > +++ ddi_soft_state.9f Tue Nov 11 18:08:26 2008 > @@ -224,22 +224,29 @@ > return (0); > } > > SEE ALSO > _fini(9E), _init(9E), attach(9E), detach(9E), > - ddi_get_instance(9F), getminor(9F), kmem_zalloc(9F) > + ddi_get_instance(9F), ddi_isoft_state(9E), ddi_ssoft_state(9E), > + getminor(9F), kmem_zalloc(9F) > > WARNINGS > There is no attempt to validate the item parameter given to > ddi_soft_state_zalloc() other than it must be a positive signed > integer. Therefore very large item numbers may cause the driver to > hang forever waiting for virtual memory resources that can never > be satisfied. > > + Use of the ddi_isoft_state(9E) interfaces is encouraged, it > + provides stronger typing. > + > NOTES > If necessary, a hierarchy of state structures can be constructed > by embedding state pointers in higher order state structures. > + > + Equivalent functionality with stronger typing is provided > + by the newer ddi_isoft_state(9F) interfaces. > > DIAGNOSTICS > All of the messages described below usually indicate bugs in the > driver and should not appear in normal operation of the system. From Chris.Horne@sun.com Tue Dec 9 19:54:11 2008 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mBA3sBbh024115 for ; Tue, 9 Dec 2008 19:54:11 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id mBA3s4cI027051 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Tue, 9 Dec 2008 19:54:11 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KBN00J016U40T00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 09 Dec 2008 19:54:04 -0800 (PST) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KBN0079Q6U3ZPC0@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 09 Dec 2008 19:54:03 -0800 (PST) Received: from fe-amer-09.sun.com ([192.18.109.79]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id mBA3s3Ra024634 for ; Wed, 10 Dec 2008 03:54:03 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0KBN005016OBCG00@mail-amer.sun.com> (original mail from Chris.Horne@Sun.COM) for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 09 Dec 2008 20:52:23 -0700 (MST) Received: from sun.com ([129.150.33.126]) by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0KBN005JI6R8R310@mail-amer.sun.com>; Tue, 09 Dec 2008 20:52:21 -0700 (MST) Date: Tue, 09 Dec 2008 20:52:20 -0700 From: Chris Horne Subject: Re: ddi_ssoft_state(9F) and ddi_isoft_state(9F) [PSARC/2008/755 FastTrack timeout 12/17/2008] In-reply-to: <20081209045121.GD852061@eng.sun.com> Sender: Chris.Horne@sun.com To: Edward Pilatowicz Cc: Christopher Horne , PSARC-ext@sun.com Message-id: <493F3CF4.2050902@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-Accept-Language: en-us, en X-PMX-Version: 5.4.1.325704 References: <200812060002.mB602OEX021391@sac.sfbay.sun.com> <20081209045121.GD852061@eng.sun.com> User-Agent: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.4) Gecko/20040414 Status: RO Content-Length: 26770 Hi Ed > hey chris, > > the current ddi soft state interfaces are "special" in my mind because i > thought they were designed to be "fast". (note that i wasn't around for > the original design, so my opinion is based off the implmentation.) > their implementation uses direct array indexing, avoids locking, and > avoids free memory, all seemingly in the name of soft state lookup > speed. i guess i'm worried that other people may be making the same > assumptions about the performance of the ddi soft state interfaces, and > these assumptions don't really carry over to the new interfaces since > they are really a thin wrapper around modhash. Yes, performance is an issue for interfaces used in the per-IO code path. For the intended use case, the proposed ddi_ssoft_state will not be 'directly' involved in the performance path. In the intended use case, the performance path 'indirectly' gains access to the "target-port" granularity ddi_ssoft_state via a private pointer maintained in the finer-grained "target-port,lun" per-scsi_device(9S) hba private soft state (PSARC/2008/675). At tran_tgt_init(9E) time of a scsi_device(9S), the hba driver is expected to ddi_ssoft_state_get(9F) the coarser "target-port" softstate, and increment a private "target-port" softstate reference count to account for storing a "target-port" softstate pointer in the finer-grained per-scsi_device(9S) hba private softstate. > hence, i'd rather see something more straitforward, like some > version/subset of the modhash interfaces promoted to public, or some new > ddi*hash* interfaces. I want the ability to say "its a softstate, but you feed it strings instead of integers". With that simple statement, I believe that anyone who has written a solaris driver would "get it". I don't want to loose that. Keep in mind that any part of the device tree that has multiple complex addressing levels compressed into a single devinfo node unit-address ("@,,") will face this problem - which is why I thought the ddi_ssoft_* prefix was warranted. The solution to performance path issues will likely involve a private ref_count approach similar to our use case above. If the above explanation is insufficient, to make progress, I am willing to change the interface prefix from ddi_ssoft_state_* to scsi_taddr_ssoft_state_*, indicating a more restricted use case that meets immediate needs. Thanks -Chris > > ed > > On Fri, Dec 05, 2008 at 04:02:24PM -0800, Christopher Horne wrote: > >>I am sponsoring the following fasttrack for myself. Micro/patch >>binding is requested. The timer is set to expire on Dec. 17 2008. >> >>-Chris >> >> >>1. Introduction >> 1.1. Project/Component Working Name: >> ddi_ssoft_state(9F) and ddi_isoft_state(9F) >> 1.2. Name of Document Author/Supplier: >> Author: Chris Horne >> 1.3. Date of This Document: >> Tue Nov 11 18:10:01 MST 2008 >> >>4. Technical Description >> >>4.1 Background: >> >> ddi_soft_state(9F) provides utility interfaces to help device >> drivers manage different context. When small integer values, like >> ddi_get_instance(9F) return values, are directly related to >> context, the current "indexed" soft_state interfaces work well. >> >>4.2 Problem: >> >> Two IEEE-1275 '@unit-address' issues can make the current >> ddi_soft_state(9F) support ineffective: >> >> o Compression: Some IEEE-1275 bindings, like the SCSI >> '@,' notation, compress multiple addressing >> levels into a single 'unit-address' string. If a driver needs >> to maintain soft state at intermediate levels, like the >> level, there is no instance number to use with >> ddi_soft_state(9F), and no dev_info node to allow use of >> ddi_[gs]et_driver_private(9F) or ddi_[gs]et_parent_data). >> >> o Complexity: An addressing level may be complex: both >> non-numeric, sparse, and large. >> >> This situation has forced drivers to maintain soft state context in >> ways that no longer fit the ddi_soft_state(9F) model. >> >> An example is SCSA HBA unit-address context. For the >> initial SCSI Parallel Interconnect transport the was >> as an integer limited to the [0-15] range, and ddi_soft_state(9F) >> worked well. On modern transports the complexity of the >> space has mushroomed, as shown below, and the >> space is now 64-bits and sparse. >> >> SPI: disk@0,0 >> WWN: disk@w216000c0ff8047dd,0 >> GUID: disk@g600c0ff0000000000047dd270fdf0901 (lun identity: MPxIO) >> ISCSI: disk@0000iqn.1986-03.com.sun%3A02%3Ac8a82272-b354-c913-80f9-db9cb378a6f60001,0 >> >>4.3 Proposal: >> >> The proposal is to extend the ddi_soft_state(9F) programming model >> by providing "string" based peers to todays "indexed" soft_state >> interfaces. In addition to "string" support, "indexed" interfaces >> with stronger typing are proposed. >> >> Implementation of proposed ddi_ssoft_state(9F) interfaces is built >> on top of the 'modhash' interfaces introduced by [1]. >> >>4.4 Proposed Interfaces: >> >> ------------------------------------------------------------------ >> Interface Name Comm.Lev. Comments >> ------------------------------------------------------------------ >> >> ddi_isoft_state Private "indexed" soft_state >> >> ddi_ssoft_state " "string" soft_state >> >> >> ddi_isoft_state_init " typed equivalent of >> ddi_soft_state_init(9F) >> ddi_ssoft_state_init " string peer of >> ddi_isoft_state_init(9F) >> >> >> ddi_isoft_state_zalloc " typed equivalent of >> ddi_soft_state_zalloc(9F) >> ddi_ssoft_state_zalloc " string peer of >> ddi_isoft_state_zalloc(9F) >> >> ddi_isoft_state_get " typed equivalent of >> ddi_soft_state_get(9F) >> ddi_ssoft_state_get " string peer of >> ddi_isoft_state_get(9F) >> >> ddi_isoft_state_free " typed equivalent of >> ddi_soft_state_free(9F) >> ddi_ssoft_state_free " string peer of >> ddi_isoft_state_free(9F) >> >> ddi_isoft_state_fini " typed equivalent of >> ddi_soft_state_fini(9F) >> ddi_ssoft_state_fini " string peer of >> ddi_isoft_state_fini(9F) >> >> ddi_isoft_state_unassigned " return first unassigned >> item of an ddi_isoft_state. >> >>4.5 Interface Prototypes: (defined in sunddi.h): >> >> typedef struct __ddi_isoft_state ddi_isoft_state; >> typedef struct __ddi_ssoft_state ddi_ssoft_state; >> >> int >> ddi_isoft_state_init(ddi_isoft_state **state_p, >> size_t size, size_t n_items); >> int >> ddi_ssoft_state_init(ddi_ssoft_state **state_p, >> size_t size, int hash_sz); >> >> int >> ddi_isoft_state_zalloc(ddi_isoft_state *state, int index); >> int >> ddi_ssoft_state_zalloc(ddi_ssoft_state *state, const char *stringkey); >> >> void * >> ddi_isoft_state_get(ddi_isoft_state *state, int index); >> void * >> ddi_ssoft_state_get(ddi_ssoft_state *state, const char *stringkey); >> >> void >> ddi_isoft_state_free(ddi_isoft_state *state, int index); >> void >> ddi_ssoft_state_free(ddi_ssoft_state *state, const char *stringkey); >> >> void >> ddi_isoft_state_fini(ddi_isoft_state **state_p); >> void >> ddi_ssoft_state_fini(ddi_ssoft_state **state_p); >> >> int >> ddi_isoft_state_unassigned(ddi_isoft_state **state_p); >> >>4.4 Example: >> >> Assuming that 'tpaddr' is the portion SCSI >> 'unit-address' string, A SCSA HBA driver can use the following to >> access its private specific context. >> >> ss = ddi_isoft_state_get(xx_state, ddi_get_instance(devi)); >> tpss = ddi_ssoft_state_get(ss->ss_tpstate, tpaddr); >> >> Where xx_state would be setup in _init(9E) via >> ddi_isoft_state_init(9F), and ss_tpstate would be setup in >> xxattach(9E) via ddi_ssoft_state_init(9F). >> >>4.5 Alternatives: >> >> An alternative 'compound' approach was considered and prototyped. >> The 'compound' approach implemented a set of 'ddi_strid_*(9E)' >> string-to-id mapping interfaces, and then used the existing >> "indexed" ddi_soft_state(9F) to get from an 'id' to the soft state. >> The 'ddi_strid_(9E)' interfaces, while a good fit for some >> applications, was found to increase SCSI host adapter driver >> complexity compared to the proposed ddi_ssoft_state(9F) interfaces. >> The proposed ddi_ssoft_state(9F) interfaces also present a familiar >> API: they are based on the well-established ddi_soft_state(9F) API >> structure. >> >>4.6 Man Pages >> >> See Appendix A. Some of the examples are TBS at this point, more >> meaningful examples should be provided before these interfaces are >> promoted. >> >>4.7 Release Binding >> >> Micro/patch binding is requested. >> >>4.8 References >> >> [1] PSARC case that introduced modhash interfaces >> PSARC/1998/212 Extensions to Device Autoconfiguration [John Danielson] >> http://sac.sfbay/PSARC/1998/212 >> http://www.opensolaris.org/os/community/arc/caselog/PSARC/1998/212 >> >> >>Appendix A: >> >>A.1: New ddi_ssoft_state(9F) >>========================================= >> :r ddi_ssoft_state.9f >> >>Kernel Functions for Drivers ddi_ssoft_state(9F) >> >>NAME >> ddi_ssoft_state, ddi_ssoft_state_get, ddi_ssoft_state_fini, >> ddi_ssoft_state_free, ddi_ssoft_state_init, ddi_ssoft_state_zalloc - >> driver string soft state utility routines >> >>SYNOPSIS >> #include >> #include >> >> typedef struct __ddi_ssoft_state ddi_ssoft_state; >> >> int ddi_ssoft_state_init(ddi_ssoft_state **state_p, >> size_t size, size_t hash_sz); >> >> int ddi_ssoft_state_zalloc(ddi_ssoft_state *state, int stringkey); >> >> void *ddi_ssoft_state_get(ddi_ssoft_state *state, int stringkey); >> >> void ddi_ssoft_state_free(ddi_ssoft_state *state, int stringkey); >> >> void ddi_ssoft_state_fini(ddi_ssoft_state **state_p); >> >>INTERFACE LEVEL >> Solaris DDI specific (Solaris DDI). >> >>PARAMETERS >> state_p Address of the opaque state pointer which will be >> initialized by ddi_ssoft_state_init() to point to >> implementation dependent data. >> >> state An opaque pointer to implementation-dependent data that >> describes the soft state. >> >> size Size of the soft state which will be allocated by subsequent >> calls to ddi_ssoft_state_zalloc(); zero is illegal. >> >> hash_sz The number of hashed lists which will be allocated; >> zero is not allowed. There is a performance >> .vs. space tradeoff in the selection of a hash_sz >> value: for 'n' soft state structures, on average each >> ddi_ssoft_state_get will need to traverse 'n'/hash_sz/2 >> entries to locate the requested soft state structure. >> >> stringkey The string key for the soft state structure; usually >> associated with some sub-portion of the device's >> 'unit-address'. >> >>DESCRIPTION >> >> Some drivers must maintain state information at a granularity >> unrelated to ddi_get_instance(9F) instance numbers or dev_info >> node ddi_get_driver_private(9F) data. >> >> When the needed soft state granularity can be associated with a >> string, such as a sub-portion of a 'unit-address' string, the >> ddi_ssoft_state(9F) utility routines can be used to help device >> drivers manage the space used by the driver to hold such state >> information. >> >> For example, if a SCSI HBA driver maintains state for each >> target-port address in a separate state structure, for transports >> with complex target-port address representations these routines >> can be used to dynamically allocate and deallocate a separate >> structure for each target-port as scsi_device(9S) associated with >> the target-port are initialized by tran_tgt_init(9E) and and freed >> by tran_tgt_free(9E). >> >> To use the routines, the driver writer needs to declare a state >> pointer, state_p, which the implementation uses as a place to hang >> a set of per-driver structures; everything else is managed by >> these routines. >> >> The routine ddi_ssoft_state_init() is usually called in the >> driver's attach(9E) routine to initialize the state pointer, and set >> the size of the soft state structure. >> >> The routine ddi_ssoft_state_zalloc() is called to allocate a soft >> state. The routine is passed a string key, which is used to refer >> to the structure in subsequent calls to ddi_ssoft_state_get() and >> ddi_ssoft_state_free(). The string key is usually some sub-portion >> of a 'unit-address'. The routine attempts to allocate space for >> the new structure, and if the space allocation was successful, >> DDI_SUCCESS is returned to the caller. Returned memory is zeroed. >> >> A pointer to the space previously allocated for a soft state >> structure can be obtained by calling ddi_ssoft_state_get() with the >> appropriate string key. >> >> The space used by a given soft state structure can be returned to >> the system using ddi_ssoft_state_free(). >> >> The space used by all the soft state structures allocated on a >> given state pointer, together with the housekeeping information >> used by the implementation can be returned to the system using >> ddi_ssoft_state_fini(). This routine is typically called from the >> driver's detach(9E) routine. >> >> The ddi_ssoft_state_zalloc(), ddi_ssoft_state_free() and >> ddi_ssoft_state_get() routines coordinate access to the underlying >> data structures in an MT-safe fashion, thus no additional locks >> should be necessary. >> >>RETURN VALUES >> >> ddi_ssoft_state_get() >> >> NULL The requested soft state structure was not >> allocated at the time of the call. >> >> pointer The pointer to the soft state structure. >> >> >> >> ddi_ssoft_state_init() >> >> 0 The allocation was successful. >> >> EINVAL Either the size parameter was zero, or the >> state_p parameter was invalid. >> >> >> ddi_ssoft_state_zalloc() >> >> DDI_SUCCESS The allocation was successful. >> >> DDI_FAILURE The routine failed to allocate the storage >> required; either the state parameter was >> invalid, the string key was NULL, or an attempt >> was made to allocate using a string key that >> was already allocated. >> >> >>CONTEXT >> The ddi_ssoft_state_init() and ddi_ssoft_state_alloc() functions >> can be called from user or kernel context only, since they may >> internally call kmem_zalloc(9F) with the KM_SLEEP flag. >> >> The ddi_ssoft_state_fini(), ddi_ssoft_state_free() and >> ddi_ssoft_state_get() routines can be called from any driver >> context. >> >>EXAMPLES >><> >> >>SEE ALSO >> _fini(9E), _init(9E), attach(9E), detach(9E), >> ddi_get_instance(9F), ddi_soft_state(9E), ddi_isoft_state(9E), >> getminor(9F), kmem_zalloc(9F) >> >>NOTES >> If necessary, a hierarchy of state structures can be constructed >> by embedding state pointers in higher order state structures. >> >>DIAGNOSTICS >><> >> >> All of the messages described below usually indicate bugs in the >> driver and should not appear in normal operation of the system. >> >> WARNING: ddi_ssoft_state_zalloc: bad handle >> WARNING: ddi_ssoft_state_free: bad handle >> WARNING: ddi_ssoft_state_fini: bad handle >> >> The implementation-dependent information kept in the state >> variable is corrupt. >> >> WARNING: ddi_ssoft_state_free: null handle >> WARNING: ddi_ssoft_state_fini: null handle >> >> The routine has been passed a null or corrupt state pointer. >> Check that ddi_ssoft_state_init() has been called. >> >> WARNING: ddi_ssoft_state_free: string key '%s' not allocated >> >> The routine has been asked to free a string key which is not >> allocated. The message prints out the invalid string key. >> >>A.2: New ddi_isoft_state(9F) >>========================================= >> :r ddi_isoft_state.9f >> >>Kernel Functions for Drivers ddi_isoft_state(9F) >> >>NAME >> ddi_isoft_state, ddi_isoft_state_get, ddi_isoft_state_fini, >> ddi_isoft_state_free, ddi_isoft_state_init, ddi_isoft_state_zalloc - >> driver indexed soft state utility routines >> >>SYNOPSIS >> #include >> #include >> >> typedef struct __ddi_isoft_state ddi_isoft_state; >> >> int ddi_isoft_state_init(ddi_isoft_state **state_p, >> size_t size, size_t n_items); >> >> int ddi_isoft_state_zalloc(ddi_isoft_state *state, int item); >> >> void *ddi_isoft_state_get(ddi_isoft_state *state, int item); >> >> void ddi_isoft_state_free(ddi_isoft_state *state, int item); >> >> void ddi_isoft_state_fini(ddi_isoft_state **state_p); >> >>INTERFACE LEVEL >> Solaris DDI specific (Solaris DDI). >> >>PARAMETERS >> state_p Address of the opaque state pointer which will be >> initialized by ddi_isoft_state_init() to point to >> implementation dependent data. >> >> state An opaque pointer to implementation-dependent data that >> describes the soft state. >> >> size Size of the item which will be allocated by subsequent >> calls to ddi_isoft_state_zalloc(). >> >> n_items A hint of the number of items which will be >> preallocated; zero is allowed. >> >> item The item number for the state structure; usually the >> instance number of the associated devinfo node. >> >>DESCRIPTION >> Most device drivers maintain state information with each instance >> of the device they control; for example, a soft copy of a device >> control register, a mutex that must be held while accessing a >> piece of hardware, a partition table, or a unit structure. These >> utility routines are intended to help device drivers manage the >> space used by the driver to hold such state information. >> >> For example, if the driver holds the state of each instance in a >> single state structure, these routines can be used to dynamically >> allocate and deallocate a separate structure for each instance of >> the driver as the instance is attached and detached. >> >> To use the routines, the driver writer needs to declare a state >> pointer, state_p, which the implementation uses as a place to hang >> a set of per-driver structures; everything else is managed by >> these routines. >> >> The routine ddi_isoft_state_init() is usually called in the >> driver's _init(9E) routine to initialize the state pointer, set >> the size of the soft state structure, and to allow the driver to >> pre-allocate a given number of such structures if required. >> >> The routine ddi_isoft_state_zalloc() is usually called in the >> driver's attach(9E) routine. The routine is passed an item number >> which is used to refer to the structure in subsequent calls to >> ddi_isoft_state_get() and ddi_isoft_state_free(). The item number is >> usually just the instance number of the devinfo node, obtained >> with ddi_get_instance(9F). The routine attempts to allocate space >> for the new structure, and if the space allocation was successful, >> DDI_SUCCESS is returned to the caller. Returned memory is zeroed. >> >> A pointer to the space previously allocated for a soft state >> structure can be obtained by calling ddi_isoft_state_get() with the >> appropriate item number. >> >> The space used by a given soft state structure can be returned to >> the system using ddi_isoft_state_free(). This routine is usually >> called from the driver's detach(9E) entry point. >> >> The space used by all the soft state structures allocated on a >> given state pointer, together with the housekeeping information >> used by the implementation can be returned to the system using >> ddi_isoft_state_fini(). This routine can be called from the >> driver's _fini(9E) routine. >> >> The ddi_isoft_state_zalloc(), ddi_isoft_state_free() and >> ddi_isoft_state_get() routines coordinate access to the underlying >> data structures in an MT-safe fashion, thus no additional locks >> should be necessary. >> >>RETURN VALUES >> >> ddi_isoft_state_get() >> >> NULL The requested state structure was not allocated >> at the time of the call. >> >> pointer The pointer to the state structure. >> >> >> >> ddi_isoft_state_init() >> >> 0 The allocation was successful. >> >> EINVAL Either the size parameter was zero, or the >> state_p parameter was invalid. >> >> >> ddi_isoft_state_zalloc() >> >> DDI_SUCCESS The allocation was successful. >> >> DDI_FAILURE The routine failed to allocate the storage >> required; either the state parameter was >> invalid, the item number was negative, or an >> attempt was made to allocate an item number >> that was already allocated. >> >> >>CONTEXT >> The ddi_isoft_state_init() and ddi_isoft_state_alloc() functions >> can be called from user or kernel context only, since they may >> internally call kmem_zalloc(9F) with the KM_SLEEP flag. >> >> The ddi_isoft_state_fini(), ddi_isoft_state_free() and >> ddi_isoft_state_get() routines can be called from any driver >> context. >> >>EXAMPLES >> Example 1 Creating and Removing Data Structures >> >> The following example shows how the routines described above can >> be used in terms of the driver entry points of a character-only >> driver. The example concentrates on the portions of the code that >> deal with creating and removing the driver's data structures. >> >> >> typedef struct { >> volatile caddr_t *csr; /* device registers */ >> kmutex_t csr_mutex; /* protects 'csr' field */ >> unsigned int state; >> dev_info_t *dip; /* back pointer to devinfo */ >> } devstate_t; >> static ddi_isoft_state *statep; >> >> int >> _init(void) >> { >> int error; >> >> error = ddi_isoft_state_init(&statep, sizeof (devstate_t), 0); >> if (error != 0) >> return (error); >> if ((error = mod_install(&modlinkage)) != 0) >> ddi_isoft_state_fini(&statep); >> return (error); >> } >> >> int >> _fini(void) >> { >> int error; >> >> if ((error = mod_remove(&modlinkage)) != 0) >> return (error); >> ddi_isoft_state_fini(&statep); >> return (0); >> } >> >> static int >> xxattach(dev_info_t *dip, ddi_attach_cmd_t cmd) >> { >> int instance; >> devstate_t *softc; >> >> switch (cmd) { >> case DDI_ATTACH: >> instance = ddi_get_instance(dip); >> if (ddi_isoft_state_zalloc(statep, instance) != >> DDI_SUCCESS) >> return (DDI_FAILURE); >> softc = ddi_isoft_state_get(statep, instance); >> softc->dip = dip; >> ... >> return (DDI_SUCCESS); >> default: >> return (DDI_FAILURE); >> } >> } >> >> static int >> xxdetach(dev_info_t *dip, ddi_detach_cmd_t cmd) >> { >> int instance; >> >> switch (cmd) { >> >> case DDI_DETACH: >> instance = ddi_get_instance(dip); >> ... >> ddi_isoft_state_free(statep, instance); >> return (DDI_SUCCESS); >> >> default: >> return (DDI_FAILURE); >> } >> } >> >> static int >> xxopen(dev_t *devp, int flag, int otyp, cred_t *cred_p) >> { >> devstate_t *softc; >> int instance; >> >> instance = getminor(*devp); >> if ((softc = ddi_isoft_state_get(statep, instance)) == NULL) >> return (ENXIO); >> ... >> softc->state |= XX_IN_USE; >> ... >> return (0); >> } >> >>SEE ALSO >> _fini(9E), _init(9E), attach(9E), detach(9E), >> ddi_get_instance(9F), getminor(9F), kmem_zalloc(9F) >> >>WARNINGS >> There is no attempt to validate the item parameter given to >> ddi_isoft_state_zalloc() other than it must be a positive signed >> integer. Therefore very large item numbers may cause the driver to >> hang forever waiting for virtual memory resources that can never >> be satisfied. >> >>NOTES >> If necessary, a hierarchy of state structures can be constructed >> by embedding state pointers in higher order state structures. >> >>DIAGNOSTICS >> All of the messages described below usually indicate bugs in the >> driver and should not appear in normal operation of the system. >> >> WARNING: ddi_isoft_state_zalloc: bad handle >> WARNING: ddi_isoft_state_free: bad handle >> WARNING: ddi_isoft_state_fini: bad handle >> >> The implementation-dependent information kept in the state >> variable is corrupt. >> >> WARNING: ddi_isoft_state_free: null handle >> WARNING: ddi_isoft_state_fini: null handle >> >> The routine has been passed a null or corrupt state pointer. >> Check that ddi_isoft_state_init() has been called. >> >> WARNING: ddi_isoft_state_free: item %d not in range [0..%d] >> >> The routine has been asked to free an item which was never >> allocated. The message prints out the invalid item number and the >> acceptable range. >> >> >>A.3: Changes to existing ddi_soft_state(9F) >>========================================= >> :r!diff -U5 ddi_soft_state.9f.orig ddi_soft_state.9f >> >>--- ddi_soft_state.9f.orig Mon Nov 10 11:45:36 2008 >>+++ ddi_soft_state.9f Tue Nov 11 18:08:26 2008 >>@@ -224,22 +224,29 @@ >> return (0); >> } >> >> SEE ALSO >> _fini(9E), _init(9E), attach(9E), detach(9E), >>- ddi_get_instance(9F), getminor(9F), kmem_zalloc(9F) >>+ ddi_get_instance(9F), ddi_isoft_state(9E), ddi_ssoft_state(9E), >>+ getminor(9F), kmem_zalloc(9F) >> >> WARNINGS >> There is no attempt to validate the item parameter given to >> ddi_soft_state_zalloc() other than it must be a positive signed >> integer. Therefore very large item numbers may cause the driver to >> hang forever waiting for virtual memory resources that can never >> be satisfied. >> >>+ Use of the ddi_isoft_state(9E) interfaces is encouraged, it >>+ provides stronger typing. >>+ >> NOTES >> If necessary, a hierarchy of state structures can be constructed >> by embedding state pointers in higher order state structures. >>+ >>+ Equivalent functionality with stronger typing is provided >>+ by the newer ddi_isoft_state(9F) interfaces. >> >> DIAGNOSTICS >> All of the messages described below usually indicate bugs in the >> driver and should not appear in normal operation of the system. From gdamore@Sun.COM Tue Dec 9 20:58:06 2008 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mBA4w63t011180 for ; Tue, 9 Dec 2008 20:58:06 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id mBA4w2uX028535 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Tue, 9 Dec 2008 20:58:05 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KBN0000B9STI200@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 09 Dec 2008 20:58:05 -0800 (PST) Received: from sca-es-mail-2.sun.com ([192.18.43.133]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KBN000GT9ST2200@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 09 Dec 2008 20:58:05 -0800 (PST) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id mBA4w5gd000640 for ; Tue, 09 Dec 2008 20:58:05 -0800 (PST) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0KBN004019OHZ100@fe-sfbay-10.sun.com> (original mail from gdamore@sun.com) for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 09 Dec 2008 20:58:05 -0800 (PST) Received: from [10.7.251.172] by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0KBN00J1K9SR9F90@fe-sfbay-10.sun.com>; Tue, 09 Dec 2008 20:58:04 -0800 (PST) Date: Tue, 09 Dec 2008 20:50:44 -0800 From: "Garrett D'Amore" Subject: Re: ddi_ssoft_state(9F) and ddi_isoft_state(9F) [PSARC/2008/755 FastTrack timeout 12/17/2008] In-reply-to: <493F3CF4.2050902@sun.com> Sender: Garrett.Damore@Sun.COM To: Chris Horne Cc: Edward Pilatowicz , Christopher Horne , PSARC-ext@Sun.COM Message-id: <493F4AA4.9010803@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200812060002.mB602OEX021391@sac.sfbay.sun.com> <20081209045121.GD852061@eng.sun.com> <493F3CF4.2050902@sun.com> User-Agent: Thunderbird 2.0.0.14 (X11/20080616) Status: RO Content-Length: 28352 Chris Horne wrote: > Hi Ed > > >> hey chris, >> >> the current ddi soft state interfaces are "special" in my mind because i >> thought they were designed to be "fast". (note that i wasn't around for >> the original design, so my opinion is based off the implmentation.) >> their implementation uses direct array indexing, avoids locking, and >> avoids free memory, all seemingly in the name of soft state lookup >> speed. i guess i'm worried that other people may be making the same >> assumptions about the performance of the ddi soft state interfaces, and >> these assumptions don't really carry over to the new interfaces since >> they are really a thin wrapper around modhash. >> > > > Yes, performance is an issue for interfaces used in the per-IO code > path. For the intended use case, the proposed ddi_ssoft_state will not > be 'directly' involved in the performance path. In the intended use > case, the performance path 'indirectly' gains access to the > "target-port" granularity ddi_ssoft_state via a private pointer > maintained in the finer-grained "target-port,lun" per-scsi_device(9S) > hba private soft state (PSARC/2008/675). At tran_tgt_init(9E) time of > a scsi_device(9S), the hba driver is expected to > ddi_ssoft_state_get(9F) the coarser "target-port" softstate, and > increment a private "target-port" softstate reference count to account > for storing a "target-port" softstate pointer in the finer-grained > per-scsi_device(9S) hba private softstate. > > > >> hence, i'd rather see something more straitforward, like some >> version/subset of the modhash interfaces promoted to public, or some new >> ddi*hash* interfaces. >> > > I want the ability to say "its a softstate, but you feed it strings > instead of integers". With that simple statement, I believe that anyone > who has written a solaris driver would "get it". I don't want to loose > that. > But in your intended usage, it *isn't* a soft state, at least not like I suspect most people thing of such things (one per device instance, the master private state for the devinfo node.) I actually fear that this naming will increase confusion (which soft state interfaces should I use?) > Keep in mind that any part of the device tree that has multiple complex > addressing levels compressed into a single devinfo node unit-address > ("@,,") will face this problem - > which is why I thought the ddi_ssoft_* prefix was warranted. The solution > to performance path issues will likely involve a private ref_count > approach similar to our use case above. > > If the above explanation is insufficient, to make progress, I am > willing to change the interface prefix from ddi_ssoft_state_* to > scsi_taddr_ssoft_state_*, indicating a more restricted use case > that meets immediate needs. > FWIW, I'd be happy with that. I'd be equally happy with a generic hash table interface. Though I'd suggest you just call it "scsi_taddr_state_*()" or somesuch. (Avoid the non-descriptive "s" prefix, and keep the identifier short.) -- Garrett > Thanks > -Chris > > > >> ed >> >> On Fri, Dec 05, 2008 at 04:02:24PM -0800, Christopher Horne wrote: >> >> >>> I am sponsoring the following fasttrack for myself. Micro/patch >>> binding is requested. The timer is set to expire on Dec. 17 2008. >>> >>> -Chris >>> >>> >>> 1. Introduction >>> 1.1. Project/Component Working Name: >>> ddi_ssoft_state(9F) and ddi_isoft_state(9F) >>> 1.2. Name of Document Author/Supplier: >>> Author: Chris Horne >>> 1.3. Date of This Document: >>> Tue Nov 11 18:10:01 MST 2008 >>> >>> 4. Technical Description >>> >>> 4.1 Background: >>> >>> ddi_soft_state(9F) provides utility interfaces to help device >>> drivers manage different context. When small integer values, like >>> ddi_get_instance(9F) return values, are directly related to >>> context, the current "indexed" soft_state interfaces work well. >>> >>> 4.2 Problem: >>> >>> Two IEEE-1275 '@unit-address' issues can make the current >>> ddi_soft_state(9F) support ineffective: >>> >>> o Compression: Some IEEE-1275 bindings, like the SCSI >>> '@,' notation, compress multiple addressing >>> levels into a single 'unit-address' string. If a driver needs >>> to maintain soft state at intermediate levels, like the >>> level, there is no instance number to use with >>> ddi_soft_state(9F), and no dev_info node to allow use of >>> ddi_[gs]et_driver_private(9F) or ddi_[gs]et_parent_data). >>> >>> o Complexity: An addressing level may be complex: both >>> non-numeric, sparse, and large. >>> >>> This situation has forced drivers to maintain soft state context in >>> ways that no longer fit the ddi_soft_state(9F) model. >>> >>> An example is SCSA HBA unit-address context. For the >>> initial SCSI Parallel Interconnect transport the was >>> as an integer limited to the [0-15] range, and ddi_soft_state(9F) >>> worked well. On modern transports the complexity of the >>> space has mushroomed, as shown below, and the >>> space is now 64-bits and sparse. >>> >>> SPI: disk@0,0 >>> WWN: disk@w216000c0ff8047dd,0 >>> GUID: disk@g600c0ff0000000000047dd270fdf0901 (lun identity: MPxIO) >>> ISCSI: disk@0000iqn.1986-03.com.sun%3A02%3Ac8a82272-b354-c913-80f9-db9cb378a6f60001,0 >>> >>> 4.3 Proposal: >>> >>> The proposal is to extend the ddi_soft_state(9F) programming model >>> by providing "string" based peers to todays "indexed" soft_state >>> interfaces. In addition to "string" support, "indexed" interfaces >>> with stronger typing are proposed. >>> >>> Implementation of proposed ddi_ssoft_state(9F) interfaces is built >>> on top of the 'modhash' interfaces introduced by [1]. >>> >>> 4.4 Proposed Interfaces: >>> >>> ------------------------------------------------------------------ >>> Interface Name Comm.Lev. Comments >>> ------------------------------------------------------------------ >>> >>> ddi_isoft_state Private "indexed" soft_state >>> >>> ddi_ssoft_state " "string" soft_state >>> >>> >>> ddi_isoft_state_init " typed equivalent of >>> ddi_soft_state_init(9F) >>> ddi_ssoft_state_init " string peer of >>> ddi_isoft_state_init(9F) >>> >>> >>> ddi_isoft_state_zalloc " typed equivalent of >>> ddi_soft_state_zalloc(9F) >>> ddi_ssoft_state_zalloc " string peer of >>> ddi_isoft_state_zalloc(9F) >>> >>> ddi_isoft_state_get " typed equivalent of >>> ddi_soft_state_get(9F) >>> ddi_ssoft_state_get " string peer of >>> ddi_isoft_state_get(9F) >>> >>> ddi_isoft_state_free " typed equivalent of >>> ddi_soft_state_free(9F) >>> ddi_ssoft_state_free " string peer of >>> ddi_isoft_state_free(9F) >>> >>> ddi_isoft_state_fini " typed equivalent of >>> ddi_soft_state_fini(9F) >>> ddi_ssoft_state_fini " string peer of >>> ddi_isoft_state_fini(9F) >>> >>> ddi_isoft_state_unassigned " return first unassigned >>> item of an ddi_isoft_state. >>> >>> 4.5 Interface Prototypes: (defined in sunddi.h): >>> >>> typedef struct __ddi_isoft_state ddi_isoft_state; >>> typedef struct __ddi_ssoft_state ddi_ssoft_state; >>> >>> int >>> ddi_isoft_state_init(ddi_isoft_state **state_p, >>> size_t size, size_t n_items); >>> int >>> ddi_ssoft_state_init(ddi_ssoft_state **state_p, >>> size_t size, int hash_sz); >>> >>> int >>> ddi_isoft_state_zalloc(ddi_isoft_state *state, int index); >>> int >>> ddi_ssoft_state_zalloc(ddi_ssoft_state *state, const char *stringkey); >>> >>> void * >>> ddi_isoft_state_get(ddi_isoft_state *state, int index); >>> void * >>> ddi_ssoft_state_get(ddi_ssoft_state *state, const char *stringkey); >>> >>> void >>> ddi_isoft_state_free(ddi_isoft_state *state, int index); >>> void >>> ddi_ssoft_state_free(ddi_ssoft_state *state, const char *stringkey); >>> >>> void >>> ddi_isoft_state_fini(ddi_isoft_state **state_p); >>> void >>> ddi_ssoft_state_fini(ddi_ssoft_state **state_p); >>> >>> int >>> ddi_isoft_state_unassigned(ddi_isoft_state **state_p); >>> >>> 4.4 Example: >>> >>> Assuming that 'tpaddr' is the portion SCSI >>> 'unit-address' string, A SCSA HBA driver can use the following to >>> access its private specific context. >>> >>> ss = ddi_isoft_state_get(xx_state, ddi_get_instance(devi)); >>> tpss = ddi_ssoft_state_get(ss->ss_tpstate, tpaddr); >>> >>> Where xx_state would be setup in _init(9E) via >>> ddi_isoft_state_init(9F), and ss_tpstate would be setup in >>> xxattach(9E) via ddi_ssoft_state_init(9F). >>> >>> 4.5 Alternatives: >>> >>> An alternative 'compound' approach was considered and prototyped. >>> The 'compound' approach implemented a set of 'ddi_strid_*(9E)' >>> string-to-id mapping interfaces, and then used the existing >>> "indexed" ddi_soft_state(9F) to get from an 'id' to the soft state. >>> The 'ddi_strid_(9E)' interfaces, while a good fit for some >>> applications, was found to increase SCSI host adapter driver >>> complexity compared to the proposed ddi_ssoft_state(9F) interfaces. >>> The proposed ddi_ssoft_state(9F) interfaces also present a familiar >>> API: they are based on the well-established ddi_soft_state(9F) API >>> structure. >>> >>> 4.6 Man Pages >>> >>> See Appendix A. Some of the examples are TBS at this point, more >>> meaningful examples should be provided before these interfaces are >>> promoted. >>> >>> 4.7 Release Binding >>> >>> Micro/patch binding is requested. >>> >>> 4.8 References >>> >>> [1] PSARC case that introduced modhash interfaces >>> PSARC/1998/212 Extensions to Device Autoconfiguration [John Danielson] >>> http://sac.sfbay/PSARC/1998/212 >>> http://www.opensolaris.org/os/community/arc/caselog/PSARC/1998/212 >>> >>> >>> Appendix A: >>> >>> A.1: New ddi_ssoft_state(9F) >>> ========================================= >>> :r ddi_ssoft_state.9f >>> >>> Kernel Functions for Drivers ddi_ssoft_state(9F) >>> >>> NAME >>> ddi_ssoft_state, ddi_ssoft_state_get, ddi_ssoft_state_fini, >>> ddi_ssoft_state_free, ddi_ssoft_state_init, ddi_ssoft_state_zalloc - >>> driver string soft state utility routines >>> >>> SYNOPSIS >>> #include >>> #include >>> >>> typedef struct __ddi_ssoft_state ddi_ssoft_state; >>> >>> int ddi_ssoft_state_init(ddi_ssoft_state **state_p, >>> size_t size, size_t hash_sz); >>> >>> int ddi_ssoft_state_zalloc(ddi_ssoft_state *state, int stringkey); >>> >>> void *ddi_ssoft_state_get(ddi_ssoft_state *state, int stringkey); >>> >>> void ddi_ssoft_state_free(ddi_ssoft_state *state, int stringkey); >>> >>> void ddi_ssoft_state_fini(ddi_ssoft_state **state_p); >>> >>> INTERFACE LEVEL >>> Solaris DDI specific (Solaris DDI). >>> >>> PARAMETERS >>> state_p Address of the opaque state pointer which will be >>> initialized by ddi_ssoft_state_init() to point to >>> implementation dependent data. >>> >>> state An opaque pointer to implementation-dependent data that >>> describes the soft state. >>> >>> size Size of the soft state which will be allocated by subsequent >>> calls to ddi_ssoft_state_zalloc(); zero is illegal. >>> >>> hash_sz The number of hashed lists which will be allocated; >>> zero is not allowed. There is a performance >>> .vs. space tradeoff in the selection of a hash_sz >>> value: for 'n' soft state structures, on average each >>> ddi_ssoft_state_get will need to traverse 'n'/hash_sz/2 >>> entries to locate the requested soft state structure. >>> >>> stringkey The string key for the soft state structure; usually >>> associated with some sub-portion of the device's >>> 'unit-address'. >>> >>> DESCRIPTION >>> >>> Some drivers must maintain state information at a granularity >>> unrelated to ddi_get_instance(9F) instance numbers or dev_info >>> node ddi_get_driver_private(9F) data. >>> >>> When the needed soft state granularity can be associated with a >>> string, such as a sub-portion of a 'unit-address' string, the >>> ddi_ssoft_state(9F) utility routines can be used to help device >>> drivers manage the space used by the driver to hold such state >>> information. >>> >>> For example, if a SCSI HBA driver maintains state for each >>> target-port address in a separate state structure, for transports >>> with complex target-port address representations these routines >>> can be used to dynamically allocate and deallocate a separate >>> structure for each target-port as scsi_device(9S) associated with >>> the target-port are initialized by tran_tgt_init(9E) and and freed >>> by tran_tgt_free(9E). >>> >>> To use the routines, the driver writer needs to declare a state >>> pointer, state_p, which the implementation uses as a place to hang >>> a set of per-driver structures; everything else is managed by >>> these routines. >>> >>> The routine ddi_ssoft_state_init() is usually called in the >>> driver's attach(9E) routine to initialize the state pointer, and set >>> the size of the soft state structure. >>> >>> The routine ddi_ssoft_state_zalloc() is called to allocate a soft >>> state. The routine is passed a string key, which is used to refer >>> to the structure in subsequent calls to ddi_ssoft_state_get() and >>> ddi_ssoft_state_free(). The string key is usually some sub-portion >>> of a 'unit-address'. The routine attempts to allocate space for >>> the new structure, and if the space allocation was successful, >>> DDI_SUCCESS is returned to the caller. Returned memory is zeroed. >>> >>> A pointer to the space previously allocated for a soft state >>> structure can be obtained by calling ddi_ssoft_state_get() with the >>> appropriate string key. >>> >>> The space used by a given soft state structure can be returned to >>> the system using ddi_ssoft_state_free(). >>> >>> The space used by all the soft state structures allocated on a >>> given state pointer, together with the housekeeping information >>> used by the implementation can be returned to the system using >>> ddi_ssoft_state_fini(). This routine is typically called from the >>> driver's detach(9E) routine. >>> >>> The ddi_ssoft_state_zalloc(), ddi_ssoft_state_free() and >>> ddi_ssoft_state_get() routines coordinate access to the underlying >>> data structures in an MT-safe fashion, thus no additional locks >>> should be necessary. >>> >>> RETURN VALUES >>> >>> ddi_ssoft_state_get() >>> >>> NULL The requested soft state structure was not >>> allocated at the time of the call. >>> >>> pointer The pointer to the soft state structure. >>> >>> >>> >>> ddi_ssoft_state_init() >>> >>> 0 The allocation was successful. >>> >>> EINVAL Either the size parameter was zero, or the >>> state_p parameter was invalid. >>> >>> >>> ddi_ssoft_state_zalloc() >>> >>> DDI_SUCCESS The allocation was successful. >>> >>> DDI_FAILURE The routine failed to allocate the storage >>> required; either the state parameter was >>> invalid, the string key was NULL, or an attempt >>> was made to allocate using a string key that >>> was already allocated. >>> >>> >>> CONTEXT >>> The ddi_ssoft_state_init() and ddi_ssoft_state_alloc() functions >>> can be called from user or kernel context only, since they may >>> internally call kmem_zalloc(9F) with the KM_SLEEP flag. >>> >>> The ddi_ssoft_state_fini(), ddi_ssoft_state_free() and >>> ddi_ssoft_state_get() routines can be called from any driver >>> context. >>> >>> EXAMPLES >>> <> >>> >>> SEE ALSO >>> _fini(9E), _init(9E), attach(9E), detach(9E), >>> ddi_get_instance(9F), ddi_soft_state(9E), ddi_isoft_state(9E), >>> getminor(9F), kmem_zalloc(9F) >>> >>> NOTES >>> If necessary, a hierarchy of state structures can be constructed >>> by embedding state pointers in higher order state structures. >>> >>> DIAGNOSTICS >>> <> >>> >>> All of the messages described below usually indicate bugs in the >>> driver and should not appear in normal operation of the system. >>> >>> WARNING: ddi_ssoft_state_zalloc: bad handle >>> WARNING: ddi_ssoft_state_free: bad handle >>> WARNING: ddi_ssoft_state_fini: bad handle >>> >>> The implementation-dependent information kept in the state >>> variable is corrupt. >>> >>> WARNING: ddi_ssoft_state_free: null handle >>> WARNING: ddi_ssoft_state_fini: null handle >>> >>> The routine has been passed a null or corrupt state pointer. >>> Check that ddi_ssoft_state_init() has been called. >>> >>> WARNING: ddi_ssoft_state_free: string key '%s' not allocated >>> >>> The routine has been asked to free a string key which is not >>> allocated. The message prints out the invalid string key. >>> >>> A.2: New ddi_isoft_state(9F) >>> ========================================= >>> :r ddi_isoft_state.9f >>> >>> Kernel Functions for Drivers ddi_isoft_state(9F) >>> >>> NAME >>> ddi_isoft_state, ddi_isoft_state_get, ddi_isoft_state_fini, >>> ddi_isoft_state_free, ddi_isoft_state_init, ddi_isoft_state_zalloc - >>> driver indexed soft state utility routines >>> >>> SYNOPSIS >>> #include >>> #include >>> >>> typedef struct __ddi_isoft_state ddi_isoft_state; >>> >>> int ddi_isoft_state_init(ddi_isoft_state **state_p, >>> size_t size, size_t n_items); >>> >>> int ddi_isoft_state_zalloc(ddi_isoft_state *state, int item); >>> >>> void *ddi_isoft_state_get(ddi_isoft_state *state, int item); >>> >>> void ddi_isoft_state_free(ddi_isoft_state *state, int item); >>> >>> void ddi_isoft_state_fini(ddi_isoft_state **state_p); >>> >>> INTERFACE LEVEL >>> Solaris DDI specific (Solaris DDI). >>> >>> PARAMETERS >>> state_p Address of the opaque state pointer which will be >>> initialized by ddi_isoft_state_init() to point to >>> implementation dependent data. >>> >>> state An opaque pointer to implementation-dependent data that >>> describes the soft state. >>> >>> size Size of the item which will be allocated by subsequent >>> calls to ddi_isoft_state_zalloc(). >>> >>> n_items A hint of the number of items which will be >>> preallocated; zero is allowed. >>> >>> item The item number for the state structure; usually the >>> instance number of the associated devinfo node. >>> >>> DESCRIPTION >>> Most device drivers maintain state information with each instance >>> of the device they control; for example, a soft copy of a device >>> control register, a mutex that must be held while accessing a >>> piece of hardware, a partition table, or a unit structure. These >>> utility routines are intended to help device drivers manage the >>> space used by the driver to hold such state information. >>> >>> For example, if the driver holds the state of each instance in a >>> single state structure, these routines can be used to dynamically >>> allocate and deallocate a separate structure for each instance of >>> the driver as the instance is attached and detached. >>> >>> To use the routines, the driver writer needs to declare a state >>> pointer, state_p, which the implementation uses as a place to hang >>> a set of per-driver structures; everything else is managed by >>> these routines. >>> >>> The routine ddi_isoft_state_init() is usually called in the >>> driver's _init(9E) routine to initialize the state pointer, set >>> the size of the soft state structure, and to allow the driver to >>> pre-allocate a given number of such structures if required. >>> >>> The routine ddi_isoft_state_zalloc() is usually called in the >>> driver's attach(9E) routine. The routine is passed an item number >>> which is used to refer to the structure in subsequent calls to >>> ddi_isoft_state_get() and ddi_isoft_state_free(). The item number is >>> usually just the instance number of the devinfo node, obtained >>> with ddi_get_instance(9F). The routine attempts to allocate space >>> for the new structure, and if the space allocation was successful, >>> DDI_SUCCESS is returned to the caller. Returned memory is zeroed. >>> >>> A pointer to the space previously allocated for a soft state >>> structure can be obtained by calling ddi_isoft_state_get() with the >>> appropriate item number. >>> >>> The space used by a given soft state structure can be returned to >>> the system using ddi_isoft_state_free(). This routine is usually >>> called from the driver's detach(9E) entry point. >>> >>> The space used by all the soft state structures allocated on a >>> given state pointer, together with the housekeeping information >>> used by the implementation can be returned to the system using >>> ddi_isoft_state_fini(). This routine can be called from the >>> driver's _fini(9E) routine. >>> >>> The ddi_isoft_state_zalloc(), ddi_isoft_state_free() and >>> ddi_isoft_state_get() routines coordinate access to the underlying >>> data structures in an MT-safe fashion, thus no additional locks >>> should be necessary. >>> >>> RETURN VALUES >>> >>> ddi_isoft_state_get() >>> >>> NULL The requested state structure was not allocated >>> at the time of the call. >>> >>> pointer The pointer to the state structure. >>> >>> >>> >>> ddi_isoft_state_init() >>> >>> 0 The allocation was successful. >>> >>> EINVAL Either the size parameter was zero, or the >>> state_p parameter was invalid. >>> >>> >>> ddi_isoft_state_zalloc() >>> >>> DDI_SUCCESS The allocation was successful. >>> >>> DDI_FAILURE The routine failed to allocate the storage >>> required; either the state parameter was >>> invalid, the item number was negative, or an >>> attempt was made to allocate an item number >>> that was already allocated. >>> >>> >>> CONTEXT >>> The ddi_isoft_state_init() and ddi_isoft_state_alloc() functions >>> can be called from user or kernel context only, since they may >>> internally call kmem_zalloc(9F) with the KM_SLEEP flag. >>> >>> The ddi_isoft_state_fini(), ddi_isoft_state_free() and >>> ddi_isoft_state_get() routines can be called from any driver >>> context. >>> >>> EXAMPLES >>> Example 1 Creating and Removing Data Structures >>> >>> The following example shows how the routines described above can >>> be used in terms of the driver entry points of a character-only >>> driver. The example concentrates on the portions of the code that >>> deal with creating and removing the driver's data structures. >>> >>> >>> typedef struct { >>> volatile caddr_t *csr; /* device registers */ >>> kmutex_t csr_mutex; /* protects 'csr' field */ >>> unsigned int state; >>> dev_info_t *dip; /* back pointer to devinfo */ >>> } devstate_t; >>> static ddi_isoft_state *statep; >>> >>> int >>> _init(void) >>> { >>> int error; >>> >>> error = ddi_isoft_state_init(&statep, sizeof (devstate_t), 0); >>> if (error != 0) >>> return (error); >>> if ((error = mod_install(&modlinkage)) != 0) >>> ddi_isoft_state_fini(&statep); >>> return (error); >>> } >>> >>> int >>> _fini(void) >>> { >>> int error; >>> >>> if ((error = mod_remove(&modlinkage)) != 0) >>> return (error); >>> ddi_isoft_state_fini(&statep); >>> return (0); >>> } >>> >>> static int >>> xxattach(dev_info_t *dip, ddi_attach_cmd_t cmd) >>> { >>> int instance; >>> devstate_t *softc; >>> >>> switch (cmd) { >>> case DDI_ATTACH: >>> instance = ddi_get_instance(dip); >>> if (ddi_isoft_state_zalloc(statep, instance) != >>> DDI_SUCCESS) >>> return (DDI_FAILURE); >>> softc = ddi_isoft_state_get(statep, instance); >>> softc->dip = dip; >>> ... >>> return (DDI_SUCCESS); >>> default: >>> return (DDI_FAILURE); >>> } >>> } >>> >>> static int >>> xxdetach(dev_info_t *dip, ddi_detach_cmd_t cmd) >>> { >>> int instance; >>> >>> switch (cmd) { >>> >>> case DDI_DETACH: >>> instance = ddi_get_instance(dip); >>> ... >>> ddi_isoft_state_free(statep, instance); >>> return (DDI_SUCCESS); >>> >>> default: >>> return (DDI_FAILURE); >>> } >>> } >>> >>> static int >>> xxopen(dev_t *devp, int flag, int otyp, cred_t *cred_p) >>> { >>> devstate_t *softc; >>> int instance; >>> >>> instance = getminor(*devp); >>> if ((softc = ddi_isoft_state_get(statep, instance)) == NULL) >>> return (ENXIO); >>> ... >>> softc->state |= XX_IN_USE; >>> ... >>> return (0); >>> } >>> >>> SEE ALSO >>> _fini(9E), _init(9E), attach(9E), detach(9E), >>> ddi_get_instance(9F), getminor(9F), kmem_zalloc(9F) >>> >>> WARNINGS >>> There is no attempt to validate the item parameter given to >>> ddi_isoft_state_zalloc() other than it must be a positive signed >>> integer. Therefore very large item numbers may cause the driver to >>> hang forever waiting for virtual memory resources that can never >>> be satisfied. >>> >>> NOTES >>> If necessary, a hierarchy of state structures can be constructed >>> by embedding state pointers in higher order state structures. >>> >>> DIAGNOSTICS >>> All of the messages described below usually indicate bugs in the >>> driver and should not appear in normal operation of the system. >>> >>> WARNING: ddi_isoft_state_zalloc: bad handle >>> WARNING: ddi_isoft_state_free: bad handle >>> WARNING: ddi_isoft_state_fini: bad handle >>> >>> The implementation-dependent information kept in the state >>> variable is corrupt. >>> >>> WARNING: ddi_isoft_state_free: null handle >>> WARNING: ddi_isoft_state_fini: null handle >>> >>> The routine has been passed a null or corrupt state pointer. >>> Check that ddi_isoft_state_init() has been called. >>> >>> WARNING: ddi_isoft_state_free: item %d not in range [0..%d] >>> >>> The routine has been asked to free an item which was never >>> allocated. The message prints out the invalid item number and the >>> acceptable range. >>> >>> >>> A.3: Changes to existing ddi_soft_state(9F) >>> ========================================= >>> :r!diff -U5 ddi_soft_state.9f.orig ddi_soft_state.9f >>> >>> --- ddi_soft_state.9f.orig Mon Nov 10 11:45:36 2008 >>> +++ ddi_soft_state.9f Tue Nov 11 18:08:26 2008 >>> @@ -224,22 +224,29 @@ >>> return (0); >>> } >>> >>> SEE ALSO >>> _fini(9E), _init(9E), attach(9E), detach(9E), >>> - ddi_get_instance(9F), getminor(9F), kmem_zalloc(9F) >>> + ddi_get_instance(9F), ddi_isoft_state(9E), ddi_ssoft_state(9E), >>> + getminor(9F), kmem_zalloc(9F) >>> >>> WARNINGS >>> There is no attempt to validate the item parameter given to >>> ddi_soft_state_zalloc() other than it must be a positive signed >>> integer. Therefore very large item numbers may cause the driver to >>> hang forever waiting for virtual memory resources that can never >>> be satisfied. >>> >>> + Use of the ddi_isoft_state(9E) interfaces is encouraged, it >>> + provides stronger typing. >>> + >>> NOTES >>> If necessary, a hierarchy of state structures can be constructed >>> by embedding state pointers in higher order state structures. >>> + >>> + Equivalent functionality with stronger typing is provided >>> + by the newer ddi_isoft_state(9F) interfaces. >>> >>> DIAGNOSTICS >>> All of the messages described below usually indicate bugs in the >>> driver and should not appear in normal operation of the system. >>> > > From Chris.Horne@sun.com Tue Dec 9 21:25:35 2008 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mBA5PYLv011605 for ; Tue, 9 Dec 2008 21:25:34 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id mBA5PD8F026910 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 10 Dec 2008 05:25:33 GMT Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KBN00203B2I4800@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 09 Dec 2008 21:25:30 -0800 (PST) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KBN000B2B2H2920@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 09 Dec 2008 21:25:29 -0800 (PST) Received: from fe-amer-10.sun.com ([192.18.109.80]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id mBA5PTak006592 for ; Wed, 10 Dec 2008 05:25:29 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0KBN00301AYVAJ00@mail-amer.sun.com> (original mail from Chris.Horne@Sun.COM) for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 09 Dec 2008 22:25:29 -0700 (MST) Received: from sun.com ([129.150.33.126]) by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0KBN009T3B2EAY10@mail-amer.sun.com>; Tue, 09 Dec 2008 22:25:27 -0700 (MST) Date: Tue, 09 Dec 2008 22:25:26 -0700 From: Chris Horne Subject: Re: ddi_ssoft_state(9F) and ddi_isoft_state(9F) [PSARC/2008/755 FastTrack timeout 12/17/2008] In-reply-to: <493F4AA4.9010803@sun.com> Sender: Chris.Horne@sun.com To: "Garrett D'Amore" Cc: Edward Pilatowicz , Christopher Horne , PSARC-ext@sun.com Message-id: <493F52C6.6060704@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-Accept-Language: en-us, en X-PMX-Version: 5.4.1.325704 References: <200812060002.mB602OEX021391@sac.sfbay.sun.com> <20081209045121.GD852061@eng.sun.com> <493F3CF4.2050902@sun.com> <493F4AA4.9010803@sun.com> User-Agent: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.4) Gecko/20040414 Status: RO Content-Length: 29332 Hi Garrett Garrett D'Amore wrote: > Chris Horne wrote: > >>Hi Ed >> >> >> >>>hey chris, >>> >>>the current ddi soft state interfaces are "special" in my mind because i >>>thought they were designed to be "fast". (note that i wasn't around for >>>the original design, so my opinion is based off the implmentation.) >>>their implementation uses direct array indexing, avoids locking, and >>>avoids free memory, all seemingly in the name of soft state lookup >>>speed. i guess i'm worried that other people may be making the same >>>assumptions about the performance of the ddi soft state interfaces, and >>>these assumptions don't really carry over to the new interfaces since >>>they are really a thin wrapper around modhash. >>> >> >> >>Yes, performance is an issue for interfaces used in the per-IO code >>path. For the intended use case, the proposed ddi_ssoft_state will not >>be 'directly' involved in the performance path. In the intended use >>case, the performance path 'indirectly' gains access to the >>"target-port" granularity ddi_ssoft_state via a private pointer >>maintained in the finer-grained "target-port,lun" per-scsi_device(9S) >>hba private soft state (PSARC/2008/675). At tran_tgt_init(9E) time of >>a scsi_device(9S), the hba driver is expected to >>ddi_ssoft_state_get(9F) the coarser "target-port" softstate, and >>increment a private "target-port" softstate reference count to account >>for storing a "target-port" softstate pointer in the finer-grained >>per-scsi_device(9S) hba private softstate. >> >> >> >> >>>hence, i'd rather see something more straitforward, like some >>>version/subset of the modhash interfaces promoted to public, or some new >>>ddi*hash* interfaces. >>> >> >>I want the ability to say "its a softstate, but you feed it strings >>instead of integers". With that simple statement, I believe that anyone >>who has written a solaris driver would "get it". I don't want to loose >>that. >> > > > But in your intended usage, it *isn't* a soft state, at least not like I > suspect most people thing of such things (one per device instance, the > master private state for the devinfo node.) ...and I am proposing a peer that is string based. If we had a separate devinfo node to represent the target port addressing, we would have an instance-oriented softstate there - but that is not how the IEEE1275 binding for scsi devices works. Who says that all forms of soft state need to be indexed by integers (or that those integers need to be instance numbers)? I view soft state as a generic term for "(private) software state". > I actually fear that this naming will increase confusion (which soft > state interfaces should I use?) > > >>Keep in mind that any part of the device tree that has multiple complex >>addressing levels compressed into a single devinfo node unit-address >>("@,,") will face this problem - >>which is why I thought the ddi_ssoft_* prefix was warranted. The solution >>to performance path issues will likely involve a private ref_count >>approach similar to our use case above. >> >>If the above explanation is insufficient, to make progress, I am >>willing to change the interface prefix from ddi_ssoft_state_* to >>scsi_taddr_ssoft_state_*, indicating a more restricted use case >>that meets immediate needs. >> > > > FWIW, I'd be happy with that. I'd be equally happy with a generic hash > table interface. not this case. > Though I'd suggest you just call it > "scsi_taddr_state_*()" or somesuch. (Avoid the non-descriptive "s" > prefix, and keep the identifier short.) Even in my immediate use case the scsi_taddr prefix is misleading because we will be using it for scsi, smp, and sata device target port addresses seen on a sas transport - but I will make the prefix change if people insist. -Chris > > -- Garrett > >>Thanks >>-Chris >> >> >> >> >>>ed >>> >>>On Fri, Dec 05, 2008 at 04:02:24PM -0800, Christopher Horne wrote: >>> >>> >>> >>>>I am sponsoring the following fasttrack for myself. Micro/patch >>>>binding is requested. The timer is set to expire on Dec. 17 2008. >>>> >>>>-Chris >>>> >>>> >>>>1. Introduction >>>> 1.1. Project/Component Working Name: >>>> ddi_ssoft_state(9F) and ddi_isoft_state(9F) >>>> 1.2. Name of Document Author/Supplier: >>>> Author: Chris Horne >>>> 1.3. Date of This Document: >>>> Tue Nov 11 18:10:01 MST 2008 >>>> >>>>4. Technical Description >>>> >>>>4.1 Background: >>>> >>>> ddi_soft_state(9F) provides utility interfaces to help device >>>> drivers manage different context. When small integer values, like >>>> ddi_get_instance(9F) return values, are directly related to >>>> context, the current "indexed" soft_state interfaces work well. >>>> >>>>4.2 Problem: >>>> >>>> Two IEEE-1275 '@unit-address' issues can make the current >>>> ddi_soft_state(9F) support ineffective: >>>> >>>> o Compression: Some IEEE-1275 bindings, like the SCSI >>>> '@,' notation, compress multiple addressing >>>> levels into a single 'unit-address' string. If a driver needs >>>> to maintain soft state at intermediate levels, like the >>>> level, there is no instance number to use with >>>> ddi_soft_state(9F), and no dev_info node to allow use of >>>> ddi_[gs]et_driver_private(9F) or ddi_[gs]et_parent_data). >>>> >>>> o Complexity: An addressing level may be complex: both >>>> non-numeric, sparse, and large. >>>> >>>> This situation has forced drivers to maintain soft state context in >>>> ways that no longer fit the ddi_soft_state(9F) model. >>>> >>>> An example is SCSA HBA unit-address context. For the >>>> initial SCSI Parallel Interconnect transport the was >>>> as an integer limited to the [0-15] range, and ddi_soft_state(9F) >>>> worked well. On modern transports the complexity of the >>>> space has mushroomed, as shown below, and the >>>> space is now 64-bits and sparse. >>>> >>>> SPI: disk@0,0 >>>> WWN: disk@w216000c0ff8047dd,0 >>>> GUID: disk@g600c0ff0000000000047dd270fdf0901 (lun identity: MPxIO) >>>> ISCSI: disk@0000iqn.1986-03.com.sun%3A02%3Ac8a82272-b354-c913-80f9-db9cb378a6f60001,0 >>>> >>>>4.3 Proposal: >>>> >>>> The proposal is to extend the ddi_soft_state(9F) programming model >>>> by providing "string" based peers to todays "indexed" soft_state >>>> interfaces. In addition to "string" support, "indexed" interfaces >>>> with stronger typing are proposed. >>>> >>>> Implementation of proposed ddi_ssoft_state(9F) interfaces is built >>>> on top of the 'modhash' interfaces introduced by [1]. >>>> >>>>4.4 Proposed Interfaces: >>>> >>>> ------------------------------------------------------------------ >>>> Interface Name Comm.Lev. Comments >>>> ------------------------------------------------------------------ >>>> >>>> ddi_isoft_state Private "indexed" soft_state >>>> >>>> ddi_ssoft_state " "string" soft_state >>>> >>>> >>>> ddi_isoft_state_init " typed equivalent of >>>> ddi_soft_state_init(9F) >>>> ddi_ssoft_state_init " string peer of >>>> ddi_isoft_state_init(9F) >>>> >>>> >>>> ddi_isoft_state_zalloc " typed equivalent of >>>> ddi_soft_state_zalloc(9F) >>>> ddi_ssoft_state_zalloc " string peer of >>>> ddi_isoft_state_zalloc(9F) >>>> >>>> ddi_isoft_state_get " typed equivalent of >>>> ddi_soft_state_get(9F) >>>> ddi_ssoft_state_get " string peer of >>>> ddi_isoft_state_get(9F) >>>> >>>> ddi_isoft_state_free " typed equivalent of >>>> ddi_soft_state_free(9F) >>>> ddi_ssoft_state_free " string peer of >>>> ddi_isoft_state_free(9F) >>>> >>>> ddi_isoft_state_fini " typed equivalent of >>>> ddi_soft_state_fini(9F) >>>> ddi_ssoft_state_fini " string peer of >>>> ddi_isoft_state_fini(9F) >>>> >>>> ddi_isoft_state_unassigned " return first unassigned >>>> item of an ddi_isoft_state. >>>> >>>>4.5 Interface Prototypes: (defined in sunddi.h): >>>> >>>> typedef struct __ddi_isoft_state ddi_isoft_state; >>>> typedef struct __ddi_ssoft_state ddi_ssoft_state; >>>> >>>> int >>>> ddi_isoft_state_init(ddi_isoft_state **state_p, >>>> size_t size, size_t n_items); >>>> int >>>> ddi_ssoft_state_init(ddi_ssoft_state **state_p, >>>> size_t size, int hash_sz); >>>> >>>> int >>>> ddi_isoft_state_zalloc(ddi_isoft_state *state, int index); >>>> int >>>> ddi_ssoft_state_zalloc(ddi_ssoft_state *state, const char *stringkey); >>>> >>>> void * >>>> ddi_isoft_state_get(ddi_isoft_state *state, int index); >>>> void * >>>> ddi_ssoft_state_get(ddi_ssoft_state *state, const char *stringkey); >>>> >>>> void >>>> ddi_isoft_state_free(ddi_isoft_state *state, int index); >>>> void >>>> ddi_ssoft_state_free(ddi_ssoft_state *state, const char *stringkey); >>>> >>>> void >>>> ddi_isoft_state_fini(ddi_isoft_state **state_p); >>>> void >>>> ddi_ssoft_state_fini(ddi_ssoft_state **state_p); >>>> >>>> int >>>> ddi_isoft_state_unassigned(ddi_isoft_state **state_p); >>>> >>>>4.4 Example: >>>> >>>> Assuming that 'tpaddr' is the portion SCSI >>>> 'unit-address' string, A SCSA HBA driver can use the following to >>>> access its private specific context. >>>> >>>> ss = ddi_isoft_state_get(xx_state, ddi_get_instance(devi)); >>>> tpss = ddi_ssoft_state_get(ss->ss_tpstate, tpaddr); >>>> >>>> Where xx_state would be setup in _init(9E) via >>>> ddi_isoft_state_init(9F), and ss_tpstate would be setup in >>>> xxattach(9E) via ddi_ssoft_state_init(9F). >>>> >>>>4.5 Alternatives: >>>> >>>> An alternative 'compound' approach was considered and prototyped. >>>> The 'compound' approach implemented a set of 'ddi_strid_*(9E)' >>>> string-to-id mapping interfaces, and then used the existing >>>> "indexed" ddi_soft_state(9F) to get from an 'id' to the soft state. >>>> The 'ddi_strid_(9E)' interfaces, while a good fit for some >>>> applications, was found to increase SCSI host adapter driver >>>> complexity compared to the proposed ddi_ssoft_state(9F) interfaces. >>>> The proposed ddi_ssoft_state(9F) interfaces also present a familiar >>>> API: they are based on the well-established ddi_soft_state(9F) API >>>> structure. >>>> >>>>4.6 Man Pages >>>> >>>> See Appendix A. Some of the examples are TBS at this point, more >>>> meaningful examples should be provided before these interfaces are >>>> promoted. >>>> >>>>4.7 Release Binding >>>> >>>> Micro/patch binding is requested. >>>> >>>>4.8 References >>>> >>>> [1] PSARC case that introduced modhash interfaces >>>> PSARC/1998/212 Extensions to Device Autoconfiguration [John Danielson] >>>> http://sac.sfbay/PSARC/1998/212 >>>> http://www.opensolaris.org/os/community/arc/caselog/PSARC/1998/212 >>>> >>>> >>>>Appendix A: >>>> >>>>A.1: New ddi_ssoft_state(9F) >>>>========================================= >>>> :r ddi_ssoft_state.9f >>>> >>>>Kernel Functions for Drivers ddi_ssoft_state(9F) >>>> >>>>NAME >>>> ddi_ssoft_state, ddi_ssoft_state_get, ddi_ssoft_state_fini, >>>> ddi_ssoft_state_free, ddi_ssoft_state_init, ddi_ssoft_state_zalloc - >>>> driver string soft state utility routines >>>> >>>>SYNOPSIS >>>> #include >>>> #include >>>> >>>> typedef struct __ddi_ssoft_state ddi_ssoft_state; >>>> >>>> int ddi_ssoft_state_init(ddi_ssoft_state **state_p, >>>> size_t size, size_t hash_sz); >>>> >>>> int ddi_ssoft_state_zalloc(ddi_ssoft_state *state, int stringkey); >>>> >>>> void *ddi_ssoft_state_get(ddi_ssoft_state *state, int stringkey); >>>> >>>> void ddi_ssoft_state_free(ddi_ssoft_state *state, int stringkey); >>>> >>>> void ddi_ssoft_state_fini(ddi_ssoft_state **state_p); >>>> >>>>INTERFACE LEVEL >>>> Solaris DDI specific (Solaris DDI). >>>> >>>>PARAMETERS >>>> state_p Address of the opaque state pointer which will be >>>> initialized by ddi_ssoft_state_init() to point to >>>> implementation dependent data. >>>> >>>> state An opaque pointer to implementation-dependent data that >>>> describes the soft state. >>>> >>>> size Size of the soft state which will be allocated by subsequent >>>> calls to ddi_ssoft_state_zalloc(); zero is illegal. >>>> >>>> hash_sz The number of hashed lists which will be allocated; >>>> zero is not allowed. There is a performance >>>> .vs. space tradeoff in the selection of a hash_sz >>>> value: for 'n' soft state structures, on average each >>>> ddi_ssoft_state_get will need to traverse 'n'/hash_sz/2 >>>> entries to locate the requested soft state structure. >>>> >>>> stringkey The string key for the soft state structure; usually >>>> associated with some sub-portion of the device's >>>> 'unit-address'. >>>> >>>>DESCRIPTION >>>> >>>> Some drivers must maintain state information at a granularity >>>> unrelated to ddi_get_instance(9F) instance numbers or dev_info >>>> node ddi_get_driver_private(9F) data. >>>> >>>> When the needed soft state granularity can be associated with a >>>> string, such as a sub-portion of a 'unit-address' string, the >>>> ddi_ssoft_state(9F) utility routines can be used to help device >>>> drivers manage the space used by the driver to hold such state >>>> information. >>>> >>>> For example, if a SCSI HBA driver maintains state for each >>>> target-port address in a separate state structure, for transports >>>> with complex target-port address representations these routines >>>> can be used to dynamically allocate and deallocate a separate >>>> structure for each target-port as scsi_device(9S) associated with >>>> the target-port are initialized by tran_tgt_init(9E) and and freed >>>> by tran_tgt_free(9E). >>>> >>>> To use the routines, the driver writer needs to declare a state >>>> pointer, state_p, which the implementation uses as a place to hang >>>> a set of per-driver structures; everything else is managed by >>>> these routines. >>>> >>>> The routine ddi_ssoft_state_init() is usually called in the >>>> driver's attach(9E) routine to initialize the state pointer, and set >>>> the size of the soft state structure. >>>> >>>> The routine ddi_ssoft_state_zalloc() is called to allocate a soft >>>> state. The routine is passed a string key, which is used to refer >>>> to the structure in subsequent calls to ddi_ssoft_state_get() and >>>> ddi_ssoft_state_free(). The string key is usually some sub-portion >>>> of a 'unit-address'. The routine attempts to allocate space for >>>> the new structure, and if the space allocation was successful, >>>> DDI_SUCCESS is returned to the caller. Returned memory is zeroed. >>>> >>>> A pointer to the space previously allocated for a soft state >>>> structure can be obtained by calling ddi_ssoft_state_get() with the >>>> appropriate string key. >>>> >>>> The space used by a given soft state structure can be returned to >>>> the system using ddi_ssoft_state_free(). >>>> >>>> The space used by all the soft state structures allocated on a >>>> given state pointer, together with the housekeeping information >>>> used by the implementation can be returned to the system using >>>> ddi_ssoft_state_fini(). This routine is typically called from the >>>> driver's detach(9E) routine. >>>> >>>> The ddi_ssoft_state_zalloc(), ddi_ssoft_state_free() and >>>> ddi_ssoft_state_get() routines coordinate access to the underlying >>>> data structures in an MT-safe fashion, thus no additional locks >>>> should be necessary. >>>> >>>>RETURN VALUES >>>> >>>> ddi_ssoft_state_get() >>>> >>>> NULL The requested soft state structure was not >>>> allocated at the time of the call. >>>> >>>> pointer The pointer to the soft state structure. >>>> >>>> >>>> >>>> ddi_ssoft_state_init() >>>> >>>> 0 The allocation was successful. >>>> >>>> EINVAL Either the size parameter was zero, or the >>>> state_p parameter was invalid. >>>> >>>> >>>> ddi_ssoft_state_zalloc() >>>> >>>> DDI_SUCCESS The allocation was successful. >>>> >>>> DDI_FAILURE The routine failed to allocate the storage >>>> required; either the state parameter was >>>> invalid, the string key was NULL, or an attempt >>>> was made to allocate using a string key that >>>> was already allocated. >>>> >>>> >>>>CONTEXT >>>> The ddi_ssoft_state_init() and ddi_ssoft_state_alloc() functions >>>> can be called from user or kernel context only, since they may >>>> internally call kmem_zalloc(9F) with the KM_SLEEP flag. >>>> >>>> The ddi_ssoft_state_fini(), ddi_ssoft_state_free() and >>>> ddi_ssoft_state_get() routines can be called from any driver >>>> context. >>>> >>>>EXAMPLES >>>><> >>>> >>>>SEE ALSO >>>> _fini(9E), _init(9E), attach(9E), detach(9E), >>>> ddi_get_instance(9F), ddi_soft_state(9E), ddi_isoft_state(9E), >>>> getminor(9F), kmem_zalloc(9F) >>>> >>>>NOTES >>>> If necessary, a hierarchy of state structures can be constructed >>>> by embedding state pointers in higher order state structures. >>>> >>>>DIAGNOSTICS >>>><> >>>> >>>> All of the messages described below usually indicate bugs in the >>>> driver and should not appear in normal operation of the system. >>>> >>>> WARNING: ddi_ssoft_state_zalloc: bad handle >>>> WARNING: ddi_ssoft_state_free: bad handle >>>> WARNING: ddi_ssoft_state_fini: bad handle >>>> >>>> The implementation-dependent information kept in the state >>>> variable is corrupt. >>>> >>>> WARNING: ddi_ssoft_state_free: null handle >>>> WARNING: ddi_ssoft_state_fini: null handle >>>> >>>> The routine has been passed a null or corrupt state pointer. >>>> Check that ddi_ssoft_state_init() has been called. >>>> >>>> WARNING: ddi_ssoft_state_free: string key '%s' not allocated >>>> >>>> The routine has been asked to free a string key which is not >>>> allocated. The message prints out the invalid string key. >>>> >>>>A.2: New ddi_isoft_state(9F) >>>>========================================= >>>> :r ddi_isoft_state.9f >>>> >>>>Kernel Functions for Drivers ddi_isoft_state(9F) >>>> >>>>NAME >>>> ddi_isoft_state, ddi_isoft_state_get, ddi_isoft_state_fini, >>>> ddi_isoft_state_free, ddi_isoft_state_init, ddi_isoft_state_zalloc - >>>> driver indexed soft state utility routines >>>> >>>>SYNOPSIS >>>> #include >>>> #include >>>> >>>> typedef struct __ddi_isoft_state ddi_isoft_state; >>>> >>>> int ddi_isoft_state_init(ddi_isoft_state **state_p, >>>> size_t size, size_t n_items); >>>> >>>> int ddi_isoft_state_zalloc(ddi_isoft_state *state, int item); >>>> >>>> void *ddi_isoft_state_get(ddi_isoft_state *state, int item); >>>> >>>> void ddi_isoft_state_free(ddi_isoft_state *state, int item); >>>> >>>> void ddi_isoft_state_fini(ddi_isoft_state **state_p); >>>> >>>>INTERFACE LEVEL >>>> Solaris DDI specific (Solaris DDI). >>>> >>>>PARAMETERS >>>> state_p Address of the opaque state pointer which will be >>>> initialized by ddi_isoft_state_init() to point to >>>> implementation dependent data. >>>> >>>> state An opaque pointer to implementation-dependent data that >>>> describes the soft state. >>>> >>>> size Size of the item which will be allocated by subsequent >>>> calls to ddi_isoft_state_zalloc(). >>>> >>>> n_items A hint of the number of items which will be >>>> preallocated; zero is allowed. >>>> >>>> item The item number for the state structure; usually the >>>> instance number of the associated devinfo node. >>>> >>>>DESCRIPTION >>>> Most device drivers maintain state information with each instance >>>> of the device they control; for example, a soft copy of a device >>>> control register, a mutex that must be held while accessing a >>>> piece of hardware, a partition table, or a unit structure. These >>>> utility routines are intended to help device drivers manage the >>>> space used by the driver to hold such state information. >>>> >>>> For example, if the driver holds the state of each instance in a >>>> single state structure, these routines can be used to dynamically >>>> allocate and deallocate a separate structure for each instance of >>>> the driver as the instance is attached and detached. >>>> >>>> To use the routines, the driver writer needs to declare a state >>>> pointer, state_p, which the implementation uses as a place to hang >>>> a set of per-driver structures; everything else is managed by >>>> these routines. >>>> >>>> The routine ddi_isoft_state_init() is usually called in the >>>> driver's _init(9E) routine to initialize the state pointer, set >>>> the size of the soft state structure, and to allow the driver to >>>> pre-allocate a given number of such structures if required. >>>> >>>> The routine ddi_isoft_state_zalloc() is usually called in the >>>> driver's attach(9E) routine. The routine is passed an item number >>>> which is used to refer to the structure in subsequent calls to >>>> ddi_isoft_state_get() and ddi_isoft_state_free(). The item number is >>>> usually just the instance number of the devinfo node, obtained >>>> with ddi_get_instance(9F). The routine attempts to allocate space >>>> for the new structure, and if the space allocation was successful, >>>> DDI_SUCCESS is returned to the caller. Returned memory is zeroed. >>>> >>>> A pointer to the space previously allocated for a soft state >>>> structure can be obtained by calling ddi_isoft_state_get() with the >>>> appropriate item number. >>>> >>>> The space used by a given soft state structure can be returned to >>>> the system using ddi_isoft_state_free(). This routine is usually >>>> called from the driver's detach(9E) entry point. >>>> >>>> The space used by all the soft state structures allocated on a >>>> given state pointer, together with the housekeeping information >>>> used by the implementation can be returned to the system using >>>> ddi_isoft_state_fini(). This routine can be called from the >>>> driver's _fini(9E) routine. >>>> >>>> The ddi_isoft_state_zalloc(), ddi_isoft_state_free() and >>>> ddi_isoft_state_get() routines coordinate access to the underlying >>>> data structures in an MT-safe fashion, thus no additional locks >>>> should be necessary. >>>> >>>>RETURN VALUES >>>> >>>> ddi_isoft_state_get() >>>> >>>> NULL The requested state structure was not allocated >>>> at the time of the call. >>>> >>>> pointer The pointer to the state structure. >>>> >>>> >>>> >>>> ddi_isoft_state_init() >>>> >>>> 0 The allocation was successful. >>>> >>>> EINVAL Either the size parameter was zero, or the >>>> state_p parameter was invalid. >>>> >>>> >>>> ddi_isoft_state_zalloc() >>>> >>>> DDI_SUCCESS The allocation was successful. >>>> >>>> DDI_FAILURE The routine failed to allocate the storage >>>> required; either the state parameter was >>>> invalid, the item number was negative, or an >>>> attempt was made to allocate an item number >>>> that was already allocated. >>>> >>>> >>>>CONTEXT >>>> The ddi_isoft_state_init() and ddi_isoft_state_alloc() functions >>>> can be called from user or kernel context only, since they may >>>> internally call kmem_zalloc(9F) with the KM_SLEEP flag. >>>> >>>> The ddi_isoft_state_fini(), ddi_isoft_state_free() and >>>> ddi_isoft_state_get() routines can be called from any driver >>>> context. >>>> >>>>EXAMPLES >>>> Example 1 Creating and Removing Data Structures >>>> >>>> The following example shows how the routines described above can >>>> be used in terms of the driver entry points of a character-only >>>> driver. The example concentrates on the portions of the code that >>>> deal with creating and removing the driver's data structures. >>>> >>>> >>>> typedef struct { >>>> volatile caddr_t *csr; /* device registers */ >>>> kmutex_t csr_mutex; /* protects 'csr' field */ >>>> unsigned int state; >>>> dev_info_t *dip; /* back pointer to devinfo */ >>>> } devstate_t; >>>> static ddi_isoft_state *statep; >>>> >>>> int >>>> _init(void) >>>> { >>>> int error; >>>> >>>> error = ddi_isoft_state_init(&statep, sizeof (devstate_t), 0); >>>> if (error != 0) >>>> return (error); >>>> if ((error = mod_install(&modlinkage)) != 0) >>>> ddi_isoft_state_fini(&statep); >>>> return (error); >>>> } >>>> >>>> int >>>> _fini(void) >>>> { >>>> int error; >>>> >>>> if ((error = mod_remove(&modlinkage)) != 0) >>>> return (error); >>>> ddi_isoft_state_fini(&statep); >>>> return (0); >>>> } >>>> >>>> static int >>>> xxattach(dev_info_t *dip, ddi_attach_cmd_t cmd) >>>> { >>>> int instance; >>>> devstate_t *softc; >>>> >>>> switch (cmd) { >>>> case DDI_ATTACH: >>>> instance = ddi_get_instance(dip); >>>> if (ddi_isoft_state_zalloc(statep, instance) != >>>> DDI_SUCCESS) >>>> return (DDI_FAILURE); >>>> softc = ddi_isoft_state_get(statep, instance); >>>> softc->dip = dip; >>>> ... >>>> return (DDI_SUCCESS); >>>> default: >>>> return (DDI_FAILURE); >>>> } >>>> } >>>> >>>> static int >>>> xxdetach(dev_info_t *dip, ddi_detach_cmd_t cmd) >>>> { >>>> int instance; >>>> >>>> switch (cmd) { >>>> >>>> case DDI_DETACH: >>>> instance = ddi_get_instance(dip); >>>> ... >>>> ddi_isoft_state_free(statep, instance); >>>> return (DDI_SUCCESS); >>>> >>>> default: >>>> return (DDI_FAILURE); >>>> } >>>> } >>>> >>>> static int >>>> xxopen(dev_t *devp, int flag, int otyp, cred_t *cred_p) >>>> { >>>> devstate_t *softc; >>>> int instance; >>>> >>>> instance = getminor(*devp); >>>> if ((softc = ddi_isoft_state_get(statep, instance)) == NULL) >>>> return (ENXIO); >>>> ... >>>> softc->state |= XX_IN_USE; >>>> ... >>>> return (0); >>>> } >>>> >>>>SEE ALSO >>>> _fini(9E), _init(9E), attach(9E), detach(9E), >>>> ddi_get_instance(9F), getminor(9F), kmem_zalloc(9F) >>>> >>>>WARNINGS >>>> There is no attempt to validate the item parameter given to >>>> ddi_isoft_state_zalloc() other than it must be a positive signed >>>> integer. Therefore very large item numbers may cause the driver to >>>> hang forever waiting for virtual memory resources that can never >>>> be satisfied. >>>> >>>>NOTES >>>> If necessary, a hierarchy of state structures can be constructed >>>> by embedding state pointers in higher order state structures. >>>> >>>>DIAGNOSTICS >>>> All of the messages described below usually indicate bugs in the >>>> driver and should not appear in normal operation of the system. >>>> >>>> WARNING: ddi_isoft_state_zalloc: bad handle >>>> WARNING: ddi_isoft_state_free: bad handle >>>> WARNING: ddi_isoft_state_fini: bad handle >>>> >>>> The implementation-dependent information kept in the state >>>> variable is corrupt. >>>> >>>> WARNING: ddi_isoft_state_free: null handle >>>> WARNING: ddi_isoft_state_fini: null handle >>>> >>>> The routine has been passed a null or corrupt state pointer. >>>> Check that ddi_isoft_state_init() has been called. >>>> >>>> WARNING: ddi_isoft_state_free: item %d not in range [0..%d] >>>> >>>> The routine has been asked to free an item which was never >>>> allocated. The message prints out the invalid item number and the >>>> acceptable range. >>>> >>>> >>>>A.3: Changes to existing ddi_soft_state(9F) >>>>========================================= >>>> :r!diff -U5 ddi_soft_state.9f.orig ddi_soft_state.9f >>>> >>>>--- ddi_soft_state.9f.orig Mon Nov 10 11:45:36 2008 >>>>+++ ddi_soft_state.9f Tue Nov 11 18:08:26 2008 >>>>@@ -224,22 +224,29 @@ >>>> return (0); >>>> } >>>> >>>>SEE ALSO >>>> _fini(9E), _init(9E), attach(9E), detach(9E), >>>>- ddi_get_instance(9F), getminor(9F), kmem_zalloc(9F) >>>>+ ddi_get_instance(9F), ddi_isoft_state(9E), ddi_ssoft_state(9E), >>>>+ getminor(9F), kmem_zalloc(9F) >>>> >>>>WARNINGS >>>> There is no attempt to validate the item parameter given to >>>> ddi_soft_state_zalloc() other than it must be a positive signed >>>> integer. Therefore very large item numbers may cause the driver to >>>> hang forever waiting for virtual memory resources that can never >>>> be satisfied. >>>> >>>>+ Use of the ddi_isoft_state(9E) interfaces is encouraged, it >>>>+ provides stronger typing. >>>>+ >>>>NOTES >>>> If necessary, a hierarchy of state structures can be constructed >>>> by embedding state pointers in higher order state structures. >>>>+ >>>>+ Equivalent functionality with stronger typing is provided >>>>+ by the newer ddi_isoft_state(9F) interfaces. >>>> >>>>DIAGNOSTICS >>>> All of the messages described below usually indicate bugs in the >>>> driver and should not appear in normal operation of the system. >>>> >> >> > > From gdamore@sun.com Wed Dec 10 04:42:59 2008 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mBACgxeA021895 for ; Wed, 10 Dec 2008 04:42:59 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id mBACgxgj016235 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Wed, 10 Dec 2008 04:42:59 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KBN0080XVBMS200@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 10 Dec 2008 05:42:58 -0700 (MST) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KBN00GP5VBLEAB0@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 10 Dec 2008 05:42:57 -0700 (MST) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id mBACgvfp007596 for ; Wed, 10 Dec 2008 04:42:57 -0800 (PST) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0KBN00B01V5V1C00@fe-sfbay-10.sun.com> (original mail from gdamore@sun.com) for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 10 Dec 2008 04:42:57 -0800 (PST) Received: from [10.7.251.172] by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0KBN005CIVBG6LA0@fe-sfbay-10.sun.com>; Wed, 10 Dec 2008 04:42:56 -0800 (PST) Date: Wed, 10 Dec 2008 04:35:31 -0800 From: "Garrett D'Amore" Subject: Re: ddi_ssoft_state(9F) and ddi_isoft_state(9F) [PSARC/2008/755 FastTrack timeout 12/17/2008] In-reply-to: <493F52C6.6060704@sun.com> Sender: Garrett.Damore@sun.com To: Chris Horne Cc: Edward Pilatowicz , Christopher Horne , psarc-ext@sun.com Message-id: <493FB793.5080109@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200812060002.mB602OEX021391@sac.sfbay.sun.com> <20081209045121.GD852061@eng.sun.com> <493F3CF4.2050902@sun.com> <493F4AA4.9010803@sun.com> <493F52C6.6060704@sun.com> User-Agent: Thunderbird 2.0.0.14 (X11/20080616) Status: RO Content-Length: 32030 Chris Horne wrote: > Hi Garrett > > Garrett D'Amore wrote: > >> Chris Horne wrote: >> >> >>> Hi Ed >>> >>> >>> >>> >>>> hey chris, >>>> >>>> the current ddi soft state interfaces are "special" in my mind because i >>>> thought they were designed to be "fast". (note that i wasn't around for >>>> the original design, so my opinion is based off the implmentation.) >>>> their implementation uses direct array indexing, avoids locking, and >>>> avoids free memory, all seemingly in the name of soft state lookup >>>> speed. i guess i'm worried that other people may be making the same >>>> assumptions about the performance of the ddi soft state interfaces, and >>>> these assumptions don't really carry over to the new interfaces since >>>> they are really a thin wrapper around modhash. >>>> >>>> >>> Yes, performance is an issue for interfaces used in the per-IO code >>> path. For the intended use case, the proposed ddi_ssoft_state will not >>> be 'directly' involved in the performance path. In the intended use >>> case, the performance path 'indirectly' gains access to the >>> "target-port" granularity ddi_ssoft_state via a private pointer >>> maintained in the finer-grained "target-port,lun" per-scsi_device(9S) >>> hba private soft state (PSARC/2008/675). At tran_tgt_init(9E) time of >>> a scsi_device(9S), the hba driver is expected to >>> ddi_ssoft_state_get(9F) the coarser "target-port" softstate, and >>> increment a private "target-port" softstate reference count to account >>> for storing a "target-port" softstate pointer in the finer-grained >>> per-scsi_device(9S) hba private softstate. >>> >>> >>> >>> >>> >>>> hence, i'd rather see something more straitforward, like some >>>> version/subset of the modhash interfaces promoted to public, or some new >>>> ddi*hash* interfaces. >>>> >>>> >>> I want the ability to say "its a softstate, but you feed it strings >>> instead of integers". With that simple statement, I believe that anyone >>> who has written a solaris driver would "get it". I don't want to loose >>> that. >>> >>> >> But in your intended usage, it *isn't* a soft state, at least not like I >> suspect most people thing of such things (one per device instance, the >> master private state for the devinfo node.) >> > > > ...and I am proposing a peer that is string based. If we had a separate > devinfo node to represent the target port addressing, we would have an > instance-oriented softstate there - but that is not how the IEEE1275 > binding for scsi devices works. > > Who says that all forms of soft state need to be indexed by integers (or > that those integers need to be instance numbers)? I view soft state > as a generic term for "(private) software state". > Of course you can use ddi_soft_state_xxx for other purposes, but its pretty clear from the examples, naming, and existing practice that this is what it is intended for. Some of the specific details of memory allocation and life time of objects really make sense only in the context of soft state that is managed like instance data. > > >> I actually fear that this naming will increase confusion (which soft >> state interfaces should I use?) >> >> >> >>> Keep in mind that any part of the device tree that has multiple complex >>> addressing levels compressed into a single devinfo node unit-address >>> ("@,,") will face this problem - >>> which is why I thought the ddi_ssoft_* prefix was warranted. The solution >>> to performance path issues will likely involve a private ref_count >>> approach similar to our use case above. >>> >>> If the above explanation is insufficient, to make progress, I am >>> willing to change the interface prefix from ddi_ssoft_state_* to >>> scsi_taddr_ssoft_state_*, indicating a more restricted use case >>> that meets immediate needs. >>> >>> >> FWIW, I'd be happy with that. I'd be equally happy with a generic hash >> table interface. >> > > not this case. > Why not? What you're proposing (or rather what you need) is a generic dictionary lookup mechanism mapping strings to some state. The only additional bits are that this case also imposes some additional memory management "features" (i.e. state structures are fixed sized, and allocation is managed by the API rather than by the calling driver.) > >> Though I'd suggest you just call it >> "scsi_taddr_state_*()" or somesuch. (Avoid the non-descriptive "s" >> prefix, and keep the identifier short.) >> > > Even in my immediate use case the scsi_taddr prefix is misleading because > we will be using it for scsi, smp, and sata device target port addresses > seen on a sas transport - but I will make the prefix change if people insist. > How about "ddi_named_state_xxx"? That will avoid the problem of confusion with ddi_soft_state, and make it a bit clearer (IMO). I'd recommend that you also consider a set of APIs to *walk* the data structure, because in my own experience the inability to easily walk the ddi_soft_state_xxx list is one of the limiting factors in its use. (On more than one occasion I've had to either make my own global linked list in addition to, or instead of, ddi_soft_state, because I needed a way to search through all soft state structures. For example, when mapping a potentially large dynamic dev_t to a specific device structure.) -- Garrett > -Chris > >> -- Garrett >> >> >>> Thanks >>> -Chris >>> >>> >>> >>> >>> >>>> ed >>>> >>>> On Fri, Dec 05, 2008 at 04:02:24PM -0800, Christopher Horne wrote: >>>> >>>> >>>> >>>> >>>>> I am sponsoring the following fasttrack for myself. Micro/patch >>>>> binding is requested. The timer is set to expire on Dec. 17 2008. >>>>> >>>>> -Chris >>>>> >>>>> >>>>> 1. Introduction >>>>> 1.1. Project/Component Working Name: >>>>> ddi_ssoft_state(9F) and ddi_isoft_state(9F) >>>>> 1.2. Name of Document Author/Supplier: >>>>> Author: Chris Horne >>>>> 1.3. Date of This Document: >>>>> Tue Nov 11 18:10:01 MST 2008 >>>>> >>>>> 4. Technical Description >>>>> >>>>> 4.1 Background: >>>>> >>>>> ddi_soft_state(9F) provides utility interfaces to help device >>>>> drivers manage different context. When small integer values, like >>>>> ddi_get_instance(9F) return values, are directly related to >>>>> context, the current "indexed" soft_state interfaces work well. >>>>> >>>>> 4.2 Problem: >>>>> >>>>> Two IEEE-1275 '@unit-address' issues can make the current >>>>> ddi_soft_state(9F) support ineffective: >>>>> >>>>> o Compression: Some IEEE-1275 bindings, like the SCSI >>>>> '@,' notation, compress multiple addressing >>>>> levels into a single 'unit-address' string. If a driver needs >>>>> to maintain soft state at intermediate levels, like the >>>>> level, there is no instance number to use with >>>>> ddi_soft_state(9F), and no dev_info node to allow use of >>>>> ddi_[gs]et_driver_private(9F) or ddi_[gs]et_parent_data). >>>>> >>>>> o Complexity: An addressing level may be complex: both >>>>> non-numeric, sparse, and large. >>>>> >>>>> This situation has forced drivers to maintain soft state context in >>>>> ways that no longer fit the ddi_soft_state(9F) model. >>>>> >>>>> An example is SCSA HBA unit-address context. For the >>>>> initial SCSI Parallel Interconnect transport the was >>>>> as an integer limited to the [0-15] range, and ddi_soft_state(9F) >>>>> worked well. On modern transports the complexity of the >>>>> space has mushroomed, as shown below, and the >>>>> space is now 64-bits and sparse. >>>>> >>>>> SPI: disk@0,0 >>>>> WWN: disk@w216000c0ff8047dd,0 >>>>> GUID: disk@g600c0ff0000000000047dd270fdf0901 (lun identity: MPxIO) >>>>> ISCSI: disk@0000iqn.1986-03.com.sun%3A02%3Ac8a82272-b354-c913-80f9-db9cb378a6f60001,0 >>>>> >>>>> 4.3 Proposal: >>>>> >>>>> The proposal is to extend the ddi_soft_state(9F) programming model >>>>> by providing "string" based peers to todays "indexed" soft_state >>>>> interfaces. In addition to "string" support, "indexed" interfaces >>>>> with stronger typing are proposed. >>>>> >>>>> Implementation of proposed ddi_ssoft_state(9F) interfaces is built >>>>> on top of the 'modhash' interfaces introduced by [1]. >>>>> >>>>> 4.4 Proposed Interfaces: >>>>> >>>>> ------------------------------------------------------------------ >>>>> Interface Name Comm.Lev. Comments >>>>> ------------------------------------------------------------------ >>>>> >>>>> ddi_isoft_state Private "indexed" soft_state >>>>> >>>>> ddi_ssoft_state " "string" soft_state >>>>> >>>>> >>>>> ddi_isoft_state_init " typed equivalent of >>>>> ddi_soft_state_init(9F) >>>>> ddi_ssoft_state_init " string peer of >>>>> ddi_isoft_state_init(9F) >>>>> >>>>> >>>>> ddi_isoft_state_zalloc " typed equivalent of >>>>> ddi_soft_state_zalloc(9F) >>>>> ddi_ssoft_state_zalloc " string peer of >>>>> ddi_isoft_state_zalloc(9F) >>>>> >>>>> ddi_isoft_state_get " typed equivalent of >>>>> ddi_soft_state_get(9F) >>>>> ddi_ssoft_state_get " string peer of >>>>> ddi_isoft_state_get(9F) >>>>> >>>>> ddi_isoft_state_free " typed equivalent of >>>>> ddi_soft_state_free(9F) >>>>> ddi_ssoft_state_free " string peer of >>>>> ddi_isoft_state_free(9F) >>>>> >>>>> ddi_isoft_state_fini " typed equivalent of >>>>> ddi_soft_state_fini(9F) >>>>> ddi_ssoft_state_fini " string peer of >>>>> ddi_isoft_state_fini(9F) >>>>> >>>>> ddi_isoft_state_unassigned " return first unassigned >>>>> item of an ddi_isoft_state. >>>>> >>>>> 4.5 Interface Prototypes: (defined in sunddi.h): >>>>> >>>>> typedef struct __ddi_isoft_state ddi_isoft_state; >>>>> typedef struct __ddi_ssoft_state ddi_ssoft_state; >>>>> >>>>> int >>>>> ddi_isoft_state_init(ddi_isoft_state **state_p, >>>>> size_t size, size_t n_items); >>>>> int >>>>> ddi_ssoft_state_init(ddi_ssoft_state **state_p, >>>>> size_t size, int hash_sz); >>>>> >>>>> int >>>>> ddi_isoft_state_zalloc(ddi_isoft_state *state, int index); >>>>> int >>>>> ddi_ssoft_state_zalloc(ddi_ssoft_state *state, const char *stringkey); >>>>> >>>>> void * >>>>> ddi_isoft_state_get(ddi_isoft_state *state, int index); >>>>> void * >>>>> ddi_ssoft_state_get(ddi_ssoft_state *state, const char *stringkey); >>>>> >>>>> void >>>>> ddi_isoft_state_free(ddi_isoft_state *state, int index); >>>>> void >>>>> ddi_ssoft_state_free(ddi_ssoft_state *state, const char *stringkey); >>>>> >>>>> void >>>>> ddi_isoft_state_fini(ddi_isoft_state **state_p); >>>>> void >>>>> ddi_ssoft_state_fini(ddi_ssoft_state **state_p); >>>>> >>>>> int >>>>> ddi_isoft_state_unassigned(ddi_isoft_state **state_p); >>>>> >>>>> 4.4 Example: >>>>> >>>>> Assuming that 'tpaddr' is the portion SCSI >>>>> 'unit-address' string, A SCSA HBA driver can use the following to >>>>> access its private specific context. >>>>> >>>>> ss = ddi_isoft_state_get(xx_state, ddi_get_instance(devi)); >>>>> tpss = ddi_ssoft_state_get(ss->ss_tpstate, tpaddr); >>>>> >>>>> Where xx_state would be setup in _init(9E) via >>>>> ddi_isoft_state_init(9F), and ss_tpstate would be setup in >>>>> xxattach(9E) via ddi_ssoft_state_init(9F). >>>>> >>>>> 4.5 Alternatives: >>>>> >>>>> An alternative 'compound' approach was considered and prototyped. >>>>> The 'compound' approach implemented a set of 'ddi_strid_*(9E)' >>>>> string-to-id mapping interfaces, and then used the existing >>>>> "indexed" ddi_soft_state(9F) to get from an 'id' to the soft state. >>>>> The 'ddi_strid_(9E)' interfaces, while a good fit for some >>>>> applications, was found to increase SCSI host adapter driver >>>>> complexity compared to the proposed ddi_ssoft_state(9F) interfaces. >>>>> The proposed ddi_ssoft_state(9F) interfaces also present a familiar >>>>> API: they are based on the well-established ddi_soft_state(9F) API >>>>> structure. >>>>> >>>>> 4.6 Man Pages >>>>> >>>>> See Appendix A. Some of the examples are TBS at this point, more >>>>> meaningful examples should be provided before these interfaces are >>>>> promoted. >>>>> >>>>> 4.7 Release Binding >>>>> >>>>> Micro/patch binding is requested. >>>>> >>>>> 4.8 References >>>>> >>>>> [1] PSARC case that introduced modhash interfaces >>>>> PSARC/1998/212 Extensions to Device Autoconfiguration [John Danielson] >>>>> http://sac.sfbay/PSARC/1998/212 >>>>> http://www.opensolaris.org/os/community/arc/caselog/PSARC/1998/212 >>>>> >>>>> >>>>> Appendix A: >>>>> >>>>> A.1: New ddi_ssoft_state(9F) >>>>> ========================================= >>>>> :r ddi_ssoft_state.9f >>>>> >>>>> Kernel Functions for Drivers ddi_ssoft_state(9F) >>>>> >>>>> NAME >>>>> ddi_ssoft_state, ddi_ssoft_state_get, ddi_ssoft_state_fini, >>>>> ddi_ssoft_state_free, ddi_ssoft_state_init, ddi_ssoft_state_zalloc - >>>>> driver string soft state utility routines >>>>> >>>>> SYNOPSIS >>>>> #include >>>>> #include >>>>> >>>>> typedef struct __ddi_ssoft_state ddi_ssoft_state; >>>>> >>>>> int ddi_ssoft_state_init(ddi_ssoft_state **state_p, >>>>> size_t size, size_t hash_sz); >>>>> >>>>> int ddi_ssoft_state_zalloc(ddi_ssoft_state *state, int stringkey); >>>>> >>>>> void *ddi_ssoft_state_get(ddi_ssoft_state *state, int stringkey); >>>>> >>>>> void ddi_ssoft_state_free(ddi_ssoft_state *state, int stringkey); >>>>> >>>>> void ddi_ssoft_state_fini(ddi_ssoft_state **state_p); >>>>> >>>>> INTERFACE LEVEL >>>>> Solaris DDI specific (Solaris DDI). >>>>> >>>>> PARAMETERS >>>>> state_p Address of the opaque state pointer which will be >>>>> initialized by ddi_ssoft_state_init() to point to >>>>> implementation dependent data. >>>>> >>>>> state An opaque pointer to implementation-dependent data that >>>>> describes the soft state. >>>>> >>>>> size Size of the soft state which will be allocated by subsequent >>>>> calls to ddi_ssoft_state_zalloc(); zero is illegal. >>>>> >>>>> hash_sz The number of hashed lists which will be allocated; >>>>> zero is not allowed. There is a performance >>>>> .vs. space tradeoff in the selection of a hash_sz >>>>> value: for 'n' soft state structures, on average each >>>>> ddi_ssoft_state_get will need to traverse 'n'/hash_sz/2 >>>>> entries to locate the requested soft state structure. >>>>> >>>>> stringkey The string key for the soft state structure; usually >>>>> associated with some sub-portion of the device's >>>>> 'unit-address'. >>>>> >>>>> DESCRIPTION >>>>> >>>>> Some drivers must maintain state information at a granularity >>>>> unrelated to ddi_get_instance(9F) instance numbers or dev_info >>>>> node ddi_get_driver_private(9F) data. >>>>> >>>>> When the needed soft state granularity can be associated with a >>>>> string, such as a sub-portion of a 'unit-address' string, the >>>>> ddi_ssoft_state(9F) utility routines can be used to help device >>>>> drivers manage the space used by the driver to hold such state >>>>> information. >>>>> >>>>> For example, if a SCSI HBA driver maintains state for each >>>>> target-port address in a separate state structure, for transports >>>>> with complex target-port address representations these routines >>>>> can be used to dynamically allocate and deallocate a separate >>>>> structure for each target-port as scsi_device(9S) associated with >>>>> the target-port are initialized by tran_tgt_init(9E) and and freed >>>>> by tran_tgt_free(9E). >>>>> >>>>> To use the routines, the driver writer needs to declare a state >>>>> pointer, state_p, which the implementation uses as a place to hang >>>>> a set of per-driver structures; everything else is managed by >>>>> these routines. >>>>> >>>>> The routine ddi_ssoft_state_init() is usually called in the >>>>> driver's attach(9E) routine to initialize the state pointer, and set >>>>> the size of the soft state structure. >>>>> >>>>> The routine ddi_ssoft_state_zalloc() is called to allocate a soft >>>>> state. The routine is passed a string key, which is used to refer >>>>> to the structure in subsequent calls to ddi_ssoft_state_get() and >>>>> ddi_ssoft_state_free(). The string key is usually some sub-portion >>>>> of a 'unit-address'. The routine attempts to allocate space for >>>>> the new structure, and if the space allocation was successful, >>>>> DDI_SUCCESS is returned to the caller. Returned memory is zeroed. >>>>> >>>>> A pointer to the space previously allocated for a soft state >>>>> structure can be obtained by calling ddi_ssoft_state_get() with the >>>>> appropriate string key. >>>>> >>>>> The space used by a given soft state structure can be returned to >>>>> the system using ddi_ssoft_state_free(). >>>>> >>>>> The space used by all the soft state structures allocated on a >>>>> given state pointer, together with the housekeeping information >>>>> used by the implementation can be returned to the system using >>>>> ddi_ssoft_state_fini(). This routine is typically called from the >>>>> driver's detach(9E) routine. >>>>> >>>>> The ddi_ssoft_state_zalloc(), ddi_ssoft_state_free() and >>>>> ddi_ssoft_state_get() routines coordinate access to the underlying >>>>> data structures in an MT-safe fashion, thus no additional locks >>>>> should be necessary. >>>>> >>>>> RETURN VALUES >>>>> >>>>> ddi_ssoft_state_get() >>>>> >>>>> NULL The requested soft state structure was not >>>>> allocated at the time of the call. >>>>> >>>>> pointer The pointer to the soft state structure. >>>>> >>>>> >>>>> >>>>> ddi_ssoft_state_init() >>>>> >>>>> 0 The allocation was successful. >>>>> >>>>> EINVAL Either the size parameter was zero, or the >>>>> state_p parameter was invalid. >>>>> >>>>> >>>>> ddi_ssoft_state_zalloc() >>>>> >>>>> DDI_SUCCESS The allocation was successful. >>>>> >>>>> DDI_FAILURE The routine failed to allocate the storage >>>>> required; either the state parameter was >>>>> invalid, the string key was NULL, or an attempt >>>>> was made to allocate using a string key that >>>>> was already allocated. >>>>> >>>>> >>>>> CONTEXT >>>>> The ddi_ssoft_state_init() and ddi_ssoft_state_alloc() functions >>>>> can be called from user or kernel context only, since they may >>>>> internally call kmem_zalloc(9F) with the KM_SLEEP flag. >>>>> >>>>> The ddi_ssoft_state_fini(), ddi_ssoft_state_free() and >>>>> ddi_ssoft_state_get() routines can be called from any driver >>>>> context. >>>>> >>>>> EXAMPLES >>>>> <> >>>>> >>>>> SEE ALSO >>>>> _fini(9E), _init(9E), attach(9E), detach(9E), >>>>> ddi_get_instance(9F), ddi_soft_state(9E), ddi_isoft_state(9E), >>>>> getminor(9F), kmem_zalloc(9F) >>>>> >>>>> NOTES >>>>> If necessary, a hierarchy of state structures can be constructed >>>>> by embedding state pointers in higher order state structures. >>>>> >>>>> DIAGNOSTICS >>>>> <> >>>>> >>>>> All of the messages described below usually indicate bugs in the >>>>> driver and should not appear in normal operation of the system. >>>>> >>>>> WARNING: ddi_ssoft_state_zalloc: bad handle >>>>> WARNING: ddi_ssoft_state_free: bad handle >>>>> WARNING: ddi_ssoft_state_fini: bad handle >>>>> >>>>> The implementation-dependent information kept in the state >>>>> variable is corrupt. >>>>> >>>>> WARNING: ddi_ssoft_state_free: null handle >>>>> WARNING: ddi_ssoft_state_fini: null handle >>>>> >>>>> The routine has been passed a null or corrupt state pointer. >>>>> Check that ddi_ssoft_state_init() has been called. >>>>> >>>>> WARNING: ddi_ssoft_state_free: string key '%s' not allocated >>>>> >>>>> The routine has been asked to free a string key which is not >>>>> allocated. The message prints out the invalid string key. >>>>> >>>>> A.2: New ddi_isoft_state(9F) >>>>> ========================================= >>>>> :r ddi_isoft_state.9f >>>>> >>>>> Kernel Functions for Drivers ddi_isoft_state(9F) >>>>> >>>>> NAME >>>>> ddi_isoft_state, ddi_isoft_state_get, ddi_isoft_state_fini, >>>>> ddi_isoft_state_free, ddi_isoft_state_init, ddi_isoft_state_zalloc - >>>>> driver indexed soft state utility routines >>>>> >>>>> SYNOPSIS >>>>> #include >>>>> #include >>>>> >>>>> typedef struct __ddi_isoft_state ddi_isoft_state; >>>>> >>>>> int ddi_isoft_state_init(ddi_isoft_state **state_p, >>>>> size_t size, size_t n_items); >>>>> >>>>> int ddi_isoft_state_zalloc(ddi_isoft_state *state, int item); >>>>> >>>>> void *ddi_isoft_state_get(ddi_isoft_state *state, int item); >>>>> >>>>> void ddi_isoft_state_free(ddi_isoft_state *state, int item); >>>>> >>>>> void ddi_isoft_state_fini(ddi_isoft_state **state_p); >>>>> >>>>> INTERFACE LEVEL >>>>> Solaris DDI specific (Solaris DDI). >>>>> >>>>> PARAMETERS >>>>> state_p Address of the opaque state pointer which will be >>>>> initialized by ddi_isoft_state_init() to point to >>>>> implementation dependent data. >>>>> >>>>> state An opaque pointer to implementation-dependent data that >>>>> describes the soft state. >>>>> >>>>> size Size of the item which will be allocated by subsequent >>>>> calls to ddi_isoft_state_zalloc(). >>>>> >>>>> n_items A hint of the number of items which will be >>>>> preallocated; zero is allowed. >>>>> >>>>> item The item number for the state structure; usually the >>>>> instance number of the associated devinfo node. >>>>> >>>>> DESCRIPTION >>>>> Most device drivers maintain state information with each instance >>>>> of the device they control; for example, a soft copy of a device >>>>> control register, a mutex that must be held while accessing a >>>>> piece of hardware, a partition table, or a unit structure. These >>>>> utility routines are intended to help device drivers manage the >>>>> space used by the driver to hold such state information. >>>>> >>>>> For example, if the driver holds the state of each instance in a >>>>> single state structure, these routines can be used to dynamically >>>>> allocate and deallocate a separate structure for each instance of >>>>> the driver as the instance is attached and detached. >>>>> >>>>> To use the routines, the driver writer needs to declare a state >>>>> pointer, state_p, which the implementation uses as a place to hang >>>>> a set of per-driver structures; everything else is managed by >>>>> these routines. >>>>> >>>>> The routine ddi_isoft_state_init() is usually called in the >>>>> driver's _init(9E) routine to initialize the state pointer, set >>>>> the size of the soft state structure, and to allow the driver to >>>>> pre-allocate a given number of such structures if required. >>>>> >>>>> The routine ddi_isoft_state_zalloc() is usually called in the >>>>> driver's attach(9E) routine. The routine is passed an item number >>>>> which is used to refer to the structure in subsequent calls to >>>>> ddi_isoft_state_get() and ddi_isoft_state_free(). The item number is >>>>> usually just the instance number of the devinfo node, obtained >>>>> with ddi_get_instance(9F). The routine attempts to allocate space >>>>> for the new structure, and if the space allocation was successful, >>>>> DDI_SUCCESS is returned to the caller. Returned memory is zeroed. >>>>> >>>>> A pointer to the space previously allocated for a soft state >>>>> structure can be obtained by calling ddi_isoft_state_get() with the >>>>> appropriate item number. >>>>> >>>>> The space used by a given soft state structure can be returned to >>>>> the system using ddi_isoft_state_free(). This routine is usually >>>>> called from the driver's detach(9E) entry point. >>>>> >>>>> The space used by all the soft state structures allocated on a >>>>> given state pointer, together with the housekeeping information >>>>> used by the implementation can be returned to the system using >>>>> ddi_isoft_state_fini(). This routine can be called from the >>>>> driver's _fini(9E) routine. >>>>> >>>>> The ddi_isoft_state_zalloc(), ddi_isoft_state_free() and >>>>> ddi_isoft_state_get() routines coordinate access to the underlying >>>>> data structures in an MT-safe fashion, thus no additional locks >>>>> should be necessary. >>>>> >>>>> RETURN VALUES >>>>> >>>>> ddi_isoft_state_get() >>>>> >>>>> NULL The requested state structure was not allocated >>>>> at the time of the call. >>>>> >>>>> pointer The pointer to the state structure. >>>>> >>>>> >>>>> >>>>> ddi_isoft_state_init() >>>>> >>>>> 0 The allocation was successful. >>>>> >>>>> EINVAL Either the size parameter was zero, or the >>>>> state_p parameter was invalid. >>>>> >>>>> >>>>> ddi_isoft_state_zalloc() >>>>> >>>>> DDI_SUCCESS The allocation was successful. >>>>> >>>>> DDI_FAILURE The routine failed to allocate the storage >>>>> required; either the state parameter was >>>>> invalid, the item number was negative, or an >>>>> attempt was made to allocate an item number >>>>> that was already allocated. >>>>> >>>>> >>>>> CONTEXT >>>>> The ddi_isoft_state_init() and ddi_isoft_state_alloc() functions >>>>> can be called from user or kernel context only, since they may >>>>> internally call kmem_zalloc(9F) with the KM_SLEEP flag. >>>>> >>>>> The ddi_isoft_state_fini(), ddi_isoft_state_free() and >>>>> ddi_isoft_state_get() routines can be called from any driver >>>>> context. >>>>> >>>>> EXAMPLES >>>>> Example 1 Creating and Removing Data Structures >>>>> >>>>> The following example shows how the routines described above can >>>>> be used in terms of the driver entry points of a character-only >>>>> driver. The example concentrates on the portions of the code that >>>>> deal with creating and removing the driver's data structures. >>>>> >>>>> >>>>> typedef struct { >>>>> volatile caddr_t *csr; /* device registers */ >>>>> kmutex_t csr_mutex; /* protects 'csr' field */ >>>>> unsigned int state; >>>>> dev_info_t *dip; /* back pointer to devinfo */ >>>>> } devstate_t; >>>>> static ddi_isoft_state *statep; >>>>> >>>>> int >>>>> _init(void) >>>>> { >>>>> int error; >>>>> >>>>> error = ddi_isoft_state_init(&statep, sizeof (devstate_t), 0); >>>>> if (error != 0) >>>>> return (error); >>>>> if ((error = mod_install(&modlinkage)) != 0) >>>>> ddi_isoft_state_fini(&statep); >>>>> return (error); >>>>> } >>>>> >>>>> int >>>>> Even in my immediate use case the scsi_taddr prefix is misleading because >>>>> we will be using it for scsi, smp, _fini(void) >>>>> { >>>>> int error; >>>>> >>>>> if ((error = mod_remove(&modlinkage)) != 0) >>>>> return (error); >>>>> ddi_isoft_state_fini(&statep); >>>>> return (0); >>>>> } >>>>> >>>>> static int >>>>> xxattach(dev_info_t *dip, ddi_attach_cmd_t cmd) >>>>> { >>>>> int instance; >>>>> devstate_t *softc; >>>>> >>>>> switch (cmd) { >>>>> case DDI_ATTACH: >>>>> instance = ddi_get_instance(dip); >>>>> if (ddi_isoft_state_zalloc(statep, instance) != >>>>> DDI_SUCCESS) >>>>> return (DDI_FAILURE); >>>>> softc = ddi_isoft_state_get(statep, instance); >>>>> softc->dip = dip; >>>>> ... >>>>> return (DDI_SUCCESS); >>>>> default: >>>>> return (DDI_FAILURE); >>>>> } >>>>> } >>>>> >>>>> static int >>>>> xxdetach(dev_info_t *dip, ddi_detach_cmd_t cmd) >>>>> { >>>>> int instance; >>>>> >>>>> switch (cmd) { >>>>> >>>>> case DDI_DETACH: >>>>> instance = ddi_get_instance(dip); >>>>> ... >>>>> ddi_isoft_state_free(statep, instance); >>>>> return (DDI_SUCCESS); >>>>> >>>>> default: >>>>> return (DDI_FAILURE); >>>>> } >>>>> } >>>>> >>>>> static int >>>>> xxopen(dev_t *devp, int flag, int otyp, cred_t *cred_p) >>>>> { >>>>> devstate_t *softc; >>>>> int instance; >>>>> >>>>> instance = getminor(*devp); >>>>> if ((softc = ddi_isoft_state_get(statep, instance)) == NULL) >>>>> return (ENXIO); >>>>> ... >>>>> softc->state |= XX_IN_USE; >>>>> ... >>>>> return (0); >>>>> } >>>>> >>>>> SEE ALSO >>>>> _fini(9E), _init(9E), attach(9E), detach(9E), >>>>> ddi_get_instance(9F), getminor(9F), kmem_zalloc(9F) >>>>> >>>>> WARNINGS >>>>> There is no attempt to validate the item parameter given to >>>>> ddi_isoft_state_zalloc() other than it must be a positive signed >>>>> integer. Therefore very large item numbers may cause the driver to >>>>> hang forever waiting for virtual memory resources that can never >>>>> be satisfied. >>>>> >>>>> NOTES >>>>> If necessary, a hierarchy of state structures can be constructed >>>>> by embedding state pointers in higher order state structures. >>>>> >>>>> DIAGNOSTICS >>>>> All of the messages described below usually indicate bugs in the >>>>> driver and should not appear in normal operation of the system. >>>>> >>>>> WARNING: ddi_isoft_state_zalloc: bad handle >>>>> WARNING: ddi_isoft_state_free: bad handle >>>>> WARNING: ddi_isoft_state_fini: bad handle >>>>> >>>>> The implementation-dependent information kept in the state >>>>> variable is corrupt. >>>>> >>>>> WARNING: ddi_isoft_state_free: null handle >>>>> WARNING: ddi_isoft_state_fini: null handle >>>>> >>>>> The routine has been passed a null or corrupt state pointer. >>>>> Check that ddi_isoft_state_init() has been called. >>>>> >>>>> WARNING: ddi_isoft_state_free: item %d not in range [0..%d] >>>>> >>>>> The routine has been asked to free an item which was never >>>>> allocated. The message prints out the invalid item number and the >>>>> acceptable range. >>>>> >>>>> >>>>> A.3: Changes to existing ddi_soft_state(9F) >>>>> ========================================= >>>>> :r!diff -U5 ddi_soft_state.9f.orig ddi_soft_state.9f >>>>> >>>>> --- ddi_soft_state.9f.orig Mon Nov 10 11:45:36 2008 >>>>> +++ ddi_soft_state.9f Tue Nov 11 18:08:26 2008 >>>>> @@ -224,22 +224,29 @@ >>>>> return (0); >>>>> } >>>>> >>>>> SEE ALSO >>>>> _fini(9E), _init(9E), attach(9E), detach(9E), >>>>> - ddi_get_instance(9F), getminor(9F), kmem_zalloc(9F) >>>>> + ddi_get_instance(9F), ddi_isoft_state(9E), ddi_ssoft_state(9E), >>>>> + getminor(9F), kmem_zalloc(9F) >>>>> >>>>> WARNINGS >>>>> There is no attempt to validate the item parameter given to >>>>> ddi_soft_state_zalloc() other than it must be a positive signed >>>>> integer. Therefore very large item numbers may cause the driver to >>>>> hang forever waiting for virtual memory resources that can never >>>>> be satisfied. >>>>> >>>>> + Use of the ddi_isoft_state(9E) interfaces is encouraged, it >>>>> + provides stronger typing. >>>>> + >>>>> NOTES >>>>> If necessary, a hierarchy of state structures can be constructed >>>>> by embedding state pointers in higher order state structures. >>>>> + >>>>> + Equivalent functionality with stronger typing is provided >>>>> + by the newer ddi_isoft_state(9F) interfaces. >>>>> >>>>> DIAGNOSTICS >>>>> All of the messages described below usually indicate bugs in the >>>>> driver and should not appear in normal operation of the system. >>>>> >>>>> >>> >>> >> > > From Chris.Horne@Sun.COM Wed Dec 10 09:44:30 2008 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mBAHiUvE016075 for ; Wed, 10 Dec 2008 09:44:30 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id mBAHiUuD062992 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 10 Dec 2008 10:44:30 -0700 (MST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KBO00K099A6EB00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 10 Dec 2008 09:44:30 -0800 (PST) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KBO00D8G9A5DPD0@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 10 Dec 2008 09:44:29 -0800 (PST) Received: from fe-amer-10.sun.com ([192.18.109.80]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id mBAHiSsD004215 for ; Wed, 10 Dec 2008 17:44:29 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0KBO00E018EOIL00@mail-amer.sun.com> (original mail from Chris.Horne@Sun.COM) for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 10 Dec 2008 10:44:11 -0700 (MST) Received: from [172.20.25.50] by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0KBO005NU99EYND0@mail-amer.sun.com>; Wed, 10 Dec 2008 10:44:03 -0700 (MST) Date: Wed, 10 Dec 2008 10:40:51 -0700 From: Chris Horne Subject: Re: ddi_ssoft_state(9F) and ddi_isoft_state(9F) [PSARC/2008/755 FastTrack timeout 12/17/2008] In-reply-to: <493F3CF4.2050902@sun.com> Sender: Chris.Horne@Sun.COM To: Edward Pilatowicz , PSARC-ext@Sun.COM Cc: Christopher Horne Message-id: <493FFF23.4030202@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200812060002.mB602OEX021391@sac.sfbay.sun.com> <20081209045121.GD852061@eng.sun.com> <493F3CF4.2050902@sun.com> User-Agent: Thunderbird 1.5.0.8 (X11/20061204) Status: RO Content-Length: 3828 After talking with Ed, we have converged. He suggested that the spec should be updated to replace the proposed ddi_ssoft_state_init(9F) 'hash_sz' argument with an 'n_items' hint-type argument (like ddi_soft_state(9F)) to avoid exposing details of a modhash implementation below. This change to a 'hint' facilitates the implementation switching from a modhash to an avl tree if performance becomes an issue. I still need to talk more with Garret. If that does not occur before/at PSARC today, we will need to extend the timer. In either case a new spec will be required. -Chris > Hi Ed > > >> hey chris, >> >> the current ddi soft state interfaces are "special" in my mind because i >> thought they were designed to be "fast". (note that i wasn't around for >> the original design, so my opinion is based off the implmentation.) >> their implementation uses direct array indexing, avoids locking, and >> avoids free memory, all seemingly in the name of soft state lookup >> speed. i guess i'm worried that other people may be making the same >> assumptions about the performance of the ddi soft state interfaces, and >> these assumptions don't really carry over to the new interfaces since >> they are really a thin wrapper around modhash. >> > > > Yes, performance is an issue for interfaces used in the per-IO code > path. For the intended use case, the proposed ddi_ssoft_state will not > be 'directly' involved in the performance path. In the intended use > case, the performance path 'indirectly' gains access to the > "target-port" granularity ddi_ssoft_state via a private pointer > maintained in the finer-grained "target-port,lun" per-scsi_device(9S) > hba private soft state (PSARC/2008/675). At tran_tgt_init(9E) time of > a scsi_device(9S), the hba driver is expected to > ddi_ssoft_state_get(9F) the coarser "target-port" softstate, and > increment a private "target-port" softstate reference count to account > for storing a "target-port" softstate pointer in the finer-grained > per-scsi_device(9S) hba private softstate. > > > >> hence, i'd rather see something more straitforward, like some >> version/subset of the modhash interfaces promoted to public, or some new >> ddi*hash* interfaces. >> > > I want the ability to say "its a softstate, but you feed it strings > instead of integers". With that simple statement, I believe that anyone > who has written a solaris driver would "get it". I don't want to loose > that. > > Keep in mind that any part of the device tree that has multiple complex > addressing levels compressed into a single devinfo node unit-address > ("@,,") will face this problem - > which is why I thought the ddi_ssoft_* prefix was warranted. The solution > to performance path issues will likely involve a private ref_count > approach similar to our use case above. > > If the above explanation is insufficient, to make progress, I am > willing to change the interface prefix from ddi_ssoft_state_* to > scsi_taddr_ssoft_state_*, indicating a more restricted use case > that meets immediate needs. > > Thanks > -Chris > from ddi_ssoft_state(9F) proposed man page... >>> int ddi_ssoft_state_init(ddi_ssoft_state **state_p, >>> size_t size, size_t hash_sz); >>> >>> hash_sz The number of hashed lists which will be allocated; >>> zero is not allowed. There is a performance >>> .vs. space tradeoff in the selection of a hash_sz >>> value: for 'n' soft state structures, on average each >>> ddi_ssoft_state_get will need to traverse 'n'/hash_sz/2 >>> entries to locate the requested soft state structure. >>> >>> Change to int ddi_ssoft_state_init(ddi_ssoft_state **state_p, size_t size, size_t n_items); ... n_items A hint of the number of items which will be allocated. From Garrett.Damore@Sun.COM Wed Dec 10 10:39:15 2008 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mBAIdEvZ019420 for ; Wed, 10 Dec 2008 10:39:15 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id mBAIdDn7015027 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Thu, 11 Dec 2008 02:39:13 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KBO00J0PBTC3900@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 10 Dec 2008 11:39:12 -0700 (MST) Received: from sca-es-mail-2.sun.com ([192.18.43.133]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KBO0094YBTARX80@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 10 Dec 2008 11:39:10 -0700 (MST) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id mBAIdAgi013510 for ; Wed, 10 Dec 2008 10:39:10 -0800 (PST) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0KBO00801BJ7JB00@fe-sfbay-09.sun.com> (original mail from Garrett.Damore@Sun.COM) for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 10 Dec 2008 10:39:10 -0800 (PST) Received: from [129.153.2.14] by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0KBO00I5PBT14T00@fe-sfbay-09.sun.com>; Wed, 10 Dec 2008 10:39:02 -0800 (PST) Date: Wed, 10 Dec 2008 10:39:01 -0800 From: "Garrett D'Amore" Subject: Re: ddi_ssoft_state(9F) and ddi_isoft_state(9F) [PSARC/2008/755 FastTrack timeout 12/17/2008] In-reply-to: <493FFF23.4030202@sun.com> Sender: Garrett.Damore@Sun.COM To: Chris Horne Cc: Edward Pilatowicz , PSARC-ext@Sun.COM, Christopher Horne Message-id: <49400CC5.1090708@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200812060002.mB602OEX021391@sac.sfbay.sun.com> <20081209045121.GD852061@eng.sun.com> <493F3CF4.2050902@sun.com> <493FFF23.4030202@sun.com> User-Agent: Thunderbird 2.0.0.16 (X11/20080807) Status: RO Content-Length: 5207 On 12/10/08 09:40, Chris Horne wrote: > After talking with Ed, we have converged. He suggested that the spec > should be > updated to replace the proposed ddi_ssoft_state_init(9F) 'hash_sz' > argument with > an 'n_items' hint-type argument (like ddi_soft_state(9F)) to avoid > exposing details > of a modhash implementation below. This change to a 'hint' facilitates > the > implementation switching from a modhash to an avl tree if performance > becomes an issue. > > I still need to talk more with Garret. If that does not occur > before/at PSARC > today, we will need to extend the timer. > > In either case a new spec will be required. The case was let run. There were other objections to the originally proposed names. I still believe calling this "soft state" violates the principle of least surprise, because "soft state" to me is a one-per-device-instance thing, which is not the intended use. If the project team is willing to call these interfaces ddi_named_state_xxx Then I can give the project a +1 without major reservations. [ as an aside, I think an even better/simpler approach would be something more parallel to ddi_set_private/get_private, like this: ddi_set_named_private(), ddi_get_named_private()... ] With ddi_str_soft_state_xxx, I am less happy. The name is both more awkward to use, and seems a bit more vague. (Is the state a string? Is it really "soft state" -- what is "soft state" anyway? The definition isn't clear, and your new usage sort of flies in the face of existing convention, in at least my opinion.) That said, if you can find at least one other member to give the project a +1 with ddi_str_soft_state_xxx, I'll abide the decision and not argue it further. -- Garrett > > -Chris >> Hi Ed >> >> >>> hey chris, >>> >>> the current ddi soft state interfaces are "special" in my mind >>> because i >>> thought they were designed to be "fast". (note that i wasn't around >>> for >>> the original design, so my opinion is based off the implmentation.) >>> their implementation uses direct array indexing, avoids locking, and >>> avoids free memory, all seemingly in the name of soft state lookup >>> speed. i guess i'm worried that other people may be making the same >>> assumptions about the performance of the ddi soft state interfaces, and >>> these assumptions don't really carry over to the new interfaces since >>> they are really a thin wrapper around modhash. >>> >> >> >> Yes, performance is an issue for interfaces used in the per-IO code >> path. For the intended use case, the proposed ddi_ssoft_state will not >> be 'directly' involved in the performance path. In the intended use >> case, the performance path 'indirectly' gains access to the >> "target-port" granularity ddi_ssoft_state via a private pointer >> maintained in the finer-grained "target-port,lun" per-scsi_device(9S) >> hba private soft state (PSARC/2008/675). At tran_tgt_init(9E) time of >> a scsi_device(9S), the hba driver is expected to >> ddi_ssoft_state_get(9F) the coarser "target-port" softstate, and >> increment a private "target-port" softstate reference count to account >> for storing a "target-port" softstate pointer in the finer-grained >> per-scsi_device(9S) hba private softstate. >> >> >> >>> hence, i'd rather see something more straitforward, like some >>> version/subset of the modhash interfaces promoted to public, or some >>> new >>> ddi*hash* interfaces. >>> >> >> I want the ability to say "its a softstate, but you feed it strings >> instead of integers". With that simple statement, I believe that anyone >> who has written a solaris driver would "get it". I don't want to loose >> that. >> >> Keep in mind that any part of the device tree that has multiple complex >> addressing levels compressed into a single devinfo node unit-address >> ("@,,") will face this problem - >> which is why I thought the ddi_ssoft_* prefix was warranted. The >> solution >> to performance path issues will likely involve a private ref_count >> approach similar to our use case above. >> >> If the above explanation is insufficient, to make progress, I am >> willing to change the interface prefix from ddi_ssoft_state_* to >> scsi_taddr_ssoft_state_*, indicating a more restricted use case >> that meets immediate needs. >> >> Thanks >> -Chris >> > from ddi_ssoft_state(9F) proposed man page... > >>>> int ddi_ssoft_state_init(ddi_ssoft_state **state_p, >>>> size_t size, size_t hash_sz); >>>> >>>> hash_sz The number of hashed lists which will be allocated; >>>> zero is not allowed. There is a performance >>>> .vs. space tradeoff in the selection of a hash_sz >>>> value: for 'n' soft state structures, on average each >>>> ddi_ssoft_state_get will need to traverse 'n'/hash_sz/2 >>>> entries to locate the requested soft state structure. >>>> >>>> > Change to > > int ddi_ssoft_state_init(ddi_ssoft_state **state_p, > size_t size, size_t n_items); > ... > n_items A hint of the number of items which will be > allocated. > > > > From Chris.Horne@Sun.COM Thu Dec 11 12:49:11 2008 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mBBKnAo3019583 for ; Thu, 11 Dec 2008 12:49:10 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id mBBKn8lc010203 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Fri, 12 Dec 2008 04:49:09 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KBQ00C0DCHVYV00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 11 Dec 2008 13:49:07 -0700 (MST) Received: from brmea-mail-2.sun.com ([192.18.98.43]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KBQ00K4QCHV3EE0@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 11 Dec 2008 13:49:07 -0700 (MST) Received: from fe-amer-09.sun.com ([192.18.109.79]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id mBBKn7WE013475 for ; Thu, 11 Dec 2008 20:49:07 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0KBQ004018H4PM00@mail-amer.sun.com> (original mail from Chris.Horne@Sun.COM) for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 11 Dec 2008 13:49:07 -0700 (MST) Received: from [172.20.25.50] by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0KBQ00HT9CHME020@mail-amer.sun.com>; Thu, 11 Dec 2008 13:48:59 -0700 (MST) Date: Thu, 11 Dec 2008 13:45:47 -0700 From: Chris Horne Subject: Re: ddi_ssoft_state(9F) and ddi_isoft_state(9F) [PSARC/2008/755 FastTrack timeout 12/17/2008] In-reply-to: <49400CC5.1090708@sun.com> Sender: Chris.Horne@Sun.COM To: "Garrett D'Amore" Cc: Edward Pilatowicz , PSARC-ext@Sun.COM, Christopher Horne Message-id: <49417BFB.9000901@sun.com> MIME-version: 1.0 Content-type: multipart/mixed; boundary="Boundary_(ID_dagnieo1jVDuRQDrPY/p/A)" X-PMX-Version: 5.4.1.325704 References: <200812060002.mB602OEX021391@sac.sfbay.sun.com> <20081209045121.GD852061@eng.sun.com> <493F3CF4.2050902@sun.com> <493FFF23.4030202@sun.com> <49400CC5.1090708@sun.com> User-Agent: Thunderbird 1.5.0.8 (X11/20061204) Status: RO Content-Length: 14236 This is a multi-part message in MIME format. --Boundary_(ID_dagnieo1jVDuRQDrPY/p/A) Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT I have reached closure with Garret and Ed, the updated spec is below. To summarize the changes: rename: ddi_ssoft_state* -> ddi_soft_state_bystr* drop: ddi_isoft_state* args: ddi_soft_state_bystr_init: rename: hash_sz -> n_items The timer has been reset to 12/17/2008. Thanks -Chris --Boundary_(ID_dagnieo1jVDuRQDrPY/p/A) Content-type: text/plain; name=soft_state.fasttrack.txt Content-transfer-encoding: 7BIT Content-disposition: inline; filename=soft_state.fasttrack.txt 1. Introduction 1.1. Project/Component Working Name: ddi_soft_state_bystr(9F) 1.2. Name of Document Author/Supplier: Author: Chris Horne 1.3. Date of This Document: Tue Nov 11 18:10:01 MST 2008 4. Technical Description 4.1 Background: ddi_soft_state(9F) provides utility interfaces to help device drivers manage different context. When small integer values, like ddi_get_instance(9F) return values, are directly related to context, the current "indexed" soft_state interfaces work well. 4.2 Problem: Two IEEE-1275 '@unit-address' issues can make the current ddi_soft_state(9F) support ineffective: o Compression: Some IEEE-1275 bindings, like the SCSI '@,' notation, compress multiple addressing levels into a single 'unit-address' string. If a driver needs to maintain soft state at intermediate levels, like the level, there is no instance number to use with ddi_soft_state(9F), and no dev_info node to allow use of ddi_[gs]et_driver_private(9F) or ddi_[gs]et_parent_data. o Complexity: An addressing level may be complex: both non-numeric, sparse, and large. This situation has forced drivers to maintain device addressing related soft state context in ways that no longer fit the "integer index" ddi_soft_state(9F) model. An example is SCSA HBA unit-address context. For the initial SCSI Parallel Interconnect transport the was as an integer limited to the [0-15] range, and ddi_soft_state(9F) worked well. On modern transports the complexity of the space has mushroomed, as shown below, and the space is now 64-bits and sparse. SPI: disk@0,0 WWN: disk@w216000c0ff8047dd,0 GUID: disk@g600c0ff0000000000047dd270fdf0901 (lun identity: MPxIO) ISCSI: disk@0000iqn.1986-03.com.sun%3A02%3Ac8a82272-b354-c913-80f9-db9cb378a6f60001,0 4.3 Proposal: The proposal is to extend the ddi_soft_state(9F) programming model by providing "string index" based peers to todays "integer index" soft_state interfaces. The initial implementation of proposed ddi_soft_state_bystr(9F) interfaces is built on top of the 'modhash' interfaces introduced by [1], but migration to an AVL tree [2] is possible if performance is a problem. 4.4 Proposed Interfaces: All interfaces are currently proposed as 'private', but promotion to 'stable' is expected after we have more experience. An initial draft of the man pages are provided. ------------------------------------------------------------------ Interface Name Comm.Lev. Comments ------------------------------------------------------------------ ddi_soft_state_bystr Private "string indexed" soft_state pool ddi_soft_state_bystr_init " string peer of ddi_soft_state_init(9F) ddi_soft_state_bystr_zalloc " string peer of ddi_soft_state_zalloc(9F) ddi_soft_state_bystr_get " string peer of ddi_soft_state_get(9F) ddi_soft_state_bystr_free " string peer of ddi_soft_state_free(9F) ddi_soft_state_bystr_fini " string peer of ddi_soft_state_fini(9F) 4.5 Interface Prototypes: (defined in sunddi.h): typedef struct __ddi_soft_state_bystr ddi_soft_state_bystr; int ddi_soft_state_bystr_init(ddi_soft_state_bystr **state_p, size_t size, int n_items); int ddi_soft_state_bystr_zalloc(ddi_soft_state_bystr *state, const char *str); void * ddi_soft_state_bystr_get(ddi_soft_state_bystr *state, const char *str); void ddi_soft_state_bystr_free(ddi_soft_state_bystr *state, const char *str); void ddi_soft_state_bystr_fini(ddi_soft_state_bystr **state_p); 4.4 Example: Assuming that 'tpaddr' is the portion SCSI 'unit-address' string, a SCSA HBA driver can use the following to access its private specific context. ss = ddi_soft_state_get(xx_state, ddi_get_instance(devi)); tpss = ddi_soft_state_bystr_get(ss->ss_tpstate, tpaddr); Where xx_state would be setup in _init(9E) via ddi_soft_state_init(9F), and ss_tpstate would be setup in xxattach(9E) via ddi_soft_state_bystr_init(9F). 4.5 Alternatives: An alternative 'compound' approach was considered and prototyped. The 'compound' approach implemented a set of 'ddi_strid_*(9E)' string-to-id mapping interfaces, and then used the existing "indexed" ddi_soft_state(9F) to get from an 'id' to the soft state. The 'ddi_strid_(9E)' interfaces, while a good fit for some applications, was found to increase SCSI host adapter driver complexity compared to the proposed ddi_soft_state_bystr(9F) interfaces. The proposed ddi_soft_state_bystr(9F) interfaces also present a familiar API: they are based on the well-established ddi_soft_state(9F) API structure. 4.6 Man Pages See Appendix A. Some of the examples are TBS at this point, more meaningful examples should be provided before these interfaces are promoted. 4.7 Release Binding Micro/patch binding is requested. 4.8 References [1] PSARC case that introduced modhash interfaces PSARC/1998/212 Extensions to Device Autoconfiguration [John Danielson] http://sac.sfbay/PSARC/1998/212 http://www.opensolaris.org/os/community/arc/caselog/PSARC/1998/212 [2] AVL Tree Interfaces PSARC/2002/399 AVL Tree Interfaces http://www.opensolaris.org/os/community/arc/caselog/PSARC/2002/399 Appendix A: A.1: New ddi_soft_state_bystr(9F) ========================================= :r ddi_soft_state_bystr.9f Kernel Functions for Drivers ddi_soft_state_bystr(9F) NAME ddi_soft_state_bystr, ddi_soft_state_bystr_get, ddi_soft_state_bystr_fini, ddi_soft_state_bystr_free, ddi_soft_state_bystr_init, ddi_soft_state_bystr_zalloc - driver string-indexed soft state utility routines SYNOPSIS #include #include typedef struct __ddi_soft_state_bystr ddi_soft_state_bystr; int ddi_soft_state_bystr_init(ddi_soft_state_bystr **state_p, size_t size, size_t n_items); int ddi_soft_state_bystr_zalloc(ddi_soft_state_bystr *state, char *str); void *ddi_soft_state_bystr_get(ddi_soft_state_bystr *state, char *str); void ddi_soft_state_bystr_free(ddi_soft_state_bystr *state, char *str); void ddi_soft_state_bystr_fini(ddi_soft_state_bystr **state_p); INTERFACE LEVEL Solaris DDI specific (Solaris DDI). PARAMETERS state_p Address of the opaque state pointer which will be initialized by ddi_soft_state_bystr_init() to point to implementation dependent data. state An opaque pointer to implementation-dependent data that describes the soft state. size Size of the soft state which will be allocated by subsequent calls to ddi_soft_state_bystr_zalloc(). n_items A hint of the number of items which will be allocated. Allocation of more items is possible, but performance may degrade. str The string index for the soft state structure; usually associated with some sub-portion of the device's 'unit-address'. DESCRIPTION Some drivers must maintain device unit-address state information at a granularity unrelated to ddi_get_instance(9F) instance numbers, dev_info node ddi_get_driver_private(9F) data, or subsystem specific private data mechanisms like scsi_device_hba_private_set(9F). When the needed soft state granularity can be associated with a string, such as a sub-portion of a device 'unit-address' string, the ddi_soft_state_bystr(9F) utility routines can be used to help device drivers manage the space used by the driver to hold such state information. For example, if a SCSI HBA driver maintains state for each target-port address in a separate soft state structure, for transports with complex target-port address representations, these routines can be used to dynamically allocate and deallocate a separate soft state structure for each target-port address. To use the routines, the driver writer needs to declare a state pointer, state_p, which the implementation uses as a place to maintain context for a related set of soft states; everything else is managed by these routines. The routine ddi_soft_state_bystr_init() is usually called in the driver's attach(9E) routine to initialize the state pointer, and set the size of the soft state structure. The routine ddi_soft_state_bystr_zalloc() is called to allocate a new soft state. The routine is passed a string index, which is used to refer to the structure in subsequent calls to ddi_soft_state_bystr_get() and ddi_soft_state_bystr_free(). The string index is usually some sub-portion of a device 'unit-address'. The routine attempts to allocate space for a new soft state, and if the space allocation was successful, DDI_SUCCESS is returned to the caller. Returned memory is zeroed. A pointer to the space previously allocated for a soft state structure can be obtained by calling ddi_soft_state_bystr_get() with the appropriate string index. The space associated with the soft state at a given string index can be returned to the system using ddi_soft_state_bystr_free(). The space used by all the soft state structures allocated on a given state pointer, together with the housekeeping information used by the implementation can be returned to the system using ddi_soft_state_bystr_fini(). This routine is typically called from the driver's detach(9E) routine. The ddi_soft_state_bystr_zalloc(), ddi_soft_state_bystr_free() and ddi_soft_state_bystr_get() routines coordinate access to the underlying data structures in an MT-safe fashion, thus no additional locks should be necessary. RETURN VALUES ddi_soft_state_bystr_get() NULL The requested soft state structure was not allocated at the time of the call. pointer The pointer to the soft state structure. ddi_soft_state_bystr_init() 0 The allocation was successful. EINVAL Either the size parameter was zero, or the state_p parameter was invalid. ddi_soft_state_bystr_zalloc() DDI_SUCCESS The allocation was successful. DDI_FAILURE The routine failed to allocate the storage required; either the state parameter was invalid, the string index was NULL, or an attempt was made to allocate using a string index that was already allocated. CONTEXT The ddi_soft_state_bystr_init() and ddi_soft_state_bystr_alloc() functions can be called from user or kernel context only, since they may internally call kmem_zalloc(9F) with the KM_SLEEP flag. The ddi_soft_state_bystr_fini(), ddi_soft_state_bystr_free() and ddi_soft_state_bystr_get() routines can be called from any driver context. EXAMPLES <> SEE ALSO _fini(9E), _init(9E), attach(9E), detach(9E), ddi_get_instance(9F), ddi_soft_state(9E), getminor(9F), kmem_zalloc(9F) NOTES If necessary, a hierarchy of state structures can be constructed by embedding state pointers in higher order state structures. DIAGNOSTICS <> All of the messages described below usually indicate bugs in the driver and should not appear in normal operation of the system. WARNING: ddi_soft_state_bystr_zalloc: bad handle WARNING: ddi_soft_state_bystr_free: bad handle WARNING: ddi_soft_state_bystr_fini: bad handle The implementation-dependent information kept in the state variable is corrupt. WARNING: ddi_soft_state_bystr_free: null handle WARNING: ddi_soft_state_bystr_fini: null handle The routine has been passed a null or corrupt state pointer. Check that ddi_soft_state_bystr_init() has been called. WARNING: ddi_soft_state_bystr_free: string index '%s' not allocated The routine has been asked to free a string index which is not allocated. The message prints out the invalid string index. A.2: Changes to existing ddi_soft_state(9F) ========================================= :r!diff -U5 ddi_soft_state.9f.orig ddi_soft_state.9f --- ddi_soft_state.9f.orig Mon Nov 10 11:45:36 2008 +++ ddi_soft_state.9f Tue Nov 11 18:08:26 2008 @@ -224,22 +224,29 @@ return (0); } SEE ALSO _fini(9E), _init(9E), attach(9E), detach(9E), - ddi_get_instance(9F), getminor(9F), kmem_zalloc(9F) + ddi_get_instance(9F), ddi_soft_state_bystr(9E), + getminor(9F), kmem_zalloc(9F) WARNINGS There is no attempt to validate the item parameter given to ddi_soft_state_zalloc() other than it must be a positive signed integer. Therefore very large item numbers may cause the driver to hang forever waiting for virtual memory resources that can never be satisfied. NOTES If necessary, a hierarchy of state structures can be constructed by embedding state pointers in higher order state structures. DIAGNOSTICS All of the messages described below usually indicate bugs in the driver and should not appear in normal operation of the system. --Boundary_(ID_dagnieo1jVDuRQDrPY/p/A)-- From Chris.Horne@sun.com Sun Jan 11 19:00:18 2009 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n0C30IR5001083 for ; Sun, 11 Jan 2009 19:00:18 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n0C30FKG021231 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Sun, 11 Jan 2009 19:00:18 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KDC0050H8CH5P00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.Com); Sun, 11 Jan 2009 19:00:17 -0800 (PST) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KDC00LO58CG7L30@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.Com); Sun, 11 Jan 2009 19:00:16 -0800 (PST) Received: from fe-amer-09.sun.com ([192.18.109.79]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n0C30GQK001239 for ; Mon, 12 Jan 2009 03:00:16 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0KDC007018C9W200@mail-amer.sun.com> (original mail from Chris.Horne@Sun.COM) for PSARC-ext@Sun.Com (ORCPT PSARC-ext@Sun.Com); Sun, 11 Jan 2009 20:00:16 -0700 (MST) Received: from sun.com ([129.150.32.61]) by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0KDC00FO48C3SC00@mail-amer.sun.com>; Sun, 11 Jan 2009 20:00:16 -0700 (MST) Date: Sun, 11 Jan 2009 20:00:03 -0700 From: Chris Horne Subject: Re: ddi_ssoft_state(9F) and ddi_isoft_state(9F) [PSARC/2008/755 FastTrack timeout 12/17/2008] In-reply-to: <200812060002.mB602OEX021391@sac.sfbay.sun.com> Sender: Chris.Horne@sun.com To: Christopher Horne Cc: PSARC-ext@sun.com Message-id: <496AB233.9020108@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-Accept-Language: en-us, en X-PMX-Version: 5.4.1.325704 References: <200812060002.mB602OEX021391@sac.sfbay.sun.com> User-Agent: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.4) Gecko/20040414 Status: RO Content-Length: 62 This case was approved at last week's PSARC meeting. -Chris