From cth@sac.sfbay.sun.com Fri Jun 27 15:52:45 2008 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5RMqiNw017376 for ; Fri, 27 Jun 2008 15:52:45 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id m5RMqgGY009368; Sat, 28 Jun 2008 06:52:43 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0K35002018VVXZ00@nwk-avmta-2.sfbay.sun.com>; Fri, 27 Jun 2008 15:52:43 -0700 (PDT) Received: from dm-sfbay-01.sfbay.sun.com ([129.145.155.118]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0K3500HGB8VU7N90@nwk-avmta-2.sfbay.sun.com>; Fri, 27 Jun 2008 15:52:42 -0700 (PDT) Received: from sac.sfbay.sun.com (new-sac.SFBay.Sun.COM [129.146.175.65]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m5RMqgY8029683; Fri, 27 Jun 2008 15:52:42 -0700 (PDT) Received: from sac.sfbay.sun.com (localhost [127.0.0.1]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5RMqeYb017371; Fri, 27 Jun 2008 15:52:40 -0700 (PDT) Received: (from cth@localhost) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id m5RMqeaa017367; Fri, 27 Jun 2008 15:52:40 -0700 (PDT) Date: Fri, 27 Jun 2008 15:52:40 -0700 (PDT) From: Christopher Horne Subject: Properties and Devinfo Snapshots [PSARC/2008/412 FastTrack timeout 07/09/2008] To: PSARC-ext@sun.com Message-id: <200806272252.m5RMqeaa017367@sac.sfbay.sun.com> Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 Status: RO Content-Length: 13137 I am sponsoring the following fasttrack for myself, requesting micro/patch binding and a timeout of 07/09/2008. -Chris Template Version: @(#)sac_nextcase 1.66 04/17/08 SMI This information is Copyright 2008 Sun Microsystems 1. Introduction 1.1 Project/Component Working Name: Properties and Devinfo Snapshots 1.2 Name of Document Author/Supplier: Author: Chris Horne 1.3 Date of This Document: Wed May 21 12:37:32 MDT 2008 4. Technical Description 4.1 Problem This proposal addresses two problems related to the interaction between Solaris kernel device tree properties and di_init(3DEVINFO) device tree devinfo snapshots [1]: 1. Some types of dynamic properties are currently not represented in a snapshot. An example is the size(9P) properties [2]. 2. When a value associated with a property changes, a driver has no way to invalidate the current DINFOCACHE cached snapshot [3] to ensure that the next request for a cached snapshot contains the latest value. An example is CR 6268239 [4]. In addition, the proposal covers a change in prtconf(1M) '-v' output to move minor/dev_t specific property information down into the "Minor Device Nodes" section introduced by [6]. 4.2 Proposal The proposal is to: 1. Provide a mechanism so that pure-dynamic properties (defined below) can be captured in a snapshot. This mechanism involves defining a new 'i_ddi_prop_dyn_t' type, and associated interfaces. Since pure-dynamic properties are not covered by the DDI, details of 'i_ddi_prop_dyn_t' are private (no man pages provided). 2. Provide an interface so that drivers can express how state changes effect 'important' property values in the current cached snapshot. A driver that is willing to present stale property values in a cached snapshot (current behavior) does not need to use this interface. The proposed ddi_prop_cache_invalidate() takes a dip argument. The semantic is that all cached properties associated with the dip are invalid, and the next cached snapshot must obtain a new set of values. The initial implementation will invalidate the entire cached snapshot. 3. Capture dynamic properties in snapshots, and have 'prtconf -v' display minor/dev_t specific properties in the "Device Minor Nodes" section. 4.3 Details 4.3.1 Pure-dynamic Properties in Snapshots If a prop_op(9E) implementation has property-name-specific code, that property is considered 'dynamic'. There are two types of dynamic properties: update-type and pure-type. An update-type dynamic property is one where the framework knows the property name and type information, and the driver dynamically updates the value using the public ddi_prop_update_*(9F) interfaces within prop_op(9E). A pure-type dynamic property is one where the framework does not know the property name and type. To obtain the value of a pure-type dynamic property, the code asking for the value must 'know' the property name and type. The prop_op(9E) entry point was initially designed to support pure-type dynamic properties. Today we have DDI support for update-type dynamic properties, but official DDI support for pure-type dynamic properties was deprecated in 1994 by [5]. When pure-type dynamic property use is not directly exposed to a driver's prop_op(9E) implementation, there are still situations where its use make sense. For example, using pure-dynamic properties to support size(9P) reduces the space required to represent the per-partition information. For a driver that needs to implement size(9P) in a 'common' way, we should encourage further use of "common label" (cmlb) interfaces. The cmlb interfaces are currently not ARCed, but they are already used by sd, cmdk, dad, and xdf drivers. Keeping cmlb interfaces project private (not ARCed) until they are stabilized (and possibly even until we agree on a more extensible partition naming approach) was part of the original cmlb plan. The fact that the cmlb interfaces currently use pure-type dynamic properties to implement size(9P) is an implementation detail. Currently, pure-dynamic properties don't show up in snapshots. This means that configuration information implemented using pure-type dynamic properties is missing. As a result, applications chose other more expensive and less scalable mechanisms to obtain information. With dynamic properties in a snapshot, a single di_init call can return the size(9P) information for all devices/partitions in the system. For a property to show up in a snapshot, the devinfo driver must have access to property name and type information. A new 'i_ddi_prop_dyn_t' mechanism defines a NULL terminated array that describes the name and type information of supported pure-type dynamic properties. A 'i_ddi_prop_dyn_t' array is registered with a devinfo node using i_ddi_prop_dyn_driver_set() (or i_ddi_prop_dyn_parent_set()). This gives the devinfo driver enough information to represent the pure-type dynamic properties in a snapshot. The initial consumer of 'i_ddi_prop_dyn_t' will be cmlb. Current cmlb consumers are modified to call the new cmlb_prop_op() interface from their prop_op(9E) implementations. The definition of the size(9P) properties in a 'i_ddi_prop_dyn_t' array is done in misc/cmlb code, with the i_ddi_prop_dyn_driver_set() registration occurring off cmlb_attach(). The cmlb ioctls that support partition size changes will invalidate the cached snapshot size(9P) properties using i_ddi_prop_dyn_cache_invalidate(). For now, drivers that support size(9P) but don't use cmlb will need to define and register the 'i_ddi_prop_dyn_t' array themselves (md, fd, pcide). The expectation is that these drivers should migrate to cmlb, or switch to using update-type dynamic properties (and handle cache snapshot synchronization using ddi_prop_cache_invalidate). Since a driver can't support pure-type dynamic properties using public interfaces, and 'i_ddi_prop_dyn_t' interfaces are specific to pure-type dynamic properties, and details are expected to stay private (and not described further). 4.3.2 Driver Control of Cached Snapshot Property Values. When a driver updates a property value using ddi_prop_update outside ddi_prop_op (direct-update) or updates driver state in such a way that ddi_prop_op(9E) will return a new value for a dynamic property (indirect-update), the driver needs a mechanism to inform the system of the implications relative to the cached devinfo snapshot. The proposal is to add a new ddi_prop_cache_invalidate interface. int ddi_prop_cache_invalidate(dev_t dev, dev_info_t *dip, char *name, int flags); When a driver calls this interface, the next cached snapshot operation is guaranteed to return the latest value for the named property. While the interface arguments are capable of describing an invalidation of a specific property, the implementation may operate at a much coarser level - including an invalidation of the entire cached snapshot. High-frequency use of ddi_prop_cache_invalidate() should be avoided, since it may cause extra overhead for the next request for the cached snapshot. An alternative approach would be to have ddi_prop_update_* perform cached snapshot invalidation on all calls. This approach was discounted because it does not work for indirect-update properties, and it does not give the driver control over what conditions warrant an invalidation. 4.3.3 Prtconf(1M) Display of minor/dev_t Specific Properties. The prtconf(1M) "-v" output currently has a section for "Device Minor Nodes". Changes associated with this case will move minor/dev_t specific "Driver properties" down into a "Minor properties" subsection of "Device Minor Node". This way minor/dev_t property values show up with the minor/dev_t to which the property applies. The following example shows how the size(9P) dynamic properties will show up in 'prtconf -v' output shows i ---%<--- sd, instance #3 ---%<--- Driver properties: ---%<--- + name='device-nblocks' type=int64 items=1 dev=none + value=00000000021bd565 ---%<--- Device Minor Nodes: dev=(32,24) dev_path=/pci@8,700000/scsi@6,1/sd@2,0:a spectype=blk type=minor dev_link=/dev/dsk/c1t2d0s0 dev_link=/dev/sd0a dev_path=/pci@8,700000/scsi@6,1/sd@2,0:a,raw spectype=chr type=minor dev_link=/dev/rdsk/c1t2d0s0 dev_link=/dev/rsd0a + Minor properties: + name='Size' type=int64 items=1 dev=(32,24) + value=00000001f421e000 + name='Nblocks' type=int64 items=1 dev=(32,24) + value=0000000000fa10f0 ---%<--- Prtconf(1M) output is not-an-interface. 4.4 Future Application. This mechanism allows information to be represented in a di_init() devinfo snapshot even when the 'native' form of the information is something other than ddi properties. This capability should prove useful as a way for management applications to gather consolidated point-in-time configuration information without needing to resort to custom ioctls. Prototype work was done that allows the SCSA capabilities defined in scsi_ifgetcap(9F) to be captured in a di_init devinfo snapshot for all SCSA HBAs by providing a common scsi_hba.c:scsi_hba_bus_prop_op implementation. ---%<--- name='scsa-dma-max' type=int items=1 dev=none value=00ffffff name='scsa-msg-out' type=int items=1 dev=none value=00000001 name='scsa-disconnect' type=int items=1 dev=none value=00000001 name='scsa-synchronous' type=int items=1 dev=none value=00000001 ... ---%<--- Like the cmlb size(9P) implementation, the use of pure-type dynamic properties is completely contained within common misc/scsi code and does not impact SCSA HBA or target drivers. The cache synchronization can be provided within the framework target->HBA driver scsi_if_getcap() code. This SCSA work may be delivered at a future date. 4.5 Interface Table ------------------------------------------------------------------------ Interface Level Comments ------------------------------------------------------------------------ i_ddi_prop_dyn_t Private Support pure-type dynamic i_ddi_prop_dyn_driver_set properties in snapshots. i_ddi_prop_dyn_driver_get Expected to stay private. i_ddi_prop_dyn_parent_set i_ddi_prop_dyn_parent_get i_ddi_prop_dyn_cache_invalidate cmlb_prop_op() Private Common label prop_op(9E): supports size(9E) properties (as pure-type dynamic properties). May be promoted when cmlb is ARCed. di_init(3DEVINFO) Committed New properties captured: by using i_ddi_prop_dyn_t interfaces, pure-type dynamic properties are now captured in snapshots. These properties 'existed' before, but were not captured. ddi_prop_cache_invalidate Private Invalidate a property in the current cached devinfo snapshot - next cached snapshot will return the latest value. May be promoted when we have more experience with the proposed interface. 'prtconf -v' output Not-an-interface Change in output: minor/dev_t specific properties have moved from "Driver properties" down to new "Minor properties" subsection of "Device Minor Nodes". ------------------------------------------------------------------------ int ddi_prop_cache_invalidate(dev_t dev, dev_info_t *dip, char *name, int flags); 4.6 References [1] Libdevinfo Reimplementation http://sac.sfbay/PSARC/1997/127 http://www.opensolaris.org/os/community/arc/caselog/1997/127 [2] Device Size Properties http://sac.sfbay/PSARC/2007/339 http://www.opensolaris.org/os/community/arc/caselog/2007/339 [3] Libdevinfo snapshot cache http://sac.sfbay/PSARC/2004/169 http://www.opensolaris.org/os/community/arc/caselog/2004/169 [4] CR 6268239 created properties not showing up in DINFOCACHE ... . [5] DDI Interfaces for PCI http://sac.sfbay/PSARC/1994/075 http://www.opensolaris.org/os/community/arc/caselog/1994/075 [6] Layered Driver Framework Updates http://sac.sfbay/PSARC/2003/537 http://www.opensolaris.org/os/community/arc/caselog/2003/537 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: FastTrack From Chris.Horne@sun.com Thu Jul 10 18:11:57 2008 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m6B1BumA018202 for ; Thu, 10 Jul 2008 18:11:57 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id m6B1BrDF010437 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Fri, 11 Jul 2008 09:11:55 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0K3T00K01HZUYR00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.Com); Thu, 10 Jul 2008 18:11:54 -0700 (PDT) Received: from brmea-mail-3.sun.com ([192.18.98.34]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0K3T00ICIHZU8060@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.Com); Thu, 10 Jul 2008 18:11:54 -0700 (PDT) Received: from fe-amer-09.sun.com ([192.18.109.79]) by brmea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id m6B1BsTA015762 for ; Fri, 11 Jul 2008 01:11:54 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K3T00H01HRNGU00@mail-amer.sun.com> (original mail from Chris.Horne@Sun.COM) for PSARC-ext@Sun.Com (ORCPT PSARC-ext@Sun.Com); Thu, 10 Jul 2008 19:11:54 -0600 (MDT) Received: from sun.com ([129.150.32.51]) by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K3T00EV6HZSV3B0@mail-amer.sun.com>; Thu, 10 Jul 2008 19:11:53 -0600 (MDT) Date: Thu, 10 Jul 2008 19:11:52 -0600 From: Chris Horne Subject: Re: Properties and Devinfo Snapshots [PSARC/2008/412 FastTrack timeout 07/09/2008] In-reply-to: <200806272252.m5RMqeaa017367@sac.sfbay.sun.com> Sender: Chris.Horne@sun.com To: Christopher Horne Cc: PSARC-ext@sun.com Message-id: <4876B358.6000009@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-Accept-Language: en-us, en X-PMX-Version: 5.4.1.325704 References: <200806272252.m5RMqeaa017367@sac.sfbay.sun.com> User-Agent: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.4) Gecko/20040414 Status: RO Content-Length: 43 This case was approved yesterday. -Chris