From sacadmin Mon Feb 25 12:53:24 2008 Received: from sac.sfbay.sun.com (localhost [127.0.0.1]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m1PKrO6D025386; Mon, 25 Feb 2008 12:53:24 -0800 (PST) Received: (from dp@localhost) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id m1PKrOG8025382; Mon, 25 Feb 2008 12:53:24 -0800 (PST) Date: Mon, 25 Feb 2008 12:53:24 -0800 (PST) From: Dan Price Message-Id: <200802252053.m1PKrOG8025382@sac.sfbay.sun.com> To: PSARC-record@sac.sfbay.sun.com Subject: sgen(7d) multiple opens [PSARC/2008/146 FastTrack timeout 3/3/2008] Status: RO Content-Length: 560 Template Version: @(#)sac_nextcase 1.64 07/13/07 SMI This information is Copyright 2008 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: sgen(7d) multiple opens 1.2. Name of Document Author/Supplier: Author: Eric Schrock 1.3 Date of This Document: 25 February, 2008 4. Technical Description See the case directory for more detail 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: FastTrack 6.6. ARC Exposure: open From dp@eng.sun.com Mon Feb 25 13:14:50 2008 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m1PLEnAK026632 for ; Mon, 25 Feb 2008 13:14:49 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m1PLEj0j019873; Mon, 25 Feb 2008 21:14:46 GMT Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JWT00J09CCLT600@brm-avmta-1.central.sun.com>; Mon, 25 Feb 2008 14:14:45 -0700 (MST) Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.63]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JWT00DNRCCK9B80@brm-avmta-1.central.sun.com>; Mon, 25 Feb 2008 14:14:44 -0700 (MST) Received: from snowdog (snowdog.SFBay.Sun.COM [129.146.228.213]) by jurassic-x4600.sfbay.sun.com (8.14.2+Sun/8.14.2) with ESMTP id m1PLEiII005900; Mon, 25 Feb 2008 13:14:44 -0800 (PST) Date: Mon, 25 Feb 2008 13:19:52 -0800 From: Dan Price Subject: PSARC/2008/146 sgen(7D) Multiple Opens To: psarc-ext@sun.com Cc: Christopher Horne , Eric Schrock Message-id: <20080225211952.GE28219@eng.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 User-Agent: Mutt/1.5.14 (2007-02-12) Status: RO Content-Length: 3319 I am pleased to sponsor this fast-track for Eric Schrock. The binding requested is minor release, and the timer is set for Monday March 3. Thanks, -dp ------ sgen(7D) Multiple Opens Eric Schrock A. DESCRIPTION When the sgen(7D) driver was introduced in PSARC 1999/525, the primary targets were specialized SCSI devices such as scanners and tape devices. These devices are typically stateful, requiring a single point of control so that consumers don't overwrite the same state and get confused. Because of this, sgen(7D) enforces O_EXCL behavior even when not requested by the consumer. At the time, this made sense given the devices targeted by sgen. There are many devices, however, that do not require such exclusive access because the protocol has built in safeguards to manage simultaneous access. The most notable of these is storage devices implementing SES (SCSI Enclosure Services). While these devices are currently bound to ses(7D) by default on Solaris (which does support multiple consumers), it is desirable to implement such consumers entirely in userland via uscsi, regardless of the underlying driver. The current design of sgen prevents this from being practical, as many consumers will want to maintain an open SES handle, and being forced to manage exclusive access across all such consumers is not practical. This case seeks to relax the exclusivity of sgen devices to that of normal devices, where concurrent opens are allowed unless O_EXCL is set. B. DETAILS This is a simple change to the sgen open routine, as the driver already enforces serialized access when issuing uscsi commands. If an O_EXCL open request is received, this will be stored in the driver state so that future opens will fail. The sgen driver also has the behavior that if the driver is suspended, all future operations return EIO until the device is re-opened. With multiple opens, it would be nice to allow future opens but have existing references return EIO, but this would be quite complex to implement. A much simpler solution that preserves the original intent is to simply prevent all opens until the last reference is closed. Given the rarity of suspend events, this seems sufficient. C. MANPAGE DIFFS *************** *** 384,394 **** above. ERRORS EBUSY The device was opened by another thread or pro- ! cess. The driver maintains a strict exclusive-open SunOS 5.11 Last change: 29 Sep 1999 6 --- 384,395 ---- above. ERRORS EBUSY The device was opened by another thread or pro- ! cess using the O_EXCL flag, or the device is ! currently open and O_EXCL is being requested. SunOS 5.11 Last change: 29 Sep 1999 6 *************** *** 399,411 **** Devices sgen(7D) - policy for each device. - - ENXIO During opening, the device did not respond to a TEST UNIT READY SCSI command. ENOTTY Indicates that the device does not support the --- 400,409 ---- -- Daniel Price - Solaris Kernel Engineering - dp@eng.sun.com - blogs.sun.com/dp From gdamore@sun.com Mon Feb 25 14:28:53 2008 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m1PMSqcd000529 for ; Mon, 25 Feb 2008 14:28:53 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id m1PMSovd020199 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Tue, 26 Feb 2008 06:28:51 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JWT00203FS2JI00@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Mon, 25 Feb 2008 15:28:50 -0700 (MST) Received: from sca-es-mail-2.sun.com ([192.18.43.133]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JWT000D2FS1JX10@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Mon, 25 Feb 2008 15:28:49 -0700 (MST) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m1PMSnvO007554 for ; Mon, 25 Feb 2008 14:28:49 -0800 (PST) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JWT00301FCJ5F00@fe-sfbay-10.sun.com> (original mail from gdamore@sun.com) for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Mon, 25 Feb 2008 14:28:49 -0800 (PST) Received: from [192.168.251.106] ([76.174.83.55]) by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JWT00KBLFRLQUG0@fe-sfbay-10.sun.com>; Mon, 25 Feb 2008 14:28:33 -0800 (PST) Date: Mon, 25 Feb 2008 14:26:44 -0800 From: "Garrett D'Amore" Subject: Re: PSARC/2008/146 sgen(7D) Multiple Opens In-reply-to: <20080225211952.GE28219@eng.sun.com> Sender: Garrett.Damore@sun.com To: Dan Price Cc: psarc-ext@sun.com, Christopher Horne , Eric Schrock Message-id: <47C340A4.1000203@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <20080225211952.GE28219@eng.sun.com> User-Agent: Thunderbird 2.0.0.6 (X11/20071023) Status: RO Content-Length: 4204 As far as Suspend/Resume goes, the "correct" behavior, IMO, is that open() "blocks" if it is called while the device is suspended, at least if it needs to access the device. (If open() doesn't need to touch the hardware, but only updates soft state, then there is no reason to prohibit it from running normally even if the device is technically suspended.) Put another way, apart from observations involving the time it takes to run, I do not believe that there should be any noticeably different behavior to consumers of a device that supports suspend/resume. A separate EIO return value seems like a poor choice to me. Please feel free to contact me offline if you desire further feedback or discussion pertaining to S/R of devices. -- Garrett Dan Price wrote: > I am pleased to sponsor this fast-track for Eric Schrock. The binding > requested is minor release, and the timer is set for Monday March 3. > > Thanks, > > -dp > > ------ > sgen(7D) Multiple Opens > Eric Schrock > > A. DESCRIPTION > > When the sgen(7D) driver was introduced in PSARC 1999/525, the primary > targets were specialized SCSI devices such as scanners and tape devices. > These devices are typically stateful, requiring a single point of > control so that consumers don't overwrite the same state and get > confused. Because of this, sgen(7D) enforces O_EXCL behavior even when > not requested by the consumer. At the time, this made sense given the > devices targeted by sgen. > > There are many devices, however, that do not require such exclusive > access because the protocol has built in safeguards to manage > simultaneous access. The most notable of these is storage devices > implementing SES (SCSI Enclosure Services). While these devices are > currently bound to ses(7D) by default on Solaris (which does support > multiple consumers), it is desirable to implement such consumers > entirely in userland via uscsi, regardless of the underlying driver. > The current design of sgen prevents this from being practical, as many > consumers will want to maintain an open SES handle, and being forced to > manage exclusive access across all such consumers is not practical. > > This case seeks to relax the exclusivity of sgen devices to that of > normal devices, where concurrent opens are allowed unless O_EXCL is set. > > > B. DETAILS > > This is a simple change to the sgen open routine, as the driver already > enforces serialized access when issuing uscsi commands. If an O_EXCL > open request is received, this will be stored in the driver state so > that future opens will fail. > > The sgen driver also has the behavior that if the driver is suspended, > all future operations return EIO until the device is re-opened. With > multiple opens, it would be nice to allow future opens but have existing > references return EIO, but this would be quite complex to implement. A > much simpler solution that preserves the original intent is to simply > prevent all opens until the last reference is closed. Given the rarity > of suspend events, this seems sufficient. > > > C. MANPAGE DIFFS > > *************** > *** 384,394 **** > above. > > > ERRORS > EBUSY The device was opened by another thread or pro- > ! cess. The driver maintains a strict exclusive-open > > > > SunOS 5.11 Last change: 29 Sep 1999 6 > > --- 384,395 ---- > above. > > > ERRORS > EBUSY The device was opened by another thread or pro- > ! cess using the O_EXCL flag, or the device is > ! currently open and O_EXCL is being requested. > > > > SunOS 5.11 Last change: 29 Sep 1999 6 > > *************** > *** 399,411 **** > > Devices sgen(7D) > > > > - policy for each device. > - > - > ENXIO During opening, the device did not respond to a > TEST UNIT READY SCSI command. > > > ENOTTY Indicates that the device does not support the > --- 400,409 ---- > > > > From eschrock@zion.sfbay.sun.com Wed Mar 5 11:08:57 2008 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m25J8u81012650 for ; Wed, 5 Mar 2008 11:08:57 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m25J8Ztm002307; Wed, 5 Mar 2008 19:08:53 GMT Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JX900E0LUISZ700@brm-avmta-1.central.sun.com>; Wed, 05 Mar 2008 12:08:52 -0700 (MST) Received: from zion.sfbay.sun.com ([129.146.17.75]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JX900859UISZK60@brm-avmta-1.central.sun.com>; Wed, 05 Mar 2008 12:08:52 -0700 (MST) Received: from zion.sfbay.sun.com (localhost [127.0.0.1]) by zion.sfbay.sun.com (8.14.2+Sun/8.14.2) with ESMTP id m25J8p1g410090; Wed, 05 Mar 2008 19:08:51 +0000 (GMT) Received: (from eschrock@localhost) by zion.sfbay.sun.com (8.14.2+Sun/8.14.2/Submit) id m25J8pgM410089; Wed, 05 Mar 2008 11:08:51 -0800 (PST) Date: Wed, 05 Mar 2008 11:08:50 -0800 From: Eric Schrock Subject: Re: PSARC/2008/146 sgen(7D) Multiple Opens In-reply-to: <47C340A4.1000203@sun.com> To: "Garrett D'Amore" Cc: Dan Price , psarc-ext@sun.com, Christopher Horne Message-id: <20080305190850.GB409817@eng.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <20080225211952.GE28219@eng.sun.com> <47C340A4.1000203@sun.com> User-Agent: Mutt/1.4.2.1i Status: RO Content-Length: 1199 For the record, we discussed this issue offline and determined that the current behavior, while leaving something to be desired, is valid given the use of the driver (userland pass-through). Any changes to this behavior are beyond the scope of this particular case. - Eric On Mon, Feb 25, 2008 at 02:26:44PM -0800, Garrett D'Amore wrote: > As far as Suspend/Resume goes, the "correct" behavior, IMO, is that > open() "blocks" if it is called while the device is suspended, at least > if it needs to access the device. (If open() doesn't need to touch the > hardware, but only updates soft state, then there is no reason to > prohibit it from running normally even if the device is technically > suspended.) > > Put another way, apart from observations involving the time it takes to > run, I do not believe that there should be any noticeably different > behavior to consumers of a device that supports suspend/resume. > > A separate EIO return value seems like a poor choice to me. > > Please feel free to contact me offline if you desire further feedback or > discussion pertaining to S/R of devices. -- Eric Schrock, Fishworks http://blogs.sun.com/eschrock From gdamore@sun.com Wed Mar 5 11:14:42 2008 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m25JEgcw013369 for ; Wed, 5 Mar 2008 11:14:42 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m25JEZEP005261 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 5 Mar 2008 19:14:41 GMT Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JX900K09USFLE00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.COM); Wed, 05 Mar 2008 11:14:39 -0800 (PST) Received: from sca-es-mail-2.sun.com ([192.18.43.133]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JX900DKGUSEUFB0@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.COM); Wed, 05 Mar 2008 11:14:38 -0800 (PST) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m25JEcOs020320 for ; Wed, 05 Mar 2008 11:14:38 -0800 (PST) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JX900301TYH5100@fe-sfbay-09.sun.com> (original mail from gdamore@sun.com) for PSARC-ext@Sun.COM (ORCPT PSARC-ext@Sun.COM); Wed, 05 Mar 2008 11:14:38 -0800 (PST) Received: from [192.168.251.106] ([76.174.83.55]) by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JX9003KEUSCTK30@fe-sfbay-09.sun.com>; Wed, 05 Mar 2008 11:14:36 -0800 (PST) Date: Wed, 05 Mar 2008 11:12:25 -0800 From: "Garrett D'Amore" Subject: Re: PSARC/2008/146 sgen(7D) Multiple Opens In-reply-to: <20080305190850.GB409817@eng.sun.com> Sender: Garrett.Damore@sun.com To: Eric Schrock Cc: Dan Price , PSARC-ext@sun.com, Christopher Horne Message-id: <47CEF099.3030104@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <20080225211952.GE28219@eng.sun.com> <47C340A4.1000203@sun.com> <20080305190850.GB409817@eng.sun.com> User-Agent: Thunderbird 2.0.0.6 (X11/20071023) Status: RO Content-Length: 1904 Right. I'm sorry, I should have remembered that I was the one that initiated this conversation at today's meeting. Anyway, other members, I believe that we have converged on this case. I do believe that at some point, the PM team needs to take a hard look at the case of "device drivers in userland" and figure out a more general way to allow userland components to participate in S/R notification, etc. Once upon a time, I believe that used to be RCM hooks for this -- but I'm not certain that those hooks are called in the x86 S/R code. But this issue can be handled outside of the context of this case. -- Garrett Eric Schrock wrote: > For the record, we discussed this issue offline and determined that the > current behavior, while leaving something to be desired, is valid given > the use of the driver (userland pass-through). Any changes to this > behavior are beyond the scope of this particular case. > > - Eric > > On Mon, Feb 25, 2008 at 02:26:44PM -0800, Garrett D'Amore wrote: > >> As far as Suspend/Resume goes, the "correct" behavior, IMO, is that >> open() "blocks" if it is called while the device is suspended, at least >> if it needs to access the device. (If open() doesn't need to touch the >> hardware, but only updates soft state, then there is no reason to >> prohibit it from running normally even if the device is technically >> suspended.) >> >> Put another way, apart from observations involving the time it takes to >> run, I do not believe that there should be any noticeably different >> behavior to consumers of a device that supports suspend/resume. >> >> A separate EIO return value seems like a poor choice to me. >> >> Please feel free to contact me offline if you desire further feedback or >> discussion pertaining to S/R of devices. >> > > -- > Eric Schrock, Fishworks http://blogs.sun.com/eschrock > From mws@zion.sfbay.sun.com Fri Mar 28 09:59:29 2008 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m2SGxT5L000565 for ; Fri, 28 Mar 2008 09:59:29 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m2SGxSJW013332; Fri, 28 Mar 2008 09:59:28 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JYG0015H9V3EV00@nwk-avmta-1.sfbay.Sun.COM>; Fri, 28 Mar 2008 09:59:28 -0700 (PDT) Received: from zion.sfbay.sun.com ([129.146.17.75]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JYG001N09T95600@nwk-avmta-1.sfbay.Sun.COM>; Fri, 28 Mar 2008 09:58:22 -0700 (PDT) Received: from zion.sfbay.sun.com (localhost [127.0.0.1]) by zion.sfbay.sun.com (8.14.2+Sun/8.14.2) with ESMTP id m2SGwLH2014257; Fri, 28 Mar 2008 16:58:21 +0000 (GMT) Received: (from mws@localhost) by zion.sfbay.sun.com (8.14.2+Sun/8.14.2/Submit) id m2SGwKRQ014256; Fri, 28 Mar 2008 09:58:20 -0700 (PDT) Date: Fri, 28 Mar 2008 09:58:20 -0700 (PDT) From: Michael Shapiro Subject: Re: PSARC/2008/146 sgen(7D) Multiple Opens In-reply-to: <47CEF099.3030104@sun.com> To: gdamore@sun.com (Garrett D'Amore) Cc: Eric.Schrock@sun.com, dp@eng.sun.com, PSARC-ext@sun.com, Chris.Horne@sun.com Message-id: <200803281658.m2SGwKRQ014256@zion.sfbay.sun.com> MIME-version: 1.0 X-Mailer: ELM [version 2.4ME+ PL31H (25)] Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 Status: RO Content-Length: 255 FYI, This case was approved back on 3/12 and the case file was not updated to reflect that: I have corrected this. (see http://sac.sfbay/Archives/Minutes/PSARC/2008/20080312.html) -Mike -- Mike Shapiro, Sun Microsystems Fishworks. blogs.sun.com/mws/