From sacadmin Fri Dec 19 14:37:45 2008 Received: from sac.sfbay.sun.com (localhost [127.0.0.1]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mBJMbjGu018177; Fri, 19 Dec 2008 14:37:45 -0800 (PST) Received: (from carlsonj@localhost) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id mBJMbjJk018168; Fri, 19 Dec 2008 14:37:45 -0800 (PST) Date: Fri, 19 Dec 2008 14:37:45 -0800 (PST) From: James Carlson Message-Id: <200812192237.mBJMbjJk018168@sac.sfbay.sun.com> To: PSARC-record@sac.sfbay.sun.com Subject: Sendfilev Behavior Change [PSARC/2008/784 FastTrack timeout 01/02/2009] Status: RO Content-Length: 565 Template Version: @(#)sac_nextcase %I% %G% SMI This information is Copyright 2008 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: Sendfilev Behavior Change 1.2. Name of Document Author/Supplier: Author: Vasumathi Sundaram 1.3 Date of This Document: 19 December, 2008 4. Technical Description See the case directory for more detail 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: OSNet 6.5. ARC review type: FastTrack 6.6. ARC Exposure: open From carlsonj@phorcys.east.sun.com Fri Dec 19 14:53:00 2008 Received: from phorcys.east.sun.com (phorcys.East.Sun.COM [129.148.174.143]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mBJMr0lq018601 for ; Fri, 19 Dec 2008 14:53:00 -0800 (PST) Received: from phorcys.east.sun.com (localhost [127.0.0.1]) by phorcys.east.sun.com (8.14.3+Sun/8.14.3) with ESMTP id mBJMqxbY012663; Fri, 19 Dec 2008 17:52:59 -0500 (EST) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.3+Sun/8.14.3/Submit) id mBJMqxBD012660; Fri, 19 Dec 2008 17:52:59 -0500 (EST) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <18764.9675.776772.804587@gargle.gargle.HOWL> Date: Fri, 19 Dec 2008 17:52:59 -0500 From: James Carlson To: psarc-ext@sac.sfbay.sun.com cc: Vasumathi.Sundaram@sun.com Subject: 2008/784 Sendfilev Behavior Change X-Mailer: VM 7.01 under Emacs 21.3.1 Status: RO Content-Length: 2867 I am sponsoring this fast-track request for Vasumathi Sundaram. The timer is set to 01/02/2009 in light of the coming holidays. Please speak up if you need more time than that. In an offline discussion, the project team went to lengths to find any sendfilev users, and determine whether the proposed change in semantics would affect any application. There are just a few known users of sendfilev, and none of them do file-to-file transfers, so no problems are known or expected. It's an incompatible change to a Committed interface and we wish to assert Patch/Micro release binding, but the project team contends (and I agree) that (a) the original interface as specified has unresolvable defects that could lead to serious system stability problems, (b) we can't just remove the sendfilev interface because there are consumers, and (c) it appears unlikely that anyone ever used the functionality affected by this change as described. 1. Problem sendfilev(3EXT) and sendfile(2) support transferring data from a regular file to another regular file, in addition to being able to transfer to a socket. The current man page states that the semantics of this function is analogous to writev(2), except that sendfilev can read from data buffers or file descriptors. Supporting this semantic, however, leads to unavoidable deadlock and performance issues. 2. Solution sendfilev is modified so that the output behavior is not the same as writev() and when multiple threads attempt to write to the same output file at the same time. Instead, the result of the combined operations is undefined. The applications that use sendfilev() will have to use their own synchronization mechanism to ensure a consistent output. 3. Man page update The following change will be made to the man page of sendfilev(3EXT). --- sendfilev.old Thu Dec 18 15:41:16 2008 +++ sendfilev.new Thu Dec 18 15:45:40 2008 @@ -44,12 +44,6 @@ open for writing. - This function is analogous to the writev() system call. See - writev(2). Instead of sending out chunks of data, however, - the sendfilev() function can read input data from data - buffers or file descriptors. + This function is analogous to the writev(2) system call, but can + read from both buffers and file descriptors. Unlike writev(), + it does not provide protection against multiple writers on a + single stream or file. Applications must use their own + synchronization mechanisms if such a situation is possible. The following is the sendfilevec structure: typedef struct sendfilevec { -- James Carlson, Solaris Networking Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 From Brian.Utterback@Sun.COM Fri Dec 19 15:11:48 2008 Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01.SFBay.Sun.COM [129.145.155.118]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mBJNBmkt019071 for ; Fri, 19 Dec 2008 15:11:48 -0800 (PST) Received: from brmea-mail-1.sun.com (brmea-mail-1.Sun.COM [192.18.98.31]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id mBJNBmA8020598 for ; Fri, 19 Dec 2008 15:11:48 -0800 (PST) Received: from fe-amer-09.sun.com ([192.18.109.79]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id mBJNBl7R028110 for ; Fri, 19 Dec 2008 23:11:47 GMT Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0KC500201BWNDJ00@mail-amer.sun.com> (original mail from Brian.Utterback@Sun.COM) for psarc-ext@sac.sfbay.sun.com; Fri, 19 Dec 2008 16:11:47 -0700 (MST) Received: from [192.168.1.4] ([71.168.64.209]) by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0KC500CTPCFDIDE0@mail-amer.sun.com> for psarc-ext@sac.sfbay.sun.com; Fri, 19 Dec 2008 16:11:37 -0700 (MST) Date: Fri, 19 Dec 2008 18:11:36 -0500 From: Brian Utterback Subject: Re: 2008/784 Sendfilev Behavior Change In-reply-to: <18764.9675.776772.804587@gargle.gargle.HOWL> Sender: Brian.Utterback@Sun.COM To: James Carlson Cc: psarc-ext@sac.sfbay.sun.com, Vasumathi.Sundaram@Sun.COM Message-id: <494C2A28.5030905@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <18764.9675.776772.804587@gargle.gargle.HOWL> User-Agent: Thunderbird 2.0.0.18 (Windows/20081105) Status: RO Content-Length: 2966 Is this an actual change? It looks to me that it is only a documentation change. Or is it the case that the intention is to remove some existing locks since they don't work anyway? James Carlson wrote: > I am sponsoring this fast-track request for Vasumathi Sundaram. The > timer is set to 01/02/2009 in light of the coming holidays. Please > speak up if you need more time than that. > > In an offline discussion, the project team went to lengths to find any > sendfilev users, and determine whether the proposed change in > semantics would affect any application. There are just a few known > users of sendfilev, and none of them do file-to-file transfers, so no > problems are known or expected. > > It's an incompatible change to a Committed interface and we wish to > assert Patch/Micro release binding, but the project team contends (and > I agree) that (a) the original interface as specified has unresolvable > defects that could lead to serious system stability problems, (b) we > can't just remove the sendfilev interface because there are consumers, > and (c) it appears unlikely that anyone ever used the functionality > affected by this change as described. > > > 1. Problem > > sendfilev(3EXT) and sendfile(2) support transferring data from a > regular file to another regular file, in addition to being able to > transfer to a socket. The current man page states that the > semantics of this function is analogous to writev(2), except that > sendfilev can read from data buffers or file descriptors. > > Supporting this semantic, however, leads to unavoidable deadlock and > performance issues. > > 2. Solution > > sendfilev is modified so that the output behavior is not the same as > writev() and when multiple threads attempt to write to the same > output file at the same time. Instead, the result of the combined > operations is undefined. The applications that use sendfilev() will > have to use their own synchronization mechanism to ensure a > consistent output. > > 3. Man page update > > The following change will be made to the man page of sendfilev(3EXT). > > --- sendfilev.old Thu Dec 18 15:41:16 2008 > +++ sendfilev.new Thu Dec 18 15:45:40 2008 > @@ -44,12 +44,6 @@ > open for writing. > > > - This function is analogous to the writev() system call. See > - writev(2). Instead of sending out chunks of data, however, > - the sendfilev() function can read input data from data > - buffers or file descriptors. > + This function is analogous to the writev(2) system call, but can > + read from both buffers and file descriptors. Unlike writev(), > + it does not provide protection against multiple writers on a > + single stream or file. Applications must use their own > + synchronization mechanisms if such a situation is possible. > > The following is the sendfilevec structure: > > typedef struct sendfilevec { > > From roland.mainz@nrubsig.org Fri Dec 19 15:20:57 2008 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mBJNKvk6019149 for ; Fri, 19 Dec 2008 15:20:57 -0800 (PST) Received: from sca-ea-mail-1.sun.com (sca-ea-mail-1.Sun.COM [192.18.43.24]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id mBJNKvwE020570 for ; Fri, 19 Dec 2008 15:20:57 -0800 (PST) Received: from relay14i.sun.com (ip124.net129179-4.block1.us.syntegra.com [129.179.4.124]) by sca-ea-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id mBJNKp9b020941 for ; Fri, 19 Dec 2008 23:20:52 GMT Received: from mmp11es.mmp.us.syntegra.com ([160.41.208.11] [160.41.208.11]) by relay14i.sun.com with ESMTP id BT-MMP-987062 for psarc-ext@sac.sfbay.sun.com; Fri, 19 Dec 2008 23:20:51 Z Received: from relay14i.sun.com (relay14i.sun.com [129.179.4.124]) by mmp11es.mmp.us.syntegra.com with ESMTP id BT-MMP-29238608 for psarc-ext@sac.sfbay.sun.com; Fri, 19 Dec 2008 23:20:51 Z Received: from mail-in-03.arcor-online.net ([151.189.21.43] [151.189.21.43]) by relay1i.sun.com with ESMTP id BT-MMP-8561099 for psarc-ext@sac.sfbay.sun.com; Fri, 19 Dec 2008 23:20:51 Z Received: from mail-in-05-z2.arcor-online.net (mail-in-05-z2.arcor-online.net [151.189.8.17]) by mail-in-03.arcor-online.net (Postfix) with ESMTP id 592CB2CAEBA; Sat, 20 Dec 2008 00:20:50 +0100 (CET) Received: from mail-in-10.arcor-online.net (mail-in-10.arcor-online.net [151.189.21.50]) by mail-in-05-z2.arcor-online.net (Postfix) with ESMTP id 3BFBA2DAA64; Sat, 20 Dec 2008 00:20:50 +0100 (CET) Received: from jupiterb48.nrubsig.org (dslb-088-068-032-006.pools.arcor-ip.net [88.68.32.6]) by mail-in-10.arcor-online.net (Postfix) with ESMTP id CBB392351A8; Sat, 20 Dec 2008 00:20:49 +0100 (CET) Received: from nrubsig.org (localhost [127.0.0.1]) by jupiterb48.nrubsig.org (8.13.8+Sun/8.13.8) with ESMTP id mBJNKlEP009088; Sat, 20 Dec 2008 00:20:48 +0100 (CET) Sender: gisburn@jupiterb48.nrubsig.org Message-ID: <494C2C4F.A493B7C1@nrubsig.org> Date: Sat, 20 Dec 2008 00:20:47 +0100 From: Roland Mainz X-Brightmail-Tracker: AAAAAA== X-Mailer: Mozilla 4.76 [en] (X11; U; SunOS 5.11 sun4u) X-Accept-Language: en To: James Carlson CC: psarc-ext@sac.sfbay.sun.com, Vasumathi.Sundaram@sun.com Subject: Re: 2008/784 Sendfilev Behavior Change References: <18764.9675.776772.804587@gargle.gargle.HOWL> X-Virus-Scanned: ClamAV 0.94.1/8786/Fri Dec 19 11:30:31 2008 on mail-in-10.arcor-online.net X-Virus-Status: Clean X-Antispam: No, score=0.0/5.0, scanned in 0.062sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Status: RO Content-Length: 1364 James Carlson wrote: > I am sponsoring this fast-track request for Vasumathi Sundaram. The > timer is set to 01/02/2009 in light of the coming holidays. Please > speak up if you need more time than that. > > In an offline discussion, the project team went to lengths to find any > sendfilev users, Erm... ksh93 had support for |sendfile()|/|sendfilev()| support but it was breaking UDP networking support (and there was no reliable way to check this) and we removed this capability again (it gave a good performance improvement for sending larger files but propper cross-platform support is more important in this case). > 2. Solution > > sendfilev is modified so that the output behavior is not the same as > writev() and when multiple threads attempt to write to the same > output file at the same time. Instead, the result of the combined > operations is undefined. The applications that use sendfilev() will > have to use their own synchronization mechanism to ensure a > consistent output. ... which effectively means: A shell with thread support or any other language environment (e.g. java) shouldn't use |sendfilev()| when files are involed, right ? ---- Bye, Roland -- __ . . __ (o.\ \/ /.o) roland.mainz@nrubsig.org \__\/\/__/ MPEG specialist, C&&JAVA&&Sun&&Unix programmer /O /==\ O\ TEL +49 641 3992797 (;O/ \/ \O;) From bart.smaalders@Sun.COM Fri Dec 19 15:53:41 2008 Received: from zion.sfbay.sun.com (zion.SFBay.Sun.COM [129.146.17.75]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mBJNrfqa019444 for ; Fri, 19 Dec 2008 15:53:41 -0800 (PST) Received: from [129.146.228.109] (cyber.SFBay.Sun.COM [129.146.228.109]) by zion.sfbay.sun.com (8.14.2+Sun/8.14.2) with ESMTP id mBJNkBvQ012170; Fri, 19 Dec 2008 23:46:11 GMT Message-ID: <494C3242.7050707@Sun.COM> Date: Fri, 19 Dec 2008 15:46:10 -0800 From: Bart Smaalders Organization: Sun Microsystems User-Agent: Thunderbird 2.0.0.17 (X11/20081023) MIME-Version: 1.0 To: Roland Mainz CC: James Carlson , psarc-ext@sac.sfbay.sun.com, Vasumathi.Sundaram@Sun.COM Subject: Re: 2008/784 Sendfilev Behavior Change References: <18764.9675.776772.804587@gargle.gargle.HOWL> <494C2C4F.A493B7C1@nrubsig.org> In-Reply-To: <494C2C4F.A493B7C1@nrubsig.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Status: RO Content-Length: 610 Roland Mainz wrote: > ... which effectively means: A shell with thread support or any other > language environment (e.g. java) shouldn't use |sendfilev()| when files > are involed, right ? Isn't this just a relaxation of the atomicity level? In other words, the current definition appears to imply that the sendfilev write operation is atomic; this proposal explicitly (perhaps not explicitly enough :-)) relaxes this constraint. - Bart -- Bart Smaalders Solaris Kernel Performance barts@cyber.eng.sun.com http://blogs.sun.com/barts "You will contribute more with mercurial than with thunderbird." From Vasumathi.Sundaram@Sun.COM Fri Dec 19 16:19:39 2008 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mBK0JdL6006294 for ; Fri, 19 Dec 2008 16:19:39 -0800 (PST) Received: from sca-es-mail-2.sun.com (sca-es-mail-2.Sun.COM [192.18.43.133]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id mBK0JdQ3054641 for ; Fri, 19 Dec 2008 16:19:39 -0800 (PST) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id mBK0JYRQ005363 for ; Fri, 19 Dec 2008 16:19:34 -0800 (PST) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0KC500201FIOZK00@fe-sfbay-09.sun.com> (original mail from Vasumathi.Sundaram@Sun.COM) for psarc-ext@sac.sfbay.sun.com; Fri, 19 Dec 2008 16:19:34 -0800 (PST) Received: from mathi.local ([209.6.212.215]) by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0KC500AD5FKDMLB0@fe-sfbay-09.sun.com> for psarc-ext@sac.sfbay.sun.com; Fri, 19 Dec 2008 16:19:27 -0800 (PST) Date: Fri, 19 Dec 2008 19:19:13 -0500 From: Vasumathi Sundaram Subject: Re: 2008/784 Sendfilev Behavior Change In-reply-to: <494C2A28.5030905@sun.com> Sender: Vasumathi.Sundaram@Sun.COM To: Brian Utterback Cc: James Carlson , psarc-ext@sac.sfbay.sun.com Message-id: <494C3A01.9040404@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <18764.9675.776772.804587@gargle.gargle.HOWL> <494C2A28.5030905@sun.com> User-Agent: Thunderbird 2.0.0.18 (Macintosh/20081105) Status: RO Content-Length: 3258 Yes, this is an actual change. We want to remove an existing lock on the destination descriptor that is now held for the duration of the system call. This is the reason for deadlock issues. Brian Utterback wrote: > Is this an actual change? It looks to me that it is only a > documentation change. Or is it the case that the intention is to > remove some existing locks since they don't work anyway? > > > James Carlson wrote: >> I am sponsoring this fast-track request for Vasumathi Sundaram. The >> timer is set to 01/02/2009 in light of the coming holidays. Please >> speak up if you need more time than that. >> >> In an offline discussion, the project team went to lengths to find any >> sendfilev users, and determine whether the proposed change in >> semantics would affect any application. There are just a few known >> users of sendfilev, and none of them do file-to-file transfers, so no >> problems are known or expected. >> >> It's an incompatible change to a Committed interface and we wish to >> assert Patch/Micro release binding, but the project team contends (and >> I agree) that (a) the original interface as specified has unresolvable >> defects that could lead to serious system stability problems, (b) we >> can't just remove the sendfilev interface because there are consumers, >> and (c) it appears unlikely that anyone ever used the functionality >> affected by this change as described. >> >> >> 1. Problem >> >> sendfilev(3EXT) and sendfile(2) support transferring data from a >> regular file to another regular file, in addition to being able to >> transfer to a socket. The current man page states that the >> semantics of this function is analogous to writev(2), except that >> sendfilev can read from data buffers or file descriptors. >> >> Supporting this semantic, however, leads to unavoidable deadlock and >> performance issues. >> >> 2. Solution >> >> sendfilev is modified so that the output behavior is not the same as >> writev() and when multiple threads attempt to write to the same >> output file at the same time. Instead, the result of the combined >> operations is undefined. The applications that use sendfilev() will >> have to use their own synchronization mechanism to ensure a >> consistent output. >> >> 3. Man page update >> >> The following change will be made to the man page of sendfilev(3EXT). >> >> --- sendfilev.old Thu Dec 18 15:41:16 2008 >> +++ sendfilev.new Thu Dec 18 15:45:40 2008 >> @@ -44,12 +44,6 @@ >> open for writing. >> >> >> - This function is analogous to the writev() system call. See >> - writev(2). Instead of sending out chunks of data, however, >> - the sendfilev() function can read input data from data >> - buffers or file descriptors. >> + This function is analogous to the writev(2) system call, but can >> + read from both buffers and file descriptors. Unlike writev(), >> + it does not provide protection against multiple writers on a >> + single stream or file. Applications must use their own >> + synchronization mechanisms if such a situation is possible. >> >> The following is the sendfilevec structure: >> >> typedef struct sendfilevec { >> >> > From Vasumathi.Sundaram@Sun.COM Fri Dec 19 16:22:17 2008 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mBK0MHgI006332 for ; Fri, 19 Dec 2008 16:22:17 -0800 (PST) Received: from sca-es-mail-1.sun.com (sca-es-mail-1.Sun.COM [192.18.43.132]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id mBK0MH54056066 for ; Fri, 19 Dec 2008 16:22:17 -0800 (PST) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id mBK0MCgm018495 for ; Fri, 19 Dec 2008 16:22:12 -0800 (PST) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0KC500401FNZL200@fe-sfbay-10.sun.com> (original mail from Vasumathi.Sundaram@Sun.COM) for psarc-ext@sac.sfbay.sun.com; Fri, 19 Dec 2008 16:22:12 -0800 (PST) Received: from mathi.local ([209.6.212.215]) by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0KC500BQWFOW3M20@fe-sfbay-10.sun.com> for psarc-ext@sac.sfbay.sun.com; Fri, 19 Dec 2008 16:22:09 -0800 (PST) Date: Fri, 19 Dec 2008 19:21:43 -0500 From: Vasumathi Sundaram Subject: Re: 2008/784 Sendfilev Behavior Change In-reply-to: <494C2C4F.A493B7C1@nrubsig.org> Sender: Vasumathi.Sundaram@Sun.COM To: Roland Mainz Cc: James Carlson , psarc-ext@sac.sfbay.sun.com Message-id: <494C3A97.90309@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <18764.9675.776772.804587@gargle.gargle.HOWL> <494C2C4F.A493B7C1@nrubsig.org> User-Agent: Thunderbird 2.0.0.18 (Macintosh/20081105) Status: RO Content-Length: 1465 Roland Mainz wrote: > James Carlson wrote: > >> I am sponsoring this fast-track request for Vasumathi Sundaram. The >> timer is set to 01/02/2009 in light of the coming holidays. Please >> speak up if you need more time than that. >> >> In an offline discussion, the project team went to lengths to find any >> sendfilev users, >> > > Erm... ksh93 had support for |sendfile()|/|sendfilev()| support but it > was breaking UDP networking support (and there was no reliable way to > check this) and we removed this capability again (it gave a good > performance improvement for sending larger files but propper > cross-platform support is more important in this case). > > >> 2. Solution >> >> sendfilev is modified so that the output behavior is not the same as >> writev() and when multiple threads attempt to write to the same >> output file at the same time. Instead, the result of the combined >> operations is undefined. The applications that use sendfilev() will >> have to use their own synchronization mechanism to ensure a >> consistent output. >> > > ... which effectively means: A shell with thread support or any other > language environment (e.g. java) shouldn't use |sendfilev()| when files > are involed, right ? > sendfilev() can be used when files are involved. But we cannot expect a consistent output in the destination file, because another thread could write to the same file. > ---- > > Bye, > Roland > > From bart.smaalders@Sun.COM Fri Dec 19 16:30:39 2008 Received: from zion.sfbay.sun.com (zion.SFBay.Sun.COM [129.146.17.75]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mBK0Ud0a006353 for ; Fri, 19 Dec 2008 16:30:39 -0800 (PST) Received: from [129.146.228.109] (cyber.SFBay.Sun.COM [129.146.228.109]) by zion.sfbay.sun.com (8.14.2+Sun/8.14.2) with ESMTP id mBK0UdDC012995; Sat, 20 Dec 2008 00:30:39 GMT Message-ID: <494C3CAF.9@Sun.COM> Date: Fri, 19 Dec 2008 16:30:39 -0800 From: Bart Smaalders Organization: Sun Microsystems User-Agent: Thunderbird 2.0.0.17 (X11/20081023) MIME-Version: 1.0 To: Vasumathi Sundaram CC: Brian Utterback , James Carlson , psarc-ext@sac.sfbay.sun.com Subject: Re: 2008/784 Sendfilev Behavior Change References: <18764.9675.776772.804587@gargle.gargle.HOWL> <494C2A28.5030905@sun.com> <494C3A01.9040404@sun.com> In-Reply-To: <494C3A01.9040404@sun.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Status: RO Content-Length: 1454 Vasumathi Sundaram wrote: > Yes, this is an actual change. We want to remove an existing lock on the > destination descriptor that is now held for the duration of the system > call. This is the reason for deadlock issues. It would be useful to be more explicit about exactly what is different: - This function is analogous to the writev() system call. See - writev(2). Instead of sending out chunks of data, however, - the sendfilev() function can read input data from data - buffers or file descriptors. + This function is analogous to the writev(2) system call, but can + read from both buffers and file descriptors. Unlike writev(), + it does not provide protection against multiple writers on a + single stream or file. Applications must use their own + synchronization mechanisms if such a situation is possible. That last paragraph might better read: This function is analogous to the writev(2) system call, but can read from both buffers and file descriptors. Unlike writev(), in the case of multiple writers to a file the effect of sendfilev is not necessarily atomic; the writes may be interleaved. Application-specific synchronization methods must be employed if this causes problems. -- Bart Smaalders Solaris Kernel Performance barts@cyber.eng.sun.com http://blogs.sun.com/barts "You will contribute more with mercurial than with thunderbird." From carlsonj@phorcys.east.sun.com Mon Dec 22 03:45:49 2008 Received: from phorcys.east.sun.com (phorcys.East.Sun.COM [129.148.174.143]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mBMBjnYR002819 for ; Mon, 22 Dec 2008 03:45:49 -0800 (PST) Received: from phorcys.east.sun.com (localhost [127.0.0.1]) by phorcys.east.sun.com (8.14.3+Sun/8.14.3) with ESMTP id mBMBjmxn014477; Mon, 22 Dec 2008 06:45:48 -0500 (EST) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.3+Sun/8.14.3/Submit) id mBMBjmZo014474; Mon, 22 Dec 2008 06:45:48 -0500 (EST) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <18767.32236.854696.301301@gargle.gargle.HOWL> Date: Mon, 22 Dec 2008 06:45:48 -0500 From: James Carlson To: Bart Smaalders Cc: Vasumathi Sundaram , Brian Utterback , psarc-ext@sac.sfbay.sun.com Subject: Re: 2008/784 Sendfilev Behavior Change In-Reply-To: <494C3CAF.9@Sun.COM> References: <18764.9675.776772.804587@gargle.gargle.HOWL> <494C2A28.5030905@sun.com> <494C3A01.9040404@sun.com> <494C3CAF.9@Sun.COM> X-Mailer: VM 7.01 under Emacs 21.3.1 Status: RO Content-Length: 955 Bart Smaalders writes: > It would be useful to be more explicit about exactly what is different: I apologize for the new text. That was mine. (The original case provided _no_ description in the man page and just deleted the offending 'writev' paragraph, though the text in the case itself said that text would be provided.) > This function is analogous to the writev(2) system call, but can > read from both buffers and file descriptors. Unlike writev(), > in the case of multiple writers to a file the effect of > sendfilev is not necessarily atomic; the writes may be > interleaved. Application-specific synchronization methods > must be employed if this causes problems. That's better. -- James Carlson, Solaris Networking Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 From carlsonj@phorcys.east.sun.com Mon Dec 22 04:07:25 2008 Received: from phorcys.east.sun.com (phorcys.East.Sun.COM [129.148.174.143]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mBMC7Ogl004296 for ; Mon, 22 Dec 2008 04:07:25 -0800 (PST) Received: from phorcys.east.sun.com (localhost [127.0.0.1]) by phorcys.east.sun.com (8.14.3+Sun/8.14.3) with ESMTP id mBMBxsbL014544; Mon, 22 Dec 2008 06:59:54 -0500 (EST) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.3+Sun/8.14.3/Submit) id mBMBxsJq014541; Mon, 22 Dec 2008 06:59:54 -0500 (EST) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <18767.33082.599502.486863@gargle.gargle.HOWL> Date: Mon, 22 Dec 2008 06:59:54 -0500 From: James Carlson To: Roland Mainz Cc: psarc-ext@sac.sfbay.sun.com, Vasumathi.Sundaram@sun.com Subject: Re: 2008/784 Sendfilev Behavior Change In-Reply-To: <494C2C4F.A493B7C1@nrubsig.org> References: <18764.9675.776772.804587@gargle.gargle.HOWL> <494C2C4F.A493B7C1@nrubsig.org> X-Mailer: VM 7.01 under Emacs 21.3.1 Status: RO Content-Length: 2103 Roland Mainz writes: > James Carlson wrote: > > I am sponsoring this fast-track request for Vasumathi Sundaram. The > > timer is set to 01/02/2009 in light of the coming holidays. Please > > speak up if you need more time than that. > > > > In an offline discussion, the project team went to lengths to find any > > sendfilev users, > > Erm... ksh93 had support for |sendfile()|/|sendfilev()| support but it > was breaking UDP networking support (and there was no reliable way to > check this) and we removed this capability again (it gave a good > performance improvement for sending larger files but propper > cross-platform support is more important in this case). I don't know what "breaking UDP" means (I suspect it's an application defect), but the usual expectation is that sendfile() and sendfilev() are used with stream-oriented communication -- such as TCP sockets. If someone used those functions on a UDP socket, I'd expect that's an error in the application, given that the man page says SOCK_STREAM. If the question you're asking is "how can I tell whether a file descriptor is a stream-type socket that can be used with sendfile or sendfilev?" then the answer is to use getsockopt with SOL_SOCKET, SO_TYPE and check for SOCK_STREAM. In any event, I don't think the question you're asking here is related to the case under review. I suggest bringing it up (with more details) on networking-discuss. > ... which effectively means: A shell with thread support or any other > language environment (e.g. java) shouldn't use |sendfilev()| when files > are involed, right ? No, it doesn't mean that. All programs on Solaris are effectively multithreaded. It means that you have to provide protection yourself if you use it and if there are multiple writers. Most (all?) applications that would use this have only a single writer, so the whole point is moot. -- James Carlson, Solaris Networking Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 From carlsonj@phorcys.east.sun.com Wed Jan 7 10:20:53 2009 Received: from phorcys.east.sun.com (phorcys.East.Sun.COM [129.148.174.143]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n07IKqb0015900 for ; Wed, 7 Jan 2009 10:20:53 -0800 (PST) Received: from phorcys.east.sun.com (localhost [127.0.0.1]) by phorcys.east.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n07IKqkY011517; Wed, 7 Jan 2009 13:20:52 -0500 (EST) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.3+Sun/8.14.3/Submit) id n07IKq7L011514; Wed, 7 Jan 2009 13:20:52 -0500 (EST) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <18788.62084.575653.140132@gargle.gargle.HOWL> Date: Wed, 7 Jan 2009 13:20:52 -0500 From: James Carlson To: psarc-ext@sac.sfbay.sun.com cc: Vasumathi.Sundaram@sun.com Subject: Re: 2008/784 Sendfilev Behavior Change In-Reply-To: <18764.9675.776772.804587@gargle.gargle.HOWL> References: <18764.9675.776772.804587@gargle.gargle.HOWL> X-Mailer: VM 7.01 under Emacs 21.3.1 Status: RO Content-Length: 653 James Carlson writes: > I am sponsoring this fast-track request for Vasumathi Sundaram. The > timer is set to 01/02/2009 in light of the coming holidays. Please > speak up if you need more time than that. This fast-track request was approved during ARC business today, with a +1 from Glenn Skinner. I have placed a copy of the current specification (with man page details amended by Bart's suggestion) in the case directory. -- James Carlson, Solaris Networking Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677