From Wed Sep 30 10:34:15 2009 Date: Wed, 30 Sep 2009 10:34:15 -0700 (PDT) From: Postmaster Subject: Message from mail server Content-Length: 94 Mime-Version: 1.0 Status: RO X-IMAP: 1251919004 79 Delete. This is a system message. --END+PSEUDO-- From psarc-member-list-request@sun.com Fri Jul 10 09:43:36 2009 Return-Path: Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02 [129.146.11.31]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6AGhZnS012768 for ; Fri, 10 Jul 2009 09:43:35 -0700 (PDT) Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6AGijAo042093; Fri, 10 Jul 2009 09:44:45 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n6AGhEvh025482; Sat, 11 Jul 2009 00:44:33 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMK00F01RU4Z100@nwk-avmta-1.sfbay.Sun.COM> (ORCPT PSARC-ext@sun.com) ; Fri, 10 Jul 2009 09:44:28 -0700 (PDT) Received: from dm-sfbay-01.sfbay.sun.com ([129.145.155.118]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMK00D47RU4SE10@nwk-avmta-1.sfbay.Sun.COM> (ORCPT PSARC-ext@sun.com); Fri, 10 Jul 2009 09:44:28 -0700 (PDT) Received: from sac.sfbay.sun.com (sac.SFBay.Sun.COM [129.146.226.132]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6AGiRKc062062; Fri, 10 Jul 2009 09:44:28 -0700 (PDT) Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6AGh9GM007357 for ; Fri, 10 Jul 2009 09:43:09 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6AGh2xH010298; Fri, 10 Jul 2009 17:43:08 +0100 (BST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMK0010PRRW9A00@brm-avmta-1.central.sun.com>; Fri, 10 Jul 2009 10:43:08 -0600 (MDT) Received: from dm-sfbay-02.sfbay.sun.com ([129.146.11.31]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMK00901RRVVHB0@brm-avmta-1.central.sun.com>; Fri, 10 Jul 2009 10:43:07 -0600 (MDT) Received: from ivrel.sfbay.sun.com (ivrel.SFBay.Sun.COM [129.146.74.76]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6AGh6tn040824; Fri, 10 Jul 2009 09:43:06 -0700 (PDT) Received: from ivrel (ivrel [129.146.74.76]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with SMTP id n6AGYQNl012755; Fri, 10 Jul 2009 09:34:26 -0700 (PDT) Date: Fri, 10 Jul 2009 09:34:26 -0700 (PDT) From: Glenn Skinner Subject: 2009/387 [Pathname Reparse Points] To: PSARC-ext@sun.com Cc: cifs-eng@sun.com, Robert.Thurlow@sun.com, Brian.Wong@sun.com, Afshin.Ardakani@sun.com Reply-to: Glenn Skinner Message-id: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> MIME-version: 1.0 X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.6_36 SunOS 5.11 sun4u sparc Content-type: TEXT/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-MD5: 59r8KipOPG8x4R16V8aGTw== X-PMX-Version: 5.4.1.325704 X-PMX-Version: 5.4.1.325704 Content-Length: 12678 Status: RO X-Status: $$$$ X-UID: 0000000001 I'm sponsoring the following fast track for Afshin Salek and the CIFS i-team. It times out on Friday, July 17th. A copy of the specification below appears in the case directory under the name "specification". I've pre-reviewed it and will give it a +1 up front. -- Glenn ---------------- Template Version: @(#)onepager.txt 1.35 07/11/07 SMI Copyright 2007 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: Support for Reparse Points 1.2. Name of Document Author/Supplier: Author: Afshin Salek 1.3. Date of This Document: 07/08/09 1.4. Name of Major Document Customer(s)/Consumer(s): PSARC CIFS team 1.5. Email Aliases: 1.5.1. Responsible Manager: Barry.Greenberg@Sun.COM 1.5.2. Responsible Engineer: Afshin.Ardakani@Sun.COM 1.5.3. Marketing Manager: 1.5.4. Interest List: cifs-team@sun.com A patch binding is requested for this change. 4. Technical Description: 4.1. Details: INTRODUCTION There are situations where a mechanism is needed to reflect the concept that data is not present at a particular path, but can be found in some alternate location(s). Examples include "referrals" used to build unified name spaces in NFSv4.x and SMB, and data relocation in HSM systems. A "reparse point" is defined as the marker for a namespace redirection and a container for the metadata to specify where the target of this redirection is. Reparse points are intended to be a general mechanism for location redirection and as such the file system that contains them is not cognizant of the reparse point format or content. Services that use reparse points know how to interpret and use the stored data. REPARSE POINT OBJECT After a lot of discussion the consensus is that the best way to represent reparse points in the file system, in order to minimize the effect on existing applications and utilities, to use symbolic links. One of the main goals in this context has been the ability to use existing utilities for backup/restore and also ZFS send/receive without having to modify them to know how to deal with reparse points. Some of what is envisioned here could be done with extensions to the Solaris automounter capability. Part of the motivation, though, is to create centrally-administrated namespaces served by a group of fileservers to near-zero-admin clients. It is expected to be easier to keep the namespaces uniform if only a small number of servers need to participate. HSM solutions would also normally be tied closely to a storage server by this mechanism. Also, for both NFS and SMB referrals, it is the client that chooses the target and not the server. The server only provides the targets' information and it is up to the client to pick the desirable target to access the data. To distinguish a regular symlink from a reparse point, an extensible system attribute will be set on the symlink. This system attribute is only one bit which indicates whether or not a symlink contains reparse data. The reparse data will be stored as the link target. The reparse data is not in file system path format, which is the typical format of a link target. In order to avoid coming up with a totaly new format for reparse data as the link target we decided to adopt the format used by magic links in BSD: (http://www.daemon-systems.org/man/symlink.7.html) @{REPARSE@{service-type1:data} [@{service-type2:data}]...} Where some examples of service-type are: #define REPARSE_SVC_SMB "SMB" #define REPARSE_SVC_NFS "NFS" #define REPARSE_SVC_HSM "HSM" The data for each service will be in string format, which is expected to be typically a UUID string. The pattern above starts with "REPARSE" to distinguish it from a other magic links, such as those supported by BSD. Note that this case is not a proposal to support BSD magic links, the intent is to avoid precluding the future addition of full BSD magic link support. Multiple services entries can co-exist within the symlink data. It is expected that normally, all entries would resolve to the same logical location, e.g. NFS and CIFS clients would find the same files. BASIC INTERFACES There is a need for both userspace and kernel APIs to work with reparse points. Userspace API In userspace the symlink(2) system call will be used to set a reparse point. The readlink(2) system call will be used in turn to read the reparse data. Kernel API In the kernel, VOP_SYMLINK and VOP_READLINK will be used to set/get reparse data. These interfaces will support all replication, archive and copy operations to preserve reparse points without further changes. fop_symlink() needs to be modified to recognize the reparse @{REPARSE} tag and pass the appropriate attribute (i.e. reparse system attribute) to VOP_SYMLINK to be set on the symlink. IMPLEMENTATION OBSERVATIONS VFS feature registration can be used to determine whether or not a file system supports reparse points. Two things are needed to obtain the reparse point data in the kernel. First, the consumer needs to know that a reparse point has been encountered and, second, it needs the vnode pointer to the symlink. The proposal is to enhance VOP_LOOKUP to return the attributes of the looked up vnode. This way when the vnode is available the caller can check the attributes to determine if the returned vnode is a reparse point or a regular symlink. Here are the old and revised signatures of VOP_LOOKUP: int VOP_LOOKUP(vnode_t *dvp, char *nm, vnode_t **vpp, pathname_t *pnp, int flags, vnode_t *rdir, cred_t *cr, caller_context_t *ct, int *deflags, pathname_t *ppnp) int VOP_LOOKUP(vnode_t *dvp, char *nm, vnode_t **vpp, pathname_t *pnp, int flags, vnode_t *rdir, cred_t *cr, caller_context_t *ct, int *deflags, pathname_t *ppnp, vattr_t *vap) A vattr_t pointer argument is added at the end to return the attributes if it is non-NULL. This is an optimization so that consumers don't have to invoke an extra VOP_GETATTR after lookup for obtaining the attributes. The symlink target size should be increased to 16K to accomodate the maximum size supported for MS-DFS referrals by Windows. Applications are expected to query the PATH_MAX and SYMLINK_MAX values on the local system using pathconf(2)/fpathconf(2). The value of SYMLINK_MAX would be changed to 16K on ZFS. The value of PATH_MAX will not be affected. To provide compatibility with other UNIXes (see section 6 below), sharemgr(1M) would be enhanced to support a "refer" option for NFS exports. This option would only result in creation of a reparse point at the specified path and does not actually share the path over NFS. This case is only about the underlying infrastructure and a future case will be presented to deal with details and specifics of handling referrals for NFSv4 server. SECURITY CONSIDERATIONS Referrals are similar to regular symbolic links in that they are only pointers to data that could be discovered in some other way. The presence of such a pointer does not compromise the security of the target object or data; the target service or file system must still enforce security. OPERATION FLOW Once a kernel service encounters a reparse point, it reads the data using VOP_READLINK and passes the data up to a user space daemon (e.g. reparsed) along with its desired record type. Depending on the requested record type the daemon could simply extract the information from the passed data and return it to kernel or do any other processing necessary to obtain the actual referral information e.g. in the case of FedFS, contacting NSDB. Going through a common user space daemon to get the referral data makes this process generic and easily expandable for possible future use cases. Referral extraction and creation by a userspace daemon can be handled via a library plugin architecture for different service types. Operation Flow Example Here is a simplified example of operation for a CIFS client that tries to access a file where the path contains a DFS link: a) Client tries to access \\srv\root\...\link\...\file.txt where: 'root' is a share (namespace root) 'link' is a reparse point seen as a folder by client b) CIFS server does a VOP_LOOKUP for 'link' when it is recognized as a reparse point by examining the attributes return by VOP_LOOKUP. At this point a STATUS_PATH_NOT_COVERED is returned to client c) Client sends a "link referral" request to the server. CIFS server uses VOP_READLINK to get the 'link' data and sends the data to 'reparsed' daemon via a door call and gets back the DFS link targets in a format understandable by the CIFS client. The targets are sent back to the client in response to its "link referral" request. b) Client picks one of the targets and contacts the target server to access 'file.txt' NFS REFERRAL IN OTHER UNIXES FS referrals have been implemented in other major UNIX distributions such as Linux, AIX and HP-UX but there is no unified approach or implementation. Linux, AIX and HP-UX specify referrals as an NFS export option. The option format is basically the same in all three operating systems (refer=path@host) but the presentation is somewhat different in each case: - In Linux a referral is presented as a mount point. - In HP-UX a referral is a file system partition or logical volume. - In AIX a special object is used to represent a referral. These are all mechanisms to trigger a change in namespace while resolving a path. This proposal is somewhat aligned with the AIX approach but does not require a new object type to be defined, which has the advantage of not impacting existing applications. As mentioned previously, an NFS "refer" option will be supported to provide option format compatibility. Additionally, the Solaris requirements include support for both NFS and SMB referrals whereas these other operating systems only support NFS referrals, and they do not provide native SMB support. For the Solaris operating system, this proposal provides a generic solution to support multiple, disparate referral mechanisms without placing restrictions on the format required by each mechanism. The following links provide a bit more details about each OS discussed above: http://www.citi.umich.edu/projects/nfsv4/linux/using-referrals.html http://nfsv4.bullopensource.org/doc/migration-and-replication-0.2.pdf http://docs.hp.com/en/5900-0306/ch01s11.html?jumpid=reg_R1002_USEN http://docs.hp.com/en/13578/nfsv4_whitepaper.pdf http://publib.boulder.ibm.com/infocenter/systems/index.jsp?topic=/com.ibm.aix.commadmn/doc/commadmndita/nfs_referrals.htm INTERFACE TABLE |Proposed |Specified | |Stability |in what | Interface Name |Classification |Document? | Comments =========================================================================== XAT_REPARSE |Consolidation |This |Reparse extensible |Private |Document |attribute | | | VOP_LOOKUP, fop_lookup |Contracted |This |Added new argument: |Consolidation |Document |vattr_t *vap |Private* | | | | | Reparse token syntax |Committed |This | |Private |Document | | | | SYMLINK_MAX |Committed |This |Increased to 16K | |Document | * The project's deliverables will all go into the OS/NET Consolidation, so no contracts are required. 6. Resources and Schedule: 6.4. Product Approval Committee requested information: 6.4.1. Consolidation or Component Name: ON 6.5. ARC review type: FastTrack From Darren.Moffat@sun.com Fri Jul 10 10:01:55 2009 Return-Path: Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02 [129.146.11.31]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6AH1tUx012799 for ; Fri, 10 Jul 2009 10:01:55 -0700 (PDT) Received: from gmp-eb-inf-2.sun.com (gmp-eb-inf-2.EU.Sun.COM [192.18.6.24]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6AH326B057056 for ; Fri, 10 Jul 2009 10:03:02 -0700 (PDT) Received: from fe-emea-09.sun.com (gmp-eb-lb-1-fe3.eu.sun.com [192.18.6.10]) by gmp-eb-inf-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n6AH2ukp017815; Fri, 10 Jul 2009 17:02:56 GMT MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII; format=flowed Received: from conversion-daemon.fe-emea-09.sun.com by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMK00C00SJJXI00@fe-emea-09.sun.com>; Fri, 10 Jul 2009 18:02:56 +0100 (BST) Received: from [129.156.173.199] ([unknown] [129.156.173.199]) by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMK00BAGSOV4820@fe-emea-09.sun.com>; Fri, 10 Jul 2009 18:02:56 +0100 (BST) Date: Fri, 10 Jul 2009 18:02:55 +0100 From: Darren J Moffat Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> Sender: Darren.Moffat@sun.com To: Glenn Skinner Cc: PSARC-ext@sun.com, cifs-eng@sun.com, Robert.Thurlow@sun.com, Brian.Wong@sun.com, Afshin.Ardakani@sun.com Message-id: <4A57743F.5070708@Sun.COM> References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> User-Agent: Thunderbird 2.0.0.18 (X11/20090127) Content-Length: 782 Status: RO X-Status: $$$$ X-UID: 0000000002 This looks very cool but I haven't quite got my head around it completely yet. What happens if open(2) is called with O_NOFOLLOW set on one of these reparse points ? (Please answer for ZFS local access, NFS and CIFS). > One of the main goals in this context has > been the ability to use existing utilities for backup/restore > and also ZFS send/receive without having to modify them to > know how to deal with reparse points. So why not just a system attribute to store the whole thing ? Particularly since it is required to store a system attribute to distinguish a reparse point from a normal symlink anyway. Also if we do end up adding BSD magic link support for the link types they have can a symlink link still have reparse data in it ? -- Darren J Moffat From gdamore@sun.com Fri Jul 10 10:02:04 2009 Return-Path: Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02 [129.146.11.31]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6AH24oX012803 for ; Fri, 10 Jul 2009 10:02:04 -0700 (PDT) Received: from sca-es-mail-2.sun.com (sca-es-mail-2.Sun.COM [192.18.43.133]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6AH3Fcb057254 for ; Fri, 10 Jul 2009 10:03:15 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n6AH39C3022374; Fri, 10 Jul 2009 10:03:10 -0700 (PDT) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII; format=flowed Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMK00700SLYWY00@fe-sfbay-09.sun.com>; Fri, 10 Jul 2009 10:03:09 -0700 (PDT) Received: from [192.168.251.11] ([unknown] [76.93.15.33]) by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMK0071USP7Z110@fe-sfbay-09.sun.com>; Fri, 10 Jul 2009 10:03:07 -0700 (PDT) Date: Fri, 10 Jul 2009 10:03:07 -0700 From: "Garrett D'Amore" Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> Sender: Garrett.Damore@sun.com To: Glenn Skinner Cc: PSARC-ext@sun.com, cifs-eng@sun.com, Robert.Thurlow@sun.com, Brian.Wong@sun.com, Afshin.Ardakani@sun.com Message-id: <4A57744B.1020700@sun.com> References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> User-Agent: Thunderbird 2.0.0.18 (X11/20081201) Content-Length: 14932 Status: RO X-Status: $$$$ X-UID: 0000000003 This sounds reasonable, and is not like the case of AFS which uses special tokens in symbolic links that can expand to other things. I'm a bit concerned about potential effects on applications, it *seems* like this is done in a manner that is safe, but there are a few items: * are applications consistent in their use of pathconf/fpathconf to get filesystem limits * presumably archivers and such are not expected to traverse these? (they get handled like an ordinary symbolic link) * what happens when the referral is archived and then reextracted? (is the attribute lost?) * as a nit, its not truly file system independent, since it relies on symbolic links (not all filesystems support symlinks, though admittedly the ones of interest to this case all do) I believe that this case likely exceeds the obviousness test for a fast track. I certainly wouldn't be comfortable having it go through with only a single +1 from another member (your own +1 doesn't count, as I understand the rules -- case owners don't count). Given this, I'm going to derail the case, just to force enough members to read it to get a meaningful vote. I'll write any resulting opinion. I don't think we need any additional materials apart from answers to the questions I've already raised. Note that I don't think there is anything intrinsically wrong with the case (though my archivers question above is I think a real potential concern) -- the derail here should not be taken as a negative statement about the case itself; I just want to make sure it is adequately and properly reviewed. Thanks. - Garrett Glenn Skinner wrote: > I'm sponsoring the following fast track for Afshin Salek and the CIFS > i-team. It times out on Friday, July 17th. > > A copy of the specification below appears in the case directory under > the name "specification". > > I've pre-reviewed it and will give it a +1 up front. > > -- Glenn > > ---------------- > > Template Version: @(#)onepager.txt 1.35 07/11/07 SMI > Copyright 2007 Sun Microsystems > > 1. Introduction > 1.1. Project/Component Working Name: > Support for Reparse Points > > 1.2. Name of Document Author/Supplier: > Author: Afshin Salek > > 1.3. Date of This Document: > 07/08/09 > > 1.4. Name of Major Document Customer(s)/Consumer(s): > PSARC > CIFS team > > 1.5. Email Aliases: > 1.5.1. Responsible Manager: Barry.Greenberg@Sun.COM > 1.5.2. Responsible Engineer: Afshin.Ardakani@Sun.COM > 1.5.3. Marketing Manager: > 1.5.4. Interest List: cifs-team@sun.com > > A patch binding is requested for this change. > > 4. Technical Description: > 4.1. Details: > > INTRODUCTION > > There are situations where a mechanism is needed to reflect > the concept that data is not present at a particular path, but > can be found in some alternate location(s). Examples include > "referrals" used to build unified name spaces in NFSv4.x and > SMB, and data relocation in HSM systems. A "reparse point" is > defined as the marker for a namespace redirection and a > container for the metadata to specify where the target of this > redirection is. > > Reparse points are intended to be a general mechanism for > location redirection and as such the file system that contains > them is not cognizant of the reparse point format or content. > Services that use reparse points know how to interpret and use > the stored data. > > REPARSE POINT OBJECT > > After a lot of discussion the consensus is that the best way > to represent reparse points in the file system, in order to > minimize the effect on existing applications and utilities, to > use symbolic links. One of the main goals in this context has > been the ability to use existing utilities for backup/restore > and also ZFS send/receive without having to modify them to > know how to deal with reparse points. > > Some of what is envisioned here could be done with extensions > to the Solaris automounter capability. Part of the > motivation, though, is to create centrally-administrated > namespaces served by a group of fileservers to near-zero-admin > clients. It is expected to be easier to keep the namespaces > uniform if only a small number of servers need to participate. > HSM solutions would also normally be tied closely to a storage > server by this mechanism. Also, for both NFS and SMB > referrals, it is the client that chooses the target and not > the server. The server only provides the targets' information > and it is up to the client to pick the desirable target to > access the data. > > To distinguish a regular symlink from a reparse point, an > extensible system attribute will be set on the symlink. This > system attribute is only one bit which indicates whether or > not a symlink contains reparse data. > > The reparse data will be stored as the link target. The > reparse data is not in file system path format, which is the > typical format of a link target. In order to avoid coming up > with a totaly new format for reparse data as the link target > we decided to adopt the format used by magic links in BSD: > (http://www.daemon-systems.org/man/symlink.7.html) > > @{REPARSE@{service-type1:data} [@{service-type2:data}]...} > > Where some examples of service-type are: > > #define REPARSE_SVC_SMB "SMB" > #define REPARSE_SVC_NFS "NFS" > #define REPARSE_SVC_HSM "HSM" > > The data for each service will be in string format, which is > expected to be typically a UUID string. > > The pattern above starts with "REPARSE" to distinguish it from > a other magic links, such as those supported by BSD. Note > that this case is not a proposal to support BSD magic links, > the intent is to avoid precluding the future addition of full > BSD magic link support. > > Multiple services entries can co-exist within the symlink > data. It is expected that normally, all entries would resolve > to the same logical location, e.g. NFS and CIFS clients would > find the same files. > > BASIC INTERFACES > > There is a need for both userspace and kernel APIs to work > with reparse points. > > Userspace API > > In userspace the symlink(2) system call will be used to set a > reparse point. The readlink(2) system call will be used in > turn to read the reparse data. > > Kernel API > > In the kernel, VOP_SYMLINK and VOP_READLINK will be used to > set/get reparse data. > > These interfaces will support all replication, archive and > copy operations to preserve reparse points without further > changes. > > fop_symlink() needs to be modified to recognize the reparse > @{REPARSE} tag and pass the appropriate attribute (i.e. > reparse system attribute) to VOP_SYMLINK to be set on the > symlink. > > IMPLEMENTATION OBSERVATIONS > > VFS feature registration can be used to determine whether or > not a file system supports reparse points. > > Two things are needed to obtain the reparse point data in the > kernel. First, the consumer needs to know that a reparse > point has been encountered and, second, it needs the vnode > pointer to the symlink. The proposal is to enhance VOP_LOOKUP > to return the attributes of the looked up vnode. This way > when the vnode is available the caller can check the > attributes to determine if the returned vnode is a reparse > point or a regular symlink. Here are the old and revised > signatures of VOP_LOOKUP: > > int VOP_LOOKUP(vnode_t *dvp, char *nm, vnode_t **vpp, > pathname_t *pnp, int flags, vnode_t *rdir, cred_t *cr, > caller_context_t *ct, int *deflags, pathname_t *ppnp) > > int VOP_LOOKUP(vnode_t *dvp, char *nm, vnode_t **vpp, > pathname_t *pnp, int flags, vnode_t *rdir, cred_t *cr, > caller_context_t *ct, int *deflags, pathname_t *ppnp, > vattr_t *vap) > > A vattr_t pointer argument is added at the end to return the > attributes if it is non-NULL. This is an optimization so that > consumers don't have to invoke an extra VOP_GETATTR after > lookup for obtaining the attributes. > > The symlink target size should be increased to 16K to > accomodate the maximum size supported for MS-DFS referrals by > Windows. Applications are expected to query the PATH_MAX and > SYMLINK_MAX values on the local system using > pathconf(2)/fpathconf(2). The value of SYMLINK_MAX would be > changed to 16K on ZFS. The value of PATH_MAX will not be > affected. > > To provide compatibility with other UNIXes (see section 6 > below), sharemgr(1M) would be enhanced to support a "refer" > option for NFS exports. This option would only result in > creation of a reparse point at the specified path and does not > actually share the path over NFS. > > This case is only about the underlying infrastructure and a > future case will be presented to deal with details and > specifics of handling referrals for NFSv4 server. > > SECURITY CONSIDERATIONS > > Referrals are similar to regular symbolic links in that they > are only pointers to data that could be discovered in some > other way. The presence of such a pointer does not compromise > the security of the target object or data; the target service > or file system must still enforce security. > > OPERATION FLOW > > Once a kernel service encounters a reparse point, it reads the > data using VOP_READLINK and passes the data up to a user space > daemon (e.g. reparsed) along with its desired record type. > Depending on the requested record type the daemon could simply > extract the information from the passed data and return it to > kernel or do any other processing necessary to obtain the > actual referral information e.g. in the case of FedFS, > contacting NSDB. Going through a common user space daemon to > get the referral data makes this process generic and easily > expandable for possible future use cases. > > Referral extraction and creation by a userspace daemon can be > handled via a library plugin architecture for different > service types. > > Operation Flow Example > > Here is a simplified example of operation for a CIFS client > that tries to access a file where the path contains a DFS > link: > > a) Client tries to access \\srv\root\...\link\...\file.txt > where: > 'root' is a share (namespace root) > 'link' is a reparse point seen as a folder by client > > b) CIFS server does a VOP_LOOKUP for 'link' when it is > recognized as a reparse point by examining the attributes > return by VOP_LOOKUP. At this point a > STATUS_PATH_NOT_COVERED is returned to client > > c) Client sends a "link referral" request to the server. CIFS > server uses VOP_READLINK to get the 'link' data and sends > the data to 'reparsed' daemon via a door call and gets back > the DFS link targets in a format understandable by the CIFS > client. The targets are sent back to the client in > response to its "link referral" request. > > b) Client picks one of the targets and contacts the target > server to access 'file.txt' > > NFS REFERRAL IN OTHER UNIXES > > FS referrals have been implemented in other major UNIX > distributions such as Linux, AIX and HP-UX but there is no > unified approach or implementation. > > Linux, AIX and HP-UX specify referrals as an NFS export > option. The option format is basically the same in all three > operating systems (refer=path@host) but the presentation is > somewhat different in each case: > > - In Linux a referral is presented as a mount point. > - In HP-UX a referral is a file system partition or logical volume. > - In AIX a special object is used to represent a referral. > > These are all mechanisms to trigger a change in namespace > while resolving a path. > > This proposal is somewhat aligned with the AIX approach but > does not require a new object type to be defined, which has > the advantage of not impacting existing applications. As > mentioned previously, an NFS "refer" option will be supported > to provide option format compatibility. > > Additionally, the Solaris requirements include support for > both NFS and SMB referrals whereas these other operating > systems only support NFS referrals, and they do not provide > native SMB support. For the Solaris operating system, this > proposal provides a generic solution to support multiple, > disparate referral mechanisms without placing restrictions on > the format required by each mechanism. > > The following links provide a bit more details about each OS > discussed above: > > http://www.citi.umich.edu/projects/nfsv4/linux/using-referrals.html > http://nfsv4.bullopensource.org/doc/migration-and-replication-0.2.pdf > http://docs.hp.com/en/5900-0306/ch01s11.html?jumpid=reg_R1002_USEN > http://docs.hp.com/en/13578/nfsv4_whitepaper.pdf > http://publib.boulder.ibm.com/infocenter/systems/index.jsp?topic=/com.ibm.aix.commadmn/doc/commadmndita/nfs_referrals.htm > > INTERFACE TABLE > > |Proposed |Specified | > |Stability |in what | > Interface Name |Classification |Document? | Comments > =========================================================================== > XAT_REPARSE |Consolidation |This |Reparse extensible > |Private |Document |attribute > | | | > VOP_LOOKUP, fop_lookup |Contracted |This |Added new argument: > |Consolidation |Document |vattr_t *vap > |Private* | | > | | | > Reparse token syntax |Committed |This | > |Private |Document | > | | | > SYMLINK_MAX |Committed |This |Increased to 16K > | |Document | > > * The project's deliverables will all go into the OS/NET > Consolidation, so no contracts are required. > > 6. Resources and Schedule: > > 6.4. Product Approval Committee requested information: > 6.4.1. Consolidation or Component Name: > ON > > 6.5. ARC review type: > FastTrack > > From Scott.Rotondo@sun.com Fri Jul 10 10:18:08 2009 Return-Path: Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01 [129.145.155.118]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6AHI88q012818 for ; Fri, 10 Jul 2009 10:18:08 -0700 (PDT) Received: from brmea-mail-1.sun.com (brmea-mail-1.Sun.COM [192.18.98.31]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6AHJINl024957 for ; Fri, 10 Jul 2009 10:19:19 -0700 (PDT) Received: from fe-amer-09.sun.com ([192.18.109.79]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6AHJINJ027332; Fri, 10 Jul 2009 17:19:18 GMT MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII; format=flowed Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMK00C00T0YFB00@mail-amer.sun.com>; Fri, 10 Jul 2009 11:19:18 -0600 (MDT) Received: from [129.146.108.62] ([unknown] [129.146.108.62]) by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMK00FKSTG58R90@mail-amer.sun.com>; Fri, 10 Jul 2009 11:19:17 -0600 (MDT) Date: Fri, 10 Jul 2009 10:18:23 -0700 From: Scott Rotondo Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A57743F.5070708@Sun.COM> Sender: Scott.Rotondo@sun.com To: Darren J Moffat Cc: Glenn Skinner , PSARC-ext@sun.com, cifs-eng@sun.com, Robert.Thurlow@sun.com, Brian.Wong@sun.com, Afshin.Ardakani@sun.com Message-id: <4A5777DF.9020904@sun.com> References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <4A57743F.5070708@Sun.COM> User-Agent: Thunderbird 2.0.0.21 (X11/20090323) Content-Length: 723 Status: RO X-Status: $$$$ X-UID: 0000000004 Darren J Moffat wrote: > This looks very cool but I haven't quite got my head around it > completely yet. > > What happens if open(2) is called with O_NOFOLLOW set on one of these > reparse points ? (Please answer for ZFS local access, NFS and CIFS). Since these reparse points are implemented with a special type of symlink, open() with O_NOFOLLOW should fail with such an object. It's been a while, but I'm nearly certain I implemented O_NOFOLLOW and O_NOLINKS entirely in filesystem-independent code. So the answer should be the same for all filesystems. Scott -- Scott Rotondo Principal Engineer, Solaris Security Technologies President, Trusted Computing Group Phone/FAX: +1 408 850 3655 (Internal x68278) From brian.wong@sun.com Fri Jul 10 10:20:17 2009 Return-Path: Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02 [129.146.11.31]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6AHKHGa012824 for ; Fri, 10 Jul 2009 10:20:17 -0700 (PDT) Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6AHLRB4003429; Fri, 10 Jul 2009 10:21:27 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n6AHLQ9e008920; Fri, 10 Jul 2009 10:21:27 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMK00C05TJRKY00@nwk-avmta-2.sfbay.sun.com>; Fri, 10 Jul 2009 10:21:27 -0700 (PDT) Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.59]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMK00AF5TJQUF20@nwk-avmta-2.sfbay.sun.com>; Fri, 10 Jul 2009 10:21:26 -0700 (PDT) Received: from [10.7.251.253] (punchin-blw.SFBay.Sun.COM [10.7.251.253]) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6AHLPnY178767; Fri, 10 Jul 2009 10:21:25 -0700 (PDT) Date: Fri, 10 Jul 2009 13:20:24 -0400 From: Brian Wong Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A57744B.1020700@sun.com> To: "Garrett D'Amore" Cc: Glenn Skinner , PSARC-ext@sun.com, cifs-eng@sun.com, Robert.Thurlow@sun.com, Afshin.Ardakani@sun.com Message-id: <4A577858.8070107@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <4A57744B.1020700@sun.com> User-Agent: Thunderbird 2.0.0.21 (X11/20090323) Content-Length: 300 Status: RO X-Status: $$$$ X-UID: 0000000005 Garrett D'Amore wrote: > > * presumably archivers and such are not expected to traverse > these? (they get handled like an ordinary symbolic link) Correct. > * what happens when the referral is archived and then reextracted? > (is the attribute lost?) No, the attributes are retained. blw From Robert.Thurlow@sun.com Fri Jul 10 10:22:59 2009 Return-Path: Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01 [129.145.155.118]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6AHMx9d012831 for ; Fri, 10 Jul 2009 10:22:59 -0700 (PDT) Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6AHO9h0028458 for ; Fri, 10 Jul 2009 10:24:09 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n6AHO8YH026197; Fri, 10 Jul 2009 10:24:09 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMK0000DTO8MV00@nwk-avmta-1.sfbay.Sun.COM>; Fri, 10 Jul 2009 10:24:08 -0700 (PDT) Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.59]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMK00DZBTO8SD30@nwk-avmta-1.sfbay.Sun.COM>; Fri, 10 Jul 2009 10:24:08 -0700 (PDT) Received: from [10.7.250.15] (punchin-client-10-7-250-15.SFBay.Sun.COM [10.7.250.15]) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6AHO7va179464; Fri, 10 Jul 2009 10:24:07 -0700 (PDT) Date: Fri, 10 Jul 2009 11:24:06 -0600 From: Robert Thurlow Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A57743F.5070708@Sun.COM> To: Darren J Moffat Cc: Glenn Skinner , PSARC-ext@sun.com, cifs-eng@sun.com, Brian.Wong@sun.com, Afshin.Ardakani@sun.com Message-id: <4A577936.9050509@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <4A57743F.5070708@Sun.COM> User-Agent: Thunderbird 2.0.0.16 (X11/20080811) Content-Length: 1317 Status: RO X-Status: $$$$ X-UID: 0000000006 Darren J Moffat wrote: > What happens if open(2) is called with O_NOFOLLOW set on one of these > reparse points ? (Please answer for ZFS local access, NFS and CIFS). For a process opening a symlink on ZFS, current behaviour in the open(2) man page should be seen (open() results in -1/ELOOP). This should also be the short-term behaviour with an open done on NFS and CIFS filesystems. In the longer term (and after delivering software to be defined in future cases), the intent is to have NFSv4 and CIFS return referrals instead of making the symlinks visible to the clients. > So why not just a system attribute to store the whole thing ? > Particularly since it is required to store a system attribute to > distinguish a reparse point from a normal symlink anyway. Archivers that slurp and spit the symlink contents will work without mods as long as they get all of the bytes, but would need more extensive modifications if our storage was in a system attribute. Also, we can get the single bit we need in ZFS now, and a 16K sysattr will not be supportable for a few more months. > Also if we do end up adding BSD magic link support for the link types > they have can a symlink link still have reparse data in it ? I believe other types of magic links would not be supported in the same symlink. Rob T From Nicolas.Williams@sun.com Fri Jul 10 10:24:04 2009 Return-Path: Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02 [129.146.11.31]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6AHO4Qc012834 for ; Fri, 10 Jul 2009 10:24:04 -0700 (PDT) Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6AHPEiK005751 for ; Fri, 10 Jul 2009 10:25:14 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n6AHP2Lo010379; Fri, 10 Jul 2009 10:25:13 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMK00C01TQ1UN00@nwk-avmta-2.sfbay.sun.com>; Fri, 10 Jul 2009 10:25:13 -0700 (PDT) Received: from binky.Central.Sun.COM ([129.153.128.104]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMK00A4STQ0UU20@nwk-avmta-2.sfbay.sun.com>; Fri, 10 Jul 2009 10:25:12 -0700 (PDT) Received: from binky.Central.Sun.COM (localhost [127.0.0.1]) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3) with ESMTP id n6AHMRA9002256; Fri, 10 Jul 2009 12:22:27 -0500 (CDT) Received: (from nw141292@localhost) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3/Submit) id n6AHMRqD002255; Fri, 10 Jul 2009 12:22:27 -0500 (CDT) Date: Fri, 10 Jul 2009 12:22:27 -0500 From: Nicolas Williams Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> To: Glenn Skinner Cc: cifs-eng@sun.com, Robert.Thurlow@sun.com, Brian.Wong@sun.com, Afshin.Ardakani@sun.com Message-id: <20090710172226.GX1274@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> X-Authentication-warning: binky.Central.Sun.COM: nw141292 set sender to Nicolas.Williams@sun.com using -f User-Agent: Mutt/1.5.7i Content-Length: 549 Status: RO X-Status: $$$$ X-UID: 0000000007 I think this is great, but I also think that perhaps there should be an automounter angle for any apps that wish to read a symlink and traverse it on their own. Imagine a new special automount map mounted on /ref (say). And imagine that it's keys are the reparse point configuration. Then any app trying to access /ref/ will get to the right place. Also, it'd be nice to have a way to convert symlinks containing /net paths into referrals automatically (again, with a one-bit extended attribute). Just a thought, Nico -- From Nicolas.Williams@sun.com Fri Jul 10 10:31:05 2009 Return-Path: Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02 [129.146.11.31]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6AHV5MY012860 for ; Fri, 10 Jul 2009 10:31:05 -0700 (PDT) Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6AHWFei010666; Fri, 10 Jul 2009 10:32:15 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n6AHWD6f029431; Fri, 10 Jul 2009 10:32:15 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMK0022LU1P5300@nwk-avmta-1.sfbay.Sun.COM>; Fri, 10 Jul 2009 10:32:13 -0700 (PDT) Received: from binky.Central.Sun.COM ([129.153.128.104]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMK00DS7U1OSK50@nwk-avmta-1.sfbay.Sun.COM>; Fri, 10 Jul 2009 10:32:12 -0700 (PDT) Received: from binky.Central.Sun.COM (localhost [127.0.0.1]) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3) with ESMTP id n6AHTNIZ002265; Fri, 10 Jul 2009 12:29:23 -0500 (CDT) Received: (from nw141292@localhost) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3/Submit) id n6AHTNc4002264; Fri, 10 Jul 2009 12:29:23 -0500 (CDT) Date: Fri, 10 Jul 2009 12:29:23 -0500 From: Nicolas Williams Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A5777DF.9020904@sun.com> To: Scott Rotondo Cc: Darren J Moffat , Glenn Skinner , PSARC-ext@sun.com, cifs-eng@sun.com, Robert.Thurlow@sun.com, Brian.Wong@sun.com, Afshin.Ardakani@sun.com Message-id: <20090710172923.GY1274@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <4A57743F.5070708@Sun.COM> <4A5777DF.9020904@sun.com> X-Authentication-warning: binky.Central.Sun.COM: nw141292 set sender to Nicolas.Williams@sun.com using -f User-Agent: Mutt/1.5.7i Content-Length: 1008 Status: RO X-Status: $$$$ X-UID: 0000000008 On Fri, Jul 10, 2009 at 10:18:23AM -0700, Scott Rotondo wrote: > Darren J Moffat wrote: > >This looks very cool but I haven't quite got my head around it > >completely yet. > > > >What happens if open(2) is called with O_NOFOLLOW set on one of these > >reparse points ? (Please answer for ZFS local access, NFS and CIFS). > > Since these reparse points are implemented with a special type of > symlink, open() with O_NOFOLLOW should fail with such an object. On the client side a server-side reparse point behaves like a mount point, very much in the same way as mirror mounts. Locally (on the server) a reparse point is stored in a symlink, but it isn't followed, and if it were then it'd behave like a mount, just as on the client side. There's nothing quite like following a symlink here, therefore O_NOFOLLOW shouldn't apply on the client side. An option to not cross mount-points and/or an option to not trigger [mirror or referral] mounts might be nice. E.g., O_XDEV and O_XTRIGGER. Nico -- From gdamore@sun.com Fri Jul 10 10:39:00 2009 Return-Path: Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02 [129.146.11.31]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6AHd00d012869 for ; Fri, 10 Jul 2009 10:39:00 -0700 (PDT) Received: from sca-es-mail-1.sun.com (sca-es-mail-1.Sun.COM [192.18.43.132]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6AHeAfi015802 for ; Fri, 10 Jul 2009 10:40:10 -0700 (PDT) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n6AHe57u017183; Fri, 10 Jul 2009 10:40:05 -0700 (PDT) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII; format=flowed Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMK00G00TJB3U00@fe-sfbay-10.sun.com>; Fri, 10 Jul 2009 10:40:05 -0700 (PDT) Received: from [192.168.251.11] ([unknown] [76.93.15.33]) by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMK00IB6UES9I60@fe-sfbay-10.sun.com>; Fri, 10 Jul 2009 10:40:05 -0700 (PDT) Date: Fri, 10 Jul 2009 10:40:04 -0700 From: "Garrett D'Amore" Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A577936.9050509@sun.com> Sender: Garrett.Damore@sun.com To: Robert Thurlow Cc: Darren J Moffat , Glenn Skinner , PSARC-ext@sun.com, cifs-eng@sun.com, Brian.Wong@sun.com, Afshin.Ardakani@sun.com Message-id: <4A577CF4.9050702@sun.com> References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <4A57743F.5070708@Sun.COM> <4A577936.9050509@sun.com> User-Agent: Thunderbird 2.0.0.18 (X11/20081201) Content-Length: 1955 Status: RO X-Status: $$$$ X-UID: 0000000009 Robert Thurlow wrote: > Darren J Moffat wrote: > >> What happens if open(2) is called with O_NOFOLLOW set on one of these >> reparse points ? (Please answer for ZFS local access, NFS and CIFS). > > For a process opening a symlink on ZFS, current behaviour in the > open(2) man page should be seen (open() results in -1/ELOOP). > This should also be the short-term behaviour with an open done > on NFS and CIFS filesystems. > > In the longer term (and after delivering software to be defined > in future cases), the intent is to have NFSv4 and CIFS return > referrals instead of making the symlinks visible to the clients. > >> So why not just a system attribute to store the whole thing ? >> Particularly since it is required to store a system attribute to >> distinguish a reparse point from a normal symlink anyway. > > Archivers that slurp and spit the symlink contents will work > without mods as long as they get all of the bytes, but would > need more extensive modifications if our storage was in a > system attribute. Also, we can get the single bit we need > in ZFS now, and a 16K sysattr will not be supportable for a > few more months. I'm confused. Brian says that archivers Just Work with the current form, because the attributes are retained. Yet, you're saying that the attributes are not necessarily retained. Which is it? Right now, either way, you have an attribute... which I *think* means that the you need support (which may or may not be present) in the archivers. I actually wonder if a different approach to this would be better -- say a per-filesystem tunable that enables parsing of symbolic links, then a simple -- *short* -- magic token that activates it. Say @@ or somesuch. (Simpler is better here.) The filesystem tunable would be available for administrators that don't want to use referrals, and have @@ (or whatever the magic token is) references already in their filesystems... -- Garrett From robert.thurlow@sun.com Fri Jul 10 10:59:04 2009 Return-Path: Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02 [129.146.11.31]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6AHx3DJ012890 for ; Fri, 10 Jul 2009 10:59:03 -0700 (PDT) Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6AI0DuW028707; Fri, 10 Jul 2009 11:00:14 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6AI05NL025122; Fri, 10 Jul 2009 19:00:12 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMK00F0FVC97S00@nwk-avmta-2.sfbay.sun.com>; Fri, 10 Jul 2009 11:00:09 -0700 (PDT) Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.63]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMK00A8GVC8UU40@nwk-avmta-2.sfbay.sun.com>; Fri, 10 Jul 2009 11:00:08 -0700 (PDT) Received: from [10.7.250.15] (punchin-client-10-7-250-15.SFBay.Sun.COM [10.7.250.15]) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6AI01dW184903; Fri, 10 Jul 2009 11:00:02 -0700 (PDT) Date: Fri, 10 Jul 2009 12:00:01 -0600 From: Robert Thurlow Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A57744B.1020700@sun.com> To: "Garrett D'Amore" Cc: Glenn Skinner , PSARC-ext@sun.com, cifs-eng@sun.com, Brian.Wong@sun.com, Afshin.Ardakani@sun.com Message-id: <4A5781A1.6050703@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <4A57744B.1020700@sun.com> User-Agent: Thunderbird 2.0.0.16 (X11/20080811) Content-Length: 1019 Status: RO X-Status: $$$$ X-UID: 0000000010 Garrett D'Amore wrote: > This sounds reasonable, and is not like the case of AFS which uses > special tokens in symbolic links that can expand to other things. That's our model, yes. > I'm a bit concerned about potential effects on applications, it *seems* > like this is done in a manner that is safe, but there are a few items: > > * are applications consistent in their use of pathconf/fpathconf to > get filesystem limits Not sure. > * presumably archivers and such are not expected to traverse these? > (they get handled like an ordinary symbolic link) Correct - we only want the symlink bytes. The main issue is the potential size, we think. > * what happens when the referral is archived and then reextracted? > (is the attribute lost?) The attribute is not sent in the archive stream, but will be re-applied by generic code when the symlink is recreated. We envision code to look at the contents in fop_symlink() to set the bit on create, whether the create was via NFS or ZFS. Rob T From Nicolas.Williams@sun.com Fri Jul 10 11:01:00 2009 Return-Path: Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02 [129.146.11.31]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6AI0xsA012898 for ; Fri, 10 Jul 2009 11:00:59 -0700 (PDT) Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6AI2ADk030454; Fri, 10 Jul 2009 11:02:10 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n6AI29Qg005922; Fri, 10 Jul 2009 12:02:09 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMK00803VFL6P00@nwk-avmta-1.sfbay.Sun.COM>; Fri, 10 Jul 2009 11:02:09 -0700 (PDT) Received: from binky.Central.Sun.COM ([129.153.128.104]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMK00DX8VFKSH50@nwk-avmta-1.sfbay.Sun.COM>; Fri, 10 Jul 2009 11:02:08 -0700 (PDT) Received: from binky.Central.Sun.COM (localhost [127.0.0.1]) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3) with ESMTP id n6AHxNrG002290; Fri, 10 Jul 2009 12:59:23 -0500 (CDT) Received: (from nw141292@localhost) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3/Submit) id n6AHxNN1002289; Fri, 10 Jul 2009 12:59:23 -0500 (CDT) Date: Fri, 10 Jul 2009 12:59:23 -0500 From: Nicolas Williams Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A577CF4.9050702@sun.com> To: "Garrett D'Amore" Cc: Robert Thurlow , Darren J Moffat , Glenn Skinner , PSARC-ext@sun.com, cifs-eng@sun.com, Brian.Wong@sun.com, Afshin.Ardakani@sun.com Message-id: <20090710175923.GZ1274@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <4A57743F.5070708@Sun.COM> <4A577936.9050509@sun.com> <4A577CF4.9050702@sun.com> X-Authentication-warning: binky.Central.Sun.COM: nw141292 set sender to Nicolas.Williams@sun.com using -f User-Agent: Mutt/1.5.7i Content-Length: 699 Status: RO X-Status: $$$$ X-UID: 0000000011 On Fri, Jul 10, 2009 at 10:40:04AM -0700, Garrett D'Amore wrote: > I actually wonder if a different approach to this would be better -- say > a per-filesystem tunable that enables parsing of symbolic links, then a > simple -- *short* -- magic token that activates it. Say @@ or > somesuch. (Simpler is better here.) The filesystem tunable would be > available for administrators that don't want to use referrals, and have > @@ (or whatever the magic token is) references already in their > filesystems... The magic could actually be a special automount map, so that any app reading a symlink and traversing it on its own would still get to the right object. /ref/ From robert.thurlow@sun.com Fri Jul 10 11:23:48 2009 Return-Path: Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02 [129.146.11.31]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6AINlc9012906 for ; Fri, 10 Jul 2009 11:23:47 -0700 (PDT) Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6AIOvQO044440; Fri, 10 Jul 2009 11:24:57 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6AIOpGG008649; Fri, 10 Jul 2009 19:24:55 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMK00C0TWHGRY00@nwk-avmta-1.sfbay.Sun.COM>; Fri, 10 Jul 2009 11:24:52 -0700 (PDT) Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.59]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMK00DFIWHGSD80@nwk-avmta-1.sfbay.Sun.COM>; Fri, 10 Jul 2009 11:24:52 -0700 (PDT) Received: from [10.7.250.15] (punchin-client-10-7-250-15.SFBay.Sun.COM [10.7.250.15]) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6AIOaZg190276; Fri, 10 Jul 2009 11:24:37 -0700 (PDT) Date: Fri, 10 Jul 2009 12:24:35 -0600 From: Robert Thurlow Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A577CF4.9050702@sun.com> To: "Garrett D'Amore" Cc: Darren J Moffat , Glenn Skinner , PSARC-ext@sun.com, cifs-eng@sun.com, brian.wong@sun.com, Afshin.Ardakani@sun.com Message-id: <4A578763.40204@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <4A57743F.5070708@Sun.COM> <4A577936.9050509@sun.com> <4A577CF4.9050702@sun.com> User-Agent: Thunderbird 2.0.0.16 (X11/20080811) Content-Length: 1132 Status: RO X-Status: $$$$ X-UID: 0000000012 Garrett D'Amore wrote: > I'm confused. Brian says that archivers Just Work with the current > form, because the attributes are retained. Yet, you're saying that the > attributes are not necessarily retained. Which is it? Right now, > either way, you have an attribute... which I *think* means that the you > need support (which may or may not be present) in the archivers. I know that some archivers in use today will not create a different bitstream for a reparse point than for a regular symlink, but I believe our intent to set the bit on restore or create means that we don't have to teach all archivers about the new bit. If any Sun-maintained archivers are aware, that's fine, but this should work with GNU tar, too. Logically, that bit is only a way for the NFS/CIFS server code to avoid having to read the symlink data to understand that it's touched a reparse point. We want this because the referral is a two-part deal - we return an "it's not here" error to the client first, and then respond to a followup query later. The sysattr makes an NFSv4 READDIR op able to avoid N accesses to symlink data. Rob T From Afshin.Ardakani@sun.com Fri Jul 10 11:25:23 2009 Return-Path: Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02 [129.146.11.31]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6AIPMe3012912 for ; Fri, 10 Jul 2009 11:25:22 -0700 (PDT) Received: from sca-es-mail-2.sun.com (sca-es-mail-2.Sun.COM [192.18.43.133]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6AIQXTX045533 for ; Fri, 10 Jul 2009 11:26:33 -0700 (PDT) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n6AIQS6V001535; Fri, 10 Jul 2009 11:26:28 -0700 (PDT) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII; format=flowed Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMK00A00W5RF900@fe-sfbay-10.sun.com>; Fri, 10 Jul 2009 11:26:28 -0700 (PDT) Received: from [10.1.106.212] ([unknown] [10.1.106.212]) by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMK001FMWK2IT50@fe-sfbay-10.sun.com>; Fri, 10 Jul 2009 11:26:28 -0700 (PDT) Date: Fri, 10 Jul 2009 11:26:14 -0700 From: Afshin Salek Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A577CF4.9050702@sun.com> Sender: Afshin.Ardakani@sun.com To: "Garrett D'Amore" Cc: Robert Thurlow , Darren J Moffat , Glenn Skinner , PSARC-ext@sun.com, cifs-eng@sun.com, Brian.Wong@sun.com Message-id: <4A5787C6.4040309@sun.com> References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <4A57743F.5070708@Sun.COM> <4A577936.9050509@sun.com> <4A577CF4.9050702@sun.com> User-Agent: Thunderbird 2.0.0.9 (X11/20080213) Content-Length: 736 Status: RO X-Status: $$$$ X-UID: 0000000013 > > I actually wonder if a different approach to this would be better -- say > a per-filesystem tunable that enables parsing of symbolic links, then a > simple -- *short* -- magic token that activates it. Say @@ or > somesuch. (Simpler is better here.) The filesystem tunable would be > available for administrators that don't want to use referrals, and have > @@ (or whatever the magic token is) references already in their > filesystems... > The reason we proposed the one bit extensible attribute is that consumers like CIFS and NFS could quickly determine whether or not a symlink is a reparse point without having to look at the content. This is, as explained in the case, can be done by a single VOP_LOOKUP. Afshin From Garrett.Damore@sun.com Fri Jul 10 12:35:11 2009 Return-Path: Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02 [129.146.11.31]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6AJZA81012974 for ; Fri, 10 Jul 2009 12:35:10 -0700 (PDT) Received: from sca-es-mail-1.sun.com (sca-es-mail-1.Sun.COM [192.18.43.132]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6AJaLeA024840 for ; Fri, 10 Jul 2009 12:36:21 -0700 (PDT) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n6AJaGOO001134; Fri, 10 Jul 2009 12:36:16 -0700 (PDT) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII; format=flowed Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMK00100ZR3TJ00@fe-sfbay-10.sun.com>; Fri, 10 Jul 2009 12:36:16 -0700 (PDT) Received: from [192.168.251.106] ([unknown] [76.93.15.33]) by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMK00A82ZS2QYC0@fe-sfbay-10.sun.com>; Fri, 10 Jul 2009 12:36:16 -0700 (PDT) Date: Fri, 10 Jul 2009 12:36:02 -0700 From: "Garrett D'Amore" Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <20090710175923.GZ1274@Sun.COM> Sender: Garrett.Damore@sun.com To: Nicolas Williams Cc: "Garrett D'Amore" , Robert Thurlow , Darren J Moffat , Glenn Skinner , PSARC-ext@sun.com, cifs-eng@sun.com, Brian.Wong@sun.com, Afshin.Ardakani@sun.com Message-id: <4A579822.4040306@sun.com> References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <4A57743F.5070708@Sun.COM> <4A577936.9050509@sun.com> <4A577CF4.9050702@sun.com> <20090710175923.GZ1274@Sun.COM> User-Agent: Thunderbird 2.0.0.21 (X11/20090505) Content-Length: 816 Status: RO X-Status: $$$$ X-UID: 0000000014 Nicolas Williams wrote: > On Fri, Jul 10, 2009 at 10:40:04AM -0700, Garrett D'Amore wrote: > >> I actually wonder if a different approach to this would be better -- say >> a per-filesystem tunable that enables parsing of symbolic links, then a >> simple -- *short* -- magic token that activates it. Say @@ or >> somesuch. (Simpler is better here.) The filesystem tunable would be >> available for administrators that don't want to use referrals, and have >> @@ (or whatever the magic token is) references already in their >> filesystems... >> > > The magic could actually be a special automount map, so that any app > reading a symlink and traversing it on its own would still get to the > right object. /ref/ > That would be a quite elegant design, IMO. - Garrett From Garrett.Damore@sun.com Fri Jul 10 12:37:21 2009 Return-Path: Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01 [129.145.155.118]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6AJbKxA012980 for ; Fri, 10 Jul 2009 12:37:20 -0700 (PDT) Received: from sca-es-mail-2.sun.com (sca-es-mail-2.Sun.COM [192.18.43.133]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6AJcVh4051170 for ; Fri, 10 Jul 2009 12:38:31 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n6AJcQuh009599; Fri, 10 Jul 2009 12:38:26 -0700 (PDT) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII; format=flowed Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMK00H00ZMWKL00@fe-sfbay-09.sun.com>; Fri, 10 Jul 2009 12:38:25 -0700 (PDT) Received: from [192.168.251.106] ([unknown] [76.93.15.33]) by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMK0048BZW0LZ60@fe-sfbay-09.sun.com>; Fri, 10 Jul 2009 12:38:25 -0700 (PDT) Date: Fri, 10 Jul 2009 12:38:24 -0700 From: "Garrett D'Amore" Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A578763.40204@sun.com> Sender: Garrett.Damore@sun.com To: Robert Thurlow Cc: "Garrett D'Amore" , Darren J Moffat , Glenn Skinner , PSARC-ext@sun.com, cifs-eng@sun.com, Brian.Wong@sun.com, Afshin.Ardakani@sun.com Message-id: <4A5798B0.1010400@sun.com> References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <4A57743F.5070708@Sun.COM> <4A577936.9050509@sun.com> <4A577CF4.9050702@sun.com> <4A578763.40204@sun.com> User-Agent: Thunderbird 2.0.0.21 (X11/20090505) Content-Length: 1308 Status: RO X-Status: $$$$ X-UID: 0000000015 Robert Thurlow wrote: > Garrett D'Amore wrote: > >> I'm confused. Brian says that archivers Just Work with the current >> form, because the attributes are retained. Yet, you're saying that >> the attributes are not necessarily retained. Which is it? Right >> now, either way, you have an attribute... which I *think* means that >> the you need support (which may or may not be present) in the archivers. > > I know that some archivers in use today will not create a different > bitstream for a reparse point than for a regular symlink, but I > believe our intent to set the bit on restore or create means that > we don't have to teach all archivers about the new bit. If any > Sun-maintained archivers are aware, that's fine, but this should > work with GNU tar, too. Okay, that's an important detail that I missed somewhere. (Was it in the original spec?) > > Logically, that bit is only a way for the NFS/CIFS server > code to avoid having to read the symlink data to understand > that it's touched a reparse point. We want this because the > referral is a two-part deal - we return an "it's not here" > error to the client first, and then respond to a followup > query later. The sysattr makes an NFSv4 READDIR op able to > avoid N accesses to symlink data. OK. -- Garrett > > Rob T From Afshin.Ardakani@sun.com Fri Jul 10 12:54:17 2009 Return-Path: Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02 [129.146.11.31]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6AJsGSG012994 for ; Fri, 10 Jul 2009 12:54:16 -0700 (PDT) Received: from sca-es-mail-2.sun.com (sca-es-mail-2.Sun.COM [192.18.43.133]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6AJtOCI035107 for ; Fri, 10 Jul 2009 12:55:24 -0700 (PDT) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n6AJtIx0010997; Fri, 10 Jul 2009 12:55:18 -0700 (PDT) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII; format=flowed Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMK00100ZR3TI00@fe-sfbay-10.sun.com>; Fri, 10 Jul 2009 12:55:18 -0700 (PDT) Received: from [10.1.106.212] ([unknown] [10.1.106.212]) by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KML00G7K0O46X10@fe-sfbay-10.sun.com>; Fri, 10 Jul 2009 12:55:17 -0700 (PDT) Date: Fri, 10 Jul 2009 12:55:05 -0700 From: Afshin Salek Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A5798B0.1010400@sun.com> Sender: Afshin.Ardakani@sun.com To: "Garrett D'Amore" Cc: Robert Thurlow , "Garrett D'Amore" , Darren J Moffat , Glenn Skinner , PSARC-ext@sun.com, cifs-eng@sun.com, Brian.Wong@sun.com Message-id: <4A579C99.5020403@sun.com> References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <4A57743F.5070708@Sun.COM> <4A577936.9050509@sun.com> <4A577CF4.9050702@sun.com> <4A578763.40204@sun.com> <4A5798B0.1010400@sun.com> User-Agent: Thunderbird 2.0.0.9 (X11/20080213) Content-Length: 1600 Status: RO X-Status: $$$$ X-UID: 0000000016 Garrett D'Amore wrote: > Robert Thurlow wrote: >> Garrett D'Amore wrote: >> >>> I'm confused. Brian says that archivers Just Work with the current >>> form, because the attributes are retained. Yet, you're saying that >>> the attributes are not necessarily retained. Which is it? Right >>> now, either way, you have an attribute... which I *think* means that >>> the you need support (which may or may not be present) in the archivers. >> >> I know that some archivers in use today will not create a different >> bitstream for a reparse point than for a regular symlink, but I >> believe our intent to set the bit on restore or create means that >> we don't have to teach all archivers about the new bit. If any >> Sun-maintained archivers are aware, that's fine, but this should >> work with GNU tar, too. > > Okay, that's an important detail that I missed somewhere. (Was it in > the original spec?) > Yes, it is: fop_symlink() needs to be modified to recognize the reparse @{REPARSE} tag and pass the appropriate attribute (i.e. reparse system attribute) to VOP_SYMLINK to be set on the symlink. Afshin >> >> Logically, that bit is only a way for the NFS/CIFS server >> code to avoid having to read the symlink data to understand >> that it's touched a reparse point. We want this because the >> referral is a two-part deal - we return an "it's not here" >> error to the client first, and then respond to a followup >> query later. The sysattr makes an NFSv4 READDIR op able to >> avoid N accesses to symlink data. > > OK. > > -- Garrett >> >> Rob T > From Afshin.Ardakani@sun.com Fri Jul 10 13:08:01 2009 Return-Path: Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01 [129.145.155.118]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6AK81sO013010 for ; Fri, 10 Jul 2009 13:08:01 -0700 (PDT) Received: from sca-es-mail-1.sun.com (sca-es-mail-1.Sun.COM [192.18.43.132]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6AK9ClG003263 for ; Fri, 10 Jul 2009 13:09:12 -0700 (PDT) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n6AK964V004287; Fri, 10 Jul 2009 13:09:06 -0700 (PDT) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII; format=flowed Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KML0060014E0J00@fe-sfbay-10.sun.com>; Fri, 10 Jul 2009 13:09:06 -0700 (PDT) Received: from [10.1.106.212] ([unknown] [10.1.106.212]) by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KML00G1C1B56X50@fe-sfbay-10.sun.com>; Fri, 10 Jul 2009 13:09:06 -0700 (PDT) Date: Fri, 10 Jul 2009 13:08:54 -0700 From: Afshin Salek Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <20090710172226.GX1274@Sun.COM> Sender: Afshin.Ardakani@sun.com To: Nicolas Williams Cc: Glenn Skinner , cifs-eng@sun.com, Robert.Thurlow@sun.com, Brian.Wong@sun.com Message-id: <4A579FD6.70004@sun.com> References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <20090710172226.GX1274@Sun.COM> User-Agent: Thunderbird 2.0.0.9 (X11/20080213) Content-Length: 886 Status: RO X-Status: $$$$ X-UID: 0000000017 Note that this case is only intended to provide a generic underlying mechanism. We can certainly discuss nice things that can be built on top of this framework but we want to keep an eye on the scope too. Perhaps these ideas can be presented as some follow up cases to this one. Afshin Nicolas Williams wrote: > I think this is great, but I also think that perhaps there should be an > automounter angle for any apps that wish to read a symlink and traverse > it on their own. Imagine a new special automount map mounted on /ref > (say). And imagine that it's keys are the reparse point configuration. > Then any app trying to access /ref/ will get to > the right place. > > Also, it'd be nice to have a way to convert symlinks containing /net > paths into referrals automatically (again, with a one-bit extended > attribute). > > Just a thought, > > Nico From Nicolas.Williams@sun.com Fri Jul 10 13:08:06 2009 Return-Path: Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02 [129.146.11.31]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6AK86gA013013 for ; Fri, 10 Jul 2009 13:08:06 -0700 (PDT) Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6AK9H6j043443; Fri, 10 Jul 2009 13:09:17 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n6AK9Dbb008552; Fri, 10 Jul 2009 13:09:16 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KML00A011BGWU00@nwk-avmta-1.sfbay.Sun.COM>; Fri, 10 Jul 2009 13:09:16 -0700 (PDT) Received: from binky.Central.Sun.COM ([129.153.128.104]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KML007GY1BFP810@nwk-avmta-1.sfbay.Sun.COM>; Fri, 10 Jul 2009 13:09:15 -0700 (PDT) Received: from binky.Central.Sun.COM (localhost [127.0.0.1]) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3) with ESMTP id n6AK6T4e002449; Fri, 10 Jul 2009 15:06:29 -0500 (CDT) Received: (from nw141292@localhost) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3/Submit) id n6AK6Tgq002448; Fri, 10 Jul 2009 15:06:29 -0500 (CDT) Date: Fri, 10 Jul 2009 15:06:29 -0500 From: Nicolas Williams Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A579822.4040306@sun.com> To: "Garrett D'Amore" Cc: "Garrett D'Amore" , Robert Thurlow , Darren J Moffat , Glenn Skinner , PSARC-ext@sun.com, cifs-eng@sun.com, Brian.Wong@sun.com, Afshin.Ardakani@sun.com Message-id: <20090710200628.GH1274@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <4A57743F.5070708@Sun.COM> <4A577936.9050509@sun.com> <4A577CF4.9050702@sun.com> <20090710175923.GZ1274@Sun.COM> <4A579822.4040306@sun.com> X-Authentication-warning: binky.Central.Sun.COM: nw141292 set sender to Nicolas.Williams@sun.com using -f User-Agent: Mutt/1.5.7i Content-Length: 1224 Status: RO X-Status: $$$$ X-UID: 0000000018 On Fri, Jul 10, 2009 at 12:36:02PM -0700, Garrett D'Amore wrote: > Nicolas Williams wrote: > >On Fri, Jul 10, 2009 at 10:40:04AM -0700, Garrett D'Amore wrote: > > > >>I actually wonder if a different approach to this would be better -- say > >>a per-filesystem tunable that enables parsing of symbolic links, then a > >>simple -- *short* -- magic token that activates it. Say @@ or > >>somesuch. (Simpler is better here.) The filesystem tunable would be > >>available for administrators that don't want to use referrals, and have > >>@@ (or whatever the magic token is) references already in their > >>filesystems... > >> > > > >The magic could actually be a special automount map, so that any app > >reading a symlink and traversing it on its own would still get to the > >right object. /ref/ > > > That would be a quite elegant design, IMO. Maybe, but the key thing is that such a map need not be implemented from day 1. By just reserving and using the potential special map mountpoint the option would remain available to do this. Also, I'm not sure that apps that read symlinks should have any expectations that the contents of a symlink will be a filesystem path. Nico -- From psarc-member-list-request@sun.com Sat Jul 11 07:50:19 2009 Return-Path: Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02 [129.146.11.31]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6BEoIsf014007 for ; Sat, 11 Jul 2009 07:50:18 -0700 (PDT) Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6BEpTBP020013; Sat, 11 Jul 2009 07:51:29 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n6BEpNTi010029; Sat, 11 Jul 2009 07:51:23 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMM00903H9NLY00@nwk-avmta-1.sfbay.Sun.COM> (ORCPT PSARC-ext@sun.com) ; Sat, 11 Jul 2009 07:51:23 -0700 (PDT) Received: from dm-sfbay-02.sfbay.sun.com ([129.146.11.31]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMM006BRH9NY9E0@nwk-avmta-1.sfbay.Sun.COM> (ORCPT PSARC-ext@sun.com); Sat, 11 Jul 2009 07:51:23 -0700 (PDT) Received: from sac.sfbay.sun.com (sac.SFBay.Sun.COM [129.146.226.132]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6BEpMsK019972; Sat, 11 Jul 2009 07:51:22 -0700 (PDT) Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6BEo8H8010752 for ; Sat, 11 Jul 2009 07:50:11 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6BEo2CR028973 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Sat, 11 Jul 2009 15:50:08 +0100 (BST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMM00J05H7IN600@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Sat, 11 Jul 2009 08:50:06 -0600 (MDT) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMM00KKPH7H5Z60@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Sat, 11 Jul 2009 08:50:05 -0600 (MDT) Received: from relay15i.sun.com (ip125.net129179-4.block1.us.syntegra.com [129.179.4.125]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6BEiBJj009582 for ; Sat, 11 Jul 2009 14:50:05 +0000 (GMT) Received: from mmp13es.mmp.us.syntegra.com ([160.41.208.13] [160.41.208.13]) by relay15i.sun.com with ESMTP id BT-MMP-356828 for PSARC-ext@sun.com; Sat, 11 Jul 2009 14:50:04 +0000 (Z) Received: from relay14i.sun.com (relay14i.sun.com [129.179.4.124]) by mmp13es.mmp.us.syntegra.com with ESMTP id BT-MMP-4944417 for PSARC-ext@sun.com; Sat, 11 Jul 2009 14:50:02 +0000 (Z) Received: from relay03-haj2.antispameurope.com ([83.246.65.53] [83.246.65.53]) by relay1i.sun.com with ESMTP id BT-MMP-2081198 for PSARC-ext@sun.com; Sat, 11 Jul 2009 14:50:01 +0000 (Z) Received: by relay03-haj2.antispameurope.com (ASE-Secure-MTA, from userid 1000) id D28B463C083; Sat, 11 Jul 2009 16:50:00 +0200 (CEST) Received: from pluto.fokus.fraunhofer.de (pluto.fokus.fraunhofer.de [195.37.77.164]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by relay03-haj2.antispameurope.com (ASE-Secure-MTA) with ESMTP id DFE1163C083; Sat, 11 Jul 2009 16:49:59 +0200 (CEST) Received: from EXCHSRV.fokus.fraunhofer.de (bohr.fokus.fraunhofer.de [10.147.9.231]) by pluto.fokus.fraunhofer.de (8.14.2/8.14.2) with SMTP id n6BEnxuY029619; Sat, 11 Jul 2009 16:49:59 +0200 (MEST) Received: from rigel ([10.147.65.195]) by EXCHSRV.fokus.fraunhofer.de with Microsoft SMTPSVC(6.0.3790.3959); Sat, 11 Jul 2009 16:49:59 +0200 Date: Sat, 11 Jul 2009 16:49:59 +0200 From: Joerg.Schilling@fokus.fraunhofer.de (Joerg Schilling) Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> Sender: Joerg.Schilling9ab33xy531fokus.fraunhofer.de@bounce.antispameurope.com To: PSARC-ext@sun.com, glenn.skinner@sun.com Cc: Robert.Thurlow@sun.com, cifs-eng@sun.com, Brian.Wong@sun.com, Afshin.Ardakani@sun.com Message-id: <4a58a697.yQ8anaL/V6TEAoRX%Joerg.Schilling@fokus.fraunhofer.de> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1 Content-transfer-encoding: 8BIT X-PMX-Version: 5.4.1.325704 X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-Antispam: No, score=-1.1/5.0, scanned in 2.087sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> User-Agent: nail 11.22 3/20/05 X-OriginalArrivalTime: 11 Jul 2009 14:49:59.0186 (UTC) FILETIME=[DCF2A720:01CA0236] Content-Length: 3712 Status: RO X-Status: $$$$ X-UID: 0000000019 Glenn Skinner wrote: > 4. Technical Description: > 4.1. Details: > > INTRODUCTION ... > To distinguish a regular symlink from a reparse point, an > extensible system attribute will be set on the symlink. This > system attribute is only one bit which indicates whether or > not a symlink contains reparse data. How exactly will these "reparse points" be distinguished from symlinks? > The reparse data will be stored as the link target. The > reparse data is not in file system path format, which is the > typical format of a link target. In order to avoid coming up > with a totaly new format for reparse data as the link target > we decided to adopt the format used by magic links in BSD: > (http://www.daemon-systems.org/man/symlink.7.html) > > @{REPARSE@{service-type1:data} [@{service-type2:data}]...} > > Where some examples of service-type are: > > #define REPARSE_SVC_SMB "SMB" > #define REPARSE_SVC_NFS "NFS" > #define REPARSE_SVC_HSM "HSM" > > The data for each service will be in string format, which is > expected to be typically a UUID string. > > The pattern above starts with "REPARSE" to distinguish it from > a other magic links, such as those supported by BSD. Note > that this case is not a proposal to support BSD magic links, > the intent is to avoid precluding the future addition of full > BSD magic link support. How will you prevent prople from removing these "reparse points" because they asume that this are symlinks that point to nowhere? Note that the official POSIX method for removing symlinks that do not point to a target is: find -L . -type l -exec rm {} + > The symlink target size should be increased to 16K to > accomodate the maximum size supported for MS-DFS referrals by > Windows. Applications are expected to query the PATH_MAX and > SYMLINK_MAX values on the local system using > pathconf(2)/fpathconf(2). The value of SYMLINK_MAX would be > changed to 16K on ZFS. The value of PATH_MAX will not be > affected. Whill SYMLINK_MAX from limits.h be set to 16k? > Operation Flow Example > > Here is a simplified example of operation for a CIFS client > that tries to access a file where the path contains a DFS > link: > > a) Client tries to access \\srv\root\...\link\...\file.txt > where: > 'root' is a share (namespace root) > 'link' is a reparse point seen as a folder by client Is this is a typo and you accidently used '\\' as path name separator instead of the official path name separator '/'? > NFS REFERRAL IN OTHER UNIXES > > FS referrals have been implemented in other major UNIX > distributions such as Linux, AIX and HP-UX but there is no > unified approach or implementation. > > Linux, AIX and HP-UX specify referrals as an NFS export > option. The option format is basically the same in all three > operating systems (refer=path@host) but the presentation is > somewhat different in each case: > > - In Linux a referral is presented as a mount point. > - In HP-UX a referral is a file system partition or logical volume. > - In AIX a special object is used to represent a referral. Is this intended to be something like the conditional symbolic links that have been in Apollo Domain system 23 years ago? Jörg -- EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin js@cs.tu-berlin.de (uni) joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily From Joerg.Schilling9ab33xy531fokus.fraunhofer.de@bounce.antispameurope.com Sat Jul 11 07:59:11 2009 Return-Path: Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01 [129.145.155.118]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6BExBT1014015 for ; Sat, 11 Jul 2009 07:59:11 -0700 (PDT) Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6BF0LA7031489 for ; Sat, 11 Jul 2009 08:00:21 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6BF0GXu003210 for <@sunmail2sca.sfbay.sun.com:Glenn.Skinner@sun.com>; Sat, 11 Jul 2009 16:00:20 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMM00E09HOKBH00@nwk-avmta-2.sfbay.sun.com> for Glenn.Skinner@sun.com (ORCPT Glenn.Skinner@sun.com); Sat, 11 Jul 2009 08:00:20 -0700 (PDT) Received: from brmea-mail-2.sun.com ([192.18.98.43]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMM009MSHOJZD10@nwk-avmta-2.sfbay.sun.com> for Glenn.Skinner@sun.com (ORCPT Glenn.Skinner@sun.com); Sat, 11 Jul 2009 08:00:19 -0700 (PDT) Received: from relay42i.sun.com ([192.5.209.72]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6BEvmqv021707 for ; Sat, 11 Jul 2009 15:00:19 +0000 (GMT) Received: from mms49es.mms.us.syntegra.com ([160.41.221.232] [160.41.221.232]) by relay42i.sun.com with ESMTP id BT-MMP-1186883 for Glenn.Skinner@sun.com; Sat, 11 Jul 2009 15:00:18 +0000 (Z) Received: from relay45i.sun.com (relay45i.sun.com [192.5.209.94]) by mms49es.mms.us.syntegra.com with ESMTP id BT-MMP-79629311 for Glenn.Skinner@sun.com; Sat, 11 Jul 2009 15:00:13 +0000 (Z) Received: from relay02-haj2.antispameurope.com ([83.246.65.52] [83.246.65.52]) by relay4i.sun.com with ESMTP id BT-MMP-2233748 for Glenn.Skinner@sun.com; Sat, 11 Jul 2009 14:59:45 +0000 (Z) Received: by relay02-haj2.antispameurope.com (ASE-Secure-MTA, from userid 1000) id 722C86F04EE; Sat, 11 Jul 2009 16:59:42 +0200 (CEST) Received: from pluto.fokus.fraunhofer.de (pluto.fokus.fraunhofer.de [195.37.77.164]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by relay02-haj2.antispameurope.com (ASE-Secure-MTA) with ESMTP id D0CBC6F04EE; Sat, 11 Jul 2009 16:59:41 +0200 (CEST) Received: from EXCHSRV.fokus.fraunhofer.de (bohr.fokus.fraunhofer.de [10.147.9.231]) by pluto.fokus.fraunhofer.de (8.14.2/8.14.2) with SMTP id n6BExfwQ029735; Sat, 11 Jul 2009 16:59:41 +0200 (MEST) Received: from rigel ([10.147.65.195]) by EXCHSRV.fokus.fraunhofer.de with Microsoft SMTPSVC(6.0.3790.3959); Sat, 11 Jul 2009 16:59:41 +0200 Date: Sat, 11 Jul 2009 16:59:41 +0200 From: Joerg.Schilling@fokus.fraunhofer.de (Joerg Schilling) Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A57744B.1020700@sun.com> Sender: Joerg.Schilling9ab33xy531fokus.fraunhofer.de@bounce.antispameurope.com To: Glenn.Skinner@sun.com, gdamore@sun.com Cc: Robert.Thurlow@sun.com, PSARC-ext@sun.com, cifs-eng@sun.com, Brian.Wong@sun.com, Afshin.Ardakani@sun.com Message-id: <4a58a8dd.90iOrIma1/BO/qu6%Joerg.Schilling@fokus.fraunhofer.de> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1 Content-transfer-encoding: 8BIT X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-Antispam: No, score=0.0/5.0, scanned in 0.084sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <4A57744B.1020700@sun.com> User-Agent: nail 11.22 3/20/05 X-OriginalArrivalTime: 11 Jul 2009 14:59:41.0420 (UTC) FILETIME=[37FC82C0:01CA0238] Content-Length: 1482 Status: RO X-Status: $$$$ X-UID: 0000000020 "Garrett D'Amore" wrote: > This sounds reasonable, and is not like the case of AFS which uses > special tokens in symbolic links that can expand to other things. > > I'm a bit concerned about potential effects on applications, it *seems* > like this is done in a manner that is safe, but there are a few items: > > * are applications consistent in their use of pathconf/fpathconf to > get filesystem limits > * presumably archivers and such are not expected to traverse these? > (they get handled like an ordinary symbolic link) > * what happens when the referral is archived and then reextracted? > (is the attribute lost?) > * as a nit, its not truly file system independent, since it relies > on symbolic links (not all filesystems support > symlinks, though admittedly the ones of interest to this case all do) I expect to see a lof of applications that asume that SYMLINK_MAX is identical to PATH_MAX. This is really hard to review. Unless SYMLINK_MAX is set to the maximum value on the system, I asume to see many applications that may fail because they do not call pathconf(). The case does not introduce a trivial anhancement. Jörg -- EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin js@cs.tu-berlin.de (uni) joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily From psarc-member-list-request@sun.com Sat Jul 11 08:04:17 2009 Return-Path: Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01 [129.145.155.118]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6BF4GMN014024 for ; Sat, 11 Jul 2009 08:04:16 -0700 (PDT) Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6BF5Q9h033826; Sat, 11 Jul 2009 08:05:27 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6BF5Fp7005363; Sat, 11 Jul 2009 16:05:17 +0100 (BST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMM00L03HWSGG00@brm-avmta-1.central.sun.com> (ORCPT PSARC-ext@sun.com) ; Sat, 11 Jul 2009 09:05:16 -0600 (MDT) Received: from dm-sfbay-02.sfbay.sun.com ([129.146.11.31]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMM00KNNHWS5V70@brm-avmta-1.central.sun.com> (ORCPT PSARC-ext@sun.com); Sat, 11 Jul 2009 09:05:16 -0600 (MDT) Received: from sac.sfbay.sun.com (sac.SFBay.Sun.COM [129.146.226.132]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6BF5FLq024675; Sat, 11 Jul 2009 08:05:15 -0700 (PDT) Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6BF3rXe012480 for ; Sat, 11 Jul 2009 08:03:53 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n6BF3j1Q008192 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Sat, 11 Jul 2009 23:03:52 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMM00C01HUE7400@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Sat, 11 Jul 2009 08:03:50 -0700 (PDT) Received: from sca-ea-mail-1.sun.com ([192.18.43.24]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMM006DNHUDY5E0@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Sat, 11 Jul 2009 08:03:49 -0700 (PDT) Received: from relay41i.sun.com ([192.5.209.70]) by sca-ea-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n6BEwodY027806 for ; Sat, 11 Jul 2009 15:03:49 +0000 (GMT) Received: from mms49es.mms.us.syntegra.com ([160.41.221.232] [160.41.221.232]) by relay41i.sun.com with ESMTP id BT-MMP-2287796 for PSARC-ext@sun.com; Sat, 11 Jul 2009 15:03:49 +0000 (Z) Received: from relay44i.sun.com (relay44i.sun.com [192.5.209.118]) by mms49es.mms.us.syntegra.com with ESMTP id BT-MMP-79635458 for PSARC-ext@sun.com; Sat, 11 Jul 2009 15:03:48 +0000 (Z) Received: from relay01-haj2.antispameurope.com ([83.246.65.51] [83.246.65.51]) by relay4i.sun.com with ESMTP id BT-MMP-6989784 for PSARC-ext@sun.com; Sat, 11 Jul 2009 15:03:04 +0000 (Z) Received: by relay01-haj2.antispameurope.com (ASE-Secure-MTA, from userid 1000) id A54F8940F8; Sat, 11 Jul 2009 17:02:59 +0200 (CEST) Received: from pluto.fokus.fraunhofer.de (pluto.fokus.fraunhofer.de [195.37.77.164]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by relay01-haj2.antispameurope.com (ASE-Secure-MTA) with ESMTP id 9B63C940F8; Sat, 11 Jul 2009 17:02:58 +0200 (CEST) Received: from EXCHSRV.fokus.fraunhofer.de (bohr.fokus.fraunhofer.de [10.147.9.231]) by pluto.fokus.fraunhofer.de (8.14.2/8.14.2) with SMTP id n6BF2wZP029751; Sat, 11 Jul 2009 17:02:58 +0200 (MEST) Received: from rigel ([10.147.65.195]) by EXCHSRV.fokus.fraunhofer.de with Microsoft SMTPSVC(6.0.3790.3959); Sat, 11 Jul 2009 17:02:58 +0200 Date: Sat, 11 Jul 2009 17:02:58 +0200 From: Joerg.Schilling@fokus.fraunhofer.de (Joerg Schilling) Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <20090710172923.GY1274@Sun.COM> Sender: Joerg.Schilling9ab33xy531fokus.fraunhofer.de@bounce.antispameurope.com To: Scott.Rotondo@sun.com, Nicolas.Williams@sun.com Cc: Robert.Thurlow@sun.com, PSARC-ext@sun.com, cifs-eng@sun.com, Brian.Wong@sun.com, Afshin.Ardakani@sun.com Message-id: <4a58a9a2.ZrWt8z8SX87Z2Bdt%Joerg.Schilling@fokus.fraunhofer.de> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1 Content-transfer-encoding: 8BIT X-PMX-Version: 5.4.1.325704 X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-Antispam: No, score=-0.2/5.0, scanned in 0.099sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <4A57743F.5070708@Sun.COM> <4A5777DF.9020904@sun.com> <20090710172923.GY1274@Sun.COM> User-Agent: nail 11.22 3/20/05 X-OriginalArrivalTime: 11 Jul 2009 15:02:58.0785 (UTC) FILETIME=[ADA00510:01CA0238] Content-Length: 1379 Status: RO X-Status: $$$$ X-UID: 0000000021 Nicolas Williams wrote: > On Fri, Jul 10, 2009 at 10:18:23AM -0700, Scott Rotondo wrote: > > Darren J Moffat wrote: > > >This looks very cool but I haven't quite got my head around it > > >completely yet. > > > > > >What happens if open(2) is called with O_NOFOLLOW set on one of these > > >reparse points ? (Please answer for ZFS local access, NFS and CIFS). > > > > Since these reparse points are implemented with a special type of > > symlink, open() with O_NOFOLLOW should fail with such an object. > > On the client side a server-side reparse point behaves like a mount > point, very much in the same way as mirror mounts. > > Locally (on the server) a reparse point is stored in a symlink, but it > isn't followed, and if it were then it'd behave like a mount, just as on > the client side. > > There's nothing quite like following a symlink here, therefore > O_NOFOLLOW shouldn't apply on the client side. If the feature is implemented as symlink, how will stat() vs. lstat() perform on these objects? Will it confuse existing applications? Jörg -- EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin js@cs.tu-berlin.de (uni) joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily From psarc-member-list-request@sun.com Sat Jul 11 08:11:59 2009 Return-Path: Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02 [129.146.11.31]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6BFBw8T014030 for ; Sat, 11 Jul 2009 08:11:58 -0700 (PDT) Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6BFD8L9026790; Sat, 11 Jul 2009 08:13:09 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n6BFCtpj012416; Sat, 11 Jul 2009 23:12:55 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMM00E01I9J2M00@nwk-avmta-1.sfbay.Sun.COM> (ORCPT PSARC-ext@sun.com) ; Sat, 11 Jul 2009 08:12:55 -0700 (PDT) Received: from dm-sfbay-01.sfbay.sun.com ([129.145.155.118]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMM00DYZI9I1S00@nwk-avmta-1.sfbay.Sun.COM> (ORCPT PSARC-ext@sun.com); Sat, 11 Jul 2009 08:12:54 -0700 (PDT) Received: from sac.sfbay.sun.com (sac.SFBay.Sun.COM [129.146.226.132]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6BFCsJM035982; Sat, 11 Jul 2009 08:12:54 -0700 (PDT) Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6BFBJiM014407 for ; Sat, 11 Jul 2009 08:11:20 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n6BFBJCY023235 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Sat, 11 Jul 2009 08:11:19 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMM00F01I6V4A00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Sat, 11 Jul 2009 08:11:19 -0700 (PDT) Received: from sca-ea-mail-2.sun.com ([192.18.43.25]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMM0090II6VZD20@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Sat, 11 Jul 2009 08:11:19 -0700 (PDT) Received: from relay43i.sun.com ([192.5.209.74]) by sca-ea-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n6BExQOx011119 for ; Sat, 11 Jul 2009 15:11:18 +0000 (GMT) Received: from mmp43es.mmp.us.syntegra.com ([160.41.221.12] [160.41.221.12]) by relay43i.sun.com with ESMTP id BT-MMP-1066435 for PSARC-ext@sun.com; Sat, 11 Jul 2009 15:11:18 +0000 (Z) Received: from relay44i.sun.com (relay44i.sun.com [192.5.209.118]) by mmp43es.mmp.us.syntegra.com with ESMTP id BT-MMP-27253159 for PSARC-ext@sun.com; Sat, 11 Jul 2009 15:11:18 +0000 (Z) Received: from relay04-haj2.antispameurope.com ([83.246.65.54] [83.246.65.54]) by relay4i.sun.com with ESMTP id BT-MMP-7004840 for PSARC-ext@sun.com; Sat, 11 Jul 2009 15:11:15 +0000 (Z) Received: by relay04-haj2.antispameurope.com (ASE-Secure-MTA, from userid 1000) id 696165EC10E; Sat, 11 Jul 2009 17:11:13 +0200 (CEST) Received: from pluto.fokus.fraunhofer.de (pluto.fokus.fraunhofer.de [195.37.77.164]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by relay04-haj2.antispameurope.com (ASE-Secure-MTA) with ESMTP id A046B5EC105; Sat, 11 Jul 2009 17:11:12 +0200 (CEST) Received: from EXCHSRV.fokus.fraunhofer.de (bohr.fokus.fraunhofer.de [10.147.9.231]) by pluto.fokus.fraunhofer.de (8.14.2/8.14.2) with SMTP id n6BFBCAD029836; Sat, 11 Jul 2009 17:11:12 +0200 (MEST) Received: from rigel ([10.147.65.195]) by EXCHSRV.fokus.fraunhofer.de with Microsoft SMTPSVC(6.0.3790.3959); Sat, 11 Jul 2009 17:11:12 +0200 Date: Sat, 11 Jul 2009 17:11:12 +0200 From: Joerg.Schilling@fokus.fraunhofer.de (Joerg Schilling) Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A577CF4.9050702@sun.com> Sender: Joerg.Schilling9ab33xy531fokus.fraunhofer.de@bounce.antispameurope.com To: Robert.Thurlow@sun.com, gdamore@sun.com Cc: PSARC-ext@sun.com, cifs-eng@sun.com, Brian.Wong@sun.com, Afshin.Ardakani@sun.com Message-id: <4a58ab90.X2ncDHPKuL+7EDtV%Joerg.Schilling@fokus.fraunhofer.de> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1 Content-transfer-encoding: 8BIT X-PMX-Version: 5.4.1.325704 X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-Antispam: No, score=-0.2/5.0, scanned in 0.348sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <4A57743F.5070708@Sun.COM> <4A577936.9050509@sun.com> <4A577CF4.9050702@sun.com> User-Agent: nail 11.22 3/20/05 X-OriginalArrivalTime: 11 Jul 2009 15:11:12.0469 (UTC) FILETIME=[D3E23850:01CA0239] Content-Length: 1720 Status: RO X-Status: $$$$ X-UID: 0000000022 "Garrett D'Amore" wrote: > > Archivers that slurp and spit the symlink contents will work > > without mods as long as they get all of the bytes, but would > > need more extensive modifications if our storage was in a > > system attribute. Also, we can get the single bit we need > > in ZFS now, and a 16K sysattr will not be supportable for a > > few more months. > > I'm confused. Brian says that archivers Just Work with the current > form, because the attributes are retained. Yet, you're saying that the > attributes are not necessarily retained. Which is it? Right now, > either way, you have an attribute... which I *think* means that the you > need support (which may or may not be present) in the archivers. If these objects will be seen as symlink file type and in case they cannot be copied using symlink(), I expect problems. I beleve we need to get more information in order to be able to check whether the case makes a POSIX compliant proposal. And BTW, I like to remind people to the "well planned" Linux implementation for "file flags" that forces archivesrs like star to open() _all_ files in search for file flags. If I am going to backup /dev/, I am forced to open e.g. the "rewind" type of a tape drive entry and thus forcing the tape to rewind just for checking for fflags. So please give more information on how to distinguish these objects from vanilla symlinks. Jörg -- EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin js@cs.tu-berlin.de (uni) joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily From psarc-member-list-request@sun.com Sat Jul 11 08:38:44 2009 Return-Path: Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01 [129.145.155.118]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6BFciLO014053 for ; Sat, 11 Jul 2009 08:38:44 -0700 (PDT) Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6BFdsNS045687; Sat, 11 Jul 2009 08:39:55 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6BFdkO2017740; Sat, 11 Jul 2009 16:39:46 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMM00J01JI9HZ00@nwk-avmta-1.sfbay.Sun.COM> (ORCPT PSARC-ext@sun.com) ; Sat, 11 Jul 2009 08:39:45 -0700 (PDT) Received: from dm-sfbay-01.sfbay.sun.com ([129.145.155.118]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMM00D41JI91S70@nwk-avmta-1.sfbay.Sun.COM> (ORCPT PSARC-ext@sun.com); Sat, 11 Jul 2009 08:39:45 -0700 (PDT) Received: from sac.sfbay.sun.com (sac.SFBay.Sun.COM [129.146.226.132]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6BFdiwb045654; Sat, 11 Jul 2009 08:39:44 -0700 (PDT) Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6BFcJSw013738 for ; Sat, 11 Jul 2009 08:38:20 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n6BFcBv6023372 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Sat, 11 Jul 2009 23:38:17 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMM00H01JFP2W00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Sat, 11 Jul 2009 08:38:13 -0700 (PDT) Received: from sca-ea-mail-2.sun.com ([192.18.43.25]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMM0091BJFPZH30@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Sat, 11 Jul 2009 08:38:13 -0700 (PDT) Received: from relay14i.sun.com (ip124.net129179-4.block1.us.syntegra.com [129.179.4.124]) by sca-ea-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n6BFcAZE016433 for ; Sat, 11 Jul 2009 15:38:13 +0000 (GMT) Received: from mmp12es.mmp.us.syntegra.com ([160.41.208.12] [160.41.208.12]) by relay14i.sun.com with ESMTP id BT-MMP-5171874 for PSARC-ext@sun.com; Sat, 11 Jul 2009 15:36:10 +0000 (Z) Received: from relay13i.sun.com (relay13i.sun.com [129.179.4.123]) by mmp12es.mmp.us.syntegra.com with ESMTP id BT-MMP-5051341 for PSARC-ext@sun.com; Sat, 11 Jul 2009 15:36:08 +0000 (Z) Received: from relay04-haj2.antispameurope.com ([83.246.65.54] [83.246.65.54]) by relay1i.sun.com with ESMTP id BT-MMP-14530771 for PSARC-ext@sun.com; Sat, 11 Jul 2009 15:36:07 +0000 (Z) Received: by relay04-haj2.antispameurope.com (ASE-Secure-MTA, from userid 1000) id F10D55EC100; Sat, 11 Jul 2009 17:36:06 +0200 (CEST) Received: from pluto.fokus.fraunhofer.de (pluto.fokus.fraunhofer.de [195.37.77.164]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by relay04-haj2.antispameurope.com (ASE-Secure-MTA) with ESMTP id 2E5A65EC100; Sat, 11 Jul 2009 17:36:06 +0200 (CEST) Received: from EXCHSRV.fokus.fraunhofer.de (bohr.fokus.fraunhofer.de [10.147.9.231]) by pluto.fokus.fraunhofer.de (8.14.2/8.14.2) with SMTP id n6BFa58e000067; Sat, 11 Jul 2009 17:36:06 +0200 (MEST) Received: from rigel ([10.147.65.195]) by EXCHSRV.fokus.fraunhofer.de with Microsoft SMTPSVC(6.0.3790.3959); Sat, 11 Jul 2009 17:36:05 +0200 Date: Sat, 11 Jul 2009 17:36:05 +0200 From: Joerg.Schilling@fokus.fraunhofer.de (Joerg Schilling) Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A578763.40204@sun.com> Sender: Joerg.Schilling9ab33xy531fokus.fraunhofer.de@bounce.antispameurope.com To: robert.thurlow@sun.com, gdamore@sun.com Cc: PSARC-ext@sun.com, cifs-eng@sun.com, brian.wong@sun.com, Afshin.Ardakani@sun.com Message-id: <4a58b165.DhNHgqW5KNH/XOkD%Joerg.Schilling@fokus.fraunhofer.de> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1 Content-transfer-encoding: 8BIT X-PMX-Version: 5.4.1.325704 X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-Antispam: No, score=0.0/5.0, scanned in 1.974sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <4A57743F.5070708@Sun.COM> <4A577936.9050509@sun.com> <4A577CF4.9050702@sun.com> <4A578763.40204@sun.com> User-Agent: nail 11.22 3/20/05 X-OriginalArrivalTime: 11 Jul 2009 15:36:05.0851 (UTC) FILETIME=[4E023AB0:01CA023D] Content-Length: 1323 Status: RO X-Status: $$$$ X-UID: 0000000023 Robert Thurlow wrote: > Garrett D'Amore wrote: > > > I'm confused. Brian says that archivers Just Work with the current > > form, because the attributes are retained. Yet, you're saying that the > > attributes are not necessarily retained. Which is it? Right now, > > either way, you have an attribute... which I *think* means that the you > > need support (which may or may not be present) in the archivers. > > I know that some archivers in use today will not create a different > bitstream for a reparse point than for a regular symlink, but I > believe our intent to set the bit on restore or create means that > we don't have to teach all archivers about the new bit. If any > Sun-maintained archivers are aware, that's fine, but this should > work with GNU tar, too. As GNU tar does not support any platform specific feature even for Linux, I would not care about GNU tar..... It is more important to check whether the new feaures could confuse typical other existing applications. Jörg -- EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin js@cs.tu-berlin.de (uni) joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily From opensolaris-arc-bounces@opensolaris.org Sat Jul 11 12:19:38 2009 Return-Path: Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02 [129.146.11.31]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6BJJbQJ014968 for ; Sat, 11 Jul 2009 12:19:37 -0700 (PDT) Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6BJKmsK043593; Sat, 11 Jul 2009 12:20:48 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n6BJKZ7I040332; Sat, 11 Jul 2009 13:20:36 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMM00I05TQA9D00@nwk-avmta-1.sfbay.Sun.COM>; Sat, 11 Jul 2009 12:20:34 -0700 (PDT) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMM00CSUTQ9MA50@nwk-avmta-1.sfbay.Sun.COM>; Sat, 11 Jul 2009 12:20:34 -0700 (PDT) Received: from relay14i.sun.com (ip124.net129179-4.block1.us.syntegra.com [129.179.4.124]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6BJBxOo023810; Sat, 11 Jul 2009 19:20:33 +0000 (GMT) Received: from mmp13es.mmp.us.syntegra.com ([160.41.208.13] [160.41.208.13]) by relay14i.sun.com with ESMTP id BT-MMP-5179308; Sat, 11 Jul 2009 19:20:29 +0000 (Z) Received: from relay13i.sun.com (relay13i.sun.com [129.179.4.123]) by mmp13es.mmp.us.syntegra.com with ESMTP id BT-MMP-5520713; Sat, 11 Jul 2009 19:20:28 +0000 (Z) Received: from mail.opensolaris.org ([72.5.123.71] [72.5.123.71]) by relay1i.sun.com with ESMTP id BT-MMP-15041272; Sat, 11 Jul 2009 19:20:27 +0000 (Z) Received: by mail.opensolaris.org (Postfix, from userid 505) id 47776155CE4; Sat, 11 Jul 2009 12:20:27 -0700 (PDT) Received: from oss-mail1.opensolaris.org (localhost [127.0.0.1]) by mail.opensolaris.org (Postfix) with ESMTP id 47B36155CCF; Sat, 11 Jul 2009 12:20:20 -0700 (PDT) Received: by mail.opensolaris.org (Postfix, from userid 505) id 046AC155CB9; Sat, 11 Jul 2009 12:20:19 -0700 (PDT) Received: from b.mail.sonic.net (b.mail.sonic.net [64.142.19.5]) by mail.opensolaris.org (Postfix) with ESMTP id 09D97155CB6 for ; Sat, 11 Jul 2009 12:19:31 -0700 (PDT) Received: from webmail.sonic.net (b.webmail.sonic.net [64.142.100.148]) by b.mail.sonic.net (8.13.8.Beta0-Sonic/8.13.7) with ESMTP id n6BJJO8n029558; Sat, 11 Jul 2009 12:19:24 -0700 Received: from 76.191.129.144 (SquirrelMail authenticated user dcragun) by webmail.sonic.net with HTTP; Sat, 11 Jul 2009 12:19:24 -0700 (PDT) Date: Sat, 11 Jul 2009 12:19:24 -0700 (PDT) From: Don Cragun Subject: Re: 2009/387 [Pathname Reparse Points] Sender: opensolaris-arc-bounces@opensolaris.org To: opensolaris-arc@opensolaris.org Cc: cifs-eng@sun.com, Afshin.Ardakani@sun.com, Brian.Wong@sun.com Errors-to: opensolaris-arc-bounces@opensolaris.org Message-id: <7037.76.191.129.144.1247339964.squirrel@webmail.sonic.net> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Importance: Normal X-Priority: 3 (Normal) Precedence: list X-BeenThere: opensolaris-arc@opensolaris.org Delivered-to: opensolaris-arc@opensolaris.org X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on oss-mail1.opensolaris.org X-Original-To: opensolaris-arc@opensolaris.org X-Antispam: No, score=0.0/5.0, scanned in 0.075sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ X-Mailman-Version: 2.1.12 List-Post: List-Subscribe: , List-Unsubscribe: , List-Archive: List-Help: List-Id: Discussion of ARC issues affecting OpenSolaris User-Agent: SquirrelMail/1.4.9a X-Spam-Status: No, score=-3.8 required=5.0 tests=AWL,BAYES_00 autolearn=unavailable version=3.2.5 X-Spam-Level: Content-Length: 2273 Status: RO X-Status: $$$$ X-UID: 0000000024 On Fri, 10 Jul 2009 15:06:29 -0500 Nicolas Williams wrote: > On Fri, Jul 10, 2009 at 12:36:02PM -0700, Garrett D'Amore wrote: >> Nicolas Williams wrote: ... ... ... > > Also, I'm not sure that apps that read symlinks should have any > expectations that the contents of a symlink will be a filesystem path. > > Nico The standards place no requirements on what can be stored in a symbolic link as long as the symlink is never used for pathname resolution. However, when a symlink is used for pathname resolution, what is being proposed here does not conform. The standards say the contents of the symlink replace the filename of the symlink and resolution continues on the resulting path. (I'm ignoring the exceptions for resolving access to the symlink itself since they aren't important to this discussion.) Given that there is existing practice supporting other behaviors here, it shouldn't be hard to file an interpretation request to allow alternative symlink processing, but explicit text for the changes to the definition of pathname resolution would be required to get a successful interpretation and have the correct wording appear in the next revision of the standards. - Don PS The paragraph in the standard that would have to change to allow this can be found on P112, L3026-3031 of the 2008 version of the Single UNIX Specification and the IEEE and ISO POSIX standards where it currently says: "In all other cases, the system shall prefix the remaining pathname, if any, with the contents of the symbolic link If the combined length exceeds {PATH_MAX}, and the implementation considers this to be an error, errno shall be set to [ENAMETOOLONG] and an error indication shall be returned. Otherwise, the resolved pathname shall be the resolution of the pathname just created. If the resulting pathname does not begin with a , the predecessor of the first filename of the pathname is taken to be the directory containing the symbolic link." This is found in the definition of the General Concept "Pathname Resolution" in subclause 4.12. _______________________________________________ opensolaris-arc mailing list opensolaris-arc@opensolaris.org From amw@sun.com Sat Jul 11 18:59:37 2009 Return-Path: Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02 [129.146.11.31]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6C1xa3m015166 for ; Sat, 11 Jul 2009 18:59:36 -0700 (PDT) Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6C20kcZ026967 for ; Sat, 11 Jul 2009 19:00:47 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6C20iBY012935; Sun, 12 Jul 2009 03:00:45 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMN00E01C98XS00@nwk-avmta-2.sfbay.sun.com>; Sat, 11 Jul 2009 19:00:44 -0700 (PDT) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMN000JAC97T480@nwk-avmta-2.sfbay.sun.com>; Sat, 11 Jul 2009 19:00:44 -0700 (PDT) Received: from relay11i.sun.com (ip121.net129179-4.block1.us.syntegra.com [129.179.4.121]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6C20Jhm003045; Sun, 12 Jul 2009 02:00:43 +0000 (GMT) Received: from mmp14es.mmp.us.syntegra.com ([160.41.208.14] [160.41.208.14]) by relay11i.sun.com with ESMTP id BT-MMP-1152646; Sun, 12 Jul 2009 02:00:37 +0000 (Z) Received: from relay14i.sun.com (relay14i.sun.com [129.179.4.124]) by mmp14es.mmp.us.syntegra.com with ESMTP id BT-MMP-508199; Sun, 12 Jul 2009 02:00:37 +0000 (Z) Received: from fed1rmmtao101.cox.net ([68.230.241.45] [68.230.241.45]) by relay1i.sun.com with ESMTP id BT-MMP-3496331; Sun, 12 Jul 2009 02:00:36 +0000 (Z) Received: from fed1rmimpo02.cox.net ([70.169.32.72]) by fed1rmmtao101.cox.net (InterMail vM.7.08.02.01 201-2186-121-102-20070209) with ESMTP id <20090712020034.JRFQ17670.fed1rmmtao101.cox.net@fed1rmimpo02.cox.net>; Sat, 11 Jul 2009 22:00:34 -0400 Received: from [192.168.123.111] ([68.228.88.55]) by fed1rmimpo02.cox.net with bizsmtp id Eq0a1c00A1Bewo404q0aZb; Sat, 11 Jul 2009 22:00:35 -0400 Date: Sat, 11 Jul 2009 19:00:32 -0700 From: "Alan.M.Wright" Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4a58a697.yQ8anaL/V6TEAoRX%Joerg.Schilling@fokus.fraunhofer.de> To: Joerg Schilling Cc: PSARC-ext@sun.com, Glenn.Skinner@sun.com, cifs-eng@sun.com, Robert.Thurlow@sun.com, Brian.Wong@sun.com, Afshin.Ardakani@sun.com Message-id: <4A5943C0.3090303@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 8BIT X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-VR-Score: -240.00 X-Authority-Analysis: v=1.0 c=1 a=qvB4dunp4MIA:10 a=cexIBkohAAAA:8 a=lU_vBwmXAAAA:8 a=8HiYPw2Qb9dN19r0mkIA:9 a=nPuh7s_ftubbw_JsyGAA:7 a=uzX1sTco9YsDjk9Is7BIFiBAHoUA:4 a=KdsN9ty_cgEA:10 X-CM-Score: 0.00 X-Antispam: No, score=-2.6/5.0, scanned in 0.469sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <4a58a697.yQ8anaL/V6TEAoRX%Joerg.Schilling@fokus.fraunhofer.de> User-Agent: Thunderbird 2.0.0.22 (Windows/20090605) Content-Length: 4251 Status: RO X-Status: $$$$ X-UID: 0000000025 Joerg Schilling wrote: > Glenn Skinner wrote: > > >> 4. Technical Description: >> 4.1. Details: >> >> INTRODUCTION >> > ... > > >> To distinguish a regular symlink from a reparse point, an >> extensible system attribute will be set on the symlink. This >> system attribute is only one bit which indicates whether or >> not a symlink contains reparse data. >> > > How exactly will these "reparse points" be distinguished from symlinks? > Precisely as described in the paragraph you quoted. Otherwise, the assumption was that they would be treated as any other symlink. >> The reparse data will be stored as the link target. The >> reparse data is not in file system path format, which is the >> typical format of a link target. In order to avoid coming up >> with a totaly new format for reparse data as the link target >> we decided to adopt the format used by magic links in BSD: >> (http://www.daemon-systems.org/man/symlink.7.html) >> >> @{REPARSE@{service-type1:data} [@{service-type2:data}]...} >> >> Where some examples of service-type are: >> >> #define REPARSE_SVC_SMB "SMB" >> #define REPARSE_SVC_NFS "NFS" >> #define REPARSE_SVC_HSM "HSM" >> >> The data for each service will be in string format, which is >> expected to be typically a UUID string. >> >> The pattern above starts with "REPARSE" to distinguish it from >> a other magic links, such as those supported by BSD. Note >> that this case is not a proposal to support BSD magic links, >> the intent is to avoid precluding the future addition of full >> BSD magic link support. >> > > How will you prevent prople from removing these "reparse points" because they > asume that this are symlinks that point to nowhere? > Note that the official POSIX method for removing symlinks that do not point > to a target is: > > find -L . -type l -exec rm {} + > Symlinks containing reparse data can be removed in the same way as any other symlink. There are no special rules, conditions or restrictions. > > >> The symlink target size should be increased to 16K to >> accomodate the maximum size supported for MS-DFS referrals by >> Windows. Applications are expected to query the PATH_MAX and >> SYMLINK_MAX values on the local system using >> pathconf(2)/fpathconf(2). The value of SYMLINK_MAX would be >> changed to 16K on ZFS. The value of PATH_MAX will not be >> affected. >> > > Whill SYMLINK_MAX from limits.h be set to 16k? > No. > > >> Operation Flow Example >> >> Here is a simplified example of operation for a CIFS client >> that tries to access a file where the path contains a DFS >> link: >> >> a) Client tries to access \\srv\root\...\link\...\file.txt >> where: >> 'root' is a share (namespace root) >> 'link' is a reparse point seen as a folder by client >> > > Is this is a typo and you accidently used '\\' as path name separator instead > of the official path name separator '/'? > As described in the example, this is what a _CIFS_ client would send to a CIFS Server. CIFS clients are often Windows, which uses '\'. CIFS Servers handle slash translation such that '\' is not presented to the local operating system. > > >> NFS REFERRAL IN OTHER UNIXES >> >> FS referrals have been implemented in other major UNIX >> distributions such as Linux, AIX and HP-UX but there is no >> unified approach or implementation. >> >> Linux, AIX and HP-UX specify referrals as an NFS export >> option. The option format is basically the same in all three >> operating systems (refer=path@host) but the presentation is >> somewhat different in each case: >> >> - In Linux a referral is presented as a mount point. >> - In HP-UX a referral is a file system partition or logical volume. >> - In AIX a special object is used to represent a referral. >> > > Is this intended to be something like the conditional symbolic links that have > been in Apollo Domain system 23 years ago? > I'm not familiar with that proposal. Can you provide details? Thanks, Alan > Jörg > > From psarc-member-list-request@sun.com Sat Jul 11 19:08:53 2009 Return-Path: Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01 [129.145.155.118]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6C28qB4015178 for ; Sat, 11 Jul 2009 19:08:52 -0700 (PDT) Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6C2A3sc046924; Sat, 11 Jul 2009 19:10:03 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n6C29vjO024891; Sat, 11 Jul 2009 19:09:57 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMN00F05COLKA00@nwk-avmta-2.sfbay.sun.com> (ORCPT psarc-ext@sun.com) ; Sat, 11 Jul 2009 19:09:57 -0700 (PDT) Received: from dm-sfbay-01.sfbay.sun.com ([129.145.155.118]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMN000RWCOLSY80@nwk-avmta-2.sfbay.sun.com> (ORCPT psarc-ext@sun.com); Sat, 11 Jul 2009 19:09:57 -0700 (PDT) Received: from sac.sfbay.sun.com (sac.SFBay.Sun.COM [129.146.226.132]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6C29vtE046726; Sat, 11 Jul 2009 19:09:57 -0700 (PDT) Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6C28hkW026028 for ; Sat, 11 Jul 2009 19:08:44 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n6C28Iv9016367; Sun, 12 Jul 2009 10:08:36 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMN00801CM90B00@nwk-avmta-1.sfbay.Sun.COM>; Sat, 11 Jul 2009 19:08:33 -0700 (PDT) Received: from sca-ea-mail-3.sun.com ([192.18.43.21]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMN0033LCM81T50@nwk-avmta-1.sfbay.Sun.COM>; Sat, 11 Jul 2009 19:08:33 -0700 (PDT) Received: from relay43i.sun.com ([192.5.209.74]) by sca-ea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6C1vNf1020913; Sun, 12 Jul 2009 02:08:32 +0000 (GMT) Received: from mms49es.mms.us.syntegra.com ([160.41.221.232] [160.41.221.232]) by relay43i.sun.com with ESMTP id BT-MMP-1085955; Sun, 12 Jul 2009 02:08:32 +0000 (Z) Received: from relay45i.sun.com (relay45i.sun.com [192.5.209.94]) by mms49es.mms.us.syntegra.com with ESMTP id BT-MMP-80528025; Sun, 12 Jul 2009 02:08:31 +0000 (Z) Received: from fed1rmmtao107.cox.net ([68.230.241.39] [68.230.241.39]) by relay4i.sun.com with ESMTP id BT-MMP-3228060; Sun, 12 Jul 2009 02:08:31 +0000 (Z) Received: from fed1rmimpo01.cox.net ([70.169.32.71]) by fed1rmmtao107.cox.net (InterMail vM.7.08.02.01 201-2186-121-102-20070209) with ESMTP id <20090712020739.BPJV18948.fed1rmmtao107.cox.net@fed1rmimpo01.cox.net>; Sat, 11 Jul 2009 22:07:39 -0400 Received: from [192.168.123.111] ([68.228.88.55]) by fed1rmimpo01.cox.net with bizsmtp id Eq7f1c0071Bewo403q7fft; Sat, 11 Jul 2009 22:07:39 -0400 Date: Sat, 11 Jul 2009 19:07:37 -0700 From: "Alan.M.Wright" Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4a58a9a2.ZrWt8z8SX87Z2Bdt%Joerg.Schilling@fokus.fraunhofer.de> To: Joerg Schilling Cc: Scott.Rotondo@sun.com, Nicolas.Williams@sun.com, PSARC-ext@sun.com, Robert.Thurlow@sun.com, Brian.Wong@sun.com, Afshin.Ardakani@sun.com, cifs-eng@sun.com Message-id: <4A594569.3070503@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 8BIT X-PMX-Version: 5.4.1.325704 X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-VR-Score: -200.00 X-Authority-Analysis: v=1.0 c=1 a=qvB4dunp4MIA:10 a=cexIBkohAAAA:8 a=lOQ_V975MOdKVRDT6SQA:9 a=rEBxx_v2AD2EkgEXDRL0MJRri-cA:4 a=KdsN9ty_cgEA:10 X-CM-Score: 0.00 X-Antispam: No, score=0.0/5.0, scanned in 0.063sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <4A57743F.5070708@Sun.COM> <4A5777DF.9020904@sun.com> <20090710172923.GY1274@Sun.COM> <4a58a9a2.ZrWt8z8SX87Z2Bdt%Joerg.Schilling@fokus.fraunhofer.de> User-Agent: Thunderbird 2.0.0.22 (Windows/20090605) Content-Length: 1298 Status: RO X-Status: $$$$ X-UID: 0000000026 Joerg Schilling wrote: > Nicolas Williams wrote: > > >> On Fri, Jul 10, 2009 at 10:18:23AM -0700, Scott Rotondo wrote: >> >>> Darren J Moffat wrote: >>> >>>> This looks very cool but I haven't quite got my head around it >>>> completely yet. >>>> >>>> What happens if open(2) is called with O_NOFOLLOW set on one of these >>>> reparse points ? (Please answer for ZFS local access, NFS and CIFS). >>>> >>> Since these reparse points are implemented with a special type of >>> symlink, open() with O_NOFOLLOW should fail with such an object. >>> >> On the client side a server-side reparse point behaves like a mount >> point, very much in the same way as mirror mounts. >> >> Locally (on the server) a reparse point is stored in a symlink, but it >> isn't followed, and if it were then it'd behave like a mount, just as on >> the client side. >> >> There's nothing quite like following a symlink here, therefore >> O_NOFOLLOW shouldn't apply on the client side. >> > > If the feature is implemented as symlink, how will stat() vs. lstat() perform > on these objects? Will it confuse existing applications? > They will behave exactly as they do today if you were to define a symlink containing garbage text. Alan > Jörg > > From psarc-member-list-request@sun.com Sat Jul 11 19:12:39 2009 Return-Path: Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02 [129.146.11.31]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6C2CdWK015185 for ; Sat, 11 Jul 2009 19:12:39 -0700 (PDT) Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6C2Dnnj030925; Sat, 11 Jul 2009 19:13:50 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6C2Deui018777; Sun, 12 Jul 2009 03:13:41 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMN00F01CUSU600@nwk-avmta-2.sfbay.sun.com> (ORCPT psarc-ext@sun.com) ; Sat, 11 Jul 2009 19:13:40 -0700 (PDT) Received: from dm-sfbay-02.sfbay.sun.com ([129.146.11.31]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMN000WOCUSSY80@nwk-avmta-2.sfbay.sun.com> (ORCPT psarc-ext@sun.com); Sat, 11 Jul 2009 19:13:40 -0700 (PDT) Received: from sac.sfbay.sun.com (sac.SFBay.Sun.COM [129.146.226.132]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6C2DdCI030893; Sat, 11 Jul 2009 19:13:39 -0700 (PDT) Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6C2CEGB026070 for ; Sat, 11 Jul 2009 19:12:14 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6C2C8oi018168; Sun, 12 Jul 2009 03:12:09 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMN00F09CS8RE00@nwk-avmta-2.sfbay.sun.com>; Sat, 11 Jul 2009 19:12:08 -0700 (PDT) Received: from brmea-mail-2.sun.com ([192.18.98.43]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMN000ZYCS7T480@nwk-avmta-2.sfbay.sun.com>; Sat, 11 Jul 2009 19:12:07 -0700 (PDT) Received: from relay42i.sun.com ([192.5.209.72]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6C27HuY021441; Sun, 12 Jul 2009 02:12:07 +0000 (GMT) Received: from mmp42es.mmp.us.syntegra.com ([160.41.221.11] [160.41.221.11]) by relay42i.sun.com with ESMTP id BT-MMP-1206919; Sun, 12 Jul 2009 02:12:06 +0000 (Z) Received: from relay41i.sun.com (relay41i.sun.com [192.5.209.70]) by mmp42es.mmp.us.syntegra.com with ESMTP id BT-MMP-80946139; Sun, 12 Jul 2009 02:12:05 +0000 (Z) Received: from fed1rmmtao101.cox.net ([68.230.241.45] [68.230.241.45]) by relay4i.sun.com with ESMTP id BT-MMP-19744266; Sun, 12 Jul 2009 02:12:05 +0000 (Z) Received: from fed1rmimpo01.cox.net ([70.169.32.71]) by fed1rmmtao101.cox.net (InterMail vM.7.08.02.01 201-2186-121-102-20070209) with ESMTP id <20090712021113.KATR17670.fed1rmmtao101.cox.net@fed1rmimpo01.cox.net>; Sat, 11 Jul 2009 22:11:13 -0400 Received: from [192.168.123.111] ([68.228.88.55]) by fed1rmimpo01.cox.net with bizsmtp id EqBE1c0011Bewo403qBEAt; Sat, 11 Jul 2009 22:11:14 -0400 Date: Sat, 11 Jul 2009 19:11:11 -0700 From: "Alan.M.Wright" Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4a58ab90.X2ncDHPKuL+7EDtV%Joerg.Schilling@fokus.fraunhofer.de> To: Joerg Schilling Cc: Robert.Thurlow@sun.com, gdamore@sun.com, PSARC-ext@sun.com, Afshin.Ardakani@sun.com, Brian.Wong@sun.com, cifs-eng@sun.com Message-id: <4A59463F.9040003@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 8BIT X-PMX-Version: 5.4.1.325704 X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-VR-Score: -210.00 X-Authority-Analysis: v=1.0 c=1 a=qvB4dunp4MIA:10 a=cexIBkohAAAA:8 a=cgDInRuswX8wTupgxDMA:9 a=txj2BKt-LOVRys1a-CUA:7 a=--d7QlIjYMtebRVlpdtYViJqQNgA:4 a=KdsN9ty_cgEA:10 X-CM-Score: 0.00 X-Antispam: No, score=-0.7/5.0, scanned in 0.066sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <4A57743F.5070708@Sun.COM> <4A577936.9050509@sun.com> <4A577CF4.9050702@sun.com> <4a58ab90.X2ncDHPKuL+7EDtV%Joerg.Schilling@fokus.fraunhofer.de> User-Agent: Thunderbird 2.0.0.22 (Windows/20090605) Content-Length: 1746 Status: RO X-Status: $$$$ X-UID: 0000000027 Joerg Schilling wrote: > "Garrett D'Amore" wrote: > > >>> Archivers that slurp and spit the symlink contents will work >>> without mods as long as they get all of the bytes, but would >>> need more extensive modifications if our storage was in a >>> system attribute. Also, we can get the single bit we need >>> in ZFS now, and a 16K sysattr will not be supportable for a >>> few more months. >>> >> I'm confused. Brian says that archivers Just Work with the current >> form, because the attributes are retained. Yet, you're saying that the >> attributes are not necessarily retained. Which is it? Right now, >> either way, you have an attribute... which I *think* means that the you >> need support (which may or may not be present) in the archivers. >> > > If these objects will be seen as symlink file type and in case they cannot > be copied using symlink(), I expect problems. > You're trying to draw a distinction between this proposal and current symlink behavior but there is no distinction. To existing applications, these will appear like symlinks that don't resolve to an existing target. Alan > I beleve we need to get more information in order to be able to check whether > the case makes a POSIX compliant proposal. > > And BTW, I like to remind people to the "well planned" Linux implementation for > "file flags" that forces archivesrs like star to open() _all_ files in search > for file flags. If I am going to backup /dev/, I am forced to open e.g. the > "rewind" type of a tape drive entry and thus forcing the tape to rewind just for > checking for fflags. > > So please give more information on how to distinguish these objects from > vanilla symlinks. > > Jörg > > From opensolaris-arc-bounces@opensolaris.org Sat Jul 11 19:20:39 2009 Return-Path: Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01 [129.145.155.118]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6C2KcjY015195 for ; Sat, 11 Jul 2009 19:20:38 -0700 (PDT) Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6C2LmwQ050566; Sat, 11 Jul 2009 19:21:49 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6C2LdJc021926; Sun, 12 Jul 2009 03:21:39 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMN00G01D83DX00@nwk-avmta-2.sfbay.sun.com>; Sat, 11 Jul 2009 19:21:39 -0700 (PDT) Received: from sca-ea-mail-3.sun.com ([192.18.43.21]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMN000AKD82T490@nwk-avmta-2.sfbay.sun.com>; Sat, 11 Jul 2009 19:21:38 -0700 (PDT) Received: from relay43i.sun.com ([192.5.209.74]) by sca-ea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6C28tC3022314; Sun, 12 Jul 2009 02:21:37 +0000 (GMT) Received: from mmp43es.mmp.us.syntegra.com ([160.41.221.12] [160.41.221.12]) by relay43i.sun.com with ESMTP id BT-MMP-1086267; Sun, 12 Jul 2009 02:21:37 +0000 (Z) Received: from relay45i.sun.com (relay45i.sun.com [192.5.209.94]) by mmp43es.mmp.us.syntegra.com with ESMTP id BT-MMP-28145299; Sun, 12 Jul 2009 02:21:34 +0000 (Z) Received: from mail.opensolaris.org ([72.5.123.71] [72.5.123.71]) by relay4i.sun.com with ESMTP id BT-MMP-3245042; Sun, 12 Jul 2009 02:21:34 +0000 (Z) Received: by mail.opensolaris.org (Postfix, from userid 505) id 257691516DD; Sat, 11 Jul 2009 19:21:34 -0700 (PDT) Received: from oss-mail1.opensolaris.org (localhost [127.0.0.1]) by mail.opensolaris.org (Postfix) with ESMTP id ADD811516BF; Sat, 11 Jul 2009 19:21:26 -0700 (PDT) Received: by mail.opensolaris.org (Postfix, from userid 505) id A10F91516B6; Sat, 11 Jul 2009 19:21:25 -0700 (PDT) Received: from fed1rmmtao106.cox.net (fed1rmmtao106.cox.net [68.230.241.40]) by mail.opensolaris.org (Postfix) with ESMTP id 5BECC1516B3 for ; Sat, 11 Jul 2009 19:20:47 -0700 (PDT) Received: from fed1rmimpo03.cox.net ([70.169.32.75]) by fed1rmmtao106.cox.net (InterMail vM.7.08.02.01 201-2186-121-102-20070209) with ESMTP id <20090712022046.DIYW25927.fed1rmmtao106.cox.net@fed1rmimpo03.cox.net>; Sat, 11 Jul 2009 22:20:46 -0400 Received: from [192.168.123.111] ([68.228.88.55]) by fed1rmimpo03.cox.net with bizsmtp id EqLm1c00D1Bewo404qLmAt; Sat, 11 Jul 2009 22:20:47 -0400 Date: Sat, 11 Jul 2009 19:20:43 -0700 From: "Alan.M.Wright" Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <7037.76.191.129.144.1247339964.squirrel@webmail.sonic.net> Sender: opensolaris-arc-bounces@opensolaris.org To: Don Cragun Cc: opensolaris-arc@opensolaris.org, cifs-eng@sun.com, Afshin.Ardakani@sun.com, Brian.Wong@sun.com Errors-to: opensolaris-arc-bounces@opensolaris.org Message-id: <4A59487B.3080202@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii; Format=flowed Content-transfer-encoding: 7BIT Precedence: list X-BeenThere: opensolaris-arc@opensolaris.org Delivered-to: opensolaris-arc@opensolaris.org X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on oss-mail1.opensolaris.org X-Original-To: opensolaris-arc@opensolaris.org X-VR-Score: -200.00 X-Authority-Analysis: v=1.0 c=1 a=qvB4dunp4MIA:10 a=CH5Yqcik-SDhM_kaKK8A:9 a=Qi4mTLqkC-XYTCBtz6EA:7 a=LGpMKP6J4t1QJ6c9sR5P2p4qDPwA:4 a=4fIDtCzjks_NFkp9:21 a=W5953m6HrUGhwjab:21 X-CM-Score: 0.00 X-Antispam: No, score=-1.1/5.0, scanned in 0.088sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <7037.76.191.129.144.1247339964.squirrel@webmail.sonic.net> X-Mailman-Version: 2.1.12 List-Post: List-Subscribe: , List-Unsubscribe: , List-Archive: List-Help: List-Id: Discussion of ARC issues affecting OpenSolaris User-Agent: Thunderbird 2.0.0.22 (Windows/20090605) X-Spam-Status: No, score=-4.4 required=5.0 tests=AWL,BAYES_00 autolearn=unavailable version=3.2.5 X-Spam-Level: Content-Length: 2917 Status: RO X-Status: $$$$ X-UID: 0000000028 Don Cragun wrote: > On Fri, 10 Jul 2009 15:06:29 -0500 Nicolas Williams wrote: > >> On Fri, Jul 10, 2009 at 12:36:02PM -0700, Garrett D'Amore wrote: >> >>> Nicolas Williams wrote: >>> > ... ... ... > >> Also, I'm not sure that apps that read symlinks should have any >> expectations that the contents of a symlink will be a filesystem path. >> >> Nico >> > > The standards place no requirements on what can be stored in a symbolic > link as long as the symlink is never used for pathname resolution. > However, when a symlink is used for pathname resolution, what is > being proposed here does not conform. The standards say the contents > of the symlink replace the filename of the symlink and resolution > continues on the resulting path. (I'm ignoring the exceptions for > resolving access to the symlink itself since they aren't important > to this discussion.) > > Given that there is existing practice supporting other behaviors here, > it shouldn't be hard to file an interpretation request to allow alternative > symlink processing, but explicit text for the changes to the definition of > pathname resolution would be required to get a successful interpretation > and have the correct wording appear in the next revision of the standards. > > - Don > > PS The paragraph in the standard that would have to change to allow > this can be found on P112, L3026-3031 of the 2008 version of the > Single UNIX Specification and the IEEE and ISO POSIX standards > where it currently says: > "In all other cases, the system shall prefix the remaining > pathname, if any, with the contents of the symbolic link > If the combined length exceeds {PATH_MAX}, and the > implementation considers this to be an error, errno shall be > set to [ENAMETOOLONG] and an error indication shall > be returned. Otherwise, the resolved pathname shall be the > resolution of the pathname just created. If the resulting > pathname does not begin with a , the predecessor > of the first filename of the pathname is taken to be the > directory containing the symbolic link." > This is found in the definition of the General Concept "Pathname > Resolution" in subclause 4.12. > That would seem to resolve the PATH_MAX question asked earlier since the length interpretation is implementation dependent. An interpretation change request assumes that we would want this to be adopted as a standard, which may be desirable, but this proposal doesn't rely on reparse data being used for local path resolution (at this time). This proposal is intended to support reparse point aware services. Regular applications should be unaware that the text is anything other than a dangling symlink. Alan _______________________________________________ opensolaris-arc mailing list opensolaris-arc@opensolaris.org From psarc-member-list-request@sun.com Sat Jul 11 19:54:18 2009 Return-Path: Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01 [129.145.155.118]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6C2sIiA015212 for ; Sat, 11 Jul 2009 19:54:18 -0700 (PDT) Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6C2tSJT059984; Sat, 11 Jul 2009 19:55:28 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n6C2tBre007991; Sun, 12 Jul 2009 10:55:14 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMN00I03ERZSK00@nwk-avmta-2.sfbay.sun.com> (ORCPT psarc-ext@sac.sfbay.sun.com); Sat, 11 Jul 2009 19:55:11 -0700 (PDT) Received: from dm-sfbay-01.sfbay.sun.com ([129.145.155.118]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMN0009FERYSVA0@nwk-avmta-2.sfbay.sun.com> (ORCPT psarc-ext@sac.sfbay.sun.com); Sat, 11 Jul 2009 19:55:10 -0700 (PDT) Received: from sac.sfbay.sun.com (sac.SFBay.Sun.COM [129.146.226.132]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6C2t9f3059946; Sat, 11 Jul 2009 19:55:09 -0700 (PDT) Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6C2rlta026240 for ; Sat, 11 Jul 2009 19:53:48 -0700 (PDT) Received: from brmea-mail-1.sun.com (brmea-mail-1.Sun.COM [192.18.98.31]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6C2rlpP041671 for ; Sat, 11 Jul 2009 19:53:47 -0700 (PDT) Received: from relay42i.sun.com ([192.5.209.72]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6C2rlls018260 for ; Sun, 12 Jul 2009 02:53:47 +0000 (GMT) Received: from mmp43es.mmp.us.syntegra.com ([160.41.221.12] [160.41.221.12]) by relay42i.sun.com with ESMTP id BT-MMP-1207869 for psarc-ext@sac.sfbay.sun.com; Sun, 12 Jul 2009 02:51:46 +0000 (Z) Received: from relay41i.sun.com (relay41i.sun.com [192.5.209.70]) by mmp43es.mmp.us.syntegra.com with ESMTP id BT-MMP-28176422; Sun, 12 Jul 2009 02:51:46 +0000 (Z) Received: from b.mail.sonic.net ([64.142.19.5] [64.142.19.5]) by relay4i.sun.com with ESMTP id BT-MMP-19791744; Sun, 12 Jul 2009 02:51:45 +0000 (Z) Received: from [10.0.0.10] (76-191-129-144.dsl.dynamic.sonic.net [76.191.129.144]) (authenticated bits=0) by b.mail.sonic.net (8.13.8.Beta0-Sonic/8.13.7) with ESMTP id n6C2pjtu003244; Sat, 11 Jul 2009 19:51:45 -0700 Date: Sat, 11 Jul 2009 19:51:43 -0700 From: Don Cragun Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A59487B.3080202@Sun.COM> To: "Alan.M.Wright" Cc: psarc-ext@sac.sfbay.sun.com, cifs-eng@sun.com, Afshin.Ardakani@sun.com, Brian.Wong@sun.com Message-id: <4A594FBF.8060809@sonic.net> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-Antispam: No, score=-2.6/5.0, scanned in 0.093sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <7037.76.191.129.144.1247339964.squirrel@webmail.sonic.net> <4A59487B.3080202@Sun.COM> User-Agent: Thunderbird 2.0.0.22 (Windows/20090605) Content-Length: 4126 Status: RO X-Status: $$$$ X-UID: 0000000029 Alan, I'm resending this since I used the wrong alias for PSARC and none of this discussion has been recorded in the formal case log. Please find additional comments below... Alan.M.Wright wrote: > Don Cragun wrote: >> On Fri, 10 Jul 2009 15:06:29 -0500 Nicolas Williams wrote: >> >>> On Fri, Jul 10, 2009 at 12:36:02PM -0700, Garrett D'Amore wrote: >>> >>>> Nicolas Williams wrote: >>>> >> ... ... ... >> >>> Also, I'm not sure that apps that read symlinks should have any >>> expectations that the contents of a symlink will be a filesystem path. >>> >>> Nico >>> >> >> The standards place no requirements on what can be stored in a symbolic >> link as long as the symlink is never used for pathname resolution. >> However, when a symlink is used for pathname resolution, what is >> being proposed here does not conform. The standards say the contents >> of the symlink replace the filename of the symlink and resolution >> continues on the resulting path. (I'm ignoring the exceptions for >> resolving access to the symlink itself since they aren't important >> to this discussion.) >> >> Given that there is existing practice supporting other behaviors here, >> it shouldn't be hard to file an interpretation request to allow >> alternative >> symlink processing, but explicit text for the changes to the >> definition of >> pathname resolution would be required to get a successful interpretation >> and have the correct wording appear in the next revision of the >> standards. >> >> - Don >> >> PS The paragraph in the standard that would have to change to allow >> this can be found on P112, L3026-3031 of the 2008 version of the >> Single UNIX Specification and the IEEE and ISO POSIX standards >> where it currently says: >> "In all other cases, the system shall prefix the remaining >> pathname, if any, with the contents of the symbolic link >> If the combined length exceeds {PATH_MAX}, and the >> implementation considers this to be an error, errno shall be >> set to [ENAMETOOLONG] and an error indication shall >> be returned. Otherwise, the resolved pathname shall be the >> resolution of the pathname just created. If the resulting >> pathname does not begin with a , the predecessor >> of the first filename of the pathname is taken to be the >> directory containing the symbolic link." >> This is found in the definition of the General Concept "Pathname >> Resolution" in subclause 4.12. >> > > That would seem to resolve the PATH_MAX question asked earlier since > the length interpretation is implementation dependent. > > An interpretation change request assumes that we would want this to be > adopted as a standard, which may be desirable, but this proposal doesn't > rely on reparse data being used for local path resolution (at this time). > This proposal is intended to support reparse point aware services. > Regular applications should be unaware that the text is anything other than > a dangling symlink. > > Alan Note that Joerg's sample find command is commonly used by system administrators on some systems to get rid of dangling symlinks on their systems. He was just pointing out that symlinks containing reparse data will be removed by this common script. I missed the fact that you never expect one of these symlinks to be resolved on a[n] [Open]Solaris system. If that is true, you won't need an interpretation to keep a verification suite from saying the system does not correctly process symbolic links if you ever plan to get a UNIX brand or POSIX certification for Solaris next. The comment in the quote about PATH_MAX is that the standard allows PATH_MAX to be unlimited. Solaris systems do have a pathname length limit and the length is limited to PATH_MAX bytes. In cases where symlinks will not be resolved, there need not be any relationship between the maximum number of bytes held in a symlink and PATH_MAX. The only part that is implementation dependent is whether there is a maximum pathname length or not. - Don From psarc-member-list-request@sun.com Sun Jul 12 03:25:27 2009 Return-Path: Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01 [129.145.155.118]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6CAPQ3C015598 for ; Sun, 12 Jul 2009 03:25:26 -0700 (PDT) Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6CAQZG6008080; Sun, 12 Jul 2009 03:26:36 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n6CAQTEP022012; Sun, 12 Jul 2009 03:26:29 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMN00E05ZO57400@brm-avmta-1.central.sun.com> (ORCPT PSARC-ext@sun.com) ; Sun, 12 Jul 2009 04:26:29 -0600 (MDT) Received: from dm-sfbay-01.sfbay.sun.com ([129.145.155.118]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMN007O2ZO4OYE0@brm-avmta-1.central.sun.com> (ORCPT PSARC-ext@sun.com); Sun, 12 Jul 2009 04:26:28 -0600 (MDT) Received: from sac.sfbay.sun.com (sac.SFBay.Sun.COM [129.146.226.132]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6CAQRdo008041; Sun, 12 Jul 2009 03:26:28 -0700 (PDT) Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6CAGC72009032 for ; Sun, 12 Jul 2009 03:16:14 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n6CAFxSt008188 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Sun, 12 Jul 2009 18:16:11 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMN00D03Z6Y4W00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Sun, 12 Jul 2009 04:16:10 -0600 (MDT) Received: from sca-ea-mail-3.sun.com ([192.18.43.21]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMN0072OZ6YOYC0@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Sun, 12 Jul 2009 04:16:10 -0600 (MDT) Received: from relay15i.sun.com (ip125.net129179-4.block1.us.syntegra.com [129.179.4.125]) by sca-ea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6CA1wF3017976 for ; Sun, 12 Jul 2009 10:16:09 +0000 (GMT) Received: from mmp11es.mmp.us.syntegra.com ([160.41.208.11] [160.41.208.11]) by relay15i.sun.com with ESMTP id BT-MMP-393003 for PSARC-ext@sun.com; Sun, 12 Jul 2009 10:16:09 +0000 (Z) Received: from relay13i.sun.com (relay13i.sun.com [129.179.4.123]) by mmp11es.mmp.us.syntegra.com with ESMTP id BT-MMP-7013970 for PSARC-ext@sun.com; Sun, 12 Jul 2009 10:16:06 +0000 (Z) Received: from relay04-haj2.antispameurope.com ([83.246.65.54] [83.246.65.54]) by relay1i.sun.com with ESMTP id BT-MMP-16641050 for PSARC-ext@sun.com; Sun, 12 Jul 2009 10:16:06 +0000 (Z) Received: by relay04-haj2.antispameurope.com (ASE-Secure-MTA, from userid 1000) id EE91C5EC146; Sun, 12 Jul 2009 12:16:05 +0200 (CEST) Received: from pluto.fokus.fraunhofer.de (pluto.fokus.fraunhofer.de [195.37.77.164]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by relay04-haj2.antispameurope.com (ASE-Secure-MTA) with ESMTP id 0B25D5EC146; Sun, 12 Jul 2009 12:16:05 +0200 (CEST) Received: from EXCHSRV.fokus.fraunhofer.de (bohr.fokus.fraunhofer.de [10.147.9.231]) by pluto.fokus.fraunhofer.de (8.14.2/8.14.2) with SMTP id n6CAG45V025622; Sun, 12 Jul 2009 12:16:04 +0200 (MEST) Received: from rigel ([10.147.65.195]) by EXCHSRV.fokus.fraunhofer.de with Microsoft SMTPSVC(6.0.3790.3959); Sun, 12 Jul 2009 12:16:04 +0200 Date: Sun, 12 Jul 2009 12:16:04 +0200 From: Joerg.Schilling@fokus.fraunhofer.de (Joerg Schilling) Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A59463F.9040003@Sun.COM> Sender: Joerg.Schilling9ab33xy531fokus.fraunhofer.de@bounce.antispameurope.com To: amw@sun.com Cc: Robert.Thurlow@sun.com, PSARC-ext@sun.com, gdamore@sun.com, cifs-eng@sun.com, Brian.Wong@sun.com, Afshin.Ardakani@sun.com Message-id: <4a59b7e4.etj9O4PGGKpK8Wtv%Joerg.Schilling@fokus.fraunhofer.de> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1 Content-transfer-encoding: 8BIT X-PMX-Version: 5.4.1.325704 X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-Antispam: No, score=0.0/5.0, scanned in 2.128sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <4A57743F.5070708@Sun.COM> <4A577936.9050509@sun.com> <4A577CF4.9050702@sun.com> <4a58ab90.X2ncDHPKuL+7EDtV%Joerg.Schilling@fokus.fraunhofer.de> <4A59463F.9040003@Sun.COM> User-Agent: nail 11.22 3/20/05 X-OriginalArrivalTime: 12 Jul 2009 10:16:04.0378 (UTC) FILETIME=[C373B7A0:01CA02D9] Content-Length: 1659 Status: RO X-Status: $$$$ X-UID: 0000000030 "Alan.M.Wright" wrote: > Joerg Schilling wrote: > > "Garrett D'Amore" wrote: > > > > > >>> Archivers that slurp and spit the symlink contents will work > >>> without mods as long as they get all of the bytes, but would > >>> need more extensive modifications if our storage was in a > >>> system attribute. Also, we can get the single bit we need > >>> in ZFS now, and a 16K sysattr will not be supportable for a > >>> few more months. > >>> > >> I'm confused. Brian says that archivers Just Work with the current > >> form, because the attributes are retained. Yet, you're saying that the > >> attributes are not necessarily retained. Which is it? Right now, > >> either way, you have an attribute... which I *think* means that the you > >> need support (which may or may not be present) in the archivers. > >> > > > > If these objects will be seen as symlink file type and in case they cannot > > be copied using symlink(), I expect problems. > > > > You're trying to draw a distinction between this proposal and current > symlink behavior but there is no distinction. To existing applications, > these will appear like symlinks that don't resolve to an existing target. You did not answer my question: is it possible to "correctly" copy such an object by using lstat(), readlink() and symlink()? Jörg -- EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin js@cs.tu-berlin.de (uni) joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily From Joerg.Schilling9ab33xy531fokus.fraunhofer.de@bounce.antispameurope.com Sun Jul 12 03:28:52 2009 Return-Path: Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01 [129.145.155.118]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6CASple015606 for ; Sun, 12 Jul 2009 03:28:51 -0700 (PDT) Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6CAU0tT009015 for ; Sun, 12 Jul 2009 03:30:00 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n6CAU0He011759 for <@sunmail2sca.sfbay.sun.com:Glenn.Skinner@sun.com>; Sun, 12 Jul 2009 03:30:00 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMN00E03ZTWJL00@brm-avmta-1.central.sun.com> for Glenn.Skinner@sun.com (ORCPT Glenn.Skinner@sun.com); Sun, 12 Jul 2009 04:29:56 -0600 (MDT) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMN007DWZTWOYF0@brm-avmta-1.central.sun.com> for Glenn.Skinner@sun.com (ORCPT Glenn.Skinner@sun.com); Sun, 12 Jul 2009 04:29:56 -0600 (MDT) Received: from relay13i.sun.com (ip123.net129179-4.block1.us.syntegra.com [129.179.4.123]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6CAIGUW010832 for ; Sun, 12 Jul 2009 10:29:56 +0000 (GMT) Received: from mmp14es.mmp.us.syntegra.com ([160.41.208.14] [160.41.208.14]) by relay13i.sun.com with ESMTP id BT-MMP-4395589 for Glenn.Skinner@sun.com; Sun, 12 Jul 2009 10:29:56 +0000 (Z) Received: from relay14i.sun.com (relay14i.sun.com [129.179.4.124]) by mmp14es.mmp.us.syntegra.com with ESMTP id BT-MMP-147119 for Glenn.Skinner@sun.com; Sun, 12 Jul 2009 10:29:55 +0000 (Z) Received: from relay03-haj2.antispameurope.com ([83.246.65.53] [83.246.65.53]) by relay1i.sun.com with ESMTP id BT-MMP-4367378 for Glenn.Skinner@sun.com; Sun, 12 Jul 2009 10:29:55 +0000 (Z) Received: by relay03-haj2.antispameurope.com (ASE-Secure-MTA, from userid 1000) id 6632E63C088; Sun, 12 Jul 2009 12:29:54 +0200 (CEST) Received: from pluto.fokus.fraunhofer.de (pluto.fokus.fraunhofer.de [195.37.77.164]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by relay03-haj2.antispameurope.com (ASE-Secure-MTA) with ESMTP id 2392663C088; Sun, 12 Jul 2009 12:29:53 +0200 (CEST) Received: from EXCHSRV.fokus.fraunhofer.de (bohr.fokus.fraunhofer.de [10.147.9.231]) by pluto.fokus.fraunhofer.de (8.14.2/8.14.2) with SMTP id n6CATqhp025789; Sun, 12 Jul 2009 12:29:52 +0200 (MEST) Received: from rigel ([10.147.65.195]) by EXCHSRV.fokus.fraunhofer.de with Microsoft SMTPSVC(6.0.3790.3959); Sun, 12 Jul 2009 12:29:52 +0200 Date: Sun, 12 Jul 2009 12:29:52 +0200 From: Joerg.Schilling@fokus.fraunhofer.de (Joerg Schilling) Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A5943C0.3090303@Sun.COM> Sender: Joerg.Schilling9ab33xy531fokus.fraunhofer.de@bounce.antispameurope.com To: amw@sun.com Cc: Robert.Thurlow@sun.com, PSARC-ext@sun.com, Glenn.Skinner@sun.com, cifs-eng@sun.com, Brian.Wong@sun.com, Afshin.Ardakani@sun.com Message-id: <4a59bb20.CuZiqCKUmLZgrA4L%Joerg.Schilling@fokus.fraunhofer.de> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1 Content-transfer-encoding: 8BIT X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-Antispam: No, score=-0.2/5.0, scanned in 0.416sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <4a58a697.yQ8anaL/V6TEAoRX%Joerg.Schilling@fokus.fraunhofer.de> <4A5943C0.3090303@Sun.COM> User-Agent: nail 11.22 3/20/05 X-OriginalArrivalTime: 12 Jul 2009 10:29:52.0915 (UTC) FILETIME=[B14C6E30:01CA02DB] Content-Length: 2222 Status: RO X-Status: $$$$ X-UID: 0000000031 "Alan.M.Wright" wrote: > > > >> To distinguish a regular symlink from a reparse point, an > >> extensible system attribute will be set on the symlink. This > >> system attribute is only one bit which indicates whether or > >> not a symlink contains reparse data. > >> > > > > How exactly will these "reparse points" be distinguished from symlinks? > > > > Precisely as described in the paragraph you quoted. > Otherwise, the assumption was that they would be treated > as any other symlink. This paragraph does not contain the information you claim: "man -k attrubute" does not lead me to useful information. > > How will you prevent prople from removing these "reparse points" because they > > asume that this are symlinks that point to nowhere? > > Note that the official POSIX method for removing symlinks that do not point > > to a target is: > > > > find -L . -type l -exec rm {} + > > > > Symlinks containing reparse data can be removed in the same way as > any other symlink. There are no special rules, conditions or restrictions. So you admit that people who intend to remove garbage symlinks will accidentaly also remove these objects? > >> The symlink target size should be increased to 16K to > >> accomodate the maximum size supported for MS-DFS referrals by > >> Windows. Applications are expected to query the PATH_MAX and > >> SYMLINK_MAX values on the local system using > >> pathconf(2)/fpathconf(2). The value of SYMLINK_MAX would be > >> changed to 16K on ZFS. The value of PATH_MAX will not be > >> affected. > >> > > > > Whill SYMLINK_MAX from limits.h be set to 16k? > > > > No. So I expect existing applications to fail. From what I did read on this topic, I expect problems with existing applications. I would like to read a description on what happens with typical existing applications, if this new feature is implemented. Jörg -- EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin js@cs.tu-berlin.de (uni) joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily From opensolaris-arc-bounces@opensolaris.org Sun Jul 12 04:04:34 2009 Return-Path: Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02 [129.146.11.31]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6CB4Yqj015629 for ; Sun, 12 Jul 2009 04:04:34 -0700 (PDT) Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6CB5gWq058536; Sun, 12 Jul 2009 04:05:42 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n6CB5OHd001070; Sun, 12 Jul 2009 19:05:26 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMO000051H1NS00@nwk-avmta-1.sfbay.Sun.COM>; Sun, 12 Jul 2009 04:05:25 -0700 (PDT) Received: from sca-ea-mail-1.sun.com ([192.18.43.24]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMO006P41H1K2C0@nwk-avmta-1.sfbay.Sun.COM>; Sun, 12 Jul 2009 04:05:25 -0700 (PDT) Received: from relay15i.sun.com (ip125.net129179-4.block1.us.syntegra.com [129.179.4.125]) by sca-ea-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n6CAxaKR019402; Sun, 12 Jul 2009 11:05:24 +0000 (GMT) Received: from mmp13es.mmp.us.syntegra.com ([160.41.208.13] [160.41.208.13]) by relay15i.sun.com with ESMTP id BT-MMP-394641; Sun, 12 Jul 2009 11:05:24 +0000 (Z) Received: from relay13i.sun.com (relay13i.sun.com [129.179.4.123]) by mmp13es.mmp.us.syntegra.com with ESMTP id BT-MMP-7103372; Sun, 12 Jul 2009 11:05:21 +0000 (Z) Received: from mail.opensolaris.org ([72.5.123.71] [72.5.123.71]) by relay1i.sun.com with ESMTP id BT-MMP-16741374; Sun, 12 Jul 2009 11:05:20 +0000 (Z) Received: by mail.opensolaris.org (Postfix, from userid 505) id EC856157337; Sun, 12 Jul 2009 04:05:19 -0700 (PDT) Received: from oss-mail1.opensolaris.org (localhost [127.0.0.1]) by mail.opensolaris.org (Postfix) with ESMTP id 4112915731E; Sun, 12 Jul 2009 04:05:10 -0700 (PDT) Received: by mail.opensolaris.org (Postfix, from userid 505) id 26A48157315; Sun, 12 Jul 2009 04:05:09 -0700 (PDT) Received: from relay02-haj2.antispameurope.com (relay02-haj2.antispameurope.com [83.246.65.52]) by mail.opensolaris.org (Postfix) with ESMTP id BAD58157312 for ; Sun, 12 Jul 2009 04:04:17 -0700 (PDT) Received: by relay02-haj2.antispameurope.com (ASE-Secure-MTA, from userid 1000) id 5F8036F0570; Sun, 12 Jul 2009 13:04:14 +0200 (CEST) Received: from pluto.fokus.fraunhofer.de (pluto.fokus.fraunhofer.de [195.37.77.164]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by relay02-haj2.antispameurope.com (ASE-Secure-MTA) with ESMTP id CBAF36F04BA; Sun, 12 Jul 2009 13:04:13 +0200 (CEST) Received: from EXCHSRV.fokus.fraunhofer.de (bohr.fokus.fraunhofer.de [10.147.9.231]) by pluto.fokus.fraunhofer.de (8.14.2/8.14.2) with SMTP id n6CB4Dv2026135; Sun, 12 Jul 2009 13:04:13 +0200 (MEST) Received: from rigel ([10.147.65.195]) by EXCHSRV.fokus.fraunhofer.de with Microsoft SMTPSVC(6.0.3790.3959); Sun, 12 Jul 2009 13:04:13 +0200 Date: Sun, 12 Jul 2009 13:04:13 +0200 From: Joerg.Schilling@fokus.fraunhofer.de (Joerg Schilling) Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A59487B.3080202@Sun.COM> Sender: opensolaris-arc-bounces@opensolaris.org To: dcragun@sonic.net, amw@sun.com Cc: opensolaris-arc@opensolaris.org, cifs-eng@sun.com, Afshin.Ardakani@sun.com, Brian.Wong@sun.com Errors-to: opensolaris-arc-bounces@opensolaris.org Message-id: <4a59c32d.tI2Jgdzpe1XfZNIu%Joerg.Schilling@fokus.fraunhofer.de> MIME-version: 1.0 Content-type: text/plain; charset=iso-8859-1 Content-transfer-encoding: quoted-printable Precedence: list X-BeenThere: opensolaris-arc@opensolaris.org Delivered-to: opensolaris-arc@opensolaris.org X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on oss-mail1.opensolaris.org X-Original-To: opensolaris-arc@opensolaris.org X-Antispam: No, score=-2.6/5.0, scanned in 1.288sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <7037.76.191.129.144.1247339964.squirrel@webmail.sonic.net> <4A59487B.3080202@Sun.COM> X-Mailman-Version: 2.1.12 List-Post: List-Subscribe: , List-Unsubscribe: , List-Archive: List-Help: List-Id: Discussion of ARC issues affecting OpenSolaris User-Agent: nail 11.22 3/20/05 X-OriginalArrivalTime: 12 Jul 2009 11:04:13.0311 (UTC) FILETIME=[7D63E8F0:01CA02E0] X-Spam-Status: No, score=-3.9 required=5.0 tests=AWL,BAYES_00, RCVD_IN_DNSWL_LOW autolearn=unavailable version=3.2.5 X-Spam-Level: Content-Length: 2819 Status: RO X-Status: $$$$ X-UID: 0000000032 "Alan.M.Wright" wrote: > That would seem to resolve the PATH_MAX question asked earlier since > the length interpretation is implementation dependent. > > An interpretation change request assumes that we would want this to be > adopted as a standard, which may be desirable, but this proposal doesn't > rely on reparse data being used for local path resolution (at this time). > This proposal is intended to support reparse point aware services. > Regular applications should be unaware that the text is anything other th= an > a dangling symlink. If you like this case to be established as POSIX standard, you would need t= o = write a much more in depth explanation than you did. There are many questions that need to be answered before you could present = it = to the standard commitee. If you make a proposal that will cause existing = applications to missbehave, you should be prepared that your proposal will = be = rejected the same way as Microsofts proposal 8 years ago was rejected when = they first tried to introduce special semantics to the ':' in file names and lat= er = tried to introduce a directory "..." with special meanings in order to = implement their file streams. Let me ask the questions that you need to answer: - What exactly is the purpose of the new techique? - How will it be used? - Where is it used? - What will happen based on what data? - What is the benefit of having this technique? ... - How _exactly_ will it be implemented? - Is is based on technology that is currently part of POSIX? - If the technology is not in POSIX, it needs to be = described in detail and first offered to POSIX - If it is in POSIX already, does it need to be enhanced? = - Which existing applications will fail or missbehave? - Note that making existing applications to missbehave is a no-go.... - Is there a way to prevent miss-behavior? ... There are many more questions. I just like to give you a cookbook for a bet= ter proposal. Looking at *BSD implementations does not verify that existing = applications are not affected. Note that e.g. FreeBSD did introduce large f= ile = support without implementing a default fallback for old applications that d= on't = know about large files. This was done because the FreeBSD guys did modify a= ll related FreeBSD software. POSIX intends to take care of existing applicatio= ns... J=F6rg -- = EMail:joerg@schily.isdn.cs.tu-berlin.de (home) J=F6rg Schilling D-13353 Be= rlin js@cs.tu-berlin.de (uni) = joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogs= pot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily _______________________________________________ opensolaris-arc mailing list opensolaris-arc@opensolaris.org From opensolaris-arc-bounces@opensolaris.org Sun Jul 12 06:35:01 2009 Return-Path: Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01 [129.145.155.118]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6CDZ1Xr015698 for ; Sun, 12 Jul 2009 06:35:01 -0700 (PDT) Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6CDa98x020961; Sun, 12 Jul 2009 06:36:09 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n6CDZmIc006020; Sun, 12 Jul 2009 21:35:53 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMO007038FQC700@nwk-avmta-1.sfbay.Sun.COM>; Sun, 12 Jul 2009 06:35:50 -0700 (PDT) Received: from sca-ea-mail-4.sun.com ([192.18.43.22]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMO00MCR8FPW240@nwk-avmta-1.sfbay.Sun.COM>; Sun, 12 Jul 2009 06:35:49 -0700 (PDT) Received: from relay41i.sun.com ([192.5.209.70]) by sca-ea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6CDErsW023791; Sun, 12 Jul 2009 13:35:49 +0000 (GMT) Received: from mmp42es.mmp.us.syntegra.com ([160.41.221.11] [160.41.221.11]) by relay41i.sun.com with ESMTP id BT-MMP-2329371; Sun, 12 Jul 2009 13:35:49 +0000 (Z) Received: from relay44i.sun.com (relay44i.sun.com [192.5.209.118]) by mmp42es.mmp.us.syntegra.com with ESMTP id BT-MMP-81728917; Sun, 12 Jul 2009 13:35:46 +0000 (Z) Received: from mail.opensolaris.org ([72.5.123.71] [72.5.123.71]) by relay4i.sun.com with ESMTP id BT-MMP-8823920; Sun, 12 Jul 2009 13:35:46 +0000 (Z) Received: by mail.opensolaris.org (Postfix, from userid 505) id 0BA591571AF; Sun, 12 Jul 2009 06:35:45 -0700 (PDT) Received: from oss-mail1.opensolaris.org (localhost [127.0.0.1]) by mail.opensolaris.org (Postfix) with ESMTP id B3B4D157198; Sun, 12 Jul 2009 06:35:36 -0700 (PDT) Received: by mail.opensolaris.org (Postfix, from userid 505) id BD9ED15718E; Sun, 12 Jul 2009 06:35:34 -0700 (PDT) Received: from sf-app1.opensolaris.org (sf-app1 [72.5.123.78]) by mail.opensolaris.org (Postfix) with ESMTP id 36E99157189 for ; Sun, 12 Jul 2009 06:35:26 -0700 (PDT) Received: from sf-app1 (localhost [127.0.0.1]) by sf-app1.opensolaris.org (8.14.3+Sun/8.14.3) with ESMTP id n6CDZPGN009055 for ; Sun, 12 Jul 2009 06:35:25 -0700 (PDT) Date: Sun, 12 Jul 2009 06:34:55 -0700 (PDT) From: "Richard L. Hamilton" Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A5943C0.3090303@Sun.COM> Sender: opensolaris-arc-bounces@opensolaris.org To: opensolaris-arc@opensolaris.org Errors-to: opensolaris-arc-bounces@opensolaris.org Message-id: <2107453041.251247405725808.JavaMail.Twebapp@sf-app1> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Precedence: list X-BeenThere: opensolaris-arc@opensolaris.org Delivered-to: opensolaris-arc@opensolaris.org X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on oss-mail1.opensolaris.org X-Original-To: opensolaris-arc@opensolaris.org X-Antispam: No, score=-0.7/5.0, scanned in 0.188sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ X-Mailman-Version: 2.1.12 List-Post: List-Subscribe: , List-Unsubscribe: , List-Archive: List-Help: List-Id: Discussion of ARC issues affecting OpenSolaris X-Spam-Status: No, score=-2.5 required=5.0 tests=AWL,BAYES_00 autolearn=unavailable version=3.2.5 X-Spam-Level: Content-Length: 2717 Status: RO X-Status: $$$$ X-UID: 0000000033 > > Is this intended to be something like the > conditional symbolic links that have > > been in Apollo Domain system 23 years ago? > > > > I'm not familiar with that proposal. Can you provide > details? > > Thanks, > > Alan As I recall, Apollo Domain OS implicitly made environment variables available to "system calls" (the quotation marks being because the OS was different enough that traditional system calls might often have been implemented more in user space than is typical on the historical Unix code-base). Therefore, symlinks on Domain OS could contain environment variable references; I think these looked like $(name); so they might have symlinks like /usr -> /$(SYSTYPE)/usr where SYSTYPE could be sys5.3 or bsd4.3. (give or take details, I recall that as an actual example) Of course, Apollo also did some other odd things with path names: //nodename referred to the / direcctory of node nodename, as seen by other nodes (similar to AFS /afs/node, or automounter /net/node) `node_data referred to the per-node private data directory (/sys/node_data on a diskful node, something like /sys/node_data.nodeid on the diskful partner of diskless node nodeid) And Apollo had a "typed" filesystem, where types for device files, symlinks, FIFOs, Unix-domain sockets, and unstructured files were only _some_ of the types; others could be for record-oriented files, windowing system entities (scrolling back through a transcript pad with mixed text and graphics would replay the graphics! - a "terminal" based on such a transcript pad was effectively seekable, but append-only for writing), files that incorporated revision history (the ancestor of Rational Rose), etc. In some cases, something that was not a directory could be other than the last level of a path name, in which case it would be up to the object at that level to interpret the "residue" of the pathname. (They also had a form of ACLs long before those were commonplace on other Unix-like OSs.) So while the Apollo was IMO a _brilliant_ example of what's possible, some of what it could do exceeds what would readily fit the Unix model (although it could present a very credible approximation of that model as a subset of what it could do). And thus, examples from an Apollo may be useful in terms of thinking about a problem, but could often not be reasonably implemented to look similar on a more traditional Unix-like OS. (way OT: ISTR one limitation on the Apollo: anything that was executable was effectively also readable, due to some architectural constraint) -- This message posted from opensolaris.org _______________________________________________ opensolaris-arc mailing list opensolaris-arc@opensolaris.org From psarc-member-list-request@sun.com Sun Jul 12 17:18:17 2009 Return-Path: Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02 [129.146.11.31]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6D0IGaf015982 for ; Sun, 12 Jul 2009 17:18:16 -0700 (PDT) Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6D0JPV5026729; Sun, 12 Jul 2009 17:19:25 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n6D0IqO4011089; Mon, 13 Jul 2009 08:19:12 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMP0030B27W0300@brm-avmta-1.central.sun.com> (ORCPT PSARC-ext@sun.com) ; Sun, 12 Jul 2009 18:19:08 -0600 (MDT) Received: from dm-sfbay-01.sfbay.sun.com ([129.145.155.118]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMP007XT27V2G40@brm-avmta-1.central.sun.com> (ORCPT PSARC-ext@sun.com); Sun, 12 Jul 2009 18:19:08 -0600 (MDT) Received: from sac.sfbay.sun.com (sac.SFBay.Sun.COM [129.146.226.132]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6D0J7I5065266; Sun, 12 Jul 2009 17:19:07 -0700 (PDT) Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6D0Ea3p026834 for ; Sun, 12 Jul 2009 17:14:37 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6D0EP08025141; Mon, 13 Jul 2009 01:14:36 +0100 (BST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMP00207209FO00@brm-avmta-1.central.sun.com>; Sun, 12 Jul 2009 18:14:33 -0600 (MDT) Received: from brmea-mail-2.sun.com ([192.18.98.43]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMP007PU2092440@brm-avmta-1.central.sun.com>; Sun, 12 Jul 2009 18:14:33 -0600 (MDT) Received: from fe-amer-09.sun.com ([192.18.109.79]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6D0EX7T004810; Mon, 13 Jul 2009 00:14:33 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMP00E001XCWB00@mail-amer.sun.com>; Sun, 12 Jul 2009 18:14:33 -0600 (MDT) Received: from TOSHIBA ([unknown] [68.228.88.55]) by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMP00F1E2079500@mail-amer.sun.com>; Sun, 12 Jul 2009 18:14:32 -0600 (MDT) Date: Sun, 12 Jul 2009 17:14:30 -0700 From: "Alan.M.Wright" Subject: Re: 2009/387 [Pathname Reparse Points] Sender: Alan.M.Wright@sun.com To: Joerg Schilling Cc: Robert.Thurlow@sun.com, PSARC-ext@sun.com, gdamore@sun.com, cifs-eng@sun.com, Brian.Wong@sun.com, Afshin.Ardakani@sun.com Message-id: MIME-version: 1.0 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2900.5579 X-Mailer: Microsoft Outlook Express 6.00.2900.5512 Content-type: text/plain; CHARSET=US-ASCII; reply-type=original; format=flowed Content-transfer-encoding: 7BIT X-Priority: 3 X-MSMail-priority: Normal X-PMX-Version: 5.4.1.325704 X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <4A57743F.5070708@Sun.COM> <4A577936.9050509@sun.com> <4A577CF4.9050702@sun.com> <4a58ab90.X2ncDHPKuL+7EDtV%Joerg.Schilling@fokus.fraunhofer.de> <4A59463F.9040003@Sun.COM> <4a59b7e4.etj9O4PGGKpK8Wtv%Joerg.Schilling@fokus.fraunhofer.de> Content-Length: 1452 Status: RO X-Status: $$$$ X-UID: 0000000034 Joerg Schilling wrote: > "Alan.M.Wright" wrote: > >> Joerg Schilling wrote: >> > "Garrett D'Amore" wrote: >> > >> > >> >>> Archivers that slurp and spit the symlink contents will work >> >>> without mods as long as they get all of the bytes, but would >> >>> need more extensive modifications if our storage was in a >> >>> system attribute. Also, we can get the single bit we need >> >>> in ZFS now, and a 16K sysattr will not be supportable for a >> >>> few more months. >> >>> >> >> I'm confused. Brian says that archivers Just Work with the current >> >> form, because the attributes are retained. Yet, you're saying that the >> >> attributes are not necessarily retained. Which is it? Right now, >> >> either way, you have an attribute... which I *think* means that the you >> >> need support (which may or may not be present) in the archivers. >> >> >> > >> > If these objects will be seen as symlink file type and in case they >> > cannot >> > be copied using symlink(), I expect problems. >> > >> >> You're trying to draw a distinction between this proposal and current >> symlink behavior but there is no distinction. To existing applications, >> these will appear like symlinks that don't resolve to an existing target. > > You did not answer my question: is it possible to "correctly" copy such an > object by using lstat(), readlink() and symlink()? Yes. Alan From amw@sun.com Sun Jul 12 18:00:06 2009 Return-Path: Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01 [129.145.155.118]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6D1057O016004 for ; Sun, 12 Jul 2009 18:00:05 -0700 (PDT) Received: from brmea-mail-2.sun.com (brmea-mail-2.Sun.COM [192.18.98.43]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6D11E8G017044 for ; Sun, 12 Jul 2009 18:01:15 -0700 (PDT) Received: from fe-amer-10.sun.com ([192.18.109.80]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6D11E9B013985; Mon, 13 Jul 2009 01:01:14 GMT MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII; reply-type=original; format=flowed Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMP00A003WMYP00@mail-amer.sun.com>; Sun, 12 Jul 2009 19:01:14 -0600 (MDT) Received: from TOSHIBA ([unknown] [129.150.0.125]) by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMP0013Q461VJ40@mail-amer.sun.com>; Sun, 12 Jul 2009 19:01:14 -0600 (MDT) Date: Sun, 12 Jul 2009 18:01:12 -0700 From: "Alan.M.Wright" Subject: Re: 2009/387 [Pathname Reparse Points] Sender: Alan.M.Wright@sun.com To: Joerg Schilling Cc: Robert.Thurlow@sun.com, PSARC-ext@sun.com, Glenn.Skinner@sun.com, cifs-eng@sun.com, Brian.Wong@sun.com, Afshin.Ardakani@sun.com Message-id: <5D73FF1F39864CE895A438B3CF981EA1@TOSHIBA> X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2900.5579 X-Mailer: Microsoft Outlook Express 6.00.2900.5512 X-Priority: 3 X-MSMail-priority: Normal References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <4a58a697.yQ8anaL/V6TEAoRX%Joerg.Schilling@fokus.fraunhofer.de> <4A5943C0.3090303@Sun.COM> <4a59bb20.CuZiqCKUmLZgrA4L%Joerg.Schilling@fokus.fraunhofer.de> Content-Length: 3346 Status: RO X-Status: $$$$ X-UID: 0000000035 Joerg Schilling wrote: > "Alan.M.Wright" wrote: >> >> To distinguish a regular symlink from a reparse point, an >> >> extensible system attribute will be set on the symlink. This >> >> system attribute is only one bit which indicates whether or >> >> not a symlink contains reparse data. >> > >> > How exactly will these "reparse points" be distinguished from symlinks? >> >> Precisely as described in the paragraph you quoted. >> Otherwise, the assumption was that they would be treated >> as any other symlink. > > This paragraph does not contain the information you claim: "man -k > attrubute" > does not lead me to useful information. The new reparse system attribute should be documented in the ls man page and be displayed when using the -/ options. >> > How will you prevent prople from removing these "reparse points" because >> > they >> > asume that this are symlinks that point to nowhere? >> > Note that the official POSIX method for removing symlinks that do not >> > point >> > to a target is: >> > >> > find -L . -type l -exec rm {} + >> > >> >> Symlinks containing reparse data can be removed in the same way as >> any other symlink. There are no special rules, conditions or restrictions. > > So you admit that people who intend to remove garbage symlinks will > accidentaly > also remove these objects? From the find man page: Using the -L or -follow option is not recommended when descending a file-system hierarchy that is under the control of other users. In particular, when using -exec, symbolic links can lead the find command out of the hierarchy in which it started. Using -type is not sufficient to restrict the type of files on which the -exec command operates, because there is an inherent race condition between the type-check per- formed by the find command and the time the executed command operates on the file argument. The command you cite seems to ignore all of the warnings in here. Yes it will remove reparse points (as it should) and it looks like there may be a multiple undesirable side-effects from such a command. >> >> The symlink target size should be increased to 16K to >> >> accomodate the maximum size supported for MS-DFS referrals by >> >> Windows. Applications are expected to query the PATH_MAX and >> >> SYMLINK_MAX values on the local system using >> >> pathconf(2)/fpathconf(2). The value of SYMLINK_MAX would be >> >> changed to 16K on ZFS. The value of PATH_MAX will not be >> >> affected. >> > >> > Whill SYMLINK_MAX from limits.h be set to 16k? >> >> No. > > So I expect existing applications to fail. Per the specification section that Don referenced, this shouldn't cause a problem. It is up to the implementation to decide whether or not the text length returned is acceptable or an error. > From what I did read on this topic, I expect problems with existing > applications. I would like to read a description on what happens with > typical > existing applications, if this new feature is implemented. We have been looking at this for several months and haven't uncovered any problems. If you have a specific example, we can consider it but, so far, everything behaves as expected when a symlink contains arbitrary text or does not refer to an existing object. Alan From psarc-member-list-request@sun.com Sun Jul 12 19:51:08 2009 Return-Path: Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01 [129.145.155.118]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6D2p7wD016057 for ; Sun, 12 Jul 2009 19:51:07 -0700 (PDT) Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6D2qGQu052816; Sun, 12 Jul 2009 19:52:16 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6D2q6j0010292; Mon, 13 Jul 2009 03:52:08 +0100 (BST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMP00K0J9AUXE00@brm-avmta-1.central.sun.com> (ORCPT PSARC-ext@sun.com) ; Sun, 12 Jul 2009 20:52:06 -0600 (MDT) Received: from dm-sfbay-02.sfbay.sun.com ([129.146.11.31]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMP0073Q9AT22B0@brm-avmta-1.central.sun.com> (ORCPT PSARC-ext@sun.com); Sun, 12 Jul 2009 20:52:05 -0600 (MDT) Received: from sac.sfbay.sun.com (sac.SFBay.Sun.COM [129.146.226.132]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6D2q4nr014626; Sun, 12 Jul 2009 19:52:04 -0700 (PDT) Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6D2odqU028131 for ; Sun, 12 Jul 2009 19:50:39 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6D2oTrI009760; Mon, 13 Jul 2009 03:50:34 +0100 (BST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMP00K03989TE00@brm-avmta-1.central.sun.com>; Sun, 12 Jul 2009 20:50:33 -0600 (MDT) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMP007WI98824A0@brm-avmta-1.central.sun.com>; Sun, 12 Jul 2009 20:50:32 -0600 (MDT) Received: from relay14i.sun.com (ip124.net129179-4.block1.us.syntegra.com [129.179.4.124]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6D2oWno025815; Mon, 13 Jul 2009 02:50:32 +0000 (GMT) Received: from mmp14es.mmp.us.syntegra.com ([160.41.208.14] [160.41.208.14]) by relay14i.sun.com with ESMTP id BT-MMP-5241737; Mon, 13 Jul 2009 02:48:31 +0000 (Z) Received: from relay11i.sun.com (relay11i.sun.com [129.179.4.121]) by mmp14es.mmp.us.syntegra.com with ESMTP id BT-MMP-566215; Mon, 13 Jul 2009 02:48:30 +0000 (Z) Received: from remote.mcintyreweb.com ([67.23.1.228] [67.23.1.228]) by relay1i.sun.com with ESMTP id BT-MMP-6123977; Mon, 13 Jul 2009 02:48:30 +0000 (Z) Received: from twins.i.mcintyreweb.com (unknown [64.166.3.74]) by remote.mcintyreweb.com (Postfix) with ESMTPS id 22A7410C25D; Sun, 12 Jul 2009 19:48:29 -0700 (PDT) Date: Sun, 12 Jul 2009 19:48:27 -0700 From: Hugh McIntyre Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4a59bb20.CuZiqCKUmLZgrA4L%Joerg.Schilling@fokus.fraunhofer.de> To: Joerg Schilling Cc: amw@sun.com, Afshin.Ardakani@sun.com, cifs-eng@sun.com, PSARC-ext@sun.com, Robert.Thurlow@sun.com, Brian.Wong@sun.com Message-id: <4A5AA07B.1020701@mcintyreweb.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-Antispam: No, score=0.0/5.0, scanned in 1.062sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <4a58a697.yQ8anaL/V6TEAoRX%Joerg.Schilling@fokus.fraunhofer.de> <4A5943C0.3090303@Sun.COM> <4a59bb20.CuZiqCKUmLZgrA4L%Joerg.Schilling@fokus.fraunhofer.de> User-Agent: Thunderbird 2.0.0.22 (Macintosh/20090605) Content-Length: 1767 Status: RO X-Status: $$$$ X-UID: 0000000036 Joerg Schilling wrote: > "Alan.M.Wright" wrote: > >>> How will you prevent prople from removing these "reparse points" because they >>> assume that this are symlinks that point to nowhere? >>> Note that the official POSIX method for removing symlinks that do not point >>> to a target is: >>> >>> find -L . -type l -exec rm {} + >>> >> Symlinks containing reparse data can be removed in the same way as >> any other symlink. There are no special rules, conditions or restrictions. > > So you admit that people who intend to remove garbage symlinks will accidentally > also remove these objects? To be fair, if you have a symlink to an automounted file or directory and the server is temporarily not accessible or automounter not running, I suspect you'll similarly think the link is dangling. So you'd better hope the automounter is in good shape and you have no network glitches before using the "find" command above. But this raises the point that if these special reparse symlinks were implemented such that the reparsing also happened when the filesystem is mounted locally via mount_zfs or mount_lofs, not just remotely over CIFS or NFSv4, then most of the issues would go away. In other words if open(), stat(), nftw(), and similar system calls locally on the server hosting the files (including LOFS) access the linked-to file, most code will think the link is OK. Since in this case, "find -L" should point to the target of the reparse point (which hopefully exists). Assuming that most code does not call readlink() and do the pathname traversal itself. I think there was some question about this earlier in the thread, but not necessarily a definitive answer. Hugh. [*] Even if some sysadmin setup is needed here. From robert.thurlow@sun.com Mon Jul 13 08:37:31 2009 Return-Path: Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02 [129.146.11.31]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6DFbVAE016545 for ; Mon, 13 Jul 2009 08:37:31 -0700 (PDT) Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6DFceWB029041; Mon, 13 Jul 2009 08:38:40 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n6DFcNSN036604; Mon, 13 Jul 2009 09:38:39 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMQ00F3Z8SCND00@nwk-avmta-2.sfbay.sun.com>; Mon, 13 Jul 2009 08:38:36 -0700 (PDT) Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.59]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMQ004OS8SBVLB0@nwk-avmta-2.sfbay.sun.com>; Mon, 13 Jul 2009 08:38:35 -0700 (PDT) Received: from [10.7.250.15] (punchin-client-10-7-250-15.SFBay.Sun.COM [10.7.250.15]) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6DFcY0f190389; Mon, 13 Jul 2009 08:38:34 -0700 (PDT) Date: Mon, 13 Jul 2009 09:38:34 -0600 From: Robert Thurlow Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A5943C0.3090303@Sun.COM> To: "Alan.M.Wright" Cc: Glenn.Skinner@sun.com, Afshin.Ardakani@sun.com Message-id: <4A5B54FA.9030308@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <4a58a697.yQ8anaL/V6TEAoRX%Joerg.Schilling@fokus.fraunhofer.de> <4A5943C0.3090303@Sun.COM> User-Agent: Thunderbird 2.0.0.16 (X11/20080811) Content-Length: 283 Status: RO X-Status: $$$$ X-UID: 0000000037 Alan.M.Wright wrote: >> Whill SYMLINK_MAX from limits.h be set to 16k? >> > > No. There's a disconnect here. Our interface table says: SYMLINK_MAX |Committed |This |Increased to 16K | |Document | Rob T From psarc-member-list-request@sun.com Mon Jul 13 08:47:51 2009 Return-Path: Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01 [129.145.155.118]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6DFlpjj016558 for ; Mon, 13 Jul 2009 08:47:51 -0700 (PDT) Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6DFn0nM019405; Mon, 13 Jul 2009 08:49:00 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n6DFmrcO044260; Mon, 13 Jul 2009 09:48:54 -0600 (MDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMQ00E5P99FGF00@brm-avmta-1.central.sun.com> (ORCPT PSARC-ext@sun.com) ; Mon, 13 Jul 2009 09:48:52 -0600 (MDT) Received: from dm-sfbay-02.sfbay.sun.com ([129.146.11.31]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMQ00MGC99EPGB0@brm-avmta-1.central.sun.com> (ORCPT PSARC-ext@sun.com); Mon, 13 Jul 2009 09:48:50 -0600 (MDT) Received: from sac.sfbay.sun.com (sac.SFBay.Sun.COM [129.146.226.132]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6DFmoAP036935; Mon, 13 Jul 2009 08:48:50 -0700 (PDT) Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6DFlRKT019672 for ; Mon, 13 Jul 2009 08:47:27 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n6DFlOl2004315; Mon, 13 Jul 2009 08:47:26 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMQ00E0X971DF00@brm-avmta-1.central.sun.com>; Mon, 13 Jul 2009 09:47:25 -0600 (MDT) Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.63]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMQ00MRG971P8C0@brm-avmta-1.central.sun.com>; Mon, 13 Jul 2009 09:47:25 -0600 (MDT) Received: from [10.7.250.15] (punchin-client-10-7-250-15.SFBay.Sun.COM [10.7.250.15]) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6DFlNRf191866; Mon, 13 Jul 2009 08:47:24 -0700 (PDT) Date: Mon, 13 Jul 2009 09:47:23 -0600 From: Robert Thurlow Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A5AA07B.1020701@mcintyreweb.com> To: Hugh McIntyre Cc: Joerg Schilling , amw@sun.com, Afshin.Ardakani@sun.com, cifs-eng@sun.com, PSARC-ext@sun.com, Brian.Wong@sun.com Message-id: <4A5B570B.4030805@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <4a58a697.yQ8anaL/V6TEAoRX%Joerg.Schilling@fokus.fraunhofer.de> <4A5943C0.3090303@Sun.COM> <4a59bb20.CuZiqCKUmLZgrA4L%Joerg.Schilling@fokus.fraunhofer.de> <4A5AA07B.1020701@mcintyreweb.com> User-Agent: Thunderbird 2.0.0.16 (X11/20080811) Content-Length: 1172 Status: RO X-Status: $$$$ X-UID: 0000000038 Hugh McIntyre wrote: > But this raises the point that if these special reparse symlinks were > implemented such that the reparsing also happened when the filesystem is > mounted locally via mount_zfs or mount_lofs, not just remotely over CIFS > or NFSv4, then most of the issues would go away. > > In other words if open(), stat(), nftw(), and similar system calls > locally on the server hosting the files (including LOFS) access the > linked-to file, most code will think the link is OK. Since in this > case, "find -L" should point to the target of the reparse point (which > hopefully exists). The project team is pretty interested in semantics seen by NFS and CIFS clients, not local semantics, so we're not terribly excited to do work to make local access act like that; I get the motivations, but it's a whole lot of work. There are also difficulties. We know we will want to set up a reparse point that projects the same namespace for both NFS and CIFS, and it isn't obvious what client code we would bridge into locally. We would also like to be able to back up reparse points, so local processes need to access them without interpretation. Rob T From psarc-member-list-request@sun.com Mon Jul 13 09:29:25 2009 Return-Path: Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01 [129.145.155.118]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6DGTP34016609 for ; Mon, 13 Jul 2009 09:29:25 -0700 (PDT) Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6DGUYdB053314; Mon, 13 Jul 2009 09:30:34 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n6DGUQX2004694; Mon, 13 Jul 2009 10:30:28 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMQ00727B6RUD00@nwk-avmta-1.sfbay.Sun.COM> (ORCPT PSARC-ext@sun.com) ; Mon, 13 Jul 2009 09:30:27 -0700 (PDT) Received: from dm-sfbay-01.sfbay.sun.com ([129.145.155.118]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMQ004YZB6Q9810@nwk-avmta-1.sfbay.Sun.COM> (ORCPT PSARC-ext@sun.com); Mon, 13 Jul 2009 09:30:26 -0700 (PDT) Received: from sac.sfbay.sun.com (sac.SFBay.Sun.COM [129.146.226.132]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6DGUKos053233; Mon, 13 Jul 2009 09:30:20 -0700 (PDT) Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6DGSvXE021697 for ; Mon, 13 Jul 2009 09:28:58 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6DGSlmq017653 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Mon, 13 Jul 2009 17:28:56 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMQ00I01B46QX00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 13 Jul 2009 09:28:54 -0700 (PDT) Received: from sca-ea-mail-2.sun.com ([192.18.43.25]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMQ00HL6B45FC10@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 13 Jul 2009 09:28:54 -0700 (PDT) Received: from relay44i.sun.com ([192.5.209.118]) by sca-ea-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n6DGHcre029145 for ; Mon, 13 Jul 2009 16:28:53 +0000 (GMT) Received: from mmp43es.mmp.us.syntegra.com ([160.41.221.12] [160.41.221.12]) by relay44i.sun.com with ESMTP id BT-MMP-38393 for PSARC-ext@sun.com; Mon, 13 Jul 2009 16:28:47 +0000 (Z) Received: from relay43i.sun.com (relay43i.sun.com [192.5.209.74]) by mmp43es.mmp.us.syntegra.com with ESMTP id BT-MMP-31215301 for PSARC-ext@sun.com; Mon, 13 Jul 2009 16:28:46 +0000 (Z) Received: from relay04-haj2.antispameurope.com ([83.246.65.54] [83.246.65.54]) by relay4i.sun.com with ESMTP id BT-MMP-5889886 for PSARC-ext@sun.com; Mon, 13 Jul 2009 16:28:02 +0000 (Z) Received: by relay04-haj2.antispameurope.com (ASE-Secure-MTA, from userid 1000) id DB57B5EC230; Mon, 13 Jul 2009 18:27:58 +0200 (CEST) Received: from pluto.fokus.fraunhofer.de (pluto.fokus.fraunhofer.de [195.37.77.164]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by relay04-haj2.antispameurope.com (ASE-Secure-MTA) with ESMTP id E3EF85EC217; Mon, 13 Jul 2009 18:27:57 +0200 (CEST) Received: from EXCHSRV.fokus.fraunhofer.de (bohr.fokus.fraunhofer.de [10.147.9.231]) by pluto.fokus.fraunhofer.de (8.14.2/8.14.2) with SMTP id n6DGRvMq005748; Mon, 13 Jul 2009 18:27:57 +0200 (MEST) Received: from rigel ([10.147.65.195]) by EXCHSRV.fokus.fraunhofer.de with Microsoft SMTPSVC(6.0.3790.3959); Mon, 13 Jul 2009 18:27:57 +0200 Date: Mon, 13 Jul 2009 18:27:57 +0200 From: Joerg.Schilling@fokus.fraunhofer.de (Joerg Schilling) Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A5AA07B.1020701@mcintyreweb.com> Sender: Joerg.Schilling9ab33xy531fokus.fraunhofer.de@bounce.antispameurope.com To: lists@mcintyreweb.com Cc: Robert.Thurlow@sun.com, PSARC-ext@sun.com, cifs-eng@sun.com, Brian.Wong@sun.com, amw@sun.com, Afshin.Ardakani@sun.com Message-id: <4a5b608d.jcfvHp0YLr8sVrS9%Joerg.Schilling@fokus.fraunhofer.de> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1 Content-transfer-encoding: 8BIT X-PMX-Version: 5.4.1.325704 X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-Antispam: No, score=-0.7/5.0, scanned in 0.682sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <4a58a697.yQ8anaL/V6TEAoRX%Joerg.Schilling@fokus.fraunhofer.de> <4A5943C0.3090303@Sun.COM> <4a59bb20.CuZiqCKUmLZgrA4L%Joerg.Schilling@fokus.fraunhofer.de> <4A5AA07B.1020701@mcintyreweb.com> User-Agent: nail 11.22 3/20/05 X-OriginalArrivalTime: 13 Jul 2009 16:27:57.0817 (UTC) FILETIME=[E1B60E90:01CA03D6] Content-Length: 2507 Status: RO X-Status: $$$$ X-UID: 0000000039 Hugh McIntyre wrote: > Joerg Schilling wrote: > > "Alan.M.Wright" wrote: > > > >>> How will you prevent prople from removing these "reparse points" because they > >>> assume that this are symlinks that point to nowhere? > >>> Note that the official POSIX method for removing symlinks that do not point > >>> to a target is: > >>> > >>> find -L . -type l -exec rm {} + > >>> > >> Symlinks containing reparse data can be removed in the same way as > >> any other symlink. There are no special rules, conditions or restrictions. > > > > So you admit that people who intend to remove garbage symlinks will accidentally > > also remove these objects? > > To be fair, if you have a symlink to an automounted file or directory > and the server is temporarily not accessible or automounter not running, > I suspect you'll similarly think the link is dangling. So you'd better > hope the automounter is in good shape and you have no network glitches > before using the "find" command above. In your example, there will be a timeout and other hints that may allow you to distinguish between a hung server and a non-existing target. > But this raises the point that if these special reparse symlinks were > implemented such that the reparsing also happened when the filesystem is > mounted locally via mount_zfs or mount_lofs, not just remotely over CIFS > or NFSv4, then most of the issues would go away. As I did not see a sufficient descption that could help me to understand what the intention of these reparse points is, I cannot discuss related issues. > In other words if open(), stat(), nftw(), and similar system calls > locally on the server hosting the files (including LOFS) access the > linked-to file, most code will think the link is OK. Since in this > case, "find -L" should point to the target of the reparse point (which > hopefully exists). nftw is not a "system call" and previous versions did have massive problems and Sun's /bin/find -follow did go into directory loops in case that there have been e.g. symlinks with a "." target name. This was the reason for writing sfind in August 2004 but it seems to have been fixed now... Jörg -- EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin js@cs.tu-berlin.de (uni) joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily From psarc-member-list-request@sun.com Mon Jul 13 10:15:58 2009 Return-Path: Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02 [129.146.11.31]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6DHFv53016660 for ; Mon, 13 Jul 2009 10:15:57 -0700 (PDT) Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6DHH6M9041389; Mon, 13 Jul 2009 10:17:06 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6DHGvIC018381; Mon, 13 Jul 2009 18:16:57 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMQ00L1FDC9G300@nwk-avmta-2.sfbay.sun.com> (ORCPT PSARC-ext@sun.com) ; Mon, 13 Jul 2009 10:16:57 -0700 (PDT) Received: from dm-sfbay-02.sfbay.sun.com ([129.146.11.31]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMQ00HR6DC8FB50@nwk-avmta-2.sfbay.sun.com> (ORCPT PSARC-ext@sun.com); Mon, 13 Jul 2009 10:16:56 -0700 (PDT) Received: from sac.sfbay.sun.com (sac.SFBay.Sun.COM [129.146.226.132]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6DHGusc041235; Mon, 13 Jul 2009 10:16:56 -0700 (PDT) Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6DHFdEV023484 for ; Mon, 13 Jul 2009 10:15:39 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6DHFWY6017378; Mon, 13 Jul 2009 18:15:34 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMQ00L0HD9WDV00@nwk-avmta-2.sfbay.sun.com>; Mon, 13 Jul 2009 10:15:32 -0700 (PDT) Received: from binky.Central.Sun.COM ([129.153.128.104]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMQ00HT1D9WF960@nwk-avmta-2.sfbay.sun.com>; Mon, 13 Jul 2009 10:15:32 -0700 (PDT) Received: from binky.Central.Sun.COM (localhost [127.0.0.1]) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3) with ESMTP id n6DGKGIs003971; Mon, 13 Jul 2009 11:20:16 -0500 (CDT) Received: (from nw141292@localhost) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3/Submit) id n6DGKD6B003970; Mon, 13 Jul 2009 11:20:13 -0500 (CDT) Date: Mon, 13 Jul 2009 11:20:13 -0500 From: Nicolas Williams Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A5B570B.4030805@sun.com> To: Robert Thurlow Cc: Hugh McIntyre , Joerg Schilling , amw@sun.com, Afshin.Ardakani@sun.com, cifs-eng@sun.com, PSARC-ext@sun.com, Brian.Wong@sun.com Message-id: <20090713162013.GX1274@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=iso-8859-1 Content-transfer-encoding: 8BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <4a58a697.yQ8anaL/V6TEAoRX%Joerg.Schilling@fokus.fraunhofer.de> <4A5943C0.3090303@Sun.COM> <4a59bb20.CuZiqCKUmLZgrA4L%Joerg.Schilling@fokus.fraunhofer.de> <4A5AA07B.1020701@mcintyreweb.com> <4A5B570B.4030805@sun.com> X-Authentication-warning: binky.Central.Sun.COM: nw141292 set sender to Nicolas.Williams@sun.com using -f User-Agent: Mutt/1.5.7i Content-Length: 4041 Status: RO X-Status: $$$$ X-UID: 0000000040 On Mon, Jul 13, 2009 at 09:47:23AM -0600, Robert Thurlow wrote: > Hugh McIntyre wrote: > > >But this raises the point that if these special reparse symlinks were > >implemented such that the reparsing also happened when the filesystem is > >mounted locally via mount_zfs or mount_lofs, not just remotely over CIFS > >or NFSv4, then most of the issues would go away. > > > >In other words if open(), stat(), nftw(), and similar system calls > >locally on the server hosting the files (including LOFS) access the > >linked-to file, most code will think the link is OK. Since in this > >case, "find -L" should point to the target of the reparse point (which > >hopefully exists). > > The project team is pretty interested in semantics seen by > NFS and CIFS clients, not local semantics, so we're not > terribly excited to do work to make local access act like > that; I get the motivations, but it's a whole lot of work. A future project could make reparse points be interpreted locally. There are two ways that that could be done: 1) Have the kernel notice that a path component is a reparse point and cause the target to be mounted. 2) Make the reparse point symlink contents start with a reserved absolute path such as, say, "/ref", so a special automounter map can be implemented later to evaluate and automount referrals. In the meantime the mountpoint for this future special map could be reserved by including an auto_master entry that mounts the -null special map (I'm not sure that -null works quite the way I think, but even if not, this entry could help document the reserved path prefix). Note too that such an automounter map could just be an executable map, possibly a simple script. If you don't ensure that reparse point contents look like an absolute path, then (2) will be precluded. That's OK since (1) remains possible, but you'd have to at least acknowledge this in making this choice. ALSO, you don't want random symlink contents to be confusable with reparse points. I think the risk of that is miniscule, but non-zero, and that concerns me more. The system attribute on reparse points, if _required_ in order for a reparse point to be treated as such, avoids that problem. It might be useful to reserve symlink contents sub-namespaces (when used with a special typing attribute, as in this case), so as to avoid confusability. Also, instead of having an extensible system attribute whose presence indicates that this is a reparse point, how about an extensible system attribute whose value indicates the real type of this object? We might need to resort to this symlink "sub-typing" scheme again in the future... IMO (1) is the better model, but the fact that (2) could be implemented as an executable map is very appealing. You should consider (2) before rejecting it. (Also, (1) could still rely on autofs infrastructure to make implementation easier.) > There are also difficulties. We know we will want to set > up a reparse point that projects the same namespace for > both NFS and CIFS, and it isn't obvious what client code > we would bridge into locally. We would also like to be > able to back up reparse points, so local processes need to > access them without interpretation. For the last part there are many ways to achieve that. Just using extended attributes is enough -- the referral could be stored in an extended attribute, and the object's type could be irrelevant. Backup tools that are aware of extended attributes would backup reparse points w/o any trouble. Using symlinks makes the reparse point somewhat more accessible to users, but it doesn't seem strictly necessary (there's always runat(1)), more like the resulf of a [likely very useful] optimization. (Also, I have to say that I agree with Jörg that overloading symlinks seems unfortunate. But because adding a new filesystem object type is hard (backup and other tools wouldn't know what to do with it), this is just unfortunate and that's that -- c'est la vie.) Nico -- From Afshin.Ardakani@sun.com Mon Jul 13 12:01:13 2009 Return-Path: Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02 [129.146.11.31]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6DJ1CgY016845 for ; Mon, 13 Jul 2009 12:01:12 -0700 (PDT) Received: from sca-es-mail-2.sun.com (sca-es-mail-2.Sun.COM [192.18.43.133]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6DJ2L7S048931 for ; Mon, 13 Jul 2009 12:02:21 -0700 (PDT) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n6DJ2GP0026975 for ; Mon, 13 Jul 2009 12:02:16 -0700 (PDT) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII; format=flowed Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMQ00L00I0Z3I00@fe-sfbay-10.sun.com> for Glenn.Skinner@SFBay.Sun.COM (ORCPT Glenn.Skinner@Sun.COM); Mon, 13 Jul 2009 12:02:16 -0700 (PDT) Received: from [10.1.106.212] ([unknown] [10.1.106.212]) by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMQ0006UI7OVJ10@fe-sfbay-10.sun.com>; Mon, 13 Jul 2009 12:02:12 -0700 (PDT) Date: Mon, 13 Jul 2009 12:01:59 -0700 From: Afshin Salek Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A5B54FA.9030308@sun.com> Sender: Afshin.Ardakani@sun.com To: Robert Thurlow Cc: "Alan.M.Wright" , Glenn.Skinner@sun.com Message-id: <4A5B84A7.3090704@sun.com> References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <4a58a697.yQ8anaL/V6TEAoRX%Joerg.Schilling@fokus.fraunhofer.de> <4A5943C0.3090303@Sun.COM> <4A5B54FA.9030308@sun.com> User-Agent: Thunderbird 2.0.0.9 (X11/20080213) Content-Length: 449 Status: RO X-Status: $$$$ X-UID: 0000000041 Our intentions is to increase this limit to 16K for ZFS. I guess I missed to mention this in the comment section. Afshin Robert Thurlow wrote: > Alan.M.Wright wrote: > >>> Whill SYMLINK_MAX from limits.h be set to 16k? >>> >> >> No. > > There's a disconnect here. Our interface table says: > > SYMLINK_MAX |Committed |This |Increased to 16K > | |Document | > > Rob T > From Nicolas.Williams@sun.com Mon Jul 13 15:27:48 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6DMRlUj009371 for ; Mon, 13 Jul 2009 15:27:47 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n6DMRSZM014763; Tue, 14 Jul 2009 06:27:41 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMQ00905RQ3HT00@brm-avmta-1.central.sun.com>; Mon, 13 Jul 2009 16:27:39 -0600 (MDT) Received: from binky.Central.Sun.COM ([129.153.128.104]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMQ00H1QRQ2JB80@brm-avmta-1.central.sun.com>; Mon, 13 Jul 2009 16:27:39 -0600 (MDT) Received: from binky.Central.Sun.COM (localhost [127.0.0.1]) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3) with ESMTP id n6DMOoa7007027; Mon, 13 Jul 2009 17:24:50 -0500 (CDT) Received: (from nw141292@localhost) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3/Submit) id n6DMOoxF007026; Mon, 13 Jul 2009 17:24:50 -0500 (CDT) Date: Mon, 13 Jul 2009 17:24:50 -0500 From: Nicolas Williams Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> To: Glenn Skinner Cc: PSARC-ext@sun.com, cifs-eng@sun.com, Robert.Thurlow@sun.com, Brian.Wong@sun.com, Afshin.Ardakani@sun.com Message-id: <20090713222450.GL1274@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> X-Authentication-warning: binky.Central.Sun.COM: nw141292 set sender to Nicolas.Williams@sun.com using -f User-Agent: Mutt/1.5.7i Content-Length: 1299 Status: RO X-Status: $$$$ X-UID: 0000000042 On Fri, Jul 10, 2009 at 09:34:26AM -0700, Glenn Skinner wrote: > The reparse data will be stored as the link target. The > reparse data is not in file system path format, which is the > typical format of a link target. In order to avoid coming up > with a totaly new format for reparse data as the link target > we decided to adopt the format used by magic links in BSD: > (http://www.daemon-systems.org/man/symlink.7.html) > > @{REPARSE@{service-type1:data} [@{service-type2:data}]...} I see no UI/CLI/API for managing this. Exposing users to this format seems most unfriendly. I think a CLI should be provided if such a complex format is to be used to store the referral data. It could be a simple script for all I care. Also, how about storing referral data in a per-protocol extended attribute instead of all of it as symlink contents? That would be much cleaner since then: - the type of the object (symlink, dir, file...) becomes irrelevant - one can export that object for some protocols, a referral for others - that object could be a symlink with a /net path -- a very nice fallback for NFSv3 Does using EAs cause problems with backup tools? If so, is that really a big deal? Wouldn't backup tools that can't backup EAs be problematic anyways? Nico -- From Nicolas.Williams@sun.com Mon Jul 13 15:29:00 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6DMSxiR009452 for ; Mon, 13 Jul 2009 15:29:00 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n6DMSqcX015360; Tue, 14 Jul 2009 06:28:54 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMQ0090FRS5NA00@brm-avmta-1.central.sun.com>; Mon, 13 Jul 2009 16:28:53 -0600 (MDT) Received: from binky.Central.Sun.COM ([129.153.128.104]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMQ00HWCRS4IY80@brm-avmta-1.central.sun.com>; Mon, 13 Jul 2009 16:28:52 -0600 (MDT) Received: from binky.Central.Sun.COM (localhost [127.0.0.1]) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3) with ESMTP id n6DMQ3po007033; Mon, 13 Jul 2009 17:26:03 -0500 (CDT) Received: (from nw141292@localhost) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3/Submit) id n6DMQ34o007032; Mon, 13 Jul 2009 17:26:03 -0500 (CDT) Date: Mon, 13 Jul 2009 17:26:03 -0500 From: Nicolas Williams Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <20090713222450.GL1274@Sun.COM> To: Glenn Skinner Cc: PSARC-ext@sun.com, cifs-eng@sun.com, Robert.Thurlow@sun.com, Brian.Wong@sun.com, Afshin.Ardakani@sun.com Message-id: <20090713222603.GP1275@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <20090713222450.GL1274@Sun.COM> X-Authentication-warning: binky.Central.Sun.COM: nw141292 set sender to Nicolas.Williams@sun.com using -f User-Agent: Mutt/1.5.7i Content-Length: 629 Status: RO X-Status: $$$$ X-UID: 0000000043 On Mon, Jul 13, 2009 at 05:24:50PM -0500, Nicolas Williams wrote: > Also, how about storing referral data in a per-protocol extended > attribute instead of all of it as symlink contents? That would be much > cleaner since then: > > - the type of the object (symlink, dir, file...) becomes irrelevant > - one can export that object for some protocols, a referral for others > - that object could be a symlink with a /net path -- a very nice > fallback for NFSv3 I forgot one: - each per-protocol referral format might be simple enough that runat(1) can be the interface to this, so no need for a CLI in that case From Afshin.Ardakani@sun.com Mon Jul 13 15:45:24 2009 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6DMjNOX009669 for ; Mon, 13 Jul 2009 15:45:23 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n6DMjMB9004831; Mon, 13 Jul 2009 15:45:23 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMQ00B0LSJM3H00@nwk-avmta-1.sfbay.Sun.COM>; Mon, 13 Jul 2009 15:45:22 -0700 (PDT) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMQ004INSJLUZ90@nwk-avmta-1.sfbay.Sun.COM>; Mon, 13 Jul 2009 15:45:22 -0700 (PDT) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n6DMjGFs013249; Mon, 13 Jul 2009 15:45:16 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMQ00200S68BP00@fe-sfbay-10.sun.com>; Mon, 13 Jul 2009 15:45:16 -0700 (PDT) Received: from [10.1.106.212] ([unknown] [10.1.106.212]) by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMQ0024VSIUL3G0@fe-sfbay-10.sun.com>; Mon, 13 Jul 2009 15:44:56 -0700 (PDT) Date: Mon, 13 Jul 2009 15:44:42 -0700 From: Afshin Salek Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <20090713222450.GL1274@Sun.COM> Sender: Afshin.Ardakani@sun.com To: Nicolas Williams Cc: Glenn Skinner , PSARC-ext@sun.com, cifs-eng@sun.com, robert.thurlow@sun.com, Brian.Wong@sun.com Message-id: <4A5BB8DA.804@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <20090713222450.GL1274@Sun.COM> User-Agent: Thunderbird 2.0.0.9 (X11/20080213) Content-Length: 2095 Status: RO X-Status: $$$$ X-UID: 0000000044 This is supposed to be a generic and expandable mechanism. NFS/DFS referrals are just primary consumers of this mechanism at this point so this is not a protocol specific mechanism to have a per protocol extended attribute that are hard coded before hand. If we want to use extended attributes then we either have to reserve some namespace or use an extended attribute with a reserved name as an index. It also makes path name reduction very expensive because for each component we have to do a lot of extra stuff to see whether we are dealing with a reparse point or not and whether that reparse point contains the data that we are interested in. I also think symlink does not quite support extended attributes. Afshin Nicolas Williams wrote: > On Fri, Jul 10, 2009 at 09:34:26AM -0700, Glenn Skinner wrote: >> The reparse data will be stored as the link target. The >> reparse data is not in file system path format, which is the >> typical format of a link target. In order to avoid coming up >> with a totaly new format for reparse data as the link target >> we decided to adopt the format used by magic links in BSD: >> (http://www.daemon-systems.org/man/symlink.7.html) >> >> @{REPARSE@{service-type1:data} [@{service-type2:data}]...} > > I see no UI/CLI/API for managing this. Exposing users to this format > seems most unfriendly. > > I think a CLI should be provided if such a complex format is to be used > to store the referral data. It could be a simple script for all I care. > > Also, how about storing referral data in a per-protocol extended > attribute instead of all of it as symlink contents? That would be much > cleaner since then: > > - the type of the object (symlink, dir, file...) becomes irrelevant > - one can export that object for some protocols, a referral for others > - that object could be a symlink with a /net path -- a very nice > fallback for NFSv3 > > Does using EAs cause problems with backup tools? If so, is that really > a big deal? Wouldn't backup tools that can't backup EAs be problematic > anyways? > > Nico From Nicolas.Williams@Sun.COM Mon Jul 13 15:55:19 2009 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6DMtIWn010335 for ; Mon, 13 Jul 2009 15:55:19 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6DMtDxC006293; Mon, 13 Jul 2009 23:55:14 +0100 (BST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMQ00C05T01KB00@brm-avmta-1.central.sun.com>; Mon, 13 Jul 2009 16:55:13 -0600 (MDT) Received: from binky.Central.Sun.COM ([129.153.128.104]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMQ00H3NT00JB90@brm-avmta-1.central.sun.com>; Mon, 13 Jul 2009 16:55:13 -0600 (MDT) Received: from binky.Central.Sun.COM (localhost [127.0.0.1]) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3) with ESMTP id n6DMqOBI007050; Mon, 13 Jul 2009 17:52:24 -0500 (CDT) Received: (from nw141292@localhost) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3/Submit) id n6DMqOUj007049; Mon, 13 Jul 2009 17:52:24 -0500 (CDT) Date: Mon, 13 Jul 2009 17:52:24 -0500 From: Nicolas Williams Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A5BB8DA.804@sun.com> To: Afshin Salek Cc: Glenn Skinner , PSARC-ext@Sun.COM, cifs-eng@Sun.COM, Robert.Thurlow@Sun.COM, Brian.Wong@Sun.COM Message-id: <20090713225224.GN1274@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <20090713222450.GL1274@Sun.COM> <4A5BB8DA.804@sun.com> X-Authentication-warning: binky.Central.Sun.COM: nw141292 set sender to Nicolas.Williams@sun.com using -f User-Agent: Mutt/1.5.7i Content-Length: 1151 Status: RO X-Status: $$$$ X-UID: 0000000045 On Mon, Jul 13, 2009 at 03:44:42PM -0700, Afshin Salek wrote: > This is supposed to be a generic and expandable mechanism. NFS/DFS > referrals are just primary consumers of this mechanism at this > point so this is not a protocol specific mechanism to have a per > protocol extended attribute that are hard coded before hand. > > If we want to use extended attributes then we either have to reserve > some namespace or use an extended attribute with a reserved name as an > index. It also makes path name reduction very expensive because for each > component we have to do a lot of extra stuff to see whether we are > dealing with a reparse point or not and whether that reparse point > contains the data that we are interested in. I'd stick with the ESA as the optimization, but EAs for storing the referral if you can manage the namespace, else make the symlink thing not-an-interface and provide a CLI, so you can change the implementation later. Oh, hmmm, I just noticed that the reparse string format is private. But I see no public interfaces for managing reparse points. Are UIs due later? If so never mind my comments to date. Nico -- From Afshin.Ardakani@sun.com Mon Jul 13 16:11:07 2009 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6DNB6SO012741 for ; Mon, 13 Jul 2009 16:11:07 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n6DNB0TT002279; Mon, 13 Jul 2009 17:11:06 -0600 (MDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMQ00E0JTQHAV00@brm-avmta-1.central.sun.com>; Mon, 13 Jul 2009 17:11:05 -0600 (MDT) Received: from sca-es-mail-2.sun.com ([192.18.43.133]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMQ00HLWTQHIXA0@brm-avmta-1.central.sun.com>; Mon, 13 Jul 2009 17:11:05 -0600 (MDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n6DNAx7W023408; Mon, 13 Jul 2009 16:10:59 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMQ00500TJ1BA00@fe-sfbay-09.sun.com>; Mon, 13 Jul 2009 16:10:59 -0700 (PDT) Received: from [10.1.106.212] ([unknown] [10.1.106.212]) by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMQ00AO1TQ74GD0@fe-sfbay-09.sun.com>; Mon, 13 Jul 2009 16:10:56 -0700 (PDT) Date: Mon, 13 Jul 2009 16:10:43 -0700 From: Afshin Salek Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <20090713225224.GN1274@Sun.COM> Sender: Afshin.Ardakani@sun.com To: Nicolas Williams Cc: Glenn Skinner , PSARC-ext@sun.com, cifs-eng@sun.com, Robert.Thurlow@sun.com, brian.wong@sun.com Message-id: <4A5BBEF3.5070408@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <20090713222450.GL1274@Sun.COM> <4A5BB8DA.804@sun.com> <20090713225224.GN1274@Sun.COM> User-Agent: Thunderbird 2.0.0.9 (X11/20080213) Content-Length: 1604 Status: RO X-Status: $$$$ X-UID: 0000000046 This is mentioned in the material that we would have a reparsed daemon is userspace which knows how do deal with the overall format and knows how to deal with service specific parts through a plugin model where each service would have its own module. This would make the whole thing easily expandable for future services/subsystems that want to use this mechanism for location redirection. Afshin Nicolas Williams wrote: > On Mon, Jul 13, 2009 at 03:44:42PM -0700, Afshin Salek wrote: >> This is supposed to be a generic and expandable mechanism. NFS/DFS >> referrals are just primary consumers of this mechanism at this >> point so this is not a protocol specific mechanism to have a per >> protocol extended attribute that are hard coded before hand. >> >> If we want to use extended attributes then we either have to reserve >> some namespace or use an extended attribute with a reserved name as an >> index. It also makes path name reduction very expensive because for each >> component we have to do a lot of extra stuff to see whether we are >> dealing with a reparse point or not and whether that reparse point >> contains the data that we are interested in. > > I'd stick with the ESA as the optimization, but EAs for storing the > referral if you can manage the namespace, else make the symlink thing > not-an-interface and provide a CLI, so you can change the implementation > later. > > Oh, hmmm, I just noticed that the reparse string format is private. But > I see no public interfaces for managing reparse points. Are UIs due > later? If so never mind my comments to date. > > Nico From Nicolas.Williams@Sun.COM Mon Jul 13 16:15:47 2009 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6DNFl8m013816 for ; Mon, 13 Jul 2009 16:15:47 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n6DNFk0r020866; Mon, 13 Jul 2009 16:15:46 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMQ00H0HTY93B00@nwk-avmta-1.sfbay.Sun.COM>; Mon, 13 Jul 2009 16:15:45 -0700 (PDT) Received: from binky.Central.Sun.COM ([129.153.128.104]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMQ004ZETY7UZB0@nwk-avmta-1.sfbay.Sun.COM>; Mon, 13 Jul 2009 16:15:43 -0700 (PDT) Received: from binky.Central.Sun.COM (localhost [127.0.0.1]) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3) with ESMTP id n6DNCtWb007076; Mon, 13 Jul 2009 18:12:55 -0500 (CDT) Received: (from nw141292@localhost) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3/Submit) id n6DNCte7007075; Mon, 13 Jul 2009 18:12:55 -0500 (CDT) Date: Mon, 13 Jul 2009 18:12:55 -0500 From: Nicolas Williams Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A5BBEF3.5070408@sun.com> To: Afshin Salek Cc: Glenn Skinner , PSARC-ext@Sun.COM, cifs-eng@Sun.COM, Robert.Thurlow@Sun.COM, Brian.Wong@Sun.COM Message-id: <20090713231255.GP1274@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <20090713222450.GL1274@Sun.COM> <4A5BB8DA.804@sun.com> <20090713225224.GN1274@Sun.COM> <4A5BBEF3.5070408@sun.com> X-Authentication-warning: binky.Central.Sun.COM: nw141292 set sender to Nicolas.Williams@sun.com using -f User-Agent: Mutt/1.5.7i Content-Length: 604 Status: RO X-Status: $$$$ X-UID: 0000000047 On Mon, Jul 13, 2009 at 04:10:43PM -0700, Afshin Salek wrote: > This is mentioned in the material that we would have a reparsed > daemon is userspace which knows how do deal with the overall format > and knows how to deal with service specific parts through a plugin > model where each service would have its own module. This would make > the whole thing easily expandable for future services/subsystems > that want to use this mechanism for location redirection. Indeed. What was not clear though was that a later case would add the necessary UI. Now I know that, so never mind my earlier comments. From Jordan.Brown@sun.com Mon Jul 13 16:31:10 2009 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6DNVAhO014400 for ; Mon, 13 Jul 2009 16:31:10 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n6DNV98k009698; Mon, 13 Jul 2009 17:31:10 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMQ00M07UNX4Y00@nwk-avmta-2.sfbay.sun.com>; Mon, 13 Jul 2009 16:31:09 -0700 (PDT) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMQ00KVZUNXMK10@nwk-avmta-2.sfbay.sun.com>; Mon, 13 Jul 2009 16:31:09 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n6DNV4Ku017528; Mon, 13 Jul 2009 16:31:04 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMQ00K00UDEGE00@fe-sfbay-09.sun.com>; Mon, 13 Jul 2009 16:31:04 -0700 (PDT) Received: from [129.145.155.47] ([unknown] [129.145.155.47]) by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMQ00HHKUNRO420@fe-sfbay-09.sun.com>; Mon, 13 Jul 2009 16:31:03 -0700 (PDT) Date: Mon, 13 Jul 2009 16:31:03 -0700 From: Jordan Brown Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <20090713231255.GP1274@Sun.COM> Sender: Jordan.Brown@sun.com To: Nicolas Williams Cc: Afshin Salek , Glenn Skinner , PSARC-ext@sun.com, cifs-eng@sun.com, Robert.Thurlow@sun.com, Brian.Wong@sun.com Message-id: <4A5BC3B7.3050807@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <20090713222450.GL1274@Sun.COM> <4A5BB8DA.804@sun.com> <20090713225224.GN1274@Sun.COM> <4A5BBEF3.5070408@sun.com> <20090713231255.GP1274@Sun.COM> User-Agent: Thunderbird 2.0.0.14 (X11/20080505) Content-Length: 795 Status: RO X-Status: $$$$ X-UID: 0000000048 Yeah, but having this format be user-visible in so prominent a place (ls -l output) makes it pretty difficult to make it really be not-an-interface. Nicolas Williams wrote: > On Mon, Jul 13, 2009 at 04:10:43PM -0700, Afshin Salek wrote: >> This is mentioned in the material that we would have a reparsed >> daemon is userspace which knows how do deal with the overall format >> and knows how to deal with service specific parts through a plugin >> model where each service would have its own module. This would make >> the whole thing easily expandable for future services/subsystems >> that want to use this mechanism for location redirection. > > Indeed. What was not clear though was that a later case would add the > necessary UI. Now I know that, so never mind my earlier comments. > From Darren.Moffat@sun.com Tue Jul 14 01:20:58 2009 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6E8Kvtl006241 for ; Tue, 14 Jul 2009 01:20:58 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6E8Kul9011092; Tue, 14 Jul 2009 09:20:57 +0100 (BST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMR0090RJ6WUE00@brm-avmta-1.central.sun.com>; Tue, 14 Jul 2009 02:20:56 -0600 (MDT) Received: from gmp-eb-inf-2.sun.com ([192.18.6.24]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMR00369J6UGO30@brm-avmta-1.central.sun.com>; Tue, 14 Jul 2009 02:20:55 -0600 (MDT) Received: from fe-emea-10.sun.com (gmp-eb-lb-1-fe3.eu.sun.com [192.18.6.10]) by gmp-eb-inf-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n6E8KmM9016539; Tue, 14 Jul 2009 08:20:48 +0000 (GMT) Received: from conversion-daemon.fe-emea-10.sun.com by fe-emea-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMR00800I382D00@fe-emea-10.sun.com>; Tue, 14 Jul 2009 09:20:48 +0100 (BST) Received: from [129.156.173.199] ([unknown] [129.156.173.199]) by fe-emea-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMR00264J5VOKA0@fe-emea-10.sun.com>; Tue, 14 Jul 2009 09:20:19 +0100 (BST) Date: Tue, 14 Jul 2009 09:20:19 +0100 From: Darren J Moffat Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A5BB8DA.804@sun.com> Sender: Darren.Moffat@sun.com To: Afshin Salek Cc: Nicolas Williams , Glenn Skinner , PSARC-ext@sun.com, cifs-eng@sun.com, Robert.Thurlow@sun.com, Brian.Wong@sun.com Message-id: <4A5C3FC3.2020508@Sun.COM> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <20090713222450.GL1274@Sun.COM> <4A5BB8DA.804@sun.com> User-Agent: Thunderbird 2.0.0.18 (X11/20090127) Content-Length: 1715 Status: RO X-Status: $$$$ X-UID: 0000000049 Afshin Salek wrote: > This is supposed to be a generic and expandable mechanism. NFS/DFS > referrals are just primary consumers of this mechanism at this > point so this is not a protocol specific mechanism to have a per > protocol extended attribute that are hard coded before hand. > > If we want to use extended attributes then we either have to reserve > some namespace or use an extended attribute with a reserved name as an > index. But you are already heading down that path by having a new system attribute. You are also reserving namespace in symlink formats with the case as specified so you don't get out of that problem. Given you need a new system attribute and backup software needs to be able to back that up all your arguments based on backup software for using symlinks don't hold up for me. > It also makes path name reduction very expensive because for each > component we have to do a lot of extra stuff to see whether we are > dealing with a reparse point or not and whether that reparse point > contains the data that we are interested in. Aren't you already doing that by looking at the new attribute and then only parsing the symlink data if it is tagged as a reparse point. I agree with Nico I think this case should use a system attribute as the "tag" like it does now but store the data in an extended attribute (ie the runat ones) that shouldn't require you wait on any new functionality in ZFS. I really don't like the overloading of symlinks particularly given that we have an extended attribute and system attribute capability already. Even more so that this case is actually proposing using a new (all be it) 1 bit system attribute anyway. -- Darren J Moffat From Afshin.Ardakani@sun.com Tue Jul 14 02:29:50 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6E9TnE9008069 for ; Tue, 14 Jul 2009 02:29:49 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n6E9TcCL028182; Tue, 14 Jul 2009 17:29:48 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMR00H01MDK5X00@brm-avmta-1.central.sun.com>; Tue, 14 Jul 2009 03:29:44 -0600 (MDT) Received: from brmea-mail-2.sun.com ([192.18.98.43]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMR003QEMDKGZ70@brm-avmta-1.central.sun.com>; Tue, 14 Jul 2009 03:29:44 -0600 (MDT) Received: from fe-amer-09.sun.com ([192.18.109.79]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6E9ThQL008594; Tue, 14 Jul 2009 09:29:43 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMR00E00LXWON00@mail-amer.sun.com>; Tue, 14 Jul 2009 03:29:43 -0600 (MDT) Received: from [192.168.1.102] ([unknown] [98.164.210.173]) by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMR001GTMDD4NF0@mail-amer.sun.com>; Tue, 14 Jul 2009 03:29:43 -0600 (MDT) Date: Tue, 14 Jul 2009 02:29:38 -0700 From: Afshin Salek Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A5C3FC3.2020508@Sun.COM> Sender: Afshin.Ardakani@sun.com To: Darren J Moffat Cc: Nicolas Williams , Glenn Skinner , PSARC-ext@sun.com, cifs-eng@sun.com, Robert.Thurlow@sun.com, Brian.Wong@sun.com Message-id: <4A5C5002.5030107@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <20090713222450.GL1274@Sun.COM> <4A5BB8DA.804@sun.com> <4A5C3FC3.2020508@Sun.COM> User-Agent: Thunderbird 2.0.0.22pre (Windows/20090515) Content-Length: 2290 Status: RO X-Status: $$$$ X-UID: 0000000050 Darren J Moffat wrote: > Afshin Salek wrote: >> This is supposed to be a generic and expandable mechanism. NFS/DFS >> referrals are just primary consumers of this mechanism at this >> point so this is not a protocol specific mechanism to have a per >> protocol extended attribute that are hard coded before hand. >> >> If we want to use extended attributes then we either have to reserve >> some namespace or use an extended attribute with a reserved name as an >> index. > > But you are already heading down that path by having a new system > attribute. You are also reserving namespace in symlink formats with the > case as specified so you don't get out of that problem. > We are not really reserving any namespace because the format that is proposed here cannot be a valid target for a regular symlink. > Given you need a new system attribute and backup software needs to be > able to back that up all your arguments based on backup software for > using symlinks don't hold up for me. > Backup softwares don't need to backup the system attribute because at the time of restore that bit will be set anyways because fop_symlink detects the @{REPARSE tag and will set the reparse attribute as mentioned in the case. > > It also makes path name reduction very expensive because for each >> component we have to do a lot of extra stuff to see whether we are >> dealing with a reparse point or not and whether that reparse point >> contains the data that we are interested in. > > Aren't you already doing that by looking at the new attribute and then > only parsing the symlink data if it is tagged as a reparse point. I The reparse detection part would be the same but I'm not sure doing a VOP_READLINK would be as expensive as working with extended attributes. Afshin > agree with Nico I think this case should use a system attribute as the > "tag" like it does now but store the data in an extended attribute (ie > the runat ones) that shouldn't require you wait on any new functionality > in ZFS. > > I really don't like the overloading of symlinks particularly given that > we have an extended attribute and system attribute capability already. > Even more so that this case is actually proposing using a new (all be > it) 1 bit system attribute anyway. > From amw@sun.com Tue Jul 14 02:51:41 2009 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6E9pfLu008415 for ; Tue, 14 Jul 2009 02:51:41 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n6E9pc3w008890; Tue, 14 Jul 2009 03:51:40 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMR0090RNE31I00@nwk-avmta-1.sfbay.Sun.COM>; Tue, 14 Jul 2009 02:51:39 -0700 (PDT) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMR00EPWNE2M7B0@nwk-avmta-1.sfbay.Sun.COM>; Tue, 14 Jul 2009 02:51:39 -0700 (PDT) Received: from fe-amer-09.sun.com ([192.18.109.79]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6E9pcgJ002275; Tue, 14 Jul 2009 09:51:38 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMR00200N8OXY00@mail-amer.sun.com>; Tue, 14 Jul 2009 03:51:38 -0600 (MDT) Received: from TOSHIBA ([unknown] [129.150.24.129]) by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMR00IAMNE0E130@mail-amer.sun.com>; Tue, 14 Jul 2009 03:51:37 -0600 (MDT) Date: Tue, 14 Jul 2009 02:51:31 -0700 From: "Alan.M.Wright" Subject: Re: 2009/387 [Pathname Reparse Points] Sender: Alan.M.Wright@sun.com To: Darren J Moffat , Afshin Salek Cc: Nicolas Williams , Glenn Skinner , PSARC-ext@sun.com, cifs-eng@sun.com, Robert.Thurlow@sun.com, Brian.Wong@sun.com Message-id: <23408EBC7ADF4918A35E43F6D6BF1E95@TOSHIBA> MIME-version: 1.0 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2900.5579 X-Mailer: Microsoft Outlook Express 6.00.2900.5512 Content-type: text/plain; CHARSET=US-ASCII; reply-type=response; format=flowed Content-transfer-encoding: 7BIT X-Priority: 3 X-MSMail-priority: Normal X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <20090713222450.GL1274@Sun.COM> <4A5BB8DA.804@sun.com> <4A5C3FC3.2020508@Sun.COM> Content-Length: 1915 Status: RO X-Status: $$$$ X-UID: 0000000051 Darren J Moffat wrote: > Afshin Salek wrote: >> This is supposed to be a generic and expandable mechanism. NFS/DFS >> referrals are just primary consumers of this mechanism at this >> point so this is not a protocol specific mechanism to have a per >> protocol extended attribute that are hard coded before hand. >> >> If we want to use extended attributes then we either have to reserve >> some namespace or use an extended attribute with a reserved name as an >> index. > > But you are already heading down that path by having a new system > attribute. You are also reserving namespace in symlink formats with the > case as specified so you don't get out of that problem. > > Given you need a new system attribute and backup software needs to be > able to back that up all your arguments based on backup software for > using symlinks don't hold up for me. > > > It also makes path name reduction very expensive because for each >> component we have to do a lot of extra stuff to see whether we are >> dealing with a reparse point or not and whether that reparse point >> contains the data that we are interested in. > > Aren't you already doing that by looking at the new attribute and then > only parsing the symlink data if it is tagged as a reparse point. I > agree with Nico I think this case should use a system attribute as the > "tag" like it does now but store the data in an extended attribute (ie > the runat ones) that shouldn't require you wait on any new functionality > in ZFS. We can't do that with this proposal because symlink can't have extended attributes. Alan > I really don't like the overloading of symlinks particularly given that > we have an extended attribute and system attribute capability already. > Even more so that this case is actually proposing using a new (all be > it) 1 bit system attribute anyway. > > -- > Darren J Moffat > From amw@sun.com Tue Jul 14 02:55:48 2009 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6E9tlTu008729 for ; Tue, 14 Jul 2009 02:55:48 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6E9tjGS006871; Tue, 14 Jul 2009 10:55:46 +0100 (BST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMR00J07NKXYF00@brm-avmta-1.central.sun.com>; Tue, 14 Jul 2009 03:55:45 -0600 (MDT) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMR003IQNKXGWA0@brm-avmta-1.central.sun.com>; Tue, 14 Jul 2009 03:55:45 -0600 (MDT) Received: from fe-amer-09.sun.com ([192.18.109.79]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6E9tiPH021689; Tue, 14 Jul 2009 09:55:44 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMR00400NFZSQ00@mail-amer.sun.com>; Tue, 14 Jul 2009 03:55:44 -0600 (MDT) Received: from TOSHIBA ([unknown] [129.150.24.129]) by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMR00IZ7NKVE130@mail-amer.sun.com>; Tue, 14 Jul 2009 03:55:44 -0600 (MDT) Date: Tue, 14 Jul 2009 02:55:38 -0700 From: "Alan.M.Wright" Subject: Re: 2009/387 [Pathname Reparse Points] Sender: Alan.M.Wright@sun.com To: Afshin Salek , Darren J Moffat Cc: Nicolas Williams , Glenn Skinner , PSARC-ext@sun.com, cifs-eng@sun.com, Robert.Thurlow@sun.com, Brian.Wong@sun.com Message-id: <77244D843F7F4DBF9DA1F80C3B5637A1@TOSHIBA> MIME-version: 1.0 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2900.5579 X-Mailer: Microsoft Outlook Express 6.00.2900.5512 Content-type: text/plain; CHARSET=US-ASCII; reply-type=response; format=flowed Content-transfer-encoding: 7BIT X-Priority: 3 X-MSMail-priority: Normal X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <20090713222450.GL1274@Sun.COM> <4A5BB8DA.804@sun.com> <4A5C3FC3.2020508@Sun.COM> <4A5C5002.5030107@sun.com> Content-Length: 1029 Status: RO X-Status: $$$$ X-UID: 0000000052 Afshin Salek wrote: > Darren J Moffat wrote: >> Afshin Salek wrote: >>> This is supposed to be a generic and expandable mechanism. NFS/DFS >>> referrals are just primary consumers of this mechanism at this >>> point so this is not a protocol specific mechanism to have a per >>> protocol extended attribute that are hard coded before hand. >>> >>> If we want to use extended attributes then we either have to reserve >>> some namespace or use an extended attribute with a reserved name as an >>> index. >> >> But you are already heading down that path by having a new system >> attribute. You are also reserving namespace in symlink formats with the >> case as specified so you don't get out of that problem. >> > > We are not really reserving any namespace because the format > that is proposed here cannot be a valid target for a regular symlink. The format may not be usable under normal circumstances but we are reserving it and the OS will take specific action if it is encountered. Alan From Darren.Moffat@sun.com Tue Jul 14 02:59:28 2009 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6E9xRbu008801 for ; Tue, 14 Jul 2009 02:59:27 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6E9xKxc008437; Tue, 14 Jul 2009 10:59:26 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMR00J03NR28600@nwk-avmta-2.sfbay.sun.com>; Tue, 14 Jul 2009 02:59:26 -0700 (PDT) Received: from gmp-eb-inf-1.sun.com ([192.18.6.21]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMR00AYINQZCA90@nwk-avmta-2.sfbay.sun.com>; Tue, 14 Jul 2009 02:59:24 -0700 (PDT) Received: from fe-emea-09.sun.com (gmp-eb-lb-1-fe3.eu.sun.com [192.18.6.10]) by gmp-eb-inf-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n6E9xHam003738; Tue, 14 Jul 2009 09:59:17 +0000 (GMT) Received: from conversion-daemon.fe-emea-09.sun.com by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMR00L00MTWEI00@fe-emea-09.sun.com>; Tue, 14 Jul 2009 10:59:17 +0100 (BST) Received: from [129.156.173.199] ([unknown] [129.156.173.199]) by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMR005C7NQSBH20@fe-emea-09.sun.com>; Tue, 14 Jul 2009 10:59:16 +0100 (BST) Date: Tue, 14 Jul 2009 10:59:16 +0100 From: Darren J Moffat Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A5C5002.5030107@sun.com> Sender: Darren.Moffat@sun.com To: Afshin Salek Cc: Nicolas Williams , Glenn Skinner , PSARC-ext@sun.com, cifs-eng@sun.com, Robert.Thurlow@sun.com, Brian.Wong@sun.com Message-id: <4A5C56F4.5080607@Sun.COM> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <20090713222450.GL1274@Sun.COM> <4A5BB8DA.804@sun.com> <4A5C3FC3.2020508@Sun.COM> <4A5C5002.5030107@sun.com> User-Agent: Thunderbird 2.0.0.18 (X11/20090127) Content-Length: 2962 Status: RO X-Status: $$$$ X-UID: 0000000053 Afshin Salek wrote: > > > Darren J Moffat wrote: >> Afshin Salek wrote: >>> This is supposed to be a generic and expandable mechanism. NFS/DFS >>> referrals are just primary consumers of this mechanism at this >>> point so this is not a protocol specific mechanism to have a per >>> protocol extended attribute that are hard coded before hand. >>> >>> If we want to use extended attributes then we either have to reserve >>> some namespace or use an extended attribute with a reserved name as an >>> index. >> >> But you are already heading down that path by having a new system >> attribute. You are also reserving namespace in symlink formats with >> the case as specified so you don't get out of that problem. >> > > We are not really reserving any namespace because the format > that is proposed here cannot be a valid target for a regular symlink. Yes you are, by adopting the BSD magic link syntax that is reserving namespace. It is reserving symlink content starting with @ and specifically the part of the @ namespace that begins @{REPARSE. How is that not reserving namespace ? Today this works: braveheart:pts/1# ls -l total 4 -rw-r--r-- 1 root root 12 Jul 14 11:10 @{REPARSE} lrwxrwxrwx 1 root root 10 Jul 14 11:07 foo -> @{REPARSE} braveheart:pts/1# cat foo hello world braveheart:pts/1# cat @\{REPARSE\} hello world After your change what happens ? Are those slightly crazy and contrived filenames ? Yes but they are valid today. >> Given you need a new system attribute and backup software needs to be >> able to back that up all your arguments based on backup software for >> using symlinks don't hold up for me. >> > > Backup softwares don't need to backup the system attribute because at > the time of restore that bit will be set anyways because fop_symlink > detects the @{REPARSE tag and will set the reparse attribute as > mentioned in the case. Okay that is helpful. So there is now code in the normal fop_symlink path to look at the content for the @{REPARSE namespace and set a system attribute. What happens if I already had symlinks that looked like that ? >> > It also makes path name reduction very expensive because for each >>> component we have to do a lot of extra stuff to see whether we are >>> dealing with a reparse point or not and whether that reparse point >>> contains the data that we are interested in. >> >> Aren't you already doing that by looking at the new attribute and then >> only parsing the symlink data if it is tagged as a reparse point. I > > The reparse detection part would be the same but I'm not sure doing > a VOP_READLINK would be as expensive as working with extended > attributes. So you haven't actually tried it yet you are saying it is too expensive. How expensive is too expensive ? What is the performance criteria and how do you know that VOP_READLINK meets it yet using an extended attribute wouldn't ? -- Darren J Moffat From Darren.Moffat@sun.com Tue Jul 14 03:00:24 2009 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6EA0OTv008893 for ; Tue, 14 Jul 2009 03:00:24 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n6EA0NrX012354; Tue, 14 Jul 2009 04:00:23 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMR00J07NSMAZ00@nwk-avmta-2.sfbay.sun.com>; Tue, 14 Jul 2009 03:00:22 -0700 (PDT) Received: from gmp-eb-inf-2.sun.com ([192.18.6.24]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMR00ARLNSKCBB0@nwk-avmta-2.sfbay.sun.com>; Tue, 14 Jul 2009 03:00:21 -0700 (PDT) Received: from fe-emea-09.sun.com (gmp-eb-lb-1-fe3.eu.sun.com [192.18.6.10]) by gmp-eb-inf-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n6EA0E0P003086; Tue, 14 Jul 2009 10:00:14 +0000 (GMT) Received: from conversion-daemon.fe-emea-09.sun.com by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMR00L00MTWEI00@fe-emea-09.sun.com>; Tue, 14 Jul 2009 11:00:14 +0100 (BST) Received: from [129.156.173.199] ([unknown] [129.156.173.199]) by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMR005DTNS6BH20@fe-emea-09.sun.com>; Tue, 14 Jul 2009 11:00:06 +0100 (BST) Date: Tue, 14 Jul 2009 11:00:06 +0100 From: Darren J Moffat Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <23408EBC7ADF4918A35E43F6D6BF1E95@TOSHIBA> Sender: Darren.Moffat@sun.com To: "Alan.M.Wright" Cc: Afshin Salek , Nicolas Williams , Glenn Skinner , PSARC-ext@sun.com, cifs-eng@sun.com, Robert.Thurlow@sun.com, Brian.Wong@sun.com Message-id: <4A5C5726.4010000@Sun.COM> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <20090713222450.GL1274@Sun.COM> <4A5BB8DA.804@sun.com> <4A5C3FC3.2020508@Sun.COM> <23408EBC7ADF4918A35E43F6D6BF1E95@TOSHIBA> User-Agent: Thunderbird 2.0.0.18 (X11/20090127) Content-Length: 607 Status: RO X-Status: $$$$ X-UID: 0000000054 Alan.M.Wright wrote: >> Aren't you already doing that by looking at the new attribute and then >> only parsing the symlink data if it is tagged as a reparse point. I >> agree with Nico I think this case should use a system attribute as the >> "tag" like it does now but store the data in an extended attribute (ie >> the runat ones) that shouldn't require you wait on any new >> functionality in ZFS. > > We can't do that with this proposal because symlink can't have > extended attributes. The point is NOT to use symlinks at all but to use a file with an extended attribute. -- Darren J Moffat From amw@sun.com Tue Jul 14 03:21:40 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6EALdOR009464 for ; Tue, 14 Jul 2009 03:21:40 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n6EALSaQ021936; Tue, 14 Jul 2009 18:21:38 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMR00F03OS21600@nwk-avmta-1.sfbay.Sun.COM>; Tue, 14 Jul 2009 03:21:38 -0700 (PDT) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMR00E1VOS1MCD0@nwk-avmta-1.sfbay.Sun.COM>; Tue, 14 Jul 2009 03:21:38 -0700 (PDT) Received: from fe-amer-10.sun.com ([192.18.109.80]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6EALbud029776; Tue, 14 Jul 2009 10:21:37 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMR00G00OLXGX00@mail-amer.sun.com>; Tue, 14 Jul 2009 04:21:37 -0600 (MDT) Received: from TOSHIBA ([unknown] [129.150.24.129]) by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMR001DRORZUY60@mail-amer.sun.com>; Tue, 14 Jul 2009 04:21:36 -0600 (MDT) Date: Tue, 14 Jul 2009 03:21:30 -0700 From: "Alan.M.Wright" Subject: Re: 2009/387 [Pathname Reparse Points] Sender: Alan.M.Wright@sun.com To: Darren J Moffat Cc: Afshin Salek , Nicolas Williams , Glenn Skinner , PSARC-ext@sun.com, cifs-eng@sun.com, Robert.Thurlow@sun.com, Brian.Wong@sun.com Message-id: <7B74FF619D8947F2A1062312E072B22C@TOSHIBA> MIME-version: 1.0 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2900.5579 X-Mailer: Microsoft Outlook Express 6.00.2900.5512 Content-type: text/plain; CHARSET=US-ASCII; reply-type=response; format=flowed Content-transfer-encoding: 7BIT X-Priority: 3 X-MSMail-priority: Normal X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <20090713222450.GL1274@Sun.COM> <4A5BB8DA.804@sun.com> <4A5C3FC3.2020508@Sun.COM> <23408EBC7ADF4918A35E43F6D6BF1E95@TOSHIBA> <4A5C5726.4010000@Sun.COM> Content-Length: 1372 Status: RO X-Status: $$$$ X-UID: 0000000055 Darren J Moffat wrote: > Alan.M.Wright wrote: >>> Aren't you already doing that by looking at the new attribute and then >>> only parsing the symlink data if it is tagged as a reparse point. I >>> agree with Nico I think this case should use a system attribute as the >>> "tag" like it does now but store the data in an extended attribute (ie >>> the runat ones) that shouldn't require you wait on any new >>> functionality in ZFS. >> >> We can't do that with this proposal because symlink can't have >> extended attributes. > > The point is NOT to use symlinks at all but to use a file with an > extended attribute. We've looked at that and similar options in detail within the project team over the past few months. We looked at using files, directories, mount points and introducing a new object type. Each one resulted in issues that seemed insurmountable: using a file to represent a disjoint name space confusing applications that expect to perform normal file operations on it, applications expecting to read or descend into directories, mount points being involved in client-server exchanges and the ramifications of ensuring that a new object type wouldn't result in erroneous behavior. Ultimately, the team came to the conclusion that it wasn't feasible to pursue any of those options and proposed the symlink option. Alan From Darren.Moffat@sun.com Tue Jul 14 03:33:34 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6EAXXnQ009680 for ; Tue, 14 Jul 2009 03:33:33 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n6EAXUCR027283; Tue, 14 Jul 2009 18:33:32 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMR0000VPBTUO00@brm-avmta-1.central.sun.com>; Tue, 14 Jul 2009 04:33:29 -0600 (MDT) Received: from gmp-eb-inf-1.sun.com ([192.18.6.21]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMR003IRPBRGUC0@brm-avmta-1.central.sun.com>; Tue, 14 Jul 2009 04:33:28 -0600 (MDT) Received: from fe-emea-09.sun.com (gmp-eb-lb-1-fe3.eu.sun.com [192.18.6.10]) by gmp-eb-inf-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n6EAXLjP008694; Tue, 14 Jul 2009 10:33:21 +0000 (GMT) Received: from conversion-daemon.fe-emea-09.sun.com by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMR00900OXQIU00@fe-emea-09.sun.com>; Tue, 14 Jul 2009 11:33:21 +0100 (BST) Received: from [129.156.173.199] ([unknown] [129.156.173.199]) by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMR005ZMPBEBH30@fe-emea-09.sun.com>; Tue, 14 Jul 2009 11:33:14 +0100 (BST) Date: Tue, 14 Jul 2009 11:33:14 +0100 From: Darren J Moffat Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <7B74FF619D8947F2A1062312E072B22C@TOSHIBA> Sender: Darren.Moffat@sun.com To: "Alan.M.Wright" Cc: Afshin Salek , Nicolas Williams , Glenn Skinner , PSARC-ext@sun.com, cifs-eng@sun.com, Robert.Thurlow@sun.com, Brian.Wong@sun.com Message-id: <4A5C5EEA.50202@Sun.COM> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <20090713222450.GL1274@Sun.COM> <4A5BB8DA.804@sun.com> <4A5C3FC3.2020508@Sun.COM> <23408EBC7ADF4918A35E43F6D6BF1E95@TOSHIBA> <4A5C5726.4010000@Sun.COM> <7B74FF619D8947F2A1062312E072B22C@TOSHIBA> User-Agent: Thunderbird 2.0.0.18 (X11/20090127) Content-Length: 2160 Status: RO X-Status: $$$$ X-UID: 0000000056 Alan.M.Wright wrote: > Darren J Moffat wrote: >> Alan.M.Wright wrote: >>>> Aren't you already doing that by looking at the new attribute and >>>> then only parsing the symlink data if it is tagged as a reparse >>>> point. I agree with Nico I think this case should use a system >>>> attribute as the "tag" like it does now but store the data in an >>>> extended attribute (ie the runat ones) that shouldn't require you >>>> wait on any new functionality in ZFS. >>> >>> We can't do that with this proposal because symlink can't have >>> extended attributes. >> >> The point is NOT to use symlinks at all but to use a file with an >> extended attribute. > > We've looked at that and similar options in detail within the project team > over the past few months. We looked at using files, directories, mount > points and introducing a new object type. Each one resulted in issues > that seemed insurmountable: using a file to represent a disjoint name space > confusing applications that expect to perform normal file operations on it, > applications expecting to read or descend into directories, mount points > being involved in client-server exchanges and the ramifications of ensuring > that a new object type wouldn't result in erroneous behavior. Ultimately, > the team came to the conclusion that it wasn't feasible to pursue any of > those options and proposed the symlink option. That is better info than Afshin gave, his reply led me to believe that no testing of other means had be fully investigated (at least performance wise). I think given that it seems symlinks are the least worst choice - better than a new object type anyway. This case is reserving namespace and I believe it is also setting precedent that we can do so with symlinks. The prefix characters ('@{' or is it really just '@" ?) you are reserving aren't reserved by default in BSD but are only enabled when a filesystem option is enabled. We don't have generic options for the VFS layer like BSD but there could be a ZFS dataset level option to enable/disable this. I'm not sure if it is really worth it though. -- Darren J Moffat From amw@sun.com Tue Jul 14 04:00:18 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6EB0HR2013172 for ; Tue, 14 Jul 2009 04:00:18 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n6EB0FQR008911; Tue, 14 Jul 2009 19:00:16 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMR00M01QKFX500@nwk-avmta-1.sfbay.Sun.COM>; Tue, 14 Jul 2009 04:00:15 -0700 (PDT) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMR00MCIQKFDS00@nwk-avmta-1.sfbay.Sun.COM>; Tue, 14 Jul 2009 04:00:15 -0700 (PDT) Received: from fe-amer-09.sun.com ([192.18.109.79]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6EB0EpO021562; Tue, 14 Jul 2009 11:00:14 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMR00B00QBT4L00@mail-amer.sun.com>; Tue, 14 Jul 2009 05:00:14 -0600 (MDT) Received: from TOSHIBA ([unknown] [68.228.88.55]) by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMR00IO3QKDE1E0@mail-amer.sun.com>; Tue, 14 Jul 2009 05:00:14 -0600 (MDT) Date: Tue, 14 Jul 2009 04:00:07 -0700 From: "Alan.M.Wright" Subject: Re: 2009/387 [Pathname Reparse Points] Sender: Alan.M.Wright@sun.com To: Darren J Moffat Cc: Afshin Salek , Nicolas Williams , Glenn Skinner , PSARC-ext@sun.com, cifs-eng@sun.com, Robert.Thurlow@sun.com, Brian.Wong@sun.com Message-id: <3464DD17027C4B35AF3A2AE3B468B7E8@TOSHIBA> MIME-version: 1.0 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2900.5579 X-Mailer: Microsoft Outlook Express 6.00.2900.5512 Content-type: text/plain; CHARSET=US-ASCII; reply-type=response; format=flowed Content-transfer-encoding: 7BIT X-Priority: 3 X-MSMail-priority: Normal X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <20090713222450.GL1274@Sun.COM> <4A5BB8DA.804@sun.com> <4A5C3FC3.2020508@Sun.COM> <23408EBC7ADF4918A35E43F6D6BF1E95@TOSHIBA> <4A5C5726.4010000@Sun.COM> <7B74FF619D8947F2A1062312E072B22C@TOSHIBA> <4A5C5EEA.50202@Sun.COM> Content-Length: 2566 Status: RO X-Status: $$$$ X-UID: 0000000057 Darren J Moffat wrote: > Alan.M.Wright wrote: >> Darren J Moffat wrote: >>> Alan.M.Wright wrote: >>>>> Aren't you already doing that by looking at the new attribute and then >>>>> only parsing the symlink data if it is tagged as a reparse point. I >>>>> agree with Nico I think this case should use a system attribute as the >>>>> "tag" like it does now but store the data in an extended attribute (ie >>>>> the runat ones) that shouldn't require you wait on any new functionality >>>>> in ZFS. >>>> >>>> We can't do that with this proposal because symlink can't have >>>> extended attributes. >>> >>> The point is NOT to use symlinks at all but to use a file with an extended >>> attribute. >> >> We've looked at that and similar options in detail within the project team >> over the past few months. We looked at using files, directories, mount >> points and introducing a new object type. Each one resulted in issues >> that seemed insurmountable: using a file to represent a disjoint name space >> confusing applications that expect to perform normal file operations on it, >> applications expecting to read or descend into directories, mount points >> being involved in client-server exchanges and the ramifications of ensuring >> that a new object type wouldn't result in erroneous behavior. Ultimately, >> the team came to the conclusion that it wasn't feasible to pursue any of >> those options and proposed the symlink option. > > That is better info than Afshin gave, his reply led me to believe that no > testing of other means had be fully investigated (at least performance > wise). > > I think given that it seems symlinks are the least worst choice - better > than a new object type anyway. > > This case is reserving namespace and I believe it is also setting precedent > that we can do so with symlinks. > > The prefix characters ('@{' or is it really just '@" ?) you are reserving > aren't reserved by default in BSD but are only enabled when a filesystem > option is enabled. We don't have generic options for the VFS layer like BSD > but there could be a ZFS dataset level option to enable/disable this. I'm > not sure if it is really worth it though. We are reserving '@{...}', i.e. the name must start with @{ and end with }. We considered both a global option and per file system options but, on the basis that no-one could identify a scenario in which something would misinterpret these objects and behave badly, we decided not to propose an enable/disable switch. Alan From Darren.Moffat@sun.com Tue Jul 14 04:07:59 2009 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6EB7xa0016787 for ; Tue, 14 Jul 2009 04:07:59 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n6EB7u6S020668; Tue, 14 Jul 2009 04:07:59 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMR0001RQXAQB00@nwk-avmta-2.sfbay.sun.com>; Tue, 14 Jul 2009 04:07:58 -0700 (PDT) Received: from gmp-eb-inf-1.sun.com ([192.18.6.21]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMR000IMQX7AJ10@nwk-avmta-2.sfbay.sun.com>; Tue, 14 Jul 2009 04:07:56 -0700 (PDT) Received: from fe-emea-09.sun.com (gmp-eb-lb-1-fe3.eu.sun.com [192.18.6.10]) by gmp-eb-inf-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n6EB7nbf013593; Tue, 14 Jul 2009 11:07:49 +0000 (GMT) Received: from conversion-daemon.fe-emea-09.sun.com by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMR00900PM3W200@fe-emea-09.sun.com>; Tue, 14 Jul 2009 12:07:49 +0100 (BST) Received: from [129.156.173.199] ([unknown] [129.156.173.199]) by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMR005QHQX0BH50@fe-emea-09.sun.com>; Tue, 14 Jul 2009 12:07:49 +0100 (BST) Date: Tue, 14 Jul 2009 12:07:48 +0100 From: Darren J Moffat Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <3464DD17027C4B35AF3A2AE3B468B7E8@TOSHIBA> Sender: Darren.Moffat@sun.com To: "Alan.M.Wright" Cc: Afshin Salek , Nicolas Williams , Glenn Skinner , PSARC-ext@sun.com, cifs-eng@sun.com, Robert.Thurlow@sun.com, Brian.Wong@sun.com Message-id: <4A5C6704.8050801@Sun.COM> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <20090713222450.GL1274@Sun.COM> <4A5BB8DA.804@sun.com> <4A5C3FC3.2020508@Sun.COM> <23408EBC7ADF4918A35E43F6D6BF1E95@TOSHIBA> <4A5C5726.4010000@Sun.COM> <7B74FF619D8947F2A1062312E072B22C@TOSHIBA> <4A5C5EEA.50202@Sun.COM> <3464DD17027C4B35AF3A2AE3B468B7E8@TOSHIBA> User-Agent: Thunderbird 2.0.0.18 (X11/20090127) Content-Length: 1627 Status: RO X-Status: $$$$ X-UID: 0000000058 Alan.M.Wright wrote: > We are reserving '@{...}', i.e. the name must start with @{ and end with }. > > We considered both a global option and per file system options but, on > the basis that no-one could identify a scenario in which something would > misinterpret these objects and behave badly, we decided not to propose > an enable/disable switch. So that fact that you can today create a symlink pointing to such a file isn't a case of behaving badly ? estale:pts/83$ echo "hello world" > "@{REPARSE:NFS:00000}" estale:pts/83$ ln -s @\{REPARSE:NFS:00000\} @\{REPARSE:NFS:00001\} estale:pts/83$ ls -l total 2 -rw-r--r-- 1 darrenm staff 12 Jul 14 12:03 @{REPARSE:NFS:00000} lrwxrwxrwx 1 darrenm staff 20 Jul 14 12:04 @{REPARSE:NFS:00001} -> @{REPARSE:NFS:00000} estale:pts/83$ cat @\{REPARSE:NFS:00000\} hello world estale:pts/83$ cat @\{REPARSE:NFS:00001\} hello world Just because you don't create files beginning @{ and ending } doesn't mean they don't exist. Given all the rest of the data that has to be in there I think the changes are very very slim. The issue isn't so much with the REPARSE but the reserving of all @{...} which this case claims to do. Again I think the chances of a problem are slim but there must be a reason why BSD made this optional, no ? Note I'm no longer arguing against the case I'm just recording the fact that there is a risk in reserving this symlink content namespace though it is a very small one. Given everything I've said and the discussion that has been had I'm now okay supporting this case and give it my @{PSARC:+1}. -- Darren J Moffat From kmcdonald@egenera.com Tue Jul 14 08:29:58 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6EFTv4h024106 for ; Tue, 14 Jul 2009 08:29:58 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n6EFTqIp018949; Tue, 14 Jul 2009 23:29:53 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMS0040P31RKT00@nwk-avmta-1.sfbay.Sun.COM>; Tue, 14 Jul 2009 08:29:51 -0700 (PDT) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMS002MR31L9DE0@nwk-avmta-1.sfbay.Sun.COM>; Tue, 14 Jul 2009 08:29:45 -0700 (PDT) Received: from relay43i.sun.com ([192.5.209.74]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6EFMk3F002464; Tue, 14 Jul 2009 15:29:45 +0000 (GMT) Received: from mms48es.mms.us.syntegra.com ([160.41.221.230] [160.41.221.230]) by relay43i.sun.com with ESMTP id BT-MMP-1312518; Tue, 14 Jul 2009 15:29:45 +0000 (Z) Received: from relay43i.sun.com (relay43i.sun.com [192.5.209.74]) by mms48es.mms.us.syntegra.com with ESMTP id BT-MMP-86103115; Tue, 14 Jul 2009 15:29:42 +0000 (Z) Received: from webaccess.egenera.com ([63.139.209.15] [63.139.209.15]) by relay4i.sun.com with ESMTP id BT-MMP-8202489; Tue, 14 Jul 2009 15:28:54 +0000 (Z) Received: from [172.23.2.193] ([172.23.2.193]) by webaccess.egenera.com with Microsoft SMTPSVC(6.0.3790.3959); Tue, 14 Jul 2009 11:28:32 -0400 Date: Tue, 14 Jul 2009 11:27:57 -0400 From: Kyle McDonald Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A5B570B.4030805@sun.com> To: Robert Thurlow Cc: Hugh McIntyre , Afshin.Ardakani@sun.com, PSARC-ext@sun.com, Brian.Wong@sun.com, cifs-eng@sun.com Message-id: <4A5CA3FD.7020102@Egenera.COM> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-Antispam: No, score=-2.6/5.0, scanned in 0.064sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <4a58a697.yQ8anaL/V6TEAoRX%Joerg.Schilling@fokus.fraunhofer.de> <4A5943C0.3090303@Sun.COM> <4a59bb20.CuZiqCKUmLZgrA4L%Joerg.Schilling@fokus.fraunhofer.de> <4A5AA07B.1020701@mcintyreweb.com> <4A5B570B.4030805@sun.com> User-Agent: Thunderbird 2.0.0.22 (Windows/20090605) X-OriginalArrivalTime: 14 Jul 2009 15:28:32.0837 (UTC) FILETIME=[BF3B0750:01CA0497] Content-Length: 2266 Status: RO X-Status: $$$$ X-UID: 0000000059 Robert Thurlow wrote: > Hugh McIntyre wrote: > >> But this raises the point that if these special reparse symlinks were >> implemented such that the reparsing also happened when the filesystem >> is mounted locally via mount_zfs or mount_lofs, not just remotely >> over CIFS or NFSv4, then most of the issues would go away. >> >> In other words if open(), stat(), nftw(), and similar system calls >> locally on the server hosting the files (including LOFS) access the >> linked-to file, most code will think the link is OK. Since in this >> case, "find -L" should point to the target of the reparse point >> (which hopefully exists). > > The project team is pretty interested in semantics seen by > NFS and CIFS clients, not local semantics, so we're not > terribly excited to do work to make local access act like > that; I get the motivations, but it's a whole lot of work. But that seems to ignore a whole class of servers, who are local NFS clients of both other servers *and* themselves. All of my data disks are always mounted in at /export/SOMETHING on my file servers. All of the clients of those servers mount those filesystems under /import, usually /import/SOMETHING, but sometimes /import/SOME/THING/ELSE. Many times the path used to access these filesystems on the clients needs to be a valid path on the server also. So I always make sure that the /import/.... path works on the server too - which for NFS and NIS and the Automounter doesn't really take any effort at all. I'll admit that I'm not an exprt on the newest features of NFSv4, and CIFS, so it's not entirely clear to me how, or when (or IF?) I'd ever want to use these reparse points, but unless I'm missing something, I know I'd want them to work locally the same as they will on the client. -Kyle > > There are also difficulties. We know we will want to set > up a reparse point that projects the same namespace for > both NFS and CIFS, and it isn't obvious what client code > we would bridge into locally. We would also like to be > able to back up reparse points, so local processes need to > access them without interpretation. > > Rob T > > _______________________________________________ > opensolaris-arc mailing list > opensolaris-arc@opensolaris.org From robert.thurlow@sun.com Tue Jul 14 10:10:18 2009 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6EHAHVG029547 for ; Tue, 14 Jul 2009 10:10:18 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6EHA2e5019810; Tue, 14 Jul 2009 18:10:14 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMS00L0V7P0V000@nwk-avmta-2.sfbay.sun.com>; Tue, 14 Jul 2009 10:10:12 -0700 (PDT) Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.63]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMS00DIC7OWWQD0@nwk-avmta-2.sfbay.sun.com>; Tue, 14 Jul 2009 10:10:08 -0700 (PDT) Received: from [10.7.250.15] (punchin-client-10-7-250-15.SFBay.Sun.COM [10.7.250.15]) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6EHA244420638; Tue, 14 Jul 2009 10:10:03 -0700 (PDT) Date: Tue, 14 Jul 2009 11:10:02 -0600 From: Robert Thurlow Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A5CA3FD.7020102@Egenera.COM> To: Kyle McDonald Cc: Hugh McIntyre , Afshin.Ardakani@sun.com, PSARC-ext@sun.com, brian.wong@sun.com, cifs-eng@sun.com Message-id: <4A5CBBEA.1030100@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <4a58a697.yQ8anaL/V6TEAoRX%Joerg.Schilling@fokus.fraunhofer.de> <4A5943C0.3090303@Sun.COM> <4a59bb20.CuZiqCKUmLZgrA4L%Joerg.Schilling@fokus.fraunhofer.de> <4A5AA07B.1020701@mcintyreweb.com> <4A5B570B.4030805@sun.com> <4A5CA3FD.7020102@Egenera.COM> User-Agent: Thunderbird 2.0.0.16 (X11/20080811) Content-Length: 599 Status: RO X-Status: $$$$ X-UID: 0000000060 Kyle McDonald wrote: > I'll admit that I'm not an exprt on the newest features of NFSv4, and > CIFS, so it's not entirely clear to me how, or when (or IF?) I'd ever > want to use these reparse points, but unless I'm missing something, I > know I'd want them to work locally the same as they will on the client. A reference [for a future case!] about FedFS is here: http://www.connectathon.org/talks09/FedFS_Cthon09.pdf Most importantly, your automounter usage won't be affected. I believe you'll be able to do the equivalent thing with referrals, and we hope most will find it easier. Rob T From Afshin.Ardakani@sun.com Tue Jul 14 10:19:21 2009 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6EHJLV3000218 for ; Tue, 14 Jul 2009 10:19:21 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n6EHJEop001159; Tue, 14 Jul 2009 10:19:21 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMS00M07848EM00@nwk-avmta-2.sfbay.sun.com>; Tue, 14 Jul 2009 10:19:20 -0700 (PDT) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMS00DV2848WYC0@nwk-avmta-2.sfbay.sun.com>; Tue, 14 Jul 2009 10:19:20 -0700 (PDT) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n6EHJFNp014726; Tue, 14 Jul 2009 10:19:15 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMS00G006AIGK00@fe-sfbay-10.sun.com>; Tue, 14 Jul 2009 10:19:15 -0700 (PDT) Received: from [10.1.106.212] ([unknown] [10.1.106.212]) by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMS00LEP83THLA0@fe-sfbay-10.sun.com>; Tue, 14 Jul 2009 10:19:06 -0700 (PDT) Date: Tue, 14 Jul 2009 10:18:54 -0700 From: Afshin Salek Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A5C56F4.5080607@Sun.COM> Sender: Afshin.Ardakani@sun.com To: Darren J Moffat Cc: Nicolas Williams , Glenn Skinner , PSARC-ext@sun.com, cifs-eng@sun.com, Robert.Thurlow@sun.com, Brian.Wong@sun.com Message-id: <4A5CBDFE.7050903@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <20090713222450.GL1274@Sun.COM> <4A5BB8DA.804@sun.com> <4A5C3FC3.2020508@Sun.COM> <4A5C5002.5030107@sun.com> <4A5C56F4.5080607@Sun.COM> User-Agent: Thunderbird 2.0.0.9 (X11/20080213) Content-Length: 3826 Status: RO X-Status: $$$$ X-UID: 0000000061 Darren J Moffat wrote: > Afshin Salek wrote: >> >> >> Darren J Moffat wrote: >>> Afshin Salek wrote: >>>> This is supposed to be a generic and expandable mechanism. NFS/DFS >>>> referrals are just primary consumers of this mechanism at this >>>> point so this is not a protocol specific mechanism to have a per >>>> protocol extended attribute that are hard coded before hand. >>>> >>>> If we want to use extended attributes then we either have to reserve >>>> some namespace or use an extended attribute with a reserved name as an >>>> index. >>> >>> But you are already heading down that path by having a new system >>> attribute. You are also reserving namespace in symlink formats with >>> the case as specified so you don't get out of that problem. >>> >> >> We are not really reserving any namespace because the format >> that is proposed here cannot be a valid target for a regular symlink. > > Yes you are, by adopting the BSD magic link syntax that is reserving > namespace. It is reserving symlink content starting with @ and > specifically the part of the @ namespace that begins @{REPARSE. > > How is that not reserving namespace ? > > Today this works: > > braveheart:pts/1# ls -l > total 4 > -rw-r--r-- 1 root root 12 Jul 14 11:10 @{REPARSE} > lrwxrwxrwx 1 root root 10 Jul 14 11:07 foo -> @{REPARSE} > braveheart:pts/1# cat foo > hello world > braveheart:pts/1# cat @\{REPARSE\} > hello world > > After your change what happens ? > > Are those slightly crazy and contrived filenames ? Yes but they are > valid today. > >>> Given you need a new system attribute and backup software needs to be >>> able to back that up all your arguments based on backup software for >>> using symlinks don't hold up for me. >>> >> >> Backup softwares don't need to backup the system attribute because at >> the time of restore that bit will be set anyways because fop_symlink >> detects the @{REPARSE tag and will set the reparse attribute as >> mentioned in the case. > > Okay that is helpful. So there is now code in the normal fop_symlink > path to look at the content for the @{REPARSE namespace and set a system > attribute. What happens if I already had symlinks that looked like that ? > >>> > It also makes path name reduction very expensive because for each >>>> component we have to do a lot of extra stuff to see whether we are >>>> dealing with a reparse point or not and whether that reparse point >>>> contains the data that we are interested in. >>> >>> Aren't you already doing that by looking at the new attribute and >>> then only parsing the symlink data if it is tagged as a reparse >>> point. I >> >> The reparse detection part would be the same but I'm not sure doing >> a VOP_READLINK would be as expensive as working with extended >> attributes. > > So you haven't actually tried it yet you are saying it is too expensive. > How expensive is too expensive ? What is the performance criteria and > how do you know that VOP_READLINK meets it yet using an extended > attribute wouldn't ? > I didn't mean to imply that we rejected the extended attribute idea just because of perceived performance issues. Actually, there are a number of issues regarding extended attributes including: - currently, symlinks can't have EAs - need to make sure a directory is empty before marking it as reparse point - once a directory is marked as reparse point, FS and/or VFS needs to make sure nothing can be created in the directory - defining semantics of different operations performed by applications against a normal file/dir vs a file/dir that has been marked as RP overall considering different aspects of other methods that we've considered including using EAs, it seemed that using symlinks is the less complicated approach. Afshin From Afshin.Ardakani@sun.com Tue Jul 14 10:25:29 2009 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6EHPTMD000550 for ; Tue, 14 Jul 2009 10:25:29 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n6EHPMZ8005205; Tue, 14 Jul 2009 10:25:29 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMS00M038EFQS00@nwk-avmta-2.sfbay.sun.com>; Tue, 14 Jul 2009 10:25:27 -0700 (PDT) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMS00DL48EFWPD0@nwk-avmta-2.sfbay.sun.com>; Tue, 14 Jul 2009 10:25:27 -0700 (PDT) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n6EHPRWF015619; Tue, 14 Jul 2009 10:25:27 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMS00B00854EQ00@fe-sfbay-10.sun.com>; Tue, 14 Jul 2009 10:25:27 -0700 (PDT) Received: from [10.1.106.212] ([unknown] [10.1.106.212]) by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMS00LBP8EEHLD0@fe-sfbay-10.sun.com>; Tue, 14 Jul 2009 10:25:26 -0700 (PDT) Date: Tue, 14 Jul 2009 10:25:14 -0700 From: Afshin Salek Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A5CA3FD.7020102@Egenera.COM> Sender: Afshin.Ardakani@sun.com To: Kyle McDonald Cc: Robert Thurlow , Hugh McIntyre , PSARC-ext@sun.com, Brian.Wong@sun.com, cifs-eng@sun.com Message-id: <4A5CBF7A.3090002@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <4a58a697.yQ8anaL/V6TEAoRX%Joerg.Schilling@fokus.fraunhofer.de> <4A5943C0.3090303@Sun.COM> <4a59bb20.CuZiqCKUmLZgrA4L%Joerg.Schilling@fokus.fraunhofer.de> <4A5AA07B.1020701@mcintyreweb.com> <4A5B570B.4030805@sun.com> <4A5CA3FD.7020102@Egenera.COM> User-Agent: Thunderbird 2.0.0.9 (X11/20080213) Content-Length: 2671 Status: RO X-Status: $$$$ X-UID: 0000000062 Note that nothing in the case precludes making reparse points to also work locally. We are just proposing a location redirection mechanism here and it happens that there are immediate consumers for this mechanism and those are DFS and NFS referrals so that's what our primary focus is at the moment. Afshin Kyle McDonald wrote: > Robert Thurlow wrote: >> Hugh McIntyre wrote: >> >>> But this raises the point that if these special reparse symlinks were >>> implemented such that the reparsing also happened when the filesystem >>> is mounted locally via mount_zfs or mount_lofs, not just remotely >>> over CIFS or NFSv4, then most of the issues would go away. >>> >>> In other words if open(), stat(), nftw(), and similar system calls >>> locally on the server hosting the files (including LOFS) access the >>> linked-to file, most code will think the link is OK. Since in this >>> case, "find -L" should point to the target of the reparse point >>> (which hopefully exists). >> >> The project team is pretty interested in semantics seen by >> NFS and CIFS clients, not local semantics, so we're not >> terribly excited to do work to make local access act like >> that; I get the motivations, but it's a whole lot of work. > But that seems to ignore a whole class of servers, who are local NFS > clients of both other servers *and* themselves. > > All of my data disks are always mounted in at /export/SOMETHING on my > file servers. > > All of the clients of those servers mount those filesystems under > /import, usually /import/SOMETHING, but sometimes /import/SOME/THING/ELSE. > > Many times the path used to access these filesystems on the clients > needs to be a valid path on the server also. So I always make sure that > the /import/.... path works on the server too - which for NFS and NIS > and the Automounter doesn't really take any effort at all. > > I'll admit that I'm not an exprt on the newest features of NFSv4, and > CIFS, so it's not entirely clear to me how, or when (or IF?) I'd ever > want to use these reparse points, but unless I'm missing something, I > know I'd want them to work locally the same as they will on the client. > > -Kyle > >> >> There are also difficulties. We know we will want to set >> up a reparse point that projects the same namespace for >> both NFS and CIFS, and it isn't obvious what client code >> we would bridge into locally. We would also like to be >> able to back up reparse points, so local processes need to >> access them without interpretation. >> >> Rob T >> >> _______________________________________________ >> opensolaris-arc mailing list >> opensolaris-arc@opensolaris.org > From Mike.Oliver@sun.com Tue Jul 14 12:53:08 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6EJr736009869 for ; Tue, 14 Jul 2009 12:53:08 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n6EJr1Ra003505; Wed, 15 Jul 2009 03:53:07 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMS0080TF8GGQ00@nwk-avmta-2.sfbay.sun.com>; Tue, 14 Jul 2009 12:53:04 -0700 (PDT) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMS000B9F8G3480@nwk-avmta-2.sfbay.sun.com>; Tue, 14 Jul 2009 12:53:04 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n6EJr420004858; Tue, 14 Jul 2009 12:53:04 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMS00900F5DGD00@fe-sfbay-09.sun.com>; Tue, 14 Jul 2009 12:53:04 -0700 (PDT) Received: from sunray3.SFBay.Sun.COM ([unknown] [10.6.102.113]) by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMS00JTSF8FK990@fe-sfbay-09.sun.com>; Tue, 14 Jul 2009 12:53:04 -0700 (PDT) Date: Tue, 14 Jul 2009 12:53:03 -0700 From: Mike Oliver Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A5C6704.8050801@Sun.COM> Sender: Mike.Oliver@sun.com To: "Alan.M.Wright" , Afshin Salek Cc: Darren J Moffat , PSARC-ext@sun.com, cifs-eng@sun.com, Robert.Thurlow@sun.com, Brian.Wong@sun.com Message-id: <4A5CE21F.4000204@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <20090713222450.GL1274@Sun.COM> <4A5BB8DA.804@sun.com> <4A5C3FC3.2020508@Sun.COM> <23408EBC7ADF4918A35E43F6D6BF1E95@TOSHIBA> <4A5C5726.4010000@Sun.COM> <7B74FF619D8947F2A1062312E072B22C@TOSHIBA> <4A5C5EEA.50202@Sun.COM> <3464DD17027C4B35AF3A2AE3B468B7E8@TOSHIBA> <4A5C6704.8050801@Sun.COM> User-Agent: Thunderbird 2.0.0.9 (X11/20080119) Content-Length: 2316 Status: RO X-Status: $$$$ X-UID: 0000000063 Darren J Moffat wrote: > Alan.M.Wright wrote: >> We are reserving '@{...}', i.e. the name must start with @{ and end >> with }. >> >> We considered both a global option and per file system options but, on >> the basis that no-one could identify a scenario in which something would >> misinterpret these objects and behave badly, we decided not to propose >> an enable/disable switch. > > So that fact that you can today create a symlink pointing to such a file > isn't a case of behaving badly ? > > estale:pts/83$ echo "hello world" > "@{REPARSE:NFS:00000}" > estale:pts/83$ ln -s @\{REPARSE:NFS:00000\} @\{REPARSE:NFS:00001\} > estale:pts/83$ ls -l > total 2 > -rw-r--r-- 1 darrenm staff 12 Jul 14 12:03 @{REPARSE:NFS:00000} > lrwxrwxrwx 1 darrenm staff 20 Jul 14 12:04 > @{REPARSE:NFS:00001} -> @{REPARSE:NFS:00000} (I don't think that's quite the right syntax, but that's not important.) Is it true that any old user can create symlinks whose content will be interpreted as a reparse point? If so, what protections are in place to prevent arbitrary content in such a user-created symlink from tricking the system into doing something bad? Presumably the protection would be implemented in 'reparsed' or its plug-ins, which aren't described in this case, but perhaps you can comment on the general approach that will be used to defend against abuse of the service data in the symlink content. Mike. -- mike.oliver@sun.com > estale:pts/83$ cat @\{REPARSE:NFS:00000\} > hello world > estale:pts/83$ cat @\{REPARSE:NFS:00001\} > hello world > > Just because you don't create files beginning @{ and ending } doesn't > mean they don't exist. Given all the rest of the data that has to be in > there I think the changes are very very slim. The issue isn't so much > with the REPARSE but the reserving of all @{...} which this case claims > to do. Again I think the chances of a problem are slim but there must > be a reason why BSD made this optional, no ? > > Note I'm no longer arguing against the case I'm just recording the fact > that there is a risk in reserving this symlink content namespace though > it is a very small one. > > Given everything I've said and the discussion that has been had I'm now > okay supporting this case and give it my @{PSARC:+1}. > From robert.thurlow@sun.com Tue Jul 14 13:01:04 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6EK14X1010408 for ; Tue, 14 Jul 2009 13:01:04 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n6EK0tT1006810; Wed, 15 Jul 2009 04:00:57 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMS00D0XFLKZC00@brm-avmta-1.central.sun.com>; Tue, 14 Jul 2009 14:00:56 -0600 (MDT) Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.59]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMS007J6FLKAC60@brm-avmta-1.central.sun.com>; Tue, 14 Jul 2009 14:00:56 -0600 (MDT) Received: from [10.7.250.15] (punchin-client-10-7-250-15.SFBay.Sun.COM [10.7.250.15]) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6EK0r95451900; Tue, 14 Jul 2009 13:00:54 -0700 (PDT) Date: Tue, 14 Jul 2009 14:00:53 -0600 From: Robert Thurlow Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A5CE21F.4000204@sun.com> To: Mike Oliver Cc: "Alan.M.Wright" , Afshin Salek , Darren J Moffat , PSARC-ext@sun.com, cifs-eng@sun.com, Brian.Wong@sun.com Message-id: <4A5CE3F5.7000405@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <20090713222450.GL1274@Sun.COM> <4A5BB8DA.804@sun.com> <4A5C3FC3.2020508@Sun.COM> <23408EBC7ADF4918A35E43F6D6BF1E95@TOSHIBA> <4A5C5726.4010000@Sun.COM> <7B74FF619D8947F2A1062312E072B22C@TOSHIBA> <4A5C5EEA.50202@Sun.COM> <3464DD17027C4B35AF3A2AE3B468B7E8@TOSHIBA> <4A5C6704.8050801@Sun.COM> <4A5CE21F.4000204@sun.com> User-Agent: Thunderbird 2.0.0.16 (X11/20080811) Content-Length: 714 Status: RO X-Status: $$$$ X-UID: 0000000064 Mike Oliver wrote: > Is it true that any old user can create symlinks whose content will be > interpreted as a reparse point? If so, what protections are in place to > prevent arbitrary content in such a user-created symlink from tricking > the system into doing something bad? Presumably the protection would be > implemented in 'reparsed' or its plug-ins, which aren't described in > this case, but perhaps you can comment on the general approach that will > be used to defend against abuse of the service data in the symlink > content. If you can write to the filesystem, you can create a symlink to point to your own secret stash of trojan-enables binaries via /net. In what way is this different? Rob T From Garrett.Damore@sun.com Tue Jul 14 13:34:08 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6EKY7TY011069 for ; Tue, 14 Jul 2009 13:34:08 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n6EKXuLc023038; Wed, 15 Jul 2009 04:34:07 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMS00E2DH4SXV00@nwk-avmta-1.sfbay.Sun.COM>; Tue, 14 Jul 2009 13:34:04 -0700 (PDT) Received: from sca-es-mail-2.sun.com ([192.18.43.133]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMS00H1HH4R8ZA0@nwk-avmta-1.sfbay.Sun.COM>; Tue, 14 Jul 2009 13:34:03 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n6EKY3ka007219; Tue, 14 Jul 2009 13:34:03 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMS00D00GTE1M00@fe-sfbay-09.sun.com>; Tue, 14 Jul 2009 13:34:03 -0700 (PDT) Received: from [203.36.146.23] ([unknown] [203.36.146.23]) by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMS00H4JH4OMAA0@fe-sfbay-09.sun.com>; Tue, 14 Jul 2009 13:34:03 -0700 (PDT) Date: Tue, 14 Jul 2009 13:33:59 -0700 From: "Garrett D'Amore" Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A5CE3F5.7000405@sun.com> Sender: Garrett.Damore@sun.com To: Robert Thurlow Cc: Mike Oliver , "Alan.M.Wright" , Afshin Salek , Darren J Moffat , PSARC-ext@sun.com, cifs-eng@sun.com, Brian.Wong@sun.com Message-id: <4A5CEBB7.8090803@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <20090713222450.GL1274@Sun.COM> <4A5BB8DA.804@sun.com> <4A5C3FC3.2020508@Sun.COM> <23408EBC7ADF4918A35E43F6D6BF1E95@TOSHIBA> <4A5C5726.4010000@Sun.COM> <7B74FF619D8947F2A1062312E072B22C@TOSHIBA> <4A5C5EEA.50202@Sun.COM> <3464DD17027C4B35AF3A2AE3B468B7E8@TOSHIBA> <4A5C6704.8050801@Sun.COM> <4A5CE21F.4000204@sun.com> <4A5CE3F5.7000405@sun.com> User-Agent: Thunderbird 2.0.0.21 (X11/20090505) Content-Length: 892 Status: RO X-Status: $$$$ X-UID: 0000000065 Robert Thurlow wrote: > Mike Oliver wrote: > >> Is it true that any old user can create symlinks whose content will be >> interpreted as a reparse point? If so, what protections are in place to >> prevent arbitrary content in such a user-created symlink from tricking >> the system into doing something bad? Presumably the protection would be >> implemented in 'reparsed' or its plug-ins, which aren't described in >> this case, but perhaps you can comment on the general approach that will >> be used to defend against abuse of the service data in the symlink >> content. > > If you can write to the filesystem, you can create a symlink > to point to your own secret stash of trojan-enables binaries > via /net. In what way is this different? I'd worry about subverting other aspects of the reparse system. For example, subverting bugs in the parser itself. -- Garrett > > Rob T From kmcdonald@egenera.com Tue Jul 14 13:39:16 2009 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6EKdGWT011198 for ; Tue, 14 Jul 2009 13:39:16 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n6EKdEfb004605; Tue, 14 Jul 2009 13:39:15 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMS00B0NHDFG500@nwk-avmta-2.sfbay.sun.com>; Tue, 14 Jul 2009 13:39:15 -0700 (PDT) Received: from sca-ea-mail-1.sun.com ([192.18.43.24]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMS000DPHDE2WA0@nwk-avmta-2.sfbay.sun.com>; Tue, 14 Jul 2009 13:39:14 -0700 (PDT) Received: from relay11i.sun.com (ip121.net129179-4.block1.us.syntegra.com [129.179.4.121]) by sca-ea-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n6EKU6df019724; Tue, 14 Jul 2009 20:39:14 +0000 (GMT) Received: from mmp13es.mmp.us.syntegra.com ([160.41.208.13] [160.41.208.13]) by relay11i.sun.com with ESMTP id BT-MMP-1360250; Tue, 14 Jul 2009 20:39:13 +0000 (Z) Received: from relay11i.sun.com (relay11i.sun.com [129.179.4.121]) by mmp13es.mmp.us.syntegra.com with ESMTP id BT-MMP-14164762; Tue, 14 Jul 2009 20:39:11 +0000 (Z) Received: from webaccess.egenera.com ([63.139.209.15] [63.139.209.15]) by relay1i.sun.com with ESMTP id BT-MMP-2211667; Tue, 14 Jul 2009 20:39:11 +0000 (Z) Received: from [172.23.2.193] ([172.23.2.193]) by webaccess.egenera.com with Microsoft SMTPSVC(6.0.3790.3959); Tue, 14 Jul 2009 16:39:11 -0400 Date: Tue, 14 Jul 2009 16:38:36 -0400 From: Kyle McDonald Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A5CE3F5.7000405@sun.com> To: Robert Thurlow Cc: Mike Oliver , Afshin Salek , PSARC-ext@sun.com, Brian.Wong@sun.com, cifs-eng@sun.com Message-id: <4A5CECCC.3020408@Egenera.COM> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-Antispam: No, score=0.0/5.0, scanned in 0.060sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <20090713222450.GL1274@Sun.COM> <4A5BB8DA.804@sun.com> <4A5C3FC3.2020508@Sun.COM> <23408EBC7ADF4918A35E43F6D6BF1E95@TOSHIBA> <4A5C5726.4010000@Sun.COM> <7B74FF619D8947F2A1062312E072B22C@TOSHIBA> <4A5C5EEA.50202@Sun.COM> <3464DD17027C4B35AF3A2AE3B468B7E8@TOSHIBA> <4A5C6704.8050801@Sun.COM> <4A5CE21F.4000204@sun.com> <4A5CE3F5.7000405@sun.com> User-Agent: Thunderbird 2.0.0.22 (Windows/20090605) X-OriginalArrivalTime: 14 Jul 2009 20:39:11.0013 (UTC) FILETIME=[2472FD50:01CA04C3] Content-Length: 968 Status: RO X-Status: $$$$ X-UID: 0000000066 Robert Thurlow wrote: > Mike Oliver wrote: > >> Is it true that any old user can create symlinks whose content will be >> interpreted as a reparse point? If so, what protections are in place to >> prevent arbitrary content in such a user-created symlink from tricking >> the system into doing something bad? Presumably the protection would be >> implemented in 'reparsed' or its plug-ins, which aren't described in >> this case, but perhaps you can comment on the general approach that will >> be used to defend against abuse of the service data in the symlink >> content. > > If you can write to the filesystem, you can create a symlink > to point to your own secret stash of trojan-enables binaries > via /net. In what way is this different? I know I always disable /net. Will this re-open an equivalent that I can't disable? -Kyle > > Rob T > _______________________________________________ > opensolaris-arc mailing list > opensolaris-arc@opensolaris.org From robert.thurlow@sun.com Tue Jul 14 13:55:45 2009 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6EKtjH8011721 for ; Tue, 14 Jul 2009 13:55:45 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6EKtXMV010665; Tue, 14 Jul 2009 21:55:40 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMS00C09I4QI400@nwk-avmta-2.sfbay.sun.com>; Tue, 14 Jul 2009 13:55:38 -0700 (PDT) Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.63]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMS000ELI4Q2YB0@nwk-avmta-2.sfbay.sun.com>; Tue, 14 Jul 2009 13:55:38 -0700 (PDT) Received: from [10.7.250.15] (punchin-client-10-7-250-15.SFBay.Sun.COM [10.7.250.15]) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6EKtT8I495536; Tue, 14 Jul 2009 13:55:32 -0700 (PDT) Date: Tue, 14 Jul 2009 14:55:29 -0600 From: Robert Thurlow Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A5CECCC.3020408@Egenera.COM> To: Kyle McDonald Cc: Mike Oliver , Afshin Salek , PSARC-ext@sun.com, Brian.Wong@sun.com, cifs-eng@sun.com Message-id: <4A5CF0C1.3030100@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <20090713222450.GL1274@Sun.COM> <4A5BB8DA.804@sun.com> <4A5C3FC3.2020508@Sun.COM> <23408EBC7ADF4918A35E43F6D6BF1E95@TOSHIBA> <4A5C5726.4010000@Sun.COM> <7B74FF619D8947F2A1062312E072B22C@TOSHIBA> <4A5C5EEA.50202@Sun.COM> <3464DD17027C4B35AF3A2AE3B468B7E8@TOSHIBA> <4A5C6704.8050801@Sun.COM> <4A5CE21F.4000204@sun.com> <4A5CE3F5.7000405@sun.com> <4A5CECCC.3020408@Egenera.COM> User-Agent: Thunderbird 2.0.0.16 (X11/20080811) Content-Length: 648 Status: RO X-Status: $$$$ X-UID: 0000000067 Kyle McDonald wrote: >> If you can write to the filesystem, you can create a symlink >> to point to your own secret stash of trojan-enables binaries >> via /net. In what way is this different? > I know I always disable /net. Will this re-open an equivalent that I > can't disable? No. The point [of the follow-on NFS case] is to move the automounter-based namespace processing to the server, by use of referrals. The symlinks will not be usable unless the key they carry can be looked up in some kind of infrastructure, e.g. an LDAP server. If you use the automounter, you rely on other systems data correctness in a comparable way. Rob T From Joerg.Schilling9ab33xy531fokus.fraunhofer.de@bounce.antispameurope.com Tue Jul 14 14:31:32 2009 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6ELVVHI014017 for ; Tue, 14 Jul 2009 14:31:32 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6ELVRmg001742 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Tue, 14 Jul 2009 22:31:31 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMS00303JSH8000@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 14 Jul 2009 14:31:29 -0700 (PDT) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMS00HI4JSH8ZE0@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 14 Jul 2009 14:31:29 -0700 (PDT) Received: from relay43i.sun.com ([192.5.209.74]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6ELQ0kx025263 for ; Tue, 14 Jul 2009 21:31:28 +0000 (GMT) Received: from mms48es.mms.us.syntegra.com ([160.41.221.230] [160.41.221.230]) by relay43i.sun.com with ESMTP id BT-MMP-4245 for PSARC-ext@sun.com; Tue, 14 Jul 2009 21:31:28 +0000 (Z) Received: from relay44i.sun.com (relay44i.sun.com [192.5.209.118]) by mms48es.mms.us.syntegra.com with ESMTP id BT-MMP-86702953 for PSARC-ext@sun.com; Tue, 14 Jul 2009 21:31:27 +0000 (Z) Received: from relay02-haj2.antispameurope.com ([83.246.65.52] [83.246.65.52]) by relay4i.sun.com with ESMTP id BT-MMP-3727121 for PSARC-ext@sun.com; Tue, 14 Jul 2009 21:31:27 +0000 (Z) Received: by relay02-haj2.antispameurope.com (ASE-Secure-MTA, from userid 1000) id 37F376F0507; Tue, 14 Jul 2009 23:31:26 +0200 (CEST) Received: from pluto.fokus.fraunhofer.de (pluto.fokus.fraunhofer.de [195.37.77.164]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by relay02-haj2.antispameurope.com (ASE-Secure-MTA) with ESMTP id 9EFAB6F0507; Tue, 14 Jul 2009 23:31:25 +0200 (CEST) Received: from EXCHSRV.fokus.fraunhofer.de (bohr.fokus.fraunhofer.de [10.147.9.231]) by pluto.fokus.fraunhofer.de (8.14.2/8.14.2) with SMTP id n6ELVP4u016331; Tue, 14 Jul 2009 23:31:25 +0200 (MEST) Received: from rigel ([10.147.65.195]) by EXCHSRV.fokus.fraunhofer.de with Microsoft SMTPSVC(6.0.3790.3959); Tue, 14 Jul 2009 23:31:25 +0200 Date: Tue, 14 Jul 2009 23:31:25 +0200 From: Joerg.Schilling@fokus.fraunhofer.de (Joerg Schilling) Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A5CF0C1.3030100@sun.com> Sender: Joerg.Schilling9ab33xy531fokus.fraunhofer.de@bounce.antispameurope.com To: robert.thurlow@sun.com, KMcDonald@egenera.com Cc: PSARC-ext@sun.com, Mike.Oliver@sun.com, cifs-eng@sun.com, Brian.Wong@sun.com, Afshin.Ardakani@sun.com Message-id: <4a5cf92d.F92r0oy2EnpQ1NS9%Joerg.Schilling@fokus.fraunhofer.de> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1 Content-transfer-encoding: 8BIT X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-Antispam: No, score=0.0/5.0, scanned in 0.508sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <20090713222450.GL1274@Sun.COM> <4A5BB8DA.804@sun.com> <4A5C3FC3.2020508@Sun.COM> <23408EBC7ADF4918A35E43F6D6BF1E95@TOSHIBA> <4A5C5726.4010000@Sun.COM> <7B74FF619D8947F2A1062312E072B22C@TOSHIBA> <4A5C5EEA.50202@Sun.COM> <3464DD17027C4B35AF3A2AE3B468B7E8@TOSHIBA> <4A5C6704.8050801@Sun.COM> <4A5CE21F.4000204@sun.com> <4A5CE3F5.7000405@sun.com> <4A5CECCC.3020408@Egenera.COM> <4A5CF0C1.3030100@sun.com> User-Agent: nail 11.22 3/20/05 X-OriginalArrivalTime: 14 Jul 2009 21:31:25.0938 (UTC) FILETIME=[71029520:01CA04CA] Content-Length: 855 Status: RO X-Status: $$$$ X-UID: 0000000068 Robert Thurlow wrote: > No. The point [of the follow-on NFS case] is to move the > automounter-based namespace processing to the server, by > use of referrals. The symlinks will not be usable unless > the key they carry can be looked up in some kind of > infrastructure, e.g. an LDAP server. If you use the > automounter, you rely on other systems data correctness > in a comparable way. Do you see a benefit from replacing technology that is known since more than 20 years by something that does not yet have even test users? Jörg -- EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin js@cs.tu-berlin.de (uni) joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily From amw@sun.com Tue Jul 14 14:35:17 2009 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6ELZGTs015582 for ; Tue, 14 Jul 2009 14:35:17 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6ELZDrH003622; Tue, 14 Jul 2009 22:35:15 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMS00305JYRZX00@nwk-avmta-1.sfbay.Sun.COM>; Tue, 14 Jul 2009 14:35:15 -0700 (PDT) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMS00HA3JYR97E0@nwk-avmta-1.sfbay.Sun.COM>; Tue, 14 Jul 2009 14:35:15 -0700 (PDT) Received: from fe-amer-09.sun.com ([192.18.109.79]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6ELZEtg028616; Tue, 14 Jul 2009 21:35:14 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMS00100JQ49C00@mail-amer.sun.com>; Tue, 14 Jul 2009 15:35:14 -0600 (MDT) Received: from [10.1.106.211] ([unknown] [10.1.106.211]) by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMS00I1JJYKVHC0@mail-amer.sun.com>; Tue, 14 Jul 2009 15:35:11 -0600 (MDT) Date: Tue, 14 Jul 2009 14:35:08 -0700 From: Alan M Wright Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A5C6704.8050801@Sun.COM> Sender: Alan.M.Wright@sun.com To: Darren J Moffat Cc: Afshin Salek , Nicolas Williams , Glenn Skinner , PSARC-ext@sun.com, cifs-eng@sun.com, Robert.Thurlow@sun.com, Brian.Wong@sun.com Message-id: <4A5CFA0C.80108@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <20090713222450.GL1274@Sun.COM> <4A5BB8DA.804@sun.com> <4A5C3FC3.2020508@Sun.COM> <23408EBC7ADF4918A35E43F6D6BF1E95@TOSHIBA> <4A5C5726.4010000@Sun.COM> <7B74FF619D8947F2A1062312E072B22C@TOSHIBA> <4A5C5EEA.50202@Sun.COM> <3464DD17027C4B35AF3A2AE3B468B7E8@TOSHIBA> <4A5C6704.8050801@Sun.COM> User-Agent: Thunderbird 2.0.0.21 (X11/20090323) Content-Length: 2222 Status: RO X-Status: $$$$ X-UID: 0000000069 On 07/14/09 04:07, Darren J Moffat wrote: > Alan.M.Wright wrote: >> We are reserving '@{...}', i.e. the name must start with @{ and end >> with }. >> >> We considered both a global option and per file system options but, on >> the basis that no-one could identify a scenario in which something would >> misinterpret these objects and behave badly, we decided not to propose >> an enable/disable switch. > > So that fact that you can today create a symlink pointing to such a file > isn't a case of behaving badly ? > > estale:pts/83$ echo "hello world" > "@{REPARSE:NFS:00000}" > estale:pts/83$ ln -s @\{REPARSE:NFS:00000\} @\{REPARSE:NFS:00001\} > estale:pts/83$ ls -l > total 2 > -rw-r--r-- 1 darrenm staff 12 Jul 14 12:03 @{REPARSE:NFS:00000} > lrwxrwxrwx 1 darrenm staff 20 Jul 14 12:04 > @{REPARSE:NFS:00001} -> @{REPARSE:NFS:00000} > estale:pts/83$ cat @\{REPARSE:NFS:00000\} > hello world > estale:pts/83$ cat @\{REPARSE:NFS:00001\} > hello world > > Just because you don't create files beginning @{ and ending } doesn't > mean they don't exist. Given all the rest of the data that has to be in > there I think the changes are very very slim. The issue isn't so much > with the REPARSE but the reserving of all @{...} which this case claims > to do. Again I think the chances of a problem are slim but there must > be a reason why BSD made this optional, no ? I suspect that magic symlink support is optional in BSD because the content of a magic symlink will be changed on the fly by the OS. For example, if @{HOST} appears in the symlink that tag will be replaced at runtime by the nodename. Nothing in this proposal will modify the content of a symlink on the fly or inhibit a regular symlink from working as it does now, including a usable symlink named exactly like a reparse point or the object being (simultaneously) a symlink and a reparse point. Alan > Note I'm no longer arguing against the case I'm just recording the fact > that there is a risk in reserving this symlink content namespace though > it is a very small one. > > Given everything I've said and the discussion that has been had I'm now > okay supporting this case and give it my @{PSARC:+1}. > From robert.thurlow@sun.com Tue Jul 14 14:44:13 2009 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6ELiD1X016299 for ; Tue, 14 Jul 2009 14:44:13 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n6ELiCDn001012; Tue, 14 Jul 2009 14:44:12 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMS00101KDNR100@brm-avmta-1.central.sun.com>; Tue, 14 Jul 2009 15:44:11 -0600 (MDT) Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.63]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMS00MLYKDM11B0@brm-avmta-1.central.sun.com>; Tue, 14 Jul 2009 15:44:10 -0600 (MDT) Received: from [10.7.250.15] (punchin-client-10-7-250-15.SFBay.Sun.COM [10.7.250.15]) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6ELi4bK520118; Tue, 14 Jul 2009 14:44:05 -0700 (PDT) Date: Tue, 14 Jul 2009 15:44:00 -0600 From: Robert Thurlow Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4a5cf92d.F92r0oy2EnpQ1NS9%Joerg.Schilling@fokus.fraunhofer.de> To: Joerg Schilling Cc: KMcDonald@egenera.com, PSARC-ext@sun.com, Mike.Oliver@sun.com, cifs-eng@sun.com, Brian.Wong@sun.com, Afshin.Ardakani@sun.com Message-id: <4A5CFC20.7090704@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <20090713222450.GL1274@Sun.COM> <4A5BB8DA.804@sun.com> <4A5C3FC3.2020508@Sun.COM> <23408EBC7ADF4918A35E43F6D6BF1E95@TOSHIBA> <4A5C5726.4010000@Sun.COM> <7B74FF619D8947F2A1062312E072B22C@TOSHIBA> <4A5C5EEA.50202@Sun.COM> <3464DD17027C4B35AF3A2AE3B468B7E8@TOSHIBA> <4A5C6704.8050801@Sun.COM> <4A5CE21F.4000204@sun.com> <4A5CE3F5.7000405@sun.com> <4A5CECCC.3020408@Egenera.COM> <4A5CF0C1.3030100@sun.com> <4a5cf92d.F92r0oy2EnpQ1NS9%Joerg.Schilling@fokus.fraunhofer.de> User-Agent: Thunderbird 2.0.0.16 (X11/20080811) Content-Length: 618 Status: RO X-Status: $$$$ X-UID: 0000000070 Joerg Schilling wrote: > Robert Thurlow wrote: > >> No. The point [of the follow-on NFS case] is to move the >> automounter-based namespace processing to the server, by >> use of referrals. The symlinks will not be usable unless >> the key they carry can be looked up in some kind of >> infrastructure, e.g. an LDAP server. If you use the >> automounter, you rely on other systems data correctness >> in a comparable way. > > Do you see a benefit from replacing technology that is known since more than 20 > years by something that does not yet have even test users? Absolutely. Rob T From Joerg.Schilling9ab33xy531fokus.fraunhofer.de@bounce.antispameurope.com Tue Jul 14 14:51:22 2009 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6ELpMV0016596 for ; Tue, 14 Jul 2009 14:51:22 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n6ELpIsW060109 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Tue, 14 Jul 2009 15:51:21 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMS00G1DKPJ9600@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 14 Jul 2009 14:51:19 -0700 (PDT) Received: from sca-ea-mail-1.sun.com ([192.18.43.24]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMS000W5KPJ2TE0@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 14 Jul 2009 14:51:19 -0700 (PDT) Received: from relay42i.sun.com ([192.5.209.72]) by sca-ea-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n6ELpBgg007606 for ; Tue, 14 Jul 2009 21:51:19 +0000 (GMT) Received: from mmp42es.mmp.us.syntegra.com ([160.41.221.11] [160.41.221.11]) by relay42i.sun.com with ESMTP id BT-MMP-51728 for PSARC-ext@sun.com; Tue, 14 Jul 2009 21:51:10 +0000 (Z) Received: from relay44i.sun.com (relay44i.sun.com [192.5.209.118]) by mmp42es.mmp.us.syntegra.com with ESMTP id BT-MMP-86929094 for PSARC-ext@sun.com; Tue, 14 Jul 2009 21:51:10 +0000 (Z) Received: from relay02-haj2.antispameurope.com ([83.246.65.52] [83.246.65.52]) by relay4i.sun.com with ESMTP id BT-MMP-3757769 for PSARC-ext@sun.com; Tue, 14 Jul 2009 21:51:10 +0000 (Z) Received: by relay02-haj2.antispameurope.com (ASE-Secure-MTA, from userid 1000) id 7006A6F0508; Tue, 14 Jul 2009 23:51:09 +0200 (CEST) Received: from pluto.fokus.fraunhofer.de (pluto.fokus.fraunhofer.de [195.37.77.164]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by relay02-haj2.antispameurope.com (ASE-Secure-MTA) with ESMTP id C9BC46F0507; Tue, 14 Jul 2009 23:51:08 +0200 (CEST) Received: from EXCHSRV.fokus.fraunhofer.de (bohr.fokus.fraunhofer.de [10.147.9.231]) by pluto.fokus.fraunhofer.de (8.14.2/8.14.2) with SMTP id n6ELp8hU016540; Tue, 14 Jul 2009 23:51:08 +0200 (MEST) Received: from rigel ([10.147.65.195]) by EXCHSRV.fokus.fraunhofer.de with Microsoft SMTPSVC(6.0.3790.3959); Tue, 14 Jul 2009 23:51:08 +0200 Date: Tue, 14 Jul 2009 23:51:08 +0200 From: Joerg.Schilling@fokus.fraunhofer.de (Joerg Schilling) Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A5CFA0C.80108@sun.com> Sender: Joerg.Schilling9ab33xy531fokus.fraunhofer.de@bounce.antispameurope.com To: Darren.Moffat@sun.com, amw@sun.com Cc: Robert.Thurlow@sun.com, PSARC-ext@sun.com, cifs-eng@sun.com, Brian.Wong@sun.com, Afshin.Ardakani@sun.com Message-id: <4a5cfdcc.gATeSQiZ6ujpJJMs%Joerg.Schilling@fokus.fraunhofer.de> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1 Content-transfer-encoding: 8BIT X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-Antispam: No, score=0.0/5.0, scanned in 0.402sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <20090713222450.GL1274@Sun.COM> <4A5BB8DA.804@sun.com> <4A5C3FC3.2020508@Sun.COM> <23408EBC7ADF4918A35E43F6D6BF1E95@TOSHIBA> <4A5C5726.4010000@Sun.COM> <7B74FF619D8947F2A1062312E072B22C@TOSHIBA> <4A5C5EEA.50202@Sun.COM> <3464DD17027C4B35AF3A2AE3B468B7E8@TOSHIBA> <4A5C6704.8050801@Sun.COM> <4A5CFA0C.80108@sun.com> User-Agent: nail 11.22 3/20/05 X-OriginalArrivalTime: 14 Jul 2009 21:51:08.0499 (UTC) FILETIME=[31DF1230:01CA04CD] Content-Length: 630 Status: RO X-Status: $$$$ X-UID: 0000000071 Alan M Wright wrote: > I suspect that magic symlink support is optional in BSD because the > content of a magic symlink will be changed on the fly by the OS. > For example, if @{HOST} appears in the symlink that tag will be > replaced at runtime by the nodename. If this is true, how ar these symlinks copied on BSD? Jörg -- EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin js@cs.tu-berlin.de (uni) joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily From Joerg.Schilling9ab33xy531fokus.fraunhofer.de@bounce.antispameurope.com Tue Jul 14 14:51:40 2009 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6ELpd9P016609 for ; Tue, 14 Jul 2009 14:51:40 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n6ELpbKA060238 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Tue, 14 Jul 2009 15:51:39 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMS00703KQ3BT00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 14 Jul 2009 14:51:39 -0700 (PDT) Received: from sca-ea-mail-2.sun.com ([192.18.43.25]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMS00416KQ2FI60@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 14 Jul 2009 14:51:38 -0700 (PDT) Received: from relay44i.sun.com ([192.5.209.118]) by sca-ea-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n6ELpcRs017747 for ; Tue, 14 Jul 2009 21:51:38 +0000 (GMT) Received: from mms49es.mms.us.syntegra.com ([160.41.221.232] [160.41.221.232]) by relay44i.sun.com with ESMTP id BT-MMP-179508 for PSARC-ext@sun.com; Tue, 14 Jul 2009 21:51:38 +0000 (Z) Received: from relay43i.sun.com (relay43i.sun.com [192.5.209.74]) by mms49es.mms.us.syntegra.com with ESMTP id BT-MMP-86421353 for PSARC-ext@sun.com; Tue, 14 Jul 2009 21:51:37 +0000 (Z) Received: from relay04-haj2.antispameurope.com ([83.246.65.54] [83.246.65.54]) by relay4i.sun.com with ESMTP id BT-MMP-133227 for PSARC-ext@sun.com; Tue, 14 Jul 2009 21:51:37 +0000 (Z) Received: by relay04-haj2.antispameurope.com (ASE-Secure-MTA, from userid 1000) id 3B2C75EC116; Tue, 14 Jul 2009 23:51:36 +0200 (CEST) Received: from pluto.fokus.fraunhofer.de (pluto.fokus.fraunhofer.de [195.37.77.164]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by relay04-haj2.antispameurope.com (ASE-Secure-MTA) with ESMTP id 4E35C5EC10D; Tue, 14 Jul 2009 23:51:35 +0200 (CEST) Received: from EXCHSRV.fokus.fraunhofer.de (bohr.fokus.fraunhofer.de [10.147.9.231]) by pluto.fokus.fraunhofer.de (8.14.2/8.14.2) with SMTP id n6ELpZ9w016546; Tue, 14 Jul 2009 23:51:35 +0200 (MEST) Received: from rigel ([10.147.65.195]) by EXCHSRV.fokus.fraunhofer.de with Microsoft SMTPSVC(6.0.3790.3959); Tue, 14 Jul 2009 23:51:35 +0200 Date: Tue, 14 Jul 2009 23:51:35 +0200 From: Joerg.Schilling@fokus.fraunhofer.de (Joerg Schilling) Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A5CFC20.7090704@sun.com> Sender: Joerg.Schilling9ab33xy531fokus.fraunhofer.de@bounce.antispameurope.com To: Robert.Thurlow@sun.com Cc: PSARC-ext@sun.com, Mike.Oliver@sun.com, KMcDonald@egenera.com, cifs-eng@sun.com, Brian.Wong@sun.com, Afshin.Ardakani@sun.com Message-id: <4a5cfde7.C9VgJWErnG5yGh9q%Joerg.Schilling@fokus.fraunhofer.de> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1 Content-transfer-encoding: 8BIT X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-Antispam: No, score=0.0/5.0, scanned in 0.073sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <20090713222450.GL1274@Sun.COM> <4A5BB8DA.804@sun.com> <4A5C3FC3.2020508@Sun.COM> <23408EBC7ADF4918A35E43F6D6BF1E95@TOSHIBA> <4A5C5726.4010000@Sun.COM> <7B74FF619D8947F2A1062312E072B22C@TOSHIBA> <4A5C5EEA.50202@Sun.COM> <3464DD17027C4B35AF3A2AE3B468B7E8@TOSHIBA> <4A5C6704.8050801@Sun.COM> <4A5CE21F.4000204@sun.com> <4A5CE3F5.7000405@sun.com> <4A5CECCC.3020408@Egenera.COM> <4A5CF0C1.3030100@sun.com> <4a5cf92d.F92r0oy2EnpQ1NS9%Joerg.Schilling@fokus.fraunhofer.de> <4A5CFC20.7090704@sun.com> User-Agent: nail 11.22 3/20/05 X-OriginalArrivalTime: 14 Jul 2009 21:51:35.0453 (UTC) FILETIME=[41EFECD0:01CA04CD] Content-Length: 1002 Status: RO X-Status: $$$$ X-UID: 0000000072 Robert Thurlow wrote: > Joerg Schilling wrote: > > Robert Thurlow wrote: > > > >> No. The point [of the follow-on NFS case] is to move the > >> automounter-based namespace processing to the server, by > >> use of referrals. The symlinks will not be usable unless > >> the key they carry can be looked up in some kind of > >> infrastructure, e.g. an LDAP server. If you use the > >> automounter, you rely on other systems data correctness > >> in a comparable way. > > > > Do you see a benefit from replacing technology that is known since more than 20 > > years by something that does not yet have even test users? > > Absolutely. Which benefits? Jörg -- EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin js@cs.tu-berlin.de (uni) joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily From robert.thurlow@sun.com Tue Jul 14 15:39:27 2009 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6EMdRn1017889 for ; Tue, 14 Jul 2009 15:39:27 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n6EMdNmO019694; Tue, 14 Jul 2009 16:39:25 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMS00J0XMXPHT00@nwk-avmta-2.sfbay.sun.com>; Tue, 14 Jul 2009 15:39:25 -0700 (PDT) Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.59]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMS00H54MXOZG10@nwk-avmta-2.sfbay.sun.com>; Tue, 14 Jul 2009 15:39:24 -0700 (PDT) Received: from [10.7.250.15] (punchin-client-10-7-250-15.SFBay.Sun.COM [10.7.250.15]) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6EMdJAZ528656; Tue, 14 Jul 2009 15:39:19 -0700 (PDT) Date: Tue, 14 Jul 2009 16:39:19 -0600 From: Robert Thurlow Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4a5cfde7.C9VgJWErnG5yGh9q%Joerg.Schilling@fokus.fraunhofer.de> To: Joerg Schilling Cc: PSARC-ext@sun.com, Mike.Oliver@sun.com, KMcDonald@egenera.com, cifs-eng@sun.com, Brian.Wong@sun.com, Afshin.Ardakani@sun.com Message-id: <4A5D0917.80605@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <20090713222450.GL1274@Sun.COM> <4A5BB8DA.804@sun.com> <4A5C3FC3.2020508@Sun.COM> <23408EBC7ADF4918A35E43F6D6BF1E95@TOSHIBA> <4A5C5726.4010000@Sun.COM> <7B74FF619D8947F2A1062312E072B22C@TOSHIBA> <4A5C5EEA.50202@Sun.COM> <3464DD17027C4B35AF3A2AE3B468B7E8@TOSHIBA> <4A5C6704.8050801@Sun.COM> <4A5CE21F.4000204@sun.com> <4A5CE3F5.7000405@sun.com> <4A5CECCC.3020408@Egenera.COM> <4A5CF0C1.3030100@sun.com> <4a5cf92d.F92r0oy2EnpQ1NS9%Joerg.Schilling@fokus.fraunhofer.de> <4A5CFC20.7090704@sun.com> <4a5cfde7.C9VgJWErnG5yGh9q%Joerg.Schilling@fokus.fraunhofer.de> User-Agent: Thunderbird 2.0.0.16 (X11/20080811) Content-Length: 2834 Status: RO X-Status: $$$$ X-UID: 0000000073 Joerg Schilling wrote: >>> Do you see a benefit from replacing technology that is known since more than 20 >>> years by something that does not yet have even test users? > Which benefits? Caveat - this is about a follow-on case. I'm glad you asked. The current automounter hasn't had a decent amount of work on it for years now, and it has a lot of issues, strictly as a piece of code in a Solaris system. Maps are read on system reboot or at the request of an admin on each client, so new filesystems or changes to locations are not picked up well. Further, unmounts are often necessary to react to changes; this causes the most pain in /net paths. If there's a problem with your network at boot time, you can run without maps or with partial maps until someone notices and manually intervenes. I'm the primary contact on automounter maintenance at Sun these days, and as much as I rely on it, I'm aware of it's warts. As a concept, there are more issues. Autofs requires that an admin edit a flat file of path-to-location relationships and store them one of several network services, and the propagation is widely variable. Automounter maps are not standardized, so multi-platform support is still dodgy. And among implentations compatible with Sun's map format, there are variations in what features work that often mean you have to dumb down to the lowest common denominator. And despite the admin's work, all clients can doctor their maps so that they see different paths to files than their cube-mate does. NFS referrals are baked into two standards (RFC3530 and the not-yet-numbered NFSv4.1 spec), and are a way to redirect the client to another place based in information at the server. FedFS is a standard-in-development in the IETF's NFSv4 working group, and ties well with NFSv4.0 and v4.1. It builds a server-side infrastructure to let servers return a coherent set of referrals to clients. It permits an admin to specify a detailed set of locations, with enough metadata to permit a V4.1 client to understand all the ways it can talk to a multi-homed server, how old the different copies of data are, and whether or not filehandles and state might be preserved after a live migration has happened. Each referral can be managed in LDAP as a distinct thing, rather than as part of a map, but are set-and-forget unless you need to change them. Changes to the namespace should be visible far more quickly. There's also a way to have a zero-admin client find the top of a namespace - it will ask for a DNS RR to get started. FedFS is also trying to be expandable enough to provide SMB referrals to those other clients. Bottom line: we think admins will be able to create uniform namespaces for all of their clients across their whole company, something that has always eluded us with the automounter. Rob T From Nicolas.Williams@sun.com Tue Jul 14 16:46:49 2009 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6ENkmNF022344 for ; Tue, 14 Jul 2009 16:46:48 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6ENkZNg013760; Wed, 15 Jul 2009 00:46:43 +0100 (BST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMS00F0DQ1U2800@brm-avmta-1.central.sun.com>; Tue, 14 Jul 2009 17:46:42 -0600 (MDT) Received: from binky.Central.Sun.COM ([129.153.128.104]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMS0028KQ1URT50@brm-avmta-1.central.sun.com>; Tue, 14 Jul 2009 17:46:42 -0600 (MDT) Received: from binky.Central.Sun.COM (localhost [127.0.0.1]) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3) with ESMTP id n6EMpQKK008388; Tue, 14 Jul 2009 17:51:26 -0500 (CDT) Received: (from nw141292@localhost) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3/Submit) id n6EMpPtd008387; Tue, 14 Jul 2009 17:51:25 -0500 (CDT) Date: Tue, 14 Jul 2009 17:51:25 -0500 From: Nicolas Williams Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A5D0917.80605@sun.com> To: Robert Thurlow Cc: Joerg Schilling , PSARC-ext@sun.com, Mike.Oliver@sun.com, KMcDonald@egenera.com, cifs-eng@sun.com, Brian.Wong@sun.com, Afshin.Ardakani@sun.com Message-id: <20090714225125.GG1274@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <3464DD17027C4B35AF3A2AE3B468B7E8@TOSHIBA> <4A5C6704.8050801@Sun.COM> <4A5CE21F.4000204@sun.com> <4A5CE3F5.7000405@sun.com> <4A5CECCC.3020408@Egenera.COM> <4A5CF0C1.3030100@sun.com> <4a5cf92d.F92r0oy2EnpQ1NS9%Joerg.Schilling@fokus.fraunhofer.de> <4A5CFC20.7090704@sun.com> <4a5cfde7.C9VgJWErnG5yGh9q%Joerg.Schilling@fokus.fraunhofer.de> <4A5D0917.80605@sun.com> X-Authentication-warning: binky.Central.Sun.COM: nw141292 set sender to Nicolas.Williams@sun.com using -f User-Agent: Mutt/1.5.7i Content-Length: 1512 Status: RO X-Status: $$$$ X-UID: 0000000074 On Tue, Jul 14, 2009 at 04:39:19PM -0600, Robert Thurlow wrote: > Joerg Schilling wrote: > > >>>Do you see a benefit from replacing technology that is known since more > >>>than 20 years by something that does not yet have even test users? > > >Which benefits? > > Caveat - this is about a follow-on case. > > I'm glad you asked. The current automounter hasn't had a decent > amount of work on it for years now, and it has a lot of issues, > strictly as a piece of code in a Solaris system. Maps are read > [...] It's not that the code is unmaintained -- these are architectural flaws in the automounter concept. Since the automounter is _client_-driven the namespace information must be distributed to all clients, which works well enough in a small environment, but does not scale to the Internet. Any automount map distribution protocol will just not lend itself to having clients dynamically adjust as the namespace changes -- clients would have to poll too often and detect changes, or the protocols would have to support exchanging diffs, and so on and on. The very notion of automount maps that need to be distributed also tends to centralization of namespace management, which presents its own scalability issues: every time you zfs create/destroy you must edit an automount map on some server, or do some LDAP creates/modifies? ugh, ugh!. Mirror mounts already solve problems that the automounter can't possibly. Referrals extend that solution past a single host. That is good. Nico -- From amw@sun.com Tue Jul 14 16:52:00 2009 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6ENpxQv022743 for ; Tue, 14 Jul 2009 16:52:00 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6ENpwRT016650; Wed, 15 Jul 2009 00:51:59 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMS00805QAJW600@nwk-avmta-1.sfbay.Sun.COM>; Tue, 14 Jul 2009 16:51:55 -0700 (PDT) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMS0038WQAJB790@nwk-avmta-1.sfbay.Sun.COM>; Tue, 14 Jul 2009 16:51:55 -0700 (PDT) Received: from fe-amer-10.sun.com ([192.18.109.80]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6ENptLT013127; Tue, 14 Jul 2009 23:51:55 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMS00100Q7JLC00@mail-amer.sun.com>; Tue, 14 Jul 2009 17:51:54 -0600 (MDT) Received: from [10.1.106.211] ([unknown] [10.1.106.211]) by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMS00E1YQAHNE20@mail-amer.sun.com>; Tue, 14 Jul 2009 17:51:54 -0600 (MDT) Date: Tue, 14 Jul 2009 16:51:53 -0700 From: Alan M Wright Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4a5cfdcc.gATeSQiZ6ujpJJMs%Joerg.Schilling@fokus.fraunhofer.de> Sender: Alan.M.Wright@sun.com To: Joerg Schilling Cc: Darren.Moffat@sun.com, Robert.Thurlow@sun.com, PSARC-ext@sun.com, cifs-eng@sun.com, Brian.Wong@sun.com, Afshin.Ardakani@sun.com Message-id: <4A5D1A19.1050905@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <20090713222450.GL1274@Sun.COM> <4A5BB8DA.804@sun.com> <4A5C3FC3.2020508@Sun.COM> <23408EBC7ADF4918A35E43F6D6BF1E95@TOSHIBA> <4A5C5726.4010000@Sun.COM> <7B74FF619D8947F2A1062312E072B22C@TOSHIBA> <4A5C5EEA.50202@Sun.COM> <3464DD17027C4B35AF3A2AE3B468B7E8@TOSHIBA> <4A5C6704.8050801@Sun.COM> <4A5CFA0C.80108@sun.com> <4a5cfdcc.gATeSQiZ6ujpJJMs%Joerg.Schilling@fokus.fraunhofer.de> User-Agent: Thunderbird 2.0.0.21 (X11/20090323) Content-Length: 533 Status: RO X-Status: $$$$ X-UID: 0000000075 On 07/14/09 14:51, Joerg Schilling wrote: > Alan M Wright wrote: > >> I suspect that magic symlink support is optional in BSD because the >> content of a magic symlink will be changed on the fly by the OS. >> For example, if @{HOST} appears in the symlink that tag will be >> replaced at runtime by the nodename. > > If this is true, how ar these symlinks copied on BSD? That seems to be outside the scope of this case. For information on BSD magic symlinks: http://www.daemon-systems.org/man/symlink.7.html Alan From daleg@elemental.org Tue Jul 14 18:58:33 2009 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6F1wWqQ027567 for ; Tue, 14 Jul 2009 18:58:33 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6F1wOCE023374; Wed, 15 Jul 2009 02:58:28 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMS00C03W5FC200@nwk-avmta-1.sfbay.Sun.COM>; Tue, 14 Jul 2009 18:58:27 -0700 (PDT) Received: from sca-ea-mail-3.sun.com ([192.18.43.21]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMS00LNPW5F0X50@nwk-avmta-1.sfbay.Sun.COM>; Tue, 14 Jul 2009 18:58:27 -0700 (PDT) Received: from relay14i.sun.com (ip124.net129179-4.block1.us.syntegra.com [129.179.4.124]) by sca-ea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6F1tYJW004846; Wed, 15 Jul 2009 01:58:27 +0000 (GMT) Received: from mmp11es.mmp.us.syntegra.com ([160.41.208.11] [160.41.208.11]) by relay14i.sun.com with ESMTP id BT-MMP-5416645; Wed, 15 Jul 2009 01:58:11 +0000 (Z) Received: from relay11i.sun.com (relay11i.sun.com [129.179.4.121]) by mmp11es.mmp.us.syntegra.com with ESMTP id BT-MMP-1728554; Wed, 15 Jul 2009 01:58:11 +0000 (Z) Received: from mercury.elemental.org ([205.134.191.194] [205.134.191.194]) by relay1i.sun.com with ESMTP id BT-MMP-2957764; Wed, 15 Jul 2009 01:58:10 +0000 (Z) Received: from [10.75.10.106] (nat-204-14-233-149-rvo.net.salesforce.com [204.14.233.149]) (authenticated bits=0) by mercury.elemental.org (8.14.3/8.14.3/ELEMENTAL-4.0) with ESMTP id n6F1w7ko023334 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Tue, 14 Jul 2009 21:58:08 -0400 (EDT) Date: Tue, 14 Jul 2009 21:58:07 -0400 From: Dale Ghent Subject: Re: 2009/387 [Pathname Reparse Points] In-reply-to: <4A5D0917.80605@sun.com> To: Robert Thurlow Cc: Joerg Schilling , Afshin.Ardakani@sun.com, cifs-eng@sun.com, Mike.Oliver@sun.com, PSARC-ext@sun.com, Brian.Wong@sun.com Message-id: MIME-version: 1.0 X-Mailer: Apple Mail (2.935.3) Content-type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0 (mercury.elemental.org [205.134.191.194]); Tue, 14 Jul 2009 21:58:08 -0400 (EDT) X-Virus-Scanned: ClamAV version 0.93.1, clamav-milter version 0.93.1 on mercury.elemental.org X-Virus-Status: Clean X-Antispam: No, score=-0.7/5.0, scanned in 0.146sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <200907101634.n6AGYQNl012755@ivrel.sfbay.sun.com> <20090713222450.GL1274@Sun.COM> <4A5BB8DA.804@sun.com> <4A5C3FC3.2020508@Sun.COM> <23408EBC7ADF4918A35E43F6D6BF1E95@TOSHIBA> <4A5C5726.4010000@Sun.COM> <7B74FF619D8947F2A1062312E072B22C@TOSHIBA> <4A5C5EEA.50202@Sun.COM> <3464DD17027C4B35AF3A2AE3B468B7E8@TOSHIBA> <4A5C6704.8050801@Sun.COM> <4A5CE21F.4000204@sun.com> <4A5CE3F5.7000405@sun.com> <4A5CECCC.3020408@Egenera.COM> <4A5CF0C1.3030100@sun.com> <4a5cf92d.F92r0oy2EnpQ1NS9%Joerg.Schilling@fokus.fraunhofer.de> <4A5CFC20.7090704@sun.com> <4a5cfde7.C9VgJWErnG5yGh9q%Joerg.Schilling@fokus.fraunhofer.de> <4A5D0917.80605@sun.com> Content-Length: 4411 Status: RO X-Status: $$$$ X-UID: 0000000076 On Jul 14, 2009, at 6:39 PM, Robert Thurlow wrote: > Joerg Schilling wrote: > >>>> Do you see a benefit from replacing technology that is known >>>> since more than 20 years by something that does not yet have even >>>> test users? > >> Which benefits? > > Caveat - this is about a follow-on case. > > I'm glad you asked. The current automounter hasn't had a decent > amount of work on it for years now, and it has a lot of issues, > strictly as a piece of code in a Solaris system. Maps are read > on system reboot or at the request of an admin on each client, > so new filesystems or changes to locations are not picked up well. > Further, unmounts are often necessary to react to changes; this > causes the most pain in /net paths. If there's a problem with > your network at boot time, you can run without maps or with > partial maps until someone notices and manually intervenes. > I'm the primary contact on automounter maintenance at Sun these > days, and as much as I rely on it, I'm aware of it's warts. > > As a concept, there are more issues. Autofs requires that an > admin edit a flat file of path-to-location relationships and > store them one of several network services, and the propagation > is widely variable. Automounter maps are not standardized, so > multi-platform support is still dodgy. And among implentations > compatible with Sun's map format, there are variations in what > features work that often mean you have to dumb down to the > lowest common denominator. And despite the admin's work, all > clients can doctor their maps so that they see different paths > to files than their cube-mate does. > > NFS referrals are baked into two standards (RFC3530 and the > not-yet-numbered NFSv4.1 spec), and are a way to redirect the > client to another place based in information at the server. > FedFS is a standard-in-development in the IETF's NFSv4 working > group, and ties well with NFSv4.0 and v4.1. It builds a > server-side infrastructure to let servers return a coherent > set of referrals to clients. It permits an admin to specify > a detailed set of locations, with enough metadata to permit > a V4.1 client to understand all the ways it can talk to a > multi-homed server, how old the different copies of data are, > and whether or not filehandles and state might be preserved > after a live migration has happened. Each referral can be > managed in LDAP as a distinct thing, rather than as part of > a map, but are set-and-forget unless you need to change them. > Changes to the namespace should be visible far more quickly. > There's also a way to have a zero-admin client find the top > of a namespace - it will ask for a DNS RR to get started. > FedFS is also trying to be expandable enough to provide SMB > referrals to those other clients. > > Bottom line: we think admins will be able to create uniform > namespaces for all of their clients across their whole company, > something that has always eluded us with the automounter. FWIW, and as another point-of-usefullness regarding referrals, is to also look at AFS. While AFS doesn't embody anything resembling referrals, the end result is something similar - AFS volumes (read: NFS exports) living on multiple AFS volume servers (read: multiple NFS servers) mounted to form a tree. Where the volume lives is transparent to the user. A user whose cwd is /afs/@cell/stuff/foo is in AFS volume stuff.foo on server pluto. Volume stuff.foo.bar could live on server mars, and be "mounted" (in AFS-speak) at /afs/@cell/stuff/foo/morestuff, and when the user accesses files in there, they're getting that stuff from mars. Federated in a way.... but the end result is the same. That user's machine is accessing a AFS cell and beyond the credentials needed to access bits within that cell, there is no need for the user to maintain a map of which AFS volume on whatever server is to be mounted where. The AFS voldb takes care of that, along with the AFS client. When I need to add a new volume, stuff.foo.baz, I merely create it on a AFS volume server, grant it sundry things such as any quota and ACLs, and mount it where I need it in the AFS tree. Done. No having to fiddle with maps. So to Joerg's issue with this being something new... the concept is far from new. This is just a very new implementation of the concept (and one that is sorely overdue, imho) /dale From reparse-points-request@sun.com Fri Jul 31 12:05:33 2009 Return-Path: Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01 [129.145.155.118]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6VJ5W2d018558 for ; Fri, 31 Jul 2009 12:05:32 -0700 (PDT) Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6VJ6g9U045520; Fri, 31 Jul 2009 12:06:42 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6VJ6c2J016638 for <@sunmail2sca.sfbay.sun.com:reparse-points@sun.com>; Fri, 31 Jul 2009 20:06:40 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KNN00K01UF3WT00@nwk-avmta-2.sfbay.sun.com> for reparse-points@sun.com (ORCPT reparse-points@sun.com); Fri, 31 Jul 2009 12:06:39 -0700 (PDT) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KNN00IJ5UF27J30@nwk-avmta-2.sfbay.sun.com> for reparse-points@sun.com (ORCPT reparse-points@sun.com); Fri, 31 Jul 2009 12:06:39 -0700 (PDT) Received: from fe-amer-10.sun.com ([192.18.109.80]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6VJ6c1r028245 for ; Fri, 31 Jul 2009 19:06:38 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KNN00800TW08O00@mail-amer.sun.com> for reparse-points@sun.com (ORCPT reparse-points@sun.com); Fri, 31 Jul 2009 13:06:38 -0600 (MDT) Received: from [10.1.106.211] ([unknown] [10.1.106.211]) by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KNN00HATUF16V30@mail-amer.sun.com> for reparse-points@sun.com (ORCPT reparse-points@sun.com); Fri, 31 Jul 2009 13:06:37 -0600 (MDT) Date: Fri, 31 Jul 2009 12:06:37 -0700 From: Alan M Wright Subject: Re: Reparse materials In-reply-to: <4A72F4C9.4090906@gmail.com> Sender: Alan.M.Wright@sun.com To: Mark Martin Cc: reparse-points@sun.com Message-id: <4A7340BD.8080700@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200907291812.n6TICwCi012972@ivrel.sfbay.sun.com> <4A71D0BB.7020506@gmail.com> <4A729B2C.4040605@sun.com> <4A72F4C9.4090906@gmail.com> User-Agent: Thunderbird 2.0.0.21 (X11/20090323) Content-Length: 3768 Status: RO X-Status: $$$$ X-UID: 0000000077 On 07/31/09 06:42, Mark Martin wrote: > Alan M Wright wrote: >> On 07/30/09 11:56, Alan.M.Wright wrote: >>> Mark Martin wrote: >>>> Asa, >>>> >>>> Would you mind please copying these to the issues file for case >>>> PSARC 2009/387 "Reparse Points"? Thanks. >>>> >>>> ________ >>>> >>>> mam-01 In the Interface table: >>>> "Note: There is no need to define a name in the file system to >>>> support >>>> a user space door service that is called from the kernel, and not >>>> defining a name avoids potential security concerns. Thus the door >>>> interface is private to the implementation and not mentioned in >>>> the >>>> interface table." >>>> >>>> Does this mean that the door name is randomly generated? A cursory >>>> source inspection will reveal the name, so what are the security >>>> concerns? It may not be customary to report door names, but I'm >>>> curious what the motivation is. >>> >>> You can create a door name in var/run, which is a means for a door >>> client to locate a door server. In the case of a kernel door client, >>> it can lookup the door handle directly, it doesn't need any name in >>> the file system. >>> >>> If you put a name in /var/run unnecessarily, that opens an avenue for >>> a potential attack because people know what name to use to access >>> the door server. It seems better to not provide a name in the file >>> system, which ensures that user space applications have no means >>> to access the door server. >>> >>>> mam-02 Reparse CLI consolidation private versus uncommitted? Does >>>> this unnecessarily lower usability expectations? Are we prepared >>>> not to give the user a well known way to manage these links from CLI? >>> >>> There are no user space consumers outside of the project. Perhaps >>> we should remove the test program description from the spec. >>> >>> Alan >> >> I've been reconsidering this discussion and I think we may want to >> have a door file in /var/run. >> >> As things are now, reparsed must pass the door fd to kernel modules, >> which means there is a specific relationship between reparsed and >> consumers of the reparse API: reparsed must open the device and issue >> an ioctl. Also, only kernel components can use the reparse API >> because user space applications can't find the door. >> >> I think it would be good to remove these dependencies and limitations, >> and I can't think of any security issues around applications asking >> reparsed to exchange the svc_data. It doesn't provide anything that >> the application couldn't have gotten by going directly to the backend >> referral service. >> >> The door file would be /var/run/reparse_door. > I'm fine here either way, although revised approach makes more sense to > me. I just wanted to understand what was hiding (and how > synchronization was occurring). >> >> I'd also like to remove the rp test program from the spec. It's not >> a deliverable and it probably shouldn't be documented in the case. >> >> Any comments on updating the spec? > Well, personally, I think it's a mistake not to offer a CLI for users. > It's not clear to me how admins are supposed to be creating these > reparse points in the first place. I realize that your cli as spec'd is > intended for testing purposes only, but what is the harm in promoting > those tools to users? If someone wanted to, they can create reparse points using the 'ln' command but I'm not sure that would be useful without a service to interpret them. Reparse points provide system infrastructure on which we will build usable services, such as SMB/DFS and NFS referrals (the follow-on projects in the umbrella case roadmap). Management interfaces will be provided for such services. Alan From glenn.skinner@sun.com Wed Aug 5 13:00:12 2009 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n75K0CmX010039 for ; Wed, 5 Aug 2009 13:00:12 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n75K04df028244 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Wed, 5 Aug 2009 14:00:11 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KNX0060B689HT00@nwk-avmta-1.sfbay.Sun.COM> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 05 Aug 2009 13:00:09 -0700 (PDT) Received: from dm-sfbay-02.sfbay.sun.com ([129.146.11.31]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KNX00ADR689FKE0@nwk-avmta-1.sfbay.Sun.COM> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 05 Aug 2009 13:00:09 -0700 (PDT) Received: from ivrel.sfbay.sun.com (ivrel.SFBay.Sun.COM [129.146.74.76]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n75K09EN045222 for ; Wed, 05 Aug 2009 13:00:09 -0700 (PDT) Received: from ivrel (ivrel [129.146.74.76]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with SMTP id n75JpUSY024599 for ; Wed, 05 Aug 2009 12:51:30 -0700 (PDT) Date: Wed, 05 Aug 2009 12:51:30 -0700 (PDT) From: Glenn Skinner Subject: 2009/387 [Pathname Reparse Points] approved To: psarc-ext@sun.com Reply-to: Glenn Skinner Message-id: <200908051951.n75JpUSY024599@ivrel.sfbay.sun.com> MIME-version: 1.0 X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.6_36 SunOS 5.11 sun4u sparc Content-type: TEXT/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-MD5: vFGOiFiRODhrdpRYBSdNPQ== X-PMX-Version: 5.4.1.325704 Content-Length: 4699 Status: RO X-Status: $$$$ X-UID: 0000000078 The pathname reparse points case (2009/387) was approved during today's PSARC meeting, subject to providing updated materials that reflect changes agreed to during our discussion. Here are my notes from the meeting. They describe the required updates. -- Glenn ---------------- Notes from the commitment review for PSARC 2009/387 [Pathname Reparse Points] held on 5-Aug-2009. The case was approved with members voting as follows: Kais Belgaied approve Mark Carlson approve Garrett D'Amore approve Richard Matthews approve Sebastien Roy approve Glenn Skinner approve Mark Martin [LSARC] approve Discussion during the review led to two specification changes: - The release binding changed from update to minor. (This change was triggered by observing that the change to the signature of VOP_LOOKUP() was not suitable for an update binding.) - The stability level of the in-symlink reparse point format changed from Committed Private to Committed. (This change was motivated by noting that there should be a way to know that one has not inadvertenty created a reparse point when creating a symlink.) During the discussion, we identified numerous places where the specification and the 20 questions document could and should be clarified. The project team agreed to do so and will deliver a revised specification containing the clarifications (as well as the changes noted above). In particular: - The discussion of release binding in the 20 questions document will be clarified to note that the project intends to deliver into the Nevada release (which implies that it will also deliver into OpenSolaris.) - The response to question 1 in the 20 questions document will be revised to name 2009/399 [Reparse Points and Referrals Umbrella Case] as a related case. - The specification's discussion of error return values in sections 5.1 and 5.1 will be clarified to avoid giving the impression that they will set errno. - The specification will be updated to make it clear that the check for existence of a file whose name is identical to the contents of a to-be-established reparse point appliies only to the create reparse point call, not to symlink(2). Other points of note: - The key issue in the case was the question of whether or not the presence of a reparse point affects POSIX conformance. The problem is that a POSIX-conformant application could create a symlink whose contents match the reparse point syntax. Subsequently, that, or another POSIX-conformat, application could do a pathname traversal through that symlink, have it be treated as a reparse point (thereby inducing its interpretation, perhaps as an NFS referral), and experience normal symlink semantics (either a traversal failure or having the traversal resolve to a file whose name matches the reparse point's contents). The project team noted that this situation can only arise when a non-POSIX conformant service (such as NFS or SMB) has control over the part of the file system name space containing the reparse point. The team further claimed that this involvement of a non-POSIX copnformant service removes the traversal from the scope of activities subject to POSIX requirements. After some discussion, the committee accepted this argument. - We discussed whether adding some mechanism to enable and disable reporse points, perhaps on a file system-wide basis would help alleviate the compatibility concerns illustrated by the previous point. The consensus among the committee and project team members was that this proposed cure was worse than the disease. - We wondered whether this case hould have a dependency on one of the follow-on cases mentioned in 2009/399 [Reparse Points and Referrals Umbrella Case] to ensure that a consumer of this project's deliverables is delived in tandem with this projhect itself. The project team argued that this case provides value in and of itself, and the committee member accepted this argument. The committee also wished to offer various pieces of advice: - Any follow-on project that provides a service employing reparse points should verify that referral data provided in response to triggering a reparse point is properly audited on the client system that triggered the reparse point. - As part of preparing the materials for the ARC review of the upcoming NFS referrals case, committee members advise that project team to provide a step by step account of what happens when pathname traversam encounters a referral. From storycrafter@gmail.com Fri Sep 18 08:52:28 2009 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n8IFqRS3026723 for ; Fri, 18 Sep 2009 08:52:27 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n8IFqPA9052542 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Fri, 18 Sep 2009 09:52:27 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KQ600E0JC3E9Y00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 18 Sep 2009 08:52:26 -0700 (PDT) Received: from sca-ea-mail-2.sun.com ([192.18.43.25]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KQ6008SGC3EYN90@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 18 Sep 2009 08:52:26 -0700 (PDT) Received: from relay15i.sun.com (ip125.net129179-4.block1.us.syntegra.com [129.179.4.125]) by sca-ea-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n8IFgCYj029801 for ; Fri, 18 Sep 2009 15:52:25 +0000 (GMT) Received: from mmp14es.mmp.us.syntegra.com ([160.41.208.14] [160.41.208.14]) by relay15i.sun.com with ESMTP id BT-MMP-2085317 for PSARC-ext@sun.com; Fri, 18 Sep 2009 15:52:25 +0000 (Z) Received: from relay14i.sun.com (relay14i.sun.com [129.179.4.124]) by mmp14es.mmp.us.syntegra.com with ESMTP id BT-MMP-76053 for PSARC-ext@sun.com; Fri, 18 Sep 2009 15:52:25 +0000 (Z) Received: from mail-yx0-f203.google.com ([209.85.210.203] [209.85.210.203]) by relay1i.sun.com with ESMTP id BT-MMP-18230925 for PSARC-ext@sun.com; Fri, 18 Sep 2009 15:52:25 +0000 (Z) Received: by yxe41 with SMTP id 41so1304423yxe.30 for ; Fri, 18 Sep 2009 08:52:17 -0700 (PDT) Received: by 10.101.90.12 with SMTP id s12mr1642112anl.77.1253289137167; Fri, 18 Sep 2009 08:52:17 -0700 (PDT) Received: from ?172.16.202.89? (68-252-106-20.ded.ameritech.net [68.252.106.20]) by mx.google.com with ESMTPS id d29sm33937and.1.2009.09.18.08.52.15 (version=TLSv1/SSLv3 cipher=RC4-MD5) ; Fri, 18 Sep 2009 08:52:15 -0700 (PDT) Date: Fri, 18 Sep 2009 10:51:56 -0500 From: Mark Martin Subject: Opinion for PSARC review - PSARC/2009/387 Reparse Points To: opensolaris-arc@opensolaris.org, PSARC-ext@sun.com Message-id: <4AB3AC9C.4010404@gmail.com> MIME-version: 1.0 Content-type: multipart/mixed; boundary="Boundary_(ID_YNptXbQEdwrYCD/eLa/R6g)" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from :user-agent:mime-version:to:subject:content-type; bh=DGM3wwL8CgPcLm8wn4LzXAdC3PFkeSr/5O77oBuErS0=; b=cTj68FZmU1QqOItBD7XW3hEfTEojNoA8NjZFnRmTstkkT2bOSsaAvgC0MBU073WIXo ttpqYP7lWLSry9JY7j7rj5N58x+/vAg2A05hCfGPpWbOWGNF2NwVMgbQMjvHmNiMAKn6 7/ilTVpgUgqI8u6v6NgUW+RGzxt/wfO/QKWcg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:subject :content-type; b=F/GWcWOV/tB6uKO/LJG6IpaslQ1AmaVZvz3r6F7xvwXdjILlDFxjnrAptd7Z8PdpJh ReZYQB7WI+5q3yZL1fLgpfhyLUu9zDBoWtdhYZPV2lOCEwnHbXR0rJyxpr2ZWS/UXN2e 4ZhL9YeScMTnjA+nLUAElB2C0aFzC31ngWrfQ= X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== User-Agent: Thunderbird 2.0.0.23 (Windows/20090812) Content-Length: 10134 Status: RO X-Status: $$$$ X-UID: 0000000079 This is a multi-part message in MIME format. --Boundary_(ID_YNptXbQEdwrYCD/eLa/R6g) Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT Attached is the opinion for PSARC, please review by 9/25/09. *Note: For the record, the original opinion was written as a .ms file (my first), which /should/ be available in the case directory, along with a pdf and the attached txt file. At the time of this email however, most of the case materials are redacted on the external website, which is an error, and I will contact the appropriate team to correct that. Thanks, Mark --Boundary_(ID_YNptXbQEdwrYCD/eLa/R6g) Content-type: text/plain; name=opinion.txt Content-transfer-encoding: 7BIT Content-disposition: inline; filename=opinion.txt sun microsystems Systems Architecture Committee _________________________________________________________________ Subject: Pathname Reparse Points Submitted by: Afshin Salek File: PSARC/2009/387/opinion.ms Date: August 5th, 2009 Committee: Glenn Skinner (opinion written by Mark Mar- tin), Garrett D'Amore, Mark Carlson, Richard Matthews, Sebastien Roy. Product Approval Committee: Solaris PAC solaris-pac-opinion@sun.com 1. Summary This is one of a series of projects under the umbrella case: PSARC 2009/399 "Reparse Points and Referrals Umbrella Case". There are situations where a mechanism is needed to reflect the concept that data is not present at a particular path, but can be found in some alternate location(s). Examples include "referrals" used to build unified name spaces in NFSv4.x and SMB, and data relocation in HSM systems. A "reparse point" is defined as the marker for a namespace redirection and a container for the metadata to specify where the target of this redirection is. Reparse points are intended to be a general mechanism for location redirection and as such the file system that con- tains them is not cognizant of the reparse point format or content. Services that use reparse points know how to inter- pret and use the stored data. 2. Decision & Precedence Information The project is approved as specified in reference [2]. The project may be delivered in a minor release of Solaris. 3. Interfaces PSARC/2009/387 Copyright 2009 Sun Microsystems - 2 - The project exports the following interfaces. _______________________________________________________________________ | Interfaces Exported | |__________________________|________________________|_________________| |Interface | Classification | Comments | |__________________________|________________________|_________________| |VOP_LOOKUP, fop_lookup | Consolidation Private | Added new argu-| | | | ment: vattr_t| | | | *vap | | | | | |XAT_REPARSE | Consolidation Private | Reparse exten-| | | | sible attribute| | | | | |symlink context syntax | Committed | | |for Reparse points | | | | | | | |reparse_kderef | Consolidation Private | Reparse kernel| | | | routines | |reparse_init | | | |reparse_parse | | | |reparse_free | | | | | | | |reparse_init | | | |reparse_parse | | | |reparse_add | | | |reparse_remove | | Reparse library| |reparse_unparse | Consolidation Private | routines | |reparse_free | | | |reparse_create | | | |reparse_delete | | | |reparse_deref | | | | | | | |reparse_plugin_ops_t | Consolidation Private | Reparse plugin| | | | ops | | | | | |/usr/lib/reparse/reparsed | Consolidation Private | Reparse daemon | |/usr/lib/libreparse.so | Consolidation Private | Reparse library| | | | | |/var/run/reparse_door | Consolidation Private | User space door| | | | | |svc: | Uncommitted | SMF Service | |/system/filesystem/reparse| | | |__________________________|________________________|_________________| 4. Opinion 4.1. POSIX Conformance The key issue in the case was the question of whether or not the presence of a reparse point affects POSIX conformance. PSARC/2009/387 Copyright 2009 Sun Microsystems - 3 - The problem is that a POSIX-conformant application could create a symlink whose contents match the reparse point syn- tax. Subsequently, that, or another POSIX-conformat, appli- cation could do a pathname traversal through that symlink, have it be treated as a reparse point (thereby inducing its interpretation, perhaps as an NFS referral), and experience normal symlink semantics (either a traversal failure or hav- ing the traversal resolve to a file whose name matches the reparse point's contents). The project team noted that this situation can only arise when a non-POSIX conformant service (such as NFS or SMB) has control over the part of the file system name space contain- ing the reparse point. The team further claimed that this involvement of a non-POSIX copnformant service removes the traversal from the scope of activities subject to POSIX requirements. After some discussion, the committee accepted this argument. 4.2. Follow-on Consumers The question was raised whether this case should have a dependency on one of the follow-on cases mentioned in 2009/399 [Reparse Points and Referrals Umbrella Case] to ensure that a consumer of this project's deliverables is delived in tandem with this projhect itself. The project team argued that this case provides value in and of itself, and the committee members accepted this argument 4.3. Specification Changes 4.3.1. Release Binding The release binding changed from update to minor. This change was triggered by observing that the change to the signature of VOP_LOOKUP() was not suitable for an update binding. 4.3.2. Stability Level of the Reparse Format The stability level of the in-symlink reparse point format changed from Committed Private to Committed. This change was motivated by noting that there should be a way to know that one has not inadvertenty created a reparse point when creat- ing a symlink. 4.4. Reparse Point Enable/Disable The committee discussed whether adding some mechanism to enable and disable reparse points, perhaps on a file PSARC/2009/387 Copyright 2009 Sun Microsystems - 4 - system-wide basis would help alleviate the compatibility concerns illustrated by the previous point. The consensus among the committee and project team members was that this proposed cure was worse than the disease. 4.5. Other Minor Clarifications There were various other minor points of confusion (some relating to terminology). Where appropriate, the project team updated the specification to incorporate the clarifica- tions. 5. Minority Opinion(s) None. 6. Advisory Information 6.1. Auditing in Follow-on Projects Any follow-on project that provides a service employing reparse points should verify that referral data provided in response to triggering a reparse point is properly audited on the client system that triggered the reparse point. 6.2. Upcoming NFS Referrals Case As part of preparing the materials for the ARC review of the upcoming NFS referrals case, committee members advise the project team to provide a step by step account of what hap- pens when pathname traversal encounters a referral. 7. Appendices 7.1. Appendix A: Technical Changes Required None. 7.2. Appendix B: Technical Changes Advised None. 7.3. Appendix C: Reference Material Unless stated otherwise, path names are relative to the case directory PSARC/2009/387. 1. Onepager. File: 20090709_afshin.salek 2. Project Specification. File: final.materials/reparse_spec.txt PSARC/2009/387 Copyright 2009 Sun Microsystems - 5 - 3. Commmitment Minutes. File: commitment-notes 4. PSARC 20 Questions. File: final.materials/reparse_20q.txt 5. Global namespace: The future of distributed file server management. http://findarticles.com/p/articles/mi_m0BRZ/is_2_23/ai_98709766/ 6. Implementation Guide for Referrals in NFSv4. http://tools.ietf.org/html/draft-ietf-nfsv4-referrals- 00 7. DFS Technical Reference. http://technet.microsoft.com/en- us/library/cc757042%28WS.10%29.aspx PSARC/2009/387 Copyright 2009 Sun Microsystems --Boundary_(ID_YNptXbQEdwrYCD/eLa/R6g)-- From sac-owner Wed Sep 30 10:59:49 2009 Received: from ivrel.sfbay.sun.com (ivrel.SFBay.Sun.COM [129.146.74.76]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n8UHxnAS025882 for ; Wed, 30 Sep 2009 10:59:49 -0700 (PDT) Received: from ivrel (ivrel [129.146.74.76]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with SMTP id n8UHbDNx005060; Wed, 30 Sep 2009 10:37:13 -0700 (PDT) Message-Id: <200909301737.n8UHbDNx005060@ivrel.sfbay.sun.com> Date: Wed, 30 Sep 2009 10:37:13 -0700 (PDT) From: Glenn Skinner Reply-To: Glenn Skinner Subject: for review: PSARC/2009/387 [Pathname Reparse Points] To: sac-review@sac.sfbay.sun.com Cc: storycrafter@gmail.com MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: VtX7RnNuurOFeHzskXLVLQ== X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.6_36 SunOS 5.11 sun4u sparc Status: RO Content-Length: 9563 Please review the following opinion and respond with any comments you may have by Wednesday, October 7th. There's a PDF rendering of the opinion in the case directory for those who prefer to view the opinion in that format. -- Glenn ---------------- sun microsystems Systems Architecture Committee _________________________________________________________________ Subject: Pathname Reparse Points Submitted by: Afshin Salek File: PSARC/2009/387/opinion.ms Date: August 5th, 2009 Committee: Glenn Skinner (opinion written by Mark Mar- tin), Garrett D'Amore, Mark Carlson, Richard Matthews, Sebastien Roy. Product Approval Committee: Solaris PAC solaris-pac-opinion@sun.com 1. Summary This is one of a series of projects under the umbrella case: PSARC 2009/399 "Reparse Points and Referrals Umbrella Case". There are situations where a mechanism is needed to reflect the concept that data is not present at a particular path, but can be found in some alternate location(s). Examples include "referrals" used to build unified name spaces in NFSv4.x and SMB, and data relocation in HSM systems. A "reparse point" is defined as the marker for a namespace redirection and a container for the metadata to specify where the target of this redirection is. Reparse points are intended to be a general mechanism for location redirection and as such the file system that con- tains them is not cognizant of the reparse point format or content. Services that use reparse points know how to inter- pret and use the stored data. 2. Decision & Precedence Information The project is approved as specified in reference [2]. The project may be delivered in a minor release of Solaris. 3. Interfaces PSARC/2009/387 Copyright 2009 Sun Microsystems - 2 - The project exports the following interfaces. _______________________________________________________________________ | Interfaces Exported | |__________________________|________________________|_________________| |Interface | Classification | Comments | |__________________________|________________________|_________________| |VOP_LOOKUP, fop_lookup | Consolidation Private | Added new argu-| | | | ment: vattr_t| | | | *vap | | | | | |XAT_REPARSE | Consolidation Private | Reparse exten-| | | | sible attribute| | | | | |symlink context syntax | Committed | | |for Reparse points | | | | | | | |reparse_kderef | Consolidation Private | Reparse kernel| | | | routines | |reparse_init | | | |reparse_parse | | | |reparse_free | | | | | | | |reparse_init | | | |reparse_parse | | | |reparse_add | | | |reparse_remove | | Reparse library| |reparse_unparse | Consolidation Private | routines | |reparse_free | | | |reparse_create | | | |reparse_delete | | | |reparse_deref | | | | | | | |reparse_plugin_ops_t | Consolidation Private | Reparse plugin| | | | ops | | | | | |/usr/lib/reparse/reparsed | Consolidation Private | Reparse daemon | |/usr/lib/libreparse.so | Consolidation Private | Reparse library| | | | | |/var/run/reparse_door | Consolidation Private | User space door| | | | | |svc: | Uncommitted | SMF Service | |/system/filesystem/reparse| | | |__________________________|________________________|_________________| 4. Opinion 4.1. POSIX Conformance The key issue in the case was the question of whether or not the presence of a reparse point affects POSIX conformance. PSARC/2009/387 Copyright 2009 Sun Microsystems - 3 - The problem is that a POSIX-conformant application could create a symlink whose contents match the reparse point syn- tax. Subsequently, that, or another POSIX-conformat, appli- cation could do a pathname traversal through that symlink, have it be treated as a reparse point (thereby inducing its interpretation, perhaps as an NFS referral), and experience normal symlink semantics (either a traversal failure or hav- ing the traversal resolve to a file whose name matches the reparse point's contents). The project team noted that this situation can only arise when a non-POSIX conformant service (such as NFS or SMB) has control over the part of the file system name space contain- ing the reparse point. The team further claimed that this involvement of a non-POSIX copnformant service removes the traversal from the scope of activities subject to POSIX requirements. After some discussion, the committee accepted this argument. 4.2. Follow-on Consumers The question was raised whether this case should have a dependency on one of the follow-on cases mentioned in 2009/399 [Reparse Points and Referrals Umbrella Case] to ensure that a consumer of this project's deliverables is delived in tandem with this projhect itself. The project team argued that this case provides value in and of itself, and the committee members accepted this argument 4.3. Specification Changes 4.3.1. Release Binding The release binding changed from update to minor. This change was triggered by observing that the change to the signature of VOP_LOOKUP() was not suitable for an update binding. 4.3.2. Stability Level of the Reparse Format The stability level of the in-symlink reparse point format changed from Committed Private to Committed. This change was motivated by noting that there should be a way to know that one has not inadvertenty created a reparse point when creat- ing a symlink. 4.4. Reparse Point Enable/Disable The committee discussed whether adding some mechanism to enable and disable reparse points, perhaps on a file PSARC/2009/387 Copyright 2009 Sun Microsystems - 4 - system-wide basis would help alleviate the compatibility concerns illustrated by the previous point. The consensus among the committee and project team members was that this proposed cure was worse than the disease. 4.5. Other Minor Clarifications There were various other minor points of confusion (some relating to terminology). Where appropriate, the project team updated the specification to incorporate the clarifica- tions. 5. Minority Opinion(s) None. 6. Advisory Information 6.1. Auditing in Follow-on Projects Any follow-on project that provides a service employing reparse points should verify that referral data provided in response to triggering a reparse point is properly audited on the client system that triggered the reparse point. 6.2. Upcoming NFS Referrals Case As part of preparing the materials for the ARC review of the upcoming NFS referrals case, committee members advise the project team to provide a step by step account of what hap- pens when pathname traversal encounters a referral. 7. Appendices 7.1. Appendix A: Technical Changes Required None. 7.2. Appendix B: Technical Changes Advised None. 7.3. Appendix C: Reference Material Unless stated otherwise, path names are relative to the case directory PSARC/2009/387. 1. Onepager. File: 20090709_afshin.salek 2. Project Specification. File: final.materials/reparse_spec.txt PSARC/2009/387 Copyright 2009 Sun Microsystems - 5 - 3. Commmitment Minutes. File: commitment-notes 4. PSARC 20 Questions. File: final.materials/reparse_20q.txt 5. Global namespace: The future of distributed file server management. http://findarticles.com/p/articles/mi_m0BRZ/is_2_23/ai_98709766/ 6. Implementation Guide for Referrals in NFSv4. http://tools.ietf.org/html/draft-ietf-nfsv4-referrals- 00 7. DFS Technical Reference. http://technet.microsoft.com/en- us/library/cc757042%28WS.10%29.aspx PSARC/2009/387 Copyright 2009 Sun Microsystems From sac-owner Tue Oct 13 12:12:36 2009 Received: from ivrel.sfbay.sun.com (ivrel.SFBay.Sun.COM [129.146.74.76]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n9DJCZGb006224 for ; Tue, 13 Oct 2009 12:12:35 -0700 (PDT) Received: from ivrel (ivrel [129.146.74.76]) by ivrel.sfbay.sun.com (8.14.3+Sun/8.14.3) with SMTP id n9DIo6Qn015218; Tue, 13 Oct 2009 11:50:06 -0700 (PDT) Message-Id: <200910131850.n9DIo6Qn015218@ivrel.sfbay.sun.com> Date: Tue, 13 Oct 2009 11:50:06 -0700 (PDT) From: Glenn Skinner Reply-To: Glenn Skinner Subject: to archive: PSARC 2009/387 [Pathname Reparse Points] To: sac-opinion@sac.sfbay.sun.com Cc: storycrafter@gmail.com, afshin.ardakani@sun.com MIME-Version: 1.0 Content-Type: MULTIPART/mixed; BOUNDARY=Covey_of_Partridges_283_000 X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.6_36 SunOS 5.11 sun4u sparc Status: RO Content-Length: 9907 --Covey_of_Partridges_283_000 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: aZcPiBA0shzC3oIDjPGj9A== All review periods having passed without comment, the opinion for PSARC/2009/287 [Pathname Reparse Points] is now final and ready to be archived. An ASCII version of the opinion is attached to this mail, and a PDF version may be found in the case directory. -- Glenn --Covey_of_Partridges_283_000 Content-Type: TEXT/plain; name="opinion.txt"; charset=us-ascii; x-unix-mode=0664 Content-Description: opinion.txt Content-MD5: 0rZfIZm0JTilhCnhw0/pDw== sun microsystems Systems Architecture Committee _________________________________________________________________ Subject: Pathname Reparse Points Submitted by: Afshin Salek File: PSARC/2009/387/opinion.ms Date: August 5th, 2009 Committee: Glenn Skinner (opinion written by Mark Mar- tin), Garrett D'Amore, Mark Carlson, Richard Matthews, Sebastien Roy. Product Approval Committee: Solaris PAC solaris-pac-opinion@sun.com 1. Summary This is one of a series of projects under the umbrella case: PSARC 2009/399 "Reparse Points and Referrals Umbrella Case". There are situations where a mechanism is needed to reflect the concept that data is not present at a particular path, but can be found in some alternate location(s). Examples include "referrals" used to build unified name spaces in NFSv4.x and SMB, and data relocation in HSM systems. A "reparse point" is defined as the marker for a namespace redirection and a container for the metadata to specify where the target of this redirection is. Reparse points are intended to be a general mechanism for location redirection and as such the file system that con- tains them is not cognizant of the reparse point format or content. Services that use reparse points know how to inter- pret and use the stored data. 2. Decision & Precedence Information The project is approved as specified in reference [2]. The project may be delivered in a minor release of Solaris. 3. Interfaces PSARC/2009/387 Copyright 2009 Sun Microsystems - 2 - The project exports the following interfaces. _______________________________________________________________________ | Interfaces Exported | |__________________________|________________________|_________________| |Interface | Classification | Comments | |__________________________|________________________|_________________| |VOP_LOOKUP, fop_lookup | Consolidation Private | Added new argu-| | | | ment: vattr_t| | | | *vap | | | | | |XAT_REPARSE | Consolidation Private | Reparse exten-| | | | sible attribute| | | | | |symlink context syntax | Committed | | |for Reparse points | | | | | | | |reparse_kderef | Consolidation Private | Reparse kernel| | | | routines | |reparse_init | | | |reparse_parse | | | |reparse_free | | | | | | | |reparse_init | | | |reparse_parse | | | |reparse_add | | | |reparse_remove | | Reparse library| |reparse_unparse | Consolidation Private | routines | |reparse_free | | | |reparse_create | | | |reparse_delete | | | |reparse_deref | | | | | | | |reparse_plugin_ops_t | Consolidation Private | Reparse plugin| | | | ops | | | | | |/usr/lib/reparse/reparsed | Consolidation Private | Reparse daemon | |/usr/lib/libreparse.so | Consolidation Private | Reparse library| | | | | |/var/run/reparse_door | Consolidation Private | User space door| | | | | |svc: | Uncommitted | SMF Service | |/system/filesystem/reparse| | | |__________________________|________________________|_________________| 4. Opinion 4.1. POSIX Conformance The key issue in the case was the question of whether or not the presence of a reparse point affects POSIX conformance. PSARC/2009/387 Copyright 2009 Sun Microsystems - 3 - The problem is that a POSIX-conformant application could create a symlink whose contents match the reparse point syn- tax. Subsequently, that, or another POSIX-conformat, appli- cation could do a pathname traversal through that symlink, have it be treated as a reparse point (thereby inducing its interpretation, perhaps as an NFS referral), and experience normal symlink semantics (either a traversal failure or hav- ing the traversal resolve to a file whose name matches the reparse point's contents). The project team noted that this situation can only arise when a non-POSIX conformant service (such as NFS or SMB) has control over the part of the file system name space contain- ing the reparse point. The team further claimed that this involvement of a non-POSIX copnformant service removes the traversal from the scope of activities subject to POSIX requirements. After some discussion, the committee accepted this argument. 4.2. Follow-on Consumers The question was raised whether this case should have a dependency on one of the follow-on cases mentioned in 2009/399 [Reparse Points and Referrals Umbrella Case] to ensure that a consumer of this project's deliverables is delived in tandem with this projhect itself. The project team argued that this case provides value in and of itself, and the committee members accepted this argument 4.3. Specification Changes 4.3.1. Release Binding The release binding changed from update to minor. This change was triggered by observing that the change to the signature of VOP_LOOKUP() was not suitable for an update binding. 4.3.2. Stability Level of the Reparse Format The stability level of the in-symlink reparse point format changed from Committed Private to Committed. This change was motivated by noting that there should be a way to know that one has not inadvertenty created a reparse point when creat- ing a symlink. 4.4. Reparse Point Enable/Disable The committee discussed whether adding some mechanism to enable and disable reparse points, perhaps on a file PSARC/2009/387 Copyright 2009 Sun Microsystems - 4 - system-wide basis would help alleviate the compatibility concerns illustrated by the previous point. The consensus among the committee and project team members was that this proposed cure was worse than the disease. 4.5. Other Minor Clarifications There were various other minor points of confusion (some relating to terminology). Where appropriate, the project team updated the specification to incorporate the clarifica- tions. 5. Minority Opinion(s) None. 6. Advisory Information 6.1. Auditing in Follow-on Projects Any follow-on project that provides a service employing reparse points should verify that referral data provided in response to triggering a reparse point is properly audited on the client system that triggered the reparse point. 6.2. Upcoming NFS Referrals Case As part of preparing the materials for the ARC review of the upcoming NFS referrals case, committee members advise the project team to provide a step by step account of what hap- pens when pathname traversal encounters a referral. 7. Appendices 7.1. Appendix A: Technical Changes Required None. 7.2. Appendix B: Technical Changes Advised None. 7.3. Appendix C: Reference Material Unless stated otherwise, path names are relative to the case directory PSARC/2009/387. 1. Onepager. File: 20090709_afshin.salek 2. Project Specification. File: final.materials/reparse_spec.txt PSARC/2009/387 Copyright 2009 Sun Microsystems - 5 - 3. Commmitment Minutes. File: commitment-notes 4. PSARC 20 Questions. File: final.materials/reparse_20q.txt 5. Global namespace: The future of distributed file server management. http://findarticles.com/p/articles/mi_m0BRZ/is_2_23/ai_98709766/ 6. Implementation Guide for Referrals in NFSv4. http://tools.ietf.org/html/draft-ietf-nfsv4-referrals- 00 7. DFS Technical Reference. http://technet.microsoft.com/en- us/library/cc757042%28WS.10%29.aspx PSARC/2009/387 Copyright 2009 Sun Microsystems --Covey_of_Partridges_283_000--