From sacadmin Fri Apr 27 15:08:27 2007 Received: from sac.sfbay.sun.com (localhost [127.0.0.1]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l3RM8RDS006462; Fri, 27 Apr 2007 15:08:27 -0700 (PDT) Received: (from maybee@localhost) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id l3RM8RRl006458; Fri, 27 Apr 2007 15:08:27 -0700 (PDT) Date: Fri, 27 Apr 2007 15:08:27 -0700 (PDT) From: Mark Maybee Message-Id: <200704272208.l3RM8RRl006458@sac.sfbay.sun.com> To: PSARC@sac.sfbay.sun.com Cc: Ienup.Sung@Sun.COM, Timothy.Haley@Sun.COM, cifs-vfs-team@Sun.COM Subject: ZFS case-insensitive support [PSARC/2007/244 Timeout: 05/04/2007] Status: RO Content-Length: 22906 Subject: PSARC FastTrack [05/04/2007]: ZFS case-insensitive support Template Version: @(#)sac_nextcase %I% %G% SMI 1. Introduction 1.1. Project/Component Working Name: ZFS case-insensitive support 1.2. Name of Document Author/Supplier: Author: Tim Haley 1.3 Date of This Document: 27 April, 2007 4. Technical Description This case seeks Minor binding. This fast-track was spun off of the CIFS Service (PSARC 2006/715) case along with: PSARC 2007/218 caller_context_t in all VOPs PSARC 2007/227 VFS Feature Registration and ACL on Create Although each of these changes are part of the bigger picture, they have been broken down into smaller pieces so each gets the attention it deserves. Several of these CIFS-related fast-tracks will describe changes to the signatures of vnode operations (VOPs). In some cases, these fast-tracks will describe multiple changes to the same VOPs. The project team intends to put all of these changes into ON in a single putback. NOTE: The VFSDEF_VERSION number in sys/vfs.h will be bumped from 3 to 4 in order to prevent unbundled file system kernel modules with the old signatures from loading. Once the unbundled file system modules are updated with the new signatures and recompiled, they will also pick up the new VFSDEF_VERSION number and be allowed to load. Case-insensitive behavior on create and lookup is a fundamental requirement for CIFS service. Traditionally, Solaris users access files using case-sensitive create and lookup. Consequently, we need to preserve the traditional behavior yet allow the CIFS server to request case-insensitive behavior. The following are the requirements for changes to the VFS/vnode interface to support mixed-case (case-sensitive and case-insensitive) behavior in Solaris. REQUIREMENTS 1) The CIFS server requires at least one file system type to support case-preserving name creation and both case-sensitive (c-s) and case-insensitive (c-i) behavior (a.k.a., mixed-behavior) on lookup. Solaris will support both the NFS and CIFS protocols. Windows clients typically expect case insensitive behavior while NFS clients typically expect case-sensitive behavior. 2) The VOP interface shall provide a way for the caller to specify an option for case-insensitive behavior. Any given file system volume (i.e. mount-point) may support both case-insensitive or case-sensitive behavior. The caller must be able to specify the behavior on lookup and create operations. The VOP interface must support those requests. 3) An interface is needed to determine if a file system supports requests for case-sensitivity, case-insensitivity, or both. The CIFS server negotiates with its clients on case behavior and needs an interface to query the file system volume on its capabilities. 4) The VOP_LOOKUP interface must return the case-sensitive name in the case of a case-insensitive lookup. Certain CIFS operations (e.g, TRANS2_QUERY_PATH_INFORMATION) require the case-sensitive name that was matched in the lookup operation. 5) When requested, the VOP_READDIR interface must return information for each directory entry to indicate if the entry is a case-insensitive duplicate of another entry. The VOP_LOOKUP interface must also return the same information. File creates done by a non-CIFS application may create file names that are a case-insensitive duplicate of an existing file name. For example, if "foo" exists then a non-CIFS application could create the file "FOO", which is a case-insensitive duplicate. This requirement states both VOP_READDIR and VOP_LOOKUP, when requested, must provide the caller with an indication that a case-insensitive conflict exists. The caller (the CIFS server) may choose to present the name as a "mangled" name so that the user in the case-insensitive environment can distinguish between c-i conflicting file names. NON-REQUIREMENTS 1) Case-insensitive emulation for file systems that do not support native case-insensitive behavior is not a requirement. The initial consumer of case-insensitive behavior is the CIFS client. The CIFS server can handle both case-sensitive and case-insensitive behavior. As long as we have one file system which supports native case-insensitive and case-sensitive behavior (see Req't 1), then there is no requirement for emulation. Note that we may need this feature in the future (and the interfaces we create should not preclude support for emulation). INTERFACE CHANGES NEEDED The following outlines the set of changes to the Solaris file system interface which are needed to support mixed-case (case-sensitive and case-insensitive) behavior in file systems. This behavior is necessary to provide CIFS services in Solaris. CASE BEHAVIOR There are two forms of case behavior: Case-sensitive and case-insensitive. Case sensitive behavior is what Solaris currently supports: When an object is created with a name, the case of the name is preserved. Objects with names that only differ in case (e.g., "foo", "Foo", "FOO") can co-exist in the same directory. Lookups on a particular name can only succeed if the case matches. Case-insensitive behavior, as required by the Solaris CIFS server, follows a particular set of rules: Object Creation: Before a named object is created, the directory must be searched for any c-i matches of that name. If there is a c-i match, then the create operation must fail with EEXIST. Lookup: In mixed-mode (volume supports both c-i and c-s behavior), it's possible for a c-s client to create names that are c-i clashes. For example, it may be possible for "foo", "Foo", and "FOO" to exist in the same directory. In those cases, a c-i lookup of any c-s form of "foo" (e.g., "foo", "FOO", "FoO", "fOo", ad nauseum) should always return the same name. In other words, if a c-i lookup of "foo" returns "foo", then a c-i lookup of "FOO" or "Foo" should also return "foo". It doesn't matter what the algorithm is as long as it is consistent. In addition, the VOP_LOOKUP() routine must return an indication if this file name is a c-i conflict with another file name in the same parent directory. Readdir: Since case-insensitive duplicates may exist in a directory, the VOP_READDIR() interface must return an indication if a directory entry contains a file name that is a c-i conflict with another entry in the same directory. VFS INTERFACE CHANGES An interface is required to determine the case behavior of a particular file system volume. There are three possible modes for behavior: c-s only, c-i only, and mixed-mode (both c-s and c-i requests are supported). The user-level interface is pathconf(2). The pathconf(2) interface will have a new variable added: {CASE_BEHAVIOR}/_PC_CASE_BEHAVIOR. The pathconf(2) system call will return a value which represents the bitwise OR of the following flags which indicate the case-behavior that the file system supports: _CASE_INSENSITIVE This file system supports case-insensitive behavior _CASE_SENSITIVE This file system supports case-sensitive behavior The fs_pathconf() routine (default routine for VOP_PATHCONF()) will be modified to support _PC_CASE_BEHAVIOR. The default behavior for file systems on Solaris is case-sensitive so a call to fs_pathconf() with _PC_CASE_BEHAVIOR set as the cmd would set the _CASE_SENSITIVE bit. Any file system that supports anything but case-sensitive-only behavior is required to use the (new) VFS Feature Registration interfaces to register its case behavior. The following new VFS features will be introduced to support case behavior: #define VFSFT_CASEINSENSITIVE 0x100000002 /* Supports case-insensitive */ #define VFSFT_NOCASESENSITIVE 0x100000004 /* NOT case-sensitive */ #define VFSFT_DIRENTFLAGS 0x100000008 /* Supports dirent flags */ File systems that support case-insensitive behavior must register the VFSFT_CASEINSENSITIVE feature. File systems that do *not* support case-sensitive behavior must register the VFSFT_NOCASESENSITIVE feature. VOP (Vnode Operation) INTERFACE CHANGES The VOP/fop interfaces that deal with names (lookup, create, remove, link, rename, mkdir, rmdir, and symlink) need to pass a new flag, FIGNORECASE, to request c-i behavior as described above. The new flag will be defined in sys/file.h: #define FIGNORECASE 0x80000 /* request case-insensitive lookups */ The readdir fop interface needs to pass a new flag, V_RDDIR_ENTFLAGS, to request c-i conflict information via the flag field of a modified dirent structure. The new flag is defined in vnode.h: /* * Flags for VOP_READDIR */ #define V_RDDIR_ENTFLAGS 0x01 /* request dirent flags */ The file systems that implement c-i behavior are responsible for following the behaviors described below. Under each fop call is a description of the behavior with FIGNORECASE set. Otherwise, the behavior is the same as today. The lookup (VOP_LOOKUP/fop_lookup) and create (VOP_CREATE/fop_create) routines already have a flag(s) field. However, lookup requires additional parameters to return the directory entry flags, direntflagp, and the real (case-preserved) name, realpnp. int fop_lookup( vnode_t *dvp, char *nm, vnode_t **vpp, pathname_t *pnp, int flags, vnode_t *rdir, cred_t *cr, caller_context *ctp, /* See PSARC/2007/218 */ /* NEW */ int *direntflagp, /* dirent-specific flags */ /* NEW */ pathname_t *realpnp); /* real case-sensitive name */ If FIGNORECASE is set and the file system supports c-i, return the vnode in **vpp with a c-i name match, if it exists. Note that in the presence of multiple case versions of a given name (e.g., "foo", "Foo", "FOO"), the case of the name (nm) passed to fop_lookup does not affect which case-sensitive name to return. In other words, if a lookup of "foo" matches the vnode for "Foo", then a lookup of "Foo", "FOO", or "fOo" should match the same vnode. If FIGNORECASE is set and the file system does not support c-i, we return EINVAL. If realpnp is non-NULL, then lookup fills in the pathname structure with the case-preserved name that was looked up. Also, if direntflagp is non-NULL then the flags will be returned in *direntflagp. If a file system supports c-i name matching, it must also support the per-directory-entry flags. int fop_create( vnode_t *dvp, char *name, vattr_t *vap, vcexcl_t excl, int mode, vnode_t **vpp, cred_t *cr, int flag, /* New flag: FIGNORECASE */ caller_context *ctp, /* See PSARC/2007/218 */ vsecattr_t *vsecp) /* See PSARC/2007/227 */ If FIGNORECASE is set and file system supports c-i, then the object is created ONLY IF there is no object that has a c-i name match in the directory. If FIGNORECASE is set, and the file system supports c-i, but an object exists with a c-i name match exists, return EEXIST. If FIGNORECASE is set and the file system does *not* support c-i, then the fop_create returns EINVAL. The following naming routines need to have a "flags" field added. int fop_remove( vnode_t *dvp, char *nm, cred_t *cr, caller_context *ctp, /* See PSARC/2007/218 */ /* NEW */ int flags) /* Takes FIGNORECASE */ If FIGNORECASE is set and file system supports c-i, remove the name of the object with a c-i name match, if it exists. Note that in the presence of multiple case versions of a given name (e.g., "foo", "Foo", "FOO"), the behavior should be the same as fop_lookup(). That is, the c-i matching algorithm should be the same in both fop_lookup, fop_remove, fop_rename, and fop_rmdir. If FIGNORECASE is set and the file system does not support c-i, return EINVAL. int fop_link( vnode_t *tdvp, vnode_t *svp, char *tnm, cred_t *cr, caller_context *ctp, /* See PSARC/2007/218 */ /* NEW */ int flags) /* Takes FIGNORECASE */ If FIGNORECASE is set and file system supports c-i, then the link is created ONLY IF there is no object that has a c-i name match in the target directory. If FIGNORECASE is set and the file system does *not* support c-i, then return EINVAL. int fop_rename( vnode_t *sdvp, char *snm, vnode_t *tdvp, char *tnm, cred_t *cr, caller_context *ctp, /* See PSARC/2007/218 */ /* NEW */ int flags) /* Takes FIGNORECASE */ If FIGNORECASE is set and the file system supports c-i, then the object to be renamed shall match the same name that fop_lookup would match. That is, the c-i matching algorithm should be the same in both fop_lookup, fop_remove, fop_rename, and fop_rmdir. Same holds for the target name, if it exists. If FIGNORECASE is set and either file system does *not* support c-i, then we return EINVAL. int fop_mkdir( vnode_t *dvp, char *dirname, vattr_t *vap, vnode_t **vpp, cred_t *cr, caller_context *ctp, /* See PSARC/2007/218 */ /* NEW */ int flags, /* Takes FIGNORECASE */ vsecattr_t *vsecp) /* See PSARC/2007/227 */ If FIGNORECASE is set and file system supports c-i, then the object is created ONLY IF there is no object that has a c-i name match in the directory. If FIGNORECASE is set and the file system does *not* support c-i, then return EINVAL. int fop_rmdir( vnode_t *dvp, char *nm, vnode_t *cdir, cred_t *cr, caller_context *ctp, /* See PSARC/2007/218 */ /* NEW */ int flags) /* Takes FIGNORECASE */ If FIGNORECASE is set and file system supports c-i, remove the name of the object with a c-i name match, if it exists. Note that in the presence of multiple case versions of a given name (e.g., "foo", "Foo", "FOO"), the behavior should be the same as fop_lookup(). That is, the c-i matching algorithm should be the same in both fop_lookup, fop_remove, fop_rename, and fop_rmdir. If FIGNORECASE is set and the file system does not support c-i, return EINVAL. int fop_readdir( vnode_t *vp, uio_t *uiop, cred_t *cr, int *eofp, caller_context *ctp, /* See PSARC/2007/218 */ /* NEW */ int flags) /* Takes V_RDDIR_ENTFLAGS */ If "flags" has V_RDDIR_ENTFLAGS set and the file system supports per-directory-entry flags, as must be the case if the file system supports c-i, then the uio structure will contain the following structure: typedef struct edirent { ino64_t ed_ino; /* "inode number" of entry */ off64_t ed_off; /* offset of disk directory entry */ uint32_t ed_eflags; /* per-entry flags */ unsigned short ed_reclen; /* length of this record */ char ed_name[1]; /* name of file */ } edirent_t; If "flags" has V_RDDIR_ENTFLAGS set and the file system does *not* support per-directory-entry flags, then fop_readdir() will return EINVAL. If "flags" does not have V_RDDIR_ENTFLAGS set then the uio structure will have the traditional dirent_t format. Note that if the file system supports per-directory-entry flags, then the file system must set VFSFT_DIRENTFLAGS using the VFS Feature Registration interface. int fop_symlink( vnode_t *dvp, char *linkname, vattr_t *vap, char *target, cred_t *cr, caller_context *ctp, /* See PSARC/2007/218 */ /* NEW */ int flags) /* Takes FIGNORECASE */ If FIGNORECASE is set and file system supports c-i, then the symlink is created ONLY IF there is no object that has a c-i name match to the target name in the directory. If FIGNORECASE is set and the file system does *not* support c-i, then EINVAL is returned. CASE-INSENSITIVE BEHAVIOR SUPPORT IN ZFS FILE SYSTEMS We will modify ZFS so that a ZFS file system can support the case-insensitive behaviors required by the Solaris CIFS server. MAN PAGE MODIFICATIONS The following changes apply to the zfs(1M) man page: The following three properties cannot be changed once the file system has been created, and so should be set at file system creation time. If not set in the "zfs create" command these properties will be inherited from the parent dataset. If the parent lacks these properties due to having been created prior to these features being supported, the new file system will have the default values for these properties. utf8only = on | off This property indicates if the file system should reject file names including characters not present in the UTF-8 character code set. If this property is explicitly set to "off", the normalization property (below) must either not be explicitly set or be set to "none". The default value for the "utf8only" property is "off". This property cannot be changed once the file system has been created. normalization = none | formD | formKC This property indicates if the file system should perform a unicode normalization of file names whenever two file names are compared, and which normalization algorithm should be used. File names are always stored unmodified, names are normalized as part of any comparison process. If this property is set to a legal value other than "none", and the "utf8only" property was left unspecified, the "utf8only" property will automatically be set to "on". The default value of the "normalization" property is "none". This property cannot be changed once the file system has been created. casesensitivity = sensitive | insensitive | mixed This property indicates if the file name matching algorithm used by the file system should be case-sensitive, case-insensitive, or allow a combination of both styles of matching. The default value for the "casesensitivity" property is "sensitive". Traditionally, UNIX and POSIX file systems have case-sensitive file names. The "mixed" value for the "casesensitivity" property indicates the file system can support requests for both case-sensitive and case-insensitive matching behavior. Currently case-insensitive matching behavior on a file system that supports mixed behavior is limited to kernel modules. Accesses from a user process have no means to directly request case-insensitive behavior on these file systems. A user process can indirectly achieve case-insensitive access, though, through an intermediate kernel module explicitly requesting case-insensitive behavior, such as a CIFS server. When a case-insensitive matching request is made of a "mixed" sensitivity file system, the behavior is generally the same as would be expected of a purely case-insensitive file system. The difference is that a "mixed" sensitivity file system may contain directories with multiple names that are unique from a case-sensitive perspective, but not unique from the case-insensitive perspective. For example, a directory might contain files "foo", "Foo", and "FOO". If there is a request to case-insensitively match any of the possible forms of "foo", (for example "foo", "FOO", "FoO", "fOo", et cetera) one of the three existing files will be chosen as the match by the matching algorithm. Exactly which file the algorithm chooses as match is not guaranteed, but what is guaranteed is that the same file will be chosen as match for any of the forms of "foo". The file chosen as a case-insensitive match for "foo", "FOO", "foO", "Foo", et. cetera will always be the same, so long as the directory remains unchanged. Regardless of the "casesensitivity" property setting, the file system will preserve the case of the name specified to create a file. The "casesensitivity" property cannot be changed once the file system has been created. The "utf8only", "normalization", and "casesensitivity" properties will also become new permissions that can be assigned to non-privileged users via the ZFS delegated administration model PSARC/2006/465. EXPORTED INTERFACES |Proposed |Specified | |Stability |in what | Interface Name |Classification |Document? | Comments =============================================================================== VFSFT_CASEINSENSITIVE |Consolidation |This | VFS Feature #define VFSFT_NOCASESENSITIVE |Private |Document | VFSFT_DIRENTFLAGS | | | ------------------------+---------------+---------------+---------------------- FIGNORECASE |Contracted |This | file.h #define |Consolidation |Document | |Private | | ------------------------+---------------+---------------+---------------------- V_RDDIR_ENTFLAGS |Contracted |This | vnode.h #define |Consolidation |Document | |Private | | ------------------------+---------------+---------------+---------------------- edirent_t |Contracted |This | Structure returned |Consolidation |Document | within uio buf of |Private | | readdir() caller | | | requesting | V_RDDIR_ENTFLAGS ------------------------+---------------+---------------+---------------------- VOP_LOOKUP, fop_lookup |Contracted |This |New output parameters |Consolidation |Document |int *direntflagp |Private | |pathname_t *realpnp | | | VOP_CREATE, fop_create, | | |New input paramater VOP_REMOVE, fop_remove, | | |int flag VOP_LINK, fop_link, | | | VOP_RENAME, fop_rename, | | | VOP_MKDIR, fop_mkdir, | | | VOP_RMDIR, fop_rmdir, | | | VOP_READDIR,fop_readdir,| | | VOP_SYMLINK,fop_symlink | | | | | | ------------------------+---------------+---------------+---------------------- {CASE_BEHAVIOR}, |Stable |This | pathconf(2) variable _PC_CASE_BEHAVIOR | |Document | name and value | | | _CASE_INSENSITIVE | | | Bit values for _CASE_SENSITIVE | | | _PC_CASE_BEHAVIOR ------------------------+---------------+---------------+---------------------- utf8only |Evolving |This | zfs(1M) file system normalization | |Document | properties casesensitivity | | | ------------------------+---------------+---------------+---------------------- [ 1.5 Last update: 04/27/07 14:37:53 ] 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: FastTrack From sacadmin Fri Apr 27 17:34:56 2007 Received: from ivrel.sfbay.sun.com (ivrel [129.146.74.76]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l3S0Yu58010236; Fri, 27 Apr 2007 17:34:56 -0700 (PDT) Received: from ivrel (ivrel [129.146.74.76]) by ivrel.sfbay.sun.com (8.13.8+Sun/8.13.8) with SMTP id l3S0VdQT021940; Fri, 27 Apr 2007 17:31:39 -0700 (PDT) Message-Id: <200704280031.l3S0VdQT021940@ivrel.sfbay.sun.com> Date: Fri, 27 Apr 2007 17:31:39 -0700 (PDT) From: Glenn Skinner Reply-To: Glenn Skinner Subject: Re: 2007/244 [ZFS case-insensitive support] To: PSARC@sac.sfbay.sun.com, maybee@sac.sfbay.sun.com Cc: Ienup.Sung@sun.com, Timothy.Haley@sun.com, cifs-vfs-team@sun.com MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: Wi3neLMi5C8424ABbPej2w== X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.6_36 SunOS 5.11 sun4u sparc Status: RO Content-Length: 1148 This case's IAM file marks it as being open, but mail pertaining to it has gone to the psarc alias rather than to psarc-ext. Please bring psarc-ext into the loop. (I'll leave it to you to make sure that psarc-ext sees the comments below.) Moving to the substance of the proposal, in its description of modifications to fop_lookup(), the proposal states: Also, if direntflagp is non-NULL then the flags will be returned in *direntflagp. If a file system supports c-i name matching, it must also support the per-directory-entry flags. What happens if a file system fails to obey this constraint? What error(s) is/are returned in response to what call(s)? The proposal's specification of per-directory-entry flags seems to be incomplete. It says how they're fetched, but beyond the statement The readdir fop interface needs to pass a new flag, V_RDDIR_ENTFLAGS, to request c-i conflict information via the flag field of a modified dirent structure. it doesn't say how the flags are set (including which bits mean what). So what edirent flags does this proposal define, and under what circumstances are they set? -- Glenn From Mark.Maybee@Sun.COM Sat Apr 28 06:42:51 2007 Received: from sfbaymail2sca.sfbay.sun.com (sfbaymail2sca.SFBay.Sun.COM [129.145.155.42]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l3SDgp8c011572 for ; Sat, 28 Apr 2007 06:42:51 -0700 (PDT) Received: from brmea-mail-4.sun.com (brmea-mail-4.Sun.COM [192.18.98.36]) by sfbaymail2sca.sfbay.sun.com (8.13.6+Sun/8.12.10/ENSMAIL,v2.2) with ESMTP id l3SDfxRe019673 for ; Sat, 28 Apr 2007 06:41:59 -0700 (PDT) Received: from fe-amer-10.sun.com ([192.18.108.184]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l3SDfw9A016998 for ; Sat, 28 Apr 2007 13:41:58 GMT Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JH700201N5OSQ00@mail-amer.sun.com> (original mail from Mark.Maybee@Sun.COM) for psarc-ext@sac.sfbay.sun.com; Sat, 28 Apr 2007 07:41:58 -0600 (MDT) Received: from [192.9.61.121] by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JH700JWKNDWO8TD@mail-amer.sun.com>; Sat, 28 Apr 2007 07:41:58 -0600 (MDT) Date: Sat, 28 Apr 2007 07:42:21 -0600 From: Mark Maybee Subject: ZFS case-insensitive support [PSARC/2007/244 Timeout: 05/04/2007] Sender: Mark.Maybee@Sun.COM To: psarc-ext@sac.sfbay.sun.com Cc: Ienup.Sung@Sun.COM, Timothy.Haley@Sun.COM, cifs-vfs-team@Sun.COM Message-id: <46334F3D.5060204@Sun.COM> MIME-version: 1.0 Content-type: multipart/mixed; boundary="Boundary_(ID_I1tMSc5PDO4za+uiLApRLg)" User-Agent: Thunderbird 1.5.0.8 (X11/20061204) Status: RO Content-Length: 23325 This is a multi-part message in MIME format. --Boundary_(ID_I1tMSc5PDO4za+uiLApRLg) Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT re-sending this to bring psarc-ext into the loop. --Boundary_(ID_I1tMSc5PDO4za+uiLApRLg) Content-type: text/plain; name=xx Content-transfer-encoding: 7BIT Content-disposition: inline; filename=xx Subject: PSARC FastTrack [05/04/2007]: ZFS case-insensitive support Template Version: @(#)sac_nextcase %I% %G% SMI 1. Introduction 1.1. Project/Component Working Name: ZFS case-insensitive support 1.2. Name of Document Author/Supplier: Author: Tim Haley 1.3 Date of This Document: 27 April, 2007 4. Technical Description This case seeks Minor binding. This fast-track was spun off of the CIFS Service (PSARC 2006/715) case along with: PSARC 2007/218 caller_context_t in all VOPs PSARC 2007/227 VFS Feature Registration and ACL on Create Although each of these changes are part of the bigger picture, they have been broken down into smaller pieces so each gets the attention it deserves. Several of these CIFS-related fast-tracks will describe changes to the signatures of vnode operations (VOPs). In some cases, these fast-tracks will describe multiple changes to the same VOPs. The project team intends to put all of these changes into ON in a single putback. NOTE: The VFSDEF_VERSION number in sys/vfs.h will be bumped from 3 to 4 in order to prevent unbundled file system kernel modules with the old signatures from loading. Once the unbundled file system modules are updated with the new signatures and recompiled, they will also pick up the new VFSDEF_VERSION number and be allowed to load. Case-insensitive behavior on create and lookup is a fundamental requirement for CIFS service. Traditionally, Solaris users access files using case-sensitive create and lookup. Consequently, we need to preserve the traditional behavior yet allow the CIFS server to request case-insensitive behavior. The following are the requirements for changes to the VFS/vnode interface to support mixed-case (case-sensitive and case-insensitive) behavior in Solaris. REQUIREMENTS 1) The CIFS server requires at least one file system type to support case-preserving name creation and both case-sensitive (c-s) and case-insensitive (c-i) behavior (a.k.a., mixed-behavior) on lookup. Solaris will support both the NFS and CIFS protocols. Windows clients typically expect case insensitive behavior while NFS clients typically expect case-sensitive behavior. 2) The VOP interface shall provide a way for the caller to specify an option for case-insensitive behavior. Any given file system volume (i.e. mount-point) may support both case-insensitive or case-sensitive behavior. The caller must be able to specify the behavior on lookup and create operations. The VOP interface must support those requests. 3) An interface is needed to determine if a file system supports requests for case-sensitivity, case-insensitivity, or both. The CIFS server negotiates with its clients on case behavior and needs an interface to query the file system volume on its capabilities. 4) The VOP_LOOKUP interface must return the case-sensitive name in the case of a case-insensitive lookup. Certain CIFS operations (e.g, TRANS2_QUERY_PATH_INFORMATION) require the case-sensitive name that was matched in the lookup operation. 5) When requested, the VOP_READDIR interface must return information for each directory entry to indicate if the entry is a case-insensitive duplicate of another entry. The VOP_LOOKUP interface must also return the same information. File creates done by a non-CIFS application may create file names that are a case-insensitive duplicate of an existing file name. For example, if "foo" exists then a non-CIFS application could create the file "FOO", which is a case-insensitive duplicate. This requirement states both VOP_READDIR and VOP_LOOKUP, when requested, must provide the caller with an indication that a case-insensitive conflict exists. The caller (the CIFS server) may choose to present the name as a "mangled" name so that the user in the case-insensitive environment can distinguish between c-i conflicting file names. NON-REQUIREMENTS 1) Case-insensitive emulation for file systems that do not support native case-insensitive behavior is not a requirement. The initial consumer of case-insensitive behavior is the CIFS client. The CIFS server can handle both case-sensitive and case-insensitive behavior. As long as we have one file system which supports native case-insensitive and case-sensitive behavior (see Req't 1), then there is no requirement for emulation. Note that we may need this feature in the future (and the interfaces we create should not preclude support for emulation). INTERFACE CHANGES NEEDED The following outlines the set of changes to the Solaris file system interface which are needed to support mixed-case (case-sensitive and case-insensitive) behavior in file systems. This behavior is necessary to provide CIFS services in Solaris. CASE BEHAVIOR There are two forms of case behavior: Case-sensitive and case-insensitive. Case sensitive behavior is what Solaris currently supports: When an object is created with a name, the case of the name is preserved. Objects with names that only differ in case (e.g., "foo", "Foo", "FOO") can co-exist in the same directory. Lookups on a particular name can only succeed if the case matches. Case-insensitive behavior, as required by the Solaris CIFS server, follows a particular set of rules: Object Creation: Before a named object is created, the directory must be searched for any c-i matches of that name. If there is a c-i match, then the create operation must fail with EEXIST. Lookup: In mixed-mode (volume supports both c-i and c-s behavior), it's possible for a c-s client to create names that are c-i clashes. For example, it may be possible for "foo", "Foo", and "FOO" to exist in the same directory. In those cases, a c-i lookup of any c-s form of "foo" (e.g., "foo", "FOO", "FoO", "fOo", ad nauseum) should always return the same name. In other words, if a c-i lookup of "foo" returns "foo", then a c-i lookup of "FOO" or "Foo" should also return "foo". It doesn't matter what the algorithm is as long as it is consistent. In addition, the VOP_LOOKUP() routine must return an indication if this file name is a c-i conflict with another file name in the same parent directory. Readdir: Since case-insensitive duplicates may exist in a directory, the VOP_READDIR() interface must return an indication if a directory entry contains a file name that is a c-i conflict with another entry in the same directory. VFS INTERFACE CHANGES An interface is required to determine the case behavior of a particular file system volume. There are three possible modes for behavior: c-s only, c-i only, and mixed-mode (both c-s and c-i requests are supported). The user-level interface is pathconf(2). The pathconf(2) interface will have a new variable added: {CASE_BEHAVIOR}/_PC_CASE_BEHAVIOR. The pathconf(2) system call will return a value which represents the bitwise OR of the following flags which indicate the case-behavior that the file system supports: _CASE_INSENSITIVE This file system supports case-insensitive behavior _CASE_SENSITIVE This file system supports case-sensitive behavior The fs_pathconf() routine (default routine for VOP_PATHCONF()) will be modified to support _PC_CASE_BEHAVIOR. The default behavior for file systems on Solaris is case-sensitive so a call to fs_pathconf() with _PC_CASE_BEHAVIOR set as the cmd would set the _CASE_SENSITIVE bit. Any file system that supports anything but case-sensitive-only behavior is required to use the (new) VFS Feature Registration interfaces to register its case behavior. The following new VFS features will be introduced to support case behavior: #define VFSFT_CASEINSENSITIVE 0x100000002 /* Supports case-insensitive */ #define VFSFT_NOCASESENSITIVE 0x100000004 /* NOT case-sensitive */ #define VFSFT_DIRENTFLAGS 0x100000008 /* Supports dirent flags */ File systems that support case-insensitive behavior must register the VFSFT_CASEINSENSITIVE feature. File systems that do *not* support case-sensitive behavior must register the VFSFT_NOCASESENSITIVE feature. VOP (Vnode Operation) INTERFACE CHANGES The VOP/fop interfaces that deal with names (lookup, create, remove, link, rename, mkdir, rmdir, and symlink) need to pass a new flag, FIGNORECASE, to request c-i behavior as described above. The new flag will be defined in sys/file.h: #define FIGNORECASE 0x80000 /* request case-insensitive lookups */ The readdir fop interface needs to pass a new flag, V_RDDIR_ENTFLAGS, to request c-i conflict information via the flag field of a modified dirent structure. The new flag is defined in vnode.h: /* * Flags for VOP_READDIR */ #define V_RDDIR_ENTFLAGS 0x01 /* request dirent flags */ The file systems that implement c-i behavior are responsible for following the behaviors described below. Under each fop call is a description of the behavior with FIGNORECASE set. Otherwise, the behavior is the same as today. The lookup (VOP_LOOKUP/fop_lookup) and create (VOP_CREATE/fop_create) routines already have a flag(s) field. However, lookup requires additional parameters to return the directory entry flags, direntflagp, and the real (case-preserved) name, realpnp. int fop_lookup( vnode_t *dvp, char *nm, vnode_t **vpp, pathname_t *pnp, int flags, vnode_t *rdir, cred_t *cr, caller_context *ctp, /* See PSARC/2007/218 */ /* NEW */ int *direntflagp, /* dirent-specific flags */ /* NEW */ pathname_t *realpnp); /* real case-sensitive name */ If FIGNORECASE is set and the file system supports c-i, return the vnode in **vpp with a c-i name match, if it exists. Note that in the presence of multiple case versions of a given name (e.g., "foo", "Foo", "FOO"), the case of the name (nm) passed to fop_lookup does not affect which case-sensitive name to return. In other words, if a lookup of "foo" matches the vnode for "Foo", then a lookup of "Foo", "FOO", or "fOo" should match the same vnode. If FIGNORECASE is set and the file system does not support c-i, we return EINVAL. If realpnp is non-NULL, then lookup fills in the pathname structure with the case-preserved name that was looked up. Also, if direntflagp is non-NULL then the flags will be returned in *direntflagp. If a file system supports c-i name matching, it must also support the per-directory-entry flags. int fop_create( vnode_t *dvp, char *name, vattr_t *vap, vcexcl_t excl, int mode, vnode_t **vpp, cred_t *cr, int flag, /* New flag: FIGNORECASE */ caller_context *ctp, /* See PSARC/2007/218 */ vsecattr_t *vsecp) /* See PSARC/2007/227 */ If FIGNORECASE is set and file system supports c-i, then the object is created ONLY IF there is no object that has a c-i name match in the directory. If FIGNORECASE is set, and the file system supports c-i, but an object exists with a c-i name match exists, return EEXIST. If FIGNORECASE is set and the file system does *not* support c-i, then the fop_create returns EINVAL. The following naming routines need to have a "flags" field added. int fop_remove( vnode_t *dvp, char *nm, cred_t *cr, caller_context *ctp, /* See PSARC/2007/218 */ /* NEW */ int flags) /* Takes FIGNORECASE */ If FIGNORECASE is set and file system supports c-i, remove the name of the object with a c-i name match, if it exists. Note that in the presence of multiple case versions of a given name (e.g., "foo", "Foo", "FOO"), the behavior should be the same as fop_lookup(). That is, the c-i matching algorithm should be the same in both fop_lookup, fop_remove, fop_rename, and fop_rmdir. If FIGNORECASE is set and the file system does not support c-i, return EINVAL. int fop_link( vnode_t *tdvp, vnode_t *svp, char *tnm, cred_t *cr, caller_context *ctp, /* See PSARC/2007/218 */ /* NEW */ int flags) /* Takes FIGNORECASE */ If FIGNORECASE is set and file system supports c-i, then the link is created ONLY IF there is no object that has a c-i name match in the target directory. If FIGNORECASE is set and the file system does *not* support c-i, then return EINVAL. int fop_rename( vnode_t *sdvp, char *snm, vnode_t *tdvp, char *tnm, cred_t *cr, caller_context *ctp, /* See PSARC/2007/218 */ /* NEW */ int flags) /* Takes FIGNORECASE */ If FIGNORECASE is set and the file system supports c-i, then the object to be renamed shall match the same name that fop_lookup would match. That is, the c-i matching algorithm should be the same in both fop_lookup, fop_remove, fop_rename, and fop_rmdir. Same holds for the target name, if it exists. If FIGNORECASE is set and either file system does *not* support c-i, then we return EINVAL. int fop_mkdir( vnode_t *dvp, char *dirname, vattr_t *vap, vnode_t **vpp, cred_t *cr, caller_context *ctp, /* See PSARC/2007/218 */ /* NEW */ int flags, /* Takes FIGNORECASE */ vsecattr_t *vsecp) /* See PSARC/2007/227 */ If FIGNORECASE is set and file system supports c-i, then the object is created ONLY IF there is no object that has a c-i name match in the directory. If FIGNORECASE is set and the file system does *not* support c-i, then return EINVAL. int fop_rmdir( vnode_t *dvp, char *nm, vnode_t *cdir, cred_t *cr, caller_context *ctp, /* See PSARC/2007/218 */ /* NEW */ int flags) /* Takes FIGNORECASE */ If FIGNORECASE is set and file system supports c-i, remove the name of the object with a c-i name match, if it exists. Note that in the presence of multiple case versions of a given name (e.g., "foo", "Foo", "FOO"), the behavior should be the same as fop_lookup(). That is, the c-i matching algorithm should be the same in both fop_lookup, fop_remove, fop_rename, and fop_rmdir. If FIGNORECASE is set and the file system does not support c-i, return EINVAL. int fop_readdir( vnode_t *vp, uio_t *uiop, cred_t *cr, int *eofp, caller_context *ctp, /* See PSARC/2007/218 */ /* NEW */ int flags) /* Takes V_RDDIR_ENTFLAGS */ If "flags" has V_RDDIR_ENTFLAGS set and the file system supports per-directory-entry flags, as must be the case if the file system supports c-i, then the uio structure will contain the following structure: typedef struct edirent { ino64_t ed_ino; /* "inode number" of entry */ off64_t ed_off; /* offset of disk directory entry */ uint32_t ed_eflags; /* per-entry flags */ unsigned short ed_reclen; /* length of this record */ char ed_name[1]; /* name of file */ } edirent_t; If "flags" has V_RDDIR_ENTFLAGS set and the file system does *not* support per-directory-entry flags, then fop_readdir() will return EINVAL. If "flags" does not have V_RDDIR_ENTFLAGS set then the uio structure will have the traditional dirent_t format. Note that if the file system supports per-directory-entry flags, then the file system must set VFSFT_DIRENTFLAGS using the VFS Feature Registration interface. int fop_symlink( vnode_t *dvp, char *linkname, vattr_t *vap, char *target, cred_t *cr, caller_context *ctp, /* See PSARC/2007/218 */ /* NEW */ int flags) /* Takes FIGNORECASE */ If FIGNORECASE is set and file system supports c-i, then the symlink is created ONLY IF there is no object that has a c-i name match to the target name in the directory. If FIGNORECASE is set and the file system does *not* support c-i, then EINVAL is returned. CASE-INSENSITIVE BEHAVIOR SUPPORT IN ZFS FILE SYSTEMS We will modify ZFS so that a ZFS file system can support the case-insensitive behaviors required by the Solaris CIFS server. MAN PAGE MODIFICATIONS The following changes apply to the zfs(1M) man page: The following three properties cannot be changed once the file system has been created, and so should be set at file system creation time. If not set in the "zfs create" command these properties will be inherited from the parent dataset. If the parent lacks these properties due to having been created prior to these features being supported, the new file system will have the default values for these properties. utf8only = on | off This property indicates if the file system should reject file names including characters not present in the UTF-8 character code set. If this property is explicitly set to "off", the normalization property (below) must either not be explicitly set or be set to "none". The default value for the "utf8only" property is "off". This property cannot be changed once the file system has been created. normalization = none | formD | formKC This property indicates if the file system should perform a unicode normalization of file names whenever two file names are compared, and which normalization algorithm should be used. File names are always stored unmodified, names are normalized as part of any comparison process. If this property is set to a legal value other than "none", and the "utf8only" property was left unspecified, the "utf8only" property will automatically be set to "on". The default value of the "normalization" property is "none". This property cannot be changed once the file system has been created. casesensitivity = sensitive | insensitive | mixed This property indicates if the file name matching algorithm used by the file system should be case-sensitive, case-insensitive, or allow a combination of both styles of matching. The default value for the "casesensitivity" property is "sensitive". Traditionally, UNIX and POSIX file systems have case-sensitive file names. The "mixed" value for the "casesensitivity" property indicates the file system can support requests for both case-sensitive and case-insensitive matching behavior. Currently case-insensitive matching behavior on a file system that supports mixed behavior is limited to kernel modules. Accesses from a user process have no means to directly request case-insensitive behavior on these file systems. A user process can indirectly achieve case-insensitive access, though, through an intermediate kernel module explicitly requesting case-insensitive behavior, such as a CIFS server. When a case-insensitive matching request is made of a "mixed" sensitivity file system, the behavior is generally the same as would be expected of a purely case-insensitive file system. The difference is that a "mixed" sensitivity file system may contain directories with multiple names that are unique from a case-sensitive perspective, but not unique from the case-insensitive perspective. For example, a directory might contain files "foo", "Foo", and "FOO". If there is a request to case-insensitively match any of the possible forms of "foo", (for example "foo", "FOO", "FoO", "fOo", et cetera) one of the three existing files will be chosen as the match by the matching algorithm. Exactly which file the algorithm chooses as match is not guaranteed, but what is guaranteed is that the same file will be chosen as match for any of the forms of "foo". The file chosen as a case-insensitive match for "foo", "FOO", "foO", "Foo", et. cetera will always be the same, so long as the directory remains unchanged. Regardless of the "casesensitivity" property setting, the file system will preserve the case of the name specified to create a file. The "casesensitivity" property cannot be changed once the file system has been created. The "utf8only", "normalization", and "casesensitivity" properties will also become new permissions that can be assigned to non-privileged users via the ZFS delegated administration model PSARC/2006/465. EXPORTED INTERFACES |Proposed |Specified | |Stability |in what | Interface Name |Classification |Document? | Comments =============================================================================== VFSFT_CASEINSENSITIVE |Consolidation |This | VFS Feature #define VFSFT_NOCASESENSITIVE |Private |Document | VFSFT_DIRENTFLAGS | | | ------------------------+---------------+---------------+---------------------- FIGNORECASE |Contracted |This | file.h #define |Consolidation |Document | |Private | | ------------------------+---------------+---------------+---------------------- V_RDDIR_ENTFLAGS |Contracted |This | vnode.h #define |Consolidation |Document | |Private | | ------------------------+---------------+---------------+---------------------- edirent_t |Contracted |This | Structure returned |Consolidation |Document | within uio buf of |Private | | readdir() caller | | | requesting | V_RDDIR_ENTFLAGS ------------------------+---------------+---------------+---------------------- VOP_LOOKUP, fop_lookup |Contracted |This |New output parameters |Consolidation |Document |int *direntflagp |Private | |pathname_t *realpnp | | | VOP_CREATE, fop_create, | | |New input paramater VOP_REMOVE, fop_remove, | | |int flag VOP_LINK, fop_link, | | | VOP_RENAME, fop_rename, | | | VOP_MKDIR, fop_mkdir, | | | VOP_RMDIR, fop_rmdir, | | | VOP_READDIR,fop_readdir,| | | VOP_SYMLINK,fop_symlink | | | | | | ------------------------+---------------+---------------+---------------------- {CASE_BEHAVIOR}, |Stable |This | pathconf(2) variable _PC_CASE_BEHAVIOR | |Document | name and value | | | _CASE_INSENSITIVE | | | Bit values for _CASE_SENSITIVE | | | _PC_CASE_BEHAVIOR ------------------------+---------------+---------------+---------------------- utf8only |Evolving |This | zfs(1M) file system normalization | |Document | properties casesensitivity | | | ------------------------+---------------+---------------+---------------------- [ 1.5 Last update: 04/27/07 14:37:53 ] 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: FastTrack --Boundary_(ID_I1tMSc5PDO4za+uiLApRLg)-- From Mark.Maybee@Sun.COM Sat Apr 28 06:45:03 2007 Received: from sfbaymail1sca.SFBay.Sun.COM (sfbaymail1sca.SFBay.Sun.COM [129.145.154.35]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l3SDj3i3011636 for ; Sat, 28 Apr 2007 06:45:03 -0700 (PDT) Received: from brmea-mail-4.sun.com (brmea-mail-4.Sun.COM [192.18.98.36]) by sfbaymail1sca.SFBay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l3SDiAa5001465 for ; Sat, 28 Apr 2007 06:44:11 -0700 (PDT) Received: from fe-amer-01.sun.com ([192.18.108.175]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l3SDiA4q017365 for ; Sat, 28 Apr 2007 13:44:10 GMT Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JH700M01L1IKM00@mail-amer.sun.com> (original mail from Mark.Maybee@Sun.COM) for psarc-ext@sac.sfbay.sun.com; Sat, 28 Apr 2007 07:44:10 -0600 (MDT) Received: from [192.9.61.121] by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JH7004G6NHL16H1@mail-amer.sun.com>; Sat, 28 Apr 2007 07:44:10 -0600 (MDT) Date: Sat, 28 Apr 2007 07:44:34 -0600 From: Mark Maybee Subject: [Fwd: Re: 2007/244 [ZFS case-insensitive support]] Sender: Mark.Maybee@Sun.COM To: psarc-ext@sac.sfbay.sun.com Cc: Ienup.Sung@Sun.COM, Timothy.Haley@Sun.COM, cifs-vfs-team@Sun.COM Message-id: <46334FC2.9010308@Sun.COM> MIME-version: 1.0 Content-type: multipart/mixed; boundary="Boundary_(ID_26lX3PfpixHu+iLO2h/q1g)" User-Agent: Thunderbird 1.5.0.8 (X11/20061204) Status: RO Content-Length: 4085 This is a multi-part message in MIME format. --Boundary_(ID_26lX3PfpixHu+iLO2h/q1g) Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT --Boundary_(ID_26lX3PfpixHu+iLO2h/q1g) Content-type: message/rfc822; name="Re: 2007/244 [ZFS case-insensitive support]" Return-path: Received: from fe-amer-09.sun.com ([192.18.108.183]) by bedge3-mail1.central.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTP id <0JH600C0CMWS16C0@bedge3-mail1.central.sun.com> for maybee@bedge3-mail1.central.Sun.COM; Fri, 27 Apr 2007 18:34:04 -0600 (MDT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JH600E01MRGO200@mail-amer.sun.com> for maybee@bedge3-mail1.central.Sun.COM (ORCPT maybee@bedge3-mail1.central.Sun.COM); Fri, 27 Apr 2007 18:34:04 -0600 (MDT) Received: from phys-bedge3-2 ([129.147.156.114]) by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTP id <0JH6002K5MWRI0I3@mail-amer.sun.com> for maybee@bedge3-mail1.central.Sun.COM (ORCPT maybee@bedge3-mail1.central.Sun.COM); Fri, 27 Apr 2007 18:34:04 -0600 (MDT) Received: from centralmail3brm.Central.Sun.COM ([129.147.62.199]) by bedge3-mail1.central.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTP id <0JH600CZSMWR1AB0@bedge3-mail1.central.sun.com> for maybee@bedge3-mail1.central.Sun.COM; Fri, 27 Apr 2007 18:34:03 -0600 (MDT) Received: from sac.sfbay.sun.com (sac.SFBay.Sun.COM [129.146.226.132]) by centralmail3brm.Central.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l3S0Y3G2006555 for ; Fri, 27 Apr 2007 18:34:03 -0600 (MDT) Received: from ivrel.sfbay.sun.com (ivrel [129.146.74.76]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l3S0Yu58010236; Fri, 27 Apr 2007 17:34:56 -0700 (PDT) Received: from ivrel (ivrel [129.146.74.76]) by ivrel.sfbay.sun.com (8.13.8+Sun/8.13.8) with SMTP id l3S0VdQT021940; Fri, 27 Apr 2007 17:31:39 -0700 (PDT) Date: Fri, 27 Apr 2007 17:31:39 -0700 (PDT) From: Glenn Skinner Subject: Re: 2007/244 [ZFS case-insensitive support] To: PSARC@sac.sfbay.sun.com, maybee@sac.sfbay.sun.com Cc: Ienup.Sung@Sun.COM, Timothy.Haley@Sun.COM, cifs-vfs-team@Sun.COM Reply-to: Glenn Skinner Message-id: <200704280031.l3S0VdQT021940@ivrel.sfbay.sun.com> X-Envelope-from: Mark.Maybee@Sun.COM X-Envelope-to: psarc-ext@sac.sfbay.sun.com, cifs-vfs-team <@smarthost.sun.com:cifs-vfs-team@Sun.COM> MIME-version: 1.0 X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.6_36 SunOS 5.11 sun4u sparc Content-type: TEXT/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-MD5: Wi3neLMi5C8424ABbPej2w== Original-recipient: rfc822;maybee@bedge3-mail1.central.Sun.COM This case's IAM file marks it as being open, but mail pertaining to it has gone to the psarc alias rather than to psarc-ext. Please bring psarc-ext into the loop. (I'll leave it to you to make sure that psarc-ext sees the comments below.) Moving to the substance of the proposal, in its description of modifications to fop_lookup(), the proposal states: Also, if direntflagp is non-NULL then the flags will be returned in *direntflagp. If a file system supports c-i name matching, it must also support the per-directory-entry flags. What happens if a file system fails to obey this constraint? What error(s) is/are returned in response to what call(s)? The proposal's specification of per-directory-entry flags seems to be incomplete. It says how they're fetched, but beyond the statement The readdir fop interface needs to pass a new flag, V_RDDIR_ENTFLAGS, to request c-i conflict information via the flag field of a modified dirent structure. it doesn't say how the flags are set (including which bits mean what). So what edirent flags does this proposal define, and under what circumstances are they set? -- Glenn --Boundary_(ID_26lX3PfpixHu+iLO2h/q1g)-- From schilling@fokus.fraunhofer.de Sat Apr 28 06:52:40 2007 Received: from sfbaymail1sca.SFBay.Sun.COM (sfbaymail1sca.SFBay.Sun.COM [129.145.154.35]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l3SDqehx011821 for ; Sat, 28 Apr 2007 06:52:40 -0700 (PDT) Received: from brmea-mail-1.sun.com (brmea-mail-1.Sun.COM [192.18.98.31]) by sfbaymail1sca.SFBay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l3SDpmNC002949 for ; Sat, 28 Apr 2007 06:51:48 -0700 (PDT) Received: from relay1.sun.com (relay1.sun.com [150.143.103.14] (may be forged)) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l3SBJsot010178 for ; Sat, 28 Apr 2007 13:51:48 GMT Received: from mms04es.sun.com ([150.143.104.74] [150.143.104.74]) by relay1.sun.com with ESMTP id BT-MMP-316320 for psarc-ext@sac.sfbay.sun.com; Sat, 28 Apr 2007 13:51:48 Z Received: from relay1.sun.com (relay1.sun.com [150.143.103.14]) by mms04es.sun.com with ESMTP id BT-MMP-603015 for psarc-ext@sac.sfbay.sun.com; Sat, 28 Apr 2007 13:51:47 Z Received: from mailhub.fokus.fraunhofer.de ([193.174.154.14] [193.174.154.14]) by relay1.sun.com with ESMTP id BT-MMP-2702621 for psarc-ext@sac.sfbay.sun.com; Sat, 28 Apr 2007 13:51:47 Z Received: from burner.fokus.fraunhofer.de (burner [10.147.65.166]) by mailhub.fokus.fraunhofer.de (8.11.6p2/8.11.6) with ESMTP id l3SDpjf29779; Sat, 28 Apr 2007 15:51:45 +0200 (MEST) Received: (from jes@localhost) by burner.fokus.fraunhofer.de (8.12.9+Sun/8.12.9/Submit) id l3SDpOXF009939; Sat, 28 Apr 2007 15:51:24 +0200 (CEST) Date: Sat, 28 Apr 2007 15:51:24 +0200 From: Joerg.Schilling@fokus.fraunhofer.de (Joerg Schilling) To: psarc-ext@sac.sfbay.sun.com, Mark.Maybee@sun.com Cc: Timothy.Haley@sun.com, Ienup.Sung@sun.com, cifs-vfs-team@sun.com Subject: Re: ZFS case-insensitive support [PSARC/2007/244 Timeout: 05/04/2007] Message-Id: <4633515c.U66IQ9uumtfOwDQW%Joerg.Schilling@fokus.fraunhofer.de> References: <46334F3D.5060204@Sun.COM> In-Reply-To: <46334F3D.5060204@Sun.COM> User-Agent: nail 11.22 3/20/05 MIME-Version: 1.0 Content-Type: text/plain; charset=ISO8859-1 Content-Transfer-Encoding: 8bit Sender: schilling@fokus.fraunhofer.de Status: RO Content-Length: 920 Mark Maybee wrote: > re-sending this to bring psarc-ext into the loop. > Could you please explain whyt you understand by "case-insensitive"? Dou you mean case-insensitive like it is implemented by Services For UNIX on Win-NT? SFU requires the case to be correct for all syscalls that use file names as an argument but still does not allow to have a file called "A" _and_ a file called "a" in the same directory. So case insensitiviy is is affecting two different areas, the way you "open" files by using named arguments and whether the fs allows different case in a directory at the same time. Jörg -- EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin js@cs.tu-berlin.de (uni) schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/old/private/ ftp://ftp.berlios.de/pub/schily From alan.coopersmith@sun.com Sat Apr 28 08:53:37 2007 Received: from sfbaymail1sca.SFBay.Sun.COM (sfbaymail1sca.SFBay.Sun.COM [129.145.154.35]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l3SFrb30015816 for ; Sat, 28 Apr 2007 08:53:37 -0700 (PDT) Received: from [192.168.0.101] (vpn-129-150-18-29.SFBay.Sun.COM [129.150.18.29]) by sfbaymail1sca.SFBay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l3SFqeB8020963; Sat, 28 Apr 2007 08:52:43 -0700 (PDT) Message-ID: <46336D84.1030108@sun.com> Date: Sat, 28 Apr 2007 08:51:32 -0700 From: Alan Coopersmith User-Agent: Thunderbird 1.5.0.4 (X11/20060602) MIME-Version: 1.0 To: Joerg Schilling CC: psarc-ext@sac.sfbay.sun.com, Mark.Maybee@sun.com, Timothy.Haley@sun.com, Ienup.Sung@sun.com, cifs-vfs-team@sun.com Subject: Re: ZFS case-insensitive support [PSARC/2007/244 Timeout: 05/04/2007] References: <46334F3D.5060204@Sun.COM> <4633515c.U66IQ9uumtfOwDQW%Joerg.Schilling@fokus.fraunhofer.de> In-Reply-To: <4633515c.U66IQ9uumtfOwDQW%Joerg.Schilling@fokus.fraunhofer.de> Content-Type: text/plain; charset=ISO8859-1; format=flowed Content-Transfer-Encoding: 7bit Status: RO Content-Length: 1877 Joerg Schilling wrote: > Mark Maybee wrote: > >> re-sending this to bring psarc-ext into the loop. >> > > Could you please explain whyt you understand by "case-insensitive"? I thought the "CASE BEHAVIOR" section of the message you replied to specified that pretty clearly - could you please explain what you didn't understand in the description already given? > Case-insensitive behavior, as required by the Solaris CIFS server, > follows a particular set of rules: > > Object Creation: Before a named object is created, the directory > must be searched for any c-i matches of that name. If there is a c-i > match, then the create operation must fail with EEXIST. > > Lookup: In mixed-mode (volume supports both c-i and c-s behavior), > it's possible for a c-s client to create names that are c-i > clashes. For example, it may be possible for "foo", "Foo", and > "FOO" to exist in the same directory. In those cases, a c-i > lookup of any c-s form of "foo" (e.g., "foo", "FOO", "FoO", "fOo", > ad nauseum) should always return the same name. In other words, > if a c-i lookup of "foo" returns "foo", then a c-i lookup of "FOO" > or "Foo" should also return "foo". It doesn't matter what the > algorithm is as long as it is consistent. In addition, the > VOP_LOOKUP() routine must return an indication if this file name > is a c-i conflict with another file name in the same parent > directory. > > Readdir: Since case-insensitive duplicates may exist in a directory, > the VOP_READDIR() interface must return an indication if a directory > entry contains a file name that is a c-i conflict with another entry > in the same directory. -- -Alan Coopersmith- alan.coopersmith@sun.com Sun Microsystems, Inc. - X Window System Engineering From schilling@fokus.fraunhofer.de Sat Apr 28 09:16:03 2007 Received: from sfbaymail2sca.sfbay.sun.com (sfbaymail2sca.SFBay.Sun.COM [129.145.155.42]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l3SGG3NL017885 for ; Sat, 28 Apr 2007 09:16:03 -0700 (PDT) Received: from sca-ea-mail-3.sun.com (sca-ea-mail-3.Sun.COM [192.18.43.21]) by sfbaymail2sca.sfbay.sun.com (8.13.6+Sun/8.12.10/ENSMAIL,v2.2) with ESMTP id l3SGFA47012622 for ; Sat, 28 Apr 2007 09:15:10 -0700 (PDT) Received: from relay22.sun.com (relay22.sun.com [192.12.251.34] (may be forged)) by sca-ea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l3SGFAWe011280 for ; Sat, 28 Apr 2007 16:15:10 GMT Received: from mms24es.sun.com ([150.143.232.74] [150.143.232.74]) by relay22.sun.com with ESMTP id BT-MMP-1431624 for psarc-ext@sac.sfbay.sun.com; Sat, 28 Apr 2007 16:15:09 Z Received: from mms23bas.mms.us.syntegra.com (relay23.mms.us.syntegra.com [192.12.251.50]) by mms24es.sun.com with ESMTP id BT-MMP-731326 for psarc-ext@sac.sfbay.sun.com; Sat, 28 Apr 2007 16:15:09 Z Received: from mailhub.fokus.fraunhofer.de ([193.174.154.14] [193.174.154.14]) by relay23.sun.com with ESMTP id BT-MMP-7003286 for psarc-ext@sac.sfbay.sun.com; Sat, 28 Apr 2007 16:15:09 Z Received: from burner.fokus.fraunhofer.de (burner [10.147.65.166]) by mailhub.fokus.fraunhofer.de (8.11.6p2/8.11.6) with ESMTP id l3SGF6f10439; Sat, 28 Apr 2007 18:15:07 +0200 (MEST) Received: (from jes@localhost) by burner.fokus.fraunhofer.de (8.12.9+Sun/8.12.9/Submit) id l3SGElvl009977; Sat, 28 Apr 2007 18:14:47 +0200 (CEST) Date: Sat, 28 Apr 2007 18:14:47 +0200 From: Joerg.Schilling@fokus.fraunhofer.de (Joerg Schilling) To: alan.coopersmith@sun.com Cc: Timothy.Haley@sun.com, psarc-ext@sac.sfbay.sun.com, Mark.Maybee@sun.com, Ienup.Sung@sun.com, cifs-vfs-team@sun.com Subject: Re: ZFS case-insensitive support [PSARC/2007/244 Timeout: 05/04/2007] Message-Id: <463372f7.QfLobcI6/VOKJECA%Joerg.Schilling@fokus.fraunhofer.de> References: <46334F3D.5060204@Sun.COM> <4633515c.U66IQ9uumtfOwDQW%Joerg.Schilling@fokus.fraunhofer.de> <46336D84.1030108@sun.com> In-Reply-To: <46336D84.1030108@sun.com> User-Agent: nail 11.22 3/20/05 MIME-Version: 1.0 Content-Type: text/plain; charset=ISO8859-1 Content-Transfer-Encoding: 8bit Sender: schilling@fokus.fraunhofer.de Status: RO Content-Length: 1137 Alan Coopersmith wrote: > Joerg Schilling wrote: > > Mark Maybee wrote: > > > >> re-sending this to bring psarc-ext into the loop. > >> > > > > Could you please explain whyt you understand by "case-insensitive"? > > I thought the "CASE BEHAVIOR" section of the message you replied to > specified that pretty clearly - could you please explain what you > didn't understand in the description already given? Sorry, I dod not find it from two "eye scans" of the content. > > Readdir: Since case-insensitive duplicates may exist in a directory, > > the VOP_READDIR() interface must return an indication if a directory > > entry contains a file name that is a c-i conflict with another entry > > in the same directory. Isn't this something that will slow down readdir? Jörg -- EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin js@cs.tu-berlin.de (uni) schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/old/private/ ftp://ftp.berlios.de/pub/schily From Timothy.Haley@Sun.COM Sat Apr 28 09:17:16 2007 Received: from sfbaymail1sca.SFBay.Sun.COM (sfbaymail1sca.SFBay.Sun.COM [129.145.154.35]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l3SGHGpU018004; Sat, 28 Apr 2007 09:17:16 -0700 (PDT) Received: from brmea-mail-3.sun.com (brmea-mail-3.Sun.COM [192.18.98.34]) by sfbaymail1sca.SFBay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l3SGGOgJ024611; Sat, 28 Apr 2007 09:16:24 -0700 (PDT) Received: from fe-amer-06.sun.com ([192.18.108.180]) by brmea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l3SGGNQB005166; Sat, 28 Apr 2007 16:16:23 GMT Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JH700201TF6X500@mail-amer.sun.com> (original mail from Timothy.Haley@Sun.COM); Sat, 28 Apr 2007 10:16:23 -0600 (MDT) Received: from [172.16.12.5] ([129.150.32.179]) by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JH7004N6UJB0RC0@mail-amer.sun.com>; Sat, 28 Apr 2007 10:16:23 -0600 (MDT) Date: Sat, 28 Apr 2007 10:16:27 -0600 From: Timothy Haley Subject: Re: 2007/244 [ZFS case-insensitive support] In-reply-to: <200704280031.l3S0VdQT021940@ivrel.sfbay.sun.com> Sender: Timothy.Haley@Sun.COM To: Glenn Skinner Cc: PSARC-ext@sac.sfbay.sun.com, maybee@sac.sfbay.sun.com, Ienup.Sung@Sun.COM, cifs-vfs-team@Sun.COM Message-id: <4633735B.6080504@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200704280031.l3S0VdQT021940@ivrel.sfbay.sun.com> User-Agent: Thunderbird 1.5.0.10 (Macintosh/20070221) Status: RO Content-Length: 2428 Glenn Skinner wrote: > This case's IAM file marks it as being open, but mail pertaining to it > has gone to the psarc alias rather than to psarc-ext. Please bring > psarc-ext into the loop. (I'll leave it to you to make sure that > psarc-ext sees the comments below.) > > Moving to the substance of the proposal, in its description of > modifications to fop_lookup(), the proposal states: > > Also, if direntflagp is non-NULL then the flags will be > returned in *direntflagp. If a file system supports c-i name > matching, it must also support the per-directory-entry flags. > > What happens if a file system fails to obey this constraint? What > error(s) is/are returned in response to what call(s)? > That language is actually too strongly worded. I will change it - Also, if direntflagp is non-NULL then the flags will be - returned in *direntflagp. If a file system supports c-i name - matching, it must also support the per-directory-entry flags. + Also, if direntflagp is non-NULL and the file system supports + per-directory-entry flags, then the flags will be returned in + *direntflagp. If a file system supports c-i name matching and + mixed case sensitivity, it should also support the + per-directory-entry flags. Note that's a should now and not a must. > The proposal's specification of per-directory-entry flags seems to be > incomplete. It says how they're fetched, but beyond the statement > > The readdir fop interface needs to pass a new flag, > V_RDDIR_ENTFLAGS, to request c-i conflict information via the flag > field of a modified dirent structure. > > it doesn't say how the flags are set (including which bits mean what). > So what edirent flags does this proposal define, and under what > circumstances are they set? > Good point. I believe the following additional lines are necessary: #define V_RDDIR_ENTFLAGS 0x01 /* request dirent flags */ + One dirent flag is being defined + + #define ED_CASE_CONFLICT 0x10 + + If set, it implies the entry conflicts, when case is disconsidered, + with at least one other entry in the directory. + The file systems that implement c-i behavior are responsible for following the behaviors described below. Under each fop call is a description of the behavior with FIGNORECASE set. Otherwise, the Are they sufficient as well? -tim From schilling@fokus.fraunhofer.de Sat Apr 28 09:25:18 2007 Received: from sfbaymail2sca.sfbay.sun.com (sfbaymail2sca.SFBay.Sun.COM [129.145.155.42]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l3SGPICE018670; Sat, 28 Apr 2007 09:25:18 -0700 (PDT) Received: from sca-ea-mail-2.sun.com (sca-ea-mail-2.Sun.COM [192.18.43.25]) by sfbaymail2sca.sfbay.sun.com (8.13.6+Sun/8.12.10/ENSMAIL,v2.2) with ESMTP id l3SGOQR1014089; Sat, 28 Apr 2007 09:24:26 -0700 (PDT) Received: from relay3.sun.com (relay3.sun.com [150.143.103.54] (may be forged)) by sca-ea-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id l3SB1n2e022147; Sat, 28 Apr 2007 16:24:25 GMT Received: from mms03es.sun.com ([150.143.104.54] [150.143.104.54]) by relay3.sun.com with ESMTP id BT-MMP-322352; Sat, 28 Apr 2007 16:24:25 Z Received: from relay1.sun.com (relay1.sun.com [150.143.103.14]) by mms03es.sun.com with ESMTP id BT-MMP-2370892; Sat, 28 Apr 2007 16:24:25 Z Received: from mailhub.fokus.fraunhofer.de ([193.174.154.14] [193.174.154.14]) by relay1.sun.com with ESMTP id BT-MMP-2801790; Sat, 28 Apr 2007 16:24:24 Z Received: from burner.fokus.fraunhofer.de (burner [10.147.65.166]) by mailhub.fokus.fraunhofer.de (8.11.6p2/8.11.6) with ESMTP id l3SGOMf11455; Sat, 28 Apr 2007 18:24:23 +0200 (MEST) Received: (from jes@localhost) by burner.fokus.fraunhofer.de (8.12.9+Sun/8.12.9/Submit) id l3SGO4jm009985; Sat, 28 Apr 2007 18:24:04 +0200 (CEST) Date: Sat, 28 Apr 2007 18:24:04 +0200 From: Joerg.Schilling@fokus.fraunhofer.de (Joerg Schilling) To: Timothy.Haley@sun.com, glenn.skinner@sun.com Cc: PSARC-ext@sac.sfbay.sun.com, maybee@sac.sfbay.sun.com, Ienup.Sung@sun.com, cifs-vfs-team@sun.com Subject: Re: 2007/244 [ZFS case-insensitive support] Message-Id: <46337524.3xJDp67ghqy5JQ/m%Joerg.Schilling@fokus.fraunhofer.de> References: <200704280031.l3S0VdQT021940@ivrel.sfbay.sun.com> <4633735B.6080504@sun.com> In-Reply-To: <4633735B.6080504@sun.com> User-Agent: nail 11.22 3/20/05 MIME-Version: 1.0 Content-Type: text/plain; charset=ISO8859-1 Content-Transfer-Encoding: 8bit Sender: schilling@fokus.fraunhofer.de Status: RO Content-Length: 783 Timothy Haley wrote: > + One dirent flag is being defined > + > + #define ED_CASE_CONFLICT 0x10 > + > + If set, it implies the entry conflicts, when case is disconsidered, > + with at least one other entry in the directory. > + This flag will only help you in case that you know that you did completeley scan the directory already since the last mount and that the case insensitivity flags did not change since the last complete directory scan. Jörg -- EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin js@cs.tu-berlin.de (uni) schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/old/private/ ftp://ftp.berlios.de/pub/schily From Calum.Mackay@Sun.COM Mon Apr 30 06:47:20 2007 Received: from sfbaymail1sca.SFBay.Sun.COM (sfbaymail1sca.SFBay.Sun.COM [129.145.154.35]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l3UDlKc9002832 for ; Mon, 30 Apr 2007 06:47:20 -0700 (PDT) Received: from gmp-ea-fw-1.sun.com (gmpes-gis-mail-2.UK.Sun.COM [129.156.42.6]) by sfbaymail1sca.SFBay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l3UDkOcH025284 for ; Mon, 30 Apr 2007 06:46:25 -0700 (PDT) Received: from d1-emea-09.sun.com ([192.18.2.119]) by gmp-ea-fw-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l3UDkIrP016696 for ; Mon, 30 Apr 2007 13:46:19 GMT Received: from conversion-daemon.d1-emea-09.sun.com by d1-emea-09.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JHB00B01CV91T00@d1-emea-09.sun.com> (original mail from Calum.Mackay@Sun.COM) for psarc-ext@sac.sfbay.sun.com; Mon, 30 Apr 2007 14:46:18 +0100 (BST) Received: from [192.168.254.1] ([62.24.230.83]) by d1-emea-09.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JHB00I0XCX54Q70@d1-emea-09.sun.com> for psarc-ext@sac.sfbay.sun.com; Mon, 30 Apr 2007 14:46:18 +0100 (BST) Date: Mon, 30 Apr 2007 14:46:16 +0100 From: Calum Mackay Subject: 2007/244 ZFS case-insensitive support - email test Sender: Calum.Mackay@Sun.COM To: psarc-ext@sac.sfbay.sun.com Message-id: <4635F328.50904@sun.com> Organization: Sun Microsystems MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT User-Agent: Thunderbird 3.0a1 (X11/20070423) Status: RO Content-Length: 219 please ignore this email... :) I'm getting reports of some emails making it into the case mail file, but not making it to the alias on the Interest list in the IAM file. this is a test of that mechanism. cheers, c. From calum.mackay@cdmnet.org Mon Apr 30 06:49:30 2007 Received: from sfbaymail2sca.sfbay.sun.com (sfbaymail2sca.SFBay.Sun.COM [129.145.155.42]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l3UDnTk8002891 for ; Mon, 30 Apr 2007 06:49:30 -0700 (PDT) Received: from brmea-mail-1.sun.com (brmea-mail-1.Sun.COM [192.18.98.31]) by sfbaymail2sca.sfbay.sun.com (8.13.6+Sun/8.12.10/ENSMAIL,v2.2) with ESMTP id l3UDmYhf002660 for ; Mon, 30 Apr 2007 06:48:34 -0700 (PDT) Received: from relay1.sun.com (relay1.sun.com [150.143.103.14] (may be forged)) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l3UDZJ8D002824 for ; Mon, 30 Apr 2007 13:48:34 GMT Received: from mms03es.sun.com ([150.143.104.54] [150.143.104.54]) by relay1.sun.com with ESMTP id BT-MMP-459215 for psarc-ext@sac.sfbay.sun.com; Mon, 30 Apr 2007 13:48:33 Z Received: from relay01i.sun.com (ip70.net150143-60.block3.us.syntegra.com [150.143.60.70]) by mms03es.sun.com with ESMTP id BT-MMP-1759439 for psarc-ext@sac.sfbay.sun.com; Mon, 30 Apr 2007 13:48:33 Z Received: from cdmnet.org ([62.24.230.83] [62.24.230.83]) by relay0i.sun.com with ESMTP id BT-MMP-4076361 for psarc-ext@sac.sfbay.sun.com; Mon, 30 Apr 2007 13:48:32 Z Received: from localhost ([127.0.0.1]:46192 ident=calum) by cdmnet.org with esmtp (Exim 4.63) (envelope-from ) id 1HiWEy-0004b2-4L for psarc-ext@sac.sfbay.sun.com; Mon, 30 Apr 2007 14:48:32 +0100 Message-Id: <4635F3AF.8080100@cdmnet.org> Date: Mon, 30 Apr 2007 14:48:31 +0100 From: Calum Mackay User-Agent: Thunderbird 3.0a1 (X11/20070423) MIME-Version: 1.0 To: psarc-ext@sac.sfbay.sun.com Subject: Re: 2007/244 ZFS case-insensitive support - email test References: <4635F328.50904@sun.com> In-Reply-To: <4635F328.50904@sun.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Status: RO Content-Length: 63 reply test from outside Sun. apols for the spam. cheers, c. From Timothy.Haley@Sun.COM Mon Apr 30 09:51:45 2007 Received: from sfbaymail2sca.sfbay.sun.com (sfbaymail2sca.SFBay.Sun.COM [129.145.155.42]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l3UGpjOx011694 for ; Mon, 30 Apr 2007 09:51:45 -0700 (PDT) Received: from brmea-mail-2.sun.com (brmea-mail-2.Sun.COM [192.18.98.43]) by sfbaymail2sca.sfbay.sun.com (8.13.6+Sun/8.12.10/ENSMAIL,v2.2) with ESMTP id l3UGonAS020784 for ; Mon, 30 Apr 2007 09:50:50 -0700 (PDT) Received: from fe-amer-06.sun.com ([192.18.108.180]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l3UGontl026781 for ; Mon, 30 Apr 2007 16:50:49 GMT Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JHB00B01L0O1K00@mail-amer.sun.com> (original mail from Timothy.Haley@Sun.COM) for psarc-ext@sac.sfbay.sun.com; Mon, 30 Apr 2007 10:50:49 -0600 (MDT) Received: from [172.20.25.27] by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JHB00C30LGPUFC0@mail-amer.sun.com> for psarc-ext@sac.sfbay.sun.com; Mon, 30 Apr 2007 10:50:49 -0600 (MDT) Date: Mon, 30 Apr 2007 10:50:49 -0600 From: Timothy Haley Subject: Re: ZFS case-insensitive support [PSARC/2007/244 Timeout: 05/04/2007] In-reply-to: <463372f7.QfLobcI6/VOKJECA%Joerg.Schilling@fokus.fraunhofer.de> Sender: Timothy.Haley@Sun.COM To: Joerg Schilling Cc: psarc-ext@sac.sfbay.sun.com, Mark.Maybee@Sun.COM, Ienup.Sung@Sun.COM Message-id: <46361E69.7080409@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <46334F3D.5060204@Sun.COM> <4633515c.U66IQ9uumtfOwDQW%Joerg.Schilling@fokus.fraunhofer.de> <46336D84.1030108@sun.com> <463372f7.QfLobcI6/VOKJECA%Joerg.Schilling@fokus.fraunhofer.de> User-Agent: Thunderbird 2.0b2 (X11/20070227) Status: RO Content-Length: 604 Joerg Schilling wrote: > >>> Readdir: Since case-insensitive duplicates may exist in a directory, >>> the VOP_READDIR() interface must return an indication if a directory >>> entry contains a file name that is a c-i conflict with another entry >>> in the same directory. > > Isn't this something that will slow down readdir? > That really depends upon how it is implemented. Within our ZFS implementation we can determine this quite quickly. Also, the general readdir() case need not be affected, this indication need only be returned if V_RDDIR_DIRENTFLAGS were requested. -tim From Timothy.Haley@Sun.COM Mon Apr 30 10:01:48 2007 Received: from sfbaymail2sca.sfbay.sun.com (sfbaymail2sca.SFBay.Sun.COM [129.145.155.42]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l3UH1l2F012400; Mon, 30 Apr 2007 10:01:48 -0700 (PDT) Received: from brmea-mail-1.sun.com (brmea-mail-1.Sun.COM [192.18.98.31]) by sfbaymail2sca.sfbay.sun.com (8.13.6+Sun/8.12.10/ENSMAIL,v2.2) with ESMTP id l3UH0qpS026103; Mon, 30 Apr 2007 10:00:52 -0700 (PDT) Received: from fe-amer-10.sun.com ([192.18.108.184]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l3UH0qWD001332; Mon, 30 Apr 2007 17:00:52 GMT Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JHB00D01LW6TJ00@mail-amer.sun.com> (original mail from Timothy.Haley@Sun.COM); Mon, 30 Apr 2007 11:00:52 -0600 (MDT) Received: from [172.20.25.27] by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JHB002NULXFZK70@mail-amer.sun.com>; Mon, 30 Apr 2007 11:00:52 -0600 (MDT) Date: Mon, 30 Apr 2007 11:00:51 -0600 From: Timothy Haley Subject: Re: 2007/244 [ZFS case-insensitive support] In-reply-to: <46337524.3xJDp67ghqy5JQ/m%Joerg.Schilling@fokus.fraunhofer.de> Sender: Timothy.Haley@Sun.COM To: Joerg Schilling Cc: glenn.skinner@Sun.COM, PSARC-ext@sac.sfbay.sun.com, maybee@sac.sfbay.sun.com, Ienup.Sung@Sun.COM, cifs-vfs-team@Sun.COM Message-id: <463620C3.2070001@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200704280031.l3S0VdQT021940@ivrel.sfbay.sun.com> <4633735B.6080504@sun.com> <46337524.3xJDp67ghqy5JQ/m%Joerg.Schilling@fokus.fraunhofer.de> User-Agent: Thunderbird 2.0b2 (X11/20070227) Status: RO Content-Length: 757 Joerg Schilling wrote: > Timothy Haley wrote: > >> + One dirent flag is being defined >> + >> + #define ED_CASE_CONFLICT 0x10 >> + >> + If set, it implies the entry conflicts, when case is disconsidered, >> + with at least one other entry in the directory. >> + > > This flag will only help you in case that you know that you did completeley > scan the directory already since the last mount and that the case insensitivity > flags did not change since the last complete directory scan. > I'm not sure I follow. Is there a question here? If requested, this flag is set by the VOP_LOOKUP() or VOP_READDIR() inside direntflagp or ed_eflags, respectively, if a conflict exists at the time of the VOP request. -tim From glenn.skinner@sun.com Mon Apr 30 11:57:49 2007 Received: from ivrel.sfbay.sun.com (ivrel [129.146.74.76]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l3UIvn31018421; Mon, 30 Apr 2007 11:57:49 -0700 (PDT) Received: from ivrel (ivrel [129.146.74.76]) by ivrel.sfbay.sun.com (8.13.8+Sun/8.13.8) with SMTP id l3UIsSBI023371; Mon, 30 Apr 2007 11:54:28 -0700 (PDT) Message-Id: <200704301854.l3UIsSBI023371@ivrel.sfbay.sun.com> Date: Mon, 30 Apr 2007 11:54:28 -0700 (PDT) From: Glenn Skinner Reply-To: Glenn Skinner Subject: Re: 2007/244 [ZFS case-insensitive support] To: Timothy.Haley@sun.com Cc: PSARC-ext@sac.sfbay.sun.com, maybee@sac.sfbay.sun.com, Ienup.Sung@sun.com, cifs-vfs-team@sun.com MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: A2zEzIACH14Q39pcg3GJlg== X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.6_36 SunOS 5.11 sun4u sparc Status: RO Content-Length: 3189 Date: Sat, 28 Apr 2007 10:16:27 -0600 From: Timothy Haley Subject: Re: 2007/244 [ZFS case-insensitive support] Glenn Skinner wrote: > Moving to the substance of the proposal, in its description of > modifications to fop_lookup(), the proposal states: > > Also, if direntflagp is non-NULL then the flags will be > returned in *direntflagp. If a file system supports c-i name > matching, it must also support the per-directory-entry flags. > > What happens if a file system fails to obey this constraint? What > error(s) is/are returned in response to what call(s)? That language is actually too strongly worded. I will change it - Also, if direntflagp is non-NULL then the flags will be - returned in *direntflagp. If a file system supports c-i name - matching, it must also support the per-directory-entry flags. + Also, if direntflagp is non-NULL and the file system supports + per-directory-entry flags, then the flags will be returned in + *direntflagp. If a file system supports c-i name matching and + mixed case sensitivity, it should also support the + per-directory-entry flags. Note that's a should now and not a must. That rephrasing's an improvement, but it still doesn't answer my question. Let me try again. Suppose a file system is misconfigured so that it claims to support c-i name matching, but doesn't support per-directory-entry flags. Is there anything that prevents such a misconfiguration from occurring? If there isn't, what happens when things occur that expose the misconfiguration? > The proposal's specification of per-directory-entry flags seems to be > incomplete. It says how they're fetched, but beyond the statement > > The readdir fop interface needs to pass a new flag, > V_RDDIR_ENTFLAGS, to request c-i conflict information via the flag > field of a modified dirent structure. > > it doesn't say how the flags are set (including which bits mean what). > So what edirent flags does this proposal define, and under what > circumstances are they set? > Good point. I believe the following additional lines are necessary: #define V_RDDIR_ENTFLAGS 0x01 /* request dirent flags */ + One dirent flag is being defined + + #define ED_CASE_CONFLICT 0x10 + + If set, it implies the entry conflicts, when case is disconsidered, + with at least one other entry in the directory. + The file systems that implement c-i behavior are responsible for following the behaviors described below. Under each fop call is a description of the behavior with FIGNORECASE set. Otherwise, the Are they sufficient as well? Perhaps some sample code that requests this flag and then acts on the value it finds would help answer this question. (I infer from Joerg Schilling's posting that he questions the flag's usefulness in practice. So the sample code could kill two birds with one stone by showing how it _is_ useful.) -- Glenn From bart.smaalders@Sun.COM Mon Apr 30 13:49:06 2007 Received: from zion.eng.sun.com (zion [129.146.17.75]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l3UKn6J9022388 for ; Mon, 30 Apr 2007 13:49:06 -0700 (PDT) Received: from [129.146.228.109] (cyber [129.146.228.109]) by zion.eng.sun.com (8.13.7+Sun/8.13.7) with ESMTP id l3UKmA5Y028894; Mon, 30 Apr 2007 13:48:10 -0700 (PDT) Message-ID: <463655BC.7070702@Sun.COM> Date: Mon, 30 Apr 2007 13:46:52 -0700 From: Bart Smaalders Organization: Sun Microsystems User-Agent: Thunderbird 2.0b2 (X11/20070227) MIME-Version: 1.0 To: Joerg Schilling CC: alan.coopersmith@sun.com, cifs-vfs-team@sun.com, Mark.Maybee@sun.com, Ienup.Sung@sun.com, psarc-ext@sac.sfbay.sun.com, Timothy.Haley@sun.com Subject: Re: ZFS case-insensitive support [PSARC/2007/244 Timeout: 05/04/2007] References: <46334F3D.5060204@Sun.COM> <4633515c.U66IQ9uumtfOwDQW%Joerg.Schilling@fokus.fraunhofer.de> <46336D84.1030108@sun.com> <463372f7.QfLobcI6/VOKJECA%Joerg.Schilling@fokus.fraunhofer.de> In-Reply-To: <463372f7.QfLobcI6/VOKJECA%Joerg.Schilling@fokus.fraunhofer.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Status: RO Content-Length: 1657 Joerg Schilling wrote: > Alan Coopersmith wrote: > >> Joerg Schilling wrote: >>> Mark Maybee wrote: >>> >>>> re-sending this to bring psarc-ext into the loop. >>>> >>> Could you please explain whyt you understand by "case-insensitive"? >> I thought the "CASE BEHAVIOR" section of the message you replied to >> specified that pretty clearly - could you please explain what you >> didn't understand in the description already given? > > Sorry, I dod not find it from two "eye scans" of the content. > >>> Readdir: Since case-insensitive duplicates may exist in a directory, >>> the VOP_READDIR() interface must return an indication if a directory >>> entry contains a file name that is a c-i conflict with another entry >>> in the same directory. > > Isn't this something that will slow down readdir? > No. You could set such flags when the conflicting files were created. Remember, a filesystem cannot change: The following three properties cannot be changed once the file system has been created, and so should be set at file system creation time. If not set in the "zfs create" command these properties will be inherited from the parent dataset. If the parent lacks these properties due to having been created prior to these features being supported, the new file system will have the default values for these properties. Thus, since one has to search for conflicts anyway during create, the partial (case insensitive) conflict can be noted then. - Bart -- Bart Smaalders Solaris Kernel Performance barts@cyber.eng.sun.com http://blogs.sun.com/barts From Timothy.Haley@Sun.COM Mon Apr 30 22:32:46 2007 Received: from sfbaymail2sca.sfbay.sun.com (sfbaymail2sca.SFBay.Sun.COM [129.145.155.42]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l415Wk26012397; Mon, 30 Apr 2007 22:32:46 -0700 (PDT) Received: from brmea-mail-1.sun.com (brmea-mail-1.Sun.COM [192.18.98.31]) by sfbaymail2sca.sfbay.sun.com (8.13.6+Sun/8.12.10/ENSMAIL,v2.2) with ESMTP id l415Vm3C026079; Mon, 30 Apr 2007 22:31:48 -0700 (PDT) Received: from fe-amer-09.sun.com ([192.18.108.183]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l415VmiJ022475; Tue, 1 May 2007 05:31:48 GMT Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JHC00L01KJ48500@mail-amer.sun.com> (original mail from Timothy.Haley@Sun.COM); Mon, 30 Apr 2007 23:31:48 -0600 (MDT) Received: from [172.16.12.5] ([129.150.32.147]) by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JHC00COVKOXLG40@mail-amer.sun.com>; Mon, 30 Apr 2007 23:31:47 -0600 (MDT) Date: Mon, 30 Apr 2007 23:31:46 -0600 From: Timothy Haley Subject: Re: 2007/244 [ZFS case-insensitive support] In-reply-to: <200704301854.l3UIsSBI023371@ivrel.sfbay.sun.com> Sender: Timothy.Haley@Sun.COM To: Glenn Skinner Cc: PSARC-ext@sac.sfbay.sun.com, maybee@sac.sfbay.sun.com, Ienup.Sung@Sun.COM, cifs-vfs-team@Sun.COM Message-id: <4636D0C2.4020506@sun.com> MIME-version: 1.0 Content-type: multipart/mixed; boundary="Boundary_(ID_tHU6qEyNZE3hXrBwYKvxtQ)" References: <200704301854.l3UIsSBI023371@ivrel.sfbay.sun.com> User-Agent: Thunderbird 1.5.0.10 (Macintosh/20070221) Status: RO Content-Length: 35490 This is a multi-part message in MIME format. --Boundary_(ID_tHU6qEyNZE3hXrBwYKvxtQ) Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT Glenn Skinner wrote: > Date: Sat, 28 Apr 2007 10:16:27 -0600 > From: Timothy Haley > Subject: Re: 2007/244 [ZFS case-insensitive support] > > Glenn Skinner wrote: > > Moving to the substance of the proposal, in its description of > > modifications to fop_lookup(), the proposal states: > > > > Also, if direntflagp is non-NULL then the flags will be > > returned in *direntflagp. If a file system supports c-i name > > matching, it must also support the per-directory-entry flags. > > > > What happens if a file system fails to obey this constraint? What > > error(s) is/are returned in response to what call(s)? > > That language is actually too strongly worded. I will change it > > - Also, if direntflagp is non-NULL then the flags will be > - returned in *direntflagp. If a file system supports c-i name > - matching, it must also support the per-directory-entry flags. > + Also, if direntflagp is non-NULL and the file system supports > + per-directory-entry flags, then the flags will be returned in > + *direntflagp. If a file system supports c-i name matching and > + mixed case sensitivity, it should also support the > + per-directory-entry flags. > > Note that's a should now and not a must. > > That rephrasing's an improvement, but it still doesn't answer my > question. Let me try again. > > Suppose a file system is misconfigured so that it claims to support c-i > name matching, but doesn't support per-directory-entry flags. Is there > anything that prevents such a misconfiguration from occurring? No, nothing prevents such a "misconfiguration". That's why I softened the language. I'm not sure I'd really term it a misconfiguration, perhaps a more appropriate term is suboptimal. > If > there isn't, what happens when things occur that expose the > misconfiguration? > If the file system were to support mixed c-i but not the direntflags, the fop_readdir() or fop_lookup() consumer would have to manually determine what case conflicts existed. So, I've revised this paragraph one more time: - Also, if direntflagp is non-NULL then the flags will be - returned in *direntflagp. If a file system supports c-i name - matching, it must also support the per-directory-entry flags. + Also if FIGNORECASE is set, direntflagp is non-NULL and the + file system supports per-directory-entry flags, then the flags + will be returned in *direntflagp. If a file system supports + c-i name matching and mixed case sensitivity, it should also + support the per-directory-entry flags. Supporting the + per-directory-entry flags is not mandatory, but if they are + not supported, the consequence is that the consumer of + fop_lookup() will be required to use brute-force techniques to + manually determine if there are any case conflicts within the + directory. > > The proposal's specification of per-directory-entry flags seems to be > > incomplete. It says how they're fetched, but beyond the statement > > > > The readdir fop interface needs to pass a new flag, > > V_RDDIR_ENTFLAGS, to request c-i conflict information via the flag > > field of a modified dirent structure. > > > > it doesn't say how the flags are set (including which bits mean what). > > So what edirent flags does this proposal define, and under what > > circumstances are they set? > > > > Good point. I believe the following additional lines are necessary: > > #define V_RDDIR_ENTFLAGS 0x01 /* request dirent flags */ > > + One dirent flag is being defined > + > + #define ED_CASE_CONFLICT 0x10 > + > + If set, it implies the entry conflicts, when case is disconsidered, > + with at least one other entry in the directory. > + > The file systems that implement c-i behavior are responsible for > following the behaviors described below. Under each fop call is a > description of the behavior with FIGNORECASE set. Otherwise, the > > Are they sufficient as well? > > Perhaps some sample code that requests this flag and then acts on the > value it finds would help answer this question. (I infer from Joerg > Schilling's posting that he questions the flag's usefulness in > practice. So the sample code could kill two birds with one stone by > showing how it _is_ useful.) > > -- Glenn > I've added snippets of code or pseudo-code to both the fop_lookup() and fop_readdir() descriptions. I've attached both the diffs between the first and latest versions, and the complete latest version. thanks, -tim --Boundary_(ID_tHU6qEyNZE3hXrBwYKvxtQ) Content-type: text/plain; name=diffs; x-mac-creator=0; x-mac-type=0 Content-transfer-encoding: 7BIT Content-disposition: inline; filename=diffs ------- cinew.txt ------- --- /tmp/geta18883 Mon Apr 30 23:09:39 2007 +++ /tmp/getb18883 Mon Apr 30 23:09:39 2007 @@ -192,6 +192,13 @@ */ #define V_RDDIR_ENTFLAGS 0x01 /* request dirent flags */ + One dirent flag is being defined + + #define ED_CASE_CONFLICT 0x10 + + If set, it implies the entry conflicts, when case is disconsidered, + with at least one other entry in the directory. + The file systems that implement c-i behavior are responsible for following the behaviors described below. Under each fop call is a description of the behavior with FIGNORECASE set. Otherwise, the @@ -226,12 +233,30 @@ If FIGNORECASE is set and the file system does not support c-i, we return EINVAL. - If realpnp is non-NULL, then lookup fills in the pathname - structure with the case-preserved name that was looked up. + If FIGNORECASE is set and realpnp is non-NULL, then lookup + fills in the pathname structure with the case-preserved name + that was looked up. - Also, if direntflagp is non-NULL then the flags will be - returned in *direntflagp. If a file system supports c-i name - matching, it must also support the per-directory-entry flags. + Also if FIGNORECASE is set, direntflagp is non-NULL and the + file system supports per-directory-entry flags, then the flags + will be returned in *direntflagp. If a file system supports + c-i name matching and mixed case sensitivity, it should also + support the per-directory-entry flags. Supporting the + per-directory-entry flags is not mandatory, but if they are + not supported, the consequence is that the consumer of + fop_lookup() will be required to use brute-force techniques to + manually determine if there are any case conflicts within the + directory. + + As an example, a CIFS server might use the flags to know it + needs to mangle a looked up name: + + error = VOP_LOOKUP(dvp, dname, &vp, FIGNORECASE, ...., &deflags, NULL); + if (error == 0 && (deflags & ED_CASE_CONFLICT)) + mname = mangle(name); + + Having the dirent flags available to a fop_lookup() caller also + hints at intriguing possibilities for future flags. int fop_create( vnode_t *dvp, @@ -353,9 +378,8 @@ /* NEW */ int flags) /* Takes V_RDDIR_ENTFLAGS */ If "flags" has V_RDDIR_ENTFLAGS set and the file system - supports per-directory-entry flags, as must be the case if the - file system supports c-i, then the uio structure will contain - the following structure: + supports per-directory-entry flags, then the uio structure + will contain the following structure: typedef struct edirent { ino64_t ed_ino; /* "inode number" of entry */ @@ -375,6 +399,33 @@ Note that if the file system supports per-directory-entry flags, then the file system must set VFSFT_DIRENTFLAGS using the VFS Feature Registration interface. + + Below is a simplified pseudo-code routine that uses the + fop_readdir() interface to rewrite the names of + case-insensitive conflicts: + + char ** + readnames() + { + ... + error = VOP_READDIR(vp, &uio, + cr, &eof, ct, V_RDDIR_DIRENTFLAGS); + + if (error) + return (error); + + compute # of entries and alloc rnames + + while entries remain and space left in rname array { + // get ed_eflags from edirent structure + if (ed_eflags & ED_CASE_CONFLICT) + strcpy(rnames[n++], mangle(ed_name)); + else + strcpy(rnames[n++], ed_name); + } + + return (rnames); + } int fop_symlink( vnode_t *dvp, @@ -399,8 +450,10 @@ MAN PAGE MODIFICATIONS - The following changes apply to the zfs(1M) man page: - + ---------------------------------------------------- + The following changes apply to the zfs(1M) man page: + ---------------------------------------------------- + The following three properties cannot be changed once the file system has been created, and so should be set at file system creation time. If not set in the "zfs create" command these properties will be @@ -476,6 +529,18 @@ non-privileged users via the ZFS delegated administration model PSARC/2006/465. + -------------------------------------------------------------------- + The following possible return value must be added to the open(2), + symlink(2), link(2), unlink(2), rename(2), mkdir(2), and rmdir(2) + man pages: + -------------------------------------------------------------------- + + EILSEQ The path argument includes non-UTF8 characters + and the file system only accepts file names + where all characters are part of the UTF-8 + character codeset. + + EXPORTED INTERFACES |Proposed |Specified | @@ -500,6 +565,12 @@ | | | requesting | V_RDDIR_ENTFLAGS ------------------------+---------------+---------------+---------------------- + ED_CASE_CONFLICT |Contracted |This | extdirent.h #define + |Consolidation |Document | + |Private | | + | | | + | +------------------------+---------------+---------------+---------------------- VOP_LOOKUP, fop_lookup |Contracted |This |New output parameters |Consolidation |Document |int *direntflagp |Private | |pathname_t *realpnp --Boundary_(ID_tHU6qEyNZE3hXrBwYKvxtQ) Content-type: text/plain; name=full; x-mac-creator=0; x-mac-type=0 Content-transfer-encoding: 7BIT Content-disposition: inline; filename=full This fast-track was spun off of the CIFS Service (PSARC 2006/715) case along with: PSARC 2007/218 caller_context_t in all VOPs PSARC 2007/227 VFS Feature Registration and ACL on Create Although each of these changes are part of the bigger picture, they have been broken down into smaller pieces so each gets the attention it deserves. Several of these CIFS-related fast-tracks will describe changes to the signatures of vnode operations (VOPs). In some cases, these fast-tracks will describe multiple changes to the same VOPs. The project team intends to put all of these changes into ON in a single putback. NOTE: The VFSDEF_VERSION number in sys/vfs.h will be bumped from 3 to 4 in order to prevent unbundled file system kernel modules with the old signatures from loading. Once the unbundled file system modules are updated with the new signatures and recompiled, they will also pick up the new VFSDEF_VERSION number and be allowed to load. Case-insensitive behavior on create and lookup is a fundamental requirement for CIFS service. Traditionally, Solaris users access files using case-sensitive create and lookup. Consequently, we need to preserve the traditional behavior yet allow the CIFS server to request case-insensitive behavior. The following are the requirements for changes to the VFS/vnode interface to support mixed-case (case-sensitive and case-insensitive) behavior in Solaris. REQUIREMENTS 1) The CIFS server requires at least one file system type to support case-preserving name creation and both case-sensitive (c-s) and case-insensitive (c-i) behavior (a.k.a., mixed-behavior) on lookup. Solaris will support both the NFS and CIFS protocols. Windows clients typically expect case insensitive behavior while NFS clients typically expect case-sensitive behavior. 2) The VOP interface shall provide a way for the caller to specify an option for case-insensitive behavior. Any given file system volume (i.e. mount-point) may support both case-insensitive or case-sensitive behavior. The caller must be able to specify the behavior on lookup and create operations. The VOP interface must support those requests. 3) An interface is needed to determine if a file system supports requests for case-sensitivity, case-insensitivity, or both. The CIFS server negotiates with its clients on case behavior and needs an interface to query the file system volume on its capabilities. 4) The VOP_LOOKUP interface must return the case-sensitive name in the case of a case-insensitive lookup. Certain CIFS operations (e.g, TRANS2_QUERY_PATH_INFORMATION) require the case-sensitive name that was matched in the lookup operation. 5) When requested, the VOP_READDIR interface must return information for each directory entry to indicate if the entry is a case-insensitive duplicate of another entry. The VOP_LOOKUP interface must also return the same information. File creates done by a non-CIFS application may create file names that are a case-insensitive duplicate of an existing file name. For example, if "foo" exists then a non-CIFS application could create the file "FOO", which is a case-insensitive duplicate. This requirement states both VOP_READDIR and VOP_LOOKUP, when requested, must provide the caller with an indication that a case-insensitive conflict exists. The caller (the CIFS server) may choose to present the name as a "mangled" name so that the user in the case-insensitive environment can distinguish between c-i conflicting file names. NON-REQUIREMENTS 1) Case-insensitive emulation for file systems that do not support native case-insensitive behavior is not a requirement. The initial consumer of case-insensitive behavior is the CIFS client. The CIFS server can handle both case-sensitive and case-insensitive behavior. As long as we have one file system which supports native case-insensitive and case-sensitive behavior (see Req't 1), then there is no requirement for emulation. Note that we may need this feature in the future (and the interfaces we create should not preclude support for emulation). INTERFACE CHANGES NEEDED The following outlines the set of changes to the Solaris file system interface which are needed to support mixed-case (case-sensitive and case-insensitive) behavior in file systems. This behavior is necessary to provide CIFS services in Solaris. CASE BEHAVIOR There are two forms of case behavior: Case-sensitive and case-insensitive. Case sensitive behavior is what Solaris currently supports: When an object is created with a name, the case of the name is preserved. Objects with names that only differ in case (e.g., "foo", "Foo", "FOO") can co-exist in the same directory. Lookups on a particular name can only succeed if the case matches. Case-insensitive behavior, as required by the Solaris CIFS server, follows a particular set of rules: Object Creation: Before a named object is created, the directory must be searched for any c-i matches of that name. If there is a c-i match, then the create operation must fail with EEXIST. Lookup: In mixed-mode (volume supports both c-i and c-s behavior), it's possible for a c-s client to create names that are c-i clashes. For example, it may be possible for "foo", "Foo", and "FOO" to exist in the same directory. In those cases, a c-i lookup of any c-s form of "foo" (e.g., "foo", "FOO", "FoO", "fOo", ad nauseum) should always return the same name. In other words, if a c-i lookup of "foo" returns "foo", then a c-i lookup of "FOO" or "Foo" should also return "foo". It doesn't matter what the algorithm is as long as it is consistent. In addition, the VOP_LOOKUP() routine must return an indication if this file name is a c-i conflict with another file name in the same parent directory. Readdir: Since case-insensitive duplicates may exist in a directory, the VOP_READDIR() interface must return an indication if a directory entry contains a file name that is a c-i conflict with another entry in the same directory. VFS INTERFACE CHANGES An interface is required to determine the case behavior of a particular file system volume. There are three possible modes for behavior: c-s only, c-i only, and mixed-mode (both c-s and c-i requests are supported). The user-level interface is pathconf(2). The pathconf(2) interface will have a new variable added: {CASE_BEHAVIOR}/_PC_CASE_BEHAVIOR. The pathconf(2) system call will return a value which represents the bitwise OR of the following flags which indicate the case-behavior that the file system supports: _CASE_INSENSITIVE This file system supports case-insensitive behavior _CASE_SENSITIVE This file system supports case-sensitive behavior The fs_pathconf() routine (default routine for VOP_PATHCONF()) will be modified to support _PC_CASE_BEHAVIOR. The default behavior for file systems on Solaris is case-sensitive so a call to fs_pathconf() with _PC_CASE_BEHAVIOR set as the cmd would set the _CASE_SENSITIVE bit. Any file system that supports anything but case-sensitive-only behavior is required to use the (new) VFS Feature Registration interfaces to register its case behavior. The following new VFS features will be introduced to support case behavior: #define VFSFT_CASEINSENSITIVE 0x100000002 /* Supports case-insensitive */ #define VFSFT_NOCASESENSITIVE 0x100000004 /* NOT case-sensitive */ #define VFSFT_DIRENTFLAGS 0x100000008 /* Supports dirent flags */ File systems that support case-insensitive behavior must register the VFSFT_CASEINSENSITIVE feature. File systems that do *not* support case-sensitive behavior must register the VFSFT_NOCASESENSITIVE feature. VOP (Vnode Operation) INTERFACE CHANGES The VOP/fop interfaces that deal with names (lookup, create, remove, link, rename, mkdir, rmdir, and symlink) need to pass a new flag, FIGNORECASE, to request c-i behavior as described above. The new flag will be defined in sys/file.h: #define FIGNORECASE 0x80000 /* request case-insensitive lookups */ The readdir fop interface needs to pass a new flag, V_RDDIR_ENTFLAGS, to request c-i conflict information via the flag field of a modified dirent structure. The new flag is defined in vnode.h: /* * Flags for VOP_READDIR */ #define V_RDDIR_ENTFLAGS 0x01 /* request dirent flags */ One dirent flag is being defined #define ED_CASE_CONFLICT 0x10 If set, it implies the entry conflicts, when case is disconsidered, with at least one other entry in the directory. The file systems that implement c-i behavior are responsible for following the behaviors described below. Under each fop call is a description of the behavior with FIGNORECASE set. Otherwise, the behavior is the same as today. The lookup (VOP_LOOKUP/fop_lookup) and create (VOP_CREATE/fop_create) routines already have a flag(s) field. However, lookup requires additional parameters to return the directory entry flags, direntflagp, and the real (case-preserved) name, realpnp. int fop_lookup( vnode_t *dvp, char *nm, vnode_t **vpp, pathname_t *pnp, int flags, vnode_t *rdir, cred_t *cr, caller_context *ctp, /* See PSARC/2007/218 */ /* NEW */ int *direntflagp, /* dirent-specific flags */ /* NEW */ pathname_t *realpnp); /* real case-sensitive name */ If FIGNORECASE is set and the file system supports c-i, return the vnode in **vpp with a c-i name match, if it exists. Note that in the presence of multiple case versions of a given name (e.g., "foo", "Foo", "FOO"), the case of the name (nm) passed to fop_lookup does not affect which case-sensitive name to return. In other words, if a lookup of "foo" matches the vnode for "Foo", then a lookup of "Foo", "FOO", or "fOo" should match the same vnode. If FIGNORECASE is set and the file system does not support c-i, we return EINVAL. If FIGNORECASE is set and realpnp is non-NULL, then lookup fills in the pathname structure with the case-preserved name that was looked up. Also if FIGNORECASE is set, direntflagp is non-NULL and the file system supports per-directory-entry flags, then the flags will be returned in *direntflagp. If a file system supports c-i name matching and mixed case sensitivity, it should also support the per-directory-entry flags. Supporting the per-directory-entry flags is not mandatory, but if they are not supported, the consequence is that the consumer of fop_lookup() will be required to use brute-force techniques to manually determine if there are any case conflicts within the directory. As an example, a CIFS server might use the flags to know it needs to mangle a looked up name: error = VOP_LOOKUP(dvp, dname, &vp, FIGNORECASE, ...., &deflags, NULL); if (error == 0 && (deflags & ED_CASE_CONFLICT)) mname = mangle(name); Having the dirent flags available to a fop_lookup() caller also hints at intriguing possibilities for future flags. int fop_create( vnode_t *dvp, char *name, vattr_t *vap, vcexcl_t excl, int mode, vnode_t **vpp, cred_t *cr, int flag, /* New flag: FIGNORECASE */ caller_context *ctp, /* See PSARC/2007/218 */ vsecattr_t *vsecp) /* See PSARC/2007/227 */ If FIGNORECASE is set and file system supports c-i, then the object is created ONLY IF there is no object that has a c-i name match in the directory. If FIGNORECASE is set, and the file system supports c-i, but an object exists with a c-i name match exists, return EEXIST. If FIGNORECASE is set and the file system does *not* support c-i, then the fop_create returns EINVAL. The following naming routines need to have a "flags" field added. int fop_remove( vnode_t *dvp, char *nm, cred_t *cr, caller_context *ctp, /* See PSARC/2007/218 */ /* NEW */ int flags) /* Takes FIGNORECASE */ If FIGNORECASE is set and file system supports c-i, remove the name of the object with a c-i name match, if it exists. Note that in the presence of multiple case versions of a given name (e.g., "foo", "Foo", "FOO"), the behavior should be the same as fop_lookup(). That is, the c-i matching algorithm should be the same in both fop_lookup, fop_remove, fop_rename, and fop_rmdir. If FIGNORECASE is set and the file system does not support c-i, return EINVAL. int fop_link( vnode_t *tdvp, vnode_t *svp, char *tnm, cred_t *cr, caller_context *ctp, /* See PSARC/2007/218 */ /* NEW */ int flags) /* Takes FIGNORECASE */ If FIGNORECASE is set and file system supports c-i, then the link is created ONLY IF there is no object that has a c-i name match in the target directory. If FIGNORECASE is set and the file system does *not* support c-i, then return EINVAL. int fop_rename( vnode_t *sdvp, char *snm, vnode_t *tdvp, char *tnm, cred_t *cr, caller_context *ctp, /* See PSARC/2007/218 */ /* NEW */ int flags) /* Takes FIGNORECASE */ If FIGNORECASE is set and the file system supports c-i, then the object to be renamed shall match the same name that fop_lookup would match. That is, the c-i matching algorithm should be the same in both fop_lookup, fop_remove, fop_rename, and fop_rmdir. Same holds for the target name, if it exists. If FIGNORECASE is set and either file system does *not* support c-i, then we return EINVAL. int fop_mkdir( vnode_t *dvp, char *dirname, vattr_t *vap, vnode_t **vpp, cred_t *cr, caller_context *ctp, /* See PSARC/2007/218 */ /* NEW */ int flags, /* Takes FIGNORECASE */ vsecattr_t *vsecp) /* See PSARC/2007/227 */ If FIGNORECASE is set and file system supports c-i, then the object is created ONLY IF there is no object that has a c-i name match in the directory. If FIGNORECASE is set and the file system does *not* support c-i, then return EINVAL. int fop_rmdir( vnode_t *dvp, char *nm, vnode_t *cdir, cred_t *cr, caller_context *ctp, /* See PSARC/2007/218 */ /* NEW */ int flags) /* Takes FIGNORECASE */ If FIGNORECASE is set and file system supports c-i, remove the name of the object with a c-i name match, if it exists. Note that in the presence of multiple case versions of a given name (e.g., "foo", "Foo", "FOO"), the behavior should be the same as fop_lookup(). That is, the c-i matching algorithm should be the same in both fop_lookup, fop_remove, fop_rename, and fop_rmdir. If FIGNORECASE is set and the file system does not support c-i, return EINVAL. int fop_readdir( vnode_t *vp, uio_t *uiop, cred_t *cr, int *eofp, caller_context *ctp, /* See PSARC/2007/218 */ /* NEW */ int flags) /* Takes V_RDDIR_ENTFLAGS */ If "flags" has V_RDDIR_ENTFLAGS set and the file system supports per-directory-entry flags, then the uio structure will contain the following structure: typedef struct edirent { ino64_t ed_ino; /* "inode number" of entry */ off64_t ed_off; /* offset of disk directory entry */ uint32_t ed_eflags; /* per-entry flags */ unsigned short ed_reclen; /* length of this record */ char ed_name[1]; /* name of file */ } edirent_t; If "flags" has V_RDDIR_ENTFLAGS set and the file system does *not* support per-directory-entry flags, then fop_readdir() will return EINVAL. If "flags" does not have V_RDDIR_ENTFLAGS set then the uio structure will have the traditional dirent_t format. Note that if the file system supports per-directory-entry flags, then the file system must set VFSFT_DIRENTFLAGS using the VFS Feature Registration interface. Below is a simplified pseudo-code routine that uses the fop_readdir() interface to rewrite the names of case-insensitive conflicts: char ** readnames() { ... error = VOP_READDIR(vp, &uio, cr, &eof, ct, V_RDDIR_DIRENTFLAGS); if (error) return (error); compute # of entries and alloc rnames while entries remain and space left in rname array { // get ed_eflags from edirent structure if (ed_eflags & ED_CASE_CONFLICT) strcpy(rnames[n++], mangle(ed_name)); else strcpy(rnames[n++], ed_name); } return (rnames); } int fop_symlink( vnode_t *dvp, char *linkname, vattr_t *vap, char *target, cred_t *cr, caller_context *ctp, /* See PSARC/2007/218 */ /* NEW */ int flags) /* Takes FIGNORECASE */ If FIGNORECASE is set and file system supports c-i, then the symlink is created ONLY IF there is no object that has a c-i name match to the target name in the directory. If FIGNORECASE is set and the file system does *not* support c-i, then EINVAL is returned. CASE-INSENSITIVE BEHAVIOR SUPPORT IN ZFS FILE SYSTEMS We will modify ZFS so that a ZFS file system can support the case-insensitive behaviors required by the Solaris CIFS server. MAN PAGE MODIFICATIONS ---------------------------------------------------- The following changes apply to the zfs(1M) man page: ---------------------------------------------------- The following three properties cannot be changed once the file system has been created, and so should be set at file system creation time. If not set in the "zfs create" command these properties will be inherited from the parent dataset. If the parent lacks these properties due to having been created prior to these features being supported, the new file system will have the default values for these properties. utf8only = on | off This property indicates if the file system should reject file names including characters not present in the UTF-8 character code set. If this property is explicitly set to "off", the normalization property (below) must either not be explicitly set or be set to "none". The default value for the "utf8only" property is "off". This property cannot be changed once the file system has been created. normalization = none | formD | formKC This property indicates if the file system should perform a unicode normalization of file names whenever two file names are compared, and which normalization algorithm should be used. File names are always stored unmodified, names are normalized as part of any comparison process. If this property is set to a legal value other than "none", and the "utf8only" property was left unspecified, the "utf8only" property will automatically be set to "on". The default value of the "normalization" property is "none". This property cannot be changed once the file system has been created. casesensitivity = sensitive | insensitive | mixed This property indicates if the file name matching algorithm used by the file system should be case-sensitive, case-insensitive, or allow a combination of both styles of matching. The default value for the "casesensitivity" property is "sensitive". Traditionally, UNIX and POSIX file systems have case-sensitive file names. The "mixed" value for the "casesensitivity" property indicates the file system can support requests for both case-sensitive and case-insensitive matching behavior. Currently case-insensitive matching behavior on a file system that supports mixed behavior is limited to kernel modules. Accesses from a user process have no means to directly request case-insensitive behavior on these file systems. A user process can indirectly achieve case-insensitive access, though, through an intermediate kernel module explicitly requesting case-insensitive behavior, such as a CIFS server. When a case-insensitive matching request is made of a "mixed" sensitivity file system, the behavior is generally the same as would be expected of a purely case-insensitive file system. The difference is that a "mixed" sensitivity file system may contain directories with multiple names that are unique from a case-sensitive perspective, but not unique from the case-insensitive perspective. For example, a directory might contain files "foo", "Foo", and "FOO". If there is a request to case-insensitively match any of the possible forms of "foo", (for example "foo", "FOO", "FoO", "fOo", et cetera) one of the three existing files will be chosen as the match by the matching algorithm. Exactly which file the algorithm chooses as match is not guaranteed, but what is guaranteed is that the same file will be chosen as match for any of the forms of "foo". The file chosen as a case-insensitive match for "foo", "FOO", "foO", "Foo", et. cetera will always be the same, so long as the directory remains unchanged. Regardless of the "casesensitivity" property setting, the file system will preserve the case of the name specified to create a file. The "casesensitivity" property cannot be changed once the file system has been created. The "utf8only", "normalization", and "casesensitivity" properties will also become new permissions that can be assigned to non-privileged users via the ZFS delegated administration model PSARC/2006/465. -------------------------------------------------------------------- The following possible return value must be added to the open(2), symlink(2), link(2), unlink(2), rename(2), mkdir(2), and rmdir(2) man pages: -------------------------------------------------------------------- EILSEQ The path argument includes non-UTF8 characters and the file system only accepts file names where all characters are part of the UTF-8 character codeset. EXPORTED INTERFACES |Proposed |Specified | |Stability |in what | Interface Name |Classification |Document? | Comments =============================================================================== VFSFT_CASEINSENSITIVE |Consolidation |This | VFS Feature #define VFSFT_NOCASESENSITIVE |Private |Document | VFSFT_DIRENTFLAGS | | | ------------------------+---------------+---------------+---------------------- FIGNORECASE |Contracted |This | file.h #define |Consolidation |Document | |Private | | ------------------------+---------------+---------------+---------------------- V_RDDIR_ENTFLAGS |Contracted |This | vnode.h #define |Consolidation |Document | |Private | | ------------------------+---------------+---------------+---------------------- edirent_t |Contracted |This | Structure returned |Consolidation |Document | within uio buf of |Private | | readdir() caller | | | requesting | V_RDDIR_ENTFLAGS ------------------------+---------------+---------------+---------------------- ED_CASE_CONFLICT |Contracted |This | extdirent.h #define |Consolidation |Document | |Private | | | | | | ------------------------+---------------+---------------+---------------------- VOP_LOOKUP, fop_lookup |Contracted |This |New output parameters |Consolidation |Document |int *direntflagp |Private | |pathname_t *realpnp | | | VOP_CREATE, fop_create, | | |New input paramater VOP_REMOVE, fop_remove, | | |int flag VOP_LINK, fop_link, | | | VOP_RENAME, fop_rename, | | | VOP_MKDIR, fop_mkdir, | | | VOP_RMDIR, fop_rmdir, | | | VOP_READDIR,fop_readdir,| | | VOP_SYMLINK,fop_symlink | | | | | | ------------------------+---------------+---------------+---------------------- {CASE_BEHAVIOR}, |Stable |This | pathconf(2) variable _PC_CASE_BEHAVIOR | |Document | name and value | | | _CASE_INSENSITIVE | | | Bit values for _CASE_SENSITIVE | | | _PC_CASE_BEHAVIOR ------------------------+---------------+---------------+---------------------- utf8only |Evolving |This | zfs(1M) file system normalization | |Document | properties casesensitivity | | | ------------------------+---------------+---------------+---------------------- [ 1.8 Last update: 04/30/07 23:08:35 ] --Boundary_(ID_tHU6qEyNZE3hXrBwYKvxtQ)-- From sacadmin Wed May 2 09:13:53 2007 Received: from spartan.SFBay.Sun.COM (spartan [129.146.226.64]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l42GDrQi013716; Wed, 2 May 2007 09:13:53 -0700 (PDT) Received: from spartan.SFBay.Sun.COM (spartan.SFBay.Sun.COM [129.146.226.64]) by spartan.SFBay.Sun.COM (8.13.6+Sun/8.13.6) with SMTP id l42GCurN028084; Wed, 2 May 2007 09:12:56 -0700 (PDT) Message-Id: <200705021612.l42GCurN028084@spartan.SFBay.Sun.COM> Date: Wed, 2 May 2007 09:12:56 -0700 (PDT) From: Don Cragun Reply-To: Don Cragun Subject: Re: ZFS case-insensitive support [PSARC/2007/244 Timeout: 05/04/2007] To: PSARC@sac.sfbay.sun.com, maybee@sac.sfbay.sun.com Cc: Ienup.Sung@Sun.COM, Timothy.Haley@Sun.COM, cifs-vfs-team@Sun.COM MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: Jdxhu1X2CIkGEyL+1QZnpw== X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.5.5 SunOS 5.9 sun4u sparc Status: RO Content-Length: 9421 >Date: Fri, 27 Apr 2007 15:08:27 -0700 (PDT) >From: Mark Maybee ... ... ... > REQUIREMENTS > > 1) The CIFS server requires at least one file system type to support > case-preserving name creation and both case-sensitive (c-s) and > case-insensitive (c-i) behavior (a.k.a., mixed-behavior) on lookup. > > Solaris will support both the NFS and CIFS protocols. Windows > clients typically expect case insensitive behavior while NFS clients > typically expect case-sensitive behavior. I understand that the CIFS server needs to be able to request CI behavior. I agree that is would be good to be able to specify that a Windows server based filesystem that is being mounted on Solaris does not support CS behavior. Is there really a need to allow ZFS to NOT support CS behavior? (I.E., isn't mixed behavior sufficient without allowing CI only behavior?) > > 5) When requested, the VOP_READDIR interface must return information > for each directory entry to indicate if the entry is a > case-insensitive duplicate of another entry. The VOP_LOOKUP > interface must also return the same information. > > File creates done by a non-CIFS application may create file names > that are a case-insensitive duplicate of an existing file name. For > example, if "foo" exists then a non-CIFS application could create > the file "FOO", which is a case-insensitive duplicate. This > requirement states both VOP_READDIR and VOP_LOOKUP, when requested, > must provide the caller with an indication that a case-insensitive > conflict exists. The caller (the CIFS server) may choose to present > the name as a "mangled" name so that the user in the > case-insensitive environment can distinguish between c-i conflicting > file names. How do you gurantee that a "mangled" name doesn't conflict with another file's name? Does lookup demangle names? How does lookup know if a name has been mangled? > INTERFACE CHANGES NEEDED ... ... ... > CASE BEHAVIOR > > There are two forms of case behavior: Case-sensitive and > case-insensitive. Case sensitive behavior is what Solaris currently > supports: When an object is created with a name, the case of the name > is preserved. Objects with names that only differ in case (e.g., > "foo", "Foo", "FOO") can co-exist in the same directory. Lookups on a > particular name can only succeed if the case matches. > > Case-insensitive behavior, as required by the Solaris CIFS server, > follows a particular set of rules: > > Object Creation: Before a named object is created, the directory > must be searched for any c-i matches of that name. If there is a c-i > match, then the create operation must fail with EEXIST. Are you saying that under a mixed-mode mount point, a CS create has to fail if it creates a CI match? > > VFS INTERFACE CHANGES > > An interface is required to determine the case behavior of a > particular file system volume. There are three possible modes for > behavior: c-s only, c-i only, and mixed-mode (both c-s and c-i > requests are supported). > > The user-level interface is pathconf(2). The pathconf(2) interface > will have a new variable added: {CASE_BEHAVIOR}/_PC_CASE_BEHAVIOR. > > The pathconf(2) system call will return a value which represents > the bitwise OR of the following flags which indicate the case-behavior > that the file system supports: > > _CASE_INSENSITIVE This file system supports case-insensitive > behavior > _CASE_SENSITIVE This file system supports case-sensitive > behavior > > The fs_pathconf() routine (default routine for VOP_PATHCONF()) will be > modified to support _PC_CASE_BEHAVIOR. The default behavior for file > systems on Solaris is case-sensitive so a call to fs_pathconf() with > _PC_CASE_BEHAVIOR set as the cmd would set the _CASE_SENSITIVE bit. > > Any file system that supports anything but case-sensitive-only > behavior is required to use the (new) VFS Feature Registration > interfaces to register its case behavior. The following new VFS > features will be introduced to support case behavior: > > #define VFSFT_CASEINSENSITIVE 0x100000002 /* Supports case-insensitive */ > #define VFSFT_NOCASESENSITIVE 0x100000004 /* NOT case-sensitive */ Aren't "Not case-sensitive" and "case-insensitive" the same thing? Why isn't the last entry above changed to: #define VFSFT_CASESENSITIVE 0x100000004 /* Supoprts case-sensitive */ > #define VFSFT_DIRENTFLAGS 0x100000008 /* Supports dirent flags */ > > File systems that support case-insensitive behavior must register the > VFSFT_CASEINSENSITIVE feature. File systems that do *not* support > case-sensitive behavior must register the VFSFT_NOCASESENSITIVE > feature. > > VOP (Vnode Operation) INTERFACE CHANGES ... ... ... > > int fop_rename( > vnode_t *sdvp, > char *snm, > vnode_t *tdvp, > char *tnm, > cred_t *cr, > caller_context *ctp, /* See PSARC/2007/218 */ > /* NEW */ int flags) /* Takes FIGNORECASE */ > > If FIGNORECASE is set and the file system supports c-i, then > the object to be renamed shall match the same name that > fop_lookup would match. That is, the c-i matching algorithm > should be the same in both fop_lookup, fop_remove, fop_rename, > and fop_rmdir. Same holds for the target name, if it exists. If "foo" and "FOO" both exist and I attempt: rename("foo", "FOO"); is it a no-op, is an error returned, or is one of the files removed as the other is renamed? > > If FIGNORECASE is set and either file system does *not* support c-i, > then we return EINVAL. What do you mean "either file system"? The rename operation doesn't work across file system boundaries (EXDEV is the error for this case). > > CASE-INSENSITIVE BEHAVIOR SUPPORT IN ZFS FILE SYSTEMS > > We will modify ZFS so that a ZFS file system can support the > case-insensitive behaviors required by the Solaris CIFS server. > > MAN PAGE MODIFICATIONS > ... ... ... > > normalization = none | formD | formKC > > This property indicates if the file system should perform a unicode > normalization of file names whenever two file names are compared, > and which normalization algorithm should be used. File names are > always stored unmodified, names are normalized as part of any > comparison process. If this property is set to a legal value other > than "none", and the "utf8only" property was left unspecified, the > "utf8only" property will automatically be set to "on". The default > value of the "normalization" property is "none". This property > cannot be changed once the file system has been created. Will the formD and formKC normalizations be described somewhere? > > casesensitivity = sensitive | insensitive | mixed > > This property indicates if the file name matching algorithm used by > the file system should be case-sensitive, case-insensitive, or allow a > combination of both styles of matching. The default value for the > "casesensitivity" property is "sensitive". Traditionally, UNIX and > POSIX file systems have case-sensitive file names. > > The "mixed" value for the "casesensitivity" property indicates the > file system can support requests for both case-sensitive and > case-insensitive matching behavior. Currently case-insensitive > matching behavior on a file system that supports mixed behavior is > limited to kernel modules. Accesses from a user process have no > means to directly request case-insensitive behavior on these file > systems. A user process can indirectly achieve case-insensitive > access, though, through an intermediate kernel module explicitly > requesting case-insensitive behavior, such as a CIFS server. > > When a case-insensitive matching request is made of a "mixed" > sensitivity file system, the behavior is generally the same as would > be expected of a purely case-insensitive file system. The > difference is that a "mixed" sensitivity file system may contain > directories with multiple names that are unique from a > case-sensitive perspective, but not unique from the case-insensitive > perspective. For example, a directory might contain files "foo", > "Foo", and "FOO". If there is a request to case-insensitively match > any of the possible forms of "foo", (for example "foo", "FOO", > "FoO", "fOo", et cetera) one of the three existing files will be > chosen as the match by the matching algorithm. Exactly which file > the algorithm chooses as match is not guaranteed, but what is > guaranteed is that the same file will be chosen as match for any > of the forms of "foo". The file chosen as a case-insensitive match > for "foo", "FOO", "foO", "Foo", et. cetera will always be the same, > so long as the directory remains unchanged. > Why does ZFS ever need casesensitivity set to insensitive? Aren't sensitive and mixed sufficient? > > EXPORTED INTERFACES > > |Proposed |Specified | > |Stability |in what | >Interface Name |Classification |Document? | Comments >=============================================================================== > VFSFT_CASEINSENSITIVE |Consolidation |This | VFS Feature #define > VFSFT_NOCASESENSITIVE |Private |Document | See note above... VSFT_CASESENSITIVE? - Don From rlhamil@mindwarp.smart.net Wed May 2 19:24:52 2007 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l432OqG8019795 for ; Wed, 2 May 2007 19:24:52 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l432NsKS028469 for <@newsunmail1brm.central.sun.com:psarc-ext@sun.com>; Wed, 2 May 2007 19:23:55 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JHG00J071BU6800@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 02 May 2007 20:23:54 -0600 (MDT) Received: from sca-ea-mail-3.sun.com ([192.18.43.21]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JHG00B6S1BTE720@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 02 May 2007 20:23:53 -0600 (MDT) Received: from relay44i.sun.com ([192.5.209.118]) by sca-ea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l432Jawb023797 for ; Thu, 03 May 2007 02:23:53 +0000 (GMT) Received: from mms49es.sun.com ([160.41.221.233] [160.41.221.233]) by relay44i.sun.com with ESMTP id BT-MMP-440223 for psarc-ext@sun.com; Thu, 03 May 2007 02:23:53 +0000 (Z) Received: from relay42i.sun.com ([192.5.209.72] [192.5.209.72]) by mms49es.sun.com with ESMTP id BT-MMP-1408046 for psarc-ext@sun.com; Thu, 03 May 2007 02:23:52 +0000 (Z) Received: from alnrmhc11.comcast.net ([204.127.225.91] [204.127.225.91]) by relay4i.sun.com with ESMTP id BT-MMP-3656912 for psarc-ext@sun.com; Thu, 03 May 2007 02:23:52 +0000 (Z) Received: from mindwarp.smart.net (c-69-243-35-55.hsd1.md.comcast.net[69.243.35.55]) by comcast.net (alnrmhc11) with ESMTP id <20070503022351b1100ke980e>; Thu, 03 May 2007 02:23:51 +0000 Received: from paradox (paradox.smart.net [192.168.0.5]) by mindwarp.smart.net (8.13.8+Sun/8.13.7/SpamBlocker) with SMTP id l432Nops000782; Wed, 02 May 2007 22:23:50 -0400 (EDT) Date: Wed, 02 May 2007 22:23:50 -0400 (EDT) From: "Richard L. Hamilton" Subject: Re: 2007/244 [ZFS case-insensitive support] Sender: rlhamil@mindwarp.smart.net To: opensolaris-arc@opensolaris.org, psarc-ext@sun.com Reply-to: "Richard L. Hamilton" Message-id: <200705030223.l432Nops000782@mindwarp.smart.net> MIME-version: 1.0 X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.7_41 SunOS 5.11 sun4u sparc Content-type: TEXT/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-MD5: 6WUFWsHccnbgeu+2bw6+MA== X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 3667 I'm told that because I posted this via the Jive web interface on os.o, it didn't make it everywhere it should, so here goes again. Apologies to those who have already seen it. ------------- Begin Forwarded Message ------------- I'd like to see at least some consideration of the idea that whenever a disambiguation was done, the chosen directory entry would be (somehow) persistently flagged as the preferred case-insensitive match, so that as long as it was not deleted or renamed, it would continue to be the chosen one regardless of any other changes made to the directory. A directory entry created by a case-insensitive request should also be thus flagged. This does imply that ideally, at least one flag bit per directory entry (not per inode or equivalent, given hard links) be available or added to support such a capability; but if not inordinately expensive (and if there was no backwards-compatibility issue involved, which might be feasible with zfs's versioning of on-disk structure), I think that would give far and away the most civilized behavior in a mixed environment. And since in practice at least one byte per directory entry would be used for the flag bit, that leaves 7 others for future interesting purposes (i.e. anything where special handling of a directory entry rather than of the underlying inode was needed). It also seems to me that a rule for disambiguating among existing names may not be quite the same situation as one permitting the creation of a new name; do you really want to blow away "foo" given a case-insensitive request to create "Foo"? Would that be more likely to be an accidental data loss, or an intentional act? Another (even if cosmetic) problem with mixed use is that some clients may expect case-insensitive but case-preserving behavior, which might not (no pun intended) always be the case when disambiguation came into play; if e.g. SMB protocol allows creation of a file in a non-exclusive, non-truncating mode, how would Windows handle that; could one behave consistently with however Windows (as the prototypical SMB server) does that while not running into other troublesome situations? And of course the old standby question applies too, "how does Linux handle this sort of thing"? The whole mixed situation looks quite scary to me, without more effort than the case makes to keep it as unsurprising as possible, especially if there were active mixed modification of a directory. Since the choice of case-insensitive, mixed, or case-sensitive is fixed for the life of the filesystem, how would one get around that, or would one have to create a new filesystem and copy everything over? Could one set that property at the creation of a clone say, and then juggle things so as to ultimately remove the original filesystem and have the clone take its place (to avoid actually copying all the data)? I do tend to agree with the notion that it should stay fixed, even if it were technically feasible to make that property changeable, because changing the property on the fly would be very confusing to programs accessing and updating the filesystem. But there will be times people will want to effectively make the change nonetheless (even if not quite online, so to speak), so documentation should probably reflect a way to work around it. I think these and probably more issues need to be considered, or if they have been, there needs to be more rationale given for the specifics described. ------------- End Forwarded Message ------------- -- mail: mailto:rlhamil@smart.net Home page: http://www.smart.net/~rlhamil MySpace: http://www.myspace.com/yesterdays_geek AIM, Yahoo, etc: ask From Timothy.Haley@sun.com Thu May 3 10:21:44 2007 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l43HLiFF021221 for ; Thu, 3 May 2007 10:21:44 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l43HKgTc027351 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Thu, 3 May 2007 10:20:46 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JHH00D1D6UL5X00@nwk-avmta-1.sfbay.Sun.COM> for psarc-ext@sun.com (ORCPT psarc-ext@Sun.COM); Thu, 03 May 2007 10:20:45 -0700 (PDT) Received: from brmea-mail-3.sun.com ([192.18.98.34]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JHH004TT6UIBW90@nwk-avmta-1.sfbay.Sun.COM> for psarc-ext@sun.com (ORCPT psarc-ext@Sun.COM); Thu, 03 May 2007 10:20:42 -0700 (PDT) Received: from fe-amer-10.sun.com ([192.18.108.184]) by brmea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l43HKgB1014154 for ; Thu, 03 May 2007 17:20:42 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JHH00J016SV4M00@mail-amer.sun.com> (original mail from Timothy.Haley@Sun.COM) for psarc-ext@Sun.COM (ORCPT psarc-ext@Sun.COM); Thu, 03 May 2007 11:20:42 -0600 (MDT) Received: from [172.20.25.27] by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JHH004V36UE6FC2@mail-amer.sun.com> for psarc-ext@Sun.COM (ORCPT psarc-ext@Sun.COM); Thu, 03 May 2007 11:20:42 -0600 (MDT) Date: Thu, 03 May 2007 11:20:38 -0600 From: Timothy Haley Subject: Re: 2007/244 [ZFS case-insensitive support] In-reply-to: <200705030223.l432Nops000782@mindwarp.smart.net> Sender: Timothy.Haley@sun.com To: "Richard L. Hamilton" Cc: psarc-ext@sun.com Message-id: <463A19E6.1070400@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200705030223.l432Nops000782@mindwarp.smart.net> User-Agent: Thunderbird 2.0b2 (X11/20070227) Status: RO Content-Length: 5874 Richard, Thanks for your comments, more inline below. Richard L. Hamilton wrote: > I'm told that because I posted this via the Jive web interface on os.o, it > didn't make it everywhere it should, so here goes again. Apologies to those > who have already seen it. > > ------------- Begin Forwarded Message ------------- > > I'd like to see at least some consideration of the idea that whenever a > disambiguation was done, the chosen directory entry would be (somehow) > persistently flagged as the preferred case-insensitive match, so that as long as > it was not deleted or renamed, it would continue to be the chosen one regardless > of any other changes made to the directory. > Under the CASE BEHAVIOR section we say this Lookup: In mixed-mode (volume supports both c-i and c-s behavior), it's possible for a c-s client to create names that are c-i clashes. For example, it may be possible for "foo", "Foo", and "FOO" to exist in the same directory. In those cases, a c-i lookup of any c-s form of "foo" (e.g., "foo", "FOO", "FoO", "fOo", ad nauseum) should always return the same name. In other words, if a c-i lookup of "foo" returns "foo", then a c-i lookup of "FOO" or "Foo" should also return "foo". It doesn't matter what the algorithm is as long as it is consistent. > case-insensitive request should also be thus flagged. This does imply that > ideally, at least one flag bit per directory entry (not per inode or equivalent, > given hard links) be available or added to support such a capability; but if not > inordinately expensive (and if there was no backwards-compatibility issue > involved, which might be feasible with zfs's versioning of on-disk structure), I > think that would give far and away the most civilized behavior in a mixed > environment. And since in practice at least one byte per directory entry would > be used for the flag bit, that leaves 7 others for future interesting purposes > (i.e. anything where special handling of a directory entry rather than of the > underlying inode was needed). > This sounds like you are delving into the file system implementation details to me. Requiring a flag be set on the entry that's the "preferred" match might be one method of insuring the behavior that's desired. It might also be troublesome to keep updated - if there are 100 case conflicts in a directory and the "preferred" match gets deleted, how and when do you determine the new "preferred" match and mark it? Another question - are you suggesting this bit has to be exposed in the direntflags? To what end? Do you have a particular problem you're trying to solve? > It also seems to me that a rule for disambiguating among existing names > may not be quite the same situation as one permitting the creation of a > new name; do you really want to blow away "foo" given a case-insensitive request > to create "Foo"? Would that be more likely to be an accidental > data loss, or an intentional act? > I think if the algorithm worked this way, a case-insensitive client could create case-sensitive conflicts, not behavior we expect or want from case-insensitive consumers of the interface. Intent isn't always so easy to discern, if you 'vi Foo' and you are a case insensitive client you probably want to edit the existing file, not create a new one. I think it's better to have consistent behavior for lookups regardless of whether or not creation is involved. > Another (even if cosmetic) problem with mixed use is that some clients > may expect case-insensitive but case-preserving behavior, which might not > (no pun intended) always be the case when disambiguation came into play; > if e.g. SMB protocol allows creation of a file in a non-exclusive, > non-truncating > mode, how would Windows handle that; could one behave consistently with > however Windows (as the prototypical SMB server) does that while not > running into other troublesome situations? And of course the old standby > question applies too, "how does Linux handle this sort of thing"? > When new files are created, we preserve the case of the filename given. I'm not sure I quite follow what you mean otherwise by "case-insensitive but case-preserving behavior", I could almost rewrite that as "case-insensitive but case-sensitive behavior". > The whole mixed situation looks quite scary to me, without more effort than > the case makes to keep it as unsurprising as possible, especially if there > were active mixed modification of a directory. > Yes, mixed use is a bit odd. We're putting it forward because being able to share a file system case-insensitive with CIFS and at the same time case-sensitive with NFS is a feature desired by some administrators. Personally, I think it's saner to choose purely case-sensitive or purely case-insensitive behavior, but that's just my opinion. > Since the choice of case-insensitive, mixed, or case-sensitive is fixed for > the life of the filesystem, how would one get around that, or would one > have to create a new filesystem and copy everything over? Could one > set that property at the creation of a clone say, and then juggle things so > as to ultimately remove the original filesystem and have the clone take > its place (to avoid actually copying all the data)? I do tend to agree with > the notion that it should stay fixed, even if it were technically feasible to > make that property changeable, because changing the property on the fly > would be very confusing to programs accessing and updating the filesystem. > But there will be times people will want to effectively make the change > nonetheless (even if not quite online, so to speak), so documentation should > probably reflect a way to work around it. > Snapshots and clones will inherit the sensitivity properties of the source file system. -tim From Timothy.Haley@Sun.COM Thu May 3 12:30:50 2007 Received: from sfbaymail2sca.sfbay.sun.com (sfbaymail2sca.SFBay.Sun.COM [129.145.155.42]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l43JUoZ4027724; Thu, 3 May 2007 12:30:50 -0700 (PDT) Received: from brmea-mail-4.sun.com (brmea-mail-4.Sun.COM [192.18.98.36]) by sfbaymail2sca.sfbay.sun.com (8.13.6+Sun/8.12.10/ENSMAIL,v2.2) with ESMTP id l43JTqnE021035; Thu, 3 May 2007 12:29:52 -0700 (PDT) Received: from fe-amer-04.sun.com ([192.18.108.178]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l43JTq7Q000620; Thu, 3 May 2007 19:29:52 GMT Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JHH00D01BUDU500@mail-amer.sun.com> (original mail from Timothy.Haley@Sun.COM); Thu, 03 May 2007 13:29:52 -0600 (MDT) Received: from [172.20.25.27] by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JHH00D3NCTRENE0@mail-amer.sun.com>; Thu, 03 May 2007 13:29:51 -0600 (MDT) Date: Thu, 03 May 2007 13:29:51 -0600 From: Timothy Haley Subject: Re: ZFS case-insensitive support [PSARC/2007/244 Timeout: 05/04/2007] In-reply-to: <200705021612.l42GCurN028084@spartan.SFBay.Sun.COM> Sender: Timothy.Haley@Sun.COM To: Don Cragun Cc: psarc-ext@sac.sfbay.sun.com, maybee@sac.sfbay.sun.com, Ienup.Sung@Sun.COM, cifs-vfs-team@Sun.COM Message-id: <463A382F.6070003@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200705021612.l42GCurN028084@spartan.SFBay.Sun.COM> User-Agent: Thunderbird 2.0b2 (X11/20070227) Status: RO Content-Length: 11392 Don Cragun wrote: >> Date: Fri, 27 Apr 2007 15:08:27 -0700 (PDT) >> From: Mark Maybee > ... ... ... >> REQUIREMENTS >> >> 1) The CIFS server requires at least one file system type to support >> case-preserving name creation and both case-sensitive (c-s) and >> case-insensitive (c-i) behavior (a.k.a., mixed-behavior) on lookup. >> >> Solaris will support both the NFS and CIFS protocols. Windows >> clients typically expect case insensitive behavior while NFS clients >> typically expect case-sensitive behavior. > > I understand that the CIFS server needs to be able to request CI > behavior. I agree that is would be good to be able to specify that a > Windows server based filesystem that is being mounted on Solaris does > not support CS behavior. Is there really a need to allow ZFS to NOT > support CS behavior? (I.E., isn't mixed behavior sufficient without > allowing CI only behavior?) > Supporting pure c-i behavior falls out nicely from our work to support mixed sensitivity. Some folks like it, I believe for example macOS HFS+ file systems are case-insensitive by default. It seems reasonable to offer it as a choice. I'm also fairly certain the CIFS server will happily share a pure case-insensitive file system. >> >> 5) When requested, the VOP_READDIR interface must return information >> for each directory entry to indicate if the entry is a >> case-insensitive duplicate of another entry. The VOP_LOOKUP >> interface must also return the same information. >> >> File creates done by a non-CIFS application may create file names >> that are a case-insensitive duplicate of an existing file name. For >> example, if "foo" exists then a non-CIFS application could create >> the file "FOO", which is a case-insensitive duplicate. This >> requirement states both VOP_READDIR and VOP_LOOKUP, when requested, >> must provide the caller with an indication that a case-insensitive >> conflict exists. The caller (the CIFS server) may choose to present >> the name as a "mangled" name so that the user in the >> case-insensitive environment can distinguish between c-i conflicting >> file names. > > How do you gurantee that a "mangled" name doesn't conflict with another > file's name? Does lookup demangle names? How does lookup know if a > name has been mangled? > I don't know what the mangling algorithm is. Mangling is a function of the CIFS server, not any of the interfaces we are defining here. > >> INTERFACE CHANGES NEEDED > ... ... ... >> CASE BEHAVIOR >> >> There are two forms of case behavior: Case-sensitive and >> case-insensitive. Case sensitive behavior is what Solaris currently >> supports: When an object is created with a name, the case of the name >> is preserved. Objects with names that only differ in case (e.g., >> "foo", "Foo", "FOO") can co-exist in the same directory. Lookups on a >> particular name can only succeed if the case matches. >> >> Case-insensitive behavior, as required by the Solaris CIFS server, >> follows a particular set of rules: >> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ We're describing the rules for c-i behavior, so... >> Object Creation: Before a named object is created, the directory >> must be searched for any c-i matches of that name. If there is a c-i >> match, then the create operation must fail with EEXIST. > > Are you saying that under a mixed-mode mount point, a CS create has to > fail if it creates a CI match? > Not at all. A CI create has to fail. >> >> VFS INTERFACE CHANGES >> >> An interface is required to determine the case behavior of a >> particular file system volume. There are three possible modes for >> behavior: c-s only, c-i only, and mixed-mode (both c-s and c-i >> requests are supported). >> >> The user-level interface is pathconf(2). The pathconf(2) interface >> will have a new variable added: {CASE_BEHAVIOR}/_PC_CASE_BEHAVIOR. >> >> The pathconf(2) system call will return a value which represents >> the bitwise OR of the following flags which indicate the case-behavior >> that the file system supports: >> >> _CASE_INSENSITIVE This file system supports case-insensitive >> behavior >> _CASE_SENSITIVE This file system supports case-sensitive >> behavior >> >> The fs_pathconf() routine (default routine for VOP_PATHCONF()) will be >> modified to support _PC_CASE_BEHAVIOR. The default behavior for file >> systems on Solaris is case-sensitive so a call to fs_pathconf() with >> _PC_CASE_BEHAVIOR set as the cmd would set the _CASE_SENSITIVE bit. >> >> Any file system that supports anything but case-sensitive-only >> behavior is required to use the (new) VFS Feature Registration >> interfaces to register its case behavior. The following new VFS >> features will be introduced to support case behavior: >> >> #define VFSFT_CASEINSENSITIVE 0x100000002 /* Supports case-insensitive */ >> #define VFSFT_NOCASESENSITIVE 0x100000004 /* NOT case-sensitive */ > > Aren't "Not case-sensitive" and "case-insensitive" the same thing? Why isn't > the last entry above changed to: > #define VFSFT_CASESENSITIVE 0x100000004 /* Supoprts case-sensitive */ > A purely case-insensitive file system supports both VFSFT_CASEINSENSITIVE and VFSFT_NOCASESENSITIVE features. A mixed sensitivity file system supports just the VFSFT_CASEINSENSITIVE feature. A case sensitive file system, our beloved existing behavior on most file systems, has neither of these features. >> #define VFSFT_DIRENTFLAGS 0x100000008 /* Supports dirent flags */ >> >> File systems that support case-insensitive behavior must register the >> VFSFT_CASEINSENSITIVE feature. File systems that do *not* support >> case-sensitive behavior must register the VFSFT_NOCASESENSITIVE >> feature. >> >> VOP (Vnode Operation) INTERFACE CHANGES > ... ... ... >> >> int fop_rename( >> vnode_t *sdvp, >> char *snm, >> vnode_t *tdvp, >> char *tnm, >> cred_t *cr, >> caller_context *ctp, /* See PSARC/2007/218 */ >> /* NEW */ int flags) /* Takes FIGNORECASE */ >> >> If FIGNORECASE is set and the file system supports c-i, then >> the object to be renamed shall match the same name that >> fop_lookup would match. That is, the c-i matching algorithm >> should be the same in both fop_lookup, fop_remove, fop_rename, >> and fop_rmdir. Same holds for the target name, if it exists. > > If "foo" and "FOO" both exist and I attempt: > rename("foo", "FOO"); > is it a no-op, is an error returned, or is one of the files removed as > the other is renamed? > I'm assuming you mean for this rename to also have the FIGNORECASE set. This is a good question. Interesting corner case. I think one could argue either way. At this very instant I can say that this rename is a no-op. I need to consider this and get some feedback from CIFs folks, though if that is correct. >> >> If FIGNORECASE is set and either file system does *not* support c-i, >> then we return EINVAL. > > What do you mean "either file system"? The rename operation doesn't > work across file system boundaries (EXDEV is the error for this case). > That's an error I thought I had removed from the text. Sorry, I will fix that. It should and will read: If FIGNORECASE is set and the file system does *not* support c-i, then we return EINVAL. >> >> CASE-INSENSITIVE BEHAVIOR SUPPORT IN ZFS FILE SYSTEMS >> >> We will modify ZFS so that a ZFS file system can support the >> case-insensitive behaviors required by the Solaris CIFS server. >> >> MAN PAGE MODIFICATIONS >> > ... ... ... >> >> normalization = none | formD | formKC >> >> This property indicates if the file system should perform a unicode >> normalization of file names whenever two file names are compared, >> and which normalization algorithm should be used. File names are >> always stored unmodified, names are normalized as part of any >> comparison process. If this property is set to a legal value other >> than "none", and the "utf8only" property was left unspecified, the >> "utf8only" property will automatically be set to "on". The default >> value of the "normalization" property is "none". This property >> cannot be changed once the file system has been created. > > Will the formD and formKC normalizations be described somewhere? > These are the forms supported by u8_strcmp() and u8_textprep_str(). See PSARC 2007/149. >> >> casesensitivity = sensitive | insensitive | mixed >> >> This property indicates if the file name matching algorithm used by >> the file system should be case-sensitive, case-insensitive, or allow a >> combination of both styles of matching. The default value for the >> "casesensitivity" property is "sensitive". Traditionally, UNIX and >> POSIX file systems have case-sensitive file names. >> >> The "mixed" value for the "casesensitivity" property indicates the >> file system can support requests for both case-sensitive and >> case-insensitive matching behavior. Currently case-insensitive >> matching behavior on a file system that supports mixed behavior is >> limited to kernel modules. Accesses from a user process have no >> means to directly request case-insensitive behavior on these file >> systems. A user process can indirectly achieve case-insensitive >> access, though, through an intermediate kernel module explicitly >> requesting case-insensitive behavior, such as a CIFS server. >> >> When a case-insensitive matching request is made of a "mixed" >> sensitivity file system, the behavior is generally the same as would >> be expected of a purely case-insensitive file system. The >> difference is that a "mixed" sensitivity file system may contain >> directories with multiple names that are unique from a >> case-sensitive perspective, but not unique from the case-insensitive >> perspective. For example, a directory might contain files "foo", >> "Foo", and "FOO". If there is a request to case-insensitively match >> any of the possible forms of "foo", (for example "foo", "FOO", >> "FoO", "fOo", et cetera) one of the three existing files will be >> chosen as the match by the matching algorithm. Exactly which file >> the algorithm chooses as match is not guaranteed, but what is >> guaranteed is that the same file will be chosen as match for any >> of the forms of "foo". The file chosen as a case-insensitive match >> for "foo", "FOO", "foO", "Foo", et. cetera will always be the same, >> so long as the directory remains unchanged. >> > > Why does ZFS ever need casesensitivity set to insensitive? Aren't > sensitive and mixed sufficient? > You would set it to insensitive if you wanted case-insensitive behavior without the harrowing implications of mixed sensitivity. >> EXPORTED INTERFACES >> >> |Proposed |Specified | >> |Stability |in what | >> Interface Name |Classification |Document? | Comments >> =============================================================================== >> VFSFT_CASEINSENSITIVE |Consolidation |This | VFS Feature #define >> VFSFT_NOCASESENSITIVE |Private |Document | > > See note above... VSFT_CASESENSITIVE? > > - Don > -tim From don.cragun@Sun.COM Fri May 4 16:09:41 2007 Received: from spartan.SFBay.Sun.COM (spartan [129.146.226.64]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l44N9ftD022672; Fri, 4 May 2007 16:09:41 -0700 (PDT) Received: from spartan.SFBay.Sun.COM (spartan.SFBay.Sun.COM [129.146.226.64]) by spartan.SFBay.Sun.COM (8.13.6+Sun/8.13.6) with SMTP id l44N8hvL004771; Fri, 4 May 2007 16:08:43 -0700 (PDT) Message-Id: <200705042308.l44N8hvL004771@spartan.SFBay.Sun.COM> Date: Fri, 4 May 2007 16:08:43 -0700 (PDT) From: Don Cragun Reply-To: Don Cragun Subject: Re: ZFS case-insensitive support [PSARC/2007/244 Timeout: 05/04/2007] To: Timothy.Haley@Sun.COM Cc: psarc-ext@sac.sfbay.sun.com, maybee@sac.sfbay.sun.com, Ienup.Sung@Sun.COM, cifs-vfs-team@Sun.COM MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: oFER36CivFqbjsYb5KpojA== X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.5.5 SunOS 5.9 sun4u sparc Status: RO Content-Length: 10141 >Date: Thu, 03 May 2007 13:29:51 -0600 >From: Timothy Haley > >Don Cragun wrote: >>> Date: Fri, 27 Apr 2007 15:08:27 -0700 (PDT) >>> From: Mark Maybee >> ... ... ... >>> REQUIREMENTS >>> >>> 1) The CIFS server requires at least one file system type to support >>> case-preserving name creation and both case-sensitive (c-s) and >>> case-insensitive (c-i) behavior (a.k.a., mixed-behavior) on lookup. >>> >>> Solaris will support both the NFS and CIFS protocols. Windows >>> clients typically expect case insensitive behavior while NFS clients >>> typically expect case-sensitive behavior. >> >> I understand that the CIFS server needs to be able to request CI >> behavior. I agree that is would be good to be able to specify that a >> Windows server based filesystem that is being mounted on Solaris does >> not support CS behavior. Is there really a need to allow ZFS to NOT >> support CS behavior? (I.E., isn't mixed behavior sufficient without >> allowing CI only behavior?) >> > >Supporting pure c-i behavior falls out nicely from our work to support >mixed sensitivity. Some folks like it, I believe for example macOS HFS+ >file systems are case-insensitive by default. It seems reasonable to >offer it as a choice. I'm also fairly certain the CIFS server will >happily share a pure case-insensitive file system. I understand that a CIFS server only needs CI behavior, but it sounds like a CIFS server will also be happy with mixed behavior. However, other applications running on a Solaris system all expect CS behavior not only for binary compatibility with prior Solaris releases but also because it is required by all of the standards mentioned on our standards(5) man page. It looks like those applications should continue to work (without modification) on a file system providing either CS or mixed behavior; but many will fail unexpectedly on a file system providing CI behavior. I understand that it is easy to provide this extra rope to allow applications to hang themselves, but it seems that doing so is creating an attractive nuisance. Is it really a good idea for a sysadmin to be able to mount a ZFS file system in CI mode rather than just in CS or in mixed mode? > >>> >>> 5) When requested, the VOP_READDIR interface must return information >>> for each directory entry to indicate if the entry is a >>> case-insensitive duplicate of another entry. The VOP_LOOKUP >>> interface must also return the same information. >>> >>> File creates done by a non-CIFS application may create file names >>> that are a case-insensitive duplicate of an existing file name. For >>> example, if "foo" exists then a non-CIFS application could create >>> the file "FOO", which is a case-insensitive duplicate. This >>> requirement states both VOP_READDIR and VOP_LOOKUP, when requested, >>> must provide the caller with an indication that a case-insensitive >>> conflict exists. The caller (the CIFS server) may choose to present >>> the name as a "mangled" name so that the user in the >>> case-insensitive environment can distinguish between c-i conflicting >>> file names. >> >> How do you gurantee that a "mangled" name doesn't conflict with another >> file's name? Does lookup demangle names? How does lookup know if a >> name has been mangled? >> > >I don't know what the mangling algorithm is. Mangling is a function of >the CIFS server, not any of the interfaces we are defining here. So, I login to a windows box and create a file in my home directory on a Solaris system through a CIFS server. Then I login on my Solaris system and look for that file in my home directory. I do an `ls -l` and find a file created at about the right time and that has about the right size, but with a name I don't recognize. Is the intent that files created by a CIFS server are only accessible through a CIFS server? > >> >>> INTERFACE CHANGES NEEDED >> ... ... ... >>> CASE BEHAVIOR >>> >>> There are two forms of case behavior: Case-sensitive and >>> case-insensitive. Case sensitive behavior is what Solaris currently >>> supports: When an object is created with a name, the case of the name >>> is preserved. Objects with names that only differ in case (e.g., >>> "foo", "Foo", "FOO") can co-exist in the same directory. Lookups on a >>> particular name can only succeed if the case matches. >>> >>> Case-insensitive behavior, as required by the Solaris CIFS server, >>> follows a particular set of rules: >>> > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > We're describing the rules for c-i behavior, so... > >>> Object Creation: Before a named object is created, the directory >>> must be searched for any c-i matches of that name. If there is a c-i >>> match, then the create operation must fail with EEXIST. >> >> Are you saying that under a mixed-mode mount point, a CS create has to >> fail if it creates a CI match? >> >Not at all. A CI create has to fail. It is clear that a CI create has to fail. From the text above it is not clear to me that a CS create is not also allowed (or even required) to fail on a FS mounted in mixed mode. > >>> >>> VFS INTERFACE CHANGES >>> >>> An interface is required to determine the case behavior of a >>> particular file system volume. There are three possible modes for >>> behavior: c-s only, c-i only, and mixed-mode (both c-s and c-i >>> requests are supported). >>> >>> The user-level interface is pathconf(2). The pathconf(2) interface >>> will have a new variable added: {CASE_BEHAVIOR}/_PC_CASE_BEHAVIOR. >>> >>> The pathconf(2) system call will return a value which represents >>> the bitwise OR of the following flags which indicate the case-behavior >>> that the file system supports: >>> >>> _CASE_INSENSITIVE This file system supports case-insensitive >>> behavior >>> _CASE_SENSITIVE This file system supports case-sensitive >>> behavior >>> >>> The fs_pathconf() routine (default routine for VOP_PATHCONF()) will be >>> modified to support _PC_CASE_BEHAVIOR. The default behavior for file >>> systems on Solaris is case-sensitive so a call to fs_pathconf() with >>> _PC_CASE_BEHAVIOR set as the cmd would set the _CASE_SENSITIVE bit. >>> >>> Any file system that supports anything but case-sensitive-only >>> behavior is required to use the (new) VFS Feature Registration >>> interfaces to register its case behavior. The following new VFS >>> features will be introduced to support case behavior: >>> >>> #define VFSFT_CASEINSENSITIVE 0x100000002 /* Supports case-insensitive */ >>> #define VFSFT_NOCASESENSITIVE 0x100000004 /* NOT case-sensitive */ >> >> Aren't "Not case-sensitive" and "case-insensitive" the same thing? Why isn't >> the last entry above changed to: >> #define VFSFT_CASESENSITIVE 0x100000004 /* Supoprts case-sensitive */ >> > >A purely case-insensitive file system supports both >VFSFT_CASEINSENSITIVE and VFSFT_NOCASESENSITIVE features. A mixed >sensitivity file system supports just the VFSFT_CASEINSENSITIVE feature. > A case sensitive file system, our beloved existing behavior on most >file systems, has neither of these features. I don't get that from the VFSFT_names above and I don't get that from the comments on these #defines. In the dictionary I have, "not XXX-sensitive" and "XXX-insensitive" are synonyms. I guess the comments below (which I didn't notice when I first commented) make it clear, but different names and/or comments above would probably make this clearer. > >>> #define VFSFT_DIRENTFLAGS 0x100000008 /* Supports dirent flags */ >>> >>> File systems that support case-insensitive behavior must register the >>> VFSFT_CASEINSENSITIVE feature. File systems that do *not* support >>> case-sensitive behavior must register the VFSFT_NOCASESENSITIVE >>> feature. >>> >>> VOP (Vnode Operation) INTERFACE CHANGES >> ... ... ... >>> >>> int fop_rename( >>> vnode_t *sdvp, >>> char *snm, >>> vnode_t *tdvp, >>> char *tnm, >>> cred_t *cr, >>> caller_context *ctp, /* See PSARC/2007/218 */ >>> /* NEW */ int flags) /* Takes FIGNORECASE */ >>> >>> If FIGNORECASE is set and the file system supports c-i, then >>> the object to be renamed shall match the same name that >>> fop_lookup would match. That is, the c-i matching algorithm >>> should be the same in both fop_lookup, fop_remove, fop_rename, >>> and fop_rmdir. Same holds for the target name, if it exists. >> >> If "foo" and "FOO" both exist and I attempt: >> rename("foo", "FOO"); >> is it a no-op, is an error returned, or is one of the files removed as >> the other is renamed? >> > >I'm assuming you mean for this rename to also have the FIGNORECASE set. Yes, but I don't know how this happens. From what I have read in this case, we will still just have one rename() system call and it doesn't have a flags argument; it just has two pathnames as arguments. How does an application set or clear the FIGNORECASE flag when it calls rename()? I assume that when an app calls open() and is creating a file there will be an O_ flag added that corresponds to FIGNORECASE (but no O_ flags seem to be added here); but I don't see anything that says how you would attach this flag to an open() of an existing file or any of the other syscalls that perform lookups when they aren't creating a file. > This is a good question. Interesting corner case. I think one could >argue either way. At this very instant I can say that this rename is a >no-op. I need to consider this and get some feedback from CIFs folks, >though if that is correct. Whatever you want to happen needs to be defined. (And, I would really like to see some rationale for the decision.) In general I assume that if a CI match for "src" is present but there is no CI match for "DeSt", then case will be preserved on DeSt if I rename("src", "DeSt"). Is this correct? - Don > ... ... ... >> >> - Don >> > >-tim From eschrock@zion.eng.sun.com Sun May 6 10:29:59 2007 Received: from zion.eng.sun.com (zion [129.146.17.75]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l46HTxdj003797; Sun, 6 May 2007 10:29:59 -0700 (PDT) Received: from zion.eng.sun.com (localhost [127.0.0.1]) by zion.eng.sun.com (8.13.7+Sun/8.13.7) with ESMTP id l46HSwZG001637; Sun, 6 May 2007 10:28:58 -0700 (PDT) Received: (from eschrock@localhost) by zion.eng.sun.com (8.13.7+Sun/8.13.7/Submit) id l46HSwmo001636; Sun, 6 May 2007 10:28:58 -0700 (PDT) Date: Sun, 6 May 2007 10:28:58 -0700 From: Eric Schrock To: Don Cragun Cc: Timothy.Haley@sun.com, psarc-ext@sac.sfbay.sun.com, maybee@sac.sfbay.sun.com, Ienup.Sung@sun.com, cifs-vfs-team@sun.com Subject: Re: ZFS case-insensitive support [PSARC/2007/244 Timeout: 05/04/2007] Message-ID: <20070506172858.GA1557@eng.sun.com> References: <200705042308.l44N8hvL004771@spartan.SFBay.Sun.COM> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200705042308.l44N8hvL004771@spartan.SFBay.Sun.COM> User-Agent: Mutt/1.4.2.1i Status: RO Content-Length: 1656 On Fri, May 04, 2007 at 04:08:43PM -0700, Don Cragun wrote: > > I understand that a CIFS server only needs CI behavior, but it sounds > like a CIFS server will also be happy with mixed behavior. However, > other applications running on a Solaris system all expect CS behavior > not only for binary compatibility with prior Solaris releases but also > because it is required by all of the standards mentioned on our > standards(5) man page. It looks like those applications should > continue to work (without modification) on a file system providing > either CS or mixed behavior; but many will fail unexpectedly on a file > system providing CI behavior. I understand that it is easy to provide > this extra rope to allow applications to hang themselves, but it seems > that doing so is creating an attractive nuisance. Is it really a good > idea for a sysadmin to be able to mount a ZFS file system in CI mode > rather than just in CS or in mixed mode? > Yes, this is a requirement in a mixed protocol environment, where POSIX standards are not required or necessarily desired. As there will always be side effects in a mixed case environment (name mangling, etc), an administrator in a predominantly CIFS environment may choose that preserving true case insensitive behavior for CIFS clients is more important than unexpected behavior on a small number of NFS clients. This will not be the default, and its effects will be well documented. Simply not providing this functionality because it is an "attractive nuisance" doesn't seem like the right approach. - Eric -- Eric Schrock, Solaris Kernel Development http://blogs.sun.com/eschrock From Timothy.Haley@Sun.COM Mon May 7 15:26:27 2007 Received: from sfbaymail1sca.SFBay.Sun.COM (sfbaymail1sca.SFBay.Sun.COM [129.145.154.35]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l47MQRef004212; Mon, 7 May 2007 15:26:27 -0700 (PDT) Received: from brmea-mail-2.sun.com (brmea-mail-2.Sun.COM [192.18.98.43]) by sfbaymail1sca.SFBay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l47MPQ0R026590; Mon, 7 May 2007 15:25:27 -0700 (PDT) Received: from fe-amer-04.sun.com ([192.18.108.178]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l47MPQaP017555; Mon, 7 May 2007 22:25:26 GMT Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JHO00601ZFK8300@mail-amer.sun.com> (original mail from Timothy.Haley@Sun.COM); Mon, 07 May 2007 16:25:26 -0600 (MDT) Received: from [10.1.98.219] by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JHO00DO7ZMCEN51@mail-amer.sun.com>; Mon, 07 May 2007 16:25:26 -0600 (MDT) Date: Mon, 07 May 2007 16:25:25 -0600 From: Timothy Haley Subject: Re: ZFS case-insensitive support [PSARC/2007/244 Timeout: 05/04/2007] In-reply-to: <200705042308.l44N8hvL004771@spartan.SFBay.Sun.COM> Sender: Timothy.Haley@Sun.COM To: Don Cragun Cc: maybee@sac.sfbay.sun.com, Ienup.Sung@Sun.COM, psarc-ext@sac.sfbay.sun.com, cifs-vfs-team@Sun.COM Message-id: <463FA755.8020204@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200705042308.l44N8hvL004771@spartan.SFBay.Sun.COM> User-Agent: Thunderbird 1.5.0.10 (Macintosh/20070221) Status: RO Content-Length: 10078 Don Cragun wrote: >> Date: Thu, 03 May 2007 13:29:51 -0600 >> From: Timothy Haley >> >>>> >>>> 5) When requested, the VOP_READDIR interface must return information >>>> for each directory entry to indicate if the entry is a >>>> case-insensitive duplicate of another entry. The VOP_LOOKUP >>>> interface must also return the same information. >>>> >>>> File creates done by a non-CIFS application may create file names >>>> that are a case-insensitive duplicate of an existing file name. For >>>> example, if "foo" exists then a non-CIFS application could create >>>> the file "FOO", which is a case-insensitive duplicate. This >>>> requirement states both VOP_READDIR and VOP_LOOKUP, when requested, >>>> must provide the caller with an indication that a case-insensitive >>>> conflict exists. The caller (the CIFS server) may choose to present >>>> the name as a "mangled" name so that the user in the >>>> case-insensitive environment can distinguish between c-i conflicting >>>> file names. >>> How do you gurantee that a "mangled" name doesn't conflict with another >>> file's name? Does lookup demangle names? How does lookup know if a >>> name has been mangled? >>> >> I don't know what the mangling algorithm is. Mangling is a function of >> the CIFS server, not any of the interfaces we are defining here. > > So, I login to a windows box and create a file in my home directory on > a Solaris system through a CIFS server. Then I login on my Solaris > system and look for that file in my home directory. I do an `ls -l` > and find a file created at about the right time and that has about the > right size, but with a name I don't recognize. Is the intent that > files created by a CIFS server are only accessible through a CIFS > server? > Name mangling occurs between the CIFS server and the CIFS client. Mangled names will not appear within the file system itself. > >>>> INTERFACE CHANGES NEEDED >>> ... ... ... >>>> CASE BEHAVIOR >>>> >>>> There are two forms of case behavior: Case-sensitive and >>>> case-insensitive. Case sensitive behavior is what Solaris currently >>>> supports: When an object is created with a name, the case of the name >>>> is preserved. Objects with names that only differ in case (e.g., >>>> "foo", "Foo", "FOO") can co-exist in the same directory. Lookups on a >>>> particular name can only succeed if the case matches. >>>> >>>> Case-insensitive behavior, as required by the Solaris CIFS server, >>>> follows a particular set of rules: >>>> >> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >> We're describing the rules for c-i behavior, so... >> >>>> Object Creation: Before a named object is created, the directory >>>> must be searched for any c-i matches of that name. If there is a c-i >>>> match, then the create operation must fail with EEXIST. >>> Are you saying that under a mixed-mode mount point, a CS create has to >>> fail if it creates a CI match? >>> >> Not at all. A CI create has to fail. > > It is clear that a CI create has to fail. From the text above it is > not clear to me that a CS create is not also allowed (or even required) > to fail on a FS mounted in mixed mode. > Perhaps this change will clarify things: Case-insensitive behavior, as required by the Solaris CIFS server, - follows a particular set of rules: + follows a particular set of rules. In mixed-mode (volume supports + both c-i and c-s behavior) the case insensitive rules do not apply + to case-sensitive accesses: Note that if the CS create were to fail as you suggest, the clashes mentioned in the lookup text just below the create text wouldn't be possible and case-sensitive creates would be behaving in a case-insensitive manner. >>>> >>>> VFS INTERFACE CHANGES >>>> >>>> An interface is required to determine the case behavior of a >>>> particular file system volume. There are three possible modes for >>>> behavior: c-s only, c-i only, and mixed-mode (both c-s and c-i >>>> requests are supported). >>>> >>>> The user-level interface is pathconf(2). The pathconf(2) interface >>>> will have a new variable added: {CASE_BEHAVIOR}/_PC_CASE_BEHAVIOR. >>>> >>>> The pathconf(2) system call will return a value which represents >>>> the bitwise OR of the following flags which indicate the case-behavior >>>> that the file system supports: >>>> >>>> _CASE_INSENSITIVE This file system supports case-insensitive >>>> behavior >>>> _CASE_SENSITIVE This file system supports case-sensitive >>>> behavior >>>> >>>> The fs_pathconf() routine (default routine for VOP_PATHCONF()) will be >>>> modified to support _PC_CASE_BEHAVIOR. The default behavior for file >>>> systems on Solaris is case-sensitive so a call to fs_pathconf() with >>>> _PC_CASE_BEHAVIOR set as the cmd would set the _CASE_SENSITIVE bit. >>>> >>>> Any file system that supports anything but case-sensitive-only >>>> behavior is required to use the (new) VFS Feature Registration >>>> interfaces to register its case behavior. The following new VFS >>>> features will be introduced to support case behavior: >>>> >>>> #define VFSFT_CASEINSENSITIVE 0x100000002 /* Supports case-insensitive > */ >>>> #define VFSFT_NOCASESENSITIVE 0x100000004 /* NOT case-sensitive */ >>> Aren't "Not case-sensitive" and "case-insensitive" the same thing? Why isn't >>> the last entry above changed to: >>> #define VFSFT_CASESENSITIVE 0x100000004 /* Supoprts case-sensitive */ >>> >> A purely case-insensitive file system supports both >> VFSFT_CASEINSENSITIVE and VFSFT_NOCASESENSITIVE features. A mixed >> sensitivity file system supports just the VFSFT_CASEINSENSITIVE feature. >> A case sensitive file system, our beloved existing behavior on most >> file systems, has neither of these features. > > I don't get that from the VFSFT_names above and I don't get that from > the comments on these #defines. In the dictionary I have, "not > XXX-sensitive" and "XXX-insensitive" are synonyms. I guess the > comments below (which I didn't notice when I first commented) make it > clear, but different names and/or comments above would probably make > this clearer. > > I'll move the comments higher up. I've also added a truth table: + Expressed in a truth table form: + ---------------------------------------------------------------------------- + VFSFT_CASEINSENSITIVE VFSFT_NOCASESENSITIVE Case sensitivity +--------------------------------------------------------------------------- + off off sensitive + on on insensitive + on off mixed + off on INVALID > >>>> #define VFSFT_DIRENTFLAGS 0x100000008 /* Supports dirent flags */ >>>> >>>> File systems that support case-insensitive behavior must register the >>>> VFSFT_CASEINSENSITIVE feature. File systems that do *not* support >>>> case-sensitive behavior must register the VFSFT_NOCASESENSITIVE >>>> feature. >>>> >>>> VOP (Vnode Operation) INTERFACE CHANGES >>> ... ... ... >>>> >>>> int fop_rename( >>>> vnode_t *sdvp, >>>> char *snm, >>>> vnode_t *tdvp, >>>> char *tnm, >>>> cred_t *cr, >>>> caller_context *ctp, /* See PSARC/2007/218 */ >>>> /* NEW */ int flags) /* Takes FIGNORECASE */ >>>> >>>> If FIGNORECASE is set and the file system supports c-i, then >>>> the object to be renamed shall match the same name that >>>> fop_lookup would match. That is, the c-i matching algorithm >>>> should be the same in both fop_lookup, fop_remove, fop_rename, >>>> and fop_rmdir. Same holds for the target name, if it exists. >>> If "foo" and "FOO" both exist and I attempt: >>> rename("foo", "FOO"); >>> is it a no-op, is an error returned, or is one of the files removed as >>> the other is renamed? >>> >> I'm assuming you mean for this rename to also have the FIGNORECASE set. > > Yes, but I don't know how this happens. From what I have read in this > case, we will still just have one rename() system call and it doesn't > have a flags argument; it just has two pathnames as arguments. How > does an application set or clear the FIGNORECASE flag when it calls > rename()? I assume that when an app calls open() and is creating a > file there will be an O_ flag added that corresponds to FIGNORECASE > (but no O_ flags seem to be added here); but I don't see anything that > says how you would attach this flag to an open() of an existing file or > any of the other syscalls that perform lookups when they aren't > creating a file. > Applications will not be able to set the FIGNORECASE bit, and there will not be any system call support for this. Only kernel consumers of the fop_XXX() interfaces may set this bit. Future work *might* expose this interface outside of the kernel, although I'm not aware of any pressing need to do so. > >> This is a good question. Interesting corner case. I think one could >> argue either way. At this very instant I can say that this rename is a >> no-op. I need to consider this and get some feedback from CIFs folks, >> though if that is correct. > > Whatever you want to happen needs to be defined. (And, I would really > like to see some rationale for the decision.) In general I assume that > if a CI match for "src" is present but there is no CI match for "DeSt", > then case will be preserved on DeSt if I rename("src", "DeSt"). Is > this correct? > In discussing this with the CIFS team, it was determined this actually needs to be a no-op. Case-insensitively, both arguments describe the same file, so: + + If FIGNORECASE is set and snm and tnm are a case-insensitive + match, the return value of the fop_rename should be zero and + no other action should be performed. -tim From sommerfeld@sun.com Tue May 8 15:09:57 2007 Received: from eastmail2bur.East.Sun.COM (eastmail2bur.East.Sun.COM [129.148.13.40]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l48M9vNK001392; Tue, 8 May 2007 15:09:57 -0700 (PDT) Received: from thunk.east.sun.com (thunk.East.Sun.COM [129.148.174.66]) by eastmail2bur.East.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l48M8rHp002111; Tue, 8 May 2007 18:08:53 -0400 (EDT) Received: from [IPv6:::1] (localhost [IPv6:::1]) by thunk.east.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l48M8qLm016644; Tue, 8 May 2007 18:08:52 -0400 (EDT) Subject: Re: ZFS case-insensitive support [PSARC/2007/244 Timeout: 05/04/2007] From: Bill Sommerfeld To: Don Cragun Cc: Timothy.Haley@sun.com, maybee@sac.sfbay.sun.com, Ienup.Sung@sun.com, psarc-ext@sac.sfbay.sun.com, cifs-vfs-team@sun.com In-Reply-To: <200705042308.l44N8hvL004771@spartan.SFBay.Sun.COM> References: <200705042308.l44N8hvL004771@spartan.SFBay.Sun.COM> Content-Type: text/plain Date: Tue, 08 May 2007 18:08:51 -0400 Message-Id: <1178662131.14396.31.camel@thunk> Mime-Version: 1.0 X-Mailer: Evolution 2.8.1.1 Content-Transfer-Encoding: 7bit Status: RO Content-Length: 478 > Is it really a good > idea for a sysadmin to be able to mount a ZFS file system in CI mode > rather than just in CS or in mixed mode? So, my expectation would be that there should be some way to lock down a directory intended to be shared between CI and CS systems so that folks operating from the CS side could not create files that couldn't be named from the CI side. As I read the spec, that's only achievable via the "insensitive" mount mode. - Bill From Timothy.Haley@Sun.COM Tue May 8 17:05:27 2007 Received: from sfbaymail1sca.SFBay.Sun.COM (sfbaymail1sca.SFBay.Sun.COM [129.145.154.35]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l4905R1O009206; Tue, 8 May 2007 17:05:27 -0700 (PDT) Received: from brmea-mail-4.sun.com (brmea-mail-4.Sun.COM [192.18.98.36]) by sfbaymail1sca.SFBay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l4904PmX012328; Tue, 8 May 2007 17:04:25 -0700 (PDT) Received: from fe-amer-02.sun.com ([192.18.108.176]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l4904PFH004776; Wed, 9 May 2007 00:04:25 GMT Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JHQ00101Y985B00@mail-amer.sun.com> (original mail from Timothy.Haley@Sun.COM); Tue, 08 May 2007 18:04:25 -0600 (MDT) Received: from [10.1.98.219] by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JHQ00L24YVBJ6N2@mail-amer.sun.com>; Tue, 08 May 2007 18:04:25 -0600 (MDT) Date: Tue, 08 May 2007 18:04:23 -0600 From: Timothy Haley Subject: Re: ZFS case-insensitive support [PSARC/2007/244 Timeout: 05/04/2007] In-reply-to: <1178662131.14396.31.camel@thunk> Sender: Timothy.Haley@Sun.COM To: Bill Sommerfeld Cc: Don Cragun , maybee@sac.sfbay.sun.com, Ienup.Sung@Sun.COM, psarc-ext@sac.sfbay.sun.com, cifs-vfs-team@Sun.COM Message-id: <46411007.4060101@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200705042308.l44N8hvL004771@spartan.SFBay.Sun.COM> <1178662131.14396.31.camel@thunk> User-Agent: Thunderbird 1.5.0.10 (Macintosh/20070221) Status: RO Content-Length: 765 Bill Sommerfeld wrote: >> Is it really a good >> idea for a sysadmin to be able to mount a ZFS file system in CI mode >> rather than just in CS or in mixed mode? > > So, my expectation would be that there should be some way to lock down a > directory intended to be shared between CI and CS systems so that folks > operating from the CS side could not create files that couldn't be named > from the CI side. > One of our design goals was to not hamper case-sensitive access in any manner. So, CI consumers, ie those who use FIGNORECASE, must be prepared to deal with case-insensitive name clashes if the file system supports mixed-case behavior. -tim > As I read the spec, that's only achievable via the "insensitive" mount > mode. > > - Bill > > From Mark.Maybee@Sun.COM Wed May 9 10:42:48 2007 Received: from sfbaymail1sca.SFBay.Sun.COM (sfbaymail1sca.SFBay.Sun.COM [129.145.154.35]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l49HgmST023582 for ; Wed, 9 May 2007 10:42:48 -0700 (PDT) Received: from brmea-mail-3.sun.com (brmea-mail-3.Sun.COM [192.18.98.34]) by sfbaymail1sca.SFBay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l49HfjG9012491 for ; Wed, 9 May 2007 10:41:45 -0700 (PDT) Received: from fe-amer-10.sun.com ([192.18.108.184]) by brmea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l49Hfi5q005840 for ; Wed, 9 May 2007 17:41:44 GMT Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JHS00901BOGI100@mail-amer.sun.com> (original mail from Mark.Maybee@Sun.COM) for psarc-ext@sac.sfbay.sun.com; Wed, 09 May 2007 11:41:44 -0600 (MDT) Received: from [172.20.25.54] by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JHS00KNEBTKSJ13@mail-amer.sun.com>; Wed, 09 May 2007 11:41:44 -0600 (MDT) Date: Wed, 09 May 2007 11:41:05 -0600 From: Mark Maybee Subject: ZFS case-insensitive support [PSARC/2007/244] Approved In-reply-to: <1178662131.14396.31.camel@thunk> Sender: Mark.Maybee@Sun.COM To: psarc-ext@sac.sfbay.sun.com Cc: Timothy.Haley@Sun.COM, ZFS Engineering Team , Ienup.Sung@Sun.COM, cifs-vfs-team@Sun.COM Message-id: <464207B1.3060406@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200705042308.l44N8hvL004771@spartan.SFBay.Sun.COM> <1178662131.14396.31.camel@thunk> User-Agent: Thunderbird 1.5.0.5 (X11/20060730) Status: RO Content-Length: 124 http://www.opensolaris.org/os/community/arc/caselog/2007/244/ was approved during arc business at the 5/9/07 PSARC meeting.