From darrenm@sac.sfbay.sun.com Mon Aug 17 09:51:52 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n7HGpp4E027204 for ; Mon, 17 Aug 2009 09:51:51 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n7HGpmiE010300; Tue, 18 Aug 2009 00:51:50 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KOJ00B075IC4300@nwk-avmta-1.sfbay.Sun.COM>; Mon, 17 Aug 2009 09:51:48 -0700 (PDT) Received: from dm-sfbay-01.sfbay.sun.com ([129.145.155.118]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KOJ00KVY5ICMX80@nwk-avmta-1.sfbay.Sun.COM>; Mon, 17 Aug 2009 09:51:48 -0700 (PDT) Received: from sac.sfbay.sun.com (sac.SFBay.Sun.COM [129.146.226.132]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n7HGplE5065155; Mon, 17 Aug 2009 09:51:47 -0700 (PDT) Received: from sac.sfbay.sun.com (localhost [127.0.0.1]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n7HGpkYg027198; Mon, 17 Aug 2009 09:51:46 -0700 (PDT) Received: (from darrenm@localhost) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id n7HGpk8q027194; Mon, 17 Aug 2009 09:51:46 -0700 (PDT) Date: Mon, 17 Aug 2009 09:51:46 -0700 (PDT) From: Darren J Moffat Subject: ZFS Crypto Updates [PSARC/2009/443 FastTrack timeout 08/24/2009] To: PSARC-ext@sun.com Cc: zfs-crypto-discuss@opensolaris.org, zfs-discuss@opensolaris.org Message-id: <200908171651.n7HGpk8q027194@sac.sfbay.sun.com> Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 Status: RO Content-Length: 8519 Template Version: @(#)sac_nextcase 1.68 02/23/09 SMI This information is Copyright 2009 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: ZFS Crypto Updates 1.2. Name of Document Author/Supplier: Author: Darren Moffat 1.3 Date of This Document: 17 August, 2009 4. Technical Description The following changes are being made to the ZFS Crypto design as documented in PSARC/2007/261 Block Pointer Changes --------------------- In order to facilitate a future project that allows for an encrypted (and compressed) "raw" send/recv stream and interaction with the in progress "BP rewriter (aka device removal work)" the crypto IV (Initalisation Vector) will now be stored in the BP. This is done by reusing the third DVA slot. The impact of this is that when encryption is enabled on a dataset "copies" can be set to a maximum of 2. Attempting to set copies=3 on an encrypted dataset fails: # zfs set copies=3 tank/fs cannot set property for 'tank/fs': copies cannot be set to 3 on encrypted datasets. Since all data blocks in an encrypted dataset are always encrypted with the same encryption algorithm a single bit is used to store the fact that this is a pointer to an encrypted block rather than storing the index value of the encryption property. On disk wrapped key changes --------------------------- Instead of a hidden "wrappedkey" property a per objset ZAP object will be used instead. This avoids the need to introduce a new binary property type and also allows for key chains. A key chain is a set of ZAP objects indexed by txg (transcation group) the value of the ZAP object is a wrapped key. This is represented in memory as an AVL tree of unwrapped keys per dataset. Simpiler defaults ----------------- It is no longer necessary to specific the keysource property, if it is not specififed and encryption is enabled then a default of "passphrase,prompt" is assumed. This allows users to enable encryption easily thus: # zfs create -o encryption=on tank/fs Enter passphrase for 'tank/fs': Enter again: # AES keylength/modes ------------------- Based on the performance of AES at various key lengths and the recent papers in the cryptography community on weakness in AES when used with a 256 bit key the default "on" value for encryption will be "aes-128-ccm". This provides the best performing default and based on current research in the crypto community it my ironically be stronger than "aes-256-ccm". The "aes-192-ccm" mode will also be added. See: http://eprint.iacr.org/2009/374 This case also seeks approval to add "aes-128-gcm, aes-192-gcm, aes-256-gcm", using AES GCM. From a cryptographic view CCM and GCM are very similar however GCM is significantly more expensive due to Galois filed multiplication requirements. GCM is the preffered mode for NSA Suite B compliance so it is included as an option. Keyscope property removed ------------------------- The keyscope property and the concept of keyscope is removed. Instead the keyscope is always now per dataset and is properly inherited. With this all the benefits of the previous keyscope=pool are available while providing a much simplified admin model that is now consistent with how property inheritance works in ZFS. This means there are no crypto related changes to zpool(1M). Explain this by example: # zfs create -o encryption=on -o keysource=raw,file:///media/my/key tank/fs At this point a single encrypted dataset exists and is wrapped using the AES key in /media/my/key. # zfs create tank/fs/sub The keysource property is inherited as is the actual value of the wrappingkey. This is more obvious in the following example: # zfs create -o encryption=on tank/a Enter passphrase for 'tank/a': Enter again: # zfs create tank/a/sub On creating tank/a/sub the user is NOT prompted to enter a passphrase because tank/a is already mounted (ie its key is available) so we inherit the wrapping key from our parent. This also ensures that on import/mount the user is only prompted for the passphrase for tank/a and not all the child datasets of it. If the user actually wishes to "stop" the inheritance of the passphrase (ie the wrapping key value) then they can explicitly set the keysource property when creating the dataset or performing a key change. Dataset rename restrictions --------------------------- On rename a dataset can non be moved out of its wrapping key hierarchy ie where it inherits the keysource property from. This is best explained by example: # zfs get -r keysource tank NAME PROPERTY VALUE SOURCE tank keysource none default tank/A keysource passphrase,prompt local tank/A/b keysource passphrase,prompt inherited from tank/A tank/A/b/c keysource passphrase,prompt inherited from tank/A tank/D keysource none default tank/D/e keysource passphrase,prompt local Simple rename of leaf dataset in place: # zfs rename tank/D/e tank/D/E OK Rename within keysource inheritance remains the same: # zfs rename tank/A/b/c tank/A/c OK Rename out of keysource inheritance path: # zfs rename tank/A/b/c tank/D/e/c FAIL Key Change ---------- Due to the above mention keyscope removal the zfs key command now only accepts a new keyscope property with -o when changing key. Key change is also now applied recursilvey in a single transaction to all datasets inheriting the wrapping key. Dataset mount ------------- The zfs_mount() library call in libzfs, and thus zfs(1M) mount command will load keys if they are available when a dataset is attempting to be mounted. Note that this means that 'zfs mount -a' can attempt to be interactive if the keysource locator is "prompt". Note that this does NOT cause a prompt for system boot and we do NOT wait looking for keys (there is no facility to do so with SMF anyway). Dataset/Clone Rekey ------------------- A new "-K" flag for 'zfs clone' is introduced. If supplied new data encryption key will be created in the transaction group the clone is created in and used for all data that is unique to the clone. The previous state of the keychain will be copied from the origin snapshot. The 'zfs key' subcommand also has the "-K" flag, in this case it will create a new data encryption key for the named dataset(s). The rationale for this is allowing compliance with NIST guideless for the amount of time a data encryption key should be used. In both the cases the last rekeyed date can be found in the new 'rekeydate' property. If a dataset has not been rekeyed then the value of 'rekeydate' is the same as the 'creation' date. For a dataset that is not encrypted then the value is "-". For example: zfs get -r -H creation,rekeydate tank tank creation Tue Aug 4 14:37 2009 - tank rekeydate - default tank/128g creation Tue Aug 4 15:52 2009 - tank/128g rekeydate Tue Aug 4 15:52 2009 local tank/192g creation Tue Aug 4 16:08 2009 - tank/192g rekeydate Tue Aug 4 16:19 2009 local Summary of zfs(1M) command synopsis ----------------------------------- zfs key usage: key -l <-a | [-r] filesystem|volume> key -u [-f] <-a | [-r] filesystem|volume> key -c [ -o ] <-a | [-r] filesystem|volume> key -K <-a | [-r] filesystem|volume> zfs set properties: keystatus NO NO undefined | unavailable | available rekeyed NO NO clonerekey YES YES on | off encryption YES YES on | off | aes-128-ccm | aes-192-ccm | aes-256-ccm | aes-128-gcm | aes-192-gcm | aes-256-gcm keysource YES YES keysource=raw|hex|passphrase,prompt|file:// Dnode Bonusbuf Encryption ------------------------- Instead of encrypting the bonusbuf section of the dnodes the ZFS Crypto feature will now depend on the ZFS fast system attributes project and will cause the bonusbuf to always completely spill. Note there are no user visible interface change from this and the ZFS fast system attribtues project isn't expected to be reviewed in ARC as it is an implementation detail only. Management of the dependency is thus not an ARC issue but an internal team coordination issue. 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: FastTrack 6.6. ARC Exposure: open From gdamore@sun.com Mon Aug 17 14:47:00 2009 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n7HLl0Bx008349 for ; Mon, 17 Aug 2009 14:47:00 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n7HLkgfx007040 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Mon, 17 Aug 2009 22:46:59 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KOJ00N09J691T00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.Com); Mon, 17 Aug 2009 14:46:57 -0700 (PDT) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KOJ00985J68O2D0@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.Com); Mon, 17 Aug 2009 14:46:56 -0700 (PDT) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n7HLksBL013436 for ; Mon, 17 Aug 2009 14:46:56 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KOJ00000ITII000@fe-sfbay-10.sun.com> for PSARC-ext@Sun.Com (ORCPT PSARC-ext@Sun.Com); Mon, 17 Aug 2009 14:46:55 -0700 (PDT) Received: from [192.168.251.11] ([unknown] [76.93.15.33]) by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KOJ000ZMJ58DBF0@fe-sfbay-10.sun.com>; Mon, 17 Aug 2009 14:46:21 -0700 (PDT) Date: Mon, 17 Aug 2009 14:46:20 -0700 From: "Garrett D'Amore" Subject: Re: ZFS Crypto Updates [PSARC/2009/443 FastTrack timeout 08/24/2009] In-reply-to: <200908171651.n7HGpk8q027194@sac.sfbay.sun.com> Sender: Garrett.Damore@sun.com To: Darren J Moffat Cc: PSARC-ext@sun.com, zfs-crypto-discuss@opensolaris.org, zfs-discuss@opensolaris.org Message-id: <4A89CFAC.6040701@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200908171651.n7HGpk8q027194@sac.sfbay.sun.com> User-Agent: Thunderbird 2.0.0.18 (X11/20081201) Status: RO Content-Length: 3488 Darren J Moffat wrote: > > Dataset rename restrictions > --------------------------- > > On rename a dataset can non be moved out of its wrapping key hierarchy > ie where it inherits the keysource property from. This is best explained > by example: > > # zfs get -r keysource tank > NAME PROPERTY VALUE SOURCE > tank keysource none default > tank/A keysource passphrase,prompt local > tank/A/b keysource passphrase,prompt inherited from tank/A > tank/A/b/c keysource passphrase,prompt inherited from tank/A > tank/D keysource none default > tank/D/e keysource passphrase,prompt local > > Simple rename of leaf dataset in place: > # zfs rename tank/D/e tank/D/E OK > > Rename within keysource inheritance remains the same: > > # zfs rename tank/A/b/c tank/A/c OK > > Rename out of keysource inheritance path: > > # zfs rename tank/A/b/c tank/D/e/c FAIL > I'd like to see draft text for a man page describing this behavior. I suspect that this is likely to be potentially confusing. As an alternative to failure, could one imagine having a "-f" (force) switch that allows the rename, and creates a new keysource root? > Dataset mount > ------------- > The zfs_mount() library call in libzfs, and thus zfs(1M) mount command > will load keys if they are available when a dataset is attempting to be > mounted. Note that this means that 'zfs mount -a' can attempt to be > interactive if the keysource locator is "prompt". Note that this does > NOT cause a prompt for system boot and we do NOT wait looking for keys > (there is no facility to do so with SMF anyway). > So what is the behavior at boot for these file systems? Are they left unmounted? Is there any indication to the administrator that this is the situation? The man page indicates that zfs mount -a is run at boot, but it seems like this might be a special case. Again, this is one I'd like to see supporting man page diffs for. > > Dnode Bonusbuf Encryption > ------------------------- > Instead of encrypting the bonusbuf section of the dnodes the ZFS Crypto feature > will now depend on the ZFS fast system attributes project and will cause > the bonusbuf to always completely spill. Note there are no user visible > interface change from this and the ZFS fast system attribtues project isn't > expected to be reviewed in ARC as it is an implementation detail only. > Management of the dependency is thus not an ARC issue but an internal team > coordination issue. > Implementation details that affect on disk storage, or have other larger ramifications elsewhere on the system, should probably still be ARC'd. Is the fast system attributes project planning on changing the on-disk format? The rest of the changes proposed look good as is to me, but I'd also like an explicit statement from the ZFS (and perhaps a;lso Fishworks) core team that they have reviewed and approved this proposal. Since I suspect you're working rather closely with them on this project, that shouldn't be too hard to achieve. (My concern here is that I think it is likely that nobody else on ARC has sufficient filesystem -- or perhaps just ZFS -- expertise to perform a meaningful review on our own. This is a situation which I would *like* to see rectified by having storage represented at ARC, but I've as yet been unable to generate sufficient interest from the storage folks.) Thanks. - Garrett From Darren.Moffat@sun.com Tue Aug 18 02:42:08 2009 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n7I9g7OK006861 for ; Tue, 18 Aug 2009 02:42:08 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n7I9g14i005285 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Tue, 18 Aug 2009 10:42:07 +0100 (BST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KOK00907GA53400@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 18 Aug 2009 03:42:05 -0600 (MDT) Received: from gmp-eb-inf-1.sun.com ([192.18.6.21]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KOK002RNGA48G30@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 18 Aug 2009 03:42:04 -0600 (MDT) Received: from fe-emea-09.sun.com (gmp-eb-lb-1-fe1.eu.sun.com [192.18.6.7] (may be forged)) by gmp-eb-inf-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n7I9g396024689 for ; Tue, 18 Aug 2009 09:42:03 +0000 (GMT) Received: from conversion-daemon.fe-emea-09.sun.com by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KOK00F00FTKIR00@fe-emea-09.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 18 Aug 2009 10:41:53 +0100 (BST) Received: from [192.168.1.105] (cpc2-rdng20-2-0-cust917.15-3.cable.virginmedia.com [86.28.167.150]) by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KOK009N1G9OX6D0@fe-emea-09.sun.com>; Tue, 18 Aug 2009 10:41:48 +0100 (BST) Date: Tue, 18 Aug 2009 10:41:38 +0100 From: Darren J Moffat Subject: Re: ZFS Crypto Updates [PSARC/2009/443 FastTrack timeout 08/24/2009] In-reply-to: <4A89CFAC.6040701@sun.com> Sender: Darren.Moffat@sun.com To: "Garrett D'Amore" Cc: Darren J Moffat , PSARC-ext@sun.com, zfs-discuss@opensolaris.org, zfs-crypto-discuss@opensolaris.org Message-id: <4A8A7752.4030304@Sun.COM> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200908171651.n7HGpk8q027194@sac.sfbay.sun.com> <4A89CFAC.6040701@sun.com> User-Agent: Thunderbird 2.0.0.21 (X11/20090623) Status: RO Content-Length: 6327 Garrett D'Amore wrote: > Darren J Moffat wrote: >> >> Dataset rename restrictions >> --------------------------- >> >> On rename a dataset can non be moved out of its wrapping key hierarchy >> ie where it inherits the keysource property from. This is best >> explained by example: >> >> # zfs get -r keysource tank NAME PROPERTY >> VALUE SOURCE >> tank keysource none default >> tank/A keysource passphrase,prompt local >> tank/A/b keysource passphrase,prompt inherited from tank/A >> tank/A/b/c keysource passphrase,prompt inherited from tank/A >> tank/D keysource none default >> tank/D/e keysource passphrase,prompt local >> >> Simple rename of leaf dataset in place: >> # zfs rename tank/D/e tank/D/E OK >> >> Rename within keysource inheritance remains the same: >> >> # zfs rename tank/A/b/c tank/A/c OK >> >> Rename out of keysource inheritance path: >> >> # zfs rename tank/A/b/c tank/D/e/c FAIL >> > > I'd like to see draft text for a man page describing this behavior. I > suspect that this is likely to be potentially confusing. The above is the draft man page text. What more would you like to see ? Is it that you don't understand the behaviour or you want to do an editoral review of the man page text ? If the later then I'd assert that isn't ARC review. > As an alternative to failure, could one imagine having a "-f" (force) > switch that allows the rename, and creates a new keysource root? We can't do that because you need to specify what the keysource is. So a force switch would be more like: # zfs rename -o keysource=... tank/A/b/c tank/D/e/c However 'zfs rename' doesn't have a -o capability today and it is out of the scope of the crypto project to add that support to rename. It is possible to add in the future but it has a non trivial impact beyond crypto support so I'm not willing to take that on just now - particularly since it requires changes to libzfs APIs that are in use by Fishworks. I've logged an RFE for the general case: 6872829 >> Dataset mount >> ------------- >> The zfs_mount() library call in libzfs, and thus zfs(1M) mount command >> will load keys if they are available when a dataset is attempting to be >> mounted. Note that this means that 'zfs mount -a' can attempt to be >> interactive if the keysource locator is "prompt". Note that this does >> NOT cause a prompt for system boot and we do NOT wait looking for keys >> (there is no facility to do so with SMF anyway). >> > > So what is the behavior at boot for these file systems? Are they left > unmounted? Yes, and this is unchanged from the previous PSARC case for ZFS Crypto. > Is there any indication to the administrator that this is > the situation? Unchanged from the previous PSARC case for ZFS Crypto. The aren't mounted and shown in df, the mounted property of the dataset is false and the keystatus property is unavailable. > The man page indicates that zfs mount -a is run at boot, > but it seems like this might be a special case. Again, this is one I'd > like to see supporting man page diffs for. This is unchanged from the previous case. There is a private implementation detail where we ensure that the SMF service that runs at boot does not attempt to prompt. This is because SMF doesn't provide any means to do so, and even if it did we may not want that to happen. >> Dnode Bonusbuf Encryption >> ------------------------- >> Instead of encrypting the bonusbuf section of the dnodes the ZFS >> Crypto feature >> will now depend on the ZFS fast system attributes project and will cause >> the bonusbuf to always completely spill. Note there are no user visible >> interface change from this and the ZFS fast system attribtues project >> isn't >> expected to be reviewed in ARC as it is an implementation detail only. >> Management of the dependency is thus not an ARC issue but an internal >> team >> coordination issue. >> > > Implementation details that affect on disk storage, or have other larger > ramifications elsewhere on the system, should probably still be ARC'd. This is not an ARC issue in any many other ZFS cases and I'm not making it one in this case either, because it isn't this project that is introducing fast system attributes. Not all ZFS changes that impact on disk actually come to ARC anyway. > Is the fast system attributes project planning on changing the on-disk > format? Yes but in a compatible way using the version system, there is nothing directly visible to admins. To enable encryption you have to upgrade your pool version number anyway - this is standard ZFS behaviour when adding new features that have an on disk impact. That was already the case with the previous ZFS Crypto ARC case. Encryption support will be a pool version after the fast system attributes project and encryption depends on it. However I don't expect to see an ARC case presented to PSARC for the fast system attributes project since there is nothing visible to the admin other than the pool version bump. The main reason for integration fast system attributes separately from crypto is because there is another non ON (and not in Sun ARC control) consumer for them (and no it isn't Fishworks). > The rest of the changes proposed look good as is to me, but I'd also > like an explicit statement from the ZFS (and perhaps a;lso Fishworks) > core team that they have reviewed and approved this proposal. The core senior members of the ZFS team contributed to this design and have provided input during an extensive review period. A representative of Fishworks was also present during the design of these changes (back in Feburary) and indicated they were happy with the changes. > (My concern here is that I think it is likely that nobody else on ARC > has sufficient filesystem -- or perhaps just ZFS -- expertise to perform > a meaningful review on our own. This is a situation which I would > *like* to see rectified by having storage represented at ARC, but I've > as yet been unable to generate sufficient interest from the storage folks.) Which is why this has had very extensive review from senior members of the core ZFS team. -- Darren J Moffat From gdamore@Sun.COM Tue Aug 18 06:47:40 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n7IDlda6015102 for ; Tue, 18 Aug 2009 06:47:40 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n7IDlB4F009093 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Tue, 18 Aug 2009 21:47:39 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KOK00N01RND4200@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 18 Aug 2009 06:47:37 -0700 (PDT) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KOK00C19RNC1DB0@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 18 Aug 2009 06:47:36 -0700 (PDT) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n7IDladO025881 for ; Tue, 18 Aug 2009 06:47:36 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KOK00I00RJBQR00@fe-sfbay-10.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 18 Aug 2009 06:47:36 -0700 (PDT) Received: from [192.168.251.11] ([unknown] [76.93.15.33]) by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KOK00G2DRNBVL90@fe-sfbay-10.sun.com>; Tue, 18 Aug 2009 06:47:36 -0700 (PDT) Date: Tue, 18 Aug 2009 06:47:35 -0700 From: "Garrett D'Amore" Subject: Re: ZFS Crypto Updates [PSARC/2009/443 FastTrack timeout 08/24/2009] In-reply-to: <4A8A7752.4030304@Sun.COM> Sender: Garrett.Damore@Sun.COM To: Darren J Moffat Cc: Darren J Moffat , PSARC-ext@Sun.COM, zfs-discuss@opensolaris.org, zfs-crypto-discuss@opensolaris.org Message-id: <4A8AB0F7.8040702@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200908171651.n7HGpk8q027194@sac.sfbay.sun.com> <4A89CFAC.6040701@sun.com> <4A8A7752.4030304@Sun.COM> User-Agent: Thunderbird 2.0.0.18 (X11/20081201) Status: RO Content-Length: 5842 Darren J Moffat wrote: > Garrett D'Amore wrote: >> Darren J Moffat wrote: >>> >>> Dataset rename restrictions >>> --------------------------- >>> >>> On rename a dataset can non be moved out of its wrapping key hierarchy >>> ie where it inherits the keysource property from. This is best >>> explained by example: >>> >>> # zfs get -r keysource tank NAME PROPERTY >>> VALUE SOURCE >>> tank keysource none default >>> tank/A keysource passphrase,prompt local >>> tank/A/b keysource passphrase,prompt inherited from tank/A >>> tank/A/b/c keysource passphrase,prompt inherited from tank/A >>> tank/D keysource none default >>> tank/D/e keysource passphrase,prompt local >>> >>> Simple rename of leaf dataset in place: >>> # zfs rename tank/D/e tank/D/E OK >>> >>> Rename within keysource inheritance remains the same: >>> >>> # zfs rename tank/A/b/c tank/A/c OK >>> >>> Rename out of keysource inheritance path: >>> >>> # zfs rename tank/A/b/c tank/D/e/c FAIL >>> >> >> I'd like to see draft text for a man page describing this behavior. >> I suspect that this is likely to be potentially confusing. > > The above is the draft man page text. What more would you like to see > ? Is it that you don't understand the behaviour or you want to do an > editoral review of the man page text ? If the later then I'd assert > that isn't ARC review. Neither. I want to make sure we present this behavior in an unconfusing to our users. If the above is intended to be in the man page, then I'm satisfied. > >> As an alternative to failure, could one imagine having a "-f" (force) >> switch that allows the rename, and creates a new keysource root? > > We can't do that because you need to specify what the keysource is. > So a force switch would be more like: > # zfs rename -o keysource=... tank/A/b/c tank/D/e/c > > However 'zfs rename' doesn't have a -o capability today and it is out > of the scope of the crypto project to add that support to rename. It > is possible to add in the future but it has a non trivial impact > beyond crypto support so I'm not willing to take that on just now - > particularly since it requires changes to libzfs APIs that are in use > by Fishworks. > > I've logged an RFE for the general case: 6872829 Okay, I'm satisfied with your answer to this. [ other bits I'm satisfied with elided. ] > >>> Dnode Bonusbuf Encryption >>> ------------------------- >>> Instead of encrypting the bonusbuf section of the dnodes the ZFS >>> Crypto feature >>> will now depend on the ZFS fast system attributes project and will >>> cause >>> the bonusbuf to always completely spill. Note there are no user >>> visible >>> interface change from this and the ZFS fast system attribtues >>> project isn't >>> expected to be reviewed in ARC as it is an implementation detail only. >>> Management of the dependency is thus not an ARC issue but an >>> internal team >>> coordination issue. >>> >> >> Implementation details that affect on disk storage, or have other >> larger ramifications elsewhere on the system, should probably still >> be ARC'd. > > This is not an ARC issue in any many other ZFS cases and I'm not > making it one in this case either, because it isn't this project that > is introducing fast system attributes. Not all ZFS changes that > impact on disk actually come to ARC anyway. > >> Is the fast system attributes project planning on changing the >> on-disk format? > > Yes but in a compatible way using the version system, there is nothing > directly visible to admins. > > To enable encryption you have to upgrade your pool version number > anyway - this is standard ZFS behaviour when adding new features that > have an on disk impact. That was already the case with the previous > ZFS Crypto ARC case. Encryption support will be a pool version after > the fast system attributes project and encryption depends on it. > However I don't expect to see an ARC case presented to PSARC for the > fast system attributes project since there is nothing visible to the > admin other than the pool version bump. The main reason for > integration fast system attributes separately from crypto is because > there is another non ON (and not in Sun ARC control) consumer for them > (and no it isn't Fishworks). Hmm.... that's too bad. I think it would still be preferable for changes of this nature to get at least some minimal review at ARC. Although as we don't have any ZFS representation at ARC, maybe review of such changes wouldn't add anything useful. > >> The rest of the changes proposed look good as is to me, but I'd also >> like an explicit statement from the ZFS (and perhaps a;lso Fishworks) >> core team that they have reviewed and approved this proposal. > > The core senior members of the ZFS team contributed to this design and > have provided input during an extensive review period. > > A representative of Fishworks was also present during the design of > these changes (back in Feburary) and indicated they were happy with > the changes. > >> (My concern here is that I think it is likely that nobody else on ARC >> has sufficient filesystem -- or perhaps just ZFS -- expertise to >> perform a meaningful review on our own. This is a situation which I >> would *like* to see rectified by having storage represented at ARC, >> but I've as yet been unable to generate sufficient interest from the >> storage folks.) > > Which is why this has had very extensive review from senior members of > the core ZFS team. Okay, I was pretty sure this was the case, just wanted to make sure it was explicitly stated as true. :-) With your replies, +1 to the case. Thanks for your patience. - Garrett From Darren.Moffat@sun.com Tue Aug 18 07:09:14 2009 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n7IE9DdO015576 for ; Tue, 18 Aug 2009 07:09:13 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n7IE9BRP009469 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Tue, 18 Aug 2009 15:09:12 +0100 (BST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KOK00D09SNBQC00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 18 Aug 2009 08:09:11 -0600 (MDT) Received: from gmp-eb-inf-1.sun.com ([192.18.6.21]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KOK004OASNBFDA0@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 18 Aug 2009 08:09:11 -0600 (MDT) Received: from fe-emea-10.sun.com (gmp-eb-lb-1-fe1.eu.sun.com [192.18.6.7] (may be forged)) by gmp-eb-inf-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n7IE9ASE012437 for ; Tue, 18 Aug 2009 14:09:10 +0000 (GMT) Received: from conversion-daemon.fe-emea-10.sun.com by fe-emea-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KOK00600SIUWF00@fe-emea-10.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 18 Aug 2009 15:08:58 +0100 (BST) Received: from [192.168.1.105] (cpc2-rdng20-2-0-cust917.15-3.cable.virginmedia.com [86.28.167.150]) by fe-emea-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KOK002XQSMQ2X90@fe-emea-10.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 18 Aug 2009 15:08:50 +0100 (BST) Date: Tue, 18 Aug 2009 15:08:39 +0100 From: Darren J Moffat Subject: Re: ZFS Crypto Updates [PSARC/2009/443 FastTrack timeout 08/24/2009] In-reply-to: <4A8AB0F7.8040702@sun.com> Sender: Darren.Moffat@sun.com To: "Garrett D'Amore" Cc: PSARC-ext@sun.com Message-id: <4A8AB5E7.70005@Sun.COM> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200908171651.n7HGpk8q027194@sac.sfbay.sun.com> <4A89CFAC.6040701@sun.com> <4A8A7752.4030304@Sun.COM> <4A8AB0F7.8040702@sun.com> User-Agent: Thunderbird 2.0.0.21 (X11/20090623) Status: RO Content-Length: 1032 Garrett D'Amore wrote: [replying off arc] > Hmm.... that's too bad. I think it would still be preferable for > changes of this nature to get at least some minimal review at ARC. > Although as we don't have any ZFS representation at ARC, maybe review of > such changes wouldn't add anything useful. I'd say I pretty much count as ZFS representation these days as I spend more of my time in the ZFS code base than anything else. A lot of the ZFS on disk changes are either ARC'd because there is a visible feature or they are really project private implementation details. The fast system attributes happens to bereally an implementation detail that unless you have a very deep understanding of how things work today, what Lustre needs and what ZFS Crypto needs it would be pointless to try and follow why what it is doing is useful. The ZFS team might choose to bring it to ARC still but I don't believe there are plans to do so, since it is really needed for Lustre first and ZFS Crypto second. -- Darren J Moffat From Darren.Moffat@sun.com Thu Aug 20 01:07:56 2009 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n7K87uUj001029 for ; Thu, 20 Aug 2009 01:07:56 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n7K87uW7020022 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Thu, 20 Aug 2009 02:07:56 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KOO00D0D196K900@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 20 Aug 2009 01:07:54 -0700 (PDT) Received: from gmp-eb-inf-1.sun.com ([192.18.6.21]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KOO00AU11957X20@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 20 Aug 2009 01:07:54 -0700 (PDT) Received: from fe-emea-10.sun.com (gmp-eb-lb-1-fe1.eu.sun.com [192.18.6.7] (may be forged)) by gmp-eb-inf-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n7K87rj9008553 for ; Thu, 20 Aug 2009 08:07:53 +0000 (GMT) Received: from conversion-daemon.fe-emea-10.sun.com by fe-emea-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KON00E00ZJO2T00@fe-emea-10.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 20 Aug 2009 09:07:30 +0100 (BST) Received: from [192.168.1.105] (cpc2-rdng20-2-0-cust917.15-3.cable.virginmedia.com [86.28.167.150]) by fe-emea-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KOO00A5L1891X80@fe-emea-10.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 20 Aug 2009 09:07:22 +0100 (BST) Date: Thu, 20 Aug 2009 09:07:21 +0100 From: Darren J Moffat Subject: PSARC/2009/443 ZFS Crypto Update [closed-approved] Sender: Darren.Moffat@sun.com To: PSARC-ext@sun.com Message-id: <4A8D0439.8060000@Sun.COM> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 User-Agent: Thunderbird 2.0.0.21 (X11/20090623) Status: RO Content-Length: 84 This case was approved in PSARC on Wednesday 19th August 2009. -- Darren J Moffat