From sacadmin Fri Mar 2 13:16:42 2007 Received: from cyber.eng.sun.com (cyber.SFBay.Sun.COM [129.146.228.109]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id l22LGgNc002109; Fri, 2 Mar 2007 13:16:42 -0800 (PST) Received: from cyber.eng.sun.com (localhost [127.0.0.1]) by cyber.eng.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l22LG2ZN029763; Fri, 2 Mar 2007 13:16:02 -0800 (PST) Received: (from ahrens@localhost) by cyber.eng.sun.com (8.13.8+Sun/8.13.8/Submit) id l22LG1fg029748; Fri, 2 Mar 2007 13:16:02 -0800 (PST) Date: Fri, 2 Mar 2007 13:16:02 -0800 (PST) From: Matthew Ahrens Message-Id: <200703022116.l22LG1fg029748@cyber.eng.sun.com> To: PSARC@sac.sfbay.sun.com Cc: zfs-eng@sun.com Subject: zfs set copies [PSARC/2007/121 Timeout: 03/16/2007] Status: RO Content-Length: 7553 Subject: PSARC FastTrack [03/16/2007]: zfs set copies Template Version: @(#)sac_nextcase %I% %G% SMI This information is Copyright 2007 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: zfs set copies 1.2. Name of Document Author/Supplier: Author: Matthew Ahrens 1.3 Date of This Document: 02 March, 2007 4. Technical Description This case adds a new option to ZFS. The stability of the option is committed, and the release binding is patch/micro. A. INTRODUCTION ZFS stores multiple copies of all metadata. This is accomplished by storing up to three DVAs (Disk Virtual Addresses) in each block pointer. This feature is known as "Ditto Blocks". When possible, the copies are stored on different disks. If they must be stored on the same disk, they will be spread out as much as possible. See bug 6410698 "ZFS metadata needs to be more highly replicated (ditto blocks)" for details on ditto blocks. This case will extend this feature to allow system administrators to store multiple copies of user data as well, on a per-filesystem basis. These copies are in addition to any redundancy provided at the pool level (mirroring, raid-z, etc). This feature has two primary use cases: 1. Single-disk pools with valuable data: If there is only one disk available (eg, a laptop, low-end desktop or misconfigured server), storing multiple copies provides protection against some types of hardware failure (eg, firmware bugs, but not whole-disk failure). 2. Machines that store data of different "importance" levels: In this use case, some data is more important than other data, and thus should be stored more redundantly. For example, consider a server with many zones. The global zone admin sets the pool's redundancy level (eg, raidz1), but a local zone admin may disagree with this assessment. Rather than copying their data manually (eg, a cron job doing cp -r), the local zone admin can simply set copies=2 on their filesystems. Another example is where the administrator has configured a pool with no redundancy (ie. no mirroring or raid-z), because most of the data in the pool is not very important. However, the pool may have several disks in it. The administrator/user may realize (perhaps later on) that some of the data really *is* important and they would like some protection against losing it if a disk fails. They may not have the option of adding more disks to mirror all of their data (cost or physical space constraints may apply here). This problem is solved by creating a new filesystem with copies=2 and putting the important data there. Now, if a disk fails, then the data in the copies=2 filesystem will not be lost. Approximately 1/4 of the data in other filesystems will be lost. (There is a small chance that a tiny fraction of the data in the copies=2 filesystem will still be lost if we were forced to put both copies on the disk that failed.) Note, this feature is not intended as a replacement for vdev-level mirroring or raid-z. When all data connected to a given machine has the same "importance" level, that can be used to pick the (one) redundancy scheme for the storage pool. B. DESCRIPTION A new property will be added, 'copies', which specifies how many copies of the given filesystem will be stored. Its value must be 1, 2, or 3. Like other properties (eg. checksum, compression), it only affects newly-written data. As such, it is recommended that the 'copies' property be set at filesystem-creation time (eg. 'zfs create -o copies=2 pool/fs'). The pool must be at least on-disk version 2 to use this feature (see 'zfs upgrade'). The forthcoming "zfs block relocation" project (bug 4852783) will allow the number of copies to be changed for existing data. By default (copies=1), only two copies of most filesystem metadata are stored. However, if we are storing multiple copies of user data, then 3 copies (the maximum) of filesystem metadata will be stored. This feature is similar to using mirroring, but differs in several important ways: * Different filesystems in the same pool can have different numbers of copies. * The space used by multiple copies is charged for (eg. in stat(2), ls -s, df(1m), du(1), zfs list, and counts against quotas and reservations). This works the same as accounting of compressed space, except that the amount of space used is more than the default, rather than less. * The storage configuration is not constrained as it is with mirroring (eg. you can have multiple copies even on a single disk, or with an odd number of disks). * Mirroring offers slightly better performance, because only one DVA needs to be allocated. * Mirroring offers slightly better redundancy, because one disk from *each* mirror can fail without data loss [1]. It is important to note that the copies provided by this feature are in addition to any redundancy provided by the pool configuration or the underlying storage. For example: * In a pool with 2-way mirrors, a filesystem with copies=1 (the default) will be stored 2 * 1 = 2 times. The filesystem can tolerate any 1 disk failing without data loss. * In a pool with 2-way mirrors, a filesystem with copies=3 will be stored with 2 * 3 = 6 times. The filesystem can tolerate any 5 disks failing without data loss (assuming that there are at least ncopies=3 mirror groups). * In a pool with single-parity raid-z a filesystem with copies=2 will be stored with 2 copies, each copy protected by its own parity block. The filesystem can tolerate any 3 disks failing without data loss (assuming that there are at least ncopies=2 raid-z groups). (Note, being able to tolerate these disk failures assumes that the copies were able to be stored on different disks. This will almost certainly be the case unless you have a small number of different-sized devices, or are running with very little free space.) [1] I'm basing this "slightly better" call on a model of random, complete-disk failures. I know that this is only an approximation. With many mirrors, most (but not all) 2-disk failures can be tolerated. With copies=2, almost no 2-top-level-vdev failures will be tolerated, because it's likely that *some* block will have both its copies on those 2 disks. With mirrors, you can arrange to mirror across controllers, which you can't do with copies. C. MANPAGE CHANGES *** zfs.man4 Tue Jun 13 10:15:38 2006 --- zfs.man5 Mon Sep 11 16:34:37 2006 *************** *** 708,714 **** --- 708,725 ---- they are inherited. + copies=1 | 2 | 3 + Controls the number of copies of data stored for this dataset. + These copies are in addition to any redundancy provided by the + pool (eg. mirroring or raid-z). The copies will be stored on + different disks if possible. The space used by multiple copies + will be charged to the associated file and dataset, changing + the 'used' property and counting against quotas and reservations. + + Changing this property only affects newly-written data. + Therefore, it is recommended that this property be set at + filesystem creation time, using the '-o copies=' option. + + Temporary Mountpoint Properties When a file system is mounted, either through mount(1M) for legacy mounts or the "zfs mount" command for normal file 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: FastTrack From sacadmin Mon Mar 5 03:28:40 2007 Received: from sfbaymail1sca.SFBay.Sun.COM (sfbaymail1sca.SFBay.Sun.COM [129.145.154.35]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id l25BSeBL029516 for ; Mon, 5 Mar 2007 03:28:40 -0800 (PST) Received: from gmp-ea-fw-1.sun.com (gmpes-gis-mail-2.UK.Sun.COM [129.156.42.6]) by sfbaymail1sca.SFBay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l25BSdru001484 for ; Mon, 5 Mar 2007 03:28:39 -0800 (PST) Received: from d1-emea-09.sun.com ([192.18.2.119]) by gmp-ea-fw-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l25BSXDi006742 for ; Mon, 5 Mar 2007 11:28:33 GMT Received: from conversion-daemon.d1-emea-09.sun.com by d1-emea-09.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JEF00001H49J600@d1-emea-09.sun.com> (original mail from Darren.Moffat@Sun.COM) for PSARC@sac.sfbay.sun.com; Mon, 05 Mar 2007 11:28:33 +0000 (GMT) Received: from [192.168.73.101] (nessieroo.force9.co.uk [81.174.224.49]) by d1-emea-09.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JEF0082NH7IYZ10@d1-emea-09.sun.com>; Mon, 05 Mar 2007 11:28:32 +0000 (GMT) Date: Mon, 05 Mar 2007 11:28:29 +0000 From: Darren J Moffat Subject: Re: zfs set copies [PSARC/2007/121 Timeout: 03/16/2007] In-reply-to: <200703022116.l22LG1fg029748@cyber.eng.sun.com> Sender: Darren.Moffat@Sun.COM To: Matthew Ahrens Cc: PSARC@sac.sfbay.sun.com, zfs-eng@Sun.COM, James.Hughes@Sun.COM Message-id: <45EBFEDD.5060804@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200703022116.l22LG1fg029748@cyber.eng.sun.com> User-Agent: Thunderbird 1.5.0.8 (X11/20061127) Status: RO Content-Length: 714 I think my questions are border line internal design / architecture but I'll ask here anyway. I assume all the copies are written in the same transaction group ? Are the copies created before compression or after ? that is will the compress part of the ZIO pipeline be run always once or ncopies times ? This isn't really that important for compression but it is very important for crypto. If we only pass through the part of the pipeline that encrypts the data once then we "spark off" the ncopies blocks we will end up with the same ciphertext written to disk ncopies times. I have a feeling that isn't a good thing to be doing in this case - I'll check with some experts (cc'd). -- Darren J Moffat From sacadmin Mon Mar 5 07:52:49 2007 Received: from zion.eng.sun.com (zion.SFBay.Sun.COM [129.146.17.75]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id l25FqnEO008403 for ; Mon, 5 Mar 2007 07:52:49 -0800 (PST) Received: from [192.9.61.138] (punchin-ahrens [192.9.61.138]) by zion.eng.sun.com (8.13.7+Sun/8.13.7) with ESMTP id l25FqmmB005212; Mon, 5 Mar 2007 07:52:48 -0800 (PST) Message-ID: <45EC3CC3.8080206@sun.com> Date: Mon, 05 Mar 2007 07:52:35 -0800 From: Matthew Ahrens User-Agent: Thunderbird 1.5.0.9 (Macintosh/20061207) MIME-Version: 1.0 To: Darren J Moffat CC: Matthew Ahrens , PSARC@sac.sfbay.sun.com, zfs-eng@sun.com, James.Hughes@sun.com Subject: Re: zfs set copies [PSARC/2007/121 Timeout: 03/16/2007] References: <200703022116.l22LG1fg029748@cyber.eng.sun.com> <45EBFEDD.5060804@Sun.COM> In-Reply-To: <45EBFEDD.5060804@Sun.COM> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Status: RO Content-Length: 520 Darren J Moffat wrote: > I think my questions are border line internal design / architecture but > I'll ask here anyway. Indeed, this case simply utilizes the existing ditto block architecture. > I assume all the copies are written in the same transaction group ? Yes. > Are the copies created before compression or after ? that is will the > compress part of the ZIO pipeline be run always once or ncopies times ? The logical data is compressed once (it would be a waste of CPU cycles to do otherwise). --matt From sacadmin Mon Mar 5 07:55:56 2007 Received: from sfbaymail1sca.SFBay.Sun.COM (sfbaymail1sca.SFBay.Sun.COM [129.145.154.35]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id l25FtuYF008916 for ; Mon, 5 Mar 2007 07:55:56 -0800 (PST) Received: from gmp-ea-fw-1.sun.com (gmpes-gis-mail-1.UK.Sun.COM [129.156.42.5]) by sfbaymail1sca.SFBay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l25FttwM027019 for ; Mon, 5 Mar 2007 07:55:55 -0800 (PST) Received: from d1-emea-09.sun.com (d1-emea-09.sun.com [192.18.2.119]) by gmp-ea-fw-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l25FtnEc020832 for ; Mon, 5 Mar 2007 15:55:49 GMT Received: from conversion-daemon.d1-emea-09.sun.com by d1-emea-09.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JEF00801TJCAU00@d1-emea-09.sun.com> (original mail from Darren.Moffat@Sun.COM) for PSARC@sac.sfbay.sun.com; Mon, 05 Mar 2007 15:55:49 +0000 (GMT) Received: from [192.168.73.101] (nessieroo.force9.co.uk [81.174.224.49]) by d1-emea-09.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JEF00871TKXYY20@d1-emea-09.sun.com>; Mon, 05 Mar 2007 15:55:46 +0000 (GMT) Date: Mon, 05 Mar 2007 15:55:45 +0000 From: Darren J Moffat Subject: Re: zfs set copies [PSARC/2007/121 Timeout: 03/16/2007] In-reply-to: <45EC3CC3.8080206@sun.com> Sender: Darren.Moffat@Sun.COM To: Matthew Ahrens Cc: Matthew Ahrens , PSARC@sac.sfbay.sun.com, zfs-eng@Sun.COM, James.Hughes@Sun.COM Message-id: <45EC3D81.4070402@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200703022116.l22LG1fg029748@cyber.eng.sun.com> <45EBFEDD.5060804@Sun.COM> <45EC3CC3.8080206@sun.com> User-Agent: Thunderbird 1.5.0.8 (X11/20061127) Status: RO Content-Length: 775 Matthew Ahrens wrote: > Darren J Moffat wrote: >> I think my questions are border line internal design / architecture >> but I'll ask here anyway. > > Indeed, this case simply utilizes the existing ditto block architecture. > >> I assume all the copies are written in the same transaction group ? > > Yes. > >> Are the copies created before compression or after ? that is will the >> compress part of the ZIO pipeline be run always once or ncopies times ? > > The logical data is compressed once (it would be a waste of CPU cycles > to do otherwise). I've thought about this a bit more and in hindsight I don't think this case introduces any new risks with respect to multiple copies of the ciphertext that mirroring and raidz didn't already. -- Darren J Moffat From sacadmin Mon Mar 5 09:09:42 2007 Received: from sfbaymail2sca.sfbay.sun.com (sfbaymail2sca.SFBay.Sun.COM [129.145.155.42]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id l25H9ga3010960 for ; Mon, 5 Mar 2007 09:09:42 -0800 (PST) Received: from nwk-ea-fw-1.sun.com (nwkes-gis-mail-2.SFBay.Sun.COM [10.4.134.6]) by sfbaymail2sca.sfbay.sun.com (8.13.6+Sun/8.12.10/ENSMAIL,v2.2) with ESMTP id l25H9fCn022268 for ; Mon, 5 Mar 2007 09:09:41 -0800 (PST) Received: from d1-sfbay-10.sun.com ([192.18.39.120]) by nwk-ea-fw-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l25H9avL009935 for ; Mon, 5 Mar 2007 09:09:36 -0800 (PST) Received: from conversion-daemon.d1-sfbay-10.sun.com by d1-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JEF00J01WXLOI00@d1-sfbay-10.sun.com> (original mail from Ed.Gould@Sun.COM) for PSARC@sac.sfbay.sun.com; Mon, 05 Mar 2007 09:09:36 -0800 (PST) Received: from [129.146.106.203] by d1-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JEF00MZ3X00D6X5@d1-sfbay-10.sun.com>; Mon, 05 Mar 2007 09:09:36 -0800 (PST) Date: Mon, 05 Mar 2007 09:09:25 -0800 From: Ed Gould Subject: Re: zfs set copies [PSARC/2007/121 Timeout: 03/16/2007] In-reply-to: <45EC3D81.4070402@Sun.COM> Sender: Ed.Gould@Sun.COM To: Darren J Moffat Cc: Matthew Ahrens , Matthew Ahrens , PSARC@sac.sfbay.sun.com, zfs-eng@Sun.COM, James.Hughes@Sun.COM Message-id: <45EC4EC5.7020807@sun.com> Organization: Sun Cluster Engineering - GDD MIME-version: 1.0 Content-type: multipart/mixed; boundary="Boundary_(ID_eRVdDLKZHxXSa9A1FyKlOw)" References: <200703022116.l22LG1fg029748@cyber.eng.sun.com> <45EBFEDD.5060804@Sun.COM> <45EC3CC3.8080206@sun.com> <45EC3D81.4070402@Sun.COM> User-Agent: Thunderbird 1.5 (X11/20060113) Status: RO Content-Length: 1422 This is a multi-part message in MIME format. --Boundary_(ID_eRVdDLKZHxXSa9A1FyKlOw) Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT Darren J Moffat wrote: > I've thought about this a bit more and in hindsight I don't think this > case introduces any new risks with respect to multiple copies of the > ciphertext that mirroring and raidz didn't already. Unless I'm missing something, this must be true, since it is known that the blocks are replicas, hence it is known that they represent the same plain text. Replicating the cipher text couldn't add any information unless there was some possibility that the same cipher text represented *different* plain text. Otherwise, an attacker would just make as many copies as necessary to crack the cipher. (In other words, mirroring and other forms of redundancy never compromise the encryption.) -- --Ed --Boundary_(ID_eRVdDLKZHxXSa9A1FyKlOw) Content-type: text/x-vcard; name=ed.gould.vcf; charset=utf-8 Content-transfer-encoding: 7BIT Content-disposition: attachment; filename=ed.gould.vcf begin:vcard fn:Ed Gould n:Gould;Ed org:Sun Microsystems, Inc.;Solaris Cluster adr;dom:M/S UMPK17-201;;17 Network Circle;Menlo Park;CA;94025 email;internet:ed.gould@sun.com title:File System Architect, PSARC Chair tel;work:+1.650.786.4937 x-mozilla-html:FALSE version:2.1 end:vcard --Boundary_(ID_eRVdDLKZHxXSa9A1FyKlOw)-- From sacadmin Mon Mar 5 11:34:04 2007 Received: from eastmail1bur.East.Sun.COM (eastmail1bur.East.Sun.COM [129.148.9.49]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id l25JY3o4014495 for ; Mon, 5 Mar 2007 11:34:04 -0800 (PST) Received: from thunk.east.sun.com (thunk.East.Sun.COM [129.148.174.66]) by eastmail1bur.East.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l25JY02g020838; Mon, 5 Mar 2007 14:34:00 -0500 (EST) Received: from [IPv6:::1] (localhost [IPv6:::1]) by thunk.east.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l25JY02q003839; Mon, 5 Mar 2007 14:34:00 -0500 (EST) Subject: Re: zfs set copies [PSARC/2007/121 Timeout: 03/16/2007] From: Bill Sommerfeld To: Darren J Moffat Cc: Matthew Ahrens , Matthew Ahrens , PSARC@sac.sfbay.sun.com, zfs-eng@sun.com, James.Hughes@sun.com In-Reply-To: <45EC3D81.4070402@Sun.COM> References: <200703022116.l22LG1fg029748@cyber.eng.sun.com> <45EBFEDD.5060804@Sun.COM> <45EC3CC3.8080206@sun.com> <45EC3D81.4070402@Sun.COM> Content-Type: text/plain Date: Mon, 05 Mar 2007 14:33:58 -0500 Message-Id: <1173123238.3489.21.camel@thunk> Mime-Version: 1.0 X-Mailer: Evolution 2.8.1.1 Content-Transfer-Encoding: 7bit Status: RO Content-Length: 1358 On Mon, 2007-03-05 at 15:55 +0000, Darren J Moffat wrote: > I've thought about this a bit more and in hindsight I don't think this > case introduces any new risks with respect to multiple copies of the > ciphertext that mirroring and raidz didn't already. I guess it depends on your security goal for zfs crypto -- a strong security goal would be to have someone without knowledge of the key be unable to distinguish between a disk with an encrypted pool and a disk which had been wiped with random bits. I'd think that mirroring and raidz would need to be looked at separately; an attacker looking at ciphertext of mirrors would see two disks with identical content. they'd be able to tell that two disks were mirrors of each other and probably be able to distinguish between a heavily used pool vs. one where most of the disk blocks were never written (though you could avoid that by overwriting the disks with random data before putting them into service). with both raidz and ditto blocks, you'd get far more information about the relative proportion of small writes and metadata, perhaps revealing additional information about the workload presented to the pool. I think we're deep into secondary threats here, but it does represent a case where the disk contents would be clearly distinguishable from a pile of random bits. - Bill From sacadmin Mon Mar 5 11:52:57 2007 Received: from cathy.sfbay.sun.com (cathy.SFBay.Sun.COM [129.146.228.63]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id l25JqvrZ014639 for ; Mon, 5 Mar 2007 11:52:57 -0800 (PST) Received: from cathy.sfbay.sun.com (localhost [127.0.0.1]) by cathy.sfbay.sun.com (8.14.0+Sun/8.14.0) with ESMTP id l25JqvYD104888; Mon, 5 Mar 2007 11:52:57 -0800 (PST) Received: (from bonwick@localhost) by cathy.sfbay.sun.com (8.14.0+Sun/8.14.0/Submit) id l25Jquca104887; Mon, 5 Mar 2007 11:52:56 -0800 (PST) Date: Mon, 5 Mar 2007 11:52:56 -0800 From: Jeff Bonwick To: Bill Sommerfeld Cc: Darren J Moffat , Matthew Ahrens , Matthew Ahrens , PSARC@sac.sfbay.sun.com, zfs-eng@sun.com, James.Hughes@sun.com Subject: Re: zfs set copies [PSARC/2007/121 Timeout: 03/16/2007] Message-ID: <20070305195256.GA104875@eng.sun.com> References: <200703022116.l22LG1fg029748@cyber.eng.sun.com> <45EBFEDD.5060804@Sun.COM> <45EC3CC3.8080206@sun.com> <45EC3D81.4070402@Sun.COM> <1173123238.3489.21.camel@thunk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1173123238.3489.21.camel@thunk> User-Agent: Mutt/1.4.1i Status: RO Content-Length: 983 Yes. For that reason, when we implement ZFS on-disk encryption, we'll have two options: per-dataset and per-pool. In the per-pool model, *everything* will be encrypted -- the disk really will look like random noise. In the per-dataset case, we'll encrypt everything but the block pointers. That way, if you forget the key to a dataset, at least you can still free it and get the blocks back. From just the block pointers you can't get user data or file names, but you can get structural information like distribution of file sizes and birth times. That's probably not useful in any material sense, btu for the truly paranoid, pool-wide encryption solves the problem. Having said that, a reminder: *this* case is about ditto blocks! ;-) Jeff On Mon, Mar 05, 2007 at 02:33:58PM -0500, Bill Sommerfeld wrote: > I think we're deep into secondary threats here, but it does represent a > case where the disk contents would be clearly distinguishable from a > pile of random bits. From sacadmin Mon Mar 5 12:31:02 2007 Received: from eastmail1bur.East.Sun.COM (eastmail1bur.East.Sun.COM [129.148.9.49]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id l25KV2eN015807 for ; Mon, 5 Mar 2007 12:31:02 -0800 (PST) Received: from thunk.east.sun.com (thunk.East.Sun.COM [129.148.174.66]) by eastmail1bur.East.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l25KUtjZ002528; Mon, 5 Mar 2007 15:30:55 -0500 (EST) Received: from [IPv6:::1] (localhost [IPv6:::1]) by thunk.east.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l25KUtbx004011; Mon, 5 Mar 2007 15:30:55 -0500 (EST) Subject: Re: zfs set copies [PSARC/2007/121 Timeout: 03/16/2007] From: Bill Sommerfeld To: Jeff Bonwick Cc: Darren J Moffat , Matthew Ahrens , Matthew Ahrens , PSARC@sac.sfbay.sun.com, zfs-eng@sun.com, James.Hughes@sun.com In-Reply-To: <20070305195256.GA104875@eng.sun.com> References: <200703022116.l22LG1fg029748@cyber.eng.sun.com> <45EBFEDD.5060804@Sun.COM> <45EC3CC3.8080206@sun.com> <45EC3D81.4070402@Sun.COM> <1173123238.3489.21.camel@thunk> <20070305195256.GA104875@eng.sun.com> Content-Type: text/plain Date: Mon, 05 Mar 2007 15:30:54 -0500 Message-Id: <1173126654.3489.47.camel@thunk> Mime-Version: 1.0 X-Mailer: Evolution 2.8.1.1 Content-Transfer-Encoding: 7bit Status: RO Content-Length: 281 On Mon, 2007-03-05 at 11:52 -0800, Jeff Bonwick wrote: > Having said that, a reminder: *this* case is about ditto blocks! ;-) oops, yes, I meant to include a "well, we're wandering off into a future case" comment but it somehow didn't make it into my message... - Bill From sacadmin Mon Mar 5 16:12:01 2007 Received: from sfbaymail2sca.sfbay.sun.com (sfbaymail2sca.SFBay.Sun.COM [129.145.155.42]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id l260C1Nh022699 for ; Mon, 5 Mar 2007 16:12:01 -0800 (PST) Received: from brmea-mail-2.sun.com (brmea-mail-2.Sun.COM [192.18.98.43]) by sfbaymail2sca.sfbay.sun.com (8.13.6+Sun/8.12.10/ENSMAIL,v2.2) with ESMTP id l260C1XS012319 for ; Mon, 5 Mar 2007 16:12:01 -0800 (PST) Received: from fe-amer-05.sun.com ([192.18.108.179]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l260C1Nx004015 for ; Tue, 6 Mar 2007 00:12:01 GMT Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JEG00I01G89SU00@mail-amer.sun.com> (original mail from Torrey.McMahon@Sun.COM) for PSARC@sac.sfbay.sun.com; Mon, 05 Mar 2007 17:12:01 -0700 (MST) Received: from [192.168.1.100] ([69.143.27.128]) by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JEG00CKDGK0AEX1@mail-amer.sun.com>; Mon, 05 Mar 2007 17:12:00 -0700 (MST) Date: Mon, 05 Mar 2007 19:12:00 -0500 From: Torrey McMahon Subject: Re: zfs set copies [PSARC/2007/121 Timeout: 03/16/2007] In-reply-to: <200703022116.l22LG1fg029748@cyber.eng.sun.com> Sender: Torrey.McMahon@Sun.COM To: Matthew Ahrens Cc: PSARC@sac.sfbay.sun.com, zfs-eng@Sun.COM Message-id: <45ECB1D0.2050302@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200703022116.l22LG1fg029748@cyber.eng.sun.com> User-Agent: Thunderbird 2.0b2 (Windows/20070116) Status: RO Content-Length: 429 Matthew Ahrens wrote: > * The space used by multiple copies is charged for (eg. in stat(2), > ls -s, df(1m), du(1), zfs list, and counts against quotas and > reservations). This works the same as accounting of compressed space, > except that the amount of space used is more than the default, rather > than less. Are there any user level commands that can inform a user why they are getting "charged" for extra space? From sacadmin Mon Mar 5 16:16:00 2007 Received: from zion.eng.sun.com (zion.SFBay.Sun.COM [129.146.17.75]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id l260G08j022790 for ; Mon, 5 Mar 2007 16:16:00 -0800 (PST) Received: from [129.146.228.109] (cyber [129.146.228.109]) by zion.eng.sun.com (8.13.7+Sun/8.13.7) with ESMTP id l260FwBB026877; Mon, 5 Mar 2007 16:15:59 -0800 (PST) Message-ID: <45ECB296.1030206@sun.com> Date: Mon, 05 Mar 2007 16:15:18 -0800 From: Matthew Ahrens User-Agent: Thunderbird 1.5.0.8 (X11/20061204) MIME-Version: 1.0 To: Torrey McMahon CC: Matthew Ahrens , PSARC@sac.sfbay.sun.com, zfs-eng@sun.com Subject: Re: zfs set copies [PSARC/2007/121 Timeout: 03/16/2007] References: <200703022116.l22LG1fg029748@cyber.eng.sun.com> <45ECB1D0.2050302@sun.com> In-Reply-To: <45ECB1D0.2050302@sun.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Status: RO Content-Length: 566 Torrey McMahon wrote: > Matthew Ahrens wrote: >> * The space used by multiple copies is charged for (eg. in stat(2), >> ls -s, df(1m), du(1), zfs list, and counts against quotas and >> reservations). This works the same as accounting of compressed space, >> except that the amount of space used is more than the default, rather >> than less. > > Are there any user level commands that can inform a user why they are > getting "charged" for extra space? They can do 'zfs get copies ' (same with compression -- 'zfs get compression'). --matt From sacadmin Tue Mar 6 04:52:56 2007 Received: from sfbaymail2sca.sfbay.sun.com (sfbaymail2sca.SFBay.Sun.COM [129.145.155.42]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id l26CquUg013623 for ; Tue, 6 Mar 2007 04:52:56 -0800 (PST) Received: from gmp-ea-fw-1.sun.com (gmpes-gis-mail-1.UK.Sun.COM [129.156.42.5]) by sfbaymail2sca.sfbay.sun.com (8.13.6+Sun/8.12.10/ENSMAIL,v2.2) with ESMTP id l26Cqq01019956 for ; Tue, 6 Mar 2007 04:52:53 -0800 (PST) Received: from d1-emea-10.sun.com (d1-emea-10.sun.com [192.18.2.120]) by gmp-ea-fw-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l26CqknU029040 for ; Tue, 6 Mar 2007 12:52:47 GMT Received: from conversion-daemon.d1-emea-10.sun.com by d1-emea-10.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JEH00J01FLL4D00@d1-emea-10.sun.com> (original mail from Darren.Moffat@Sun.COM) for PSARC@sac.sfbay.sun.com; Tue, 06 Mar 2007 12:52:46 +0000 (GMT) Received: from [192.168.73.101] (nessieroo.force9.co.uk [81.174.224.49]) by d1-emea-10.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JEH003R4FRXSG00@d1-emea-10.sun.com>; Tue, 06 Mar 2007 12:52:46 +0000 (GMT) Date: Tue, 06 Mar 2007 12:52:45 +0000 From: Darren J Moffat Subject: Re: zfs set copies [PSARC/2007/121 Timeout: 03/16/2007] In-reply-to: <1173123238.3489.21.camel@thunk> Sender: Darren.Moffat@Sun.COM To: Bill Sommerfeld Cc: Matthew Ahrens , Matthew Ahrens , PSARC@sac.sfbay.sun.com, zfs-eng@Sun.COM, James.Hughes@Sun.COM Message-id: <45ED641D.2010109@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200703022116.l22LG1fg029748@cyber.eng.sun.com> <45EBFEDD.5060804@Sun.COM> <45EC3CC3.8080206@sun.com> <45EC3D81.4070402@Sun.COM> <1173123238.3489.21.camel@thunk> User-Agent: Thunderbird 1.5.0.8 (X11/20061127) Status: RO Content-Length: 746 Bill Sommerfeld wrote: > On Mon, 2007-03-05 at 15:55 +0000, Darren J Moffat wrote: >> I've thought about this a bit more and in hindsight I don't think this >> case introduces any new risks with respect to multiple copies of the >> ciphertext that mirroring and raidz didn't already. > > I guess it depends on your security goal for zfs crypto -- a strong > security goal would be to have someone without knowledge of the key be > unable to distinguish between a disk with an encrypted pool and a disk > which had been wiped with random bits. That is not a goal of zfs crypto as it is currently specified. If that is what one wants then I think doing the crypto at the block device or lower level is more appropriate. -- Darren J Moffat From sacadmin Wed Mar 14 13:03:26 2007 Received: from zion.eng.sun.com (zion.SFBay.Sun.COM [129.146.17.75]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id l2EK3QHI016753 for ; Wed, 14 Mar 2007 13:03:26 -0700 (PDT) Received: from [129.146.228.185] (dhcp-umpk17-228-185 [129.146.228.185]) by zion.eng.sun.com (8.13.7+Sun/8.13.7) with ESMTP id l2EK3PMw011793; Wed, 14 Mar 2007 13:03:26 -0700 (PDT) Message-ID: <45F8550D.7060402@sun.com> Date: Wed, 14 Mar 2007 13:03:25 -0700 From: Matthew Ahrens User-Agent: Thunderbird 1.5.0.9 (Macintosh/20061207) MIME-Version: 1.0 To: Matthew Ahrens CC: PSARC@sac.sfbay.sun.com, zfs-eng@sun.com Subject: Re: zfs set copies [PSARC/2007/121 Timeout: 03/16/2007] References: <200703022116.l22LG1fg029748@cyber.eng.sun.com> In-Reply-To: <200703022116.l22LG1fg029748@cyber.eng.sun.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Status: RO Content-Length: 129 Matthew Ahrens wrote: > Subject: PSARC FastTrack [03/16/2007]: zfs set copies The case was approved at today's meeting. --matt