From tim.haley@oracle.com Sun Mar 28 12:12:46 2010 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2SJCkW7027019 for ; Sun, 28 Mar 2010 12:12:46 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o2SJCkbQ007038; Sun, 28 Mar 2010 12:12:46 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L0000I03AP9C500@brm-avmta-1.central.sun.com>; Sun, 28 Mar 2010 13:12:45 -0600 (MDT) Received: from sca-ea-mail-2.sun.com ([192.18.43.25]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L0000HC5AP9IB60@brm-avmta-1.central.sun.com>; Sun, 28 Mar 2010 13:12:45 -0600 (MDT) Received: from rcsinet13.oracle.com (rcsinet13.oracle.com [148.87.113.125]) by sca-ea-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o2SJCib5020972; Sun, 28 Mar 2010 19:12:45 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by rcsinet13.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o2SJCfbO011367; Sun, 28 Mar 2010 19:12:41 +0000 (GMT) Received: from abhmt010.oracle.com by acsmt353.oracle.com with ESMTP id 117755561269803540; Sun, 28 Mar 2010 12:12:20 -0700 Received: from Giles.local (/10.7.250.184) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Sun, 28 Mar 2010 12:12:20 -0700 Date: Sun, 28 Mar 2010 13:12:16 -0600 From: Tim Haley Subject: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] To: PSARC-ext@sun.com, ZFS Team Message-id: <4BAFAA10.2070706@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090203.4BAFAA2A.00CB:SCFMA4539814,ss=1,fgs=0 User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3 Status: RO Content-Length: 5842 I am sponsoring the following fast-track on behalf of myself. This case introduces a new zfs sub-command for describing differences between snapshots in a zfs hierarchy. A delegated permission and read-only system attribute are also introduced to support the sub-command. The case requests micro/patch binding. Template Version: @(#)sac_nextcase 1.69 02/15/10 SMI This information is Copyright 2010 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: zfs diff 1.2. Name of Document Author/Supplier: Author: Tim Haley 1.3 Date of This Document: 28 March, 2010 4. Technical Description There is a long-standing RFE for zfs to be able to describe what has changed between the snapshots of a dataset. To provide this capability, we propose a new 'zfs diff' sub-command. When run with appropriate privilege the sub-command describes what file system level changes have occurred between the requested snapshots. A diff between the current version of the file system and one of its snapshots is also supported. Five types of change are described: o File/Directory modified o File/Directory present in older snapshot but not newer o File/Directory present in newer snapshot but not older o File/Directory renamed o File link count changed Diffs can be performed if the user is delegated the "diff" permission. The "diff" permission is being introduced by this case. Diffs can also be performed without the "diff" permission, if the user has appropriate privilege. For diffs between existing snapshots, the necessary privilege is {PRIV_SYS_CONFIG}. For diff between the current file system and a snapshot {PRIV_SYS_MOUNT} is also necessary. Also introduced by this case is a system attribute on zfs files called 'generation'. This attribute is part of the XATTR_VIEW_READONLY described in PSARC 2007/315. It is generated automatically by the ZFS module. Man page changes: --- fgetattr.3c.rogi Sun Mar 21 18:47:29 2010 +++ fgetattr.3c Sun Mar 21 18:50:50 2010 @@ -97,6 +97,7 @@ XATTR_VIEW_READONLY A_FSID uint64_value A_OPAQUE boolean_value A_AV_SCANSTAMP uint8_array[] + A_GEN uint64_value XATTR_VIEW_READWRITE A_READONLY boolean_value A_HIDDEN boolean_value A_SYSTEM boolean_value --- zfs.1m.rogi Sun Mar 21 17:01:04 2010 +++ zfs.1m Sun Mar 28 12:20:04 2010 @@ -165,6 +165,9 @@ zfs release [-r] tag snapshot... + zfs diff snapshot snapshot|filesystem + + DESCRIPTION The zfs command configures ZFS datasets within a ZFS storage pool, as described in zpool(1M). A dataset is identified by @@ -1638,7 +1641,41 @@ size, the resulting behavior is undefined. + zfs diff snapshot snapshot | filesystem + Gives a high level description of the differences between a + snapshot and a descendant dataset. The descendant may either + be a later snapshot of the dataset or the current dataset. + For each file system object that has undergone a change + between the original snapshot and the descendant, the type of + change is described along with the name of the file or + directory. In the case of a rename, both the old and new + names are shown. + + The type of change is described with a single character: + + + Indicates the file/directory was added in the later dataset + - Indicates the file/directory was removed in the later dataset + M Indicates the file/directory was modified in the later dataset + R Indicates the file/directory was renamed in the later dataset + + If the modification involved a change in the link count of a + file, the change will be expressed as a delta within + parentheses on the modification line. Example outputs are + below: + + M /myfiles/ + M /myfiles/link_to_me (+1) + R /myfiles/rename_me -> /myfiles/renamed + - /myfiles/delete_me + + /myfiles/new_file + + Users must be granted the diff permission with zfs allow in + order to use this sub-command, unless they already have the + {PRIV_SYS_CONFIG} privilege and, in the current file system + versus snapshot case, the {PRIV_SYS_MOUNT} privilege. + + zfs destroy [-rRf] filesystem|volume @@ -2733,6 +2770,7 @@ 'mount' ability in the origin file system create subcommand Must also have the 'mount' ability + diff subcommand destroy subcommand Must also have the 'mount' ability hold subcommand Allows adding a user hold to a snapshot mount subcommand Allows mount/umount of ZFS datasets @@ -3551,7 +3589,12 @@ tion Guide. +NOTES + A file or directory described as modified by the diff sub-command + may have been modified in multiple ways. Any action which causes + a change in the st_ctime (see stat(2)) is grounds for reporting a + modification. 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: FastTrack 6.6. ARC Exposure: open From Darren.Reed@sun.com Sun Mar 28 23:43:49 2010 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2T6hnDt004434 for ; Sun, 28 Mar 2010 23:43:49 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o2T6hlpo012796; Sun, 28 Mar 2010 23:43:49 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L010070P6P0P900@brm-avmta-1.central.sun.com>; Mon, 29 Mar 2010 00:43:48 -0600 (MDT) Received: from gmp-eb-inf-1.sun.com ([192.18.6.21]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L0100D026OZNO80@brm-avmta-1.central.sun.com>; Mon, 29 Mar 2010 00:43:48 -0600 (MDT) Received: from fe-emea-10.sun.com (gmp-eb-lb-1-fe1.eu.sun.com [192.18.6.7] (may be forged)) by gmp-eb-inf-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o2T6hkrm029057; Mon, 29 Mar 2010 06:43:47 +0000 (GMT) Received: from conversion-daemon.fe-emea-10.sun.com by fe-emea-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0L0100F006NXP300@fe-emea-10.sun.com>; Mon, 29 Mar 2010 07:43:35 +0100 (BST) Received: from [129.145.154.61] ([unknown] [129.145.154.61]) by fe-emea-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0L0100LSV6OG2660@fe-emea-10.sun.com>; Mon, 29 Mar 2010 07:43:35 +0100 (BST) Date: Sun, 28 Mar 2010 23:43:28 -0700 From: Darren Reed Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <4BAFAA10.2070706@oracle.com> Sender: Darren.Reed@sun.com To: Tim Haley Cc: PSARC-ext@sun.com, ZFS Team Message-id: <4BB04C10.5050300@Sun.COM> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <4BAFAA10.2070706@oracle.com> User-Agent: Thunderbird 2.0.0.23 (X11/20090910) Status: RO Content-Length: 2576 On 03/28/10 12:12, Tim Haley wrote: > I am sponsoring the following fast-track on behalf of myself. This > case introduces a new zfs sub-command for describing differences > between snapshots in a zfs hierarchy. A delegated permission and > read-only system attribute are also introduced to support the > sub-command. The case requests micro/patch binding. > > Template Version: @(#)sac_nextcase 1.69 02/15/10 SMI > This information is Copyright 2010 Sun Microsystems > 1. Introduction > 1.1. Project/Component Working Name: > zfs diff > 1.2. Name of Document Author/Supplier: > Author: Tim Haley > 1.3 Date of This Document: > 28 March, 2010 > > 4. Technical Description > > There is a long-standing RFE for zfs to be able to describe > what has changed between the snapshots of a dataset. > To provide this capability, we propose a new 'zfs diff' > sub-command. When run with appropriate privilege the > sub-command describes what file system level changes have > occurred between the requested snapshots. A diff between the > current version of the file system and one of its snapshots is > also supported. > > Five types of change are described: > > o File/Directory modified > o File/Directory present in older snapshot but not newer > o File/Directory present in newer snapshot but not older > o File/Directory renamed > o File link count changed > > Diffs can be performed if the user is delegated the "diff" > permission. The "diff" permission is being introduced by this > case. Diffs can also be performed without the "diff" > permission, if the user has appropriate privilege. For diffs > between existing snapshots, the necessary privilege is > {PRIV_SYS_CONFIG}. For diff between the current file system > and a snapshot {PRIV_SYS_MOUNT} is also necessary. > > Also introduced by this case is a system attribute on zfs files > called 'generation'. This attribute is part of the > XATTR_VIEW_READONLY described in PSARC 2007/315. It is > generated automatically by the ZFS module. Does "zfs diff" work between different versions of zfs filesystems? If so, what are the restrictions? The documentation for "zfs upgrade" isn't clear on if you can upgrade a filesystem without upgrading its snapshot or vice versa. What commitment level are you seeking for the subcommand? And its output? Darren From Darren.Moffat@oracle.com Mon Mar 29 01:24:48 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2T8Om9R023506 for ; Mon, 29 Mar 2010 01:24:48 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o2T8Om82056374; Mon, 29 Mar 2010 02:24:48 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L0100H0NBDBRI00@nwk-avmta-2.sfbay.sun.com>; Mon, 29 Mar 2010 01:24:47 -0700 (PDT) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L01007F8BDBF590@nwk-avmta-2.sfbay.sun.com>; Mon, 29 Mar 2010 01:24:47 -0700 (PDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o2T8Ol9s007646; Mon, 29 Mar 2010 08:24:47 +0000 (GMT) Received: from acsmt355.oracle.com (acsmt355.oracle.com [141.146.40.155]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o2T3lXti001372; Mon, 29 Mar 2010 08:24:45 +0000 (GMT) Received: from abhmt010.oracle.com by acsmt355.oracle.com with ESMTP id 124730781269850971; Mon, 29 Mar 2010 01:22:51 -0700 Received: from [10.7.251.221] (/10.7.251.221) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 29 Mar 2010 01:22:51 -0700 Date: Mon, 29 Mar 2010 09:22:48 +0100 From: Darren J Moffat Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <4BAFAA10.2070706@oracle.com> To: Tim Haley Cc: PSARC-ext@sun.com, ZFS Team Message-id: <4BB06358.9020502@Oracle.COM> Organization: Oracle Solaris Security MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt355.oracle.com [141.146.40.155] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A0B0204.4BB063CE.0258:SCFMA4539814,ss=1,fgs=0 References: <4BAFAA10.2070706@oracle.com> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.7) Gecko/20100214 Lightning/1.0b1 Thunderbird/3.0.1 Status: RO Content-Length: 437 On 28/03/2010 20:12, Tim Haley wrote: > I am sponsoring the following fast-track on behalf of myself. This > case introduces a new zfs sub-command for describing differences > between snapshots in a zfs hierarchy. A delegated permission and > read-only system attribute are also introduced to support the > sub-command. The case requests micro/patch binding. I'm happy with the case as specified so it gets my +1. -- Darren J Moffat From Joep.Vesseur@sun.com Mon Mar 29 01:44:46 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2T8ik41023709 for ; Mon, 29 Mar 2010 01:44:46 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o2T8igkv006391; Mon, 29 Mar 2010 02:44:45 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L0100M0XCAKHO00@nwk-avmta-1.sfbay.Sun.COM>; Mon, 29 Mar 2010 01:44:44 -0700 (PDT) Received: from gmp-eb-inf-2.sun.com ([192.18.6.24]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L0100I5SCAIK980@nwk-avmta-1.sfbay.Sun.COM>; Mon, 29 Mar 2010 01:44:43 -0700 (PDT) Received: from fe-emea-10.sun.com (gmp-eb-lb-1-fe1.eu.sun.com [192.18.6.7] (may be forged)) by gmp-eb-inf-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o2T8ig5Z021596; Mon, 29 Mar 2010 08:44:42 +0000 (GMT) Received: from conversion-daemon.fe-emea-10.sun.com by fe-emea-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0L0100800A8VTO00@fe-emea-10.sun.com>; Mon, 29 Mar 2010 09:44:21 +0100 (BST) Received: from enigma.Holland.Sun.COM ([unknown] [10.16.117.32]) by fe-emea-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0L01005EFC9Q2NI0@fe-emea-10.sun.com>; Mon, 29 Mar 2010 09:44:14 +0100 (BST) Date: Mon, 29 Mar 2010 10:44:13 +0200 From: Joep Vesseur Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <4BAFAA10.2070706@oracle.com> Sender: Joep.Vesseur@sun.com To: Tim Haley Cc: PSARC-ext@sun.com, ZFS Team Message-id: <4BB0685D.7090409@Sun.COM> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <4BAFAA10.2070706@oracle.com> User-Agent: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.9.1.5) Gecko/20091201 Lightning/1.0b1 Thunderbird/3.0 Status: RO Content-Length: 588 On 03/28/10 21:12, Tim Haley wrote: > Five types of change are described: > > o File/Directory modified > o File/Directory present in older snapshot but not newer > o File/Directory present in newer snapshot but not older > o File/Directory renamed > o File link count changed Is there any provision made to detect/display file property changes? Or will those be covered by a change to the parent-directory? > [...] For diffs > between existing snapshots, the necessary privilege is > {PRIV_SYS_CONFIG}. That seems an odd privilege to require...? Joep From Darren.Moffat@Oracle.COM Mon Mar 29 01:57:37 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2T8vahG024090 for ; Mon, 29 Mar 2010 01:57:36 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o2T8vXIJ014817; Mon, 29 Mar 2010 02:57:35 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L0100J0NCVYSZ00@nwk-avmta-2.sfbay.sun.com>; Mon, 29 Mar 2010 01:57:34 -0700 (PDT) Received: from sca-ea-mail-3.sun.com ([192.18.43.21]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L01007M6CVYF5B0@nwk-avmta-2.sfbay.sun.com>; Mon, 29 Mar 2010 01:57:34 -0700 (PDT) Received: from rcsinet13.oracle.com (rcsinet13.oracle.com [148.87.113.125]) by sca-ea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o2T8vX0L021318; Mon, 29 Mar 2010 08:57:34 +0000 (GMT) Received: from acsmt353.oracle.com (acsmt353.oracle.com [141.146.40.153]) by rcsinet13.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o2T8vW8V014327; Mon, 29 Mar 2010 08:57:33 +0000 (GMT) Received: from abhmt020.oracle.com by acsmt354.oracle.com with ESMTP id 118937021269853007; Mon, 29 Mar 2010 01:56:47 -0700 Received: from [10.7.251.221] (/10.7.251.221) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 29 Mar 2010 01:56:47 -0700 Date: Mon, 29 Mar 2010 09:56:45 +0100 From: Darren J Moffat Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <4BB0685D.7090409@Sun.COM> To: Joep Vesseur Cc: Tim Haley , PSARC-ext@sun.com, ZFS Team Message-id: <4BB06B4D.6040900@Oracle.COM> Organization: Oracle Solaris Security MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt353.oracle.com [141.146.40.153] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090203.4BB06B7D.00FC:SCFMA4539814,ss=1,fgs=0 References: <4BAFAA10.2070706@oracle.com> <4BB0685D.7090409@Sun.COM> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.7) Gecko/20100214 Lightning/1.0b1 Thunderbird/3.0.1 Status: RO Content-Length: 786 On 29/03/2010 09:44, Joep Vesseur wrote: > On 03/28/10 21:12, Tim Haley wrote: > >> Five types of change are described: >> >> o File/Directory modified >> o File/Directory present in older snapshot but not newer >> o File/Directory present in newer snapshot but not older >> o File/Directory renamed >> o File link count changed > > Is there any provision made to detect/display file property changes? Or > will > those be covered by a change to the parent-directory? > > > > [...] For diffs > > between existing snapshots, the necessary privilege is > > {PRIV_SYS_CONFIG}. > > That seems an odd privilege to require...? While odd it is consistent with the rest of ZFS. You need SYS_CONFIG to do "disk" or pool level stuff if you don't have a ZFS delegation. -- Darren J Moffat From tim.haley@oracle.com Mon Mar 29 07:58:28 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2TEwSmc029786 for ; Mon, 29 Mar 2010 07:58:28 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o2TEwMSv018340; Mon, 29 Mar 2010 08:58:26 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L0100413TLDKT00@nwk-avmta-1.sfbay.Sun.COM>; Mon, 29 Mar 2010 07:58:25 -0700 (PDT) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L0100C5NTLDKU40@nwk-avmta-1.sfbay.Sun.COM>; Mon, 29 Mar 2010 07:58:25 -0700 (PDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o2TEwOtT009583; Mon, 29 Mar 2010 14:58:25 +0000 (GMT) Received: from acsmt353.oracle.com (acsmt353.oracle.com [141.146.40.153]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o2TDDDpa017021; Mon, 29 Mar 2010 14:58:24 +0000 (GMT) Received: from abhmt005.oracle.com by acsmt354.oracle.com with ESMTP id 119960431269874629; Mon, 29 Mar 2010 07:57:09 -0700 Received: from [172.20.25.27] (/10.85.25.27) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 29 Mar 2010 07:57:09 -0700 Date: Mon, 29 Mar 2010 08:57:08 -0600 From: Tim Haley Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <4BB0685D.7090409@Sun.COM> To: Joep Vesseur Cc: PSARC-ext@sun.com, ZFS Team Message-id: <4BB0BFC4.5090900@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt353.oracle.com [141.146.40.153] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090206.4BB0C010.00E0:SCFMA4539814,ss=1,fgs=0 References: <4BAFAA10.2070706@oracle.com> <4BB0685D.7090409@Sun.COM> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.8) Gecko/20100302 Thunderbird/3.0.2 Status: RO Content-Length: 818 On 03/29/10 02:44 AM, Joep Vesseur wrote: > On 03/28/10 21:12, Tim Haley wrote: > >> Five types of change are described: >> >> o File/Directory modified >> o File/Directory present in older snapshot but not newer >> o File/Directory present in newer snapshot but not older >> o File/Directory renamed >> o File link count changed > > Is there any provision made to detect/display file property changes? > Or will > those be covered by a change to the parent-directory? > > Joep Not sure exactly what sort of properties you have in mind - many I can think of would result in us detecting a modification to the file itself, for example: # echo 'tim was here' > file # zfs snapshot toad/timh@before # touch file # zfs snapshot toad/timh@after # zfs diff toad/timh@before toad/timh@after M /toad/timh/file -tim From Joep.Vesseur@sun.com Mon Mar 29 08:02:16 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2TF2F8w029917 for ; Mon, 29 Mar 2010 08:02:15 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o2TF2EoE027760; Mon, 29 Mar 2010 10:02:15 -0500 (CDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L0100D2NTRQ6B00@brm-avmta-1.central.sun.com>; Mon, 29 Mar 2010 09:02:14 -0600 (MDT) Received: from gmp-eb-inf-2.sun.com ([192.18.6.24]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L0100BYBTRO7K10@brm-avmta-1.central.sun.com>; Mon, 29 Mar 2010 09:02:13 -0600 (MDT) Received: from fe-emea-13.sun.com (gmp-eb-lb-1-fe1.eu.sun.com [192.18.6.7] (may be forged)) by gmp-eb-inf-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o2TF2BvZ028362; Mon, 29 Mar 2010 15:02:11 +0000 (GMT) Received: from conversion-daemon.fe-emea-13.sun.com by fe-emea-13.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0L0100100TJIC800@fe-emea-13.sun.com>; Mon, 29 Mar 2010 16:01:55 +0100 (BST) Received: from enigma.Holland.Sun.COM ([unknown] [10.16.117.32]) by fe-emea-13.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0L01001QITR6DJ00@fe-emea-13.sun.com>; Mon, 29 Mar 2010 16:01:54 +0100 (BST) Date: Mon, 29 Mar 2010 17:01:52 +0200 From: Joep Vesseur Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <4BB0BFC4.5090900@oracle.com> Sender: Joep.Vesseur@sun.com To: Tim Haley Cc: PSARC-ext@sun.com, ZFS Team Message-id: <4BB0C0E0.3070205@Sun.COM> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <4BAFAA10.2070706@oracle.com> <4BB0685D.7090409@Sun.COM> <4BB0BFC4.5090900@oracle.com> User-Agent: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.9.1.5) Gecko/20091201 Lightning/1.0b1 Thunderbird/3.0 Status: RO Content-Length: 614 On 03/29/10 16:57, Tim Haley wrote: > Not sure exactly what sort of properties you have in mind - many I can > think of would > result in us detecting a modification to the file itself, for example: > > # echo 'tim was here' > file > # zfs snapshot toad/timh@before > # touch file > # zfs snapshot toad/timh@after > # zfs diff toad/timh@before toad/timh@after > M /toad/timh/file I meant something like chmod u+s /usr/bin/sh Would this be reported as a change to /usr/bin? And would someone have to go and look at the diff of the directory listing? Or would it be reported as a change to /usr/bin/sh? Joep From tim.haley@oracle.com Mon Mar 29 08:06:24 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2TF6OC9029939 for ; Mon, 29 Mar 2010 08:06:24 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o2TF6MpV025290; Mon, 29 Mar 2010 09:06:22 -0600 (MDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L0100D09TYMM600@brm-avmta-1.central.sun.com>; Mon, 29 Mar 2010 09:06:22 -0600 (MDT) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L0100BN3TYM7M20@brm-avmta-1.central.sun.com>; Mon, 29 Mar 2010 09:06:22 -0600 (MDT) Received: from rcsinet13.oracle.com (rcsinet13.oracle.com [148.87.113.125]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o2TF6LYx016332; Mon, 29 Mar 2010 15:06:21 +0000 (GMT) Received: from acsmt355.oracle.com (acsmt355.oracle.com [141.146.40.155]) by rcsinet13.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o2SJF9on014211; Mon, 29 Mar 2010 15:06:20 +0000 (GMT) Received: from abhmt007.oracle.com by acsmt353.oracle.com with ESMTP id 125888731269875158; Mon, 29 Mar 2010 08:05:58 -0700 Received: from [172.20.25.27] (/10.85.25.27) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 29 Mar 2010 08:05:56 -0700 Date: Mon, 29 Mar 2010 09:05:52 -0600 From: Tim Haley Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <4BB0C0E0.3070205@Sun.COM> To: Joep Vesseur Cc: PSARC-ext@sun.com, ZFS Team Message-id: <4BB0C1D0.2030905@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt355.oracle.com [141.146.40.155] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090205.4BB0C1ED.003F:SCFMA4539814,ss=1,fgs=0 References: <4BAFAA10.2070706@oracle.com> <4BB0685D.7090409@Sun.COM> <4BB0BFC4.5090900@oracle.com> <4BB0C0E0.3070205@Sun.COM> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.8) Gecko/20100302 Thunderbird/3.0.2 Status: RO Content-Length: 742 On 03/29/10 09:01 AM, Joep Vesseur wrote: > On 03/29/10 16:57, Tim Haley wrote: >> Not sure exactly what sort of properties you have in mind - many I can >> think of would >> result in us detecting a modification to the file itself, for example: >> >> # echo 'tim was here' > file >> # zfs snapshot toad/timh@before >> # touch file >> # zfs snapshot toad/timh@after >> # zfs diff toad/timh@before toad/timh@after >> M /toad/timh/file > > I meant something like > > chmod u+s /usr/bin/sh > > Would this be reported as a change to /usr/bin? And would someone have > to go > and look at the diff of the directory listing? > > Or would it be reported as a change to /usr/bin/sh? > > Joep It's reported as a modification to /usr/bin/sh. -tim From Nicolas.Williams@sun.com Mon Mar 29 09:14:36 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2TGEajg001515 for ; Mon, 29 Mar 2010 09:14:36 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o2TGEXLV011827; Mon, 29 Mar 2010 10:14:35 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L0100M01X4BRV00@nwk-avmta-2.sfbay.sun.com>; Mon, 29 Mar 2010 09:14:35 -0700 (PDT) Received: from sca-ea-mail-4.sun.com ([192.18.43.22]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L0100JTWX4B9B50@nwk-avmta-2.sfbay.sun.com>; Mon, 29 Mar 2010 09:14:35 -0700 (PDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by sca-ea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o2TGEYr1025958; Mon, 29 Mar 2010 16:14:34 +0000 (GMT) Received: from acsmt355.oracle.com (acsmt355.oracle.com [141.146.40.155]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o2TFlOdE021070; Mon, 29 Mar 2010 16:14:29 +0000 (GMT) Received: from abhmt003.oracle.com by acsmt353.oracle.com with ESMTP id 126116191269879253; Mon, 29 Mar 2010 09:14:13 -0700 Received: from Sun.COM (/129.153.128.104) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 29 Mar 2010 09:14:12 -0700 Date: Mon, 29 Mar 2010 11:14:07 -0500 From: Nicolas Williams Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <4BAFAA10.2070706@oracle.com> To: Tim Haley Cc: PSARC-ext@sun.com, ZFS Team Message-id: <20100329161407.GN21244@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt355.oracle.com [141.146.40.155] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090209.4BB0D1E6.0142,ss=1,fgs=0 References: <4BAFAA10.2070706@oracle.com> User-Agent: Mutt/1.5.20 (2010-03-02) Status: RO Content-Length: 1150 On Sun, Mar 28, 2010 at 01:12:16PM -0600, Tim Haley wrote: > + If the modification involved a change in the link count of a > + file, the change will be expressed as a delta within > + parentheses on the modification line. Example outputs are > + below: > + > + M /myfiles/ > + M /myfiles/link_to_me (+1) > + R /myfiles/rename_me -> /myfiles/renamed > + - /myfiles/delete_me > + + /myfiles/new_file Is there any escaping of whitespace and non-printable characters in the pathnames? If not then the above format is ambiguous and cannot be safely scripted. There are several ways that you could address that problem, such as escaping (HTML/XML entities? backslash escapes? pick your poison), adding a length field preceding either each path or each path that contains whitespace, or use a multi-line (for renames) format with paths being the last field such that you need only [backslash?-]escape newlines. Probably the simplest answer is backslash-escaping whitespace, since shells can handle that trivially. That is what I recommend. Nico -- From Nicolas.Williams@sun.com Mon Mar 29 09:17:48 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2TGHl39001555 for ; Mon, 29 Mar 2010 09:17:47 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o2TGHg5Z019302; Mon, 29 Mar 2010 11:17:46 -0500 (CDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L010004BX9LZ400@nwk-avmta-1.sfbay.Sun.COM>; Mon, 29 Mar 2010 09:17:45 -0700 (PDT) Received: from sca-ea-mail-2.sun.com ([192.18.43.25]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L0100CC7X9FKK90@nwk-avmta-1.sfbay.Sun.COM>; Mon, 29 Mar 2010 09:17:39 -0700 (PDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by sca-ea-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o2TGHdM0001792; Mon, 29 Mar 2010 16:17:39 +0000 (GMT) Received: from acsmt353.oracle.com (acsmt353.oracle.com [141.146.40.153]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o2T1ATjL012143; Mon, 29 Mar 2010 16:17:38 +0000 (GMT) Received: from abhmt010.oracle.com by acsmt354.oracle.com with ESMTP id 126129361269879456; Mon, 29 Mar 2010 09:17:36 -0700 Received: from Sun.COM (/129.153.128.104) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 29 Mar 2010 09:17:36 -0700 Date: Mon, 29 Mar 2010 11:17:31 -0500 From: Nicolas Williams Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <4BB0C1D0.2030905@oracle.com> To: Tim Haley Cc: Joep Vesseur , PSARC-ext@sun.com, ZFS Team Message-id: <20100329161731.GO21244@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt353.oracle.com [141.146.40.153] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090207.4BB0D2A3.004B,ss=1,fgs=0 References: <4BAFAA10.2070706@oracle.com> <4BB0685D.7090409@Sun.COM> <4BB0BFC4.5090900@oracle.com> <4BB0C0E0.3070205@Sun.COM> <4BB0C1D0.2030905@oracle.com> User-Agent: Mutt/1.5.20 (2010-03-02) Status: RO Content-Length: 888 On Mon, Mar 29, 2010 at 09:05:52AM -0600, Tim Haley wrote: > On 03/29/10 09:01 AM, Joep Vesseur wrote: > >On 03/29/10 16:57, Tim Haley wrote: > >>Not sure exactly what sort of properties you have in mind - many I can > >>think of would > >>result in us detecting a modification to the file itself, for example: > > > >I meant something like > > > > chmod u+s /usr/bin/sh > > > >Would this be reported as a change to /usr/bin? And would someone > >have to go and look at the diff of the directory listing? > > > >Or would it be reported as a change to /usr/bin/sh? > > It's reported as a modification to /usr/bin/sh. I'm guessing that ZFS would need a second generation number per-dnode to track data and meta-data changes separately (or a data checksum that excludes parts of blkptr_t -- the parts that refer to disk locations). Still, this feature is absolutely fantastic. Nico -- From sebastien.roy@oracle.com Mon Mar 29 09:25:18 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2TGPIZE001590 for ; Mon, 29 Mar 2010 09:25:18 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o2TGPGeT021268; Mon, 29 Mar 2010 10:25:17 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L0100307XM53600@nwk-avmta-1.sfbay.Sun.COM>; Mon, 29 Mar 2010 09:25:17 -0700 (PDT) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L0100CT5XM3KUA0@nwk-avmta-1.sfbay.Sun.COM>; Mon, 29 Mar 2010 09:25:16 -0700 (PDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o2TGPFfi010395; Mon, 29 Mar 2010 16:25:15 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o2SNumFY028211; Mon, 29 Mar 2010 16:25:14 +0000 (GMT) Received: from abhmt010.oracle.com by acsmt354.oracle.com with ESMTP id 120285001269879838; Mon, 29 Mar 2010 09:23:58 -0700 Received: from [129.148.19.4] (/129.148.19.4) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 29 Mar 2010 09:23:58 -0700 Date: Mon, 29 Mar 2010 12:23:56 -0400 From: Sebastien Roy Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <20100329161407.GN21244@Sun.COM> To: Nicolas Williams Cc: Tim Haley , PSARC-ext@sun.com, ZFS Team Message-id: <4BB0D41C.2000203@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090209.4BB0D46A.0146:SCFMA4539814,ss=1,fgs=0 References: <4BAFAA10.2070706@oracle.com> <20100329161407.GN21244@Sun.COM> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.8) Gecko/20100302 Lightning/1.0b1 Thunderbird/3.0.2 Status: RO Content-Length: 1727 On 03/29/10 12:14 PM, Nicolas Williams wrote: > On Sun, Mar 28, 2010 at 01:12:16PM -0600, Tim Haley wrote: >> + If the modification involved a change in the link count of a >> + file, the change will be expressed as a delta within >> + parentheses on the modification line. Example outputs are >> + below: >> + >> + M /myfiles/ >> + M /myfiles/link_to_me (+1) >> + R /myfiles/rename_me -> /myfiles/renamed >> + - /myfiles/delete_me >> + + /myfiles/new_file > > Is there any escaping of whitespace and non-printable characters in the > pathnames? If not then the above format is ambiguous and cannot be > safely scripted. > > There are several ways that you could address that problem, such as > escaping (HTML/XML entities? backslash escapes? pick your poison), > adding a length field preceding either each path or each path that > contains whitespace, or use a multi-line (for renames) format with paths > being the last field such that you need only [backslash?-]escape > newlines. > > Probably the simplest answer is backslash-escaping whitespace, since > shells can handle that trivially. That is what I recommend. Taking a step back here, this subcommand is not the only zfs subcommand whose output could be subject to parsing by scripts. Adding parsable output should be something that is thought-through for the entire suite of subcommands (and zfs-related commands) so that there is a uniformly applicable solution. IMO, that's not this case (although that's ultimately the project team's decision). If you think there is something that is ambiguous to the human eye, then I think that's in scope. -Seb From Nicolas.Williams@sun.com Mon Mar 29 09:34:34 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2TGYYN3001626 for ; Mon, 29 Mar 2010 09:34:34 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o2TGYQjn001894; Mon, 29 Mar 2010 11:34:33 -0500 (CDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L010011TY1K4P00@nwk-avmta-2.sfbay.sun.com>; Mon, 29 Mar 2010 09:34:32 -0700 (PDT) Received: from sca-ea-mail-1.sun.com ([192.18.43.24]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L0100JIPY1I9B70@nwk-avmta-2.sfbay.sun.com>; Mon, 29 Mar 2010 09:34:30 -0700 (PDT) Received: from rcsinet13.oracle.com (rcsinet13.oracle.com [148.87.113.125]) by sca-ea-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o2TGYU9a016287; Mon, 29 Mar 2010 16:34:30 +0000 (GMT) Received: from acsmt353.oracle.com (acsmt353.oracle.com [141.146.40.153]) by rcsinet13.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o2TGYSIu006251; Mon, 29 Mar 2010 16:34:29 +0000 (GMT) Received: from abhmt018.oracle.com by acsmt354.oracle.com with ESMTP id 126184251269880392; Mon, 29 Mar 2010 09:33:12 -0700 Received: from Sun.COM (/129.153.128.104) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 29 Mar 2010 09:33:11 -0700 Date: Mon, 29 Mar 2010 11:33:06 -0500 From: Nicolas Williams Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <4BB0D41C.2000203@oracle.com> To: Sebastien Roy Cc: Tim Haley , PSARC-ext@sun.com, ZFS Team Message-id: <20100329163306.GP21244@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt353.oracle.com [141.146.40.153] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090206.4BB0D695.0157,ss=1,fgs=0 References: <4BAFAA10.2070706@oracle.com> <20100329161407.GN21244@Sun.COM> <4BB0D41C.2000203@oracle.com> User-Agent: Mutt/1.5.20 (2010-03-02) Status: RO Content-Length: 1285 On Mon, Mar 29, 2010 at 12:23:56PM -0400, Sebastien Roy wrote: > >Is there any escaping of whitespace and non-printable characters in the > >pathnames? If not then the above format is ambiguous and cannot be > >safely scripted. > > Taking a step back here, this subcommand is not the only zfs > subcommand whose output could be subject to parsing by scripts. Many zfs sub-commands already have a -H option ("Display output in a form more easily parsed by scripts"). > Adding parsable output should be something that is thought-through > for the entire suite of subcommands (and zfs-related commands) so > that there is a uniformly applicable solution. IMO, that's not this > case (although that's ultimately the project team's decision). Therefore I don't think your argument carries water. My request is not generalizable because ZFS already has parseable output support. > If > you think there is something that is ambiguous to the human eye, > then I think that's in scope. I could have filenames with ' -> ' in them that would render the rename output ambiguous to the human eye. I could have filenames with '\n' in the name that would render the output ambiguous to the human eye. Nico -- From Steve.McKinty@sun.com Mon Mar 29 09:34:58 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2TGYvXG001638 for ; Mon, 29 Mar 2010 09:34:58 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o2TGYuZx028592; Mon, 29 Mar 2010 10:34:57 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L010051BY28FG00@nwk-avmta-1.sfbay.Sun.COM>; Mon, 29 Mar 2010 09:34:56 -0700 (PDT) Received: from gmp-eb-inf-1.sun.com ([192.18.6.21]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L0100CMFY27KKA0@nwk-avmta-1.sfbay.Sun.COM>; Mon, 29 Mar 2010 09:34:56 -0700 (PDT) Received: from fe-emea-09.sun.com (gmp-eb-lb-1-fe1.eu.sun.com [192.18.6.7] (may be forged)) by gmp-eb-inf-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o2TGYtXX013565; Mon, 29 Mar 2010 16:34:55 +0000 (GMT) Received: from conversion-daemon.fe-emea-09.sun.com by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0L0100H00XQEMJ00@fe-emea-09.sun.com>; Mon, 29 Mar 2010 17:34:34 +0100 (BST) Received: from [10.0.0.5] ([unknown] [80.15.167.5]) by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0L010056HY1ESY60@fe-emea-09.sun.com>; Mon, 29 Mar 2010 17:34:27 +0100 (BST) Date: Mon, 29 Mar 2010 18:34:23 +0200 From: Steve Mckinty Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <4BB0D41C.2000203@oracle.com> Sender: Steve.McKinty@sun.com To: Sebastien Roy Cc: Nicolas Williams , Tim Haley , PSARC-ext@sun.com, ZFS Team Message-id: <4BB0D68F.2080407@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <4BAFAA10.2070706@oracle.com> <20100329161407.GN21244@Sun.COM> <4BB0D41C.2000203@oracle.com> User-Agent: Thunderbird 2.0.0.21 (X11/20090323) Status: RO Content-Length: 1977 Sebastien Roy wrote: > On 03/29/10 12:14 PM, Nicolas Williams wrote: >> On Sun, Mar 28, 2010 at 01:12:16PM -0600, Tim Haley wrote: >>> + If the modification involved a change in the link count of a >>> + file, the change will be expressed as a delta within >>> + parentheses on the modification line. Example outputs are >>> + below: >>> + >>> + M /myfiles/ >>> + M /myfiles/link_to_me (+1) >>> + R /myfiles/rename_me -> /myfiles/renamed >>> + - /myfiles/delete_me >>> + + /myfiles/new_file >> >> Is there any escaping of whitespace and non-printable characters in the >> pathnames? If not then the above format is ambiguous and cannot be >> safely scripted. >> >> There are several ways that you could address that problem, such as >> escaping (HTML/XML entities? backslash escapes? pick your poison), >> adding a length field preceding either each path or each path that >> contains whitespace, or use a multi-line (for renames) format with paths >> being the last field such that you need only [backslash?-]escape >> newlines. >> >> Probably the simplest answer is backslash-escaping whitespace, since >> shells can handle that trivially. That is what I recommend. > > Taking a step back here, this subcommand is not the only zfs subcommand > whose output could be subject to parsing by scripts. Adding parsable > output should be something that is thought-through for the entire suite > of subcommands (and zfs-related commands) so that there is a uniformly > applicable solution. IMO, that's not this case (although that's > ultimately the project team's decision). If you think there is > something that is ambiguous to the human eye, then I think that's in scope. It's not uncommon to have an additional option to a command (stty -g for example) which produces easily-parsable (and not necessarily easily readable :) ) output for this situation. Steve From sebastien.roy@oracle.com Mon Mar 29 10:13:24 2010 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2THDNEN002197 for ; Mon, 29 Mar 2010 10:13:23 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o2THDJwb009717; Mon, 29 Mar 2010 10:13:21 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L010030XZU9B700@nwk-avmta-2.sfbay.sun.com>; Mon, 29 Mar 2010 10:13:21 -0700 (PDT) Received: from sca-ea-mail-4.sun.com ([192.18.43.22]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L0100JZ2ZU89HC0@nwk-avmta-2.sfbay.sun.com>; Mon, 29 Mar 2010 10:13:20 -0700 (PDT) Received: from rcsinet13.oracle.com (rcsinet13.oracle.com [148.87.113.125]) by sca-ea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o2THDJaM024741; Mon, 29 Mar 2010 17:13:20 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by rcsinet13.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o2TH2Tvp029757; Mon, 29 Mar 2010 17:13:18 +0000 (GMT) Received: from abhmt004.oracle.com by acsmt354.oracle.com with ESMTP id 120462831269882728; Mon, 29 Mar 2010 10:12:08 -0700 Received: from [129.148.19.4] (/129.148.19.4) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 29 Mar 2010 10:12:08 -0700 Date: Mon, 29 Mar 2010 13:12:06 -0400 From: Sebastien Roy Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <4BB0D68F.2080407@sun.com> To: Steve Mckinty Cc: Nicolas Williams , Tim Haley , PSARC-ext@sun.com, ZFS Team Message-id: <4BB0DF66.5030906@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090203.4BB0DFAF.0101:SCFMA4539814,ss=1,fgs=0 References: <4BAFAA10.2070706@oracle.com> <20100329161407.GN21244@Sun.COM> <4BB0D41C.2000203@oracle.com> <4BB0D68F.2080407@sun.com> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.8) Gecko/20100302 Lightning/1.0b1 Thunderbird/3.0.2 Status: RO Content-Length: 903 On 03/29/10 12:34 PM, Steve Mckinty wrote: > Sebastien Roy wrote: >> Taking a step back here, this subcommand is not the only zfs >> subcommand whose output could be subject to parsing by scripts. Adding >> parsable output should be something that is thought-through for the >> entire suite of subcommands (and zfs-related commands) so that there >> is a uniformly applicable solution. IMO, that's not this case >> (although that's ultimately the project team's decision). If you think >> there is something that is ambiguous to the human eye, then I think >> that's in scope. > > It's not uncommon to have an additional option to a command (stty -g for > example) which produces easily-parsable (and not necessarily easily > readable :) ) output for this situation. That's right, but to reiterate, I think that such an option should theoretically apply to more then just the "diff" subcommand. -Seb From sebastien.roy@oracle.com Mon Mar 29 10:17:45 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2THHjYa002283 for ; Mon, 29 Mar 2010 10:17:45 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o2THHTCK000803; Mon, 29 Mar 2010 11:17:44 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L0200F8L01JVQ00@nwk-avmta-1.sfbay.Sun.COM>; Mon, 29 Mar 2010 10:17:43 -0700 (PDT) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L0200CRL01IKZD0@nwk-avmta-1.sfbay.Sun.COM>; Mon, 29 Mar 2010 10:17:42 -0700 (PDT) Received: from rcsinet15.oracle.com (rcsinet15.oracle.com [148.87.113.117]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o2THHfaR028009; Mon, 29 Mar 2010 17:17:41 +0000 (GMT) Received: from acsmt355.oracle.com (acsmt355.oracle.com [141.146.40.155]) by rcsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o2TDBuhW004469; Mon, 29 Mar 2010 17:17:38 +0000 (GMT) Received: from abhmt008.oracle.com by acsmt353.oracle.com with ESMTP id 120482851269883054; Mon, 29 Mar 2010 10:17:34 -0700 Received: from [129.148.19.4] (/129.148.19.4) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 29 Mar 2010 10:17:33 -0700 Date: Mon, 29 Mar 2010 13:17:30 -0400 From: Sebastien Roy Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <20100329163306.GP21244@Sun.COM> To: Nicolas Williams Cc: Tim Haley , PSARC-ext@sun.com, ZFS Team Message-id: <4BB0E0AA.2050703@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt355.oracle.com [141.146.40.155] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090205.4BB0E0B4.00D5:SCFMA4539814,ss=1,fgs=0 References: <4BAFAA10.2070706@oracle.com> <20100329161407.GN21244@Sun.COM> <4BB0D41C.2000203@oracle.com> <20100329163306.GP21244@Sun.COM> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.8) Gecko/20100302 Lightning/1.0b1 Thunderbird/3.0.2 Status: RO Content-Length: 1599 On 03/29/10 12:33 PM, Nicolas Williams wrote: > On Mon, Mar 29, 2010 at 12:23:56PM -0400, Sebastien Roy wrote: >>> Is there any escaping of whitespace and non-printable characters in the >>> pathnames? If not then the above format is ambiguous and cannot be >>> safely scripted. >> >> Taking a step back here, this subcommand is not the only zfs >> subcommand whose output could be subject to parsing by scripts. > > Many zfs sub-commands already have a -H option ("Display output in a > form more easily parsed by scripts"). Ah, indeed. > >> Adding parsable output should be something that is thought-through >> for the entire suite of subcommands (and zfs-related commands) so >> that there is a uniformly applicable solution. IMO, that's not this >> case (although that's ultimately the project team's decision). > > Therefore I don't think your argument carries water. My request is not > generalizable because ZFS already has parseable output support. Understood, and I see that now. It would indeed make sense to be consistent and have -H for this subcommand as well. >> If >> you think there is something that is ambiguous to the human eye, >> then I think that's in scope. > > I could have filenames with ' -> ' in them that would render the rename > output ambiguous to the human eye. I could have filenames with > '\n' in the name that would render the output ambiguous > to the human eye. My intention wasn't to rathole into some absurd discussion over how to handle ridiculous filenames. -Seb From Nicolas.Williams@sun.com Mon Mar 29 10:25:24 2010 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2THPOoJ002487 for ; Mon, 29 Mar 2010 10:25:24 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o2THPOst016526; Mon, 29 Mar 2010 10:25:24 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L0200H170EBZC00@nwk-avmta-1.sfbay.Sun.COM>; Mon, 29 Mar 2010 10:25:23 -0700 (PDT) Received: from sca-ea-mail-1.sun.com ([192.18.43.24]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L0200CSG0EAKZE0@nwk-avmta-1.sfbay.Sun.COM>; Mon, 29 Mar 2010 10:25:22 -0700 (PDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by sca-ea-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o2THPLJv011327; Mon, 29 Mar 2010 17:25:21 +0000 (GMT) Received: from acsmt353.oracle.com (acsmt353.oracle.com [141.146.40.153]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o2TCTMJn015184; Mon, 29 Mar 2010 17:25:19 +0000 (GMT) Received: from abhmt012.oracle.com by acsmt354.oracle.com with ESMTP id 126350071269883491; Mon, 29 Mar 2010 10:24:51 -0700 Received: from Sun.COM (/129.153.128.104) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 29 Mar 2010 10:24:50 -0700 Date: Mon, 29 Mar 2010 12:24:45 -0500 From: Nicolas Williams Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <4BB0E0AA.2050703@oracle.com> To: Sebastien Roy Cc: Tim Haley , PSARC-ext@sun.com, ZFS Team Message-id: <20100329172445.GS21244@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt353.oracle.com [141.146.40.153] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090207.4BB0E280.0121,ss=1,fgs=0 References: <4BAFAA10.2070706@oracle.com> <20100329161407.GN21244@Sun.COM> <4BB0D41C.2000203@oracle.com> <20100329163306.GP21244@Sun.COM> <4BB0E0AA.2050703@oracle.com> User-Agent: Mutt/1.5.20 (2010-03-02) Status: RO Content-Length: 1206 On Mon, Mar 29, 2010 at 01:17:30PM -0400, Sebastien Roy wrote: > Understood, and I see that now. It would indeed make sense to be > consistent and have -H for this subcommand as well. Agreed. Do you agree re: backslash-escaping? > >I could have filenames with ' -> ' in them that would render the rename > >output ambiguous to the human eye. I could have filenames with > >'\n' in the name that would render the output ambiguous > >to the human eye. > > My intention wasn't to rathole into some absurd discussion over how > to handle ridiculous filenames. My intention is to avoid security problems in consumers of zfs diff. I assert that zfs diff is mostly useful only in connection with scripting. I don't think it's reasonable to expect that zfs diff be useful only "by eye". It has got to be scriptable because for any sufficiently large and active dataset the output of zfs diff will generally be too large to handle "by eye" (sure, you could grep for specific things, but not much beyond that you're scripting). (If zfs diff is only useful for scripting then the -H option is actually unnecessary and zfs diff should always disambiguate pathnames in its output.) Nico -- From dcragun@sonic.net Mon Mar 29 10:31:07 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2THV7bQ002549 for ; Mon, 29 Mar 2010 10:31:07 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o2THUlej014438 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Mon, 29 Mar 2010 12:31:06 -0500 (CDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L020040L0NHBV00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 29 Mar 2010 10:30:53 -0700 (PDT) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L0200JLW0NH9DB0@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 29 Mar 2010 10:30:53 -0700 (PDT) Received: from relay42i.sun.com ([192.5.209.72]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o2THUqFx008213 for ; Mon, 29 Mar 2010 17:30:52 +0000 (GMT) Received: from mms48es.mms.us.syntegra.com ([160.41.221.230] [160.41.221.230]) by relay42i.sun.com with ESMTP id BT-MMP-3239494; Mon, 29 Mar 2010 17:30:52 +0000 (Z) Received: from relay45i.sun.com (relay45i.sun.com [192.5.209.94]) by mms48es.mms.us.syntegra.com with ESMTP id BT-MMP-62894002; Mon, 29 Mar 2010 17:30:52 +0000 (Z) Received: from a.mail.sonic.net ([64.142.16.245] [64.142.16.245]) by relay4i.sun.com with ESMTP id BT-MMP-15526621; Mon, 29 Mar 2010 17:30:51 +0000 (Z) Received: from [10.0.0.4] (76-191-129-144.dsl.dynamic.sonic.net [76.191.129.144]) (authenticated bits=0) by a.mail.sonic.net (8.13.8.Beta0-Sonic/8.13.7) with ESMTP id o2THUjcn022493 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Mon, 29 Mar 2010 10:30:45 -0700 Date: Mon, 29 Mar 2010 10:30:44 -0700 From: Don Cragun Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: To: tim.haley@oracle.com Cc: zfs-team@sun.com, PSARC-ext@sun.com Message-id: <409C3415-CAEE-423A-B166-DD3326469031@sonic.net> MIME-version: 1.0 X-Mailer: Apple Mail (2.1077) Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-Antispam: No, score=-2.6/5.0, scanned in 0.105sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: Status: RO Content-Length: 4059 On Mar 28, 2010, at 13:12:16 -0600, Tim Haley wrote: Please find documentation nits in-line below... - Don ... ... ... > > 4. Technical Description > > There is a long-standing RFE for zfs to be able to describe > what has changed between the snapshots of a dataset. > To provide this capability, we propose a new 'zfs diff' > sub-command. When run with appropriate privilege the > sub-command describes what file system level changes have > occurred between the requested snapshots. A diff between the > current version of the file system and one of its snapshots is > also supported. > > Five types of change are described: > > o File/Directory modified > o File/Directory present in older snapshot but not newer > o File/Directory present in newer snapshot but not older > o File/Directory renamed A directory is a type of file. Saying "File/Directory" is a long way of saying "File". The other possibility is that you meant "Regular File/Directory", but that would only be appropriate if you mean that zfs diff does not report changes to symbolic links, character special file, block special files, doors, FIFOs, sockets, and any other types of files zfs supports. > o File link count changed > > Diffs can be performed if the user is delegated the "diff" > permission. The "diff" permission is being introduced by this > case. Diffs can also be performed without the "diff" > permission, if the user has appropriate privilege. For diffs > between existing snapshots, the necessary privilege is > {PRIV_SYS_CONFIG}. For diff between the current file system > and a snapshot {PRIV_SYS_MOUNT} is also necessary. > > Also introduced by this case is a system attribute on zfs files > called 'generation'. This attribute is part of the > XATTR_VIEW_READONLY described in PSARC 2007/315. It is > generated automatically by the ZFS module. > > Man page changes: ... ... ... > --- zfs.1m.rogi Sun Mar 21 17:01:04 2010 > +++ zfs.1m Sun Mar 28 12:20:04 2010 > @@ -165,6 +165,9 @@ > zfs release [-r] tag snapshot... > > > + zfs diff snapshot snapshot|filesystem > + > + > DESCRIPTION > The zfs command configures ZFS datasets within a ZFS storage > pool, as described in zpool(1M). A dataset is identified by > @@ -1638,7 +1641,41 @@ > size, the resulting behavior is undefined. > > > + zfs diff snapshot snapshot | filesystem > > + Gives a high level description of the differences between a > + snapshot and a descendant dataset. The descendant may either > + be a later snapshot of the dataset or the current dataset. > + For each file system object that has undergone a change > + between the original snapshot and the descendant, the type of > + change is described along with the name of the file or > + directory. In the case of a rename, both the old and new > + names are shown. > + > + The type of change is described with a single character: > + > + + Indicates the file/directory was added in the later dataset > + - Indicates the file/directory was removed in the later dataset > + M Indicates the file/directory was modified in the later dataset > + R Indicates the file/directory was renamed in the later dataset Again, "file/directory" should just be "file" in all four lines above. > + > + If the modification involved a change in the link count of a > + file, the change will be expressed as a delta within > + parentheses on the modification line. Example outputs are > + below: > + > + M /myfiles/ > + M /myfiles/link_to_me (+1) > + R /myfiles/rename_me -> /myfiles/renamed > + - /myfiles/delete_me > + + /myfiles/new_file ... ... ... From Darren.Moffat@Oracle.COM Mon Mar 29 10:42:49 2010 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2THgnIu002645 for ; Mon, 29 Mar 2010 10:42:49 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o2THgm6R013643; Mon, 29 Mar 2010 10:42:49 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L020060J17C4Z00@brm-avmta-1.central.sun.com>; Mon, 29 Mar 2010 11:42:48 -0600 (MDT) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L020058417CSQ00@brm-avmta-1.central.sun.com>; Mon, 29 Mar 2010 11:42:48 -0600 (MDT) Received: from rcsinet13.oracle.com (rcsinet13.oracle.com [148.87.113.125]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o2THglC7009871; Mon, 29 Mar 2010 17:42:47 +0000 (GMT) Received: from acsmt355.oracle.com (acsmt355.oracle.com [141.146.40.155]) by rcsinet13.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o2THgbUr004215; Mon, 29 Mar 2010 17:42:37 +0000 (GMT) Received: from abhmt008.oracle.com by acsmt353.oracle.com with ESMTP id 126403101269884541; Mon, 29 Mar 2010 10:42:21 -0700 Received: from [10.7.251.221] (/10.7.251.221) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 29 Mar 2010 10:42:20 -0700 Date: Mon, 29 Mar 2010 18:42:17 +0100 From: Darren J Moffat Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <409C3415-CAEE-423A-B166-DD3326469031@sonic.net> To: Don Cragun Cc: tim.haley@Oracle.COM, zfs-team@sun.com, PSARC-ext@sun.com Message-id: <4BB0E679.4050207@Oracle.COM> Organization: Oracle Solaris Security MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt355.oracle.com [141.146.40.155] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090209.4BB0E68E.01D5:SCFMA4539814,ss=1,fgs=0 References: <409C3415-CAEE-423A-B166-DD3326469031@sonic.net> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.7) Gecko/20100214 Lightning/1.0b1 Thunderbird/3.0.1 Status: RO Content-Length: 793 On 29/03/2010 18:30, Don Cragun wrote: >> + + Indicates the file/directory was added in the later dataset >> + - Indicates the file/directory was removed in the later dataset >> + M Indicates the file/directory was modified in the later dataset >> + R Indicates the file/directory was renamed in the later dataset > > Again, "file/directory" should just be "file" in all four lines above. While a that may be technically true I personally found it very useful that it said file/directory. Particularly since this isn't a POSIX C API man page. I'd rather it made it clear that both files and directories, and all other types of filesystem objects are supported here, and that it do so by explicitly saying file and directory. -- Darren J Moffat From sebastien.roy@oracle.com Mon Mar 29 10:47:40 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2THldGW002685 for ; Mon, 29 Mar 2010 10:47:40 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o2THlZxP029368; Mon, 29 Mar 2010 11:47:38 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L02000111FEEW00@nwk-avmta-1.sfbay.Sun.COM>; Mon, 29 Mar 2010 10:47:38 -0700 (PDT) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L0200MND1FCML10@nwk-avmta-1.sfbay.Sun.COM>; Mon, 29 Mar 2010 10:47:36 -0700 (PDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o2THlZem013020; Mon, 29 Mar 2010 17:47:35 +0000 (GMT) Received: from acsmt355.oracle.com (acsmt355.oracle.com [141.146.40.155]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o2THLnd2015322; Mon, 29 Mar 2010 17:47:33 +0000 (GMT) Received: from abhmt005.oracle.com by acsmt355.oracle.com with ESMTP id 120582931269884742; Mon, 29 Mar 2010 10:45:42 -0700 Received: from [129.148.19.4] (/129.148.19.4) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 29 Mar 2010 10:45:42 -0700 Date: Mon, 29 Mar 2010 13:45:40 -0400 From: Sebastien Roy Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <20100329172445.GS21244@Sun.COM> To: Nicolas Williams Cc: Tim Haley , PSARC-ext@sun.com, ZFS Team Message-id: <4BB0E744.60001@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt355.oracle.com [141.146.40.155] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090201.4BB0E7B6.01AA:SCFMA4539814,ss=1,fgs=0 References: <4BAFAA10.2070706@oracle.com> <20100329161407.GN21244@Sun.COM> <4BB0D41C.2000203@oracle.com> <20100329163306.GP21244@Sun.COM> <4BB0E0AA.2050703@oracle.com> <20100329172445.GS21244@Sun.COM> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.8) Gecko/20100302 Lightning/1.0b1 Thunderbird/3.0.2 Status: RO Content-Length: 1763 On 03/29/10 01:24 PM, Nicolas Williams wrote: > On Mon, Mar 29, 2010 at 01:17:30PM -0400, Sebastien Roy wrote: >> Understood, and I see that now. It would indeed make sense to be >> consistent and have -H for this subcommand as well. > > Agreed. Do you agree re: backslash-escaping? Yes, but presumably the existing output syntax for -H has already taken that into account, no? The existing syntax appears to use a single tab as a separator. I would hope that a tab within a field would be escaped, and if it's not that should be a bug (otherwise it's not really parsable). >>> I could have filenames with ' -> ' in them that would render the rename >>> output ambiguous to the human eye. I could have filenames with >>> '\n' in the name that would render the output ambiguous >>> to the human eye. >> >> My intention wasn't to rathole into some absurd discussion over how >> to handle ridiculous filenames. > > My intention is to avoid security problems in consumers of zfs diff. I > assert that zfs diff is mostly useful only in connection with scripting. > > I don't think it's reasonable to expect that zfs diff be useful only "by > eye". It has got to be scriptable because for any sufficiently large > and active dataset the output of zfs diff will generally be too large to > handle "by eye" (sure, you could grep for specific things, but not much > beyond that you're scripting). > > (If zfs diff is only useful for scripting then the -H option is actually > unnecessary and zfs diff should always disambiguate pathnames in its > output.) I don't think it's only useful for scripting. It makes perfect sense to me to incorporate -H as with other zfs subcommands. Tim, what is your plan regarding this suggestion? -Seb From matthew.ahrens@oracle.com Mon Mar 29 11:03:07 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2TI37jP003148 for ; Mon, 29 Mar 2010 11:03:07 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o2TI35wR005237; Mon, 29 Mar 2010 13:03:06 -0500 (CDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L020061Z2567C00@nwk-avmta-2.sfbay.sun.com>; Mon, 29 Mar 2010 11:03:06 -0700 (PDT) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L0200JPA2559JE0@nwk-avmta-2.sfbay.sun.com>; Mon, 29 Mar 2010 11:03:05 -0700 (PDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o2TI3454029223; Mon, 29 Mar 2010 18:03:04 +0000 (GMT) Received: from acsmt355.oracle.com (acsmt355.oracle.com [141.146.40.155]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o2TI2of8029669; Mon, 29 Mar 2010 18:02:52 +0000 (GMT) Received: from abhmt020.oracle.com by acsmt355.oracle.com with ESMTP id 120634461269885706; Mon, 29 Mar 2010 11:01:46 -0700 Received: from stationary.local (/10.7.251.178) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 29 Mar 2010 11:01:45 -0700 Date: Mon, 29 Mar 2010 11:01:43 -0700 From: Matthew Ahrens Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <20100329163306.GP21244@Sun.COM> To: Nicolas Williams Cc: Sebastien Roy , Tim Haley , PSARC-ext@sun.com, ZFS Team Message-id: <4BB0EB07.9080403@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt355.oracle.com [141.146.40.155] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090201.4BB0EB57.01C3:SCFMA4539814,ss=1,fgs=0 References: <4BAFAA10.2070706@oracle.com> <20100329161407.GN21244@Sun.COM> <4BB0D41C.2000203@oracle.com> <20100329163306.GP21244@Sun.COM> User-Agent: Thunderbird 2.0.0.24 (Macintosh/20100228) Status: RO Content-Length: 831 Nicolas Williams wrote: > On Mon, Mar 29, 2010 at 12:23:56PM -0400, Sebastien Roy wrote: > >>> Is there any escaping of whitespace and non-printable characters in the >>> pathnames? If not then the above format is ambiguous and cannot be >>> safely scripted. >>> >> Taking a step back here, this subcommand is not the only zfs >> subcommand whose output could be subject to parsing by scripts. >> > > Many zfs sub-commands already have a -H option ("Display output in a > form more easily parsed by scripts"). > However, those subcommands don't deal with filenames. They deal with things like filesystem names and property names/values, which are all much more constrained than filenames. How do commands like ls and find handle printing of filenames with arbitrary characters (newlines and such)? --matt From bart.smaalders@oracle.com Mon Mar 29 11:18:04 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2TII4Og003330 for ; Mon, 29 Mar 2010 11:18:04 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o2TII30Z014543; Mon, 29 Mar 2010 13:18:03 -0500 (CDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L020090N2U2QS00@brm-avmta-1.central.sun.com>; Mon, 29 Mar 2010 12:18:02 -0600 (MDT) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L020051P2U2SN40@brm-avmta-1.central.sun.com>; Mon, 29 Mar 2010 12:18:02 -0600 (MDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o2TII1wj008140; Mon, 29 Mar 2010 18:18:01 +0000 (GMT) Received: from acsmt353.oracle.com (acsmt353.oracle.com [141.146.40.153]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o2TD6vsZ028718; Mon, 29 Mar 2010 18:17:59 +0000 (GMT) Received: from abhmt009.oracle.com by acsmt353.oracle.com with ESMTP id 120689161269886599; Mon, 29 Mar 2010 11:16:39 -0700 Received: from [129.146.228.109] (/129.146.228.109) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 29 Mar 2010 11:16:38 -0700 Date: Mon, 29 Mar 2010 11:14:24 -0700 From: Bart Smaalders Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <4BB0EB07.9080403@oracle.com> To: Matthew Ahrens Cc: Nicolas Williams , Sebastien Roy , Tim Haley , PSARC-ext@sun.com, ZFS Team Message-id: <4BB0EE00.1080705@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt353.oracle.com [141.146.40.153] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090207.4BB0EED8.0156:SCFMA4539814,ss=1,fgs=0 References: <4BAFAA10.2070706@oracle.com> <20100329161407.GN21244@Sun.COM> <4BB0D41C.2000203@oracle.com> <20100329163306.GP21244@Sun.COM> <4BB0EB07.9080403@oracle.com> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.8) Gecko/20100302 Lightning/1.0b1 Thunderbird/3.0.2 Status: RO Content-Length: 219 On 03/29/10 11:01, Matthew Ahrens wrote: > How do commands like ls and find handle printing of filenames with > arbitrary characters (newlines and such)? In general, badly. % touch `echo '\07'` % ls % - Bart From dcragun@sonic.net Mon Mar 29 11:23:31 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2TINV4M003428 for ; Mon, 29 Mar 2010 11:23:31 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o2TINTb2056480 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Mon, 29 Mar 2010 12:23:30 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L020070J335FI00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 29 Mar 2010 11:23:30 -0700 (PDT) Received: from sca-ea-mail-1.sun.com ([192.18.43.24]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L02006IM334XL00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 29 Mar 2010 11:23:28 -0700 (PDT) Received: from relay14i.sun.com (ip124.net129179-4.block1.us.syntegra.com [129.179.4.124]) by sca-ea-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o2TINR5H004611 for ; Mon, 29 Mar 2010 18:23:27 +0000 (GMT) Received: from mmp11es.mmp.us.syntegra.com ([160.41.208.11] [160.41.208.11]) by relay14i.sun.com with ESMTP id BT-MMP-842212; Mon, 29 Mar 2010 18:21:27 +0000 (Z) Received: from relay15i.sun.com (relay15i.sun.com [129.179.4.125]) by mmp11es.mmp.us.syntegra.com with ESMTP id BT-MMP-98264794; Mon, 29 Mar 2010 18:21:27 +0000 (Z) Received: from b.mail.sonic.net ([64.142.19.5] [64.142.19.5]) by relay1i.sun.com with ESMTP id BT-MMP-20376265; Mon, 29 Mar 2010 18:21:26 +0000 (Z) Received: from [10.0.0.4] (76-191-129-144.dsl.dynamic.sonic.net [76.191.129.144]) (authenticated bits=0) by b.mail.sonic.net (8.13.8.Beta0-Sonic/8.13.7) with ESMTP id o2TILNZF020833 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Mon, 29 Mar 2010 11:21:23 -0700 Date: Mon, 29 Mar 2010 11:21:22 -0700 From: Don Cragun Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <4BB0E679.4050207@Oracle.COM> To: Darren J Moffat Cc: tim.haley@Oracle.COM, zfs-team@sun.com, PSARC-ext@sun.com Message-id: <6C772477-CDBD-42C7-82D5-90E8614BFC1C@sonic.net> MIME-version: 1.0 X-Mailer: Apple Mail (2.1077) Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-Antispam: No, score=-0.2/5.0, scanned in 0.146sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <409C3415-CAEE-423A-B166-DD3326469031@sonic.net> <4BB0E679.4050207@Oracle.COM> Status: RO Content-Length: 1378 On Mar 29, 2010, at 10:42 AM, Darren J Moffat wrote: > On 29/03/2010 18:30, Don Cragun wrote: >>> + + Indicates the file/directory was added in the later dataset >>> + - Indicates the file/directory was removed in the later dataset >>> + M Indicates the file/directory was modified in the later dataset >>> + R Indicates the file/directory was renamed in the later dataset >> >> Again, "file/directory" should just be "file" in all four lines above. > > While a that may be technically true I personally found it very useful that it said file/directory. Particularly since this isn't a POSIX C API man page. > > I'd rather it made it clear that both files and directories, and all other types of filesystem objects are supported here, and that it do so by explicitly saying file and directory. I'm not an ARC member, so you are free to ignore my comments. But, explicitly saying "file of any type and file of type directory" makes absolutely no sense to me. I don't see that it is clearer; it just raises the question of what does "file" mean on this man page if "directory" is not a type of file. I know you don't like POSIX/SUS based man pages, but is pretty basic. It clearly shows that the S_IFMT portion of the st_mode field (which has type mode_t) specifies the file type. - Don > > -- > Darren J Moffat From Darren.Moffat@Oracle.COM Mon Mar 29 11:37:44 2010 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2TIbhh1003467 for ; Mon, 29 Mar 2010 11:37:43 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o2TIbegK019181; Mon, 29 Mar 2010 11:37:43 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L0200B1F3QUQ300@brm-avmta-1.central.sun.com>; Mon, 29 Mar 2010 12:37:42 -0600 (MDT) Received: from sca-ea-mail-4.sun.com ([192.18.43.22]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L02005Z73QTSS50@brm-avmta-1.central.sun.com>; Mon, 29 Mar 2010 12:37:41 -0600 (MDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by sca-ea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o2TIbffM001693; Mon, 29 Mar 2010 18:37:41 +0000 (GMT) Received: from acsmt355.oracle.com (acsmt355.oracle.com [141.146.40.155]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o2THACKl031249; Mon, 29 Mar 2010 18:37:37 +0000 (GMT) Received: from abhmt017.oracle.com by acsmt355.oracle.com with ESMTP id 126583511269887856; Mon, 29 Mar 2010 11:37:36 -0700 Received: from [10.7.251.221] (/10.7.251.221) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 29 Mar 2010 11:37:35 -0700 Date: Mon, 29 Mar 2010 19:37:29 +0100 From: Darren J Moffat Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <6C772477-CDBD-42C7-82D5-90E8614BFC1C@sonic.net> To: Don Cragun Cc: tim.haley@Oracle.COM, zfs-team@sun.com, PSARC-ext@sun.com Message-id: <4BB0F369.8010006@Oracle.COM> Organization: Oracle Solaris Security MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt355.oracle.com [141.146.40.155] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A0B0206.4BB0F372.0232:SCFMA4539814,ss=1,fgs=0 References: <409C3415-CAEE-423A-B166-DD3326469031@sonic.net> <4BB0E679.4050207@Oracle.COM> <6C772477-CDBD-42C7-82D5-90E8614BFC1C@sonic.net> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.7) Gecko/20100214 Lightning/1.0b1 Thunderbird/3.0.1 Status: RO Content-Length: 1820 On 29/03/2010 19:21, Don Cragun wrote: > On Mar 29, 2010, at 10:42 AM, Darren J Moffat wrote: > >> On 29/03/2010 18:30, Don Cragun wrote: >>>> + + Indicates the file/directory was added in the later dataset >>>> + - Indicates the file/directory was removed in the later dataset >>>> + M Indicates the file/directory was modified in the later dataset >>>> + R Indicates the file/directory was renamed in the later dataset >>> >>> Again, "file/directory" should just be "file" in all four lines above. >> >> While a that may be technically true I personally found it very useful that it said file/directory. Particularly since this isn't a POSIX C API man page. >> >> I'd rather it made it clear that both files and directories, and all other types of filesystem objects are supported here, and that it do so by explicitly saying file and directory. > > I'm not an ARC member, so you are free to ignore my comments. But, > explicitly saying "file of any type and file of type directory" makes > absolutely no sense to me. I don't see that it is clearer; it just > raises the question of what does "file" mean on this man page if > "directory" is not a type of file. Think like a user that doesn't know C programming and how these things are implemented and doesn't know what POSIX/SUS is. > I know you don't like POSIX/SUS based man pages, but is > pretty basic. It clearly shows that the S_IFMT portion of the st_mode > field (which has type mode_t) specifies the file type. Which is fine for a developer but not for an end admin or user. Remember ZFS commands can be delegated to users. Users thing in terms of files and directories (and depending on where they came from they might still be calling them folders not directories). -- Darren J Moffat From Nicolas.Williams@sun.com Mon Mar 29 11:49:12 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2TInB3L003820 for ; Mon, 29 Mar 2010 11:49:11 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o2TInAM7002816; Mon, 29 Mar 2010 13:49:11 -0500 (CDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L0200G1549YQK00@nwk-avmta-1.sfbay.Sun.COM>; Mon, 29 Mar 2010 11:49:10 -0700 (PDT) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L0200FA849YO900@nwk-avmta-1.sfbay.Sun.COM>; Mon, 29 Mar 2010 11:49:10 -0700 (PDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o2TIn9tN027528; Mon, 29 Mar 2010 18:49:10 +0000 (GMT) Received: from acsmt353.oracle.com (acsmt353.oracle.com [141.146.40.153]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o2TGcIpV008403; Mon, 29 Mar 2010 18:49:09 +0000 (GMT) Received: from abhmt017.oracle.com by acsmt355.oracle.com with ESMTP id 120784161269888548; Mon, 29 Mar 2010 11:49:08 -0700 Received: from Sun.COM (/129.153.128.104) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 29 Mar 2010 11:49:07 -0700 Date: Mon, 29 Mar 2010 13:49:03 -0500 From: Nicolas Williams Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <4BB0EE00.1080705@oracle.com> To: Bart Smaalders Cc: Matthew Ahrens , Sebastien Roy , Tim Haley , PSARC-ext@sun.com, ZFS Team Message-id: <20100329184903.GW21244@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt353.oracle.com [141.146.40.153] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090205.4BB0F625.018C,ss=1,fgs=0 References: <4BAFAA10.2070706@oracle.com> <20100329161407.GN21244@Sun.COM> <4BB0D41C.2000203@oracle.com> <20100329163306.GP21244@Sun.COM> <4BB0EB07.9080403@oracle.com> <4BB0EE00.1080705@oracle.com> User-Agent: Mutt/1.5.20 (2010-03-02) Status: RO Content-Length: 604 On Mon, Mar 29, 2010 at 11:14:24AM -0700, Bart Smaalders wrote: > On 03/29/10 11:01, Matthew Ahrens wrote: > > >How do commands like ls and find handle printing of filenames with > >arbitrary characters (newlines and such)? > > In general, badly. > > % touch `echo '\07'` > % ls > > % Use ls -b: -b --escape Forces printing of non-printable characters to be in the octal \ddd notation. Octal escapes for non-printable characters seems like the way to go. There's also GNU ls find's -print0 and GNU xargs' --nul (-0). (GNU ls also has the -b option). Nico -- From dcragun@sonic.net Mon Mar 29 12:10:37 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2TJAbi4004521 for ; Mon, 29 Mar 2010 12:10:37 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o2TJAa1s026989 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Mon, 29 Mar 2010 13:10:37 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L0200M0559O4R00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 29 Mar 2010 12:10:36 -0700 (PDT) Received: from sca-ea-mail-4.sun.com ([192.18.43.22]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L0200FI959OO910@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 29 Mar 2010 12:10:36 -0700 (PDT) Received: from relay15i.sun.com (ip125.net129179-4.block1.us.syntegra.com [129.179.4.125]) by sca-ea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o2TIx5i9009780 for ; Mon, 29 Mar 2010 19:10:36 +0000 (GMT) Received: from mmp14es.mmp.us.syntegra.com ([160.41.208.14] [160.41.208.14]) by relay15i.sun.com with ESMTP id BT-MMP-3295099; Mon, 29 Mar 2010 19:10:35 +0000 (Z) Received: from relay13i.sun.com (relay13i.sun.com [129.179.4.123]) by mmp14es.mmp.us.syntegra.com with ESMTP id BT-MMP-500850; Mon, 29 Mar 2010 19:10:35 +0000 (Z) Received: from b.mail.sonic.net ([64.142.19.5] [64.142.19.5]) by relay1i.sun.com with ESMTP id BT-MMP-32249797; Mon, 29 Mar 2010 19:10:35 +0000 (Z) Received: from [10.0.0.4] (76-191-129-144.dsl.dynamic.sonic.net [76.191.129.144]) (authenticated bits=0) by b.mail.sonic.net (8.13.8.Beta0-Sonic/8.13.7) with ESMTP id o2TJARlc023742 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Mon, 29 Mar 2010 12:10:28 -0700 Date: Mon, 29 Mar 2010 12:10:27 -0700 From: Don Cragun Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <4BB0F369.8010006@Oracle.COM> To: Darren J Moffat Cc: tim.haley@Oracle.COM, zfs-team@sun.com, PSARC-ext@sun.com Message-id: MIME-version: 1.0 X-Mailer: Apple Mail (2.1077) Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-Antispam: No, score=-1.1/5.0, scanned in 0.069sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <409C3415-CAEE-423A-B166-DD3326469031@sonic.net> <4BB0E679.4050207@Oracle.COM> <6C772477-CDBD-42C7-82D5-90E8614BFC1C@sonic.net> <4BB0F369.8010006@Oracle.COM> Status: RO Content-Length: 2168 On Mar 29, 2010, at 11:37 AM, Darren J Moffat wrote: > > > On 29/03/2010 19:21, Don Cragun wrote: >> On Mar 29, 2010, at 10:42 AM, Darren J Moffat wrote: >> >>> On 29/03/2010 18:30, Don Cragun wrote: >>>>> + + Indicates the file/directory was added in the later dataset >>>>> + - Indicates the file/directory was removed in the later dataset >>>>> + M Indicates the file/directory was modified in the later dataset >>>>> + R Indicates the file/directory was renamed in the later dataset >>>> >>>> Again, "file/directory" should just be "file" in all four lines above. >>> >>> While a that may be technically true I personally found it very useful that it said file/directory. Particularly since this isn't a POSIX C API man page. >>> >>> I'd rather it made it clear that both files and directories, and all other types of filesystem objects are supported here, and that it do so by explicitly saying file and directory. >> >> I'm not an ARC member, so you are free to ignore my comments. But, >> explicitly saying "file of any type and file of type directory" makes >> absolutely no sense to me. I don't see that it is clearer; it just >> raises the question of what does "file" mean on this man page if >> "directory" is not a type of file. > > Think like a user that doesn't know C programming and how these things are implemented and doesn't know what POSIX/SUS is. > >> I know you don't like POSIX/SUS based man pages, but is >> pretty basic. It clearly shows that the S_IFMT portion of the st_mode >> field (which has type mode_t) specifies the file type. > > Which is fine for a developer but not for an end admin or user. > > Remember ZFS commands can be delegated to users. Users thing in terms of files and directories (and depending on where they came from they might still be calling them folders not directories). > OK. I see what you're trying to do now. Please change "the file/directory" on all four lines to "something". Naive users won't get lost in the details and savvy programmers won't be confused by the real, overlapping definitions. - Don > > -- > Darren J Moffat From tim.haley@oracle.com Mon Mar 29 15:27:10 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2TMR9kC007433 for ; Mon, 29 Mar 2010 15:27:10 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o2TMR7vI037179; Mon, 29 Mar 2010 16:27:07 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L0200M0FED7TE00@nwk-avmta-2.sfbay.sun.com>; Mon, 29 Mar 2010 15:27:07 -0700 (PDT) Received: from sca-ea-mail-2.sun.com ([192.18.43.25]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L02006JRED5XOD0@nwk-avmta-2.sfbay.sun.com>; Mon, 29 Mar 2010 15:27:06 -0700 (PDT) Received: from rcsinet15.oracle.com (rcsinet15.oracle.com [148.87.113.117]) by sca-ea-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o2TMR53A029096; Mon, 29 Mar 2010 22:27:05 +0000 (GMT) Received: from acsmt355.oracle.com (acsmt355.oracle.com [141.146.40.155]) by rcsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o2TKLRVr030897; Mon, 29 Mar 2010 22:27:04 +0000 (GMT) Received: from abhmt014.oracle.com by acsmt354.oracle.com with ESMTP id 127305391269901622; Mon, 29 Mar 2010 15:27:02 -0700 Received: from [172.20.25.27] (/10.85.25.27) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 29 Mar 2010 15:27:02 -0700 Date: Mon, 29 Mar 2010 16:27:02 -0600 From: Tim Haley Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <4BB04C10.5050300@Sun.COM> To: Darren Reed Cc: PSARC-ext@sun.com, ZFS Team Message-id: <4BB12936.8010702@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt355.oracle.com [141.146.40.155] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090209.4BB12939.0021:SCFMA4539814,ss=1,fgs=0 References: <4BAFAA10.2070706@oracle.com> <4BB04C10.5050300@Sun.COM> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.8) Gecko/20100302 Thunderbird/3.0.2 Status: RO Content-Length: 2223 On 03/29/10 12:43 AM, Darren Reed wrote: > On 03/28/10 12:12, Tim Haley wrote: >> I am sponsoring the following fast-track on behalf of myself. This >> case introduces a new zfs sub-command for describing differences >> between snapshots in a zfs hierarchy. A delegated permission and >> read-only system attribute are also introduced to support the >> sub-command. The case requests micro/patch binding. >> >> Template Version: @(#)sac_nextcase 1.69 02/15/10 SMI >> This information is Copyright 2010 Sun Microsystems >> 1. Introduction >> 1.1. Project/Component Working Name: >> zfs diff >> 1.2. Name of Document Author/Supplier: >> Author: Tim Haley >> 1.3 Date of This Document: >> 28 March, 2010 >> >> 4. Technical Description >> >> There is a long-standing RFE for zfs to be able to describe >> what has changed between the snapshots of a dataset. >> To provide this capability, we propose a new 'zfs diff' >> sub-command. When run with appropriate privilege the >> sub-command describes what file system level changes have >> occurred between the requested snapshots. A diff between the >> current version of the file system and one of its snapshots is >> also supported. >> >> Five types of change are described: >> >> o File/Directory modified >> o File/Directory present in older snapshot but not newer >> o File/Directory present in newer snapshot but not older >> o File/Directory renamed >> o File link count changed >> >> Diffs can be performed if the user is delegated the "diff" >> permission. The "diff" permission is being introduced by this >> case. Diffs can also be performed without the "diff" >> permission, if the user has appropriate privilege. For diffs >> between existing snapshots, the necessary privilege is >> {PRIV_SYS_CONFIG}. For diff between the current file system >> and a snapshot {PRIV_SYS_MOUNT} is also necessary. >> >> Also introduced by this case is a system attribute on zfs files >> called 'generation'. This attribute is part of the >> XATTR_VIEW_READONLY described in PSARC 2007/315. It is >> generated automatically by the ZFS module. > > Does "zfs diff" work between different versions of zfs filesystems? > If so, what are the restrictions? > Yes. No restrictions. -tim From dp@eng.sun.com Mon Mar 29 21:54:03 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2U4s3VZ013351 for ; Mon, 29 Mar 2010 21:54:03 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o2U4s2fH003003; Mon, 29 Mar 2010 23:54:03 -0500 (CDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L0200201WA2XU00@nwk-avmta-1.sfbay.Sun.COM>; Mon, 29 Mar 2010 21:54:02 -0700 (PDT) Received: from jurassic.Eng.Sun.COM ([129.146.17.59]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L02006UAWA291D0@nwk-avmta-1.sfbay.Sun.COM>; Mon, 29 Mar 2010 21:54:02 -0700 (PDT) Received: from xanadu (xanadu.SFBay.Sun.COM [129.146.228.194]) by jurassic.Eng.Sun.COM (8.14.4+Sun/8.14.4) with ESMTP id o2U4s1gh176331; Mon, 29 Mar 2010 21:54:01 -0700 (PDT) Date: Mon, 29 Mar 2010 21:54:01 -0700 From: Dan Price Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <4BB0EE00.1080705@oracle.com> To: Bart Smaalders Cc: Matthew Ahrens , Nicolas Williams , Sebastien Roy , Tim Haley , PSARC-ext@sun.com, ZFS Team Message-id: <20100330045401.GA2197@eng.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <4BAFAA10.2070706@oracle.com> <20100329161407.GN21244@Sun.COM> <4BB0D41C.2000203@oracle.com> <20100329163306.GP21244@Sun.COM> <4BB0EB07.9080403@oracle.com> <4BB0EE00.1080705@oracle.com> User-Agent: Mutt/1.5.20 (2009-06-14) Status: RO Content-Length: 2555 On Mon 29 Mar 2010 at 11:14AM, Bart Smaalders wrote: > On 03/29/10 11:01, Matthew Ahrens wrote: > > >How do commands like ls and find handle printing of filenames with > >arbitrary characters (newlines and such)? > > In general, badly. Tim, My concern, which others have hinted at, is that there are a legion of people who are going to want to consume this information and there is great value in making said information be machine parseable. Automated build systems, tripwires, fancy backup/recovery tools, et cetera. In summary, the current output seems mostly OK if it's for humans, but the case is ambiguous about who the expected consumer is. It would be a tragedy if there wasn't a machine consumable way to get at this information. I also have questions about how intelligent a consuming piece of software must be in order to make sense of this information. Has anyone written a proof of concept tool using this? For example, if a directory /foo/a is renamed to /foo/b, then an analyzer would need to stat /foo/b in order to discover that /foo/b is a directory, then traverse into as needed. It would be a shame if everyone who wanted to consume this had to write the same thousand lines of code (I'm happy to be convinced that this isn't the case). Some specific questions... 1) In what order are the changes printed? If I saw: + /myfiles/rename_dir R /myfiles/rename_dir -> /myfiles/rename_dir My analyzer would need to be smart enough to realize that the second must have happened before the first, and that both paths need evaluation. Right? 2) The meaning of "file/directory" (Don's concern aside) seems ambiguous in the proposal. Are we tracking the filesystem *namespace* entry? Or the actual object? I found that not being sure of this made the proposal hard to evaluate. Simple thought experiment which confused me: snapshot@1 rm a/b rm a/c rmdir a echo "foo" > a snapshot@2 Does that yield this? Or this? - a/b | - a/b - a/c | - a/c - a | M a + a | 3) Output is shown with leading slashes. Is output shown relative to the mount point? Or something else? (If the former, what if between @a and @b the mountpoint changed?) 4) I would also vote for a mode which simply outputs a list of pathnames to investigate for differences. This would enable: zfs diff -someflag a@1 a@2 | xargs do_some_analysis_on_these Thanks for tackling this, -dp -- Daniel Price, Solaris Kernel Engineering http://blogs.sun.com/dp From michael.schuster@oracle.com Mon Mar 29 22:29:51 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2U5ToVZ014053 for ; Mon, 29 Mar 2010 22:29:50 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o2U5TZJf022231; Tue, 30 Mar 2010 00:29:50 -0500 (CDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L0200D01XXO7B00@nwk-avmta-1.sfbay.Sun.COM>; Mon, 29 Mar 2010 22:29:48 -0700 (PDT) Received: from sca-ea-mail-1.sun.com ([192.18.43.24]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L02008EZXXN2760@nwk-avmta-1.sfbay.Sun.COM>; Mon, 29 Mar 2010 22:29:47 -0700 (PDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by sca-ea-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o2U5TkTr023063; Tue, 30 Mar 2010 05:29:46 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o2U5Tjnt011256; Tue, 30 Mar 2010 05:29:45 +0000 (GMT) Received: from abhmt013.oracle.com by acsmt355.oracle.com with ESMTP id 122225881269926966; Mon, 29 Mar 2010 22:29:26 -0700 Received: from [10.7.250.72] (/10.7.250.72) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 29 Mar 2010 22:29:26 -0700 Date: Tue, 30 Mar 2010 07:29:20 +0200 From: Michael Schuster Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <20100330045401.GA2197@eng.sun.com> To: PSARC-ext@sun.com Cc: Tim Haley , ZFS Team Message-id: <4BB18C30.1060600@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090205.4BB18C4A.007D:SCFMA4539814,ss=1,fgs=0 References: <4BAFAA10.2070706@oracle.com> <20100329161407.GN21244@Sun.COM> <4BB0D41C.2000203@oracle.com> <20100329163306.GP21244@Sun.COM> <4BB0EB07.9080403@oracle.com> <4BB0EE00.1080705@oracle.com> <20100330045401.GA2197@eng.sun.com> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.2pre) Gecko/20100204 Thunderbird/3.1a1 Status: RO Content-Length: 1941 On 30.03.10 06:54, Dan Price wrote: > On Mon 29 Mar 2010 at 11:14AM, Bart Smaalders wrote: >> On 03/29/10 11:01, Matthew Ahrens wrote: >> >>> How do commands like ls and find handle printing of filenames with >>> arbitrary characters (newlines and such)? >> >> In general, badly. > > Tim, > > My concern, which others have hinted at, is that there are a legion of > people who are going to want to consume this information and there is > great value in making said information be machine parseable. Automated > build systems, tripwires, fancy backup/recovery tools, et cetera. > > In summary, the current output seems mostly OK if it's for humans, but > the case is ambiguous about who the expected consumer is. It would > be a tragedy if there wasn't a machine consumable way to get at this > information. May I humbly point at dladm here, specifically at the -p option that various show-* subcommands utilise? Here's the relevant snippet from the man-page: Parseable Output Format Many dladm subcommands have an option that displays output in a machine-parseable format. The output format is one or more lines of colon (:) delimited fields. The fields displayed are specific to the subcommand used and are listed under the entry for the -o option for a given subcommand. Output includes only those fields requested by means of the -o option, in the order requested. When you request multiple fields, any literal colon charac- ters are escaped by a backslash (\) before being output. Similarly, literal backslash characters will also be escaped (\\). This escape format is parseable by using shell read(1) functions with the environment variable IFS=: (see EXAMPLES, below). Note that escaping is not done when you request only a single field. regards Michael -- michael.schuster@oracle.com Recursion, n.: see 'Recursion' From tim.haley@oracle.com Mon Mar 29 22:36:23 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2U5aNTu014355 for ; Mon, 29 Mar 2010 22:36:23 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o2U5aM7T025552; Tue, 30 Mar 2010 00:36:23 -0500 (CDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L0200E03Y8NYA00@nwk-avmta-1.sfbay.Sun.COM>; Mon, 29 Mar 2010 22:36:23 -0700 (PDT) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L02008T9Y8M2780@nwk-avmta-1.sfbay.Sun.COM>; Mon, 29 Mar 2010 22:36:22 -0700 (PDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o2U5aLEw023734; Tue, 30 Mar 2010 05:36:22 +0000 (GMT) Received: from acsmt353.oracle.com (acsmt353.oracle.com [141.146.40.153]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o2U5DQxY009932; Tue, 30 Mar 2010 05:36:21 +0000 (GMT) Received: from abhmt019.oracle.com by acsmt354.oracle.com with ESMTP id 128158991269927360; Mon, 29 Mar 2010 22:36:00 -0700 Received: from Giles.local (/10.7.250.184) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 29 Mar 2010 22:36:00 -0700 Date: Mon, 29 Mar 2010 23:35:54 -0600 From: Tim Haley Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <4BB0E744.60001@oracle.com> To: Sebastien Roy Cc: Nicolas Williams , PSARC-ext@sun.com, ZFS Team Message-id: <4BB18DBA.80801@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt353.oracle.com [141.146.40.153] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090204.4BB18DD5.011B:SCFMA4539814,ss=1,fgs=0 References: <4BAFAA10.2070706@oracle.com> <20100329161407.GN21244@Sun.COM> <4BB0D41C.2000203@oracle.com> <20100329163306.GP21244@Sun.COM> <4BB0E0AA.2050703@oracle.com> <20100329172445.GS21244@Sun.COM> <4BB0E744.60001@oracle.com> User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3 Status: RO Content-Length: 2438 On 3/29/10 11:45 AM, Sebastien Roy wrote: > On 03/29/10 01:24 PM, Nicolas Williams wrote: >> On Mon, Mar 29, 2010 at 01:17:30PM -0400, Sebastien Roy wrote: >>> Understood, and I see that now. It would indeed make sense to be >>> consistent and have -H for this subcommand as well. >> >> Agreed. Do you agree re: backslash-escaping? > > Yes, but presumably the existing output syntax for -H has already > taken that into account, no? The existing syntax appears to use a > single tab as a separator. I would hope that a tab within a field > would be escaped, and if it's not that should be a bug (otherwise it's > not really parsable). > >>>> I could have filenames with ' -> ' in them that would render the >>>> rename >>>> output ambiguous to the human eye. I could have filenames with >>>> '\n' in the name that would render the output ambiguous >>>> to the human eye. >>> >>> My intention wasn't to rathole into some absurd discussion over how >>> to handle ridiculous filenames. >> >> My intention is to avoid security problems in consumers of zfs diff. I >> assert that zfs diff is mostly useful only in connection with scripting. >> >> I don't think it's reasonable to expect that zfs diff be useful only "by >> eye". It has got to be scriptable because for any sufficiently large >> and active dataset the output of zfs diff will generally be too large to >> handle "by eye" (sure, you could grep for specific things, but not much >> beyond that you're scripting). >> >> (If zfs diff is only useful for scripting then the -H option is actually >> unnecessary and zfs diff should always disambiguate pathnames in its >> output.) > > I don't think it's only useful for scripting. It makes perfect sense > to me to incorporate -H as with other zfs subcommands. Tim, what is > your plan regarding this suggestion? > > -Seb My proposal would be add a -H option. I'm going to say the output without this option remains as described, and ++ With the -H option, parseable output is produced. Fields are ++ separated by a single tab, and no '->' is placed between ++ the old and new names of a rename. Whitespace characters, the ++ backslash character, and other characters not in the print ++ class for the locale are represented in the output as a ++ backslash character followed by the three-digit octal ++ representation of the byte value. ++ -tim From Nicolas.Williams@sun.com Mon Mar 29 23:21:09 2010 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2U6L9jW014898 for ; Mon, 29 Mar 2010 23:21:09 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o2U6L8p3000896; Mon, 29 Mar 2010 23:21:08 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L0300A030B8C500@nwk-avmta-2.sfbay.sun.com>; Mon, 29 Mar 2010 23:21:08 -0700 (PDT) Received: from sca-ea-mail-2.sun.com ([192.18.43.25]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L0300JOX0B89U90@nwk-avmta-2.sfbay.sun.com>; Mon, 29 Mar 2010 23:21:08 -0700 (PDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by sca-ea-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o2U6L2jm027144; Tue, 30 Mar 2010 06:21:02 +0000 (GMT) Received: from acsmt355.oracle.com (acsmt355.oracle.com [141.146.40.155]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o2TNuIwU011858; Tue, 30 Mar 2010 06:21:01 +0000 (GMT) Received: from abhmt004.oracle.com by acsmt354.oracle.com with ESMTP id 128261511269930061; Mon, 29 Mar 2010 23:21:01 -0700 Received: from Sun.COM (/129.153.128.104) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 29 Mar 2010 23:21:00 -0700 Date: Tue, 30 Mar 2010 01:20:55 -0500 From: Nicolas Williams Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <20100330045401.GA2197@eng.sun.com> To: Dan Price Cc: Bart Smaalders , Matthew Ahrens , Sebastien Roy , Tim Haley , PSARC-ext@sun.com, ZFS Team Message-id: <20100330062055.GJ21244@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt355.oracle.com [141.146.40.155] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090203.4BB1984E.005B,ss=1,fgs=0 References: <4BAFAA10.2070706@oracle.com> <20100329161407.GN21244@Sun.COM> <4BB0D41C.2000203@oracle.com> <20100329163306.GP21244@Sun.COM> <4BB0EB07.9080403@oracle.com> <4BB0EE00.1080705@oracle.com> <20100330045401.GA2197@eng.sun.com> User-Agent: Mutt/1.5.20 (2010-03-02) Status: RO Content-Length: 2484 On Mon, Mar 29, 2010 at 09:54:01PM -0700, Dan Price wrote: > I also have questions about how intelligent a consuming piece of > software must be in order to make sense of this information. Has anyone > written a proof of concept tool using this? For example, if a directory > /foo/a is renamed to /foo/b, then an analyzer would need to stat /foo/b > in order to discover that /foo/b is a directory, then traverse into as > needed. It would be a shame if everyone who wanted to consume this had > to write the same thousand lines of code (I'm happy to be convinced that > this isn't the case). Using simple backslash escaping works well with the shell's read built-in. Backslash octal escaping also works well with the shell's read built-in using the -r option (but not with /usr/has/bin/sh, which has no -r option). I'm happy with octal escapes for shell scripting. For other purposes one might really like HTML/XML entities (e.g., '&'), but I don't expect the i-team will want to cater to every developer :/ Instead, an API would be nice, since an API could completely avoid any ambiguity directly. That could come later. > Some specific questions... > > 1) In what order are the changes printed? If I saw: > > + /myfiles/rename_dir > R /myfiles/rename_dir -> /myfiles/rename_dir > > My analyzer would need to be smart enough to realize that the second > must have happened before the first, and that both paths need > evaluation. Right? Between two snapshots only the later event would be found, surely. (But I'm not the i-team.) > 2) The meaning of "file/directory" (Don's concern aside) seems ambiguous in > the proposal. Are we tracking the filesystem *namespace* entry? Or the > actual object? I found that not being sure of this made the proposal > hard to evaluate. Simple thought experiment which confused me: A related question: what about zero-link files that were open at the time that a snapshot was taken? Presumably there's no need to even mention their existence since they can't be accessed by the consumer. > 4) I would also vote for a mode which simply outputs a list of > pathnames to investigate for differences. This would enable: > > zfs diff -someflag a@1 a@2 | xargs do_some_analysis_on_these If you're going to target xargs then definitely, definitely include a GNU find-like -print0 option to match xargs' -0 option. That completely avoids ambiguity. (And, of course, for renames output only the new name.) Nico -- From Nicolas.Williams@sun.com Mon Mar 29 23:25:05 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2U6P5WF014932 for ; Mon, 29 Mar 2010 23:25:05 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o2U6P141021390; Tue, 30 Mar 2010 00:25:03 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L03006070HQ4800@nwk-avmta-1.sfbay.Sun.COM>; Mon, 29 Mar 2010 23:25:02 -0700 (PDT) Received: from sca-ea-mail-2.sun.com ([192.18.43.25]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L0300KTL0HQKW10@nwk-avmta-1.sfbay.Sun.COM>; Mon, 29 Mar 2010 23:25:02 -0700 (PDT) Received: from rcsinet15.oracle.com (rcsinet15.oracle.com [148.87.113.117]) by sca-ea-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o2U6Oufq028032; Tue, 30 Mar 2010 06:24:56 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by rcsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o2TL1DHs027447; Tue, 30 Mar 2010 06:24:55 +0000 (GMT) Received: from abhmt008.oracle.com by acsmt355.oracle.com with ESMTP id 128269331269930260; Mon, 29 Mar 2010 23:24:20 -0700 Received: from Sun.COM (/129.153.128.104) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 29 Mar 2010 23:24:20 -0700 Date: Tue, 30 Mar 2010 01:24:15 -0500 From: Nicolas Williams Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <20100330062055.GJ21244@Sun.COM> To: Dan Price Cc: Bart Smaalders , Matthew Ahrens , Sebastien Roy , Tim Haley , PSARC-ext@sun.com, ZFS Team Message-id: <20100330062415.GM21318@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090202.4BB19938.005E,ss=1,fgs=0 References: <4BAFAA10.2070706@oracle.com> <20100329161407.GN21244@Sun.COM> <4BB0D41C.2000203@oracle.com> <20100329163306.GP21244@Sun.COM> <4BB0EB07.9080403@oracle.com> <4BB0EE00.1080705@oracle.com> <20100330045401.GA2197@eng.sun.com> <20100330062055.GJ21244@Sun.COM> User-Agent: Mutt/1.5.20 (2010-03-02) Status: RO Content-Length: 816 On Tue, Mar 30, 2010 at 01:20:55AM -0500, Nicolas Williams wrote: > On Mon, Mar 29, 2010 at 09:54:01PM -0700, Dan Price wrote: > > Some specific questions... > > > > 1) In what order are the changes printed? If I saw: > > > > + /myfiles/rename_dir > > R /myfiles/rename_dir -> /myfiles/rename_dir > > > > My analyzer would need to be smart enough to realize that the second > > must have happened before the first, and that both paths need > > evaluation. Right? > > Between two snapshots only the later event would be found, surely. (But > I'm not the i-team.) Oh, but for modified and rename order matters, and may not be what one expects (rename first, then update, so that one could inspect the file with the new name). But I think one could script around this problem, if it is a problem. Nico -- From Vladimir.Marek@Sun.COM Tue Mar 30 04:35:50 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2UBZoLE007643 for ; Tue, 30 Mar 2010 04:35:50 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o2UBZoLs008322; Tue, 30 Mar 2010 06:35:50 -0500 (CDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L0300K03EVP5I00@nwk-avmta-1.sfbay.Sun.COM>; Tue, 30 Mar 2010 04:35:49 -0700 (PDT) Received: from gmp-eb-inf-2.sun.com ([192.18.6.24]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L0300LMUEVOY640@nwk-avmta-1.sfbay.Sun.COM>; Tue, 30 Mar 2010 04:35:49 -0700 (PDT) Received: from fe-emea-09.sun.com (gmp-eb-lb-1-fe1.eu.sun.com [192.18.6.7] (may be forged)) by gmp-eb-inf-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o2UBZmgB006911; Tue, 30 Mar 2010 11:35:48 +0000 (GMT) Received: from conversion-daemon.fe-emea-09.sun.com by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0L0300400E7JM300@fe-emea-09.sun.com>; Tue, 30 Mar 2010 12:35:27 +0100 (BST) Received: from tortuga.czech.sun.com ([unknown] [129.157.71.66]) by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0L03000RVEURDQ50@fe-emea-09.sun.com>; Tue, 30 Mar 2010 12:35:16 +0100 (BST) Date: Tue, 30 Mar 2010 13:35:14 +0200 From: Vladimir Marek Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <4BAFAA10.2070706@oracle.com> Sender: Vladimir.Marek@Sun.COM To: Tim Haley Cc: PSARC-ext@Sun.COM, ZFS Team Mail-followup-to: Tim Haley , PSARC-ext@Sun.COM, ZFS Team Message-id: <20100330113513.GF3834@tortuga.czech.sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <4BAFAA10.2070706@oracle.com> User-Agent: Mutt/1.5.20 (2010-02-01) Status: RO Content-Length: 386 Hi, I'm wondering, will zfs diff work between two zfs pools? I want to know if the current snapshot of my data differs from the snapshot I created by "zfs send | zfs receive" while ago (on the same machine, just different pool), so that I should refresh my backup. Does 'zfs diff' read all the data in snapshot, or compares just some checksums and thus is fast? Thank you -- Vlad From tim.haley@oracle.com Tue Mar 30 06:22:08 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2UDM8eJ008850 for ; Tue, 30 Mar 2010 06:22:08 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o2UDM5Hg000367; Tue, 30 Mar 2010 07:22:08 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L030010ZJSUD300@nwk-avmta-1.sfbay.Sun.COM>; Tue, 30 Mar 2010 06:22:06 -0700 (PDT) Received: from sca-ea-mail-3.sun.com ([192.18.43.21]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L0300L69JSTY290@nwk-avmta-1.sfbay.Sun.COM>; Tue, 30 Mar 2010 06:22:05 -0700 (PDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by sca-ea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o2UDM5rG016281; Tue, 30 Mar 2010 13:22:05 +0000 (GMT) Received: from acsmt355.oracle.com (acsmt355.oracle.com [141.146.40.155]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o2TM8xnq014155; Tue, 30 Mar 2010 13:22:04 +0000 (GMT) Received: from abhmt006.oracle.com by acsmt354.oracle.com with ESMTP id 123436701269955321; Tue, 30 Mar 2010 06:22:01 -0700 Received: from Giles.local (/10.7.250.184) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 30 Mar 2010 06:22:01 -0700 Date: Tue, 30 Mar 2010 07:21:59 -0600 From: Tim Haley Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <20100330113513.GF3834@tortuga.czech.sun.com> To: PSARC-ext@sun.com, ZFS Team Message-id: <4BB1FAF7.8030804@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt355.oracle.com [141.146.40.155] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090207.4BB1FAFC.01F3:SCFMA4539814,ss=1,fgs=0 References: <4BAFAA10.2070706@oracle.com> <20100330113513.GF3834@tortuga.czech.sun.com> User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3 Status: RO Content-Length: 603 On 3/30/10 5:35 AM, Vladimir Marek wrote: > Hi, > > I'm wondering, will zfs diff work between two zfs pools? > > No. Only between snapshots within the same dataset within the same pool. > I want to know if the current snapshot of my data differs from the > snapshot I created by "zfs send | zfs receive" while ago (on the same > machine, just different pool), so that I should refresh my backup. > > Does 'zfs diff' read all the data in snapshot, or compares just some > checksums and thus is fast? > > We do our best to only read meta-data in finding the differences. -tim > Thank you > From glenn.brunette@oracle.com Tue Mar 30 09:58:49 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2UGwnpS011901 for ; Tue, 30 Mar 2010 09:58:49 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o2UGwjYs047831; Tue, 30 Mar 2010 10:58:48 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L030020DTTZFY00@nwk-avmta-2.sfbay.sun.com>; Tue, 30 Mar 2010 09:58:47 -0700 (PDT) Received: from brmea-mail-2.sun.com ([192.18.98.43]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L0300L2STTWKQ50@nwk-avmta-2.sfbay.sun.com>; Tue, 30 Mar 2010 09:58:44 -0700 (PDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o2UGwhl2000762; Tue, 30 Mar 2010 16:58:43 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o2UBRFVm009537; Tue, 30 Mar 2010 16:58:40 +0000 (GMT) Received: from abhmt002.oracle.com by acsmt353.oracle.com with ESMTP id 130353051269968290; Tue, 30 Mar 2010 09:58:10 -0700 Received: from orion.local (/141.144.112.248) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 30 Mar 2010 09:58:09 -0700 Date: Tue, 30 Mar 2010 12:58:06 -0400 From: Glenn Brunette Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <20100330045401.GA2197@eng.sun.com> To: Dan Price Cc: Bart Smaalders , Matthew Ahrens , Nicolas Williams , Sebastien Roy , Tim Haley , PSARC-ext@sun.com, ZFS Team Message-id: <4BB22D9E.6010601@oracle.com> Organization: Oracle Corporation MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090205.4BB22DC2.0054:SCFMA4539814,ss=1,fgs=0 References: <4BAFAA10.2070706@oracle.com> <20100329161407.GN21244@Sun.COM> <4BB0D41C.2000203@oracle.com> <20100329163306.GP21244@Sun.COM> <4BB0EB07.9080403@oracle.com> <4BB0EE00.1080705@oracle.com> <20100330045401.GA2197@eng.sun.com> User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4 Status: RO Content-Length: 3404 Dan, I have to +1 all of your well-thought-out comments. As a potential consuming of this functionality for the Immutable Service Container project, answers to these questions are critical. I am also interested in whether additional fields can be added to the output similar to a "-o field1,field2" scenario? It would be nice to have data such as file type, modification time (where applicable). Also, will this functionality be able to tell how files were modified? Things like changes in file ownership, group membership, permissions and ACLs, size, times, etc.? Even if this processing is not directly implemented as part of the zfs diff command, perhaps the fields could be made available (per -o comment above) to be consumed by layered tools? g On 3/30/10 12:54 AM, Dan Price wrote: > On Mon 29 Mar 2010 at 11:14AM, Bart Smaalders wrote: >> On 03/29/10 11:01, Matthew Ahrens wrote: >> >>> How do commands like ls and find handle printing of filenames with >>> arbitrary characters (newlines and such)? >> >> In general, badly. > > Tim, > > My concern, which others have hinted at, is that there are a legion of > people who are going to want to consume this information and there is > great value in making said information be machine parseable. Automated > build systems, tripwires, fancy backup/recovery tools, et cetera. > > In summary, the current output seems mostly OK if it's for humans, but > the case is ambiguous about who the expected consumer is. It would > be a tragedy if there wasn't a machine consumable way to get at this > information. > > I also have questions about how intelligent a consuming piece of > software must be in order to make sense of this information. Has anyone > written a proof of concept tool using this? For example, if a directory > /foo/a is renamed to /foo/b, then an analyzer would need to stat /foo/b > in order to discover that /foo/b is a directory, then traverse into as > needed. It would be a shame if everyone who wanted to consume this had > to write the same thousand lines of code (I'm happy to be convinced that > this isn't the case). > > Some specific questions... > > 1) In what order are the changes printed? If I saw: > > + /myfiles/rename_dir > R /myfiles/rename_dir -> /myfiles/rename_dir > > My analyzer would need to be smart enough to realize that the second > must have happened before the first, and that both paths need > evaluation. Right? > > 2) The meaning of "file/directory" (Don's concern aside) seems ambiguous in > the proposal. Are we tracking the filesystem *namespace* entry? Or the > actual object? I found that not being sure of this made the proposal > hard to evaluate. Simple thought experiment which confused me: > > snapshot@1 > rm a/b > rm a/c > rmdir a > echo "foo"> a > snapshot@2 > > Does that yield this? Or this? > > - a/b | - a/b > - a/c | - a/c > - a | M a > + a | > > 3) Output is shown with leading slashes. Is output shown relative to the > mount point? Or something else? (If the former, what if between @a and > @b the mountpoint changed?) > > 4) I would also vote for a mode which simply outputs a list of > pathnames to investigate for differences. This would enable: > > zfs diff -someflag a@1 a@2 | xargs do_some_analysis_on_these > > > Thanks for tackling this, > > -dp > From MAILER-DAEMON Tue Mar 30 11:30:01 2010 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2UIU1fS013710 for ; Tue, 30 Mar 2010 11:30:01 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o2UITxnW016239; Tue, 30 Mar 2010 11:30:00 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L030070FY1ZO400@nwk-avmta-2.sfbay.sun.com>; Tue, 30 Mar 2010 11:29:59 -0700 (PDT) Received: from jurassic.Eng.Sun.COM ([129.146.17.59]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L0300LR9Y1XKGB0@nwk-avmta-2.sfbay.sun.com>; Tue, 30 Mar 2010 11:29:57 -0700 (PDT) Received: from xanadu (xanadu.SFBay.Sun.COM [129.146.228.194]) by jurassic.Eng.Sun.COM (8.14.4+Sun/8.14.4) with ESMTP id o2UITu9k261836; Tue, 30 Mar 2010 11:29:56 -0700 (PDT) Date: Tue, 30 Mar 2010 11:29:56 -0700 From: Dan Price Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <20100330062055.GJ21244@Sun.COM> To: Nicolas Williams Cc: Bart Smaalders , Matthew Ahrens , Sebastien Roy , Tim Haley , PSARC-ext@sun.com, ZFS Team Message-id: <20100330182952.GA26891@eng.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <4BAFAA10.2070706@oracle.com> <20100329161407.GN21244@Sun.COM> <4BB0D41C.2000203@oracle.com> <20100329163306.GP21244@Sun.COM> <4BB0EB07.9080403@oracle.com> <4BB0EE00.1080705@oracle.com> <20100330045401.GA2197@eng.sun.com> <20100330062055.GJ21244@Sun.COM> User-Agent: Mutt/1.4.2.1i Status: RO Content-Length: 1364 On Tue 30 Mar 2010 at 01:20AM, Nicolas Williams wrote: > > 1) In what order are the changes printed? If I saw: > > > > + /myfiles/rename_dir > > R /myfiles/rename_dir -> /myfiles/rename_dir > > > > My analyzer would need to be smart enough to realize that the second > > must have happened before the first, and that both paths need > > evaluation. Right? > > Between two snapshots only the later event would be found, surely. (But > I'm not the i-team.) The example was slightly messed up, sorry; that caused misunderstanding. I'm worried about this situation: snapshot@1 mv /myfiles/name1 /myfiles/name2 mkdir /myfiles/name1 snapshot@2 So, I'm fairly sure that between the two snapshots both events are relevant. So the above might yield: + /myfiles/name1 R /myfiles/name1 -> /myfiles/name2 It's tempting to read this as "name1 was created, then renamed to name2." But that isn't what it means. For a human, the following version portrays the events much more clearly: R /myfiles/name1 -> /myfiles/name2 + /myfiles/name1 I'm not sufficiently expert about all the pieces here to understand whether it's possible to efficiently or even sensibly sort things into a "human friendly" order. -dp -- Daniel Price, Solaris Kernel Engineering http://blogs.sun.com/dp From tim.haley@oracle.com Tue Mar 30 11:42:32 2010 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2UIgWKw013804 for ; Tue, 30 Mar 2010 11:42:32 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o2UIgVhf011001; Tue, 30 Mar 2010 11:42:31 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L0300105YMVZ600@brm-avmta-1.central.sun.com>; Tue, 30 Mar 2010 12:42:31 -0600 (MDT) Received: from sca-ea-mail-3.sun.com ([192.18.43.21]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L03007THYMUD0B0@brm-avmta-1.central.sun.com>; Tue, 30 Mar 2010 12:42:31 -0600 (MDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by sca-ea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o2UIgUns011280; Tue, 30 Mar 2010 18:42:30 +0000 (GMT) Received: from acsmt355.oracle.com (acsmt355.oracle.com [141.146.40.155]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o2UIe4u3011354; Tue, 30 Mar 2010 18:42:26 +0000 (GMT) Received: from abhmt020.oracle.com by acsmt355.oracle.com with ESMTP id 124453431269974521; Tue, 30 Mar 2010 11:42:01 -0700 Received: from [172.20.25.27] (/10.85.25.27) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 30 Mar 2010 11:42:00 -0700 Date: Tue, 30 Mar 2010 12:42:00 -0600 From: Tim Haley Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <20100330182952.GA26891@eng.sun.com> To: Dan Price Cc: Nicolas Williams , Bart Smaalders , Matthew Ahrens , Sebastien Roy , PSARC-ext@sun.com, ZFS Team Message-id: <4BB245F8.3000107@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt355.oracle.com [141.146.40.155] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A0B0203.4BB24614.0331:SCFMA4539814,ss=1,fgs=0 References: <4BAFAA10.2070706@oracle.com> <20100329161407.GN21244@Sun.COM> <4BB0D41C.2000203@oracle.com> <20100329163306.GP21244@Sun.COM> <4BB0EB07.9080403@oracle.com> <4BB0EE00.1080705@oracle.com> <20100330045401.GA2197@eng.sun.com> <20100330062055.GJ21244@Sun.COM> <20100330182952.GA26891@eng.sun.com> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.8) Gecko/20100302 Thunderbird/3.0.2 Status: RO Content-Length: 1692 On 03/30/10 12:29 PM, Dan Price wrote: > On Tue 30 Mar 2010 at 01:20AM, Nicolas Williams wrote: >>> 1) In what order are the changes printed? If I saw: >>> >>> + /myfiles/rename_dir >>> R /myfiles/rename_dir -> /myfiles/rename_dir >>> >>> My analyzer would need to be smart enough to realize that the second >>> must have happened before the first, and that both paths need >>> evaluation. Right? >> >> Between two snapshots only the later event would be found, surely. (But >> I'm not the i-team.) > > The example was slightly messed up, sorry; that caused misunderstanding. > I'm worried about this situation: > > snapshot@1 > mv /myfiles/name1 /myfiles/name2 > mkdir /myfiles/name1 > snapshot@2 > > So, I'm fairly sure that between the two snapshots both events are > relevant. So the above might yield: > > + /myfiles/name1 > R /myfiles/name1 -> /myfiles/name2 > > It's tempting to read this as "name1 was created, then renamed to > name2." But that isn't what it means. For a human, the following > version portrays the events much more clearly: > > R /myfiles/name1 -> /myfiles/name2 > + /myfiles/name1 > > I'm not sufficiently expert about all the pieces here to understand > whether it's possible to efficiently or even sensibly sort things into > a "human friendly" order. > Apologies for not responding sooner, I've been playing a bit with the code. Based on what I've been doing this morning, I believe it will be possible to present the list in roughly chronological order. The order originally was just by object number, so you could get either of the outputs you show above. -tim From MAILER-DAEMON Tue Mar 30 11:48:38 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2UImcdJ013822 for ; Tue, 30 Mar 2010 11:48:38 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o2UImaRG011520; Tue, 30 Mar 2010 12:48:37 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L030080TYX0RJ00@nwk-avmta-2.sfbay.sun.com>; Tue, 30 Mar 2010 11:48:36 -0700 (PDT) Received: from jurassic.Eng.Sun.COM ([129.146.17.59]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L0300LBPYX0KTD0@nwk-avmta-2.sfbay.sun.com>; Tue, 30 Mar 2010 11:48:36 -0700 (PDT) Received: from xanadu (xanadu.SFBay.Sun.COM [129.146.228.194]) by jurassic.Eng.Sun.COM (8.14.4+Sun/8.14.4) with ESMTP id o2UImaKm263432; Tue, 30 Mar 2010 11:48:36 -0700 (PDT) Date: Tue, 30 Mar 2010 11:48:36 -0700 From: Dan Price Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <4BB1FAF7.8030804@oracle.com> To: Tim Haley Cc: PSARC-ext@sun.com, ZFS Team Message-id: <20100330184836.GB26891@eng.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <4BAFAA10.2070706@oracle.com> <20100330113513.GF3834@tortuga.czech.sun.com> <4BB1FAF7.8030804@oracle.com> User-Agent: Mutt/1.4.2.1i Status: RO Content-Length: 1064 On Tue 30 Mar 2010 at 07:21AM, Tim Haley wrote: > On 3/30/10 5:35 AM, Vladimir Marek wrote: > >Hi, > > > >I'm wondering, will zfs diff work between two zfs pools? > > > > No. Only between snapshots within the same dataset within the same pool. Only within the same dataset? What about a snapshot of a clone, compared with a snapshot from the source of the clone? In other words, given this scenario: # zfs create rpool/testdiff # zfs snapshot rpool/testdiff@1 # zfs clone rpool/testdiff@1 rpool/testdiffchild # zfs snapshot rpool/testdiffchild@2 Then I can do this: # zfs send -i rpool/testdiff@1 rpool/testdiffchild@2 This seems to me to be a useful application; perhaps what you meant was "it will only work when 'zfs send' would work"? (As an aside, similar to the way send can send full snapshots, it'd be nice to have a way to have "null" as the lefthand side of a diff. That is, since birth of the fs, what's changed? Sort of like 'diff /dev/null /myfile') -dp -- Daniel Price, Solaris Kernel Engineering http://blogs.sun.com/dp From Nicolas.Williams@sun.com Tue Mar 30 12:25:55 2010 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2UJPtWm014574 for ; Tue, 30 Mar 2010 12:25:55 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o2UJPpT4015177; Tue, 30 Mar 2010 12:25:55 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L040061V0N7I600@brm-avmta-1.central.sun.com>; Tue, 30 Mar 2010 13:25:55 -0600 (MDT) Received: from sca-ea-mail-2.sun.com ([192.18.43.25]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L04006US0N62W00@brm-avmta-1.central.sun.com>; Tue, 30 Mar 2010 13:25:54 -0600 (MDT) Received: from rcsinet13.oracle.com (rcsinet13.oracle.com [148.87.113.125]) by sca-ea-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o2UJPmCZ007321; Tue, 30 Mar 2010 19:25:48 +0000 (GMT) Received: from acsmt355.oracle.com (acsmt355.oracle.com [141.146.40.155]) by rcsinet13.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o2UFw23B032255; Tue, 30 Mar 2010 19:25:47 +0000 (GMT) Received: from abhmt010.oracle.com by acsmt355.oracle.com with ESMTP id 130911951269977137; Tue, 30 Mar 2010 12:25:37 -0700 Received: from Sun.COM (/129.153.128.104) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 30 Mar 2010 12:25:33 -0700 Date: Tue, 30 Mar 2010 14:25:28 -0500 From: Nicolas Williams Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <4BB245F8.3000107@oracle.com> To: Tim Haley Cc: Dan Price , Bart Smaalders , Matthew Ahrens , Sebastien Roy , PSARC-ext@sun.com, ZFS Team Message-id: <20100330192528.GR21244@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt355.oracle.com [141.146.40.155] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090207.4BB2503C.004D,ss=1,fgs=0 References: <4BAFAA10.2070706@oracle.com> <20100329161407.GN21244@Sun.COM> <4BB0D41C.2000203@oracle.com> <20100329163306.GP21244@Sun.COM> <4BB0EB07.9080403@oracle.com> <4BB0EE00.1080705@oracle.com> <20100330045401.GA2197@eng.sun.com> <20100330062055.GJ21244@Sun.COM> <20100330182952.GA26891@eng.sun.com> <4BB245F8.3000107@oracle.com> User-Agent: Mutt/1.5.20 (2010-03-02) Status: RO Content-Length: 1262 On Tue, Mar 30, 2010 at 12:42:00PM -0600, Tim Haley wrote: > On 03/30/10 12:29 PM, Dan Price wrote: > >The example was slightly messed up, sorry; that caused misunderstanding. > >I'm worried about this situation: > > > > snapshot@1 > > mv /myfiles/name1 /myfiles/name2 > > mkdir /myfiles/name1 > > snapshot@2 > > > >So, I'm fairly sure that between the two snapshots both events are > >relevant. So the above might yield: > > > > + /myfiles/name1 > > R /myfiles/name1 -> /myfiles/name2 Ah, sure. > Apologies for not responding sooner, I've been playing a bit with > the code. Based on what I've been doing this morning, I believe it > will be possible to present the list in roughly chronological order. > The order originally was just by object number, so you could get > either of the outputs you show above. If you include the object number in the output then you can let the consumer figure it out. Increasing zfs diff's footprint to get it to sort its output correctly will decrease its performance, and the consumer might not care. (OTOH, a light-weight algorithm for properly sorting these output lines is fairly obvious, so maybe there's no reason to be concerned about performance.) Nico -- From tim.haley@oracle.com Tue Mar 30 13:05:02 2010 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2UK52wj015456 for ; Tue, 30 Mar 2010 13:05:02 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o2UK51UC021781; Tue, 30 Mar 2010 13:05:01 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L0400A0N2GDHZ00@brm-avmta-1.central.sun.com>; Tue, 30 Mar 2010 14:05:01 -0600 (MDT) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L04006JK2GC2WB0@brm-avmta-1.central.sun.com>; Tue, 30 Mar 2010 14:05:00 -0600 (MDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o2UK4xrw006531; Tue, 30 Mar 2010 20:04:59 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o2UB0Rgs025324; Tue, 30 Mar 2010 20:04:57 +0000 (GMT) Received: from abhmt006.oracle.com by acsmt355.oracle.com with ESMTP id 124670161269979481; Tue, 30 Mar 2010 13:04:41 -0700 Received: from [172.20.25.27] (/10.85.25.27) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 30 Mar 2010 13:04:40 -0700 Date: Tue, 30 Mar 2010 14:04:39 -0600 From: Tim Haley Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <20100330192528.GR21244@Sun.COM> To: Nicolas Williams Cc: Dan Price , Bart Smaalders , Matthew Ahrens , Sebastien Roy , PSARC-ext@sun.com, ZFS Team Message-id: <4BB25957.1080200@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090205.4BB2596B.0059:SCFMA4539814,ss=1,fgs=0 References: <4BAFAA10.2070706@oracle.com> <20100329161407.GN21244@Sun.COM> <4BB0D41C.2000203@oracle.com> <20100329163306.GP21244@Sun.COM> <4BB0EB07.9080403@oracle.com> <4BB0EE00.1080705@oracle.com> <20100330045401.GA2197@eng.sun.com> <20100330062055.GJ21244@Sun.COM> <20100330182952.GA26891@eng.sun.com> <4BB245F8.3000107@oracle.com> <20100330192528.GR21244@Sun.COM> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.8) Gecko/20100302 Thunderbird/3.0.2 Status: RO Content-Length: 1682 On 03/30/10 01:25 PM, Nicolas Williams wrote: > On Tue, Mar 30, 2010 at 12:42:00PM -0600, Tim Haley wrote: >> On 03/30/10 12:29 PM, Dan Price wrote: >>> The example was slightly messed up, sorry; that caused misunderstanding. >>> I'm worried about this situation: >>> >>> snapshot@1 >>> mv /myfiles/name1 /myfiles/name2 >>> mkdir /myfiles/name1 >>> snapshot@2 >>> >>> So, I'm fairly sure that between the two snapshots both events are >>> relevant. So the above might yield: >>> >>> + /myfiles/name1 >>> R /myfiles/name1 -> /myfiles/name2 > > Ah, sure. > >> Apologies for not responding sooner, I've been playing a bit with >> the code. Based on what I've been doing this morning, I believe it >> will be possible to present the list in roughly chronological order. >> The order originally was just by object number, so you could get >> either of the outputs you show above. > > If you include the object number in the output then you can let the > consumer figure it out. Increasing zfs diff's footprint to get it to > sort its output correctly will decrease its performance, and the > consumer might not care. (OTOH, a light-weight algorithm for properly > sorting these output lines is fairly obvious, so maybe there's no reason > to be concerned about performance.) > > Nico It would be easy enough for me to print a 'time' column as the first column, and the output could then be sent to 'sort -n'. I'm not sure how people feel about that. Is that cheating? :-) The alternative is to AVL sort by that time, which as you note will increase the footprint, perhaps dramatically for a really big diff. -tim From tim.haley@oracle.com Tue Mar 30 13:23:40 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2UKNeNn015678 for ; Tue, 30 Mar 2010 13:23:40 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o2UKNbWE013903; Tue, 30 Mar 2010 14:23:39 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L0400E153BEIE00@nwk-avmta-2.sfbay.sun.com>; Tue, 30 Mar 2010 13:23:38 -0700 (PDT) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L0400BCG3BDAX30@nwk-avmta-2.sfbay.sun.com>; Tue, 30 Mar 2010 13:23:37 -0700 (PDT) Received: from rcsinet15.oracle.com (rcsinet15.oracle.com [148.87.113.117]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o2UKNaeC021130; Tue, 30 Mar 2010 20:23:36 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by rcsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o2UKNZSJ004560; Tue, 30 Mar 2010 20:23:35 +0000 (GMT) Received: from abhmt008.oracle.com by acsmt353.oracle.com with ESMTP id 131116661269980565; Tue, 30 Mar 2010 13:22:45 -0700 Received: from [172.20.25.27] (/10.85.25.27) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 30 Mar 2010 13:22:44 -0700 Date: Tue, 30 Mar 2010 14:22:43 -0600 From: Tim Haley Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <20100330045401.GA2197@eng.sun.com> To: Dan Price Cc: Bart Smaalders , Matthew Ahrens , Nicolas Williams , Sebastien Roy , PSARC-ext@sun.com, ZFS Team Message-id: <4BB25D93.20107@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090204.4BB25DC8.006F:SCFMA4539814,ss=1,fgs=0 References: <4BAFAA10.2070706@oracle.com> <20100329161407.GN21244@Sun.COM> <4BB0D41C.2000203@oracle.com> <20100329163306.GP21244@Sun.COM> <4BB0EB07.9080403@oracle.com> <4BB0EE00.1080705@oracle.com> <20100330045401.GA2197@eng.sun.com> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.8) Gecko/20100302 Thunderbird/3.0.2 Status: RO Content-Length: 3468 On 3/29/10 10:54 PM, Dan Price wrote: > On Mon 29 Mar 2010 at 11:14AM, Bart Smaalders wrote: > >> On 03/29/10 11:01, Matthew Ahrens wrote: >> >> >>> How do commands like ls and find handle printing of filenames with >>> arbitrary characters (newlines and such)? >>> >> In general, badly. >> > Tim, > > My concern, which others have hinted at, is that there are a legion of > people who are going to want to consume this information and there is > great value in making said information be machine parseable. Automated > build systems, tripwires, fancy backup/recovery tools, et cetera. > > In summary, the current output seems mostly OK if it's for humans, but > the case is ambiguous about who the expected consumer is. It would > be a tragedy if there wasn't a machine consumable way to get at this > information. > > I'm adding a -H option for scripting, with parseable output. > I also have questions about how intelligent a consuming piece of > software must be in order to make sense of this information. Has anyone > written a proof of concept tool using this? For example, if a directory > /foo/a is renamed to /foo/b, then an analyzer would need to stat /foo/b > in order to discover that /foo/b is a directory, then traverse into as > needed. It would be a shame if everyone who wanted to consume this had > to write the same thousand lines of code (I'm happy to be convinced that > this isn't the case). > > Some specific questions... > > 1) In what order are the changes printed? If I saw: > > + /myfiles/rename_dir > R /myfiles/rename_dir -> /myfiles/rename_dir > > My analyzer would need to be smart enough to realize that the second > must have happened before the first, and that both paths need > evaluation. Right? > > This got clarified, I believe. > 2) The meaning of "file/directory" (Don's concern aside) seems ambiguous in > the proposal. Are we tracking the filesystem *namespace* entry? Or the > actual object? I found that not being sure of this made the proposal > hard to evaluate. Simple thought experiment which confused me: > > snapshot@1 > rm a/b > rm a/c > rmdir a > echo "foo"> a > snapshot@2 > > Does that yield this? Or this? > > - a/b | - a/b > - a/c | - a/c > - a | M a > + a | > > We are tracking the actual file objects. Running your test with the current code: M /files/ - /files/a - /files/a/b - /files/a/c + /files/a Having slept on this, I think an extra field in the output will help. A type character could be added to another column, using the same sorts of symbols that ls -F shows, @ for symbolic link, / for directory, | for pipe, etc. Plus an 'F' to indicate a regular file. So the above would become: M /files/ / - /files/a F - /files/a/b F - /files/a/c F + /files/a F > 3) Output is shown with leading slashes. Is output shown relative to the > mount point? Or something else? (If the former, what if between @a and > @b the mountpoint changed?) > > The output shown is relative to where the dataset is mounted at the time of the diff. We don't necessarily know where it was mounted at the time of any particular snapshot. You are correct that the command should work with clones, too, as though are desendant. For a clone we'd present its paths relative to where it is mounted. -tim From Nicolas.Williams@sun.com Tue Mar 30 13:37:06 2010 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2UKb6Tc015739 for ; Tue, 30 Mar 2010 13:37:06 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o2UKb1m4023440; Tue, 30 Mar 2010 13:37:06 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L0400D173XTR400@brm-avmta-1.central.sun.com>; Tue, 30 Mar 2010 14:37:05 -0600 (MDT) Received: from sca-ea-mail-2.sun.com ([192.18.43.25]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L0400CU13XRD900@brm-avmta-1.central.sun.com>; Tue, 30 Mar 2010 14:37:03 -0600 (MDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by sca-ea-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o2UKawK8002865; Tue, 30 Mar 2010 20:36:58 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o2UJlsWE011959; Tue, 30 Mar 2010 20:36:57 +0000 (GMT) Received: from abhmt018.oracle.com by acsmt353.oracle.com with ESMTP id 124756571269981416; Tue, 30 Mar 2010 13:36:56 -0700 Received: from Sun.COM (/129.153.128.104) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 30 Mar 2010 13:36:56 -0700 Date: Tue, 30 Mar 2010 15:36:51 -0500 From: Nicolas Williams Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <4BB25957.1080200@oracle.com> To: Tim Haley Cc: Dan Price , Bart Smaalders , Matthew Ahrens , Sebastien Roy , PSARC-ext@sun.com, ZFS Team Message-id: <20100330203651.GT21244@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090209.4BB260E9.01D5,ss=1,fgs=0 References: <4BB0D41C.2000203@oracle.com> <20100329163306.GP21244@Sun.COM> <4BB0EB07.9080403@oracle.com> <4BB0EE00.1080705@oracle.com> <20100330045401.GA2197@eng.sun.com> <20100330062055.GJ21244@Sun.COM> <20100330182952.GA26891@eng.sun.com> <4BB245F8.3000107@oracle.com> <20100330192528.GR21244@Sun.COM> <4BB25957.1080200@oracle.com> User-Agent: Mutt/1.5.20 (2010-03-02) Status: RO Content-Length: 904 On Tue, Mar 30, 2010 at 02:04:39PM -0600, Tim Haley wrote: > It would be easy enough for me to print a 'time' column as the first > column, and the output could then be sent to 'sort -n'. I'm not sure > how people feel about that. Is that cheating? :-) The alternative > is to AVL sort by that time, which as you note will increase the > footprint, perhaps dramatically for a really big diff. I'd be happy with that. Someone suggested a -o field1,field2,..,fieldN option, and that's starting to look desirable. There's at least these fields that you could include in output: - object number - object type - timestamp* - generation number - type of change (create*, unlink*, rename*, other*, other meta-data, data) - old path* - new path* - link count* The starred ones are the ones included in your proposals so far. I'd be happy with just those; the others would be icing :) Nico -- From Nicolas.Williams@sun.com Tue Mar 30 13:45:50 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2UKjoJN015775 for ; Tue, 30 Mar 2010 13:45:50 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o2UKjlsK027884; Tue, 30 Mar 2010 15:45:49 -0500 (CDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L0400F034CCV100@nwk-avmta-2.sfbay.sun.com>; Tue, 30 Mar 2010 13:45:48 -0700 (PDT) Received: from sca-ea-mail-2.sun.com ([192.18.43.25]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L0400B024CCAS40@nwk-avmta-2.sfbay.sun.com>; Tue, 30 Mar 2010 13:45:48 -0700 (PDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by sca-ea-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o2UKjh8o005961; Tue, 30 Mar 2010 20:45:43 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o2UBTLLh016118; Tue, 30 Mar 2010 20:45:41 +0000 (GMT) Received: from abhmt002.oracle.com by acsmt354.oracle.com with ESMTP id 131185341269981940; Tue, 30 Mar 2010 13:45:40 -0700 Received: from Sun.COM (/129.153.128.104) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 30 Mar 2010 13:45:39 -0700 Date: Tue, 30 Mar 2010 15:45:34 -0500 From: Nicolas Williams Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <4BB25D93.20107@oracle.com> To: Tim Haley Cc: Dan Price , Bart Smaalders , Matthew Ahrens , Sebastien Roy , PSARC-ext@sun.com, ZFS Team Message-id: <20100330204534.GV21244@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090205.4BB262F6.00CF,ss=1,fgs=0 References: <4BAFAA10.2070706@oracle.com> <20100329161407.GN21244@Sun.COM> <4BB0D41C.2000203@oracle.com> <20100329163306.GP21244@Sun.COM> <4BB0EB07.9080403@oracle.com> <4BB0EE00.1080705@oracle.com> <20100330045401.GA2197@eng.sun.com> <4BB25D93.20107@oracle.com> User-Agent: Mutt/1.5.20 (2010-03-02) Status: RO Content-Length: 498 On Tue, Mar 30, 2010 at 02:22:43PM -0600, Tim Haley wrote: > You are correct that the command should work with clones, too, as > though are desendant. > For a clone we'd present its paths relative to where it is mounted. I'd say make all paths relative to the root of the dataset, even when the newer snapshot is of a clone. Let the consumer worry about how to find absolute paths to the named objects (by qualifying with the mount point of the relevant dataset and .zfs/snapshot/...). Nico -- From glenn.brunette@oracle.com Tue Mar 30 15:45:51 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2UMjplE017727 for ; Tue, 30 Mar 2010 15:45:51 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o2UMjomf050266; Tue, 30 Mar 2010 16:45:50 -0600 (MDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L04004079WE9R00@brm-avmta-1.central.sun.com>; Tue, 30 Mar 2010 16:45:50 -0600 (MDT) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L0400CE19WEDG90@brm-avmta-1.central.sun.com>; Tue, 30 Mar 2010 16:45:50 -0600 (MDT) Received: from rcsinet15.oracle.com (rcsinet15.oracle.com [148.87.113.117]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o2UMjnoA000401; Tue, 30 Mar 2010 22:45:49 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by rcsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o2UKODKw006630; Tue, 30 Mar 2010 22:45:48 +0000 (GMT) Received: from abhmt013.oracle.com by acsmt353.oracle.com with ESMTP id 125028661269989102; Tue, 30 Mar 2010 15:45:02 -0700 Received: from orion.local (/68.82.150.225) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 30 Mar 2010 15:45:02 -0700 Date: Tue, 30 Mar 2010 18:45:00 -0400 From: Glenn Brunette Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <20100330203651.GT21244@Sun.COM> To: Nicolas Williams Cc: Tim Haley , Dan Price , Bart Smaalders , Matthew Ahrens , Sebastien Roy , PSARC-ext@sun.com, ZFS Team Message-id: <4BB27EEC.5050008@oracle.com> Organization: Oracle Corporation MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090205.4BB27F1C.0196:SCFMA4539814,ss=1,fgs=0 References: <4BB0D41C.2000203@oracle.com> <20100329163306.GP21244@Sun.COM> <4BB0EB07.9080403@oracle.com> <4BB0EE00.1080705@oracle.com> <20100330045401.GA2197@eng.sun.com> <20100330062055.GJ21244@Sun.COM> <20100330182952.GA26891@eng.sun.com> <4BB245F8.3000107@oracle.com> <20100330192528.GR21244@Sun.COM> <4BB25957.1080200@oracle.com> <20100330203651.GT21244@Sun.COM> User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4 Status: RO Content-Length: 1011 On 3/30/10 4:36 PM, Nicolas Williams wrote: > On Tue, Mar 30, 2010 at 02:04:39PM -0600, Tim Haley wrote: >> It would be easy enough for me to print a 'time' column as the first >> column, and the output could then be sent to 'sort -n'. I'm not sure >> how people feel about that. Is that cheating? :-) The alternative >> is to AVL sort by that time, which as you note will increase the >> footprint, perhaps dramatically for a really big diff. > > I'd be happy with that. Someone suggested a -o field1,field2,..,fieldN That would be me. > option, and that's starting to look desirable. There's at least these > fields that you could include in output: > > - object number > - object type > - timestamp* > - generation number > - type of change (create*, unlink*, rename*, other*, other meta-data, data) > - old path* > - new path* > - link count* > > The starred ones are the ones included in your proposals so far. I'd be > happy with just those; the others would be icing :) +1 g From john.plocher@gmail.com Tue Mar 30 19:02:29 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2V22Tb0020816 for ; Tue, 30 Mar 2010 19:02:29 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o2V22Scr003966; Tue, 30 Mar 2010 21:02:29 -0500 (CDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L0400E0BJ04GX00@nwk-avmta-2.sfbay.sun.com>; Tue, 30 Mar 2010 19:02:28 -0700 (PDT) Received: from sca-ea-mail-4.sun.com ([192.18.43.22]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L040091JJ044RD0@nwk-avmta-2.sfbay.sun.com>; Tue, 30 Mar 2010 19:02:28 -0700 (PDT) Received: from relay11i.sun.com (ip121.net129179-4.block1.us.syntegra.com [129.179.4.121]) by sca-ea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o2V1wOkM020705; Wed, 31 Mar 2010 02:02:28 +0000 (GMT) Received: from mmp12es.mmp.us.syntegra.com ([160.41.208.12] [160.41.208.12]) by relay11i.sun.com with ESMTP id BT-MMP-2502104; Wed, 31 Mar 2010 02:02:28 +0000 (Z) Received: from relay13i.sun.com (relay13i.sun.com [129.179.4.123]) by mmp12es.mmp.us.syntegra.com with ESMTP id BT-MMP-101145217; Wed, 31 Mar 2010 02:02:27 +0000 (Z) Received: from mail-qy0-f174.google.com ([209.85.221.174] [209.85.221.174]) by relay1i.sun.com with ESMTP id BT-MMP-35520981; Wed, 31 Mar 2010 02:02:27 +0000 (Z) Received: by mail-qy0-f174.google.com with SMTP id 4so1575548qyk.24 for ; Tue, 30 Mar 2010 19:02:18 -0700 (PDT) Received: by 10.229.85.193 with HTTP; Tue, 30 Mar 2010 19:02:17 -0700 (PDT) Received: by 10.229.225.7 with SMTP id iq7mr812565qcb.26.1270000937720; Tue, 30 Mar 2010 19:02:17 -0700 (PDT) Date: Tue, 30 Mar 2010 19:02:17 -0700 From: John Plocher Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <20100330203651.GT21244@Sun.COM> To: Nicolas Williams Cc: Tim Haley , PSARC-ext@sun.com, ZFS Team , Bart Smaalders , Matthew Ahrens , Dan Price Message-id: MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1 Content-transfer-encoding: 7BIT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:received:message-id:subject:from:to:cc:content-type; bh=MWLLm+1LFRLzDePAzFHKKWukWBdjG/lZJFUYWFWpLeI=; b=rLfWe+rwPwBYEF57OSZGMpMOBy9JWxzoE0U4o98HxUaJXLMrLmnsnqZ87/5fThJucf ybT6SsYWGK2rFv1uvGoVXkCzw1jXOlx1vgOhunvwjhY5oaH6Jg7JFqqMTYSRnyVJyiHS PzymBgZ7bUeTlCMqaHDhndus+XFq+gMZ0CC/k= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=Gsum34ym3zXEs3+ruMWwm3shEbo2TD7dxN/5ciZt6YNpCAWIUAPPrSmISxLwijlREK lQ0xSzKjRWaCtZYUNa7Hfj/X2HYNK6zMw0ElUoLy8z6nC5q6FLfz2RhXxEMImaaawc5w j1WIMBF0Xth17Y8wMjxIhHyC98YHTXKyntf74= X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-Antispam: No, score=0.0/5.0, scanned in 0.052sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <4BB0D41C.2000203@oracle.com> <4BB0EB07.9080403@oracle.com> <4BB0EE00.1080705@oracle.com> <20100330045401.GA2197@eng.sun.com> <20100330062055.GJ21244@Sun.COM> <20100330182952.GA26891@eng.sun.com> <4BB245F8.3000107@oracle.com> <20100330192528.GR21244@Sun.COM> <4BB25957.1080200@oracle.com> <20100330203651.GT21244@Sun.COM> Status: RO Content-Length: 512 On Tue, Mar 30, 2010 at 1:36 PM, Nicolas Williams wrote: > On Tue, Mar 30, 2010 at 02:04:39PM -0600, Tim Haley wrote: >> It would be easy enough for me to print a 'time' column as the first This is getting pretty close to "design by ARC" rather than "review by ARC"; it might be a better use of ARC bandwidth to take this discussion offline and place the case in "waiting need spec" mode... -John (who has ratholed his share of these, and so recognizes the symptoms easily :-) From Vladimir.Marek@sun.com Wed Mar 31 00:13:06 2010 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2V7D60S007370 for ; Wed, 31 Mar 2010 00:13:06 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o2V7D2Yw001052; Wed, 31 Mar 2010 00:13:06 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L0400E03XDTIJ00@brm-avmta-1.central.sun.com>; Wed, 31 Mar 2010 01:13:05 -0600 (MDT) Received: from gmp-eb-inf-1.sun.com ([192.18.6.21]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L0400GDAXDSLZC0@brm-avmta-1.central.sun.com>; Wed, 31 Mar 2010 01:13:04 -0600 (MDT) Received: from fe-emea-10.sun.com (gmp-eb-lb-1-fe1.eu.sun.com [192.18.6.7] (may be forged)) by gmp-eb-inf-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o2V7D3vc013663; Wed, 31 Mar 2010 07:13:03 +0000 (GMT) Received: from conversion-daemon.fe-emea-10.sun.com by fe-emea-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0L0400K00WSA0I00@fe-emea-10.sun.com>; Wed, 31 Mar 2010 08:12:44 +0100 (BST) Received: from tortuga.czech.sun.com ([unknown] [129.157.71.66]) by fe-emea-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0L04007SEXD01310@fe-emea-10.sun.com>; Wed, 31 Mar 2010 08:12:38 +0100 (BST) Date: Wed, 31 Mar 2010 09:12:36 +0200 From: Vladimir Marek Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <20100330184836.GB26891@eng.sun.com> Sender: Vladimir.Marek@sun.com To: Dan Price Cc: Tim Haley , PSARC-ext@sun.com, ZFS Team Mail-followup-to: Dan Price , Tim Haley , PSARC-ext@Sun.COM, ZFS Team Message-id: <20100331071235.GH3834@tortuga.czech.sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <4BAFAA10.2070706@oracle.com> <20100330113513.GF3834@tortuga.czech.sun.com> <4BB1FAF7.8030804@oracle.com> <20100330184836.GB26891@eng.sun.com> User-Agent: Mutt/1.5.20 (2010-02-01) Status: RO Content-Length: 1080 > > >I'm wondering, will zfs diff work between two zfs pools? > > > > > > > No. Only between snapshots within the same dataset within the same pool. > > Only within the same dataset? What about a snapshot of a clone, > compared with a snapshot from the source of the clone? > > In other words, given this scenario: > > # zfs create rpool/testdiff > # zfs snapshot rpool/testdiff@1 > # zfs clone rpool/testdiff@1 rpool/testdiffchild > # zfs snapshot rpool/testdiffchild@2 > > Then I can do this: > > # zfs send -i rpool/testdiff@1 rpool/testdiffchild@2 > > This seems to me to be a useful application; perhaps what you meant > was "it will only work when 'zfs send' would work"? That would be exactly what I would like to use. If I want to sync two datasets using incremental send, the only way is to remember which snapshot has to be a base fore next incremental send. Or name snapshots by a pattern (001, 002, ...) and trust that they won't be renamed and/or deleted and recreated. If I'm wrong and I overlooked something, please correct me. Thank you! -- Vlad From Darren.Moffat@oracle.com Wed Mar 31 04:07:00 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o2VB70sI014178 for ; Wed, 31 Mar 2010 04:07:00 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o2VB6xe8025825; Wed, 31 Mar 2010 05:06:59 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L050050987NO000@nwk-avmta-1.sfbay.Sun.COM>; Wed, 31 Mar 2010 04:06:59 -0700 (PDT) Received: from sca-ea-mail-4.sun.com ([192.18.43.22]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L0500M6X87M7Z20@nwk-avmta-1.sfbay.Sun.COM>; Wed, 31 Mar 2010 04:06:58 -0700 (PDT) Received: from rcsinet15.oracle.com (rcsinet15.oracle.com [148.87.113.117]) by sca-ea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o2VB6wPI029530; Wed, 31 Mar 2010 11:06:58 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by rcsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o2ULPkPe009956; Wed, 31 Mar 2010 11:06:54 +0000 (GMT) Received: from abhmt003.oracle.com by acsmt353.oracle.com with ESMTP id 133293561270033613; Wed, 31 Mar 2010 04:06:53 -0700 Received: from [10.7.251.221] (/10.7.251.221) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 31 Mar 2010 04:06:52 -0700 Date: Wed, 31 Mar 2010 12:06:48 +0100 From: Darren J Moffat Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <20100330203651.GT21244@Sun.COM> To: Nicolas Williams Cc: Tim Haley , PSARC-ext@sun.com, ZFS Team , Bart Smaalders , Matthew Ahrens , Dan Price Message-id: <4BB32CC8.9040402@Oracle.COM> Organization: Oracle Solaris Security MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090202.4BB32CD1.013B:SCFMA4539814,ss=1,fgs=0 References: <4BB0D41C.2000203@oracle.com> <20100329163306.GP21244@Sun.COM> <4BB0EB07.9080403@oracle.com> <4BB0EE00.1080705@oracle.com> <20100330045401.GA2197@eng.sun.com> <20100330062055.GJ21244@Sun.COM> <20100330182952.GA26891@eng.sun.com> <4BB245F8.3000107@oracle.com> <20100330192528.GR21244@Sun.COM> <4BB25957.1080200@oracle.com> <20100330203651.GT21244@Sun.COM> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.7) Gecko/20100214 Lightning/1.0b1 Thunderbird/3.0.1 Status: RO Content-Length: 912 On 30/03/2010 21:36, Nicolas Williams wrote: > On Tue, Mar 30, 2010 at 02:04:39PM -0600, Tim Haley wrote: >> It would be easy enough for me to print a 'time' column as the first >> column, and the output could then be sent to 'sort -n'. I'm not sure >> how people feel about that. Is that cheating? :-) The alternative >> is to AVL sort by that time, which as you note will increase the >> footprint, perhaps dramatically for a really big diff. > > I'd be happy with that. Someone suggested a -o field1,field2,..,fieldN > option, and that's starting to look desirable. There's at least these > fields that you could include in output: > > - object number I don't see what use the object number is to anyone parsing the output of 'zfs diff' particularly since there is no way for them to translate it to anything meaningful. The object number is an internal ZFS representation. -- Darren J Moffat From joerg.schilling9ab33xy531fokus.fraunhofer.de@bounce.antispameurope.com Thu Apr 1 02:11:25 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o319BOKq023053 for ; Thu, 1 Apr 2010 02:11:25 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o319BOGn021282 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Thu, 1 Apr 2010 04:11:24 -0500 (CDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L0600I37XJ0B100@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 01 Apr 2010 02:11:24 -0700 (PDT) Received: from sca-ea-mail-4.sun.com ([192.18.43.22]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L06006MHXIYEAA0@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 01 Apr 2010 02:11:22 -0700 (PDT) Received: from relay41i.sun.com ([192.5.209.70]) by sca-ea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o319AGLS008017 for ; Thu, 01 Apr 2010 09:11:22 +0000 (GMT) Received: from mms48es.mms.us.syntegra.com ([160.41.221.230] [160.41.221.230]) by relay41i.sun.com with ESMTP id BT-MMP-3480677 for PSARC-ext@sun.com; Thu, 01 Apr 2010 09:11:21 +0000 (Z) Received: from relay43i.sun.com (relay43i.sun.com [192.5.209.74]) by mms48es.mms.us.syntegra.com with ESMTP id BT-MMP-67206984 for PSARC-ext@sun.com; Thu, 01 Apr 2010 09:11:21 +0000 (Z) Received: from relay02-haj2.antispameurope.com ([83.246.65.52] [83.246.65.52]) by relay4i.sun.com with ESMTP id BT-MMP-10345316 for PSARC-ext@sun.com; Thu, 01 Apr 2010 09:11:20 +0000 (Z) Received: by relay02-haj2.antispameurope.com (ASE-Secure-MTA, from userid 1000) id 24F2B19401B; Thu, 01 Apr 2010 11:11:20 +0200 (CEST) Received: from pluto.fokus.fraunhofer.de (pluto.fokus.fraunhofer.de [195.37.77.164]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by relay02-haj2.antispameurope.com (ASE-Secure-MTA) with ESMTP id C39B419402E; Thu, 01 Apr 2010 11:11:13 +0200 (CEST) Received: from EXCHSRV.fokus.fraunhofer.de (bohr.fokus.fraunhofer.de [10.147.9.231]) by pluto.fokus.fraunhofer.de (8.14.2/8.14.2) with SMTP id o319BDP6010225; Thu, 01 Apr 2010 11:11:13 +0200 (MEST) Received: from rigel ([10.147.65.195]) by EXCHSRV.fokus.fraunhofer.de with Microsoft SMTPSVC(6.0.3790.3959); Thu, 01 Apr 2010 11:11:12 +0200 Date: Thu, 01 Apr 2010 11:11:13 +0200 From: Joerg.Schilling@fokus.fraunhofer.de (Joerg Schilling) Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <4BB18DBA.80801@oracle.com> Sender: joerg.schilling9ab33xy531fokus.fraunhofer.de@bounce.antispameurope.com To: tim.haley@oracle.com, sebastien.roy@oracle.com Cc: zfs-team@sun.com, PSARC-ext@sun.com Message-id: <4bb46331.w3QC5evXgGPsqTNj%Joerg.Schilling@fokus.fraunhofer.de> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1 Content-transfer-encoding: 8BIT X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-Antispam: No, score=0.0/5.0, scanned in 0.336sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <4BAFAA10.2070706@oracle.com> <20100329161407.GN21244@Sun.COM> <4BB0D41C.2000203@oracle.com> <20100329163306.GP21244@Sun.COM> <4BB0E0AA.2050703@oracle.com> <20100329172445.GS21244@Sun.COM> <4BB0E744.60001@oracle.com> <4BB18DBA.80801@oracle.com> User-Agent: nail 11.22 3/20/05 X-OriginalArrivalTime: 01 Apr 2010 09:11:12.0817 (UTC) FILETIME=[468AE210:01CAD17B] Status: RO Content-Length: 1071 Tim Haley wrote: > My proposal would be add a -H option. I'm going to say the output > without this option remains > as described, and > > ++ With the -H option, parseable output is produced. Fields are > ++ separated by a single tab, and no '->' is placed between > ++ the old and new names of a rename. Whitespace characters, the > ++ backslash character, and other characters not in the print > ++ class for the locale are represented in the output as a > ++ backslash character followed by the three-digit octal > ++ representation of the byte value. > ++ I strongly recommend not to use -H/-L/-P for for other porsuses than what POSIX describes for utililties that deal with directory trees. Jörg -- EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin js@cs.tu-berlin.de (uni) joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily From Darren.Moffat@Oracle.COM Thu Apr 1 03:36:52 2010 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o31AaqiH023701 for ; Thu, 1 Apr 2010 03:36:52 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o31AaqBI002382; Thu, 1 Apr 2010 03:36:52 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L070010R1HFQB00@brm-avmta-1.central.sun.com>; Thu, 01 Apr 2010 04:36:51 -0600 (MDT) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L07006XO1HCDAB0@brm-avmta-1.central.sun.com>; Thu, 01 Apr 2010 04:36:48 -0600 (MDT) Received: from rcsinet15.oracle.com (rcsinet15.oracle.com [148.87.113.117]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o31Aam8p029549; Thu, 01 Apr 2010 10:36:48 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by rcsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o31AafOg030197; Thu, 01 Apr 2010 10:36:41 +0000 (GMT) Received: from abhmt003.oracle.com by acsmt353.oracle.com with ESMTP id 137583811270118140; Thu, 01 Apr 2010 03:35:40 -0700 Received: from [10.7.251.221] (/10.7.251.221) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 01 Apr 2010 03:35:40 -0700 Date: Thu, 01 Apr 2010 11:35:36 +0100 From: Darren J Moffat Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <4bb46331.w3QC5evXgGPsqTNj%Joerg.Schilling@fokus.fraunhofer.de> To: Joerg.Schilling@fokus.fraunhofer.de Cc: tim.haley@Oracle.COM, sebastien.roy@Oracle.COM, zfs-team@sun.com, PSARC-ext@sun.com Message-id: <4BB476F8.5060508@Oracle.COM> Organization: Oracle Solaris Security MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090202.4BB4773D.0089:SCFMA4539814,ss=1,fgs=0 References: <4BAFAA10.2070706@oracle.com> <20100329161407.GN21244@Sun.COM> <4BB0D41C.2000203@oracle.com> <20100329163306.GP21244@Sun.COM> <4BB0E0AA.2050703@oracle.com> <20100329172445.GS21244@Sun.COM> <4BB0E744.60001@oracle.com> <4BB18DBA.80801@oracle.com> <4bb46331.w3QC5evXgGPsqTNj%Joerg.Schilling@fokus.fraunhofer.de> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.7) Gecko/20100214 Lightning/1.0b1 Thunderbird/3.0.1 Status: RO Content-Length: 1218 On 01/04/2010 10:11, Joerg.Schilling@fokus.fraunhofer.de wrote: > Tim Haley wrote: > >> My proposal would be add a -H option. I'm going to say the output >> without this option remains >> as described, and >> >> ++ With the -H option, parseable output is produced. Fields are >> ++ separated by a single tab, and no '->' is placed between >> ++ the old and new names of a rename. Whitespace characters, the >> ++ backslash character, and other characters not in the print >> ++ class for the locale are represented in the output as a >> ++ backslash character followed by the three-digit octal >> ++ representation of the byte value. >> ++ > > I strongly recommend not to use -H/-L/-P for for other porsuses than what POSIX > describes for utililties that deal with directory trees. zfs(1M) is not a POSIX utility and likely never will be so POSIX rules just don't apply. More importantly there is existing precedence for -H in zfs(1M) already: -H Used for scripting mode. Do not print headers and separate fields by a single tab instead of arbitrary white space. -- Darren J Moffat From joerg.schilling9ab33xy531fokus.fraunhofer.de@bounce.antispameurope.com Thu Apr 1 03:58:52 2010 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o31Awqla024035 for ; Thu, 1 Apr 2010 03:58:52 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o31AwqIS022361 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Thu, 1 Apr 2010 03:58:52 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L070010H2I4IZ00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 01 Apr 2010 03:58:52 -0700 (PDT) Received: from sca-ea-mail-3.sun.com ([192.18.43.21]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L070010P2I3IE00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 01 Apr 2010 03:58:51 -0700 (PDT) Received: from relay44i.sun.com ([192.5.209.118]) by sca-ea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o31AmjPF006342 for ; Thu, 01 Apr 2010 10:58:50 +0000 (GMT) Received: from mmp41es.mmp.us.syntegra.com ([160.41.221.10] [160.41.221.10]) by relay44i.sun.com with ESMTP id BT-MMP-3412222 for PSARC-ext@sun.com; Thu, 01 Apr 2010 10:58:50 +0000 (Z) Received: from relay42i.sun.com (relay42i.sun.com [192.5.209.72]) by mmp41es.mmp.us.syntegra.com with ESMTP id BT-MMP-68996440 for PSARC-ext@sun.com; Thu, 01 Apr 2010 10:58:49 +0000 (Z) Received: from relay02-haj2.antispameurope.com ([83.246.65.52] [83.246.65.52]) by relay4i.sun.com with ESMTP id BT-MMP-5163269 for PSARC-ext@sun.com; Thu, 01 Apr 2010 10:58:48 +0000 (Z) Received: by relay02-haj2.antispameurope.com (ASE-Secure-MTA, from userid 1000) id 8D8B16F055E; Thu, 01 Apr 2010 12:58:47 +0200 (CEST) Received: from pluto.fokus.fraunhofer.de (pluto.fokus.fraunhofer.de [195.37.77.164]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by relay02-haj2.antispameurope.com (ASE-Secure-MTA) with ESMTP id EE9BF6F0541; Thu, 01 Apr 2010 12:58:45 +0200 (CEST) Received: from EXCHSRV.fokus.fraunhofer.de (bohr.fokus.fraunhofer.de [10.147.9.231]) by pluto.fokus.fraunhofer.de (8.14.2/8.14.2) with SMTP id o31AwjPv012478; Thu, 01 Apr 2010 12:58:45 +0200 (MEST) Received: from rigel ([10.147.65.195]) by EXCHSRV.fokus.fraunhofer.de with Microsoft SMTPSVC(6.0.3790.3959); Thu, 01 Apr 2010 12:58:45 +0200 Date: Thu, 01 Apr 2010 12:58:45 +0200 From: Joerg.Schilling@fokus.fraunhofer.de (Joerg Schilling) Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/05/2010] In-reply-to: <20100330203651.GT21244@Sun.COM> Sender: joerg.schilling9ab33xy531fokus.fraunhofer.de@bounce.antispameurope.com To: tim.haley@oracle.com, Nicolas.Williams@sun.com Cc: zfs-team@sun.com, PSARC-ext@sun.com, matthew.ahrens@oracle.com, dp@eng.sun.com, bart.smaalders@oracle.com Message-id: <4bb47c65.kc/+SRHx3S5C64e/%Joerg.Schilling@fokus.fraunhofer.de> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1 Content-transfer-encoding: 8BIT X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-Antispam: No, score=-0.2/5.0, scanned in 0.616sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <4BB0D41C.2000203@oracle.com> <20100329163306.GP21244@Sun.COM> <4BB0EB07.9080403@oracle.com> <4BB0EE00.1080705@oracle.com> <20100330045401.GA2197@eng.sun.com> <20100330062055.GJ21244@Sun.COM> <20100330182952.GA26891@eng.sun.com> <4BB245F8.3000107@oracle.com> <20100330192528.GR21244@Sun.COM> <4BB25957.1080200@oracle.com> <20100330203651.GT21244@Sun.COM> User-Agent: nail 11.22 3/20/05 X-OriginalArrivalTime: 01 Apr 2010 10:58:45.0251 (UTC) FILETIME=[4C7E2D30:01CAD18A] Status: RO Content-Length: 2833 Nicolas Williams wrote: > I'd be happy with that. Someone suggested a -o field1,field2,..,fieldN > option, and that's starting to look desirable. There's at least these > fields that you could include in output: > > - object number > - object type > - timestamp* > - generation number > - type of change (create*, unlink*, rename*, other*, other meta-data, data) > - old path* > - new path* > - link count* If you like to discuss meta data diff options, I recommend you to look at software that implements this since a long time already...... Star implements meta data diffing since 1993. star diffopts=help Diff options: not if this option is present, exclude listed options ! if this option is present, exclude listed options all compare everything perm compare file permissions mode compare file permissions symperm compare symlink permissions type compare file type nlink compare linkcount (star dump mode only) uid compare owner of file gid compare group of file uname compare name of owner of file gname compare name of group of file id compare owner, group, ownername and groupname of file size compare file size data compare content of file cont compare content of file rdev compare rdev of device node hardlink compare target of hardlink symlink compare target of symlink sympath compare target pathname of symlink sparse compare if both files are sparse or not atime compare access time of file (only star) mtime compare modification time of file ctime compare creation time of file (only star) times compare all times of file lmtime compare modification time of symlinks xtimes compare all times and lmtime dir compare directory content (star dump mode only) acl compare access control lists (specify -acl also) xattr compare extended attributes (specify -xattr also) fflags compare extended file flags (specify -xfflags also) Default is to compare everything except atime. More information is in "man star". Jörg -- EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin js@cs.tu-berlin.de (uni) joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily From tim.haley@oracle.com Thu Apr 1 13:40:47 2010 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o31Kel4c004249 for ; Thu, 1 Apr 2010 13:40:47 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o31KehQn010155; Thu, 1 Apr 2010 13:40:46 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L0700G0NTFXEQ00@brm-avmta-1.central.sun.com>; Thu, 01 Apr 2010 14:40:45 -0600 (MDT) Received: from sca-ea-mail-2.sun.com ([192.18.43.25]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L07007R3TFW2R40@brm-avmta-1.central.sun.com>; Thu, 01 Apr 2010 14:40:44 -0600 (MDT) Received: from rcsinet15.oracle.com (rcsinet15.oracle.com [148.87.113.117]) by sca-ea-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o31Kei2k005358; Thu, 01 Apr 2010 20:40:44 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by rcsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o31KefWk022927; Thu, 01 Apr 2010 20:40:41 +0000 (GMT) Received: from abhmt004.oracle.com by acsmt355.oracle.com with ESMTP id 131653421270154381; Thu, 01 Apr 2010 13:39:41 -0700 Received: from [172.20.25.27] (/10.85.25.27) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 01 Apr 2010 13:39:40 -0700 Date: Thu, 01 Apr 2010 14:39:39 -0600 From: Tim Haley Subject: zfs diff [PSARC/2010/105 FastTrack timeout 04/08/2010] In-reply-to: <4BB476F8.5060508@Oracle.COM> To: PSARC-ext@sun.com Cc: zfs-team@sun.com Message-id: <4BB5048B.5080904@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090208.4BB504CB.00D5:SCFMA4539814,ss=1,fgs=0 References: <4BAFAA10.2070706@oracle.com> <20100329161407.GN21244@Sun.COM> <4BB0D41C.2000203@oracle.com> <20100329163306.GP21244@Sun.COM> <4BB0E0AA.2050703@oracle.com> <20100329172445.GS21244@Sun.COM> <4BB0E744.60001@oracle.com> <4BB18DBA.80801@oracle.com> <4bb46331.w3QC5evXgGPsqTNj%Joerg.Schilling@fokus.fraunhofer.de> <4BB476F8.5060508@Oracle.COM> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.8) Gecko/20100315 Thunderbird/3.0.3 Status: RO Content-Length: 7272 After a couple of days pondering a lot of the mail that's been flying about on this topic, I'm now re-submitting the specification with some minor changes. I've reset the time-out to a week from today. There have been many good and varied suggestions and I considered a number of more extensive changes and additions. For now it makes sense to me to start with something simple and let people "kick the tires" without having to predict the many clever uses they are going to come up with for this. There is going to be commonality in the applications that use this and in the future we can enhance the command to provide more information as necessary. We start with a simple utility that decreases the amount of processing one has to do by virtue of it quickly narrowing the search to what has actually changed. -tim ----------------------->8------------- There is a long-standing RFE for zfs to be able to describe what has changed between the snapshots of a dataset. To provide this capability, we propose a new 'zfs diff' sub-command. When run with appropriate privilege the sub-command describes what file system level changes have occurred between the requested snapshots. A diff between the current version of the file system and one of its snapshots is also supported. Five types of change are described: o File modified o File present in older snapshot but not newer o File present in newer snapshot but not older o File renamed o File link count changed Diffs can be performed if the user is delegated the "diff" permission. The "diff" permission is being introduced by this case. Diffs can also be performed without the "diff" permission, if the user has appropriate privilege. For diffs between existing snapshots, the necessary privilege is {PRIV_SYS_CONFIG}. For diff between the current file system and a snapshot {PRIV_SYS_MOUNT} is also necessary. Also introduced by this case is a system attribute on zfs files called 'generation'. This attribute is part of the XATTR_VIEW_READONLY described in PSARC 2007/315. It is generated automatically by the ZFS module. Man page changes: --- fgetattr.3c.rogi Sun Mar 21 18:47:29 2010 +++ fgetattr.3c Sun Mar 21 18:50:50 2010 @@ -97,6 +97,7 @@ XATTR_VIEW_READONLY A_FSID uint64_value A_OPAQUE boolean_value A_AV_SCANSTAMP uint8_array[] + A_GEN uint64_value XATTR_VIEW_READWRITE A_READONLY boolean_value A_HIDDEN boolean_value A_SYSTEM boolean_value --- zfs.1m.rogi Sun Mar 21 17:01:04 2010 +++ zfs.1m Wed Mar 31 11:12:05 2010 @@ -165,6 +165,9 @@ zfs release [-r] tag snapshot... + zfs diff [-FHt] snapshot snapshot|filesystem + + DESCRIPTION The zfs command configures ZFS datasets within a ZFS storage pool, as described in zpool(1M). A dataset is identified by @@ -1638,7 +1641,73 @@ size, the resulting behavior is undefined. + zfs diff [-FHt] snapshot snapshot | filesystem + Gives a high level description of the differences between a + snapshot and a descendant dataset. The descendant may either + be a later snapshot of the dataset or the current dataset. + + For each file that has undergone a change between the + original snapshot and the descendant, the type of change is + described along with the name of the file. In the case of a + rename, both the old and new names are shown. White-space + characters, backslash characters, and other non-printable or + non 7-bit ASCII characters found in file names are printed + as a backslash character followed by the three-digit octal + representation of the byte value. + + If the -t option is requested, the first column of output + from the command is the file's st_ctime (see stat(2)) + value. For deleted files, this is the st_ctime in the + earlier snapshot. + + The type of change follows any timestamp, and is described + with a single character: + + + Indicates the file was added in the later dataset + - Indicates the file was removed in the later dataset + M Indicates the file was modified in the later dataset + R Indicates the file was renamed in the later dataset + + If the -F option is requested the next column of output will + be a single character describing the type of the file. The + mappings are: + + F regular file + / directory + B block device + > door + | FIFO + @ symbolic link + P event portal + = socket + + If the modification involved a change in the link count of a + non-directory file, the change will be expressed as a delta + within parentheses on the modification line. If the file was + renamed, the old name will be separated from the new with the + string '->'. + + Example outputs with both -F and -t specified are below: + + 1269962518 M / /myfiles/ + 1269962444 M F /myfiles/link_to_me (+1) + 1269962499 R F /myfiles/rename_me -> /myfiles/renamed + 1269962431 - F /myfiles/delete_me + 1269962518 + F /myfiles/new_file + 1269962501 + | /myfiles/new_pipe + + If the -H option is selected, easier to parse output is + produced. Fields are separated by a single tab, and no '->' + is placed between the old and new names of a rename. No + guarantees are made on the spacing between fields of non -H + output. + + Users must be granted the diff permission with zfs allow in + order to use this sub-command, unless they already have the + {PRIV_SYS_CONFIG} privilege and, in the current file system + versus snapshot case, the {PRIV_SYS_MOUNT} privilege. + + zfs destroy [-rRf] filesystem|volume @@ -2733,6 +2802,7 @@ 'mount' ability in the origin file system create subcommand Must also have the 'mount' ability + diff subcommand destroy subcommand Must also have the 'mount' ability hold subcommand Allows adding a user hold to a snapshot mount subcommand Allows mount/umount of ZFS datasets @@ -3551,8 +3621,14 @@ tion Guide. +NOTES + A file described as modified by the diff sub-command may have been + modified in multiple ways. Any action which causes a change in + the st_ctime is grounds for reporting a modification. From Nicolas.Williams@sun.com Thu Apr 1 13:53:57 2010 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o31KrvhI004362 for ; Thu, 1 Apr 2010 13:53:57 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o31Kru4s021461; Thu, 1 Apr 2010 13:53:57 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L0700H01U1XPM00@brm-avmta-1.central.sun.com>; Thu, 01 Apr 2010 14:53:57 -0600 (MDT) Received: from sca-ea-mail-4.sun.com ([192.18.43.22]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L07007RRU1W2J50@brm-avmta-1.central.sun.com>; Thu, 01 Apr 2010 14:53:56 -0600 (MDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by sca-ea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o31KrtpC004747; Thu, 01 Apr 2010 20:53:56 +0000 (GMT) Received: from acsmt355.oracle.com (acsmt355.oracle.com [141.146.40.155]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o31CkIV7026845; Thu, 01 Apr 2010 20:53:55 +0000 (GMT) Received: from abhmt008.oracle.com by acsmt354.oracle.com with ESMTP id 139636381270155233; Thu, 01 Apr 2010 13:53:53 -0700 Received: from Sun.COM (/129.153.128.104) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 01 Apr 2010 13:53:53 -0700 Date: Thu, 01 Apr 2010 15:53:48 -0500 From: Nicolas Williams Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/08/2010] In-reply-to: <4BB5048B.5080904@oracle.com> To: Tim Haley Cc: PSARC-ext@sun.com, zfs-team@sun.com Message-id: <20100401205348.GJ4225@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt355.oracle.com [141.146.40.155] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090204.4BB507E3.00B1,ss=1,fgs=0 References: <20100329161407.GN21244@Sun.COM> <4BB0D41C.2000203@oracle.com> <20100329163306.GP21244@Sun.COM> <4BB0E0AA.2050703@oracle.com> <20100329172445.GS21244@Sun.COM> <4BB0E744.60001@oracle.com> <4BB18DBA.80801@oracle.com> <4bb46331.w3QC5evXgGPsqTNj%Joerg.Schilling@fokus.fraunhofer.de> <4BB476F8.5060508@Oracle.COM> <4BB5048B.5080904@oracle.com> User-Agent: Mutt/1.5.20 (2010-03-02) Status: RO Content-Length: 298 On Thu, Apr 01, 2010 at 02:39:39PM -0600, Tim Haley wrote: > After a couple of days pondering a lot of the mail that's been flying > about on this topic, I'm now re-submitting the specification with some > minor changes. I've reset the time-out to a week from today. Looks great to me, Nico -- From garrett.damore@oracle.com Thu Apr 1 14:04:23 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o31L4NEt004812 for ; Thu, 1 Apr 2010 14:04:23 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o31L4Mmd029586; Thu, 1 Apr 2010 16:04:22 -0500 (CDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L0700D03UJAVG00@nwk-avmta-2.sfbay.sun.com>; Thu, 01 Apr 2010 14:04:22 -0700 (PDT) Received: from sca-ea-mail-1.sun.com ([192.18.43.24]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L0700D3YUJ9UO00@nwk-avmta-2.sfbay.sun.com>; Thu, 01 Apr 2010 14:04:21 -0700 (PDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by sca-ea-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o31L4L2m014674; Thu, 01 Apr 2010 21:04:21 +0000 (GMT) Received: from acsmt353.oracle.com (acsmt353.oracle.com [141.146.40.153]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o3169LY7012766; Thu, 01 Apr 2010 21:04:19 +0000 (GMT) Received: from abhmt018.oracle.com by acsmt354.oracle.com with ESMTP id 139673061270155858; Thu, 01 Apr 2010 14:04:18 -0700 Received: from [10.7.251.172] (/10.7.251.172) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 01 Apr 2010 14:04:17 -0700 Date: Thu, 01 Apr 2010 14:04:16 -0700 From: "Garrett D'Amore" Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/08/2010] In-reply-to: <20100401205348.GJ4225@Sun.COM> To: Nicolas Williams Cc: Tim Haley , PSARC-ext@sun.com, zfs-team@sun.com Message-id: <4BB50A50.4000409@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt353.oracle.com [141.146.40.153] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090208.4BB50A54.0053:SCFMA4539814,ss=1,fgs=0 References: <20100329161407.GN21244@Sun.COM> <4BB0D41C.2000203@oracle.com> <20100329163306.GP21244@Sun.COM> <4BB0E0AA.2050703@oracle.com> <20100329172445.GS21244@Sun.COM> <4BB0E744.60001@oracle.com> <4BB18DBA.80801@oracle.com> <4bb46331.w3QC5evXgGPsqTNj%Joerg.Schilling@fokus.fraunhofer.de> <4BB476F8.5060508@Oracle.COM> <4BB5048B.5080904@oracle.com> <20100401205348.GJ4225@Sun.COM> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.7) Gecko/20100131 Lightning/1.0b1 Thunderbird/3.0.1 Status: RO Content-Length: 428 On 04/ 1/10 01:53 PM, Nicolas Williams wrote: > On Thu, Apr 01, 2010 at 02:39:39PM -0600, Tim Haley wrote: > >> After a couple of days pondering a lot of the mail that's been flying >> about on this topic, I'm now re-submitting the specification with some >> minor changes. I've reset the time-out to a week from today. >> > Looks great to me, > > Nico > Agreed. +1 on the new proposal for me. -- Garrett From tim.haley@oracle.com Wed Apr 7 10:18:53 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o37HIrrb016482 for ; Wed, 7 Apr 2010 10:18:53 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o37HImAN016044; Wed, 7 Apr 2010 11:18:52 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L0I00619O3G1E00@nwk-avmta-1.sfbay.Sun.COM>; Wed, 07 Apr 2010 10:18:52 -0700 (PDT) Received: from sca-ea-mail-1.sun.com ([192.18.43.24]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L0I00L8TO3E4U90@nwk-avmta-1.sfbay.Sun.COM>; Wed, 07 Apr 2010 10:18:50 -0700 (PDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by sca-ea-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o37HIonq017707; Wed, 07 Apr 2010 17:18:50 +0000 (GMT) Received: from acsmt353.oracle.com (acsmt353.oracle.com [141.146.40.153]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o375UFYE019847; Wed, 07 Apr 2010 17:18:49 +0000 (GMT) Received: from abhmt016.oracle.com by acsmt355.oracle.com with ESMTP id 143563261270660701; Wed, 07 Apr 2010 10:18:21 -0700 Received: from [172.20.25.27] (/10.85.25.27) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 07 Apr 2010 10:18:21 -0700 Date: Wed, 07 Apr 2010 11:18:20 -0600 From: Tim Haley Subject: Re: zfs diff [PSARC/2010/105 FastTrack timeout 04/08/2010] In-reply-to: <4BB12936.8010702@oracle.com> Cc: PSARC-ext@sun.com, ZFS Team Message-id: <4BBCBE5C.8040301@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt353.oracle.com [141.146.40.153] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090201.4BBCBE7A.001C:SCFMA4539814,ss=1,fgs=0 References: <4BAFAA10.2070706@oracle.com> <4BB04C10.5050300@Sun.COM> <4BB12936.8010702@oracle.com> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.8) Gecko/20100315 Thunderbird/3.0.3 Status: RO Content-Length: 55 This case was approved in today's PSARC meeting. -tim