From ahrens@sac.sfbay.sun.com Mon Mar 30 20:39:08 2009 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n2V3d7qJ014119 for ; Mon, 30 Mar 2009 20:39:07 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n2V3cvvq020288; Tue, 31 Mar 2009 04:39:06 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KHC0060RQ559000@nwk-avmta-2.sfbay.sun.com>; Mon, 30 Mar 2009 20:39:05 -0700 (PDT) Received: from localhost.sfbay.sun.com ([129.146.17.46]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KHC00F8ZQ55M3C0@nwk-avmta-2.sfbay.sun.com>; Mon, 30 Mar 2009 20:39:05 -0700 (PDT) Received: from localhost.sfbay.sun.com (localhost [127.0.0.1] (may be forged)) by localhost.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n2V3d5BA019799; Mon, 30 Mar 2009 20:39:05 -0700 (PDT) Received: (from ahrens@localhost) by localhost.sfbay.sun.com (8.14.3+Sun/8.14.3/Submit) id n2V3d5Dl019795; Mon, 30 Mar 2009 20:39:05 -0700 (PDT) Date: Mon, 30 Mar 2009 20:39:05 -0700 (PDT) From: Matthew Ahrens Subject: ZFS user/group quotas & space accounting [PSARC/2009/204 FastTrack timeout 04/08/2009] To: PSARC-ext@sun.com Cc: lustre-dmu-ext@sun.com, zfs-team@sun.com Message-id: <200903310339.n2V3d5Dl019795@localhost.sfbay.sun.com> Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 Status: RO Content-Length: 7486 Template Version: @(#)sac_nextcase 1.68 02/23/09 SMI This information is Copyright 2009 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: ZFS user/group quotas & space accounting 1.2. Name of Document Author/Supplier: Author: Matthew Ahrens 1.3 Date of This Document: 30 March, 2009 4. Technical Description ZFS user/group space accounting A. SUMMARY This case adds support to ZFS for user/group quotas & per-uid/gid space tracking. B. PROBLEM Enterprise customers often want to know who is using space, based on what uid and gid owns each file. Education customers often want to apply per-user quotas to hundreds of thousands of users. In these situations, the number of users and/or existing infrastructure prohibits using one filesystem per user and setting filesystem-wide quotas. C. PROPOSED SOLUTION 1. Overview Each filesystem keeps track of how much space inside it is owned by each user (uid) and group (gid). This is the amount of space "referenced", so relationships between filesystems, descendents, clones, and snapshots are ignored, and each tracks their "user used" and "group used" independently. This is the same policy behind the "referenced", "refquota", and "refreservation" properties. The amount of space charged is the amount of space reported by struct stat's st_blocks and du(1). Both POSIX ids (uid & gid) and untranslated SIDs are supported (eg, when sharing filesystems over SMB without a name service translation set up). ZFS will now enforce quotas on the amount of space referenced by files owned by particular users and groups. Enforcement may be delayed by several seconds. In other words, users may go a bit over their quota before the system notices that they are over quota and begins to refuse additional writes with EDQUOT. This decision was made to get the feature to market in a reasonable time, with a minimum of engineering resources expended. The design and implementation do not preclude implementing strict enforcement at a later date. User space accounting and quotas "stick with" each dataset (snapshot, filesystem, and clone). This means that user quotas (and space accounting) are not inherited. They will be "copied" to a new snapshot, and keep the values they had at the time the snapshot was taken. Likewise, user quotas will be "copied" to a clone (from its origin snapshot), and they will be copied with "zfs send" (even without -R). (User accounting and quota information is not actually copied to snapshots and clones, just referenced and copied-on-write like other filesystem contents.) The user space accounting and quotas is reported by the new userused@, groupused@, userquota@, and groupquota@ properties, and by the new "zfs userspace" and "zfs groupspace" subcommands, which are detailed below. 2. Version Compatibility To use these features, the pool must be upgraded to a new on-disk version (15). Old filesystems must have their space accounting information initialized by running "zfs userspace " or upgrading the old filesystem to a new on-disk version (4). To set user quotas, the pool and filesystem must both be upgraded. 3. Permissions Setting or changing user quotas are administrative actions, subject to the same privilege requirements as other zfs subcommands. There are new "userquota" and "groupquota" permissions which can be granted with "zfs allow", to allow those properties to be viewed and changed. Unprivileged users can only view their own userquota and userused, and the groupquota and groupused of any groups they belong to. The new "userused" and "groupused" permissions can be granted with "zfs allow" to permit users to view these properties. The existing "version" permission (granted with "zfs allow") permits the accounting information to be initialized by "zfs userspace". 4. New Properties user/group space accounting information and quotas can be manipulated with 4 new properties: zfs get userused@ zfs get groupused@ zfs get userquota@ zfs get groupquota@ zfs set userquota@= zfs set groupquota@= The or is specified using one of the following forms: posix name (eg. ahrens) posix numeric id (eg. 126829) sid name (eg. ahrens@sun) sid numeric id (eg. S-1-12345-12423-125829) For "zfs set", if a nonexistent name is specified, an error is generated. Any numeric ID is permitted. For "zfs get", if a nonexistent name is specified, "-" is printed for the value, indicating that there is no value available (like "zfs get nonexistent:userproperty"). As with filesystem quotas ("quota" and "refquota" properties), user quotas can be set to a value larger than the available space. User quotas can also be set to a value less than the amount of space used by that user, effectively forcing that user to reduce their space utilization. These new properties are not printed by "zfs get all", since that could generate a huge amount of output, which would not be very well organized. The new "zfs userspace" subcommand should be used instead. 5. New Subcommands Two new subcommands are added: "zfs userspace" and "zfs groupspace": zfs {user|group}space [-hniHp] [-o field[,...]] [-sS field] ... [-t type [,...]] Typical output is like this: TYPE NAME USED QUOTA POSIX User ahrens 14M 1G POSIX User george 56.5M none POSIX User lling 258M 500M SMB User marks@sun 103M none Option flags: -h Show help message and exit. -n Print numeric ID instead of user/group name (like "ls -n") -i Translate SID to POSIX ID. The POSIX ID may be ephemeral if no mapping exists. Normal POSIX interfaces (eg, stat(2), "ls -l") perform this translation, so the -i option allows the output from "zfs userspace" to be compared directly with those utilities. However, -i may lead to confusion if some files were created by a SMB user before a SMB -> POSIX name mapping was established. In that case some files are owned by the SMB entity and some by the POSIX entity. The -i flag will report that the POSIX entity has the total usage and quota for both entities. -H Do not print headers, use tab-delimited output (like "zfs list/get -H") -p Use exact (parsable) numeric output (like "zfs get -p") -o field[,...] Print only the specified fields (like "zfs list/get -o"), from the following set: type,name,used,quota. The default is to print all fields. -s field Sort output by this field (like "zfs list -s"). The -s (and -S) flag may be specified multiple times to sort first by one field, then by another. The default is "-s type -s name". -S field Sort by this field in reverse order, see -s. -t type[,...] Print only the specified types (like "zfs list -t"), from the following set: all,posixuser,smbuser,posixgroup,smbgroup. The default for "zfs userspace" is "-t posixuser,smbuser". The default for "zfs groupspace" is "-t posixgroup,smbgroup". This is the only difference between the two subcommands, and in fact "zfs userspace -t posixgroup" is perfectly valid. 6. Stability This case requests patch/micro release binding. The new interfaces are committed. 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: FastTrack 6.6. ARC Exposure: open From Darren.Moffat@sun.com Tue Mar 31 03:22:17 2009 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n2VAMH6H002768 for ; Tue, 31 Mar 2009 03:22:17 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n2VAMGLV022274; Tue, 31 Mar 2009 03:22:17 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KHD00G058T4RM00@nwk-avmta-1.sfbay.Sun.COM>; Tue, 31 Mar 2009 03:22:16 -0700 (PDT) Received: from gmp-eb-inf-1.sun.com ([192.18.6.21]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KHD00LZY8T3JOB0@nwk-avmta-1.sfbay.Sun.COM>; Tue, 31 Mar 2009 03:22:16 -0700 (PDT) Received: from fe-emea-09.sun.com (gmp-eb-lb-2-fe2.eu.sun.com [192.18.6.11]) by gmp-eb-inf-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n2VAMFW6028471; Tue, 31 Mar 2009 10:22:15 +0000 (GMT) Received: from conversion-daemon.fe-emea-09.sun.com by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7.0-5.01 64bit (built Feb 19 2009)) id <0KHD008007NW1100@fe-emea-09.sun.com>; Tue, 31 Mar 2009 11:22:15 +0100 (BST) Received: from [192.168.1.104] (cpc2-rdng20-2-0-cust917.15-3.cable.virginmedia.com [86.28.167.150]) by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7.0-5.01 64bit (built Feb 19 2009)) with ESMTPSA id <0KHD00JTG8SSC600@fe-emea-09.sun.com>; Tue, 31 Mar 2009 11:22:04 +0100 (BST) Date: Tue, 31 Mar 2009 11:22:04 +0100 From: Darren J Moffat Subject: Re: ZFS user/group quotas & space accounting [PSARC/2009/204 FastTrack timeout 04/08/2009] In-reply-to: <200903310339.n2V3d5Dl019795@localhost.sfbay.sun.com> Sender: Darren.Moffat@sun.com To: PSARC-ext@sun.com Cc: Matthew Ahrens , lustre-dmu-ext@sun.com, zfs-team@sun.com Message-id: <49D1EECC.60404@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200903310339.n2V3d5Dl019795@localhost.sfbay.sun.com> User-Agent: Thunderbird 2.0.0.18 (X11/20090211) Status: RO Content-Length: 73 I'm happy with this case as defined so it has my +1. -- Darren J Moffat From edward.pilatowicz@sun.com Tue Mar 31 11:32:09 2009 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n2VIW8af002555 for ; Tue, 31 Mar 2009 11:32:09 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n2VIW4Jl028684; Tue, 31 Mar 2009 12:32:07 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KHD0050VVHHYW00@nwk-avmta-1.sfbay.Sun.COM>; Tue, 31 Mar 2009 11:32:05 -0700 (PDT) Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.59]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KHD00KR5VHGUG80@nwk-avmta-1.sfbay.Sun.COM>; Tue, 31 Mar 2009 11:32:04 -0700 (PDT) Received: from jurassic-x4600.sfbay.sun.com (localhost [127.0.0.1]) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n2VIW4Et931867 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 31 Mar 2009 11:32:04 -0700 (PDT) Received: (from edp@localhost) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3/Submit) id n2VIW4jL931866; Tue, 31 Mar 2009 11:32:04 -0700 (PDT) Date: Tue, 31 Mar 2009 11:32:03 -0700 From: Edward Pilatowicz Subject: Re: ZFS user/group quotas & space accounting [PSARC/2009/204 FastTrack timeout 04/08/2009] In-reply-to: <200903310339.n2V3d5Dl019795@localhost.sfbay.sun.com> To: Matthew Ahrens Cc: PSARC-ext@sun.com, lustre-dmu-ext@sun.com, zfs-team@sun.com Message-id: <20090331183203.GA905476@eng.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <200903310339.n2V3d5Dl019795@localhost.sfbay.sun.com> X-Authentication-warning: jurassic-x4600.sfbay.sun.com: edp set sender to edward.pilatowicz@sun.com using -f User-Agent: Mutt/1.5.19 (2009-01-05) Status: RO Content-Length: 8013 perhaps i'm just dense and missed this, but how do you remove or clear a quota? zfs inherit {user|group}quota@ ? ed On Mon, Mar 30, 2009 at 08:39:05PM -0700, Matthew Ahrens wrote: > > Template Version: @(#)sac_nextcase 1.68 02/23/09 SMI > This information is Copyright 2009 Sun Microsystems > 1. Introduction > 1.1. Project/Component Working Name: > ZFS user/group quotas & space accounting > 1.2. Name of Document Author/Supplier: > Author: Matthew Ahrens > 1.3 Date of This Document: > 30 March, 2009 > 4. Technical Description > ZFS user/group space accounting > > A. SUMMARY > > This case adds support to ZFS for user/group quotas & per-uid/gid space > tracking. > > B. PROBLEM > > Enterprise customers often want to know who is using space, based on > what uid and gid owns each file. > > Education customers often want to apply per-user quotas to hundreds of > thousands of users. In these situations, the number of users and/or > existing infrastructure prohibits using one filesystem per user and > setting filesystem-wide quotas. > > C. PROPOSED SOLUTION > > 1. Overview > > Each filesystem keeps track of how much space inside it is owned by each > user (uid) and group (gid). This is the amount of space "referenced", > so relationships between filesystems, descendents, clones, and snapshots > are ignored, and each tracks their "user used" and "group used" > independently. This is the same policy behind the "referenced", > "refquota", and "refreservation" properties. The amount of space > charged is the amount of space reported by struct stat's st_blocks and > du(1). > > Both POSIX ids (uid & gid) and untranslated SIDs are supported (eg, when > sharing filesystems over SMB without a name service translation set up). > > ZFS will now enforce quotas on the amount of space referenced by files > owned by particular users and groups. Enforcement may be delayed by > several seconds. In other words, users may go a bit over their quota > before the system notices that they are over quota and begins to refuse > additional writes with EDQUOT. This decision was made to get the > feature to market in a reasonable time, with a minimum of engineering > resources expended. The design and implementation do not preclude > implementing strict enforcement at a later date. > > User space accounting and quotas "stick with" each dataset (snapshot, > filesystem, and clone). This means that user quotas (and space > accounting) are not inherited. They will be "copied" to a new snapshot, > and keep the values they had at the time the snapshot was taken. > Likewise, user quotas will be "copied" to a clone (from its origin > snapshot), and they will be copied with "zfs send" (even without -R). > (User accounting and quota information is not actually copied to > snapshots and clones, just referenced and copied-on-write like other > filesystem contents.) > > The user space accounting and quotas is reported by the new > userused@, groupused@, userquota@, and > groupquota@ properties, and by the new "zfs userspace" and "zfs > groupspace" subcommands, which are detailed below. > > 2. Version Compatibility > > To use these features, the pool must be upgraded to a new on-disk > version (15). Old filesystems must have their space accounting > information initialized by running "zfs userspace " or upgrading the > old filesystem to a new on-disk version (4). To set user quotas, the > pool and filesystem must both be upgraded. > > 3. Permissions > > Setting or changing user quotas are administrative actions, subject to > the same privilege requirements as other zfs subcommands. There are new > "userquota" and "groupquota" permissions which can be granted with "zfs > allow", to allow those properties to be viewed and changed. > > Unprivileged users can only view their own userquota and userused, and > the groupquota and groupused of any groups they belong to. The new > "userused" and "groupused" permissions can be granted with "zfs allow" > to permit users to view these properties. > > The existing "version" permission (granted with "zfs allow") permits the > accounting information to be initialized by "zfs userspace". > > 4. New Properties > > user/group space accounting information and quotas can be manipulated > with 4 new properties: > > zfs get userused@ > zfs get groupused@ > > zfs get userquota@ > zfs get groupquota@ > > zfs set userquota@= > zfs set groupquota@= > > The or is specified using one of the following forms: > posix name (eg. ahrens) > posix numeric id (eg. 126829) > sid name (eg. ahrens@sun) > sid numeric id (eg. S-1-12345-12423-125829) > > For "zfs set", if a nonexistent name is specified, an error is > generated. Any numeric ID is permitted. For "zfs get", if a > nonexistent name is specified, "-" is printed for the value, indicating > that there is no value available (like "zfs get nonexistent:userproperty"). > > As with filesystem quotas ("quota" and "refquota" properties), user > quotas can be set to a value larger than the available space. > > User quotas can also be set to a value less than the amount of space > used by that user, effectively forcing that user to reduce their space > utilization. > > These new properties are not printed by "zfs get all", since that could > generate a huge amount of output, which would not be very well > organized. The new "zfs userspace" subcommand should be used instead. > > 5. New Subcommands > > Two new subcommands are added: "zfs userspace" and "zfs groupspace": > > zfs {user|group}space [-hniHp] [-o field[,...]] [-sS field] ... > [-t type [,...]] > > Typical output is like this: > TYPE NAME USED QUOTA > POSIX User ahrens 14M 1G > POSIX User george 56.5M none > POSIX User lling 258M 500M > SMB User marks@sun 103M none > > Option flags: > > -h Show help message and exit. > > -n Print numeric ID instead of user/group name (like "ls -n") > > -i Translate SID to POSIX ID. The POSIX ID may be ephemeral if no > mapping exists. Normal POSIX interfaces (eg, stat(2), "ls -l") > perform this translation, so the -i option allows the output > from "zfs userspace" to be compared directly with those > utilities. However, -i may lead to confusion if some files were > created by a SMB user before a SMB -> POSIX name mapping was > established. In that case some files are owned by the SMB > entity and some by the POSIX entity. The -i flag will report > that the POSIX entity has the total usage and quota for both > entities. > > -H Do not print headers, use tab-delimited output (like "zfs list/get -H") > > -p Use exact (parsable) numeric output (like "zfs get -p") > > -o field[,...] > Print only the specified fields (like "zfs list/get -o"), from > the following set: type,name,used,quota. The default is to > print all fields. > > -s field > Sort output by this field (like "zfs list -s"). The -s (and -S) > flag may be specified multiple times to sort first by one field, > then by another. The default is "-s type -s name". > > -S field > Sort by this field in reverse order, see -s. > > -t type[,...] > Print only the specified types (like "zfs list -t"), from the > following set: all,posixuser,smbuser,posixgroup,smbgroup. The > default for "zfs userspace" is "-t posixuser,smbuser". The > default for "zfs groupspace" is "-t posixgroup,smbgroup". This > is the only difference between the two subcommands, and in fact > "zfs userspace -t posixgroup" is perfectly valid. > > 6. Stability > > This case requests patch/micro release binding. The new interfaces are > committed. > > > 6. Resources and Schedule > 6.4. Steering Committee requested information > 6.4.1. Consolidation C-team Name: > ON > 6.5. ARC review type: FastTrack > 6.6. ARC Exposure: open From Matthew.Ahrens@sun.com Tue Mar 31 11:36:57 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n2VIauZa003037 for ; Tue, 31 Mar 2009 11:36:57 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n2VIaphO022463; Wed, 1 Apr 2009 02:36:54 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KHD0080NVPHPT00@brm-avmta-1.central.sun.com>; Tue, 31 Mar 2009 12:36:53 -0600 (MDT) Received: from zion.sfbay.sun.com ([129.146.17.75]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KHD00JJ5VPGE5B0@brm-avmta-1.central.sun.com>; Tue, 31 Mar 2009 12:36:52 -0600 (MDT) Received: from stationary.local (punchin-ahrens.SFBay.Sun.COM [10.7.251.178]) by zion.sfbay.sun.com (8.14.2+Sun/8.14.2) with ESMTP id n2VIapfv007362; Tue, 31 Mar 2009 18:36:51 +0000 (GMT) Date: Tue, 31 Mar 2009 11:36:50 -0700 From: Matthew Ahrens Subject: Re: ZFS user/group quotas & space accounting [PSARC/2009/204 FastTrack timeout 04/08/2009] In-reply-to: <20090331183203.GA905476@eng.sun.com> To: Edward Pilatowicz Cc: Matthew Ahrens , PSARC-ext@sun.com, lustre-dmu-ext@sun.com, zfs-team@sun.com Message-id: <49D262C2.4060102@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200903310339.n2V3d5Dl019795@localhost.sfbay.sun.com> <20090331183203.GA905476@eng.sun.com> User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) Status: RO Content-Length: 315 Edward Pilatowicz wrote: > perhaps i'm just dense and missed this, but how do you remove or clear a > quota? zfs inherit {user|group}quota@ ? Same as with regular quotas and refquotas: zfs set {user|group}quota@=none The zfs(1m) manpage and the output from "zfs set" document this. --matt From Nicolas.Williams@sun.com Tue Mar 31 12:15:02 2009 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n2VJF1Kf000356 for ; Tue, 31 Mar 2009 12:15:02 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n2VJElSb018708; Tue, 31 Mar 2009 20:15:00 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KHD00B1DXGZ1500@nwk-avmta-1.sfbay.Sun.COM>; Tue, 31 Mar 2009 12:14:59 -0700 (PDT) Received: from binky.Central.Sun.COM ([129.153.128.104]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KHD00KYVXGYUDD0@nwk-avmta-1.sfbay.Sun.COM>; Tue, 31 Mar 2009 12:14:58 -0700 (PDT) Received: from binky.Central.Sun.COM (localhost [127.0.0.1]) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3) with ESMTP id n2VJD7nv014572; Tue, 31 Mar 2009 14:13:07 -0500 (CDT) Received: (from nw141292@localhost) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3/Submit) id n2VJD7Dk014571; Tue, 31 Mar 2009 14:13:07 -0500 (CDT) Date: Tue, 31 Mar 2009 14:13:07 -0500 From: Nicolas Williams Subject: Re: ZFS user/group quotas & space accounting [PSARC/2009/204 FastTrack timeout 04/08/2009] In-reply-to: <200903310339.n2V3d5Dl019795@localhost.sfbay.sun.com> To: Matthew Ahrens Cc: PSARC-ext@sun.com, lustre-dmu-ext@sun.com, zfs-team@sun.com Message-id: <20090331191306.GI9992@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <200903310339.n2V3d5Dl019795@localhost.sfbay.sun.com> X-Authentication-warning: binky.Central.Sun.COM: nw141292 set sender to Nicolas.Williams@sun.com using -f User-Agent: Mutt/1.5.7i Status: RO Content-Length: 1478 On Mon, Mar 30, 2009 at 08:39:05PM -0700, Matthew Ahrens wrote: > The or is specified using one of the following forms: > posix name (eg. ahrens) > posix numeric id (eg. 126829) > sid name (eg. ahrens@sun) > sid numeric id (eg. S-1-12345-12423-125829) It's worth noting that the third user/group form ("sid name") could be resolved either via getpwnam(3C)/getgrnam(3C) or by using ID mapping APIs. The former requires that nss_ad(5) be used as a backend for passwd and group in nsswitch.conf(4), which makes it somewhat not friendly. In spite of this ISTM that using getpwnam(3C)/getgrnam(3C) is arguably more correct. Yes, any resulting ephemeral UIDs/GIDs have to then be mapped back to SIDs via ID mapping APIs, but presumably this happens in the "posix numeric id" ID case too if the given ID is an ephemeral ID (or is that not allowed? and if so, why?). The issue: how should system components resolve name@domain user/group name forms? Via getXbyname()? Or via Solaris ID mapping APIs? I'm not sure this matters much for now. I doubt there's much use of '@' in user/group names outside the context of nss_ad/ID mapping. And I believe any future cross-Unix domain ID mapping ought to fit into the existing Solaris ID mapping framework, so that using existing Solaris ID mapping APIs should be future-proof. But just in case, and if nothing else for the benefit of future ARCheologists, I'm only sending this comment to the case record. Nico -- From Matthew.Ahrens@sun.com Tue Mar 31 14:00:54 2009 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n2VL0s2T006076 for ; Tue, 31 Mar 2009 14:00:54 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n2VL0r8K011541; Tue, 31 Mar 2009 14:00:53 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KHE0001N2DGZK00@nwk-avmta-1.sfbay.Sun.COM>; Tue, 31 Mar 2009 14:00:52 -0700 (PDT) Received: from zion.sfbay.sun.com ([129.146.17.75]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KHE00FZD2DEWT40@nwk-avmta-1.sfbay.Sun.COM>; Tue, 31 Mar 2009 14:00:50 -0700 (PDT) Received: from stationary.local (punchin-ahrens.SFBay.Sun.COM [10.7.251.178]) by zion.sfbay.sun.com (8.14.2+Sun/8.14.2) with ESMTP id n2VL0ncF011285; Tue, 31 Mar 2009 21:00:49 +0000 (GMT) Date: Tue, 31 Mar 2009 14:00:48 -0700 From: Matthew Ahrens Subject: Re: ZFS user/group quotas & space accounting [PSARC/2009/204 FastTrack timeout 04/08/2009] In-reply-to: <20090331191306.GI9992@Sun.COM> To: Nicolas Williams Cc: Matthew Ahrens , PSARC-ext@sun.com, lustre-dmu-ext@sun.com, zfs-team@sun.com Message-id: <49D28480.5020105@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200903310339.n2V3d5Dl019795@localhost.sfbay.sun.com> <20090331191306.GI9992@Sun.COM> User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) Status: RO Content-Length: 3385 Nicolas Williams wrote: > On Mon, Mar 30, 2009 at 08:39:05PM -0700, Matthew Ahrens wrote: >> The or is specified using one of the following forms: >> posix name (eg. ahrens) >> posix numeric id (eg. 126829) >> sid name (eg. ahrens@sun) >> sid numeric id (eg. S-1-12345-12423-125829) > > It's worth noting that the third user/group form ("sid name") could be > resolved either via getpwnam(3C)/getgrnam(3C) or by using ID mapping > APIs. The former requires that nss_ad(5) be used as a backend for > passwd and group in nsswitch.conf(4), which makes it somewhat not > friendly. > > In spite of this ISTM that using getpwnam(3C)/getgrnam(3C) is arguably > more correct. Yes, any resulting ephemeral UIDs/GIDs have to then be > mapped back to SIDs via ID mapping APIs, But if the system has not been configured in that way (using nss_ad), then this getpwnam() would simply fail, which doesn't seem very useful. Unless I'm misunderstanding. How would getpwnam() return an ephemeral UID? Does it fake up a struct passwd entry? Does it do that if you don't use nss_ad? The goal here is to implement a symmetric user interface: "zfs userspace" tells you the actual sid name (or domain+rid with -n) that is being charged (computed from the ZFS FUID). The user should be able to set quotas on that value with "zfs set userquota@=". I suppose you could argue that if they didn't set up nss_ad, then we should not be able to map the domain+rid to a sid name, so they would only see domain+rid, and be able to set quotas using that same domain+rid. But it doesn't seem very helpful to make the user run idmap(1m) to do the mapping themself. > but presumably this happens in > the "posix numeric id" ID case too if the given ID is an ephemeral ID > (or is that not allowed? and if so, why?). Yes, this is done by idmap_get_sidby[ug]id(). > The issue: how should system components resolve name@domain user/group > name forms? Via getXbyname()? Or via Solaris ID mapping APIs? Indeed. It would be great to have one library function to resolve -> S-1-numeric-domain + rid. (With a NULL domain in the case that it's a POSIX id.) In the absence of that, the current algorithm is: if there's an '@' in the name: sid_to_id() idmap_get_sidby[ug]id() else: getpwnam()/getgrnam() Note that in the '@' case, we are translating from user@domain -> (possibly ephemeral) UID -> S-1-numeric-domain + rid. This is pretty roundabout, so we have filed RFE 6816841 to provide a more direct interface to map user@domain -> S-1-numeric-domain + rid. Similarly, when printing SIDs from "zfs userspace, we translate from S-1-numeric-domain + rid to user@domain by doing sid_to_id() and then idmap_getwinnameby[ug]id(). The new interfaces provided by 6816841 will simplify this. > I'm not sure this matters much for now. I doubt there's much use of '@' > in user/group names outside the context of nss_ad/ID mapping. And I > believe any future cross-Unix domain ID mapping ought to fit into the > existing Solaris ID mapping framework, so that using existing Solaris ID > mapping APIs should be future-proof. Indeed, it would be great if that functionality was available. If you have suggestions on how I could improve our algorithm using currently-available interfaces, let me know. --matt From Nicolas.Williams@sun.com Tue Mar 31 14:39:00 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n2VLcx5A009296 for ; Tue, 31 Mar 2009 14:39:00 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n2VLcoBQ002181; Wed, 1 Apr 2009 05:38:57 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KHE0051944VQV00@nwk-avmta-1.sfbay.Sun.COM>; Tue, 31 Mar 2009 14:38:55 -0700 (PDT) Received: from binky.Central.Sun.COM ([129.153.128.104]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KHE00FGS44PWM70@nwk-avmta-1.sfbay.Sun.COM>; Tue, 31 Mar 2009 14:38:49 -0700 (PDT) Received: from binky.Central.Sun.COM (localhost [127.0.0.1]) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3) with ESMTP id n2VLb2Nn014920; Tue, 31 Mar 2009 16:37:02 -0500 (CDT) Received: (from nw141292@localhost) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3/Submit) id n2VLb2Kv014919; Tue, 31 Mar 2009 16:37:02 -0500 (CDT) Date: Tue, 31 Mar 2009 16:37:02 -0500 From: Nicolas Williams Subject: Re: ZFS user/group quotas & space accounting [PSARC/2009/204 FastTrack timeout 04/08/2009] In-reply-to: <49D28480.5020105@sun.com> To: Matthew Ahrens Cc: Matthew Ahrens , PSARC-ext@sun.com, lustre-dmu-ext@sun.com, zfs-team@sun.com Message-id: <20090331213702.GS9992@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <200903310339.n2V3d5Dl019795@localhost.sfbay.sun.com> <20090331191306.GI9992@Sun.COM> <49D28480.5020105@sun.com> X-Authentication-warning: binky.Central.Sun.COM: nw141292 set sender to Nicolas.Williams@sun.com using -f User-Agent: Mutt/1.5.7i Status: RO Content-Length: 3498 On Tue, Mar 31, 2009 at 02:00:48PM -0700, Matthew Ahrens wrote: > Nicolas Williams wrote: > >It's worth noting that the third user/group form ("sid name") could be > >resolved either via getpwnam(3C)/getgrnam(3C) or by using ID mapping > >APIs. The former requires that nss_ad(5) be used as a backend for > >passwd and group in nsswitch.conf(4), which makes it somewhat not > >friendly. > > > >In spite of this ISTM that using getpwnam(3C)/getgrnam(3C) is arguably > >more correct. Yes, any resulting ephemeral UIDs/GIDs have to then be > >mapped back to SIDs via ID mapping APIs, > > But if the system has not been configured in that way (using nss_ad), then > this getpwnam() would simply fail, which doesn't seem very useful. Unless > I'm misunderstanding. How would getpwnam() return an ephemeral UID? Does > it fake up a struct passwd entry? Does it do that if you don't use nss_ad? I said so myself: using getpwnam()/getgrnam() here means relying on nsswitch.conf being configured to use nss_ad. That's not user friendly... It's arguably more correct from one p.o.v., and arguably less correct from another. > >The issue: how should system components resolve name@domain user/group > >name forms? Via getXbyname()? Or via Solaris ID mapping APIs? > > Indeed. It would be great to have one library function to resolve user identifier> -> S-1-numeric-domain + rid. (With a NULL domain in the > case that it's a POSIX id.) In the absence of that, the current algorithm > is: > > if there's an '@' in the name: > sid_to_id() > idmap_get_sidby[ug]id() > else: > getpwnam()/getgrnam() This algorithm seems unfortunate. I'd rather that nss_ad always be configured in nsswitch.conf and only be active if AD is in use. Actually, nss_ad should probably be generalized to be less AD-specific. Think of Administrator@localhost and Guest User@localhost in workgroup mode, as well as of cross-Unix domain ID mapping. Because I doubt nss_ad will be generalized, or that it will always be configured in nsswitch.conf, I think it's too soon to actually establish a rule that system software should prefer getXbyYs over ID mapping APIs. But it does seem like a good _future_ rule. I commented mostly because I wanted future ARCheologists to find something in the case record about this. > Note that in the '@' case, we are translating from user@domain -> (possibly > ephemeral) UID -> S-1-numeric-domain + rid. This is pretty roundabout, so > we have filed RFE 6816841 to provide a more direct interface to map > user@domain -> S-1-numeric-domain + rid. Yup. > Similarly, when printing SIDs from "zfs userspace, we translate from > S-1-numeric-domain + rid to user@domain by doing sid_to_id() and then > idmap_getwinnameby[ug]id(). The new interfaces provided by 6816841 will > simplify this. Yup. > >I'm not sure this matters much for now. I doubt there's much use of '@' > >in user/group names outside the context of nss_ad/ID mapping. And I > >believe any future cross-Unix domain ID mapping ought to fit into the > >existing Solaris ID mapping framework, so that using existing Solaris ID > >mapping APIs should be future-proof. > > Indeed, it would be great if that functionality was available. If you have > suggestions on how I could improve our algorithm using currently-available > interfaces, let me know. Right now I don't. Thus I think this sub-thread should end here :) at least as far as the ARC case record goes. Thanks! Nico -- From mgerdts@gmail.com Tue Mar 31 19:08:04 2009 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n31283wo027606 for ; Tue, 31 Mar 2009 19:08:04 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n31283Eb019431 for <@sunmail2sca.sfbay.sun.com:PSARC-EXT@sun.com>; Tue, 31 Mar 2009 20:08:03 -0600 (MDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KHE0070PGLEXD00@brm-avmta-1.central.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Tue, 31 Mar 2009 20:08:02 -0600 (MDT) Received: from sca-ea-mail-2.sun.com ([192.18.43.25]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KHE0074TGLB6X20@brm-avmta-1.central.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Tue, 31 Mar 2009 20:07:59 -0600 (MDT) Received: from relay15i.sun.com (ip125.net129179-4.block1.us.syntegra.com [129.179.4.125]) by sca-ea-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n3122k73002949 for ; Wed, 01 Apr 2009 02:07:59 +0000 (GMT) Received: from mmp14es.mmp.us.syntegra.com ([160.41.208.14] [160.41.208.14]) by relay15i.sun.com with ESMTP id BT-MMP-3570129 for PSARC-EXT@sun.com; Wed, 01 Apr 2009 02:07:59 +0000 (Z) Received: from relay12i.sun.com (relay12i.sun.com [129.179.4.122]) by mmp14es.mmp.us.syntegra.com with ESMTP id BT-MMP-229569 for PSARC-EXT@sun.com; Wed, 01 Apr 2009 02:07:59 +0000 (Z) Received: from mail-gx0-f159.google.com ([209.85.217.159] [209.85.217.159]) by relay1i.sun.com with ESMTP id BT-MMP-17727803 for PSARC-EXT@sun.com; Wed, 01 Apr 2009 02:07:59 +0000 (Z) Received: by gxk3 with SMTP id 3so6156361gxk.8 for ; Tue, 31 Mar 2009 19:07:51 -0700 (PDT) Received: by 10.150.144.17 with SMTP id r17mr7449161ybd.154.1238551671394; Tue, 31 Mar 2009 19:07:51 -0700 (PDT) Date: Tue, 31 Mar 2009 21:07:51 -0500 From: Mike Gerdts Subject: Fwd: [zfs-discuss] [Fwd: ZFS user/group quotas & space accounting [PSARC/2009/204 FastTrack timeout 04/08/2009]] In-reply-to: <49D27C3F.3070507@sun.com> To: PSARC-EXT@sun.com Message-id: <65f8f3ad0903311907s72e1944fm26b0014b37766d40@mail.gmail.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=YxpfNYdsNKoc2pKBIjzzXQ4rn00k5H2b6aLiKUb1x9Y=; b=E9MBo8ITRaj/bWh3OO4Ew2cdnRxhOJfrenfAUeAhOGh5wY1R1FMR5I8nakpxExJRVd CJHNEBouBqI7cDcUTnhZFRqQWqFIHiuKFISHNcIZVs+Vhu9t8Lmww8YvSeNeAetPdTrj KnCLC68gYD480/DlBR0gQoEhXa3N3IzGAgvLQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=o5IPC8gblaO9tZk6UjYyneJcNhdaqLjrFzxM9+p3pCNE3h6Qkn4eBg0/Qmq3I7n9jG /CyTmvAMcyMUf/mCX1ii3F/IjiZOdUQhD8hgXGZjnv5+1OgSd6juScINwbZZIhonb5m2 NEKpZIlL0SsJdIlGJhp7Vl0WXvEDakBreozZg= X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-Antispam: No, score=0.0/5.0, scanned in 0.094sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <49D266FD.9080809@sun.com> <65f8f3ad0903311237hbcfdb1ds953661fdc231a8ba@mail.gmail.com> <20090331195553.GK9992@Sun.COM> <49D27C3F.3070507@sun.com> Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by sac.sfbay.sun.com id n31283wo027606 Status: RO Content-Length: 3505 Per the attached discussion on zfs-discuss, the behavior how uid and gid resolution is handled for datasets that are delegated to zones should be defined. Mike ---------- Forwarded message ---------- From: Matthew Ahrens Date: Tue, Mar 31, 2009 at 3:25 PM Subject: Re: [zfs-discuss] [Fwd: ZFS user/group quotas & space accounting [PSARC/2009/204 FastTrack timeout 04/08/2009]] To: Nicolas Williams Cc: Mike Gerdts , ZFS filesystem discussion list Nicolas Williams wrote: > > On Tue, Mar 31, 2009 at 02:37:02PM -0500, Mike Gerdts wrote: >>> >>> The or is specified using one of the following forms: >>> posix name (eg. ahrens) >>> posix numeric id (eg. 126829) >>> sid name (eg. ahrens@sun) >>> sid numeric id (eg. S-1-12345-12423-125829) >> >> How does this work with zones?  Suppose in the global zone I have >> passwd entries like: > > ZFS stores UIDs, GIDs and SIDs on disk.  The POSIX ID/SID <-> name > resolution happens in user-land.  As such the answer to your question is > the same as for any other operating system facility that deals with > POSIX ID/SID <-> name resolution: it depends on each zone's > configuration. Right, so you'll get the same answer as if you did a "ls" on those files (if the local zone's files were available to the global zone). > (In general kernel code never deals with user/group names directly, but > with UIDs/GIDs/SIDs.  One exception is the NFSv4 code, which upcalls to > user-land to resolve NFSv4 name@domain user/group names and vice versa.) > >> >> jill:x:123:123:Jill Admin:/home/jill:/bin/bash >> joe:x:124:124:Joe Admin:/home/joe:/bin/bash >> >> And in a non-global zone (called bedrock) I have: >> >> fred:x:123:123:Fred Flintstone:/home/fred:/bin/bash >> barney:x:124:124:Barney Rubble:/home/barney:/bin/bash >> >> Dataset rpool/quarry is delegated to the zone bedrock. >> >> Does "zfs get all rpool/quarry" report the same thing whether it is >> run in the global zone or the non-global zone? > > If you use the -n option, yes :) > > Oh, but then, the -n option is for the new zfs {user|group}space > sub-command.  I don't think "zfs get" is getting that option; maybe it > should. Since "zfs get all" doesn't report the {user|group}{used|quota}@... properties, you will definitely get the same answer ;-) These new properties are not printed by "zfs get all", since that could generate a huge amount of output, which would not be very well organized.  The new "zfs userspace" subcommand should be used instead. >> Has there been any thought to using a UID resolution mechanism similar >> to that used by ps?  That is, if "zfs get ... " is run in the >> global zone and the dataset is deleted to a non-global zone, display >> the UID rather than a possibly mistaken username. > > That seems like a good idea to me.  You should send that comment to the > ARC case record (send an e-mail to psarc-ext@sun.com with > "PSARC/2009/204" somewhere in the Subject: header). Indeed, that might be a good idea.  I wasn't aware of that convention. Though note that this only applies to "zfs userspace" -- we would simply say that if the fs is zoned, that implies the -n option.  We could also disallow them from doing "zfs get userused@name pool/zoned/fs", just make it an error to prevent them from seeing something other than what they intended. --matt -- Mike Gerdts http://mgerdts.blogspot.com/ From carlsonj@phorcys.east.sun.com Wed Apr 1 04:51:49 2009 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n31Bpnwf000966 for ; Wed, 1 Apr 2009 04:51:49 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n31BpiSG007536 for <@sunmail2sca.sfbay.sun.com:PSARC-EXT@sun.com>; Wed, 1 Apr 2009 12:51:48 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KHF001057M92U00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Wed, 01 Apr 2009 04:51:45 -0700 (PDT) Received: from phorcys.east.sun.com ([129.148.174.143]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KHF000K67M92E00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Wed, 01 Apr 2009 04:51:45 -0700 (PDT) Received: from phorcys.east.sun.com (phorcys.local [127.0.0.1]) by phorcys.east.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n31BhwtA016308; Wed, 01 Apr 2009 07:43:58 -0400 (EDT) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.3+Sun/8.14.3/Submit) id n31BhwAN016305; Wed, 01 Apr 2009 07:43:58 -0400 (EDT) Date: Wed, 01 Apr 2009 07:43:58 -0400 From: James Carlson Subject: Re: Fwd: [zfs-discuss] [Fwd: ZFS user/group quotas & space accounting [PSARC/2009/204 FastTrack timeout 04/08/2009]] In-reply-to: <65f8f3ad0903311907s72e1944fm26b0014b37766d40@mail.gmail.com> To: Mike Gerdts Cc: PSARC-EXT@sun.com Message-id: <18899.21374.209960.80845@gargle.gargle.HOWL> MIME-version: 1.0 X-Mailer: VM 7.01 under Emacs 21.3.1 Content-type: text/plain; charset=iso-8859-1 X-PMX-Version: 5.4.1.325704 References: <49D266FD.9080809@sun.com> <65f8f3ad0903311237hbcfdb1ds953661fdc231a8ba@mail.gmail.com> <20090331195553.GK9992@Sun.COM> <49D27C3F.3070507@sun.com> <65f8f3ad0903311907s72e1944fm26b0014b37766d40@mail.gmail.com> Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by sac.sfbay.sun.com id n31Bpnwf000966 Status: RO Content-Length: 1249 Mike Gerdts writes: > >> Has there been any thought to using a UID resolution mechanism similar > >> to that used by ps?  That is, if "zfs get ... " is run in the > >> global zone and the dataset is deleted to a non-global zone, display > >> the UID rather than a possibly mistaken username. > > > > That seems like a good idea to me.  You should send that comment to the > > ARC case record (send an e-mail to psarc-ext@sun.com with > > "PSARC/2009/204" somewhere in the Subject: header). > > Indeed, that might be a good idea.  I wasn't aware of that convention. > Though note that this only applies to "zfs userspace" -- we would > simply say that if the fs is zoned, that implies the -n option.  We > could also disallow them from doing "zfs get userused@name > pool/zoned/fs", just make it an error to prevent them from seeing > something other than what they intended. It looks like a small thing, but I suggest running that by the TX team. On a TX system, the user names to UID mappings are all the same. -- James Carlson, Solaris Networking Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 From Nicolas.Williams@Sun.COM Wed Apr 1 10:37:49 2009 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n31HbnKS021952 for ; Wed, 1 Apr 2009 10:37:49 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n31HbfcJ029979; Wed, 1 Apr 2009 18:37:47 +0100 (BST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KHF00703NMYPN00@brm-avmta-1.central.sun.com>; Wed, 01 Apr 2009 11:37:46 -0600 (MDT) Received: from binky.Central.Sun.COM ([129.153.128.104]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KHF00GIUNMXZAE0@brm-avmta-1.central.sun.com>; Wed, 01 Apr 2009 11:37:45 -0600 (MDT) Received: from binky.Central.Sun.COM (localhost [127.0.0.1]) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3) with ESMTP id n31HSSUc015352; Wed, 01 Apr 2009 12:28:28 -0500 (CDT) Received: (from nw141292@localhost) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3/Submit) id n31HSSel015351; Wed, 01 Apr 2009 12:28:28 -0500 (CDT) Date: Wed, 01 Apr 2009 12:28:28 -0500 From: Nicolas Williams Subject: Re: Fwd: [zfs-discuss] [Fwd: ZFS user/group quotas & space accounting [PSARC/2009/204 FastTrack timeout 04/08/2009]] In-reply-to: <18899.21374.209960.80845@gargle.gargle.HOWL> To: James Carlson Cc: Mike Gerdts , psarc-ext@Sun.COM Message-id: <20090401172827.GC9992@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=iso-8859-1 Content-transfer-encoding: 8BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <49D266FD.9080809@sun.com> <65f8f3ad0903311237hbcfdb1ds953661fdc231a8ba@mail.gmail.com> <20090331195553.GK9992@Sun.COM> <49D27C3F.3070507@sun.com> <65f8f3ad0903311907s72e1944fm26b0014b37766d40@mail.gmail.com> <18899.21374.209960.80845@gargle.gargle.HOWL> X-Authentication-warning: binky.Central.Sun.COM: nw141292 set sender to Nicolas.Williams@sun.com using -f User-Agent: Mutt/1.5.7i Status: RO Content-Length: 1747 On Wed, Apr 01, 2009 at 07:43:58AM -0400, James Carlson wrote: > Mike Gerdts writes: > > >> Has there been any thought to using a UID resolution mechanism similar > > >> to that used by ps?  That is, if "zfs get ... " is run in the > > >> global zone and the dataset is deleted to a non-global zone, display > > >> the UID rather than a possibly mistaken username. > > > > > > That seems like a good idea to me.  You should send that comment to the > > > ARC case record (send an e-mail to psarc-ext@sun.com with > > > "PSARC/2009/204" somewhere in the Subject: header). > > > > Indeed, that might be a good idea.  I wasn't aware of that convention. > > Though note that this only applies to "zfs userspace" -- we would > > simply say that if the fs is zoned, that implies the -n option.  We > > could also disallow them from doing "zfs get userused@name > > pool/zoned/fs", just make it an error to prevent them from seeing > > something other than what they intended. > > It looks like a small thing, but I suggest running that by the TX > team. On a TX system, the user names to UID mappings are all the > same. Yes, but this is an artifact of how TX provides name services for zones. I think what you mean is that since the namespaces are the same in all TX zones and the TX g-z zfs(1) should not eschew ID->name resolution when run in the TX g-z and applied to datasets delegated to other zones. But I don't think that's worthwhile: suppose TX someday adds support for non-TX exclusive IP zones[*], then we'd have to go back and fix zfs(1). [*] Running at a specific label from the TX p.o.v, and using no multilevel services from the g-z, except, perhaps, X11 displays shared via shm and Unix domain sockets. Nico -- From Darren.Reed@sun.com Wed Apr 1 10:56:35 2009 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n31HuZrN023565 for ; Wed, 1 Apr 2009 10:56:35 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n31HuXQf019569; Wed, 1 Apr 2009 11:56:34 -0600 (MDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KHF00919OI9G500@brm-avmta-1.central.sun.com>; Wed, 01 Apr 2009 11:56:33 -0600 (MDT) Received: from gmp-eb-inf-2.sun.com ([192.18.6.24]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KHF008PVOI80410@brm-avmta-1.central.sun.com>; Wed, 01 Apr 2009 11:56:33 -0600 (MDT) Received: from fe-emea-10.sun.com (gmp-eb-lb-2-fe2.eu.sun.com [192.18.6.11]) by gmp-eb-inf-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n31HuWBJ002919; Wed, 01 Apr 2009 17:56:32 +0000 (GMT) Received: from conversion-daemon.fe-emea-10.sun.com by fe-emea-10.sun.com (Sun Java(tm) System Messaging Server 7.0-5.01 64bit (built Feb 19 2009)) id <0KHF00D00OBQVC00@fe-emea-10.sun.com>; Wed, 01 Apr 2009 18:56:32 +0100 (BST) Received: from [192.168.0.201] ([unknown] [76.191.150.176]) by fe-emea-10.sun.com (Sun Java(tm) System Messaging Server 7.0-5.01 64bit (built Feb 19 2009)) with ESMTPSA id <0KHF00GNAOI4M580@fe-emea-10.sun.com>; Wed, 01 Apr 2009 18:56:32 +0100 (BST) Date: Wed, 01 Apr 2009 10:56:25 -0700 From: Darren Reed Subject: Re: ZFS user/group quotas & space accounting [PSARC/2009/204 FastTrack timeout 04/08/2009] In-reply-to: <200903310339.n2V3d5Dl019795@localhost.sfbay.sun.com> Sender: Darren.Reed@sun.com To: Matthew Ahrens Cc: psarc-ext@sun.com, lustre-dmu-ext@sun.com, zfs-team@sun.com Message-id: <49D3AAC9.2010503@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200903310339.n2V3d5Dl019795@localhost.sfbay.sun.com> User-Agent: Thunderbird 2.0.0.21 (Windows/20090302) Status: RO Content-Length: 1164 Matthew Ahrens wrote: > ... > User space accounting and quotas "stick with" each dataset (snapshot, > filesystem, and clone). This means that user quotas (and space > accounting) are not inherited. They will be "copied" to a new snapshot, > and keep the values they had at the time the snapshot was taken. > Likewise, user quotas will be "copied" to a clone (from its origin > snapshot), and they will be copied with "zfs send" (even without -R). > (User accounting and quota information is not actually copied to > snapshots and clones, just referenced and copied-on-write like other > filesystem contents.) If I read this right then the implication is that a user can use snapshot/clone to increase their allotted quota in a givern pool because they can create more datasets with their own quota, correct? So if I start out with tank/test1 that has a quota of 2GB, create a snapshot of that (tank/test1@now) and then clone it (test/test1_clone), I end up with 4GB of quota for tank/test1. That's different. If at some point in the future there was a desire to implement quotas for pools, does anything this project delivers make that "impossible"? Darren From Matthew.Ahrens@sun.com Wed Apr 1 11:04:51 2009 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n31I4pvN024218 for ; Wed, 1 Apr 2009 11:04:51 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n31I4khh025526; Wed, 1 Apr 2009 11:04:49 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KHF0002ROVZSN00@nwk-avmta-2.sfbay.sun.com>; Wed, 01 Apr 2009 11:04:47 -0700 (PDT) Received: from zion.sfbay.sun.com ([129.146.17.75]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KHF00LP1OVYSQ20@nwk-avmta-2.sfbay.sun.com>; Wed, 01 Apr 2009 11:04:46 -0700 (PDT) Received: from stationary.local (punchin-ahrens.SFBay.Sun.COM [10.7.251.178]) by zion.sfbay.sun.com (8.14.2+Sun/8.14.2) with ESMTP id n31I4jdS003081; Wed, 01 Apr 2009 18:04:45 +0000 (GMT) Date: Wed, 01 Apr 2009 11:04:44 -0700 From: Matthew Ahrens Subject: Re: ZFS user/group quotas & space accounting [PSARC/2009/204 FastTrack timeout 04/08/2009] In-reply-to: <49D3AAC9.2010503@Sun.COM> To: Darren Reed Cc: Matthew Ahrens , psarc-ext@sun.com, lustre-dmu-ext@sun.com, zfs-team@sun.com Message-id: <49D3ACBC.6040300@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200903310339.n2V3d5Dl019795@localhost.sfbay.sun.com> <49D3AAC9.2010503@Sun.COM> User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) Status: RO Content-Length: 1761 Darren Reed wrote: > Matthew Ahrens wrote: >> ... >> User space accounting and quotas "stick with" each dataset (snapshot, >> filesystem, and clone). This means that user quotas (and space >> accounting) are not inherited. They will be "copied" to a new snapshot, >> and keep the values they had at the time the snapshot was taken. >> Likewise, user quotas will be "copied" to a clone (from its origin >> snapshot), and they will be copied with "zfs send" (even without -R). >> (User accounting and quota information is not actually copied to >> snapshots and clones, just referenced and copied-on-write like other >> filesystem contents.) > > If I read this right then the implication is that a user can use > snapshot/clone to increase their allotted quota in a givern pool > because they can create more datasets with their own quota, correct? Yes, if you have permission to create clones, you can make more space available to yourself. I doubt sysadmins will do this in practice (setting a quota on someone with sysadmin privs). > So if I start out with tank/test1 that has a quota of 2GB, create a > snapshot of that (tank/test1@now) and then clone it (test/test1_clone), > I end up with 4GB of quota for tank/test1. That's different. No. If you do: zfs create pool/fs zfs set userquota@dreed=2G pool/fs zfs snapshot pool/fs@snap zfs clone pool/fs@snap pool/clone then user dreed is now allowed to use 2G in pool/fs and 2G in pool/clone. The quota on pool/fs is not changed. > If at some point in the future there was a desire to implement quotas > for pools, does anything this project delivers make that "impossible"? No, it makes it easier. (And yes, there are pitfalls with zones; let's save this discussion for a later date.) --matt From Matthew.Ahrens@sun.com Wed Apr 1 14:19:00 2009 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n31LJ0Uu009080 for ; Wed, 1 Apr 2009 14:19:00 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n31LJ01U023723; Wed, 1 Apr 2009 14:19:00 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KHF00B05XVO0O00@nwk-avmta-2.sfbay.sun.com>; Wed, 01 Apr 2009 14:19:00 -0700 (PDT) Received: from zion.sfbay.sun.com ([129.146.17.75]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KHF00LDXXVOS9E0@nwk-avmta-2.sfbay.sun.com>; Wed, 01 Apr 2009 14:19:00 -0700 (PDT) Received: from stationary.local (punchin-ahrens.SFBay.Sun.COM [10.7.251.178]) by zion.sfbay.sun.com (8.14.2+Sun/8.14.2) with ESMTP id n31LIxLR009495; Wed, 01 Apr 2009 21:18:59 +0000 (GMT) Date: Wed, 01 Apr 2009 14:18:58 -0700 From: Matthew Ahrens Subject: Re: ZFS user/group quotas & space accounting [PSARC/2009/204 FastTrack timeout 04/08/2009] In-reply-to: <200903310339.n2V3d5Dl019795@localhost.sfbay.sun.com> To: Matthew Ahrens Cc: PSARC-ext@sun.com, lustre-dmu-ext@sun.com, zfs-team@sun.com Message-id: <49D3DA42.7050006@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200903310339.n2V3d5Dl019795@localhost.sfbay.sun.com> User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) Status: RO Content-Length: 8988 To summarize the issues that have come up here and on zfs-discuss: Additional specification: Zones have different POSIX numeric ID <-> name mappings, so we should not interpret the numeric ID from a local zone in the global zone. Therefore: "zfs userspace" from the global zone on a zoned filesystem will always use the "-n" semantics (print numeric ID, not name). "zfs {get|set} {user|group}{used|quota}@..." from the global zone on a zoned filesystem will only accept numeric IDs, not names. Additional RFE: It would be nice if rquotad supported ZFS quotas so that users could determine their quotas and space usage over NFS. This case will not be extended to cover that, but we've filed RFE 6824968 which we think will be reasonable to do in the near future. Clarification / reiteration: The amount of space charged to a user is the compressed space (same as with all other ZFS space accounting). It would be nice if there were better APIs for doing the numeric SID <-> user@domain mappings (eg, one API for both POSIX IDs and SIDs), but there aren't for now. --matt Matthew Ahrens wrote: > Template Version: @(#)sac_nextcase 1.68 02/23/09 SMI > This information is Copyright 2009 Sun Microsystems > 1. Introduction > 1.1. Project/Component Working Name: > ZFS user/group quotas & space accounting > 1.2. Name of Document Author/Supplier: > Author: Matthew Ahrens > 1.3 Date of This Document: > 30 March, 2009 > 4. Technical Description > ZFS user/group space accounting > > A. SUMMARY > > This case adds support to ZFS for user/group quotas & per-uid/gid space > tracking. > > B. PROBLEM > > Enterprise customers often want to know who is using space, based on > what uid and gid owns each file. > > Education customers often want to apply per-user quotas to hundreds of > thousands of users. In these situations, the number of users and/or > existing infrastructure prohibits using one filesystem per user and > setting filesystem-wide quotas. > > C. PROPOSED SOLUTION > > 1. Overview > > Each filesystem keeps track of how much space inside it is owned by each > user (uid) and group (gid). This is the amount of space "referenced", > so relationships between filesystems, descendents, clones, and snapshots > are ignored, and each tracks their "user used" and "group used" > independently. This is the same policy behind the "referenced", > "refquota", and "refreservation" properties. The amount of space > charged is the amount of space reported by struct stat's st_blocks and > du(1). > > Both POSIX ids (uid & gid) and untranslated SIDs are supported (eg, when > sharing filesystems over SMB without a name service translation set up). > > ZFS will now enforce quotas on the amount of space referenced by files > owned by particular users and groups. Enforcement may be delayed by > several seconds. In other words, users may go a bit over their quota > before the system notices that they are over quota and begins to refuse > additional writes with EDQUOT. This decision was made to get the > feature to market in a reasonable time, with a minimum of engineering > resources expended. The design and implementation do not preclude > implementing strict enforcement at a later date. > > User space accounting and quotas "stick with" each dataset (snapshot, > filesystem, and clone). This means that user quotas (and space > accounting) are not inherited. They will be "copied" to a new snapshot, > and keep the values they had at the time the snapshot was taken. > Likewise, user quotas will be "copied" to a clone (from its origin > snapshot), and they will be copied with "zfs send" (even without -R). > (User accounting and quota information is not actually copied to > snapshots and clones, just referenced and copied-on-write like other > filesystem contents.) > > The user space accounting and quotas is reported by the new > userused@, groupused@, userquota@, and > groupquota@ properties, and by the new "zfs userspace" and "zfs > groupspace" subcommands, which are detailed below. > > 2. Version Compatibility > > To use these features, the pool must be upgraded to a new on-disk > version (15). Old filesystems must have their space accounting > information initialized by running "zfs userspace " or upgrading the > old filesystem to a new on-disk version (4). To set user quotas, the > pool and filesystem must both be upgraded. > > 3. Permissions > > Setting or changing user quotas are administrative actions, subject to > the same privilege requirements as other zfs subcommands. There are new > "userquota" and "groupquota" permissions which can be granted with "zfs > allow", to allow those properties to be viewed and changed. > > Unprivileged users can only view their own userquota and userused, and > the groupquota and groupused of any groups they belong to. The new > "userused" and "groupused" permissions can be granted with "zfs allow" > to permit users to view these properties. > > The existing "version" permission (granted with "zfs allow") permits the > accounting information to be initialized by "zfs userspace". > > 4. New Properties > > user/group space accounting information and quotas can be manipulated > with 4 new properties: > > zfs get userused@ > zfs get groupused@ > > zfs get userquota@ > zfs get groupquota@ > > zfs set userquota@= > zfs set groupquota@= > > The or is specified using one of the following forms: > posix name (eg. ahrens) > posix numeric id (eg. 126829) > sid name (eg. ahrens@sun) > sid numeric id (eg. S-1-12345-12423-125829) > > For "zfs set", if a nonexistent name is specified, an error is > generated. Any numeric ID is permitted. For "zfs get", if a > nonexistent name is specified, "-" is printed for the value, indicating > that there is no value available (like "zfs get nonexistent:userproperty"). > > As with filesystem quotas ("quota" and "refquota" properties), user > quotas can be set to a value larger than the available space. > > User quotas can also be set to a value less than the amount of space > used by that user, effectively forcing that user to reduce their space > utilization. > > These new properties are not printed by "zfs get all", since that could > generate a huge amount of output, which would not be very well > organized. The new "zfs userspace" subcommand should be used instead. > > 5. New Subcommands > > Two new subcommands are added: "zfs userspace" and "zfs groupspace": > > zfs {user|group}space [-hniHp] [-o field[,...]] [-sS field] ... > [-t type [,...]] > > Typical output is like this: > TYPE NAME USED QUOTA > POSIX User ahrens 14M 1G > POSIX User george 56.5M none > POSIX User lling 258M 500M > SMB User marks@sun 103M none > > Option flags: > > -h Show help message and exit. > > -n Print numeric ID instead of user/group name (like "ls -n") > > -i Translate SID to POSIX ID. The POSIX ID may be ephemeral if no > mapping exists. Normal POSIX interfaces (eg, stat(2), "ls -l") > perform this translation, so the -i option allows the output > from "zfs userspace" to be compared directly with those > utilities. However, -i may lead to confusion if some files were > created by a SMB user before a SMB -> POSIX name mapping was > established. In that case some files are owned by the SMB > entity and some by the POSIX entity. The -i flag will report > that the POSIX entity has the total usage and quota for both > entities. > > -H Do not print headers, use tab-delimited output (like "zfs list/get -H") > > -p Use exact (parsable) numeric output (like "zfs get -p") > > -o field[,...] > Print only the specified fields (like "zfs list/get -o"), from > the following set: type,name,used,quota. The default is to > print all fields. > > -s field > Sort output by this field (like "zfs list -s"). The -s (and -S) > flag may be specified multiple times to sort first by one field, > then by another. The default is "-s type -s name". > > -S field > Sort by this field in reverse order, see -s. > > -t type[,...] > Print only the specified types (like "zfs list -t"), from the > following set: all,posixuser,smbuser,posixgroup,smbgroup. The > default for "zfs userspace" is "-t posixuser,smbuser". The > default for "zfs groupspace" is "-t posixgroup,smbgroup". This > is the only difference between the two subcommands, and in fact > "zfs userspace -t posixgroup" is perfectly valid. > > 6. Stability > > This case requests patch/micro release binding. The new interfaces are > committed. > > > 6. Resources and Schedule > 6.4. Steering Committee requested information > 6.4.1. Consolidation C-team Name: > ON > 6.5. ARC review type: FastTrack > 6.6. ARC Exposure: open From edward.pilatowicz@sun.com Wed Apr 1 16:24:17 2009 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n31NOGsi019761 for ; Wed, 1 Apr 2009 16:24:17 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n31NOBwL023326; Thu, 2 Apr 2009 00:24:14 +0100 (BST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KHG00H0B3OCRA00@brm-avmta-1.central.sun.com>; Wed, 01 Apr 2009 17:24:12 -0600 (MDT) Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.59]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KHG00CWG3OBIDD0@brm-avmta-1.central.sun.com>; Wed, 01 Apr 2009 17:24:11 -0600 (MDT) Received: from jurassic-x4600.sfbay.sun.com (localhost [127.0.0.1]) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n31NOBdw342131 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 01 Apr 2009 16:24:11 -0700 (PDT) Received: (from edp@localhost) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3/Submit) id n31NOAXU342130; Wed, 01 Apr 2009 16:24:10 -0700 (PDT) Date: Wed, 01 Apr 2009 16:24:10 -0700 From: Edward Pilatowicz Subject: Re: ZFS user/group quotas & space accounting [PSARC/2009/204 FastTrack timeout 04/08/2009] In-reply-to: <49D3DA42.7050006@sun.com> To: Matthew Ahrens Cc: Matthew Ahrens , PSARC-ext@sun.com, lustre-dmu-ext@sun.com, zfs-team@sun.com Message-id: <20090401232410.GA340348@eng.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <200903310339.n2V3d5Dl019795@localhost.sfbay.sun.com> <49D3DA42.7050006@sun.com> X-Authentication-warning: jurassic-x4600.sfbay.sun.com: edp set sender to edward.pilatowicz@sun.com using -f User-Agent: Mutt/1.5.19 (2009-01-05) Status: RO Content-Length: 793 On Wed, Apr 01, 2009 at 02:18:58PM -0700, Matthew Ahrens wrote: > To summarize the issues that have come up here and on zfs-discuss: > > Additional specification: > > Zones have different POSIX numeric ID <-> name mappings, so we should not > interpret the numeric ID from a local zone in the global zone. > Therefore: > > "zfs userspace" from the global zone on a zoned filesystem will always > use the "-n" semantics (print numeric ID, not name). > > "zfs {get|set} {user|group}{used|quota}@..." from the global zone on a > zoned filesystem will only accept numeric IDs, not names. > so just to be clear, inside a non-global zone which contains zoned filesystems, administrators will be able to set and see these properties using both numeric ids and their corresponding name mappings? ed From Matthew.Ahrens@sun.com Wed Apr 1 16:25:26 2009 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n31NPQdT019813 for ; Wed, 1 Apr 2009 16:25:26 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n31NPOWe017878; Wed, 1 Apr 2009 16:25:25 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KHG00H0V3QCW900@brm-avmta-1.central.sun.com>; Wed, 01 Apr 2009 17:25:24 -0600 (MDT) Received: from zion.sfbay.sun.com ([129.146.17.75]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KHG00C7O3QBIBD0@brm-avmta-1.central.sun.com>; Wed, 01 Apr 2009 17:25:23 -0600 (MDT) Received: from stationary.local (punchin-ahrens.SFBay.Sun.COM [10.7.251.178]) by zion.sfbay.sun.com (8.14.2+Sun/8.14.2) with ESMTP id n31NPMqQ012512; Wed, 01 Apr 2009 23:25:22 +0000 (GMT) Date: Wed, 01 Apr 2009 16:25:22 -0700 From: Matthew Ahrens Subject: Re: ZFS user/group quotas & space accounting [PSARC/2009/204 FastTrack timeout 04/08/2009] In-reply-to: <20090401232410.GA340348@eng.sun.com> To: Edward Pilatowicz Cc: Matthew Ahrens , PSARC-ext@sun.com, lustre-dmu-ext@sun.com, zfs-team@sun.com Message-id: <49D3F7E2.5050905@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200903310339.n2V3d5Dl019795@localhost.sfbay.sun.com> <49D3DA42.7050006@sun.com> <20090401232410.GA340348@eng.sun.com> User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) Status: RO Content-Length: 856 Edward Pilatowicz wrote: > On Wed, Apr 01, 2009 at 02:18:58PM -0700, Matthew Ahrens wrote: >> To summarize the issues that have come up here and on zfs-discuss: >> >> Additional specification: >> >> Zones have different POSIX numeric ID <-> name mappings, so we should not >> interpret the numeric ID from a local zone in the global zone. >> Therefore: >> >> "zfs userspace" from the global zone on a zoned filesystem will always >> use the "-n" semantics (print numeric ID, not name). >> >> "zfs {get|set} {user|group}{used|quota}@..." from the global zone on a >> zoned filesystem will only accept numeric IDs, not names. >> > > so just to be clear, inside a non-global zone which contains zoned > filesystems, administrators will be able to set and see these properties > using both numeric ids and their corresponding name mappings? Correct. --matt From Matthew.Ahrens@sun.com Tue Apr 14 16:50:21 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n3ENoKRx014055 for ; Tue, 14 Apr 2009 16:50:21 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n3ENoFoN015805; Wed, 15 Apr 2009 07:50:19 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KI4003057JS9I00@nwk-avmta-1.sfbay.Sun.COM>; Tue, 14 Apr 2009 16:50:16 -0700 (PDT) Received: from zion.sfbay.sun.com ([129.146.17.75]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KI400IAM7JSVDD0@nwk-avmta-1.sfbay.Sun.COM>; Tue, 14 Apr 2009 16:50:16 -0700 (PDT) Received: from stationary.local (punchin-ahrens.SFBay.Sun.COM [10.7.251.178]) by zion.sfbay.sun.com (8.14.2+Sun/8.14.2) with ESMTP id n3ENoGWA017851; Tue, 14 Apr 2009 23:50:16 +0000 (GMT) Date: Tue, 14 Apr 2009 16:50:16 -0700 From: Matthew Ahrens Subject: Re: ZFS user/group quotas & space accounting [PSARC/2009/204 FastTrack timeout 04/08/2009] In-reply-to: <200903310339.n2V3d5Dl019795@localhost.sfbay.sun.com> To: Matthew Ahrens Cc: PSARC-ext@sun.com, lustre-dmu-ext@sun.com, zfs-team@sun.com Message-id: <49E52138.2060407@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200903310339.n2V3d5Dl019795@localhost.sfbay.sun.com> User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) Status: RO Content-Length: 7998 No outstanding issues, timer expired last Wednesday. I'm marking this case closed approved. --matt Matthew Ahrens wrote: > Template Version: @(#)sac_nextcase 1.68 02/23/09 SMI > This information is Copyright 2009 Sun Microsystems > 1. Introduction > 1.1. Project/Component Working Name: > ZFS user/group quotas & space accounting > 1.2. Name of Document Author/Supplier: > Author: Matthew Ahrens > 1.3 Date of This Document: > 30 March, 2009 > 4. Technical Description > ZFS user/group space accounting > > A. SUMMARY > > This case adds support to ZFS for user/group quotas & per-uid/gid space > tracking. > > B. PROBLEM > > Enterprise customers often want to know who is using space, based on > what uid and gid owns each file. > > Education customers often want to apply per-user quotas to hundreds of > thousands of users. In these situations, the number of users and/or > existing infrastructure prohibits using one filesystem per user and > setting filesystem-wide quotas. > > C. PROPOSED SOLUTION > > 1. Overview > > Each filesystem keeps track of how much space inside it is owned by each > user (uid) and group (gid). This is the amount of space "referenced", > so relationships between filesystems, descendents, clones, and snapshots > are ignored, and each tracks their "user used" and "group used" > independently. This is the same policy behind the "referenced", > "refquota", and "refreservation" properties. The amount of space > charged is the amount of space reported by struct stat's st_blocks and > du(1). > > Both POSIX ids (uid & gid) and untranslated SIDs are supported (eg, when > sharing filesystems over SMB without a name service translation set up). > > ZFS will now enforce quotas on the amount of space referenced by files > owned by particular users and groups. Enforcement may be delayed by > several seconds. In other words, users may go a bit over their quota > before the system notices that they are over quota and begins to refuse > additional writes with EDQUOT. This decision was made to get the > feature to market in a reasonable time, with a minimum of engineering > resources expended. The design and implementation do not preclude > implementing strict enforcement at a later date. > > User space accounting and quotas "stick with" each dataset (snapshot, > filesystem, and clone). This means that user quotas (and space > accounting) are not inherited. They will be "copied" to a new snapshot, > and keep the values they had at the time the snapshot was taken. > Likewise, user quotas will be "copied" to a clone (from its origin > snapshot), and they will be copied with "zfs send" (even without -R). > (User accounting and quota information is not actually copied to > snapshots and clones, just referenced and copied-on-write like other > filesystem contents.) > > The user space accounting and quotas is reported by the new > userused@, groupused@, userquota@, and > groupquota@ properties, and by the new "zfs userspace" and "zfs > groupspace" subcommands, which are detailed below. > > 2. Version Compatibility > > To use these features, the pool must be upgraded to a new on-disk > version (15). Old filesystems must have their space accounting > information initialized by running "zfs userspace " or upgrading the > old filesystem to a new on-disk version (4). To set user quotas, the > pool and filesystem must both be upgraded. > > 3. Permissions > > Setting or changing user quotas are administrative actions, subject to > the same privilege requirements as other zfs subcommands. There are new > "userquota" and "groupquota" permissions which can be granted with "zfs > allow", to allow those properties to be viewed and changed. > > Unprivileged users can only view their own userquota and userused, and > the groupquota and groupused of any groups they belong to. The new > "userused" and "groupused" permissions can be granted with "zfs allow" > to permit users to view these properties. > > The existing "version" permission (granted with "zfs allow") permits the > accounting information to be initialized by "zfs userspace". > > 4. New Properties > > user/group space accounting information and quotas can be manipulated > with 4 new properties: > > zfs get userused@ > zfs get groupused@ > > zfs get userquota@ > zfs get groupquota@ > > zfs set userquota@= > zfs set groupquota@= > > The or is specified using one of the following forms: > posix name (eg. ahrens) > posix numeric id (eg. 126829) > sid name (eg. ahrens@sun) > sid numeric id (eg. S-1-12345-12423-125829) > > For "zfs set", if a nonexistent name is specified, an error is > generated. Any numeric ID is permitted. For "zfs get", if a > nonexistent name is specified, "-" is printed for the value, indicating > that there is no value available (like "zfs get nonexistent:userproperty"). > > As with filesystem quotas ("quota" and "refquota" properties), user > quotas can be set to a value larger than the available space. > > User quotas can also be set to a value less than the amount of space > used by that user, effectively forcing that user to reduce their space > utilization. > > These new properties are not printed by "zfs get all", since that could > generate a huge amount of output, which would not be very well > organized. The new "zfs userspace" subcommand should be used instead. > > 5. New Subcommands > > Two new subcommands are added: "zfs userspace" and "zfs groupspace": > > zfs {user|group}space [-hniHp] [-o field[,...]] [-sS field] ... > [-t type [,...]] > > Typical output is like this: > TYPE NAME USED QUOTA > POSIX User ahrens 14M 1G > POSIX User george 56.5M none > POSIX User lling 258M 500M > SMB User marks@sun 103M none > > Option flags: > > -h Show help message and exit. > > -n Print numeric ID instead of user/group name (like "ls -n") > > -i Translate SID to POSIX ID. The POSIX ID may be ephemeral if no > mapping exists. Normal POSIX interfaces (eg, stat(2), "ls -l") > perform this translation, so the -i option allows the output > from "zfs userspace" to be compared directly with those > utilities. However, -i may lead to confusion if some files were > created by a SMB user before a SMB -> POSIX name mapping was > established. In that case some files are owned by the SMB > entity and some by the POSIX entity. The -i flag will report > that the POSIX entity has the total usage and quota for both > entities. > > -H Do not print headers, use tab-delimited output (like "zfs list/get -H") > > -p Use exact (parsable) numeric output (like "zfs get -p") > > -o field[,...] > Print only the specified fields (like "zfs list/get -o"), from > the following set: type,name,used,quota. The default is to > print all fields. > > -s field > Sort output by this field (like "zfs list -s"). The -s (and -S) > flag may be specified multiple times to sort first by one field, > then by another. The default is "-s type -s name". > > -S field > Sort by this field in reverse order, see -s. > > -t type[,...] > Print only the specified types (like "zfs list -t"), from the > following set: all,posixuser,smbuser,posixgroup,smbgroup. The > default for "zfs userspace" is "-t posixuser,smbuser". The > default for "zfs groupspace" is "-t posixgroup,smbgroup". This > is the only difference between the two subcommands, and in fact > "zfs userspace -t posixgroup" is perfectly valid. > > 6. Stability > > This case requests patch/micro release binding. The new interfaces are > committed. > > > 6. Resources and Schedule > 6.4. Steering Committee requested information > 6.4.1. Consolidation C-team Name: > ON > 6.5. ARC review type: FastTrack > 6.6. ARC Exposure: open