From dp@snowdog.eng.sun.com Wed Oct 25 14:46:41 2006 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id k9PLkerR021537 for ; Wed, 25 Oct 2006 14:46:40 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.149.247.22]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id k9PLkGZR005199 for <@sunmail3.sfbay.sun.com:PSARC-EXT@Sun.Com>; Thu, 26 Oct 2006 05:46:39 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0J7P0090BOHM1M00@nwk-avmta-2.sfbay.sun.com> for PSARC-EXT@Sun.Com (ORCPT PSARC-EXT@Sun.Com); Wed, 25 Oct 2006 14:46:34 -0700 (PDT) Received: from engmail3mpk.sfbay.Sun.COM ([129.146.11.26]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0J7P00KW6OHL2Y80@nwk-avmta-2.sfbay.sun.com> for PSARC-EXT@Sun.Com (ORCPT PSARC-EXT@Sun.Com); Wed, 25 Oct 2006 14:46:33 -0700 (PDT) Received: from snowdog.eng.sun.com (snowdog.SFBay.Sun.COM [129.146.228.213]) by engmail3mpk.sfbay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id k9PLkWDT006972; Wed, 25 Oct 2006 14:46:33 -0700 (PDT) Received: from snowdog.eng.sun.com (localhost [127.0.0.1]) by snowdog.eng.sun.com (8.13.7+Sun/8.13.7) with ESMTP id k9PLkW5b023152; Wed, 25 Oct 2006 14:46:32 -0700 (PDT) Received: (from dp@localhost) by snowdog.eng.sun.com (8.13.7+Sun/8.13.7/Submit) id k9PLkWGK023151; Wed, 25 Oct 2006 14:46:32 -0700 (PDT) Date: Wed, 25 Oct 2006 14:46:32 -0700 From: Dan Price Subject: PSARC/2006/598 Swap resource control; locked memory RM improvements To: PSARC-EXT@sun.com Cc: zones-discuss@opensolaris.org, Stephen Lawrence Message-id: <20061025214631.GA23100@eng.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 User-Agent: Mutt/1.4.2.1i Status: RO Content-Length: 9805 I am pleased to sponsor the following case for Steve Lawrence. This case improves memory resource management and further integrates memory RM with Zones. The interfaces are Commited (for the rctls and zonecfg(1m) properties) and Uncommitted (for the kstats and changes to the output of prstat(1m)). Patch release binding is requested. The timer is set for 11/01/2006. Please note that the Zones team plans to run this and future cases "in the open." Both Steve Lawrence and zones-discuss@opensolaris.org should be CC'd. For members of zones-discuss who are wondering what the heck this mail is about, please see http://www.opensolaris.org/jive/thread.jspa?threadID=15976 before replying to this message. Thanks, -dp - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Swap resource control; locked memory RM improvements Steve Lawrence SUMMARY: This case enhances Solaris Zones[1] and builds upon recent work to improve the integration between Zones and Solaris Resource Management[2]. The case addresses an existing RFE[6], which requests a mechanism to limit system swap reserved by a zone. The case also proposes extensions to [2], which will make swap reservation and locked memory resource controls easy to configure on a zone via zonecfg(1m). 1. This case proposes adding the following resource control: INTERFACE COMMITMENT BINDING "zone.max-swap" Committed Patch This control will limit the swap reserved by processes and tmpfs mounts within the global zone and non-global zones. This resource control serves to address the referenced RFE[6]. 2. To simplify the configuration of memory-related resource controls on zones, this case proposes adding the following properties to zonecfg(1M): INTERFACE COMMITMENT BINDING "swap" zonecfg property Committed Patch "locked" zonecfg property Committed Patch These properties will be added to the zonecfg "capped-memory" zonecfg resource introduced by [2]. 3. For observability of zone resource utilization and limits, this case proposes the addition of following kstats: INTERFACE COMMITMENT BINDING zone:{zoneid}:vm:zonename Uncommitted Patch zone:{zoneid}:vm:swap_reserved Uncommitted Patch zone:{zoneid}:vm:max_swap_reserved Uncommitted Patch zone:{zoneid}:vm:locked_memory Uncommitted Patch zone:{zoneid}:vm:max_locked_memory Uncommitted Patch These kstats will be of class "misc". The global zone will see kstats for all zones, while non global zones will only see kstats with matching zoneid. 4. prstat(1m) output changes to report swap reserved. INTERFACE COMMITMENT BINDING prstat(1m) output Uncommitted Patch This case proposes changing the "SIZE" column of "prstat -Z" zone output lines to "SWAP". The swap reported will be the total swap consumed by the zone's processes and tmpfs mounts. This value will assist administrators in monitoring the swap reserved by each zone, allowing them to choose a reasonable "zone.max-swap" settings. The "SIZE" column will also be changed to "SWAP" for prstat options a, T, and J, for users, tasks, and projects. DETAIL: 1. "zone.max-swap" resource control. Limits swap consumed by user process address space mappings and tmpfs mounts within a zone. Currently a global or non-global zone can consume all swap resources available on the system, limiting the usefulness of zones as an application container. zone.max-swap provides a mechanism to limit swap consumption per zone. This will protect other zones from runaway memory leakers/consumers and/or tmpfs writers in a zone with zone.max-swap configured. Another solution to this problem would be a "swap set" [5] feature, which would allow the reservation of swap devices into sets to which zones could be bound. While "swap sets" would be useful, zone.max-swap provides a simple solution which is easier to administer, as it does not require the configuration of pools and swap devices/files. "zone.max-swap" is not incompatable with swap sets. In fact, a future addition of swap sets could be used in combination with zone.max-swap. For instance, several zones could be bound to the same set of swap devices, each with it's own individual zone.max-swap configured as a cap within that set. The implementation of "zone.max-swap" is also much less risky to make available via patch. zone.max-swap will be configurable on both the global zone, and non-global zones. The affect on processes in a zone reaching its zone.max-swap limit is the same as if all system swap is reserved. Callers of mmap(2) and sbrk(2) will receive EAGAIN. Writes to tmpfs will return ENOSPC, which is the same errno returned when a tmpfs mount reaches it's "size" mount option. The "size" mount option limits the quantity of swap that a tmpfs mount can reserve. While a low zone.max-swap setting for the global zone can lead to a difficult-to-administer global zone, the same problem exists today when configuring the zone.max-lwps resource control on the global zone, or when all system swap is reserved. 2. "swap" and "locked" properties for zonecfg(1m) "capped_memory" resource. [2] added a new 'capped-memory' resource to zonecfg. This resource groups the properties used when capping memory for the zone. It currently has the 'physical' property which specifies the physical memory cap for the zone. We will add two new properties, 'swap' and 'locked' to the "capped-memory" resource. These properties will be added by using the rctl alias mechanism which is also described in [2]. swap: An unsigned decimal number with a required k, m, g, or t modifier. A value of '10m' means ten megabytes." This will be used to configure the zone.max-swap resource control, which limits swap consumed by processes and tmpfs mounts within a zone. locked: An unsigned decimal number with a required k, m, g, or t modifier. A value of '10m' means ten megabytes." This will be used to configure the zone.max-locked-memory[3,4] resource control, which limits locked physical memory (made non-pageable) by processes within a zone. 3. Swap and locked memory kstats for zones. There is currently no way to observe how much locked memory or swap a zone is consuming. This makes capacity planning and monitoring difficult. To solve this problem, the following kstats are proposed: INTERFACE COMMITMENT BINDING zone:{zoneid}:vm:zonename Uncommitted Patch zone:{zoneid}:vm:swap_reserved Uncommitted Patch zone:{zoneid}:vm:max_swap_reserved Uncommitted Patch zone:{zoneid}:vm:locked_memory Uncommitted Patch zone:{zoneid}:vm:max_locked_memory Uncommitted Patch Higher level monitoring scripts/tools can be developed in the future to consume these kstats, or a future version of these kstats. STATISTIC DESCRIPTION zonename The name of the zone with {zoneid} swap reserved: swap reserved by zone in bytes. max_swap_reserved: current zone.max-swap limit in bytes, locked_memory: physical memory locked by zone in bytes. max_locked_memory: current zone.max-locked-memory limit in bytes. These kstats can be consumed by higher level tools/scripts to provide information about zone memory usage. Each kstats instance number matches the zoneid of the zone it represents. Non-global zones will only be able to read the kstat with matching zoneid The global zone will be able to read all kstats. 4. prstat(1m) output changes to report swap reserved. INTERFACE COMMITMENT BINDING prstat(1m) output Uncommitted Patch This case proposes changing the "SIZE" column of "prstat -Z" zone output lines to "SWAP". The swap reported will be the total swap consumed by the zone's processes and tmpfs mounts. This value will assist administrators in monitoring the swap reserved by each zone, allowing them to choose a reasonable "zone.max-swap" settings. The "SIZE" column will also be changed to "SWAP" for prstat options a, T, and J, for users, tasks, and projects. The current "SIZE" column arbitrarily sums the address spaces of the processes in each zone. This sum include device mappings, but does not include NORESERVE segments. This sum does not map to any real system resource, and therefore provides no meaningful information. REFERENCES: [1] PSARC/2002/174 Virtualization and Namespace Isolation in Solaris http://sac.sfbay.sun.com/PSARC/2002/174 http://www.opensolaris.org/os/community/arc/caselog/2002/174/ [2] PSARC/2006/496 Improved Zones/RM Integration http://sac.sfbay.sun.com/PSARC/2006/496/ http://www.opensolaris.org/os/community/arc/caselog/2006/496/ [3] PSARC/2006/463 Amendment to zone/project.max-locked-memory Resource Controls http://sac.sfbay.sun.com/PSARC/2006/463/ http://www.opensolaris.org/os/community/arc/caselog/2006/463/ [4] PSARC/2004/580 zone/project.max-locked-memory Resource Controls http://sac.sfbay.sun.com/PSARC/2004/580/ http://www.opensolaris.org/os/community/arc/caselog/2004/580/ [5] PSARC/2002/181 Swap Sets http://sac.sfbay.sun.com/PSARC/2002/181/ http://www.opensolaris.org/os/community/arc/caselog/2002/181/ [6] 5103071 RFE: local zones can run the global zone out of swap http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=5103071 -- Daniel Price - Solaris Kernel Engineering - dp@eng.sun.com - blogs.sun.com/dp From Jeff.Victor@Sun.COM Wed Oct 25 18:33:41 2006 Received: from sunmail1brm.Central.Sun.COM (sunmail1brm.Central.Sun.COM [129.147.62.17]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id k9Q1XfKE026904 for ; Wed, 25 Oct 2006 18:33:41 -0700 (PDT) Received: from nwk-avmta-1.sfbay.sun.com (nwk-avmta-1.SFBay.Sun.COM [129.149.246.28]) by sunmail1brm.Central.Sun.COM (8.11.7p1+Sun/8.11.7/ENSMAIL,v2.2) with ESMTP id k9Q1Xec18155 for <@sunmail2.sfbay.sun.com:PSARC-EXT@sun.com>; Wed, 25 Oct 2006 19:33:40 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2 (built Dec 2 2004)) id <0J7P00907Z034X00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Wed, 25 Oct 2006 18:33:39 -0700 (PDT) Received: from brmea-mail-3.sun.com ([192.18.98.34]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2 (built Dec 2 2004)) with ESMTP id <0J7P0071RZ03SU20@nwk-avmta-1.sfbay.Sun.COM> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Wed, 25 Oct 2006 18:33:39 -0700 (PDT) Received: from fe-amer-06.sun.com ([192.18.108.180]) by brmea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id k9Q1XcVn012717 for ; Wed, 25 Oct 2006 19:33:38 -0600 (MDT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0J7P00L01XAPSP00@mail-amer.sun.com> (original mail from Jeff.Victor@Sun.COM) for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Wed, 25 Oct 2006 19:33:38 -0600 (MDT) Received: from [129.150.12.16] by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0J7P0010YYZYYER3@mail-amer.sun.com>; Wed, 25 Oct 2006 19:33:38 -0600 (MDT) Date: Wed, 25 Oct 2006 21:32:39 -0400 From: Jeff Victor Subject: Re: [zones-discuss] PSARC/2006/598 Swap resource control; locked memory RM improvements In-reply-to: <20061025214631.GA23100@eng.sun.com> Sender: Jeff.Victor@Sun.COM To: Dan Price Cc: PSARC-EXT@Sun.COM, Stephen Lawrence , zones-discuss@opensolaris.org Message-id: <45401037.5000806@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=us-ascii Content-transfer-encoding: 7BIT X-Accept-Language: en-us, en X-PMX-Version: 5.2.0.264296 References: <20061025214631.GA23100@eng.sun.com> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.7) Gecko/20051027 Status: RO Content-Length: 11189 Overall, these two pieces will be very welcome additions to the arsenal of zone-related resource controls. I have inserted some requests for clarification in-line. Dan Price wrote: > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > Swap resource control; locked memory RM improvements > Steve Lawrence > > SUMMARY: > This case enhances Solaris Zones[1] and builds upon recent work to > improve the integration between Zones and Solaris Resource > Management[2]. The case addresses an existing RFE[6], which requests > a mechanism to limit system swap reserved by a zone. The case also > proposes extensions to [2], which will make swap reservation and > locked memory resource controls easy to configure on a zone via > zonecfg(1m). > > 1. This case proposes adding the following resource control: > > INTERFACE COMMITMENT BINDING > "zone.max-swap" Committed Patch > > This control will limit the swap reserved by processes and tmpfs > mounts within the global zone and non-global zones. This resource > control serves to address the referenced RFE[6]. > > 2. To simplify the configuration of memory-related resource controls > on zones, this case proposes adding the following properties to > zonecfg(1M): > > INTERFACE COMMITMENT BINDING > "swap" zonecfg property Committed Patch > "locked" zonecfg property Committed Patch > > These properties will be added to the zonecfg "capped-memory" > zonecfg resource introduced by [2]. > > 3. For observability of zone resource utilization and limits, this > case proposes the addition of following kstats: > > INTERFACE COMMITMENT BINDING > zone:{zoneid}:vm:zonename Uncommitted Patch > zone:{zoneid}:vm:swap_reserved Uncommitted Patch > zone:{zoneid}:vm:max_swap_reserved Uncommitted Patch > zone:{zoneid}:vm:locked_memory Uncommitted Patch > zone:{zoneid}:vm:max_locked_memory Uncommitted Patch > > These kstats will be of class "misc". The global zone will see > kstats for all zones, while non global zones will only see kstats > with matching zoneid. > > 4. prstat(1m) output changes to report swap reserved. > > INTERFACE COMMITMENT BINDING > prstat(1m) output Uncommitted Patch > > This case proposes changing the "SIZE" column of "prstat -Z" zone > output lines to "SWAP". The swap reported will be the total swap > consumed by the zone's processes and tmpfs mounts. This value will > assist administrators in monitoring the swap reserved by each zone, > allowing them to choose a reasonable "zone.max-swap" settings. > > The "SIZE" column will also be changed to "SWAP" for prstat > options a, T, and J, for users, tasks, and projects. The reason for not changing this column in the default output would be helpful. > DETAIL: > > 1. "zone.max-swap" resource control. > > Limits swap consumed by user process address space mappings and > tmpfs mounts within a zone. > > Currently a global or non-global zone can consume all swap > resources available on the system, limiting the usefulness of zones > as an application container. zone.max-swap provides a mechanism to I would rephrase that as "the container of an application" to avoid confusion with the Solaris feature set called "Containers." I assume that the former was meant moreso than the latter even though Containers are Solaris' implementation of "an application container." > limit swap consumption per zone. This will protect other zones > from runaway memory leakers/consumers and/or tmpfs writers in a > zone with zone.max-swap configured. > > Another solution to this problem would be a "swap set" [5] feature, > which would allow the reservation of swap devices into sets to > which zones could be bound. While "swap sets" would be useful, > zone.max-swap provides a simple solution which is easier to > administer, as it does not require the configuration of pools and > swap devices/files. > > "zone.max-swap" is not incompatable with swap sets. In fact, a > future addition of swap sets could be used in combination with > zone.max-swap. For instance, several zones could be bound to the > same set of swap devices, each with it's own individual > zone.max-swap configured as a cap within that set. The > implementation of "zone.max-swap" is also much less risky to make > available via patch. > > zone.max-swap will be configurable on both the global zone, and > non-global zones. The affect on processes in a zone reaching its > zone.max-swap limit is the same as if all system swap is reserved. > Callers of mmap(2) and sbrk(2) will receive EAGAIN. Writes to > tmpfs will return ENOSPC, which is the same errno returned when > a tmpfs mount reaches it's "size" mount option. The "size" mount > option limits the quantity of swap that a tmpfs mount can reserve. With S10 11/06, some zone limitations are now configurable, e.g. setting the system time clock. Similarly, the ability to modify a zone's swap limit could be given to the zone's root user, which might be valuable in some situations. This would be analogous to the 'basic' privilege level. It would allow an advisory limit to be placed on a zone - a limit that the zone admin could modify in unusual circumstances. I realize that this opens a can of worms in that most rctls are protected by the sys_res_config priv, which is not allowed in a zone even with 11/06. Further, it makes sense to consistently allow or forbid rctl-modification in zones. I just wanted to mention this idea so that it is not unintentionally overlooked. > While a low zone.max-swap setting for the global zone can lead to > a difficult-to-administer global zone, the same problem exists > today when configuring the zone.max-lwps resource control on the > global zone, or when all system swap is reserved. > > 2. "swap" and "locked" properties for zonecfg(1m) "capped_memory" > resource. > > [2] added a new 'capped-memory' resource to zonecfg. This resource > groups the properties used when capping memory for the zone. It > currently has the 'physical' property which specifies the physical > memory cap for the zone. We will add two new properties, 'swap' > and 'locked' to the "capped-memory" resource. These properties > will be added by using the rctl alias mechanism which is also > described in [2]. > > swap: An unsigned decimal number with a required k, m, g, or t > modifier. A value of '10m' means ten megabytes." > This will be used to configure the zone.max-swap > resource control, which limits swap consumed by > processes and tmpfs mounts within a zone. > > locked: An unsigned decimal number with a required k, m, g, or t > modifier. A value of '10m' means ten megabytes." > This will be used to configure the > zone.max-locked-memory[3,4] resource control, which > limits locked physical memory (made non-pageable) by > processes within a zone. > > 3. Swap and locked memory kstats for zones. > > There is currently no way to observe how much locked memory or swap > a zone is consuming. This makes capacity planning and monitoring > difficult. To solve this problem, the following kstats are > proposed: > > INTERFACE COMMITMENT BINDING > zone:{zoneid}:vm:zonename Uncommitted Patch > zone:{zoneid}:vm:swap_reserved Uncommitted Patch > zone:{zoneid}:vm:max_swap_reserved Uncommitted Patch > zone:{zoneid}:vm:locked_memory Uncommitted Patch > zone:{zoneid}:vm:max_locked_memory Uncommitted Patch > > Higher level monitoring scripts/tools can be developed in the > future to consume these kstats, or a future version of these > kstats. > > STATISTIC DESCRIPTION > zonename The name of the zone with {zoneid} > swap reserved: swap reserved by zone in bytes. Does swap_reserved include pages shared with other zones, e.g. text pages? > max_swap_reserved: current zone.max-swap limit in bytes, > locked_memory: physical memory locked by zone in bytes. Does locked_memory include pages shared with other zones, e.g. text pages? > max_locked_memory: current zone.max-locked-memory limit in > bytes. > > These kstats can be consumed by higher level tools/scripts to > provide information about zone memory usage. Each kstats instance > number matches the zoneid of the zone it represents. Non-global > zones will only be able to read the kstat with matching zoneid The > global zone will be able to read all kstats. > > 4. prstat(1m) output changes to report swap reserved. > > INTERFACE COMMITMENT BINDING > prstat(1m) output Uncommitted Patch > > This case proposes changing the "SIZE" column of "prstat -Z" zone > output lines to "SWAP". The swap reported will be the total swap > consumed by the zone's processes and tmpfs mounts. This value > will assist administrators in monitoring the swap reserved by each > zone, allowing them to choose a reasonable "zone.max-swap" > settings. > > The "SIZE" column will also be changed to "SWAP" for prstat > options a, T, and J, for users, tasks, and projects. > > The current "SIZE" column arbitrarily sums the address spaces of > the processes in each zone. This sum include device mappings, > but does not include NORESERVE segments. This sum does not map > to any real system resource, and therefore provides no meaningful > information. > > REFERENCES: > > [1] PSARC/2002/174 Virtualization and Namespace Isolation in Solaris > http://sac.sfbay.sun.com/PSARC/2002/174 > http://www.opensolaris.org/os/community/arc/caselog/2002/174/ > > [2] PSARC/2006/496 Improved Zones/RM Integration > http://sac.sfbay.sun.com/PSARC/2006/496/ > http://www.opensolaris.org/os/community/arc/caselog/2006/496/ > > [3] PSARC/2006/463 Amendment to zone/project.max-locked-memory Resource > Controls > http://sac.sfbay.sun.com/PSARC/2006/463/ > http://www.opensolaris.org/os/community/arc/caselog/2006/463/ > > [4] PSARC/2004/580 zone/project.max-locked-memory Resource Controls > http://sac.sfbay.sun.com/PSARC/2004/580/ > http://www.opensolaris.org/os/community/arc/caselog/2004/580/ > > [5] PSARC/2002/181 Swap Sets > http://sac.sfbay.sun.com/PSARC/2002/181/ > http://www.opensolaris.org/os/community/arc/caselog/2002/181/ > > [6] 5103071 RFE: local zones can run the global zone out of swap > http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=5103071 -------------------------------------------------------------------------- Jeff VICTOR Sun Microsystems jeff.victor @ sun.com OS Ambassador Sr. Technical Specialist Solaris 10 Zones FAQ: http://www.opensolaris.org/os/community/zones/faq -------------------------------------------------------------------------- From sl108498@steve1.sfbay.sun.com Thu Oct 26 11:50:31 2006 Received: from sunmail1brm.Central.Sun.COM (sunmail1brm.Central.Sun.COM [129.147.62.17]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id k9QIoUPR018047 for ; Thu, 26 Oct 2006 11:50:30 -0700 (PDT) Received: from nwk-avmta-1.sfbay.sun.com (nwk-avmta-1.SFBay.Sun.COM [129.149.246.28]) by sunmail1brm.Central.Sun.COM (8.11.7p1+Sun/8.11.7/ENSMAIL,v2.2) with ESMTP id k9QIoUX03453 for <@sunmail2.sfbay.sun.com:PSARC-EXT@sun.com>; Thu, 26 Oct 2006 12:50:30 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2 (built Dec 2 2004)) id <0J7R00M01B061Z00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Thu, 26 Oct 2006 11:50:30 -0700 (PDT) Received: from sfbaymail2sca.sfbay.sun.com ([129.145.155.42]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2 (built Dec 2 2004)) with ESMTP id <0J7R00LFSB05AP00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Thu, 26 Oct 2006 11:50:29 -0700 (PDT) Received: from steve1.sfbay.sun.com (steve1.SFBay.Sun.COM [129.146.228.230]) by sfbaymail2sca.sfbay.sun.com (8.13.6+Sun/8.12.10/ENSMAIL,v2.2) with ESMTP id k9QIoJ7a002974; Thu, 26 Oct 2006 11:50:29 -0700 (PDT) Received: from steve1.sfbay.sun.com (localhost [127.0.0.1]) by steve1.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id k9QIo7Nb004212; Thu, 26 Oct 2006 11:50:07 -0700 (PDT) Received: (from sl108498@localhost) by steve1.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id k9QIo7iK004211; Thu, 26 Oct 2006 11:50:07 -0700 (PDT) Date: Thu, 26 Oct 2006 11:50:07 -0700 From: Steve Lawrence Subject: Re: [zones-discuss] PSARC/2006/598 Swap resource control; locked memory RM improvements In-reply-to: <45401037.5000806@sun.com> To: Jeff Victor Cc: Dan Price , PSARC-EXT@sun.com, Stephen Lawrence , zones-discuss@opensolaris.org Message-id: <20061026185007.GN2475@steve1.eng.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <20061025214631.GA23100@eng.sun.com> <45401037.5000806@sun.com> User-Agent: Mutt/1.4.2.1i Status: RO Content-Length: 5676 Comments inline. I've snipped stuff not relevant to comments. > > 4. prstat(1m) output changes to report swap reserved. > > > > INTERFACE COMMITMENT BINDING > > prstat(1m) output Uncommitted Patch > > > > This case proposes changing the "SIZE" column of "prstat -Z" zone > > output lines to "SWAP". The swap reported will be the total swap > > consumed by the zone's processes and tmpfs mounts. This value will > > assist administrators in monitoring the swap reserved by each zone, > > allowing them to choose a reasonable "zone.max-swap" settings. > > > > The "SIZE" column will also be changed to "SWAP" for prstat > > options a, T, and J, for users, tasks, and projects. > > The reason for not changing this column in the default output would be > helpful. I have a seperate private interface used by prstat(1m) to get aggregate swap reserved by users, tasks, projects, and zones. Default prstat output is "per-process", and the information is accessed via /proc. Currently, per-process, or per-address-space, swap reservation is not counted or made available via /proc. From proc(4): typedef struct psinfo { ... size_t pr_size; /* size of process image in Kbytes */ ... "size of process image" is pretty meaningless. If we can change "pr_size" to be "swap reserved by process", then we could change "SIZE" to "SWAP" for all prstat(1m) output. Would such a change to psinfo_t be reasonable? > > Currently a global or non-global zone can consume all swap > > resources available on the system, limiting the usefulness of zones > > as an application container. zone.max-swap provides a mechanism to > > I would rephrase that as "the container of an application" to avoid > confusion with the Solaris feature set called "Containers." I assume that > the former was meant moreso than the latter even though Containers are > Solaris' implementation of "an application container." I'm not sure what you mean, but ok. By "the Solaris feature set called Containers.", do you mean "zones + RM", or do you mean "zones, xen, ldoms". > > zone.max-swap will be configurable on both the global zone, and > > non-global zones. The affect on processes in a zone reaching its > > zone.max-swap limit is the same as if all system swap is reserved. > > Callers of mmap(2) and sbrk(2) will receive EAGAIN. Writes to > > tmpfs will return ENOSPC, which is the same errno returned when > > a tmpfs mount reaches it's "size" mount option. The "size" mount > > option limits the quantity of swap that a tmpfs mount can reserve. > > With S10 11/06, some zone limitations are now configurable, e.g. setting > the system time clock. Similarly, the ability to modify a zone's swap > limit could be given to the zone's root user, which might be valuable in > some situations. This would be analogous to the 'basic' privilege level. > It would allow an advisory limit to be placed on a zone - a limit that the > zone admin could modify in unusual circumstances. > > I realize that this opens a can of worms in that most rctls are protected > by the sys_res_config priv, which is not allowed in a zone even with 11/06. > Further, it makes sense to consistently allow or forbid rctl-modification > in zones. > > I just wanted to mention this idea so that it is not unintentionally > overlooked. Currently, all zone.* rctls are not modifiable from a non global zone. The established mechanism for a zone admin to set rctls within the zone is via project.* rctls set on projects within the zone. Granted, in the "zone.max-swap" case, we are not proposing a "project.max-swap", due to implementation complexity and risk. With sufficient customer damand, we could investigate implementing "project.max-swap" in the future. Currently no zone.* rctls allow "basic" rctl values to be set. The only project.* rctl which allows basic is "project.max-contracts", and perhaps that is a bug. A "basic" rctl is an unprivileged rctl that only affects the process within the task, project, or zone which sets it. It is pretty useless, except for process.* rctls. I'd be happy to address the general issues of privilege related to project and zone rctls as a seperate case. A possible solution may be to redefine "basic" for project and zone rctls, and/or introduce more fine grained privileges. I agree that work is needed here. > > STATISTIC DESCRIPTION > > zonename The name of the zone with {zoneid} > > swap reserved: swap reserved by zone in bytes. > > Does swap_reserved include pages shared with other zones, e.g. text pages? Each process mapping text reserves unique swap for that mapping. Even though the underlying physical page may be shared between processes/zones, each process needs it's own swap reservation. This is because each process may cow the page, and then may need to page the private copy to disk. > > > max_swap_reserved: current zone.max-swap limit in bytes, > > locked_memory: physical memory locked by zone in bytes. > > Does locked_memory include pages shared with other zones, e.g. text pages? locked pages are charged to each mlock-er. If a process in zone-A locks a text page shared by a process in zone-B, only zone-A is charged. If zone-B then locks the page, both zone-A and zone-B will have a charge. The exception is system V shared memory. When system V shared memory is locked, the zone in which the shmget() was done will be charged. Additional locks to the same system V shared memory by other processes will not cause additional charge. This is discussed in [4]. Thanks very much for the comments! -Steve L. From dp@snowdog.eng.sun.com Thu Oct 26 16:37:17 2006 Received: from sunmail2.sfbay.sun.com (sunmail2.SFBay.Sun.COM [129.149.246.180]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id k9QNbHKl026439 for ; Thu, 26 Oct 2006 16:37:17 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2.sfbay.sun.com (8.11.7p1+Sun/8.11.7/ENSMAIL,v2.2) with ESMTP id k9QNbH317005 for <@sunmail1brm.central.sun.com:PSARC-EXT@sun.com>; Thu, 26 Oct 2006 16:37:17 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0J7R00N0DOA44Q00@brm-avmta-1.central.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Thu, 26 Oct 2006 17:37:16 -0600 (MDT) Received: from engmail3mpk.sfbay.Sun.COM ([129.146.11.26]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0J7R00GFYOA3F660@brm-avmta-1.central.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Thu, 26 Oct 2006 17:37:15 -0600 (MDT) Received: from snowdog.eng.sun.com (snowdog.SFBay.Sun.COM [129.146.228.213]) by engmail3mpk.sfbay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id k9QNbEXP015499; Thu, 26 Oct 2006 16:37:14 -0700 (PDT) Received: from snowdog.eng.sun.com (localhost [127.0.0.1]) by snowdog.eng.sun.com (8.13.7+Sun/8.13.7) with ESMTP id k9QNbDTt003262; Thu, 26 Oct 2006 16:37:14 -0700 (PDT) Received: (from dp@localhost) by snowdog.eng.sun.com (8.13.7+Sun/8.13.7/Submit) id k9QNbDnl003261; Thu, 26 Oct 2006 16:37:13 -0700 (PDT) Date: Thu, 26 Oct 2006 16:37:13 -0700 From: Dan Price Subject: Re: [zones-discuss] PSARC/2006/598 Swap resource control; locked memory RM improvements In-reply-to: <20061026185007.GN2475@steve1.eng.sun.com> To: Steve Lawrence Cc: Jeff Victor , PSARC-EXT@sun.com, zones-discuss@opensolaris.org Message-id: <20061026233712.GC804@eng.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <20061025214631.GA23100@eng.sun.com> <45401037.5000806@sun.com> <20061026185007.GN2475@steve1.eng.sun.com> User-Agent: Mutt/1.4.2.1i Status: RO Content-Length: 2554 On Thu 26 Oct 2006 at 11:50AM, Steve Lawrence wrote: > "size of process image" is pretty meaningless. If we can change "pr_size" to > be "swap reserved by process", then we could change "SIZE" to "SWAP" for all > prstat(1m) output. Would such a change to psinfo_t be reasonable? You'd have to check in with Roger, I think (and doing so would probably be worth doing anyway). Adding a new field might be feasible. > > > Currently a global or non-global zone can consume all swap > > > resources available on the system, limiting the usefulness of zones > > > as an application container. zone.max-swap provides a mechanism to > > > > I would rephrase that as "the container of an application" to avoid > > confusion with the Solaris feature set called "Containers." I assume that > > the former was meant moreso than the latter even though Containers are > > Solaris' implementation of "an application container." > > I'm not sure what you mean, but ok. By "the Solaris feature set called > Containers.", do you mean "zones + RM", or do you mean "zones, xen, ldoms". Steve, I think the text is fine. This document isn't intended for consumption by customers, and the text is clear enough to anyone trying to absorb its meaning. > > Similarly, the ability to modify a zone's swap > > limit could be given to the zone's root user, which might be valuable in > > some situations. This would be analogous to the 'basic' privilege level. > > It would allow an advisory limit to be placed on a zone - a limit that the > > zone admin could modify in unusual circumstances. > > > > I just wanted to mention this idea so that it is not unintentionally > > overlooked. > > Currently, all zone.* rctls are not modifiable from a non global zone. > > The established mechanism for a zone admin to set rctls within the > zone is via project.* rctls set on projects within the zone. Granted, in > the "zone.max-swap" case, we are not proposing a "project.max-swap", due to > implementation complexity and risk. With sufficient customer damand, we could > investigate implementing "project.max-swap" in the future. I think I'd agree that allowing a zone to modify its own zone.* rctls (perhaps only to lower them) is something we *could do* at some point. But I'm aware of neither an RFE for this nor stated customer demand. If someone wants this, then let's get that recorded as an RFE in the bug database, please. Thanks, -dp -- Daniel Price - Solaris Kernel Engineering - dp@eng.sun.com - blogs.sun.com/dp From gww@eng.sun.com Tue Oct 31 10:56:19 2006 Received: from sunmail1brm.Central.Sun.COM (sunmail1brm.Central.Sun.COM [129.147.62.17]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id k9VIuJYg011143 for ; Tue, 31 Oct 2006 10:56:19 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.149.247.22]) by sunmail1brm.Central.Sun.COM (8.11.7p1+Sun/8.11.7/ENSMAIL,v2.2) with ESMTP id k9VIuIC20177 for <@sunmail3.sfbay.sun.com:PSARC-EXT@sun.com>; Tue, 31 Oct 2006 11:56:18 -0700 (MST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0J800070PKLTFR00@nwk-avmta-2.sfbay.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Tue, 31 Oct 2006 10:56:17 -0800 (PST) Received: from engmail3mpk.sfbay.Sun.COM ([129.146.11.26]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0J80001CZKLTZ780@nwk-avmta-2.sfbay.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Tue, 31 Oct 2006 10:56:17 -0800 (PST) Received: from marduk.eng.sun.com (marduk.SFBay.Sun.COM [129.146.108.224]) by engmail3mpk.sfbay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id k9VIuGn0007407; Tue, 31 Oct 2006 10:56:16 -0800 (PST) Received: from marduk.eng.sun.com (localhost [127.0.0.1]) by marduk.eng.sun.com (8.13.6+Sun/8.12.11) with ESMTP id k9VIxNhX013022; Tue, 31 Oct 2006 10:59:23 -0800 (PST) Received: (from gww@localhost) by marduk.eng.sun.com (8.13.6+Sun/8.12.11/Submit) id k9VIxNmL013021; Tue, 31 Oct 2006 10:59:23 -0800 (PST) Date: Tue, 31 Oct 2006 10:59:23 -0800 (PST) From: Gary Winiger Subject: Re: PSARC/2006/598 Swap resource control; locked memory RM improvements To: PSARC-EXT@sun.com, dp@eng.sun.com Cc: Stephen.Lawrence@sun.com, zones-discuss@opensolaris.org Message-id: <200610311859.k9VIxNmL013021@marduk.eng.sun.com> Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 1347 > DETAIL: > > 1. "zone.max-swap" resource control. > > Limits swap consumed by user process address space mappings and > tmpfs mounts within a zone. > zone.max-swap will be configurable on both the global zone, and > non-global zones. The affect on processes in a zone reaching its > zone.max-swap limit is the same as if all system swap is reserved. > Callers of mmap(2) and sbrk(2) will receive EAGAIN. Writes to > tmpfs will return ENOSPC, which is the same errno returned when > a tmpfs mount reaches it's "size" mount option. The "size" mount > option limits the quantity of swap that a tmpfs mount can reserve. > > While a low zone.max-swap setting for the global zone can lead to > a difficult-to-administer global zone, the same problem exists > today when configuring the zone.max-lwps resource control on the > global zone, or when all system swap is reserved. While other controls seem to be capped by the process/thread's project, it seems that use of swap like max-lwps might be important for the global zone project 0. Has consideration been given to allowing global zone "system" processes to not be bound by GZ zone.max-swap? It might be better than having to reboot or boot single to fix a problem that could otherwise be fixed in the GZ. Gary.. From sl108498@steve1.sfbay.sun.com Tue Oct 31 11:27:55 2006 Received: from sunmail1brm.Central.Sun.COM (sunmail1brm.Central.Sun.COM [129.147.62.17]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id k9VJRt34011881 for ; Tue, 31 Oct 2006 11:27:55 -0800 (PST) Received: from nwk-avmta-1.sfbay.sun.com (nwk-avmta-1.SFBay.Sun.COM [129.149.246.28]) by sunmail1brm.Central.Sun.COM (8.11.7p1+Sun/8.11.7/ENSMAIL,v2.2) with ESMTP id k9VJRsC04714 for <@sunmail2.sfbay.sun.com:PSARC-EXT@sun.com>; Tue, 31 Oct 2006 12:27:54 -0700 (MST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2 (built Dec 2 2004)) id <0J8000H0JM2FM000@nwk-avmta-1.sfbay.Sun.COM> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Tue, 31 Oct 2006 11:27:51 -0800 (PST) Received: from sfbaymail1sca.SFBay.Sun.COM ([129.145.154.35]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2 (built Dec 2 2004)) with ESMTP id <0J80003LUM2F32D0@nwk-avmta-1.sfbay.Sun.COM> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Tue, 31 Oct 2006 11:27:51 -0800 (PST) Received: from steve1.sfbay.sun.com (steve1.SFBay.Sun.COM [129.146.228.230]) by sfbaymail1sca.SFBay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id k9VJRopO018515; Tue, 31 Oct 2006 11:27:50 -0800 (PST) Received: from steve1.sfbay.sun.com (localhost [127.0.0.1]) by steve1.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id k9VJRUxs006582; Tue, 31 Oct 2006 11:27:30 -0800 (PST) Received: (from sl108498@localhost) by steve1.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id k9VJRTLb006581; Tue, 31 Oct 2006 11:27:29 -0800 (PST) Date: Tue, 31 Oct 2006 11:27:29 -0800 From: Steve Lawrence Subject: Re: PSARC/2006/598 Swap resource control; locked memory RM improvements In-reply-to: <200610311859.k9VIxNmL013021@marduk.eng.sun.com> To: Gary Winiger Cc: PSARC-EXT@sun.com, dp@eng.sun.com, Stephen.Lawrence@sun.com, zones-discuss@opensolaris.org Message-id: <20061031192726.GI2475@steve1.eng.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <200610311859.k9VIxNmL013021@marduk.eng.sun.com> User-Agent: Mutt/1.4.2.1i Status: RO Content-Length: 2185 On Tue, Oct 31, 2006 at 10:59:23AM -0800, Gary Winiger wrote: > > DETAIL: > > > > 1. "zone.max-swap" resource control. > > > > Limits swap consumed by user process address space mappings and > > tmpfs mounts within a zone. > > > zone.max-swap will be configurable on both the global zone, and > > non-global zones. The affect on processes in a zone reaching its > > zone.max-swap limit is the same as if all system swap is reserved. > > Callers of mmap(2) and sbrk(2) will receive EAGAIN. Writes to > > tmpfs will return ENOSPC, which is the same errno returned when > > a tmpfs mount reaches it's "size" mount option. The "size" mount > > option limits the quantity of swap that a tmpfs mount can reserve. > > > > While a low zone.max-swap setting for the global zone can lead to > > a difficult-to-administer global zone, the same problem exists > > today when configuring the zone.max-lwps resource control on the > > global zone, or when all system swap is reserved. > > While other controls seem to be capped by the process/thread's > project, it seems that use of swap like max-lwps might be > important for the global zone project 0. Has consideration been > given to allowing global zone "system" processes to not be > bound by GZ zone.max-swap? > It might be better than having to reboot or boot single to fix > a problem that could otherwise be fixed in the GZ. This is how we treat cpu-shares. project 0 in the global zone has "infinite" shares. This will not help root logins directly, but could by setting: usermod -K project=system root Would it be reasonable to propose special treatment of the global project 0 for all project and zone rctls? Once could argue that capping system daemons can only lead some sort of undesireable system failure. This would of course exempt all global zone system daemons from resource management. To mitigate this, SMF could be leveraged to run application daemons (or leaky/bad system daemons) in other projects. Perhaps this issue should be run as a seperate fasttrack? I need to investigate the implementation impact. -Steve > > Gary.. From gww@eng.sun.com Tue Oct 31 11:59:57 2006 Received: from sunmail1brm.Central.Sun.COM (sunmail1brm.Central.Sun.COM [129.147.62.17]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id k9VJxurs013594 for ; Tue, 31 Oct 2006 11:59:57 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.149.247.22]) by sunmail1brm.Central.Sun.COM (8.11.7p1+Sun/8.11.7/ENSMAIL,v2.2) with ESMTP id k9VJxuC16587 for <@sunmail3.sfbay.sun.com:PSARC-EXT@sun.com>; Tue, 31 Oct 2006 12:59:56 -0700 (MST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0J8000B1JNJTVQ00@nwk-avmta-2.sfbay.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Tue, 31 Oct 2006 11:59:53 -0800 (PST) Received: from engmail3mpk.sfbay.Sun.COM ([129.146.11.26]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0J80001FONJRYXE0@nwk-avmta-2.sfbay.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Tue, 31 Oct 2006 11:59:51 -0800 (PST) Received: from marduk.eng.sun.com (marduk.SFBay.Sun.COM [129.146.108.224]) by engmail3mpk.sfbay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id k9VJxoCL005653; Tue, 31 Oct 2006 11:59:50 -0800 (PST) Received: from marduk.eng.sun.com (localhost [127.0.0.1]) by marduk.eng.sun.com (8.13.6+Sun/8.12.11) with ESMTP id k9VK2wwM013231; Tue, 31 Oct 2006 12:02:58 -0800 (PST) Received: (from gww@localhost) by marduk.eng.sun.com (8.13.6+Sun/8.12.11/Submit) id k9VK2w2W013230; Tue, 31 Oct 2006 12:02:58 -0800 (PST) Date: Tue, 31 Oct 2006 12:02:58 -0800 (PST) From: Gary Winiger Subject: Re: PSARC/2006/598 Swap resource control; locked memory RM improvements To: gww@eng.sun.com, Stephen.Lawrence@sun.com Cc: PSARC-EXT@sun.com, dp@eng.sun.com, Stephen.Lawrence@sun.com, zones-discuss@opensolaris.org Message-id: <200610312002.k9VK2w2W013230@marduk.eng.sun.com> Content-transfer-encoding: 7BIT X-Sun-Charset: US-ASCII X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 1242 > This is how we treat cpu-shares. project 0 in the global zone has "infinite" > shares. > > This will not help root logins directly, but could by setting: > > usermod -K project=system root Or perhaps deliver root's entry this way to start with. > Would it be reasonable to propose special treatment of the global project 0 > for all project and zone rctls? Once could argue that capping system daemons > can only lead some sort of undesireable system failure. > > This would of course exempt all global zone system daemons from resource > management. To mitigate this, SMF could be leveraged to run application > daemons (or leaky/bad system daemons) in other projects. leaky or bad system daemons should be fixed, but defining projects may well be a workaround ;-} > Perhaps this issue should be run as a seperate fasttrack? I need to > investigate the implementation impact. I'm looking for this case to define how to preserve the current model of "unlimited" unless one asks for a limit model in the global zone. I believe it is important from a system integrity and maintenance perspective. Other's may have different opinions. If there is a compelling reason to deliver in phases, please discuss that. Gary.. From sl108498@steve1.sfbay.sun.com Tue Oct 31 14:01:47 2006 Received: from sunmail1brm.Central.Sun.COM (sunmail1brm.Central.Sun.COM [129.147.62.17]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id k9VM1lT8017188 for ; Tue, 31 Oct 2006 14:01:47 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.149.247.22]) by sunmail1brm.Central.Sun.COM (8.11.7p1+Sun/8.11.7/ENSMAIL,v2.2) with ESMTP id k9VM1lC11990 for <@sunmail3.sfbay.sun.com:PSARC-EXT@sun.com>; Tue, 31 Oct 2006 15:01:47 -0700 (MST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0J8000K01T6UPM00@nwk-avmta-2.sfbay.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Tue, 31 Oct 2006 14:01:42 -0800 (PST) Received: from sfbaymail1sca.SFBay.Sun.COM ([129.145.154.35]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0J8000D70T6SI990@nwk-avmta-2.sfbay.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Tue, 31 Oct 2006 14:01:41 -0800 (PST) Received: from steve1.sfbay.sun.com (steve1.SFBay.Sun.COM [129.146.228.230]) by sfbaymail1sca.SFBay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id k9VM1ebR003787; Tue, 31 Oct 2006 14:01:40 -0800 (PST) Received: from steve1.sfbay.sun.com (localhost [127.0.0.1]) by steve1.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id k9VM1Km4006677; Tue, 31 Oct 2006 14:01:20 -0800 (PST) Received: (from sl108498@localhost) by steve1.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id k9VM1KBv006676; Tue, 31 Oct 2006 14:01:20 -0800 (PST) Date: Tue, 31 Oct 2006 14:01:20 -0800 From: Steve Lawrence Subject: Re: PSARC/2006/598 Swap resource control; locked memory RM improvements In-reply-to: <200610312002.k9VK2w2W013230@marduk.eng.sun.com> To: Gary Winiger Cc: stephen.lawrence@sun.com, PSARC-EXT@sun.com, dp@eng.sun.com, zones-discuss@opensolaris.org Message-id: <20061031220120.GK2475@steve1.eng.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <200610312002.k9VK2w2W013230@marduk.eng.sun.com> User-Agent: Mutt/1.4.2.1i Status: RO Content-Length: 2039 On Tue, Oct 31, 2006 at 12:02:58PM -0800, Gary Winiger wrote: > > This is how we treat cpu-shares. project 0 in the global zone has "infinite" > > shares. > > > > This will not help root logins directly, but could by setting: > > > > usermod -K project=system root > > Or perhaps deliver root's entry this way to start with. Would that be a reasonable change to make via patch? Perhaps this change could be delivered to nevada, but not backported. It would be confusing to deliver this change, and also deliver the "user.root" project. If we made root's default project "system", then the "user.root" project should be removed. "user.root" is kind of a bug anyhow, as SMF does not run root services in "user.root". Currently, only root processes spawned by login/pam run in "user.root". > > > Would it be reasonable to propose special treatment of the global project 0 > > for all project and zone rctls? Once could argue that capping system daemons > > can only lead some sort of undesireable system failure. > > > > This would of course exempt all global zone system daemons from resource > > management. To mitigate this, SMF could be leveraged to run application > > daemons (or leaky/bad system daemons) in other projects. > > leaky or bad system daemons should be fixed, but defining projects > may well be a workaround ;-} Agreed. > > > Perhaps this issue should be run as a seperate fasttrack? I need to > > investigate the implementation impact. > > I'm looking for this case to define how to preserve the current > model of "unlimited" unless one asks for a limit model in the > global zone. I believe it is important from a system integrity and > maintenance perspective. Other's may have different opinions. > If there is a compelling reason to deliver in phases, please discuss > that. The global zone will have no swap limit by default. The default zone.max-swap rctl delivered on the global zone is UINT64_MAX, which is essentially unlimited. Is that what you mean? -Steve > > Gary.. From gww@eng.sun.com Tue Oct 31 14:58:51 2006 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id k9VMwotR018618 for ; Tue, 31 Oct 2006 14:58:51 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id k9VMwicd028862 for <@sunmail1brm.central.sun.com:PSARC-EXT@sun.com>; Wed, 1 Nov 2006 06:58:49 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0J8000K1NVU0FP00@brm-avmta-1.central.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Tue, 31 Oct 2006 15:58:48 -0700 (MST) Received: from engmail3mpk.sfbay.Sun.COM ([129.146.11.26]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0J8000IIQVTXPD30@brm-avmta-1.central.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Tue, 31 Oct 2006 15:58:45 -0700 (MST) Received: from marduk.eng.sun.com (marduk.SFBay.Sun.COM [129.146.108.224]) by engmail3mpk.sfbay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id k9VMwijO002923; Tue, 31 Oct 2006 14:58:44 -0800 (PST) Received: from marduk.eng.sun.com (localhost [127.0.0.1]) by marduk.eng.sun.com (8.13.6+Sun/8.12.11) with ESMTP id k9VN1qSv013618; Tue, 31 Oct 2006 15:01:52 -0800 (PST) Received: (from gww@localhost) by marduk.eng.sun.com (8.13.6+Sun/8.12.11/Submit) id k9VN1qln013617; Tue, 31 Oct 2006 15:01:52 -0800 (PST) Date: Tue, 31 Oct 2006 15:01:52 -0800 (PST) From: Gary Winiger Subject: Re: PSARC/2006/598 Swap resource control; locked memory RM improvements To: gww@eng.sun.com, stephen.lawrence@sun.com Cc: PSARC-EXT@sun.com, dp@eng.sun.com, zones-discuss@opensolaris.org Message-id: <200610312301.k9VN1qln013617@marduk.eng.sun.com> Content-transfer-encoding: 7BIT X-Sun-Charset: US-ASCII X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 2225 > > > This will not help root logins directly, but could by setting: > > > > > > usermod -K project=system root > > > > Or perhaps deliver root's entry this way to start with. > > Would that be a reasonable change to make via patch? Perhaps this change > could be delivered to nevada, but not backported. > > It would be confusing to deliver this change, and also deliver the "user.root" > project. If we made root's default project "system", then the "user.root" > project should be removed. "user.root" is kind of a bug anyhow, as SMF does > not run root services in "user.root". Currently, only root processes spawned > by login/pam run in "user.root". > > > Perhaps this issue should be run as a seperate fasttrack? I need to > > > investigate the implementation impact. > > > > I'm looking for this case to define how to preserve the current > > model of "unlimited" unless one asks for a limit model in the > > global zone. I believe it is important from a system integrity and > > maintenance perspective. Other's may have different opinions. > > If there is a compelling reason to deliver in phases, please discuss > > that. > > The global zone will have no swap limit by default. The default zone.max-swap > rctl delivered on the global zone is UINT64_MAX, which is essentially > unlimited. Is that what you mean? My point(s) here is not so much how things get done, but that the global zone is in some ways special. IIRC, before this project, the GZ doesn't have a swap limit. After this project an administrator could set swap limit on the GZ. Granted this is administrative action and they get what they deserve/ask for. However, it seemed to me that part of this case "should" (my judgement) include some way to override the limit in case override is really desired. As implied, perhaps by putting root into project 0 at login or as part of daemon/service start is a way to bypass the administrator's choice in the GZ for some processes. What I didn't see as part of this case is the architecture to allow this bypass. Perhaps I'm off base for thinking it's necessary to protect against inadvertantly not being able to administer the system from the GZ. Gary.. From sl108498@steve1.sfbay.sun.com Tue Oct 31 15:24:42 2006 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id k9VNOgiN019331 for ; Tue, 31 Oct 2006 15:24:42 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.149.247.22]) by sunmail5.uk.sun.com (8.13.7+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id k9VNOQ8E000454 for <@sunmail3.sfbay.sun.com:PSARC-EXT@sun.com>; Tue, 31 Oct 2006 23:24:41 GMT Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0J800031HX13YQ00@nwk-avmta-2.sfbay.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Tue, 31 Oct 2006 15:24:39 -0800 (PST) Received: from sfbaymail2sca.sfbay.sun.com ([129.145.155.42]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0J80003D6X12I400@nwk-avmta-2.sfbay.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Tue, 31 Oct 2006 15:24:38 -0800 (PST) Received: from steve1.sfbay.sun.com (steve1.SFBay.Sun.COM [129.146.228.230]) by sfbaymail2sca.sfbay.sun.com (8.13.6+Sun/8.12.10/ENSMAIL,v2.2) with ESMTP id k9VNOcvN017778; Tue, 31 Oct 2006 15:24:38 -0800 (PST) Received: from steve1.sfbay.sun.com (localhost [127.0.0.1]) by steve1.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id k9VNOIoY006713; Tue, 31 Oct 2006 15:24:18 -0800 (PST) Received: (from sl108498@localhost) by steve1.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id k9VNOIi0006712; Tue, 31 Oct 2006 15:24:18 -0800 (PST) Date: Tue, 31 Oct 2006 15:24:18 -0800 From: Steve Lawrence Subject: Re: PSARC/2006/598 Swap resource control; locked memory RM improvements In-reply-to: <200610312301.k9VN1qln013617@marduk.eng.sun.com> To: Gary Winiger Cc: stephen.lawrence@sun.com, PSARC-EXT@sun.com, dp@eng.sun.com, zones-discuss@opensolaris.org Message-id: <20061031232418.GM2475@steve1.eng.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <200610312301.k9VN1qln013617@marduk.eng.sun.com> User-Agent: Mutt/1.4.2.1i Status: RO Content-Length: 2093 > > > I'm looking for this case to define how to preserve the current > > > model of "unlimited" unless one asks for a limit model in the > > > global zone. I believe it is important from a system integrity and > > > maintenance perspective. Other's may have different opinions. > > > If there is a compelling reason to deliver in phases, please discuss > > > that. > > > > The global zone will have no swap limit by default. The default zone.max-swap > > rctl delivered on the global zone is UINT64_MAX, which is essentially > > unlimited. Is that what you mean? > > My point(s) here is not so much how things get done, but that > the global zone is in some ways special. IIRC, before this > project, the GZ doesn't have a swap limit. After this project > an administrator could set swap limit on the GZ. Granted this > is administrative action and they get what they deserve/ask for. > However, it seemed to me that part of this case "should" (my > judgement) include some way to override the limit in case > override is really desired. As implied, perhaps by putting > root into project 0 at login or as part of daemon/service start > is a way to bypass the administrator's choice in the GZ for > some processes. What I didn't see as part of this case is > the architecture to allow this bypass. Perhaps I'm off base > for thinking it's necessary to protect against inadvertantly > not being able to administer the system from the GZ. > It seems reasonable to amend this case to say: 1. Any process with priv_sys_resource running in the global zone's system project (project 0) will not subject to project.* or zone.* resource controls. System daemons which wish to be subject to the global zone's resource controls can drop priv_sys_resource. 2. The "user.root" project will be removed, and root's default project will be set to the "system" project via /etc/user_attr. I'm not sure if (2) can be delivered via patch. I need some guidance here. I'm also not sure how implementable (1) is until I do more investigation. -Steve > Gary.. > From dp@snowdog.eng.sun.com Tue Oct 31 15:28:41 2006 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id k9VNSd3I019349 for ; Tue, 31 Oct 2006 15:28:40 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.149.247.22]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id k9VNScO7013919 for <@sunmail3.sfbay.sun.com:PSARC-EXT@sun.com>; Wed, 1 Nov 2006 07:28:39 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0J800042FX7Q5800@nwk-avmta-2.sfbay.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Tue, 31 Oct 2006 15:28:38 -0800 (PST) Received: from engmail3mpk.sfbay.Sun.COM ([129.146.11.26]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0J80003OSX7LHQ00@nwk-avmta-2.sfbay.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Tue, 31 Oct 2006 15:28:33 -0800 (PST) Received: from snowdog.eng.sun.com (snowdog.SFBay.Sun.COM [129.146.228.213]) by engmail3mpk.sfbay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id k9VNSW2Q018208; Tue, 31 Oct 2006 15:28:32 -0800 (PST) Received: from snowdog.eng.sun.com (localhost [127.0.0.1]) by snowdog.eng.sun.com (8.13.8+Sun/8.13.8) with ESMTP id k9VNSVEL008494; Tue, 31 Oct 2006 15:28:31 -0800 (PST) Received: (from dp@localhost) by snowdog.eng.sun.com (8.13.8+Sun/8.13.8/Submit) id k9VNSVfS008493; Tue, 31 Oct 2006 15:28:31 -0800 (PST) Date: Tue, 31 Oct 2006 15:28:31 -0800 From: Dan Price Subject: Re: PSARC/2006/598 Swap resource control; locked memory RM improvements In-reply-to: <20061031232418.GM2475@steve1.eng.sun.com> To: Steve Lawrence Cc: Gary Winiger , PSARC-EXT@sun.com, zones-discuss@opensolaris.org Message-id: <20061031232830.GR3101@eng.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <200610312301.k9VN1qln013617@marduk.eng.sun.com> <20061031232418.GM2475@steve1.eng.sun.com> User-Agent: Mutt/1.4.2.1i Status: RO Content-Length: 829 On Tue 31 Oct 2006 at 03:24PM, Steve Lawrence wrote: > It seems reasonable to amend this case to say: > > 1. > Any process with priv_sys_resource running in the global zone's > system project (project 0) will not subject to project.* or zone.* > resource controls. System daemons which wish to be subject to the > global zone's resource controls can drop priv_sys_resource. This "feels" risky for a patch... the effect here is to "un-manage" things which the customer may be expecting to be resource managed, potentially including a workload. Or is the point that no-one using RM will be relying on the system project to limit things? This implies that can't give the system project 10 (or 100) shares after this proposal? -dp -- Daniel Price - Solaris Kernel Engineering - dp@eng.sun.com - blogs.sun.com/dp From sl108498@steve1.sfbay.sun.com Tue Oct 31 16:10:57 2006 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id kA10AvYS020600 for ; Tue, 31 Oct 2006 16:10:57 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.149.247.22]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id kA10AueD017512 for <@sunmail3.sfbay.sun.com:PSARC-EXT@sun.com>; Tue, 31 Oct 2006 16:10:57 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0J8000703Z68NU00@nwk-avmta-2.sfbay.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Tue, 31 Oct 2006 16:10:56 -0800 (PST) Received: from sfbaymail2sca.sfbay.sun.com ([129.145.155.42]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0J80003O4Z64HM30@nwk-avmta-2.sfbay.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Tue, 31 Oct 2006 16:10:52 -0800 (PST) Received: from steve1.sfbay.sun.com (steve1.SFBay.Sun.COM [129.146.228.230]) by sfbaymail2sca.sfbay.sun.com (8.13.6+Sun/8.12.10/ENSMAIL,v2.2) with ESMTP id kA10AqNw009626; Tue, 31 Oct 2006 16:10:52 -0800 (PST) Received: from steve1.sfbay.sun.com (localhost [127.0.0.1]) by steve1.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id kA10AXBQ006806; Tue, 31 Oct 2006 16:10:33 -0800 (PST) Received: (from sl108498@localhost) by steve1.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id kA10AXZR006805; Tue, 31 Oct 2006 16:10:33 -0800 (PST) Date: Tue, 31 Oct 2006 16:10:33 -0800 From: Steve Lawrence Subject: Re: PSARC/2006/598 Swap resource control; locked memory RM improvements In-reply-to: <20061031232830.GR3101@eng.sun.com> To: Dan Price Cc: Steve Lawrence , Gary Winiger , PSARC-EXT@sun.com, zones-discuss@opensolaris.org Message-id: <20061101001032.GN2475@steve1.eng.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <200610312301.k9VN1qln013617@marduk.eng.sun.com> <20061031232418.GM2475@steve1.eng.sun.com> <20061031232830.GR3101@eng.sun.com> User-Agent: Mutt/1.4.2.1i Status: RO Content-Length: 1465 On Tue, Oct 31, 2006 at 03:28:31PM -0800, Dan Price wrote: > On Tue 31 Oct 2006 at 03:24PM, Steve Lawrence wrote: > > It seems reasonable to amend this case to say: > > > > 1. > > Any process with priv_sys_resource running in the global zone's > > system project (project 0) will not subject to project.* or zone.* > > resource controls. System daemons which wish to be subject to the > > global zone's resource controls can drop priv_sys_resource. > > This "feels" risky for a patch... the effect here is to "un-manage" > things which the customer may be expecting to be resource managed, > potentially including a workload. Or is the point that no-one using > RM will be relying on the system project to limit things? > > This implies that can't give the system project 10 (or 100) shares > after this proposal? For shares, you can't do this now anyway. The system project implicitly has infinite shares, regardless of what shares you have set. This is documented in FSS(7): By default, project system (project ID 0) includes all sys- tem daemons started by initialization scripts and has an "unlimited" amount of shares. That is, it is always scheduled first no matter how many shares are given to other projects. I've proposed extending this for all resource controls, not just *.cpu-shares. -Steve > > -dp > > -- > Daniel Price - Solaris Kernel Engineering - dp@eng.sun.com - blogs.sun.com/dp From dp@snowdog.eng.sun.com Tue Oct 31 16:12:02 2006 Received: from sunmail3.sfbay.sun.com (sunmail3.SFBay.Sun.COM [129.149.247.180]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id kA10C2pA020625 for ; Tue, 31 Oct 2006 16:12:02 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.149.247.22]) by sunmail3.sfbay.sun.com (8.11.7p1+Sun/8.11.7/ENSMAIL,v2.2) with ESMTP id kA10C0824343 for <@sunmail3.sfbay.sun.com:PSARC-EXT@sun.com>; Tue, 31 Oct 2006 16:12:00 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0J8000709Z7XPM00@nwk-avmta-2.sfbay.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Tue, 31 Oct 2006 16:11:57 -0800 (PST) Received: from engmail3mpk.sfbay.Sun.COM ([129.146.11.26]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0J80003R0Z7WHM30@nwk-avmta-2.sfbay.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Tue, 31 Oct 2006 16:11:57 -0800 (PST) Received: from snowdog.eng.sun.com (snowdog.SFBay.Sun.COM [129.146.228.213]) by engmail3mpk.sfbay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id kA10BtaN003965; Tue, 31 Oct 2006 16:11:56 -0800 (PST) Received: from snowdog.eng.sun.com (localhost [127.0.0.1]) by snowdog.eng.sun.com (8.13.8+Sun/8.13.8) with ESMTP id kA10Bs4o008628; Tue, 31 Oct 2006 16:11:54 -0800 (PST) Received: (from dp@localhost) by snowdog.eng.sun.com (8.13.8+Sun/8.13.8/Submit) id kA10BsCC008627; Tue, 31 Oct 2006 16:11:54 -0800 (PST) Date: Tue, 31 Oct 2006 16:11:54 -0800 From: Dan Price Subject: Re: PSARC/2006/598 Swap resource control; locked memory RM improvements In-reply-to: <20061101001032.GN2475@steve1.eng.sun.com> To: Steve Lawrence Cc: Gary Winiger , PSARC-EXT@sun.com, zones-discuss@opensolaris.org Message-id: <20061101001154.GT3101@eng.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <200610312301.k9VN1qln013617@marduk.eng.sun.com> <20061031232418.GM2475@steve1.eng.sun.com> <20061031232830.GR3101@eng.sun.com> <20061101001032.GN2475@steve1.eng.sun.com> User-Agent: Mutt/1.4.2.1i Status: RO Content-Length: 1616 On Tue 31 Oct 2006 at 04:10PM, Steve Lawrence wrote: > On Tue, Oct 31, 2006 at 03:28:31PM -0800, Dan Price wrote: > > On Tue 31 Oct 2006 at 03:24PM, Steve Lawrence wrote: > > > It seems reasonable to amend this case to say: > > > > > > 1. > > > Any process with priv_sys_resource running in the global zone's > > > system project (project 0) will not subject to project.* or zone.* > > > resource controls. System daemons which wish to be subject to the > > > global zone's resource controls can drop priv_sys_resource. > > > > This "feels" risky for a patch... the effect here is to "un-manage" > > things which the customer may be expecting to be resource managed, > > potentially including a workload. Or is the point that no-one using > > RM will be relying on the system project to limit things? > > > > This implies that can't give the system project 10 (or 100) shares > > after this proposal? > > For shares, you can't do this now anyway. The system project implicitly has > infinite shares, regardless of what shares you have set. This is documented > in FSS(7): > > By default, project system (project ID 0) includes all sys- > tem daemons started by initialization scripts and has an > "unlimited" amount of shares. That is, it is always > scheduled first no matter how many shares are given to other > projects. > > I've proposed extending this for all resource controls, not just *.cpu-shares. My apologies... I should've done some more digging first. -dp -- Daniel Price - Solaris Kernel Engineering - dp@eng.sun.com - blogs.sun.com/dp From mbarto@logiqwest.com Tue Oct 31 18:09:39 2006 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id kA129ctc022812 for ; Tue, 31 Oct 2006 18:09:39 -0800 (PST) Received: from nwk-avmta-1.sfbay.sun.com (nwk-avmta-1.SFBay.Sun.COM [129.149.246.28]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id kA129UnX028537; Wed, 1 Nov 2006 10:09:35 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2 (built Dec 2 2004)) id <0J8100C0F4NX5F00@nwk-avmta-1.sfbay.Sun.COM>; Tue, 31 Oct 2006 18:09:33 -0800 (PST) Received: from nwkea-mail-2.sun.com ([192.18.42.14]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2 (built Dec 2 2004)) with ESMTP id <0J8100E7U4NX6HB0@nwk-avmta-1.sfbay.Sun.COM>; Tue, 31 Oct 2006 18:09:33 -0800 (PST) Received: from relay1.sun.com (relay1.sun.com [150.143.103.14] (may be forged)) by nwkea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id kA11iG4X024448; Tue, 31 Oct 2006 18:09:32 -0800 (PST) Received: from mms06es.sun.com ([150.143.104.114] [150.143.104.114]) by relay1.sun.com with ESMTP; Wed, 01 Nov 2006 02:09:32 +0000 (Z) Received: from mms02bas.mms.us.syntegra.com (mms02bas.mms.us.syntegra.com [150.143.103.20]) by mms06es.sun.com with ESMTP; Wed, 01 Nov 2006 02:09:28 +0000 (Z) Received: from rrcs-mta-01.hrndva.rr.com ([24.28.200.153] [24.28.200.153]) by relay2.sun.com with ESMTP; Wed, 01 Nov 2006 02:00:12 +0000 (Z) Received: from rrcs-fep-10.hrndva.rr.com (rrcs-fep-10b.hrndva.rr.com [172.28.200.148]) by rrcs-mta-01.hrndva.rr.com (8.13.5+Sun/8.12.10) with ESMTP id kA11wPub002705; Tue, 31 Oct 2006 20:58:25 -0500 (EST) Received: from [192.168.3.17] (really [24.199.42.162]) by rrcs-fep-10.hrndva.rr.com with ESMTP id <20061101015825.BBHG16607.rrcs-fep-10.hrndva.rr.com@[192.168.3.17]>; Tue, 31 Oct 2006 20:58:25 -0500 Date: Tue, 31 Oct 2006 17:58:24 -0800 From: Michael Barto Subject: Re: [zones-discuss] Re: PSARC/2006/598 Swap resource control; locked memory RM improvements In-reply-to: <200610312301.k9VN1qln013617@marduk.eng.sun.com> To: Gary Winiger Cc: stephen.lawrence@sun.com, PSARC-EXT@sun.com, zones-discuss@opensolaris.org Reply-to: mbarto@logiqwest.com Message-id: <4547FF40.8050707@logiqwest.com> Organization: LogiQwest MIME-version: 1.0 Content-type: multipart/alternative; boundary="Boundary_(ID_c4dUZTtGDtjBjJDVBPbZkw)" X-Accept-Language: en-us, en X-PMX-Version: 5.2.0.264296 References: <200610312301.k9VN1qln013617@marduk.eng.sun.com> User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7.12) Gecko/20050915 Status: RO Content-Length: 10567 This is a multi-part message in MIME format. --Boundary_(ID_c4dUZTtGDtjBjJDVBPbZkw) Content-type: text/plain; charset=us-ascii; format=flowed Content-transfer-encoding: 7BIT After all thus juggling, let make it simple for the system admin and use some sort of fair share process to assignment and manage the swap for all the zones. Personally I think that the global zone should use minimum resources and be considered in the IT management processes to be only like a system controller on a complex server. Keep your applications out of the global zone!!! Gary Winiger wrote: >>>>This will not help root logins directly, but could by setting: >>>> >>>> usermod -K project=system root >>>> >>>> >>> Or perhaps deliver root's entry this way to start with. >>> >>> >>Would that be a reasonable change to make via patch? Perhaps this change >>could be delivered to nevada, but not backported. >> >>It would be confusing to deliver this change, and also deliver the "user.root" >>project. If we made root's default project "system", then the "user.root" >>project should be removed. "user.root" is kind of a bug anyhow, as SMF does >>not run root services in "user.root". Currently, only root processes spawned >>by login/pam run in "user.root". >> >> > > > > >>>>Perhaps this issue should be run as a seperate fasttrack? I need to >>>>investigate the implementation impact. >>>> >>>> >>> I'm looking for this case to define how to preserve the current >>> model of "unlimited" unless one asks for a limit model in the >>> global zone. I believe it is important from a system integrity and >>> maintenance perspective. Other's may have different opinions. >>> If there is a compelling reason to deliver in phases, please discuss >>> that. >>> >>> >>The global zone will have no swap limit by default. The default zone.max-swap >>rctl delivered on the global zone is UINT64_MAX, which is essentially >>unlimited. Is that what you mean? >> >> > > My point(s) here is not so much how things get done, but that > the global zone is in some ways special. IIRC, before this > project, the GZ doesn't have a swap limit. After this project > an administrator could set swap limit on the GZ. Granted this > is administrative action and they get what they deserve/ask for. > However, it seemed to me that part of this case "should" (my > judgement) include some way to override the limit in case > override is really desired. As implied, perhaps by putting > root into project 0 at login or as part of daemon/service start > is a way to bypass the administrator's choice in the GZ for > some processes. What I didn't see as part of this case is > the architecture to allow this bypass. Perhaps I'm off base > for thinking it's necessary to protect against inadvertantly > not being able to administer the system from the GZ. > >Gary.. > >_______________________________________________ >zones-discuss mailing list >zones-discuss@opensolaris.org > > > -- ------------------------------------------------------------------------ *Michael Barto* Software Architect LogiQwest Circle LogiQwest Inc. 16458 Bolsa Chica Street, # 15 Huntington Beach, CA 92649 http://www.logiqwest.com/ mbarto@logiqwest.com Tel: 714 377 3705 Fax: 714 840 3937 Cell: 714 883 1949 *'tis a gift to be simple* ------------------------------------------------------------------------ This e-mail may contain LogiQwest proprietary information and should be treated as confidential. --Boundary_(ID_c4dUZTtGDtjBjJDVBPbZkw) Content-type: multipart/related; boundary="Boundary_(ID_JJIBbfzti8zCVp1+Dh3eLg)" --Boundary_(ID_JJIBbfzti8zCVp1+Dh3eLg) Content-type: text/html; charset=us-ascii Content-transfer-encoding: 7BIT After all thus juggling, let make it simple for the system admin and use some sort of fair share process to assignment and manage the swap for all the zones. Personally I think that the global zone should use minimum resources and be considered in the IT management processes to be only like a system controller on a complex server. Keep your applications out of the global zone!!!

Gary Winiger wrote:
This will not help root logins directly, but could by setting:

	usermod -K project=system root
        
	Or perhaps deliver root's entry this way to start with.
      
Would that be a reasonable change to make via patch?  Perhaps this change
could be delivered to nevada, but not backported.

It would be confusing to deliver this change, and also deliver the "user.root"
project.  If we made root's default project "system", then the "user.root"
project should be removed.  "user.root" is kind of a bug anyhow, as SMF does
not run root services in "user.root".  Currently, only root processes spawned
by login/pam run in "user.root".
    


  
Perhaps this issue should be run as a seperate fasttrack?  I need to 
investigate the implementation impact.
        
	I'm looking for this case to define how to preserve the current
	model of "unlimited" unless one asks for a limit model in the
	global zone.  I believe it is important from a system integrity and
	maintenance perspective.  Other's may have different opinions.
	If there is a compelling reason to deliver in phases, please discuss
	that.
      
The global zone will have no swap limit by default.  The default zone.max-swap
rctl delivered on the global zone is UINT64_MAX, which is essentially
unlimited.  Is that what you mean?
    

	My point(s) here is not so much how things get done, but that
	the global zone is in some ways special.  IIRC, before this
	project, the GZ doesn't have a swap limit.  After this project
	an administrator could set swap limit on the GZ.  Granted this
	is administrative action and they get what they deserve/ask for.
	However, it seemed to me that part of this case "should" (my
	judgement) include some way to override the limit in case 
	override is really desired.  As implied, perhaps by putting
	root into project 0 at login or as part of daemon/service start
	is a way to bypass the administrator's choice in the GZ for
	some processes.  What I didn't see as part of this case is
	the architecture to allow this bypass.  Perhaps I'm off base
	for thinking it's necessary to protect against inadvertantly
	not being able to administer the system from the GZ.

Gary..
	
_______________________________________________
zones-discuss mailing list
zones-discuss@opensolaris.org

--
Michael Barto
Software Architect

 LogiQwest Circle
LogiQwest Inc.
16458 Bolsa Chica Street, # 15
Huntington Beach, CA  92649
http://www.logiqwest.com/

     mbarto@logiqwest.com
Tel:  714 377 3705
Fax: 714 840 3937
Cell: 714 883 1949
'tis a gift to be simple
This e-mail may contain LogiQwest proprietary information and should be treated as confidential.
--Boundary_(ID_JJIBbfzti8zCVp1+Dh3eLg) Content-id: Content-type: image/gif; x-mac-type=47494666; x-mac-creator=70727677; name=circle.gif Content-transfer-encoding: base64 Content-disposition: inline; filename=circle.gif R0lGODlhUQBQAJEAAAAAAP////1HA////yH5BAEAAAMALAAAAABRAFAAAAL/jI+pyybSopy0 OvWy3bwfmGSPR5bMGIrmaqEhpqWgM7Ocii0uggexvdm9WrsfsCLk3VLHW43ZjCp5T6qsijSu fiKt75qcdI/cMYzaI6ZLRvP54w56Pen4FI11eZPzRtwO9wYD0jcoh9OF5aN4UhOm89hoBtHH KEWXeLFGo3PZkeg4xxcpZqmmtfkFhllIAbgYJvRa2hqRCRn7NCuRmvXaO2jqR0p7C1trBZlc CgWGqJInvIxMM9MKesxIjAYnDdvNO7Z74jfkWhT+TN3t3WmTGlk0PrwuiRofCmyrX1zV68/P Xjtnun4ZIhGQ3h6DQZrdCQSRYLBYTrjRAreo2p+E/1mWhbOoB1S+cg0vhcSG0R0eT43AGStU kOM7ayg2ntpGhqZFdio14gTyqIcxj8cQGsVFaOgHcphYaJik1BNKJy+jqlKTrY3IZxLZUep5 jpNYK/KWOlsVbRWUkDIyRlS0JunVdBQvmJ3i7+7BpfWsPjQp61tLr6YGfnqq1hxPluc2imK8 ZahfyKwkAyJWU6dZN5nH4oH6r7Bcpno150LpV7SSghm/jkWMVNastKqgYbudGdjlf8rIZpsr e/PFxK6Q0MEabLDiiHaZzU0Xey9z5rn3HF64sHcOiJOpan3MJu/2n1k/g23+cXnk76O0N6NN Gnn57f30cofNivB55dOvrFzXTx9ReVBWzUr7KSMMeb60ZUIdABEIj3vPeUagT3xIGF9/UUxl 31qucfUbYJWQ81SJJmpiWFP7KVghhm6JsWKLtmQ4QigyluRQbX/dyItxE/LYIGspAknSjlEU AAA7 --Boundary_(ID_JJIBbfzti8zCVp1+Dh3eLg)-- --Boundary_(ID_c4dUZTtGDtjBjJDVBPbZkw)-- From Darren.Moffat@Sun.COM Wed Nov 1 03:30:04 2006 Received: from sunmail2.sfbay.sun.com (sunmail2.SFBay.Sun.COM [129.149.246.180]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id kA1BU48D006885 for ; Wed, 1 Nov 2006 03:30:04 -0800 (PST) Received: from nwk-avmta-1.sfbay.sun.com (nwk-avmta-1.SFBay.Sun.COM [129.149.246.28]) by sunmail2.sfbay.sun.com (8.11.7p1+Sun/8.11.7/ENSMAIL,v2.2) with ESMTP id kA1BU4U18593 for <@sunmail2.sfbay.sun.com:PSARC-EXT@sun.com>; Wed, 1 Nov 2006 03:30:04 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2 (built Dec 2 2004)) id <0J810020DUM3KR00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Wed, 01 Nov 2006 03:30:03 -0800 (PST) Received: from gmp-ea-fw-1.sun.com ([129.156.42.6]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2 (built Dec 2 2004)) with ESMTP id <0J8100CRJUM00TD0@nwk-avmta-1.sfbay.Sun.COM> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Wed, 01 Nov 2006 03:30:01 -0800 (PST) Received: from d1-emea-09.sun.com ([192.18.2.119]) by gmp-ea-fw-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id kA1BU08U002573 for ; Wed, 01 Nov 2006 11:30:00 +0000 (GMT) Received: from conversion-daemon.d1-emea-09.sun.com by d1-emea-09.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0J8100701ULVOR00@d1-emea-09.sun.com> (original mail from Darren.Moffat@Sun.COM) for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Wed, 01 Nov 2006 11:30:00 +0000 (GMT) Received: from [129.150.120.2] by d1-emea-09.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0J8100D4YULWT210@d1-emea-09.sun.com>; Wed, 01 Nov 2006 11:29:59 +0000 (GMT) Date: Wed, 01 Nov 2006 11:29:48 +0000 From: Darren J Moffat Subject: Re: PSARC/2006/598 Swap resource control; locked memory RM improvements In-reply-to: <20061031192726.GI2475@steve1.eng.sun.com> Sender: Darren.Moffat@Sun.COM To: Steve Lawrence Cc: Gary Winiger , PSARC-EXT@Sun.COM, dp@eng.sun.com, zones-discuss@opensolaris.org Message-id: <4548852C.2020002@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200610311859.k9VIxNmL013021@marduk.eng.sun.com> <20061031192726.GI2475@steve1.eng.sun.com> User-Agent: Thunderbird 1.5.0.5 (X11/20060925) Status: RO Content-Length: 2677 Steve Lawrence wrote: > On Tue, Oct 31, 2006 at 10:59:23AM -0800, Gary Winiger wrote: >>> DETAIL: >>> >>> 1. "zone.max-swap" resource control. >>> >>> Limits swap consumed by user process address space mappings and >>> tmpfs mounts within a zone. >>> zone.max-swap will be configurable on both the global zone, and >>> non-global zones. The affect on processes in a zone reaching its >>> zone.max-swap limit is the same as if all system swap is reserved. >>> Callers of mmap(2) and sbrk(2) will receive EAGAIN. Writes to >>> tmpfs will return ENOSPC, which is the same errno returned when >>> a tmpfs mount reaches it's "size" mount option. The "size" mount >>> option limits the quantity of swap that a tmpfs mount can reserve. >>> >>> While a low zone.max-swap setting for the global zone can lead to >>> a difficult-to-administer global zone, the same problem exists >>> today when configuring the zone.max-lwps resource control on the >>> global zone, or when all system swap is reserved. >> While other controls seem to be capped by the process/thread's >> project, it seems that use of swap like max-lwps might be >> important for the global zone project 0. Has consideration been >> given to allowing global zone "system" processes to not be >> bound by GZ zone.max-swap? >> It might be better than having to reboot or boot single to fix >> a problem that could otherwise be fixed in the GZ. > > This is how we treat cpu-shares. project 0 in the global zone has "infinite" > shares. > > This will not help root logins directly, but could by setting: > > usermod -K project=system root > > Would it be reasonable to propose special treatment of the global project 0 > for all project and zone rctls? Once could argue that capping system daemons > can only lead some sort of undesireable system failure. > > This would of course exempt all global zone system daemons from resource > management. To mitigate this, SMF could be leveraged to run application > daemons (or leaky/bad system daemons) in other projects. Please don't do that in a hardcoded way. I don't mind if it is the default but it can't be hard coded. One of the main reasons we have nfsd (and kcfd) is so that resource controls can be placed on system services. With the advent of SMF this is actually really easy to do since you just need to set the project/pool the start method runs in. Also it is perfectly reasonable to have a case where no "useful" customer work happens in the global zone, ie it is a service processor really, and all the real work happens in the non global zones. -- Darren J Moffat From Darren.Moffat@sun.com Wed Nov 1 03:32:12 2006 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id kA1BWB3w006916 for ; Wed, 1 Nov 2006 03:32:12 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.149.247.22]) by sunmail5.uk.sun.com (8.13.7+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id kA1BVxlw006891 for <@sunmail3.sfbay.sun.com:PSARC-EXT@sun.com>; Wed, 1 Nov 2006 11:32:10 GMT Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0J8100I0JUPK4O00@nwk-avmta-2.sfbay.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Wed, 01 Nov 2006 03:32:08 -0800 (PST) Received: from gmp-ea-fw-1.sun.com ([129.156.42.5]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0J8100AKQUPJVD30@nwk-avmta-2.sfbay.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Wed, 01 Nov 2006 03:32:08 -0800 (PST) Received: from d1-emea-09.sun.com (d1-emea-09.sun.com [192.18.2.119] (may be forged)) by gmp-ea-fw-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id kA1BW65t010769 for ; Wed, 01 Nov 2006 11:32:07 +0000 (GMT) Received: from conversion-daemon.d1-emea-09.sun.com by d1-emea-09.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0J8100701ULVOR00@d1-emea-09.sun.com> (original mail from Darren.Moffat@Sun.COM) for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Wed, 01 Nov 2006 11:32:06 +0000 (GMT) Received: from [129.150.120.2] by d1-emea-09.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0J8100D6IUPHT310@d1-emea-09.sun.com>; Wed, 01 Nov 2006 11:32:06 +0000 (GMT) Date: Wed, 01 Nov 2006 11:31:57 +0000 From: Darren J Moffat Subject: Re: PSARC/2006/598 Swap resource control; locked memory RM improvements In-reply-to: <20061031220120.GK2475@steve1.eng.sun.com> Sender: Darren.Moffat@sun.com To: Steve Lawrence Cc: Gary Winiger , PSARC-EXT@sun.com, dp@eng.sun.com, zones-discuss@opensolaris.org Message-id: <454885AD.5000702@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200610312002.k9VK2w2W013230@marduk.eng.sun.com> <20061031220120.GK2475@steve1.eng.sun.com> User-Agent: Thunderbird 1.5.0.5 (X11/20060925) Status: RO Content-Length: 1244 Steve Lawrence wrote: > On Tue, Oct 31, 2006 at 12:02:58PM -0800, Gary Winiger wrote: >>> This is how we treat cpu-shares. project 0 in the global zone has "infinite" >>> shares. >>> >>> This will not help root logins directly, but could by setting: >>> >>> usermod -K project=system root >> Or perhaps deliver root's entry this way to start with. > > Would that be a reasonable change to make via patch? Perhaps this change > could be delivered to nevada, but not backported. > > It would be confusing to deliver this change, and also deliver the "user.root" > project. If we made root's default project "system", then the "user.root" > project should be removed. "user.root" is kind of a bug anyhow, as SMF does > not run root services in "user.root". Currently, only root processes spawned > by login/pam run in "user.root". I actually felt that that was an interesting accident that had useful sideeffects. It would though be better to formalise the distinction between services SMF starts and logins to root. I think it is actually a good thing that they are in different projects by default. Just because they are in different projects doesn't mean the configuration of those projects can't be the same. -- Darren J Moffat From sl108498@steve1.sfbay.sun.com Wed Nov 1 10:40:13 2006 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id kA1IeChi019964 for ; Wed, 1 Nov 2006 10:40:13 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id kA1IeBR2017994 for <@sunmail1brm.central.sun.com:PSARC-EXT@sun.com>; Wed, 1 Nov 2006 10:40:12 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0J8200616EIY3200@brm-avmta-1.central.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Wed, 01 Nov 2006 11:40:10 -0700 (MST) Received: from sfbaymail1sca.SFBay.Sun.COM ([129.145.154.35]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0J82000J1EIWJT80@brm-avmta-1.central.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Wed, 01 Nov 2006 11:40:08 -0700 (MST) Received: from steve1.sfbay.sun.com (steve1.SFBay.Sun.COM [129.146.228.230]) by sfbaymail1sca.SFBay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id kA1Ie6aj003910; Wed, 01 Nov 2006 10:40:06 -0800 (PST) Received: from steve1.sfbay.sun.com (localhost [127.0.0.1]) by steve1.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id kA1IdjoX007092; Wed, 01 Nov 2006 10:39:45 -0800 (PST) Received: (from sl108498@localhost) by steve1.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id kA1Idj7i007091; Wed, 01 Nov 2006 10:39:45 -0800 (PST) Date: Wed, 01 Nov 2006 10:39:45 -0800 From: Steve Lawrence Subject: Re: PSARC/2006/598 Swap resource control; locked memory RM improvements In-reply-to: <4548852C.2020002@Sun.COM> To: Darren J Moffat Cc: Steve Lawrence , Gary Winiger , PSARC-EXT@sun.com, dp@eng.sun.com, zones-discuss@opensolaris.org Message-id: <20061101183945.GC7055@steve1.eng.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <200610311859.k9VIxNmL013021@marduk.eng.sun.com> <20061031192726.GI2475@steve1.eng.sun.com> <4548852C.2020002@Sun.COM> User-Agent: Mutt/1.4.2.1i Status: RO Content-Length: 1209 > >Would it be reasonable to propose special treatment of the global project 0 > >for all project and zone rctls? Once could argue that capping system > >daemons > >can only lead some sort of undesireable system failure. > > > >This would of course exempt all global zone system daemons from resource > >management. To mitigate this, SMF could be leveraged to run application > >daemons (or leaky/bad system daemons) in other projects. > > Please don't do that in a hardcoded way. I don't mind if it is the > default but it can't be hard coded. One of the main reasons we have > nfsd (and kcfd) is so that resource controls can be placed on system > services. Do you mean "one of the reasons that nfs and kcfd run in the system project"?? > > With the advent of SMF this is actually really easy to do since you > just need to set the project/pool the start method runs in. > > Also it is perfectly reasonable to have a case where no "useful" > customer work happens in the global zone, ie it is a service processor > really, and all the real work happens in the non global zones. Please elaborate on this. I'm not sure I understand what you're getting at. -Steve > > -- > Darren J Moffat From sl108498@steve1.sfbay.sun.com Wed Nov 1 10:42:17 2006 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id kA1IgG6M019995 for ; Wed, 1 Nov 2006 10:42:17 -0800 (PST) Received: from nwk-avmta-1.sfbay.sun.com (nwk-avmta-1.SFBay.Sun.COM [129.149.246.28]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id kA1IfuWV012409 for <@sunmail2.sfbay.sun.com:PSARC-EXT@sun.com>; Thu, 2 Nov 2006 02:42:15 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2 (built Dec 2 2004)) id <0J8200L07EME0H00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Wed, 01 Nov 2006 10:42:14 -0800 (PST) Received: from sfbaymail1sca.SFBay.Sun.COM ([129.145.154.35]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2 (built Dec 2 2004)) with ESMTP id <0J8200JD9EME8930@nwk-avmta-1.sfbay.Sun.COM> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Wed, 01 Nov 2006 10:42:14 -0800 (PST) Received: from steve1.sfbay.sun.com (steve1.SFBay.Sun.COM [129.146.228.230]) by sfbaymail1sca.SFBay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id kA1IgCdQ004928; Wed, 01 Nov 2006 10:42:13 -0800 (PST) Received: from steve1.sfbay.sun.com (localhost [127.0.0.1]) by steve1.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id kA1IfqX1007101; Wed, 01 Nov 2006 10:41:52 -0800 (PST) Received: (from sl108498@localhost) by steve1.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id kA1IfqNL007100; Wed, 01 Nov 2006 10:41:52 -0800 (PST) Date: Wed, 01 Nov 2006 10:41:51 -0800 From: Steve Lawrence Subject: Re: PSARC/2006/598 Swap resource control; locked memory RM improvements In-reply-to: <454885AD.5000702@Sun.COM> To: Darren J Moffat Cc: Steve Lawrence , Gary Winiger , PSARC-EXT@sun.com, dp@eng.sun.com, zones-discuss@opensolaris.org Message-id: <20061101184151.GD7055@steve1.eng.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <200610312002.k9VK2w2W013230@marduk.eng.sun.com> <20061031220120.GK2475@steve1.eng.sun.com> <454885AD.5000702@Sun.COM> User-Agent: Mutt/1.4.2.1i Status: RO Content-Length: 1571 On Wed, Nov 01, 2006 at 11:31:57AM +0000, Darren J Moffat wrote: > Steve Lawrence wrote: > >On Tue, Oct 31, 2006 at 12:02:58PM -0800, Gary Winiger wrote: > >>>This is how we treat cpu-shares. project 0 in the global zone has > >>>"infinite" > >>>shares. > >>> > >>>This will not help root logins directly, but could by setting: > >>> > >>> usermod -K project=system root > >> Or perhaps deliver root's entry this way to start with. > > > >Would that be a reasonable change to make via patch? Perhaps this change > >could be delivered to nevada, but not backported. > > > >It would be confusing to deliver this change, and also deliver the > >"user.root" > >project. If we made root's default project "system", then the "user.root" > >project should be removed. "user.root" is kind of a bug anyhow, as SMF > >does > >not run root services in "user.root". Currently, only root processes > >spawned > >by login/pam run in "user.root". > > I actually felt that that was an interesting accident that had useful > sideeffects. It would though be better to formalise the distinction > between services SMF starts and logins to root. I think it is actually > a good thing that they are in different projects by default. > > Just because they are in different projects doesn't mean the > configuration of those projects can't be the same. I'm proposing special kernel treatment of projid 0. Another project cannot be configured to be like this. Are you proposing a new project attribute which means "don't enforce zone rctls"?? -Steve > > -- > Darren J Moffat From Darren.Moffat@Sun.COM Thu Nov 2 02:56:32 2006 Received: from sunmail1brm.Central.Sun.COM (sunmail1brm.Central.Sun.COM [129.147.62.17]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id kA2AuVYN011625 for ; Thu, 2 Nov 2006 02:56:32 -0800 (PST) Received: from nwk-avmta-1.sfbay.sun.com (nwk-avmta-1.SFBay.Sun.COM [129.149.246.28]) by sunmail1brm.Central.Sun.COM (8.11.7p1+Sun/8.11.7/ENSMAIL,v2.2) with ESMTP id kA2AuV804427 for <@sunmail2.sfbay.sun.com:PSARC-EXT@sun.com>; Thu, 2 Nov 2006 03:56:31 -0700 (MST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2 (built Dec 2 2004)) id <0J830040HNQ3V900@nwk-avmta-1.sfbay.Sun.COM> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Thu, 02 Nov 2006 02:56:27 -0800 (PST) Received: from gmp-ea-fw-1.sun.com ([129.156.42.6]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2 (built Dec 2 2004)) with ESMTP id <0J83003J3NQ2GPA0@nwk-avmta-1.sfbay.Sun.COM> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Thu, 02 Nov 2006 02:56:27 -0800 (PST) Received: from d1-emea-09.sun.com ([192.18.2.119]) by gmp-ea-fw-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id kA2AuQLY021767 for ; Thu, 02 Nov 2006 10:56:26 +0000 (GMT) Received: from conversion-daemon.d1-emea-09.sun.com by d1-emea-09.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0J8300L01NKP1100@d1-emea-09.sun.com> (original mail from Darren.Moffat@Sun.COM) for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Thu, 02 Nov 2006 10:56:26 +0000 (GMT) Received: from [129.156.173.199] by d1-emea-09.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0J830015ONQ18J10@d1-emea-09.sun.com>; Thu, 02 Nov 2006 10:56:25 +0000 (GMT) Date: Thu, 02 Nov 2006 10:56:25 +0000 From: Darren J Moffat Subject: Re: PSARC/2006/598 Swap resource control; locked memory RM improvements In-reply-to: <20061101183945.GC7055@steve1.eng.sun.com> Sender: Darren.Moffat@Sun.COM To: Steve Lawrence Cc: Gary Winiger , PSARC-EXT@Sun.COM, dp@eng.sun.com, zones-discuss@opensolaris.org Message-id: <4549CED9.8040007@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200610311859.k9VIxNmL013021@marduk.eng.sun.com> <20061031192726.GI2475@steve1.eng.sun.com> <4548852C.2020002@Sun.COM> <20061101183945.GC7055@steve1.eng.sun.com> User-Agent: Thunderbird 1.5.0.5 (X11/20060926) Status: RO Content-Length: 2192 Steve Lawrence wrote: >>> Would it be reasonable to propose special treatment of the global project 0 >>> for all project and zone rctls? Once could argue that capping system >>> daemons >>> can only lead some sort of undesireable system failure. >>> >>> This would of course exempt all global zone system daemons from resource >>> management. To mitigate this, SMF could be leveraged to run application >>> daemons (or leaky/bad system daemons) in other projects. >> Please don't do that in a hardcoded way. I don't mind if it is the >> default but it can't be hard coded. One of the main reasons we have >> nfsd (and kcfd) is so that resource controls can be placed on system >> services. > > Do you mean "one of the reasons that nfs and kcfd run in the system project"?? This is getting a bit off topic for this case I think. Certainly by default. kcfd also puts itself into the FX scheduling class if FSS is not the default. The idea was that you could control how much CPU resource the kernel software crypto takes up (either give it more or less depending on your needs). >> With the advent of SMF this is actually really easy to do since you >> just need to set the project/pool the start method runs in. >> >> Also it is perfectly reasonable to have a case where no "useful" >> customer work happens in the global zone, ie it is a service processor >> really, and all the real work happens in the non global zones. > > Please elaborate on this. I'm not sure I understand what you're getting at. I think it might be my miss understanding of what this case really does. The idea here is that the real application work all happens in a non global zone, the global zone is just there to manage the machine. Unfortunately nfsd and kcfd are bad examples of services since their work is global zone specific in this case (kcfd does run in a local zone but the kcf thread pool it creates is global zone only). I was thinking more of services that could run in the global zone and if they are they are lower priority than the local zone ones. Now that I think about this case a bit more I think that it doesn't actually change things in that area. -- Darren J Moffat From sl108498@steve1.sfbay.sun.com Thu Nov 2 17:49:56 2006 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id kA31nt3N002223 for ; Thu, 2 Nov 2006 17:49:56 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.149.247.22]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id kA31nowO015811 for <@sunmail3.sfbay.sun.com:PSARC-EXT@sun.com>; Fri, 3 Nov 2006 09:49:54 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0J8400E0DT33LI00@nwk-avmta-2.sfbay.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Thu, 02 Nov 2006 17:49:51 -0800 (PST) Received: from sfbaymail2sca.sfbay.sun.com ([129.145.155.42]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0J8400CZJT33DAB0@nwk-avmta-2.sfbay.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Thu, 02 Nov 2006 17:49:51 -0800 (PST) Received: from steve1.sfbay.sun.com (steve1.SFBay.Sun.COM [129.146.228.230]) by sfbaymail2sca.sfbay.sun.com (8.13.6+Sun/8.12.10/ENSMAIL,v2.2) with ESMTP id kA31ndsd020924; Thu, 02 Nov 2006 17:49:45 -0800 (PST) Received: from steve1.sfbay.sun.com (localhost [127.0.0.1]) by steve1.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id kA31nFpq008031; Thu, 02 Nov 2006 17:49:15 -0800 (PST) Received: (from sl108498@localhost) by steve1.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id kA31nEUA008030; Thu, 02 Nov 2006 17:49:14 -0800 (PST) Date: Thu, 02 Nov 2006 17:49:14 -0800 From: Steve Lawrence Subject: Re: [zones-discuss] Re: PSARC/2006/598 Swap resource control; locked memory RM improvements In-reply-to: <4547FF40.8050707@logiqwest.com> To: Michael Barto Cc: Gary Winiger , stephen.lawrence@sun.com, PSARC-EXT@sun.com, zones-discuss@opensolaris.org Message-id: <20061103014914.GJ7738@steve1.eng.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <200610312301.k9VN1qln013617@marduk.eng.sun.com> <4547FF40.8050707@logiqwest.com> User-Agent: Mutt/1.4.2.1i Status: RO Content-Length: 3854 I'm not sure it is within the domain of this case to to tell admins what they should and shouldn't use the global zone for. In any event, we are making it easy for admins to manage swap limits for zones via zonecfg. On Tue, Oct 31, 2006 at 05:58:24PM -0800, Michael Barto wrote: > After all thus juggling, let make it simple for the system admin and use > some sort of fair share process to assignment and manage the swap for > all the zones. Personally I think that the global zone should use > minimum resources and be considered in the IT management processes to be > only like a system controller on a complex server. Keep your > applications out of the global zone!!! > > Gary Winiger wrote: > > >>>>This will not help root logins directly, but could by setting: > >>>> > >>>> usermod -K project=system root > >>>> > >>>> > >>> Or perhaps deliver root's entry this way to start with. > >>> > >>> > >>Would that be a reasonable change to make via patch? Perhaps this change > >>could be delivered to nevada, but not backported. > >> > >>It would be confusing to deliver this change, and also deliver the > >>"user.root" > >>project. If we made root's default project "system", then the "user.root" > >>project should be removed. "user.root" is kind of a bug anyhow, as SMF > >>does > >>not run root services in "user.root". Currently, only root processes > >>spawned > >>by login/pam run in "user.root". > >> > >> > > > > > > > > > >>>>Perhaps this issue should be run as a seperate fasttrack? I need to > >>>>investigate the implementation impact. > >>>> > >>>> > >>> I'm looking for this case to define how to preserve the current > >>> model of "unlimited" unless one asks for a limit model in the > >>> global zone. I believe it is important from a system integrity and > >>> maintenance perspective. Other's may have different opinions. > >>> If there is a compelling reason to deliver in phases, please discuss > >>> that. > >>> > >>> > >>The global zone will have no swap limit by default. The default > >>zone.max-swap > >>rctl delivered on the global zone is UINT64_MAX, which is essentially > >>unlimited. Is that what you mean? > >> > >> > > > > My point(s) here is not so much how things get done, but that > > the global zone is in some ways special. IIRC, before this > > project, the GZ doesn't have a swap limit. After this project > > an administrator could set swap limit on the GZ. Granted this > > is administrative action and they get what they deserve/ask for. > > However, it seemed to me that part of this case "should" (my > > judgement) include some way to override the limit in case > > override is really desired. As implied, perhaps by putting > > root into project 0 at login or as part of daemon/service start > > is a way to bypass the administrator's choice in the GZ for > > some processes. What I didn't see as part of this case is > > the architecture to allow this bypass. Perhaps I'm off base > > for thinking it's necessary to protect against inadvertantly > > not being able to administer the system from the GZ. > > > >Gary.. > > > >_______________________________________________ > >zones-discuss mailing list > >zones-discuss@opensolaris.org > > > > > > > > -- > ------------------------------------------------------------------------ > *Michael Barto* > Software Architect > > LogiQwest Circle > LogiQwest Inc. > 16458 Bolsa Chica Street, # 15 > Huntington Beach, CA 92649 > http://www.logiqwest.com/ > > mbarto@logiqwest.com > Tel: 714 377 3705 > Fax: 714 840 3937 > Cell: 714 883 1949 > > *'tis a gift to be simple* > ------------------------------------------------------------------------ > This e-mail may contain LogiQwest proprietary information and should be > treated as confidential. > From sl108498@steve1.sfbay.sun.com Thu Nov 2 18:00:09 2006 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id kA3208hL002636 for ; Thu, 2 Nov 2006 18:00:08 -0800 (PST) Received: from nwk-avmta-1.sfbay.sun.com (nwk-avmta-1.SFBay.Sun.COM [129.149.246.28]) by sunmail5.uk.sun.com (8.13.7+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id kA31xvO3004875 for <@sunmail2.sfbay.sun.com:PSARC-EXT@sun.com>; Fri, 3 Nov 2006 02:00:07 GMT Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2 (built Dec 2 2004)) id <0J8400309TK7PP00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Thu, 02 Nov 2006 18:00:07 -0800 (PST) Received: from sfbaymail2sca.sfbay.sun.com ([129.145.155.42]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2 (built Dec 2 2004)) with ESMTP id <0J84004ZPTK6C9A0@nwk-avmta-1.sfbay.Sun.COM> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Thu, 02 Nov 2006 18:00:06 -0800 (PST) Received: from steve1.sfbay.sun.com (steve1.SFBay.Sun.COM [129.146.228.230]) by sfbaymail2sca.sfbay.sun.com (8.13.6+Sun/8.12.10/ENSMAIL,v2.2) with ESMTP id kA31xpJJ023713; Thu, 02 Nov 2006 17:59:56 -0800 (PST) Received: from steve1.sfbay.sun.com (localhost [127.0.0.1]) by steve1.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id kA31xQhA008042; Thu, 02 Nov 2006 17:59:26 -0800 (PST) Received: (from sl108498@localhost) by steve1.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id kA31xQoI008041; Thu, 02 Nov 2006 17:59:26 -0800 (PST) Date: Thu, 02 Nov 2006 17:59:26 -0800 From: Steve Lawrence Subject: Re: PSARC/2006/598 Swap resource control; locked memory RM improvements In-reply-to: <20061031232418.GM2475@steve1.eng.sun.com> To: Steve Lawrence Cc: Gary Winiger , PSARC-EXT@sun.com, dp@eng.sun.com, zones-discuss@opensolaris.org Message-id: <20061103015926.GK7738@steve1.eng.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <200610312301.k9VN1qln013617@marduk.eng.sun.com> <20061031232418.GM2475@steve1.eng.sun.com> User-Agent: Mutt/1.4.2.1i Status: RO Content-Length: 3121 Given a lack of supportive feedback, I'm going to revoke the proposed amendment below. To mitigate a zone admin setting a problematic swap limit on the global zone, we will enhance zonecfg to: 1. Print a warning when setting swap (and lwp) limits on the global zone. Since the swap limit will not go into effect until reboot, the admin has a change to modify his setting before it takes affect. 2. Enforce a reasonable minimum when setting swap (and lwp) limits on the global zone. Currently, the rctl framework provides many mechanisms by which the admin can make the system difficult to manage. For instance, setting task.max-lwps on project "user.root" can prevent root login. If we want to make changes to prevent admins from resource-controlling their way out of the box, I think we need a broader case to address the whole problem. -Steve On Tue, Oct 31, 2006 at 03:24:18PM -0800, Steve Lawrence wrote: > > > > I'm looking for this case to define how to preserve the current > > > > model of "unlimited" unless one asks for a limit model in the > > > > global zone. I believe it is important from a system integrity and > > > > maintenance perspective. Other's may have different opinions. > > > > If there is a compelling reason to deliver in phases, please discuss > > > > that. > > > > > > The global zone will have no swap limit by default. The default zone.max-swap > > > rctl delivered on the global zone is UINT64_MAX, which is essentially > > > unlimited. Is that what you mean? > > > > My point(s) here is not so much how things get done, but that > > the global zone is in some ways special. IIRC, before this > > project, the GZ doesn't have a swap limit. After this project > > an administrator could set swap limit on the GZ. Granted this > > is administrative action and they get what they deserve/ask for. > > However, it seemed to me that part of this case "should" (my > > judgement) include some way to override the limit in case > > override is really desired. As implied, perhaps by putting > > root into project 0 at login or as part of daemon/service start > > is a way to bypass the administrator's choice in the GZ for > > some processes. What I didn't see as part of this case is > > the architecture to allow this bypass. Perhaps I'm off base > > for thinking it's necessary to protect against inadvertantly > > not being able to administer the system from the GZ. > > > > It seems reasonable to amend this case to say: > > 1. > Any process with priv_sys_resource running in the global zone's > system project (project 0) will not subject to project.* or zone.* > resource controls. System daemons which wish to be subject to the > global zone's resource controls can drop priv_sys_resource. > > 2. > The "user.root" project will be removed, and root's default project > will be set to the "system" project via /etc/user_attr. > > I'm not sure if (2) can be delivered via patch. I need some guidance here. > I'm also not sure how implementable (1) is until I do more investigation. > > -Steve > > > Gary.. > > From Darren.Moffat@sun.com Fri Nov 3 01:37:05 2006 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id kA39b4Lv011310 for ; Fri, 3 Nov 2006 01:37:05 -0800 (PST) Received: from nwk-avmta-1.sfbay.sun.com (nwk-avmta-1.SFBay.Sun.COM [129.149.246.28]) by sunmail5.uk.sun.com (8.13.7+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id kA39akYi013642 for <@sunmail2.sfbay.sun.com:PSARC-EXT@sun.com>; Fri, 3 Nov 2006 09:37:03 GMT Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2 (built Dec 2 2004)) id <0J8500601EPP0N00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Fri, 03 Nov 2006 01:37:01 -0800 (PST) Received: from gmp-ea-fw-1.sun.com ([129.156.42.5]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2 (built Dec 2 2004)) with ESMTP id <0J8500KXMEPOSM30@nwk-avmta-1.sfbay.Sun.COM> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Fri, 03 Nov 2006 01:37:01 -0800 (PST) Received: from d1-emea-09.sun.com (d1-emea-09.sun.com [192.18.2.119] (may be forged)) by gmp-ea-fw-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id kA39b0t8025388 for ; Fri, 03 Nov 2006 09:37:00 +0000 (GMT) Received: from conversion-daemon.d1-emea-09.sun.com by d1-emea-09.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0J8500401EIURS00@d1-emea-09.sun.com> (original mail from Darren.Moffat@Sun.COM) for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Fri, 03 Nov 2006 09:37:00 +0000 (GMT) Received: from [192.168.73.102] (nessieroo.force9.co.uk [81.174.224.49]) by d1-emea-09.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0J8500HMZEPJE800@d1-emea-09.sun.com>; Fri, 03 Nov 2006 09:36:56 +0000 (GMT) Date: Fri, 03 Nov 2006 09:36:45 +0000 From: Darren J Moffat Subject: Re: PSARC/2006/598 Swap resource control; locked memory RM improvements In-reply-to: <20061103015926.GK7738@steve1.eng.sun.com> Sender: Darren.Moffat@sun.com To: Steve Lawrence Cc: Gary Winiger , PSARC-EXT@sun.com, dp@eng.sun.com, zones-discuss@opensolaris.org Message-id: <454B0DAD.4090600@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200610312301.k9VN1qln013617@marduk.eng.sun.com> <20061031232418.GM2475@steve1.eng.sun.com> <20061103015926.GK7738@steve1.eng.sun.com> User-Agent: Thunderbird 1.5.0.5 (X11/20060925) Status: RO Content-Length: 667 Steve Lawrence wrote: > Given a lack of supportive feedback, I'm going to revoke the proposed amendment > below. To mitigate a zone admin setting a problematic swap limit on the global > zone, we will enhance zonecfg to: > > 1. Print a warning when setting swap (and lwp) limits on the global > zone. Since the swap limit will not go into effect until reboot, > the admin has a change to modify his setting before it takes > affect. > > 2. Enforce a reasonable minimum when setting swap (and lwp) limits > on the global zone. What is the definition of reasonable ? I think that being a default should be part of the case. -- Darren J Moffat From sl108498@steve1.sfbay.sun.com Fri Nov 3 10:49:21 2006 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id kA3InL4E021280 for ; Fri, 3 Nov 2006 10:49:21 -0800 (PST) Received: from nwk-avmta-1.sfbay.sun.com (nwk-avmta-1.SFBay.Sun.COM [129.149.246.28]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id kA3InK9n018905 for <@sunmail2.sfbay.sun.com:PSARC-EXT@sun.com>; Fri, 3 Nov 2006 10:49:20 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2 (built Dec 2 2004)) id <0J8600E0H4A8F900@nwk-avmta-1.sfbay.Sun.COM> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Fri, 03 Nov 2006 10:49:20 -0800 (PST) Received: from sfbaymail1sca.SFBay.Sun.COM ([129.145.154.35]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2 (built Dec 2 2004)) with ESMTP id <0J86003MK4A6A060@nwk-avmta-1.sfbay.Sun.COM> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Fri, 03 Nov 2006 10:49:19 -0800 (PST) Received: from steve1.sfbay.sun.com (steve1.SFBay.Sun.COM [129.146.228.230]) by sfbaymail1sca.SFBay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id kA3InH5O010372; Fri, 03 Nov 2006 10:49:17 -0800 (PST) Received: from steve1.sfbay.sun.com (localhost [127.0.0.1]) by steve1.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id kA3ImqhE008289; Fri, 03 Nov 2006 10:48:52 -0800 (PST) Received: (from sl108498@localhost) by steve1.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id kA3ImqKK008288; Fri, 03 Nov 2006 10:48:52 -0800 (PST) Date: Fri, 03 Nov 2006 10:48:52 -0800 From: Steve Lawrence Subject: Re: PSARC/2006/598 Swap resource control; locked memory RM improvements In-reply-to: <454B0DAD.4090600@Sun.COM> To: Darren J Moffat Cc: Steve Lawrence , Gary Winiger , PSARC-EXT@sun.com, dp@eng.sun.com, zones-discuss@opensolaris.org Message-id: <20061103184852.GN7738@steve1.eng.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <200610312301.k9VN1qln013617@marduk.eng.sun.com> <20061031232418.GM2475@steve1.eng.sun.com> <20061103015926.GK7738@steve1.eng.sun.com> <454B0DAD.4090600@Sun.COM> User-Agent: Mutt/1.4.2.1i Status: RO Content-Length: 1732 On Fri, Nov 03, 2006 at 09:36:45AM +0000, Darren J Moffat wrote: > Steve Lawrence wrote: > >Given a lack of supportive feedback, I'm going to revoke the proposed > >amendment > >below. To mitigate a zone admin setting a problematic swap limit on the > >global > >zone, we will enhance zonecfg to: > > > > 1. Print a warning when setting swap (and lwp) limits on the global > > zone. Since the swap limit will not go into effect until reboot, > > the admin has a change to modify his setting before it takes > > affect. > > > > 2. Enforce a reasonable minimum when setting swap (and lwp) limits > > on the global zone. > > What is the definition of reasonable ? I think that being a > default should be part of the case. I'm not sure what your second sentence means. As for the first, how about: reasonable minimum: The amount of resource neccessary to boot a zone that is has a default installation and configuration. My "minimum", I do not mean the system default value. For example, for the following rctl, 128 is the system default value: # prctl -n project.max-shm-ids $$ process: 425781: -sh NAME PRIVILEGE VALUE FLAG ACTION RECIPIENT project.max-shm-ids privileged 128 - deny - system 16.8M max deny - zone.max-swap will not have a system default value. By default, every zone will have access to all swap available on the system. By minimum, I mean the minimum that zonecfg will allow you to set. zonecfg> add capped memory zonecfg:capped memory> set zone.max-swap=1K Error, minimum value for zone.max-swap is ### -Steve > > -- > Darren J Moffat From Darren.Moffat@Sun.COM Mon Nov 6 04:40:15 2006 Received: from sunmail1brm.Central.Sun.COM (sunmail1brm.Central.Sun.COM [129.147.62.17]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id kA6CeFoX022513 for ; Mon, 6 Nov 2006 04:40:15 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.149.247.22]) by sunmail1brm.Central.Sun.COM (8.11.7p1+Sun/8.11.7/ENSMAIL,v2.2) with ESMTP id kA6CeE922388 for <@sunmail3.sfbay.sun.com:PSARC-EXT@sun.com>; Mon, 6 Nov 2006 05:40:14 -0700 (MST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0J8B00M1P772XK00@nwk-avmta-2.sfbay.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Mon, 06 Nov 2006 04:40:14 -0800 (PST) Received: from gmp-ea-fw-1.sun.com ([129.156.42.5]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0J8B00CAO7711490@nwk-avmta-2.sfbay.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Mon, 06 Nov 2006 04:40:14 -0800 (PST) Received: from d1-emea-10.sun.com (d1-emea-10.sun.com [192.18.2.120] (may be forged)) by gmp-ea-fw-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id kA6CeCLE003508 for ; Mon, 06 Nov 2006 12:40:12 +0000 (GMT) Received: from conversion-daemon.d1-emea-10.sun.com by d1-emea-10.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0J8B00H0171ITI00@d1-emea-10.sun.com> (original mail from Darren.Moffat@Sun.COM) for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Mon, 06 Nov 2006 12:40:12 +0000 (GMT) Received: from [129.156.39.226] by d1-emea-10.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0J8B00CT1770J300@d1-emea-10.sun.com>; Mon, 06 Nov 2006 12:40:12 +0000 (GMT) Date: Mon, 06 Nov 2006 12:40:00 +0000 From: Darren J Moffat Subject: Re: PSARC/2006/598 Swap resource control; locked memory RM improvements In-reply-to: <20061103184852.GN7738@steve1.eng.sun.com> Sender: Darren.Moffat@Sun.COM To: Steve Lawrence Cc: Gary Winiger , PSARC-EXT@Sun.COM, dp@eng.sun.com, zones-discuss@opensolaris.org Message-id: <454F2D20.2050704@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200610312301.k9VN1qln013617@marduk.eng.sun.com> <20061031232418.GM2475@steve1.eng.sun.com> <20061103015926.GK7738@steve1.eng.sun.com> <454B0DAD.4090600@Sun.COM> <20061103184852.GN7738@steve1.eng.sun.com> User-Agent: Thunderbird 1.5.0.5 (X11/20060925) Status: RO Content-Length: 2074 Steve Lawrence wrote: > On Fri, Nov 03, 2006 at 09:36:45AM +0000, Darren J Moffat wrote: >> Steve Lawrence wrote: >>> Given a lack of supportive feedback, I'm going to revoke the proposed >>> amendment >>> below. To mitigate a zone admin setting a problematic swap limit on the >>> global >>> zone, we will enhance zonecfg to: >>> >>> 1. Print a warning when setting swap (and lwp) limits on the global >>> zone. Since the swap limit will not go into effect until reboot, >>> the admin has a change to modify his setting before it takes >>> affect. >>> >>> 2. Enforce a reasonable minimum when setting swap (and lwp) limits >>> on the global zone. >> What is the definition of reasonable ? I think that being a >> default should be part of the case. > > I'm not sure what your second sentence means. As for the first, how about: I thought that what you meant was an actual number or percentage of the total available system swap. Hence why I though it was a default value worthy of being part of the interfaces this case defines. > reasonable minimum: The amount of resource neccessary to boot a zone that > is has a default installation and configuration. > > My "minimum", I do not mean the system default value. For example, for > the following rctl, 128 is the system default value: > > # prctl -n project.max-shm-ids $$ > process: 425781: -sh > NAME PRIVILEGE VALUE FLAG ACTION RECIPIENT > project.max-shm-ids > privileged 128 - deny - > system 16.8M max deny - > > zone.max-swap will not have a system default value. By default, every zone > will have access to all swap available on the system. Okay. > By minimum, I mean the minimum that zonecfg will allow you to set. > > zonecfg> add capped memory > zonecfg:capped memory> set zone.max-swap=1K > Error, minimum value for zone.max-swap is ### Where does the ### come from. I think I'm missing something here. -- Darren J Moffat From sl108498@steve1.sfbay.sun.com Mon Nov 6 11:38:47 2006 Received: from sunmail1brm.Central.Sun.COM (sunmail1brm.Central.Sun.COM [129.147.62.17]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id kA6Jcl6r002146 for ; Mon, 6 Nov 2006 11:38:47 -0800 (PST) Received: from nwk-avmta-1.sfbay.sun.com (nwk-avmta-1.SFBay.Sun.COM [129.149.246.28]) by sunmail1brm.Central.Sun.COM (8.11.7p1+Sun/8.11.7/ENSMAIL,v2.2) with ESMTP id kA6Jck904879 for <@sunmail2.sfbay.sun.com:PSARC-EXT@sun.com>; Mon, 6 Nov 2006 12:38:46 -0700 (MST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2 (built Dec 2 2004)) id <0J8B0080ZQKLWN00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Mon, 06 Nov 2006 11:38:45 -0800 (PST) Received: from sfbaymail1sca.SFBay.Sun.COM ([129.145.154.35]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2 (built Dec 2 2004)) with ESMTP id <0J8B006VWQKILY40@nwk-avmta-1.sfbay.Sun.COM> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Mon, 06 Nov 2006 11:38:42 -0800 (PST) Received: from steve1.sfbay.sun.com (steve1.SFBay.Sun.COM [129.146.228.230]) by sfbaymail1sca.SFBay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id kA6JcfC6016135; Mon, 06 Nov 2006 11:38:41 -0800 (PST) Received: from steve1.sfbay.sun.com (localhost [127.0.0.1]) by steve1.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id kA6Jc99d009356; Mon, 06 Nov 2006 11:38:09 -0800 (PST) Received: (from sl108498@localhost) by steve1.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id kA6Jc9eX009355; Mon, 06 Nov 2006 11:38:09 -0800 (PST) Date: Mon, 06 Nov 2006 11:38:09 -0800 From: Steve Lawrence Subject: Re: PSARC/2006/598 Swap resource control; locked memory RM improvements In-reply-to: <454F2D20.2050704@Sun.COM> To: Darren J Moffat Cc: Steve Lawrence , Gary Winiger , PSARC-EXT@sun.com, dp@eng.sun.com, zones-discuss@opensolaris.org Message-id: <20061106193809.GV7738@steve1.eng.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <200610312301.k9VN1qln013617@marduk.eng.sun.com> <20061031232418.GM2475@steve1.eng.sun.com> <20061103015926.GK7738@steve1.eng.sun.com> <454B0DAD.4090600@Sun.COM> <20061103184852.GN7738@steve1.eng.sun.com> <454F2D20.2050704@Sun.COM> User-Agent: Mutt/1.4.2.1i Status: RO Content-Length: 2486 On Mon, Nov 06, 2006 at 12:40:00PM +0000, Darren J Moffat wrote: > Steve Lawrence wrote: > >On Fri, Nov 03, 2006 at 09:36:45AM +0000, Darren J Moffat wrote: > >>Steve Lawrence wrote: > >>>Given a lack of supportive feedback, I'm going to revoke the proposed > >>>amendment > >>>below. To mitigate a zone admin setting a problematic swap limit on the > >>>global > >>>zone, we will enhance zonecfg to: > >>> > >>> 1. Print a warning when setting swap (and lwp) limits on the global > >>> zone. Since the swap limit will not go into effect until reboot, > >>> the admin has a change to modify his setting before it takes > >>> affect. > >>> > >>> 2. Enforce a reasonable minimum when setting swap (and lwp) limits > >>> on the global zone. > >>What is the definition of reasonable ? I think that being a > >>default should be part of the case. > > > >I'm not sure what your second sentence means. As for the first, how about: > > I thought that what you meant was an actual number or percentage > of the total available system swap. Hence why I though it was a > default value worthy of being part of the interfaces this case > defines. > > >reasonable minimum: The amount of resource neccessary to boot a zone that > >is has a default installation and configuration. > > > >My "minimum", I do not mean the system default value. For example, for > >the following rctl, 128 is the system default value: > > > ># prctl -n project.max-shm-ids $$ > >process: 425781: -sh > >NAME PRIVILEGE VALUE FLAG ACTION > >RECIPIENT > >project.max-shm-ids > > privileged 128 - deny > > - > > system 16.8M max deny > > - > > > >zone.max-swap will not have a system default value. By default, every zone > >will have access to all swap available on the system. > > Okay. > > >By minimum, I mean the minimum that zonecfg will allow you to set. > > > >zonecfg> add capped memory > >zonecfg:capped memory> set zone.max-swap=1K > >Error, minimum value for zone.max-swap is ### > > Where does the ### come from. I think I'm missing something here. > ### is the "reasonable minimum" described above. Basing it in the system software requirements instead of an arbitrary % of total system swap seems better, especially since the total system swap can change at any time by "swap [ -a | -d ]'. -Steve > -- > Darren J Moffat From gww@eng.sun.com Mon Nov 6 12:10:29 2006 Received: from sunmail1brm.Central.Sun.COM (sunmail1brm.Central.Sun.COM [129.147.62.17]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id kA6KASUP002902 for ; Mon, 6 Nov 2006 12:10:29 -0800 (PST) Received: from nwk-avmta-1.sfbay.sun.com (nwk-avmta-1.SFBay.Sun.COM [129.149.246.28]) by sunmail1brm.Central.Sun.COM (8.11.7p1+Sun/8.11.7/ENSMAIL,v2.2) with ESMTP id kA6KAS919182 for <@sunmail2.sfbay.sun.com:PSARC-EXT@sun.com>; Mon, 6 Nov 2006 13:10:28 -0700 (MST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2 (built Dec 2 2004)) id <0J8B00B2RS1FHK00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Mon, 06 Nov 2006 12:10:27 -0800 (PST) Received: from engmail3mpk.sfbay.Sun.COM ([129.146.11.26]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2 (built Dec 2 2004)) with ESMTP id <0J8B0060QS1EM1B0@nwk-avmta-1.sfbay.Sun.COM> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Mon, 06 Nov 2006 12:10:26 -0800 (PST) Received: from marduk.eng.sun.com (marduk.SFBay.Sun.COM [129.146.108.224]) by engmail3mpk.sfbay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id kA6KAOf1011511; Mon, 06 Nov 2006 12:10:24 -0800 (PST) Received: from marduk.eng.sun.com (localhost [127.0.0.1]) by marduk.eng.sun.com (8.13.6+Sun/8.12.11) with ESMTP id kA6KDfiK023902; Mon, 06 Nov 2006 12:13:41 -0800 (PST) Received: (from gww@localhost) by marduk.eng.sun.com (8.13.6+Sun/8.12.11/Submit) id kA6KDf1T023901; Mon, 06 Nov 2006 12:13:41 -0800 (PST) Date: Mon, 06 Nov 2006 12:13:41 -0800 (PST) From: Gary Winiger Subject: Re: PSARC/2006/598 Swap resource control; locked memory RM improvements To: Stephen.Lawrence@sun.com Cc: PSARC-EXT@sun.com, dp@eng.sun.com, gww@eng.sun.com, zones-discuss@opensolaris.org Message-id: <200611062013.kA6KDf1T023901@marduk.eng.sun.com> Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 154 At the request of the project team, I've put this case into waiting need spec. When the spec is updated, it will be sent out and the timer reset. Gary.. From sl108498@steve1.sfbay.sun.com Wed Nov 8 13:45:39 2006 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id kA8LjbX3014565 for ; Wed, 8 Nov 2006 13:45:38 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id kA8Ljarq028343 for <@sunmail1brm.central.sun.com:PSARC-EXT@sun.com>; Thu, 9 Nov 2006 05:45:37 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0J8F00H01LRZ8H00@brm-avmta-1.central.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Wed, 08 Nov 2006 14:45:35 -0700 (MST) Received: from sfbaymail2sca.sfbay.sun.com ([129.145.155.42]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0J8F00E10LRYRJ60@brm-avmta-1.central.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Wed, 08 Nov 2006 14:45:34 -0700 (MST) Received: from steve1.sfbay.sun.com (steve1.SFBay.Sun.COM [129.146.228.230]) by sfbaymail2sca.sfbay.sun.com (8.13.6+Sun/8.12.10/ENSMAIL,v2.2) with ESMTP id kA8LjTU0021886; Wed, 08 Nov 2006 13:45:34 -0800 (PST) Received: from steve1.sfbay.sun.com (localhost [127.0.0.1]) by steve1.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id kA8LiswO010305; Wed, 08 Nov 2006 13:44:54 -0800 (PST) Received: (from sl108498@localhost) by steve1.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id kA8LisJa010304; Wed, 08 Nov 2006 13:44:54 -0800 (PST) Date: Wed, 08 Nov 2006 13:44:54 -0800 From: Steve Lawrence Subject: Re: PSARC/2006/598 Swap resource control; locked memory RM improvements In-reply-to: <200611062013.kA6KDf1T023901@marduk.eng.sun.com> To: Gary Winiger Cc: Stephen.Lawrence@sun.com, PSARC-EXT@sun.com, dp@eng.sun.com, zones-discuss@opensolaris.org Message-id: <20061108214454.GC6111@steve1.eng.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <200611062013.kA6KDf1T023901@marduk.eng.sun.com> User-Agent: Mutt/1.4.2.1i Status: RO Content-Length: 1249 I am working on a new spec. I have an unanswered question from the discussion: > > The "SIZE" column will also be changed to "SWAP" for prstat > > options a, T, and J, for users, tasks, and projects. > > The reason for not changing this column in the default output would be > helpful. I have a seperate private interface used by prstat(1m) to get aggregate swap reserved by users, tasks, projects, and zones. Default prstat output is "per-process", and the information is accessed via /proc. Currently, per-process swap reservation is not counted or made available via /proc. From proc(4): typedef struct psinfo { ... size_t pr_size; /* size of process image in Kbytes */ ... "size of process image" is pretty meaningless. If we can change "pr_size" to be "swap reserved by process", then we could change "SIZE" to "SWAP" for all prstat(1m) output. Would such a change to psinfo_t be reasonable???? If not, could potentially convert pr_pad1 to pr_swap???? -Steve On Mon, Nov 06, 2006 at 12:13:41PM -0800, Gary Winiger wrote: > At the request of the project team, I've put this case into waiting need spec. > When the spec is updated, it will be sent out and the timer reset. > > Gary.. From sl108498@steve1.sfbay.sun.com Thu Nov 9 12:54:30 2006 Received: from sunmail1brm.Central.Sun.COM (sunmail1brm.Central.Sun.COM [129.147.62.17]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id kA9KsUCh021494 for ; Thu, 9 Nov 2006 12:54:30 -0800 (PST) Received: from nwk-avmta-1.sfbay.sun.com (nwk-avmta-1.SFBay.Sun.COM [129.149.246.28]) by sunmail1brm.Central.Sun.COM (8.11.7p1+Sun/8.11.7/ENSMAIL,v2.2) with ESMTP id kA9KsT913815 for <@sunmail2.sfbay.sun.com:PSARC-EXT@sun.com>; Thu, 9 Nov 2006 13:54:29 -0700 (MST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2 (built Dec 2 2004)) id <0J8H00J43E2T0600@nwk-avmta-1.sfbay.Sun.COM> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Thu, 09 Nov 2006 12:54:29 -0800 (PST) Received: from sfbaymail1sca.SFBay.Sun.COM ([129.145.154.35]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2 (built Dec 2 2004)) with ESMTP id <0J8H009HRE2QP690@nwk-avmta-1.sfbay.Sun.COM> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Thu, 09 Nov 2006 12:54:26 -0800 (PST) Received: from steve1.sfbay.sun.com (steve1.SFBay.Sun.COM [129.146.228.230]) by sfbaymail1sca.SFBay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id kA9KsPb0013466; Thu, 09 Nov 2006 12:54:25 -0800 (PST) Received: from steve1.sfbay.sun.com (localhost [127.0.0.1]) by steve1.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id kA9KrmpM012425; Thu, 09 Nov 2006 12:53:48 -0800 (PST) Received: (from sl108498@localhost) by steve1.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id kA9KrmJf012424; Thu, 09 Nov 2006 12:53:48 -0800 (PST) Date: Thu, 09 Nov 2006 12:53:48 -0800 From: Steve Lawrence Subject: Re: PSARC/2006/598 Swap resource control; locked memory RM improvements In-reply-to: <20061031232830.GR3101@eng.sun.com> To: Dan Price Cc: Steve Lawrence , Gary Winiger , PSARC-EXT@sun.com, zones-discuss@opensolaris.org Message-id: <20061109205348.GU9508@steve1.eng.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <200610312301.k9VN1qln013617@marduk.eng.sun.com> <20061031232418.GM2475@steve1.eng.sun.com> <20061031232830.GR3101@eng.sun.com> User-Agent: Mutt/1.4.2.1i Status: RO Content-Length: 1340 On Tue, Oct 31, 2006 at 03:28:31PM -0800, Dan Price wrote: > On Tue 31 Oct 2006 at 03:24PM, Steve Lawrence wrote: > > It seems reasonable to amend this case to say: > > > > 1. > > Any process with priv_sys_resource running in the global zone's > > system project (project 0) will not subject to project.* or zone.* > > resource controls. System daemons which wish to be subject to the > > global zone's resource controls can drop priv_sys_resource. > > This "feels" risky for a patch... the effect here is to "un-manage" > things which the customer may be expecting to be resource managed, > potentially including a workload. Or is the point that no-one using > RM will be relying on the system project to limit things? Currently, if rctls are specified in project(4) for the system project, they are never actually applied during boot. While this could be considered a bug, this also means that currently, by default, admins will have to take additional manual steps to resource manage project 0. They are more likely to put what they want to manage in a new/different project, and use project(4) to configure rctls. -Steve > > This implies that can't give the system project 10 (or 100) shares > after this proposal? > > -dp > > -- > Daniel Price - Solaris Kernel Engineering - dp@eng.sun.com - blogs.sun.com/dp From gww@eng.sun.com Sat Nov 11 20:36:19 2006 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id kAC4aJKj018362 for ; Sat, 11 Nov 2006 20:36:19 -0800 (PST) Received: from nwk-avmta-1.sfbay.sun.com (nwk-avmta-1.SFBay.Sun.COM [129.149.246.28]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id kAC4aIG9025743 for <@sunmail2.sfbay.sun.com:PSARC-EXT@sun.com>; Sat, 11 Nov 2006 20:36:19 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2 (built Dec 2 2004)) id <0J8L00F0TOSI9Q00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Sat, 11 Nov 2006 20:36:18 -0800 (PST) Received: from engmail3mpk.sfbay.Sun.COM ([129.146.11.26]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2 (built Dec 2 2004)) with ESMTP id <0J8L00CRIOSGYU70@nwk-avmta-1.sfbay.Sun.COM> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Sat, 11 Nov 2006 20:36:16 -0800 (PST) Received: from marduk.eng.sun.com (marduk.SFBay.Sun.COM [129.146.108.224]) by engmail3mpk.sfbay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id kAC4aFPi027567; Sat, 11 Nov 2006 20:36:15 -0800 (PST) Received: from marduk.eng.sun.com (localhost [127.0.0.1]) by marduk.eng.sun.com (8.13.6+Sun/8.12.11) with ESMTP id kAC4ded0002611; Sat, 11 Nov 2006 20:39:40 -0800 (PST) Received: (from gww@localhost) by marduk.eng.sun.com (8.13.6+Sun/8.12.11/Submit) id kAC4de0j002610; Sat, 11 Nov 2006 20:39:40 -0800 (PST) Date: Sat, 11 Nov 2006 20:39:40 -0800 (PST) From: Gary Winiger Subject: Restart: PSARC/2006/598 Swap resource control; locked memory RM improvements To: Stephen.Lawrence@sun.com, gww@eng.sun.com Cc: PSARC-EXT@sun.com, dp@eng.sun.com, zones-discuss@opensolaris.org Message-id: <200611120439.kAC4de0j002610@marduk.eng.sun.com> Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 11182 At the request of the project team, I'm restarting this case with the new spec below. The new timer is set for 20 Nov, 2006. The original spec is in the case directory as spec.orig. This spec is in the case directory as spec.txt. The project team didn't supply a summary of the changes, so I'll be asking for one in a follow on. Gary.. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Swap resource control; locked memory RM improvements Steve Lawrence SUMMARY: This case enhances Solaris Zones[1] and builds upon recent work to improve the integration between Zones and Solaris Resource Management[2]. The case addresses an existing RFE[6], which requests a mechanism to limit system swap reserved by a zone. The case also proposes extensions to [2], which will make swap reservation and locked memory resource controls easy to configure on a zone via zonecfg(1m). 1. This case proposes adding the following resource control: INTERFACE COMMITMENT BINDING "zone.max-swap" Committed Patch This control will limit the swap reserved by processes and tmpfs mounts within the global zone and non-global zones. This resource control serves to address the referenced RFE[6]. 2. To simplify the configuration of memory-related resource controls on zones, this case proposes adding the following properties to zonecfg(1M): INTERFACE COMMITMENT BINDING "swap" zonecfg property Committed Patch "locked" zonecfg property Committed Patch These properties will be added to the zonecfg "capped-memory" zonecfg resource introduced by [2]. 3. For observability of zone resource utilization and limits, this case proposes the addition of following kstats: INTERFACE COMMITMENT BINDING caps:{zoneid}:swapresv_zone_{zoneid} Uncommitted Patch caps:{zoneid}:lockedmem_zone_{zoneid} Uncommitted Patch To observe project resource utilization, this case also proposes the following kstat: INTERFACE COMMITMENT BINDING caps:{zoneid}:lockedmem_project_{projid} Uncommitted Patch The projid cannot be used as the instance number, as each zone has a unique project namespace. This means project 0 in the global zone is different from project 0 in each non global zone. The global zone will see kstats for all zones, while non global zones will only see kstats with matching zoneid. 4. prstat(1m) output changes to report swap reserved. INTERFACE COMMITMENT BINDING prstat(1m) output Uncommitted Patch This case proposes changing the "SIZE" column of "prstat -Z" zone output lines to "SWAP". The swap reported will be the total swap consumed by the zone's processes and tmpfs mounts. This value will assist administrators in monitoring the swap reserved by each zone, allowing them to choose a reasonable "zone.max-swap" settings. DETAIL: 1. "zone.max-swap" resource control. Limits swap consumed by user process address space mappings and tmpfs mounts within a zone. Currently a global or non-global zone can consume all swap resources available on the system, limiting the usefulness of zones as an application container. zone.max-swap provides a mechanism to limit swap consumption per zone. This will protect other zones from runaway memory leakers/consumers and/or tmpfs writers in a zone with zone.max-swap configured. Another solution to this problem would be a "swap set" [5] feature, which would allow the reservation of swap devices into sets to which zones could be bound. While "swap sets" would be useful, zone.max-swap provides a simple solution which is easier to administer, as it does not require the configuration of pools and swap devices/files. "zone.max-swap" is not incompatable with swap sets. In fact, a future addition of swap sets could be used in combination with zone.max-swap. For instance, several zones could be bound to the same set of swap devices, each with it's own individual zone.max-swap configured as a cap within that set. The implementation of "zone.max-swap" is also much less risky to make available via patch. zone.max-swap will be configurable on both the global zone, and non-global zones. The affect on processes in a zone reaching its zone.max-swap limit is the same as if all system swap is reserved. Callers of mmap(2) and sbrk(2) will receive EAGAIN. Writes to tmpfs will return ENOSPC, which is the same errno returned when a tmpfs mount reaches it's "size" mount option. The "size" mount option limits the quantity of swap that a tmpfs mount can reserve. While a low zone.max-swap setting for the global zone can lead to a difficult-to-administer global zone, the same problem exists today when configuring the zone.max-lwps resource control on the global zone, or when all system swap is reserved. The zonecfg(1m) enhancements detailed below will help administrators configure zone.max-swap safely. 2. "swap" and "locked" properties for zonecfg(1m) "capped_memory" resource. [2] added a new 'capped-memory' resource to zonecfg. This resource groups the properties used when capping memory for the zone. It currently has the 'physical' property which specifies the physical memory cap for the zone. We will add two new properties, 'swap' and 'locked' to the "capped-memory" resource. These properties will be added by using the rctl alias mechanism which is also described in [2]. swap: An unsigned decimal number with a required k, m, g, or t modifier. A value of '10m' means ten megabytes." This will be used to configure the zone.max-swap resource control, which limits swap consumed by processes and tmpfs mounts within a zone. locked: An unsigned decimal number with a required k, m, g, or t modifier. A value of '10m' means ten megabytes." This will be used to configure the zone.max-locked-memory[3,4] resource control, which limits locked physical memory (made non-pageable) by processes within a zone. To prevent administrators from configuring a low swap limit that will prevent a system from booting, zonecfg will not allow a swap limit to be configured to less than: Global zone: 100M Non-global zone: 50M. These numbers are based on the swap needed to boota zone after a default installation. Also, if zone.max-swap is configured (via zonecfg(1m)) on the global zone, a warning will be printed: global:capped-memory> set swap=200M Warning: Setting capped swap on the global zone can impact system availability. Similar warnings will be printed for setting other rctls on the global zone which can affect availability, such as zone.max-lwps. 3. For observability of zone resource utilization and limits, this case proposes the addition of following kstats: INTERFACE COMMITMENT BINDING caps:{zoneid}:swapresv_zone_{zoneid} Uncommitted Patch caps:{zoneid}:lockedmem_zone_{zoneid} Uncommitted Patch To observe project resource utilization, this case also proposes the following kstat: INTERFACE COMMITMENT BINDING caps:{zoneid}:lockedmem_project_{projid} Uncommitted Patch The projid cannot be used as the instance number, as each zone has a unique project namespace. This means project 0 in the global zone is different from project 0 in each non global zone. The global zone will see kstats for all zones, while non global zones will only see kstats with matching zoneid. Each kstat will have the statistics: usage: The current quantity of resource consumed. value: The current enforced cap. zonename: The name of the zone. A zone may change zoneid each time it boots, so this statistic helps to match the kstat to the zone. These kstats can be consumed by higher level tools/scripts to provide information about zone memory usage. Each kstats instance number matches the zoneid of the zone it represents. Non-global zones will only be able to read the kstat with matching zoneid. The global zone will be able to read all kstats. Additional kstats will be added in the future to report usage and cap for other rctls. Addressing existing rctls is outside the scope of this case. 4. prstat(1m) output changes to report swap reserved. INTERFACE COMMITMENT BINDING prstat(1m) output Uncommitted Patch This case proposes changing the "SIZE" column of "prstat -Z" zone output lines to "SWAP". The swap reported will be the total swap consumed by the zone's processes and tmpfs mounts. This value will assist administrators in monitoring the swap reserved by each zone, allowing them to choose a reasonable "zone.max-swap" settings. The "SIZE" column will also be changed to "SWAP" for prstat options a, T, and J, for users, tasks, and projects. The current "SIZE" column arbitrarily sums the address spaces of the processes in each zone. This sum include device mappings, but does not include NORESERVE segments. This sum does not map to real system resources, and therefore provides no meaningful information when summed across all processes belonging to a zone, project, task, or user. For the default prstat process listing, "SIZE" will not be changed to swap, as the virtual address space size for each process is a useful number. Detailed per process memory consumption reporting is outside the scope of this case, and would be better addressed by a case proposing a solution for 6487372[7]: RFE: prstat -x: Providing VSZ/RSS/ANON/LOCK Memory & CPU Usage This RFE requests displaying detailed memory usage per process. "SWAP" reservation certainly falls into this category. REFERENCES: [1] PSARC/2002/174 Virtualization and Namespace Isolation in Solaris http://sac.sfbay.sun.com/PSARC/2002/174 http://www.opensolaris.org/os/community/arc/caselog/2002/174/ [2] PSARC/2006/496 Improved Zones/RM Integration http://sac.sfbay.sun.com/PSARC/2006/496/ http://www.opensolaris.org/os/community/arc/caselog/2006/496/ [3] PSARC/2006/463 Amendment to zone/project.max-locked-memory Resource Controls http://sac.sfbay.sun.com/PSARC/2006/463/ http://www.opensolaris.org/os/community/arc/caselog/2006/463/ [4] PSARC/2004/580 zone/project.max-locked-memory Resource Controls http://sac.sfbay.sun.com/PSARC/2004/580/ http://www.opensolaris.org/os/community/arc/caselog/2004/580/ [5] PSARC/2002/181 Swap Sets http://sac.sfbay.sun.com/PSARC/2002/181/ http://www.opensolaris.org/os/community/arc/caselog/2002/181/ [6] 5103071 RFE: local zones can run the global zone out of swap http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=5103071 [7] RFE: prstat -x: Providing VSZ/RSS/ANON/LOCK Memory & CPU Usage http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6487372 From gww@eng.sun.com Sat Nov 11 20:59:27 2006 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id kAC4xRmb018805 for ; Sat, 11 Nov 2006 20:59:27 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.7+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id kAC4xDms006010 for <@sunmail1brm.central.sun.com:PSARC-EXT@sun.com>; Sun, 12 Nov 2006 04:59:26 GMT Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0J8L00F0FPV11Q00@brm-avmta-1.central.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Sat, 11 Nov 2006 21:59:25 -0700 (MST) Received: from engmail3mpk.sfbay.Sun.COM ([129.146.11.26]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0J8L005GWPV1D960@brm-avmta-1.central.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Sat, 11 Nov 2006 21:59:25 -0700 (MST) Received: from marduk.eng.sun.com (marduk.SFBay.Sun.COM [129.146.108.224]) by engmail3mpk.sfbay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id kAC4xNaK020035; Sat, 11 Nov 2006 20:59:23 -0800 (PST) Received: from marduk.eng.sun.com (localhost [127.0.0.1]) by marduk.eng.sun.com (8.13.6+Sun/8.12.11) with ESMTP id kAC52m1C002646; Sat, 11 Nov 2006 21:02:48 -0800 (PST) Received: (from gww@localhost) by marduk.eng.sun.com (8.13.6+Sun/8.12.11/Submit) id kAC52mrJ002645; Sat, 11 Nov 2006 21:02:48 -0800 (PST) Date: Sat, 11 Nov 2006 21:02:48 -0800 (PST) From: Gary Winiger Subject: Re: Restart: PSARC/2006/598 Swap resource control; locked memory RM improvements To: Stephen.Lawrence@sun.com, gww@eng.sun.com Cc: PSARC-EXT@sun.com, dp@eng.sun.com, zones-discuss@opensolaris.org Message-id: <200611120502.kAC52mrJ002645@marduk.eng.sun.com> Content-transfer-encoding: 7BIT X-Sun-Charset: US-ASCII X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 2667 First off, sorry for the stutter in the spec update mail. > The project team didn't supply a summary of the changes, so I'll be > asking for one in a follow on. Summary of changes please. > 1. This case proposes adding the following resource control: > > INTERFACE COMMITMENT BINDING > "zone.max-swap" Committed Patch > > This control will limit the swap reserved by processes and tmpfs > mounts within the global zone and non-global zones. This resource > control serves to address the referenced RFE[6]. There was some considerable discussion on the global zone aspect of this part of the proposal. Perhaps I missed in the spec how the new proposal mitigates the risk of the global zone not being able to administer the system. > DETAIL: > > 1. "zone.max-swap" resource control. > > Limits swap consumed by user process address space mappings and > tmpfs mounts within a zone. > While a low zone.max-swap setting for the global zone can lead to > a difficult-to-administer global zone, the same problem exists > today when configuring the zone.max-lwps resource control on the > global zone, or when all system swap is reserved. The zonecfg(1m) > enhancements detailed below will help administrators configure > zone.max-swap safely. Perhaps I misunderstood the interaction between project 0 and zone.max-lwps in the global zone. If a max-lwps is set is project 0 bound by it? Perhaps a short summary of the offline discussion on project 0 and the project teams feeling that the discussions conclusions might not be patch qualified. I realize the need for this project to have a patch binding. > 2. "swap" and "locked" properties for zonecfg(1m) "capped_memory" > resource. > To prevent administrators from configuring a low swap limit that > will prevent a system from booting, zonecfg will not allow a > swap limit to be configured to less than: > > Global zone: 100M > Non-global zone: 50M. > > These numbers are based on the swap needed to boota zone after a > default installation. > > Also, if zone.max-swap is configured (via zonecfg(1m)) on the > global zone, a warning will be printed: > > global:capped-memory> set swap=200M > Warning: Setting capped swap on the global zone can impact > system availability. > > Similar warnings will be printed for setting other rctls on the > global zone which can affect availability, such as zone.max-lwps. I don't doubt that 100M and 50M are currently reasonable numbers, however, how will they be tracked (computed/changed) in future. Gary.. From sl108498@steve1.sfbay.sun.com Mon Nov 13 16:35:23 2006 Received: from sunmail1brm.Central.Sun.COM (sunmail1brm.Central.Sun.COM [129.147.62.17]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id kAE0ZNAg000473 for ; Mon, 13 Nov 2006 16:35:23 -0800 (PST) Received: from nwk-avmta-1.sfbay.sun.com (nwk-avmta-1.SFBay.Sun.COM [129.149.246.28]) by sunmail1brm.Central.Sun.COM (8.11.7p1+Sun/8.11.7/ENSMAIL,v2.2) with ESMTP id kAE0ZMM11993 for <@sunmail2.sfbay.sun.com:PSARC-EXT@sun.com>; Mon, 13 Nov 2006 17:35:22 -0700 (MST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2 (built Dec 2 2004)) id <0J8P000052YYHX00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Mon, 13 Nov 2006 16:35:22 -0800 (PST) Received: from sfbaymail2sca.sfbay.sun.com ([129.145.155.42]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2 (built Dec 2 2004)) with ESMTP id <0J8P00I602YY7WC0@nwk-avmta-1.sfbay.Sun.COM> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Mon, 13 Nov 2006 16:35:22 -0800 (PST) Received: from steve1.sfbay.sun.com (steve1.SFBay.Sun.COM [129.146.228.230]) by sfbaymail2sca.sfbay.sun.com (8.13.6+Sun/8.12.10/ENSMAIL,v2.2) with ESMTP id kAE0ZL84025007; Mon, 13 Nov 2006 16:35:22 -0800 (PST) Received: from steve1.sfbay.sun.com (localhost [127.0.0.1]) by steve1.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id kAE0Ybxg014525; Mon, 13 Nov 2006 16:34:37 -0800 (PST) Received: (from sl108498@localhost) by steve1.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id kAE0Yb5E014524; Mon, 13 Nov 2006 16:34:37 -0800 (PST) Date: Mon, 13 Nov 2006 16:34:37 -0800 From: Steve Lawrence Subject: Re: Restart: PSARC/2006/598 Swap resource control; locked memory RM improvements In-reply-to: <200611120502.kAC52mrJ002645@marduk.eng.sun.com> To: Gary Winiger Cc: Stephen.Lawrence@sun.com, PSARC-EXT@sun.com, dp@eng.sun.com, zones-discuss@opensolaris.org Message-id: <20061114003437.GB14411@steve1.eng.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <200611120502.kAC52mrJ002645@marduk.eng.sun.com> User-Agent: Mutt/1.4.2.1i Status: RO Content-Length: 7107 On Sat, Nov 11, 2006 at 09:02:48PM -0800, Gary Winiger wrote: > > First off, sorry for the stutter in the spec update mail. > > > The project team didn't supply a summary of the changes, so I'll be > > asking for one in a follow on. > I've addressed your comments way below. Here is my change summary and case discussion summary: SUMMARY OF CHANGES 1. Change to the proposed uncommitted kstat names and statistics. From the form: zone:{zoneid}:vm with statistics: zonename swap_reserved max_swap_reserved locked_memory max_locked_memory To the form: caps:{zoneid}:swaprsev_zone_{zoneid} caps:{zoneid}:lockedmem_zone_{zoneid} caps:{zoneid}:lockedmem_project_{projid} with statistics: zonename usage value This sets up a generic scheme for adding kstats to project and zone rctls. A kstat is created per rctl, instead of per zone. 2. Addition of zonecfg(1m) minimums for setting zone.max-swap. When setting zone.max-swap via zonecfg(1m), a minimum value will be enforced: global zone: 100M non-global zone: 50M Currently, this is about 20M more than is needed to boot after a default installation. 3. Addition of zonecfg(1m) warnings when setting zone.max-swap and zone.max-lwps on the global zone. global:capped-memory> set swap=200M Warning: Setting capped swap on the global zone can impact system availability. SUMMARY OF CASE DISCUSSION: The case disussion has focused on the problem that the zone.max-swap rctl on the global zone can affect system availability. An identical problem exists today with task/project/zone.max-lwps. Solutions to this problem may involve one or more of: - Exempting project 0 in the global zone from zone.* rctls. - Preventing task/project.* rctls from being set on project 0 in the global zone. - Modifying root's default project. - Adding a new privilege to exempt a process from rctls. - Updating system service manifests to drop the new privilege. Solving this problem in a way that will prevent the global zone (on a default system) from becoming unavailable due to a resource control setting will require a signficant change to the system. I believe solving this problem is outside the scope of the "zone.max-swap" case, and would be better solved by another case which is "not" seeking patch binding. To minimize this problem for zone.max-swap (and zone.max-lwps), I've instead proposed zonecfg enhacements to assist the admin in configuring these rctls safely. > > > 1. This case proposes adding the following resource control: > > > > INTERFACE COMMITMENT BINDING > > "zone.max-swap" Committed Patch > > > > This control will limit the swap reserved by processes and tmpfs > > mounts within the global zone and non-global zones. This resource > > control serves to address the referenced RFE[6]. > > There was some considerable discussion on the global zone aspect > of this part of the proposal. Perhaps I missed in the spec how > the new proposal mitigates the risk of the global zone not being > able to administer the system. > > > DETAIL: > > > > 1. "zone.max-swap" resource control. > > > > Limits swap consumed by user process address space mappings and > > tmpfs mounts within a zone. > > > While a low zone.max-swap setting for the global zone can lead to > > a difficult-to-administer global zone, the same problem exists > > today when configuring the zone.max-lwps resource control on the > > global zone, or when all system swap is reserved. The zonecfg(1m) > > enhancements detailed below will help administrators configure > > zone.max-swap safely. > > Perhaps I misunderstood the interaction between project 0 > and zone.max-lwps in the global zone. If a max-lwps is set > is project 0 bound by it? Currently yes. zone.* rctls bound all processes in the global zone, regardless of project. This is the issue that my "other" proposal is attempting to address. > Perhaps a short summary of the offline discussion on project 0 > and the project teams feeling that the discussions conclusions > might not be patch qualified. I realize the need for this project > to have a patch binding. I've added this summary above. > > > 2. "swap" and "locked" properties for zonecfg(1m) "capped_memory" > > resource. > > > To prevent administrators from configuring a low swap limit that > > will prevent a system from booting, zonecfg will not allow a > > swap limit to be configured to less than: > > > > Global zone: 100M > > Non-global zone: 50M. > > > > These numbers are based on the swap needed to boota zone after a > > default installation. > > > > Also, if zone.max-swap is configured (via zonecfg(1m)) on the > > global zone, a warning will be printed: > > > > global:capped-memory> set swap=200M > > Warning: Setting capped swap on the global zone can impact > > system availability. > > > > Similar warnings will be printed for setting other rctls on the > > global zone which can affect availability, such as zone.max-lwps. > > I don't doubt that 100M and 50M are currently reasonable numbers, > however, how will they be tracked (computed/changed) in future. Good question. These are essentially "virtual system requirements". For the global zone, we could compute this number "on demand", such as during system boot, instead of using a hard-coded value. Unfortunately, the resource management and zones services execute before X is started, so if these services watermark the minimum swap value, they will under-estimate. We could hack the milestone services to cache the amount of reserved swap when they complete. We could then use that value (plus some buffer) as the minimum. For non-global-zones, there is a bit of a "chicken-and-egg" problem. It is impossible to know how much swap a zone will need before booting it. Of course, under-estimating for a non-global zone is not catastrophic. We could just up this minumum when somebody files a bug that it is too low. We should probally also not "hard enforce" these minimum in zonecfg, but rather warn verbosely. This will allow admins that "really want it" to get it. It will also allow us to amply over-estimate the minimums, which is safer. In general, it wouldn't be bad if the various milestones recorded the resource utilization (swap, rss, lwps, processes, etc) when they are reached. Ugg, looks like zones and X services start "after" multi-user server. There is no actual milestone service that maps to the "all" milestone. So much for snapshotting swap usage vi milestone start scripts. I'd rather not hard code this resource utilization snapshot into startd. Choosing better minimums can be addressed by: - Make the global zone swap minimum to the release system memory requirements. This number should be available. - Leave the non-global zone minimum at 50M - Make violating the zonecfg minimum a "very verbose warning" instead of an error. -Steve -Steve > > Gary.. From Darren.Reed@sun.com Mon Nov 13 18:01:13 2006 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id kAE21DIA002409 for ; Mon, 13 Nov 2006 18:01:13 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.7+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id kAE2176o014328 for <@sunmail1brm.central.sun.com:PSARC-EXT@sun.com>; Tue, 14 Nov 2006 02:01:12 GMT Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0J8P00F016XY2B00@brm-avmta-1.central.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Mon, 13 Nov 2006 19:01:10 -0700 (MST) Received: from sineb-mail-1.sun.com ([192.18.19.6]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0J8P00B1X6XVSBB0@brm-avmta-1.central.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Mon, 13 Nov 2006 19:01:08 -0700 (MST) Received: from fe-apac-03.sun.com (fe-apac-03.sun.com [192.18.19.174] (may be forged)) by sineb-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id kAE216Rv022147 for ; Tue, 14 Nov 2006 10:01:07 +0800 (SGT) Received: from conversion-daemon.mail-apac.sun.com by mail-apac.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0J8P001016OJG400@mail-apac.sun.com> (original mail from Darren.Reed@Sun.COM) for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Tue, 14 Nov 2006 10:01:06 +0800 (SGT) Received: from [129.146.106.55] by mail-apac.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0J8P00EXV6XRJ6D1@mail-apac.sun.com>; Tue, 14 Nov 2006 10:01:05 +0800 (SGT) Date: Mon, 13 Nov 2006 17:58:58 -0800 From: Darren.Reed@sun.com Subject: Re: Restart: PSARC/2006/598 Swap resource control; locked memory RM improvements In-reply-to: <20061114003437.GB14411@steve1.eng.sun.com> Sender: Darren.Reed@sun.com To: Steve Lawrence Cc: Gary Winiger , PSARC-EXT@sun.com, dp@eng.sun.com, zones-discuss@opensolaris.org Message-id: <455922E2.7080009@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=us-ascii Content-transfer-encoding: 7BIT X-Accept-Language: en-us, en X-PMX-Version: 5.2.0.264296 References: <200611120502.kAC52mrJ002645@marduk.eng.sun.com> <20061114003437.GB14411@steve1.eng.sun.com> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.7) Gecko/20060120 Status: RO Content-Length: 5850 Steve Lawrence wrote: >On Sat, Nov 11, 2006 at 09:02:48PM -0800, Gary Winiger wrote: > > >> First off, sorry for the stutter in the spec update mail. >> >> >> >>>The project team didn't supply a summary of the changes, so I'll be >>>asking for one in a follow on. >>> >>> > >I've addressed your comments way below. Here is my change summary and case >discussion summary: > >SUMMARY OF CHANGES > >1. Change to the proposed uncommitted kstat names and statistics. > From the form: > > zone:{zoneid}:vm > > with statistics: > zonename > swap_reserved > max_swap_reserved > locked_memory > max_locked_memory > > To the form: > > caps:{zoneid}:swaprsev_zone_{zoneid} > caps:{zoneid}:lockedmem_zone_{zoneid} > caps:{zoneid}:lockedmem_project_{projid} > > with statistics: > zonename > usage > value > > This sets up a generic scheme for adding kstats to project and > zone rctls. A kstat is created per rctl, instead of per zone. > >2. Addition of zonecfg(1m) minimums for setting zone.max-swap. > > When setting zone.max-swap via zonecfg(1m), a minimum value will be > enforced: > > global zone: 100M > non-global zone: 50M > > Currently, this is about 20M more than is needed to boot after a > default installation. > >3. Addition of zonecfg(1m) warnings when setting zone.max-swap and > zone.max-lwps on the global zone. > > global:capped-memory> set swap=200M > Warning: Setting capped swap on the global zone can impact > system availability. > >SUMMARY OF CASE DISCUSSION: > >The case disussion has focused on the problem that the zone.max-swap >rctl on the global zone can affect system availability. An identical >problem exists today with task/project/zone.max-lwps. > >Solutions to this problem may involve one or more of: > > - Exempting project 0 in the global zone from zone.* rctls. > - Preventing task/project.* rctls from being set on project 0 > in the global zone. > - Modifying root's default project. > - Adding a new privilege to exempt a process from rctls. > - Updating system service manifests to drop the new privilege. > >Solving this problem in a way that will prevent the global zone (on a >default system) from becoming unavailable due to a resource control setting >will require a signficant change to the system. I believe solving this >problem is outside the scope of the "zone.max-swap" case, and would be better >solved by another case which is "not" seeking patch binding. > >To minimize this problem for zone.max-swap (and zone.max-lwps), I've instead >proposed zonecfg enhacements to assist the admin in configuring these rctls >safely. > > > >>> 1. This case proposes adding the following resource control: >>> >>> INTERFACE COMMITMENT BINDING >>> "zone.max-swap" Committed Patch >>> >>> This control will limit the swap reserved by processes and tmpfs >>> mounts within the global zone and non-global zones. This resource >>> control serves to address the referenced RFE[6]. >>> >>> >> There was some considerable discussion on the global zone aspect >> of this part of the proposal. Perhaps I missed in the spec how >> the new proposal mitigates the risk of the global zone not being >> able to administer the system. >> >> >> >>>DETAIL: >>> >>> 1. "zone.max-swap" resource control. >>> >>> Limits swap consumed by user process address space mappings and >>> tmpfs mounts within a zone. >>> >>> >>> While a low zone.max-swap setting for the global zone can lead to >>> a difficult-to-administer global zone, the same problem exists >>> today when configuring the zone.max-lwps resource control on the >>> global zone, or when all system swap is reserved. The zonecfg(1m) >>> enhancements detailed below will help administrators configure >>> zone.max-swap safely. >>> >>> >> Perhaps I misunderstood the interaction between project 0 >> and zone.max-lwps in the global zone. If a max-lwps is set >> is project 0 bound by it? >> >> > >Currently yes. zone.* rctls bound all processes in the global zone, regardless >of project. This is the issue that my "other" proposal is attempting to >address. > > > >> Perhaps a short summary of the offline discussion on project 0 >> and the project teams feeling that the discussions conclusions >> might not be patch qualified. I realize the need for this project >> to have a patch binding. >> >> > >I've added this summary above. > > > >>> 2. "swap" and "locked" properties for zonecfg(1m) "capped_memory" >>> resource. >>> >>> >>> To prevent administrators from configuring a low swap limit that >>> will prevent a system from booting, zonecfg will not allow a >>> swap limit to be configured to less than: >>> >>> Global zone: 100M >>> Non-global zone: 50M. >>> >>> These numbers are based on the swap needed to boota zone after a >>> default installation. >>> >>> Also, if zone.max-swap is configured (via zonecfg(1m)) on the >>> global zone, a warning will be printed: >>> >>> global:capped-memory> set swap=200M >>> Warning: Setting capped swap on the global zone can impact >>> system availability. >>> >>> Similar warnings will be printed for setting other rctls on the >>> global zone which can affect availability, such as zone.max-lwps. >>> >>> >> I don't doubt that 100M and 50M are currently reasonable numbers, >> however, how will they be tracked (computed/changed) in future. >> >> > >Good question. These are essentially "virtual system requirements". > > What is the behaviour of Solaris intended to be when someone makes these changes (or attempts to make them) on a system that has no swap space? Furthermore, why shouldn't I be able to say a zone has no swap space available to it - i.e. to force it to all run from RAM? Darren From mgerdts@gmail.com Mon Nov 13 19:54:22 2006 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id kAE3sMqM004120 for ; Mon, 13 Nov 2006 19:54:22 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id kAE3sJQV028138 for <@sunmail1brm.central.sun.com:PSARC-EXT@sun.com>; Mon, 13 Nov 2006 19:54:20 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0J8P00K3HC6JNE00@brm-avmta-1.central.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Mon, 13 Nov 2006 20:54:19 -0700 (MST) Received: from brmea-mail-3.sun.com ([192.18.98.34]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0J8P00H1YC6HE630@brm-avmta-1.central.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Mon, 13 Nov 2006 20:54:17 -0700 (MST) Received: from relay11.sun.com (relay11.sun.com [217.140.40.14] (may be forged)) by brmea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id kAE3sGpf010328 for ; Mon, 13 Nov 2006 20:54:16 -0700 (MST) Received: from mms12es.sun.com ([160.41.223.34] [160.41.223.34]) by relay11.sun.com with ESMTP for PSARC-EXT@sun.com; Tue, 14 Nov 2006 03:54:15 +0000 (Z) Received: from relay13.sun.com (relay13.sun.com [217.140.40.54]) by mms12es.sun.com with ESMTP for PSARC-EXT@sun.com; Tue, 14 Nov 2006 03:54:14 +0000 (Z) Received: from ug-out-1314.google.com ([66.249.92.172] [66.249.92.172]) by relay13.sun.com with ESMTP for PSARC-EXT@sun.com; Tue, 14 Nov 2006 03:36:22 +0000 (Z) Received: by ug-out-1314.google.com with SMTP id z27so1258862ugc for ; Mon, 13 Nov 2006 19:35:09 -0800 (PST) Received: by 10.78.17.1 with SMTP id 1mr476716huq.1163475308767; Mon, 13 Nov 2006 19:35:08 -0800 (PST) Received: by 10.78.144.8 with HTTP; Mon, 13 Nov 2006 19:35:08 -0800 (PST) Date: Mon, 13 Nov 2006 21:35:08 -0600 From: Mike Gerdts Subject: Re: [zones-discuss] Re: Restart: PSARC/2006/598 Swap resource control; locked memory RM improvements In-reply-to: <455922E2.7080009@Sun.COM> To: "Darren.Reed@sun.com" Cc: Steve Lawrence , PSARC-EXT@sun.com, zones-discuss@opensolaris.org, Gary Winiger Message-id: <65f8f3ad0611131935u619eb2bawf2097557e9ece44d@mail.gmail.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT Content-disposition: inline DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=O+GjdhJKRMG/BrKyA4mBClSY8ScAwYB9ZkbA/Q8nP8pNTOKCWW2C4ZNsiHzb4KedGcFrUIhwJ3HbMUT9t7vewevL9lUCChcBcFOSuGSxtfyKQ3e3KfrzHRpB8OjDQqyzDX+XuiZ50q5EhSh3J5ucvhbBiGC0gR0aSeYGbwXRMpc= X-PMX-Version: 5.2.0.264296 References: <200611120502.kAC52mrJ002645@marduk.eng.sun.com> <20061114003437.GB14411@steve1.eng.sun.com> <455922E2.7080009@Sun.COM> Status: RO Content-Length: 343 On 11/13/06, Darren.Reed@sun.com wrote: > What is the behaviour of Solaris intended to be when someone > makes these changes (or attempts to make them) on a system > that has no swap space? > > Furthermore, why shouldn't I be able to say a zone has no swap > space available to it - i.e. to force it to all run from RAM? >From the context, I would say that swap in this definition is the Status: RO definition of "virtual swap" given in "System Administration Guide: Devices and File Systems" (and no where else as far as I can tell): The Solaris OS uses the concept of virtual swap space, a layer between anonymous memory pages and the physical storage (or disk-backed swap space) that actually back these pages. A system's virtual swap space is equal to the sum of all its physical (disk-backed) swap space plus a portion of the currently available physical memory. I've seen confusion on this among many sysadmins and had a short while ago brought it up without PSARC-EXT on the reply list. Mike -- Mike Gerdts http://mgerdts.blogspot.com/ From sl108498@steve1.sfbay.sun.com Tue Nov 14 10:33:31 2006 Received: from sunmail1brm.Central.Sun.COM (sunmail1brm.Central.Sun.COM [129.147.62.17]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id kAEIXVv5024223 for ; Tue, 14 Nov 2006 10:33:31 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail1brm.Central.Sun.COM (8.11.7p1+Sun/8.11.7/ENSMAIL,v2.2) with ESMTP id kAEIXVM27094 for <@sunmail1brm.central.sun.com:PSARC-EXT@sun.com>; Tue, 14 Nov 2006 11:33:31 -0700 (MST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0J8Q00K03GVUW500@brm-avmta-1.central.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Tue, 14 Nov 2006 11:33:30 -0700 (MST) Received: from sfbaymail2sca.sfbay.sun.com ([129.145.155.42]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0J8Q00J1JGVU8N30@brm-avmta-1.central.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Tue, 14 Nov 2006 11:33:30 -0700 (MST) Received: from steve1.sfbay.sun.com (steve1.SFBay.Sun.COM [129.146.228.230]) by sfbaymail2sca.sfbay.sun.com (8.13.6+Sun/8.12.10/ENSMAIL,v2.2) with ESMTP id kAEIXSJJ002803; Tue, 14 Nov 2006 10:33:28 -0800 (PST) Received: from steve1.sfbay.sun.com (localhost [127.0.0.1]) by steve1.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id kAEIWf7V014804; Tue, 14 Nov 2006 10:32:41 -0800 (PST) Received: (from sl108498@localhost) by steve1.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id kAEIWfcc014803; Tue, 14 Nov 2006 10:32:41 -0800 (PST) Date: Tue, 14 Nov 2006 10:32:41 -0800 From: Steve Lawrence Subject: Re: Restart: PSARC/2006/598 Swap resource control; locked memory RM improvements In-reply-to: <455922E2.7080009@Sun.COM> To: Darren.Reed@sun.com Cc: Steve Lawrence , Gary Winiger , PSARC-EXT@sun.com, dp@eng.sun.com, zones-discuss@opensolaris.org Message-id: <20061114183241.GD14411@steve1.eng.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <200611120502.kAC52mrJ002645@marduk.eng.sun.com> <20061114003437.GB14411@steve1.eng.sun.com> <455922E2.7080009@Sun.COM> User-Agent: Mutt/1.4.2.1i Status: RO Content-Length: 1184 > >Good question. These are essentially "virtual system > >requirements". > > > > > > What is the behaviour of Solaris intended to be when someone > makes these changes (or attempts to make them) on a system > that has no swap space? All systems have reservable swap space. Systems with no swap devices use physical memory to back swap reservations. > > Furthermore, why shouldn't I be able to say a zone has no swap > space available to it - i.e. to force it to all run from RAM? Solaris's vm system has no such concept. All anonymous allocations reserve swap. I think you suggesting a zone "switch" so that an admin can choose from one of: A. reserve swap from disk only B. reserve swap from memory only C. reserve swap from disk, then memory D. reserve swap from memory, then disk. Currently, system behavior is C for everyone. zone.max-swap simply limits swap reservation. It does not provide an interface for choosing a swap allocation policy. These concepts are orthogonal. I can see a "swap sets" feature addressing allocation policy, since "swap sets" could be used to associate a given zone with a particular set of swap devices. -Steve > > Darren > From sacadmin Wed Nov 22 09:17:24 2006 Received: from sunmail1brm.Central.Sun.COM (sunmail1brm.Central.Sun.COM [129.147.62.17]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id kAMHHOH6002133 for ; Wed, 22 Nov 2006 09:17:24 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail1brm.Central.Sun.COM (8.11.7p1+Sun/8.11.7/ENSMAIL,v2.2) with ESMTP id kAMHHN404302; Wed, 22 Nov 2006 10:17:23 -0700 (MST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0J9500L1N6OXQG00@nwk-avmta-2.sfbay.sun.com>; Wed, 22 Nov 2006 09:17:21 -0800 (PST) Received: from nwk-ea-fw-1.sun.com ([10.4.134.6]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0J9500FFC6OWJ170@nwk-avmta-2.sfbay.sun.com>; Wed, 22 Nov 2006 09:17:20 -0800 (PST) Received: from d1-sfbay-10.sun.com ([192.18.39.120]) by nwk-ea-fw-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id kAMHHK3Q026820; Wed, 22 Nov 2006 09:17:20 -0800 (PST) Received: from conversion-daemon.d1-sfbay-10.sun.com by d1-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0J9500B016LGB200@d1-sfbay-10.sun.com> (original mail from Sherri.Shieh@Sun.COM); Wed, 22 Nov 2006 09:17:20 -0800 (PST) Received: from [129.146.11.172] by d1-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0J9500D2E6OVNRN2@d1-sfbay-10.sun.com>; Wed, 22 Nov 2006 09:17:20 -0800 (PST) Date: Wed, 22 Nov 2006 09:17:19 -0800 From: Sherri Shieh Subject: PSARC Fast track: Swap resource control; locked memory RM improvements (2006/598) Sender: Sherri.Shieh@Sun.COM To: psarc@Sun.COM, Dan Price , Stephen Lawrence Message-id: <4564861F.1020209@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=us-ascii Content-transfer-encoding: 7BIT X-Accept-Language: en-us, en X-PMX-Version: 5.2.0.264296 User-Agent: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.7) Gecko/20060120 Status: RO Content-Length: 353 This fast track was approved in last week's meeting. I have marked it closed approved. - Sherri ========================================================= Sherri Shieh Sun Microsystems, Inc. Program Manager Email: sherri.shieh@sun.com Systems Architecture Phone: 650-786-5245/x85245 =========================================================== From sacadmin Wed Nov 22 11:12:18 2006 Received: from sunmail1brm.Central.Sun.COM (sunmail1brm.Central.Sun.COM [129.147.62.17]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id kAMJCIFh008390 for ; Wed, 22 Nov 2006 11:12:18 -0800 (PST) Received: from nwk-avmta-1.sfbay.sun.com (nwk-avmta-1.SFBay.Sun.COM [129.149.246.28]) by sunmail1brm.Central.Sun.COM (8.11.7p1+Sun/8.11.7/ENSMAIL,v2.2) with ESMTP id kAMJCH408076 for <@sunmail2.sfbay.sun.com:psarc@Sun.COM>; Wed, 22 Nov 2006 12:12:17 -0700 (MST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2 (built Dec 2 2004)) id <0J9500I0CC0FL300@nwk-avmta-1.sfbay.Sun.COM> for psarc@Sun.COM (ORCPT psarc@Sun.COM); Wed, 22 Nov 2006 11:12:15 -0800 (PST) Received: from engmail3mpk.sfbay.Sun.COM ([129.146.11.26]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2 (built Dec 2 2004)) with ESMTP id <0J9500EUAC0EL620@nwk-avmta-1.sfbay.Sun.COM> for psarc@Sun.COM (ORCPT psarc@Sun.COM); Wed, 22 Nov 2006 11:12:14 -0800 (PST) Received: from snowdog.eng.sun.com (snowdog.SFBay.Sun.COM [129.146.228.213]) by engmail3mpk.sfbay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id kAMJCDeI007862; Wed, 22 Nov 2006 11:12:13 -0800 (PST) Received: from snowdog.eng.sun.com (localhost [127.0.0.1]) by snowdog.eng.sun.com (8.13.8+Sun/8.13.8) with ESMTP id kAMJCDsF004400; Wed, 22 Nov 2006 11:12:13 -0800 (PST) Received: (from dp@localhost) by snowdog.eng.sun.com (8.13.8+Sun/8.13.8/Submit) id kAMJCDWt004399; Wed, 22 Nov 2006 11:12:13 -0800 (PST) Date: Wed, 22 Nov 2006 11:12:13 -0800 From: Dan Price Subject: Re: PSARC Fast track: Swap resource control; locked memory RM improvements (2006/598) In-reply-to: <4564861F.1020209@Sun.COM> To: Sherri Shieh Cc: psarc@Sun.COM, Dan Price , Stephen Lawrence Message-id: <20061122191212.GC4292@eng.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <4564861F.1020209@Sun.COM> User-Agent: Mutt/1.4.2.1i Status: RO Content-Length: 447 On Wed 22 Nov 2006 at 09:17AM, Sherri Shieh wrote: > This fast track was approved in last week's meeting. I have marked it > closed approved. > > - Sherri Steve informed me yesterday that he is still preparing a finalized spec which reflects the review. I'll update the case to "approved" then. For now, I've moved it back to waiting needs spec. -dp -- Daniel Price - Solaris Kernel Engineering - dp@eng.sun.com - blogs.sun.com/dp From sacadmin Wed Nov 29 11:12:06 2006 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id kATJC5hc018361 for ; Wed, 29 Nov 2006 11:12:05 -0800 (PST) Received: from nwk-avmta-1.sfbay.sun.com (nwk-avmta-1.SFBay.Sun.COM [129.149.246.28]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id kATJBjqp012033 for <@sunmail2.sfbay.sun.com:psarc@Sun.COM>; Thu, 30 Nov 2006 03:12:04 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2 (built Dec 2 2004)) id <0J9I00F07AO2B200@nwk-avmta-1.sfbay.Sun.COM> for psarc@Sun.COM (ORCPT psarc@Sun.COM); Wed, 29 Nov 2006 11:12:02 -0800 (PST) Received: from engmail3mpk.sfbay.Sun.COM ([129.146.11.26]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2 (built Dec 2 2004)) with ESMTP id <0J9I008XZAO1WT60@nwk-avmta-1.sfbay.Sun.COM> for psarc@Sun.COM (ORCPT psarc@Sun.COM); Wed, 29 Nov 2006 11:12:01 -0800 (PST) Received: from snowdog.eng.sun.com (snowdog.SFBay.Sun.COM [129.146.228.213]) by engmail3mpk.sfbay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id kATJC1NF016523; Wed, 29 Nov 2006 11:12:01 -0800 (PST) Received: from snowdog.eng.sun.com (localhost [127.0.0.1]) by snowdog.eng.sun.com (8.13.8+Sun/8.13.8) with ESMTP id kATJC1Eh001750; Wed, 29 Nov 2006 11:12:01 -0800 (PST) Received: (from dp@localhost) by snowdog.eng.sun.com (8.13.8+Sun/8.13.8/Submit) id kATJC0gU001749; Wed, 29 Nov 2006 11:12:00 -0800 (PST) Date: Wed, 29 Nov 2006 11:12:00 -0800 From: Dan Price Subject: Re: PSARC Fast track: Swap resource control; locked memory RM improvements (2006/598) In-reply-to: <20061122191212.GC4292@eng.sun.com> To: Sherri Shieh Cc: psarc@sun.com, Dan Price , Stephen Lawrence Message-id: <20061129191200.GC28247@eng.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <4564861F.1020209@Sun.COM> <20061122191212.GC4292@eng.sun.com> User-Agent: Mutt/1.4.2.1i Status: RO Content-Length: 681 On Wed 22 Nov 2006 at 11:12AM, Dan Price wrote: > On Wed 22 Nov 2006 at 09:17AM, Sherri Shieh wrote: > > This fast track was approved in last week's meeting. I have marked it > > closed approved. > > > > - Sherri > > Steve informed me yesterday that he is still preparing a finalized > spec which reflects the review. > > I'll update the case to "approved" then. For now, I've moved it back > to waiting needs spec. Steve supplied the updated spec file to me. I've updated the spec in the case directory, and marked the case approved. Thanks to all who helped out on this one. -dp -- Daniel Price - Solaris Kernel Engineering - dp@eng.sun.com - blogs.sun.com/dp