From sac-owner Mon May 17 11:45:33 2004 Received: from sunmail1brm.Central.Sun.COM (sunmail1brm.Central.Sun.COM [129.147.62.17]) by sac.sfbay.sun.com (8.12.9+Sun/8.12.9) with ESMTP id i4HIjXJJ015017 for ; Mon, 17 May 2004 11:45:33 -0700 (PDT) Received: (from noaccess@localhost) by sunmail1brm.Central.Sun.COM (8.11.7p1+Sun/8.11.7/ENSMAIL,v2.2) id i4HIjF404704 for one-pager-not-2b-used-directly; Mon, 17 May 2004 12:45:15 -0600 (MDT) Received: from sfbaymail1sca.SFBay.Sun.COM (sfbaymail1sca.SFBay.Sun.COM [129.145.154.35]) by sunmail1brm.Central.Sun.COM (8.11.7p1+Sun/8.11.7/ENSMAIL,v2.2) with ESMTP id i4HIjBL04624; Mon, 17 May 2004 12:45:11 -0600 (MDT) Received: from kotuk.sfbay.sun.com (kotuk.SFBay.Sun.COM [129.146.89.79]) by sfbaymail1sca.SFBay.Sun.COM (8.12.10+Sun/8.12.10/ENSMAIL,v2.2) with ESMTP id i4HIj9eD018117; Mon, 17 May 2004 11:45:09 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by kotuk.sfbay.sun.com (8.12.11+Sun/8.12.11) with ESMTP id i4HIipVP018229; Mon, 17 May 2004 11:44:51 -0700 (PDT) Date: Mon, 17 May 2004 11:44:51 -0700 (PDT) From: Andrei Dorofeev X-X-Sender: andrei@kotuk To: one-pager@Sun.COM Subject: [/]CPU caps Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Status: RO Content-Length: 6240 1. Introduction 1.1. Project/Component Working Name: CPU caps 1.2. Name of Document Author/Supplier: Andrei Dorofeev 1.3. Date of This Document: 05/16/04 1.4. Name of Major Document Customer(s)/Consumer(s): 1.4.1. The Steering Committees you expect to review your project ON Steering Committee 1.5. Email Aliases: 1.5.1. Responsible Manager: David Linder 1.5.2. Responsible Engineer: Andrei Dorofeev 2. Project Summary 2.1. Project Description This project will enhance the existing FSS scheduler by giving system administrators more control over the allocation of CPU resources between applications running on their systems. Fair Share Scheduler (FSS) is built around the classic share model where each application's CPU allocation depends on the ratio between shares assigned to that application and other applications and does not depend on how many processes it contains. This project will allow system administrators to define absolute upper limits (or caps) on how much CPU resources can be consumed by each application. Such caps could be used in addition to shares when it is desireable that applications that do not compete for the same CPU resources do not consume all available cycles. CPU caps could be also used to isolate workloads from each other by reducing their variability and producing more predictable behavior. Additionally, CPU caps can be used for dealing with various software licensing issues and for chargeback billing. If implemented, this feature would also allow system administrators to overcommit their CPU resources by setting caps below share-based entitlements. An ability to do so was requested by some customers. 2.2. Risks and Assumptions This project depends on another project (enabling microstate accounting by default) being delivered in Solaris 10. It is assumed that customers who are interested in CPU caps are going to be willing to switch from the default Time-Sharing (TS) scheduler to FSS. 3. Business Summary 3.1. Problem Area Processor sets feature in Solaris has been traditionally used for CPU capping in server consolidation environment on large multi-processor systems. Processor sets, however, can not contain partial CPUs to provide sub-CPU levels of service. Fair Share Scheduler, on another hand, could be used to provide sub-CPU levels of service, but it does not have a capping feature. Under existing FSS model, a single application can consume all available CPU resources if there are no other applications competing for them. 3.2. Market/Requester Some of Sun's strategic customers (financial institutions and banks) have requested this feature. Server consolidation is potentially a multi-billion dollar market for Sun. There are also internal groups at Sun interested in this feature (SunRay, GridEngine). 3.3. Business Justification This project will ensure that Solaris can be used in environments that require a precise control over how much CPU resources are allocated to each application. This feature will be very useful for customers who are trying to get the most out of their systems by increasing overall utilization levels and doing server consolidation. 3.4. Competitive Analysis Most of Sun's major competitors provide interfaces to define CPU caps in one form or another. IBM's Workload Manager and HP's Process Resource Manager both provide ways to limit CPU usage via shares and caps. 3.5. Opportunity Window/Exposure This feature is expected to be required by the time Solaris 10 becomes adopted. The ability to put CPU caps on zones is going to have a significant value in server consolidation environments. This feature is tentatively scheduled to be integrated into Solaris 10 Update 1. 3.6. How will you know when you are done? The project will be complete when all of its requirements have been met and it has been fully tested on all supported hardware, with no open P1/P2/P3 bugs filed against the project. 4. Technical Description The project will deliver new resource controls for projects and zones that will allow their CPU caps to be specified in units of 1/100th of a CPU. Processes that try to use more CPU resources than defined by their caps will be removed from the processors and put on special wait queues where they will wait for their containers to lower their CPU usage before they can be scheduled again. The time threads spend on these special queues waiting to be scheduled again will be included in their wait-cpu (LMS_WAIT_CPU) micro-state counters. When a project or a zone achieves its CPU cap without using its full share entitlement, the scheduler will automatically distribute the remaining CPU cycles between other projects and zones based on their share allocations. The use of CPU caps may increase the system's idle time and load averages. In comparison, the old "shares-only" approach would never waste unused CPU cycles. 5. Reference Documents A design document is currently being refined and will be available at: http://taxman.eng/projects/cpu-caps 6. Resources and Schedule 6.1. Projected Availability Q3CY04 6.2. Cost of Effort 1 Solaris kernel developer - 4-5 months 1 test engineer - 1 month 6.3. Cost of Capital Resources $0 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: OS/Net 6.4.2. Contributing OpCo/BU/Division Name: SSG/Solaris/SMOV 6.4.3. Type of SC Approval needed: Standard 6.4.4. Project Boundary Conditions: 6.4.5. Is this a necessary project for OEM agreements: No 6.4.6. Notes/Dependencies: Depends on microstate accounting project. 6.4.7. Target RTI Date/Release: S10 Update 1 6.5. ARC review type: Standard 7. Prototype Availability 7.1. Prototype Availability Q2CY04 7.2. Prototype Cost 1 engineer 2 months