From Daniel.Hain@sun.com Fri Mar 27 12:40:29 2009 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n2RJeTml010053 for ; Fri, 27 Mar 2009 12:40:29 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n2RJeSDW002898 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Fri, 27 Mar 2009 12:40:29 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KH600D0JJZGWU00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 27 Mar 2009 12:40:28 -0700 (PDT) Received: from sca-es-mail-2.sun.com ([192.18.43.133]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KH6007I6JZGJQ70@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 27 Mar 2009 12:40:28 -0700 (PDT) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n2RJeRsZ017771 for ; Fri, 27 Mar 2009 12:40:27 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7.0-5.01 64bit (built Feb 19 2009)) id <0KH600I00JIZOF00@fe-sfbay-10.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 27 Mar 2009 12:40:27 -0700 (PDT) Received: from [192.168.1.6] ([unknown] [72.197.209.207]) by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7.0-5.01 64bit (built Feb 19 2009)) with ESMTPSA id <0KH600IKZJZ8AIA0@fe-sfbay-10.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 27 Mar 2009 12:40:20 -0700 (PDT) Date: Fri, 27 Mar 2009 12:39:43 -0700 From: Daniel Hain Subject: Page Retirement Kernel KStat Addition [PSARC/2009/181 FastTrack timeout 04/03/2009] Sender: Daniel.Hain@sun.com To: PSARC-ext@sun.com Cc: Christopher Baumbauer Message-id: <49CD2B7F.8060207@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 User-Agent: Thunderbird 2.0.0.18 (X11/20090224) Status: RO Content-Length: 7252 I'm sponsoring this on behalf of Christopher Baumbauer. Exposure is open, seeking patch binding. Timeout set for 04/03/2009. - Dan Template Version: @(#)onepager.txt 1.35 07/11/07 SMI Copyright 2009 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: Page Retirement Kernel KStat Addition 1.2. Name of Document Author/Supplier: Author: Chris Baumbauer 1.3. Date of This Document: 03/16/2008 1.3.1. Date this project was conceived: 03/16/2008 1.4. Name of Major Document Customer(s)/Consumer(s): 1.4.1. The PAC or CPT you expect to review your project: Solaris PAC 1.4.2. The ARC(s) you expect to review your project: PSARC 1.4.3. The Director/VP who is "Sponsoring" this project: brian.reynard@sun.com 1.4.4. The name of your business unit: Solaris RPE Platform-Kernel 1.5. Email Aliases: 1.5.1. Responsible Manager: ken.yan@sun.com 1.5.2. Responsible Engineer: christopher.baumbauer@sun.com 1.5.3. Marketing Manager: 1.5.4. Interest List: 2. Project Summary 2.1. Project Description: Provide a new kstat to indicate when a page of kernel memory is slated for retirement. 2.2. Risks and Assumptions: The addition of a new kstat may not get picked up by various debugging and diagnostic tools. 3. Business Summary 3.1. Problem Area: Customer experiences a nasty performance hit whenever the pagelock cache is disabled, and in the case mentioned during the technical description, the pagelock cache continuously gets disabled when it shouldn't. 3.2. Market/Requester: 3.3. Business Justification: 3.4. Competitive Analysis: 3.5. Opportunity Window/Exposure: 3.6. How will you know when you are done?: 4. Technical Description: 4.1. Details: The implementation is a part of a fix with page retirement where the new kstat will be used to differentiate pages slated for retirement. Since a kernel page will be locked throughout its lifetime, if it is slated for retirement, the page will first find its way into a page retirement pending queue (S10) or a page capture hash bucket (Nevada) and then increment a kstat to indicate that a page is waiting to be retired. Then a background thread (page_retire_thread in S10 or page_capture_thread in Nevada) will check the pending kstat to see if there is any work to do, go through the queue or hash, and attempt to unlock the page in an effort to to induce a retirement. In the case of kernel pages, they are always locked and will thus not undergo retirement, but still force the page retirement thread to run and attempt to cleanup the page while negatively impacting system performance by unneccarily disabling the pagelock cache. Also, since the kernel page has been marked for retirement, they will need to be scrubbed on reboot to prevent possible future issues on the next boot. Adding a new kstat that specifically tracks kernel pages placed in the respective pending location will allow the page retirement system to pull kernel pages out of circulation while still permit their cleanup on reboot. A kernel page retirement example: # ./prt_sparc Welcome to page_retire_test (PRT). 'h' for Help. prt> M0x23f406000 ITER 0: badpa 0 busy 0 again 1 not 0 ok 0 ----- prt> k module: unix instance: 0 name: page_retire class: misc crtime 73.371403096 pages_deferred 1 pages_deferred_kernel 0 pages_fma 0 pages_limit 514 pages_limit_exceeded 0 pages_multiple_ce 0 pages_notdequeued 0 pages_notenqueued 0 pages_pending 0 pages_pending_kas 1 pages_retire_request 0 pages_retire_request_free 0 pages_retired 0 pages_ue 0 pages_ue_cleared_freed 0 pages_ue_cleared_retired 0 pages_ue_persistent 0 pages_unretired 0 snaptime 302.701530127 prt> ^D # mdb -k Loading modules: [ unix genunix specfs dtrace ufs pcisch sd ip hook neti sctp arp usba fctl nca lofs zfs cpc random fcip ptm sppp nfs ] > pr_pending_q::print [ 0x70001fd0180, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ] > 0x70001fd0180::print page_t { p_offset = 0x3000c406000 p_vnode = kvp p_selock = 0 p_vpmref = 0 p_hash = 0 p_vpnext = 0x70001fd0100 p_vpprev = 0x70001fd0200 p_next = 0x70001fd0180 p_prev = 0x70001fd0180 p_lckcnt = 0x1 p_cowcnt = 0 p_cv = { _opaque = 0 } p_io_cv = { _opaque = 0 } p_iolock_state = 0 p_szc = 0x3 p_fsdata = 0 p_state = 0 p_nrm = 0 p_vcolor = 0x2 p_index = 0x8 p_toxic = 0x41 ... 4.2. Bug/RFE Number(s): CR 6587140 4.3. In Scope: 4.4. Out of Scope: 4.5. Interfaces: New kstat variable (pr_pending_kas) which will be used in addition to pr_pending to track the number of pages awaiting retirement. 4.6. Doc Impact: 4.7. Admin/Config Impact: Programs and scripts that make use of the page_retirement kstats in general, and the pr_pending kstat in particular. 4.8. HA Impact: 4.9. I18N/L10N Impact: 4.10. Packaging & Delivery: 4.11. Security Impact: 4.12. Dependencies: 5. Reference Documents: CR 6587140 6. Resources and Schedule: 6.1. Projected Availability: 6.2. Cost of Effort: 6.3. Cost of Capital Resources: 6.4. Product Approval Committee requested information: 6.4.1. Consolidation or Component Name: 6.4.3. Type of CPT Review and Approval expected: FastTrack 6.4.4. Project Boundary Conditions: 6.4.5. Is this a necessary project for OEM agreements: no 6.4.6. Notes: 6.4.7. Target RTI Date/Release: Nevada: snv_112 S10: S10u8_1 6.4.8. Target Code Design Review Date: 6.4.9. Update approval addition: 6.5. ARC review type: Fasttrack 6.6. ARC Exposure: open 6.6.1. Rationale: 7. Prototype Availability: 7.1. Prototype Availability: 7.2. Prototype Cost: From Alan.Burlison@sun.com Fri Mar 27 14:13:54 2009 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n2RLDs5j016837 for ; Fri, 27 Mar 2009 14:13:54 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n2RLDsCI018579 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Fri, 27 Mar 2009 14:13:54 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KH600803OB5QM00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 27 Mar 2009 14:13:53 -0700 (PDT) Received: from dm-uk-02.uk.sun.com ([129.156.101.196]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KH60058DOB4O230@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 27 Mar 2009 14:13:52 -0700 (PDT) Received: from barman.uk.sun.com (barman.UK.Sun.COM [129.156.132.12]) by dm-uk-02.uk.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n2RLDnCQ005669; Fri, 27 Mar 2009 21:13:49 +0000 (GMT) Received: from vpn-129-150-120-38.uk.sun.com ([129.150.120.38]) by barman.uk.sun.com with esmtp (Exim 4.42) id 1LnJSr-00068F-Tf; Fri, 27 Mar 2009 21:19:46 +0000 Date: Fri, 27 Mar 2009 21:13:48 +0000 From: Alan Burlison Subject: Re: Page Retirement Kernel KStat Addition [PSARC/2009/181 FastTrack timeout 04/03/2009] In-reply-to: <49CD2B7F.8060207@sun.com> To: Daniel Hain Cc: PSARC-ext@sun.com, Christopher Baumbauer Message-id: <49CD418C.4080303@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <49CD2B7F.8060207@sun.com> User-Agent: Thunderbird 2.0.0.18 (X11/20081215) Status: RO Content-Length: 240 Daniel Hain wrote: > 2.2. Risks and Assumptions: > The addition of a new kstat may not get picked up by various debugging > and diagnostic tools. Are the kstats being published as named kstats or raw kstats? -- Alan Burlison -- From Christopher.Baumbauer@sun.com Fri Mar 27 14:36:48 2009 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n2RLamf1018629 for ; Fri, 27 Mar 2009 14:36:48 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n2RLalbc006568 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Fri, 27 Mar 2009 21:36:47 GMT Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KH600203PDA4Y00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 27 Mar 2009 15:36:46 -0600 (MDT) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KH600BFAPDAE4B0@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 27 Mar 2009 15:36:46 -0600 (MDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n2RLakxg009298 for ; Fri, 27 Mar 2009 14:36:46 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7.0-5.01 64bit (built Feb 19 2009)) id <0KH600I00P1LEB00@fe-sfbay-09.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 27 Mar 2009 14:36:46 -0700 (PDT) Received: from [129.153.85.31] ([unknown] [129.153.85.31]) by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7.0-5.01 64bit (built Feb 19 2009)) with ESMTPSA id <0KH600FKLPD7FI10@fe-sfbay-09.sun.com>; Fri, 27 Mar 2009 14:36:44 -0700 (PDT) Date: Fri, 27 Mar 2009 14:36:43 -0700 From: Christopher Baumbauer Subject: Re: Page Retirement Kernel KStat Addition [PSARC/2009/181 FastTrack timeout 04/03/2009] In-reply-to: <49CD418C.4080303@sun.com> Sender: Christopher.Baumbauer@sun.com To: Alan Burlison Cc: Daniel Hain , PSARC-ext@sun.com Reply-to: Christopher.Baumbauer@sun.com Message-id: <49CD46EB.6040204@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <49CD2B7F.8060207@sun.com> <49CD418C.4080303@sun.com> User-Agent: Thunderbird 2.0.0.16 (X11/20080807) Status: RO Content-Length: 302 On 03/27/09 14:13, Alan Burlison wrote: > Daniel Hain wrote: > >> 2.2. Risks and Assumptions: >> The addition of a new kstat may not get picked up by various debugging >> and diagnostic tools. > > Are the kstats being published as named kstats or raw kstats? > This would be a named kstat. From Alan.Burlison@sun.com Fri Mar 27 14:48:27 2009 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n2RLmR1c019747 for ; Fri, 27 Mar 2009 14:48:27 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n2RLmR9H004875 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Fri, 27 Mar 2009 14:48:27 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KH600A05PWORP00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 27 Mar 2009 14:48:24 -0700 (PDT) Received: from dm-uk-01.uk.sun.com ([129.156.101.115]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KH6005FKPWNNS50@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 27 Mar 2009 14:48:24 -0700 (PDT) Received: from barman.uk.sun.com (barman.UK.Sun.COM [129.156.132.12]) by dm-uk-01.uk.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n2RLmLaV002394; Fri, 27 Mar 2009 21:48:21 +0000 (GMT) Received: from vpn-129-150-120-38.uk.sun.com ([129.150.120.38]) by barman.uk.sun.com with esmtp (Exim 4.42) id 1LnK0I-0006MI-1H; Fri, 27 Mar 2009 21:54:18 +0000 Date: Fri, 27 Mar 2009 21:48:20 +0000 From: Alan Burlison Subject: Re: Page Retirement Kernel KStat Addition [PSARC/2009/181 FastTrack timeout 04/03/2009] In-reply-to: <49CD46EB.6040204@Sun.COM> To: Christopher.Baumbauer@sun.com Cc: Daniel Hain , PSARC-ext@sun.com Message-id: <49CD49A4.2080900@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <49CD2B7F.8060207@sun.com> <49CD418C.4080303@sun.com> <49CD46EB.6040204@Sun.COM> User-Agent: Thunderbird 2.0.0.18 (X11/20081215) Status: RO Content-Length: 204 Christopher Baumbauer wrote: >> Are the kstats being published as named kstats or raw kstats? > > This would be a named kstat. Thanks, in that case, kstat(1) should pick them up. -- Alan Burlison --