From Thu Feb 14 13:35:04 2008 Date: Thu, 14 Feb 2008 13:35:04 -0800 (PST) From: Postmaster Subject: Message from mail server Content-Length: 95 Mime-Version: 1.0 Status: RO X-IMAP: 1203024904 3 Delete. This is a system message. --END+PSEUDO-- From sacadmin Mon Feb 11 13:55:01 2008 Received: from sac.sfbay.sun.com (localhost [127.0.0.1]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m1BLt1Bv005592; Mon, 11 Feb 2008 13:55:01 -0800 (PST) Received: (from hyw@localhost) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id m1BLt0vv005588; Mon, 11 Feb 2008 13:55:00 -0800 (PST) Date: Mon, 11 Feb 2008 13:55:00 -0800 (PST) From: Huay-Yong Wang Message-Id: <200802112155.m1BLt0vv005588@sac.sfbay.sun.com> To: PSARC-record@sac.sfbay.sun.com Cc: Chris.kiick@sun.com, Steve.Sistare@sun.com Subject: PSARC/2008/098 Lockstat: Changing spin count to spin time Content-Length: 4622 Status: RO X-Status: $$$$ X-UID: 0000000001 I am sponsoring this fasttrack for Steve Sistare and the timeout is set to expire 02/19/2008. The proposal is attached below. The project team is asking for a release binding of patch/micro. This project proposes to change the spin count metric provided by lockstat(1M) to a more appropriate and useful metric of spin time. ---- Cut here ---- Lockstat(1M): Changing spin count to spin time. Certain kernel lock probes return a value called the "spin count", which is the number of iterations that are taken through the spin loop before the lock is acquired. This metric has been used in the past to diagnose certain performance problems due to lock contention . However it has been of limited usefulness. The metric of "spin count" does not provide an absolute measure of overhead and cannot be used to compare between systems. With the addition of exponential backoff optimizations, it becomes even cruder because the number of spins is no longer linearly proportional to the amount of time spent acquiring the lock. The dtrace guide itself says: "The spin count has little meaning on its own, but can be used to compare spin times." This fast-track proposes that the spin count metric should be replaced with a direct measurement of the spin time. This will be a measure of time spent busy-waiting to acquire a lock, using high resolution time (in nanoseconds). This provides an accurate measure of lock contention that will be very useful to performance analysts; in particular, it allows them to see how much %sys time is due to lock contention. Nanoseconds can also be compared between OS versions, CPU speeds and platforms. It will also be more consistent with other dtrace probes and commands (like plockstat) that report times instead of spin counts. Updating the spin metric will require three changes: 1) Change the mutex and disp_lock code to gather high-resolution timing data using gethrtime() instead of counting loop iterations. 2) Update the lockstat command to use the column header "nsec" instead of "spin" for the spin values. 3) Modify the documentation - the lockstat man page and dtrace guide - to reflect this change. These changes have already been tested. Note that change #1 does not introduce any performance hit from the overhead of calling gethrtime() in the mutex spin code - this has been tested. Modified Interfaces: 1. man lockstat(1M) The existing text says: spin or nsec Average number of times caller spun trying to get the lock, or average duration of the events in nanoseconds, as appropriate for the event... The proposed change: nsec Average duration of the events in nanoseconds, as appropriate for the event... Stability: Unstable The output of lockstat is not a stable interface and may be modified. The materials for the original lockstat case PSARC/1997/086 state: "The lockstat command and interfaces are all classified as Unstable" The man page makes this clear: "The format and content of lockstat output reflect the current Solaris kernel implementation and are therefore subject to change in future releases." 2. Solaris Dynamic Tracing Guide, Chapter 18, lockstat Provider The probe lockstat:::adaptive-spin arg1 was defined as spin count, but will now defined to be spin time in nanoseconds. The probe lockstat:::spin-spin arg1 was defined as spin count, and is now defined to be spin time in nanoseconds. The existing text for both says: "arg1 ... contains the spin count: the number of iterations that were taken through the spin loop before the lock was acquired. The spin count has little meaning on its own, but can be used to compare spin times." The proposed change: "arg1 ... contains the spin time in nanoseconds: the elapsed time spent in the spin loop before the lock was acquired." Stability: Evolving The dtrace guide marks the Arguments in the lockstat provider as Evolving stability. Evolving interfaces can change but an effort must be made to ensure binary compatibility. Since this is only a change in units, there is no issue with binary compatibility. References: CR6624951 replace lockstat spin metric with busy-wait time CR6658169 dtrace documentation update for lock spin metric change CR6657854 man page and documentation changes for lockstat spin metric From Huay-Yong.Wang@sun.com Mon Feb 11 15:36:23 2008 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m1BNaNTA013806 for ; Mon, 11 Feb 2008 15:36:23 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m1BNaNg6044900 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Mon, 11 Feb 2008 16:36:23 -0700 (MST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JW300J0TLKJIS00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.COM); Mon, 11 Feb 2008 16:36:19 -0700 (MST) Received: from sca-es-mail-2.sun.com ([192.18.43.133]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JW300F31LKERK30@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.COM); Mon, 11 Feb 2008 16:36:15 -0700 (MST) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m1BNaE5o017505 for ; Mon, 11 Feb 2008 15:36:14 -0800 (PST) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JW300201L6R3F00@fe-sfbay-09.sun.com> (original mail from Huay-Yong.Wang@Sun.COM) for PSARC-ext@Sun.COM (ORCPT PSARC-ext@Sun.COM); Mon, 11 Feb 2008 15:36:14 -0800 (PST) Received: from [129.153.85.9] by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JW3001GILKD0GD0@fe-sfbay-09.sun.com> for PSARC-ext@Sun.COM (ORCPT PSARC-ext@Sun.COM); Mon, 11 Feb 2008 15:36:13 -0800 (PST) Date: Mon, 11 Feb 2008 15:36:10 -0800 From: Huay-Yong.Wang@sun.com Subject: PSARC/2008/098 Lockstat: Changing spin count to spin time Sender: Huay-Yong.Wang@sun.com To: PSARC-ext@sun.com Cc: Steve Sistare , chris kiick Message-id: <47B0DBEA.2090306@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-Accept-Language: en-us, en X-PMX-Version: 5.2.0.264296 User-Agent: Mozilla Thunderbird 0.8 (X11/20040917) Content-Length: 4622 Status: RO X-Status: $$$$ X-UID: 0000000002 I am sponsoring this fasttrack for Steve Sistare and the timeout is set to expire 02/19/2008. The proposal is attached below. The project team is asking for a release binding of patch/micro. This project proposes to change the spin count metric provided by lockstat(1M) to a more appropriate and useful metric of spin time. ---- Cut here ---- Lockstat(1M): Changing spin count to spin time. Certain kernel lock probes return a value called the "spin count", which is the number of iterations that are taken through the spin loop before the lock is acquired. This metric has been used in the past to diagnose certain performance problems due to lock contention . However it has been of limited usefulness. The metric of "spin count" does not provide an absolute measure of overhead and cannot be used to compare between systems. With the addition of exponential backoff optimizations, it becomes even cruder because the number of spins is no longer linearly proportional to the amount of time spent acquiring the lock. The dtrace guide itself says: "The spin count has little meaning on its own, but can be used to compare spin times." This fast-track proposes that the spin count metric should be replaced with a direct measurement of the spin time. This will be a measure of time spent busy-waiting to acquire a lock, using high resolution time (in nanoseconds). This provides an accurate measure of lock contention that will be very useful to performance analysts; in particular, it allows them to see how much %sys time is due to lock contention. Nanoseconds can also be compared between OS versions, CPU speeds and platforms. It will also be more consistent with other dtrace probes and commands (like plockstat) that report times instead of spin counts. Updating the spin metric will require three changes: 1) Change the mutex and disp_lock code to gather high-resolution timing data using gethrtime() instead of counting loop iterations. 2) Update the lockstat command to use the column header "nsec" instead of "spin" for the spin values. 3) Modify the documentation - the lockstat man page and dtrace guide - to reflect this change. These changes have already been tested. Note that change #1 does not introduce any performance hit from the overhead of calling gethrtime() in the mutex spin code - this has been tested. Modified Interfaces: 1. man lockstat(1M) The existing text says: spin or nsec Average number of times caller spun trying to get the lock, or average duration of the events in nanoseconds, as appropriate for the event... The proposed change: nsec Average duration of the events in nanoseconds, as appropriate for the event... Stability: Unstable The output of lockstat is not a stable interface and may be modified. The materials for the original lockstat case PSARC/1997/086 state: "The lockstat command and interfaces are all classified as Unstable" The man page makes this clear: "The format and content of lockstat output reflect the current Solaris kernel implementation and are therefore subject to change in future releases." 2. Solaris Dynamic Tracing Guide, Chapter 18, lockstat Provider The probe lockstat:::adaptive-spin arg1 was defined as spin count, but will now defined to be spin time in nanoseconds. The probe lockstat:::spin-spin arg1 was defined as spin count, and is now defined to be spin time in nanoseconds. The existing text for both says: "arg1 ... contains the spin count: the number of iterations that were taken through the spin loop before the lock was acquired. The spin count has little meaning on its own, but can be used to compare spin times." The proposed change: "arg1 ... contains the spin time in nanoseconds: the elapsed time spent in the spin loop before the lock was acquired." Stability: Evolving The dtrace guide marks the Arguments in the lockstat provider as Evolving stability. Evolving interfaces can change but an effort must be made to ensure binary compatibility. Since this is only a change in units, there is no issue with binary compatibility. References: CR6624951 replace lockstat spin metric with busy-wait time CR6658169 dtrace documentation update for lock spin metric change CR6657854 man page and documentation changes for lockstat spin metric From Steve.Sistare@sun.com Thu Feb 14 09:56:43 2008 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m1EHug5I016542 for ; Thu, 14 Feb 2008 09:56:42 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id m1EHucOP018825 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Fri, 15 Feb 2008 01:56:41 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JW800D05PUECG00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.COM); Thu, 14 Feb 2008 10:56:38 -0700 (MST) Received: from brmea-mail-2.sun.com ([192.18.98.43]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JW8002SCPUEZC90@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.COM); Thu, 14 Feb 2008 10:56:38 -0700 (MST) Received: from fe-amer-09.sun.com ([192.18.109.79]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id m1EHubw2026013 for ; Thu, 14 Feb 2008 17:56:37 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JW800G01O9GA900@mail-amer.sun.com> (original mail from Steve.Sistare@Sun.COM) for PSARC-ext@Sun.COM (ORCPT PSARC-ext@Sun.COM); Thu, 14 Feb 2008 10:56:37 -0700 (MST) Received: from [129.148.180.120] by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JW8000QAPU9LY50@mail-amer.sun.com> for PSARC-ext@Sun.COM (ORCPT PSARC-ext@Sun.COM); Thu, 14 Feb 2008 10:56:34 -0700 (MST) Date: Thu, 14 Feb 2008 12:56:32 -0500 From: Steve Sistare Subject: Re: PSARC/2008/098 Lockstat: Changing spin count to spin time In-reply-to: <47B0DBEA.2090306@Sun.COM> Sender: Steve.Sistare@sun.com To: PSARC-ext@sun.com Cc: Huay-Yong.Wang@sun.com, chris kiick Message-id: <47B480D0.4070803@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <47B0DBEA.2090306@Sun.COM> User-Agent: Thunderbird 2.0.0.6 (X11/20070802) Content-Length: 5450 Status: RO X-Status: $$$$ X-UID: 0000000003 Folks, For truth in advertising, I need to correct the statement below that says: "change #1 does not introduce any performance hit". We had not tested while running xVM. John Levon suggested this test - thanks John. We did so, and there is no overhead while lockstat is not running. When lockstat is running, there is 2% to 10% overhead on a few of our unrealistic "extreme mutex" tests, on top of the 40% to 70% overhead that is incurred just by enabling lockstat, regardless of old vs new metric. I believe that obtaining meaningful spin times justifies this small additional overhead. - Steve Sistare Huay-Yong.Wang@Sun.COM wrote: > > > I am sponsoring this fasttrack for Steve Sistare and the timeout > is set to expire 02/19/2008. The proposal is attached below. > The project team is asking for a release binding of patch/micro. > > This project proposes to change the spin count metric provided > by lockstat(1M) to a more appropriate and useful metric of spin time. > > > ---- Cut here ---- > > Lockstat(1M): Changing spin count to spin time. > > Certain kernel lock probes return a value called the "spin count", > which is the number of iterations that are taken through the spin loop > before the lock is acquired. > > This metric has been used in the past to diagnose certain performance > problems due to lock contention . However it has been of limited > usefulness. The metric of "spin count" does not provide an absolute > measure of overhead and cannot be used to compare between systems. > With the addition of exponential backoff optimizations, it becomes > even cruder because the number of spins is no longer linearly > proportional to the amount of time spent acquiring the lock. The > dtrace guide itself says: "The spin count has little meaning on its > own, but can be used to compare spin times." > > This fast-track proposes that the spin count metric should be > replaced with a direct measurement of the spin time. This will be a > measure of time spent busy-waiting to acquire a lock, using high > resolution time (in nanoseconds). This provides an accurate measure > of lock contention that will be very useful to performance analysts; > in particular, it allows them to see how much %sys time is due to > lock contention. Nanoseconds can also be compared between OS > versions, CPU speeds and platforms. It will also be more consistent > with other dtrace probes and commands (like plockstat) that report > times instead of spin counts. > > Updating the spin metric will require three changes: > 1) Change the mutex and disp_lock code to gather high-resolution > timing data using gethrtime() instead of counting loop iterations. > 2) Update the lockstat command to use the column header "nsec" > instead of "spin" for the spin values. > 3) Modify the documentation - the lockstat man page and dtrace guide - > to reflect this change. > > These changes have already been tested. Note that change #1 > does not introduce any performance hit from the overhead of calling > gethrtime() in the mutex spin code - this has been tested. > > Modified Interfaces: > > 1. man lockstat(1M) > > The existing text says: > > spin or nsec Average number of times caller spun > trying to get the lock, or average > duration of the events in > nanoseconds, as appropriate for the > event... > > The proposed change: > > nsec Average duration of the events in nanoseconds, > as appropriate for the event... > > Stability: Unstable > > The output of lockstat is not a stable interface and may be modified. > The materials for the original lockstat case PSARC/1997/086 state: > "The lockstat command and interfaces are all classified as Unstable" > The man page makes this clear: > "The format and content of lockstat output reflect the current > Solaris kernel implementation and are therefore subject to > change in future releases." > > 2. Solaris Dynamic Tracing Guide, Chapter 18, lockstat Provider > > The probe lockstat:::adaptive-spin arg1 was defined as spin count, > but will now defined to be spin time in nanoseconds. > > The probe lockstat:::spin-spin arg1 was defined as spin count, > and is now defined to be spin time in nanoseconds. > > The existing text for both says: > > "arg1 ... contains the spin count: the number of iterations that > were taken through the spin loop before the lock was acquired. > The spin count has little meaning on its own, but can be used to > compare spin times." > > The proposed change: > > "arg1 ... contains the spin time in nanoseconds: the elapsed > time spent in the spin loop before the lock was acquired." > > Stability: Evolving > > The dtrace guide marks the Arguments in the lockstat provider as > Evolving stability. Evolving interfaces can change but an effort must > be made to ensure binary compatibility. Since this is only a change > in units, there is no issue with binary compatibility. > > References: > CR6624951 replace lockstat spin metric with busy-wait time > CR6658169 dtrace documentation update for lock spin metric change > CR6657854 man page and documentation changes for lockstat spin metric > > > > From Huay-Yong.Wang@sun.com Sun Feb 24 16:51:03 2008 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m1P0p2H3019563 for ; Sun, 24 Feb 2008 16:51:02 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m1P0p2Xk005710 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Sun, 24 Feb 2008 16:51:02 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JWR00103RP1B800@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Sun, 24 Feb 2008 16:51:01 -0800 (PST) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JWR00G8ZRP04450@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Sun, 24 Feb 2008 16:51:00 -0800 (PST) Received: from fe-amer-09.sun.com ([192.18.109.79]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id m1P0p0Q3025002 for ; Mon, 25 Feb 2008 00:51:00 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JWR00501RMFF800@mail-amer.sun.com> (original mail from Huay-Yong.Wang@Sun.COM) for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Sun, 24 Feb 2008 17:51:00 -0700 (MST) Received: from [192.168.1.2] ([129.150.33.93]) by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JWR009ASROV1P80@mail-amer.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Sun, 24 Feb 2008 17:51:00 -0700 (MST) Date: Sun, 24 Feb 2008 16:50:54 -0800 From: Huay-Yong Wang Subject: Re: PSARC/2008/098 Lockstat: Changing spin count to spin time In-reply-to: <47B0DBEA.2090306@Sun.COM> Sender: Huay-Yong.Wang@sun.com To: PSARC-ext@sun.com Cc: Steve Sistare , chris kiick Message-id: <47C210EE.9000201@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=us-ascii Content-transfer-encoding: 7BIT X-Accept-Language: en-us, en X-PMX-Version: 5.2.0.264296 References: <47B0DBEA.2090306@Sun.COM> User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7.13) Gecko/20060414 Status: RO Content-Length: 102 This case was approved during the 2/20/08 PSARC meeting. I am marking this case approved and closed.