From petede@sac.sfbay.sun.com Tue Jun 8 02:12:37 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o589CbOw002098 for ; Tue, 8 Jun 2010 02:12:37 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o589CaBi023806; Tue, 8 Jun 2010 03:12:37 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L3O00E0RUX0AA00@nwk-avmta-2.sfbay.sun.com>; Tue, 08 Jun 2010 02:12:36 -0700 (PDT) Received: from dm-sfbay-02.sfbay.sun.com ([129.146.11.31]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L3O002HVUWZHOB0@nwk-avmta-2.sfbay.sun.com>; Tue, 08 Jun 2010 02:12:35 -0700 (PDT) Received: from sac.sfbay.sun.com (sac.SFBay.Sun.COM [129.146.226.132]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o589CZha019686; Tue, 08 Jun 2010 02:12:35 -0700 (PDT) Received: from sac.sfbay.sun.com (localhost [127.0.0.1]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o589CS3u002093; Tue, 08 Jun 2010 02:12:28 -0700 (PDT) Received: (from petede@localhost) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id o589CSp9002089; Tue, 08 Jun 2010 02:12:28 -0700 (PDT) Date: Tue, 08 Jun 2010 02:12:28 -0700 (PDT) From: Peter Dennis Subject: Performance Improvements for libmtmalloc [PSARC/2010/212 FastTrack timeout 06/15/2010] To: PSARC-ext@sun.com Cc: rick.weisner@oracle.com Message-id: <201006080912.o589CSp9002089@sac.sfbay.sun.com> Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 Status: RO Content-Length: 9970 I am sponsoring this case for Rick Weisner. Requested release binding: Patch Modified man pages are in the case's materials directory and diffs are at the end of this proposal. Template Version: @(#)sac_nextcase 1.70 03/30/10 SMI This information is Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved. 1. Introduction 1.1. Project/Component Working Name: Performance Improvements for libmtmalloc 1.2. Name of Document Author/Supplier: Author: Rick Weisner 1.3 Date of This Document: 08 June, 2010 4. Technical Description Template Version: @(#)sac_nextcase 1.70 05/10/10 SMI This information is Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved. 1. Introduction 1.1. Project/Component Working Name: Performance Improvements for libmtmalloc 1.2. Name of Document Author/Supplier: Author: Rick Weisner 1.3 Date of This Document: 01 June, 2010 4. Technical Description SUMMARY Under the following two situations libmtmalloc has shown poor scalability. 1. When there are large numbers of allocating threads. (see CR6922229) and 2. When the allocation size is larger than 64 KB. (see CR6555149) We will remedy the above scalability issues by: 1) Using atomic operations to eliminate the cache lock in libmtmalloc. 2) Provide a mechanism whereby the parent lock can also be eliminated for threads whose id is less than 2* the number of cpus. 3) Make the maximum cacheable requestsize tunable via an environment variable. BACKGROUND libmtmalloc organizes avaiable address space into buckets. Each thread which calls malloc is assigned a bucket based upon its thread id. The per bucket parent lock controls the use of each bucket. Each bucket is a list of caches based on size. Each list is protected by a cache lock. Applications with a large number of allocating threads may have their performance limited by contention for these locks. These sort of applications are not unusual in the Telco space. Larger allocations sizes are also becoming more common. With 64 bit applications, terabytes of memory, and hundreds of threads it is advantageous to be able to adjust the maximum cacheable requestsize to better suit the needs of the application. PROBLEM A customer's application did not perform as needed on a Netra 5440. DTrace indicated lock contention relating to memory allocation in libmtmalloc. The customer provided some code that provided dramatic performance increases by eliminating the "cache" locks and "parent" locks from libmtmalloc and replacing them with atomic operations. The customer's code was not threadsafe in general but was promising. In a different case the customer states: We observed that db is hitting oversize_lock mutex due to the memory needed to be allocated is more than MAX_CACHED. Sometimes acquiring the oversize_lock mutex is taking more than 2sec, causing the db performance to degrade. (see 6555149) PROPOSAL 1) Eliminate the cache lock by using atomic operations. 2) Add a new option to mallocctl(3MALLOC) that activates the use of exclusive buckets for threads whose ID is < 2 * the number of CPUs. The value argument associated with the mallocctl option is ignored. The use of exclusive buckets can also be activitated if there is an environment variable named MTEXCLUSIVE. This feature is needed for situations where the source code is unavailable. This feature will also assist in performance analysis. Once the option has been called there is no facility to 'unset' it. 3) Introduce the environment variable, MTMAXCACHE, which will set the maximum request size that is cached. It will have the values of 16 to 21. The default is 16 which means that requests less than 2^^16 are cached. With this value we can support up to 2mb (2^^21) request sizes in cache. If the value of MTMAXCACHE is set to something outside of the ranges then it will use either 16 or 21 (which ever bound has been broken by the value set). It is necessary to use an environment variable instead of a mallocctl interface because the MTMAXCACHE must be determined before malloc_init calls setup_caches. DETAILS The code has been developed and tested in 64 bit mode on Solaris 10 u6 on a Netra T5440. The test harness uses a configurable number of allocation threads, a configurable sample count, a configurable "maximum" allocation size. Each allocation thread has a configurable number of ramdom or fixed size allocations between 8 and the requested "max" allocation size + 1/2 the "max" allocation size. A freeing thread then releases the allocations while the allocating thread performs a fresh set of allocations. In initial testing with "stock" libmtmalloc it was possible to do 6300 64 bit operations per sec on the N5440. With the "atomic" library this increases to 15000. COMMENTS Exported Interfaces: MTEXCLUSIVE Committed option for mallocctl(3MALLOC). MTEXCLUSIVE Committed Shell environment variable. If set, then the effect is the same as if mallocctl was called with the option MTEXCLUSIVE. MTMAXCACHE Committed Shell environmet variable. If set, the value sets the maximum cachable requestsize to 2^^MTMAXCACHE. Reference: 6922229 libmtmalloc would benefit from atomic operations 6555149 poor performance with libmtmalloc compared to libc 6956786 Provide a tunable to tweak the MAX_CACHED threshold in libmtmalloc 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: FastTrack 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: FastTrack 6.6. ARC Exposure: open Man page diffs: ** libmtmalloc.man Thu Jun 3 15:46:52 2010 --- new_libmtmalloc.man Thu Jun 3 16:09:44 2010 *************** *** 28,34 **** --- 28,58 ---- mallocctl memalign realloc valloc + ENVIRONMENT VARIABLES + MTEXCLUSIVE By default, libmtmalloc allocates 2*NCPUS + buckets from which allocations occur. + threads share buckets based on their thread + id. If MTEXCLUSIVE is invoked, then 4*NCPUS + buckets are used. Threads with thread id less + than 2*NCPUS receive an exclusive bucket and + thus do not need to use locks. Allocation + performance for these buckets may be dramatically + increased. One enabled MTEXCLUSIVE can not be + disabled. This feature can be enabled by + setting the environment value MTEXCLUSIVE to + anything. Altenatively it can be enabled by + a call to mallocctl(see mallocctl). + MTMAXCACHE By default, allocations less than 2^^16 bytes + are allocated from buckets indexed by thread id. + Using this environment variable size of the + cached allocations can be increased to 2^^17, + 2^^18, 2^^18, 2^^19, 2^^20, or 2^^21 by + setting MTMAXCACHE to 17,18,19,20,or 21. + If MTMAXCACHE is set to less than 16 it is + reset to 16. If MTMAXCACHE is set to more than + 21, then it is reset to 21. This all occurs + silently. FILES /usr/lib/libmtmalloc.so.1 *** mallocctl.man Thu Jun 3 15:37:18 2010 --- new_mallocctl.man Thu Jun 3 15:45:41 2010 *************** *** 164,170 **** --- 164,183 ---- 256. The default value is 9. This value is multiplied by 8192. + MTEXCLUSIVE By default, libmtmalloc allocates 2*NCPUS + buckets from which allocations occur. + threads share buckets based on their thread + id. If MTEXCLUSIVE is invoked, then 4*NCPUS + buckets are used. Threads with thread id less + than 2*NCPUS receive an exclusive bucket and + thus do not need to use locks. Allocation + performance for these buckets may be dramatically + increased. One enabled MTEXCLUSIVE can not be + disabled. This feature can also be enabled by + setting the environment value MTEXCLUSIVE to + anything. + RETURN VALUES If there is no available memory, malloc(), realloc(), memalign(), and valloc() return a null pointer. When real- *************** *** 224,230 **** brk(2), getrlimit(2), bsdmalloc(3MALLOC), dlopen(3C), malloc(3C), malloc(3MALLOC), mapmalloc(3MALLOC), signal.h(3HEAD), umem_alloc(3MALLOC), watchmalloc(3MALLOC), ! attributes(5) WARNINGS Undefined results will occur if the size requested for a --- 237,243 ---- brk(2), getrlimit(2), bsdmalloc(3MALLOC), dlopen(3C), malloc(3C), malloc(3MALLOC), mapmalloc(3MALLOC), signal.h(3HEAD), umem_alloc(3MALLOC), watchmalloc(3MALLOC), ! libmtmalloc(3LIB), attributes(5) WARNINGS Undefined results will occur if the size requested for a From peter.dennis@oracle.com Tue Jun 15 10:37:39 2010 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o5FHbcXD027788 for ; Tue, 15 Jun 2010 10:37:38 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o5FHbcHH014923 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Tue, 15 Jun 2010 10:37:38 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L4200701GYQOH00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 15 Jun 2010 11:37:38 -0600 (MDT) Received: from sca-ea-mail-2.sun.com ([192.18.43.25]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L4200CN6GYPL1E0@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 15 Jun 2010 11:37:38 -0600 (MDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by sca-ea-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o5FHbbW5014890 for ; Tue, 15 Jun 2010 17:37:37 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o5FGKTTx001724 for ; Tue, 15 Jun 2010 17:37:36 +0000 (GMT) Received: from abhmt004.oracle.com by acsmt353.oracle.com with ESMTP id 327519671276623386; Tue, 15 Jun 2010 10:36:26 -0700 Received: from [129.150.120.241] (/129.150.120.241) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 15 Jun 2010 10:36:25 -0700 Date: Tue, 15 Jun 2010 18:36:23 +0100 From: Peter Dennis Subject: Performance Improvements for libmtmalloc [PSARC/2010/212 FastTrack, timeout 06/15/2010] To: PSARC-ext@sun.com Cc: rick weisner Message-id: <4C17BA17.4060803@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090204.4C17BA61.0060:SCFMA4539814,ss=1,fgs=0 User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.7) Gecko/20100214 Lightning/1.0b1 Thunderbird/3.0.1 Status: RO Content-Length: 112 This case times out today but there have been no comments from any PSARC members - any reviewers ? thanks pete From olga.kryzhanovska@gmail.com Tue Jun 15 10:58:27 2010 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o5FHwQfg029063 for ; Tue, 15 Jun 2010 10:58:26 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o5FHwQ74005094 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Tue, 15 Jun 2010 10:58:26 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L4200903HXEHT00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 15 Jun 2010 11:58:26 -0600 (MDT) Received: from sca-ea-mail-2.sun.com ([192.18.43.25]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L4200CWYHXDL0D0@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 15 Jun 2010 11:58:25 -0600 (MDT) Received: from relay13i.sun.com (ip123.net129179-4.block1.us.syntegra.com [129.179.4.123]) by sca-ea-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o5FHpDd6020141 for ; Tue, 15 Jun 2010 17:58:25 +0000 (GMT) Received: from mmp11es.mmp.us.syntegra.com ([160.41.208.11] [160.41.208.11]) by relay13i.sun.com with ESMTP id BT-MMP-3081570 for PSARC-ext@sun.com; Tue, 15 Jun 2010 17:58:25 +0000 (Z) Received: from relay15i.sun.com (relay15i.sun.com [129.179.4.125]) by mmp11es.mmp.us.syntegra.com with ESMTP id BT-MMP-279676250 for PSARC-ext@sun.com; Tue, 15 Jun 2010 17:58:24 +0000 (Z) Received: from mail-iw0-f177.google.com ([209.85.214.177] [209.85.214.177]) by relay1i.sun.com with ESMTP id BT-MMP-12493401 for PSARC-ext@sun.com; Tue, 15 Jun 2010 17:58:24 +0000 (Z) Received: by iwn3 with SMTP id 3so985405iwn.8 for ; Tue, 15 Jun 2010 10:58:24 -0700 (PDT) Received: by 10.42.0.68 with SMTP id 4mr2592226icb.92.1276624704297; Tue, 15 Jun 2010 10:58:24 -0700 (PDT) Received: by 10.231.190.10 with HTTP; Tue, 15 Jun 2010 10:58:24 -0700 (PDT) Date: Tue, 15 Jun 2010 19:58:24 +0200 From: =?KOI8-R?B?z8zYx8Egy9LZ1sHOz9fTy8HR?= Subject: Re: Performance Improvements for libmtmalloc [PSARC/2010/212 FastTrack, timeout 06/15/2010] In-reply-to: <4C17BA17.4060803@oracle.com> To: Peter Dennis Cc: PSARC-ext@sun.com, rick weisner Message-id: MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1 Content-transfer-encoding: 7BIT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=W3ykwvI/K+Sj5wLS3sSOMYgiRu404HybSwlP6XAuSmU=; b=UqHjBIJv8oWMrjOJRE6SLHVhluGA1iFQAddEOTnuWqiNJmzDZ9c+zDJvZyu494DSgq 7FYvVHcpiakob86zSEjgfnqmnQ+m8i8+5o53N7rDVd7djhIjLwdV3hP5XR+DPfyeFP0A 6G2FlXSp+10etF0dJ77pVHiz98gGnoPyv1Nb4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=iM5KmsDQnRV2bQQwfvZHqY9zsjh+13Fl8B1KspgI4JQXT9lgnQnzVbEg74aoLHq5dR IqbKmMQy1UBWIasvxNOehuw5cywibXlWkHF5ARxtYsr125Abg3ElB+Dn+QmG49OZOjJ0 nbexQuLVPvaYTcVKPQ3c4qlfhOtKFPBqwqJxc= X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-Antispam: No, score=0.0/5.0, scanned in 0.067sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <4C17BA17.4060803@oracle.com> Status: RO Content-Length: 1185 I have a comment: Why do you use 3 environment variables? Environment variables are very expensive to use, they significantly increase start up time of an application and slow down all applications which use getenv() or putenv() if the size of the environment is large. libast had the same problem, with bitter and measurable impact on start up time, and replaced all the allocator tunable variables into 1 variable to improve start up time. IMO one environment variable to control the libmtmalloc behaviour should be enough. Olga On Tue, Jun 15, 2010 at 7:36 PM, Peter Dennis wrote: > This case times out today but there have been no comments from > any PSARC members - any reviewers ? > > thanks > pete > _______________________________________________ > opensolaris-arc mailing list > opensolaris-arc@opensolaris.org > -- , _ _ , { \/`o;====- Olga Kryzhanovska -====;o`\/ } .----'-/`-/ olga.kryzhanovska@gmail.com \-`\-'----. `'-..-| / http://twitter.com/fleyta \ |-..-'` /\/\ Solaris/BSD//C/C++ programmer /\/\ `--` `--` From rick.weisner@oracle.com Tue Jun 15 13:53:39 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o5FKrdo8003329 for ; Tue, 15 Jun 2010 13:53:39 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o5FKrdga022959 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Tue, 15 Jun 2010 15:53:39 -0500 (CDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L420020DQ1E9W00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 15 Jun 2010 14:53:38 -0600 (MDT) Received: from sca-ea-mail-2.sun.com ([192.18.43.25]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L4200CM5Q1C5070@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 15 Jun 2010 14:53:37 -0600 (MDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by sca-ea-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o5FKramx025493 for ; Tue, 15 Jun 2010 20:53:36 +0000 (GMT) Received: from acsmt353.oracle.com (acsmt353.oracle.com [141.146.40.153]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o5FKrSWk021814; Tue, 15 Jun 2010 20:53:28 +0000 (GMT) Received: from abhmt008.oracle.com by acsmt353.oracle.com with ESMTP id 328106081276635134; Tue, 15 Jun 2010 13:52:14 -0700 Received: from arwen.us.oracle.com (/10.140.206.36) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 15 Jun 2010 13:52:14 -0700 Date: Tue, 15 Jun 2010 15:58:14 -0500 From: rick weisner Subject: Re: Performance Improvements for libmtmalloc [PSARC/2010/212 FastTrack, timeout 06/15/2010] In-reply-to: To: olga.kryzhanovska@gmail.com Cc: Peter Dennis , PSARC-ext@sun.com Message-id: <20100615205813.GD2221@arwen.us.oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-transfer-encoding: 8BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt353.oracle.com [141.146.40.153] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090209.4C17E84B.0042:SCFMA4539814,ss=1,fgs=0 References: <4C17BA17.4060803@oracle.com> User-Agent: Mutt/1.5.18 (2008-05-17) Status: RO Content-Length: 3493 Olga, I introduce two variables, MTMAXCACHE and MTEXCLUSIVE. If I understand your point I could combine them into one variable such as MTMALLOC_OPTIONS in the same manner as UMEM_DEBUG or the undocumented UMEM_OPTIONS. While we are at it we could also accept other mallocctl options such as MTCHUNKSIZE, MTINITBUFFER, MTDOUBLEFREE, and MTDEBUGPATTERN. It is a bit of scope creep but I will poll my advisers. I personally like opening up libmtmalloc a bit. It would make performance analysis without source code easier. I have to have at least one environment variable because changing the maximum cache allocation value (MTMAXCACHE) must happen at init. rick On Tue, Jun 15, 2010 at 07:58:24PM +0200, ольга крыжановская wrote: > DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; > d=gmail.com; s=gamma; > h=domainkey-signature:mime-version:received:received:in-reply-to > :references:date:message-id:subject:from:to:cc:content-type; > bh=W3ykwvI/K+Sj5wLS3sSOMYgiRu404HybSwlP6XAuSmU=; > b=UqHjBIJv8oWMrjOJRE6SLHVhluGA1iFQAddEOTnuWqiNJmzDZ9c+zDJvZyu494DSgq > 7FYvVHcpiakob86zSEjgfnqmnQ+m8i8+5o53N7rDVd7djhIjLwdV3hP5XR+DPfyeFP0A > 6G2FlXSp+10etF0dJ77pVHiz98gGnoPyv1Nb4= > DomainKey-Signature: a=rsa-sha1; c=nofws; > d=gmail.com; s=gamma; > h=mime-version:in-reply-to:references:date:message-id:subject:from:to > :cc:content-type; > b=iM5KmsDQnRV2bQQwfvZHqY9zsjh+13Fl8B1KspgI4JQXT9lgnQnzVbEg74aoLHq5dR > IqbKmMQy1UBWIasvxNOehuw5cywibXlWkHF5ARxtYsr125Abg3ElB+Dn+QmG49OZOjJ0 > nbexQuLVPvaYTcVKPQ3c4qlfhOtKFPBqwqJxc= > In-Reply-To: <4C17BA17.4060803@oracle.com> > Date: Tue, 15 Jun 2010 19:58:24 +0200 > Subject: Re: Performance Improvements for libmtmalloc [PSARC/2010/212 > FastTrack, timeout 06/15/2010] > From: ольга крыжановская > To: Peter Dennis > Cc: PSARC-ext@sun.com, rick weisner > X-Source-IP: mail-iw0-f172.google.com [209.85.214.172] > X-CT-RefId: str=0001.0A090206.4C17BF42.0109,ss=1,fgs=0 > > I have a comment: Why do you use 3 environment variables? Environment > variables are very expensive to use, they significantly increase start > up time of an application and slow down all applications which use > getenv() or putenv() if the size of the environment is large. libast > had the same problem, with bitter and measurable impact on start up > time, and replaced all the allocator tunable variables into 1 variable > to improve start up time. > > IMO one environment variable to control the libmtmalloc behaviour > should be enough. > > Olga > > On Tue, Jun 15, 2010 at 7:36 PM, Peter Dennis wrote: > > This case times out today but there have been no comments from > > any PSARC members - any reviewers ? > > > > thanks > > pete > > _______________________________________________ > > opensolaris-arc mailing list > > opensolaris-arc@opensolaris.org > > > > > > -- > , _ _ , > { \/`o;====- Olga Kryzhanovska -====;o`\/ } > .----'-/`-/ olga.kryzhanovska@gmail.com \-`\-'----. > `'-..-| / http://twitter.com/fleyta \ |-..-'` > /\/\ Solaris/BSD//C/C++ programmer /\/\ > `--` `--` -- Rickey C. Weisner Software Development and Performance Specialist Principal Software Engineer Systems Quality Office Oracle Corporation cell phone: 615-308-1147 email: rick.weisner@oracle.com From bart.smaalders@oracle.com Tue Jun 15 14:23:09 2010 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o5FLN9lx004756 for ; Tue, 15 Jun 2010 14:23:09 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o5FLN9YH015387 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Tue, 15 Jun 2010 14:23:09 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L4200205RELG800@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 15 Jun 2010 14:23:09 -0700 (PDT) Received: from jurassic.Eng.Sun.COM ([129.146.17.63]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L4200FUUREKL8C0@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 15 Jun 2010 14:23:08 -0700 (PDT) Received: from [129.146.228.109] (cyber.SFBay.Sun.COM [129.146.228.109]) by jurassic.Eng.Sun.COM (8.14.4+Sun/8.14.4) with ESMTP id o5FLN8rK769667; Tue, 15 Jun 2010 14:23:08 -0700 (PDT) Date: Tue, 15 Jun 2010 14:23:07 -0700 From: Bart Smaalders Subject: Re: Performance Improvements for libmtmalloc [PSARC/2010/212 FastTrack, timeout 06/15/2010] In-reply-to: To: =?UTF-8?B?0L7Qu9GM0LPQsCDQutGA0YvQttCw0L3QvtCy0YHQutCw0Y8=?= Cc: Peter Dennis , PSARC-ext@sun.com, rick weisner Message-id: <4C17EF3B.2040400@oracle.com> Organization: Oracle MIME-version: 1.0 Content-type: text/plain; charset=UTF-8; format=flowed Content-transfer-encoding: 8BIT X-PMX-Version: 5.4.1.325704 References: <4C17BA17.4060803@oracle.com> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.9) Gecko/20100524 Lightning/1.0b1 Thunderbird/3.0.4 Status: RO Content-Length: 976 On 06/15/10 10:58, ольга крыжановская wrote: > I have a comment: Why do you use 3 environment variables? Environment > variables are very expensive to use, they significantly increase start > up time of an application and slow down all applications which use > getenv() or putenv() if the size of the environment is large. libast > had the same problem, with bitter and measurable impact on start up > time, and replaced all the allocator tunable variables into 1 variable > to improve start up time. Sorry... this makes little sense. Yes, if you have ten thousand environment variables, getenv can take significant time.... but this isn't the common case, and any program that needs mt malloc isn't worried about optimizing startup time. Typical use case is server applications. - Bart -- Bart Smaalders Solaris Kernel Performance bart.smaalders@oracle.com http://blogs.sun.com/barts "You will contribute more with mercurial than with thunderbird." From olga.kryzhanovska@gmail.com Tue Jun 15 15:31:47 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o5FMVlhg008800 for ; Tue, 15 Jun 2010 15:31:47 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o5FMVkxW005674 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Tue, 15 Jun 2010 17:31:46 -0500 (CDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L4200J05UKYSK00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 15 Jun 2010 15:31:46 -0700 (PDT) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L420050FUKYK150@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 15 Jun 2010 15:31:46 -0700 (PDT) Received: from relay13i.sun.com (ip123.net129179-4.block1.us.syntegra.com [129.179.4.123]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o5FMSB6D014491 for ; Tue, 15 Jun 2010 22:31:45 +0000 (GMT) Received: from mmp14es.mmp.us.syntegra.com ([160.41.208.14] [160.41.208.14]) by relay13i.sun.com with ESMTP id BT-MMP-3093699 for PSARC-ext@sun.com; Tue, 15 Jun 2010 22:31:45 +0000 (Z) Received: from relay13i.sun.com (relay13i.sun.com [129.179.4.123]) by mmp14es.mmp.us.syntegra.com with ESMTP id BT-MMP-124242 for PSARC-ext@sun.com; Tue, 15 Jun 2010 22:31:45 +0000 (Z) Received: from mail-iw0-f177.google.com ([209.85.214.177] [209.85.214.177]) by relay1i.sun.com with ESMTP id BT-MMP-4560522 for PSARC-ext@sun.com; Tue, 15 Jun 2010 22:31:45 +0000 (Z) Received: by iwn3 with SMTP id 3so1221316iwn.8 for ; Tue, 15 Jun 2010 15:31:45 -0700 (PDT) Received: by 10.42.0.68 with SMTP id 4mr2765070icb.93.1276641105071; Tue, 15 Jun 2010 15:31:45 -0700 (PDT) Received: by 10.231.190.10 with HTTP; Tue, 15 Jun 2010 15:31:45 -0700 (PDT) Date: Wed, 16 Jun 2010 00:31:45 +0200 From: =?KOI8-R?B?z8zYx8Egy9LZ1sHOz9fTy8HR?= Subject: Re: Performance Improvements for libmtmalloc [PSARC/2010/212 FastTrack, timeout 06/15/2010] In-reply-to: <4C17EF3B.2040400@oracle.com> To: Bart Smaalders Cc: Peter Dennis , PSARC-ext@sun.com, rick weisner Message-id: MIME-version: 1.0 Content-type: text/plain; charset=UTF-8 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=LAitXZErsGx9MI+47fe704Jux+hFg0A9Gur8pA2HDpc=; b=L1kUx1nlq57Hdb83jk0I4p+F+55ceNQeGFPwJ1sgXoTL1JF1RXHlCCj+9xsA/oU5o9 yvjiJJOMctkssHhKK/JmbI1ZtGGD4j22hmYt/xy8BsNoYkBLUT8FuKu0p7DGVyj3GXYo 2miL2i6t2Qf6tjLgfHhiPqAQ+TEwre0eoXfk4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=ONGun6P/7/KG0kpCfWZb8S1K2QhBrnyQNTjduJzRS9oDlkbOQpy9v73UEBf6UAhku9 zQel5cAetKnndtCnmb5SkZE7ASWavYaV/EZEt85IKP6MeVkulJRtWSzud5OXao1ffzJC imG2Q9SORo+Qy4Ovi+6UHJJ2yM4TDYbrRgmX8= X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-Antispam: No, score=-0.7/5.0, scanned in 0.077sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <4C17BA17.4060803@oracle.com> <4C17EF3B.2040400@oracle.com> Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by sac.sfbay.sun.com id o5FMVlhg008800 Status: RO Content-Length: 1879 Solaris getenv() and putenv() do not scale linear and it gets a lot *worse* above 100 or more variables. Adding 1 environment variable per libmtmalloc tunable makes IMO no sense, it is not economic nor is it fast to have that many variables for similar purposes. I don't buy the comment about server applications, this is about economic engineering, this should apply to all users of libmtmalloc, the small ones and the larger server applications. Olga 2010/6/15 Bart Smaalders : > On 06/15/10 10:58, ольга крыжановская wrote: >> >> I have a comment: Why do you use 3 environment variables? Environment >> variables are very expensive to use, they significantly increase start >> up time of an application and slow down all applications which use >> getenv() or putenv() if the size of the environment is large. libast >> had the same problem, with bitter and measurable impact on start up >> time, and replaced all the allocator tunable variables into 1 variable >> to improve start up time. > > Sorry... this makes little sense. Yes, if you have ten thousand > environment variables, getenv can take significant time.... but this > isn't the common case, and any program that needs mt malloc isn't > worried about optimizing startup time. > > Typical use case is server applications. > > - Bart > > > -- > Bart Smaalders Solaris Kernel Performance > bart.smaalders@oracle.com http://blogs.sun.com/barts > "You will contribute more with mercurial than with thunderbird." > -- , _ _ , { \/`o;====- Olga Kryzhanovska -====;o`\/ } .----'-/`-/ olga.kryzhanovska@gmail.com \-`\-'----. `'-..-| / http://twitter.com/fleyta \ |-..-'` /\/\ Solaris/BSD//C/C++ programmer /\/\ `--` `--` From darren.reed@oracle.com Tue Jun 15 16:51:20 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o5FNpKg3010275 for ; Tue, 15 Jun 2010 16:51:20 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o5FNpJNQ065337 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Tue, 15 Jun 2010 17:51:19 -0600 (MDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L4200J01Y9J5L00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 15 Jun 2010 17:51:19 -0600 (MDT) Received: from brmea-mail-2.sun.com ([192.18.98.43]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L4200C7JY9J58F0@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 15 Jun 2010 17:51:19 -0600 (MDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o5FNpILB006446 for ; Tue, 15 Jun 2010 23:51:18 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o5FNpEVW020141; Tue, 15 Jun 2010 23:51:14 +0000 (GMT) Received: from abhmt009.oracle.com by acsmt353.oracle.com with ESMTP id 328537441276645756; Tue, 15 Jun 2010 16:49:16 -0700 Received: from mintslice.sfbay.sun.com (/129.146.106.55) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 15 Jun 2010 16:49:16 -0700 Date: Tue, 15 Jun 2010 16:49:42 -0700 From: Darren Reed Subject: Re: Performance Improvements for libmtmalloc [PSARC/2010/212 FastTrack, timeout 06/15/2010] In-reply-to: To: =?UTF-8?B?0L7Qu9GM0LPQsCDQutGA0YvQttCw0L3QvtCy0YHQutCw0Y8=?= Cc: Bart Smaalders , Peter Dennis , PSARC-ext@sun.com, rick weisner Message-id: <4C181196.10501@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=UTF-8; format=flowed Content-transfer-encoding: 8BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090207.4C1811F4.001A:SCFMA4539814,ss=1,fgs=0 References: <4C17BA17.4060803@oracle.com> <4C17EF3B.2040400@oracle.com> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-GB; rv:1.9.1.5) Gecko/20091206 Thunderbird/3.0 Status: RO Content-Length: 1815 On 15/06/10 03:31 PM, ольга крыжановская wrote: > Solaris getenv() and putenv() do not scale linear and it gets a lot > *worse* above 100 or more variables. Adding 1 environment variable per > libmtmalloc tunable makes IMO no sense, it is not economic nor is it > fast to have that many variables for similar purposes. > I don't buy the comment about server applications, this is about > economic engineering, this should apply to all users of libmtmalloc, > the small ones and the larger server applications. > Then file a bug for getenv() and putenv() (maybe even 1 each?), stating that their performance does not scale well. Darren > 2010/6/15 Bart Smaalders: > >> On 06/15/10 10:58, ольга крыжановская wrote: >> >>> I have a comment: Why do you use 3 environment variables? Environment >>> variables are very expensive to use, they significantly increase start >>> up time of an application and slow down all applications which use >>> getenv() or putenv() if the size of the environment is large. libast >>> had the same problem, with bitter and measurable impact on start up >>> time, and replaced all the allocator tunable variables into 1 variable >>> to improve start up time. >>> >> Sorry... this makes little sense. Yes, if you have ten thousand >> environment variables, getenv can take significant time.... but this >> isn't the common case, and any program that needs mt malloc isn't >> worried about optimizing startup time. >> >> Typical use case is server applications. >> >> - Bart >> >> >> -- >> Bart Smaalders Solaris Kernel Performance >> bart.smaalders@oracle.com http://blogs.sun.com/barts >> "You will contribute more with mercurial than with thunderbird." >> >> > > > From olga.kryzhanovska@gmail.com Tue Jun 15 16:55:43 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o5FNthoj010569 for ; Tue, 15 Jun 2010 16:55:43 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o5FNtgUi001402 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Tue, 15 Jun 2010 17:55:43 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L4200K09YGVVT00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 15 Jun 2010 16:55:43 -0700 (PDT) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L42005KBYGUK2B0@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 15 Jun 2010 16:55:42 -0700 (PDT) Received: from relay13i.sun.com (ip123.net129179-4.block1.us.syntegra.com [129.179.4.123]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o5FNt5q0001903 for ; Tue, 15 Jun 2010 23:55:42 +0000 (GMT) Received: from mmp12es.mmp.us.syntegra.com ([160.41.208.12] [160.41.208.12]) by relay13i.sun.com with ESMTP id BT-MMP-3096451 for PSARC-ext@sun.com; Tue, 15 Jun 2010 23:55:42 +0000 (Z) Received: from relay15i.sun.com (relay15i.sun.com [129.179.4.125]) by mmp12es.mmp.us.syntegra.com with ESMTP id BT-MMP-279869924 for PSARC-ext@sun.com; Tue, 15 Jun 2010 23:55:41 +0000 (Z) Received: from mail-iw0-f177.google.com ([209.85.214.177] [209.85.214.177]) by relay1i.sun.com with ESMTP id BT-MMP-13114379 for PSARC-ext@sun.com; Tue, 15 Jun 2010 23:55:41 +0000 (Z) Received: by mail-iw0-f177.google.com with SMTP id 3so1291925iwn.8 for ; Tue, 15 Jun 2010 16:55:41 -0700 (PDT) Received: by 10.231.130.152 with SMTP id t24mr8917599ibs.146.1276646141568; Tue, 15 Jun 2010 16:55:41 -0700 (PDT) Received: by 10.231.190.10 with HTTP; Tue, 15 Jun 2010 16:55:41 -0700 (PDT) Date: Wed, 16 Jun 2010 01:55:41 +0200 From: =?KOI8-R?B?z8zYx8Egy9LZ1sHOz9fTy8HR?= Subject: Re: Performance Improvements for libmtmalloc [PSARC/2010/212 FastTrack, timeout 06/15/2010] In-reply-to: <4C181196.10501@oracle.com> To: Darren Reed Cc: Bart Smaalders , Peter Dennis , PSARC-ext@sun.com, rick weisner Message-id: MIME-version: 1.0 Content-type: text/plain; charset=UTF-8 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=1VdAfu1vSe10yAhpa8OjkEg1BjSAfEJDjAAN75Ig6k0=; b=Go2/x6NrcPpFMRJJ5KfdUwVf3NYU/y+1t30P1FeBiq5v3jnt7DL+U7mJIGBYTHIsxb ihyipYgEm80ZQvCSNDnH9F0o00JQZXRiqNiOWTwAOlZE9FMha/yItn8VFqJFGVSFTv/z N9MFpgFM7srdTmnDw4bh69jOuNWnyZ7kgyI8g= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=qCrFnIgCzEnX1nWQ9NJmLe6Z36784b63FIZj3031QHSIs23MWiFheoFgmtxPphIHhl SA1xDVFH0VwGJT5vZ89RIPMBexCyNoG69AM7eW80+TS9gdaGqyi6ZoDQ40wmlVWvwzqo knADXyW2ACFO9wzvs5GdC2QWMEg2t4cZVGH2E= X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-Antispam: No, score=-0.2/5.0, scanned in 0.085sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <4C17BA17.4060803@oracle.com> <4C17EF3B.2040400@oracle.com> <4C181196.10501@oracle.com> Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by sac.sfbay.sun.com id o5FNthoj010569 Status: RO Content-Length: 2337 I already looked at the problem. This can not be fixed without breaking binary compatibility. Olga 2010/6/16 Darren Reed : > On 15/06/10 03:31 PM, ольга крыжановская wrote: >> >> Solaris getenv() and putenv() do not scale linear and it gets a lot >> *worse* above 100 or more variables. Adding 1 environment variable per >> libmtmalloc tunable makes IMO no sense, it is not economic nor is it >> fast to have that many variables for similar purposes. >> I don't buy the comment about server applications, this is about >> economic engineering, this should apply to all users of libmtmalloc, >> the small ones and the larger server applications. >> > > Then file a bug for getenv() and putenv() (maybe even 1 each?), > stating that their performance does not scale well. > > Darren > > >> 2010/6/15 Bart Smaalders: >> >>> >>> On 06/15/10 10:58, ольга крыжановская wrote: >>> >>>> >>>> I have a comment: Why do you use 3 environment variables? Environment >>>> variables are very expensive to use, they significantly increase start >>>> up time of an application and slow down all applications which use >>>> getenv() or putenv() if the size of the environment is large. libast >>>> had the same problem, with bitter and measurable impact on start up >>>> time, and replaced all the allocator tunable variables into 1 variable >>>> to improve start up time. >>>> >>> >>> Sorry... this makes little sense. Yes, if you have ten thousand >>> environment variables, getenv can take significant time.... but this >>> isn't the common case, and any program that needs mt malloc isn't >>> worried about optimizing startup time. >>> >>> Typical use case is server applications. >>> >>> - Bart >>> >>> >>> -- >>> Bart Smaalders Solaris Kernel Performance >>> bart.smaalders@oracle.com http://blogs.sun.com/barts >>> "You will contribute more with mercurial than with thunderbird." >>> >>> >> >> >> > > -- , _ _ , { \/`o;====- Olga Kryzhanovska -====;o`\/ } .----'-/`-/ olga.kryzhanovska@gmail.com \-`\-'----. `'-..-| / http://twitter.com/fleyta \ |-..-'` /\/\ Solaris/BSD//C/C++ programmer /\/\ `--` `--` From seth.goldberg@oracle.com Tue Jun 15 17:03:51 2010 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o5G03oiI010873 for ; Tue, 15 Jun 2010 17:03:50 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o5G03oCg013311 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Tue, 15 Jun 2010 17:03:50 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L4200001YUB1X00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 15 Jun 2010 17:03:47 -0700 (PDT) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L420050PYUBK1C0@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 15 Jun 2010 17:03:47 -0700 (PDT) Received: from rcsinet15.oracle.com (rcsinet15.oracle.com [148.87.113.117]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o5G03kDI004915 for ; Wed, 16 Jun 2010 00:03:47 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by rcsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o5G03i3T015057; Wed, 16 Jun 2010 00:03:44 +0000 (GMT) Received: from abhmt016.oracle.com by acsmt353.oracle.com with ESMTP id 328564041276646542; Tue, 15 Jun 2010 17:02:22 -0700 Received: from kasha (/10.1.48.74) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 15 Jun 2010 17:02:22 -0700 Date: Tue, 15 Jun 2010 17:02:06 -0700 (PDT) From: Seth Goldberg Subject: Re: Performance Improvements for libmtmalloc [PSARC/2010/212 FastTrack, timeout 06/15/2010] In-reply-to: X-X-Sender: sethg@bergsoft.local To: =?KOI8-R?B?z8zYx8Egy9LZ1sHOz9fTy8HR?= Cc: Darren Reed , Bart Smaalders , Peter Dennis , PSARC-ext@sun.com, rick weisner Message-id: MIME-version: 1.0 Content-type: multipart/mixed; boundary="Boundary_(ID_USGGjEueOLq3PuvahStjEw)" X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090209.4C1814E0.0153:SCFMA4539814,ss=1,fgs=0 References: <4C17BA17.4060803@oracle.com> <4C17EF3B.2040400@oracle.com> <4C181196.10501@oracle.com> User-Agent: Alpine 2.00 (GSO 1167 2008-08-23) Status: RO Content-Length: 3477 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --Boundary_(ID_USGGjEueOLq3PuvahStjEw) Content-type: TEXT/PLAIN; charset=UTF-8; format=flowed Content-transfer-encoding: QUOTED-PRINTABLE Hi, If you forward your analysis to us, we can add it to a bug for you. Su= rely=20 we can come up with a way to optimize environment handling (perhaps with a = new=20 interface that future executables will use, while deprecating the existing= =20 mechanism). Such pathological performance is certainly a bug and, if its= =20 severity turns out to be as dire as you say, it needs to be fixed. --S Quoting =D0=BE=D0=BB=D1=8C=D0=B3=D0=B0 =D0=BA=D1=80=D1=8B=D0=B6=D0=B0=D0=BD= =D0=BE=D0=B2=D1=81=D0=BA=D0=B0=D1=8F, who wrote the following on Wed, 16 Ju= n 2010: > I already looked at the problem. This can not be fixed without > breaking binary compatibility. > > Olga > > 2010/6/16 Darren Reed : >> On 15/06/10 03:31 PM, =D0=BE=D0=BB=D1=8C=D0=B3=D0=B0 =D0=BA=D1=80=D1=8B= =D0=B6=D0=B0=D0=BD=D0=BE=D0=B2=D1=81=D0=BA=D0=B0=D1=8F wrote: >>> >>> Solaris getenv() and putenv() do not scale linear and it gets a lot >>> *worse* above 100 or more variables. Adding 1 environment variable per >>> libmtmalloc tunable makes IMO no sense, it is not economic nor is it >>> fast to have that many variables for similar purposes. >>> I don't buy the comment about server applications, this is about >>> economic engineering, this should apply to all users of libmtmalloc, >>> the small ones and the larger server applications. >>> >> >> Then file a bug for getenv() and putenv() (maybe even 1 each?), >> stating that their performance does not scale well. >> >> Darren >> >> >>> 2010/6/15 Bart Smaalders: >>> >>>> >>>> On 06/15/10 10:58, =D0=BE=D0=BB=D1=8C=D0=B3=D0=B0 =D0=BA=D1=80=D1=8B= =D0=B6=D0=B0=D0=BD=D0=BE=D0=B2=D1=81=D0=BA=D0=B0=D1=8F wrote: >>>> >>>>> >>>>> I have a comment: Why do you use 3 environment variables? Environment >>>>> variables are very expensive to use, they significantly increase star= t >>>>> up time of an application and slow down all applications which use >>>>> getenv() or putenv() if the size of the environment is large. libast >>>>> had the same problem, with bitter and measurable impact on start up >>>>> time, and replaced all the allocator tunable variables into 1 variabl= e >>>>> to improve start up time. >>>>> >>>> >>>> Sorry... this makes little sense. Yes, if you have ten thousand >>>> environment variables, getenv can take significant time.... but this >>>> isn't the common case, and any program that needs mt malloc isn't >>>> worried about optimizing startup time. >>>> >>>> Typical use case is server applications. >>>> >>>> - Bart >>>> >>>> >>>> -- >>>> Bart Smaalders Solaris Kernel Performance >>>> bart.smaalders@oracle.com http://blogs.sun.com/barts >>>> "You will contribute more with mercurial than with thunderbird." >>>> >>>> >>> >>> >>> >> >> > > > > -- > , _ _ , > { \/`o;=3D=3D=3D=3D- Olga Kryzhanovska -=3D=3D=3D=3D;o`\/ } > .----'-/`-/ olga.kryzhanovska@gmail.com \-`\-'----. > `'-..-| / http://twitter.com/fleyta \ |-..-'` > /\/\ Solaris/BSD//C/C++ programmer /\/\ > `--` `--` >= --Boundary_(ID_USGGjEueOLq3PuvahStjEw)-- From bart.smaalders@oracle.com Tue Jun 15 18:55:52 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o5G1tq4V013162 for ; Tue, 15 Jun 2010 18:55:52 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o5G1tpML028643 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Tue, 15 Jun 2010 20:55:51 -0500 (CDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L43009094132X00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 15 Jun 2010 18:55:51 -0700 (PDT) Received: from jurassic.Eng.Sun.COM ([129.146.17.59]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L43006Q6412L400@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 15 Jun 2010 18:55:50 -0700 (PDT) Received: from [129.146.228.109] (cyber.SFBay.Sun.COM [129.146.228.109]) by jurassic.Eng.Sun.COM (8.14.4+Sun/8.14.4) with ESMTP id o5G1tqRR789220; Tue, 15 Jun 2010 18:55:52 -0700 (PDT) Date: Tue, 15 Jun 2010 18:55:49 -0700 From: Bart Smaalders Subject: Re: Performance Improvements for libmtmalloc [PSARC/2010/212 FastTrack, timeout 06/15/2010] In-reply-to: To: =?UTF-8?B?0L7Qu9GM0LPQsCDQutGA0YvQttCw0L3QvtCy0YHQutCw0Y8=?= Cc: Peter Dennis , PSARC-ext@sun.com, rick weisner Message-id: <4C182F25.7000507@oracle.com> Organization: Oracle MIME-version: 1.0 Content-type: text/plain; charset=UTF-8; format=flowed Content-transfer-encoding: 8BIT X-PMX-Version: 5.4.1.325704 References: <4C17BA17.4060803@oracle.com> <4C17EF3B.2040400@oracle.com> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.9) Gecko/20100524 Lightning/1.0b1 Thunderbird/3.0.4 Status: RO Content-Length: 1576 On 06/15/10 15:31, ольга крыжановская wrote: > Solaris getenv() and putenv() do not scale linear and it gets a lot > *worse* above 100 or more variables. Running libmicro's getenv test case indicates that this is not the case; performance is linear with size of environment aside from cache effects. Increasing the size of the environment by a factor of 10 each time from 100 environment variables to 100000: : barts@cyber[114]; : barts@cyber[110]; bin/getenv -s 100 -D 100 -B 100 -1 prc thr usecs/call samples errors cnt/samp getenv 1 1 0.10884 8571 0 100 : barts@cyber[114]; : barts@cyber[110]; bin/getenv -s 1000 -D 100 -B 100 -1 prc thr usecs/call samples errors cnt/samp getenv 1 1 0.91209 700 0 100 : barts@cyber[115]; : barts@cyber[110]; bin/getenv -s 10000 -D 100 -B 100 -1 prc thr usecs/call samples errors cnt/samp getenv 1 1 12.37686 62 0 100 : barts@cyber[116]; : barts@cyber[110]; bin/getenv -s 100000 -D 100 -B 100 -1 prc thr usecs/call samples errors cnt/samp getenv 1 1 127.88930 98 0 100 Please note that retrieving a single environment variable out of 100 took about 110 nanoseconds on my desktop - not exactly a long time. - Bart -- Bart Smaalders Solaris Kernel Performance bart.smaalders@oracle.com http://blogs.sun.com/barts "You will contribute more with mercurial than with thunderbird." From peter.dennis@oracle.com Wed Jun 16 08:58:44 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o5GFwiOG021488 for ; Wed, 16 Jun 2010 08:58:44 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o5GFwZcs024325 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 16 Jun 2010 09:58:43 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L4400G2371V0N00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 16 Jun 2010 08:58:43 -0700 (PDT) Received: from sca-ea-mail-2.sun.com ([192.18.43.25]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L4400CJX71VOM40@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 16 Jun 2010 08:58:43 -0700 (PDT) Received: from rcsinet15.oracle.com (rcsinet15.oracle.com [148.87.113.117]) by sca-ea-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o5GFwg3n021375 for ; Wed, 16 Jun 2010 15:58:42 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by rcsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o5GB8o3p023904 for ; Wed, 16 Jun 2010 15:58:41 +0000 (GMT) Received: from abhmt020.oracle.com by acsmt355.oracle.com with ESMTP id 350113631276703848; Wed, 16 Jun 2010 08:57:28 -0700 Received: from [129.150.120.241] (/129.150.120.241) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 16 Jun 2010 08:57:28 -0700 Date: Wed, 16 Jun 2010 16:57:26 +0100 From: Peter Dennis Subject: Performance Improvements for libmtmalloc [PSARC/2010/212 FastTrack, timeout 06/15/2010] To: PSARC-ext@sun.com Cc: rick weisner Message-id: <4C18F466.7070905@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090208.4C18F4B2.0016:SCFMA4539814,ss=1,fgs=0 User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.7) Gecko/20100214 Lightning/1.0b1 Thunderbird/3.0.1 Status: RO Content-Length: 152 I'm moving this case to waiting needs spec as the comments received have indicated that the scope of the environment variables could be expanded. Pete From sebastien.roy@oracle.com Wed Jun 16 09:59:19 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o5GGxJut024021 for ; Wed, 16 Jun 2010 09:59:19 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o5GGxIet023357 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 16 Jun 2010 11:59:18 -0500 (CDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L4400F1N9UUYO00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 16 Jun 2010 09:59:18 -0700 (PDT) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L4400EWF9UPXKD0@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 16 Jun 2010 09:59:13 -0700 (PDT) Received: from rcsinet13.oracle.com (rcsinet13.oracle.com [148.87.113.125]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o5GGxCT4018695; Wed, 16 Jun 2010 16:59:12 +0000 (GMT) Received: from acsmt353.oracle.com (acsmt353.oracle.com [141.146.40.153]) by rcsinet13.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o5GGx8Of020185; Wed, 16 Jun 2010 16:59:11 +0000 (GMT) Received: from abhmt001.oracle.com by acsmt353.oracle.com with ESMTP id 350313971276707541; Wed, 16 Jun 2010 09:59:01 -0700 Received: from [129.148.19.4] (/129.148.19.4) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 16 Jun 2010 09:59:01 -0700 Date: Wed, 16 Jun 2010 12:58:58 -0400 From: Sebastien Roy Subject: Re: Performance Improvements for libmtmalloc [PSARC/2010/212 FastTrack timeout 06/15/2010] In-reply-to: <201006080912.o589CSp9002089@sac.sfbay.sun.com> To: Peter Dennis Cc: PSARC-ext@sun.com, rick.weisner@oracle.com Message-id: <4C1902D2.8090307@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt353.oracle.com [141.146.40.153] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090203.4C1902E0.0070:SCFMA4539814,ss=1,fgs=0 References: <201006080912.o589CSp9002089@sac.sfbay.sun.com> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.9) Gecko/20100524 Lightning/1.0b1 Thunderbird/3.0.4 Status: RO Content-Length: 27 +1, I have no issues. -Seb From peter.dennis@oracle.com Mon Jun 21 07:59:16 2010 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o5LExGCB008991 for ; Mon, 21 Jun 2010 07:59:16 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o5LExG05004833 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Mon, 21 Jun 2010 07:59:16 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L4D00805DMSKY00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 21 Jun 2010 08:59:16 -0600 (MDT) Received: from brmea-mail-2.sun.com ([192.18.98.43]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L4D004EZDMRCZ20@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 21 Jun 2010 08:59:15 -0600 (MDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o5LExFHG003721 for ; Mon, 21 Jun 2010 14:59:15 +0000 (GMT) Received: from acsmt353.oracle.com (acsmt353.oracle.com [141.146.40.153]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o5LExBLM023148 for ; Mon, 21 Jun 2010 14:59:14 +0000 (GMT) Received: from abhmt009.oracle.com by acsmt354.oracle.com with ESMTP id 360290161277132332; Mon, 21 Jun 2010 07:58:52 -0700 Received: from [129.150.120.56] (/129.150.120.56) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 21 Jun 2010 07:58:51 -0700 Date: Mon, 21 Jun 2010 15:58:48 +0100 From: Peter Dennis Subject: Performance Improvements for libmtmalloc [PSARC/2010/212 FastTrack, timeout 06/28/2010] To: PSARC-ext@sun.com Cc: rick weisner Message-id: <4C1F7E28.8010308@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt353.oracle.com [141.146.40.153] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090207.4C1F7E42.0163:SCFMA4539814,ss=1,fgs=0 User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.7) Gecko/20100214 Lightning/1.0b1 Thunderbird/3.0.1 Status: RO Content-Length: 6237 I've reset the timer for this case to 28-Jun-2010. Modified proposal is below (essentially to include all the mtmalloc(3MALLOC) options to the environment setting). Modified man pages are in the materials sub directory. Template Version: @(#)sac_nextcase 1.70 05/10/10 SMI This information is Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved. 1. Introduction 1.1. Project/Component Working Name: Performance Improvements for libmtmalloc 1.2. Name of Document Author/Supplier: Author: Rick Weisner 1.3 Date of This Document: 18 June, 2010 4. Technical Description SUMMARY Under the following two situations libmtmalloc has shown poor scalability. 1. When there are large numbers of allocating threads. (see CR6922229) and 2. When the allocation size is larger than 64 KB. (see CR6555149) We will remedy the above scalability issues by: 1) Using atomic operations to eliminate the cache lock in libmtmalloc. 2) Provide a mechanism whereby the parent lock can also be eliminated for threads whose id is less than 2* the number of cpus. 3) Make the maximum cacheable requestsize tunable via an environment variable. BACKGROUND libmtmalloc organizes avaiable address space into buckets. Each thread which calls malloc is assigned a bucket based upon its thread id. The per bucket parent lock controls the use of each bucket. Each bucket is a list of caches based on size. Each list is protected by a cache lock. Applications with a large number of allocating threads may have their performance limited by contention for these locks. These sort of applications are not unusual in the Telco space. Larger allocations sizes are also becoming more common. With 64 bit applications, terabytes of memory, and hundreds of threads it is advantageous to be able to adjust the maximum cacheable requestsize to better suit the needs of the application. PROBLEM A customer's application did not perform as needed on a Netra 5440. DTrace indicated lock contention relating to memory allocation in libmtmalloc. The customer provided some code that provided dramatic performance increases by eliminating the "cache" locks and "parent" locks from libmtmalloc and replacing them with atomic operations. The customer's code was not threadsafe in general but was promising. In a different case the customer states: We observed that db is hitting oversize_lock mutex due to the memory needed to be allocated is more than MAX_CACHED. Sometimes acquiring the oversize_lock mutex is taking more than 2sec, causing the db performance to degrade. (see 6555149) PROPOSAL 1) Eliminate the cache lock by using atomic operations. 2) Add a new option to mallocctl(3MALLOC) that activates the use of exclusive buckets for threads whose ID is < 2 * the number of CPUs. The value argument associated with the mallocctl option is ignored. Once the option has been called there is no facility to 'unset' it. A new environment variable will be introduced, MTMALLOC_OPTIONS. It will consist of a comma separated list of options in the style of umem_debug. We will support the following options: MTEXCLUSIVE=Y or y or yes or Yes, or anything that starts with y, to enable the use of exclusive buckets, MTMAXCACHE=16,17,18,19 or 20 (see below), and/or MYCHUNKSIZE=xx where xx is a number use to size the buckets. The default is 9. Invalid options are silently ignored. This feature is needed for situations where the source code is unavailable. This feature will also assist in performance analysis. 3) Introduce the option MTMAXCACHE in the environment variable, MTMALLOC_OPTIONS, which will set the maximum request size that is cached. It will have the values of 16 to 21. The default is 16 which means that requests less than 2^^16 are cached. With this value we can support up to 2mb (2^^21) request sizes in cache. If the value of MTMAXCACHE is set to something outside of the ranges then it will use either 16 or 21 (which ever bound has been broken by the value set). It is necessary to use an environment variable instead of a mallocctl interface because the MTMAXCACHE must be determined before malloc_init calls setup_caches. Here is an example MTMALLOC_OPTIONS: export MTMALLOC_OPTIONS="MTEXCLUSIVE=Y,MTMAXCACHE=17,MTCHUNKSIZE=64" DETAILS The code has been developed and tested in 64 bit mode on Solaris 10 u6 on a Netra T5440. The test harness uses a configurable number of allocation threads, a configurable sample count, a configurable "maximum" allocation size. Each allocation thread has a configurable number of ramdom or fixed size allocations between 8 and the requested "max" allocation size + 1/2 the "max" allocation size. A freeing thread then releases the allocations while the allocating thread performs a fresh set of allocations. In initial testing with "stock" libmtmalloc it was possible to do 6300 64 bit operations per sec on the N5440. With the "atomic" library this increases to 15000. COMMENTS Exported Interfaces: MTEXCLUSIVE Stable option for mallocctl(3MALLOC). MTMALLOC_OPTIONS Stable Shell environmet variable supporting the following options: MTEXCLUSIVE=Y MTMAXCACHE=16,17,18,19,20 or 21 MTCHUNKSIZE=xx where xx is a number Reference: 6922229 libmtmalloc would benefit from atomic operations 6555149 poor performance with libmtmalloc compared to libc 6956786 Provide a tunable to tweak the MAX_CACHED threshold in libmtmalloc DELIVERY VEHICLE Solaris RELEASE Patch COMMITMENT LEVEL 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: FastTrack From peter.dennis@oracle.com Mon Jun 28 13:46:04 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o5SKk3jQ021795 for ; Mon, 28 Jun 2010 13:46:03 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o5SKk2Qc029732 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Mon, 28 Jun 2010 15:46:03 -0500 (CDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L4Q00J0FSCRKW00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 28 Jun 2010 13:46:03 -0700 (PDT) Received: from sca-ea-mail-4.sun.com ([192.18.43.22]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L4Q00HITSCQIE10@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 28 Jun 2010 13:46:02 -0700 (PDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by sca-ea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o5SKk1uW004339 for ; Mon, 28 Jun 2010 20:46:02 +0000 (GMT) Received: from acsmt355.oracle.com (acsmt355.oracle.com [141.146.40.155]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o5SK6aKU006106 for ; Mon, 28 Jun 2010 20:46:00 +0000 (GMT) Received: from abhmt016.oracle.com by acsmt353.oracle.com with ESMTP id 380454131277757920; Mon, 28 Jun 2010 13:45:20 -0700 Received: from [10.7.251.4] (/10.7.251.4) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 28 Jun 2010 13:45:20 -0700 Date: Mon, 28 Jun 2010 21:45:17 +0100 From: Pete Dennis Subject: Re: Performance Improvements for libmtmalloc [PSARC/2010/212 FastTrack, timeout 06/28/2010] In-reply-to: <4C1F7E28.8010308@oracle.com> To: PSARC-ext@sun.com Cc: rick weisner Message-id: <4C2909DD.5080301@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt355.oracle.com [141.146.40.155] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090202.4C290A08.01EF:SCFMA4539814,ss=1,fgs=0 References: <4C1F7E28.8010308@oracle.com> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.7) Gecko/20100214 Lightning/1.0b1 Thunderbird/3.0.1 Status: RO Content-Length: 199 The timer is about to complete on this but no +1's received (I'm not going to count the +1 for the previous proposal as it has changed with respect to the usage of the environment variables). pete From garrett@damore.org Mon Jun 28 13:54:42 2010 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o5SKsgOj022318 for ; Mon, 28 Jun 2010 13:54:42 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o5SKsfDi026097 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Mon, 28 Jun 2010 13:54:41 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L4Q0040DSR5C400@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 28 Jun 2010 14:54:41 -0600 (MDT) Received: from sca-ea-mail-3.sun.com ([192.18.43.21]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L4Q00D0USR5TG90@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 28 Jun 2010 14:54:41 -0600 (MDT) Received: from relay13i.sun.com (ip123.net129179-4.block1.us.syntegra.com [129.179.4.123]) by sca-ea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o5SKbadK005226 for ; Mon, 28 Jun 2010 20:54:40 +0000 (GMT) Received: from mmp14es.mmp.us.syntegra.com ([160.41.208.14] [160.41.208.14]) by relay13i.sun.com with ESMTP id BT-MMP-3873661 for PSARC-ext@sun.com; Mon, 28 Jun 2010 20:54:40 +0000 (Z) Received: from relay15i.sun.com (relay15i.sun.com [129.179.4.125]) by mmp14es.mmp.us.syntegra.com with ESMTP id BT-MMP-766817 for PSARC-ext@sun.com; Mon, 28 Jun 2010 20:54:40 +0000 (Z) Received: from oproxy1-pub.bluehost.com ([66.147.249.253] [66.147.249.253]) by relay1i.sun.com id BT-MMP-923678 for PSARC-ext@sun.com; Mon, 28 Jun 2010 20:54:39 +0000 (Z) Received: (qmail 16257 invoked by uid 0); Mon, 28 Jun 2010 20:54:39 +0000 Received: from unknown (HELO box374.bluehost.com) (69.89.31.174) by oproxy1.bluehost.com.bluehost.com with SMTP; Mon, 28 Jun 2010 20:54:39 +0000 Received: from cpe-76-93-15-33.socal.res.rr.com ([76.93.15.33] helo=[192.168.251.110]) by box374.bluehost.com with esmtpsa (SSLv3:AES256-SHA:256) (Exim 4.69) (envelope-from ) id 1OTLLj-00017q-Ad; Mon, 28 Jun 2010 14:54:39 -0600 Date: Mon, 28 Jun 2010 13:56:49 -0700 From: "Garrett D'Amore" Subject: Re: Performance Improvements for libmtmalloc [PSARC/2010/212 FastTrack, timeout 06/28/2010] In-reply-to: <4C2909DD.5080301@oracle.com> To: Pete Dennis Cc: PSARC-ext@sun.com, rick weisner Message-id: <1277758609.5596.4407.camel@velocity> MIME-version: 1.0 X-Mailer: Evolution 2.28.3 Content-type: text/plain; charset=UTF-8 Content-transfer-encoding: 7BIT DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=default; d=damore.org; h=Received:Subject:From:To:Cc:In-Reply-To:References:Content-Type:Date:Message-ID:Mime-Version:X-Mailer:Content-Transfer-Encoding:X-Identified-User; b=sKgOSeIRnN6ZIUvPX4syQJqjMIpR5mFvMmG7n/8yohTPZG1+ukFjwcmEHB1gcYesATGuycGZzfzazCkNu29Sg1yGLnJmdQ5IOdpUcKvlRZuQWGRYu8PPm5hVcC9T1dU2; X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-Identified-User: {2225:box374.bluehost.com:damoreor:damore.org} {sentby:smtp auth 76.93.15.33 authed with garrett+damore.org} X-Antispam: No, score=0.0/5.0, scanned in 0.250sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <4C1F7E28.8010308@oracle.com> <4C2909DD.5080301@oracle.com> Status: RO Content-Length: 401 +1 -- Garrett On Mon, 2010-06-28 at 21:45 +0100, Pete Dennis wrote: > The timer is about to complete on this but no +1's received > (I'm not going to count the +1 for the previous proposal > as it has changed with respect to the usage of the environment > variables). > > pete > > _______________________________________________ > opensolaris-arc mailing list > opensolaris-arc@opensolaris.org From peter.dennis@oracle.com Wed Jun 30 10:07:13 2010 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o5UH7DW5001438 for ; Wed, 30 Jun 2010 10:07:13 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o5UH7DLO002669 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 30 Jun 2010 10:07:13 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L4U00J0B7K17U00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 30 Jun 2010 10:07:13 -0700 (PDT) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L4U0085K7K0ZD50@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 30 Jun 2010 10:07:12 -0700 (PDT) Received: from rcsinet13.oracle.com (rcsinet13.oracle.com [148.87.113.125]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o5UH7B7B003158 for ; Wed, 30 Jun 2010 17:07:12 +0000 (GMT) Received: from acsmt355.oracle.com (acsmt355.oracle.com [141.146.40.155]) by rcsinet13.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o5UEqBDc000849 for ; Wed, 30 Jun 2010 17:07:08 +0000 (GMT) Received: from abhmt010.oracle.com by acsmt355.oracle.com with ESMTP id 370614301277917570; Wed, 30 Jun 2010 10:06:10 -0700 Received: from [129.150.120.56] (/129.150.120.56) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 30 Jun 2010 10:06:09 -0700 Date: Wed, 30 Jun 2010 18:06:06 +0100 From: Peter Dennis Subject: Performance Improvements for libmtmalloc [PSARC/2010/212 FastTrack, timeout 06/28/2010] To: PSARC-ext@sun.com Cc: rick weisner Message-id: <4C2B797E.6090804@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt355.oracle.com [141.146.40.155] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090204.4C2B79BF.00DC:SCFMA4539814,ss=1,fgs=0 User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.7) Gecko/20100214 Lightning/1.0b1 Thunderbird/3.0.1 Status: RO Content-Length: 132 This case is closed approved. It was approved during the PSARC meeting 30-Jun-2010 as well as receiving a +1 and timing out. Pete