From jyri@sac.sfbay.sun.com Fri Nov 6 01:16:44 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nA69Gh5d018620 for ; Fri, 6 Nov 2009 01:16:43 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id nA69G9xt002206; Fri, 6 Nov 2009 17:16:42 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KSO00F1DKFRBG00@nwk-avmta-2.sfbay.sun.com>; Fri, 06 Nov 2009 01:16:39 -0800 (PST) Received: from dm-sfbay-01.sfbay.sun.com ([129.145.155.118]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KSO009FJKFRTF80@nwk-avmta-2.sfbay.sun.com>; Fri, 06 Nov 2009 01:16:39 -0800 (PST) Received: from sac.sfbay.sun.com (sac.SFBay.Sun.COM [129.146.226.132]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id nA69GckK029172; Fri, 06 Nov 2009 01:16:38 -0800 (PST) Received: from sac.sfbay.sun.com (localhost [127.0.0.1]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nA69Gaal018615; Fri, 06 Nov 2009 01:16:36 -0800 (PST) Received: (from jyri@localhost) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id nA69Gaq2018611; Fri, 06 Nov 2009 01:16:36 -0800 (PST) Date: Fri, 06 Nov 2009 01:16:36 -0800 (PST) From: Jyri Virkki Subject: Deliver collectd 4.8.x into OpenSolaris [LSARC/2009/606 FastTrack timeout 11/13/2009] To: LSARC-ext@sun.com Cc: webstack-discuss@opensolaris.org Message-id: <200911060916.nA69Gaq2018611@sac.sfbay.sun.com> Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 Status: RO Content-Length: 14030 I am sponsoring this case for Amit Gupta. It introduces collectd into OpenSolaris. Timeout set to Nov.13. Template Version: @(#)sac_nextcase 1.68 02/23/09 SMI This information is Copyright 2009 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: Deliver collectd 4.8.x into OpenSolaris 1.2. Name of Document Author/Supplier: Author: Amit Gupta 1.3 Date of This Document: 06 November, 2009 4. Technical Description Deliver collectd 4.8.x into OpenSolaris 6 November 2009 1. Summary and motivation 1.1. Introduction This project delivers collectd 4.8.x into OpenSolaris collectd is a daemon which collects system performance statistics periodically and provides mechanisms to store the values in a variety of ways, for example in RRD files. For more information about collectd see: http://collectd.org 2. Technical issues 2.1. Key objects /usr/collectd4/lib/collectd From collectd's man page: collectd is a daemon that receives system statistics and makes them available in a number of ways. The main daemon itself doesn't have any real functionality apart from loading, querying and submitting to plugins. /etc/collectd4/collectd.conf The default collectd configuration file. Do note that this project won't deliver collectdmon. This wrapper will not be required since SMF will take care of starting and monitoring the collectd daemon. 2.2. Versioning From collectd's FAQ: The version numbers in collectd consist of three numbers: The major- and minor-number and the patchlevel. * Versions with different major-numbers are basically not compatible. This means that the definitions of RRD-files or config-options have been changed or, in general, that the user has to do something in addition to install the new version. This is not nice and avoided when possible, but sometimes necessary to prevent old mistakes to become ancient mistakes. * Versions with differing minor-numbers are backwards compatible, i. e. you can replace the lower version with the higher one and everything should still work. This means that features are added, but not removed or changed and that the default behavior does not change. * Versions with different patchlevels are both, forward- and backwards-compatible, because no new features have been introduced. The only difference between the two versions is one or more bug fixes, so you should generally install the higher version of the two. 2.3 Loadable Plugins collectd ships with a number of loadable plugins which are enabled through directives in the config file. Some of these plugins have library dependencies that must be satisfied through build time package options (supplied to the configure script). An example is mysql.so which has a dependency on the mysql client libraries. Support for loadable plugins and other features is enabled at build time through flags to the configure script. All the plugins that are delivered have sample configuration directives in collectd.conf. Note that in many cases, these are just sample directives and will need to be modified based on site specific configuration. Detailed documentation about specific plugin configuration can be found in the collectd website. Not all the plugins are enabled by default in collectd.conf. The configuration directives for plugins which are not enabled are commented out in collectd.conf. The plugins which are enabled by default are: syslog, cpu, interface, load, memory and rrdtool Other plugins can be enabled in one of the following two ways: * By adding/uncommenting the plugin configuration directives in collectd.conf * By dropping a .conf file having the necessary plugin configuration directives into the /etc/collectd4/conf.d Similarly, plugins can be disabled as needed in the following two ways: * By removing/commenting out the plugin configuration directives from collectd.conf * By deleting the plugin specific .conf file from the /etc/collectd4/conf.d directory Do note that collectd.conf is read and processed in order, i. e. from top to bottom. So the plugins are loaded in the order listed in collectd.conf. This project will deliver the following plugins: * Apache * CPU * CSV * cURL * DF * Disk * Exec * FileCount * GenericJMX * Interface * Java * Load * LogFile * memcachec * memcached * Memory * MySQL * Network * Notify Desktop * NTPd * PostgreSQL * RRDtool * SNMP * Swap * SysLog * Table * Tail * Tape * UnixSock * Uptime * Users * UUID * Write HTTP * ZFS ARC 2.4 Directory Naming and Structure The proposed directory layout for collectd is: /usr/collectd4 /lib /man /share /etc/collectd4 /conf.d /var/collectd4 /rrd /logs /run The detailed directory and file layout for collectd is provided in Appendix 1. Multiple versions can coexist, and are distinguished by the naming scheme (collectd4 for version 4.*.*, collectd5 for version 5.*.*). 2.5 Configuration File This project will use the standard convention and deliver configuration files under /etc/collectd4 The default configuration file provided with this integration will work out of the box. In addition, a directory conf.d will be created in /etc/collectd4/ The default configuration loads any file with a .conf extension that is placed in this directory. Hence, plugins can optionally be configured by simply dropping a .conf file into this directory without modifying the main configuration file. By default, conf.d will not contain any plugin configuration files. 2.6 Log File The log File will be located in /var/collectd4/logs and will be named collectd.log. The default log level is info and can be changed by modifying the main configuration file. 2.7 Build features Build time package selection determines which collectd loadable plugins are available to the user at runtime. A complete list of collectd modules is available on the http://collectd.org. However, not all plugins can be built on OpenSolaris because of dependencies on external libraries and hence, only a subset of the plugins will be delivered. 3. collectd documentation collectd 4.* provides a full and complete documentation set, in UNIX manual format. This project will deliver the complete documentation set. 4. Packaging and Delivery We propose to package collectd under the following packages: SUNWcollectd4u - [usr] Server package (including collectd daemon, loadable plugins, man pages) SUNWcollectd4r - [root] (config files, smf files) 5. collectd Interfaces 5.1. Interface Stability collectd has no obvious history of any interface instability and the expectation is that dot releases of collectd will be compatible. This is not completely guaranteed by the community and it is possible that interfaces may change due to the fixing of bugs. 5.2. Imported Interfaces libcurl Volatile PSARC/2007/165 Output format of Apache's mod_status N/A PSARC/2007/586 Output format of Lighhtpd's mod_status N/A LSARC/2008/191 libmysqlclient Committed LSARC 2009/062 libnetsnmp Uncommitted LSARC/2008/355 librrd Uncommitted LSARC/2008/129 libpq Uncommitted LSARC/2009/349 Java SE 6 Committed PSARC/2006/430 libkstat Uncommitted PSARC/1992/085 kstat_read Stable PSARC/1992/085 getmntent Evolving PSARC/1999/131 getloadavg Standard PSARC/1996/196 libmemcached Uncommitted LSARC/2008/126 libgcrypt Volatile LSARC/2008/354 libnotify Volatile LSARC/2007/137 swapctl Stable PSARC/1997/099 libhal Volatile PSARC/2005/399 getutxent N/A N/A syslog Standard N/A statvfs Stable N/A 5.3. Exported Interfaces NAME STABILITY NOTES ----------------------------------------------------------------------- SUNWcollectd4r Committed package SUNWcollectd4u Committed package collectd config file Uncommitted config file syntax collectd log file Not An Interface log file syntax collectd pid file Project Private daemon pid file /var/collectd4/logs Uncommitted log file dir /etc/collectd4/collectd.conf Uncommitted default conf file /usr/collectd4/lib/*.so Project Private plugins /lib/svc/method/collectd4-svc Project Private SMF svc method svc:/application/monitoring/collectd4:default Committed FMRI /var/svc/manifest/application/monitoring/collectd4.xml Project Private SMF Manifest 6. References [1] http://collectd.org ================================================================ Appendix 1: collectd Integration Directory and File Structure. 1. The following files are included in the collectd integration: /usr/collectd4 /share /man /man1 /collectd.1 /man5 /collectd.conf.5 /collectd-exec.conf /collectd-unixsock.5 /collectd-java.5 /collectd-snmp.5 /types.db.5 /collectd /types.db /postgresql_default.conf /java /java-plugin.jar /jmx.jar /var/collectd4 /logs /collectd.log /rrd /etc/collectd4 /conf.d /collectd.conf /usr/collectd4 /lib /collectd /apache.so /bind.so /cpu.so /csv.so /curl.so /df.so /disk.so /exec.so /filecount.so /interface.so /java.so /load.so /logfile.so /match_regex.so /match_timediff.so /match_value.so /memcached.so /memory.so /mysql.so /network.so /ntpd.so /postgresql.so /rrdtool.so /snmp.so /syslog.so /swap.so /table.so /tail.so /tape.so /target_notification.so /target_replace.so /target_set.so /ted.so /types.db /unixsock.so /uptime.so /users.so /uuid.so /libcollectdclient.so.0.0.0 /libcollectdclient.so.0 -> libcollectdclient.so.0.0.0 /libcollectdclient.so -> libcollectdclient.so.0.0.0 /lib/svc/method/collectd4-svc /var/svc/manifest/application/monitoring/collectd4.xml 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: sfw 6.5. ARC review type: FastTrack 6.6. ARC Exposure: open From Darren.Moffat@sun.com Fri Nov 6 01:43:54 2009 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nA69hrLw018857 for ; Fri, 6 Nov 2009 01:43:53 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id nA69hoT0024068 for <@sunmail2sca.sfbay.sun.com:LSARC-ext@sun.com>; Fri, 6 Nov 2009 01:43:53 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KSO00M09LP42B00@brm-avmta-1.central.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Fri, 06 Nov 2009 02:43:52 -0700 (MST) Received: from gmp-eb-inf-2.sun.com ([192.18.6.24]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KSO00KBXLP26P10@brm-avmta-1.central.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Fri, 06 Nov 2009 02:43:51 -0700 (MST) Received: from fe-emea-09.sun.com (gmp-eb-lb-1-fe1.eu.sun.com [192.18.6.7] (may be forged)) by gmp-eb-inf-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id nA69hna3017803 for ; Fri, 06 Nov 2009 09:43:50 +0000 (GMT) Received: from conversion-daemon.fe-emea-09.sun.com by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KSO00C00KNW9H00@fe-emea-09.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Fri, 06 Nov 2009 09:43:42 +0000 (GMT) Received: from [192.168.1.105] (cpc2-rdng20-2-0-cust917.15-3.cable.virginmedia.com [86.28.167.150]) by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KSO00G27LO94YC0@fe-emea-09.sun.com>; Fri, 06 Nov 2009 09:43:26 +0000 (GMT) Date: Fri, 06 Nov 2009 09:43:21 +0000 From: Darren J Moffat Subject: Re: Deliver collectd 4.8.x into OpenSolaris [LSARC/2009/606 FastTrack timeout 11/13/2009] In-reply-to: <200911060916.nA69Gaq2018611@sac.sfbay.sun.com> Sender: Darren.Moffat@sun.com To: Jyri Virkki Cc: LSARC-ext@sun.com, webstack-discuss@opensolaris.org Message-id: <4AF3EFB9.1060403@Sun.COM> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200911060916.nA69Gaq2018611@sac.sfbay.sun.com> User-Agent: Thunderbird 2.0.0.23 (X11/20090929) Status: RO Content-Length: 3537 Jyri Virkki wrote: > 2. Technical issues > > 2.1. Key objects > > /usr/collectd4/lib/collectd > From collectd's man page: > collectd is a daemon that receives system statistics and makes > them available in a number of ways. The main daemon itself > doesn't have any real functionality apart from loading, > querying and submitting to plugins. Is collectd really that huge and important that it deserves its own place under /usr ? Do Linux distros do this (I doubt it because the FHS and LSB both very much frown on the new dirs in /usr). Can this be: /usr/lib/collectd4 instead ? > /etc/collectd4/collectd.conf > The default collectd configuration file. That seems a reasonable place. > Do note that this project won't deliver collectdmon. This > wrapper will not be required since SMF will take care of > starting and monitoring the collectd daemon. Seems reasonable but is it worth delivering it anyway incase someone needs to start it outside of SMF for some reason ? > * ZFS ARC What ZFS interfaces are consumed to gather this information ? > 2.4 Directory Naming and Structure > > The proposed directory layout for collectd is: > > /usr/collectd4 > /lib > /man > /share Hidding the man pages away outside of /usr/share/man isn't good, man pages should always be in /usr/share/man unless there are multiple incompatible versions of something installed at the same time. Do we know we will have collectd4 and collectd5 both installed ? If not I highly recommend the man pages go in /usr/share/man, the shared stuff for collectd would go in /usr/share/collectd4 and the lib stuff in /usr/lib/collectd4 > 2.6 Log File > > The log File will be located in /var/collectd4/logs and will > be named collectd.log. The default log level is info and can > be changed by modifying the main configuration file. What uid/gid owns the logs directory ? > 4. Packaging and Delivery > > We propose to package collectd under the following packages: > > SUNWcollectd4u - [usr] Server package (including collectd > daemon, loadable plugins, man pages) > SUNWcollectd4r - [root] (config files, smf files) As I've said in other cases lets stop pretending we need split usr and root packages unless the project is going to be delivered in Solaris 10 as well as OpenSolaris. > 5.3. Exported Interfaces > > NAME STABILITY NOTES > ----------------------------------------------------------------------- > SUNWcollectd4r Committed package > SUNWcollectd4u Committed package These need to be Volatile as they are going to change before Solaris Next delivers and even on OpenSolaris 2010.03 there will be a single SUNWcollectd4 package. > /lib/svc/method/collectd4-svc Project Private SMF svc method > svc:/application/monitoring/collectd4:default > Committed FMRI > /var/svc/manifest/application/monitoring/collectd4.xml > Project Private SMF Manifest Glad to see a case that listed all these out correctly. -- Darren J Moffat From Jyri.Virkki@sun.com Fri Nov 6 01:55:26 2009 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nA69tPIp019147 for ; Fri, 6 Nov 2009 01:55:26 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id nA69tMo0028240 for <@sunmail2sca.sfbay.sun.com:LSARC-ext@sun.com>; Fri, 6 Nov 2009 09:55:25 GMT Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KSO00H05M8CW600@nwk-avmta-2.sfbay.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Fri, 06 Nov 2009 01:55:24 -0800 (PST) Received: from brmea-mail-2.sun.com ([192.18.98.43]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KSO0094JM8CTJB0@nwk-avmta-2.sfbay.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Fri, 06 Nov 2009 01:55:24 -0800 (PST) Received: from dm-usca15-11.red.iplanet.com (host-185-56-18-192.iplanet.com [192.18.56.185] (may be forged)) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id nA69sl58025338; Fri, 06 Nov 2009 09:54:48 +0000 (GMT) Received: from buye.red.iplanet.com (buye [192.18.65.224]) by dm-usca15-11.red.iplanet.com (8.11.7p1+Sun/8.11.7/IPLANET,v1.2) with ESMTP id nA69skV18384; Fri, 06 Nov 2009 01:54:46 -0800 (PST) Received: from buye.red.iplanet.com (localhost [127.0.0.1]) by buye.red.iplanet.com (8.13.7+Sun/8.13.7) with ESMTP id nA69slHr025181; Fri, 06 Nov 2009 01:54:47 -0800 (PST) Received: (from jyri@localhost) by buye.red.iplanet.com (8.13.7+Sun/8.13.7/Submit) id nA69slSp025180; Fri, 06 Nov 2009 01:54:47 -0800 (PST) Date: Fri, 06 Nov 2009 01:54:47 -0800 From: Jyri Virkki Subject: Re: Deliver collectd 4.8.x into OpenSolaris [LSARC/2009/606 FastTrack timeout 11/13/2009] In-reply-to: <200911060916.nA69Gaq2018611@sac.sfbay.sun.com> To: LSARC-ext@sun.com Cc: webstack-discuss@opensolaris.org Message-id: <20091106095447.GA23503@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <200911060916.nA69Gaq2018611@sac.sfbay.sun.com> User-Agent: Mutt/1.5.11 Status: RO Content-Length: 1748 > Deliver collectd 4.8.x into OpenSolaris I have one issue from the pre-review of this case which seems worth discussing in the ARC forum, namely the versioned directory layout. There are many components in Solaris already with versioned file layouts but AFAIK there is no consensus on the details. It would be useful to have a precedent-setting case on that topic to avoid the differing solutions from project to project. Existing precedents vary. A few examples: Most popular pattern is /usr/$COMPONENT/$VERSION/... /usr/php/$version/* /usr/mysql/$version/* /usr/ruby/$version/* /usr/lighttpd/$version/* Some use /usr/$COMPONENT$VERSION/... /usr/tomcat6/* And some have both.. (historical reasons in both cases, IIRC) /usr/apache2/$version/* /usr/perl5/$version/* As currently specified, this case (collectd) is proposing the second variant, /usr/$COMPONENT$VERSION/ so /usr/collectd4/* If we settle on the first variant, it should be /usr/collectd/4/* As noted in s.2.2, collectd upstream considers their releases within a major release family to be compatible, so only different major releases need to coexist. Other projects, such as apache httpd, are compatible only within minor release family thus they specify major.minor as the $VERSION in /usr/$COMPONENT/$VERSION/ At the other extreme is perl, which is using major.minor.micro as the $VERSION (which seems excessive, but perhaps it was needed) % ls -l /usr/perl5 total 16 drwxr-xr-x 5 root bin 512 Jun 18 2007 5.6.1 drwxr-xr-x 3 root bin 512 Jun 18 2007 5.8.3 drwxr-xr-x 5 root bin 512 Jun 18 2007 5.8.4 -- Jyri J. Virkki - jyri.virkki@sun.com - Sun Microsystems From Amit.Gupta@Sun.COM Fri Nov 6 09:22:16 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nA6HME6d026083 for ; Fri, 6 Nov 2009 09:22:14 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id nA6HLds6007135 for <@sunmail2sca.sfbay.sun.com:LSARC-ext@sun.com>; Sat, 7 Nov 2009 01:22:13 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KSP000176X04K00@nwk-avmta-2.sfbay.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Fri, 06 Nov 2009 09:22:12 -0800 (PST) Received: from gmp-eb-inf-1.sun.com ([192.18.6.21]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KSP00FUV6WYXJA0@nwk-avmta-2.sfbay.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Fri, 06 Nov 2009 09:22:11 -0800 (PST) Received: from fe-emea-10.sun.com (gmp-eb-lb-1-fe1.eu.sun.com [192.18.6.7] (may be forged)) by gmp-eb-inf-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id nA6HMAKc010954 for ; Fri, 06 Nov 2009 17:22:10 +0000 (GMT) Received: from conversion-daemon.fe-emea-10.sun.com by fe-emea-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KSP0000069LIP00@fe-emea-10.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Fri, 06 Nov 2009 17:22:09 +0000 (GMT) Received: from test.sun.com ([unknown] [129.158.228.93]) by fe-emea-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KSP00EF36WTG320@fe-emea-10.sun.com>; Fri, 06 Nov 2009 17:22:09 +0000 (GMT) Date: Fri, 06 Nov 2009 22:39:19 +0530 From: Amit Gupta Subject: Re: [webstack-discuss] Deliver collectd 4.8.x into OpenSolaris [LSARC/2009/606 FastTrack timeout 11/13/2009] In-reply-to: <4AF3EFB9.1060403@Sun.COM> Sender: Amit.Gupta@Sun.COM To: Darren J Moffat Cc: Jyri Virkki , webstack-discuss@opensolaris.org, LSARC-ext@Sun.COM Reply-to: Amit.Gupta@Sun.COM Message-id: <4AF4583F.8000105@Sun.COM> MIME-version: 1.0 Content-type: multipart/alternative; boundary="Boundary_(ID_kKEDXBdvw7HLt38yDoGwnA)" X-PMX-Version: 5.4.1.325704 References: <200911060916.nA69Gaq2018611@sac.sfbay.sun.com> <4AF3EFB9.1060403@Sun.COM> User-Agent: Thunderbird 2.0.0.4 (X11/20070622) Status: RO Content-Length: 11615 This is a multi-part message in MIME format. --Boundary_(ID_kKEDXBdvw7HLt38yDoGwnA) Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT On 11/06/09 15:13, Darren J Moffat wrote: > Jyri Virkki wrote: >> 2. Technical issues >> >> 2.1. Key objects >> >> /usr/collectd4/lib/collectd >> From collectd's man page: >> collectd is a daemon that receives system statistics and makes >> them available in a number of ways. The main daemon itself >> doesn't have any real functionality apart from loading, >> querying and submitting to plugins. > > Is collectd really that huge and important that it deserves its own > place under /usr ? Do Linux distros do this (I doubt it because the > FHS and LSB both very much frown on the new dirs in /usr). > Can this be: > > /usr/lib/collectd4 instead ? I have followed lighttpd, apache, mysql, tomcat, php, etc. all of which install into separate directories under /usr. It does seem to make sense to move it under /usr/lib as /usr/lib/collectd4. >> Do note that this project won't deliver collectdmon. This >> wrapper will not be required since SMF will take care of >> starting and monitoring the collectd daemon. > > Seems reasonable but is it worth delivering it anyway incase someone > needs to start it outside of SMF for some reason ? Yes, it is possible for other projects to embed collectd and mange its lifecycle outside of SMF in which case collectdmon will be useful. I will modify ARC case accordingly. >> * ZFS ARC > > What ZFS interfaces are consumed to gather this information ? As per plugin documentation (http://collectd.org/wiki/index.php/Plugin:ZFS_ARC), *ZFS ARC plugin* collects information about the /Adaptive Replacement Cache/ (ARC) of the /Zeta File-System/ (ZFS). It uses kstat for the same. >> 2.4 Directory Naming and Structure >> >> The proposed directory layout for collectd is: >> >> /usr/collectd4 >> /lib >> /man >> /share > > Hidding the man pages away outside of /usr/share/man isn't good, man > pages should always be in /usr/share/man unless there are multiple > incompatible versions of something installed at the same time. > > Do we know we will have collectd4 and collectd5 both installed ? It is possible that collectd4 and collectd5 will both be installed at the same time. > If not I highly recommend the man pages go in /usr/share/man, the > shared stuff for collectd would go in /usr/share/collectd4 and the lib > stuff in /usr/lib/collectd4 Given that collectd's major versions are incompatible, does it make sense to ship man pages under /usr/share/collectd4/man? >> 2.6 Log File >> >> The log File will be located in /var/collectd4/logs and will >> be named collectd.log. The default log level is info and can >> be changed by modifying the main configuration file. > > What uid/gid owns the logs directory ? root:bin > >> 4. Packaging and Delivery >> >> We propose to package collectd under the following packages: >> >> SUNWcollectd4u - [usr] Server package (including collectd daemon, >> loadable plugins, man pages) >> SUNWcollectd4r - [root] (config files, smf files) > > As I've said in other cases lets stop pretending we need split usr and > root packages unless the project is going to be delivered in Solaris > 10 as well as OpenSolaris. Again, we have followed the lead of apache, lighttpd. mysql etc. all of which deliver a separate root and usr package. > >> 5.3. Exported Interfaces >> >> NAME STABILITY NOTES >> ----------------------------------------------------------------------- >> SUNWcollectd4r Committed package >> SUNWcollectd4u Committed package > > These need to be Volatile as they are going to change before Solaris > Next delivers and even on OpenSolaris 2010.03 there will be a single > SUNWcollectd4 package. Will do. Regards Amit --Boundary_(ID_kKEDXBdvw7HLt38yDoGwnA) Content-type: text/html; CHARSET=US-ASCII Content-transfer-encoding: 7BIT On 11/06/09 15:13, Darren J Moffat wrote:
Jyri Virkki wrote:
2.      Technical issues

        2.1. Key objects

        /usr/collectd4/lib/collectd
            From collectd's man page:
            collectd is a daemon that receives system statistics and makes
            them available in a number of ways. The main daemon itself
            doesn't have any real functionality apart from loading,
            querying and submitting to plugins.

Is collectd really that huge and important that it deserves its own place under /usr ?  Do Linux distros do this (I doubt it because the FHS and LSB both very much frown on the new dirs in /usr).
Can this be:

    /usr/lib/collectd4 instead ?
I have followed lighttpd, apache, mysql, tomcat, php, etc. all of which install into separate directories under /usr.  It does seem to make sense to move it under /usr/lib as /usr/lib/collectd4.
        Do note that this project won't deliver collectdmon. This
        wrapper will not be required since SMF will take care of
        starting and monitoring the collectd daemon.

Seems reasonable but is it worth delivering it anyway incase someone needs to start it outside of SMF for some reason ?
Yes, it is possible for other projects to embed collectd and mange its lifecycle outside of SMF in which case collectdmon will be useful. I will modify ARC case accordingly.
        * ZFS ARC

What ZFS interfaces are consumed to gather this information ?
As per plugin documentation (http://collectd.org/wiki/index.php/Plugin:ZFS_ARC), ZFS ARC plugin collects information about the Adaptive Replacement Cache (ARC) of the Zeta File-System (ZFS). It uses kstat for the same.

        2.4 Directory Naming and Structure

        The proposed directory layout for collectd is:

        /usr/collectd4
                         /lib
                         /man
                         /share

Hidding the man pages away outside of /usr/share/man isn't good, man pages should always be in /usr/share/man unless there are multiple incompatible versions of something installed at the same time.

Do we know we will have collectd4 and collectd5 both installed ?
It is possible that collectd4 and collectd5 will both be installed at the same time.
  If not I highly recommend the man pages go in /usr/share/man, the shared stuff for collectd would go in /usr/share/collectd4 and the lib stuff in /usr/lib/collectd4
Given that collectd's major versions are incompatible, does it make sense to ship man pages under /usr/share/collectd4/man?
        2.6 Log File

        The log File will be located in /var/collectd4/logs and will
        be named collectd.log. The default log level is info and can
        be changed by modifying the main configuration file.

What uid/gid owns the logs directory ?
root:bin

4.      Packaging and Delivery

        We propose to package collectd under the following packages:

        SUNWcollectd4u - [usr] Server package (including collectd                         daemon, loadable plugins, man pages)
        SUNWcollectd4r - [root] (config files, smf files)

As I've said in other cases lets stop pretending we need split usr and root packages unless the project is going to be delivered in Solaris 10 as well as OpenSolaris.
Again, we have followed the lead of apache, lighttpd. mysql etc. all of which deliver a separate root and usr package.

        5.3. Exported Interfaces

        NAME                                STABILITY        NOTES
        -----------------------------------------------------------------------
        SUNWcollectd4r                      Committed        package
        SUNWcollectd4u                      Committed        package

These need to be Volatile as they are going to change before Solaris Next delivers and even on OpenSolaris 2010.03 there will be a single SUNWcollectd4 package.
Will do.

Regards
Amit
--Boundary_(ID_kKEDXBdvw7HLt38yDoGwnA)-- From Amit.Gupta@sun.com Fri Nov 6 09:35:21 2009 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nA6HZLUH026528 for ; Fri, 6 Nov 2009 09:35:21 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id nA6HZKe1020156 for <@sunmail2sca.sfbay.sun.com:LSARC-ext@sun.com>; Fri, 6 Nov 2009 09:35:20 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KSP0010P7IVC100@nwk-avmta-2.sfbay.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Fri, 06 Nov 2009 09:35:19 -0800 (PST) Received: from gmp-eb-inf-1.sun.com ([192.18.6.21]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KSP00FRN7IUXLA0@nwk-avmta-2.sfbay.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Fri, 06 Nov 2009 09:35:18 -0800 (PST) Received: from fe-emea-09.sun.com (gmp-eb-lb-1-fe1.eu.sun.com [192.18.6.7] (may be forged)) by gmp-eb-inf-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id nA6HZH56011682 for ; Fri, 06 Nov 2009 17:35:17 +0000 (GMT) Received: from conversion-daemon.fe-emea-09.sun.com by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KSP006007F9V600@fe-emea-09.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Fri, 06 Nov 2009 17:35:08 +0000 (GMT) Received: from test.sun.com ([unknown] [129.158.228.93]) by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KSP006EL7IHPO40@fe-emea-09.sun.com>; Fri, 06 Nov 2009 17:35:08 +0000 (GMT) Date: Fri, 06 Nov 2009 22:52:19 +0530 From: Amit Gupta Subject: Re: [webstack-discuss] Deliver collectd 4.8.x into OpenSolaris [LSARC/2009/606 FastTrack timeout 11/13/2009] In-reply-to: <4AF3EFB9.1060403@Sun.COM> Sender: Amit.Gupta@sun.com To: Darren J Moffat Cc: Jyri Virkki , webstack-discuss@opensolaris.org, LSARC-ext@sun.com Reply-to: Amit.Gupta@sun.com Message-id: <4AF45B4B.1090508@Sun.COM> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200911060916.nA69Gaq2018611@sac.sfbay.sun.com> <4AF3EFB9.1060403@Sun.COM> User-Agent: Thunderbird 2.0.0.4 (X11/20070622) Status: RO Content-Length: 517 On 11/06/09 15:13, Darren J Moffat wrote: > Is collectd really that huge and important that it deserves its own > place under /usr ? Do Linux distros do this (I doubt it because the > FHS and LSB both very much frown on the new dirs in /usr). > > Can this be: > > /usr/lib/collectd4 instead ? > > BTW, I don't seem to find any component under /usr/lib which allows multiple version to co-exist. Since multiple versions of collectd can co-exist, is that a good reason for it to go under /usr? Regards Amit From Darren.Moffat@Sun.COM Tue Nov 10 08:46:21 2009 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nAAGkLjQ015923 for ; Tue, 10 Nov 2009 08:46:21 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id nAAGkKqJ024047 for <@sunmail2sca.sfbay.sun.com:LSARC-ext@sun.com>; Tue, 10 Nov 2009 08:46:21 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KSW00J0JJX9HZ00@nwk-avmta-2.sfbay.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Tue, 10 Nov 2009 08:46:21 -0800 (PST) Received: from gmp-eb-inf-1.sun.com ([192.18.6.21]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KSW00JQRJX70R00@nwk-avmta-2.sfbay.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Tue, 10 Nov 2009 08:46:19 -0800 (PST) Received: from fe-emea-09.sun.com (gmp-eb-lb-1-fe1.eu.sun.com [192.18.6.7] (may be forged)) by gmp-eb-inf-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id nAAGkIXW020238 for ; Tue, 10 Nov 2009 16:46:18 +0000 (GMT) Received: from conversion-daemon.fe-emea-09.sun.com by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KSW00G00ITUM600@fe-emea-09.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Tue, 10 Nov 2009 16:46:12 +0000 (GMT) Received: from [192.168.2.108] (99-52-200-208.lightspeed.snjsca.sbcglobal.net [99.52.200.208]) by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KSW009WGJWSSYE0@fe-emea-09.sun.com>; Tue, 10 Nov 2009 16:46:09 +0000 (GMT) Date: Tue, 10 Nov 2009 16:46:03 +0000 From: Darren J Moffat Subject: Re: [webstack-discuss] Deliver collectd 4.8.x into OpenSolaris [LSARC/2009/606 FastTrack timeout 11/13/2009] In-reply-to: <4AF45B4B.1090508@Sun.COM> Sender: Darren.Moffat@Sun.COM To: Amit.Gupta@Sun.COM Cc: Jyri Virkki , webstack-discuss@opensolaris.org, LSARC-ext@Sun.COM Message-id: <4AF998CB.4060005@Sun.COM> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200911060916.nA69Gaq2018611@sac.sfbay.sun.com> <4AF3EFB9.1060403@Sun.COM> <4AF45B4B.1090508@Sun.COM> User-Agent: Thunderbird 2.0.0.23 (X11/20091013) Status: RO Content-Length: 1256 Amit Gupta wrote: > On 11/06/09 15:13, Darren J Moffat wrote: >> Is collectd really that huge and important that it deserves its own >> place under /usr ? Do Linux distros do this (I doubt it because the >> FHS and LSB both very much frown on the new dirs in /usr). >> >> Can this be: >> >> /usr/lib/collectd4 instead ? >> >> > BTW, I don't seem to find any component under /usr/lib which allows > multiple version to co-exist. Since multiple versions of collectd can > co-exist, is that a good reason for it to go under /usr? Just because there could be multiple versions doesn't mean there should be. Usually the need to deliver multiple versions of a feature is because of incompatible change that impacts the things layered on top of them. This makes sense for PHP, Apache, Python, Perl because they are environments that have runtime impact and there are massive amounts of software layered above them. I don't see collectd really fitting into that same model so I don't think it is necessary to support multiple versions of it installed at the same time given what it is used for. How doe BSD or Linux distributions install it ? I doubt you will find many Linux distros installing it directly under /usr. -- Darren J Moffat From Jyri.Virkki@sun.com Tue Nov 10 10:01:39 2009 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nAAI1dxU018186 for ; Tue, 10 Nov 2009 10:01:39 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id nAAI1Y3U002136 for <@sunmail2sca.sfbay.sun.com:LSARC-ext@sun.com>; Tue, 10 Nov 2009 10:01:39 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KSW00611NEQBB00@brm-avmta-1.central.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Tue, 10 Nov 2009 11:01:38 -0700 (MST) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KSW0030ENENZD30@brm-avmta-1.central.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Tue, 10 Nov 2009 11:01:36 -0700 (MST) Received: from dm-usca19-13.red.iplanet.com (host-179-56-18-192.iplanet.com [192.18.56.179] (may be forged)) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id nAAI1ZKe027685 for ; Tue, 10 Nov 2009 18:01:35 +0000 (GMT) Received: from buye.red.iplanet.com (buye [192.18.65.224]) by dm-usca19-13.red.iplanet.com (8.11.7p1+Sun/8.11.7/IPLANET,v1.2) with ESMTP id nAAI1gJ24662 for ; Tue, 10 Nov 2009 10:01:42 -0800 (PST) Received: from buye.red.iplanet.com (localhost [127.0.0.1]) by buye.red.iplanet.com (8.13.7+Sun/8.13.7) with ESMTP id nAAI1ZFP024070 for ; Tue, 10 Nov 2009 10:01:35 -0800 (PST) Received: (from jyri@localhost) by buye.red.iplanet.com (8.13.7+Sun/8.13.7/Submit) id nAAI1ZnW024069 for LSARC-ext@sun.com; Tue, 10 Nov 2009 10:01:35 -0800 (PST) Date: Tue, 10 Nov 2009 10:01:35 -0800 From: Jyri Virkki Subject: Re: Deliver collectd 4.8.x into OpenSolaris [LSARC/2009/606 FastTrack timeout 11/13/2009] In-reply-to: <200911060916.nA69Gaq2018611@sac.sfbay.sun.com> To: LSARC-ext@sun.com Message-id: <20091110180134.GA23915@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <200911060916.nA69Gaq2018611@sac.sfbay.sun.com> User-Agent: Mutt/1.5.11 Status: RO Content-Length: 403 > > I am sponsoring this case for Amit Gupta. It introduces collectd into > OpenSolaris. Timeout set to Nov.13. Had an urgent conflicting meeting show up this morning so won't be able to call into LSARC. If possible please discuss the versioning layout issues noted in this case earlier. I'll listen to the recording and catch up later. -- Jyri J. Virkki - jyri.virkki@sun.com - Sun Microsystems From Amit.Gupta@sun.com Wed Nov 11 08:42:25 2009 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nABGgNlr027168 for ; Wed, 11 Nov 2009 08:42:23 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id nABGgMUs014435 for <@sunmail2sca.sfbay.sun.com:LSARC-ext@sun.com>; Wed, 11 Nov 2009 09:42:23 -0700 (MST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KSY00C0PEELNY00@nwk-avmta-1.sfbay.Sun.COM> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Wed, 11 Nov 2009 08:42:21 -0800 (PST) Received: from gmp-eb-inf-1.sun.com ([192.18.6.21]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KSY006B8EEKWPA0@nwk-avmta-1.sfbay.Sun.COM> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Wed, 11 Nov 2009 08:42:21 -0800 (PST) Received: from fe-emea-09.sun.com (gmp-eb-lb-1-fe1.eu.sun.com [192.18.6.7] (may be forged)) by gmp-eb-inf-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id nABGgJmO012468 for ; Wed, 11 Nov 2009 16:42:19 +0000 (GMT) Received: from conversion-daemon.fe-emea-09.sun.com by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KSY00700C0ASV00@fe-emea-09.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Wed, 11 Nov 2009 16:42:18 +0000 (GMT) Received: from test.sun.com ([unknown] [129.158.228.93]) by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KSY00J1TEEA5D50@fe-emea-09.sun.com>; Wed, 11 Nov 2009 16:42:14 +0000 (GMT) Date: Wed, 11 Nov 2009 21:59:14 +0530 From: Amit Gupta Subject: Re: [webstack-discuss] Deliver collectd 4.8.x into OpenSolaris [LSARC/2009/606 FastTrack timeout 11/13/2009] In-reply-to: <4AF4583F.8000105@Sun.COM> Sender: Amit.Gupta@sun.com To: Amit.Gupta@sun.com Cc: Darren J Moffat , webstack-discuss@opensolaris.org, LSARC-ext@sun.com, Jyri Virkki Reply-to: Amit.Gupta@sun.com Message-id: <4AFAE65A.7020105@Sun.COM> MIME-version: 1.0 Content-type: multipart/mixed; boundary="Boundary_(ID_eLx2XaSwbQzWnZThvufvKQ)" X-PMX-Version: 5.4.1.325704 References: <200911060916.nA69Gaq2018611@sac.sfbay.sun.com> <4AF3EFB9.1060403@Sun.COM> <4AF4583F.8000105@Sun.COM> User-Agent: Thunderbird 2.0.0.4 (X11/20070622) Status: RO Content-Length: 28190 This is a multi-part message in MIME format. --Boundary_(ID_eLx2XaSwbQzWnZThvufvKQ) Content-type: multipart/alternative; boundary="Boundary_(ID_85zWsyqRieRDhE+d4jMAig)" --Boundary_(ID_85zWsyqRieRDhE+d4jMAig) Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT Do find the updated ARC case attached. The summary of the changes is as follows: - updated the file layout as per Darren's suggestion. - included collectdmon as suggested. - added include files in the file layout which I had missed out. I am not sure as yet if we need to deliver only one collectd package i.e SUNWcollectd?. As I mentioned earlier, I have followed the lead of apache, lighttpd. mysql etc. all of which deliver a separate root and usr package. Regards Amit On 11/06/09 22:39, Amit Gupta wrote: > On 11/06/09 15:13, Darren J Moffat wrote: >> Jyri Virkki wrote: >>> 2. Technical issues >>> >>> 2.1. Key objects >>> >>> /usr/collectd4/lib/collectd >>> From collectd's man page: >>> collectd is a daemon that receives system statistics and >>> makes >>> them available in a number of ways. The main daemon itself >>> doesn't have any real functionality apart from loading, >>> querying and submitting to plugins. >> >> Is collectd really that huge and important that it deserves its own >> place under /usr ? Do Linux distros do this (I doubt it because the >> FHS and LSB both very much frown on the new dirs in /usr). >> Can this be: >> >> /usr/lib/collectd4 instead ? > I have followed lighttpd, apache, mysql, tomcat, php, etc. all of > which install into separate directories under /usr. It does seem to > make sense to move it under /usr/lib as /usr/lib/collectd4. >>> Do note that this project won't deliver collectdmon. This >>> wrapper will not be required since SMF will take care of >>> starting and monitoring the collectd daemon. >> >> Seems reasonable but is it worth delivering it anyway incase someone >> needs to start it outside of SMF for some reason ? > Yes, it is possible for other projects to embed collectd and mange its > lifecycle outside of SMF in which case collectdmon will be useful. I > will modify ARC case accordingly. >>> * ZFS ARC >> >> What ZFS interfaces are consumed to gather this information ? > As per plugin documentation > (http://collectd.org/wiki/index.php/Plugin:ZFS_ARC), *ZFS ARC plugin* > collects information about the /Adaptive Replacement Cache/ (ARC) of > the /Zeta File-System/ (ZFS). It uses kstat for the same. > >>> 2.4 Directory Naming and Structure >>> >>> The proposed directory layout for collectd is: >>> >>> /usr/collectd4 >>> /lib >>> /man >>> /share >> >> Hidding the man pages away outside of /usr/share/man isn't good, man >> pages should always be in /usr/share/man unless there are multiple >> incompatible versions of something installed at the same time. >> >> Do we know we will have collectd4 and collectd5 both installed ? > It is possible that collectd4 and collectd5 will both be installed at > the same time. >> If not I highly recommend the man pages go in /usr/share/man, the >> shared stuff for collectd would go in /usr/share/collectd4 and the >> lib stuff in /usr/lib/collectd4 > Given that collectd's major versions are incompatible, does it make > sense to ship man pages under /usr/share/collectd4/man? >>> 2.6 Log File >>> >>> The log File will be located in /var/collectd4/logs and will >>> be named collectd.log. The default log level is info and can >>> be changed by modifying the main configuration file. >> >> What uid/gid owns the logs directory ? > root:bin >> >>> 4. Packaging and Delivery >>> >>> We propose to package collectd under the following packages: >>> >>> SUNWcollectd4u - [usr] Server package (including collectd >>> daemon, loadable plugins, man pages) >>> SUNWcollectd4r - [root] (config files, smf files) >> >> As I've said in other cases lets stop pretending we need split usr >> and root packages unless the project is going to be delivered in >> Solaris 10 as well as OpenSolaris. > Again, we have followed the lead of apache, lighttpd. mysql etc. all > of which deliver a separate root and usr package. >> >>> 5.3. Exported Interfaces >>> >>> NAME STABILITY NOTES >>> >>> ----------------------------------------------------------------------- >>> SUNWcollectd4r Committed package >>> SUNWcollectd4u Committed package >> >> These need to be Volatile as they are going to change before Solaris >> Next delivers and even on OpenSolaris 2010.03 there will be a single >> SUNWcollectd4 package. > Will do. > > Regards > Amit > ------------------------------------------------------------------------ > > _______________________________________________ > > > webstack-discuss mailing list > webstack-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/webstack-discuss > --Boundary_(ID_85zWsyqRieRDhE+d4jMAig) Content-type: text/html; CHARSET=US-ASCII Content-transfer-encoding: 7BIT Do find the updated ARC case attached. The summary of the changes is as follows:

- updated the file layout as per Darren's suggestion.
- included collectdmon as suggested.
- added include files in the file layout which I had missed out.

I am not sure as yet if we need to deliver only one collectd package i.e SUNWcollectd?. As I mentioned earlier, I have followed the lead of apache, lighttpd. mysql etc. all of which deliver a separate root and usr package.

Regards
Amit
On 11/06/09 22:39, Amit Gupta wrote:
On 11/06/09 15:13, Darren J Moffat wrote:
Jyri Virkki wrote:
2.      Technical issues

        2.1. Key objects

        /usr/collectd4/lib/collectd
            From collectd's man page:
            collectd is a daemon that receives system statistics and makes
            them available in a number of ways. The main daemon itself
            doesn't have any real functionality apart from loading,
            querying and submitting to plugins.

Is collectd really that huge and important that it deserves its own place under /usr ?  Do Linux distros do this (I doubt it because the FHS and LSB both very much frown on the new dirs in /usr).
Can this be:

    /usr/lib/collectd4 instead ?
I have followed lighttpd, apache, mysql, tomcat, php, etc. all of which install into separate directories under /usr.  It does seem to make sense to move it under /usr/lib as /usr/lib/collectd4.
        Do note that this project won't deliver collectdmon. This
        wrapper will not be required since SMF will take care of
        starting and monitoring the collectd daemon.

Seems reasonable but is it worth delivering it anyway incase someone needs to start it outside of SMF for some reason ?
Yes, it is possible for other projects to embed collectd and mange its lifecycle outside of SMF in which case collectdmon will be useful. I will modify ARC case accordingly.
        * ZFS ARC

What ZFS interfaces are consumed to gather this information ?
As per plugin documentation (http://collectd.org/wiki/index.php/Plugin:ZFS_ARC), ZFS ARC plugin collects information about the Adaptive Replacement Cache (ARC) of the Zeta File-System (ZFS). It uses kstat for the same.

        2.4 Directory Naming and Structure

        The proposed directory layout for collectd is:

        /usr/collectd4
                         /lib
                         /man
                         /share

Hidding the man pages away outside of /usr/share/man isn't good, man pages should always be in /usr/share/man unless there are multiple incompatible versions of something installed at the same time.

Do we know we will have collectd4 and collectd5 both installed ?
It is possible that collectd4 and collectd5 will both be installed at the same time.
  If not I highly recommend the man pages go in /usr/share/man, the shared stuff for collectd would go in /usr/share/collectd4 and the lib stuff in /usr/lib/collectd4
Given that collectd's major versions are incompatible, does it make sense to ship man pages under /usr/share/collectd4/man?
        2.6 Log File

        The log File will be located in /var/collectd4/logs and will
        be named collectd.log. The default log level is info and can
        be changed by modifying the main configuration file.

What uid/gid owns the logs directory ?
root:bin

4.      Packaging and Delivery

        We propose to package collectd under the following packages:

        SUNWcollectd4u - [usr] Server package (including collectd                         daemon, loadable plugins, man pages)
        SUNWcollectd4r - [root] (config files, smf files)

As I've said in other cases lets stop pretending we need split usr and root packages unless the project is going to be delivered in Solaris 10 as well as OpenSolaris.
Again, we have followed the lead of apache, lighttpd. mysql etc. all of which deliver a separate root and usr package.

        5.3. Exported Interfaces

        NAME                                STABILITY        NOTES
        -----------------------------------------------------------------------
        SUNWcollectd4r                      Committed        package
        SUNWcollectd4u                      Committed        package

These need to be Volatile as they are going to change before Solaris Next delivers and even on OpenSolaris 2010.03 there will be a single SUNWcollectd4 package.
Will do.

Regards
Amit

_______________________________________________


webstack-discuss mailing list
webstack-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/webstack-discuss

--Boundary_(ID_85zWsyqRieRDhE+d4jMAig)-- --Boundary_(ID_eLx2XaSwbQzWnZThvufvKQ) Content-type: text/plain; name=collectd-arc.txt Content-transfer-encoding: 7BIT Content-disposition: inline; filename=collectd-arc.txt Template Version: @(#)sac_nextcase 1.68 02/23/09 SMI This information is Copyright 2009 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: Deliver collectd 4.8.x into OpenSolaris 1.2. Name of Document Author/Supplier: Author: Amit Gupta 1.3 Date of This Document: 06 November, 2009 4. Technical Description Deliver collectd 4.8.x into OpenSolaris 11 November 2009 1. Summary and motivation 1.1. Introduction This project delivers collectd 4.8.x into OpenSolaris collectd is a daemon which collects system performance statistics periodically and provides mechanisms to store the values in a variety of ways, for example in RRD files. For more information about collectd see: http://collectd.org 2. Technical issues 2.1. Key objects /usr/lib/collectd4/collectd From collectd's man page: collectd is a daemon that receives system statistics and makes them available in a number of ways. The main daemon itself doesn't have any real functionality apart from loading, querying and submitting to plugins. /usr/lib/collectd4/collectdmon From collectdmon's man page: collectdmon is a small ``wrapper'' daemon which starts and monitors the collectd daemon. If collectd terminates it will automatically be restarted, unless collectdmon was told to shut it down. /etc/collectd4/collectd.conf The default collectd configuration file. 2.2. Versioning From collectd's FAQ: The version numbers in collectd consist of three numbers: The major- and minor-number and the patchlevel. * Versions with different major-numbers are basically not compatible. This means that the definitions of RRD-files or config-options have been changed or, in general, that the user has to do something in addition to install the new version. This is not nice and avoided when possible, but sometimes necessary to prevent old mistakes to become ancient mistakes. * Versions with differing minor-numbers are backwards compatible, i. e. you can replace the lower version with the higher one and everything should still work. This means that features are added, but not removed or changed and that the default behavior does not change. * Versions with different patchlevels are both, forward- and backwards-compatible, because no new features have been introduced. The only difference between the two versions is one or more bug fixes, so you should generally install the higher version of the two. 2.3 Loadable Plugins collectd ships with a number of loadable plugins which are enabled through directives in the config file. Some of these plugins have library dependencies that must be satisfied through build time package options (supplied to the configure script). An example is mysql.so which has a dependency on the mysql client libraries. Support for loadable plugins and other features is enabled at build time through flags to the configure script. All the plugins that are delivered have sample configuration directives in collectd.conf. Note that in many cases, these are just sample directives and will need to be modified based on site specific configuration. Detailed documentation about specific plugin configuration can be found in the collectd website. Not all the plugins are enabled by default in collectd.conf. The configuration directives for plugins which are not enabled are commented out in collectd.conf. The plugins which are enabled by default are: syslog, cpu, interface, load, memory and rrdtool Other plugins can be enabled in one of the following two ways: * By adding/uncommenting the plugin configuration directives in collectd.conf * By dropping a .conf file having the necessary plugin configuration directives into the /etc/collectd4/conf.d Similarly, plugins can be disabled as needed in the following two ways: * By removing/commenting out the plugin configuration directives from collectd.conf * By deleting the plugin specific .conf file from the /etc/collectd4/conf.d directory Do note that collectd.conf is read and processed in order, i. e. from top to bottom. So the plugins are loaded in the order listed in collectd.conf. This project will deliver the following plugins: * Apache * CPU * CSV * cURL * DF * Disk * Exec * FileCount * GenericJMX * Interface * Java * Load * LogFile * memcachec * memcached * Memory * MySQL * Network * Notify Desktop * NTPd * PostgreSQL * RRDtool * SNMP * Swap * SysLog * Table * Tail * Tape * UnixSock * Uptime * Users * UUID * Write HTTP * ZFS ARC 2.4 Directory Naming and Structure The proposed directory layout for collectd is: /usr/lib/collectd4 /etc/collectd4 /conf.d /var/collectd4 /rrd /logs /usr/share/collectd4 /usr/include/collectd The detailed directory and file layout for collectd is provided in Appendix 1. Multiple versions can coexist, and are distinguished by the naming scheme (collectd4 for version 4.*.*, collectd5 for version 5.*.*). 2.5 Configuration File This project will use the standard convention and deliver configuration files under /etc/collectd4 The default configuration file provided with this integration will work out of the box. In addition, a directory conf.d will be created in /etc/collectd4/ The default configuration loads any file with a .conf extension that is placed in this directory. Hence, plugins can optionally be configured by simply dropping a .conf file into this directory without modifying the main configuration file. By default, conf.d will not contain any plugin configuration files. 2.6 Log File The log File will be located in /var/collectd4/logs and will be named collectd.log. The default log level is info and can be changed by modifying the main configuration file. 2.7 Build features Build time package selection determines which collectd loadable plugins are available to the user at runtime. A complete list of collectd modules is available on the http://collectd.org. However, not all plugins can be built on OpenSolaris because of dependencies on external libraries and hence, only a subset of the plugins will be delivered. 3. collectd documentation collectd 4.* provides a full and complete documentation set, in UNIX manual format. This project will deliver the complete documentation set. 4. Packaging and Delivery We propose to package collectd under the following packages: SUNWcollectd4u - [usr] Server package (including collectd daemon, loadable plugins, man pages) SUNWcollectd4r - [root] (config files, smf files) 5. collectd Interfaces 5.1. Interface Stability collectd has no obvious history of any interface instability and the expectation is that dot releases of collectd will be compatible. This is not completely guaranteed by the community and it is possible that interfaces may change due to the fixing of bugs. 5.2. Imported Interfaces libcurl Volatile PSARC/2007/165 Output format of Apache's mod_status N/A PSARC/2007/586 Output format of Lighhtpd's mod_status N/A LSARC/2008/191 libmysqlclient Committed LSARC 2009/062 libnetsnmp Uncommitted LSARC/2008/355 librrd Uncommitted LSARC/2008/129 libpq Uncommitted LSARC/2009/349 Java SE 6 Committed PSARC/2006/430 libkstat Uncommitted PSARC/1992/085 kstat_read Stable PSARC/1992/085 getmntent Evolving PSARC/1999/131 getloadavg Standard PSARC/1996/196 libmemcached Uncommitted LSARC/2008/126 libgcrypt Volatile LSARC/2008/354 libnotify Volatile LSARC/2007/137 swapctl Stable PSARC/1997/099 libhal Volatile PSARC/2005/399 getutxent N/A N/A syslog Standard N/A statvfs Stable N/A 5.3. Exported Interfaces NAME STABILITY NOTES ----------------------------------------------------------------------- SUNWcollectd4r Committed package SUNWcollectd4u Committed package collectd config file Uncommitted config file syntax collectd log file Not An Interface log file syntax collectd pid file Project Private daemon pid file /var/collectd4/logs Uncommitted log file dir /etc/collectd4/collectd.conf Uncommitted default conf file /usr/lib/collectd4/*.so Project Private plugins /usr/include/collectd Uncommitted Include directory /lib/svc/method/collectd4-svc Project Private SMF svc method svc:/application/monitoring/collectd4:default Committed FMRI /var/svc/manifest/application/monitoring/collectd4.xml Project Private SMF Manifest 6. References [1] http://collectd.org ================================================================ Appendix 1: collectd Integration Directory and File Structure. 1. The following files are included in the collectd integration: /usr/lib/collectd4 collectd collectdmon apache.so bind.so cpu.so csv.so curl.so df.so disk.so exec.so filecount.so interface.so java.so load.so logfile.so match_regex.so match_timediff.so match_value.so memcached.so memory.so mysql.so network.so ntpd.so postgresql.so rrdtool.so snmp.so syslog.so swap.so table.so tail.so tape.so target_notification.so target_replace.so target_set.so ted.so unixsock.so uptime.so users.so uuid.so libcollectdclient.so.0.0.0 libcollectdclient.so.0 -> libcollectdclient.so.0.0.0 libcollectdclient.so -> libcollectdclient.so.0.0.0 /etc/collectd4 /conf.d collectd.conf /var/collectd4 /logs collectd.log /rrd /usr/share/collectd4 types.db postgresql_default.conf java-plugin.jar jmx.jar /usr/share/man /man1 collectd.1 collectdmon.1 /man5 collectd.conf.5 collectd-exec.conf collectd-unixsock.5 collectd-java.5 collectd-snmp.5 types.db.5 /usr/include/collectd client.h lcc_features.h /lib/svc/method/collectd4-svc /var/svc/manifest/application/monitoring/collectd4.xml 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: sfw 6.5. ARC review type: FastTrack 6.6. ARC Exposure: open --Boundary_(ID_eLx2XaSwbQzWnZThvufvKQ)-- From Darren.Moffat@Sun.COM Wed Nov 11 09:04:52 2009 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nABH4qxO028036 for ; Wed, 11 Nov 2009 09:04:52 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id nABH4mqn030243 for <@sunmail2sca.sfbay.sun.com:LSARC-ext@sun.com>; Wed, 11 Nov 2009 10:04:52 -0700 (MST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KSY0020NFG1DN00@nwk-avmta-2.sfbay.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Wed, 11 Nov 2009 09:04:49 -0800 (PST) Received: from gmp-eb-inf-2.sun.com ([192.18.6.24]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KSY000DUFG0ED30@nwk-avmta-2.sfbay.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Wed, 11 Nov 2009 09:04:49 -0800 (PST) Received: from fe-emea-10.sun.com (gmp-eb-lb-1-fe1.eu.sun.com [192.18.6.7] (may be forged)) by gmp-eb-inf-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id nABH4lj9029314 for ; Wed, 11 Nov 2009 17:04:48 +0000 (GMT) Received: from conversion-daemon.fe-emea-10.sun.com by fe-emea-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KSY00E00FDIC200@fe-emea-10.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Wed, 11 Nov 2009 17:04:48 +0000 (GMT) Received: from [10.0.189.215] ([unknown] [192.18.41.196]) by fe-emea-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KSY00CY9FFNH740@fe-emea-10.sun.com>; Wed, 11 Nov 2009 17:04:38 +0000 (GMT) Date: Wed, 11 Nov 2009 09:04:35 -0800 From: Darren J Moffat Subject: Re: [webstack-discuss] Deliver collectd 4.8.x into OpenSolaris [LSARC/2009/606 FastTrack timeout 11/13/2009] In-reply-to: <4AFAE65A.7020105@Sun.COM> Sender: Darren.Moffat@Sun.COM To: Amit.Gupta@Sun.COM Cc: webstack-discuss@opensolaris.org, LSARC-ext@Sun.COM, Jyri Virkki Message-id: <4AFAEEA3.8070104@Sun.COM> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200911060916.nA69Gaq2018611@sac.sfbay.sun.com> <4AF3EFB9.1060403@Sun.COM> <4AF4583F.8000105@Sun.COM> <4AFAE65A.7020105@Sun.COM> User-Agent: Thunderbird 2.0.0.23 (X11/20091013) Status: RO Content-Length: 297 The filelayout looks fine. What uid/gid and privileges does SMF start collectd with ? (ie what is the method credential section in the SMF manifest). Hopefully this need to run as root with all privileges - ideally it should run as daemon with basic privileges (or less). -- Darren J Moffat From Jyri.Virkki@sun.com Wed Nov 11 19:32:56 2009 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nAC3WtvP016297 for ; Wed, 11 Nov 2009 19:32:55 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id nAC3Wrpi012592; Wed, 11 Nov 2009 19:32:53 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KSZ00L098ITPM00@nwk-avmta-2.sfbay.sun.com>; Wed, 11 Nov 2009 19:32:53 -0800 (PST) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KSZ00EMF8ISE8C0@nwk-avmta-2.sfbay.sun.com>; Wed, 11 Nov 2009 19:32:52 -0800 (PST) Received: from dm-usca15-11.red.iplanet.com (host-185-56-18-192.iplanet.com [192.18.56.185] (may be forged)) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id nAC3WGha009183; Thu, 12 Nov 2009 03:32:16 +0000 (GMT) Received: from buye.red.iplanet.com (buye [192.18.65.224]) by dm-usca15-11.red.iplanet.com (8.11.7p1+Sun/8.11.7/IPLANET,v1.2) with ESMTP id nAC3WEV18905; Wed, 11 Nov 2009 19:32:14 -0800 (PST) Received: from buye.red.iplanet.com (localhost [127.0.0.1]) by buye.red.iplanet.com (8.13.7+Sun/8.13.7) with ESMTP id nAC3WG64006771; Wed, 11 Nov 2009 19:32:16 -0800 (PST) Received: (from jyri@localhost) by buye.red.iplanet.com (8.13.7+Sun/8.13.7/Submit) id nAC3WGPK006770; Wed, 11 Nov 2009 19:32:16 -0800 (PST) Date: Wed, 11 Nov 2009 19:32:16 -0800 From: Jyri Virkki Subject: Re: [webstack-discuss] Deliver collectd 4.8.x into OpenSolaris [LSARC/2009/606 FastTrack timeout 11/13/2009] In-reply-to: <4AFAE65A.7020105@Sun.COM> To: Amit Gupta Cc: webstack-discuss@opensolaris.org, LSARC-ext@sun.com Message-id: <20091112033215.GA3636@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <200911060916.nA69Gaq2018611@sac.sfbay.sun.com> <4AF3EFB9.1060403@Sun.COM> <4AF4583F.8000105@Sun.COM> <4AFAE65A.7020105@Sun.COM> User-Agent: Mutt/1.5.11 Status: RO Content-Length: 3166 Amit Gupta wrote: > > Do find the updated ARC case attached. The summary of the changes is as > follows: > > - updated the file layout as per Darren's suggestion. jv01: The layout is now inconsistent with respect to versioning support. Note how various parts are versioned to 'collectd4' but other parts (under /usr/share/man and /usr/include) are not. That's the worst of both worlds, as it cannot support multiple collectd versions but pollutes the namespace with a partial attempt. The key decision to make is whether it is likely that collectd 4.x and [a future] collectd 5.x may both be needed in the same release of Solaris. There is no blanket answer to that unfortunately. At best, with enough familiarity with the upstream community, their future direction and the future direction of other components which use collectd, you can make an educated prediction. Given the upstream community has stated their major releases are incompatible it suggests coexistence might be needed. AFAIK there is no version 5 yet but looking back, looks like they still support 3.x (in maintenance mode) along with 4.x because the two are incompatible (http://collectd.org/download.shtml), so it is not unreasonable to imagine 4.x and 5.x may have a similar coexistence. On the down side, versioning always adds a bit of clutter and inconvenience unless needed. Think about it and decide whether to support collectd version coexistence or not. If anyone who is interested in collectd has a preference, speak up now. Once you decide, make the layout fully reflect the decision. Either it needs to allow coexistence everywhere, or not. jv02: Iff version coexistence is chosen, I was hoping for ARC members to have opinions on $COMPONENT$VERSION/* vs. $COMPONENT/$VERSION/* directory naming (see earlier email). If there are no opinions, project team can do as they wish. > - included collectdmon as suggested. In general it is unnecessary to deliver functionality which duplicates what smf provides. e.g. we don't deliver init.d files from upstream components just because they exist, given on Solaris smf takes care of it. collectdmon seems to fall in the same category. That said I don't have a strong opinion there, if you feel there is a customer benefit to delivering it, then ok. jv03: However, if you do, then document how the collectd smf service and collectdmon can (and can't) interact. In particular, in what ways can things go confusingly wrong if customer attempts to use both? The collectdmon man page is a good place to insert this information. > I am not sure as yet if we need to deliver only one collectd package i.e > SUNWcollectd?. As I mentioned earlier, I have followed the lead of > apache, lighttpd. mysql etc. all of which deliver a separate root and > usr package. Since you are integrating into sfw you will be required to integrate both usr and root packages. We also know they'll get renamed several times in ways outside of the project team control, so no point worrying about it until OpenSolaris gets its ARC act together. Call them Volatile, if you like. -- Jyri J. Virkki - jyri.virkki@sun.com - Sun Microsystems From Amit.Gupta@sun.com Wed Nov 11 23:16:34 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nAC7GX8f019510 for ; Wed, 11 Nov 2009 23:16:34 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id nAC7GPXY014310 for <@sunmail2sca.sfbay.sun.com:LSARC-ext@sun.com>; Thu, 12 Nov 2009 15:16:33 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KSZ00C01IVK2700@brm-avmta-1.central.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Thu, 12 Nov 2009 00:16:32 -0700 (MST) Received: from gmp-eb-inf-2.sun.com ([192.18.6.24]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KSZ000ZXIVI1EC0@brm-avmta-1.central.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Thu, 12 Nov 2009 00:16:31 -0700 (MST) Received: from fe-emea-10.sun.com (gmp-eb-lb-1-fe1.eu.sun.com [192.18.6.7] (may be forged)) by gmp-eb-inf-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id nAC7GUMA002442 for ; Thu, 12 Nov 2009 07:16:30 +0000 (GMT) Received: from conversion-daemon.fe-emea-10.sun.com by fe-emea-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KSZ00J00IE81200@fe-emea-10.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Thu, 12 Nov 2009 07:16:25 +0000 (GMT) Received: from test.sun.com ([unknown] [129.158.228.93]) by fe-emea-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KSZ00C4UIV3H7B0@fe-emea-10.sun.com>; Thu, 12 Nov 2009 07:16:18 +0000 (GMT) Date: Thu, 12 Nov 2009 12:33:16 +0530 From: Amit Gupta Subject: Re: [webstack-discuss] Deliver collectd 4.8.x into OpenSolaris [LSARC/2009/606 FastTrack timeout 11/13/2009] In-reply-to: <4AFAEEA3.8070104@Sun.COM> Sender: Amit.Gupta@sun.com To: Darren J Moffat Cc: webstack-discuss@opensolaris.org, LSARC-ext@sun.com, Jyri Virkki Reply-to: Amit.Gupta@sun.com Message-id: <4AFBB334.8080106@Sun.COM> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200911060916.nA69Gaq2018611@sac.sfbay.sun.com> <4AF3EFB9.1060403@Sun.COM> <4AF4583F.8000105@Sun.COM> <4AFAE65A.7020105@Sun.COM> <4AFAEEA3.8070104@Sun.COM> User-Agent: Thunderbird 2.0.0.4 (X11/20070622) Status: RO Content-Length: 404 On 11/11/09 22:34, Darren J Moffat wrote: > The filelayout looks fine. > > What uid/gid and privileges does SMF start collectd with ? (ie what is > the method credential section in the SMF manifest). SMF will start collectd as root. Regards Amit > Hopefully this need to run as root with all privileges - ideally it > should run as daemon with basic privileges (or less). > > -- > Darren J Moffat From storycrafter@gmail.com Thu Nov 12 06:43:42 2009 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nACEhgwg009353 for ; Thu, 12 Nov 2009 06:43:42 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id nACEhdF0017523; Thu, 12 Nov 2009 06:43:40 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KT000L0T3KRKA00@nwk-avmta-1.sfbay.Sun.COM>; Thu, 12 Nov 2009 06:43:39 -0800 (PST) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KT00079R3KQ7P50@nwk-avmta-1.sfbay.Sun.COM>; Thu, 12 Nov 2009 06:43:39 -0800 (PST) Received: from relay15i.sun.com (ip125.net129179-4.block1.us.syntegra.com [129.179.4.125]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id nACENOxJ027355; Thu, 12 Nov 2009 14:43:38 +0000 (GMT) Received: from mmp11es.mmp.us.syntegra.com ([160.41.208.11] [160.41.208.11]) by relay15i.sun.com with ESMTP id BT-MMP-2210158; Thu, 12 Nov 2009 14:43:38 +0000 (Z) Received: from relay15i.sun.com (relay15i.sun.com [129.179.4.125]) by mmp11es.mmp.us.syntegra.com with ESMTP id BT-MMP-10502650; Thu, 12 Nov 2009 14:43:38 +0000 (Z) Received: from gv-out-0910.google.com ([216.239.58.187] [216.239.58.187]) by relay1i.sun.com with ESMTP id BT-MMP-27828275; Thu, 12 Nov 2009 14:43:38 +0000 (Z) Received: by gv-out-0910.google.com with SMTP id l14so276672gvf.8 for ; Thu, 12 Nov 2009 06:43:33 -0800 (PST) Received: by 10.216.86.9 with SMTP id v9mr948487wee.148.1258037012111; Thu, 12 Nov 2009 06:43:32 -0800 (PST) Date: Thu, 12 Nov 2009 08:43:32 -0600 From: Mark Martin Subject: Re: [webstack-discuss] Deliver collectd 4.8.x into OpenSolaris [LSARC/2009/606 FastTrack timeout 11/13/2009] In-reply-to: <20091112033215.GA3636@sun.com> To: Jyri Virkki Cc: Amit Gupta , webstack-discuss@opensolaris.org, LSARC-ext@sun.com Message-id: MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1 Content-transfer-encoding: 7BIT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=O1BiAcqxrd6XRBIm0mH1BicaEU7cZlX0bcxq5N/aB7k=; b=tq3YkB7je2eW/UoHg2CE2R3jGq/rdmobPlYusxe7I+EDKwW0d7qc5ApoSSEqh2LCWD lCK5F7MpmGFKzdRR4+BKIdsz044msf5nz7S7E+9RTHSpu+PQ/R8Z+HLa2wwO5sv973ro 6NQq4fg4Nuc5h1/VhcGDgyyuHmr5O11KUFOmw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=CbmgVnKh7T+p6mjaBbDXQnkZH5Ntp8LarNbIY36qeNgQ6JM64xUmJK59VJ6lv7TKnV 1EE1AGHUA3oDqLtD3wPjNXHi+7QGp7qw9ML/eNdCvYHeF00Mv3FG7/le28T7f1MzxAp1 t/RE19ImLeE1PZSUoQX3heDu0xzi6ZOcLAnl0= X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== References: <200911060916.nA69Gaq2018611@sac.sfbay.sun.com> <4AF3EFB9.1060403@Sun.COM> <4AF4583F.8000105@Sun.COM> <4AFAE65A.7020105@Sun.COM> <20091112033215.GA3636@sun.com> Status: RO Content-Length: 3097 On Wed, Nov 11, 2009 at 9:32 PM, Jyri Virkki wrote: > The key decision to make is whether it is likely that collectd 4.x and > [a future] collectd 5.x may both be needed in the same release of Solaris. > > There is no blanket answer to that unfortunately. At best, with enough > familiarity with the upstream community, their future direction and > the future direction of other components which use collectd, you can > make an educated prediction. > > Given the upstream community has stated their major releases are > incompatible it suggests coexistence might be needed. AFAIK there is > no version 5 yet but looking back, looks like they still support 3.x > (in maintenance mode) along with 4.x because the two are incompatible > (http://collectd.org/download.shtml), so it is not unreasonable to > imagine 4.x and 5.x may have a similar coexistence. > > On the down side, versioning always adds a bit of clutter and > inconvenience unless needed. > > Think about it and decide whether to support collectd version > coexistence or not. > > If anyone who is interested in collectd has a preference, speak up now. My personal preference would be to take the conservative approach and plan on accommodating it (major version coexistence). You've given some history that suggests that it is more likely than not -- yesterday's weather is tomorrow's forecast and all that. > > Once you decide, make the layout fully reflect the decision. Either it > needs to allow coexistence everywhere, or not. > > > jv02: Iff version coexistence is chosen, I was hoping for ARC members > to have opinions on $COMPONENT$VERSION/* vs. $COMPONENT/$VERSION/* > directory naming (see earlier email). If there are no opinions, > project team can do as they wish. > With the qualification that I'm the newbie outsider member, I'll offer my preference for the $COMPONENT/$VERSION notation as a general guideline. As you mentioned in an earlier email, $COMPONENT$VERSION/ is mostly a historical artifact. It seems arbitrary to me either way, but for what it's worth, I'd rather not see version information as a decorator at the topmost level. Every year 100's of component names are created in the FOSS world (that's a guess), so the chance of creating "clutter" in /usr/ is an upwardly trending risk to my mind. > >> I am not sure as yet if we need to deliver only one collectd package i.e >> SUNWcollectd?. As I mentioned earlier, I have followed the lead of >> apache, lighttpd. mysql etc. all of which deliver a separate root and >> usr package. > > Since you are integrating into sfw you will be required to integrate > both usr and root packages. > > We also know they'll get renamed several times in ways outside of the > project team control, so no point worrying about it until OpenSolaris > gets its ARC act together. Call them Volatile, if you like. > > This is where I agree with you most strongly, Jyri. To my mind, we can't honestly ignore the stability level on these package names/structure. Volatile seems appropriate as a stability level until something changes that. From Amit.Gupta@sun.com Thu Nov 12 07:35:27 2009 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nACFZROw010098 for ; Thu, 12 Nov 2009 07:35:27 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id nACFZOjV013835 for <@sunmail2sca.sfbay.sun.com:LSARC-ext@sun.com>; Thu, 12 Nov 2009 07:35:26 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KT000H375Z1E400@brm-avmta-1.central.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Thu, 12 Nov 2009 08:35:25 -0700 (MST) Received: from gmp-eb-inf-2.sun.com ([192.18.6.24]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KT000D3K5YZ3D40@brm-avmta-1.central.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Thu, 12 Nov 2009 08:35:24 -0700 (MST) Received: from fe-emea-10.sun.com (gmp-eb-lb-1-fe1.eu.sun.com [192.18.6.7] (may be forged)) by gmp-eb-inf-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id nACFZNYQ016033 for ; Thu, 12 Nov 2009 15:35:23 +0000 (GMT) Received: from conversion-daemon.fe-emea-10.sun.com by fe-emea-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KT000M005QGX000@fe-emea-10.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Thu, 12 Nov 2009 15:35:09 +0000 (GMT) Received: from test.sun.com ([unknown] [129.158.228.93]) by fe-emea-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KT000CBE5YHL680@fe-emea-10.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Thu, 12 Nov 2009 15:35:08 +0000 (GMT) Date: Thu, 12 Nov 2009 20:52:06 +0530 From: Amit Gupta Subject: Re: [webstack-discuss] Deliver collectd 4.8.x into OpenSolaris [LSARC/2009/606 FastTrack timeout 11/13/2009] In-reply-to: <20091112033215.GA3636@sun.com> Sender: Amit.Gupta@sun.com To: Jyri Virkki Cc: webstack-discuss@opensolaris.org, LSARC-ext@sun.com Reply-to: Amit.Gupta@sun.com Message-id: <4AFC281E.7050407@Sun.COM> MIME-version: 1.0 Content-type: multipart/mixed; boundary="Boundary_(ID_1rBfliKqLVG9HwIBR7q4aA)" X-PMX-Version: 5.4.1.325704 References: <200911060916.nA69Gaq2018611@sac.sfbay.sun.com> <4AF3EFB9.1060403@Sun.COM> <4AF4583F.8000105@Sun.COM> <4AFAE65A.7020105@Sun.COM> <20091112033215.GA3636@sun.com> User-Agent: Thunderbird 2.0.0.4 (X11/20070622) Status: RO Content-Length: 17868 This is a multi-part message in MIME format. --Boundary_(ID_1rBfliKqLVG9HwIBR7q4aA) Content-type: multipart/alternative; boundary="Boundary_(ID_mZwC6Sl7H8ElbBC/U18j7g)" --Boundary_(ID_mZwC6Sl7H8ElbBC/U18j7g) Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT On 11/12/09 09:02, Jyri Virkki wrote: > Given the upstream community has stated their major releases are > incompatible it suggests coexistence might be needed. AFAIK there is > no version 5 yet but looking back, looks like they still support 3.x > (in maintenance mode) along with 4.x because the two are incompatible > (http://collectd.org/download.shtml), so it is not unreasonable to > imagine 4.x and 5.x may have a similar coexistence. > > On the down side, versioning always adds a bit of clutter and > inconvenience unless needed. > > Think about it and decide whether to support collectd version > coexistence or not. > I will assume that multiple versions will co-exist. I have updated the attached ARC case accordingly. > >> - included collectdmon as suggested. >> > > jv03: However, if you do, then document how the collectd smf service > and collectdmon can (and can't) interact. In particular, in what ways > can things go confusingly wrong if customer attempts to use both? The > collectdmon man page is a good place to insert this information. > While collectdmon is not required if collectd is started via SMF, there may be products which embed collectd and may want to manage lifecyle of collectd via collectdmon. Regards Amit --Boundary_(ID_mZwC6Sl7H8ElbBC/U18j7g) Content-type: text/html; CHARSET=US-ASCII Content-transfer-encoding: 7BIT On 11/12/09 09:02, Jyri Virkki wrote:
Given the upstream community has stated their major releases are
incompatible it suggests coexistence might be needed. AFAIK there is
no version 5 yet but looking back, looks like they still support 3.x
(in maintenance mode) along with 4.x because the two are incompatible
(http://collectd.org/download.shtml), so it is not unreasonable to
imagine 4.x and 5.x may have a similar coexistence.

On the down side, versioning always adds a bit of clutter and
inconvenience unless needed.

Think about it and decide whether to support collectd version
coexistence or not.
I will assume that multiple versions will co-exist. I have updated the attached ARC case accordingly.
  
- included collectdmon as suggested.
    

jv03: However, if you do, then document how the collectd smf service
and collectdmon can (and can't) interact.  In particular, in what ways
can things go confusingly wrong if customer attempts to use both?  The
collectdmon man page is a good place to insert this information.
While collectdmon is not required if collectd is started via SMF, there may be products which embed collectd and may want to manage lifecyle of collectd via collectdmon.

Regards
Amit --Boundary_(ID_mZwC6Sl7H8ElbBC/U18j7g)-- --Boundary_(ID_1rBfliKqLVG9HwIBR7q4aA) Content-type: text/plain; name=collectd-arc.txt Content-transfer-encoding: 7BIT Content-disposition: inline; filename=collectd-arc.txt Template Version: @(#)sac_nextcase 1.68 02/23/09 SMI This information is Copyright 2009 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: Deliver collectd 4.8.x into OpenSolaris 1.2. Name of Document Author/Supplier: Author: Amit Gupta 1.3 Date of This Document: 06 November, 2009 4. Technical Description Deliver collectd 4.8.x into OpenSolaris 12 November 2009 1. Summary and motivation 1.1. Introduction This project delivers collectd 4.8.x into OpenSolaris collectd is a daemon which collects system performance statistics periodically and provides mechanisms to store the values in a variety of ways, for example in RRD files. For more information about collectd see: http://collectd.org 2. Technical issues 2.1. Key objects /usr/lib/collectd4/collectd From collectd's man page: collectd is a daemon that receives system statistics and makes them available in a number of ways. The main daemon itself doesn't have any real functionality apart from loading, querying and submitting to plugins. /usr/lib/collectd4/collectdmon From collectdmon's man page: collectdmon is a small ``wrapper'' daemon which starts and monitors the collectd daemon. If collectd terminates it will automatically be restarted, unless collectdmon was told to shut it down. Note that collectdmon is not require if collectd is started via SMF. However, there may be products which embed collectd and may want to manage lifecyle of collectd via collectmon. /etc/collectd4/collectd.conf The default collectd configuration file. 2.2. Versioning From collectd's FAQ: The version numbers in collectd consist of three numbers: The major- and minor-number and the patchlevel. * Versions with different major-numbers are basically not compatible. This means that the definitions of RRD-files or config-options have been changed or, in general, that the user has to do something in addition to install the new version. This is not nice and avoided when possible, but sometimes necessary to prevent old mistakes to become ancient mistakes. * Versions with differing minor-numbers are backwards compatible, i. e. you can replace the lower version with the higher one and everything should still work. This means that features are added, but not removed or changed and that the default behavior does not change. * Versions with different patchlevels are both, forward- and backwards-compatible, because no new features have been introduced. The only difference between the two versions is one or more bug fixes, so you should generally install the higher version of the two. 2.3 Loadable Plugins collectd ships with a number of loadable plugins which are enabled through directives in the config file. Some of these plugins have library dependencies that must be satisfied through build time package options (supplied to the configure script). An example is mysql.so which has a dependency on the mysql client libraries. Support for loadable plugins and other features is enabled at build time through flags to the configure script. All the plugins that are delivered have sample configuration directives in collectd.conf. Note that in many cases, these are just sample directives and will need to be modified based on site specific configuration. Detailed documentation about specific plugin configuration can be found in the collectd website. Not all the plugins are enabled by default in collectd.conf. The configuration directives for plugins which are not enabled are commented out in collectd.conf. The plugins which are enabled by default are: syslog, cpu, interface, load, memory and rrdtool Other plugins can be enabled in one of the following two ways: * By adding/uncommenting the plugin configuration directives in collectd.conf * By dropping a .conf file having the necessary plugin configuration directives into the /etc/collectd4/conf.d Similarly, plugins can be disabled as needed in the following two ways: * By removing/commenting out the plugin configuration directives from collectd.conf * By deleting the plugin specific .conf file from the /etc/collectd4/conf.d directory Do note that collectd.conf is read and processed in order, i. e. from top to bottom. So the plugins are loaded in the order listed in collectd.conf. This project will deliver the following plugins: * Apache * CPU * CSV * cURL * DF * Disk * Exec * FileCount * GenericJMX * Interface * Java * Load * LogFile * memcachec * memcached * Memory * MySQL * Network * Notify Desktop * NTPd * PostgreSQL * RRDtool * SNMP * Swap * SysLog * Table * Tail * Tape * UnixSock * Uptime * Users * UUID * Write HTTP * ZFS ARC 2.4 Directory Naming and Structure The proposed directory layout for collectd is: /usr/lib/collectd4 /etc/collectd4 /conf.d /var/collectd4 /rrd /logs /usr/share/collectd4 /usr/include/collectd The detailed directory and file layout for collectd is provided in Appendix 1. Multiple versions can coexist, and are distinguished by the naming scheme (collectd4 for version 4.*.*, collectd5 for version 5.*.*). 2.5 Configuration File This project will use the standard convention and deliver configuration files under /etc/collectd4 The default configuration file provided with this integration will work out of the box. In addition, a directory conf.d will be created in /etc/collectd4/ The default configuration loads any file with a .conf extension that is placed in this directory. Hence, plugins can optionally be configured by simply dropping a .conf file into this directory without modifying the main configuration file. By default, conf.d will not contain any plugin configuration files. 2.6 Log File The log File will be located in /var/collectd4/logs and will be named collectd.log. The default log level is info and can be changed by modifying the main configuration file. 2.7 Build features Build time package selection determines which collectd loadable plugins are available to the user at runtime. A complete list of collectd modules is available on the http://collectd.org. However, not all plugins can be built on OpenSolaris because of dependencies on external libraries and hence, only a subset of the plugins will be delivered. 3. collectd documentation collectd 4.* provides a full and complete documentation set, in UNIX manual format. This project will deliver the complete documentation set. 4. Packaging and Delivery We propose to package collectd under the following packages: SUNWcollectd4u - [usr] Server package (including collectd daemon, loadable plugins, man pages) SUNWcollectd4r - [root] (config files, smf files) 5. collectd Interfaces 5.1. Interface Stability collectd has no obvious history of any interface instability and the expectation is that dot releases of collectd will be compatible. This is not completely guaranteed by the community and it is possible that interfaces may change due to the fixing of bugs. 5.2. Imported Interfaces libcurl Volatile PSARC/2007/165 Output format of Apache's mod_status N/A PSARC/2007/586 Output format of Lighhtpd's mod_status N/A LSARC/2008/191 libmysqlclient Committed LSARC 2009/062 libnetsnmp Uncommitted LSARC/2008/355 librrd Uncommitted LSARC/2008/129 libpq Uncommitted LSARC/2009/349 Java SE 6 Committed PSARC/2006/430 libkstat Uncommitted PSARC/1992/085 kstat_read Stable PSARC/1992/085 getmntent Evolving PSARC/1999/131 getloadavg Standard PSARC/1996/196 libmemcached Uncommitted LSARC/2008/126 libgcrypt Volatile LSARC/2008/354 libnotify Volatile LSARC/2007/137 swapctl Stable PSARC/1997/099 libhal Volatile PSARC/2005/399 getutxent N/A N/A syslog Standard N/A statvfs Stable N/A 5.3. Exported Interfaces NAME STABILITY NOTES ----------------------------------------------------------------------- SUNWcollectd4r Committed package SUNWcollectd4u Committed package collectd config file Uncommitted config file syntax collectd log file Not An Interface log file syntax collectd pid file Project Private daemon pid file /var/collectd4/logs Uncommitted log file dir /etc/collectd4/collectd.conf Uncommitted default conf file /usr/lib/collectd4/*.so Project Private plugins /usr/include/collectd Uncommitted Include directory /lib/svc/method/collectd4-svc Project Private SMF svc method svc:/application/monitoring/collectd4:default Committed FMRI /var/svc/manifest/application/monitoring/collectd4.xml Project Private SMF Manifest 6. References [1] http://collectd.org ================================================================ Appendix 1: collectd Integration Directory and File Structure. 1. The following files are included in the collectd integration: /usr/lib/collectd4 collectd collectdmon apache.so bind.so cpu.so csv.so curl.so df.so disk.so exec.so filecount.so interface.so java.so load.so logfile.so match_regex.so match_timediff.so match_value.so memcached.so memory.so mysql.so network.so ntpd.so postgresql.so rrdtool.so snmp.so syslog.so swap.so table.so tail.so tape.so target_notification.so target_replace.so target_set.so ted.so unixsock.so uptime.so users.so uuid.so libcollectdclient.so.0.0.0 libcollectdclient.so.0 -> libcollectdclient.so.0.0.0 libcollectdclient.so -> libcollectdclient.so.0.0.0 /etc/collectd4 /conf.d collectd.conf /var/collectd4 /logs collectd.log /rrd /usr/share/collectd4 types.db postgresql_default.conf java-plugin.jar jmx.jar /man /man1 collectd.1 collectdmon.1 /man5 collectd.conf.5 collectd-exec.conf collectd-unixsock.5 collectd-java.5 collectd-snmp.5 types.db.5 /usr/include/collectd4 client.h lcc_features.h /lib/svc/method/collectd4-svc /var/svc/manifest/application/monitoring/collectd4.xml 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: sfw 6.5. ARC review type: FastTrack 6.6. ARC Exposure: open --Boundary_(ID_1rBfliKqLVG9HwIBR7q4aA)-- From Darren.Moffat@Sun.COM Thu Nov 12 08:45:51 2009 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nACGjpDK011193 for ; Thu, 12 Nov 2009 08:45:51 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id nACGjoAg019755 for <@sunmail2sca.sfbay.sun.com:LSARC-ext@sun.com>; Thu, 12 Nov 2009 08:45:50 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KT00008D98EPK00@brm-avmta-1.central.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Thu, 12 Nov 2009 09:45:50 -0700 (MST) Received: from gmp-eb-inf-1.sun.com ([192.18.6.21]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KT000DUL98C38A0@brm-avmta-1.central.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Thu, 12 Nov 2009 09:45:49 -0700 (MST) Received: from fe-emea-09.sun.com (gmp-eb-lb-1-fe1.eu.sun.com [192.18.6.7] (may be forged)) by gmp-eb-inf-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id nACGjmDr015354 for ; Thu, 12 Nov 2009 16:45:48 +0000 (GMT) Received: from conversion-daemon.fe-emea-09.sun.com by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KT000M008P2M600@fe-emea-09.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Thu, 12 Nov 2009 16:45:46 +0000 (GMT) Received: from [192.168.2.108] (99-52-200-208.lightspeed.snjsca.sbcglobal.net [99.52.200.208]) by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KT00072Y984EZD0@fe-emea-09.sun.com>; Thu, 12 Nov 2009 16:45:44 +0000 (GMT) Date: Thu, 12 Nov 2009 08:45:40 -0800 From: Darren J Moffat Subject: Re: [webstack-discuss] Deliver collectd 4.8.x into OpenSolaris [LSARC/2009/606 FastTrack timeout 11/13/2009] In-reply-to: <4AFBB334.8080106@Sun.COM> Sender: Darren.Moffat@Sun.COM To: Amit.Gupta@Sun.COM Cc: webstack-discuss@opensolaris.org, LSARC-ext@Sun.COM, Jyri Virkki Message-id: <4AFC3BB4.7070806@Sun.COM> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200911060916.nA69Gaq2018611@sac.sfbay.sun.com> <4AF3EFB9.1060403@Sun.COM> <4AF4583F.8000105@Sun.COM> <4AFAE65A.7020105@Sun.COM> <4AFAEEA3.8070104@Sun.COM> <4AFBB334.8080106@Sun.COM> User-Agent: Thunderbird 2.0.0.23 (X11/20091013) Status: RO Content-Length: 615 Amit Gupta wrote: > On 11/11/09 22:34, Darren J Moffat wrote: >> The filelayout looks fine. >> >> What uid/gid and privileges does SMF start collectd with ? (ie what is >> the method credential section in the SMF manifest). > SMF will start collectd as root. Why does collectd need to run as root ? What stats is it that is collectd needs that requires it to run with root ? It really shouldn't need to. > Regards > Amit >> Hopefully this need to run as root with all privileges - ideally it >> should run as daemon with basic privileges (or less). I missed a "not" in that sentence. -- Darren J Moffat From Amit.Gupta@sun.com Thu Nov 12 10:30:03 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nACIU2qG015703 for ; Thu, 12 Nov 2009 10:30:03 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id nACITr1A006144 for <@sunmail2sca.sfbay.sun.com:LSARC-ext@sun.com>; Fri, 13 Nov 2009 02:30:01 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KT000B2VE20A400@brm-avmta-1.central.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Thu, 12 Nov 2009 11:30:00 -0700 (MST) Received: from sineb-mail-2.sun.com ([192.18.19.7]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KT00065ME1WZB20@brm-avmta-1.central.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Thu, 12 Nov 2009 11:29:58 -0700 (MST) Received: from fe-apac-05.sun.com (fe-apac-05.sun.com [192.18.19.176] (may be forged)) by sineb-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id nACITuQt010606 for ; Thu, 12 Nov 2009 18:29:56 +0000 (GMT) Received: from conversion-daemon.mail-apac.sun.com by mail-apac.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KT000E00DNQI100@mail-apac.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Fri, 13 Nov 2009 02:29:56 +0800 (SGT) Received: from [192.168.1.34] ([unknown] [122.167.234.175]) by mail-apac.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KT0006D7E1SZS50@mail-apac.sun.com>; Fri, 13 Nov 2009 02:29:56 +0800 (SGT) Date: Thu, 12 Nov 2009 23:59:52 +0530 From: "amit.gupta@sun.com" Subject: Re: [webstack-discuss] Deliver collectd 4.8.x into OpenSolaris [LSARC/2009/606 FastTrack timeout 11/13/2009] In-reply-to: <4AFC3BB4.7070806@Sun.COM> Sender: Amit.Gupta@sun.com To: Darren J Moffat Cc: webstack-discuss@opensolaris.org, LSARC-ext@sun.com, Jyri Virkki Message-id: <4AFC5420.7020202@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200911060916.nA69Gaq2018611@sac.sfbay.sun.com> <4AF3EFB9.1060403@Sun.COM> <4AF4583F.8000105@Sun.COM> <4AFAE65A.7020105@Sun.COM> <4AFAEEA3.8070104@Sun.COM> <4AFBB334.8080106@Sun.COM> <4AFC3BB4.7070806@Sun.COM> User-Agent: Thunderbird 2.0.0.23 (Windows/20090812) Status: RO Content-Length: 855 Darren J Moffat wrote: > Amit Gupta wrote: >> On 11/11/09 22:34, Darren J Moffat wrote: >>> The filelayout looks fine. >>> >>> What uid/gid and privileges does SMF start collectd with ? (ie what >>> is the method credential section in the SMF manifest). >> SMF will start collectd as root. > > Why does collectd need to run as root ? > > What stats is it that is collectd needs that requires it to run with > root ? It really shouldn't need to. I don't see any issues running collectd as a non root user, however, I am not certain if any of the collectd plugins require collectd to run as root. What user do you suggest collectd to run as? Regards Amit > >> Regards >> Amit >>> Hopefully this need to run as root with all privileges - ideally it >>> should run as daemon with basic privileges (or less). > > I missed a "not" in that sentence. > From Darren.Moffat@Sun.COM Thu Nov 12 10:52:59 2009 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nACIqw5f015987 for ; Thu, 12 Nov 2009 10:52:58 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id nACIqvPK004643 for <@sunmail2sca.sfbay.sun.com:LSARC-ext@sun.com>; Thu, 12 Nov 2009 10:52:58 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KT000C17F494B00@nwk-avmta-2.sfbay.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Thu, 12 Nov 2009 10:52:57 -0800 (PST) Received: from gmp-eb-inf-1.sun.com ([192.18.6.21]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KT00087WF48U740@nwk-avmta-2.sfbay.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Thu, 12 Nov 2009 10:52:57 -0800 (PST) Received: from fe-emea-10.sun.com (gmp-eb-lb-1-fe1.eu.sun.com [192.18.6.7] (may be forged)) by gmp-eb-inf-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id nACIqtej024899 for ; Thu, 12 Nov 2009 18:52:56 +0000 (GMT) Received: from conversion-daemon.fe-emea-10.sun.com by fe-emea-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KT000K00ET4GR00@fe-emea-10.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Thu, 12 Nov 2009 18:52:37 +0000 (GMT) Received: from [10.0.189.215] ([unknown] [192.18.41.196]) by fe-emea-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KT000C3SF3ML6E0@fe-emea-10.sun.com>; Thu, 12 Nov 2009 18:52:37 +0000 (GMT) Date: Thu, 12 Nov 2009 10:52:35 -0800 From: Darren J Moffat Subject: Re: [webstack-discuss] Deliver collectd 4.8.x into OpenSolaris [LSARC/2009/606 FastTrack timeout 11/13/2009] In-reply-to: <4AFC5420.7020202@sun.com> Sender: Darren.Moffat@Sun.COM To: "amit.gupta@sun.com" Cc: webstack-discuss@opensolaris.org, LSARC-ext@Sun.COM, Jyri Virkki Message-id: <4AFC5973.4070504@Sun.COM> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200911060916.nA69Gaq2018611@sac.sfbay.sun.com> <4AF3EFB9.1060403@Sun.COM> <4AF4583F.8000105@Sun.COM> <4AFAE65A.7020105@Sun.COM> <4AFAEEA3.8070104@Sun.COM> <4AFBB334.8080106@Sun.COM> <4AFC3BB4.7070806@Sun.COM> <4AFC5420.7020202@sun.com> User-Agent: Thunderbird 2.0.0.23 (X11/20091013) Status: RO Content-Length: 770 amit.gupta@sun.com wrote: > Darren J Moffat wrote: >> Amit Gupta wrote: >>> On 11/11/09 22:34, Darren J Moffat wrote: >>>> The filelayout looks fine. >>>> >>>> What uid/gid and privileges does SMF start collectd with ? (ie what >>>> is the method credential section in the SMF manifest). >>> SMF will start collectd as root. >> >> Why does collectd need to run as root ? >> >> What stats is it that is collectd needs that requires it to run with >> root ? It really shouldn't need to. > I don't see any issues running collectd as a non root user, however, I > am not certain if any of the collectd plugins require collectd to run as > root. What user do you suggest collectd to run as? user daemon, group daemon and the basic privilege set. -- Darren J Moffat From Amit.Gupta@sun.com Fri Nov 13 06:35:24 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nADEZOIP021799 for ; Fri, 13 Nov 2009 06:35:24 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id nADEZJJF022942 for <@sunmail2sca.sfbay.sun.com:LSARC-ext@sun.com>; Fri, 13 Nov 2009 22:35:23 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KT100101XUX6700@brm-avmta-1.central.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Fri, 13 Nov 2009 07:35:21 -0700 (MST) Received: from gmp-eb-inf-1.sun.com ([192.18.6.21]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KT1000EZXUSQ900@brm-avmta-1.central.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Fri, 13 Nov 2009 07:35:18 -0700 (MST) Received: from fe-emea-09.sun.com (gmp-eb-lb-1-fe1.eu.sun.com [192.18.6.7] (may be forged)) by gmp-eb-inf-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id nADEZGss026152 for ; Fri, 13 Nov 2009 14:35:16 +0000 (GMT) Received: from conversion-daemon.fe-emea-09.sun.com by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KT100A00XO1MF00@fe-emea-09.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Fri, 13 Nov 2009 14:35:00 +0000 (GMT) Received: from test.sun.com ([unknown] [129.158.228.93]) by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KT100B88XU0JDC0@fe-emea-09.sun.com>; Fri, 13 Nov 2009 14:34:53 +0000 (GMT) Date: Fri, 13 Nov 2009 19:51:47 +0530 From: Amit Gupta Subject: Re: [webstack-discuss] Deliver collectd 4.8.x into OpenSolaris [LSARC/2009/606 FastTrack timeout 11/13/2009] In-reply-to: <4AFC5973.4070504@Sun.COM> Sender: Amit.Gupta@sun.com To: Darren J Moffat Cc: webstack-discuss@opensolaris.org, LSARC-ext@sun.com, Jyri Virkki Reply-to: Amit.Gupta@sun.com Message-id: <4AFD6B7B.706@Sun.COM> MIME-version: 1.0 Content-type: multipart/mixed; boundary="Boundary_(ID_E5lUmjSqPvtLq5GDks9X8g)" X-PMX-Version: 5.4.1.325704 References: <200911060916.nA69Gaq2018611@sac.sfbay.sun.com> <4AF3EFB9.1060403@Sun.COM> <4AF4583F.8000105@Sun.COM> <4AFAE65A.7020105@Sun.COM> <4AFAEEA3.8070104@Sun.COM> <4AFBB334.8080106@Sun.COM> <4AFC3BB4.7070806@Sun.COM> <4AFC5420.7020202@sun.com> <4AFC5973.4070504@Sun.COM> User-Agent: Thunderbird 2.0.0.4 (X11/20070622) Status: RO Content-Length: 15596 This is a multi-part message in MIME format. --Boundary_(ID_E5lUmjSqPvtLq5GDks9X8g) Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT Do find the updated ARC case attached. I have added a new section "Lifecycle" which specifies that collectd will run as daemon:daemon. Regards Amit On 11/13/09 00:22, Darren J Moffat wrote: > amit.gupta@sun.com wrote: >> Darren J Moffat wrote: >>> Amit Gupta wrote: >>>> On 11/11/09 22:34, Darren J Moffat wrote: >>>>> The filelayout looks fine. >>>>> >>>>> What uid/gid and privileges does SMF start collectd with ? (ie >>>>> what is the method credential section in the SMF manifest). >>>> SMF will start collectd as root. >>> >>> Why does collectd need to run as root ? >>> >>> What stats is it that is collectd needs that requires it to run with >>> root ? It really shouldn't need to. >> I don't see any issues running collectd as a non root user, however, >> I am not certain if any of the collectd plugins require collectd to >> run as root. What user do you suggest collectd to run as? > > user daemon, group daemon and the basic privilege set. > --Boundary_(ID_E5lUmjSqPvtLq5GDks9X8g) Content-type: text/plain; name=collectd-arc.txt Content-transfer-encoding: 7BIT Content-disposition: inline; filename=collectd-arc.txt Template Version: @(#)sac_nextcase 1.68 02/23/09 SMI This information is Copyright 2009 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: Deliver collectd 4.8.x into OpenSolaris 1.2. Name of Document Author/Supplier: Author: Amit Gupta 1.3 Date of This Document: 06 November, 2009 4. Technical Description Deliver collectd 4.8.x into OpenSolaris 13 November 2009 1. Summary and motivation 1.1. Introduction This project delivers collectd 4.8.x into OpenSolaris collectd is a daemon which collects system performance statistics periodically and provides mechanisms to store the values in a variety of ways, for example in RRD files. For more information about collectd see: http://collectd.org 2. Technical description 2.1. Key objects /usr/lib/collectd4/collectd From collectd's man page: collectd is a daemon that receives system statistics and makes them available in a number of ways. The main daemon itself doesn't have any real functionality apart from loading, querying and submitting to plugins. /usr/lib/collectd4/collectdmon From collectdmon's man page: collectdmon is a small ``wrapper'' daemon which starts and monitors the collectd daemon. If collectd terminates it will automatically be restarted, unless collectdmon was told to shut it down. Note that collectdmon is not require if collectd is started via SMF. However, there may be products which embed collectd and may want to manage lifecyle of collectd via collectmon. /etc/collectd4/collectd.conf The default collectd configuration file. 2.2. Versioning From collectd's FAQ: The version numbers in collectd consist of three numbers: The major- and minor-number and the patchlevel. * Versions with different major-numbers are basically not compatible. This means that the definitions of RRD-files or config-options have been changed or, in general, that the user has to do something in addition to install the new version. This is not nice and avoided when possible, but sometimes necessary to prevent old mistakes to become ancient mistakes. * Versions with differing minor-numbers are backwards compatible, i. e. you can replace the lower version with the higher one and everything should still work. This means that features are added, but not removed or changed and that the default behavior does not change. * Versions with different patchlevels are both, forward- and backwards-compatible, because no new features have been introduced. The only difference between the two versions is one or more bug fixes, so you should generally install the higher version of the two. 2.3 Configuration File This project will use the standard convention and deliver configuration files under /etc/collectd4 The default configuration file provided with this integration will work out of the box. In addition, a directory conf.d will be created in /etc/collectd4/ The default configuration loads any file with a .conf extension that is placed in this directory. Hence, plugins can optionally be configured by simply dropping a .conf file into this directory without modifying the main configuration file. By default, conf.d will not contain any plugin configuration files. 2.4 Log File The log File will be located in /var/collectd4/logs and will be named collectd.log. The default log level is info and can be changed by modifying the main configuration file. 2.5 Lifecycle collectd will be started as a SMF service and will run as user "daemon" and group "daemon". 2.6 Loadable Plugins collectd ships with a number of loadable plugins which are enabled through directives in the config file. Some of these plugins have library dependencies that must be satisfied through build time package options (supplied to the configure script). An example is mysql.so which has a dependency on the mysql client libraries. Support for loadable plugins and other features is enabled at build time through flags to the configure script. All the plugins that are delivered have sample configuration directives in collectd.conf. Note that in many cases, these are just sample directives and will need to be modified based on site specific configuration. Detailed documentation about specific plugin configuration can be found in the collectd website. Not all the plugins are enabled by default in collectd.conf. The configuration directives for plugins which are not enabled are commented out in collectd.conf. The plugins which are enabled by default are: syslog, cpu, interface, load, memory and rrdtool Other plugins can be enabled in one of the following two ways: * By adding/uncommenting the plugin configuration directives in collectd.conf * By dropping a .conf file having the necessary plugin configuration directives into the /etc/collectd4/conf.d Similarly, plugins can be disabled as needed in the following two ways: * By removing/commenting out the plugin configuration directives from collectd.conf * By deleting the plugin specific .conf file from the /etc/collectd4/conf.d directory Do note that collectd.conf is read and processed in order, i. e. from top to bottom. So the plugins are loaded in the order listed in collectd.conf. This project will deliver the following plugins: * Apache * CPU * CSV * cURL * DF * Disk * Exec * FileCount * GenericJMX * Interface * Java * Load * LogFile * memcachec * memcached * Memory * MySQL * Network * Notify Desktop * NTPd * PostgreSQL * RRDtool * SNMP * Swap * SysLog * Table * Tail * Tape * UnixSock * Uptime * Users * Write HTTP * ZFS ARC 2.7 Directory Naming and Structure The proposed directory layout for collectd is: /usr/lib/collectd4 /etc/collectd4 /conf.d /var/collectd4 /rrd /logs /usr/share/collectd4 /usr/include/collectd The detailed directory and file layout for collectd is provided in Appendix 1. Multiple versions can coexist, and are distinguished by the naming scheme (collectd4 for version 4.*.*, collectd5 for version 5.*.*). 2.8 Build features Build time package selection determines which collectd loadable plugins are available to the user at runtime. A complete list of collectd modules is available on the http://collectd.org. However, not all plugins can be built on OpenSolaris because of dependencies on external libraries and hence, only a subset of the plugins will be delivered. 3. collectd documentation collectd 4.* provides a full and complete documentation set, in UNIX manual format. This project will deliver the complete documentation set. 4. Packaging and Delivery We propose to package collectd under the following packages: SUNWcollectd4u - [usr] Server package (including collectd daemon, loadable plugins, man pages) SUNWcollectd4r - [root] (config files, smf files) 5. collectd Interfaces 5.1. Interface Stability collectd has no obvious history of any interface instability and the expectation is that dot releases of collectd will be compatible. This is not completely guaranteed by the community and it is possible that interfaces may change due to the fixing of bugs. 5.2. Imported Interfaces libcurl Volatile PSARC/2007/165 Output format of Apache's mod_status N/A PSARC/2007/586 Output format of Lighhtpd's mod_status N/A LSARC/2008/191 libmysqlclient Committed LSARC 2009/062 libnetsnmp Uncommitted LSARC/2008/355 librrd Uncommitted LSARC/2008/129 libpq Uncommitted LSARC/2009/349 Java SE 6 Committed PSARC/2006/430 libkstat Uncommitted PSARC/1992/085 kstat_read Stable PSARC/1992/085 getmntent Evolving PSARC/1999/131 getloadavg Standard PSARC/1996/196 libmemcached Uncommitted LSARC/2008/126 libgcrypt Volatile LSARC/2008/354 libnotify Volatile LSARC/2007/137 swapctl Stable PSARC/1997/099 libhal Volatile PSARC/2005/399 getutxent N/A N/A syslog Standard N/A statvfs Stable N/A 5.3. Exported Interfaces NAME STABILITY NOTES ----------------------------------------------------------------------- SUNWcollectd4r Committed package SUNWcollectd4u Committed package collectd config file Uncommitted config file syntax collectd log file Not An Interface log file syntax collectd pid file Project Private daemon pid file /var/collectd4/logs Uncommitted log file dir /etc/collectd4/collectd.conf Uncommitted default conf file /usr/lib/collectd4/*.so Project Private plugins /usr/include/collectd Uncommitted Include directory /lib/svc/method/collectd4-svc Project Private SMF svc method svc:/application/monitoring/collectd4:default Committed FMRI /var/svc/manifest/application/monitoring/collectd4.xml Project Private SMF Manifest 6. References [1] http://collectd.org ================================================================ Appendix 1: collectd Integration Directory and File Structure. 1. The following files are included in the collectd integration: /usr/lib/collectd4 collectd collectdmon apache.so bind.so cpu.so csv.so curl.so df.so disk.so exec.so filecount.so interface.so java.so load.so logfile.so match_regex.so match_timediff.so match_value.so memcached.so memory.so mysql.so network.so notify_desktop.so ntpd.so postgresql.so rrdtool.so snmp.so syslog.so swap.so table.so tail.so tape.so target_notification.so target_replace.so target_set.so ted.so unixsock.so uptime.so users.so libcollectdclient.so.0.0.0 libcollectdclient.so.0 -> libcollectdclient.so.0.0.0 libcollectdclient.so -> libcollectdclient.so.0.0.0 /etc/collectd4 /conf.d collectd.conf /var/collectd4 /logs collectd.log /rrd /usr/share/collectd4 types.db postgresql_default.conf java-plugin.jar jmx.jar /man /man1 collectd.1 collectdmon.1 /man5 collectd.conf.5 collectd-exec.conf collectd-unixsock.5 collectd-java.5 collectd-snmp.5 types.db.5 /usr/include/collectd4 client.h lcc_features.h /lib/svc/method/collectd4-svc /var/svc/manifest/application/monitoring/collectd4.xml 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: sfw 6.5. ARC review type: FastTrack 6.6. ARC Exposure: open --Boundary_(ID_E5lUmjSqPvtLq5GDks9X8g)-- From Darren.Moffat@sun.com Fri Nov 13 13:27:25 2009 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nACGjpDK011193 for ; Thu, 12 Nov 2009 08:45:51 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id nACGjoAg019755 for <@sunmail2sca.sfbay.sun.com:LSARC-ext@sun.com>; Thu, 12 Nov 2009 08:45:50 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KT00008D98EPK00@brm-avmta-1.central.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Thu, 12 Nov 2009 09:45:50 -0700 (MST) Received: from gmp-eb-inf-1.sun.com ([192.18.6.21]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KT000DUL98C38A0@brm-avmta-1.central.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Thu, 12 Nov 2009 09:45:49 -0700 (MST) Received: from fe-emea-09.sun.com (gmp-eb-lb-1-fe1.eu.sun.com [192.18.6.7] (may be forged)) by gmp-eb-inf-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id nACGjmDr015354 for ; Thu, 12 Nov 2009 16:45:48 +0000 (GMT) Received: from conversion-daemon.fe-emea-09.sun.com by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KT000M008P2M600@fe-emea-09.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Thu, 12 Nov 2009 16:45:46 +0000 (GMT) Received: from [192.168.2.108] (99-52-200-208.lightspeed.snjsca.sbcglobal.net [99.52.200.208]) by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KT00072Y984EZD0@fe-emea-09.sun.com>; Thu, 12 Nov 2009 16:45:44 +0000 (GMT) Date: Thu, 12 Nov 2009 08:45:40 -0800 From: Darren J Moffat Subject: Re: [webstack-discuss] Deliver collectd 4.8.x into OpenSolaris [LSARC/2009/606 FastTrack timeout 11/13/2009] In-reply-to: <4AFBB334.8080106@Sun.COM> Sender: Darren.Moffat@sun.com To: Amit.Gupta@sun.com Cc: webstack-discuss@opensolaris.org, LSARC-ext@sun.com, Jyri Virkki Message-id: <4AFC3BB4.7070806@Sun.COM> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200911060916.nA69Gaq2018611@sac.sfbay.sun.com> <4AF3EFB9.1060403@Sun.COM> <4AF4583F.8000105@Sun.COM> <4AFAE65A.7020105@Sun.COM> <4AFAEEA3.8070104@Sun.COM> <4AFBB334.8080106@Sun.COM> User-Agent: Thunderbird 2.0.0.23 (X11/20091013) Status: RO Content-Length: 615 Amit Gupta wrote: > On 11/11/09 22:34, Darren J Moffat wrote: >> The filelayout looks fine. >> >> What uid/gid and privileges does SMF start collectd with ? (ie what is >> the method credential section in the SMF manifest). > SMF will start collectd as root. Why does collectd need to run as root ? What stats is it that is collectd needs that requires it to run with root ? It really shouldn't need to. > Regards > Amit >> Hopefully this need to run as root with all privileges - ideally it >> should run as daemon with basic privileges (or less). I missed a "not" in that sentence. -- Darren J Moffat From Darren.Moffat@sun.com Fri Nov 13 13:29:15 2009 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nACIqw5f015987 for ; Thu, 12 Nov 2009 10:52:58 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id nACIqvPK004643 for <@sunmail2sca.sfbay.sun.com:LSARC-ext@sun.com>; Thu, 12 Nov 2009 10:52:58 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KT000C17F494B00@nwk-avmta-2.sfbay.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Thu, 12 Nov 2009 10:52:57 -0800 (PST) Received: from gmp-eb-inf-1.sun.com ([192.18.6.21]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KT00087WF48U740@nwk-avmta-2.sfbay.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Thu, 12 Nov 2009 10:52:57 -0800 (PST) Received: from fe-emea-10.sun.com (gmp-eb-lb-1-fe1.eu.sun.com [192.18.6.7] (may be forged)) by gmp-eb-inf-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id nACIqtej024899 for ; Thu, 12 Nov 2009 18:52:56 +0000 (GMT) Received: from conversion-daemon.fe-emea-10.sun.com by fe-emea-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KT000K00ET4GR00@fe-emea-10.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Thu, 12 Nov 2009 18:52:37 +0000 (GMT) Received: from [10.0.189.215] ([unknown] [192.18.41.196]) by fe-emea-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KT000C3SF3ML6E0@fe-emea-10.sun.com>; Thu, 12 Nov 2009 18:52:37 +0000 (GMT) Date: Thu, 12 Nov 2009 10:52:35 -0800 From: Darren J Moffat Subject: Re: [webstack-discuss] Deliver collectd 4.8.x into OpenSolaris [LSARC/2009/606 FastTrack timeout 11/13/2009] In-reply-to: <4AFC5420.7020202@sun.com> Sender: Darren.Moffat@sun.com To: "amit.gupta@sun.com" Cc: webstack-discuss@opensolaris.org, LSARC-ext@sun.com, Jyri Virkki Message-id: <4AFC5973.4070504@Sun.COM> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200911060916.nA69Gaq2018611@sac.sfbay.sun.com> <4AF3EFB9.1060403@Sun.COM> <4AF4583F.8000105@Sun.COM> <4AFAE65A.7020105@Sun.COM> <4AFAEEA3.8070104@Sun.COM> <4AFBB334.8080106@Sun.COM> <4AFC3BB4.7070806@Sun.COM> <4AFC5420.7020202@sun.com> User-Agent: Thunderbird 2.0.0.23 (X11/20091013) Status: RO Content-Length: 770 amit.gupta@sun.com wrote: > Darren J Moffat wrote: >> Amit Gupta wrote: >>> On 11/11/09 22:34, Darren J Moffat wrote: >>>> The filelayout looks fine. >>>> >>>> What uid/gid and privileges does SMF start collectd with ? (ie what >>>> is the method credential section in the SMF manifest). >>> SMF will start collectd as root. >> >> Why does collectd need to run as root ? >> >> What stats is it that is collectd needs that requires it to run with >> root ? It really shouldn't need to. > I don't see any issues running collectd as a non root user, however, I > am not certain if any of the collectd plugins require collectd to run as > root. What user do you suggest collectd to run as? user daemon, group daemon and the basic privilege set. -- Darren J Moffat From Amit.Gupta@sun.com Fri Nov 13 13:29:21 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nACIU2qG015703 for ; Thu, 12 Nov 2009 10:30:03 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id nACITr1A006144 for <@sunmail2sca.sfbay.sun.com:LSARC-ext@sun.com>; Fri, 13 Nov 2009 02:30:01 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KT000B2VE20A400@brm-avmta-1.central.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Thu, 12 Nov 2009 11:30:00 -0700 (MST) Received: from sineb-mail-2.sun.com ([192.18.19.7]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KT00065ME1WZB20@brm-avmta-1.central.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Thu, 12 Nov 2009 11:29:58 -0700 (MST) Received: from fe-apac-05.sun.com (fe-apac-05.sun.com [192.18.19.176] (may be forged)) by sineb-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id nACITuQt010606 for ; Thu, 12 Nov 2009 18:29:56 +0000 (GMT) Received: from conversion-daemon.mail-apac.sun.com by mail-apac.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KT000E00DNQI100@mail-apac.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Fri, 13 Nov 2009 02:29:56 +0800 (SGT) Received: from [192.168.1.34] ([unknown] [122.167.234.175]) by mail-apac.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KT0006D7E1SZS50@mail-apac.sun.com>; Fri, 13 Nov 2009 02:29:56 +0800 (SGT) Date: Thu, 12 Nov 2009 23:59:52 +0530 From: "amit.gupta@sun.com" Subject: Re: [webstack-discuss] Deliver collectd 4.8.x into OpenSolaris [LSARC/2009/606 FastTrack timeout 11/13/2009] In-reply-to: <4AFC3BB4.7070806@Sun.COM> Sender: Amit.Gupta@sun.com To: Darren J Moffat Cc: webstack-discuss@opensolaris.org, LSARC-ext@sun.com, Jyri Virkki Message-id: <4AFC5420.7020202@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200911060916.nA69Gaq2018611@sac.sfbay.sun.com> <4AF3EFB9.1060403@Sun.COM> <4AF4583F.8000105@Sun.COM> <4AFAE65A.7020105@Sun.COM> <4AFAEEA3.8070104@Sun.COM> <4AFBB334.8080106@Sun.COM> <4AFC3BB4.7070806@Sun.COM> User-Agent: Thunderbird 2.0.0.23 (Windows/20090812) Status: RO Content-Length: 855 Darren J Moffat wrote: > Amit Gupta wrote: >> On 11/11/09 22:34, Darren J Moffat wrote: >>> The filelayout looks fine. >>> >>> What uid/gid and privileges does SMF start collectd with ? (ie what >>> is the method credential section in the SMF manifest). >> SMF will start collectd as root. > > Why does collectd need to run as root ? > > What stats is it that is collectd needs that requires it to run with > root ? It really shouldn't need to. I don't see any issues running collectd as a non root user, however, I am not certain if any of the collectd plugins require collectd to run as root. What user do you suggest collectd to run as? Regards Amit > >> Regards >> Amit >>> Hopefully this need to run as root with all privileges - ideally it >>> should run as daemon with basic privileges (or less). > > I missed a "not" in that sentence. > From storycrafter@gmail.com Fri Nov 13 13:33:43 2009 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nACEhgwg009353 for ; Thu, 12 Nov 2009 06:43:42 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id nACEhdF0017523; Thu, 12 Nov 2009 06:43:40 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KT000L0T3KRKA00@nwk-avmta-1.sfbay.Sun.COM>; Thu, 12 Nov 2009 06:43:39 -0800 (PST) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KT00079R3KQ7P50@nwk-avmta-1.sfbay.Sun.COM>; Thu, 12 Nov 2009 06:43:39 -0800 (PST) Received: from relay15i.sun.com (ip125.net129179-4.block1.us.syntegra.com [129.179.4.125]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id nACENOxJ027355; Thu, 12 Nov 2009 14:43:38 +0000 (GMT) Received: from mmp11es.mmp.us.syntegra.com ([160.41.208.11] [160.41.208.11]) by relay15i.sun.com with ESMTP id BT-MMP-2210158; Thu, 12 Nov 2009 14:43:38 +0000 (Z) Received: from relay15i.sun.com (relay15i.sun.com [129.179.4.125]) by mmp11es.mmp.us.syntegra.com with ESMTP id BT-MMP-10502650; Thu, 12 Nov 2009 14:43:38 +0000 (Z) Received: from gv-out-0910.google.com ([216.239.58.187] [216.239.58.187]) by relay1i.sun.com with ESMTP id BT-MMP-27828275; Thu, 12 Nov 2009 14:43:38 +0000 (Z) Received: by gv-out-0910.google.com with SMTP id l14so276672gvf.8 for ; Thu, 12 Nov 2009 06:43:33 -0800 (PST) Received: by 10.216.86.9 with SMTP id v9mr948487wee.148.1258037012111; Thu, 12 Nov 2009 06:43:32 -0800 (PST) Date: Thu, 12 Nov 2009 08:43:32 -0600 From: Mark Martin Subject: Re: [webstack-discuss] Deliver collectd 4.8.x into OpenSolaris [LSARC/2009/606 FastTrack timeout 11/13/2009] In-reply-to: <20091112033215.GA3636@sun.com> To: Jyri Virkki Cc: Amit Gupta , webstack-discuss@opensolaris.org, LSARC-ext@sun.com Message-id: MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1 Content-transfer-encoding: 7BIT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=O1BiAcqxrd6XRBIm0mH1BicaEU7cZlX0bcxq5N/aB7k=; b=tq3YkB7je2eW/UoHg2CE2R3jGq/rdmobPlYusxe7I+EDKwW0d7qc5ApoSSEqh2LCWD lCK5F7MpmGFKzdRR4+BKIdsz044msf5nz7S7E+9RTHSpu+PQ/R8Z+HLa2wwO5sv973ro 6NQq4fg4Nuc5h1/VhcGDgyyuHmr5O11KUFOmw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=CbmgVnKh7T+p6mjaBbDXQnkZH5Ntp8LarNbIY36qeNgQ6JM64xUmJK59VJ6lv7TKnV 1EE1AGHUA3oDqLtD3wPjNXHi+7QGp7qw9ML/eNdCvYHeF00Mv3FG7/le28T7f1MzxAp1 t/RE19ImLeE1PZSUoQX3heDu0xzi6ZOcLAnl0= X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== References: <200911060916.nA69Gaq2018611@sac.sfbay.sun.com> <4AF3EFB9.1060403@Sun.COM> <4AF4583F.8000105@Sun.COM> <4AFAE65A.7020105@Sun.COM> <20091112033215.GA3636@sun.com> Status: RO Content-Length: 3097 On Wed, Nov 11, 2009 at 9:32 PM, Jyri Virkki wrote: > The key decision to make is whether it is likely that collectd 4.x and > [a future] collectd 5.x may both be needed in the same release of Solaris. > > There is no blanket answer to that unfortunately. At best, with enough > familiarity with the upstream community, their future direction and > the future direction of other components which use collectd, you can > make an educated prediction. > > Given the upstream community has stated their major releases are > incompatible it suggests coexistence might be needed. AFAIK there is > no version 5 yet but looking back, looks like they still support 3.x > (in maintenance mode) along with 4.x because the two are incompatible > (http://collectd.org/download.shtml), so it is not unreasonable to > imagine 4.x and 5.x may have a similar coexistence. > > On the down side, versioning always adds a bit of clutter and > inconvenience unless needed. > > Think about it and decide whether to support collectd version > coexistence or not. > > If anyone who is interested in collectd has a preference, speak up now. My personal preference would be to take the conservative approach and plan on accommodating it (major version coexistence). You've given some history that suggests that it is more likely than not -- yesterday's weather is tomorrow's forecast and all that. > > Once you decide, make the layout fully reflect the decision. Either it > needs to allow coexistence everywhere, or not. > > > jv02: Iff version coexistence is chosen, I was hoping for ARC members > to have opinions on $COMPONENT$VERSION/* vs. $COMPONENT/$VERSION/* > directory naming (see earlier email). If there are no opinions, > project team can do as they wish. > With the qualification that I'm the newbie outsider member, I'll offer my preference for the $COMPONENT/$VERSION notation as a general guideline. As you mentioned in an earlier email, $COMPONENT$VERSION/ is mostly a historical artifact. It seems arbitrary to me either way, but for what it's worth, I'd rather not see version information as a decorator at the topmost level. Every year 100's of component names are created in the FOSS world (that's a guess), so the chance of creating "clutter" in /usr/ is an upwardly trending risk to my mind. > >> I am not sure as yet if we need to deliver only one collectd package i.e >> SUNWcollectd?. As I mentioned earlier, I have followed the lead of >> apache, lighttpd. mysql etc. all of which deliver a separate root and >> usr package. > > Since you are integrating into sfw you will be required to integrate > both usr and root packages. > > We also know they'll get renamed several times in ways outside of the > project team control, so no point worrying about it until OpenSolaris > gets its ARC act together. Call them Volatile, if you like. > > This is where I agree with you most strongly, Jyri. To my mind, we can't honestly ignore the stability level on these package names/structure. Volatile seems appropriate as a stability level until something changes that. From Amit.Gupta@sun.com Fri Nov 13 13:40:51 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nADEZOIP021799 for ; Fri, 13 Nov 2009 06:35:24 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id nADEZJJF022942 for <@sunmail2sca.sfbay.sun.com:LSARC-ext@sun.com>; Fri, 13 Nov 2009 22:35:23 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KT100101XUX6700@brm-avmta-1.central.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Fri, 13 Nov 2009 07:35:21 -0700 (MST) Received: from gmp-eb-inf-1.sun.com ([192.18.6.21]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KT1000EZXUSQ900@brm-avmta-1.central.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Fri, 13 Nov 2009 07:35:18 -0700 (MST) Received: from fe-emea-09.sun.com (gmp-eb-lb-1-fe1.eu.sun.com [192.18.6.7] (may be forged)) by gmp-eb-inf-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id nADEZGss026152 for ; Fri, 13 Nov 2009 14:35:16 +0000 (GMT) Received: from conversion-daemon.fe-emea-09.sun.com by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KT100A00XO1MF00@fe-emea-09.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Fri, 13 Nov 2009 14:35:00 +0000 (GMT) Received: from test.sun.com ([unknown] [129.158.228.93]) by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KT100B88XU0JDC0@fe-emea-09.sun.com>; Fri, 13 Nov 2009 14:34:53 +0000 (GMT) Date: Fri, 13 Nov 2009 19:51:47 +0530 From: Amit Gupta Subject: Re: [webstack-discuss] Deliver collectd 4.8.x into OpenSolaris [LSARC/2009/606 FastTrack timeout 11/13/2009] In-reply-to: <4AFC5973.4070504@Sun.COM> Sender: Amit.Gupta@sun.com To: Darren J Moffat Cc: webstack-discuss@opensolaris.org, LSARC-ext@sun.com, Jyri Virkki Reply-to: Amit.Gupta@sun.com Message-id: <4AFD6B7B.706@Sun.COM> MIME-version: 1.0 Content-type: multipart/mixed; boundary="Boundary_(ID_E5lUmjSqPvtLq5GDks9X8g)" X-PMX-Version: 5.4.1.325704 References: <200911060916.nA69Gaq2018611@sac.sfbay.sun.com> <4AF3EFB9.1060403@Sun.COM> <4AF4583F.8000105@Sun.COM> <4AFAE65A.7020105@Sun.COM> <4AFAEEA3.8070104@Sun.COM> <4AFBB334.8080106@Sun.COM> <4AFC3BB4.7070806@Sun.COM> <4AFC5420.7020202@sun.com> <4AFC5973.4070504@Sun.COM> User-Agent: Thunderbird 2.0.0.4 (X11/20070622) Status: RO Content-Length: 15596 This is a multi-part message in MIME format. --Boundary_(ID_E5lUmjSqPvtLq5GDks9X8g) Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT Do find the updated ARC case attached. I have added a new section "Lifecycle" which specifies that collectd will run as daemon:daemon. Regards Amit On 11/13/09 00:22, Darren J Moffat wrote: > amit.gupta@sun.com wrote: >> Darren J Moffat wrote: >>> Amit Gupta wrote: >>>> On 11/11/09 22:34, Darren J Moffat wrote: >>>>> The filelayout looks fine. >>>>> >>>>> What uid/gid and privileges does SMF start collectd with ? (ie >>>>> what is the method credential section in the SMF manifest). >>>> SMF will start collectd as root. >>> >>> Why does collectd need to run as root ? >>> >>> What stats is it that is collectd needs that requires it to run with >>> root ? It really shouldn't need to. >> I don't see any issues running collectd as a non root user, however, >> I am not certain if any of the collectd plugins require collectd to >> run as root. What user do you suggest collectd to run as? > > user daemon, group daemon and the basic privilege set. > --Boundary_(ID_E5lUmjSqPvtLq5GDks9X8g) Content-type: text/plain; name=collectd-arc.txt Content-transfer-encoding: 7BIT Content-disposition: inline; filename=collectd-arc.txt Template Version: @(#)sac_nextcase 1.68 02/23/09 SMI This information is Copyright 2009 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: Deliver collectd 4.8.x into OpenSolaris 1.2. Name of Document Author/Supplier: Author: Amit Gupta 1.3 Date of This Document: 06 November, 2009 4. Technical Description Deliver collectd 4.8.x into OpenSolaris 13 November 2009 1. Summary and motivation 1.1. Introduction This project delivers collectd 4.8.x into OpenSolaris collectd is a daemon which collects system performance statistics periodically and provides mechanisms to store the values in a variety of ways, for example in RRD files. For more information about collectd see: http://collectd.org 2. Technical description 2.1. Key objects /usr/lib/collectd4/collectd From collectd's man page: collectd is a daemon that receives system statistics and makes them available in a number of ways. The main daemon itself doesn't have any real functionality apart from loading, querying and submitting to plugins. /usr/lib/collectd4/collectdmon From collectdmon's man page: collectdmon is a small ``wrapper'' daemon which starts and monitors the collectd daemon. If collectd terminates it will automatically be restarted, unless collectdmon was told to shut it down. Note that collectdmon is not require if collectd is started via SMF. However, there may be products which embed collectd and may want to manage lifecyle of collectd via collectmon. /etc/collectd4/collectd.conf The default collectd configuration file. 2.2. Versioning From collectd's FAQ: The version numbers in collectd consist of three numbers: The major- and minor-number and the patchlevel. * Versions with different major-numbers are basically not compatible. This means that the definitions of RRD-files or config-options have been changed or, in general, that the user has to do something in addition to install the new version. This is not nice and avoided when possible, but sometimes necessary to prevent old mistakes to become ancient mistakes. * Versions with differing minor-numbers are backwards compatible, i. e. you can replace the lower version with the higher one and everything should still work. This means that features are added, but not removed or changed and that the default behavior does not change. * Versions with different patchlevels are both, forward- and backwards-compatible, because no new features have been introduced. The only difference between the two versions is one or more bug fixes, so you should generally install the higher version of the two. 2.3 Configuration File This project will use the standard convention and deliver configuration files under /etc/collectd4 The default configuration file provided with this integration will work out of the box. In addition, a directory conf.d will be created in /etc/collectd4/ The default configuration loads any file with a .conf extension that is placed in this directory. Hence, plugins can optionally be configured by simply dropping a .conf file into this directory without modifying the main configuration file. By default, conf.d will not contain any plugin configuration files. 2.4 Log File The log File will be located in /var/collectd4/logs and will be named collectd.log. The default log level is info and can be changed by modifying the main configuration file. 2.5 Lifecycle collectd will be started as a SMF service and will run as user "daemon" and group "daemon". 2.6 Loadable Plugins collectd ships with a number of loadable plugins which are enabled through directives in the config file. Some of these plugins have library dependencies that must be satisfied through build time package options (supplied to the configure script). An example is mysql.so which has a dependency on the mysql client libraries. Support for loadable plugins and other features is enabled at build time through flags to the configure script. All the plugins that are delivered have sample configuration directives in collectd.conf. Note that in many cases, these are just sample directives and will need to be modified based on site specific configuration. Detailed documentation about specific plugin configuration can be found in the collectd website. Not all the plugins are enabled by default in collectd.conf. The configuration directives for plugins which are not enabled are commented out in collectd.conf. The plugins which are enabled by default are: syslog, cpu, interface, load, memory and rrdtool Other plugins can be enabled in one of the following two ways: * By adding/uncommenting the plugin configuration directives in collectd.conf * By dropping a .conf file having the necessary plugin configuration directives into the /etc/collectd4/conf.d Similarly, plugins can be disabled as needed in the following two ways: * By removing/commenting out the plugin configuration directives from collectd.conf * By deleting the plugin specific .conf file from the /etc/collectd4/conf.d directory Do note that collectd.conf is read and processed in order, i. e. from top to bottom. So the plugins are loaded in the order listed in collectd.conf. This project will deliver the following plugins: * Apache * CPU * CSV * cURL * DF * Disk * Exec * FileCount * GenericJMX * Interface * Java * Load * LogFile * memcachec * memcached * Memory * MySQL * Network * Notify Desktop * NTPd * PostgreSQL * RRDtool * SNMP * Swap * SysLog * Table * Tail * Tape * UnixSock * Uptime * Users * Write HTTP * ZFS ARC 2.7 Directory Naming and Structure The proposed directory layout for collectd is: /usr/lib/collectd4 /etc/collectd4 /conf.d /var/collectd4 /rrd /logs /usr/share/collectd4 /usr/include/collectd The detailed directory and file layout for collectd is provided in Appendix 1. Multiple versions can coexist, and are distinguished by the naming scheme (collectd4 for version 4.*.*, collectd5 for version 5.*.*). 2.8 Build features Build time package selection determines which collectd loadable plugins are available to the user at runtime. A complete list of collectd modules is available on the http://collectd.org. However, not all plugins can be built on OpenSolaris because of dependencies on external libraries and hence, only a subset of the plugins will be delivered. 3. collectd documentation collectd 4.* provides a full and complete documentation set, in UNIX manual format. This project will deliver the complete documentation set. 4. Packaging and Delivery We propose to package collectd under the following packages: SUNWcollectd4u - [usr] Server package (including collectd daemon, loadable plugins, man pages) SUNWcollectd4r - [root] (config files, smf files) 5. collectd Interfaces 5.1. Interface Stability collectd has no obvious history of any interface instability and the expectation is that dot releases of collectd will be compatible. This is not completely guaranteed by the community and it is possible that interfaces may change due to the fixing of bugs. 5.2. Imported Interfaces libcurl Volatile PSARC/2007/165 Output format of Apache's mod_status N/A PSARC/2007/586 Output format of Lighhtpd's mod_status N/A LSARC/2008/191 libmysqlclient Committed LSARC 2009/062 libnetsnmp Uncommitted LSARC/2008/355 librrd Uncommitted LSARC/2008/129 libpq Uncommitted LSARC/2009/349 Java SE 6 Committed PSARC/2006/430 libkstat Uncommitted PSARC/1992/085 kstat_read Stable PSARC/1992/085 getmntent Evolving PSARC/1999/131 getloadavg Standard PSARC/1996/196 libmemcached Uncommitted LSARC/2008/126 libgcrypt Volatile LSARC/2008/354 libnotify Volatile LSARC/2007/137 swapctl Stable PSARC/1997/099 libhal Volatile PSARC/2005/399 getutxent N/A N/A syslog Standard N/A statvfs Stable N/A 5.3. Exported Interfaces NAME STABILITY NOTES ----------------------------------------------------------------------- SUNWcollectd4r Committed package SUNWcollectd4u Committed package collectd config file Uncommitted config file syntax collectd log file Not An Interface log file syntax collectd pid file Project Private daemon pid file /var/collectd4/logs Uncommitted log file dir /etc/collectd4/collectd.conf Uncommitted default conf file /usr/lib/collectd4/*.so Project Private plugins /usr/include/collectd Uncommitted Include directory /lib/svc/method/collectd4-svc Project Private SMF svc method svc:/application/monitoring/collectd4:default Committed FMRI /var/svc/manifest/application/monitoring/collectd4.xml Project Private SMF Manifest 6. References [1] http://collectd.org ================================================================ Appendix 1: collectd Integration Directory and File Structure. 1. The following files are included in the collectd integration: /usr/lib/collectd4 collectd collectdmon apache.so bind.so cpu.so csv.so curl.so df.so disk.so exec.so filecount.so interface.so java.so load.so logfile.so match_regex.so match_timediff.so match_value.so memcached.so memory.so mysql.so network.so notify_desktop.so ntpd.so postgresql.so rrdtool.so snmp.so syslog.so swap.so table.so tail.so tape.so target_notification.so target_replace.so target_set.so ted.so unixsock.so uptime.so users.so libcollectdclient.so.0.0.0 libcollectdclient.so.0 -> libcollectdclient.so.0.0.0 libcollectdclient.so -> libcollectdclient.so.0.0.0 /etc/collectd4 /conf.d collectd.conf /var/collectd4 /logs collectd.log /rrd /usr/share/collectd4 types.db postgresql_default.conf java-plugin.jar jmx.jar /man /man1 collectd.1 collectdmon.1 /man5 collectd.conf.5 collectd-exec.conf collectd-unixsock.5 collectd-java.5 collectd-snmp.5 types.db.5 /usr/include/collectd4 client.h lcc_features.h /lib/svc/method/collectd4-svc /var/svc/manifest/application/monitoring/collectd4.xml 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: sfw 6.5. ARC review type: FastTrack 6.6. ARC Exposure: open --Boundary_(ID_E5lUmjSqPvtLq5GDks9X8g)-- From Amit.Gupta@sun.com Fri Nov 13 13:41:08 2009 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nACFZROw010098 for ; Thu, 12 Nov 2009 07:35:27 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id nACFZOjV013835 for <@sunmail2sca.sfbay.sun.com:LSARC-ext@sun.com>; Thu, 12 Nov 2009 07:35:26 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KT000H375Z1E400@brm-avmta-1.central.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Thu, 12 Nov 2009 08:35:25 -0700 (MST) Received: from gmp-eb-inf-2.sun.com ([192.18.6.24]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KT000D3K5YZ3D40@brm-avmta-1.central.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Thu, 12 Nov 2009 08:35:24 -0700 (MST) Received: from fe-emea-10.sun.com (gmp-eb-lb-1-fe1.eu.sun.com [192.18.6.7] (may be forged)) by gmp-eb-inf-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id nACFZNYQ016033 for ; Thu, 12 Nov 2009 15:35:23 +0000 (GMT) Received: from conversion-daemon.fe-emea-10.sun.com by fe-emea-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KT000M005QGX000@fe-emea-10.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Thu, 12 Nov 2009 15:35:09 +0000 (GMT) Received: from test.sun.com ([unknown] [129.158.228.93]) by fe-emea-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KT000CBE5YHL680@fe-emea-10.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Thu, 12 Nov 2009 15:35:08 +0000 (GMT) Date: Thu, 12 Nov 2009 20:52:06 +0530 From: Amit Gupta Subject: Re: [webstack-discuss] Deliver collectd 4.8.x into OpenSolaris [LSARC/2009/606 FastTrack timeout 11/13/2009] In-reply-to: <20091112033215.GA3636@sun.com> Sender: Amit.Gupta@sun.com To: Jyri Virkki Cc: webstack-discuss@opensolaris.org, LSARC-ext@sun.com Reply-to: Amit.Gupta@sun.com Message-id: <4AFC281E.7050407@Sun.COM> MIME-version: 1.0 Content-type: multipart/mixed; boundary="Boundary_(ID_1rBfliKqLVG9HwIBR7q4aA)" X-PMX-Version: 5.4.1.325704 References: <200911060916.nA69Gaq2018611@sac.sfbay.sun.com> <4AF3EFB9.1060403@Sun.COM> <4AF4583F.8000105@Sun.COM> <4AFAE65A.7020105@Sun.COM> <20091112033215.GA3636@sun.com> User-Agent: Thunderbird 2.0.0.4 (X11/20070622) Status: RO Content-Length: 17868 This is a multi-part message in MIME format. --Boundary_(ID_1rBfliKqLVG9HwIBR7q4aA) Content-type: multipart/alternative; boundary="Boundary_(ID_mZwC6Sl7H8ElbBC/U18j7g)" --Boundary_(ID_mZwC6Sl7H8ElbBC/U18j7g) Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT On 11/12/09 09:02, Jyri Virkki wrote: > Given the upstream community has stated their major releases are > incompatible it suggests coexistence might be needed. AFAIK there is > no version 5 yet but looking back, looks like they still support 3.x > (in maintenance mode) along with 4.x because the two are incompatible > (http://collectd.org/download.shtml), so it is not unreasonable to > imagine 4.x and 5.x may have a similar coexistence. > > On the down side, versioning always adds a bit of clutter and > inconvenience unless needed. > > Think about it and decide whether to support collectd version > coexistence or not. > I will assume that multiple versions will co-exist. I have updated the attached ARC case accordingly. > >> - included collectdmon as suggested. >> > > jv03: However, if you do, then document how the collectd smf service > and collectdmon can (and can't) interact. In particular, in what ways > can things go confusingly wrong if customer attempts to use both? The > collectdmon man page is a good place to insert this information. > While collectdmon is not required if collectd is started via SMF, there may be products which embed collectd and may want to manage lifecyle of collectd via collectdmon. Regards Amit --Boundary_(ID_mZwC6Sl7H8ElbBC/U18j7g) Content-type: text/html; CHARSET=US-ASCII Content-transfer-encoding: 7BIT On 11/12/09 09:02, Jyri Virkki wrote:
Given the upstream community has stated their major releases are
incompatible it suggests coexistence might be needed. AFAIK there is
no version 5 yet but looking back, looks like they still support 3.x
(in maintenance mode) along with 4.x because the two are incompatible
(http://collectd.org/download.shtml), so it is not unreasonable to
imagine 4.x and 5.x may have a similar coexistence.

On the down side, versioning always adds a bit of clutter and
inconvenience unless needed.

Think about it and decide whether to support collectd version
coexistence or not.
I will assume that multiple versions will co-exist. I have updated the attached ARC case accordingly.
  
- included collectdmon as suggested.
    

jv03: However, if you do, then document how the collectd smf service
and collectdmon can (and can't) interact.  In particular, in what ways
can things go confusingly wrong if customer attempts to use both?  The
collectdmon man page is a good place to insert this information.
While collectdmon is not required if collectd is started via SMF, there may be products which embed collectd and may want to manage lifecyle of collectd via collectdmon.

Regards
Amit --Boundary_(ID_mZwC6Sl7H8ElbBC/U18j7g)-- --Boundary_(ID_1rBfliKqLVG9HwIBR7q4aA) Content-type: text/plain; name=collectd-arc.txt Content-transfer-encoding: 7BIT Content-disposition: inline; filename=collectd-arc.txt Template Version: @(#)sac_nextcase 1.68 02/23/09 SMI This information is Copyright 2009 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: Deliver collectd 4.8.x into OpenSolaris 1.2. Name of Document Author/Supplier: Author: Amit Gupta 1.3 Date of This Document: 06 November, 2009 4. Technical Description Deliver collectd 4.8.x into OpenSolaris 12 November 2009 1. Summary and motivation 1.1. Introduction This project delivers collectd 4.8.x into OpenSolaris collectd is a daemon which collects system performance statistics periodically and provides mechanisms to store the values in a variety of ways, for example in RRD files. For more information about collectd see: http://collectd.org 2. Technical issues 2.1. Key objects /usr/lib/collectd4/collectd From collectd's man page: collectd is a daemon that receives system statistics and makes them available in a number of ways. The main daemon itself doesn't have any real functionality apart from loading, querying and submitting to plugins. /usr/lib/collectd4/collectdmon From collectdmon's man page: collectdmon is a small ``wrapper'' daemon which starts and monitors the collectd daemon. If collectd terminates it will automatically be restarted, unless collectdmon was told to shut it down. Note that collectdmon is not require if collectd is started via SMF. However, there may be products which embed collectd and may want to manage lifecyle of collectd via collectmon. /etc/collectd4/collectd.conf The default collectd configuration file. 2.2. Versioning From collectd's FAQ: The version numbers in collectd consist of three numbers: The major- and minor-number and the patchlevel. * Versions with different major-numbers are basically not compatible. This means that the definitions of RRD-files or config-options have been changed or, in general, that the user has to do something in addition to install the new version. This is not nice and avoided when possible, but sometimes necessary to prevent old mistakes to become ancient mistakes. * Versions with differing minor-numbers are backwards compatible, i. e. you can replace the lower version with the higher one and everything should still work. This means that features are added, but not removed or changed and that the default behavior does not change. * Versions with different patchlevels are both, forward- and backwards-compatible, because no new features have been introduced. The only difference between the two versions is one or more bug fixes, so you should generally install the higher version of the two. 2.3 Loadable Plugins collectd ships with a number of loadable plugins which are enabled through directives in the config file. Some of these plugins have library dependencies that must be satisfied through build time package options (supplied to the configure script). An example is mysql.so which has a dependency on the mysql client libraries. Support for loadable plugins and other features is enabled at build time through flags to the configure script. All the plugins that are delivered have sample configuration directives in collectd.conf. Note that in many cases, these are just sample directives and will need to be modified based on site specific configuration. Detailed documentation about specific plugin configuration can be found in the collectd website. Not all the plugins are enabled by default in collectd.conf. The configuration directives for plugins which are not enabled are commented out in collectd.conf. The plugins which are enabled by default are: syslog, cpu, interface, load, memory and rrdtool Other plugins can be enabled in one of the following two ways: * By adding/uncommenting the plugin configuration directives in collectd.conf * By dropping a .conf file having the necessary plugin configuration directives into the /etc/collectd4/conf.d Similarly, plugins can be disabled as needed in the following two ways: * By removing/commenting out the plugin configuration directives from collectd.conf * By deleting the plugin specific .conf file from the /etc/collectd4/conf.d directory Do note that collectd.conf is read and processed in order, i. e. from top to bottom. So the plugins are loaded in the order listed in collectd.conf. This project will deliver the following plugins: * Apache * CPU * CSV * cURL * DF * Disk * Exec * FileCount * GenericJMX * Interface * Java * Load * LogFile * memcachec * memcached * Memory * MySQL * Network * Notify Desktop * NTPd * PostgreSQL * RRDtool * SNMP * Swap * SysLog * Table * Tail * Tape * UnixSock * Uptime * Users * UUID * Write HTTP * ZFS ARC 2.4 Directory Naming and Structure The proposed directory layout for collectd is: /usr/lib/collectd4 /etc/collectd4 /conf.d /var/collectd4 /rrd /logs /usr/share/collectd4 /usr/include/collectd The detailed directory and file layout for collectd is provided in Appendix 1. Multiple versions can coexist, and are distinguished by the naming scheme (collectd4 for version 4.*.*, collectd5 for version 5.*.*). 2.5 Configuration File This project will use the standard convention and deliver configuration files under /etc/collectd4 The default configuration file provided with this integration will work out of the box. In addition, a directory conf.d will be created in /etc/collectd4/ The default configuration loads any file with a .conf extension that is placed in this directory. Hence, plugins can optionally be configured by simply dropping a .conf file into this directory without modifying the main configuration file. By default, conf.d will not contain any plugin configuration files. 2.6 Log File The log File will be located in /var/collectd4/logs and will be named collectd.log. The default log level is info and can be changed by modifying the main configuration file. 2.7 Build features Build time package selection determines which collectd loadable plugins are available to the user at runtime. A complete list of collectd modules is available on the http://collectd.org. However, not all plugins can be built on OpenSolaris because of dependencies on external libraries and hence, only a subset of the plugins will be delivered. 3. collectd documentation collectd 4.* provides a full and complete documentation set, in UNIX manual format. This project will deliver the complete documentation set. 4. Packaging and Delivery We propose to package collectd under the following packages: SUNWcollectd4u - [usr] Server package (including collectd daemon, loadable plugins, man pages) SUNWcollectd4r - [root] (config files, smf files) 5. collectd Interfaces 5.1. Interface Stability collectd has no obvious history of any interface instability and the expectation is that dot releases of collectd will be compatible. This is not completely guaranteed by the community and it is possible that interfaces may change due to the fixing of bugs. 5.2. Imported Interfaces libcurl Volatile PSARC/2007/165 Output format of Apache's mod_status N/A PSARC/2007/586 Output format of Lighhtpd's mod_status N/A LSARC/2008/191 libmysqlclient Committed LSARC 2009/062 libnetsnmp Uncommitted LSARC/2008/355 librrd Uncommitted LSARC/2008/129 libpq Uncommitted LSARC/2009/349 Java SE 6 Committed PSARC/2006/430 libkstat Uncommitted PSARC/1992/085 kstat_read Stable PSARC/1992/085 getmntent Evolving PSARC/1999/131 getloadavg Standard PSARC/1996/196 libmemcached Uncommitted LSARC/2008/126 libgcrypt Volatile LSARC/2008/354 libnotify Volatile LSARC/2007/137 swapctl Stable PSARC/1997/099 libhal Volatile PSARC/2005/399 getutxent N/A N/A syslog Standard N/A statvfs Stable N/A 5.3. Exported Interfaces NAME STABILITY NOTES ----------------------------------------------------------------------- SUNWcollectd4r Committed package SUNWcollectd4u Committed package collectd config file Uncommitted config file syntax collectd log file Not An Interface log file syntax collectd pid file Project Private daemon pid file /var/collectd4/logs Uncommitted log file dir /etc/collectd4/collectd.conf Uncommitted default conf file /usr/lib/collectd4/*.so Project Private plugins /usr/include/collectd Uncommitted Include directory /lib/svc/method/collectd4-svc Project Private SMF svc method svc:/application/monitoring/collectd4:default Committed FMRI /var/svc/manifest/application/monitoring/collectd4.xml Project Private SMF Manifest 6. References [1] http://collectd.org ================================================================ Appendix 1: collectd Integration Directory and File Structure. 1. The following files are included in the collectd integration: /usr/lib/collectd4 collectd collectdmon apache.so bind.so cpu.so csv.so curl.so df.so disk.so exec.so filecount.so interface.so java.so load.so logfile.so match_regex.so match_timediff.so match_value.so memcached.so memory.so mysql.so network.so ntpd.so postgresql.so rrdtool.so snmp.so syslog.so swap.so table.so tail.so tape.so target_notification.so target_replace.so target_set.so ted.so unixsock.so uptime.so users.so uuid.so libcollectdclient.so.0.0.0 libcollectdclient.so.0 -> libcollectdclient.so.0.0.0 libcollectdclient.so -> libcollectdclient.so.0.0.0 /etc/collectd4 /conf.d collectd.conf /var/collectd4 /logs collectd.log /rrd /usr/share/collectd4 types.db postgresql_default.conf java-plugin.jar jmx.jar /man /man1 collectd.1 collectdmon.1 /man5 collectd.conf.5 collectd-exec.conf collectd-unixsock.5 collectd-java.5 collectd-snmp.5 types.db.5 /usr/include/collectd4 client.h lcc_features.h /lib/svc/method/collectd4-svc /var/svc/manifest/application/monitoring/collectd4.xml 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: sfw 6.5. ARC review type: FastTrack 6.6. ARC Exposure: open --Boundary_(ID_1rBfliKqLVG9HwIBR7q4aA)-- From Jyri.Virkki@sun.com Fri Nov 13 15:56:44 2009 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nADNui4N005940 for ; Fri, 13 Nov 2009 15:56:44 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id nADNuiVX020937 for <@sunmail2sca.sfbay.sun.com:LSARC-ext@sun.com>; Fri, 13 Nov 2009 15:56:44 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KT200G0DNUKAT00@nwk-avmta-2.sfbay.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Fri, 13 Nov 2009 15:56:44 -0800 (PST) Received: from sca-es-mail-2.sun.com ([192.18.43.133]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KT2004G6NUI2480@nwk-avmta-2.sfbay.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Fri, 13 Nov 2009 15:56:42 -0800 (PST) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id nADNugAW024000 for ; Fri, 13 Nov 2009 15:56:42 -0800 (PST) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KT200600NP7SS00@fe-sfbay-10.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Fri, 13 Nov 2009 15:56:41 -0800 (PST) Received: from [192.168.11.3] ([unknown] [209.128.86.122]) by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KT20079MNUHVZ00@fe-sfbay-10.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Fri, 13 Nov 2009 15:56:41 -0800 (PST) Date: Fri, 13 Nov 2009 15:56:40 -0800 From: Jyri Virkki Subject: Re: [webstack-discuss] Deliver collectd 4.8.x into OpenSolaris [LSARC/2009/606 FastTrack timeout 11/13/2009] In-reply-to: <4AFC281E.7050407@Sun.COM> Sender: Jyri.Virkki@sun.com To: Amit.Gupta@sun.com Cc: webstack-discuss@opensolaris.org, LSARC-ext@sun.com Message-id: MIME-version: 1.0 X-Mailer: Apple Mail (2.930.3) Content-type: text/plain; CHARSET=US-ASCII; delsp=yes; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200911060916.nA69Gaq2018611@sac.sfbay.sun.com> <4AF3EFB9.1060403@Sun.COM> <4AF4583F.8000105@Sun.COM> <4AFAE65A.7020105@Sun.COM> <20091112033215.GA3636@sun.com> <4AFC281E.7050407@Sun.COM> Status: RO Content-Length: 2028 On Nov 12, 2009, at 7:22 AM, Amit Gupta wrote: >> jv03: However, if you do, then document how the collectd smf service >> and collectdmon can (and can't) interact. In particular, in what >> ways >> can things go confusingly wrong if customer attempts to use both? >> The >> collectdmon man page is a good place to insert this information. >> > While collectdmon is not required if collectd is started via SMF, > there may be products which embed collectd and may want to manage > lifecyle of collectd via collectdmon. That's fine if you feel there is value and wish to support it. The issue jv03 is about documenting to customers what to do and more importantly not do with it. i.e. customer starts collectd via smf and later attempts to also start it via collectdmon; what can go wrong? Same for reverse, run it via collectdmon and later 'svcadm enable' it as well, what happens? Or start it via collectdmon, get confused/forget and try to stop it via 'svcadm disable'. You can probably come up with other scenarios of weird things customer may do to confuse themselves. If we ship two separate mechanisms to do the same thing then we have a responsibility to the customer to give them some guidance on how things interact and what to do/not do. (Sure, the smart customer will know intuitively, pick one and don't mix and match. But documenting the advice formally is a good thing.) For the ARC case, I suggest two things: > Note that collectdmon is not require if collectd is started > via SMF. However, > there may be products which embed collectd and may want to > manage lifecyle of > collectd via collectmon. Add text here saying: "The man page will document that smf is the preferred mechanism and offer some guidance on the use of collectdmon on Solaris. Refer to Appendix 2 for more." Then add an Appendix 2 showing the text to add into the collectdmon man page. Probably just a paragraph or so. -- Jyri Virkki - jyri.virkki@sun.com . From Amit.Gupta@Sun.COM Mon Nov 16 09:00:10 2009 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nAGH0A1w004138 for ; Mon, 16 Nov 2009 09:00:10 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id nAGH06wb014507 for <@sunmail2sca.sfbay.sun.com:LSARC-ext@sun.com>; Mon, 16 Nov 2009 09:00:09 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KT700E01OK7UQ00@nwk-avmta-1.sfbay.Sun.COM> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Mon, 16 Nov 2009 09:00:07 -0800 (PST) Received: from gmp-eb-inf-1.sun.com ([192.18.6.21]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KT700HFWOK2G480@nwk-avmta-1.sfbay.Sun.COM> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Mon, 16 Nov 2009 09:00:03 -0800 (PST) Received: from fe-emea-09.sun.com (gmp-eb-lb-1-fe1.eu.sun.com [192.18.6.7] (may be forged)) by gmp-eb-inf-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id nAGH02Vd002659 for ; Mon, 16 Nov 2009 17:00:02 +0000 (GMT) Received: from conversion-daemon.fe-emea-09.sun.com by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KT700E00O0IEG00@fe-emea-09.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Mon, 16 Nov 2009 16:59:44 +0000 (GMT) Received: from test.sun.com ([unknown] [129.158.228.93]) by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KT700MOMOJ9OV70@fe-emea-09.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Mon, 16 Nov 2009 16:59:37 +0000 (GMT) Date: Mon, 16 Nov 2009 22:16:24 +0530 From: Amit Gupta Subject: Re: [webstack-discuss] Deliver collectd 4.8.x into OpenSolaris [LSARC/2009/606 FastTrack timeout 11/13/2009] In-reply-to: Sender: Amit.Gupta@Sun.COM To: Jyri Virkki Cc: webstack-discuss@opensolaris.org, LSARC-ext@Sun.COM Reply-to: Amit.Gupta@Sun.COM Message-id: <4B0181E0.1080009@Sun.COM> MIME-version: 1.0 Content-type: multipart/mixed; boundary="Boundary_(ID_4MQDOg7ImgheODdddRtB1A)" X-PMX-Version: 5.4.1.325704 References: <200911060916.nA69Gaq2018611@sac.sfbay.sun.com> <4AF3EFB9.1060403@Sun.COM> <4AF4583F.8000105@Sun.COM> <4AFAE65A.7020105@Sun.COM> <20091112033215.GA3636@sun.com> <4AFC281E.7050407@Sun.COM> User-Agent: Thunderbird 2.0.0.4 (X11/20070622) Status: RO Content-Length: 15723 This is a multi-part message in MIME format. --Boundary_(ID_4MQDOg7ImgheODdddRtB1A) Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT On 11/14/09 05:26, Jyri Virkki wrote: > > Add text here saying: "The man page will document that smf is the > preferred mechanism and offer some guidance on the use of collectdmon > on Solaris. Refer to Appendix 2 for more." > > Then add an Appendix 2 showing the text to add into the collectdmon > man page. Probably just a paragraph or so. > As suggested, I have added Appendix 2 in the attached ARC case. Kindly review. Regards Amit --Boundary_(ID_4MQDOg7ImgheODdddRtB1A) Content-type: text/plain; name=collectd-arc.txt Content-transfer-encoding: 7BIT Content-disposition: inline; filename=collectd-arc.txt Template Version: @(#)sac_nextcase 1.68 02/23/09 SMI This information is Copyright 2009 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: Deliver collectd 4.8.x into OpenSolaris 1.2. Name of Document Author/Supplier: Author: Amit Gupta 1.3 Date of This Document: 06 November, 2009 4. Technical Description Deliver collectd 4.8.x into OpenSolaris 13 November 2009 1. Summary and motivation 1.1. Introduction This project delivers collectd 4.8.x into OpenSolaris collectd is a daemon which collects system performance statistics periodically and provides mechanisms to store the values in a variety of ways, for example in RRD files. For more information about collectd see: http://collectd.org 2. Technical description 2.1. Key objects /usr/lib/collectd4/collectd From collectd's man page: collectd is a daemon that receives system statistics and makes them available in a number of ways. The main daemon itself doesn't have any real functionality apart from loading, querying and submitting to plugins. /usr/lib/collectd4/collectdmon From collectdmon's man page: collectdmon is a small ``wrapper'' daemon which starts and monitors the collectd daemon. If collectd terminates it will automatically be restarted, unless collectdmon was told to shut it down. Note that collectdmon is not require if collectd is started via SMF. However, there may be products which embed collectd and may want to manage lifecyle of collectd via collectmon. The man page will document that smf is the preferred mechanism and offer some guidance on the use of collectdmon on Solaris. Refer to Appendix 2 for more. /etc/collectd4/collectd.conf The default collectd configuration file. 2.2. Versioning From collectd's FAQ: The version numbers in collectd consist of three numbers: The major- and minor-number and the patchlevel. * Versions with different major-numbers are basically not compatible. This means that the definitions of RRD-files or config-options have been changed or, in general, that the user has to do something in addition to install the new version. This is not nice and avoided when possible, but sometimes necessary to prevent old mistakes to become ancient mistakes. * Versions with differing minor-numbers are backwards compatible, i. e. you can replace the lower version with the higher one and everything should still work. This means that features are added, but not removed or changed and that the default behavior does not change. * Versions with different patchlevels are both, forward- and backwards-compatible, because no new features have been introduced. The only difference between the two versions is one or more bug fixes, so you should generally install the higher version of the two. 2.3 Configuration File This project will use the standard convention and deliver configuration files under /etc/collectd4 The default configuration file provided with this integration will work out of the box. In addition, a directory conf.d will be created in /etc/collectd4/ The default configuration loads any file with a .conf extension that is placed in this directory. Hence, plugins can optionally be configured by simply dropping a .conf file into this directory without modifying the main configuration file. By default, conf.d will not contain any plugin configuration files. 2.4 Log File The log File will be located in /var/collectd4/logs and will be named collectd.log. The default log level is info and can be changed by modifying the main configuration file. 2.5 Lifecycle collectd will be started as a SMF service and will run as user "daemon" and group "daemon". 2.6 Loadable Plugins collectd ships with a number of loadable plugins which are enabled through directives in the config file. Some of these plugins have library dependencies that must be satisfied through build time package options (supplied to the configure script). An example is mysql.so which has a dependency on the mysql client libraries. Support for loadable plugins and other features is enabled at build time through flags to the configure script. All the plugins that are delivered have sample configuration directives in collectd.conf. Note that in many cases, these are just sample directives and will need to be modified based on site specific configuration. Detailed documentation about specific plugin configuration can be found in the collectd website. Not all the plugins are enabled by default in collectd.conf. The configuration directives for plugins which are not enabled are commented out in collectd.conf. The plugins which are enabled by default are: syslog, cpu, interface, load, memory and rrdtool Other plugins can be enabled in one of the following two ways: * By adding/uncommenting the plugin configuration directives in collectd.conf * By dropping a .conf file having the necessary plugin configuration directives into the /etc/collectd4/conf.d Similarly, plugins can be disabled as needed in the following two ways: * By removing/commenting out the plugin configuration directives from collectd.conf * By deleting the plugin specific .conf file from the /etc/collectd4/conf.d directory Do note that collectd.conf is read and processed in order, i. e. from top to bottom. So the plugins are loaded in the order listed in collectd.conf. This project will deliver the following plugins: * Apache * CPU * CSV * cURL * DF * Disk * Exec * FileCount * GenericJMX * Interface * Java * Load * LogFile * memcachec * memcached * Memory * MySQL * Network * Notify Desktop * NTPd * PostgreSQL * RRDtool * SNMP * Swap * SysLog * Table * Tail * Tape * UnixSock * Uptime * Users * Write HTTP * ZFS ARC 2.7 Directory Naming and Structure The proposed directory layout for collectd is: /usr/lib/collectd4 /etc/collectd4 /conf.d /var/collectd4 /rrd /logs /usr/share/collectd4 /usr/include/collectd The detailed directory and file layout for collectd is provided in Appendix 1. Multiple versions can coexist, and are distinguished by the naming scheme (collectd4 for version 4.*.*, collectd5 for version 5.*.*). 2.8 Build features Build time package selection determines which collectd loadable plugins are available to the user at runtime. A complete list of collectd modules is available on the http://collectd.org. However, not all plugins can be built on OpenSolaris because of dependencies on external libraries and hence, only a subset of the plugins will be delivered. 3. collectd documentation collectd 4.* provides a full and complete documentation set, in UNIX manual format. This project will deliver the complete documentation set. 4. Packaging and Delivery We propose to package collectd under the following packages: SUNWcollectd4u - [usr] Server package (including collectd daemon, loadable plugins, man pages) SUNWcollectd4r - [root] (config files, smf files) 5. collectd Interfaces 5.1. Interface Stability collectd has no obvious history of any interface instability and the expectation is that dot releases of collectd will be compatible. This is not completely guaranteed by the community and it is possible that interfaces may change due to the fixing of bugs. 5.2. Imported Interfaces libcurl Volatile PSARC/2007/165 Output format of Apache's mod_status N/A PSARC/2007/586 Output format of Lighhtpd's mod_status N/A LSARC/2008/191 libmysqlclient Committed LSARC 2009/062 libnetsnmp Uncommitted LSARC/2008/355 librrd Uncommitted LSARC/2008/129 libpq Uncommitted LSARC/2009/349 Java SE 6 Committed PSARC/2006/430 libkstat Uncommitted PSARC/1992/085 kstat_read Stable PSARC/1992/085 getmntent Evolving PSARC/1999/131 getloadavg Standard PSARC/1996/196 libmemcached Uncommitted LSARC/2008/126 libgcrypt Volatile LSARC/2008/354 libnotify Volatile LSARC/2007/137 swapctl Stable PSARC/1997/099 libhal Volatile PSARC/2005/399 getutxent N/A N/A syslog Standard N/A statvfs Stable N/A 5.3. Exported Interfaces NAME STABILITY NOTES ----------------------------------------------------------------------- SUNWcollectd4r Committed package SUNWcollectd4u Committed package collectd config file Uncommitted config file syntax collectd log file Not An Interface log file syntax collectd pid file Project Private daemon pid file /var/collectd4/logs Uncommitted log file dir /etc/collectd4/collectd.conf Uncommitted default conf file /usr/lib/collectd4/*.so Project Private plugins /usr/include/collectd Uncommitted Include directory /lib/svc/method/collectd4-svc Project Private SMF svc method svc:/application/monitoring/collectd4:default Committed FMRI /var/svc/manifest/application/monitoring/collectd4.xml Project Private SMF Manifest 6. References [1] http://collectd.org ================================================================ Appendix 1: collectd Integration Directory and File Structure. 1. The following files are included in the collectd integration: /usr/lib/collectd4 collectd collectdmon apache.so bind.so cpu.so csv.so curl.so df.so disk.so exec.so filecount.so interface.so java.so load.so logfile.so match_regex.so match_timediff.so match_value.so memcached.so memory.so mysql.so network.so notify_desktop.so ntpd.so postgresql.so rrdtool.so snmp.so syslog.so swap.so table.so tail.so tape.so target_notification.so target_replace.so target_set.so ted.so unixsock.so uptime.so users.so libcollectdclient.so.0.0.0 libcollectdclient.so.0 -> libcollectdclient.so.0.0.0 libcollectdclient.so -> libcollectdclient.so.0.0.0 /etc/collectd4 /conf.d collectd.conf /var/collectd4 /logs collectd.log /rrd /usr/share/collectd4 types.db postgresql_default.conf java-plugin.jar jmx.jar /man /man1 collectd.1 collectdmon.1 /man5 collectd.conf.5 collectd-exec.conf collectd-unixsock.5 collectd-java.5 collectd-snmp.5 types.db.5 /usr/include/collectd4 client.h lcc_features.h /lib/svc/method/collectd4-svc /var/svc/manifest/application/monitoring/collectd4.xml ================================================================ Appendix 2: collectdmon man page update The following text will be added to the collectdmon man page: It is to be noted that collectd4 SMF service is the preferred method to start collectd. If collectdmon is used to start collectd with the default collectd.conf, make sure that the SMF service is disabled else both processes will try to update the same rrd files which may cause the data to get corrupted. 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: sfw 6.5. ARC review type: FastTrack 6.6. ARC Exposure: open --Boundary_(ID_4MQDOg7ImgheODdddRtB1A)-- From Jyri.Virkki@sun.com Mon Nov 16 18:38:42 2009 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nAH2cgfP001638 for ; Mon, 16 Nov 2009 18:38:42 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id nAH2ceDd026604; Mon, 16 Nov 2009 18:38:41 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KT800A09FCGGZ00@brm-avmta-1.central.sun.com>; Mon, 16 Nov 2009 19:38:40 -0700 (MST) Received: from brmea-mail-2.sun.com ([192.18.98.43]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KT800KZ5FCGKL50@brm-avmta-1.central.sun.com>; Mon, 16 Nov 2009 19:38:40 -0700 (MST) Received: from dm-usca15-11.red.iplanet.com (host-185-56-18-192.iplanet.com [192.18.56.185] (may be forged)) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id nAH2bxLt002957; Tue, 17 Nov 2009 02:37:59 +0000 (GMT) Received: from buye.red.iplanet.com (buye [192.18.65.224]) by dm-usca15-11.red.iplanet.com (8.11.7p1+Sun/8.11.7/IPLANET,v1.2) with ESMTP id nAH2bwK11008; Mon, 16 Nov 2009 18:37:58 -0800 (PST) Received: from buye.red.iplanet.com (localhost [127.0.0.1]) by buye.red.iplanet.com (8.13.7+Sun/8.13.7) with ESMTP id nAH2bwsG018144; Mon, 16 Nov 2009 18:37:58 -0800 (PST) Received: (from jyri@localhost) by buye.red.iplanet.com (8.13.7+Sun/8.13.7/Submit) id nAH2bwXD018143; Mon, 16 Nov 2009 18:37:58 -0800 (PST) Date: Mon, 16 Nov 2009 18:37:58 -0800 From: Jyri Virkki Subject: Re: [webstack-discuss] Deliver collectd 4.8.x into OpenSolaris [LSARC/2009/606 FastTrack timeout 11/13/2009] In-reply-to: <4B0181E0.1080009@Sun.COM> To: Amit Gupta Cc: webstack-discuss@opensolaris.org, LSARC-ext@sun.com Message-id: <20091117023758.GC14241@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <200911060916.nA69Gaq2018611@sac.sfbay.sun.com> <4AF3EFB9.1060403@Sun.COM> <4AF4583F.8000105@Sun.COM> <4AFAE65A.7020105@Sun.COM> <20091112033215.GA3636@sun.com> <4AFC281E.7050407@Sun.COM> <4B0181E0.1080009@Sun.COM> User-Agent: Mutt/1.5.11 Status: RO Content-Length: 1400 Amit Gupta wrote: > > Deliver collectd 4.8.x into OpenSolaris > 13 November 2009 While there hasn't been extensive discussion here, the comments here from Mark as well as some discussion in unrelated case 2009/575 lean in the direction of $COMPONENT/$VERSION/ layout, so barring any last minute reasons against it, I will draw a line in the sand and suggest to adopt the more common versioning choice and go with /usr/lib/collectd/4/ /etc/collectd/4/ /var/collectd/4/ /usr/share/collectd/4/ /usr/include/collectd/4/ As with other versioned projects, it is benefitial to make the 'preferred' version easily found on the system by users, so those files which fall under this casual discovery goal should in addition be symlinked from the common locations. For this case, that might just be the man pages, since the recommended usage is to start it via smf and smf will know where to find the binaries and shared libraries. But it is good if the casual user can type 'man collectd' and have it work out of the box. (In the man page, if you haven't already, you should document the locations of the rest of the collectd files.) Finally, as discussed in the thread, you can mark the exported package names Volatile. Aside from these points, the case seems to be converging on all the comment topics. -- Jyri J. Virkki - jyri.virkki@sun.com - Sun Microsystems From Amit.Gupta@sun.com Tue Nov 17 06:23:15 2009 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nAHENEIJ025089 for ; Tue, 17 Nov 2009 06:23:15 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id nAHENECW017910 for <@sunmail2sca.sfbay.sun.com:LSARC-ext@sun.com>; Tue, 17 Nov 2009 06:23:14 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KT900D0HBYQ5B00@nwk-avmta-2.sfbay.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Tue, 17 Nov 2009 06:23:14 -0800 (PST) Received: from gmp-eb-inf-1.sun.com ([192.18.6.21]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KT9006STBYPQ8C0@nwk-avmta-2.sfbay.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Tue, 17 Nov 2009 06:23:14 -0800 (PST) Received: from fe-emea-09.sun.com (gmp-eb-lb-1-fe1.eu.sun.com [192.18.6.7] (may be forged)) by gmp-eb-inf-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id nAHENCtW011999 for ; Tue, 17 Nov 2009 14:23:12 +0000 (GMT) Received: from conversion-daemon.fe-emea-09.sun.com by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KT900K00BQBMW00@fe-emea-09.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Tue, 17 Nov 2009 14:22:52 +0000 (GMT) Received: from test.sun.com ([unknown] [129.158.228.93]) by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KT900JJWBXT5870@fe-emea-09.sun.com> for LSARC-ext@sun.com (ORCPT LSARC-ext@sun.com); Tue, 17 Nov 2009 14:22:44 +0000 (GMT) Date: Tue, 17 Nov 2009 19:39:29 +0530 From: Amit Gupta Subject: Re: [webstack-discuss] Deliver collectd 4.8.x into OpenSolaris [LSARC/2009/606 FastTrack timeout 11/13/2009] In-reply-to: <20091117023758.GC14241@sun.com> Sender: Amit.Gupta@sun.com To: Jyri Virkki Cc: webstack-discuss@opensolaris.org, LSARC-ext@sun.com Reply-to: Amit.Gupta@sun.com Message-id: <4B02AE99.6000107@Sun.COM> MIME-version: 1.0 Content-type: multipart/mixed; boundary="Boundary_(ID_hkT6RaWe0TlvXcDbzY9TbQ)" X-PMX-Version: 5.4.1.325704 References: <200911060916.nA69Gaq2018611@sac.sfbay.sun.com> <4AF3EFB9.1060403@Sun.COM> <4AF4583F.8000105@Sun.COM> <4AFAE65A.7020105@Sun.COM> <20091112033215.GA3636@sun.com> <4AFC281E.7050407@Sun.COM> <4B0181E0.1080009@Sun.COM> <20091117023758.GC14241@sun.com> User-Agent: Thunderbird 2.0.0.4 (X11/20070622) Status: RO Content-Length: 19251 This is a multi-part message in MIME format. --Boundary_(ID_hkT6RaWe0TlvXcDbzY9TbQ) Content-type: multipart/alternative; boundary="Boundary_(ID_i25tQFV4xazMib6EavHgnw)" --Boundary_(ID_i25tQFV4xazMib6EavHgnw) Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT On 11/17/09 08:07, Jyri Virkki wrote: > Amit Gupta wrote: > >> Deliver collectd 4.8.x into OpenSolaris >> 13 November 2009 >> > > > While there hasn't been extensive discussion here, the comments here > from Mark as well as some discussion in unrelated case 2009/575 lean > in the direction of $COMPONENT/$VERSION/ layout, so barring any last > minute reasons against it, I will draw a line in the sand and > suggest to adopt the more common versioning choice and go with > > /usr/lib/collectd/4/ > /etc/collectd/4/ > /var/collectd/4/ > /usr/share/collectd/4/ > /usr/include/collectd/4/ > Updated the attached ARC case with the new file layout as suggested. > As with other versioned projects, it is benefitial to make the > 'preferred' version easily found on the system by users, so those > files which fall under this casual discovery goal should in addition > be symlinked from the common locations. > > For this case, that might just be the man pages, since the recommended > usage is to start it via smf and smf will know where to find the > binaries and shared libraries. But it is good if the casual user can > type 'man collectd' and have it work out of the box. (In the man page, > if you haven't already, you should document the locations of the rest > of the collectd files.) > I am working on this. > > Finally, as discussed in the thread, you can mark the exported package > names Volatile. > Done. Regards Amit --Boundary_(ID_i25tQFV4xazMib6EavHgnw) Content-type: text/html; CHARSET=US-ASCII Content-transfer-encoding: 7BIT On 11/17/09 08:07, Jyri Virkki wrote: 
Amit Gupta wrote:
  
Deliver collectd 4.8.x into OpenSolaris
13 November 2009
    


While there hasn't been extensive discussion here, the comments here
from Mark as well as some discussion in unrelated case 2009/575 lean
in the direction of $COMPONENT/$VERSION/ layout, so barring any last
minute reasons against it, I will draw a line in the sand and
suggest to adopt the more common versioning choice and go with

       /usr/lib/collectd/4/
       /etc/collectd/4/
       /var/collectd/4/
       /usr/share/collectd/4/
       /usr/include/collectd/4/
Updated the attached ARC case with the new file layout as suggested.
As with other versioned projects, it is benefitial to make the
'preferred' version easily found on the system by users, so those
files which fall under this casual discovery goal should in addition
be symlinked from the common locations.

For this case, that might just be the man pages, since the recommended
usage is to start it via smf and smf will know where to find the
binaries and shared libraries. But it is good if the casual user can
type 'man collectd' and have it work out of the box. (In the man page,
if you haven't already, you should document the locations of the rest
of the collectd files.)
I am working on this.

Finally, as discussed in the thread, you can mark the exported package
names Volatile.
Done.

Regards
Amit

--Boundary_(ID_i25tQFV4xazMib6EavHgnw)-- --Boundary_(ID_hkT6RaWe0TlvXcDbzY9TbQ) Content-type: text/plain; name=collectd-arc.txt Content-transfer-encoding: 7BIT Content-disposition: inline; filename=collectd-arc.txt Template Version: @(#)sac_nextcase 1.68 02/23/09 SMI This information is Copyright 2009 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: Deliver collectd 4.8.x into OpenSolaris 1.2. Name of Document Author/Supplier: Author: Amit Gupta 1.3 Date of This Document: 06 November, 2009 4. Technical Description Deliver collectd 4.8.x into OpenSolaris 17 November 2009 1. Summary and motivation 1.1. Introduction This project delivers collectd 4.8.x into OpenSolaris collectd is a daemon which collects system performance statistics periodically and provides mechanisms to store the values in a variety of ways, for example in RRD files. For more information about collectd see: http://collectd.org 2. Technical description 2.1. Key objects /usr/lib/collectd/4/collectd From collectd's man page: collectd is a daemon that receives system statistics and makes them available in a number of ways. The main daemon itself doesn't have any real functionality apart from loading, querying and submitting to plugins. /usr/lib/collectd/4/collectdmon From collectdmon's man page: collectdmon is a small ``wrapper'' daemon which starts and monitors the collectd daemon. If collectd terminates it will automatically be restarted, unless collectdmon was told to shut it down. Note that collectdmon is not require if collectd is started via SMF. However, there may be products which embed collectd and may want to manage lifecyle of collectd via collectmon. The man page will document that smf is the preferred mechanism and offer some guidance on the use of collectdmon on Solaris. Refer to Appendix 2 for more. /etc/collectd/4/collectd.conf The default collectd configuration file. 2.2. Versioning From collectd's FAQ: The version numbers in collectd consist of three numbers: The major- and minor-number and the patchlevel. * Versions with different major-numbers are basically not compatible. This means that the definitions of RRD-files or config-options have been changed or, in general, that the user has to do something in addition to install the new version. This is not nice and avoided when possible, but sometimes necessary to prevent old mistakes to become ancient mistakes. * Versions with differing minor-numbers are backwards compatible, i. e. you can replace the lower version with the higher one and everything should still work. This means that features are added, but not removed or changed and that the default behavior does not change. * Versions with different patchlevels are both, forward- and backwards-compatible, because no new features have been introduced. The only difference between the two versions is one or more bug fixes, so you should generally install the higher version of the two. 2.3 Configuration File This project will use the standard convention and deliver configuration files under /etc/collectd/4 The default configuration file provided with this integration will work out of the box. In addition, a directory conf.d will be created in /etc/collectd/4/ The default configuration loads any file with a .conf extension that is placed in this directory. Hence, plugins can optionally be configured by simply dropping a .conf file into this directory without modifying the main configuration file. By default, conf.d will not contain any plugin configuration files. 2.4 Log File The log File will be located in /var/collectd/4/logs and will be named collectd.log. The default log level is info and can be changed by modifying the main configuration file. 2.5 Lifecycle collectd will be started as a SMF service and will run as user "daemon" and group "daemon". 2.6 Loadable Plugins collectd ships with a number of loadable plugins which are enabled through directives in the config file. Some of these plugins have library dependencies that must be satisfied through build time package options (supplied to the configure script). An example is mysql.so which has a dependency on the mysql client libraries. Support for loadable plugins and other features is enabled at build time through flags to the configure script. All the plugins that are delivered have sample configuration directives in collectd.conf. Note that in many cases, these are just sample directives and will need to be modified based on site specific configuration. Detailed documentation about specific plugin configuration can be found in the collectd website. Not all the plugins are enabled by default in collectd.conf. The configuration directives for plugins which are not enabled are commented out in collectd.conf. The plugins which are enabled by default are: syslog, cpu, interface, load, memory and rrdtool Other plugins can be enabled in one of the following two ways: * By adding/uncommenting the plugin configuration directives in collectd.conf * By dropping a .conf file having the necessary plugin configuration directives into the /etc/collectd/4/conf.d Similarly, plugins can be disabled as needed in the following two ways: * By removing/commenting out the plugin configuration directives from collectd.conf * By deleting the plugin specific .conf file from the /etc/collectd/4/conf.d directory Do note that collectd.conf is read and processed in order, i. e. from top to bottom. So the plugins are loaded in the order listed in collectd.conf. This project will deliver the following plugins: * Apache * CPU * CSV * cURL * DF * Disk * Exec * FileCount * GenericJMX * Interface * Java * Load * LogFile * memcachec * memcached * Memory * MySQL * Network * Notify Desktop * NTPd * PostgreSQL * RRDtool * SNMP * Swap * SysLog * Table * Tail * Tape * UnixSock * Uptime * Users * Write HTTP * ZFS ARC 2.7 Directory Naming and Structure The proposed directory layout for collectd is: /usr/lib/collectd/4 /etc/collectd/4 /conf.d /var/collectd/4 /rrd /logs /run /usr/share/collectd/4 /usr/include/collectd/4 The detailed directory and file layout for collectd is provided in Appendix 1. Multiple versions can coexist, and are distinguished by the directory naming scheme (collectd/4 for version 4.*.*, collectd/5 for version 5.*.*). 2.8 Build features Build time package selection determines which collectd loadable plugins are available to the user at runtime. A complete list of collectd modules is available on the http://collectd.org. However, not all plugins can be built on OpenSolaris because of dependencies on external libraries and hence, only a subset of the plugins will be delivered. 3. collectd documentation collectd 4.* provides a full and complete documentation set, in UNIX manual format. This project will deliver the complete documentation set. 4. Packaging and Delivery We propose to package collectd under the following packages: SUNWcollectd4u - [usr] Server package (including collectd daemon, loadable plugins, man pages) SUNWcollectd4r - [root] (config files, smf files) 5. collectd Interfaces 5.1. Interface Stability collectd has no obvious history of any interface instability and the expectation is that dot releases of collectd will be compatible. This is not completely guaranteed by the community and it is possible that interfaces may change due to the fixing of bugs. 5.2. Imported Interfaces libcurl Volatile PSARC/2007/165 Output format of Apache's mod_status N/A PSARC/2007/586 Output format of Lighhtpd's mod_status N/A LSARC/2008/191 libmysqlclient Committed LSARC 2009/062 libnetsnmp Uncommitted LSARC/2008/355 librrd Uncommitted LSARC/2008/129 libpq Uncommitted LSARC/2009/349 Java SE 6 Committed PSARC/2006/430 libkstat Uncommitted PSARC/1992/085 kstat_read Stable PSARC/1992/085 getmntent Evolving PSARC/1999/131 getloadavg Standard PSARC/1996/196 libmemcached Uncommitted LSARC/2008/126 libgcrypt Volatile LSARC/2008/354 libnotify Volatile LSARC/2007/137 swapctl Stable PSARC/1997/099 libhal Volatile PSARC/2005/399 getutxent N/A N/A syslog Standard N/A statvfs Stable N/A 5.3. Exported Interfaces NAME STABILITY NOTES ----------------------------------------------------------------------- SUNWcollectd4r Volatile package SUNWcollectd4u Volatile package collectd config file Uncommitted config file syntax collectd log file Not An Interface log file syntax collectd pid file Project Private daemon pid file /var/collectd/4/logs Uncommitted log file dir /etc/collectd/4/collectd.conf Uncommitted default conf file /usr/lib/collectd/4/*.so Project Private plugins /usr/include/collectd Uncommitted Include directory /lib/svc/method/collectd4-svc Project Private SMF svc method svc:/application/monitoring/collectd4:default Committed FMRI /var/svc/manifest/application/monitoring/collectd4.xml Project Private SMF Manifest 6. References [1] http://collectd.org ================================================================ Appendix 1: collectd Integration Directory and File Structure. 1. The following files are included in the collectd integration: /usr/lib/collectd/4 collectd collectdmon apache.so bind.so cpu.so csv.so curl.so df.so disk.so exec.so filecount.so interface.so java.so load.so logfile.so match_regex.so match_timediff.so match_value.so memcached.so memory.so mysql.so network.so notify_desktop.so ntpd.so postgresql.so rrdtool.so snmp.so syslog.so swap.so table.so tail.so tape.so target_notification.so target_replace.so target_set.so ted.so unixsock.so uptime.so users.so libcollectdclient.so.0.0.0 libcollectdclient.so.0 -> libcollectdclient.so.0.0.0 libcollectdclient.so -> libcollectdclient.so.0.0.0 /etc/collectd/4 /conf.d collectd.conf /var/collectd/4 /logs collectd.log /rrd /run /usr/share/collectd/4 types.db postgresql_default.conf java-plugin.jar jmx.jar /man /man1 collectd.1 collectdmon.1 /man5 collectd.conf.5 collectd-exec.conf collectd-unixsock.5 collectd-java.5 collectd-snmp.5 types.db.5 /usr/include/collectd/4 client.h lcc_features.h /lib/svc/method/collectd4-svc /var/svc/manifest/application/monitoring/collectd4.xml ================================================================ Appendix 2: collectdmon man page update The following text will be added to the collectdmon man page: It is to be noted that the preferred method to start collectd is using the SMF service. If collectdmon is used to start collectd with the default collectd.conf, make sure that the SMF service is disabled else both processes will try to update the same rrd files which may cause the data to get corrupted. 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: sfw 6.5. ARC review type: FastTrack 6.6. ARC Exposure: open --Boundary_(ID_hkT6RaWe0TlvXcDbzY9TbQ)-- From Jyri.Virkki@sun.com Mon Nov 23 14:12:45 2009 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nANMCjKk013233 for ; Mon, 23 Nov 2009 14:12:45 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id nANMCgmZ014192; Mon, 23 Nov 2009 14:12:44 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KTL0000J1P7OD00@brm-avmta-1.central.sun.com>; Mon, 23 Nov 2009 15:12:43 -0700 (MST) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KTL00J701P64S30@brm-avmta-1.central.sun.com>; Mon, 23 Nov 2009 15:12:42 -0700 (MST) Received: from dm-usca19-13.red.iplanet.com (host-179-56-18-192.iplanet.com [192.18.56.179] (may be forged)) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id nANMC1De026282; Mon, 23 Nov 2009 22:12:01 +0000 (GMT) Received: from buye.red.iplanet.com (buye [192.18.65.224]) by dm-usca19-13.red.iplanet.com (8.11.7p1+Sun/8.11.7/IPLANET,v1.2) with ESMTP id nANMC1X24281; Mon, 23 Nov 2009 14:12:01 -0800 (PST) Received: from buye.red.iplanet.com (localhost [127.0.0.1]) by buye.red.iplanet.com (8.13.7+Sun/8.13.7) with ESMTP id nANMC1Cd026588; Mon, 23 Nov 2009 14:12:01 -0800 (PST) Received: (from jyri@localhost) by buye.red.iplanet.com (8.13.7+Sun/8.13.7/Submit) id nANMC1cw026587; Mon, 23 Nov 2009 14:12:01 -0800 (PST) Date: Mon, 23 Nov 2009 14:12:01 -0800 From: Jyri Virkki Subject: Re: Deliver collectd 4.8.x into OpenSolaris [LSARC/2009/606 FastTrack timeout 11/13/2009] In-reply-to: <4B02AE99.6000107@Sun.COM> To: Amit Gupta Cc: webstack-discuss@opensolaris.org, LSARC-ext@sun.com Message-id: <20091123221200.GE18477@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <200911060916.nA69Gaq2018611@sac.sfbay.sun.com> <4AF3EFB9.1060403@Sun.COM> <4AF4583F.8000105@Sun.COM> <4AFAE65A.7020105@Sun.COM> <20091112033215.GA3636@sun.com> <4AFC281E.7050407@Sun.COM> <4B0181E0.1080009@Sun.COM> <20091117023758.GC14241@sun.com> <4B02AE99.6000107@Sun.COM> User-Agent: Mutt/1.5.11 Status: RO Content-Length: 341 Amit Gupta wrote: > > Updated the attached ARC case with the new file layout as suggested. The latest updated version of the doc is in the case directory (materials/collectd.txt) The case timed out on 11/13 and all the discussion in this case has converged, marking it closed. -- Jyri J. Virkki - jyri.virkki@sun.com - Sun Microsystems