From John.Fischer@sun.com Mon Feb 4 10:41:28 2008 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m14IfRVE015339 for ; Mon, 4 Feb 2008 10:41:27 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id m14IfCip018123 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Tue, 5 Feb 2008 02:41:26 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JVQ0021998YZB00@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Mon, 04 Feb 2008 10:41:22 -0800 (PST) Received: from sca-es-mail-2.sun.com ([192.18.43.133]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JVQ00K4498XJD70@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Mon, 04 Feb 2008 10:41:21 -0800 (PST) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m14IfLn9009761 for ; Mon, 04 Feb 2008 10:41:21 -0800 (PST) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JVQ000019533U00@fe-sfbay-09.sun.com> (original mail from John.Fischer@Sun.COM) for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Mon, 04 Feb 2008 10:41:21 -0800 (PST) Received: from [192.168.10.6] ([24.10.86.139]) by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JVQ006AY98VQNG0@fe-sfbay-09.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Mon, 04 Feb 2008 10:41:20 -0800 (PST) Date: Mon, 04 Feb 2008 10:41:16 -0800 From: John Fischer Subject: PSARC/2008/078 - postrun - delayed execution environment for procedural package scripts Sender: John.Fischer@sun.com To: psarc-ext@sun.com Cc: Laszlo.Peter@sun.com, John Fischer Reply-to: John.Fischer@sun.com Message-id: <47A75C4C.8040300@sun.com> MIME-version: 1.0 Content-type: multipart/mixed; boundary="Boundary_(ID_StvVfnHxpSFav0W9kTb4Zg)" X-PMX-Version: 5.2.0.264296 User-Agent: Thunderbird 2.0.0.4 (X11/20070622) Status: RO Content-Length: 7782 This is a multi-part message in MIME format. --Boundary_(ID_StvVfnHxpSFav0W9kTb4Zg) Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT PSARC, I am sponsoring this fast track for Laszlo (Laca) Peter of the Desktop group in Dublin, Ireland. The case directory contains this proposal and the SMF manifest. I have set the timeout for Monday, February 11th, 2008. This case provides an easy-to-use method for calling commands in procedural package scripts that cannot be run in Live Upgrade and alternate rootdir environments because they are not available in older versions of Solaris and/or not relocatable. The project is requesting a Patch release of Solaris. Thanks, John --Boundary_(ID_StvVfnHxpSFav0W9kTb4Zg) Content-type: text/plain; name=proposal.txt Content-transfer-encoding: 7BIT Content-disposition: inline; filename=proposal.txt Title: postrun - delayed execution environment for procedural package scripts Case: PSARC/2008/078 Submitter: Laszlo Peter Owner: John Fischer Timeout: 02/11/2008 1.0 Introduction 1.1 Project/Component Working Name: Delayed execution environment for procedural package scripts 1.2 Purpose This project provides an easy-to-use method for calling commands in procedural package scripts that cannot be run in Live Upgrade and alternate rootdir environments because they are not available in older versions of Solaris and/or not relocatable. Patch release binding is requested. 2.0 Description Several software projects ship with programs needed for updating the project's databases or caches whenever the configuration changes or new files/data added that invalidate the cache. Examples from the GNOME world are: - update-mime-database - update-desktop-database - gtk-update-icon-cache - adding new gconf schemas to the gconf database (gconftool-2) - gdk-pixbuf image loaders On Linux, these tasks are handled by postinstall/postremove scripts, however that approach cannot be used on Solaris, because these are binary programs that won't work in Live Upgrade and some of them were not desiged to work properly in an alternate root directory (when pkgadd -R option is used) either. In these cases, the only choice is to execute the commands at the first system boot after installation, however, when installing the packages on a live system, the commands need to be executed immediately and no reboot is necessary. Such behavior can be achieved by adding complex procedural scripts to each package that needs to run any of the above programs after installation. This would become difficult to maintain and error prone, since very similar complex postinstall scripts would be repeated in each package. An smf service would also be required, at least for each of the above services that takes care of the first-boot case. Some packages would, however, require special processing (e.g. adding the battery applet to the default GNOME panel on a laptop), such cases would need their own smf services. Instead, JDS in Nevada uses postrun for running these programs. Postrun generalizes the above idea by putting the complex script into a shared script (postrun itself) and using a single smf service for all first-boot commands. The postrun script determines if it is safe to executes the commands in the actual installation scenario or saves it in its spool directory for the next system boot. The transient smf service looks for spooled jobs and executes them. When there are no spooled jobs, this service adds less than 0.1 sec to the system startup time. This case makes postrun a Contracted Consolidation Private interface of GNOME so that StarOffice and other desktop components can also make use of it. 2.1 Interfaces $PKG_INSTALL_ROOT/var/lib/postrun/postrun script used by "root" packages (sunw_pkgtype=root) $BASEDIR/usr/lib/postrun script used by "usr" packages (sunw_pkgtype=usr) CLI for both of the above scripts: Usage: postrun [options] postrun reads the standard input for the commands that the package script requests to be executed. Options: -u, --uniq If the same command is requested multiple times, the command is only run once. If it is safe to execute the command immediately, it will be delayed by 5 minutes, or as set using the --timeout option -t , --timeout Delay the execution of uniq commands by minutes. -b, --bg Run the command in the background and return control immediately -f Read the commands from instead of the standard input. -c Assign this job to class . Useful for querying jobs with postrun-query -i Ignore this job if it cannot be executed immediately. -h, -?, --help Display this help /usr/lib/postrun-query Usage: postrun-query [options] Options: -c , --class Only consider jobs the belong to class -e, --exists return 0 if spooled jobs exist 1 otherwise -n, --count print the number of spooled jobs only. The default behaviour is to list all jobs -j , --job display job number -h, -?, --help Display this help All interfaces Contracted Consolidation Private 3.0 Delivery Availability and file locations: +------------------+----------------------------------------------------+ | Package name | File locations | +------------------+----------------------------------------------------+ | SUNWpostrun-root | /var/lib/postrun/postrun | | | /var/lib/postrun/postrun-runq | | | /var/log/postrun.log | | | /var/spool/postrun/ | +------------------+----------------------------------------------------+ | SUNWpostrun | /usr/lib/postrun | | | /usr/lib/postrun-query | +------------------+----------------------------------------------------+ 4.0 Interface Classification The project exports the following interfaces: +-----------------------------------------------------------------------+ | Interfaces Exported | +------------------------------+------------------+---------------------+ | Interface | Classification | Comment | +------------------------------+------------------+---------------------+ | /usr/lib/postrun | Contracted | | | /var/lib/postrun/postrun | Consolidation | | | /usr/lib/postrun-query | Private | | +------------------------------+------------------+---------------------+ | SUNWpostrun | Uncommitted | | | SUNWpostrun-root | Uncommitted | | +------------------------------+------------------+---------------------+ | svc:/system/postrun | Uncommitted | transient service | | | | for executing | | | | spooled jobs | +------------------------------+------------------+---------------------+ --Boundary_(ID_StvVfnHxpSFav0W9kTb4Zg)-- From sacadmin Mon Feb 4 10:34:57 2008 Received: from sac.sfbay.sun.com (localhost [127.0.0.1]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m14IYvYd014478; Mon, 4 Feb 2008 10:34:57 -0800 (PST) Received: (from johnf@localhost) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id m14IYvJ1014474; Mon, 4 Feb 2008 10:34:57 -0800 (PST) Date: Mon, 4 Feb 2008 10:34:57 -0800 (PST) From: John Fischer Message-Id: <200802041834.m14IYvJ1014474@sac.sfbay.sun.com> To: PSARC-record@sac.sfbay.sun.com Subject: postrun - delayed execution environment for procedural package scripts [PSARC/2008/078 FastTrack timeout 02/11/2008] Status: RO Content-Length: 618 Template Version: @(#)sac_nextcase 1.64 07/13/07 SMI This information is Copyright 2008 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: postrun - delayed execution environment for procedural package scripts 1.2. Name of Document Author/Supplier: Author: Laszlo Peter 1.3 Date of This Document: 04 February, 2008 4. Technical Description See the case directory for more detail 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: JDS - Desktop 6.5. ARC review type: FastTrack 6.6. ARC Exposure: open From Nicolas.Williams@sun.com Mon Feb 4 11:56:36 2008 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m14JuaM2020822 for ; Mon, 4 Feb 2008 11:56:36 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m14JuZJL003572; Mon, 4 Feb 2008 11:56:35 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JVQ00I0BCQAM000@nwk-avmta-1.sfbay.Sun.COM>; Mon, 04 Feb 2008 11:56:34 -0800 (PST) Received: from binky.Central.Sun.COM ([129.153.128.104]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JVQ004N2CQAKDA0@nwk-avmta-1.sfbay.Sun.COM>; Mon, 04 Feb 2008 11:56:34 -0800 (PST) Received: from binky.Central.Sun.COM (localhost [127.0.0.1]) by binky.Central.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id m14JuXqe023140; Mon, 04 Feb 2008 13:56:33 -0600 (CST) Received: (from nw141292@localhost) by binky.Central.Sun.COM (8.14.1+Sun/8.14.1/Submit) id m14JuX4j023139; Mon, 04 Feb 2008 13:56:33 -0600 (CST) Date: Mon, 04 Feb 2008 13:56:33 -0600 From: Nicolas Williams Subject: Re: PSARC/2008/078 - postrun - delayed execution environment for procedural package scripts In-reply-to: <47A75C4C.8040300@sun.com> To: John Fischer Cc: psarc-ext@sun.com Mail-followup-to: John Fischer , psarc-ext@sun.com Message-id: <20080204195633.GZ19708@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <47A75C4C.8040300@sun.com> X-Authentication-warning: binky.Central.Sun.COM: nw141292 set sender to Nicolas.Williams@sun.com using -f User-Agent: Mutt/1.5.7i Status: RO Content-Length: 205 What about class action scripts that can't run in alternate boot / LU? Does not providing a post-run mode for class action scripts mean that postrun scripts will be more complicated than they have to be? From John.Fischer@sun.com Mon Feb 4 13:28:56 2008 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m14LStXa023995 for ; Mon, 4 Feb 2008 13:28:55 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m14LStBo029182 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Mon, 4 Feb 2008 13:28:55 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JVQ00901H07TK00@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Mon, 04 Feb 2008 13:28:55 -0800 (PST) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JVQ008VXH06F020@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Mon, 04 Feb 2008 13:28:54 -0800 (PST) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m14LSs5c023879 for ; Mon, 04 Feb 2008 13:28:54 -0800 (PST) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JVQ00301G1ATJ00@fe-sfbay-09.sun.com> (original mail from John.Fischer@Sun.COM) for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Mon, 04 Feb 2008 13:28:54 -0800 (PST) Received: from [192.168.10.6] ([24.10.86.139]) by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JVQ006BWGZOP590@fe-sfbay-09.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Mon, 04 Feb 2008 13:28:37 -0800 (PST) Date: Mon, 04 Feb 2008 13:28:28 -0800 From: John Fischer Subject: [Fwd: Re: PSARC/2008/078 - postrun - delayed execution environment for procedural package scripts] Sender: John.Fischer@sun.com To: Laszlo.Peter@sun.com Cc: John Fischer , psarc-ext@sun.com Reply-to: John.Fischer@sun.com Message-id: <47A7837C.6000906@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 User-Agent: Thunderbird 2.0.0.4 (X11/20070622) Status: RO Content-Length: 568 Forwarding to Laca. Thanks, John -------- Original Message -------- Subject: Re: PSARC/2008/078 - postrun - delayed execution environment for procedural package scripts Date: Mon, 04 Feb 2008 13:56:33 -0600 From: Nicolas Williams To: John Fischer CC: psarc-ext@sun.com References: <47A75C4C.8040300@sun.com> What about class action scripts that can't run in alternate boot / LU? Does not providing a post-run mode for class action scripts mean that postrun scripts will be more complicated than they have to be? From Laszlo.Peter@sun.com Mon Feb 4 16:57:09 2008 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m150v8v6006491 for ; Mon, 4 Feb 2008 16:57:08 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m150v4d0027315 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Mon, 4 Feb 2008 16:57:08 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JVQ0031BQN7TQ00@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Mon, 04 Feb 2008 17:57:07 -0700 (MST) Received: from gmp-eb-mail-2.sun.com ([192.18.6.24]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JVQ007AVQN5RPC0@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Mon, 04 Feb 2008 17:57:06 -0700 (MST) Received: from fe-emea-09.sun.com (gmp-eb-lb-2-fe3.eu.sun.com [192.18.6.12]) by gmp-eb-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m150v5gf014352 for ; Tue, 05 Feb 2008 00:57:05 +0000 (GMT) Received: from conversion-daemon.fe-emea-09.sun.com by fe-emea-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JVQ00A01QIF6C00@fe-emea-09.sun.com> (original mail from Laszlo.Peter@Sun.COM) for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Tue, 05 Feb 2008 00:57:05 +0000 (GMT) Received: from [192.168.2.101] ([60.234.117.94]) by fe-emea-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JVQ0074NQN26OC0@fe-emea-09.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Tue, 05 Feb 2008 00:57:05 +0000 (GMT) Date: Tue, 05 Feb 2008 13:57:03 +1300 From: "Laszlo (Laca) Peter" Subject: Re: PSARC/2008/078 - postrun - delayed execution environment for procedural package scripts In-reply-to: <20080204195633.GZ19708@Sun.COM> Sender: Laszlo.Peter@sun.com To: Nicolas Williams Cc: John Fischer , psarc-ext@sun.com Reply-to: laca@sun.com Message-id: <1202173023.1528.25.camel@localhost> Organization: Sun Microsystems, Inc. MIME-version: 1.0 X-Mailer: Evolution 2.12.2 Content-type: text/plain Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <47A75C4C.8040300@sun.com> <20080204195633.GZ19708@Sun.COM> Status: RO Content-Length: 1097 On Mon, 2008-02-04 at 13:56 -0600, Nicolas Williams wrote: > What about class action scripts that can't run in alternate boot / LU? Well, JDS doesn't have any so an example would help me picture it. I would normally recommend using class action scripts when it's simple enough to be scriptable and then there should be no problem with altroot or LU. > Does not providing a post-run mode for class action scripts mean that > postrun scripts will be more complicated than they have to be? I don't see why they would be, but that's probably because I can't think of a good example for the CAS that needs a post-run mode (or I misunderstand you question). The problem that postrun tries to solve is updating binary files or updating text files based on information extraced from binary files. We have binary executables to do the work, we just need to execute them at the right time. The postrun scripts themselves tend to be very simple, for example: ( echo 'test -x /usr/bin/scrollkeeper-update || exit 0'; echo '/usr/bin/scrollkeeper-update' ) | $BASEDIR/lib/postrun -b -u -c JDS Laca From Sebastien.Roy@sun.com Mon Feb 4 17:29:27 2008 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m151TQtr006860 for ; Mon, 4 Feb 2008 17:29:27 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id m151TO3P005817 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Tue, 5 Feb 2008 09:29:25 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JVQ00J07S4ZT200@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Mon, 04 Feb 2008 17:29:23 -0800 (PST) Received: from brmea-mail-2.sun.com ([192.18.98.43]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JVQ00HS8S4ZH040@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Mon, 04 Feb 2008 17:29:23 -0800 (PST) Received: from fe-amer-09.sun.com ([192.18.109.79]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id m151TNW0016113 for ; Tue, 05 Feb 2008 01:29:23 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JVQ00901RZFY000@mail-amer.sun.com> (original mail from Sebastien.Roy@Sun.COM) for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Mon, 04 Feb 2008 18:29:23 -0700 (MST) Received: from [192.168.1.4] ([72.93.210.146]) by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JVQ000EVS4YT5C0@mail-amer.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Mon, 04 Feb 2008 18:29:23 -0700 (MST) Date: Mon, 04 Feb 2008 20:29:22 -0500 From: Sebastien Roy Subject: Re: PSARC/2008/078 - postrun - delayed execution environment for procedural package scripts In-reply-to: <47A75C4C.8040300@sun.com> Sender: Sebastien.Roy@sun.com To: John.Fischer@sun.com Cc: psarc-ext@sun.com Message-id: <47A7BBF2.9050001@sun.com> Organization: Sun Microsystems MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <47A75C4C.8040300@sun.com> User-Agent: Thunderbird 2.0.0.9 (X11/20080115) Status: RO Content-Length: 627 John Fischer wrote: > Instead, JDS in Nevada uses postrun for running these programs. > Postrun generalizes the above idea by putting the complex > script into a shared script (postrun itself) and using a > single smf service for all first-boot commands. When can consumers of postrun expect their scripts to be run at first boot? My question hints at a potential limitation of postrun for packages that expect their changes to have taken effect very early in boot (earlier than the postrun SMF service is started). It may be worthwhile to call out this limitation to set appropriate expectations. -Seb From Laszlo.Peter@sun.com Mon Feb 4 18:13:16 2008 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m152DGo1008899 for ; Mon, 4 Feb 2008 18:13:16 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m152DEc3001342 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Tue, 5 Feb 2008 02:13:15 GMT Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JVQ00L03U62WJ00@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Mon, 04 Feb 2008 18:13:14 -0800 (PST) Received: from gmp-eb-mail-1.sun.com ([192.18.6.21]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JVQ00H84U60GX80@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Mon, 04 Feb 2008 18:13:13 -0800 (PST) Received: from fe-emea-10.sun.com (gmp-eb-lb-2-fe1.eu.sun.com [192.18.6.10]) by gmp-eb-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m152DCDj006536 for ; Tue, 05 Feb 2008 02:13:12 +0000 (GMT) Received: from conversion-daemon.fe-emea-10.sun.com by fe-emea-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JVQ00901TZJKB00@fe-emea-10.sun.com> (original mail from Laszlo.Peter@Sun.COM) for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Tue, 05 Feb 2008 02:13:12 +0000 (GMT) Received: from [192.168.2.101] ([60.234.117.94]) by fe-emea-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JVQ00MP5U5WKTF0@fe-emea-10.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Tue, 05 Feb 2008 02:13:12 +0000 (GMT) Date: Tue, 05 Feb 2008 15:13:09 +1300 From: "Laszlo (Laca) Peter" Subject: Re: PSARC/2008/078 - postrun - delayed execution environment for procedural package scripts In-reply-to: <47A7BBF2.9050001@sun.com> Sender: Laszlo.Peter@sun.com To: Sebastien Roy Cc: John.Fischer@sun.com, psarc-ext@sun.com Reply-to: laca@sun.com Message-id: <1202177589.1528.41.camel@localhost> Organization: Sun Microsystems, Inc. MIME-version: 1.0 X-Mailer: Evolution 2.12.2 Content-type: text/plain Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <47A75C4C.8040300@sun.com> <47A7BBF2.9050001@sun.com> Status: RO Content-Length: 1342 On Mon, 2008-02-04 at 20:29 -0500, Sebastien Roy wrote: > John Fischer wrote: > > Instead, JDS in Nevada uses postrun for running these programs. > > Postrun generalizes the above idea by putting the complex > > script into a shared script (postrun itself) and using a > > single smf service for all first-boot commands. > > When can consumers of postrun expect their scripts to be run at first > boot? My question hints at a potential limitation of postrun for > packages that expect their changes to have taken effect very early in > boot (earlier than the postrun SMF service is started). It may be > worthwhile to call out this limitation to set appropriate expectations. The dependencies are svc:/system/filesystem/local and svc:/system/sysidtool:system. So that's the earliest they will be run. At the other end, gnome-session waits for all postrun scripts in the JDS_wait class to complete before the session is started. (A progress bar is displayed, the label is something like 'completing post installation confirugation'.). The current consumers are at the desktop level (note that postrun is JDS Consolidation Private), so that should be early enough but not too early for the installation of desktop components. I agree this could be a limitation if postrun was used outside the desktop stack. Laca From casper@holland.sun.com Tue Feb 5 00:13:03 2008 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m158D3EO017085 for ; Tue, 5 Feb 2008 00:13:03 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m158D1Dq014647; Tue, 5 Feb 2008 00:13:02 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JVR00J07ATOQH00@brm-avmta-1.central.sun.com>; Tue, 05 Feb 2008 01:13:00 -0700 (MST) Received: from dm-holland-02.uk.sun.com ([129.156.101.225]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JVR00BRXATM2I50@brm-avmta-1.central.sun.com>; Tue, 05 Feb 2008 01:12:59 -0700 (MST) Received: from holland (room101.Holland.Sun.COM [10.16.117.40]) by dm-holland-02.uk.sun.com (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id m158CuRI005413; Tue, 05 Feb 2008 08:12:56 +0000 (GMT) Date: Tue, 05 Feb 2008 09:12:55 +0100 From: Casper.Dik@sun.com Subject: Re: PSARC/2008/078 - postrun - delayed execution environment for procedural package scripts In-reply-to: <1202177589.1528.41.camel@localhost> Sender: casper@holland.sun.com To: laca@sun.com Cc: Sebastien Roy , John.Fischer@sun.com, psarc-ext@sun.com Message-id: <200802050812.m158CuRI005413@dm-holland-02.uk.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <47A75C4C.8040300@sun.com> <47A7BBF2.9050001@sun.com> <1202177589.1528.41.camel@localhost> Status: RO Content-Length: 1128 >So that's the earliest they will be run. At the other end, >gnome-session waits for all postrun scripts in the JDS_wait >class to complete before the session is started. (A progress bar >is displayed, the label is something like 'completing post >installation confirugation'.). > >The current consumers are at the desktop level (note that >postrun is JDS Consolidation Private), so that should be early >enough but not too early for the installation of desktop components. >I agree this could be a limitation if postrun was used outside >the desktop stack. I find that postrun currently is an abomination (or at least the consumers are); it runs 10,000s of processes (literally) and takes a long time; making it public would likely make this worse. Will its use be sufficiently discouraged? (And will current postrun consumers be fixed to run more efficiently)? I note that this is a lot worse for people who update biweekly,, but I also wonder why postrun needs to run at standard priority? Some systems will run certain services faster and appear up more quickly if postrun is run at a lower CPU priority. Casper From John.Plocher@sun.com Tue Feb 5 06:51:26 2008 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m15EpQ9D025341 for ; Tue, 5 Feb 2008 06:51:26 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m15EpP70021559 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Tue, 5 Feb 2008 06:51:26 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JVR00C21T9PTK00@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Tue, 05 Feb 2008 06:51:25 -0800 (PST) Received: from sca-es-mail-2.sun.com ([192.18.43.133]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JVR006B3T9OB3A0@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Tue, 05 Feb 2008 06:51:24 -0800 (PST) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m15EpOw1009836 for ; Tue, 05 Feb 2008 06:51:24 -0800 (PST) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JVR00001T9CNW00@fe-sfbay-09.sun.com> (original mail from John.Plocher@Sun.COM) for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Tue, 05 Feb 2008 06:51:24 -0800 (PST) Received: from [192.168.168.4] ([208.74.177.212]) by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JVR00I3BT9N7L20@fe-sfbay-09.sun.com>; Tue, 05 Feb 2008 06:51:23 -0800 (PST) Date: Tue, 05 Feb 2008 06:51:23 -0800 From: John Plocher Subject: Re: PSARC/2008/078 - postrun - delayed execution environment for procedural package scripts In-reply-to: <1202177589.1528.41.camel@localhost> Sender: John.Plocher@sun.com To: laca@sun.com Cc: Sebastien Roy , John.Fischer@sun.com, psarc-ext@sun.com, Danek Duvall Message-id: <47A877EB.9000100@Sun.Com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <47A75C4C.8040300@sun.com> <47A7BBF2.9050001@sun.com> <1202177589.1528.41.camel@localhost> User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) Status: RO Content-Length: 832 Laszlo (Laca) Peter wrote: > gnome-session waits for all postrun scripts in the JDS_wait A few questions/comments ... jmp-1) Brand and organization names like JDS should never be used in architectural level artifacts - rather a descriptive word should be used instead. jmp-2) There should be a conceptual distinction between graphics system startup, login screen, and desktop startup. This would allow the user or admin to use KDE (or CDE or ...) instead of GNOME as their desktop. If the postrun stuff is tied to only the gnome desktop, yet it does things for the other two conceptual areas, we have problems. In the same way, if postrun is a JDS-only thing, what mechanism would KDE (...) use? jmp-3) How does this relate to the work being done in IPS, which is also trying to fix/eliminate postinstall scripts? -John From sch@zion.sfbay.sun.com Tue Feb 5 10:13:54 2008 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m15IDrut004901 for ; Tue, 5 Feb 2008 10:13:54 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m15IDlJf003854; Tue, 5 Feb 2008 18:13:48 GMT Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JVS00J092MZGH00@brm-avmta-1.central.sun.com>; Tue, 05 Feb 2008 11:13:47 -0700 (MST) Received: from zion.sfbay.sun.com ([129.146.17.75]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JVS00DZ52MYD550@brm-avmta-1.central.sun.com>; Tue, 05 Feb 2008 11:13:46 -0700 (MST) Received: from zion.sfbay.sun.com (localhost [127.0.0.1]) by zion.sfbay.sun.com (8.14.2+Sun/8.14.2) with ESMTP id m15IDkOV239398; Tue, 05 Feb 2008 18:13:46 +0000 (GMT) Received: (from sch@localhost) by zion.sfbay.sun.com (8.14.2+Sun/8.14.2/Submit) id m15IDj2U239396; Tue, 05 Feb 2008 10:13:45 -0800 (PST) Date: Tue, 05 Feb 2008 10:13:45 -0800 From: Stephen Hahn Subject: Re: PSARC/2008/078 - postrun - delayed execution environment for procedural package scripts In-reply-to: <47A877EB.9000100@Sun.Com> To: John Plocher Cc: laca@sun.com, John.Fischer@sun.com, psarc-ext@sun.com, Sebastien Roy , Danek Duvall Message-id: <20080205181345.GE236823@eng.sun.com> Organization: Solaris Kernel Development; Sun Microsystems, Inc. MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <47A75C4C.8040300@sun.com> <47A7BBF2.9050001@sun.com> <1202177589.1528.41.camel@localhost> <47A877EB.9000100@Sun.Com> User-Agent: Mutt/1.4.2.1i Status: RO Content-Length: 1031 * John Plocher [2008-02-05 15:00]: > jmp-3) How does this relate to the work being done in IPS, which is also > trying to fix/eliminate postinstall scripts? Image packaging will need something like postrun, but I would expect to see the actual commands delivered as files, rather than via a pipe. If postrun remains at Contracted Consolidation Private, then it should be a straightforward series of tasks to update it to something more aligned. I do agree with Casper that casual observation of postrun tasks leads to a series of questions about the implementation. If we were to move to a public interface commitment level, I would like to see more diagnostic interfaces. I'd also like to separate the tasks that need to be done every reboot/reconstruction from version-specific transitions, which I don't believe is done in the current tool. (Yes, I know I need to stop having only informal conversations and incept.) - Stephen -- sch@sun.com http://blogs.sun.com/sch/ From danek.duvall@sun.com Tue Feb 5 11:26:58 2008 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m15JQwd6006863 for ; Tue, 5 Feb 2008 11:26:58 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m15JQusR022640 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Tue, 5 Feb 2008 11:26:58 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JVS0010P60XPW00@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Tue, 05 Feb 2008 12:26:57 -0700 (MST) Received: from dm-sfbay-02.sfbay.sun.com ([129.146.11.31]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JVS00D4J60VCWB0@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Tue, 05 Feb 2008 12:26:55 -0700 (MST) Received: from zruty.sfbay.sun.com (zruty.SFBay.Sun.COM [129.146.168.40]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m15JQtox059760; Tue, 05 Feb 2008 11:26:55 -0800 (PST) Received: from zruty.sfbay.sun.com (localhost [127.0.0.1]) by zruty.sfbay.sun.com (8.14.2+Sun/8.14.2) with ESMTP id m15JQs8Y012216; Tue, 05 Feb 2008 11:26:55 -0800 (PST) Received: (from dduvall@localhost) by zruty.sfbay.sun.com (8.14.2+Sun/8.14.2/Submit) id m15JQsnv012215; Tue, 05 Feb 2008 11:26:54 -0800 (PST) Date: Tue, 05 Feb 2008 11:26:54 -0800 From: Danek Duvall Subject: Re: PSARC/2008/078 - postrun - delayed execution environment for procedural package scripts In-reply-to: <47A75C4C.8040300@sun.com> To: John Fischer Cc: psarc-ext@sun.com Message-id: <20080205192654.GY5284@zruty.sfbay.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <47A75C4C.8040300@sun.com> User-Agent: Mutt/1.5.16 (2007-06-27) Status: RO Content-Length: 1969 On Mon, Feb 04, 2008 at 10:41:16AM -0800, John Fischer wrote: > Several software projects ship with programs needed for updating the > project's databases or caches whenever the configuration changes or > new files/data added that invalidate the cache. Examples from the > GNOME world are: > - update-mime-database > - update-desktop-database > - gtk-update-icon-cache > - adding new gconf schemas to the gconf database (gconftool-2) > - gdk-pixbuf image loaders > > [ ... ] > > In these cases, the only choice is to execute the commands at the > first system boot after installation, however, when installing the > packages on a live system, the commands need to be executed > immediately and no reboot is necessary. Such behavior can be > achieved by adding complex procedural scripts to each package that > needs to run any of the above programs after installation. This > would become difficult to maintain and error prone, since very > similar complex postinstall scripts would be repeated in each > package. For the examples you list, as well as a few others, such as makewhatis and the infodir service that has been proposed, it seems to me that a service per database/cache is precisely what you want. It's something that can be kicked at an arbitrary time, rather than just at boot time (and the first boot after a package was installed, no less), turned on and off, and otherwise manipulated by the standard SMF tools. Indeed, it seems to me that what you'd really want is for a single long-running service to watch directories for file events -- creations, deletions, etc -- and kick off the appropriate updater program to handle the directories where stuff happened. It could have multiple FMRI front-ends for administrator intervention. It could also be extended to the user level, if we should ever end up with a user or session level restarter restarter. Danek From jek3@sun.com Tue Feb 5 11:39:38 2008 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m15JdclD007040 for ; Tue, 5 Feb 2008 11:39:38 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m15JdZ8W026595; Tue, 5 Feb 2008 11:39:36 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JVS002036M0KT00@brm-avmta-1.central.sun.com>; Tue, 05 Feb 2008 12:39:36 -0700 (MST) Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.63]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JVS00DP36LZCRA0@brm-avmta-1.central.sun.com>; Tue, 05 Feb 2008 12:39:35 -0700 (MST) Received: from [129.150.12.37] (vpn-129-150-12-37.SFBay.Sun.COM [129.150.12.37]) by jurassic-x4600.sfbay.sun.com (8.14.2+Sun/8.14.2) with ESMTP id m15JdXFF686679; Tue, 05 Feb 2008 11:39:34 -0800 (PST) Date: Tue, 05 Feb 2008 09:38:02 -1000 From: Joseph Kowalski Subject: Re: PSARC/2008/078 - postrun - delayed execution environment for procedural package scripts In-reply-to: <20080205181345.GE236823@eng.sun.com> To: Stephen Hahn Cc: John Plocher , laca@sun.com, john.fischer@sun.com, psarc-ext@sun.com, Sebastien Roy , Danek Duvall Message-id: <47A8BB1A.5040708@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <47A75C4C.8040300@sun.com> <47A7BBF2.9050001@sun.com> <1202177589.1528.41.camel@localhost> <47A877EB.9000100@Sun.Com> <20080205181345.GE236823@eng.sun.com> User-Agent: Thunderbird 2.0.0.6 (X11/20071119) Status: RO Content-Length: 626 Stephen Hahn wrote: > I'd also like to separate the tasks that need > to be done every reboot/reconstruction from version-specific > transitions, which I don't believe is done in the current tool. > I'm behind in this discussion (so maybe I'm missing something), but isn't "reboot" something we should strive to avoid? Sometimes its needed, but we should do everything possible to avoid it. (In particular, it seems that many of the JDS/Gnome things don't require a reboot,... only a restart of something at the X-level or perhaps higher.) Are we doing enough to avoid reboots,... does this encourage it? - jek3 From Nicolas.Williams@sun.com Tue Feb 5 11:58:22 2008 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m15JwMGP007837 for ; Tue, 5 Feb 2008 11:58:22 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m15JwGHR002389; Tue, 5 Feb 2008 11:58:18 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JVS000PB7H4RE00@nwk-avmta-2.sfbay.sun.com>; Tue, 05 Feb 2008 11:58:16 -0800 (PST) Received: from binky.Central.Sun.COM ([129.153.128.104]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JVS00M6L7F1HM40@nwk-avmta-2.sfbay.sun.com>; Tue, 05 Feb 2008 11:57:02 -0800 (PST) Received: from binky.Central.Sun.COM (localhost [127.0.0.1]) by binky.Central.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id m15Jv1nS024058; Tue, 05 Feb 2008 13:57:01 -0600 (CST) Received: (from nw141292@localhost) by binky.Central.Sun.COM (8.14.1+Sun/8.14.1/Submit) id m15Jv10k024057; Tue, 05 Feb 2008 13:57:01 -0600 (CST) Date: Tue, 05 Feb 2008 13:57:01 -0600 From: Nicolas Williams Subject: Re: PSARC/2008/078 - postrun - delayed execution environment for procedural package scripts In-reply-to: <200802050812.m158CuRI005413@dm-holland-02.uk.sun.com> To: Casper.Dik@sun.com Cc: laca@sun.com, john.fischer@sun.com, psarc-ext@sun.com, Sebastien Roy Mail-followup-to: Casper.Dik@Sun.COM, laca@sun.com, john.fischer@sun.com, psarc-ext@sun.com, Sebastien Roy Message-id: <20080205195701.GW19708@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <47A75C4C.8040300@sun.com> <47A7BBF2.9050001@sun.com> <1202177589.1528.41.camel@localhost> <200802050812.m158CuRI005413@dm-holland-02.uk.sun.com> X-Authentication-warning: binky.Central.Sun.COM: nw141292 set sender to Nicolas.Williams@sun.com using -f User-Agent: Mutt/1.5.7i Status: RO Content-Length: 529 On Tue, Feb 05, 2008 at 09:12:55AM +0100, Casper.Dik@Sun.COM wrote: > I note that this is a lot worse for people who update biweekly,, > but I also wonder why postrun needs to run at standard priority? > > Some systems will run certain services faster and appear up more quickly > if postrun is run at a lower CPU priority. Would it be useful to be able to associate postrun with services? Think "update method". After an alt-root/LU upgrade every affected service would have it's update method run prior to its start method. From casper@holland.sun.com Tue Feb 5 12:22:18 2008 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m15KMIN5009504 for ; Tue, 5 Feb 2008 12:22:18 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m15KMH9b006717; Tue, 5 Feb 2008 12:22:17 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JVS00JW18L2VD10@nwk-avmta-1.sfbay.Sun.COM>; Tue, 05 Feb 2008 12:22:14 -0800 (PST) Received: from dm-holland-02.uk.sun.com ([129.156.101.225]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JVS00E5O8359BE0@nwk-avmta-1.sfbay.Sun.COM>; Tue, 05 Feb 2008 12:11:30 -0800 (PST) Received: from holland (room101.Holland.Sun.COM [10.16.117.40]) by dm-holland-02.uk.sun.com (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id m15KBQSU013329; Tue, 05 Feb 2008 20:11:26 +0000 (GMT) Date: Tue, 05 Feb 2008 21:11:26 +0100 From: Casper.Dik@sun.com Subject: Re: PSARC/2008/078 - postrun - delayed execution environment for procedural package scripts In-reply-to: <20080205195701.GW19708@Sun.COM> Sender: casper@holland.sun.com To: Nicolas Williams Cc: laca@sun.com, John.Fischer@sun.com, psarc-ext@sun.com, Sebastien Roy Message-id: <200802052011.m15KBQSU013329@dm-holland-02.uk.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <47A75C4C.8040300@sun.com> <47A7BBF2.9050001@sun.com> <1202177589.1528.41.camel@localhost> <200802050812.m158CuRI005413@dm-holland-02.uk.sun.com> <20080205195701.GW19708@Sun.COM> Status: RO Content-Length: 1632 >On Tue, Feb 05, 2008 at 09:12:55AM +0100, Casper.Dik@Sun.COM wrote: >> I note that this is a lot worse for people who update biweekly,, >> but I also wonder why postrun needs to run at standard priority? >> >> Some systems will run certain services faster and appear up more quickly >> if postrun is run at a lower CPU priority. > >Would it be useful to be able to associate postrun with services? >Think "update method". After an alt-root/LU upgrade every affected >service would have it's update method run prior to its start method. That would have been my next suggestion if you hadn't beaten me to it. What seems natural to me is that each "postrun" script has a natural dependency, e.g., GNOME or etc. (fc-cache -> cde-login) (cde-printinfo -> cde-login) And that these should somehow be grouped; this would additionally allow us to do partial initializations. No dtlogin, no fc-cache). We also have a few services which do this too (cde-print-info, fc-cache, ppd-cache-update) and I am assuming that these will all be moved to the new scheme). Are the "postrun" methods required to be idempotent? "SMC" carried its own "postrun" method and ran it when it first started (good because it didn't need to run for all systems, bad because that took a LONG time). Unfortunately, I noticed that when you ran several upgrades and then started SMC it would not work, because it would try to run some "postrun stuff" twice and that cause breakage. So what will happen when you lvieupgrade the same partition twice without even once booting into it? That's easily done and should not cause any breakage. Casper From Nicolas.Williams@sun.com Tue Feb 5 12:40:33 2008 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m15KeW2I009767 for ; Tue, 5 Feb 2008 12:40:32 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id m15KeBXM003261; Wed, 6 Feb 2008 04:40:22 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JVS004579F8OU10@nwk-avmta-2.sfbay.sun.com>; Tue, 05 Feb 2008 12:40:20 -0800 (PST) Received: from binky.Central.Sun.COM ([129.153.128.104]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JVS00MSJ9F6HR60@nwk-avmta-2.sfbay.sun.com>; Tue, 05 Feb 2008 12:40:19 -0800 (PST) Received: from binky.Central.Sun.COM (localhost [127.0.0.1]) by binky.Central.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id m15KeEtj024094; Tue, 05 Feb 2008 14:40:14 -0600 (CST) Received: (from nw141292@localhost) by binky.Central.Sun.COM (8.14.1+Sun/8.14.1/Submit) id m15KeESj024093; Tue, 05 Feb 2008 14:40:14 -0600 (CST) Date: Tue, 05 Feb 2008 14:40:14 -0600 From: Nicolas Williams Subject: Re: PSARC/2008/078 - postrun - delayed execution environment for procedural package scripts In-reply-to: <200802052011.m15KBQSU013329@dm-holland-02.uk.sun.com> To: Casper.Dik@sun.com Cc: laca@sun.com, John.Fischer@sun.com, psarc-ext@sun.com, Sebastien Roy Mail-followup-to: Casper.Dik@Sun.COM, laca@sun.com, John.Fischer@sun.com, psarc-ext@sun.com, Sebastien Roy Message-id: <20080205204014.GX19708@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <47A75C4C.8040300@sun.com> <47A7BBF2.9050001@sun.com> <1202177589.1528.41.camel@localhost> <200802050812.m158CuRI005413@dm-holland-02.uk.sun.com> <20080205195701.GW19708@Sun.COM> <200802052011.m15KBQSU013329@dm-holland-02.uk.sun.com> X-Authentication-warning: binky.Central.Sun.COM: nw141292 set sender to Nicolas.Williams@sun.com using -f User-Agent: Mutt/1.5.7i Status: RO Content-Length: 954 On Tue, Feb 05, 2008 at 09:11:26PM +0100, Casper.Dik@Sun.COM wrote: > So what will happen when you lvieupgrade the same partition twice without > even once booting into it? That's easily done and should not cause > any breakage. Well, our class action scripts tend to be written so this just works. Postrun uses should also be idempotent. I asked earlier about class action scripts because there will be, surely (right?) be cases where we'll need class actions to run when the system boots, and if we don't have the infrastructure for it then it will be surely re-invented multiple times, some of them badly. Also, to what service should updates of, say, /etc/passwd, belong? Sure, today updates of that specific file can run in normal class action files, but use your imagination -- I'm talking about files which don't belong to any one service but which must be updated early on. Which service should update them? Probably a new one. Nico -- From Laszlo.Peter@sun.com Tue Feb 5 12:49:48 2008 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m15Knm3k010158 for ; Tue, 5 Feb 2008 12:49:48 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m15Knirw017769 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Tue, 5 Feb 2008 12:49:48 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JVS000W79UZR920@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Tue, 05 Feb 2008 12:49:47 -0800 (PST) Received: from gmp-eb-mail-2.sun.com ([192.18.6.24]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JVS00M6X9UXHR70@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Tue, 05 Feb 2008 12:49:46 -0800 (PST) Received: from fe-emea-09.sun.com (gmp-eb-lb-2-fe3.eu.sun.com [192.18.6.12]) by gmp-eb-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m15Knidv002493 for ; Tue, 05 Feb 2008 20:49:44 +0000 (GMT) Received: from conversion-daemon.fe-emea-09.sun.com by fe-emea-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JVS00H0172CC600@fe-emea-09.sun.com> (original mail from Laszlo.Peter@Sun.COM) for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Tue, 05 Feb 2008 20:49:44 +0000 (GMT) Received: from [192.168.2.101] ([60.234.117.94]) by fe-emea-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JVS002LD9URHE00@fe-emea-09.sun.com>; Tue, 05 Feb 2008 20:49:43 +0000 (GMT) Date: Wed, 06 Feb 2008 09:49:41 +1300 From: "Laszlo (Laca) Peter" Subject: Re: PSARC/2008/078 - postrun - delayed execution environment for procedural package scripts In-reply-to: <47A877EB.9000100@Sun.Com> Sender: Laszlo.Peter@sun.com To: John Plocher Cc: Sebastien Roy , John.Fischer@sun.com, psarc-ext@sun.com, Danek Duvall Reply-to: laca@sun.com Message-id: <1202244581.1526.15.camel@localhost> Organization: Sun Microsystems, Inc. MIME-version: 1.0 X-Mailer: Evolution 2.12.2 Content-type: text/plain Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <47A75C4C.8040300@sun.com> <47A7BBF2.9050001@sun.com> <1202177589.1528.41.camel@localhost> <47A877EB.9000100@Sun.Com> Status: RO Content-Length: 1495 On Tue, 2008-02-05 at 06:51 -0800, John Plocher wrote: > Laszlo (Laca) Peter wrote: > > > gnome-session waits for all postrun scripts in the JDS_wait > > > A few questions/comments ... > > jmp-1) Brand and organization names like JDS should never be used in > architectural level artifacts - rather a descriptive word should > be used instead. Understood. JDS here refers to the desktop environment, in contrast with CDE or KDE. > jmp-2) There should be a conceptual distinction between graphics system > startup, login screen, and desktop startup. This would allow the user > or admin to use KDE (or CDE or ...) instead of GNOME as their desktop. Right. That's why it's gnome-session that waits for postrun and not X or gdm. Note that this doesn't normally happen: by the time the user logs in the postrun jobs are finished on most systems, only some lower end machine will ever have to wait. > If the postrun stuff is tied to only the gnome desktop, yet it does > things for the other two conceptual areas, we have problems. postrun itself is not tied to the GNOME desktop, but the jobs submitted to postrun are. > In the same > way, if postrun is a JDS-only thing, what mechanism would KDE (...) use? KDE could (in theory) use postrun and define its own class of jobs that it may need to wait for. > jmp-3) How does this relate to the work being done in IPS, which is also > trying to fix/eliminate postinstall scripts? Stephen Hahn has already responded to this. Laca From bart.smaalders@sun.com Tue Feb 5 12:55:56 2008 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m15KttNv010445 for ; Tue, 5 Feb 2008 12:55:55 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m15KtovB021986; Tue, 5 Feb 2008 12:55:50 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JVS00L0FA526600@brm-avmta-1.central.sun.com>; Tue, 05 Feb 2008 13:55:50 -0700 (MST) Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.59]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JVS0074QA51SY30@brm-avmta-1.central.sun.com>; Tue, 05 Feb 2008 13:55:50 -0700 (MST) Received: from [129.146.228.109] (cyber.SFBay.Sun.COM [129.146.228.109]) by jurassic-x4600.sfbay.sun.com (8.14.2+Sun/8.14.2) with ESMTP id m15KtnHq699834; Tue, 05 Feb 2008 12:55:49 -0800 (PST) Date: Tue, 05 Feb 2008 12:51:09 -0800 From: Bart Smaalders Subject: Re: PSARC/2008/078 - postrun - delayed execution environment for procedural package scripts In-reply-to: <20080205204014.GX19708@Sun.COM> To: Casper.Dik@sun.com, laca@sun.com, John.Fischer@sun.com, psarc-ext@sun.com, Sebastien Roy Message-id: <47A8CC3D.6010004@Sun.COM> Organization: Sun Microsystems MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <47A75C4C.8040300@sun.com> <47A7BBF2.9050001@sun.com> <1202177589.1528.41.camel@localhost> <200802050812.m158CuRI005413@dm-holland-02.uk.sun.com> <20080205195701.GW19708@Sun.COM> <200802052011.m15KBQSU013329@dm-holland-02.uk.sun.com> <20080205204014.GX19708@Sun.COM> User-Agent: Thunderbird 2.0.0.6 (X11/20070924) Status: RO Content-Length: 1332 Nicolas Williams wrote: > On Tue, Feb 05, 2008 at 09:11:26PM +0100, Casper.Dik@Sun.COM wrote: >> So what will happen when you lvieupgrade the same partition twice without >> even once booting into it? That's easily done and should not cause >> any breakage. > > Well, our class action scripts tend to be written so this just works. > > Postrun uses should also be idempotent. > > I asked earlier about class action scripts because there will be, surely > (right?) be cases where we'll need class actions to run when the system > boots, and if we don't have the infrastructure for it then it will be > surely re-invented multiple times, some of them badly. > > Also, to what service should updates of, say, /etc/passwd, belong? > Sure, today updates of that specific file can run in normal class action > files, but use your imagination -- I'm talking about files which don't > belong to any one service but which must be updated early on. Which > service should update them? Probably a new one. > > Nico /etc/passwd is one of the examples of something we're handling at install time w/ IPS; the packaging system has to know how to manipulate this file in order to install new files owned by new users correctly. - Bart -- Bart Smaalders Solaris Kernel Performance barts@cyber.eng.sun.com http://blogs.sun.com/barts From sommerfeld@sun.com Tue Feb 5 12:58:13 2008 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m15KwCb4010493 for ; Tue, 5 Feb 2008 12:58:13 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m15Kw2rb015797; Tue, 5 Feb 2008 20:58:11 GMT Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JVS0010DA8WMJ00@nwk-avmta-1.sfbay.Sun.COM>; Tue, 05 Feb 2008 12:58:08 -0800 (PST) Received: from dm-east-02.east.sun.com ([129.148.13.5]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JVS000BDA8UOJ10@nwk-avmta-1.sfbay.Sun.COM>; Tue, 05 Feb 2008 12:58:07 -0800 (PST) Received: from thunk.east.sun.com (thunk.East.Sun.COM [129.148.174.66]) by dm-east-02.east.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m15Kw53k031877; Tue, 05 Feb 2008 15:58:05 -0500 (EST) Received: from [IPv6:::1] (localhost [IPv6:::1]) by thunk.east.sun.com (8.14.2+Sun/8.14.2) with ESMTP id m15Kw44u018472; Tue, 05 Feb 2008 15:58:04 -0500 (EST) Date: Tue, 05 Feb 2008 15:58:04 -0500 From: Bill Sommerfeld Subject: Re: PSARC/2008/078 - postrun - delayed execution environment for procedural package scripts In-reply-to: <200802050812.m158CuRI005413@dm-holland-02.uk.sun.com> To: Casper.Dik@sun.com Cc: laca@sun.com, John.Fischer@sun.com, psarc-ext@sun.com, Sebastien Roy Message-id: <1202245084.17750.8.camel@thunk> MIME-version: 1.0 X-Mailer: Evolution 2.12.2 Content-type: text/plain Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <47A75C4C.8040300@sun.com> <47A7BBF2.9050001@sun.com> <1202177589.1528.41.camel@localhost> <200802050812.m158CuRI005413@dm-holland-02.uk.sun.com> Status: RO Content-Length: 593 On Tue, 2008-02-05 at 09:12 +0100, Casper.Dik@sun.com wrote: > I find that postrun currently is an abomination (or at least the > consumers are); it runs 10,000s of processes (literally) and takes > a long time; making it public would likely make this worse. The amount of run-on-first-boot work we've accumulated in nevada is absolutely astounding; i'm not sure it's all via postrun but there's clearly a lot of it. I've been waiting about an hour for two small one-disk systems I've just installed with three non-global zones to complete first-boot work on all the zones. - Bill From Laszlo.Peter@sun.com Tue Feb 5 12:59:47 2008 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m15KxlUp010533 for ; Tue, 5 Feb 2008 12:59:47 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m15KxjNK022981 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Tue, 5 Feb 2008 12:59:47 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JVS00L09ABMED00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 05 Feb 2008 13:59:46 -0700 (MST) Received: from gmp-eb-mail-2.sun.com ([192.18.6.24]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JVS007CGABLTJ30@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 05 Feb 2008 13:59:46 -0700 (MST) Received: from fe-emea-10.sun.com (gmp-eb-lb-2-fe1.eu.sun.com [192.18.6.10]) by gmp-eb-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m15KxjMv002697 for ; Tue, 05 Feb 2008 20:59:45 +0000 (GMT) Received: from conversion-daemon.fe-emea-10.sun.com by fe-emea-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JVS00B019UP8U00@fe-emea-10.sun.com> (original mail from Laszlo.Peter@Sun.COM) for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 05 Feb 2008 20:59:45 +0000 (GMT) Received: from [192.168.2.101] ([60.234.117.94]) by fe-emea-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JVS00HF0ABG6XB0@fe-emea-10.sun.com>; Tue, 05 Feb 2008 20:59:45 +0000 (GMT) Date: Wed, 06 Feb 2008 09:59:42 +1300 From: "Laszlo (Laca) Peter" Subject: Re: PSARC/2008/078 - postrun - delayed execution environment for procedural package scripts In-reply-to: <47A8BB1A.5040708@sun.com> Sender: Laszlo.Peter@sun.com To: Joseph Kowalski Cc: Stephen Hahn , John Plocher , John.Fischer@sun.com, psarc-ext@sun.com, Sebastien Roy , Danek Duvall Reply-to: laca@sun.com Message-id: <1202245182.1526.23.camel@localhost> Organization: Sun Microsystems, Inc. MIME-version: 1.0 X-Mailer: Evolution 2.12.2 Content-type: text/plain Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <47A75C4C.8040300@sun.com> <47A7BBF2.9050001@sun.com> <1202177589.1528.41.camel@localhost> <47A877EB.9000100@Sun.Com> <20080205181345.GE236823@eng.sun.com> <47A8BB1A.5040708@sun.com> Status: RO Content-Length: 979 On Tue, 2008-02-05 at 09:38 -1000, Joseph Kowalski wrote: > Stephen Hahn wrote: > > I'd also like to separate the tasks that need > > to be done every reboot/reconstruction from version-specific > > transitions, which I don't believe is done in the current tool. > > > I'm behind in this discussion (so maybe I'm missing something), but isn't > "reboot" something we should strive to avoid? Sometimes its needed, but > we should do everything possible to avoid it. Yes. If you run pkgrm/pkgadd on a live system, postrun will execute the job immediately, no reboot needed. If you install with Jumpstart or LU or other altroot installation methods, there will be a reboot at the end anyway, so no extra reboot is needed. Laca > (In particular, it seems that many of the JDS/Gnome things don't require > a reboot,... only a restart of something at the X-level or perhaps higher.) > > Are we doing enough to avoid reboots,... does this encourage it? > > - jek3 > From casper@holland.sun.com Tue Feb 5 13:02:10 2008 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m15L2AqI010731 for ; Tue, 5 Feb 2008 13:02:10 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m15L26GF025945; Tue, 5 Feb 2008 14:02:08 -0700 (MST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JVS00L2TAFIMB00@brm-avmta-1.central.sun.com>; Tue, 05 Feb 2008 14:02:06 -0700 (MST) Received: from dm-holland-02.uk.sun.com ([129.156.101.225]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JVS007EZAFFT930@brm-avmta-1.central.sun.com>; Tue, 05 Feb 2008 14:02:04 -0700 (MST) Received: from holland (room101.Holland.Sun.COM [10.16.117.40]) by dm-holland-02.uk.sun.com (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id m15L214s003159; Tue, 05 Feb 2008 21:02:01 +0000 (GMT) Date: Tue, 05 Feb 2008 22:02:01 +0100 From: Casper.Dik@sun.com Subject: Re: PSARC/2008/078 - postrun - delayed execution environment for procedural package scripts In-reply-to: <1202245084.17750.8.camel@thunk> Sender: casper@holland.sun.com To: Bill Sommerfeld Cc: laca@sun.com, John.Fischer@sun.com, psarc-ext@sun.com, Sebastien Roy Message-id: <200802052102.m15L214s003159@dm-holland-02.uk.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <47A75C4C.8040300@sun.com> <47A7BBF2.9050001@sun.com> <1202177589.1528.41.camel@localhost> <200802050812.m158CuRI005413@dm-holland-02.uk.sun.com> <1202245084.17750.8.camel@thunk> Status: RO Content-Length: 768 > >On Tue, 2008-02-05 at 09:12 +0100, Casper.Dik@sun.com wrote: >> I find that postrun currently is an abomination (or at least the >> consumers are); it runs 10,000s of processes (literally) and takes >> a long time; making it public would likely make this worse. > >The amount of run-on-first-boot work we've accumulated in nevada is >absolutely astounding; i'm not sure it's all via postrun but there's >clearly a lot of it. > >I've been waiting about an hour for two small one-disk systems I've just >installed with three non-global zones to complete first-boot work on all >the zones. Manifest import is a large part of that, no doubt. (Is it my imagination or is manifest-import MUCH slower on systems with more than one hardware execution thread) Casper From Nicolas.Williams@sun.com Tue Feb 5 13:09:19 2008 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m15L9J9j010810 for ; Tue, 5 Feb 2008 13:09:19 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m15L9B7q025977; Tue, 5 Feb 2008 13:09:15 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JVS0020LARFNX00@nwk-avmta-1.sfbay.Sun.COM>; Tue, 05 Feb 2008 13:09:15 -0800 (PST) Received: from binky.Central.Sun.COM ([129.153.128.104]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JVS000YEAREOP10@nwk-avmta-1.sfbay.Sun.COM>; Tue, 05 Feb 2008 13:09:14 -0800 (PST) Received: from binky.Central.Sun.COM (localhost [127.0.0.1]) by binky.Central.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id m15L9EXf024136; Tue, 05 Feb 2008 15:09:14 -0600 (CST) Received: (from nw141292@localhost) by binky.Central.Sun.COM (8.14.1+Sun/8.14.1/Submit) id m15L9EZe024135; Tue, 05 Feb 2008 15:09:14 -0600 (CST) Date: Tue, 05 Feb 2008 15:09:14 -0600 From: Nicolas Williams Subject: Re: PSARC/2008/078 - postrun - delayed execution environment for procedural package scripts In-reply-to: <47A8CC3D.6010004@Sun.COM> To: Bart Smaalders Cc: Casper.Dik@sun.com, laca@sun.com, John.Fischer@sun.com, psarc-ext@sun.com, Sebastien Roy Mail-followup-to: Bart Smaalders , Casper.Dik@sun.com, laca@sun.com, John.Fischer@sun.com, psarc-ext@sun.com, Sebastien Roy Message-id: <20080205210913.GZ19708@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <47A75C4C.8040300@sun.com> <47A7BBF2.9050001@sun.com> <1202177589.1528.41.camel@localhost> <200802050812.m158CuRI005413@dm-holland-02.uk.sun.com> <20080205195701.GW19708@Sun.COM> <200802052011.m15KBQSU013329@dm-holland-02.uk.sun.com> <20080205204014.GX19708@Sun.COM> <47A8CC3D.6010004@Sun.COM> X-Authentication-warning: binky.Central.Sun.COM: nw141292 set sender to Nicolas.Williams@sun.com using -f User-Agent: Mutt/1.5.7i Status: RO Content-Length: 519 On Tue, Feb 05, 2008 at 12:51:09PM -0800, Bart Smaalders wrote: > /etc/passwd is one of the examples of something we're handling > at install time w/ IPS; the packaging system has to know how > to manipulate this file in order to install new files owned > by new users correctly. I thought IPS would be script-less. But whatever. Surely there will be files to upgrade where IPS won't know how, though we should probably be seeking to keep those to an absolute minimum by putting more and more config content in SMF. From Laszlo.Peter@sun.com Tue Feb 5 13:09:47 2008 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m15L9kKT010843 for ; Tue, 5 Feb 2008 13:09:46 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m15L9ch8020709 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Tue, 5 Feb 2008 21:09:45 GMT Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JVS0020LAS8OJ00@nwk-avmta-1.sfbay.Sun.COM> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Tue, 05 Feb 2008 13:09:44 -0800 (PST) Received: from gmp-eb-mail-2.sun.com ([192.18.6.24]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JVS00006AS6OP20@nwk-avmta-1.sfbay.Sun.COM> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Tue, 05 Feb 2008 13:09:43 -0800 (PST) Received: from fe-emea-10.sun.com (gmp-eb-lb-2-fe2.eu.sun.com [192.18.6.11]) by gmp-eb-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m15L9gi6002938 for ; Tue, 05 Feb 2008 21:09:42 +0000 (GMT) Received: from conversion-daemon.fe-emea-10.sun.com by fe-emea-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JVS00301AP9S600@fe-emea-10.sun.com> (original mail from Laszlo.Peter@Sun.COM) for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Tue, 05 Feb 2008 21:09:42 +0000 (GMT) Received: from [192.168.2.101] ([60.234.117.94]) by fe-emea-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JVS00H0UAS26XE0@fe-emea-10.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Tue, 05 Feb 2008 21:09:42 +0000 (GMT) Date: Wed, 06 Feb 2008 10:09:39 +1300 From: "Laszlo (Laca) Peter" Subject: Re: PSARC/2008/078 - postrun - delayed execution environment for procedural package scripts In-reply-to: <200802052011.m15KBQSU013329@dm-holland-02.uk.sun.com> Sender: Laszlo.Peter@sun.com To: Casper.Dik@sun.com Cc: Nicolas Williams , John.Fischer@sun.com, psarc-ext@sun.com, Sebastien Roy Reply-to: laca@sun.com Message-id: <1202245779.1526.31.camel@localhost> Organization: Sun Microsystems, Inc. MIME-version: 1.0 X-Mailer: Evolution 2.12.2 Content-type: text/plain Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <47A75C4C.8040300@sun.com> <47A7BBF2.9050001@sun.com> <1202177589.1528.41.camel@localhost> <200802050812.m158CuRI005413@dm-holland-02.uk.sun.com> <20080205195701.GW19708@Sun.COM> <200802052011.m15KBQSU013329@dm-holland-02.uk.sun.com> Status: RO Content-Length: 437 On Tue, 2008-02-05 at 21:11 +0100, Casper.Dik@Sun.COM wrote: > > So what will happen when you lvieupgrade the same partition twice without > even once booting into it? That's easily done and should not cause > any breakage. It should work with postrun. Most, if not all postrun jobs in GNOME are called with the -u (unique) flag, which causes them to be executed once only, no matter how many package scripts called them. Laca From Laszlo.Peter@sun.com Tue Feb 5 18:13:12 2008 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m162DCBN023129 for ; Tue, 5 Feb 2008 18:13:12 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m162DB8l015551 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Tue, 5 Feb 2008 18:13:11 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JVS0080FOTZW200@nwk-avmta-1.sfbay.Sun.COM> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Tue, 05 Feb 2008 18:13:11 -0800 (PST) Received: from gmp-eb-mail-1.sun.com ([192.18.6.21]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JVS00KRFOTYGPD0@nwk-avmta-1.sfbay.Sun.COM> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Tue, 05 Feb 2008 18:13:11 -0800 (PST) Received: from fe-emea-10.sun.com (gmp-eb-lb-2-fe1.eu.sun.com [192.18.6.10]) by gmp-eb-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m162D9g1010756 for ; Wed, 06 Feb 2008 02:13:09 +0000 (GMT) Received: from conversion-daemon.fe-emea-10.sun.com by fe-emea-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JVS00K01OENY000@fe-emea-10.sun.com> (original mail from Laszlo.Peter@Sun.COM) for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 06 Feb 2008 02:13:09 +0000 (GMT) Received: from [192.168.2.101] ([60.234.117.94]) by fe-emea-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JVS00AH7OTU3FE0@fe-emea-10.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 06 Feb 2008 02:13:09 +0000 (GMT) Date: Wed, 06 Feb 2008 15:13:06 +1300 From: "Laszlo (Laca) Peter" Subject: Re: PSARC/2008/078 - postrun - delayed execution environment for procedural package scripts In-reply-to: <200802050812.m158CuRI005413@dm-holland-02.uk.sun.com> Sender: Laszlo.Peter@sun.com To: Casper.Dik@sun.com Cc: Sebastien Roy , John.Fischer@sun.com, psarc-ext@sun.com Reply-to: laca@sun.com Message-id: <1202263986.1526.55.camel@localhost> Organization: Sun Microsystems, Inc. MIME-version: 1.0 X-Mailer: Evolution 2.12.2 Content-type: text/plain Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <47A75C4C.8040300@sun.com> <47A7BBF2.9050001@sun.com> <1202177589.1528.41.camel@localhost> <200802050812.m158CuRI005413@dm-holland-02.uk.sun.com> Status: RO Content-Length: 994 On Tue, 2008-02-05 at 09:12 +0100, Casper.Dik@Sun.COM wrote: > I find that postrun currently is an abomination (or at least the > consumers are); it runs 10,000s of processes (literally) and takes > a long time; making it public would likely make this worse. I find that number surprising. It shouldn't be the case and if it is, there are bugs to be fixed (probably incorrect or unnecessary scripts in the consumers). I fixed a few in some scripts in recent development builds, targetting snv_88. > Will its use be sufficiently discouraged? (And will current postrun > consumers be fixed to run more efficiently)? I think Consolidation Private is sufficiently discouraging. > I note that this is a lot worse for people who update biweekly,, > but I also wonder why postrun needs to run at standard priority? > > Some systems will run certain services faster and appear up more > quickly if postrun is run at a lower CPU priority. That's a good idea, I'll investigate. Thanks, Laca From casper@holland.sun.com Wed Feb 6 00:23:01 2008 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m168N02S000778 for ; Wed, 6 Feb 2008 00:23:01 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id m168MqxI024144; Wed, 6 Feb 2008 16:22:57 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JVT0070J5Y6I100@brm-avmta-1.central.sun.com>; Wed, 06 Feb 2008 01:22:54 -0700 (MST) Received: from dm-holland-02.uk.sun.com ([129.156.101.225]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JVT000PF5Y5IZ80@brm-avmta-1.central.sun.com>; Wed, 06 Feb 2008 01:22:53 -0700 (MST) Received: from holland (room101.Holland.Sun.COM [10.16.117.40]) by dm-holland-02.uk.sun.com (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id m168Mpht009643; Wed, 06 Feb 2008 08:22:51 +0000 (GMT) Date: Wed, 06 Feb 2008 09:22:51 +0100 From: Casper.Dik@sun.com Subject: Re: PSARC/2008/078 - postrun - delayed execution environment for procedural package scripts In-reply-to: <1202263986.1526.55.camel@localhost> Sender: casper@holland.sun.com To: laca@sun.com Cc: Sebastien Roy , John.Fischer@sun.com, psarc-ext@sun.com Message-id: <200802060822.m168Mpht009643@dm-holland-02.uk.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <47A75C4C.8040300@sun.com> <47A7BBF2.9050001@sun.com> <1202177589.1528.41.camel@localhost> <200802050812.m158CuRI005413@dm-holland-02.uk.sun.com> <1202263986.1526.55.camel@localhost> Status: RO Content-Length: 819 > >On Tue, 2008-02-05 at 09:12 +0100, Casper.Dik@Sun.COM wrote: >> I find that postrun currently is an abomination (or at least the >> consumers are); it runs 10,000s of processes (literally) and takes >> a long time; making it public would likely make this worse. > >I find that number surprising. It shouldn't be the case and if >it is, there are bugs to be fixed (probably incorrect or >unnecessary scripts in the consumers). I fixed a few in some >scripts in recent development builds, targetting snv_88. Well, look at the PIDs you get when you login after boot. E.g., this system was live upgraded and booted; (17:34 on 1/31) and dtlogin started on 17:36 on 1/31 has PID 15164, initrd 13010, etc) So certainly SOMETHING ran 10,000+ processes. I've seen complaints about this in other places too. Casper From Darren.Reed@sun.com Wed Feb 6 02:30:07 2008 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m16AU70G002749 for ; Wed, 6 Feb 2008 02:30:07 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m16AU3fm020926 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Wed, 6 Feb 2008 02:30:07 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JVT00H15BU6TJ00@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 06 Feb 2008 03:30:06 -0700 (MST) Received: from sineb-mail-2.sun.com ([192.18.19.7]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JVT00AIDBU41N50@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 06 Feb 2008 03:30:05 -0700 (MST) Received: from fe-apac-06.sun.com (fe-apac-06.sun.com [192.18.19.177] (may be forged)) by sineb-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id m16AU73s029589 for ; Wed, 06 Feb 2008 10:30:07 +0000 (GMT) Received: from conversion-daemon.mail-apac.sun.com by mail-apac.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JVT00001BRUQJ00@mail-apac.sun.com> (original mail from Darren.Reed@Sun.COM) for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 06 Feb 2008 18:30:03 +0800 (SGT) Received: from [129.158.87.228] by mail-apac.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JVT007AIBU1BIQN@mail-apac.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 06 Feb 2008 18:30:03 +0800 (SGT) Date: Wed, 06 Feb 2008 21:28:39 +1100 From: Darren Reed Subject: Re: PSARC/2008/078 - postrun - delayed execution environment for procedural package scripts In-reply-to: <200802060822.m168Mpht009643@dm-holland-02.uk.sun.com> Sender: Darren.Reed@sun.com To: Casper.Dik@sun.com Cc: laca@sun.com, Sebastien Roy , John.Fischer@sun.com, psarc-ext@sun.com Message-id: <47A98BD7.5090006@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <47A75C4C.8040300@sun.com> <47A7BBF2.9050001@sun.com> <1202177589.1528.41.camel@localhost> <200802050812.m158CuRI005413@dm-holland-02.uk.sun.com> <1202263986.1526.55.camel@localhost> <200802060822.m168Mpht009643@dm-holland-02.uk.sun.com> User-Agent: Thunderbird 1.5.0.13 (Windows/20070809) Status: RO Content-Length: 1015 Casper.Dik@Sun.COM wrote: >> On Tue, 2008-02-05 at 09:12 +0100, Casper.Dik@Sun.COM wrote: >> >>> I find that postrun currently is an abomination (or at least the >>> consumers are); it runs 10,000s of processes (literally) and takes >>> a long time; making it public would likely make this worse. >>> >> I find that number surprising. It shouldn't be the case and if >> it is, there are bugs to be fixed (probably incorrect or >> unnecessary scripts in the consumers). I fixed a few in some >> scripts in recent development builds, targetting snv_88. >> > > Well, look at the PIDs you get when you login after boot. > > E.g., this system was live upgraded and booted; (17:34 on 1/31) and > dtlogin started on 17:36 on 1/31 has PID 15164, initrd 13010, etc) > > So certainly SOMETHING ran 10,000+ processes. I've seen complaints about > this in other places too. > Hmmm, too bad process accounting isn't available early on in bootup, that'd at least allow some evidence gathering. Darren From pete@sorted.org Wed Feb 6 03:25:42 2008 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m16BPfec003481 for ; Wed, 6 Feb 2008 03:25:41 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id m16BPSsf007821; Wed, 6 Feb 2008 19:25:31 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JVT00L09EEH7300@nwk-avmta-1.sfbay.Sun.COM>; Wed, 06 Feb 2008 03:25:29 -0800 (PST) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JVT00ATVEEG2X90@nwk-avmta-1.sfbay.Sun.COM>; Wed, 06 Feb 2008 03:25:28 -0800 (PST) Received: from relay42i.sun.com ([192.5.209.72]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id m16BMwe0018935; Wed, 06 Feb 2008 11:25:28 +0000 (GMT) Received: from mms49es.sun.com ([160.41.221.233] [160.41.221.233]) by relay42i.sun.com with ESMTP id BT-MMP-765347; Wed, 06 Feb 2008 11:25:28 +0000 (Z) Received: from relay43i.sun.com (relay43i.sun.com [192.5.209.74]) by mms49es.sun.com with ESMTP id BT-MMP-1047921; Wed, 06 Feb 2008 11:25:27 +0000 (Z) Received: from radix.sorted.org ([194.70.217.146] [194.70.217.146]) by relay4i.sun.com with ESMTP id BT-MMP-14206014; Wed, 06 Feb 2008 11:25:27 +0000 (Z) Received: by radix.sorted.org (Postfix, from userid 501) id 954332B94F; Wed, 06 Feb 2008 11:25:26 +0000 (GMT) Date: Wed, 06 Feb 2008 11:25:26 +0000 From: Pete Bentley Subject: Re: PSARC/2008/078 - postrun - delayed execution environment for procedural package scripts In-reply-to: <47A98BD7.5090006@Sun.COM> To: Darren Reed Cc: Casper.Dik@sun.com, John.Fischer@sun.com, psarc-ext@sun.com, Sebastien Roy , laca@sun.com Message-id: <20080206112526.GA27903@sorted.org> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 X-Brightmail-Tracker: AAAAAA== X-Antispam: No, score=-1.1/5.0, scanned in 0.108sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <47A75C4C.8040300@sun.com> <47A7BBF2.9050001@sun.com> <1202177589.1528.41.camel@localhost> <200802050812.m158CuRI005413@dm-holland-02.uk.sun.com> <1202263986.1526.55.camel@localhost> <200802060822.m168Mpht009643@dm-holland-02.uk.sun.com> <47A98BD7.5090006@Sun.COM> User-Agent: Mutt/1.4.2.1i Status: RO Content-Length: 1018 On Wed, Feb 06, 2008 at 09:28:39PM +1100, Darren Reed wrote: > Casper.Dik@Sun.COM wrote: > > So certainly SOMETHING ran 10,000+ processes. I've seen complaints about > > this in other places too. > > Hmmm, too bad process accounting isn't available early on in bootup, > that'd at least allow some evidence gathering. I have observed these processes running and noted at the time that they were children of postrun - a couple of my test boxes are Netra X1's and it's particularly obvious on those machines as postrun can take about 15 minutes to complete. Postrun logs its activity to /var/log/postrun.log and the biggest culprits seem to be the jobs which use gconftool-2 to install desktop preference "schemas". I last watched this happening on b71, b81 may have improved as postrun completes in about 2 minutes on an amd64 box, and I'm pretty sure earlier builds were taking 5 minutes or more on the same box. I plan to upgrade at least one X1 to b81 later this week so I'll get a better idea then. Pete. From Darren.Kenny@sun.com Wed Feb 6 07:38:21 2008 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m16FcLAg007225 for ; Wed, 6 Feb 2008 07:38:21 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m16FcHHI022090 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Wed, 6 Feb 2008 07:38:21 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JVT00I19Q3VBL00@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 06 Feb 2008 08:38:19 -0700 (MST) Received: from gmp-eb-mail-2.sun.com ([192.18.6.24]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JVT009PWQ3SMU70@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 06 Feb 2008 08:38:17 -0700 (MST) Received: from fe-emea-10.sun.com (gmp-eb-lb-2-fe3.eu.sun.com [192.18.6.12]) by gmp-eb-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m16FcGjb005965 for ; Wed, 06 Feb 2008 15:38:16 +0000 (GMT) Received: from conversion-daemon.fe-emea-10.sun.com by fe-emea-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JVT00201PVR1U00@fe-emea-10.sun.com> (original mail from Darren.Kenny@Sun.COM) for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 06 Feb 2008 15:38:16 +0000 (GMT) Received: from [129.150.121.192] by fe-emea-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JVT0070EQ3HKYF0@fe-emea-10.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 06 Feb 2008 15:38:06 +0000 (GMT) Date: Wed, 06 Feb 2008 15:37:52 +0000 From: Darren Kenny Subject: Re: PSARC/2008/078 - postrun - delayed execution environment for procedural package scripts In-reply-to: <20080206112526.GA27903@sorted.org> Sender: Darren.Kenny@sun.com To: Pete Bentley Cc: Darren Reed , Casper.Dik@sun.com, John.Fischer@sun.com, psarc-ext@sun.com, Sebastien Roy , laca@sun.com Message-id: <47A9D450.5000809@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <47A75C4C.8040300@sun.com> <47A7BBF2.9050001@sun.com> <1202177589.1528.41.camel@localhost> <200802050812.m158CuRI005413@dm-holland-02.uk.sun.com> <1202263986.1526.55.camel@localhost> <200802060822.m168Mpht009643@dm-holland-02.uk.sun.com> <47A98BD7.5090006@Sun.COM> <20080206112526.GA27903@sorted.org> User-Agent: Thunderbird 2.0.0.9 (X11/20080128) Status: RO Content-Length: 2015 Hi Peter, It's certainly possible that this might have been the case on b71 - since then I do know that Laca has cleaned up the postrun scripts being installed, using a unique instance of the scripts so that even in multiple packages are installed, so long as they all have the same postrun tasks, only one is queued. [1] This results in something like the GConf schema tree being generated only once, after all the packages have been installed (by restarting the postrun service) or on reboot. This is where postrun really is useful - to do otherwise would have meant having to run several commands on each package install, considerably slowing down installation as more and more packages are installed - for one package it's probably not too severe, but it builds up over time if there are a lot. Darren. [1] - If you happen to look in /var/spool/postrun after upgrading JDS, but before a reboot you'll see evidence of this. Pete Bentley wrote: > On Wed, Feb 06, 2008 at 09:28:39PM +1100, Darren Reed wrote: >> Casper.Dik@Sun.COM wrote: >>> So certainly SOMETHING ran 10,000+ processes. I've seen complaints about >>> this in other places too. >> Hmmm, too bad process accounting isn't available early on in bootup, >> that'd at least allow some evidence gathering. > > I have observed these processes running and noted at the time that > they were children of postrun - a couple of my test boxes are Netra X1's > and it's particularly obvious on those machines as postrun can take about > 15 minutes to complete. > > Postrun logs its activity to /var/log/postrun.log and the biggest > culprits seem to be the jobs which use gconftool-2 to install > desktop preference "schemas". > > I last watched this happening on b71, b81 may have improved as > postrun completes in about 2 minutes on an amd64 box, and I'm pretty > sure earlier builds were taking 5 minutes or more on the same box. > > I plan to upgrade at least one X1 to b81 later this week so I'll get > a better idea then. > > Pete. From johnf@sac.sfbay.sun.com Fri Feb 15 11:29:18 2008 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m1FJTI3X015389 for ; Fri, 15 Feb 2008 11:29:18 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m1FJT2vf003366; Fri, 15 Feb 2008 12:29:16 -0700 (MST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JWA0058FOSQ9P00@nwk-avmta-2.sfbay.sun.com>; Fri, 15 Feb 2008 11:29:14 -0800 (PST) Received: from dm-sfbay-02.sfbay.sun.com ([129.146.11.31]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JWA003NNOSL1H70@nwk-avmta-2.sfbay.sun.com>; Fri, 15 Feb 2008 11:29:10 -0800 (PST) Received: from sac.sfbay.sun.com (new-sac.SFBay.Sun.COM [129.146.175.65]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m1FJT5ne015195; Fri, 15 Feb 2008 11:29:05 -0800 (PST) Received: from sac.sfbay.sun.com (localhost [127.0.0.1]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m1FJLZmc015271; Fri, 15 Feb 2008 11:21:35 -0800 (PST) Received: (from johnf@localhost) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id m1FJLY69015270; Fri, 15 Feb 2008 11:21:34 -0800 (PST) Date: Fri, 15 Feb 2008 11:21:34 -0800 (PST) From: John Fischer Subject: Re: PSARC/2008/078 - postrun - delayed execution environment for procedural package scripts To: Darren.Kenny@sun.com, pete@sorted.org Cc: Casper.Dik@sun.com, Darren.Reed@sun.com, John.Fischer@sun.com, Sebastien.Roy@sun.com, laca@sun.com, psarc-ext@sun.com Message-id: <200802151921.m1FJLY69015270@sac.sfbay.sun.com> Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 131 All, The discussion having died down with no architectural change necessary I am closing this fast track approved. Thanks, John