From sacadmin Thu Oct 23 09:58:28 2008 Received: from sac.sfbay.sun.com (localhost [127.0.0.1]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m9NGwSFC007456; Thu, 23 Oct 2008 09:58:28 -0700 (PDT) Received: (from ab196087@localhost) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id m9NGwR87007452; Thu, 23 Oct 2008 09:58:27 -0700 (PDT) Date: Thu, 23 Oct 2008 09:58:27 -0700 (PDT) From: Ali Bahrami Message-Id: <200810231658.m9NGwR87007452@sac.sfbay.sun.com> To: PSARC-record@sac.sfbay.sun.com Subject: New ld archive rescan options [PSARC/2008/651 Self Review] Status: RO Content-Length: 558 Template Version: @(#)sac_nextcase %I% %G% SMI This information is Copyright 2008 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: New ld archive rescan options 1.2. Name of Document Author/Supplier: Author: Ali Bahrami 1.3 Date of This Document: 23 October, 2008 4. Technical Description See the case directory for more detail 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: on 6.5. ARC review type: Automatic 6.6. ARC Exposure: open From Ali.Bahrami@sun.com Thu Oct 23 10:04:46 2008 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m9NH4kj5007591 for ; Thu, 23 Oct 2008 10:04:46 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m9NH4ixa010592 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Thu, 23 Oct 2008 10:04:46 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0K9700L0RBFW8U00@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 23 Oct 2008 10:04:44 -0700 (PDT) Received: from brmea-mail-2.sun.com ([192.18.98.43]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0K9700HCUBFW53C0@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 23 Oct 2008 10:04:44 -0700 (PDT) Received: from fe-amer-09.sun.com ([192.18.109.79]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id m9NH4iva003003 for ; Thu, 23 Oct 2008 17:04:44 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K9700601AB4IA00@mail-amer.sun.com> (original mail from Ali.Bahrami@Sun.COM) for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 23 Oct 2008 11:04:44 -0600 (MDT) Received: from [172.20.25.67] by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K9700GPXBFRWDF0@mail-amer.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 23 Oct 2008 11:04:39 -0600 (MDT) Date: Thu, 23 Oct 2008 11:04:39 -0600 From: Ali Bahrami Subject: PSARC 2008/651 New ld archive rescan options Sender: Ali.Bahrami@sun.com To: psarc-ext@sun.com Message-id: <4900AEA7.3020109@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 User-Agent: Thunderbird 2.0.0.16 (X11/20080922) Status: RO Content-Length: 6267 I am sponsoring the following self-reviewed case for myself. It adds new archive rescan options to the Solaris link-editor (ld). I believe it qualifies for self review: - The new -z rescan-now option is a straightforward variation on the existing -z rescan option. - The idea of archive rescan groups comes from an existing feature of the GNU ld, and is straightforward. Adding it to the Solaris ld is an additional move to improve Linux compatibility. Copies of the current ld manpage (ld.1.orig), the proposed new manpage (ld.1), and diffs between the two (ld.1.diffs) can be found in the case materials. ----- Release Binding: Patch/Micro New ld options: -rescan-now Committed -rescan-start, --start-group, -( Committed -rescan-end, --end-group, -) Committed --------------------------------------------------------------------------- This case delivers an overhaul of archive rescanning by the ld link-editor. It adds new command options. There are two reasons for this work: 1) The existing -z rescan option has a design flaw that must be maintained for backward compatibility. A new option (-z rescan-now) is needed to provide a solution. 2) The GNU linker supports the concept of archive rescan groups. We have encountered makefiles from FOSS (Free and Open Source Software) that have attempted to pass these options to the Solaris ld. Supporting these options lowers the barrier to bringing software to Solaris. These items involve interrelated changes to the same code, making it convenient to address them together at this time. New ld -z rescan-now Option --------------------------- The CR 6748160 problem with -zrescan describes a problem encountered by a program built using objects coming from multiple archives: cc -z rescan name.a main.a The multiple archives have mutual dependencies, requiring multiple passes through each archive, as triggered via -zrescan. When the compiler calls the link-editor, the ld command looks something like: ld .../crti.o .../crt1.o -z rescan name.a main.a -lc .../crtn.o where the .../crt*.o files are supplied by the C compiler. When ld sees a -z rescan, it waits until it has processed all of the command arguments, and then it goes back and rescans all the archives seen. This causes an unfortunate interaction with the crt*.o objects, which was not foreseen when -z rescan was originally defined. ELF init sections are constructed by concatenating the init sections from the objects into a function called by the runtime linker at object initialization. The function entry is provided by crti.o, and the the return by crtn.o. These are PROGBITS sections, with no special attributes other than the fact that they all have the same name (.init). As such, the link-editor is not aware of the crt*.o objects as having any special properties --- it simply concatenates them together in the order given, without interpretation, in the usual manner. In the case given above, any init sections contributed by an object in the archive that are brought into the link via the operation of rescan will be added to the init section in the output object after the function termination provided by crtn.o. As such, that code will never run, and the initializations they contain will not be carried out. The the case of the customer related to CR 6748160, the result of the missing initialization was a NULL pointer dereference and a segmentation fault. Worse is the potential for such a program to not crash in this case, and to instead silently produce incorrect results. The link editor has no basis to prevent this from occurring: - Compilers are not required to name their objects as crt*.o, so we cannot make any assumptions about special file names. - The glue code that is provided by these crt*.o objects are not tagged in a way that uniquely identifies their purpose. - Users are allowed to use the same names for their objects, so even if we could assume the names of these special compiler provided objects, we could only use that information if we knew where the compiler keeps them on disk. Compilers can be installed in arbitrary locations, so such assumptions are unworkable. Note that example here involves init sections, but the issue exists for other sections (e.g. fini), including some that the linker may not even know about. A good solution to this problem would be to make -z rescan positional, taking effect at the point where the link editor encounters it instead of waiting until all arguments have been processed. This would work, assuming that the -z rescan follows the archives needing rescanning. Unfortunately, there are existing makefiles in the field that have been written knowing that the position of -z rescan is unimportant. Changing this aspect of -z rescan would break any existing makefiles where the -z rescan precedes the archives, as in the example I gave above. Had the interaction with init sections been noted when -z rescan was first defined, making it positional would have been the likely answer. However, it is too late to change that now. Since -z rescan cannot be changed, the solution to the problem outlined about is to introduce a new option, -z rescan-now, and to change the ld(1) manpage to indicate that -z rescan is deprecated, and that -z rescan-now should be used instead. Archive Rescan Groups --------------------- The GNU linker supports the notion of archive rescan groups: gld ... --start-group ... --end-group ... or using an alternative syntax gld ... -( ... -) ... When the link editor encounters the closing delimiter of such a group, it immediately initiates an archive rescan operation that considers only archives found inside the group. To lower the barrier to Solaris encountered by code written outside Sun, and because it is a simple extension to the rescan code already being modified, I am taking this opportunity to add support for archive rescan groups to the Solaris link-editor. The native names of these options are -z rescan-start, and -z rescan-end. The GNU ld names described above will also be accepted, following the precedent set by "PSARC 2008/583 add gld options to ld(1)".