sun microsystems Systems Architecture Committee _________________________________________________________________ Subject: Add /usr/xpg4/bin/crontab and /usr/xpg6/bin/crontab Submitted by: Carol Fields File: PSARC/2005/683/opinion.ms Date: November 30th, 2005 Committee: Bill Sommerfeld, Robert Berube Jr, James D Carlson, Ed Gould, Joseph Kowalski, Glenn Skinner, Gary Winiger, Shudong Zhou. Product Approval Committee: Solaris PAC solaris-pac-opinion@Sun.COM 1. Summary This project proposed to deliver xpg4 and xpg6 variants of the "crontab" command which invoke the xpg4 and xpg6 vari- ants of "vi" as their default editors. In addition, the project will deliver a change to the /usr/bin variant of crontab to use "vi" rather than "ed" as the default editor. 2. Decision & Precedence Information This project is approved as specified in references [1] and [2]. The entirety of this project may be delivered in a Minor release of the Solaris OS. In addition, the two xpg vari- ants of crontab may be delivered in a Patch release of the Solaris OS. PSARC/2005/683 Copyright 2005 Sun Microsystems - 2 - 3. Interfaces The project exports the following interfaces. __________________________________________________________ | Interfaces Exported | |_____________________|________________|_________________| |Interface | Classification| Comments | |_____________________|________________|_________________| |/usr/xpg4/bin/crontab| Standard | XPG4 behavior | |/usr/xpg6/bin/crontab| Standard | XPG6 behavior | |/usr/bin/crontab | Standard | change default| | | | editor from ed| | | | to vi to paral-| | | | lel XPG| | | | behavior | |_____________________|________________|_________________| 4. Opinion 4.1. Background During the era of SunOS 4.1, our standards-compliance approach relied exclusively on the setting of $PATH. Based on customer feedback, we subsequently added the additional ability for users to "cherry-pick" specific commands from the variant directories. At present, we document a combined approach: setting $PATH as specified in standards(5) for a complete standards-compliant environment, and, in addition, variant forms of individual commands are documented by path- name in the individual command manpages. 4.2. Original Proposal The original proposal left the existing behavior of /usr/bin/crontab -e unchanged, using the first "ed" in $PATH as the default editor. 4.3. Amendment Several reviewers expressed concern with the divergence in behavor between /usr/bin and xpgN variants introduced by this case. The project team accepted a suggestion during the meeting to amend the spec to add, in addition to the delivery with Patch binding of the xpg4 and xpg6 variants of crontab, a change with Minor release binding to /usr/bin/crontab to use the first vi on $PATH as the default editor. 4.4. Standards Strategy Unclear To Many Additionally, several reviewers expressed concern with the PSARC/2005/683 Copyright 2005 Sun Microsystems - 3 - slowly growing proliferation of additional variant commands in the /usr/xpg4/bin and /usr/xpg6/bin directories, trig- gered solely by the behavior of a spawned subprocess. This concern was reinforced by the observed behavior that many of the variant commands in /usr/xpg4/bin support options which are simply rejected by the default variant; for instance, /usr/xpg4/bin/id supports a '-u' option; while /usr/bin/id rejects -u with an "illegal option" usage message. The heated discussion which followed made it clear that our strategy for standards compliance is not well understood among solaris developers as well as among solaris users & developers of applications for solaris. Selective cherry- picking is documented, but there is additionally a signifi- cant concern among a number of members of the Committee that the cherry-picking of complex commands which invoke other commands is of very limited applicability. Both ability to cherrypick and the limitations on this are not well expressed in our documentation, including the standards(5) man page. This led to the advice given in section 6.1. While conformance with the letter of the standards could be satisfied by changing /usr/bin/crontab's default editor to the first "vi" in $PATH, precedent as defined by [3] and [4] among other cases leads us to also deliver xpg4 and xpg6 variants to allow for cherry-picking. Several reviewers felt that this takes cherry-picking too far but there was no consensus among the committee to break with past precedent; a median view among the Committee is that any change to this strategy needs to involve significant review and considera- tion. 4.5. Gratuitous Divergence Obscures The Architecture The discussion also exposed significant divergence between our standards compliance architecture and its implementa- tion. In many cases, once a decision to "fork" a command was made, many new non-conflicting behaviors were integrated only into the xpgN variants. However, as a general rule, there should be no cases where a particular command feature, option, or the like accepted by a /usr/xpgN/bin variant should generate a syntax error when fed to the default sys- tem (/usr/bin) variant; similarly, non-conflicting features found in the default system variant should generally be present in any /usr/xpgN/bin variant unless they conflict with the requirements of the relevant standards.[1] _________________________ [1] Whenever possible we should also attempt to head off future unresolvable divergence by avoiding new changes to the default system variant of a command which are known to conflict with a relevant standard. PSARC/2005/683 Copyright 2005 Sun Microsystems - 4 - This divergence has created both confusion about the overall architecture, and also substantially increases the need to cherry-pick specific command variants in scripts. This degree of confusion is harmful to Solaris and to our users. When members of the community encounter such diver- gence, they should file bugs against the commands in ques- tion, and should flag those bugs with the "missing-std- feature" keyword. 4.6. Examples Of Divergence Specific examples of divergence, where options and syntax are supported by an xpgN variant of a command and are rejected as syntax errors by the /usr/bin variant: 4.6.1. id -u, -g, -G, -n, and -r 4521640 *id* /usr/bin/id missing options from /usr/xpg4/bin/id 4.6.2. grep -e, -f, -q, -x, -E, and -F 4843344 *grep* should have -q, -e, -f, -x, -E, and -F options 4.6.3. tail -n and possibly -c 6231496 *tail* non-XPG4 tail could support -n and pos- sibly -c 4.6.4. du -x 6269516 *du* should always accept -x 4.6.5. sh's lack of support for "export env=expr", $(cmd) syntax, and other non-conflicting POSIX syntax.[2] 6378708 *sh* could implement non-conflicting posix syn- tax _________________________ [2] One early reviewer of this opinion questioned the appropriateness of including this point, as the existing "/bin/sh" implementation is largely unrelated to the codebase used for the /usr/xpgN/bin shells. This is true, and will be a significant obstacle to fixing this problem, but it is a matter of resource allocation, not architecture. It is also the case that the limitations of the existing "sh" are a recurring source of irritation to newcomers to solaris and barrier to adoption by some. Finding the proper approach and balance here will be difficult. PSARC/2005/683 Copyright 2005 Sun Microsystems - 5 - This is by no means a comprehensive list. Fixing these bugs, and identifying other inconsistencies is outside the scope of the case, but resulted in the advice given in section 6.2. 4.7. Heroic Compatibility Measures Unnecessary A proposal was made to suggest that any potential issue of script compatibility could be addressed by having crontab invoke "ed" if isatty(0) returned zero and "vi" if it returned nonzero. However, as there was no consensus that a patch binding was otherwise compatible with a change to the default editor spawned by the crontab command, these heroic measures were deemed unnecessary in a change with Minor release binding. 4.8. Proliferation of additional setuid commands Subsequent to the meeting, concern was expressed regarding the delivery of additional setuid variant commands; this concern was underscored by prior incidents where XPG4 vari- ants which were setuid had security holes not found in the default system variant. The specific problem was caused by linking with values-xpg4.o and values-xpg6.o, exposing dif- ferent interfaces in the variant executables. The current implementation plan is to deliver three setuid executables, none of which will be linked with values- xpg?.o, mitigating this risk. Whether or not any specific crontab executable is setuid is best thought of as an imple- mentation detail, not any sort of Public interface. 5. Minority Opinion(s) No members voted to deny; however, several members of the Committee believe that the xpgN variants of the crontab com- mand push variant cherry-picking beyond its actual useful- ness. 6. Advisory Information 6.1. Improve standards(5) documentation relating to cherry-picking. The PAC should ensure that both our user-facing and our developer-facing process documentation relating to standards-related command variants is reviewed for clarity and approchability. PSARC/2005/683 Copyright 2005 Sun Microsystems - 6 - 6.2. Repair Existing Gratuitous Divergence Unnecessarily divergent behavior between standards-compliant and default variants of the commands we ship in Solaris is a recurring source of irritation, confusion, and dissatisfac- tion. While each individual instance of divergence is minor, the cumulative impact is akin to that of a thousand paper cuts. Many of these fixes are likely to be small "starter" bugs appropriate for new developers, though subtleties requiring review from experienced engineers will abound. Engineers should be encouraged to fix these and other "irritant" bugs as they are found. 7. Appendices 7.1. Appendix A: Technical Changes Required None. 7.2. Appendix B: Technical Changes Advised None. 7.3. Appendix C: Reference Material Unless stated otherwise, path names are relative to the case directory PSARC/2005/683. 1. Amended specification. File: final.spec 2. crontab(1) man page. File: final/crontab.1 3. PSARC 1994/161 XCU4 Conformance 4. PSARC 2000/492 Austin Group Common Revision of SUSv2 and POSIX Standards. PSARC/2005/683 Copyright 2005 Sun Microsystems