Several questions have come up recently about how to package extensions to our system. This note is an attempt to write down some rules that all ARCs will enforce, and to describe some situations in which the ARCs may apply their judgement and grant exceptions to the rules. Executive Summary The rule we agreed to is: Each component of the system can appear in only one package. The following sections describe the problems that lead to this rule, what exactly this rule means, and some situations in which exceptions to this rule may be granted. The Problem The simplest approach to extending the system has been to take a copy of the part of the system you want to extend, modify it accordingly, and supply this modified version as your extension, while still supplying the standard version. The biggest problem we've run into with this approach is patch management. If a patch is made to the standard version of the component in order to fix a bug, it must also be made to the extended version. If the patch requires coordinated changes to two different components of the system, and only one of the components is included in the extension, it becomes next to impossible to coordinate the patch. The problem becomes even worse when the owner of the standard component is unaware that the component has been duplicated and extended in another product, making the need to coordinate such patches an unknown. Examples of the Problem Example - UFS+ RMTC created a product known internally as "UFS+". UFS+ consisted of a number of kernel modules, including the UFS kernel modules. All of these modules were derived from the SunOS 4.1 versions. Some of these modules simply included bug fixes for bugs that effected UFS+; others contained the enhancements that *were* UFS+. Occasionally bugs were discovered which required fixes in these common modules. Since the UFS+ versions of these modules could not be delivered with the base system, any patches produced applied to the base system only. This left customers in the unpleasant situation of choosing between the patch to the base system and UFS+. Both the patch and the UFS+ product could not be installed at the same time. A more severe version of this problem occurred with 4.1.1. 4.1.1 included important performance improvements (clustering) which required changes to these common modules. People who subsequently installed UFS+ on 4.1.1 lost the advantages of these performance improvements. Example - ARM login The ARM product (see below) delivered a new version of the login program. When a bug was fixed in the base system login program, the ARM version was not updated. Again this left customers in the position of choosing between ARM and the login bug fix. This is a particularly difficult choice when the bug fix fixes a security bug. The Rule Again, the rule is: Each component of the system may be provided in only one package. The simplest way to think of a component is as a file (I only want one package providing /etc/dump), but there are other important namespaces as well (I only want one package to implement the "ufs" filesystem type). In many areas of the system the mechanisms exist for maintaining differently named components with similar functionality (e.g., /opt/hsm/bin/dump, "ufs+" filesystem type). A component is not really (only) a binary piece of the system. We don't want two products taking the same source code, with their own slightly different hacks, and shipping it. We also don't want two products taking completely different source code and shipping it under the same name. A "package" in this context is a System V package, as defined by the SVID and further specified by the ABI and the SCD. A package is composed of deliverable components of the system (executable files, shell scripts, data files, etc.). A package is the output of the build process and the input to the release engineering process. A product is assembled out of packages and may contain more than one package. A package may be contained in more than one product (although this is discouraged). The package utilities perform some version management and dependency checking, making it feasible to include a single package in multiple products. In the future the package utilities will be enhanced to support hierarchical packages so that packages may contain other packages. At that time, it is only the "atomic", low level packages that fall under the constraints described here. Examples of How to Apply the Rule Example - ARM The Account Resource Manager is an unbundled product that runs on SunOS 4.x. It replaces several standard programs such as login and passwd with new versions that provide enhanced security and account management. Due to the problems with replacing standard system components, we would like the SunOS 5.0 version of ARM to be implemented as a simple add-on to the system. One approach to doing this is to separate the parts of the standard system that ARM would want to extend into a separate library. The standard components of the system would call into this library to perform their security related functions. The standard system would contain a version of this library which provided the standard security functions and policies. Many entry points in the standard library might be stubs, providing no functionality in the standard version. ARM would then provide an alternate version of this library that implemented their functions and policies. This approaches avoids any patch problems with the standard commands, since they would reside only in the base system. It also reduces the likelihood of patch conflicts by breaking the system down into smaller components. However, there is still a standard component that must be replaced by ARM. It would seem to be an advantage if the installation of the ARM unbundled product (as for any unbundled product) were "undoable". One way to accomplish this would be to have ARM save the standard version of the library and install the ARM version in its place. Another approach is to have the library actually be a symlink that points to the current version of the library; installing ARM would change the symlink to point to the ARM version instead of the standard version. Since the system needs to be re-architected to introduce this library, the latter approach is preferred in this case. In cases like this, it is generally expected that the group that wants to introduce such new functionality will bear the responsibility for restructuring the base system to allow the new functionality to be added appropriately. Of course, such restructuring is subject to ARC review. It's likely that such restructuring will only be done by SunSoft, or by other groups under contract to SunSoft. Example - dump RMTC has developed a new version of the filesystem dump command that has many extensions. Unlike the ARM example, rather than replacing a standard system component this new version is to be provided in addition to the standard version. In this case, the new version must be installed in a package-specific directory, not overwriting the standard version of dump. Since this new version is derived from the source code of the standard version, the patch problem remains. The customer can be insulated from this fact by also naming the extended program to be a different name than that of the standard program -- thereby reducing the likelihood that the customer will perceive our implementation practice of sharing the source from the product definition of being a separate product. For the special case where the extended dump is intended to replace the standard dump program, the latter requirement can be suspended (in other words, the program could be called "dump", although it would still have to be placed in a package directory until such time as it actually replaced the standard dump.) Another issue is the /etc/dumpdates database used by dump. If the new version uses this file in a compatible way with the standard version (as we thought it should), then it should use *exactly* this file, rather than keeping its own version of this file in a separate place. If this file needed to be extended, and the extension couldn't be done compatibly with the standard version, then putting the extensions in a "parallel" file might be best. This would allow both the old and new versions of dump to be used compatibly, assuming that is actually a goal. Of course, the exact format of the /etc/dumpdates file needs to be specified as (at least) a Sun private interface to allow it to be shared by two products in this manner. Example - filemerge So far, all the examples have been cases of unbundled products modifying pieces of the base system. The filemerge example is a case of two unbundled products competing for the same program. Currently, the filemerge program is slated to ship both with SPARCworks and Avocet. If both products want to make modifications to filemerge, then the patch problem arises. Filemerge is essential to the Avocet product as the goal of Avocet is to track concurrent changes and place the user in filemerge to produce a merged result. Filemerge is desirable for SPARCworks as it can be used as a GUI for diff. The way to ship filemerge with both products, while still following the rule, is to package filemerge as a separate package that is included by both the Avocet and SPARCworks products. The filemerge package must be owned by only one group, with all revisions to that package coming from that group. Example - syncinit The syncinit example is a case of several unbundled products competing for the same program. The syncinit program is a utility used by the SunLink and SunNet synchronous datacomm products (SNA3270, BSC3270, BSCRJE, SNAP2P, INR, X.25) to set their link parameters (speed, clocking, etc). This case is similar to the filemerge example and should be handled in the same manner. Examples of Exceptions to the Rule Example - ARM The general rule would be that the base system must be restructured to accommodate changes such as ARM requires, and that the restructured version of the base system must be delivered to customers before the unbundled product that depends on it can be delivered to customers. However, it can often take a long time to get such a change to the base system delivered to customers, and there are several options for what people should be allowed to do in the mean time. One possibility is for them to be allowed to deliver a version of the product "the old way", introducing the patch problem. If they did this, they would be required to upgrade the base system and the unbundled product to use the restructured approach "as soon as possible", so that the patch problem doesn't exist for a long time. Hopefully the problem can be managed if it's just for a short time. Another possibility would be to complete the restructuring and deliver the restructured versions of the base system components in the package with the first release of the unbundled product. The ARM example also brings up an interesting sub-problem. Let's assume that the ARM group restructured the system in the way suggested, creating a separate "security policy" library. There would be a version of this library in the standard system, implementing the standard policy. The ARM product would have its own version of this library, implementing the ARM policy. While some parts of the library might have significant ARM extensions, other parts might be little changed from the standard versions. In the latter case, what should be done to allow the standard version and the ARM version to share changes to the common code? This is basically the patch problem all over again, although chased down into a small corner of the system. In general, after reducing the size of the problem as much as practical, it is up to the managers and engineers involved to manage the remaining parts of the problem. It is a matter of the ARC's judgement as to when you get to this point. Example - dump As mentioned above, the new dump program introduces the patch problem since it is derived from the source code for the standard dump program. The rule would not normally allow such a situation to occur. However, in most cases, the ARCs would require that the same group own both such programs. It is hoped that this will make the patch problem manageable. Summary The rule is: Each component of the system can appear in only one package. The ARCs recognize that this rule can not be expected to cover all situations, and that exceptions will have to be made occasionally. Several common cases that would justify exceptions have been described above. Exceptions are only granted via the ARC review process.