sun microsystems Systems Architecture Committee _________________________________________________________________ Subject: Power PC Kernel Port Submitted by: Prasad Singamsetty File: psarc/1994/188/opinion.ms Date: July 20th, 1994 Committee: Mark K., Steve C., Bruce D., Robert H., Yousef K., Joseph Kowal- ski, Terrence M., Glenn Skinner. 1. Summary The Power PC (PPC) is a new instruction set architecture that has the potential to challenge the Intel x86 architecture's market share. The basic Instruction Set Architecture is described by the Power PC Architecture [2,3], and there are currently two chips in the PPC family. The 601 is designed for more powerful (potentially MP) sys- tems. The 603 is designed for notebook computers. There is a PReP specification [4] that provides a high level functional specification for Power PC platforms, but it is too general to be useful for an implementation. The only major PPC platform available at this time is the IBM Refer- ence Implementation. The primary goal of the Power PC kernel port is to extend the Solaris kernel to be able to support the 601- and 603- based IBM Reference Implementation platforms for the Power PC. A secondary goal is the creation of a framework for supporting other Power PC platforms. It should be noted that even the architecture for the IBM reference implementations is still in a state of flux. Many important details (like MP architecture and BIOS interface specifications) are still not fully defined. Because PPC platform specifications are still in such a fluid state, it should be assumed that additional PPC platform support pro- jects will have to be created to support the real product platforms when they become available. 2. Decision & Precedence Information The project is accepted, subject to the changes enumerated in Appendix A. psarc/1994/188 Copyright 1994 Sun Microsystems, Inc. - 2 - The classification of the change introduced by this project is micro. It should also be noted that this project represents only part of the work required in order to fully support Solaris on the PPC. The other basic PPC projects are: 1993/685 Power PC ABI 1994/196 Power PC Header files 1994/197 Power PC Commands There are several aspects of the PPC Kernel that are depen- dent on the PPC ABI specification: The virtual address space layout seen by user processes. The page size seen by user processes. The mapping of hardware exceptions into signals types. The initial state of a newly forked process (registers, stack frame, auxiliary vector). Because of these dependencies, this project is dependent on the approval of 1993/685 (PPC ABI). 3. Opinion ______________________________________________________________________ | Interfaces Exported | |________________________|__________________________|________________| |Interface | Classification | Comments | |________________________|__________________________|________________| |PROMIF functions | Project Private/Obsolete| | |console interfaces | Project Private | until 1993/408| |E_kvseg | Project Private | | |RAMdisk extensions | Project Private/Obsolete| | |System call numbers[5] | Consolidation Private | | |System call linkage[5] | Consolidation Private | | |Signal/Trap linkage[5,6]| Consolidation Private | | |________________________|__________________________|________________| Although numerous changes are being made to the Solaris Ker- nel in order to accomplish the Power PC port, the majority of these changes are obvious ones with no architectural con- tent (e.g. device drivers and ISA specific implementations of platform dependent functions). These received little discussion. The majority of the discussion focused on a half-dozen issues. psarc/1994/188 Copyright 1994 Sun Microsystems, Inc. - 3 - 3.1. MP Support There is not currently a standard or reference implementa- tion for PPC based MP machines. The team proposes not to include any MP work in this project. They will, however, not do anything to preclude MP support and will ensure that all new code is written to be appropriately MP-safe/hot. The committee agreed with this proposal, but expects to see a new project created to add support for PPC MP platforms at a later date. 3.2. PROMIF Support The PReP spec calls for Open Firmware PROM/BIOS, but does not define any interfaces to it. The IBM reference imple- mentation does not yet provide or define these services. Since these services are not yet available, the initial ker- nel port (this project) will not attempt to provide an interface to them. As an interim solution, the team will use the existing (x86-based) NOP non-implementations to stub out these func- tions. It is expected, however, that when the PPC PROM functionality is defined, a new project will be created to use those services to more fully implement the corresponding functions. In particular, it is not expected that the stub implementations will be shipped with the product's FCS. It should be noted, however, that there may be PPC platforms that do not provide the Open Firmware support. If such machines exist, and Solaris is to support them, it may be necessary to retain the stub versions of the PROMIF func- tions and provide other implementations for needed boot-time functionality. The committee felt that code maintainability, product sup- portability, and our ability to run on multiple PPC plat- forms would be compromised if the FCS product did not include support for Open Boot Prom functionality. The approval of this project for shipment is dependent on the completion of another (not yet submitted) project to provide Open Boot Firmware support (and if necessary a solution for legacy non-OBP systems). 3.3. Dependency on 1993/408 The PPC reference implementation uses keyboard and display controllers that are very similar to those on the x86, and so it makes sense to base the PPC drivers on the x86 ver- sions. The x86 keyboard and display architecture, however, is currently in a major state of flux (1993/408). The team was forced to choose between basing the PPC port on psarc/1994/188 Copyright 1994 Sun Microsystems, Inc. - 4 - an implementation that was becoming obsolete or an implemen- tation that was not yet available. They chose the latter, and have been working very closely with the console redesign team. The committee agreed that this was a reasonable approach, and noted that the PPC kernel project does not actually depend on the completion and integration of 1993/408. Until 1993/408 integrates, the PPC kernel can use its own copies of the necessary code (in platform specific directories). The major concern was the fire drill that would result in the source tree when this project checked in new console code (in platform dependent directories) and then the con- sole redesign team checked a slightly different version into common directories. The committee recommended that an effort be made to accelerate the check-in of the portions of the new console implementation that the PPC port needed in order to avoid this confusion. If such an accelerated check-in is not possible, the team is authorized to check in their versions of this code (as plat- form specific code), but they must assume the responsibility for all necessary changes when the common code from 1993/408 is put back. Allowing the PPC project to check in a potentially redundant copy of common code into a platform specific directory is a violation of 1991/061. It will only be permitted for a very brief period of time (though possibly through FCS if 1993/408 does not integrate before then), and the team will be required to comply with 1991/061 (by eliminating the redundant PPC version of the code) after 1993/408 integrates. 3.4. RAM Disk Driver One of the PROM services that is not yet defined or imple- mented in the IBM reference platform is general disk I/O. Without this service, it is very difficult to load a kernel into memory. To get around this problem, the team has defined an extremely large boot block, one that contains an entire ufs file system with all of the files that need to be loaded in order to assemble a working kernel in memory. The PROM boot code reads the entire image into memory, and the Solaris bootstrap then treats that image as a ram-disk, from which the necessary modules are loaded. This requires a minor change to the ram-disk driver. In other systems, the ram-disk driver allocates (a pre-defined size) and initializes the memory for the ram-disk when it is opened. On the Power PC, the memory for the ram-disk has psarc/1994/188 Copyright 1994 Sun Microsystems, Inc. - 5 - already been allocated and initialized, and the size is a parameter passed in a global structure. The team indicated that this was only a short term porting aid and would not be part of any real product. The commit- tee had no objection to this. 3.5. 32MB Short Call Limitation The PPC ISA has an efficient relative short call that is limited to a range of plus/minus 32Mb. Calls within the main kernel, and within dynamically loaded modules easily fit within this range, but calls between the main kernel and dynamically loaded modules may not. This is not a problem on Sparc or X86 because the standard call instructions on those ISAs have a much wider range. In the short term, the team has limited Sysmap to 16M and set SYSBASE (the base for dynamically loaded modules) to KERNELBASE+16M ... thus restricting the core kernel and all of its data structures to 16Mb. Ultimately, however, this limitation may prove to be unreasonably restrictive and a better solution must be found. Two alternatives were pro- posed: 1. use Procedure Linkage Tables (PLTs) for linking kernel modules. 2. use a separate segment within the kernel address space for dynamically loadable modules and keep them within 32Mb of the core kernel. The first option requires complex changes to kobj and imposes a performance hit on the function calls that go through the PLTs (e.g. every call to mutex_enter). The second option requires the creation of a new segment and a new set of allocation routines to manage the space in that segment. The second option is also more restrictive (in that it requires the core kernel and the text+data of all dynamically loaded modules to fit within 32Mb). It was the consensus of the committee that we probably want to do option 2 promptly and move to option 1 later (if and when the 32Mb limitation becomes pressing). The committee expressed concern that a more general solution in the future should not necessitate recompilation or re-linking of exist- ing dynamically loadable object modules. The team offered an "existence proof" design to show that binary compatibil- ity could be maintained, and the committee was satisfied. Option 2 requires changes to common code, and those changes must be reviewed by the ON owners, but this is not an archi- tectural issue. A more detailed description of how this psarc/1994/188 Copyright 1994 Sun Microsystems, Inc. - 6 - will be implemented can be found in the supplementary materials directory for this case (PSARC/1994/188/materials/kobj.proposal[1]). 3.6. Platform Specific Module Interface Although all of the PPC platforms have not yet been defined, the team anticipates considerable diversity among them and wants to provide a general framework for platform specific modules. Due to similarities with x86 interrupt controllers and MMUs, the team proposed to use the x86 MP Platform Specific Module interface and implementation (1993/622) as a starting point for the PPC implementation. The motivation behind 1993/622 was MP support, and MP sup- port is not an issue in this project, but the team argued that most of the functions in that framework are not MP specific and that many of the implementations are already very close to what is needed on the PPC. The committee expressed concern that there was a significant overlap between these functions and KBI phase 2, and that it seemed inappropriate to sanction a different interface to solve the same problems a little bit sooner. It was also observed that the need for a general framework for PSMs was only hypothetical, since Solaris is not expected to run on other PPC platforms in the next twelve months. The committee suggested that the PPC kernel team work with the KBI team to ensure that the KBI would adequately answer x86, x86mp, and PPC needs for Platform Specific Modules. If extensions are required, these should be made as part of KBI-2 or some new project. 4. Minority Opinion(s) None. 5. Advisory Information 5.1. Dependency on 1993/408 The decision to use the new console architecture (1993/408) in this project is a reasonable one, but the committee is concerned that this project proposes to deliver its code before 1993/408. The PPC team should negotiate with the new console team to see if the put-back of the code that the PPC project requires can be accelerated. If this is not possible the PPC team is authorized to put their version of the console support back into a platform dependent directory. Later, when the console team does their put-back, the PPC team must psarc/1994/188 Copyright 1994 Sun Microsystems, Inc. - 7 - handle the resync and reconciliation work that is required in order to use the new code from the console team, and eliminate their private version. 5.2. Stubbed PROMIF Implementation Because there are not yet any PPC machines with Open Boot Firmware, the interim decision to use the existing x86 code to stub out the PROMIF functions is an appropriate one (and necessary in order to proceed with the other aspects of the PPC port). This is not, however, acceptable for final ship- ment. When proper PROMIF specifications and implementations become available, a new project must be created to replace the stub implementation with a more complete exploitation of the Open Boot Firmware. If this proves infeasible, the team must return to PSARC for approval of a different PROMIF support strategy. 5.3. Work-around for 32Mb Call Limitation The common module changes required to implement the new seg- ment and ensure that dynamically loadable modules and the kernel proper are all kept within a 32Mb range should be reviewed by the appropriate ON code owners prior to put- back. 6. Appendices 6.1. Appendix A: Technical Changes Required The proposed platform specific module interface overlaps with interfaces that are to be addressed by the KBI and it seems inappropriate to create a conflicting standard for these functions. The platform specific module support framework must be removed into a separate project, in which x86, PPC and KBI concerns will all be reconciled. 6.2. Appendix B: Technical Changes Advised None. 6.3. Appendix C: Reference Material 1. PSARC/1994/188/materials/kobj.proposal 2. PowerPC 601 User's Manual, Motorola 3. PowerPC Processor Architecture (Books I-III), IBM psarc/1994/188 Copyright 1994 Sun Microsystems, Inc. - 8 - 4. PowerPC Reference Platform (PReP) Specification 5 PSARC/1994/188/materials/abi.specs 6 PSARC/1994/188/materials/machsig.h psarc/1994/188 Copyright 1994 Sun Microsystems, Inc.