This is a proposal to get rid of some limits in some tools by compiling things in 64-bit mode on architectures that support it. The value in this case would be that (eventually) the Solaris userland could support platforms which are 64bit only. As one of the first projects to go down this path, it also sets 32/64 bit expectations going forward and applies them to a selected group of utilities. Motivation ---------- There are emerging platforms where supporting *any* 32-bit userland is problematic or impossible. There is a class of "limitations/bugs" in many utilities related to the use of 32-bit data structures. Best Practices concerning 64-bitness ------------------------------------ This case asserts that it is a long term goal for OpenSolaris to be free of those "limitations and bugs" and to able to support 64-bit-only systems. It also asserts that transitioning to a self-consistent set of 64-bit utilities (on platforms that support it) is more desirable than supporting a mixed set of 32- and 64-bit programs that may have different features or limitations due simply to their 32-/64-bitness. This case sets the expectation that having 64-bit binaries on 64 bit operating systems is a natural and normal thing to do, and that bug/limit parity with 32-bit capacity limitations on other platforms or on other similar utilities on the same platform is not a deciding architectural issue - though it may be a C-Team or community one. Proposal -------- This case proposes to leverage the 64-bit SPARC platform to enable 64-bit builds of the operating system. Starting with a small manageable set of utilities (bash and the GNU core utilities), we will modify their makefiles to create 64-bit code as part of the main OpenSolaris build process. The GNU Core Utilities are the basic file, shell and text manipulation utilities produced by the FSF: # Output of entire files: cat tac nl od # Formatting file contents: fmt pr fold # Output of parts of files: head tail split csplit # Summarizing files: wc sum cksum md5sum sha1sum sha2 # Operating on sorted files: sort shuf uniq comm ptx tsort # Operating on fields within a line: cut paste join # Operating on characters: tr expand unexpand # Directory listing: ls dir vdir dircolors # Basic operations: cp dd install mv rm shred # Special file types: ln mkdir rmdir mkfifo mknod # Changing file attributes: chgrp chmod chown touch # Disk usage: df du stat sync # Printing text: echo printf yes # Conditions: false true test expr # Redirection: tee # File name manipulation: dirname basename pathchk # Working context: pwd stty printenv tty # User information: id logname whoami groups users who # System context: date uname hostname hostid # Modified command invocation: chroot env nice nohup su # Process control: kill # Delaying: sleep # Numeric operations: factor seq Longer term (not this project...) support for bootstrapping new 64-bit processors will require 64-bit utilities for single user (and probably a significant set beyond that). This case starts us down this path with some simple to convert utilities, but does not presume to take on the whole conversion effort. There should be no architectural impact because of this change, other than the removal of 32-bit limits: - No problem with dates >= 2030 (64bit |time_t|) - High-resolution timestamps by default - Stack would be non-executable by default - ARG_MAX would be larger (twice the size), therefore allowing applications to deal with much more data (note this extra memory is mapped via |mmap()|-like calls and only reserves the size, not actually allocate it (reserved address space != real memory usage)). - "bash" supports arrays and it would be nice that we don't limit the array content to 2GB The long-term goal is to make both OS/Net and SFWNV "64bit clean", e.g. that we can support hardware which only supports a 64bit mode and no 32bit support (for at least one Solaris port we know that the porters were _forced_ to invent artificial 32bit support to work aound the problem that OS/Net is not 64bit clean (which is IMO an unimaginable abdomination)). I only wanted to start the journey through the code and picked "GNU coreutils" and "bash" as easy targets to begin this long-term work. The "switch to 64bit" is primarily intended as "safeguard" to make sure that: 1) the code's 64bit cleanless cannot be broken by future changes in the code itself or any API changes 2) the code is used on real-life systems (which will catch all possible regressions which may slip through initial testing) FAQ --- Why these utilities and not a smaller or larger set? "GNU coreutils" and "bash" are two existing packages which cannot be split into smaller pieces without pain. Why 64-bit only and not isaexec()'d 32 and 64? On 64-bit systems (like SPARC and a couple of others still being ported), isaexec will always exec the 64-bit version, but it will do so (with a double-exec performance penalty) AFTER it has truncated the argument list to a 32-bit ARG_MAX, among other things. The extra |exec()| becomes an issue in two scenarios: 1. Tiny application with very small runtime (e.g. GNU "echo") 2. Machine with many CPUs - |exec()| must tear down the address space and make crosscalls to all other CPUs... which will be a problem for the 256CPU Niagara2+ machine... and even more a problem when I read TheRegister.co.uk's article (http://www.theregister.co.uk/2007/07/26/sun_2048_thread_niagara/) about a possible 2048-core machine. 2048-1 crosscalls per |exec()| BartS: Cross calls are limited to the set of CPUs the process has run on so isaexec isn't likely to cause a cross call storm unless there are some pathological context switching problems :-) . RolandM: From what I know there were several high-priority escalations on SF15K/25K machines last year where Bourne/bash scripts were causing system-wide performance problems just because they were |fork()|+|exec()|'ing madly ... and from my own experience a tight |exec()|-loop can nail large machines (e.g. M8000) against the next available wall. It sounds that the "pathological case" happens somewhat often (the problem gets even worse when the application uses largepages (but don't ask me _why_ this happens...)). What about systems that must still support 32-bit processors? This project does not intend to change the default build (32- or 64-bit) for the x86/x64 world because those systems still are required to provide boot and runtime support for both 32- and 64-bit capable CPUs in the same OS image. As programs are switched from 32bit applications to 64bit on SPARC, (as this project proposes), the effort can be leveraged to allow other platforms (like x64) to switch to 64bit on demand if the need arises (only major difference which needs to be checked is endianess, differences in the floating-point subsystems and minor other nits) Is 64-bit-only *always* the right answer? No. The question came up of why we should continue to deliver a 32 bit version of a command on 64-bit CPUs: One case was ksh93 where we _explicitly_ ship 32bit and 64bit vesions on both SPARC and x86 via "isaexec" because ksh93 supports a plugin API (and these plugin may need other libraries which may only be available as 32bit versions). What, isn't all of ON already 64-bit clean? OS/Net and SFWNV aren't 64bit clean which prevent Solaris from being ported to 64bit-only hardware. The problem is _known_ since the first 64bit support was added for Solaris 7 and I _finally_ want to start a small project to crawl over the sources and kill the problem. And IMO the _obvious_ solution to prevent this issue in the future is to make the matching binaries 64bit by default if the kernel is 64bit-only. The file size limit can be lifted on 32-bit with large file support (lfcompile(5)). The register usage can be altered in an often helpful way by compiling with -xarch=v8plus. -xarch=v8plus cannot safely be mixed with other sparcv7 code in all cases.