id: @(#)opl-device-tree.txt 1.9 08/02/15 purpose: Device Tree for OPL systems copyright: Copyright 2008 Sun Microsystems, Inc. All Rights Reserved copyright: Use is subject to license terms. Title: OPL Specific Device tree nodes and properties O Purpose This document specifies the application of Open Firmware to the Olympus Product Line (OPL) class of servers that are being jointly developed by Sun & Fujitsu. It defines the device tree nodes specific to OPL systems. This document supplements the information supplied in Jupiter Bus Device binding [4]. Task Group Members: The following individuals are members of the task group that produced this document. Sunit Jain (CSW OBP) - Sun Microsystems, Inc. 1. Overview and References Olympus Product Line (OPL) is a line of servers that are jointly developed by Sun & Fujitsu. Much of the (HW & SW) design is being leveraged by Fujitsu's previous generation Prime Power servers. Refer to [1] for the details on OPL servers. This document supplements the information supplied in [2], [3] & [4]. 1.1 References [1] Columbus2 Logical Specification Rev. 1.1 URL: Due to Sun/Fujitsu confidentiality agreement this document cannot be posted in public domain. Please contact Susan Hamwi (x40045, susan.hamwi@sun.com) to get access to the document. [2] IEEE Std 1275-1994 Standard for Boot (Initialization, Configuration) Firmware, Core Practices and Requirements [3] Core Errata, IEEE P1275.7/D4 [4] Jupiter bus device binding FWARC/2005/076 http://sac.eng/Archives/CaseLog/arc/FWARC/2005/076/commitment.materials/ [5] Columbus2 OBP Functional Specification Ver 3.1 URL: Due to Sun/Fujitsu confidentiality agreement this document cannot be posted in public domain. Please contact Susan Hamwi (x40045, susan.hamwi@sun.com) to get access to the document. [6] Interrupt Mapping, version 9, Open Firmware Recommended Practice http://playground.sun.com/1275/practice/#imap [ public ] http://noho.eng.sun.com/1275/practice/#imap [ within SWAN ] [7] EBus Bindings FWARC/2006/056 [8] RIO EBus Bindings FWARC/2006/057 1.2 Definition of terms Bus node A device node that represents a Jupiter bus interconnect. Child node A device node that represents a Jupiter bus device. Example: Device nodes for Olympus-C processor, Oberon Host IO bridge. CMU-CH CPU Memory Unit Channel. PCI bridge based on Sun Psycho ASIC to support legacy devices like tty, Flashprom, etc. DR Dynamic Reconfiguration - Run time addition or deletion to the system hardware configuration. Floating boards System boards that can be moved around between different domains as and when needed. It is recommended that kernel & OBP are not placed on floating boards. FMEMA Floating Memory Address. HW feature for the memory copy rename operation. Sometime also referred as FMA which is not to be confused with Sun's FMA - Fault Management Architecture. Hardware Descriptor Domain specific data passed from SP to the domain. Table FWARC/2005/261 IKP Solaris' In-kernel Probing module Jupiter Bus Client A device that connects to a port of the Jupiter Bus interface. Example: Olympus-C processor, Oberon Host IO bridge. Jupiter Bus Device A 11-bit number that uniquely identifies a Jupiter Bus ID Client device. LSB Logical System board - The unit of grouping for memory, CPU and IO which is visible to software. Can be a PSB or an XSB. LSB_ID ID assigned to an LSB. 5 bit value. MAC Memory Access Controller. MAC connects the SC with the memory modules. Performs memory access control, memory initialization & diagnostics. (This document refers to MAC as Memory Access Controller & should not to be confused with Media Access Control) MTP Multi Threaded Processor. A processor module containing more than one strand. Oberon ASIC Jupiter bus to PCI Express host IO bridge. One Oberon has 2 channels, each channel has two x8 PCIE ports. Olympus-C SPARC64 VI CMP processor supporting 2 physical cores & 2 virtual cores (strands) within the physical cores. PSB Physical System Board. Each system board contains CPUs memory & IO. SC System Controller (not to be confused with Service Processor as in Starcat/Serengeti). It is responsible for routing the address/data between CPUs & memory controllers and the system crossbar. SCF System Control Facility. Service Processor. SCFI System Control Facility Interface. Strand A term that identifies the HW state used to hold a SW thread in order to execute it. Strand is specifically the SW visible architecture state (PC, nPC, GPRs, FPRs, %cc, status registers, ASRs, etc.) of a thread and any micro architecture state required bu HW for its execution. Thread A term that refers to a SW entity that can be run on HW. A thread is scheduled, may or may not be actively running on HW at any given time, and may migrate around the HW of a system. XSB eXtended System Board. One-quarter of PSB with CPU, memory and IO. XSCF eXtended System Control Facility. Generic name given to SCF for OPL systems. 2 OPL Device Tree OPL OBP defines the following device nodes that are specific to OPL systems. The design has been adopted from OBP implementation on Fujitsu's Prime Power machines in order to leverage Solaris drivers and applications used on Prime Power machines. 2.1 System Root Node ('/') Root node for OPL machines represents the Jupiter bus, which is the system bus for OPL machines. 2.1.1 Root Node Properties "name" Type: Prop-encoded string Contents: Standard device type property Value: Obtained from SP via Hardware Descriptor table NOTE: "name" property value will be defined later by a separate fast track case. "device_type" Type: Prop-encoded string Contents: Standard device type property Value: "jupiter" "#size-cells" Type: Prop-encoded-array, 1 prop-encoded integer Contents: Number of cells needed to encode the size Value: 2 "banner-name" Type: Prop-encoded-string Contents: Banner name for OPL platform Value: Obtained from SP via Hardware Descriptor table NOTE: "banner-name" property value will be defined later by a separate fast track case. "model" Type: Prop-encoded array, 1 prop encoded strings Contents: Standard device type property Value: Obtained from SP via Hardware Descriptor table "FF1" for Form Factor 1 machines "FF2" for Form Factor 2 machines "DC1" for Data Center 1 machines "DC2" for Data Center 2 machines "DC3" for Data Center 3 machines "IKKAKU" for Ikkaku platform "clock-frequency" Type: Prop-encoded-array, 1 prop-encoded integer Contents: System bus clock frequency, in hertz. Value: 32-bit integer obtained from SP via Hardware Descriptor table "stick-frequency" Type: Prop-encoded-array, 1 prop-encoded integer Contents: System bus clock frequency, in hertz. Value: 32-bit integer obtained from SP via Hardware Descriptor table "breakpoint-trap" Type: Prop-encoded-array, 1 prop-encoded integer Contents: Trap number for breakpoint Value: 0x7f "reset-reason" Type: Prop-encoded array, 1 prop encoded strings Contents: Reason for power-on Value: One of the following value based on the information obtained from the SP via Hardware Descriptor table. "POR" : Partition Power-on "S-POR" : POR requested by POST or OBP "B-POR" : POR requested by SCF "FATAL" : POR by System Fatal "INTR-POR" : POR for Interleave mode change "DR-POR" : POR for temporary partition for DR "DR-RTI0" : POR reset for DR attach (new) "DR-RTI1" : POR reset for DR attach (change) "idprom" Type: Prop-encoded-array, 32 prop-encoded bytes Contents: Machine specific identification data (machine-type code, serial number, ethernet address) Value: Obtained from SP via Hardware Descriptor table "dr-status" Type: Prop-encoded string Contents: If this property exists, then DR operations in the domain are prohibited. The value of this property indicates the reason why DR is prohibited. If the property is missing, the DR operations in the domain are allowed. Value: String indicating reason why DR operations in the domain are prohibited. Obtained from SP via Hardware Descriptor table "floating-boards" Type: Prop-encoded-array, list of property-encoded integers Contents: List of LSB_IDs of boards marked as floating boards. Value: An array of 32-bit integers representing LSB_IDs of floating boards. Value obtained from SP via Hardware Descriptor table "cpu-signature-ranges" Type: Prop-encoded-array, list of property-encoded integers Contents: Physical base address and size of the CPU signature space in SRAM Interface Buffer for all the LSBs. Format: { phys.hi phys.lo size.hi size.lo } Note: OPL uses Starcat CPU Signature implementation. Refer to PSARC/2000/205 for details on CPU signatures. "chassis-sn" Type: Prop-encoded string Contents: Chassis serial number as stored in the panel. Value: A string representing chassis serial number. Value obtained from SP via Hardware Descriptor table "SPARC64-VII-mode" Type: Prop-encoded-array, no value (Boolean) Value: None Description: If the property exists, it indicates that SPARC64-VII mode has been selected for the domain. Absence of this property indicates that domain is executing in SPARC64-VI mode. 2.1.2 Root Node Methods open ( -- okay? ) Prepare this device for subsequent use. close ( -- ) Close this previously opened device. decode-unit ( addr len -- phys ) Convert text unit-string to physical address. encode-unit ( phys -- unit-str unit-len ) Covert physical address to text unit-string. map-in ( phys.lo phys.hi size -- virt ) Map the specified region; return a virtual address. map-out ( virt size -- ) Destroy mapping from previous map-in. get-hwd-va ( portid -- va ) Return the virtual address of the hardware descriptor structure for the device as specified by the 'portid'. get-interrupt-target ( -- cpuid ) Return the cpuid of the executing cpu. master-interrupt ( reset-int-xt portid -- error? ) Register a Jupiter bus interrupt master's "reset interrupt handler". 2.2 Legacy Devices OPL platforms implement CMU-CH PCI bridge [4] to support legacy devices. Ebus is implemented via traditional Cheerio device and supports the legacy devices like Flash PROM & Serial port (tty). Cheerio device connects to the secondary side of the CMU-CH PCI bridge. In addition, OPL systems have the SCF interface and Panel interrupt devices implemented as Ebus devices. The following sections describe the device node structure of these legacy devices. 2.2.1 Ebus Device Node Ebus is supported via Sun Cheerio ASIC under the CMH-CH PCI bridge. The device path to Ebus node would be: /pci@,/ebus@ 2.2.1.1 Ebus Device Node Properties Following properties are defined in RIO EBus Bindings (FWARC/2006/057) are listed here for completeness : "name" - "ebus" "device_type" - "ebus" "compatible" - "pci108e,1000" + "pciclass,068000" "#address-cells" - 2 "#size-cells" - 1 "reg" "ranges" "assigned-addresses" "interrupt-map" "interrupt-map-mask" "#interrupt-cells" - 1 "vendor-id" - 0x108e "device-id" - 0x1000 "class-code" - 0x068000 "revision-id" "min-grant" "max-latency" "devsel-speed" "fast-back-to-back" 2.2.1.2 Ebus Device Node Methods Following methods are defined in RIO EBus bindings (FWARC/2006/057) and are listed here for completeness : open - Prepare this device for subsequent use. close - Close this previously opened device. map-in - Map the specified region; return a virtual address. map-out - Destroy mapping from previous map-in. decode-unit - Convert text unit-string to physical address. encode-unit - Covert physical address to text unit-string. dma-map-in - Convert virtual address to device bus DMa address. dma-map-out - Free DMA mapping set up with dma-map-in. dma-alloc - Allocate a memory region for later use. dma-free - Free memory allocated with dma-alloc. dma-sync - Synchronize (flush) DMA memory caches. 2.2.2 Flash PROM Device Node Every LSB implements a CMU-CH, thus every LSB has its own Flash PROM device. Flash PROM device carries the POST and OBP firmware. Each Flash PROM device in a domain is guaranteed to have identical POST/OBP image. We only represent one flash PROM node in the OBP device tree which belongs to the CMU-CH of the master CPU. The device path to flash PROM node would be : /pci@,/ebus@/flashprom@, 2.2.2.1 Flash PROM Device Node Properties "name" Type: Prop-encoded string Contents: Standard device type property Value: "flashprom" "compatible" Type: Prop-encoded array, composite string properties Contents: Standard device type property Value: "FJSV,flashprom" "reg" Type: Prop-encoded-array, list of property-encoded integers Contents: The physical address and size "reg-format" in Ebus address space. Format: { chip# offset size } "status" Type: Prop-encoded-array, Property encoded string Contents: Standard property to indicate the operational status of the flashprom device Value: Absence of property or property value "okay" indicates that flashprom is operational. Other values represent non- operational flashprom. 2.2.2.2 Flash PROM Device Node Methods None needed. 2.2.3 Serial Port (tty) Device Node Just like Flash PROM, every LSB implements its own serial port. So, a domain with N LSBs would have N serial ports. Each serial port would be represented with a separate device node. The device path to Serial port node would be : /pci@,/ebus@/serial@, 2.2.3.1 Serial Port Device Node Properties "name" Type: Prop-encoded string Contents: Standard device type property Value: "serial" "device_type" Type: Prop-encoded-array, Property encoded string Contents: Standard device type property Value: "serial" "compatible" Type: Prop-encoded array, composite string properties Contents: Standard device type property Value: "FJSV,su" "reg" Type: Prop-encoded-array, list of property-encoded integers Contents: The physical address and size "reg-format" in Ebus address space. Format: { chip# offset size } "status" Type: Prop-encoded-array, Property encoded string Contents: Standard property to indicate the operational status of the serial port Value: Absence of property or property value "okay" indicates that serial port is operational. Other values represent non- operational serial port. "interrupts" Type: Property-encoded-array, 1 property-encoded integer Contents: Interrupt number for serial port Value: 00000001 2.2.3.2 Serial Port Device Node Methods open ( -- okay? ) Prepare this device for subsequent use. close ( -- ) Close this previously opened device. read ( adr len -- actual ) Read device into memory buffer; return actual byte count. write ( adr len -- actual ) Write memory buffer to device; return actual byte count. install-abort ( -- ) Begin polling for a console abort sequence. remove-abort ( -- ) Cease polling for a console abort sequence. restore ( -- ) Restore device to useable state after unexpected reset. 2.2.4 SCF Interface Device Node OPL implements Domain to SCF (Service processor) interface via SRAM under the Ebus. The SCF interface device node represents the SCF interface registers as well as the SRAM space for the interface itself. The device path to SCF interface node would be: /pci@,/ebus@/scfc@, 2.2.4.1 SCF Interface Device Node Properties "name" Type: Prop-encoded string Contents: Standard device type property Value: "scfc" "compatible" Type: Prop-encoded array, composite string properties Contents: Standard device type property Value: "FJSV,scfc" "reg" Type: Prop-encoded-array, list of property-encoded integers Contents: The physical address and size "reg-format" in Ebus address space. Format: { chip# offset size } "status" Type: Prop-encoded-array, Property encoded string Contents: Standard property to indicate the operational status of the SCFI Value: Absence of property or property value "okay" indicates that SCFI is operational. Other values represent non- operational SCFI. "interrupts" Type: Property-encoded-array, 1 property-encoded integer Contents: Interrupt numbers for SCFI Value: 00000001 SCFI Interrupt 2.2.4.2 SCF Interface Device Node Methods None needed. 2.2.5 Panel Interrupt Device Node Panel interrupt register exists in SCF Interface register space and is represented by a separate device node of its own. The device path to Panel interrupt node would be: /pci@,/ebus@/panel@, 2.2.5.1 Panel Interrupt Device Node Properties "name" Type: Prop-encoded string Contents: Standard device type property Value: "panel" "compatible" Type: Prop-encoded array, composite string properties Contents: Standard device type property Value: "FJSV,panel" "reg" Type: Prop-encoded-array, list of property-encoded integers Contents: The physical address and size "reg-format" in Ebus address space. Format: { chip# offset size } "status" Type: Prop-encoded-array, Property encoded string Contents: Standard property to indicate the operational status of the Panel interrupt Value: Absence of property or property value "okay" indicates that panel interrupt is operational. Other values represent non-operational panel interrupt. "interrupts" Type: Property-encoded-array, 1 property-encoded integer Contents: Interrupt number for panel interrupt Value: 00000001 2.2.5.2 Panel Interrupt Device Node Methods None needed. 2.2.6 Pseudo Console Device Node OPL systems have one serial port (tty) on each LSB. For a domain with multiple LSBs, system would have multiple serial ports. OBP selects one serial port and uses as the console device. During DR event (dynamic reconfiguration), LSBs can be added to or removed from the domain. It is possible for domain to lose the LSB that contains the OBP console serial port. Before this happens, Solaris would notify OBP to switch the console to another serial port. It is possible that the newly selected serial port is on an LSB that was added to the domain after Solaris was booted. As a result, OBP would have no knowledge and no device node for this new serial port and stdin and stdout cannot be set with the ihandle of new serial port. To handle DR related issues we define a Pseudo console node. The stdin and stdout would always point to the Pseudo console node. OBP would operate the physical tty device through this Pseudo console node. The device path to Pseudo Console node would be: /pseudo-console 2.2.6.1 Pseudo Console Device Node Properties "name" Type: Prop-encoded string Contents: Standard device type property Value: "pseudo-console" "device_type" Type: Prop-encoded-array, Property encoded string Contents: Standard device type property Value: "serial" "compatible" Type: Prop-encoded array, composite string properties Contents: Standard device type property Value: "FJSV,oplmsu" "tty-port#" Type: Prop-encoded-array, 1 prop-encoded integer Contents: LSB_ID of the system board that has the serial port being used as console. Value: LSB_ID (0-f) 2.2.6.2 Pseudo Console Device Node Methods open ( -- okay? ) Prepare this device for subsequent use. close ( -- ) Close this previously opened device. read ( adr len -- actual ) Read device into memory buffer; return actual byte count. write ( adr len -- actual ) Write memory buffer to device; return actual byte count. install-abort ( -- ) Begin polling for a console abort sequence. remove-abort ( -- ) Cease polling for a console abort sequence. restore ( -- ) Restore device to useable state after unexpected reset. 2.2.7 NVRAM Device Node OPL Systems maintain NVRAM on the Service Processor (SCF). At the system startup time, the contents of NVRAM are copied to domain memory. This NVRAM image, in domain memory, is treated as pseudo device and OBP represents device node for pseudo NVRAM device in OBP device tree. 2.2.7.1 NVRAM Device Node NVRAM node would be a child of the root node. The device path would be: /nvram 2.2.7.1.1 NVRAM Device Node Properties "name" Type: Prop-encoded string Contents: Standard device type property Value: "nvram" "device_type" Type: Prop-encoded-array, Property encoded string Contents: Standard device type property Value: "nvram" 2.2.7.1.2 NVRAM Device Node Methods open ( -- okay? ) Prepare this device for subsequent use. close ( -- ) Close this previously opened device. write ( adr len -- actual ) Write memory buffer to device. read ( adr len -- actual ) Read device into memory buffer. size ( -- n ) Return the number of bytes available. seek ( hi lo -- status ) Set the device position for next read or write. sync ( -- ) Synchronize the NVRAM data in memory to the real device. When this call is made, the NVRAM data in volatile memory is copied to the real device with non-volatile memory. This call is needed to make sure that any change made to the data in NVRAM memory buffer are retained across the system resets and power cycles. 3. Client Interface Methods 3.1 New Client Interface Methods The client interface calls defined for OPL platform support have been grouped according to their function and are defined in following sub-sections. 3.1.1 Console Support 3.1.1.1 FJSV,switch-console ( tty-port# -- failed? ) Notify OBP to switch the console to the new serial port (tty) as specified by tty-port#. A serial port, used as OBP console, may get degraded or DR'ed out of the domain. In such scenario, Solaris selects a new serial port as console and informs OBP to make the corresponding change. tty-port# is the id of the new serial port. The value of tty-port# is same as the LSB_ID of the LSB that contains the new serial port. Call fails if OBP detects an error at specified path or the path is already degraded. 3.1.2 Time Of the Day facility Domains in OPL systems do not implement a Time Of the Day (TOD) hardware. TOD is maintained on the SCF and domain can get the time by sending a mailbox command to SCF. 3.1.2.1 FJSV,get-tod ( -- time stick ) During boot, before SCFI driver is attached, Solaris can request time of the day by making FJSV,get-tod client interface callback to OBP. OBP returns the time in seconds since 00:00:00 UTC, January 1, 1970. Also returns the %stick register value when the time was obtained from SCF. Using 'time' and 'stick' along with current value of %stick, Solaris can calculate the current time. 3.1.2.2 FJSV,set-domain-time ( diff -- ) This client interface call is implemented to support different time zones in different domains. Solaris calls OBP with the time difference (in seconds) between the new and the old time. OBP notifies the SCF about the time differential for the domain. 3.1.3 DR Support 3.1.3.1 FJSV,attach-notice ( LSB_ID -- failed? ) Notify OBP that the LSB specified by LSB_ID has been added to the domain. OBP updates its internal state to prepare for the newly added SCF & Serial port (tty) device as a result of the newly added LSB. A non-zero value of 'failed?' indicates that the CIF call failed. This interface does not allocate any memory resources. Note: The changes made by OBP are not visible to OBP clients. 3.1.3.2 FJSV,detach-notice ( LSB_ID -- failed? ) Notify OBP that the LSB specified by LSB_ID has been removed from the domain. OBP updates its internal state to prepare for the SCF & Serial port (tty) path that has been removed from the domain as a result of the removal of the LSB. A non-zero value of 'failed?' indicates that the CIF call failed. This interface does not allocate any memory resources. Note: The changes made by OBP are not visible to OBP clients.