/* * Copyright 2005 Sun Microsystems, Inc. All rights reserved. * Use is subject to license terms. */ #pragma ident "@(#)ontario_pcp_spec.txt 1.6 05/06/24 SMI" Name: Solaris 'libpcp' Library Specification Document. Location: https://hees.east.sun.com/X/ontario/sw/dat/teams/sw/pcp/ ontario_pcp_spec.txt Revision History: Version Author Date Comments 1.1 Venu Mula 05/13/05 Initial version 1.2 Venu Mula 05/13/05 Added libpcp api usage 1.3 Venu Mula 06/07/05 Added libpcp block diagram 1.4 Venu Mula 06/10/05 Added interface table 1.5 Venu Mula 06/17/05 Added libpcp guide lines 1.6 Venu Mula 06/24/05 Changed whole document according PSARC inception review feedback. 1. Introduction This document describes 'libpcp' (platform channel protocol library) specification and its interfaces available in Solaris for sending/receiving messages from Solaris to System Controller via platform virtual channels in Niagara CPU based Sun4v platforms. 1.1 References 1.1.1 Project Documents Archive : http://hees.east.sun.com/X/ontario/sw/dat/teams/sw/pcp/ 1.1.2 'libpcp' 1-pager : http://hees.east.sun.com/X/ontario/sw/dat/teams/sw/pcp/libpcp.1pager 1.1.3 PSARC Case for this project : http://sac.eng/arc/PSARC/2005/372/ 1.1.4 FWARC case for Hypervisor Service API : http://sac.sfbay.sun.com/arc/FWARC/2005/173/ 1.2 Terminology 1.2.1. The words "application" or "libpcp users" or "Solaris applications" identify the Solaris side of virtual channel end-point. 1.2.2. The words "channel" or "virtual channel" identify the virtual channel provided by hypervisor/glvc for communication between Solaris and SC. 1.2.3. The words "platform virtual channel" or "platform channel" or "PCP channels" identify the virtual channels provided by glv/hypervisor for platform support usage. 1.2.4. The words "SC application" or "ALOM Tasks" identify the Solaris side of virtual channel end-point. 1.3 Glossary PCP - Platform Channel Protocol SC - System Controller or Service Processor libpcp - Solaris platform channel protocol library PD - Partition Descriptor 2. Project Description 2.1 Overview In Niagara CPU based sun4v platforms (Ontario and Erie), Host to System Controller (SC) communication is done via virtual channels provided by Hypervisor/glvc. On Solaris, glvc driver provides generic read/write/ioctl interfaces for reading/writing data to and from virtual channels. glvc driver does not define any protocol on how the messages are exchanged between virtual channel end points (i.e Solaris and SC) or does not define any message format for the data sent on virtual channels. On Ontario and Erie platforms Solaris applications such as SunVTS, Explorer, SunMC etc.. are required to communicate with System Controller for platform support. The main aim of this project is to implement Platform Channel Protocol (PCP) on Solaris for proper message exchange with SC on platform virtual channels and to provide interfaces for Solaris applications able to send/receive data to and from SC. 2.2 Assumptions 2.2.1. To use libpcp interfaces, it is assumed that Platform System Controller firmware must implement Platform Channel Protocol (Refer APPENDEX B). 2.2.2. To communicate with SC on a virtual channel, it is assumed that corresponding virtual channel device node is created by glvc/ hypervisor. To create new platform virtual channels, hypervisor partition descriptor (PD) table should include an entry for each virtual channel. Hypervisor PD is defined by platform system firmware. 2.2.3. It is assumed that libpcp shall not mandate any specific byte order for the application specific message data. It's left upto to the channel end applications. 2.3 Out of Scope The following things are out of scope for this project. 2.3.1. This project doesn't cover System Controller side support for Platform Channel Protocol. System Controller support for PCP is implemented in SC firmware and is covered under platform SC FWARC. 2.3.2. This project does not define or cover application specific information such as message data format or message types that is passed across channels. Refer respective platform host applications and platform system firmware implementation for this type of information. 3. Communication mechanism on Platform Virtual Channels 3.1 Platform virtuanl Channel communication Block Diagram ________ | SunVTS | | ALOM |____ _______ . | Task | | | |___ . |________| | |SunVTS | | . . . _________ | |_______| | . . . | SunMC | | | . . . | ALOM | | | . ________ . . | Task |__ | ________ |_.__| | . . |________| | | | | . | | . . _________ | | | SunMC |___.__| | . . | |__| | |________| . | | _______ . _________ ______ | Data |____| . |libpcp | | | . | | | | | Socket |____ __________ . | |__| Glvc |_.__| Hyper- |__| VBSC |___| Monitor|__ | | |_.__| | |_______| . | -Visor | |______| |________| | | | Explorer | . | | . |_________| . | | |__________| .__| | . . __________ | | .| |_______| . . | Explorer | | | __________ .| . . | ALOM |__| | | | .| . . | Task | | | Picl |_.| . . |__________| | | Plugin | . . . | |__________| . . . _________ | . . . | Picl | | . . . | Plugin |_____| . . . | Task | . . . |_________| . . . <-- Libpcp --.-><---- Solaris ------>. <-System FW -><------ SC firmware -----> Users. . <- PSARC-2005/372 -> <- FWARC-2005/173 --> <- FWARC-2005xxx --> Figure 1 : libpcp user apps <-> System Controller communication mechanism over platform virtual channels. 3.2 Description of PCP channel communication The following steps explain the end to end communication on platform virtual channels on sun4v platforms using libpcp interfaces. 1. Host application (such as SunVTS, SunMC etc..) requests libpcp to send data on a particular virtual channel. Refer section 4 & APPENDEX:A for detailed information on how host applications requests libpcp to send and receive data on virtual channels. 2. libpcp first creates a request message header which includes PCP protocol specific information and host application specific information such as (msg_type, sub_type & msg_len) and requests glvc driver to send request message header on virtual channel. 3. Corresponding SC application (such as SunVTS ALOM task etc..) gets the request message header, does a sanity check on header and based on header information, it waits for the request message data to arrive from host application. 4. libpcp now sends request message data, provided by host applications, to SC. 5. After receiving request message data, SC application executes appropriate application specific functionality based on request message and sends response message header and response message data in two seperate packets to libpcp. 6. libpcp receives response message header, does a sanity check (such as checksum), and after receiving response message data, it passes host application specific information (available in response message header) and response message data to host applications. 4. Solaris 'libpcp' Library interfaces. 4.1. Interface Table ------------------------------------------------------------ Name Stability comments ------------------------------------------------------------ struct pcp_msg_t sun private - function pcp_init() sun private - function pcp_send_recv sun private - function pcp_close sun private - ------------------------------------------------------------ 4.2. Guidelines for communicate with Service Controller using libpcp interfaces via virtual channels on Ontario platform. a) For communicating from Host to Service Controller (SC) using virtual channels on Ontario, glvc/hypervisor (FWARC/2005/173) needs to create a seperate virtual channel for every channel consumer. This information needs to be added to platform specific Partition Descriptor (PD) which resides in System firmware. Based on PD, glvc/hypervisor creates virtual channel device nodes under /devices/virtual-devices@xxx directory in Ontario platform. In current implementation, libpcp consumers provides this virtual channel device node to libpcp initialization routine (pcp_init()). b) Solaris applications and Service Service Controller side needs to define and implement appropriate support for following information. - 'msg_type' and 'sub_type' fields in request and response message headers. - message data format for request and response messages. (i.e 'msg_data' field in pcp_msg_t struct) - Solaris applications and System Controller firmware needs to implement support for interpreting the request/response messages and appropriate functionality. Note: libpcp does not define or interpret 'msg_type', 'sub_type' or message data format fields. c) For examples on how to use libpcp interfaces, Refer Platform specific applications such as SunVTS hsclbtest, Explorer, SunMC which access SC using virtual channels to get platform related information. 4.3. Interfaces Description. libpcp supports following interfaces and data structures for host applications to send/receive messages on platform virtual channels. 4.3.1 "pcp_msg_t" structure Host applications exchanges information with libpcp for sending or receiving data on virtual channels using pcp_msg_t data structure. struct pcp_msg { uint8_t msg_type; uint8_t sub_type; uint16_t rsvd_pad; uint32_t msg_len; void *msg_data; } pcp_msg_t; * 'msg_type' & 'sub_type' fields are application specific fields interpreted by host applications and system controller. * 'msg_len' field identifies that length of message data. * 'msg_data' fields points to the message data. For every message on virtual channel, libpcp or SC first sends a message header, which includes host application specific information such as msg_type, sub_type & msg_len and then sends the message data (i.e msg_data) on platform virtual channel. * 'msg_type' & 'sub_type' fields are application specific fields interpreted by host applications and system controller. * 'msg_len' field identifies that length of message data. * 'msg_data' fields points to the message data. 4.3.1 "pcp_init()" interface NAME: pcp_init - initializate virtual channel SYNOPSIS: #include "libpcp.h" int pcp_init(char *channel_name); DESCRIPTION: pcp_init() function initializes the platform virtual channel. 'channel_name' refers to the virtual channel created by glvc/hypervisor. In current implementation host applications shall supply complete path of the virtual channel device node as 'channel_name' to pcp_init(). RETURN VALUES: Upon successful completion, the pcp_init() initializes the virtual channel and returns a non-negative integer representing the channel file descriptor. Otherwise, a negative interger is returned to indicate channel initialization error. PCPL_INVALID_ARGS - Invalid arguments. PCPL_GLVC_ERROR - glvc driver error. PCPL_ERROR - libpcp internal error. 4.3.2 "pcp_send_recv()" Interface NAME: pcp_send_recv - send/receive message on virtual channel SYNOPSIS: #include "libpcp.h" int pcp_send_recv(int channel_fd, pcp_msg_t *req_msg, pcp_msg_t *resp_msg, uint32_t timeout); DESCRIPTION: pcp_send_recv() function sends request message - 'req_msg' - on virtual channel pointed by 'chnl_fd' and if host application expects a response message, the response message is returned in 'resp_msg'. The 'timeout' field identifies the time in secs libpcp to wait for the response message before returning timeout error. The timeout field can have following valid values: PCP_TO_NO_RESPONSE - no response message is expected 0 - wait forever for the response message. (1 to 0xfffffffe) - time to wait in secs for the response message. Note: Upon receiving response message, host application shall free the memory allocated for response message data (i.e resp_msg->msg_data) using free() call. RETURN VALUES: Upon successful completion, the pcp_send_recv() returns zero value. Otherwise, a negative interger is returned to indicate an error while sending/receving messages on virtual channel. PCPL_INVALID_ARGS - Invalid arguments. PCPL_GLVC_TIMEOUT - glvc call timeout. PCPL_XPORT_ERROR - Channel transport error. PCPL_MALLOC_FAIL - malloc failure. PCPL_CKSUM_ERROR - channel data checksum error. PCPL_ERROR - libpcp internal error. 4.3.1 "pcp_close()" interface NAME: pcp_close - close virtual channel SYNOPSIS: #include "libpcp.h" int pcp_close(int channel_fd); DESCRIPTION: pcp_close() function closes the virtual channel connection. RETURN VALUES: Upon successful completion, the pcp_close() returns 0. Otherwise, it will return -1 to indicate an error during channel close. 4.4 Example for 'libpcp' interface usage by user applications. libpcp api is available to user applications as a shared library (libpcp.so). The libpcp.so library will reside under "/usr/platform/`uname -i`/lib/" directory. Note: In current implementation 'libpcp.so' is available under Ontario and Erie platforms. For other platforms to use this library, proper links shall be made from respective platform lib directory to libpcp.so files under Ontario platform lib directory. libpcp user applications should follow following sequence of steps to exchange messages with SC via virtual platform channels. a). Call pcp_init() with appropriate application channel name. Note: Refer platform specific PD (partition descriptor) about different channels. glvc driver creates channels device nodes under /devices/virtual-devices@100/ directory on Ontario platform. b). If pcp_init() fails (i.e return value < 0 ), the libpcp user application can not proceed further in using libpcp api to communicate with SC. At this point, the user application may want log an error and exit in using libpcp api. c). Call pcp_send_recv() with appropriate arguments for sending/receiving messages. d). If pcp_send_recv() fails either repeat step 'c' or goto step 'g'. e). If pcp_send_recv is successful, free the response message data memory. libpcp allocates memory for the response message data. Applications should free the memory using free() call. (i.e "free(resp_msg->msg_data") ) f). To exchange more messages with SC, goto step 'c', or if message exchange with SC is done goto step 'g'. g). Call pcp_close(). h). Message exchange with SC is completed. Appendix A: Example source code for SunMC usage of libpcp interfaces for getting fru related info from System Controller. #define SC_FIOBD_SEEPROM_HANDLE 65 #define SUNMC_CHANNEL_TIMEOUT 30 /* secs*/ typedef struct pcp_get_fru_data { dp_handle_t handle; } pcp_get_fru_data_t; typedef struct pcp_fru_data { dp_handle_t handle; unsigned char fru_presence; unsigned char fru_data_status; unsigned char part_no[PCP_MAX_FRU_PN]; unsigned char serial_no[PCP_MAX_FRU_SN]; unsigned char vendor_name[PCP_MAX_FRU_VEND_NAME]; unsigned char initial_hw_dash_level[PCP_MAX_FRU_HW_DL]; unsigned char initial_hw_rev_level[PCP_MAX_FRU_HW_RL]; unsigned char dimm_size[PCP_MAX_DIMM_SZ]; } pcp_fru_data_t; void main() { int chnl_fd; pcp_get_fru_data_t *fru_data_req; pcp_fru_data_t *fru_data_resp; pcp_msg_t send_msg; pcp_msg_t recv_msg; int ret; /* initialize virtual channel */ if ((chnl_fd = pcp_init("sunmc")) < 0) { printf("channel initialization failed\n"); exit(1); } /* create request message data */ fru_data_req = malloc(sizeof(pcp_get_fru_data_t)); if (fru_data_req == NULL) { printf("malloc failure\n"); exit(1); } fru_data_req->handle = SC_FIOBD_SEEPROM_HANDLE; /* fru handle */ /* Fill in request message */ send_msg.msg_type = PCP_GET_FRU_DATA; send_msg.sub_type = 0; /* not used */ send_msg.msg_len = sizeof(pcp_get_fru_data_t); send_msg.msg_data = fru_data_req; /* send and recv message on virtual channel */ ret = pcp_send_recv(chnl_fd, &send_msg, &recv_msg, SUNMC_CHANNEL_TIMEOUT); if (ret < 0) { printf("pcp_send_recv failed. ret=%d\n", ret); free(fru_data_req); exit(1); } /* Parse response message data */ fru_data_resp = (pcp_fru_data_t *) recv_msg.msg_data; if (fru_data_resp == NULL) { printf("NULL response message data\n"); free(fru_data_req); exit(1); } .... /* End of parsing response message data */ free(fru_data_req); /* free response message data */ free(recv_msg.msg_data); /* close virtual channel fd */ pcp_close(chnl_fd); exit(0); } APPENDEX B: Platform Channel Protocol Description : To initiate messages on platform channels, the sender first sends the platform channel message header to the receiver. The message header is sent in network byte order. The message header always starts with a magic number, which will be same for all types messages on platform channels, to avoid message framing errors. /* * magic number for Platform Channel Protocol (PCP) * ~(rot13("PCP_") = 0xAFBCAFA0 * rot13 is a simple Caesar-cypher encryption that replaces each English * letter with the one 13 places forward or back along the alphabet. */ #define PLAT_CHNL_PROT_MAGIC_NUM (0xAFBCAFA0) The magic number is followed by protocol version. The protocol version field can be used for compatabiity of future protocol format changes. #define PLAT_CHNL_PROT_VER_1 1 Based on the message type ('msg_type') the message data is interpreted differently. The 'msg_type' and 'sub_type' fields are provided by user applications. /* * Platform Channel Request Message Header. */ typedef struct pcp_req_msg_hdr { uint32_t magic_num; /* magic number */ uint8_t proto_ver; /* version info for */ /* backward compatibility */ uint8_t msg_type; /* provided by user apps */ uint8_t sub_type; /* provided by user apps */ uint8_t rsvd_pad; /* padding bits */ uint32_t xid; /* transaction id */ uint32_t timeout; /* timeout in seconds */ uint32_t msg_len; /* length of request or response data */ uint16_t msg_cksum; /* 16-bit checksum of req msg data */ uint16_t hdr_cksum; /* 16-bit checksum of req hdr */ } pcp_req_msg_hdr_t; /* * Platform Channel Response Message Header. */ typedef struct pcp_resp_msg_hdr { uint32_t magic_num; /* magic number */ uint8_t proto_ver; /* version info for */ /* backward compatibility */ uint8_t msg_type; /* passed to user apps */ uint8_t sub_type; /* passed to user apps */ uint8_t rsvd_pad; /* for padding */ uint32_t xid; /* transaction id */ uint32_t timeout; /* timeout in seconds */ uint32_t msg_len; /* length of request or response data */ uint32_t status; /* response status */ uint16_t msg_cksum; /* 16-bit checksum of resp msg data */ uint16_t hdr_cksum; /* 16-bit checksum of resp hdr */ } pcp_resp_msg_hdr_t; The transaction id (xid) field is used to match the request and response messages. The timeout value indicates to the receiver the time in seconds the sender expects the response. (0xFFFFFFFF) indicates no response is wanted; (0) indicates a willingness to wait forever. #define PCP_TO_NO_RESPONSE (0xFFFFFFFF) #define PCP_TO_WAIT_FOREVER (0) The message length (msg_len) field tells the length of message data. The header and message checksum values (msg_cksum & hdr_cksum) are used for data integrity of message header and data. The receiver after receiving the message header waits until it gets 'msg_len' bytes before interpreting the message data. The status field in the message header is used in the response message to communicate following generic errors to the message sender. #define PCP_OK (0) /* message received okay */ #define PCP_ERROR (1) /* generic error */ #define PCP_HDR_CKSUM_ERROR (2) /* header checksum error */ #define PCP_MSG_CKSUM_ERROR (3) /* message checksum error */ #define PCP_XPORT_ERROR (4) /* message in complete error */ After sending the message header, the sender sends the message data. The sender waits for the response message, if it is expecting for a response.