From tedk@sac.sfbay.sun.com Thu Jun 24 17:33:55 2010 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o5P0XsqQ006117 for ; Thu, 24 Jun 2010 17:33:54 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o5P0XsFf017788; Thu, 24 Jun 2010 17:33:54 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L4J0070JO8IOC00@nwk-avmta-2.sfbay.sun.com>; Thu, 24 Jun 2010 17:33:54 -0700 (PDT) Received: from dm-sfbay-02.sfbay.sun.com ([129.146.11.31]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L4J00MLDO8IG560@nwk-avmta-2.sfbay.sun.com>; Thu, 24 Jun 2010 17:33:54 -0700 (PDT) Received: from sac.sfbay.sun.com (sac.SFBay.Sun.COM [129.146.226.132]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o5P0XsKW027828; Thu, 24 Jun 2010 17:33:54 -0700 (PDT) Received: from sac.sfbay.sun.com (localhost [127.0.0.1]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o5P0XqWX006112; Thu, 24 Jun 2010 17:33:52 -0700 (PDT) Received: (from tedk@localhost) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id o5P0XqjA006108; Thu, 24 Jun 2010 17:33:52 -0700 (PDT) Date: Thu, 24 Jun 2010 17:33:52 -0700 (PDT) From: Ted Kim Subject: OFUV Userland Interface [PSARC/2010/239 FastTrack timeout 07/02/2010] To: PSARC-ext@sun.com Message-id: <201006250033.o5P0XqjA006108@sac.sfbay.sun.com> Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 Status: RO Content-Length: 16627 Template Version: @(#)sac_nextcase 1.70 03/30/10 SMI This information is Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved. 1. Introduction 1.1. Project/Component Working Name: OFUV Userland Interface 1.2. Name of Document Author/Supplier: Author: Brendan Doyle 1.3 Date of This Document: 24 June, 2010 4. Technical Description Table of Contents ----------------- 4.1 Introduction 4.2 Proposal and Summary of Interfaces A. Device Node Entries B. User/Kernel Interfaces C. Config Files D. Security and Resource Control 4.3 Description of Interfaces A. Device Node Entries B. User/Kernel Interfaces 1. RDMA CM 2. Verbs 1. Verbs User/Kernel Interface 2. Asynchronous Event Interface 3. Completion Event Interface 3. umad Interface C. Config Files 4.1 Introduction ---------------- In Linux, the most popular InfiniBand (IB) OS-bypass framework is the Open Fabrics User Verbs (OFUV) framework from the Open Fabrics Enterprise Distribution (OFED). In Solaris, OFUV is being ported over in two parts: kernel and userland. The main body of the kernel part is described in PSARC/2009/421, which describes the APIs used by kernel consumers of OFUV. This case describes the userland interface (including OS-bypass) to OFUV. The complete userland interface includes userland-facing interfaces of components originally described in PSARC/2009/421 and one new kernel module (sol_umad), which includes the interfaces used by the ported OFED libumad library to the Solaris kernel IBMF module. The actual userland portion of OFUV is a companion project which delivers the open source libraries and utilities. The main business objective is enabling the OFED libraries and utilities to satisfy some of the requirements of Oracle appliance platforms and other consumers, such as financial messaging systems and MPI. Reference Documents: o Open Fabrics User Verbs (OFUV) - primary kernel components PSARC/2009/421 one-pager http://sac.sfbay/PSARC/2009/421/20090731_brendan.doyle o Kernel RDMA CM API Architecture and use: materials directory: ofuv_rdma_arch.txt o Solaris Open Fabrics User Verbs Architecture Document: materials directory: solaris_ofuv_arch.pdf o OFUV Implementation Details: materials directory: OFUVImplementationDetails.pdf Related ARC cases: PSARC/2002/132 IBTF: InfiniBand Transport Framework PSARC/2002/607 InfiniBand Management Framework (IBMF) PSARC/2003/145 uDAPL: User Direct Access Programming Library PSARC/2009/421 Open Fabrics User Verbs (OFUV) - primary kernel components 4.2 Proposal and Summary of Interfaces -------------------------------------- This case proposes a userland to kernel interface for OFUV. The interface taxonomy classification for this interface is Volatile. The rationale for this choice is the fact that the interface is externally specified in OFED. And as Volatile has absorbed the former External classification in the current taxonomy, we use that. This initial version of OFUV corresponds to OFED-1.3.1. Later phases of the project will update this to the current version (which as of this writing is OFED-1.5.1). A micro/patch binding is asserted for this interface. A. /devices entries o '/devices/ib/sol_uverbs' o '/devices/ib/sol_uverbs@0:uverbs#' o '/devices/ib/sol_uverbs@0:ucma' # - minor number o '/devices/pseudo/sol_ucma@0' o '/devices/pseudo/sol_ucma@0:sol_ucma' o '/devices/ib/sol_umad@0' o '/devices/ib/sol_umad@0:issm# o '/devices/ib/sol_umad@0:umad# # - minor number The umad device files have permission 600 and are owned by root. All other device files have permission 666 and are owned by root. See section 4.2.D. Security and Resource Control for further discussion. B. User/kernel Interfaces These interfaces are only used by the respective userland library components to access to the kernel components of this project. B.1 RDMA CM - 'librdmacm' library to 'sol_ucma' kernel module B.2 Verbs - 'libibverbs' library to 'sol_uverbs' kernel module B.3 umad - 'libibumad' library to 'sol_umad' kernel module C. Config Files /kernel/drv/sol_uverbs.conf /kernel/drv/sol_ucma.conf /kernel/drv/sol_umad.conf D. Security and Resource Control The sol_umad minor nodes are owned by root and has 600 file permissions. Only root can execute the tools and utilities that use these minor nodes as a means of access to the kernel IBMF (see 4.3.B.3). We restrict this because the Management Datagram (MAD) based tools/utilities allow users to reconfigure the IB fabric and nodes on the fabric. The sol_uverbs and sol_ucma minor nodes are owned by root and have 666 file permissions. All users can run the utilities that use these files as a means of access to the kernel IBTF. These utilities are not intrusive and do not allow configuration of the IB fabric or fabric nodes. However protection is afforded against non-root users use of privileged Q_Keys and use of reserved well-known ports. Protecting privileged Q_Keys prevents spoofing of certain types of traffic, including management messages. The protections on using reserved well-known ports is analogous to the restrictions on socket binds of ports numbered less than 1024. Checks are made in the kernel to verify the user process has the PRIV_NET_PRIVADDR privilege for these operations, before the request is allowed to proceed. This project relies on the same resource controls as the uDAPL project (See PSARC/2003/145) namely the use of the project.max-locked-memory to control the amount of memory a user process can pin down. 4.3 Description of Interfaces ----------------------------- A. Device node entries o '/devices/ib/sol_uverbs' character device Major node for the uverbs and ucma minor nodes provided by the 'sol_uverbs' driver o '/devices/ib/sol_uverbs@0:uverbs#' # - minor number (0-max), max is 15 character minor node device is created for each HCA in the system -- These device nodes are utilized by the 'libibverbs' library to open specific HCA instances and provide verb operations. Properties exported for each minor device node are: name type value ---- ---- ----- "device-id" int Vendor supplied part ID "vendor-id" int 24 bit Vendor ID "abi-version" int Defines the userland verbs to kernel interface version. This property is defined only for minor number 0. "guid" int The HCA's node GUID o '/devices/ib/sol_uverbs@0:ucma' character device minor node that may only be opened from within the kernel and is created for the 'sol_ucma kernel' module use -- This is project private. This device node is created for interaction between the 'sol_ucma' and 'sol_uverbs' drivers. The 'sol_ucma' driver does an ldi_open(9f) of this device file, to ensure that 'sol_uverbs' driver is loaded. o '/devices/pseudo/sol_ucma@0' pseudo device major node for the sol_ucma minor note provided by the sol_ucma driver o '/devices/pseudo/sol_ucma@0:sol_ucma' pseudo device minor node for RDMA CM operations Properties exported by the minor device node are: name type value ---- ---- ----- "abi-version" int Defines the userland verbs to kernel interface version o '/devices/ib/sol_umad' character device Major node for the issm and umad minor nodes provided by the 'sol_umad' driver o '/devices/ib/sol_umad@0:issm# o '/devices/ib/sol_umad@0:umad# # - minor number (0-max), max is 15 character device minor nodes that are created for each HCA port in the system -- These device nodes are utilized by the 'libibumad' for IsSM control and MAD operations. Properties exported for each minor device node are: name type value ---- ---- ----- "abi-version" int Defines the userland verbs to kernel interface version "guid" int The HCA's node GUID Opening the issm devices causes the IsSM port capability bit to be set on that port. Closing the issm device causes the IsSM port capability to be reset. IsSM signals the presence of an IB Subnet Manager on the port. B. User/kernel Interfaces This project also implements the OFED userland to kernel interfaces defined by the OFED headers: 'ib_user_verbs.h', 'rdma_user_cm.h' & 'ib_user_mad.h' (all ported from Linux). These interfaces allow the OFED 'libibverbs', 'librdmacm' & 'libibumad' libraries to access the kernel IB modules needed to implement the APIs exported by the libraries. The 'sol_uverbs' kernel module of this project implements the interface to the 'libibverbs' library defined in 'ib_user_verbs.h', the 'sol_ucma' kernel module implements the interface to the 'librdmacm' CM library defined in 'rdma_user_cm.h', and the 'sol_umad' kernel module implements the interface to the 'libibumad' library defined in 'ib_user_mad.h'. B.1 RDMA CM The 'sol_ucma' driver exports a device propertt named "abi-version" that may be examined to determine compatibility. The property has the following format: name='abi-version' type=int value=00000004 Entry points: open(2) obtain a CM fd close(2) close a CM fd poll(2) Poll for CM events write(2) write RDMA CM API commands to 'sol_ucma' kernel module on fd obtained from open() Details of the commands are in 'sol_ucma-cmds.txt' (in the materials directory). For reference, there is also a copy of rdma_user_cm.h in the materials directory. B.2 Verbs The 'sol_uverbs' driver exports a device property named "abi-version" that may be examined by the the userland consumer ('libibverbs') and used to determine the appropriateness of the user/kernel ABI compatibility. The property has the following format: name='abi-version' type=int value=00000006 If the user/kernel ABI revision are compatible, the consumer opens the desired character minor device for use: /devices/ib/sol_uverbs@0:uverbs# # - minor number (0-max) At the time of the open, 'sol_uverbs' creates unique minor device number beyond the range of all possible HCA/UCMA minor devices, and uses that to identify the caller's user context on subsequent calls into the driver. The consumer then uses the device write(2) entry point to invoke commands associated with the user/kernel ABI. The user commands are defined in the OFED header 'ib_user_verbs.h'. Where implementation specific information may be required, the commands allow for an opaque data area to be passed transparently between the underlying kernel device specific channel interface implementation and the device specific user space consumer of the ABI (e.g. the 'hermon' kernel driver may pass hermon specific variables associated with a command to the device specific consumer entity 'libmlx4'). This ABI has been extended with support for two commands not required by the OFED implementation, IB_USER_VERBS_CMD_QUERY_GID and IB_USER_VERBS_CMD_QUERY_PKEY. In the OFED kernel implementation, the underlying hardware specific driver exposes the device's GID table and PKEY table entries via the Linux /sys file system. In the absence of the /sys file system, the consumer utilizes the 'sol_uverb' driver to read the values from the underlying hardware specific driver. Additionally a per process asynchronous event file is created when the consumer opens of a specific 'sol_uverbs' minor node device See B.2.2) And a completion notification event file is created when the consumer requests creation of a completion channel via the 'IB_USER_VERB_CMD_CREATE_COMP_CHANNEL' (See B.2.1 & B.2.3). For reference, there is also a copy of ib_user_verbs.h in the materials directory. B.2.1 Verbs User/Kernel Interface /devices/ib/sol_uverbs@0:uverbs# # - minor number (0-max) open(2) Obtain fd to write(2) user verbs commands to the 'sol_uverbs' close(2) close fd from open(2) write(2) write user verbs commands to the 'sol_uverbs' kernel module There are two types of buffer passed to write(2) destined for the 'sol_uverbs' driver. They are (a) commands with no response and (b) commands with a response. Details are given in 'sol_uverbs-cmds.txt' (in the materials directory). B.2.2 Asynchronous Event Interface A per process asynchronous event file is created when the consumer opens of a specific 'sol_uverbs' minor node device. Operations supported on the async_fd are: open(2) Obtain fd for async event interface close(2) Close per process fd created by 'sol_uverbs' for delivery of async events read(2) Read on async event fd of an async event into an 'ib_uverbs_async_event_desc' structure: uint64_t element; uint32_t event_type; uint32_t reserved; Where 'event_type' is one of: IBV_EVENT_CQ_ERR IBV_EVENT_QP_FATAL IBV_EVENT_QP_REQ_ERR IBV_EVENT_QP_ACCESS_ERR IBV_EVENT_COMM_EST IBV_EVENT_SQ_DRAINED IBV_EVENT_PATH_MIG IBV_EVENT_PATH_MIG_ERR IBV_EVENT_DEVICE_FATAL IBV_EVENT_PORT_ACTIVE IBV_EVENT_PORT_ERR IBV_EVENT_LID_CHANGE IBV_EVENT_PKEY_CHANGE IBV_EVENT_SM_CHANGE IBV_EVENT_SRQ_ERR IBV_EVENT_SRQ_LIMIT_REACHED IBV_EVENT_QP_LAST_WQE_REACHED IBV_EVENT_CLIENT_REREGISTER And 'element' is one of: struct ibv_cq *cq; /* CQ that got the event */ struct ibv_qp *qp; /* QP that got the event */ struct ibv_srq *srq; /* SRQ that got the event */ int port_num; /* port that got the event */ When 'element' is 'cq', 'qp' or 'srq' 'element is a userland address that is a pointer to a userland allocated struct. poll(2) Poll for events on async event fd B.2.3 Completion Event Interface A completion notification event file is created when the consumer requests creation of a completion channel via the 'IB_USER_VERB_CMD_CREATE_COMP_CHANNEL' (See B.2.1). Operations supported on the cq_fd are: open(2) Obtain fd for completion event interface close(2) Close per process fd created by 'sol_uverbs' for delivery of completion events read(2) Read on cq event fd of a completion event into an 'ib_uverbs_comp_event_desc' structure: uint64_t cq_handle; Where 'cq_handle' is is a userland address that is a pointer to a userland allocated struct ibv_cq poll(2) Poll for events on cq event fd B.3 umad Interface The 'sol_mad' driver exports a device property named "abi-version" that may be examined by the the userland consumer ('libibumad')and used to determine the appropriateness of the user/kernel ABI compatibility. The property has the following format: name='abi-version' type=int value=00000005 If the user/kernel ABI revision are compatible, the consumer can open the desired character minor device for use: /devices/ib/sol_umad@0:issm# /devices/ib/sol_umad@0:umad# # - minor number (0-max) At the time of the open 'sol_umad' creates unique minor device number and uses that to identify the caller's user context on subsequent calls into the driver. The consumer then uses the device write(2) and ioctl(2) entry point to invoke commands associated with the user/kernel ABI. Details of the commands are defined in sol_umad-cmds.txt (in the materials directory). For reference, there is also a copy of ib_user_mad.h in the materials directory. C. Config Files The following config files, that contain no user configurable data, are provided: /kernel/drv/sol_uverbs.conf /kernel/drv/sol_ucma.conf /kernel/drv/sol_umad.conf 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: FastTrack 6.6. ARC Exposure: open From Darren.Moffat@Oracle.COM Fri Jun 25 02:16:43 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o5P9GhFm008452 for ; Fri, 25 Jun 2010 02:16:43 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o5P9Gh00009235 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Fri, 25 Jun 2010 03:16:43 -0600 (MDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L4K00D03CFVPW00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 25 Jun 2010 03:16:43 -0600 (MDT) Received: from sca-ea-mail-1.sun.com ([192.18.43.24]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L4K00M5JCFUKKC0@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 25 Jun 2010 03:16:42 -0600 (MDT) Received: from rcsinet15.oracle.com (rcsinet15.oracle.com [148.87.113.117]) by sca-ea-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o5P9Gag7023795; Fri, 25 Jun 2010 09:16:36 +0000 (GMT) Received: from acsmt353.oracle.com (acsmt353.oracle.com [141.146.40.153]) by rcsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o5P9GV4p008764; Fri, 25 Jun 2010 09:16:31 +0000 (GMT) Received: from abhmt002.oracle.com by acsmt354.oracle.com with ESMTP id 356472101277457365; Fri, 25 Jun 2010 02:16:05 -0700 Received: from [10.7.251.221] (/10.7.251.221) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 25 Jun 2010 02:16:05 -0700 Date: Fri, 25 Jun 2010 10:16:02 +0100 From: Darren J Moffat Subject: Re: OFUV Userland Interface [PSARC/2010/239 FastTrack timeout 07/02/2010] In-reply-to: <201006250033.o5P0XqjA006108@sac.sfbay.sun.com> To: Ted Kim Cc: PSARC-ext@sun.com Message-id: <4C2473D2.3050606@Oracle.COM> Organization: Oracle Solaris Security MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt353.oracle.com [141.146.40.153] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090205.4C2473F4.006C:SCFMA4539814,ss=1,fgs=0 References: <201006250033.o5P0XqjA006108@sac.sfbay.sun.com> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.9) Gecko/20100607 Lightning/1.0b1 OracleBeehiveExtension/1.0.0.0pre3 Thunderbird/3.0.4 Status: RO Content-Length: 1929 On 25/06/2010 01:33, Ted Kim wrote: > D. Security and Resource Control > > The sol_umad minor nodes are owned by root and has 600 file > permissions. Only root can execute the tools and utilities that use > these minor nodes as a means of access to the kernel IBMF (see > 4.3.B.3). We restrict this because the Management Datagram (MAD) based > tools/utilities allow users to reconfigure the IB fabric and nodes on > the fabric. That doesn't fit the Solaris security model. Explicit checks for root or root only device files are highly discouraged because it requires the userland components to run with more privilege than necessary in many cases. A common exception to this is where the operations that can be done on the device are equivalent to asserting all privileges in the effective set. Why can the minor nodes not be 666 and the kernel have a new privilege check ? > However protection is afforded against non-root users use of > privileged Q_Keys and use of reserved well-known ports. Protecting > privileged Q_Keys prevents spoofing of certain types of traffic, > including management messages. The protections on using reserved > well-known ports is analogous to the restrictions on socket binds of > ports numbered less than 1024. Checks are made in the kernel to verify > the user process has the PRIV_NET_PRIVADDR privilege for these > operations, before the request is allowed to proceed. I don't know enough about this area to know if reusing PRIV_NET_PRIVADDR is a good idea or if a new privilege is more appropriate. Is it possible that a single userland process could be choosing between using a AF_INET socket with a port number < 1024 or choosing to use a privileged Q_Key ? General question now. The "sol_" prefix is that short for Solaris ? If so why is it necessary and why are you abbreviating it isn't like it is something that someone needs to type in a lot. -- Darren J Moffat From ted.h.kim@oracle.com Mon Jun 28 14:46:01 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o5SLk0Id023245 for ; Mon, 28 Jun 2010 14:46:00 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o5SLk0c8036591 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Mon, 28 Jun 2010 15:46:00 -0600 (MDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L4Q00901V4O6800@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 28 Jun 2010 15:46:00 -0600 (MDT) Received: from sca-ea-mail-3.sun.com ([192.18.43.21]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L4Q00DYLV4NTEE0@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 28 Jun 2010 15:45:59 -0600 (MDT) Received: from rcsinet13.oracle.com (rcsinet13.oracle.com [148.87.113.125]) by sca-ea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o5SLjwMc001532 for ; Mon, 28 Jun 2010 21:45:59 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by rcsinet13.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o5SL62cF029861 for ; Mon, 28 Jun 2010 21:45:57 +0000 (GMT) Received: from abhmt012.oracle.com by acsmt354.oracle.com with ESMTP id 380594051277761463; Mon, 28 Jun 2010 14:44:23 -0700 Received: from [129.153.3.74] (/129.153.3.74) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 28 Jun 2010 14:44:23 -0700 Date: Mon, 28 Jun 2010 14:42:55 -0700 From: "Ted H. Kim" Subject: Re: OFUV Userland Interface [PSARC/2010/239 FastTrack timeout 07/02/2010] In-reply-to: <4C2473D2.3050606@Oracle.COM> To: Darren J Moffat Cc: PSARC-ext@sun.com Message-id: <4C29175F.6020404@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090204.4C291816.0172:SCFMA4539814,ss=1,fgs=0 References: <201006250033.o5P0XqjA006108@sac.sfbay.sun.com> <4C2473D2.3050606@Oracle.COM> User-Agent: Thunderbird 2.0.0.24 (X11/20100302) Status: RO Content-Length: 3224 Hi Darren, I tried to contact you by private email, but I am not sure it went through. Anyway ... Darren J Moffat wrote: > On 25/06/2010 01:33, Ted Kim wrote: >> D. Security and Resource Control >> >> The sol_umad minor nodes are owned by root and has 600 file >> permissions. Only root can execute the tools and utilities that use >> these minor nodes as a means of access to the kernel IBMF (see >> 4.3.B.3). We restrict this because the Management Datagram (MAD) based >> tools/utilities allow users to reconfigure the IB fabric and nodes on >> the fabric. > > That doesn't fit the Solaris security model. Explicit checks for root > or root only device files are highly discouraged because it requires the > userland components to run with more privilege than necessary in many > cases. > > A common exception to this is where the operations that can be done on > the device are equivalent to asserting all privileges in the effective set. > > Why can the minor nodes not be 666 and the kernel have a new privilege > check ? Okay, project team could do that. I am suggesting again using PRIV_NET_PRIVADDR here for the kernel privilege check. >> However protection is afforded against non-root users use of >> privileged Q_Keys and use of reserved well-known ports. Protecting >> privileged Q_Keys prevents spoofing of certain types of traffic, >> including management messages. The protections on using reserved >> well-known ports is analogous to the restrictions on socket binds of >> ports numbered less than 1024. Checks are made in the kernel to verify >> the user process has the PRIV_NET_PRIVADDR privilege for these >> operations, before the request is allowed to proceed. > > I don't know enough about this area to know if reusing PRIV_NET_PRIVADDR > is a good idea or if a new privilege is more appropriate. In general, we have tried to avoid introducing InfiniBand specific items into the administration model, except where absolutely necessary. In this case, we think it conveys the proper effect both when the privilege is on and when it is off and does not cause a unexpected surprise effect. > Is it possible that a single userland process could be choosing between > using a AF_INET socket with a port number < 1024 or choosing to use a > privileged Q_Key ? The two cases are separate in terms of the underlying mechanisms affected. The port restriction has to do with connection traffic. The Q_Key has to do with datagrams. However, they do have a similar theme w.r.t. being able to get privileged access/traffic. > General question now. The "sol_" prefix is that short for Solaris ? If > so why is it necessary and why are you abbreviating it isn't like it is > something that someone needs to type in a lot. Well perhaps the name could have been chosen differently. But as long as the objection is not a major one, the project team does not want to change at this point, since that would require changes to stuff that already went back for the previous PSARC/2009/421 case. -ted -- Ted H. Kim, PhD Sun/Oracle America Inc. ted.h.kim@oracle.com 222 North Sepulveda Blvd., 10th Floor (310) 341-1116 El Segundo, CA 90245 (310) 341-1115 FAX From Nicolas.Williams@oracle.com Mon Jun 28 15:27:48 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o5SMRmcs024629 for ; Mon, 28 Jun 2010 15:27:48 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o5SMRlKU012934 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Mon, 28 Jun 2010 17:27:48 -0500 (CDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L4Q00D05X2B3R00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 28 Jun 2010 16:27:47 -0600 (MDT) Received: from sca-ea-mail-3.sun.com ([192.18.43.21]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L4Q0095TX2AP120@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 28 Jun 2010 16:27:47 -0600 (MDT) Received: from rcsinet13.oracle.com (rcsinet13.oracle.com [148.87.113.125]) by sca-ea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o5SMRk64015182 for ; Mon, 28 Jun 2010 22:27:46 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by rcsinet13.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o5SKYQZS006170 for ; Mon, 28 Jun 2010 22:27:45 +0000 (GMT) Received: from abhmt020.oracle.com by acsmt353.oracle.com with ESMTP id 380684711277764051; Mon, 28 Jun 2010 15:27:31 -0700 Received: from oracle.com (/129.153.128.104) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 28 Jun 2010 15:27:30 -0700 Date: Mon, 28 Jun 2010 17:27:58 -0500 From: Nicolas Williams Subject: Re: OFUV Userland Interface [PSARC/2010/239 FastTrack timeout 07/02/2010] In-reply-to: <4C29175F.6020404@oracle.com> To: "Ted H. Kim" Cc: Darren J Moffat , PSARC-ext@sun.com Message-id: <20100628222758.GE13125@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090206.4C2921E1.0252:SCFMA4539814,ss=1,fgs=0 References: <201006250033.o5P0XqjA006108@sac.sfbay.sun.com> <4C2473D2.3050606@Oracle.COM> <4C29175F.6020404@oracle.com> User-Agent: Mutt/1.5.20 (2010-03-02) Status: RO Content-Length: 3039 On Mon, Jun 28, 2010 at 02:42:55PM -0700, Ted H. Kim wrote: > Darren J Moffat wrote: > >Why can the minor nodes not be 666 and the kernel have a new > >privilege check ? > > Okay, project team could do that. > I am suggesting again using PRIV_NET_PRIVADDR here for the kernel > privilege check. For the umad devices it seems to me that what you want is either to use one of PRIV_SYS_DL_CONFIG (DL == datalink) or PRIV_SYS_NET_CONFIG (which aggregates several config privs). Alternatively you could add a new privilege called, say, PRIV_SYS_IB_CONFIG and update PRIV_SYS_NET_CONFIG to also aggregate it. PRIV_NET_PRIVADDR is definitely not approproate for operations that can configure/re-configure IB. See the privileges(5) manpage. > >>However protection is afforded against non-root users use of > >>privileged Q_Keys and use of reserved well-known ports. Protecting > >>privileged Q_Keys prevents spoofing of certain types of traffic, > >>including management messages. The protections on using reserved > >>well-known ports is analogous to the restrictions on socket binds of > >>ports numbered less than 1024. Checks are made in the kernel to verify > >>the user process has the PRIV_NET_PRIVADDR privilege for these > >>operations, before the request is allowed to proceed. > > > >I don't know enough about this area to know if reusing > >PRIV_NET_PRIVADDR is a good idea or if a new privilege is more > >appropriate. > > In general, we have tried to avoid introducing InfiniBand > specific items into the administration model, except where > absolutely necessary. In this case, we think it conveys > the proper effect both when the privilege is on and > when it is off and does not cause a unexpected surprise > effect. Adding a privilege is not a big deal because many sysadmins will have root access with all privileges. However, you might want to add RBAC (role-based access control) profiles so that non-root-with-all-privs users can be granted IB admin access. The way you'd do this is by delivering one or more new profiles(4) entries and one or more exec_attr(4) entries as well, with the latter granting either euid=0 or specific privileges to run specific IB programs to users with these profiles. > >Is it possible that a single userland process could be choosing > >between using a AF_INET socket with a port number < 1024 or > >choosing to use a privileged Q_Key ? > > The two cases are separate in terms of the underlying > mechanisms affected. The port restriction has to do with > connection traffic. The Q_Key has to do with datagrams. > However, they do have a similar theme w.r.t. being > able to get privileged access/traffic. They seem roughly comparable, but keep in mind that with TCP/UDP ports the meaning of a port < 1024 is really up to the acceptor, whereas the meaning of controlled Q_Keys may well be less flexible (I don't know). If the peer has no choice as to whether to accept such things then I'd say that you need a new privilege, else I'd say that PRIV_NET_PRIVADDR is fine. Nico -- From ted.h.kim@oracle.com Tue Jun 29 12:17:17 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o5TJHGU5011983 for ; Tue, 29 Jun 2010 12:17:16 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o5TJHEp5025090 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Tue, 29 Jun 2010 14:17:16 -0500 (CDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L4S0003BIWR8200@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 29 Jun 2010 12:17:15 -0700 (PDT) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L4S00DQ6IWQBCB0@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 29 Jun 2010 12:17:15 -0700 (PDT) Received: from rcsinet15.oracle.com (rcsinet15.oracle.com [148.87.113.117]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o5TJHEik029254 for ; Tue, 29 Jun 2010 19:17:14 +0000 (GMT) Received: from acsmt353.oracle.com (acsmt353.oracle.com [141.146.40.153]) by rcsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o5THXv3p005769 for ; Tue, 29 Jun 2010 19:17:13 +0000 (GMT) Received: from abhmt013.oracle.com by acsmt353.oracle.com with ESMTP id 367141461277838977; Tue, 29 Jun 2010 12:16:17 -0700 Received: from [129.153.3.74] (/129.153.3.74) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 29 Jun 2010 12:16:15 -0700 Date: Tue, 29 Jun 2010 12:14:48 -0700 From: "Ted H. Kim" Subject: Re: OFUV Userland Interface [PSARC/2010/239 FastTrack timeout 07/02/2010] In-reply-to: <20100628222758.GE13125@oracle.com> To: Nicolas Williams Cc: Darren J Moffat , PSARC-ext@sun.com Message-id: <4C2A4628.8040908@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt353.oracle.com [141.146.40.153] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090203.4C2A46BA.0034:SCFMA4539814,ss=1,fgs=0 References: <201006250033.o5P0XqjA006108@sac.sfbay.sun.com> <4C2473D2.3050606@Oracle.COM> <4C29175F.6020404@oracle.com> <20100628222758.GE13125@oracle.com> User-Agent: Thunderbird 2.0.0.24 (X11/20100302) Status: RO Content-Length: 4305 Nicholas, > For the umad devices it seems to me that what you want is either to use > one of PRIV_SYS_DL_CONFIG (DL == datalink) or PRIV_SYS_NET_CONFIG (which > aggregates several config privs). Alternatively you could add a new > privilege called, say, PRIV_SYS_IB_CONFIG and update PRIV_SYS_NET_CONFIG > to also aggregate it. Okay, I see your point. I think PRIV_SYS_NET_CONFIG is the more analogous one here. > PRIV_NET_PRIVADDR is definitely not approproate for operations that can > configure/re-configure IB. > > See the privileges(5) manpage. > >>>> However protection is afforded against non-root users use of >>>> privileged Q_Keys and use of reserved well-known ports. Protecting >>>> privileged Q_Keys prevents spoofing of certain types of traffic, >>>> including management messages. The protections on using reserved >>>> well-known ports is analogous to the restrictions on socket binds of >>>> ports numbered less than 1024. Checks are made in the kernel to verify >>>> the user process has the PRIV_NET_PRIVADDR privilege for these >>>> operations, before the request is allowed to proceed. >>> I don't know enough about this area to know if reusing >>> >>> PRIV_NET_PRIVADDR is a good idea or if a new privilege is more >>> appropriate. >> In general, we have tried to avoid introducing InfiniBand >> specific items into the administration model, except where >> absolutely necessary. In this case, we think it conveys >> the proper effect both when the privilege is on and >> when it is off and does not cause a unexpected surprise >> effect. > > Adding a privilege is not a big deal because many sysadmins will have > root access with all privileges. > > However, you might want to add RBAC (role-based access control) profiles > so that non-root-with-all-privs users can be granted IB admin access. > The way you'd do this is by delivering one or more new profiles(4) > entries and one or more exec_attr(4) entries as well, with the latter > granting either euid=0 or specific privileges to run specific IB > programs to users with these profiles. The thing about not adding IB specific items into the admin model is mostly about "least surprise". Despite our view about IB, many admins don't think about IB explicitly. We are trying to avoid unpleasant IB surprises if an admin has his system reasonably configured for "regular" networking and hasn't thought about IB directly. Of course, the other competing tension is to not add excessive or complicated protections, to preserve ease of use. >>> Is it possible that a single userland process could be choosing >>> between using a AF_INET socket with a port number < 1024 or >>> choosing to use a privileged Q_Key ? >> >> The two cases are separate in terms of the underlying >> mechanisms affected. The port restriction has to do with >> connection traffic. The Q_Key has to do with datagrams. >> However, they do have a similar theme w.r.t. being >> able to get privileged access/traffic. > > They seem roughly comparable, but keep in mind that with TCP/UDP ports > the meaning of a port < 1024 is really up to the acceptor, whereas the > meaning of controlled Q_Keys may well be less flexible (I don't know). > If the peer has no choice as to whether to accept such things then I'd > say that you need a new privilege, else I'd say that PRIV_NET_PRIVADDR > is fine. The receiver of a message with a certain Q_Key can always decide to scrutinize it whatever way it wants and/or toss it out. But in light of the umad discussion above, I do think one further refinement should be made. Two particular values of privileged Q_Key (x80010000, x80010001) are used by "General Services" Management Datagrams (MADs). (Note, even people familiar with IB may never have heard of the second case. It is for an obscure service that the industry will probably never implement - DevMgt v2 IORM & DA communication.) If either of those values are obtained then certain MADs could be spoofed. So if either of those values are requested, then I think the same privilege used for umad (PRIV_SYS_NET_CONFIG) should also be checked. -ted -- Ted H. Kim, PhD Sun/Oracle America Inc. ted.h.kim@oracle.com 222 North Sepulveda Blvd., 10th Floor (310) 341-1116 El Segundo, CA 90245 (310) 341-1115 FAX From Nicolas.Williams@oracle.com Tue Jun 29 12:43:40 2010 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o5TJheRW012283 for ; Tue, 29 Jun 2010 12:43:40 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o5TJheMJ018967 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Tue, 29 Jun 2010 12:43:40 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L4S00H1FK4SJ300@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 29 Jun 2010 12:43:40 -0700 (PDT) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L4S000INK4SWV50@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 29 Jun 2010 12:43:40 -0700 (PDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o5TJheLW000110 for ; Tue, 29 Jun 2010 19:43:40 +0000 (GMT) Received: from acsmt353.oracle.com (acsmt353.oracle.com [141.146.40.153]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o5TIUQDo006237 for ; Tue, 29 Jun 2010 19:43:39 +0000 (GMT) Received: from abhmt012.oracle.com by acsmt353.oracle.com with ESMTP id 383705281277840618; Tue, 29 Jun 2010 12:43:38 -0700 Received: from oracle.com (/129.153.128.104) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 29 Jun 2010 12:43:37 -0700 Date: Tue, 29 Jun 2010 14:44:51 -0500 From: Nicolas Williams Subject: Re: OFUV Userland Interface [PSARC/2010/239 FastTrack timeout 07/02/2010] In-reply-to: <4C2A4628.8040908@oracle.com> To: "Ted H. Kim" Cc: Darren J Moffat , PSARC-ext@sun.com Message-id: <20100629194450.GV11785@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt353.oracle.com [141.146.40.153] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090201.4C2A4CEB.0140:SCFMA4539814,ss=1,fgs=0 References: <201006250033.o5P0XqjA006108@sac.sfbay.sun.com> <4C2473D2.3050606@Oracle.COM> <4C29175F.6020404@oracle.com> <20100628222758.GE13125@oracle.com> <4C2A4628.8040908@oracle.com> User-Agent: Mutt/1.5.20 (2010-03-02) Status: RO Content-Length: 3486 On Tue, Jun 29, 2010 at 12:14:48PM -0700, Ted H. Kim wrote: > >For the umad devices it seems to me that what you want is either to use > >one of PRIV_SYS_DL_CONFIG (DL == datalink) or PRIV_SYS_NET_CONFIG (which > >aggregates several config privs). Alternatively you could add a new > >privilege called, say, PRIV_SYS_IB_CONFIG and update PRIV_SYS_NET_CONFIG > >to also aggregate it. > > Okay, I see your point. > I think PRIV_SYS_NET_CONFIG is the more analogous one here. OK. > >Adding a privilege is not a big deal [...] > > The thing about not adding IB specific items into the admin > model is mostly about "least surprise". Despite our view about > IB, many admins don't think about IB explicitly. We are trying > to avoid unpleasant IB surprises if an admin has his system > reasonably configured for "regular" networking and hasn't > thought about IB directly. But what makes IB special? Why not just never add new privileges, never add new RBAC profiles, on account of "least surprise"? (Don't answer yet; read on.) Quite separately from that, you need to understand how RBAC works. If you grant a sysadmin access to the root role, then that's all they'll ever need to admin IB -- any extra privileges or RBAC profiles that you add will be completely irrelevant to that sysadmin's life, even if they admin IB. Similarly, if you grant the 'Primary Administrator' profile to a sysadmin they'll then be able to do everything they need to, including IB administration. We add new privileges and/or new RBAC profiles not to complicate the lives of sysadmins, nor your (the i-team's) life. We add them to make it possible to grant processes and users less-than-all-privilege. And some customers really do make use of the ability to grant less-than-full-root to their users. > >>The two cases are separate in terms of the underlying > >>mechanisms affected. The port restriction has to do with > >>connection traffic. The Q_Key has to do with datagrams. > >>However, they do have a similar theme w.r.t. being > >>able to get privileged access/traffic. > > > >They seem roughly comparable, but keep in mind that with TCP/UDP ports > >the meaning of a port < 1024 is really up to the acceptor, whereas the > >meaning of controlled Q_Keys may well be less flexible (I don't know). > >If the peer has no choice as to whether to accept such things then I'd > >say that you need a new privilege, else I'd say that PRIV_NET_PRIVADDR > >is fine. > > The receiver of a message with a certain Q_Key can always decide to > scrutinize it whatever way it wants and/or toss it out. Sure, but do implementations typically provide ways to say "don't accept Q_keys from node xyz"? If not, then there may be enough of a different relative to TCP/UDP ports < 1024... > But in light of the umad discussion above, I do think one further > refinement should be made. Two particular values of privileged Q_Key > (x80010000, x80010001) are used by "General Services" Management > Datagrams (MADs). (Note, even people familiar with IB may never > have heard of the second case. It is for an obscure service that the > industry will probably never implement - DevMgt v2 IORM & DA > communication.) If either of those values are obtained then certain > MADs could be spoofed. So if either of those values are requested, > then I think the same privilege used for umad (PRIV_SYS_NET_CONFIG) > should also be checked. If Q_Keys have configuration effects then I think that's OK, yes. Nico -- From ted.h.kim@oracle.com Tue Jun 29 15:24:05 2010 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o5TMO4M0017228 for ; Tue, 29 Jun 2010 15:24:05 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o5TMO4XF029234 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Tue, 29 Jun 2010 15:24:04 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L4S0060NRK4RN00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 29 Jun 2010 15:24:04 -0700 (PDT) Received: from sca-ea-mail-1.sun.com ([192.18.43.24]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L4S000OYRK3VT90@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 29 Jun 2010 15:24:03 -0700 (PDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by sca-ea-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o5TMO2GU013894 for ; Tue, 29 Jun 2010 22:24:03 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o5TKlgJ4031885 for ; Tue, 29 Jun 2010 22:23:59 +0000 (GMT) Received: from abhmt019.oracle.com by acsmt354.oracle.com with ESMTP id 384084831277850182; Tue, 29 Jun 2010 15:23:02 -0700 Received: from [129.153.3.74] (/129.153.3.74) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 29 Jun 2010 15:23:01 -0700 Date: Tue, 29 Jun 2010 15:21:33 -0700 From: "Ted H. Kim" Subject: Re: OFUV Userland Interface [PSARC/2010/239 FastTrack timeout 07/02/2010] In-reply-to: <20100629194450.GV11785@oracle.com> To: Nicolas Williams Cc: Darren J Moffat , PSARC-ext@sun.com Message-id: <4C2A71ED.40506@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090203.4C2A7282.010D:SCFMA4539814,ss=1,fgs=0 References: <201006250033.o5P0XqjA006108@sac.sfbay.sun.com> <4C2473D2.3050606@Oracle.COM> <4C29175F.6020404@oracle.com> <20100628222758.GE13125@oracle.com> <4C2A4628.8040908@oracle.com> <20100629194450.GV11785@oracle.com> User-Agent: Thunderbird 2.0.0.24 (X11/20100302) Status: RO Content-Length: 3806 Nicolas Williams wrote: >>> Adding a privilege is not a big deal [...] >> >> The thing about not adding IB specific items into the admin >> model is mostly about "least surprise". Despite our view about >> IB, many admins don't think about IB explicitly. We are trying >> to avoid unpleasant IB surprises if an admin has his system >> reasonably configured for "regular" networking and hasn't >> thought about IB directly. > > But what makes IB special? Why not just never add new privileges, never > add new RBAC profiles, on account of "least surprise"? (Don't answer > yet; read on.) In principle, IB is like everything else. So it's not "never" add anything, but we perhaps have a higher level of trying to reuse things, except when it's a bad fit to existing stuff. The why goes like this ... We have learned that adding too much IB specific admin becomes a hindrance to adoption, just because it's different or people don't want to bother with knowing more. But at the same time, people get outraged, if ignoring IB causes some hole in your security. So of course it is a balance of all these tensions. Also, people can legitimately disagree about how to weigh these factors. So I guess it's going to be a judgment call about whether these IB items are different enough to need new privileges or not. > Quite separately from that, you need to understand how RBAC works. If > you grant a sysadmin access to the root role, then that's all they'll > ever need to admin IB -- any extra privileges or RBAC profiles that you > add will be completely irrelevant to that sysadmin's life, even if they > admin IB. Similarly, if you grant the 'Primary Administrator' profile > to a sysadmin they'll then be able to do everything they need to, > including IB administration. > > We add new privileges and/or new RBAC profiles not to complicate the > lives of sysadmins, nor your (the i-team's) life. We add them to make > it possible to grant processes and users less-than-all-privilege. > > And some customers really do make use of the ability to grant > less-than-full-root to their users. Okay I think I see where you are coming from. Granular permissions for folks that use them, but the right think happens even for the old fashion root/non-root case. >>>> The two cases are separate in terms of the underlying >>>> mechanisms affected. The port restriction has to do with >>>> connection traffic. The Q_Key has to do with datagrams. >>>> However, they do have a similar theme w.r.t. being >>>> able to get privileged access/traffic. >>> >>> They seem roughly comparable, but keep in mind that with TCP/UDP ports >>> the meaning of a port < 1024 is really up to the acceptor, whereas the >>> meaning of controlled Q_Keys may well be less flexible (I don't know). >>> If the peer has no choice as to whether to accept such things then I'd >>> say that you need a new privilege, else I'd say that PRIV_NET_PRIVADDR >>> is fine. >> >> The receiver of a message with a certain Q_Key can always decide to >> scrutinize it whatever way it wants and/or toss it out. > > Sure, but do implementations typically provide ways to say "don't accept > Q_keys from node xyz"? If not, then there may be enough of a different > relative to TCP/UDP ports < 1024... No not typically, but it is app dependent. So would your answer here is we should have a new PRIV_IB_PRIVQKEY? Or are you saying something different? If so, what is the usual interface taxonomy classification for a new privilege? What else is usually done with a new privilege? (man pages, exec_attr/prof_attr changes, etc?) Thanks, -ted -- Ted H. Kim, PhD Sun/Oracle America Inc. ted.h.kim@oracle.com 222 North Sepulveda Blvd., 10th Floor (310) 341-1116 El Segundo, CA 90245 (310) 341-1115 FAX From Nicolas.Williams@oracle.com Tue Jun 29 15:38:59 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o5TMcxCA017312 for ; Tue, 29 Jun 2010 15:38:59 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o5TMcwF1020496 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Tue, 29 Jun 2010 16:38:58 -0600 (MDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L4S00603S8Y3M00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 29 Jun 2010 16:38:58 -0600 (MDT) Received: from sca-ea-mail-3.sun.com ([192.18.43.21]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L4S008KNS8X9VE0@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 29 Jun 2010 16:38:58 -0600 (MDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by sca-ea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o5TMcvKi020169 for ; Tue, 29 Jun 2010 22:38:57 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o5TKlhPH031899 for ; Tue, 29 Jun 2010 22:38:57 +0000 (GMT) Received: from abhmt010.oracle.com by acsmt354.oracle.com with ESMTP id 384122151277851130; Tue, 29 Jun 2010 15:38:50 -0700 Received: from oracle.com (/129.153.128.104) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 29 Jun 2010 15:38:48 -0700 Date: Tue, 29 Jun 2010 17:40:07 -0500 From: Nicolas Williams Subject: Re: OFUV Userland Interface [PSARC/2010/239 FastTrack timeout 07/02/2010] In-reply-to: <4C2A71ED.40506@oracle.com> To: "Ted H. Kim" Cc: Darren J Moffat , PSARC-ext@sun.com Message-id: <20100629224007.GC11785@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090203.4C2A7601.0095:SCFMA4539814,ss=1,fgs=0 References: <201006250033.o5P0XqjA006108@sac.sfbay.sun.com> <4C2473D2.3050606@Oracle.COM> <4C29175F.6020404@oracle.com> <20100628222758.GE13125@oracle.com> <4C2A4628.8040908@oracle.com> <20100629194450.GV11785@oracle.com> <4C2A71ED.40506@oracle.com> User-Agent: Mutt/1.5.20 (2010-03-02) Status: RO Content-Length: 2103 On Tue, Jun 29, 2010 at 03:21:33PM -0700, Ted H. Kim wrote: > Nicolas Williams wrote: > >We add new privileges and/or new RBAC profiles not to complicate the > >lives of sysadmins, nor your (the i-team's) life. We add them to make > >it possible to grant processes and users less-than-all-privilege. > > Okay I think I see where you are coming from. > Granular permissions for folks that use them, but the > right think happens even for the old fashion root/non-root case. Exactly :) We're talking about extra work for the i-team, not extra work for the user. But the extra work actually enables something that some customers want and use. > >>The receiver of a message with a certain Q_Key can always decide to > >>scrutinize it whatever way it wants and/or toss it out. > > > >Sure, but do implementations typically provide ways to say "don't accept > >Q_keys from node xyz"? If not, then there may be enough of a different > >relative to TCP/UDP ports < 1024... > > No not typically, but it is app dependent. > > So would your answer here is we should have a new PRIV_IB_PRIVQKEY? Possibly. I'm not an expert, so if you assert it's close enough to the TCP/UDP privileged port thing then I think PRIV_NET_PRIVADDR is good enough. > What else is usually done with a new privilege? (man pages, > exec_attr/prof_attr changes, etc?) Typically you'll want to create exec_attr entries and possibly create and/or modify prof_attr entries, to allow sysadmins to grant others the right to execute IB programs with the new privilege. Look for existing profiles with "Admin" or "Manage" in their names, such as: Apache 22 Administration CUPS Administration ISCSI Target Administration MySQL Administration Primary Administrator ... ZFS File System Management ZFS Storage Management ... What kinds of tasks are there with IB? I could see, perhaps: - admin tasks - tasks requiring Q_keys but which are not admin tasks - tasks requiring no non-basic privileges If that's right then you'd want to add two profiles. (Tasks requiring no non-basic privileges require no RBAC profiles.) Nico -- From ted.h.kim@oracle.com Tue Jun 29 17:40:56 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o5U0etxt020461 for ; Tue, 29 Jun 2010 17:40:55 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o5U0eshS016330 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Tue, 29 Jun 2010 19:40:55 -0500 (CDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L4S00F0BXW7ES00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 29 Jun 2010 17:40:55 -0700 (PDT) Received: from sca-ea-mail-1.sun.com ([192.18.43.24]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L4S009KIXW6NJ20@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 29 Jun 2010 17:40:54 -0700 (PDT) Received: from rcsinet15.oracle.com (rcsinet15.oracle.com [148.87.113.117]) by sca-ea-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o5U0esNJ020413 for ; Wed, 30 Jun 2010 00:40:54 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by rcsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o5U0eku5012680 for ; Wed, 30 Jun 2010 00:40:46 +0000 (GMT) Received: from abhmt020.oracle.com by acsmt353.oracle.com with ESMTP id 367919831277858377; Tue, 29 Jun 2010 17:39:37 -0700 Received: from [129.153.3.74] (/129.153.3.74) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 29 Jun 2010 17:39:37 -0700 Date: Tue, 29 Jun 2010 17:38:08 -0700 From: "Ted H. Kim" Subject: Re: OFUV Userland Interface [PSARC/2010/239 FastTrack timeout 07/02/2010] In-reply-to: <20100629224007.GC11785@oracle.com> To: Nicolas Williams Cc: Darren J Moffat , PSARC-ext@sun.com Message-id: <4C2A91F0.1040805@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090207.4C2A9296.0027:SCFMA4539814,ss=1,fgs=0 References: <201006250033.o5P0XqjA006108@sac.sfbay.sun.com> <4C2473D2.3050606@Oracle.COM> <4C29175F.6020404@oracle.com> <20100628222758.GE13125@oracle.com> <4C2A4628.8040908@oracle.com> <20100629194450.GV11785@oracle.com> <4C2A71ED.40506@oracle.com> <20100629224007.GC11785@oracle.com> User-Agent: Thunderbird 2.0.0.24 (X11/20100302) Status: RO Content-Length: 2490 Nicolas Williams wrote: >>>> The receiver of a message with a certain Q_Key can always decide to >>>> scrutinize it whatever way it wants and/or toss it out. >>> >>> Sure, but do implementations typically provide ways to say "don't accept >>> Q_keys from node xyz"? If not, then there may be enough of a different >>> relative to TCP/UDP ports < 1024... >> No not typically, but it is app dependent. >> >> So would your answer here is we should have a new PRIV_IB_PRIVQKEY? > > Possibly. I'm not an expert, so if you assert it's close enough to the > TCP/UDP privileged port thing then I think PRIV_NET_PRIVADDR is good > enough. I will confer a bit more with the project team. But we might say PRIV_NET_PRIVADDR is good enough. >> What else is usually done with a new privilege? (man pages, >> exec_attr/prof_attr changes, etc?) > > Typically you'll want to create exec_attr entries and possibly create > and/or modify prof_attr entries, to allow sysadmins to grant others the > right to execute IB programs with the new privilege. > > Look for existing profiles with "Admin" or "Manage" in their names, such > as: > > Apache 22 Administration > CUPS Administration > ISCSI Target Administration > MySQL Administration > Primary Administrator > ... > ZFS File System Management > ZFS Storage Management > ... > > What kinds of tasks are there with IB? I could see, perhaps: > > - admin tasks > - tasks requiring Q_keys but which are not admin tasks > - tasks requiring no non-basic privileges > > If that's right then you'd want to add two profiles. (Tasks requiring > no non-basic privileges require no RBAC profiles.) So for all the cases we have discussed (umad using PRIV_SYS_NET_CONFIG, reserved ports using PRIV_NET_PRIVADDR, plus what is decided for privileged Q_Keys), I guess the companion userland library/utilities/commands case should give the exact exec_attr and prof_attr changes, right? So I think we have admin tasks and no non-basic privileges. I am hard pressed to think of a real userland app using privileged Q_Keys but not using umad (and being an admin thing). (The kernel does have such a scenario, but we are talking about OFUV which is meant to deliver IB to userland.) Should we still include a profile for this theoretical future situation? -ted -- Ted H. Kim, PhD Sun/Oracle America Inc. ted.h.kim@oracle.com 222 North Sepulveda Blvd., 10th Floor (310) 341-1116 El Segundo, CA 90245 (310) 341-1115 FAX From ted.h.kim@oracle.com Wed Jun 30 15:50:18 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o5UMoILM014105 for ; Wed, 30 Jun 2010 15:50:18 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o5UMoGES029897 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 30 Jun 2010 17:50:18 -0500 (CDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L4U00H0JNFT0300@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 30 Jun 2010 15:50:17 -0700 (PDT) Received: from sca-ea-mail-3.sun.com ([192.18.43.21]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L4U00A4KNFS6XC0@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 30 Jun 2010 15:50:16 -0700 (PDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by sca-ea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o5UMoG7X024916 for ; Wed, 30 Jun 2010 22:50:16 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o5UMdc90019186 for ; Wed, 30 Jun 2010 22:50:14 +0000 (GMT) Received: from abhmt007.oracle.com by acsmt353.oracle.com with ESMTP id 371690661277938110; Wed, 30 Jun 2010 15:48:30 -0700 Received: from [129.153.3.74] (/129.153.3.74) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 30 Jun 2010 15:48:30 -0700 Date: Wed, 30 Jun 2010 15:47:02 -0700 From: "Ted H. Kim" Subject: Re: OFUV Userland Interface [PSARC/2010/239 FastTrack timeout 07/02/2010] In-reply-to: <4C2A91F0.1040805@oracle.com> To: Nicolas Williams Cc: Darren J Moffat , PSARC-ext@sun.com Message-id: <4C2BC966.2040600@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090204.4C2BCA27.0141:SCFMA4539814,ss=1,fgs=0 References: <201006250033.o5P0XqjA006108@sac.sfbay.sun.com> <4C2473D2.3050606@Oracle.COM> <4C29175F.6020404@oracle.com> <20100628222758.GE13125@oracle.com> <4C2A4628.8040908@oracle.com> <20100629194450.GV11785@oracle.com> <4C2A71ED.40506@oracle.com> <20100629224007.GC11785@oracle.com> <4C2A91F0.1040805@oracle.com> User-Agent: Thunderbird 2.0.0.24 (X11/20100302) Status: RO Content-Length: 697 Nicholas, I think this is where things are ending up. umad to use protection 666 and PRIV_SYS_NET_CONFIG ucma reserved ports uses PRIV_NET_PRIVADDR uverbs privileged Q_Keys uses PRIV_NET_PRIVADDR except for the ones which might be used to spoof MADs (x80010000, x80010001) -- those will require PRIV_SYS_NET_CONFIG too then when the case for userland libraries/commands/utilities comes along, that case will describe the exec_attr/prof_attr changes too Have I stated things correctly? -ted -- Ted H. Kim, PhD Sun/Oracle America Inc. ted.h.kim@oracle.com 222 North Sepulveda Blvd., 10th Floor (310) 341-1116 El Segundo, CA 90245 (310) 341-1115 FAX From Nicolas.Williams@Oracle.COM Wed Jun 30 15:52:46 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o5UMqkEE014144 for ; Wed, 30 Jun 2010 15:52:46 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o5UMqjIQ000796 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 30 Jun 2010 17:52:46 -0500 (CDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L4U00007NJXZQ00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 30 Jun 2010 15:52:45 -0700 (PDT) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L4U00FH9NJWPXC0@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 30 Jun 2010 15:52:44 -0700 (PDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o5UMqhd8017446 for ; Wed, 30 Jun 2010 22:52:43 +0000 (GMT) Received: from acsmt353.oracle.com (acsmt353.oracle.com [141.146.40.153]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o5UJ9EW6016129 for ; Wed, 30 Jun 2010 22:52:43 +0000 (GMT) Received: from abhmt013.oracle.com by acsmt353.oracle.com with ESMTP id 386999811277938300; Wed, 30 Jun 2010 15:51:40 -0700 Received: from oracle.com (/129.153.128.104) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 30 Jun 2010 15:51:39 -0700 Date: Wed, 30 Jun 2010 17:53:47 -0500 From: Nicolas Williams Subject: Re: OFUV Userland Interface [PSARC/2010/239 FastTrack timeout 07/02/2010] In-reply-to: <4C2BC966.2040600@oracle.com> To: "Ted H. Kim" Cc: Darren J Moffat , PSARC-ext@sun.com Message-id: <20100630225347.GE14528@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt353.oracle.com [141.146.40.153] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090203.4C2BCABB.0096:SCFMA4539814,ss=1,fgs=0 References: <201006250033.o5P0XqjA006108@sac.sfbay.sun.com> <4C2473D2.3050606@Oracle.COM> <4C29175F.6020404@oracle.com> <20100628222758.GE13125@oracle.com> <4C2A4628.8040908@oracle.com> <20100629194450.GV11785@oracle.com> <4C2A71ED.40506@oracle.com> <20100629224007.GC11785@oracle.com> <4C2A91F0.1040805@oracle.com> <4C2BC966.2040600@oracle.com> User-Agent: Mutt/1.5.20 (2010-03-02) Status: RO Content-Length: 760 On Wed, Jun 30, 2010 at 03:47:02PM -0700, Ted H. Kim wrote: > I think this is where things are ending up. > > umad to use protection 666 and PRIV_SYS_NET_CONFIG > ucma reserved ports uses PRIV_NET_PRIVADDR > uverbs privileged Q_Keys uses PRIV_NET_PRIVADDR except for > the ones which might be used to spoof MADs (x80010000, x80010001) -- > those will require PRIV_SYS_NET_CONFIG too Sounds reasonable to me. > then when the case for userland libraries/commands/utilities > comes along, that case will describe the exec_attr/prof_attr > changes too Are you adding new profiles or adding to existing profiles? (or both?) If you're adding profiles you should list them. (Note: I'm not an ARC member, so I can't give you the +1 that you need.) Nico -- From ted.h.kim@Oracle.COM Wed Jun 30 17:35:50 2010 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o610ZoO6016476 for ; Wed, 30 Jun 2010 17:35:50 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o610Znep016049 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 30 Jun 2010 17:35:49 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L4U0080HSBPS100@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 30 Jun 2010 18:35:49 -0600 (MDT) Received: from sca-ea-mail-1.sun.com ([192.18.43.24]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L4U0004TSBP37E0@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 30 Jun 2010 18:35:49 -0600 (MDT) Received: from rcsinet13.oracle.com (rcsinet13.oracle.com [148.87.113.125]) by sca-ea-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o610ZmFE024888 for ; Thu, 01 Jul 2010 00:35:48 +0000 (GMT) Received: from acsmt353.oracle.com (acsmt353.oracle.com [141.146.40.153]) by rcsinet13.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o610XMIV001259 for ; Thu, 01 Jul 2010 00:35:46 +0000 (GMT) Received: from abhmt010.oracle.com by acsmt353.oracle.com with ESMTP id 387158211277944537; Wed, 30 Jun 2010 17:35:37 -0700 Received: from [129.153.3.74] (/129.153.3.74) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 30 Jun 2010 17:35:36 -0700 Date: Wed, 30 Jun 2010 17:34:06 -0700 From: "Ted H. Kim" Subject: Re: OFUV Userland Interface [PSARC/2010/239 FastTrack timeout 07/02/2010] In-reply-to: <20100630225347.GE14528@oracle.com> To: Nicolas Williams Cc: Darren J Moffat , PSARC-ext@sun.com Message-id: <4C2BE27E.9050804@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt353.oracle.com [141.146.40.153] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090207.4C2BE2E4.00A8:SCFMA4539814,ss=1,fgs=0 References: <201006250033.o5P0XqjA006108@sac.sfbay.sun.com> <4C2473D2.3050606@Oracle.COM> <4C29175F.6020404@oracle.com> <20100628222758.GE13125@oracle.com> <4C2A4628.8040908@oracle.com> <20100629194450.GV11785@oracle.com> <4C2A71ED.40506@oracle.com> <20100629224007.GC11785@oracle.com> <4C2A91F0.1040805@oracle.com> <4C2BC966.2040600@oracle.com> <20100630225347.GE14528@oracle.com> User-Agent: Thunderbird 2.0.0.24 (X11/20100302) Status: RO Content-Length: 923 Nicolas Williams wrote: >> then when the case for userland libraries/commands/utilities >> comes along, that case will describe the exec_attr/prof_attr >> changes too > > Are you adding new profiles or adding to existing profiles? (or both?) > If you're adding profiles you should list them. I think we are adding to the Network Management profile (in exec_attr). That's where I think the umad using commands end up. The userland situations for privileged Q_Key and reserved port usage are theoretical at this point, and so I don't think warrant changes for those right now. > (Note: I'm not an ARC member, so I can't give you the +1 that you need.) > > Nico Okay, I hope ARC folks are actually following along. -ted -- Ted H. Kim, PhD Sun/Oracle America Inc. ted.h.kim@oracle.com 222 North Sepulveda Blvd., 10th Floor (310) 341-1116 El Segundo, CA 90245 (310) 341-1115 FAX From Nicolas.Williams@oracle.com Wed Jun 30 18:47:42 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o611lfu3017383 for ; Wed, 30 Jun 2010 18:47:42 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o611lfsZ015341 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 30 Jun 2010 20:47:41 -0500 (CDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L4U00H01VNHNO00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 30 Jun 2010 18:47:41 -0700 (PDT) Received: from sca-ea-mail-3.sun.com ([192.18.43.21]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L4U00DY0VNGSA90@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 30 Jun 2010 18:47:40 -0700 (PDT) Received: from rcsinet13.oracle.com (rcsinet13.oracle.com [148.87.113.125]) by sca-ea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o611lejC010704 for ; Thu, 01 Jul 2010 01:47:40 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by rcsinet13.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o5UJFc2F026468 for ; Thu, 01 Jul 2010 01:47:39 +0000 (GMT) Received: from abhmt020.oracle.com by acsmt354.oracle.com with ESMTP id 372097481277948743; Wed, 30 Jun 2010 18:45:43 -0700 Received: from oracle.com (/129.153.128.104) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 30 Jun 2010 18:45:40 -0700 Date: Wed, 30 Jun 2010 20:47:54 -0500 From: Nicolas Williams Subject: Re: OFUV Userland Interface [PSARC/2010/239 FastTrack timeout 07/02/2010] In-reply-to: <4C2BE27E.9050804@oracle.com> To: "Ted H. Kim" Cc: Darren J Moffat , PSARC-ext@sun.com Message-id: <20100701014753.GG14528@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090206.4C2BF3BB.0254:SCFMA4539814,ss=1,fgs=0 References: <4C29175F.6020404@oracle.com> <20100628222758.GE13125@oracle.com> <4C2A4628.8040908@oracle.com> <20100629194450.GV11785@oracle.com> <4C2A71ED.40506@oracle.com> <20100629224007.GC11785@oracle.com> <4C2A91F0.1040805@oracle.com> <4C2BC966.2040600@oracle.com> <20100630225347.GE14528@oracle.com> <4C2BE27E.9050804@oracle.com> User-Agent: Mutt/1.5.20 (2010-03-02) Status: RO Content-Length: 806 On Wed, Jun 30, 2010 at 05:34:06PM -0700, Ted H. Kim wrote: > Nicolas Williams wrote: > >>then when the case for userland libraries/commands/utilities > >>comes along, that case will describe the exec_attr/prof_attr > >>changes too > > > >Are you adding new profiles or adding to existing profiles? (or both?) > >If you're adding profiles you should list them. > > I think we are adding to the Network Management profile > (in exec_attr). That's where I think the umad using commands end up. OK. > The userland situations for privileged Q_Key and reserved port usage > are theoretical at this point, and so I don't think warrant > changes for those right now. Unless you have tools like ping or rlogin that non-root users should nonetheless get to execute with privilege, then you're right. Nico -- From Darren.Moffat@Oracle.COM Thu Jul 1 01:58:52 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o618wqOw020587 for ; Thu, 1 Jul 2010 01:58:52 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o618wo6Z009564 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Thu, 1 Jul 2010 02:58:51 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L4V00I0HFM2H400@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 01 Jul 2010 01:58:50 -0700 (PDT) Received: from sca-ea-mail-1.sun.com ([192.18.43.24]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L4V0082HFM26TB0@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 01 Jul 2010 01:58:50 -0700 (PDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by sca-ea-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o618wos5003127 for ; Thu, 01 Jul 2010 08:58:50 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o617cEZG020133 for ; Thu, 01 Jul 2010 08:58:48 +0000 (GMT) Received: from abhmt016.oracle.com by acsmt353.oracle.com with ESMTP id 388002011277974727; Thu, 01 Jul 2010 01:58:47 -0700 Received: from [10.7.251.221] (/10.7.251.221) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 01 Jul 2010 01:58:47 -0700 Date: Thu, 01 Jul 2010 09:58:44 +0100 From: Darren J Moffat Subject: Re: OFUV Userland Interface [PSARC/2010/239 FastTrack timeout 07/02/2010] In-reply-to: <4C2BC966.2040600@oracle.com> To: "Ted H. Kim" Cc: Nicolas Williams , PSARC-ext@sun.com Message-id: <4C2C58C4.3000304@Oracle.COM> Organization: Oracle Solaris Security MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090206.4C2C58C9.01ED:SCFMA4539814,ss=1,fgs=0 References: <201006250033.o5P0XqjA006108@sac.sfbay.sun.com> <4C2473D2.3050606@Oracle.COM> <4C29175F.6020404@oracle.com> <20100628222758.GE13125@oracle.com> <4C2A4628.8040908@oracle.com> <20100629194450.GV11785@oracle.com> <4C2A71ED.40506@oracle.com> <20100629224007.GC11785@oracle.com> <4C2A91F0.1040805@oracle.com> <4C2BC966.2040600@oracle.com> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.9) Gecko/20100607 Lightning/1.0b1 OracleBeehiveExtension/1.0.0.0pre3 Thunderbird/3.0.4 Status: RO Content-Length: 738 On 30/06/2010 23:47, Ted H. Kim wrote: > I think this is where things are ending up. > > umad to use protection 666 and PRIV_SYS_NET_CONFIG > ucma reserved ports uses PRIV_NET_PRIVADDR > uverbs privileged Q_Keys uses PRIV_NET_PRIVADDR except for > the ones which might be used to spoof MADs (x80010000, x80010001) -- > those will require PRIV_SYS_NET_CONFIG too > > then when the case for userland libraries/commands/utilities > comes along, that case will describe the exec_attr/prof_attr > changes too These changes address the concerns I raised originally. With that I have no other security related issues with this case, however I don't feel I know IB itself well enough to give the +1 on the case generally. -- Darren J Moffat From gary.winiger@oracle.com Thu Jul 1 07:42:16 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o61EgFFD025784 for ; Thu, 1 Jul 2010 07:42:15 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o61EgFf2010425 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Thu, 1 Jul 2010 08:42:15 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L4V00J09VIFKP00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 01 Jul 2010 07:42:15 -0700 (PDT) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L4V00IZ6VIEWV00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 01 Jul 2010 07:42:14 -0700 (PDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o61EgEql013387 for ; Thu, 01 Jul 2010 14:42:14 +0000 (GMT) Received: from acsmt353.oracle.com (acsmt353.oracle.com [141.146.40.153]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o5UHmbHa020622 for ; Thu, 01 Jul 2010 14:42:12 +0000 (GMT) Received: from abhmt021.oracle.com by acsmt354.oracle.com with ESMTP id 374153431277995276; Thu, 01 Jul 2010 07:41:16 -0700 Received: from gwws-macbook-pro.local (/10.7.251.252) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 01 Jul 2010 07:41:15 -0700 Date: Thu, 01 Jul 2010 07:41:24 -0700 From: Gary Winiger Subject: Re: OFUV Userland Interface [PSARC/2010/239 FastTrack timeout 07/02/2010] In-reply-to: <20100630225347.GE14528@oracle.com> To: Nicolas Williams Cc: "Ted H. Kim" , Darren J Moffat , PSARC-ext@sun.com Message-id: <4C2CA914.2070405@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt353.oracle.com [141.146.40.153] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090205.4C2CA945.0193:SCFMA4539814,ss=1,fgs=0 References: <201006250033.o5P0XqjA006108@sac.sfbay.sun.com> <4C2473D2.3050606@Oracle.COM> <4C29175F.6020404@oracle.com> <20100628222758.GE13125@oracle.com> <4C2A4628.8040908@oracle.com> <20100629194450.GV11785@oracle.com> <4C2A71ED.40506@oracle.com> <20100629224007.GC11785@oracle.com> <4C2A91F0.1040805@oracle.com> <4C2BC966.2040600@oracle.com> <20100630225347.GE14528@oracle.com> User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4 Status: RO Content-Length: 432 On 6/30/10 3:53 PM, Nicolas Williams wrote: > Are you adding new profiles or adding to existing profiles? (or both?) > If you're adding profiles you should list them. > > (Note: I'm not an ARC member, so I can't give you the +1 that you need.) It seems to me the project team has a new spec. It seems to me the case owner would post the new spec, set a new timer and the project would await a review of the new spec. Gary.. From garrett@damore.org Thu Jul 1 08:12:09 2010 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o61FC8o6026383 for ; Thu, 1 Jul 2010 08:12:09 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o61FC7ah001035 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Thu, 1 Jul 2010 08:12:08 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L4V00K1RWW8UQ00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 01 Jul 2010 08:12:08 -0700 (PDT) Received: from sca-ea-mail-2.sun.com ([192.18.43.25]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L4V00IARWW8WR20@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 01 Jul 2010 08:12:08 -0700 (PDT) Received: from relay41i.sun.com ([192.5.209.70]) by sca-ea-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o61FAsmx019487 for ; Thu, 01 Jul 2010 15:12:07 +0000 (GMT) Received: from mms49es.mms.us.syntegra.com ([160.41.221.232] [160.41.221.232]) by relay41i.sun.com with ESMTP id BT-MMP-179311 for PSARC-ext@sun.com; Thu, 01 Jul 2010 15:12:07 +0000 (Z) Received: from relay45i.sun.com (relay45i.sun.com [192.5.209.94]) by mms49es.mms.us.syntegra.com with ESMTP id BT-MMP-1947437 for PSARC-ext@sun.com; Thu, 01 Jul 2010 15:12:06 +0000 (Z) Received: from oproxy2-pub.bluehost.com ([67.222.39.60] [67.222.39.60]) by relay4i.sun.com id BT-MMP-7677651 for PSARC-ext@sun.com; Thu, 01 Jul 2010 15:12:06 +0000 (Z) Received: (qmail 12915 invoked by uid 0); Thu, 01 Jul 2010 15:12:05 +0000 Received: from unknown (HELO box374.bluehost.com) (69.89.31.174) by oproxy2.bluehost.com with SMTP; Thu, 01 Jul 2010 15:12:05 +0000 Received: from cpe-76-93-15-33.socal.res.rr.com ([76.93.15.33] helo=[192.168.251.110]) by box374.bluehost.com with esmtpsa (SSLv3:AES256-SHA:256) (Exim 4.69) (envelope-from ) id 1OULQr-0003Oi-N4; Thu, 01 Jul 2010 09:12:05 -0600 Date: Thu, 01 Jul 2010 08:14:43 -0700 From: "Garrett D'Amore" Subject: Re: OFUV Userland Interface [PSARC/2010/239 FastTrack timeout 07/02/2010] In-reply-to: <4C2C58C4.3000304@Oracle.COM> To: Darren J Moffat Cc: "Ted H. Kim" , PSARC-ext@sun.com Message-id: <1277997283.5596.4698.camel@velocity> MIME-version: 1.0 X-Mailer: Evolution 2.28.3 Content-type: text/plain; charset=UTF-8 Content-transfer-encoding: 7BIT DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=default; d=damore.org; h=Received:Subject:From:To:Cc:In-Reply-To:References:Content-Type:Date:Message-ID:Mime-Version:X-Mailer:Content-Transfer-Encoding:X-Identified-User; b=tCKvVg0xE4baNLbhKHKYwsTm9C/iys8SsO0830Y/2H+LP213alaFlIhejQwhNVPo85rmCspiluY7Y+DiHd6NN+r8k0Varhz68US6uJtYlZdKiM2QqTYhRJxVbMh060OK; X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-Identified-User: {2225:box374.bluehost.com:damoreor:damore.org} {sentby:smtp auth 76.93.15.33 authed with garrett+damore.org} X-Antispam: No, score=-0.7/5.0, scanned in 0.206sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <201006250033.o5P0XqjA006108@sac.sfbay.sun.com> <4C2473D2.3050606@Oracle.COM> <4C29175F.6020404@oracle.com> <20100628222758.GE13125@oracle.com> <4C2A4628.8040908@oracle.com> <20100629194450.GV11785@oracle.com> <4C2A71ED.40506@oracle.com> <20100629224007.GC11785@oracle.com> <4C2A91F0.1040805@oracle.com> <4C2BC966.2040600@oracle.com> <4C2C58C4.3000304@Oracle.COM> Status: RO Content-Length: 1437 On Thu, 2010-07-01 at 09:58 +0100, Darren J Moffat wrote: > On 30/06/2010 23:47, Ted H. Kim wrote: > > I think this is where things are ending up. > > > > umad to use protection 666 and PRIV_SYS_NET_CONFIG > > ucma reserved ports uses PRIV_NET_PRIVADDR > > uverbs privileged Q_Keys uses PRIV_NET_PRIVADDR except for > > the ones which might be used to spoof MADs (x80010000, x80010001) -- > > those will require PRIV_SYS_NET_CONFIG too > > > > then when the case for userland libraries/commands/utilities > > comes along, that case will describe the exec_attr/prof_attr > > changes too > > These changes address the concerns I raised originally. With that I have > no other security related issues with this case, however I don't feel I > know IB itself well enough to give the +1 on the case generally. This is actually kind of a problem... I don't think (but could be wrong) that any of the PSARC members here "really" understand IB. I have delved into it just a little, but with the specific context of doing the 10GbE mellanox driver. It would be good if someone from the IB team were to join as a PSARC intern, and work towards full membership. Meanwhile, I have not followed the privilege discussion on this case that closely, but assuming that that issue is resolved, I think I can give a +1 to the *rest* of the case. For the most part, its architecture is already determined by the extant Linux OFUV API. - Garrett From ted.h.kim@oracle.com Fri Jul 2 13:10:59 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o62KAxC0021206 for ; Fri, 2 Jul 2010 13:10:59 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o62KAwvN062465 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Fri, 2 Jul 2010 14:10:58 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L4Y00M015EATV00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 02 Jul 2010 13:10:58 -0700 (PDT) Received: from sca-ea-mail-1.sun.com ([192.18.43.24]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L4Y00C0I5EAWIC0@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 02 Jul 2010 13:10:58 -0700 (PDT) Received: from rcsinet15.oracle.com (rcsinet15.oracle.com [148.87.113.117]) by sca-ea-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o62KAvQx005836 for ; Fri, 02 Jul 2010 20:10:57 +0000 (GMT) Received: from acsmt355.oracle.com (acsmt355.oracle.com [141.146.40.155]) by rcsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o623IjNc001244 for ; Fri, 02 Jul 2010 20:10:57 +0000 (GMT) Received: from abhmt007.oracle.com by acsmt353.oracle.com with ESMTP id 378903471278101454; Fri, 02 Jul 2010 13:10:54 -0700 Received: from [129.153.3.74] (/129.153.3.74) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 02 Jul 2010 13:10:53 -0700 Date: Fri, 02 Jul 2010 13:09:20 -0700 From: "Ted H. Kim" Subject: Re: OFUV Userland Interface [PSARC/2010/239 FastTrack timeout 07/02/2010] In-reply-to: <4C2CA914.2070405@oracle.com> To: Gary Winiger Cc: Nicolas Williams , Darren J Moffat , PSARC-ext@sun.com Message-id: <4C2E4770.2020105@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt355.oracle.com [141.146.40.155] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090209.4C2E47D1.00F1:SCFMA4539814,ss=1,fgs=0 References: <201006250033.o5P0XqjA006108@sac.sfbay.sun.com> <4C2473D2.3050606@Oracle.COM> <4C29175F.6020404@oracle.com> <20100628222758.GE13125@oracle.com> <4C2A4628.8040908@oracle.com> <20100629194450.GV11785@oracle.com> <4C2A71ED.40506@oracle.com> <20100629224007.GC11785@oracle.com> <4C2A91F0.1040805@oracle.com> <4C2BC966.2040600@oracle.com> <20100630225347.GE14528@oracle.com> <4C2CA914.2070405@oracle.com> User-Agent: Thunderbird 2.0.0.24 (X11/20100302) Status: RO Content-Length: 861 I will re-post a corrected version of the proposal and extend the timer. Only the one section on security is revised, so I do hope people don't regard this as completely new. -ted Gary Winiger wrote: > On 6/30/10 3:53 PM, Nicolas Williams wrote: > >> Are you adding new profiles or adding to existing profiles? (or both?) >> If you're adding profiles you should list them. >> >> (Note: I'm not an ARC member, so I can't give you the +1 that you need.) > > It seems to me the project team has a new spec. It seems to me > the case owner would post the new spec, set a new timer and > the project would await a review of the new spec. > > Gary.. -- Ted H. Kim, PhD Sun/Oracle America Inc. ted.h.kim@oracle.com 222 North Sepulveda Blvd., 10th Floor (310) 341-1116 El Segundo, CA 90245 (310) 341-1115 FAX From ted.h.kim@oracle.com Fri Jul 2 18:32:48 2010 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o631WlUr001640 for ; Fri, 2 Jul 2010 18:32:47 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o62MmOFE011657 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Fri, 2 Jul 2010 15:48:24 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L4Y00K01COOJD00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 02 Jul 2010 15:48:24 -0700 (PDT) Received: from sca-ea-mail-1.sun.com ([192.18.43.24]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L4Y00AYGCOO23D0@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 02 Jul 2010 15:48:24 -0700 (PDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by sca-ea-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o62MmNME015067 for ; Fri, 02 Jul 2010 22:48:24 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o62HmPR1012687 for ; Fri, 02 Jul 2010 22:48:22 +0000 (GMT) Received: from abhmt013.oracle.com by acsmt354.oracle.com with ESMTP id 393282421278110853; Fri, 02 Jul 2010 15:47:33 -0700 Received: from [129.153.3.74] (/129.153.3.74) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 02 Jul 2010 15:47:33 -0700 Date: Fri, 02 Jul 2010 15:45:53 -0700 From: "Ted H. Kim" Subject: Re: OFUV Userland Interface [PSARC/2010/239 FastTrack timeout 07/09/2010] In-reply-to: <4C2E4770.2020105@oracle.com> To: PSARC-ext@sun.com Message-id: <4C2E6C21.30305@oracle.com> MIME-version: 1.0 Content-type: multipart/mixed; boundary="Boundary_(ID_C2cpAbYcE6qyN9/EAjpg9Q)" X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090208.4C2E6CB7.00FF:SCFMA4539814,ss=1,fgs=0 References: <201006250033.o5P0XqjA006108@sac.sfbay.sun.com> <4C2473D2.3050606@Oracle.COM> <4C29175F.6020404@oracle.com> <20100628222758.GE13125@oracle.com> <4C2A4628.8040908@oracle.com> <20100629194450.GV11785@oracle.com> <4C2A71ED.40506@oracle.com> <20100629224007.GC11785@oracle.com> <4C2A91F0.1040805@oracle.com> <4C2BC966.2040600@oracle.com> <20100630225347.GE14528@oracle.com> <4C2CA914.2070405@oracle.com> <4C2E4770.2020105@oracle.com> User-Agent: Thunderbird 2.0.0.24 (X11/20100302) Status: RO Content-Length: 18003 This is a multi-part message in MIME format. --Boundary_(ID_C2cpAbYcE6qyN9/EAjpg9Q) Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT Folks, Here is the revised proposal with the security section edited. I will post diffs shortly. For some reason I couldn't get into the ARC site right now, but the subject reflects an extended timer. -ted -- Ted H. Kim, PhD Sun/Oracle America Inc. ted.h.kim@oracle.com 222 North Sepulveda Blvd., 10th Floor (310) 341-1116 El Segundo, CA 90245 (310) 341-1115 FAX --Boundary_(ID_C2cpAbYcE6qyN9/EAjpg9Q) Content-type: text/plain; name=psarc-2010-239-case.txt Content-transfer-encoding: 7BIT Content-disposition: inline; filename=psarc-2010-239-case.txt Template Version: @(#)sac_nextcase 1.70 03/30/10 SMI This information is Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved. 1. Introduction 1.1. Project/Component Working Name: OFUV Userland Interface 1.2. Name of Document Author/Supplier: Author: Brendan Doyle 1.3 Date of This Document: 2 July, 2010 4. Technical Description Table of Contents ----------------- 4.1 Introduction 4.2 Proposal and Summary of Interfaces A. Device Node Entries B. User/Kernel Interfaces C. Config Files D. Security and Resource Control 4.3 Description of Interfaces A. Device Node Entries B. User/Kernel Interfaces 1. RDMA CM 2. Verbs 1. Verbs User/Kernel Interface 2. Asynchronous Event Interface 3. Completion Event Interface 3. umad Interface C. Config Files 4.1 Introduction ---------------- In Linux, the most popular InfiniBand (IB) OS-bypass framework is the Open Fabrics User Verbs (OFUV) framework from the Open Fabrics Enterprise Distribution (OFED). In Solaris, OFUV is being ported over in two parts: kernel and userland. The main body of the kernel part is described in PSARC/2009/421, which describes the APIs used by kernel consumers of OFUV. This case describes the userland interface (including OS-bypass) to OFUV. The complete userland interface includes userland-facing interfaces of components originally described in PSARC/2009/421 and one new kernel module (sol_umad), which includes the interfaces used by the ported OFED libumad library to the Solaris kernel IBMF module. The actual userland portion of OFUV is a companion project which delivers the open source libraries and utilities. The main business objective is enabling the OFED libraries and utilities to satisfy some of the requirements of Oracle appliance platforms and other consumers, such as financial messaging systems and MPI. Reference Documents: o Open Fabrics User Verbs (OFUV) - primary kernel components PSARC/2009/421 one-pager http://sac.sfbay/PSARC/2009/421/20090731_brendan.doyle o Kernel RDMA CM API Architecture and use: materials directory: ofuv_rdma_arch.txt o Solaris Open Fabrics User Verbs Architecture Document: materials directory: solaris_ofuv_arch.pdf o OFUV Implementation Details: materials directory: OFUVImplementationDetails.pdf Related ARC cases: PSARC/2002/132 IBTF: InfiniBand Transport Framework PSARC/2002/607 InfiniBand Management Framework (IBMF) PSARC/2003/145 uDAPL: User Direct Access Programming Library PSARC/2009/421 Open Fabrics User Verbs (OFUV) - primary kernel components 4.2 Proposal and Summary of Interfaces -------------------------------------- This case proposes a userland to kernel interface for OFUV. The interface taxonomy classification for this interface is Volatile. The rationale for this choice is the fact that the interface is externally specified in OFED. And as Volatile has absorbed the former External classification in the current taxonomy, we use that. This initial version of OFUV corresponds to OFED-1.3.1. Later phases of the project will update this to the current version (which as of this writing is OFED-1.5.1). A micro/patch binding is asserted for this interface. A. /devices entries o '/devices/ib/sol_uverbs' o '/devices/ib/sol_uverbs@0:uverbs#' o '/devices/ib/sol_uverbs@0:ucma' # - minor number o '/devices/pseudo/sol_ucma@0' o '/devices/pseudo/sol_ucma@0:sol_ucma' o '/devices/ib/sol_umad@0' o '/devices/ib/sol_umad@0:issm# o '/devices/ib/sol_umad@0:umad# # - minor number The umad device files have permission 600 and are owned by root. All other device files have permission 666 and are owned by root. See section 4.2.D. Security and Resource Control for further discussion. B. User/kernel Interfaces These interfaces are only used by the respective userland library components to access to the kernel components of this project. B.1 RDMA CM - 'librdmacm' library to 'sol_ucma' kernel module B.2 Verbs - 'libibverbs' library to 'sol_uverbs' kernel module B.3 umad - 'libibumad' library to 'sol_umad' kernel module C. Config Files /kernel/drv/sol_uverbs.conf /kernel/drv/sol_ucma.conf /kernel/drv/sol_umad.conf D. Security and Resource Control The sol_umad minor nodes are owned by root and have 666 file permissions. Operations on these nodes require the PRIV_SYS_NET_CONFIG privilege. We restrict this because the Management Datagram (MAD) based tools/utilities allow users to reconfigure the IB fabric and nodes on the fabric. When commands and utilities using umad are proposed in the companion ARC case, changes to exec_attr(5) for these binaries will also be described. The sol_uverbs and sol_ucma minor nodes are owned by root and have 666 file permissions. All users can run the utilities that use these files as a means of access to the kernel IBTF. These utilities are not intrusive and do not allow configuration of the IB fabric or fabric nodes. However protection is afforded against non-root users use of privileged Q_Keys and use of reserved well-known ports. Protecting privileged Q_Keys prevents spoofing of certain types of traffic, including management messages. PRIV_NET_PRIVADDR is required to use privileged Q_Keys in the IB_USER_VERBS_CMD_MODIFY_QP operation. In addition for the two particular Q_Key values used by MADs (x80010000, x80010001), PRIV_SYS_NET_CONFIG is also required. The protection on using reserved well-known ports is analogous to the restrictions on socket binds of ports numbered less than 1024. Checks are made in the kernel to verify the user process has the PRIV_NET_PRIVADDR privilege to use ports numbered less than 1024 in the RDMA_USER_CM_CMD_BIND_ADDR operation. While we protect privileged Q_Keys and reserved ports, we do not currently plan to deliver system commands that use these capabilities. Therefore, we are not expecting to deliver exec_attr or prof_attr changes for these two use cases. This project relies on the same resource controls as the uDAPL project (See PSARC/2003/145) namely the use of the project.max-locked-memory to control the amount of memory a user process can pin down. 4.3 Description of Interfaces ----------------------------- A. Device node entries o '/devices/ib/sol_uverbs' character device Major node for the uverbs and ucma minor nodes provided by the 'sol_uverbs' driver o '/devices/ib/sol_uverbs@0:uverbs#' # - minor number (0-max), max is 15 character minor node device is created for each HCA in the system -- These device nodes are utilized by the 'libibverbs' library to open specific HCA instances and provide verb operations. Properties exported for each minor device node are: name type value ---- ---- ----- "device-id" int Vendor supplied part ID "vendor-id" int 24 bit Vendor ID "abi-version" int Defines the userland verbs to kernel interface version. This property is defined only for minor number 0. "guid" int The HCA's node GUID o '/devices/ib/sol_uverbs@0:ucma' character device minor node that may only be opened from within the kernel and is created for the 'sol_ucma kernel' module use -- This is project private. This device node is created for interaction between the 'sol_ucma' and 'sol_uverbs' drivers. The 'sol_ucma' driver does an ldi_open(9f) of this device file, to ensure that 'sol_uverbs' driver is loaded. o '/devices/pseudo/sol_ucma@0' pseudo device major node for the sol_ucma minor note provided by the sol_ucma driver o '/devices/pseudo/sol_ucma@0:sol_ucma' pseudo device minor node for RDMA CM operations Properties exported by the minor device node are: name type value ---- ---- ----- "abi-version" int Defines the userland verbs to kernel interface version o '/devices/ib/sol_umad' character device Major node for the issm and umad minor nodes provided by the 'sol_umad' driver o '/devices/ib/sol_umad@0:issm# o '/devices/ib/sol_umad@0:umad# # - minor number (0-max), max is 15 character device minor nodes that are created for each HCA port in the system -- These device nodes are utilized by the 'libibumad' for IsSM control and MAD operations. Properties exported for each minor device node are: name type value ---- ---- ----- "abi-version" int Defines the userland verbs to kernel interface version "guid" int The HCA's node GUID Opening the issm devices causes the IsSM port capability bit to be set on that port. Closing the issm device causes the IsSM port capability to be reset. IsSM signals the presence of an IB Subnet Manager on the port. B. User/kernel Interfaces This project also implements the OFED userland to kernel interfaces defined by the OFED headers: 'ib_user_verbs.h', 'rdma_user_cm.h' & 'ib_user_mad.h' (all ported from Linux). These interfaces allow the OFED 'libibverbs', 'librdmacm' & 'libibumad' libraries to access the kernel IB modules needed to implement the APIs exported by the libraries. The 'sol_uverbs' kernel module of this project implements the interface to the 'libibverbs' library defined in 'ib_user_verbs.h', the 'sol_ucma' kernel module implements the interface to the 'librdmacm' CM library defined in 'rdma_user_cm.h', and the 'sol_umad' kernel module implements the interface to the 'libibumad' library defined in 'ib_user_mad.h'. B.1 RDMA CM The 'sol_ucma' driver exports a device propertt named "abi-version" that may be examined to determine compatibility. The property has the following format: name='abi-version' type=int value=00000004 Entry points: open(2) obtain a CM fd close(2) close a CM fd poll(2) Poll for CM events write(2) write RDMA CM API commands to 'sol_ucma' kernel module on fd obtained from open() Details of the commands are in 'sol_ucma-cmds.txt' (in the materials directory). For reference, there is also a copy of rdma_user_cm.h in the materials directory. B.2 Verbs The 'sol_uverbs' driver exports a device property named "abi-version" that may be examined by the the userland consumer ('libibverbs') and used to determine the appropriateness of the user/kernel ABI compatibility. The property has the following format: name='abi-version' type=int value=00000006 If the user/kernel ABI revision are compatible, the consumer opens the desired character minor device for use: /devices/ib/sol_uverbs@0:uverbs# # - minor number (0-max) At the time of the open, 'sol_uverbs' creates unique minor device number beyond the range of all possible HCA/UCMA minor devices, and uses that to identify the caller's user context on subsequent calls into the driver. The consumer then uses the device write(2) entry point to invoke commands associated with the user/kernel ABI. The user commands are defined in the OFED header 'ib_user_verbs.h'. Where implementation specific information may be required, the commands allow for an opaque data area to be passed transparently between the underlying kernel device specific channel interface implementation and the device specific user space consumer of the ABI (e.g. the 'hermon' kernel driver may pass hermon specific variables associated with a command to the device specific consumer entity 'libmlx4'). This ABI has been extended with support for two commands not required by the OFED implementation, IB_USER_VERBS_CMD_QUERY_GID and IB_USER_VERBS_CMD_QUERY_PKEY. In the OFED kernel implementation, the underlying hardware specific driver exposes the device's GID table and PKEY table entries via the Linux /sys file system. In the absence of the /sys file system, the consumer utilizes the 'sol_uverb' driver to read the values from the underlying hardware specific driver. Additionally a per process asynchronous event file is created when the consumer opens of a specific 'sol_uverbs' minor node device See B.2.2) And a completion notification event file is created when the consumer requests creation of a completion channel via the 'IB_USER_VERB_CMD_CREATE_COMP_CHANNEL' (See B.2.1 & B.2.3). For reference, there is also a copy of ib_user_verbs.h in the materials directory. B.2.1 Verbs User/Kernel Interface /devices/ib/sol_uverbs@0:uverbs# # - minor number (0-max) open(2) Obtain fd to write(2) user verbs commands to the 'sol_uverbs' close(2) close fd from open(2) write(2) write user verbs commands to the 'sol_uverbs' kernel module There are two types of buffer passed to write(2) destined for the 'sol_uverbs' driver. They are (a) commands with no response and (b) commands with a response. Details are given in 'sol_uverbs-cmds.txt' (in the materials directory). B.2.2 Asynchronous Event Interface A per process asynchronous event file is created when the consumer opens of a specific 'sol_uverbs' minor node device. Operations supported on the async_fd are: open(2) Obtain fd for async event interface close(2) Close per process fd created by 'sol_uverbs' for delivery of async events read(2) Read on async event fd of an async event into an 'ib_uverbs_async_event_desc' structure: uint64_t element; uint32_t event_type; uint32_t reserved; Where 'event_type' is one of: IBV_EVENT_CQ_ERR IBV_EVENT_QP_FATAL IBV_EVENT_QP_REQ_ERR IBV_EVENT_QP_ACCESS_ERR IBV_EVENT_COMM_EST IBV_EVENT_SQ_DRAINED IBV_EVENT_PATH_MIG IBV_EVENT_PATH_MIG_ERR IBV_EVENT_DEVICE_FATAL IBV_EVENT_PORT_ACTIVE IBV_EVENT_PORT_ERR IBV_EVENT_LID_CHANGE IBV_EVENT_PKEY_CHANGE IBV_EVENT_SM_CHANGE IBV_EVENT_SRQ_ERR IBV_EVENT_SRQ_LIMIT_REACHED IBV_EVENT_QP_LAST_WQE_REACHED IBV_EVENT_CLIENT_REREGISTER And 'element' is one of: struct ibv_cq *cq; /* CQ that got the event */ struct ibv_qp *qp; /* QP that got the event */ struct ibv_srq *srq; /* SRQ that got the event */ int port_num; /* port that got the event */ When 'element' is 'cq', 'qp' or 'srq' 'element is a userland address that is a pointer to a userland allocated struct. poll(2) Poll for events on async event fd B.2.3 Completion Event Interface A completion notification event file is created when the consumer requests creation of a completion channel via the 'IB_USER_VERB_CMD_CREATE_COMP_CHANNEL' (See B.2.1). Operations supported on the cq_fd are: open(2) Obtain fd for completion event interface close(2) Close per process fd created by 'sol_uverbs' for delivery of completion events read(2) Read on cq event fd of a completion event into an 'ib_uverbs_comp_event_desc' structure: uint64_t cq_handle; Where 'cq_handle' is is a userland address that is a pointer to a userland allocated struct ibv_cq poll(2) Poll for events on cq event fd B.3 umad Interface The 'sol_mad' driver exports a device property named "abi-version" that may be examined by the the userland consumer ('libibumad')and used to determine the appropriateness of the user/kernel ABI compatibility. The property has the following format: name='abi-version' type=int value=00000005 If the user/kernel ABI revision are compatible, the consumer can open the desired character minor device for use: /devices/ib/sol_umad@0:issm# /devices/ib/sol_umad@0:umad# # - minor number (0-max) At the time of the open 'sol_umad' creates unique minor device number and uses that to identify the caller's user context on subsequent calls into the driver. The consumer then uses the device write(2) and ioctl(2) entry point to invoke commands associated with the user/kernel ABI. Details of the commands are defined in sol_umad-cmds.txt (in the materials directory). For reference, there is also a copy of ib_user_mad.h in the materials directory. C. Config Files The following config files, that contain no user configurable data, are provided: /kernel/drv/sol_uverbs.conf /kernel/drv/sol_ucma.conf /kernel/drv/sol_umad.conf 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: FastTrack 6.6. ARC Exposure: open --Boundary_(ID_C2cpAbYcE6qyN9/EAjpg9Q)-- From ted.h.kim@oracle.com Fri Jul 2 19:00:38 2010 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o6320bG9002229 for ; Fri, 2 Jul 2010 19:00:38 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o62MvxxW017177 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Fri, 2 Jul 2010 15:57:59 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L4Y00M01D4NUU00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 02 Jul 2010 15:57:59 -0700 (PDT) Received: from sca-ea-mail-2.sun.com ([192.18.43.25]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L4Y00L76D4NVS00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 02 Jul 2010 15:57:59 -0700 (PDT) Received: from rcsinet13.oracle.com (rcsinet13.oracle.com [148.87.113.125]) by sca-ea-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o62Mvwbg018034 for ; Fri, 02 Jul 2010 22:57:58 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by rcsinet13.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o62Mvv9R032718 for ; Fri, 02 Jul 2010 22:57:57 +0000 (GMT) Received: from abhmt016.oracle.com by acsmt353.oracle.com with ESMTP id 379257031278111410; Fri, 02 Jul 2010 15:56:50 -0700 Received: from [129.153.3.74] (/129.153.3.74) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 02 Jul 2010 15:56:50 -0700 Date: Fri, 02 Jul 2010 15:55:15 -0700 From: "Ted H. Kim" Subject: Re: OFUV Userland Interface [PSARC/2010/239 FastTrack timeout 07/09/2010] In-reply-to: <4C2E4770.2020105@oracle.com> To: PSARC-ext@sun.com Message-id: <4C2E6E53.8000105@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090207.4C2E6EF6.0059:SCFMA4539814,ss=1,fgs=0 References: <201006250033.o5P0XqjA006108@sac.sfbay.sun.com> <4C2473D2.3050606@Oracle.COM> <4C29175F.6020404@oracle.com> <20100628222758.GE13125@oracle.com> <4C2A4628.8040908@oracle.com> <20100629194450.GV11785@oracle.com> <4C2A71ED.40506@oracle.com> <20100629224007.GC11785@oracle.com> <4C2A91F0.1040805@oracle.com> <4C2BC966.2040600@oracle.com> <20100630225347.GE14528@oracle.com> <4C2CA914.2070405@oracle.com> <4C2E4770.2020105@oracle.com> User-Agent: Thunderbird 2.0.0.24 (X11/20100302) Status: RO Content-Length: 3672 Here are the diffs for the revised proposal, basically the date and a revision of the security section. 0.234# diff -bw -U6 orig-psarc-2010-239-case.txt psarc-2010-239-case.txt --- orig-psarc-2010-239-case.txt Fri Jul 2 13:33:11 2010 +++ psarc-2010-239-case.txt Fri Jul 2 15:46:11 2010 @@ -3,13 +3,13 @@ 1. Introduction 1.1. Project/Component Working Name: OFUV Userland Interface 1.2. Name of Document Author/Supplier: Author: Brendan Doyle 1.3 Date of This Document: - 24 June, 2010 + 2 July, 2010 4. Technical Description Table of Contents ----------------- 4.1 Introduction @@ -127,34 +127,44 @@ /kernel/drv/sol_uverbs.conf /kernel/drv/sol_ucma.conf /kernel/drv/sol_umad.conf D. Security and Resource Control -The sol_umad minor nodes are owned by root and has 600 file -permissions. Only root can execute the tools and utilities that use -these minor nodes as a means of access to the kernel IBMF (see -4.3.B.3). We restrict this because the Management Datagram (MAD) based -tools/utilities allow users to reconfigure the IB fabric and nodes on -the fabric. +The sol_umad minor nodes are owned by root and have 666 file +permissions. Operations on these nodes require the PRIV_SYS_NET_CONFIG +privilege. We restrict this because the Management Datagram (MAD) +based tools/utilities allow users to reconfigure the IB fabric and +nodes on the fabric. When commands and utilities using umad are +proposed in the companion ARC case, changes to exec_attr(5) for these +binaries will also be described. The sol_uverbs and sol_ucma minor nodes are owned by root and have 666 file permissions. All users can run the utilities that use these files as a means of access to the kernel IBTF. These utilities are not intrusive and do not allow configuration of the IB fabric or fabric nodes. However protection is afforded against non-root users use of privileged Q_Keys and use of reserved well-known ports. Protecting privileged Q_Keys prevents spoofing of certain types of traffic, -including management messages. The protections on using reserved -well-known ports is analogous to the restrictions on socket binds of -ports numbered less than 1024. Checks are made in the kernel to verify -the user process has the PRIV_NET_PRIVADDR privilege for these -operations, before the request is allowed to proceed. +including management messages. PRIV_NET_PRIVADDR is required to use +privileged Q_Keys in the IB_USER_VERBS_CMD_MODIFY_QP operation. In +addition for the two particular Q_Key values used by MADs (x80010000, +x80010001), PRIV_SYS_NET_CONFIG is also required. The protection on +using reserved well-known ports is analogous to the restrictions on +socket binds of ports numbered less than 1024. Checks are made in the +kernel to verify the user process has the PRIV_NET_PRIVADDR privilege +to use ports numbered less than 1024 in the RDMA_USER_CM_CMD_BIND_ADDR +operation. +While we protect privileged Q_Keys and reserved ports, we do not +currently plan to deliver system commands that use these +capabilities. Therefore, we are not expecting to deliver exec_attr or +prof_attr changes for these two use cases. + This project relies on the same resource controls as the uDAPL project (See PSARC/2003/145) namely the use of the project.max-locked-memory to control the amount of memory a user process can pin down. 4.3 Description of Interfaces -- Ted H. Kim, PhD Sun/Oracle America Inc. ted.h.kim@oracle.com 222 North Sepulveda Blvd., 10th Floor (310) 341-1116 El Segundo, CA 90245 (310) 341-1115 FAX From sebastien.roy@oracle.com Wed Jul 7 09:18:58 2010 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o67GIw7m023194 for ; Wed, 7 Jul 2010 09:18:58 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o67GIw2T002192 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 7 Jul 2010 09:18:58 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L570070N3ZMGR00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 07 Jul 2010 10:18:58 -0600 (MDT) Received: from brmea-mail-2.sun.com ([192.18.98.43]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L57006E13ZLYJ00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 07 Jul 2010 10:18:57 -0600 (MDT) Received: from rcsinet13.oracle.com (rcsinet13.oracle.com [148.87.113.125]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o67GIvtA004104 for ; Wed, 07 Jul 2010 16:18:57 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by rcsinet13.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o67GIsMP023389 for ; Wed, 07 Jul 2010 16:18:54 +0000 (GMT) Received: from abhmt001.oracle.com by acsmt355.oracle.com with ESMTP id 386220351278519465; Wed, 07 Jul 2010 09:17:45 -0700 Received: from [129.148.174.103] (/129.148.174.103) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 07 Jul 2010 09:17:44 -0700 Date: Wed, 07 Jul 2010 12:17:43 -0400 From: Sebastien Roy Subject: Re: OFUV Userland Interface [PSARC/2010/239 FastTrack timeout 07/09/2010] In-reply-to: <4C2E6E53.8000105@oracle.com> To: "Ted H. Kim" Cc: psarc-ext@sun.com Message-id: <4C34A8A7.60602@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090205.4C34A8F0.00E0:SCFMA4539814,ss=1,fgs=0 References: <201006250033.o5P0XqjA006108@sac.sfbay.sun.com> <4C2473D2.3050606@Oracle.COM> <4C29175F.6020404@oracle.com> <20100628222758.GE13125@oracle.com> <4C2A4628.8040908@oracle.com> <20100629194450.GV11785@oracle.com> <4C2A71ED.40506@oracle.com> <20100629224007.GC11785@oracle.com> <4C2A91F0.1040805@oracle.com> <4C2BC966.2040600@oracle.com> <20100630225347.GE14528@oracle.com> <4C2CA914.2070405@oracle.com> <4C2E4770.2020105@oracle.com> <4C2E6E53.8000105@oracle.com> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.9) Gecko/20100607 Lightning/1.0b1 Thunderbird/3.0.4 Status: RO Content-Length: 43 I'm happy with the updated spec. +1 -Seb From ted.h.kim@oracle.com Wed Jul 7 11:01:47 2010 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o67I1lQf011539 for ; Wed, 7 Jul 2010 11:01:47 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o67I1lUx011807 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 7 Jul 2010 11:01:47 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L5700M018QZ6P00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 07 Jul 2010 11:01:47 -0700 (PDT) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L5700LJ68QYXJ00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 07 Jul 2010 11:01:46 -0700 (PDT) Received: from rcsinet13.oracle.com (rcsinet13.oracle.com [148.87.113.125]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o67I1jqD014142 for ; Wed, 07 Jul 2010 18:01:46 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by rcsinet13.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o67GVjKm000493 for ; Wed, 07 Jul 2010 18:01:45 +0000 (GMT) Received: from abhmt012.oracle.com by acsmt354.oracle.com with ESMTP id 406834951278525698; Wed, 07 Jul 2010 11:01:38 -0700 Received: from [129.153.3.74] (/129.153.3.74) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 07 Jul 2010 11:01:38 -0700 Date: Wed, 07 Jul 2010 10:59:56 -0700 From: "Ted H. Kim" Subject: Re: OFUV Userland Interface [PSARC/2010/239 FastTrack timeout 07/09/2010] In-reply-to: <4C2F73AE.7050401@oracle.com> To: PSARC-ext@sun.com Message-id: <4C34C09C.7060102@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090207.4C34C109.015A:SCFMA4539814,ss=1,fgs=0 References: <201006250033.o5P0XqjA006108@sac.sfbay.sun.com> <4C2473D2.3050606@Oracle.COM> <4C29175F.6020404@oracle.com> <20100628222758.GE13125@oracle.com> <4C2A4628.8040908@oracle.com> <20100629194450.GV11785@oracle.com> <4C2A71ED.40506@oracle.com> <20100629224007.GC11785@oracle.com> <4C2A91F0.1040805@oracle.com> <4C2BC966.2040600@oracle.com> <20100630225347.GE14528@oracle.com> <4C2CA914.2070405@oracle.com> <4C2E4770.2020105@oracle.com> <4C2E6C21.30305@oracle.com> <4C2F73AE.7050401@oracle.com> User-Agent: Thunderbird 2.0.0.24 (X11/20100302) Status: RO Content-Length: 298 Folks, This case was approved at the 7/7/10 PSARC meeting. So I am marking it closed approved. -ted -- Ted H. Kim, PhD Sun/Oracle America Inc. ted.h.kim@oracle.com 222 North Sepulveda Blvd., 10th Floor (310) 341-1116 El Segundo, CA 90245 (310) 341-1115 FAX