From John.Forte@sun.com Tue Feb 17 15:10:55 2009 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n1HNAsSC003641 for ; Tue, 17 Feb 2009 15:10:55 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n1HNAeJl019214 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Tue, 17 Feb 2009 16:10:54 -0700 (MST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KF80080TGDW0Q00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.Com); Tue, 17 Feb 2009 15:10:44 -0800 (PST) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KF8007EGGDV9Q10@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.Com); Tue, 17 Feb 2009 15:10:43 -0800 (PST) Received: from fe-amer-09.sun.com ([192.18.109.79]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n1HNAhOC026552 for ; Tue, 17 Feb 2009 23:10:43 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7.0-3.01 64bit (built Dec 23 2008)) id <0KF800200EKPAF00@mail-amer.sun.com> for PSARC-ext@Sun.Com (ORCPT PSARC-ext@Sun.Com); Tue, 17 Feb 2009 16:10:43 -0700 (MST) Received: from [129.146.56.52] ([unknown] [129.146.56.52]) by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7.0-3.01 64bit (built Dec 23 2008)) with ESMTPSA id <0KF8003VWGDLXGC0@mail-amer.sun.com> for PSARC-ext@Sun.Com (ORCPT PSARC-ext@Sun.Com); Tue, 17 Feb 2009 16:10:34 -0700 (MST) Date: Tue, 17 Feb 2009 15:06:23 -0800 From: John Forte Subject: COMSTAR Infiniband SRP Target [PSARC 2009/111 FastTrack timeout 02/25/2009] Sender: John.Forte@sun.com To: PSARC-ext@sun.com Cc: Peter Cudhea , Susan Gleeson Message-id: <499B42EF.60801@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 User-Agent: Thunderbird 2.0.0.17 (X11/20081014) Status: RO Content-Length: 16407 I am sponsoring this fasttrack for Peter Cudhea. Requested binding is Patch, timeout is 02/25/2009. There is an IO controller profile attributes table, described in section 4.2.3, in the case materials directory. - John This information is Copyright 2009 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: COMSTAR Infiniband SRP Target 1.2. Name of Document Author/Supplier: Peter.Cudhea@sun.com 1.3. Date of This Document: 02/12/09 4. Technical Description COMSTAR Infiniband SRP Target ------------------------- 4.1. Problem OpenSolaris currently lacks a target driver for the SCSI RDMA Protocol (SRP). SRP accelerates the SCSI protocol by mapping the data transfer phases of SCSI commands to RDMA operations. As a result an SRP initiator should be able to read and write data from a COMSTAR SRP target at high data rates with relatively low CPU utilization. SRP is an alternative to iSER (PSARC 2008/395) for accessing SCSI storage over an Infiniband fabric. Both protocols are seeing demand in the market. In particular, we need an SRP target in COMSTAR (PSARC 2007/523) to enable VMware connectivity to OpenSolaris based open storage. VMware ESX, for example, supports only SRP (not iSER) for block-based storage connectivity over Infiniband. 4.2. Proposal The project will deliver a target implementation of SCSI RDMA Protocol represented as a COMSTAR STMF port provider. We include a minimal implementation of the Infiniband Device Management Agen as a consumer to IBTF (PSARC 2002/132). This agent allows initiator systems to query the capabilities of the target. The SRP port provider will register its targets with the IB DM Agent to allow the targets to be discovered by SRP initiators. 4.2.1. COMSTAR SRP Target (srpt) When the SRP target service is enabled, it will register as a COMSTAR port provider using STMF. This port provider will use the IB transport framework (IBTF) to enumerate all the HCAs on the system by GUID. Each IB HCA will be reflected to STMF as a COMSTAR target named 'eui.'. For example, for an IB HCA with a HCA GUID of 0003BA0001002E49 the STMF target name will be 'eui.0003BA0001002E49'. STMF commands may then be used to assign these targets to host groups and to create views that determine which backing stores are accessible to which targets. STMF commands may also be used to mark each target as either offline or online. All of the physical IB ports on an HCA will treated as part of the same STMF target. In IB target terms, each Host Channel Adapter (HCA) is treated as a Target Channel Adapter (TCA) with a single IO Unit containing a single IO controller. Multiple physical ports on the HCA are not exposed as separate virtual target-side resources. When the SRP service is enabled and the STMF target for a particular HCA is marked online, each port on that HCA will be configured to listen for incoming connections to the SRP service. A virtual I/O Controller representing the target will also be registered with the minimal IB DM Agent as described in section 4.2.2. The SRP target capability will be represented as an SMF service using the FMRI svc:/system/ibsrp/target:default. This service will be disabled by default. No new 'rights profiles' will be defined; each administrative method for this service (e.g. start and stop) will use a credential with user=root, group=root, and privileges=basic,sys_devices. The ibsrp/target service will be dependent on STMF (svc:/system/stmf:default) and on the IB Device Management Agent described in section 4.2.2 (svc:/system/ibdma:default). The SRP target will be implemented as pseudo device driver 'srpt' which is a child under the Infiniband 'ib' nexus. 4.2.2 Minimal Infiniband Device-Management Agent (ibdma) In order for IB initiators to enumerate the available storage, this project also includes a minimal "device management agent" for Infiniband, as described in section 16.3 of the Infiniband Architecture Spec. (Section 16.3 corresponds to version 1 of the IB Device Management protocol, which is distinct from the version 2 Device Management protocol as defined in Annex A8.) Infiniband "device management" services are used by IB initiator systems to enumerate the IO Units (IOUs) and IO Controllers (IOCs) on the target system, and to query the services supported on each IO Controller. For this case, SRP initiators in particular use DM services to enumerate all the IOCs that are SRP-capable IOCs, and to enumerate the SRP services that are available through each IOC. SRP is the only target-side service that makes use of IB DM services for discovery that the Infiniband group expects OpenSolaris to support. If other target-side services are added in the future that require a more fully-functioning DM Agent, then this minimal agent can be expanded and extended as required. While the API we introduce in section 4.2.2.3 to register new services as IO Controllers is somewhat general, and could in principle be used by other target-side services, we are keeping the new API interface as Project Private to provide maximum flexibility to enhance or extend the details in the future. We implement only the query-oriented subset of the DM protocol that is necessary for SRP discovery. The full DM protocol, among other uses, allows initiator systems to request target-side device management services such as testing devices and retrieving device diagnostic codes. The SRP spec in section B.5 spells out its requirements: The IB I/O unit shall include an IB device management agent to provide the IOUnitInfo, IOControllerProfile, and ServiceEntries attributes. The query-oriented subset we implement falls short of full compliance with the IB DM protocol as specified in the IB Arch spec. This subset is consistent with how initiators actually use DM Agent services for discovery. We return appropriate error codes (either "method not supported" or "combination of method and attribute not supported" for all other DM requests. The IB Device Management agent will be represented as an SMF service using the FMRI svc:/system/ibdma:default. This service will be disabled by default. No new 'rights profiles' will be defined; each administrative method for this service (e.g. start and stop) will use a credential with user=root, group=root, and privileges=basic,sys_devices. The IB DM agent will be implemented as a pseudo device driver 'ibdma' under /kernel/misc. 4.2.2.1 Listening for DM agent requests To support this case, the IB group will add a new value IBT_DMA to the enumeration ibt_clnt_modinfo_t that is used as an argument to ibt_attach. The IBT_DMA value is reserved for Sun Internal use only, and is used to directly support the IB Device Management Agent. The minimal IB DM agent, as specified in section B.5 of the SRP spec (SCSI Architecture Mapping) will respond to DevMgtGet requests with the following Attribute IDs: Attribute ID Attribute Name OpenSolaris Comments 0x01 ClassPortInfo "HELLO" message to determine protocol class match 0x02 IOUnitInfo Enumerate the "virtual IOCs" available on an IO Unit. Each HCA is represented as an IO Unit. 0x10 IOControllerProfile Retrieve IO Controller Profile information for a specific IOC 0x12 ServiceEntries Enumerate Service Names and Service IDs for a given IOC The manual page changes for adding IBT_DMA are: ------- ibt_attach.9f ------- 75a76,77 > IBT_DMA For Sun Internal use only. ------- ibt_clnt_modinfo_t.9s ------- 55a56,57 > IBT_DMA For Sun Internal use only. 4.2.2.2 Enumeration of HCAs as I/O Units Infiniband target services are made available through I/O Controllers that are associated with I/O Units. See for example Figure B.3 in Annex B of the SRP Specification for an SRP-specific picture of this architecture. In the general IB architecture, a particular I/O Unit could be visible through several different Target CAs and thus via the different IB ports on those CAs. The IO Controllers and services available through an IOU are generally consistent no matter which port is used to make a connection to that IOU. In simple terms, once a connection has reached an IO Unit, it can make use of the services provided by that I/O Unit. For the minimal IB DM Agent, we choose to represent each HCA as a separate I/O Unit. The key to the minimal IB DM Agent is that it requires no administration. By using the HCA GUID as the I/O Unit GUID, we avoid the need to administer I/O Units as separate entities. Alternative no-administration models would be to treat the entire target system as a single I/O Unit or to treat each individual port on each HCA as an I/O Unit. The current HCA-as-an-IOU model was chosen because it is familiar to current users of IB target-side services on other systems such as Linux. This model works the way users of IB services expect it to. As soon as the ibdma service is enabled, the target system will: o Modify the "port profile" for each HCA port to indicate that "device management is supported" on that port. This is part of the "port capabilities mask". o Listen for IB Device Management MADs and respond with errors to those requests that are not supported. o Enumerate each HCA on the system as an available IO Unit. o Until specific target-side services are registered using the API defined in section 4.2.2.3, the IO Units will report they have no contained IO Controllers. Virtual IOCs are tied to a specific service and do not exist until a specific service is enabled. 4.2.2.3 Registering a Virtual I/O Controller for a Supported Service In IB, each IOC is associated with an "IO Controller profile" that defines the specific services that are available via that IOC. For example, as described above the Infiniband Annex to the SRP specification defines both the IO/Controller profile and the service name to use for SRP. ibdma_ioc_register Register a "Virtual IO Controller". ibdma_ioc_unregister Unregister a "Virtual IO Controller" ibdma_ioc_update Modify the characteristics of an existing virtual IO Controller. To register a new virtual IO Controller, the caller specifies the GUID of the parent IO Unit, an IOC profile describing the virtual service, and a specific list of target-side Service Entries associated with the virtual IOC. As far as the IB DM Agent is concerned, the set of advertised IOCs is arbitrary. A single IO Unit could support multiple different target-side services each of which would be represented as a separate virtual IO Controller. Similarly, the target-side service can decide whether or not to create multiple virtual IOCs representing the different physical ports on the HCA. It is up to each service individually to determine the particular IO controllers to emulate. The IB DM Agent simply advertises these virtual IO controllers to initiator systems. 4.2.3. The SRP Virtual I/O Controller For SRP, a single virtual IO Controller is created for each HCA. Each virtual IOC is marked as being SRP-capable by using parameters from the standard SRP IO Controller Profile which appears in Annex B of the SRP spec. The specific parameters used in this IO profile are available in the materials directory for this case in a document called SRPT-IOC-parameters.txt. 4.3. Risks and Assumptions The current implementation relies on IB "shared receive queues" which are not available in all IB HCAs or drivers. All Sun supported HCAs and drivers do support this capability. In particular, the tavor, arbel, and (since snv_107) hermon drivers do support this capability. 4.4. How will you know when you are done?: Linux and VMWare ESX initiators based on the OFED stack can reliably access COMSTAR storage through the SRP target port provider. 4.5. Interfaces: -------------------------------------------------------------- EXPORTED INTERFACES Interface Level Comments -------------------------------------------------------------- ibdma_ioc_register Project Private ibdma_ioc_unregister Project Private ibdma_ioc_update Project Private eui. Committed Naming convention for STMF SRP targets IOU GUID = HCA GUID Committed SRP initiators see stable targets IOC GUID = HCA GUID Committed SRP initiators see stable targets SCSI RDMA Protocol Committed Defined by T10 standard for SRP Device Management Protocol Committed Section 16.3 (query subset) of IB architecture IBT_DMA Consolidation Private arg to ibt_attach -------------------------------------------------------------- IMPORTED INTERFACES Interface Level Comments -------------------------------------------------------------- IBTF Consolidation Private STMF Consolidation Private IBT_DMA Consolidation Private Exported from IBTF Imported to SRP --------------------------------------------------------------- 4.6. Doc Impact: Man pages for srpt and ibdma device drivers. IBTF internal man page updates (as described in 4.2.2.1) for ibt_attach(9f) and ibt_clnt_modinfo_t(9s). We will need to add a section in the OpenSolaris COMSTAR Administration Guide that describes how to provision different kinds of COMSTAR storage targets. Currently, this guide describes only how to provision Fibre Channel storage targets. The new documentation should be coordinated among all the new COMSTAR port providers such as iSCSI, iSER, FCoE, and SAS. 4.7. Admin/Config Impact: None 4.10. Packaging & Delivery: Packages SUNWsrptr x86: /kernel/drv/srpt /kernel/drv/amd64/srpt /kernel/drv/srpt.conf /lib/svc/method/svc-srpt /var/svc/manifest/system/ibsrp/target.xml sparc: /kernel/drv/sparcv9/srpt /kernel/drv/srpt.conf /lib/svc/method/svc-srpt /var/svc/manifest/system/ibsrp/target.xml SUNWibdmar x86: /kernel/misc/ibdma /kernel/misc/amd64/ibdma /lib/svc/method/svc-ibdma /var/svc/manifest/system/ibdma.xml sparc: /kernel/misc/sparcv9/ibdma /lib/svc/method/svc-ibdma /var/svc/manifest/system/ibdma.xml 5. References Infiniband Transport Framework (IBTF) [PSARC 2002/132] COMSTAR SCSI Transport Framework (STMF) [PSARC 2007/523] SCSI Architecture Model 2 (SAM-2, T10/1157-D) (t10.org) SCSI-3 Primary Commands (SPC-3, T10/1416-D) (t10.org) SCSI RDMA Protocol revision 16a of the SRP T10/1415-D specification of July 3rd, 2002 (http://www.t10.org/ftp/t10/drafts/srp/srp-r16a.pdf) Infiniband Architecture Specification v1.2.1 (Infiniband Trade Association) (http://www.infinibandta.org/specs/) From Darren.Moffat@sun.com Wed Feb 18 01:58:56 2009 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n1I9wuji007418 for ; Wed, 18 Feb 2009 01:58:56 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n1I9wtEE005372 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 18 Feb 2009 01:58:56 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KF900D03AE7WP00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 18 Feb 2009 02:58:55 -0700 (MST) Received: from gmp-eb-inf-2.sun.com ([192.18.6.24]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KF900D6GAE4OG00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 18 Feb 2009 02:58:53 -0700 (MST) Received: from fe-emea-09.sun.com (gmp-eb-lb-2-fe2.eu.sun.com [192.18.6.11]) by gmp-eb-inf-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n1I9wqnh014749 for ; Wed, 18 Feb 2009 09:58:52 +0000 (GMT) Received: from conversion-daemon.fe-emea-09.sun.com by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7.0-3.01 64bit (built Dec 23 2008)) id <0KF900A009BDNV00@fe-emea-09.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 18 Feb 2009 09:58:52 +0000 (GMT) Received: from [129.156.173.21] ([unknown] [129.156.173.21]) by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7.0-3.01 64bit (built Dec 23 2008)) with ESMTPSA id <0KF900085ADR9Q50@fe-emea-09.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 18 Feb 2009 09:58:40 +0000 (GMT) Date: Wed, 18 Feb 2009 09:58:39 +0000 From: Darren J Moffat Subject: Re: COMSTAR Infiniband SRP Target [PSARC 2009/111 FastTrack timeout 02/25/2009] In-reply-to: <499B42EF.60801@sun.com> Sender: Darren.Moffat@sun.com To: John Forte Cc: PSARC-ext@sun.com, Peter Cudhea , Susan Gleeson Message-id: <499BDBCF.8060805@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <499B42EF.60801@sun.com> User-Agent: Thunderbird 2.0.0.18 (X11/20081201) Status: RO Content-Length: 792 John Forte wrote: > The IB Device Management agent will be represented as an SMF > service using the FMRI svc:/system/ibdma:default. This > service will be disabled by default. No new 'rights profiles' > will be defined; each administrative method for this service > (e.g. start and stop) will use a credential with user=root, > group=root, and privileges=basic,sys_devices. Why does the service method need uid/gid of root and sys_devices ? Can it instead run as daemon (lets not have the daemon vs noaccess discussion again here though) with sys_devices ? In otherwords please justify why uid and gid of root is needed when sys_devices is used. Does it really need all the privileges in basic ? [ Eg file_link_any, proc_session, proc_info ] ? -- Darren J Moffat From Peter.Cudhea@sun.com Wed Feb 18 07:09:39 2009 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n1IF9dnA003597 for ; Wed, 18 Feb 2009 07:09:39 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n1IF9S9P005876 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 18 Feb 2009 07:09:39 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KF90004ZOS10400@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 18 Feb 2009 07:09:37 -0800 (PST) Received: from brmea-mail-2.sun.com ([192.18.98.43]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KF900JX7OS05L30@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 18 Feb 2009 07:09:37 -0800 (PST) Received: from fe-amer-09.sun.com ([192.18.109.79]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n1IF9aik013119 for ; Wed, 18 Feb 2009 15:09:36 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7.0-3.01 64bit (built Dec 23 2008)) id <0KF900200OB7DG00@mail-amer.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 18 Feb 2009 08:09:36 -0700 (MST) Received: from peter-cudheas-macbook-pro.local ([unknown] [129.150.64.105]) by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7.0-3.01 64bit (built Dec 23 2008)) with ESMTPSA id <0KF900A8PORPXJG0@mail-amer.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 18 Feb 2009 08:09:26 -0700 (MST) Date: Wed, 18 Feb 2009 10:09:25 -0500 From: Peter Cudhea Subject: Re: COMSTAR Infiniband SRP Target [PSARC 2009/111 FastTrack timeout 02/25/2009] In-reply-to: <499BDBCF.8060805@Sun.COM> Sender: Peter.Cudhea@sun.com To: Darren J Moffat Cc: John Forte , PSARC-ext@sun.com, Susan Gleeson Message-id: <499C24A5.3090501@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <499B42EF.60801@sun.com> <499BDBCF.8060805@Sun.COM> User-Agent: Thunderbird 2.0.0.19 (Macintosh/20081209) Status: RO Content-Length: 2079 My intention was to copy existing code that is known to be working, specifically from the COMSTAR iscsi target port provider. That being said, it looks to me that both the iscsi port provider, the srpt port provider introduced here, as well as the ibdma service, could all function fine with a more limited set of privileges. I could use help determining the best profile to use. We intend to borrow patterns (and this rights profile) from the existing service svc:/network/iscsi/target:default. The /lib/svc/method/iscsi-tgt program opens a /devices entry and then issues an ioctl to perform the actual operation (start, stop, change-parameters). I believe we need sys_devices to open the device file under /devices/ib/. The ioctl implementation inside the kernel then calls drv_priv(cred), which also appears to only check sys_devices. So would a profile with only the sys_devices privilege be enough to get the job done? Is something more recommended? Note that the enhanced privileges are used only long enough to run the routine that executes the ioctl -- all long-running code is in the kernel itself. Once we get clear the proper profile to use, we can open an RFE to apply that same profile to COMSTAR iSCSI/ISER. Peter Darren J Moffat wrote: > John Forte wrote: >> The IB Device Management agent will be represented as an SMF >> service using the FMRI svc:/system/ibdma:default. This >> service will be disabled by default. No new 'rights profiles' >> will be defined; each administrative method for this service >> (e.g. start and stop) will use a credential with user=root, >> group=root, and privileges=basic,sys_devices. > > Why does the service method need uid/gid of root and sys_devices ? > Can it instead run as daemon (lets not have the daemon vs noaccess > discussion again here though) with sys_devices ? In otherwords please > justify why uid and gid of root is needed when sys_devices is used. > > Does it really need all the privileges in basic ? [ Eg file_link_any, > proc_session, proc_info ] ? > From Peter.Cudhea@sun.com Thu Feb 19 09:46:35 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n1JHkYNY006793 for ; Thu, 19 Feb 2009 09:46:34 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n1JHkSD0028259 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Fri, 20 Feb 2009 01:46:33 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KFB00427QPKI900@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 19 Feb 2009 09:46:32 -0800 (PST) Received: from brmea-mail-2.sun.com ([192.18.98.43]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KFB001V3QPGAR40@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 19 Feb 2009 09:46:29 -0800 (PST) Received: from fe-amer-09.sun.com ([192.18.109.79]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n1JHkSZP023915 for ; Thu, 19 Feb 2009 17:46:28 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7.0-3.01 64bit (built Dec 23 2008)) id <0KFB00C00ONVMI00@mail-amer.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 19 Feb 2009 10:46:28 -0700 (MST) Received: from dhcp-ubur01-168-192.East.Sun.COM ([unknown] [129.148.168.192]) by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7.0-3.01 64bit (built Dec 23 2008)) with ESMTPSA id <0KFB00JUOQPDK150@mail-amer.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 19 Feb 2009 10:46:26 -0700 (MST) Date: Thu, 19 Feb 2009 12:46:24 -0500 From: Peter Cudhea Subject: Re: COMSTAR Infiniband SRP Target [PSARC 2009/111 FastTrack timeout 02/25/2009] In-reply-to: <499C24A5.3090501@sun.com> Sender: Peter.Cudhea@sun.com To: Peter Cudhea Cc: Darren J Moffat , John Forte , psarc-ext@sun.com, Susan Gleeson Message-id: <499D9AF0.6040905@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <499B42EF.60801@sun.com> <499BDBCF.8060805@Sun.COM> <499C24A5.3090501@sun.com> User-Agent: Thunderbird 2.0.0.19 (Macintosh/20081209) Status: RO Content-Length: 2650 We believe we can switch to using the following credential for the ibdma service. If we determine a need for additional privileges beyond this, we will send a follow-up note to this case file. Would this address your concerns? user='daemon', group='daemon', privs='basic,sys_devices,!file_link_any, !proc_session, !proc_info'. I have also opened CR 6807687 to capture the desire to rationalize the ways the various COMSTAR SMF services and utility commands use RBAC. Peter Peter Cudhea wrote: > My intention was to copy existing code that is known to be working, > specifically from the COMSTAR iscsi target port provider. That being > said, it looks to me that both the iscsi port provider, the srpt port > provider introduced here, as well as the ibdma service, could all > function fine with a more limited set of privileges. I could use help > determining the best profile to use. > > We intend to borrow patterns (and this rights profile) from the > existing service svc:/network/iscsi/target:default. The > /lib/svc/method/iscsi-tgt program opens a /devices entry and then > issues an ioctl to perform the actual operation (start, stop, > change-parameters). I believe we need sys_devices to open the > device file under /devices/ib/. The ioctl implementation inside the > kernel then calls drv_priv(cred), which also appears to only check > sys_devices. So would a profile with only the sys_devices privilege > be enough to get the job done? Is something more recommended? > > Note that the enhanced privileges are used only long enough to run the > routine that executes the ioctl -- all long-running code is in the > kernel itself. > > Once we get clear the proper profile to use, we can open an RFE to > apply that same profile to COMSTAR iSCSI/ISER. > > Peter > > Darren J Moffat wrote: >> John Forte wrote: >>> The IB Device Management agent will be represented as an SMF >>> service using the FMRI svc:/system/ibdma:default. This >>> service will be disabled by default. No new 'rights profiles' >>> will be defined; each administrative method for this service >>> (e.g. start and stop) will use a credential with user=root, >>> group=root, and privileges=basic,sys_devices. >> >> Why does the service method need uid/gid of root and sys_devices ? >> Can it instead run as daemon (lets not have the daemon vs noaccess >> discussion again here though) with sys_devices ? In otherwords >> please justify why uid and gid of root is needed when sys_devices is >> used. >> >> Does it really need all the privileges in basic ? [ Eg file_link_any, >> proc_session, proc_info ] ? >> From Darren.Moffat@sun.com Thu Feb 19 09:50:16 2009 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n1JHoFtj006931 for ; Thu, 19 Feb 2009 09:50:16 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n1JHoAap004269 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Thu, 19 Feb 2009 17:50:14 GMT Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KFB00G2VQVQTL00@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 19 Feb 2009 10:50:14 -0700 (MST) Received: from gmp-eb-inf-1.sun.com ([192.18.6.21]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KFB003UEQVEOU90@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 19 Feb 2009 10:50:03 -0700 (MST) Received: from fe-emea-10.sun.com (gmp-eb-lb-2-fe2.eu.sun.com [192.18.6.11]) by gmp-eb-inf-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n1JHo2Gj009784 for ; Thu, 19 Feb 2009 17:50:02 +0000 (GMT) Received: from conversion-daemon.fe-emea-10.sun.com by fe-emea-10.sun.com (Sun Java(tm) System Messaging Server 7.0-3.01 64bit (built Dec 23 2008)) id <0KFB00200PU9FJ00@fe-emea-10.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 19 Feb 2009 17:50:02 +0000 (GMT) Received: from [129.156.173.21] ([unknown] [129.156.173.21]) by fe-emea-10.sun.com (Sun Java(tm) System Messaging Server 7.0-3.01 64bit (built Dec 23 2008)) with ESMTPSA id <0KFB00HETQV6NX90@fe-emea-10.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 19 Feb 2009 17:49:55 +0000 (GMT) Date: Thu, 19 Feb 2009 17:49:54 +0000 From: Darren J Moffat Subject: Re: COMSTAR Infiniband SRP Target [PSARC 2009/111 FastTrack timeout 02/25/2009] In-reply-to: <499D9AF0.6040905@sun.com> Sender: Darren.Moffat@sun.com To: Peter Cudhea Cc: John Forte , psarc-ext@sun.com, Susan Gleeson Message-id: <499D9BC2.9000304@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <499B42EF.60801@sun.com> <499BDBCF.8060805@Sun.COM> <499C24A5.3090501@sun.com> <499D9AF0.6040905@sun.com> User-Agent: Thunderbird 2.0.0.18 (X11/20081201) Status: RO Content-Length: 428 Peter Cudhea wrote: > We believe we can switch to using the following credential for the > ibdma service. If we determine a need for additional privileges beyond > this, we will send a follow-up note to this case file. Would this > address your concerns? > user='daemon', > group='daemon', > privs='basic,sys_devices,!file_link_any, !proc_session, !proc_info'. That looks great and nicely restricted. -- Darren J Moffat From John.Forte@sun.com Wed Feb 25 11:59:18 2009 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n1PJxIum016679 for ; Wed, 25 Feb 2009 11:59:18 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n1PJxFGD002124 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 25 Feb 2009 11:59:18 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KFN001070USO000@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.Com); Wed, 25 Feb 2009 11:59:16 -0800 (PST) Received: from brmea-mail-2.sun.com ([192.18.98.43]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KFN00JII0USIR40@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.Com); Wed, 25 Feb 2009 11:59:16 -0800 (PST) Received: from fe-amer-10.sun.com ([192.18.109.80]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n1PJxFqK018354 for ; Wed, 25 Feb 2009 19:59:15 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7.0-3.01 64bit (built Dec 23 2008)) id <0KFN00C000DV5S00@mail-amer.sun.com> for PSARC-ext@Sun.Com (ORCPT PSARC-ext@Sun.Com); Wed, 25 Feb 2009 12:59:15 -0700 (MST) Received: from [129.146.56.52] ([unknown] [129.146.56.52]) by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7.0-3.01 64bit (built Dec 23 2008)) with ESMTPSA id <0KFN00GLT0UKDQA0@mail-amer.sun.com> for PSARC-ext@Sun.Com (ORCPT PSARC-ext@Sun.Com); Wed, 25 Feb 2009 12:59:08 -0700 (MST) Date: Wed, 25 Feb 2009 11:54:48 -0800 From: John Forte Subject: Re: COMSTAR Infiniband SRP Target [PSARC 2009/111 FastTrack timeout 02/25/2009] In-reply-to: <499B42EF.60801@sun.com> Sender: John.Forte@sun.com To: Peter Cudhea Cc: PSARC-ext@sun.com, Susan Gleeson Message-id: <49A5A208.1090004@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <499B42EF.60801@sun.com> User-Agent: Thunderbird 2.0.0.17 (X11/20081014) Status: RO Content-Length: 48 This case was approved at PSARC today. - John From John.Forte@sun.com Tue Apr 14 10:40:06 2009 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n3EHe6FH009556 for ; Tue, 14 Apr 2009 10:40:06 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n3EHe2AT004572 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Tue, 14 Apr 2009 11:40:06 -0600 (MDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KI300B0JQESA800@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.Com); Tue, 14 Apr 2009 11:40:04 -0600 (MDT) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KI300B2UQES5O00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.Com); Tue, 14 Apr 2009 11:40:04 -0600 (MDT) Received: from fe-amer-10.sun.com ([192.18.109.80]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n3EHe47O003525 for ; Tue, 14 Apr 2009 17:40:04 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7.0-5.01 64bit (built Feb 19 2009)) id <0KI300700QA9GR00@mail-amer.sun.com> for PSARC-ext@Sun.Com (ORCPT PSARC-ext@Sun.Com); Tue, 14 Apr 2009 11:40:04 -0600 (MDT) Received: from [129.146.56.52] ([unknown] [129.146.56.52]) by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7.0-5.01 64bit (built Feb 19 2009)) with ESMTPSA id <0KI300LSJQERMF30@mail-amer.sun.com> for PSARC-ext@Sun.Com (ORCPT PSARC-ext@Sun.Com); Tue, 14 Apr 2009 11:40:03 -0600 (MDT) Date: Tue, 14 Apr 2009 10:34:53 -0700 From: John Forte Subject: Re: COMSTAR Infiniband SRP Target [PSARC 2009/111 FastTrack timeout 02/25/2009] In-reply-to: <49A5A208.1090004@sun.com> Sender: John.Forte@sun.com To: PSARC-ext@sun.com Cc: Peter Cudhea , Susan Gleeson Message-id: <49E4C93D.5070809@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <499B42EF.60801@sun.com> <49A5A208.1090004@sun.com> User-Agent: Thunderbird 2.0.0.17 (X11/20081014) Status: RO Content-Length: 1257 The project team would like to make an amendment to this fasttrack. I believe this qualifies for self-review and does not need a separate fasttrack. The project has not yet integrated. The amendment text below will be added to the case materials directory in addition to the updated fasttrack. Let me know if there are any technical objections or if a separate fasttrack would be preferred. - John ---- This text modifies PSARC 2009/111 to remove the SMF service for the IB Device Management Agent (svc:/system/ibdma:default). Instead, IBDMA will be managed as a subcomponent of the SRP Target service svc:/system/ibdma/target:default. IBDMA will be active when the SMF service for ibsrp/target is enabled. In addition, after consultation with the IB group, the name of the constant IBT_DMA has changed to IBT_DM_AGENT. The meaning of the constant has not changed. The clarified sentences about the implementation of the IB DM Agent in 4.2.2 should read: The IB Device Manager agent will be implemented as a miscellaneous kernel module 'ibdma' under /kernel/misc. This is consistent with other IB modules and clients. The IBDMA module will not be separately managed, but will be managed as a subcomponent of SRPT.