From Ted.Kim@sun.com Fri Nov 6 14:39:06 2009 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nA6Md6ET007584 for ; Fri, 6 Nov 2009 14:39:06 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id nA6Md2qe035305 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Fri, 6 Nov 2009 15:39:06 -0700 (MST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KSP00L01LL5IO00@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Fri, 06 Nov 2009 14:39:05 -0800 (PST) Received: from sca-es-mail-2.sun.com ([192.18.43.133]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KSP00GXKLL42660@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Fri, 06 Nov 2009 14:39:04 -0800 (PST) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id nA6Md4IK019324 for ; Fri, 06 Nov 2009 14:39:04 -0800 (PST) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KSP00A00L810B00@fe-sfbay-10.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Fri, 06 Nov 2009 14:39:04 -0800 (PST) Received: from [129.153.3.74] ([unknown] [129.153.3.74]) by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KSP00MMALKKNCD0@fe-sfbay-10.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Fri, 06 Nov 2009 14:38:45 -0800 (PST) Date: Fri, 06 Nov 2009 14:36:04 -0800 From: "Ted H. Kim" Subject: Open Fabrics User Verbs (OFUV) primary kernel components [PSARC/2009/421 FastTrack timeout 11/13/2009] Sender: Ted.Kim@sun.com To: psarc-ext@sun.com Message-id: <4AF4A4D4.6070907@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 User-Agent: Thunderbird 2.0.0.23 (X11/20090910) Status: RO Content-Length: 9005 Note to PSARC admin folks: I may need manual intervention to get this into the agenda. (As far as I can tell, the tools don't support a fasttrack using an existing case with one-pager already in place.) Thanks, -ted ++++ This information is Copyright 2009 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: Open Fabrics User Verbs (OFUV) primary kernel components 1.2. Name of Document Author/Supplier: Author: Brendan Doyle 1.3 Date of This Document: 6 November, 2009 4. Technical Description Open Fabrics User Verbs (OFUV) primary kernel components ======================================================== Table of Contents ----------------- I. Introduction II. Summary of Interfaces A. Exported OFED RDMA CM APIs B. Imported Contracted Project Private IBTF APIs C. New Imported IBTF API III. Description of Interfaces A. OFED RDMA CM APIs B. Contracts for private IBTF APIs C. Updated IBTF API IV. Summary of changes by man page I. Introduction --------------- In Linux, the most popular InfiniBand (IB) OS-bypass framework is the Open Fabrics User Verbs (OFUV) framework from the Open Fabrics Enterprise Distribution (OFED). The OFUV API itself is modeled to a large degree on the KPI of OFED, used by OFED Linux kernel modules. Many of the calls originating in userland eventually join kernel calls in common code further down the stack in the OFED framework. In Solaris, OFUV is being ported over in two parts: kernel and userland. The kernel part is more fully described in the one-pager of this case. The userland portion is a companion project which delivers the open source libraries. Because of the similarity of the OFUV API to the OFED KPI and their largely common back-end, there is an opportunity to port selected portions of OFED KPI along the way to accelerate development of certain key applications. In particular, our business objective is to port what is necessary to satisfy some of the requirements of Oracle's Reliable Datagram Sockets v3 (RDSv3, used in Exadata 2) and the Lustre Solaris port. Both applications are originally written for Linux OFED and benefit considerably from porting a certain facility known as the "RDMA-CM", which provides a KPI to manage IB connections. This KPI would be an alternative compatibility KPI to the one we already have in IBTF (Solaris IB framework). The kernel OFUV project is being delivered in phases. This phase provides the RDMA-CM KPI for the kernel applications mentioned above and lays the foundation for later phases. The actual OS-bypass functionality is not enabled in this phase, but code is architected with this goal in mind and as a result ends up being distributed in a number of driver modules that match the architecture of the final phase of the project. A later fasttrack will describe interface enabled and used for the support of the OS-bypass functionality. In summary, this fast track describes the RDMA-CM interfaces delivered in this first phase of the OFUV kernel project. References: o Open Fabrics User Verbs (OFUV) - primary kernel components PSARC/2009/421 one-pager http://sac.sfbay/PSARC/2009/421/20090731_brendan.doyle o IBTF: InfiniBand Transport Framework PSARC/2002/132 o RDS - Reliable Datagram Service PSARC/2006/356 o Kernel RDMA CM API Architecture and use: materials directory: ofuv_rdma_arch.txt o Solaris Open Fabrics User Verbs Architecture Document: materials directory: solaris_ofuv_arch.pdf o OFUV Implementation Details: materials directory: OFUVImplementationDetails.pdf II. Summary of Interfaces ------------------------- This case asserts a micro/patch binding. A. Exported OFED RDMA CM APIs - ON Consolidation Private rdma_accept() - OFED Defined rdma_bind_addr() - OFED Defined rdma_cm_event_handler() - OFED Defined rdma_connect() - OFED Defined rdma_create_id() - OFED Defined rdma_create_qp() - OFED Defined rdma_destroy_id() - OFED Defined rdma_destroy_qp() - OFED Defined rdma_disconnect() - OFED Defined rdma_init_qp_attr() - OFED Defined rdma_join_multicast() - OFED Defined rdma_leave_multicast() - OFED Defined rdma_listen() - OFED Defined rdma_reject() - OFED Defined rdma_resolve_addr() - OFED Defined rdma_resolve_route() - OFED Defined ib_get_ibt_channel_hdl() - Solaris Extension ib_get_ibt_hca_hdl() - Solaris Extension B. Imported Contracted Project Private IBTF APIs These two calls are private IBTF interfaces used in this project by contract: ibt_ofuvcm_get_req_data() ibt_ofuvcm_proceed() Additionally a Contracted Project Private IBTF interface flag is added to the ibt_open_rc_channel(9f) function as follows: IBT_OCHAN_OFUV Indicates this channel is for an Open Fabric User Verbs (OFUV) consumer. IBTF does not flush the QP associated with channel when a DREQ is received for OFUV channels. C. New Imported IBTF API IBT_GENERIC_MISC - ON Consolidation Private (IBTF Transport Interface) add new value to ibt_clnt_class_t arg of ibt_attach(9f) III. Description of Interfaces ------------------------------ A. OFED RDMA CM APIs This project provides the OFED kernel RDMA CM interfaces defined in the rdma_cm.h OFED header, with a number of Solaris specific extensions required in order to interface into IBTF (which map from "CM ID" concept to the related IBTF handles). The 'sol_ofs' kernel module exports the OFED RDMA CM interfaces to kernel consumers, and translates the OFED APIs into Solaris equivalent IBTF APIs. See the provided man pages (in the materials/man_pages directory) for details on each API. B. Contract for private IBTF APIs See the case directory for contract (contract-01.txt) to use the project private IBTF APIs. C. Updated IBTF API To support this framework, a new client class(IBT_GENERIC_MISC) is added to the list of support IBTF client classes (ibt_clnt_class_t). This change is documented in the revised man pages for ibt_attach.9f and ibt_clnt_modinfo_t.9s in the materials/man_pages directory. IV. Summary of changes by man page ---------------------------------- The OFED manual pages for the RDMA CM kernel APIs are taken from OFED and converted to Solaris conventions. A few new man pages for the Solaris specific extension are also provided. Modified versions of existing man pages have change bars. All new or changed man pages can be found in the case materials/man_pages directory. Man page Disposition Reasons for change (sorted by section and name) (subsection of III) ------------------------------------------------------------------ sol_ofs(7D) new A sol_uverbs(7D) new A sol_ucma(7d) new A rdmacm(9) new A rdma_cm_event_handler(9E) new A ib_get_ibt_channel_hdl(9F) new A ib_get_ibt_hca_hdl(9F) new A ibt_attach(9F) changed C rdma_accept(9F) new A rdma_bind_addr(9F) new A rdma_connect(9F) new A rdma_create_id(9F) new A rdma_create_qp(9F) new A rdma_destroy_id(9F) new A rdma_destroy_qp(9F) new A rdma_disconnect(9F) new A rdma_init_qp_attr(9F) new A rdma_join_multicast(9F) new A rdma_leave_multicast(9F) new A rdma_listen(9F) new A rdma_reject(9F) new A rdma_resolve_addr(9F) new A rdma_resolve_route(9F) new A ibt_clnt_modinfo_t(9S) changed C 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: FastTrack 6.6. ARC Exposure: open -- Ted H. Kim Sun Microsystems, Inc. ted.kim@sun.com 222 North Sepulveda Blvd., 10th Floor (310) 341-1116 El Segundo, CA 90245 (310) 341-1120 FAX From gww@eng.sun.com Fri Nov 6 15:01:16 2009 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nA6N1FcU008472 for ; Fri, 6 Nov 2009 15:01:15 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id nA6N1Fl5024144 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Fri, 6 Nov 2009 15:01:15 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KSP00B01MM1LG00@nwk-avmta-1.sfbay.Sun.COM> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Fri, 06 Nov 2009 15:01:13 -0800 (PST) Received: from dm-eng-02.sfbay.sun.com ([129.146.11.32]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KSP0095ZMM14J70@nwk-avmta-1.sfbay.Sun.COM> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Fri, 06 Nov 2009 15:01:13 -0800 (PST) Received: from marduk.eng.sun.com (marduk.SFBay.Sun.COM [129.146.108.224]) by dm-eng-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id nA6N1D8M004724; Fri, 06 Nov 2009 15:01:13 -0800 (PST) Received: from marduk.eng.sun.com (localhost [127.0.0.1]) by marduk.eng.sun.com (8.13.6+Sun/8.12.11) with ESMTP id nA6N17JI002642; Fri, 06 Nov 2009 15:01:07 -0800 (PST) Received: (from gww@localhost) by marduk.eng.sun.com (8.13.6+Sun/8.12.11/Submit) id nA6N16tB002641; Fri, 06 Nov 2009 15:01:06 -0800 (PST) Date: Fri, 06 Nov 2009 15:01:06 -0800 (PST) From: Gary Winiger Subject: Re: Open Fabrics User Verbs (OFUV) primary kernel components [PSARC/2009/421 FastTrack timeout 11/13/2009] To: psarc-ext@sun.com, ted.kim@sun.com Message-id: <200911062301.nA6N16tB002641@marduk.eng.sun.com> Content-transfer-encoding: 7BIT X-Sun-Charset: US-ASCII X-PMX-Version: 5.4.1.325704 Status: RO Content-Length: 424 Ted, > Note to PSARC admin folks: I may need manual > intervention to get this into the agenda. > (As far as I can tell, the tools don't support > a fasttrack using an existing case with > one-pager already in place.) Not sure what you're asking. If you have a case that needs an owner and it is a fast track, you need to directly find an owner from the list of available owners (members, intern, licensees). Gary.. From Ted.Kim@Sun.COM Fri Nov 6 15:05:22 2009 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nA6N5MFP008724 for ; Fri, 6 Nov 2009 15:05:22 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id nA6N5M6s025238 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Fri, 6 Nov 2009 15:05:22 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KSP00J01MSYXN00@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Fri, 06 Nov 2009 16:05:22 -0700 (MST) Received: from sca-es-mail-2.sun.com ([192.18.43.133]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KSP000SVMSXMKA0@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Fri, 06 Nov 2009 16:05:21 -0700 (MST) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id nA6N5LUP021654 for ; Fri, 06 Nov 2009 15:05:21 -0800 (PST) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KSP00H00MN35T00@fe-sfbay-10.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Fri, 06 Nov 2009 15:05:21 -0800 (PST) Received: from [129.153.3.74] ([unknown] [129.153.3.74]) by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KSP004LDMSW8B50@fe-sfbay-10.sun.com>; Fri, 06 Nov 2009 15:05:21 -0800 (PST) Date: Fri, 06 Nov 2009 15:02:40 -0800 From: "Ted H. Kim" Subject: Re: Open Fabrics User Verbs (OFUV) primary kernel components [PSARC/2009/421 FastTrack timeout 11/13/2009] In-reply-to: <200911062301.nA6N16tB002641@marduk.eng.sun.com> Sender: Ted.Kim@Sun.COM To: Gary Winiger Cc: psarc-ext@Sun.COM Message-id: <4AF4AB10.3080301@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200911062301.nA6N16tB002641@marduk.eng.sun.com> User-Agent: Thunderbird 2.0.0.23 (X11/20090910) Status: RO Content-Length: 881 I am the licensee. But to submit this case, I had to do it manually (edit IAM file and email). But doing so means some internal databases don't get updated. So I am guessing it won't show up in the PSARC agenda file. -ted Gary Winiger wrote: > Ted, > >> Note to PSARC admin folks: I may need manual >> intervention to get this into the agenda. >> (As far as I can tell, the tools don't support >> a fasttrack using an existing case with >> one-pager already in place.) > > Not sure what you're asking. If you have a case that needs > an owner and it is a fast track, you need to directly find > an owner from the list of available owners (members, intern, > licensees). > > Gary.. -- Ted H. Kim Sun Microsystems, Inc. ted.kim@sun.com 222 North Sepulveda Blvd., 10th Floor (310) 341-1116 El Segundo, CA 90245 (310) 341-1120 FAX From gww@eng.sun.com Fri Nov 6 15:09:46 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nA6N9k7q008942 for ; Fri, 6 Nov 2009 15:09:46 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id nA6N9P6i003932 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Sat, 7 Nov 2009 07:09:45 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KSP00C01N07Q600@nwk-avmta-1.sfbay.Sun.COM> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Fri, 06 Nov 2009 15:09:43 -0800 (PST) Received: from dm-eng-02.sfbay.sun.com ([129.146.11.32]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KSP0090BN064JB0@nwk-avmta-1.sfbay.Sun.COM> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Fri, 06 Nov 2009 15:09:42 -0800 (PST) Received: from marduk.eng.sun.com (marduk.SFBay.Sun.COM [129.146.108.224]) by dm-eng-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id nA6N9gwd009640; Fri, 06 Nov 2009 15:09:42 -0800 (PST) Received: from marduk.eng.sun.com (localhost [127.0.0.1]) by marduk.eng.sun.com (8.13.6+Sun/8.12.11) with ESMTP id nA6N9axv002658; Fri, 06 Nov 2009 15:09:36 -0800 (PST) Received: (from gww@localhost) by marduk.eng.sun.com (8.13.6+Sun/8.12.11/Submit) id nA6N9ao3002657; Fri, 06 Nov 2009 15:09:36 -0800 (PST) Date: Fri, 06 Nov 2009 15:09:36 -0800 (PST) From: Gary Winiger Subject: Re: Open Fabrics User Verbs (OFUV) primary kernel components [PSARC/2009/421 FastTrack timeout 11/13/2009] To: gww@eng.sun.com, ted.kim@sun.com Cc: psarc-ext@sun.com Message-id: <200911062309.nA6N9ao3002657@marduk.eng.sun.com> Content-transfer-encoding: 7BIT X-Sun-Charset: US-ASCII X-PMX-Version: 5.4.1.325704 Status: RO Content-Length: 215 > I am the licensee. As a licensee you should know what to do. Contact your mentor and RTM http://sac.eng.sun.com/arc/Processes/ARC-LicenseeDuties.html In general it's automagic it you use the tools. Gary.. From Ted.Kim@Sun.COM Fri Nov 6 15:29:57 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nA6NTutQ009343 for ; Fri, 6 Nov 2009 15:29:57 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id nA6NTmPe012744 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Sat, 7 Nov 2009 07:29:55 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KSP00F05NXUHF00@nwk-avmta-1.sfbay.Sun.COM> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Fri, 06 Nov 2009 15:29:54 -0800 (PST) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KSP00EALNXT6W30@nwk-avmta-1.sfbay.Sun.COM> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Fri, 06 Nov 2009 15:29:53 -0800 (PST) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id nA6NTrZ2027557 for ; Fri, 06 Nov 2009 15:29:53 -0800 (PST) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KSP00B00NWY7W00@fe-sfbay-09.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Fri, 06 Nov 2009 15:29:53 -0800 (PST) Received: from [129.153.3.74] ([unknown] [129.153.3.74]) by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KSP00IFJNXSEOD0@fe-sfbay-09.sun.com>; Fri, 06 Nov 2009 15:29:53 -0800 (PST) Date: Fri, 06 Nov 2009 15:27:12 -0800 From: "Ted H. Kim" Subject: Re: Open Fabrics User Verbs (OFUV) primary kernel components [PSARC/2009/421 FastTrack timeout 11/13/2009] In-reply-to: <200911062309.nA6N9ao3002657@marduk.eng.sun.com> Sender: Ted.Kim@Sun.COM To: Gary Winiger Cc: PSARC-ext@Sun.COM Message-id: <4AF4B0D0.8090608@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200911062309.nA6N9ao3002657@marduk.eng.sun.com> User-Agent: Thunderbird 2.0.0.23 (X11/20090910) Status: RO Content-Length: 850 I believe I have done as much as is possible in this situation. But I am willing to find out there was something else I should have done. As far as I can tell (by examining the scripts), the tools do not support the situation of wanting to use an existing case number for a fasttrack You can reuse the one-pager already filed. But if you don't want that to be the text of your fasttrack, you are stuck. -ted Gary Winiger wrote: >> I am the licensee. > > As a licensee you should know what to do. Contact your mentor > and RTM > http://sac.eng.sun.com/arc/Processes/ARC-LicenseeDuties.html > > In general it's automagic it you use the tools. > > Gary.. -- Ted H. Kim Sun Microsystems, Inc. ted.kim@sun.com 222 North Sepulveda Blvd., 10th Floor (310) 341-1116 El Segundo, CA 90245 (310) 341-1120 FAX From Nicolas.Williams@sun.com Fri Nov 6 15:43:42 2009 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nA6Nhfh2009673 for ; Fri, 6 Nov 2009 15:43:41 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id nA6NheS8063246; Fri, 6 Nov 2009 16:43:40 -0700 (MST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KSP00305OKS5B00@nwk-avmta-2.sfbay.sun.com>; Fri, 06 Nov 2009 15:43:40 -0800 (PST) Received: from binky.Central.Sun.COM ([129.153.128.104]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KSP00GPUOKR24B0@nwk-avmta-2.sfbay.sun.com>; Fri, 06 Nov 2009 15:43:39 -0800 (PST) Received: from binky.Central.Sun.COM (localhost [127.0.0.1]) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3) with ESMTP id nA6NdbFX010488; Fri, 06 Nov 2009 17:39:37 -0600 (CST) Received: (from nw141292@localhost) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3/Submit) id nA6Ndbom010487; Fri, 06 Nov 2009 17:39:37 -0600 (CST) Date: Fri, 06 Nov 2009 17:39:37 -0600 From: Nicolas Williams Subject: Re: Open Fabrics User Verbs (OFUV) primary kernel components [PSARC/2009/421 FastTrack timeout 11/13/2009] In-reply-to: <4AF4B0D0.8090608@sun.com> To: "Ted H. Kim" Cc: Gary Winiger , PSARC-ext@sun.com Message-id: <20091106233936.GM1105@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <200911062309.nA6N9ao3002657@marduk.eng.sun.com> <4AF4B0D0.8090608@sun.com> X-Authentication-warning: binky.Central.Sun.COM: nw141292 set sender to Nicolas.Williams@sun.com using -f User-Agent: Mutt/1.5.7i Status: RO Content-Length: 717 On Fri, Nov 06, 2009 at 03:27:12PM -0800, Ted H. Kim wrote: > I believe I have done as much as is > possible in this situation. But I > am willing to find out there was > something else I should have done. > > As far as I can tell (by examining > the scripts), the tools do not > support the situation of wanting to use > an existing case number for a fasttrack > > You can reuse the one-pager already filed. > But if you don't want that to be the > text of your fasttrack, you are stuck. FWICT you have these options: a) if the old case is open, and you're not changing it to be a full case, just send e-mail and place new contents in the case directory; or b) close the old case, file a new one. Nico -- From sebastien.roy@sun.com Wed Nov 11 12:14:28 2009 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nABKERle004303 for ; Wed, 11 Nov 2009 12:14:27 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id nABKEQXV009296 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Wed, 11 Nov 2009 12:14:27 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KSY0090FO82FM00@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 11 Nov 2009 13:14:26 -0700 (MST) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KSY0061KO82N320@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 11 Nov 2009 13:14:26 -0700 (MST) Received: from fe-amer-10.sun.com ([192.18.109.80]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id nABKEQh5014766 for ; Wed, 11 Nov 2009 20:14:26 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KSY00K00O5L2100@mail-amer.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 11 Nov 2009 13:14:26 -0700 (MST) Received: from [129.148.174.103] ([unknown] [129.148.174.103]) by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KSY00LAZO7RRQ00@mail-amer.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 11 Nov 2009 13:14:16 -0700 (MST) Date: Wed, 11 Nov 2009 15:11:38 -0500 From: Sebastien Roy Subject: Re: Open Fabrics User Verbs (OFUV) primary kernel components [PSARC/2009/421 FastTrack timeout 11/13/2009] In-reply-to: <4AF4A4D4.6070907@sun.com> Sender: sebastien.roy@sun.com To: "Ted H. Kim" Cc: psarc-ext@sun.com Message-id: <1257970298.2088.114.camel@strat> Organization: Sun Microsystems MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <4AF4A4D4.6070907@sun.com> Status: RO Content-Length: 2370 The ofuv_rdma_arch.txt architecture document shows librdmacm and libibvers as being part of this "Phase 1" delivery. Reading between the lines, it appears that this fast-track is only delivering part of what this document describes as "Phase 1". Is that correct? On Fri, 2009-11-06 at 14:36 -0800, Ted H. Kim wrote: > In particular, our business objective is to port what is necessary to > satisfy some of the requirements of Oracle's Reliable Datagram Sockets > v3 (RDSv3, used in Exadata 2) and the Lustre Solaris port. Both > applications are originally written for Linux OFED and benefit > considerably from porting a certain facility known as the "RDMA-CM", > which provides a KPI to manage IB connections. The consumers of the kernel APIs defined by this case will be 3rd party kernel modules, correct (at least initially)? If so, are you certain that you want the new rdma_*() interfaces to be Consolidation Private? It seems to be implied that these are part of an external specification and are documented. Perhaps they should be some form of Public... > This KPI would be > an alternative compatibility KPI to the one we already have in > IBTF (Solaris IB framework). This is somewhat understating the role of IBTF in the architecture. Isn't this KPI being implemented as a consumer of IBTF? Stating that it is an alternative would imply that one could obsolete IBTF and perhaps eventually remove it. > B. Imported Contracted Project Private IBTF APIs > These two calls are private IBTF interfaces used in this > project by contract: > > ibt_ofuvcm_get_req_data() > ibt_ofuvcm_proceed() > > Additionally a Contracted Project Private IBTF interface flag > is added to the ibt_open_rc_channel(9f) function as follows: > > IBT_OCHAN_OFUV > Indicates this channel is for an Open Fabric User Verbs (OFUV) > consumer. IBTF does not flush the QP associated with channel > when a DREQ is received for OFUV channels. Use of a contract often indicates that some piece of architecture is either missing/incomplete or mis-classified. Instead of using a contract, why not elevate the stability level of the two functions needed to the same level as ibt_open_rc_channel() (consolidation-private I think)? The consumer and supplier are both in ON, right? -Seb From Ted.Kim@Sun.COM Thu Nov 12 14:27:18 2009 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nACMRIc4022219 for ; Thu, 12 Nov 2009 14:27:18 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id nACMRIOR028957 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Thu, 12 Nov 2009 14:27:18 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KT000201P1IWW00@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 12 Nov 2009 14:27:18 -0800 (PST) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KT000MJRP1HGX40@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 12 Nov 2009 14:27:17 -0800 (PST) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id nACMRHH5007812 for ; Thu, 12 Nov 2009 14:27:17 -0800 (PST) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KT000300OTDME00@fe-sfbay-10.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 12 Nov 2009 14:27:17 -0800 (PST) Received: from [129.153.3.74] ([unknown] [129.153.3.74]) by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KT000B33P182X60@fe-sfbay-10.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 12 Nov 2009 14:27:10 -0800 (PST) Date: Thu, 12 Nov 2009 14:24:18 -0800 From: "Ted H. Kim" Subject: Re: Open Fabrics User Verbs (OFUV) primary kernel components [PSARC/2009/421 FastTrack timeout 11/13/2009] In-reply-to: <1257970298.2088.114.camel@strat> Sender: Ted.Kim@Sun.COM To: Sebastien Roy Cc: psarc-ext@Sun.COM Message-id: <4AFC8B12.7000008@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <4AF4A4D4.6070907@sun.com> <1257970298.2088.114.camel@strat> User-Agent: Thunderbird 2.0.0.23 (X11/20090910) Status: RO Content-Length: 3989 Seb, Sebastien Roy wrote: > The ofuv_rdma_arch.txt architecture document shows librdmacm and > libibvers as being part of this "Phase 1" delivery. Reading between the > lines, it appears that this fast-track is only delivering part of what > this document describes as "Phase 1". Is that correct? Yes, that is true. Now, we have subdivided Phase 1 further and added this delivery to help the porting of RDSv3 and Lustre. > On Fri, 2009-11-06 at 14:36 -0800, Ted H. Kim wrote: >> In particular, our business objective is to port what is necessary to >> satisfy some of the requirements of Oracle's Reliable Datagram Sockets >> v3 (RDSv3, used in Exadata 2) and the Lustre Solaris port. Both >> applications are originally written for Linux OFED and benefit >> considerably from porting a certain facility known as the "RDMA-CM", >> which provides a KPI to manage IB connections. > > The consumers of the kernel APIs defined by this case will be 3rd party > kernel modules, correct (at least initially)? If so, are you certain > that you want the new rdma_*() interfaces to be Consolidation Private? > It seems to be implied that these are part of an external specification > and are documented. Perhaps they should be some form of Public... While the consumers do have an origin based in Linux, these consumers are actually internal Sun projects and not being delivered by 3rd party external folks. The interfaces are documented to some extent externally. But for the moment, the main objective is get the applications done. We are not sure how far we want to go to keep them fully in sync with further developments from Linux, unless our key applications really require it. Also, as a "compatibility" layer to our IBTF interfaces, I see them as having the same audience. And to date, all of our kernel IB interfaces are Consolidation Private. OFUV userland will be a different story. But in the kernel, we want more control, at least for now. >> This KPI would be >> an alternative compatibility KPI to the one we already have in >> IBTF (Solaris IB framework). > > This is somewhat understating the role of IBTF in the architecture. > Isn't this KPI being implemented as a consumer of IBTF? Stating that it > is an alternative would imply that one could obsolete IBTF and perhaps > eventually remove it. Okay it's probably better to say this piece is a "compatibility" layer on top of IBTF for the RDMA-CM style communications management. And as you probably know, IBTF has lots more pieces than that. >> B. Imported Contracted Project Private IBTF APIs >> These two calls are private IBTF interfaces used in this >> project by contract: >> >> ibt_ofuvcm_get_req_data() >> ibt_ofuvcm_proceed() >> >> Additionally a Contracted Project Private IBTF interface flag >> is added to the ibt_open_rc_channel(9f) function as follows: >> >> IBT_OCHAN_OFUV >> Indicates this channel is for an Open Fabric User Verbs (OFUV) >> consumer. IBTF does not flush the QP associated with channel >> when a DREQ is received for OFUV channels. > > Use of a contract often indicates that some piece of architecture is > either missing/incomplete or mis-classified. Instead of using a > contract, why not elevate the stability level of the two functions > needed to the same level as ibt_open_rc_channel() (consolidation-private > I think)? The consumer and supplier are both in ON, right? We don't think any other consumer of IBTF is going to use these items. These are primarily to bridge some differences in design approach between the IBTF CM and RDMA-CM. And while appropriate to support the internal workings of RDMA-CM, we don't want apps to try to mix/use the two CM models at the same time. -ted -- Ted H. Kim Sun Microsystems, Inc. ted.kim@sun.com 222 North Sepulveda Blvd., 10th Floor (310) 341-1116 El Segundo, CA 90245 (310) 341-1120 FAX From Sebastien.Roy@sun.com Thu Nov 12 14:46:54 2009 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nACMks1b022655 for ; Thu, 12 Nov 2009 14:46:54 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id nACMkqZD036236 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Thu, 12 Nov 2009 15:46:53 -0700 (MST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KT000E0RPY5L200@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 12 Nov 2009 15:46:53 -0700 (MST) Received: from brmea-mail-2.sun.com ([192.18.98.43]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KT00088BPY4EE40@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 12 Nov 2009 15:46:52 -0700 (MST) Received: from fe-amer-09.sun.com ([192.18.109.79]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id nACMkqwk023806 for ; Thu, 12 Nov 2009 22:46:52 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KT000G00PJW1C00@mail-amer.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 12 Nov 2009 15:46:52 -0700 (MST) Received: from [129.148.174.103] ([unknown] [129.148.174.103]) by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KT0002O7PY40900@mail-amer.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 12 Nov 2009 15:46:52 -0700 (MST) Date: Thu, 12 Nov 2009 17:44:14 -0500 From: Sebastien Roy Subject: Re: Open Fabrics User Verbs (OFUV) primary kernel components [PSARC/2009/421 FastTrack timeout 11/13/2009] In-reply-to: <4AFC8B12.7000008@sun.com> Sender: Sebastien.Roy@sun.com To: "Ted H. Kim" Cc: psarc-ext@sun.com Message-id: <1258065854.12532.14.camel@strat> Organization: Sun Microsystems MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <4AF4A4D4.6070907@sun.com> <1257970298.2088.114.camel@strat> <4AFC8B12.7000008@sun.com> Status: RO Content-Length: 4224 On Thu, 2009-11-12 at 14:24 -0800, Ted H. Kim wrote: > Seb, > > Sebastien Roy wrote: > > The ofuv_rdma_arch.txt architecture document shows librdmacm and > > libibvers as being part of this "Phase 1" delivery. Reading between the > > lines, it appears that this fast-track is only delivering part of what > > this document describes as "Phase 1". Is that correct? > > Yes, that is true. Now, we have subdivided Phase 1 further > and added this delivery to help the porting of RDSv3 and Lustre. Okay, so the architecture delivered by this case is strictly what is specified in the mail log, and the additional architecture documents are for greater context. > > On Fri, 2009-11-06 at 14:36 -0800, Ted H. Kim wrote: > >> In particular, our business objective is to port what is necessary to > >> satisfy some of the requirements of Oracle's Reliable Datagram Sockets > >> v3 (RDSv3, used in Exadata 2) and the Lustre Solaris port. Both > >> applications are originally written for Linux OFED and benefit > >> considerably from porting a certain facility known as the "RDMA-CM", > >> which provides a KPI to manage IB connections. > > > > The consumers of the kernel APIs defined by this case will be 3rd party > > kernel modules, correct (at least initially)? If so, are you certain > > that you want the new rdma_*() interfaces to be Consolidation Private? > > It seems to be implied that these are part of an external specification > > and are documented. Perhaps they should be some form of Public... > > While the consumers do have an origin based in Linux, these consumers > are actually internal Sun projects and not being delivered > by 3rd party external folks. The interfaces are documented to some > extent externally. But for the moment, the main objective is > get the applications done. We are not sure how far we want to go > to keep them fully in sync with further developments > from Linux, unless our key applications really require it. > > Also, as a "compatibility" layer to our IBTF interfaces, I see > them as having the same audience. And to date, all of our > kernel IB interfaces are Consolidation Private. OFUV userland > will be a different story. But in the kernel, we want more > control, at least for now. Okay. > >> This KPI would be > >> an alternative compatibility KPI to the one we already have in > >> IBTF (Solaris IB framework). > > > > This is somewhat understating the role of IBTF in the architecture. > > Isn't this KPI being implemented as a consumer of IBTF? Stating that it > > is an alternative would imply that one could obsolete IBTF and perhaps > > eventually remove it. > > Okay it's probably better to say this piece is a "compatibility" > layer on top of IBTF for the RDMA-CM style communications > management. And as you probably know, IBTF has lots more pieces > than that. Yes indeed. > >> B. Imported Contracted Project Private IBTF APIs > >> These two calls are private IBTF interfaces used in this > >> project by contract: > >> > >> ibt_ofuvcm_get_req_data() > >> ibt_ofuvcm_proceed() > >> > >> Additionally a Contracted Project Private IBTF interface flag > >> is added to the ibt_open_rc_channel(9f) function as follows: > >> > >> IBT_OCHAN_OFUV > >> Indicates this channel is for an Open Fabric User Verbs (OFUV) > >> consumer. IBTF does not flush the QP associated with channel > >> when a DREQ is received for OFUV channels. > > > > Use of a contract often indicates that some piece of architecture is > > either missing/incomplete or mis-classified. Instead of using a > > contract, why not elevate the stability level of the two functions > > needed to the same level as ibt_open_rc_channel() (consolidation-private > > I think)? The consumer and supplier are both in ON, right? > > We don't think any other consumer of IBTF is going to > use these items. These are primarily to bridge some differences > in design approach between the IBTF CM and RDMA-CM. And > while appropriate to support the internal workings of RDMA-CM, > we don't want apps to try to mix/use the two CM models > at the same time. Okay, thanks. +1 on the case. -Seb From Ted.Kim@sun.com Thu Nov 12 15:41:07 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nACNf6X9023740 for ; Thu, 12 Nov 2009 15:41:07 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id nACNf0lW019883 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Fri, 13 Nov 2009 07:41:05 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KT00081LSGH2C00@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 12 Nov 2009 15:41:05 -0800 (PST) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KT000MG0SGHH5B0@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 12 Nov 2009 15:41:05 -0800 (PST) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id nACNf5ac014618 for ; Thu, 12 Nov 2009 15:41:05 -0800 (PST) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KT000J00SFVNJ00@fe-sfbay-09.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 12 Nov 2009 15:41:05 -0800 (PST) Received: from [129.153.3.74] ([unknown] [129.153.3.74]) by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KT00069FSGFUX40@fe-sfbay-09.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 12 Nov 2009 15:41:04 -0800 (PST) Date: Thu, 12 Nov 2009 15:38:13 -0800 From: "Ted H. Kim" Subject: Re: Open Fabrics User Verbs (OFUV) primary kernel components [PSARC/2009/421 FastTrack timeout 11/13/2009] In-reply-to: <1258065854.12532.14.camel@strat> Sender: Ted.Kim@sun.com To: Sebastien Roy Cc: PSARC-ext@sun.com Message-id: <4AFC9C65.40904@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <4AF4A4D4.6070907@sun.com> <1257970298.2088.114.camel@strat> <4AFC8B12.7000008@sun.com> <1258065854.12532.14.camel@strat> User-Agent: Thunderbird 2.0.0.23 (X11/20090910) Status: RO Content-Length: 805 Sebastien Roy wrote: >>> The ofuv_rdma_arch.txt architecture document shows librdmacm and >>> libibvers as being part of this "Phase 1" delivery. Reading between the >>> lines, it appears that this fast-track is only delivering part of what >>> this document describes as "Phase 1". Is that correct? >> Yes, that is true. Now, we have subdivided Phase 1 further >> and added this delivery to help the porting of RDSv3 and Lustre. > > Okay, so the architecture delivered by this case is strictly what is > specified in the mail log, and the additional architecture documents are > for greater context. yup. -ted -- Ted H. Kim Sun Microsystems, Inc. ted.kim@sun.com 222 North Sepulveda Blvd., 10th Floor (310) 341-1116 El Segundo, CA 90245 (310) 341-1120 FAX From Ted.Kim@sun.com Fri Nov 13 13:27:41 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nACNf6X9023740 for ; Thu, 12 Nov 2009 15:41:07 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id nACNf0lW019883 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Fri, 13 Nov 2009 07:41:05 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KT00081LSGH2C00@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 12 Nov 2009 15:41:05 -0800 (PST) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KT000MG0SGHH5B0@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 12 Nov 2009 15:41:05 -0800 (PST) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id nACNf5ac014618 for ; Thu, 12 Nov 2009 15:41:05 -0800 (PST) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KT000J00SFVNJ00@fe-sfbay-09.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 12 Nov 2009 15:41:05 -0800 (PST) Received: from [129.153.3.74] ([unknown] [129.153.3.74]) by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KT00069FSGFUX40@fe-sfbay-09.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 12 Nov 2009 15:41:04 -0800 (PST) Date: Thu, 12 Nov 2009 15:38:13 -0800 From: "Ted H. Kim" Subject: Re: Open Fabrics User Verbs (OFUV) primary kernel components [PSARC/2009/421 FastTrack timeout 11/13/2009] In-reply-to: <1258065854.12532.14.camel@strat> Sender: Ted.Kim@sun.com To: Sebastien Roy Cc: PSARC-ext@sun.com Message-id: <4AFC9C65.40904@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <4AF4A4D4.6070907@sun.com> <1257970298.2088.114.camel@strat> <4AFC8B12.7000008@sun.com> <1258065854.12532.14.camel@strat> User-Agent: Thunderbird 2.0.0.23 (X11/20090910) Status: RO Content-Length: 805 Sebastien Roy wrote: >>> The ofuv_rdma_arch.txt architecture document shows librdmacm and >>> libibvers as being part of this "Phase 1" delivery. Reading between the >>> lines, it appears that this fast-track is only delivering part of what >>> this document describes as "Phase 1". Is that correct? >> Yes, that is true. Now, we have subdivided Phase 1 further >> and added this delivery to help the porting of RDSv3 and Lustre. > > Okay, so the architecture delivered by this case is strictly what is > specified in the mail log, and the additional architecture documents are > for greater context. yup. -ted -- Ted H. Kim Sun Microsystems, Inc. ted.kim@sun.com 222 North Sepulveda Blvd., 10th Floor (310) 341-1116 El Segundo, CA 90245 (310) 341-1120 FAX From Ted.Kim@sun.com Fri Nov 13 13:33:32 2009 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nACMRIc4022219 for ; Thu, 12 Nov 2009 14:27:18 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id nACMRIOR028957 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Thu, 12 Nov 2009 14:27:18 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KT000201P1IWW00@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 12 Nov 2009 14:27:18 -0800 (PST) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KT000MJRP1HGX40@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 12 Nov 2009 14:27:17 -0800 (PST) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id nACMRHH5007812 for ; Thu, 12 Nov 2009 14:27:17 -0800 (PST) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KT000300OTDME00@fe-sfbay-10.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 12 Nov 2009 14:27:17 -0800 (PST) Received: from [129.153.3.74] ([unknown] [129.153.3.74]) by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KT000B33P182X60@fe-sfbay-10.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 12 Nov 2009 14:27:10 -0800 (PST) Date: Thu, 12 Nov 2009 14:24:18 -0800 From: "Ted H. Kim" Subject: Re: Open Fabrics User Verbs (OFUV) primary kernel components [PSARC/2009/421 FastTrack timeout 11/13/2009] In-reply-to: <1257970298.2088.114.camel@strat> Sender: Ted.Kim@sun.com To: Sebastien Roy Cc: psarc-ext@sun.com Message-id: <4AFC8B12.7000008@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <4AF4A4D4.6070907@sun.com> <1257970298.2088.114.camel@strat> User-Agent: Thunderbird 2.0.0.23 (X11/20090910) Status: RO Content-Length: 3989 Seb, Sebastien Roy wrote: > The ofuv_rdma_arch.txt architecture document shows librdmacm and > libibvers as being part of this "Phase 1" delivery. Reading between the > lines, it appears that this fast-track is only delivering part of what > this document describes as "Phase 1". Is that correct? Yes, that is true. Now, we have subdivided Phase 1 further and added this delivery to help the porting of RDSv3 and Lustre. > On Fri, 2009-11-06 at 14:36 -0800, Ted H. Kim wrote: >> In particular, our business objective is to port what is necessary to >> satisfy some of the requirements of Oracle's Reliable Datagram Sockets >> v3 (RDSv3, used in Exadata 2) and the Lustre Solaris port. Both >> applications are originally written for Linux OFED and benefit >> considerably from porting a certain facility known as the "RDMA-CM", >> which provides a KPI to manage IB connections. > > The consumers of the kernel APIs defined by this case will be 3rd party > kernel modules, correct (at least initially)? If so, are you certain > that you want the new rdma_*() interfaces to be Consolidation Private? > It seems to be implied that these are part of an external specification > and are documented. Perhaps they should be some form of Public... While the consumers do have an origin based in Linux, these consumers are actually internal Sun projects and not being delivered by 3rd party external folks. The interfaces are documented to some extent externally. But for the moment, the main objective is get the applications done. We are not sure how far we want to go to keep them fully in sync with further developments from Linux, unless our key applications really require it. Also, as a "compatibility" layer to our IBTF interfaces, I see them as having the same audience. And to date, all of our kernel IB interfaces are Consolidation Private. OFUV userland will be a different story. But in the kernel, we want more control, at least for now. >> This KPI would be >> an alternative compatibility KPI to the one we already have in >> IBTF (Solaris IB framework). > > This is somewhat understating the role of IBTF in the architecture. > Isn't this KPI being implemented as a consumer of IBTF? Stating that it > is an alternative would imply that one could obsolete IBTF and perhaps > eventually remove it. Okay it's probably better to say this piece is a "compatibility" layer on top of IBTF for the RDMA-CM style communications management. And as you probably know, IBTF has lots more pieces than that. >> B. Imported Contracted Project Private IBTF APIs >> These two calls are private IBTF interfaces used in this >> project by contract: >> >> ibt_ofuvcm_get_req_data() >> ibt_ofuvcm_proceed() >> >> Additionally a Contracted Project Private IBTF interface flag >> is added to the ibt_open_rc_channel(9f) function as follows: >> >> IBT_OCHAN_OFUV >> Indicates this channel is for an Open Fabric User Verbs (OFUV) >> consumer. IBTF does not flush the QP associated with channel >> when a DREQ is received for OFUV channels. > > Use of a contract often indicates that some piece of architecture is > either missing/incomplete or mis-classified. Instead of using a > contract, why not elevate the stability level of the two functions > needed to the same level as ibt_open_rc_channel() (consolidation-private > I think)? The consumer and supplier are both in ON, right? We don't think any other consumer of IBTF is going to use these items. These are primarily to bridge some differences in design approach between the IBTF CM and RDMA-CM. And while appropriate to support the internal workings of RDMA-CM, we don't want apps to try to mix/use the two CM models at the same time. -ted -- Ted H. Kim Sun Microsystems, Inc. ted.kim@sun.com 222 North Sepulveda Blvd., 10th Floor (310) 341-1116 El Segundo, CA 90245 (310) 341-1120 FAX From Sebastien.Roy@sun.com Fri Nov 13 13:33:59 2009 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nACMks1b022655 for ; Thu, 12 Nov 2009 14:46:54 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id nACMkqZD036236 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Thu, 12 Nov 2009 15:46:53 -0700 (MST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KT000E0RPY5L200@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 12 Nov 2009 15:46:53 -0700 (MST) Received: from brmea-mail-2.sun.com ([192.18.98.43]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KT00088BPY4EE40@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 12 Nov 2009 15:46:52 -0700 (MST) Received: from fe-amer-09.sun.com ([192.18.109.79]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id nACMkqwk023806 for ; Thu, 12 Nov 2009 22:46:52 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KT000G00PJW1C00@mail-amer.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 12 Nov 2009 15:46:52 -0700 (MST) Received: from [129.148.174.103] ([unknown] [129.148.174.103]) by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KT0002O7PY40900@mail-amer.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 12 Nov 2009 15:46:52 -0700 (MST) Date: Thu, 12 Nov 2009 17:44:14 -0500 From: Sebastien Roy Subject: Re: Open Fabrics User Verbs (OFUV) primary kernel components [PSARC/2009/421 FastTrack timeout 11/13/2009] In-reply-to: <4AFC8B12.7000008@sun.com> Sender: Sebastien.Roy@sun.com To: "Ted H. Kim" Cc: psarc-ext@sun.com Message-id: <1258065854.12532.14.camel@strat> Organization: Sun Microsystems MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <4AF4A4D4.6070907@sun.com> <1257970298.2088.114.camel@strat> <4AFC8B12.7000008@sun.com> Status: RO Content-Length: 4224 On Thu, 2009-11-12 at 14:24 -0800, Ted H. Kim wrote: > Seb, > > Sebastien Roy wrote: > > The ofuv_rdma_arch.txt architecture document shows librdmacm and > > libibvers as being part of this "Phase 1" delivery. Reading between the > > lines, it appears that this fast-track is only delivering part of what > > this document describes as "Phase 1". Is that correct? > > Yes, that is true. Now, we have subdivided Phase 1 further > and added this delivery to help the porting of RDSv3 and Lustre. Okay, so the architecture delivered by this case is strictly what is specified in the mail log, and the additional architecture documents are for greater context. > > On Fri, 2009-11-06 at 14:36 -0800, Ted H. Kim wrote: > >> In particular, our business objective is to port what is necessary to > >> satisfy some of the requirements of Oracle's Reliable Datagram Sockets > >> v3 (RDSv3, used in Exadata 2) and the Lustre Solaris port. Both > >> applications are originally written for Linux OFED and benefit > >> considerably from porting a certain facility known as the "RDMA-CM", > >> which provides a KPI to manage IB connections. > > > > The consumers of the kernel APIs defined by this case will be 3rd party > > kernel modules, correct (at least initially)? If so, are you certain > > that you want the new rdma_*() interfaces to be Consolidation Private? > > It seems to be implied that these are part of an external specification > > and are documented. Perhaps they should be some form of Public... > > While the consumers do have an origin based in Linux, these consumers > are actually internal Sun projects and not being delivered > by 3rd party external folks. The interfaces are documented to some > extent externally. But for the moment, the main objective is > get the applications done. We are not sure how far we want to go > to keep them fully in sync with further developments > from Linux, unless our key applications really require it. > > Also, as a "compatibility" layer to our IBTF interfaces, I see > them as having the same audience. And to date, all of our > kernel IB interfaces are Consolidation Private. OFUV userland > will be a different story. But in the kernel, we want more > control, at least for now. Okay. > >> This KPI would be > >> an alternative compatibility KPI to the one we already have in > >> IBTF (Solaris IB framework). > > > > This is somewhat understating the role of IBTF in the architecture. > > Isn't this KPI being implemented as a consumer of IBTF? Stating that it > > is an alternative would imply that one could obsolete IBTF and perhaps > > eventually remove it. > > Okay it's probably better to say this piece is a "compatibility" > layer on top of IBTF for the RDMA-CM style communications > management. And as you probably know, IBTF has lots more pieces > than that. Yes indeed. > >> B. Imported Contracted Project Private IBTF APIs > >> These two calls are private IBTF interfaces used in this > >> project by contract: > >> > >> ibt_ofuvcm_get_req_data() > >> ibt_ofuvcm_proceed() > >> > >> Additionally a Contracted Project Private IBTF interface flag > >> is added to the ibt_open_rc_channel(9f) function as follows: > >> > >> IBT_OCHAN_OFUV > >> Indicates this channel is for an Open Fabric User Verbs (OFUV) > >> consumer. IBTF does not flush the QP associated with channel > >> when a DREQ is received for OFUV channels. > > > > Use of a contract often indicates that some piece of architecture is > > either missing/incomplete or mis-classified. Instead of using a > > contract, why not elevate the stability level of the two functions > > needed to the same level as ibt_open_rc_channel() (consolidation-private > > I think)? The consumer and supplier are both in ON, right? > > We don't think any other consumer of IBTF is going to > use these items. These are primarily to bridge some differences > in design approach between the IBTF CM and RDMA-CM. And > while appropriate to support the internal workings of RDMA-CM, > we don't want apps to try to mix/use the two CM models > at the same time. Okay, thanks. +1 on the case. -Seb From Ted.Kim@sun.com Mon Nov 16 09:56:43 2009 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id nAGHugKs010180 for ; Mon, 16 Nov 2009 09:56:42 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id nAGHufeg007673 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Mon, 16 Nov 2009 10:56:42 -0700 (MST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KT70040BR6HH500@nwk-avmta-1.sfbay.Sun.COM> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Mon, 16 Nov 2009 09:56:41 -0800 (PST) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KT700HODR6AFKE0@nwk-avmta-1.sfbay.Sun.COM> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Mon, 16 Nov 2009 09:56:39 -0800 (PST) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id nAGHuYin029304 for ; Mon, 16 Nov 2009 09:56:34 -0800 (PST) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KT700900QR6PJ00@fe-sfbay-10.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Mon, 16 Nov 2009 09:56:34 -0800 (PST) Received: from [129.153.3.74] ([unknown] [129.153.3.74]) by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KT700881R64IY40@fe-sfbay-10.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Mon, 16 Nov 2009 09:56:33 -0800 (PST) Date: Mon, 16 Nov 2009 09:53:32 -0800 From: "Ted H. Kim" Subject: Re: Open Fabrics User Verbs (OFUV) primary kernel components [PSARC/2009/421 FastTrack timeout 11/13/2009] In-reply-to: <4AFC8B12.7000008@sun.com> Sender: Ted.Kim@sun.com To: psarc-ext@sun.com Message-id: <4B01919C.5020206@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <4AF4A4D4.6070907@sun.com> <1257970298.2088.114.camel@strat> <4AFC8B12.7000008@sun.com> User-Agent: Thunderbird 2.0.0.23 (X11/20090910) Status: RO Content-Length: 355 Folks, The timer has expired on the case, and the questions have been answered. Also, the the case got the required +1. So with that I am closing it as approved. -ted -- Ted H. Kim Sun Microsystems, Inc. ted.kim@sun.com 222 North Sepulveda Blvd., 10th Floor (310) 341-1116 El Segundo, CA 90245 (310) 341-1120 FAX