From randyf@sac.sfbay.sun.com Fri Apr 11 15:23:55 2008 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m3BMNtJJ027427 for ; Fri, 11 Apr 2008 15:23:55 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m3BMNqHW016112; Fri, 11 Apr 2008 23:23:54 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JZ60001RM7TKD00@nwk-avmta-1.sfbay.Sun.COM>; Fri, 11 Apr 2008 15:23:53 -0700 (PDT) Received: from dm-sfbay-02.sfbay.sun.com ([129.146.11.31]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JZ600J2LM7S0A30@nwk-avmta-1.sfbay.Sun.COM>; Fri, 11 Apr 2008 15:23:52 -0700 (PDT) Received: from sac.sfbay.sun.com (new-sac.SFBay.Sun.COM [129.146.175.65]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m3BMNpxZ001216; Fri, 11 Apr 2008 15:23:51 -0700 (PDT) Received: from sac.sfbay.sun.com (localhost [127.0.0.1]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m3BMNosN027422; Fri, 11 Apr 2008 15:23:50 -0700 (PDT) Received: (from randyf@localhost) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id m3BMNnRq027418; Fri, 11 Apr 2008 15:23:49 -0700 (PDT) Date: Fri, 11 Apr 2008 15:23:49 -0700 (PDT) From: Randy Fishel Subject: TCP/IP support for Intel I/OAT hardware [PSARC/2008/257 Self Review] To: PSARC-ext@sun.com Cc: Bruce.Curtis@sun.com, Mark.Johnson@sun.com Message-id: <200804112223.m3BMNnRq027418@sac.sfbay.sun.com> Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 Status: RO Content-Length: 3060 I am sponsoring this case on behalf of Bruce Curtis. It requests Patch binding, and is being marked as a Closed Approved Automatic as the only non-private interfaces are new driver names. If anyone belives this is not a self-review, it can be turned into a fasttrack. 1. Introduction - TCP/IP support for Intel I/OAT hardware Author: Bruce.Curtis@Sun.COM Date: 04/11/2008 All new Sun Intel based platforms have Intel I/OAT (I/O Acceleration Technology) hardware. The first such hardware is an on-systemboard asynchronous DMA engine code named Crystal Beach. Through a set of RFEs Solaris will use this hardware to implement TCP receive side zero CPU copy via a socket. This is a transparent implementation, i.e. no user-land application changes are required. No new APIs are introduced nor any changes to existing APIs. 2. Type of Release - Patch This is a patch as this new feature is targeting an S10 update. 3. Project Dependencies - None This new feature doesn't depend on any other project nor does any other project depend on it. 4. Project Architecture The new feature is implemented as specified in 4 RFEs: 6582335 TCP/IP receive-side zero CPU copy for support of Intel I/OAT hardware 6582330 sodirect - socket direct, for support of Intel I/OAT hardware 6582323 uioa - uio asynchronous, for support of Intel I/OAT hardware 6567008 driver for intel ioat v1 & v2 DMA engine needed Such that a new device driver "ioat" interfaces to the on-board I/OAT asynchronous DMA chip, the pseudo driver "dcopy" interfaces to the "ioat" driver, the uioa (uioa asynchronous) extensions to uio (user i/o) interfaces to dcopy, and lastly sockfs and tcp are modified to setup and use sodirect (socket direct) and uioa when a socket read()/recv() system call is made and I/OAT hardware is present and the user buffer is of sufficent size. Then, TCP payload data copy is accomplished via a DMA by the NIC (as it is for non I/OAT) into a kernel mblk_t's dblk_t data buffer then early in TCP payload processing a 2nd DMA is started from the dblk_t data buffer to the user-land buffer specified and the dblk_t is marked as such. The mblk_t is processed by the network stack framework as usual but before the read()/recv() returns sockfs checks for the I/OAT DMA completion. 5. Public Interfaces - SUNWdcopy, sys/sodirect.h Package contains the "ioat" device driver and "dcopy" pseudo driver. A new include file "sys/sodirect.h" is added to the SUNWhea package as it is included by "sys/sonode.h" and "sys/uio.h". 6. Interface Specification The commitment level for "SUNWdcopy" and the driver names "ioat" and "dcopy" are: stable The include file "sys/sodirect.h", and any interfaces contained there in, are: project private 7. Resources and Schedule 7.4. Steering Committee requested information 7.4.1. Consolidation C-team Name: ON 7.5. ARC review type: Automatic 7.6. ARC Exposure: open