From cth@sac.sfbay.sun.com Tue Oct 9 15:33:24 2007 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l99MXO0F019970 for ; Tue, 9 Oct 2007 15:33:24 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l99MT0gL005282; Tue, 9 Oct 2007 16:29:06 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JPO00B0V1679I00@nwk-avmta-1.sfbay.Sun.COM>; Tue, 09 Oct 2007 15:30:07 -0700 (PDT) Received: from dm-sfbay-02.sfbay.sun.com ([129.146.11.31]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JPO009MU1649130@nwk-avmta-1.sfbay.Sun.COM>; Tue, 09 Oct 2007 15:30:04 -0700 (PDT) Received: from sac.sfbay.sun.com (new-sac.SFBay.Sun.COM [129.146.175.65]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id l99MU2aa005901; Tue, 09 Oct 2007 15:30:02 -0700 (PDT) Received: from sac.sfbay.sun.com (localhost [127.0.0.1]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l99MXE5Y019965; Tue, 09 Oct 2007 15:33:14 -0700 (PDT) Received: (from cth@localhost) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id l99MXES2019961; Tue, 09 Oct 2007 15:33:14 -0700 (PDT) Date: Tue, 09 Oct 2007 15:33:14 -0700 (PDT) From: Christopher Horne Subject: dma-max-arch scsi capability [PSARC/2007/589 FastTrack timeout 10/16/2007] To: PSARC-ext@sun.com Cc: chris.horne@sun.com, eric.taylor@sun.com, grant.zhang@sun.com, mark.johnson@sun.com, randall.ralphs@sun.com Message-id: <200710092233.l99MXES2019961@sac.sfbay.sun.com> Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 4457 I am sponsoring the following fasttrack for Mark Johnson, requesting patch binding and a timeout of 10/16/2007. -Chris Template Version: @(#)sac_nextcase 1.64 07/13/07 SMI 1. Introduction 1.1 Project/Component Working Name: dma-max-arch scsi capability 1.2 Name of Document Author/Supplier: Author: Mark Johnson 1.3 Date of This Document: Tue Oct 9 10:36:08 MDT 2007 4. Technical Description 4.1 Introduction This case introduces the 'dma-max-arch' scsi capability, and is the first of two fasttracks to address st tape driver performance on x86. dma-max-arch scsi capability bp_copyin()/bp_copyout() There is a recently escalated bug outstanding for this issue. 6567168 s10 x86 st tape driver performance issue http://monaco.sfbay/detail.jsf?cr=6567168 4.2 Background Most tape drives cannot handle partial DMAs. An entire tape block must be transferred in a single DMA. For our SPARC based systems, this is relatively simple. Since these systems have an IOMMU, the only real consideration is the underlying HBA's maximum DMA size, which is returned via the 'dma-max' scsi capability. For current x86 bases systems, this becomes more complicated. Not only can the maximum DMA be limited by the underlying HBA's maximum DMA size, but it can also be limited by the DMA engines scatter/gather list constraints (if the memory is completely fragmented). Today on x86, the st tape driver will allocate physically contiguous memory and then bp_mapin/bcopy/bp_mapout all transfers where the tape blocksize is greater than 64KBytes. There are two parts to the solution. The first is to provide a way for the st driver to query what the DMA constraints of the HBA are, taking the sgllen into account. The second is to provide a 64-bit optimized bp copy for block sizes which are too large to fit within the HBA's sgllen constraints, but are within the HBA's maximum DMA size. For example, the ST driver may find out that the maximum DMA supported by the HBA ('dma-max') is 4M and the maximum DMA supported by the HBA/system is 1M ('dma-max-arch'). The st driver can then allow any blocksize <= 1M to go directly to the HBA and then use the optimized copy for block sizes greater than 1M and less than or equal to 4M. Today it does an un-optimized copy for block sizes greater than 64K. This case addresses the first part of the problem. It provides a new 'dma-max-arch' SCSI capability which will return additional DMA arch constraints. If there are no additional constraints (e.g. on SPARC), scsi_ifgetcap() will returned undefined for 'dma-max-arch'. An implementation note of interest, the 'dma-max-arch' capability is implemented by the scsi_ifgetcap(9F) implementation itself. It does not rely on a tran_getcap/tran_setcap(9E) response from the HBA. 4.3 Interface Table INTERFACE COMMITMENT LEVEL COMMENT 'dma-max-arch' Committed new scsi_ifgetcap(9F) capability 4.4 Man page changes See below. 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: FastTrack 6.6. ARC Exposure: open A.1 man page changes for scsi_ifgetcap(9F) ... dma-max Maximum dma transfer size that is supported by the host adapter. + dma-max-arch Maximum dma transfer size that is + supported by system. Takes the host + adapter and system architecture into + account. This is useful for target + drivers which don't support partial DMAs + on systems which don't have an IOMMU. In + this case, the DMA can also be limited by + the host adapters scatter/gather list + constraints. + + The 'dma-max-arch' capability is not + settable. It is implemented with + scsi_ifgetcap(9F) and does not rely on a + tran_getcap(9E) response from the HBA. + + msg-out Message out capability that is sup- ported by the host adapter: 0 dis- ables, 1 enables. ... From Chris.Horne@sun.com Wed Oct 10 10:48:00 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l9AHlxHQ016862 for ; Wed, 10 Oct 2007 10:47:59 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l9AHifaM002178 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 10 Oct 2007 18:44:43 +0100 (BST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JPP00L0HIMI8100@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.Com); Wed, 10 Oct 2007 11:44:42 -0600 (MDT) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JPP00HSXIMHWK50@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.Com); Wed, 10 Oct 2007 11:44:41 -0600 (MDT) Received: from fe-amer-10.sun.com ([192.18.109.80]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l9AHifwZ002061 for ; Wed, 10 Oct 2007 17:44:41 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JPP00001H7FLL00@mail-amer.sun.com> (original mail from Chris.Horne@Sun.COM) for PSARC-ext@Sun.Com (ORCPT PSARC-ext@Sun.Com); Wed, 10 Oct 2007 11:44:41 -0600 (MDT) Received: from sun.com ([129.150.33.166]) by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JPP00A7RIMCE580@mail-amer.sun.com>; Wed, 10 Oct 2007 11:44:37 -0600 (MDT) Date: Wed, 10 Oct 2007 11:44:36 -0600 From: Chris Horne Subject: Re: dma-max-arch scsi capability [PSARC/2007/589 FastTrack timeout 10/16/2007] In-reply-to: <200710092233.l99MXES2019961@sac.sfbay.sun.com> Sender: Chris.Horne@sun.com To: Christopher Horne Cc: PSARC-ext@sun.com, Eric.Taylor@sun.com, Grant.Zhang@sun.com, Mark.Johnson@sun.com, Randall.Ralphs@sun.com Message-id: <470D0F84.5030200@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-Accept-Language: en-us, en X-PMX-Version: 5.2.0.264296 References: <200710092233.l99MXES2019961@sac.sfbay.sun.com> User-Agent: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.4) Gecko/20040414 Status: RO Content-Length: 136 > 1.1 Project/Component Working Name: > > dma-max-arch scsi capability This case was approved at today's PSARC meeting. -Chris