Template Version: @(#)onepager.txt 1.35 07/11/07 SMI This information is Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved. 1. Introduction 1.1. Project/Component Working Name: Fibre Channel over InfiniBand (FCoIB) Solaris support 1.2. Name of Document Author/Supplier: Lars Paul Huse 1.3. Date of This Document: 5/18/10 1.3.1. Date this project was conceived: As in 1.3 1.4. Name of Major Document Customer(s)/Consumer(s): 1.4.1. The PAC or CPT you expect to review your project: PTL Project 7506 1.4.2. The ARC(s) you expect to review your project: PSARC (Platform Software Architecture Review Committee) The OpenSolaris Systems Architecture community 1.4.3. The Director/VP who is "Sponsoring" this project: Eivind.Rongved@Sun.COM Raju.Penumatcha@Sun.COM 1.4.4. The name of your business unit: System - NSN (Netra Systems & Networking) European Design Center 1.5. Email Aliases: 1.5.1. Responsible Manager: Eivind.Rongved@Sun.COM 1.5.2. Responsible Engineer: Lars.Paul.Huse@Sun.COM 1.5.3. Marketing Manager: Stephen.Callahan@Sun.COM 1.5.4. Interest List: Solaris-XoIB@Sun.COM 2. Project Summary 2.1. Project Description: This project adds FCoIB (Fibre-Channel over InfiniBand) capability to Leadville, the Solaris fibre-channel initiator stack. It leverages the I/O consolidation capabilities of advanced InfiniBand VPI host hardware and fabric attached InfiniBand to Fibre-Channel gateway hardware to share gateway fibre-channel facing ports. Using Fibre Channel (FC) over InfiniBand (FCoIB) enable the use of cost-effective bridges for connecting to FC SANs. With FCoIB, InfiniBand host adapters will encapsulate FC packets in hardware on the host and move the encapsulated FC packages over the InfiniBand fabric. Attaching InfiniBand to FC gateways (e.g. Mellanox BridgeX as used in the Oracle Magnum product line) to the IB fabric, will then enable access to the FC SAN. The project targets to port/reimplement the existing FCoIB in Ofed / Linux to Solaris. 2.2. Risks and Assumptions: FCoIB is a replacement for local disc access and is hence subjected to high quality requirement. Performing extensive targeted testing for FCoIB will be a mitigation for this. 3. Business Summary This project enables Solaris hosts that are connected to an InfiniBand fabric to access a SAN via the Sun InfiniBand Gateway. Providing "gateway" functionality in Solaris allows Solaris hosts to be "full-fledged citizens" in InfiniBand cloud deployments. 3.2. Market/Requester: Customers leveraging the performance advantages of the InfiniBand fabric in enterprise grid deployments. 3.3. Business Justification: This project enables SPARC and Solaris x86 servers to be included in future enterprise and cloud offerings which leverage InfiniBand as the primary interconnect for the solution. 3.4. Competitive Analysis: Intentionally left blank 3.5. Opportunity Window/Exposure: InfiniBand based appliances have been released by Oracle (Exadata) which leverage the InfiniBand fabric. 3.6. How will you know when you are done?: 1. Servers leveraging FCoIB for Fibre Channel connectivity will have functionality, performance, and RAS equivalent to the Solaris native Fibre Channel implementation. 2. Interoperability and 3rd party certifications (Sun SAN qualification, EMC E-Labs, \'85) will be completed to ensure customer acceptance and supportability of the solution. 4. Technical Description: 4.1. Details: The FCoIB function is comprised of two key components. The management infrastructure to discover fabric attached shared InfiniBand to Fibre-Channel gateway external ports, and the FCoIB initiator virtual HBA that connects to the Leadville stack and uses the FC off-load capabilities of the InfiniBand Transport Framework (IBTF) to provide the shared Fibre-Channel fabric access. The management function discovers which InfiniBand ports are attached to one or more FC-to-IB gateway external FC ports, and interacts with Fibre-Channel administration software to enable and configure virtual HBA associated with a specific IB port,pkey, and a shared gateway external Fibre-Channel port. The FCoIB service may be enabled or disabled via smf(5). The SMF service "svc-fcoibi" is part of the SMF framework, and uses the fcoib(7d) device driver (DD) IOCTL interface to enable or disable FCoIB Initiator operation at the system level (svc-fcoibi ) The FCoIB initiator function appears a FCP-3 HBA to Leadville and utilizes the IBTF to establish a transport-level direct connection with a server on the Fibre-Channel SAN. I/O operations are off-loaded to hardware as well as the FCP protocol (FC-2 Class 3 protocol). 4.2. Bug/RFE Number(s): CR 6931335 4.3. In Scope: N/A 4.4. Out of Scope: NA 4.5. Interfaces: 4.5.1 Exported Interfaces: Interface Level Comments =================================================================== FCoIB DD service ioctls Project Private Enable/disable FCoIB Initiator DD ioctls Project Private List/Create/Destroy svc:/system/fcoib_initiator:default Comitted 4.5.2 Imported Interfaces: Interface Level Comments =================================================================== Leadville FCA API Committed vHBA/stack interface InfiniBand Transport Consolidation Private IB H/W abstraction Framework Leadville have in this context 3 relevant PSARC cases: PSARC 1997/385 Fibre Channel Driver Re-architecture (Leadville) PSARC 2004/291 Fibre Channel HBA port utility PSARC 2007/501 N_Port_ID Virtualization for Solaris Solaris Infiniband transport is covered by: PSARC 2002/132 IBTF: InfiniBand Transport Framework PSARC 2004/715 IBTF Contract Updates PSARC 2008/726 New HCA Capabilities IBTF v4 is being prepared for PSARC review by the Solaris IB team, but have not been sumbitted. 4.6. Doc Impact: New man pages: fcoibi (7D) 4.7. Admin/Config Impact: None 4.8. HA Impact: None 4.9. I18N/L10N Impact: None 4.10. Packaging & Delivery: A new package SUNWfcoib will be provided. 4.11. Security Impact: N/A 4.12. Dependencies: This project has dependencies on: - Leadville FCA interface - InfiniBand Transport Framework updates required to support of SCSI over Fibre-Channel flows. 5. Reference Documents: 6. Resources and Schedule: 6.1. Projected Availability: 6.2. Cost of Effort: The development effort is port/reimplement the existing FCoIB driver in Ofed on Linux to an Solaris environment. This is an effort financed by Mellanox and executed by System Fabric Works. Additional testing/QA will be performed by NSN PV. Effort of documentation TBD. 6.3. Cost of Capital Resources: 6.4. Product Approval Committee requested information: 6.4.1. Consolidation or Component Name: Solaris OS (ON C-Team) 6.4.3. Type of CPT Review and Approval expected: FastTrack 6.4.4. Project Boundary Conditions: Pt. no web home created 6.4.5. Is this a necessary project for OEM agreements: No 6.4.6. Notes: None 6.4.7. Target RTI Date/Release: ONV build 145 (7/19 closing date) 6.4.8. Target Code Design Review Date: TBD 6.4.9. Update approval addition: No 6.5. ARC review type: FastTrack 6.6. ARC Exposure: Open 6.6.1. Rationale: 7. Prototype Availability: 7.1. Prototype Availability: 7.2. Prototype Cost: