1. Introduction 1.1. Project/Component Working Name: Twisted update to 8.1 and adding Twisted-Web2 1.2. Name of Document Author/Supplier: Brian Cameron 1.3. Date of This Document: 10/28/2008 1.4. Name of Major Document Customer(s)/Consumer(s): 1.4.1. The PAC or CPT you expect to review your project: Solaris PAC 1.4.2. The ARC(s) you expect to review your project: PSARC 1.4.3. The Director/VP who is "Sponsoring" this project: Robert O'Dea 1.4.4. The name of your business unit: Software - OPG 1.5. Email Aliases: 1.5.1. Responsible Manager: leo.binchy@sun.com 1.5.2. Responsible Engineer: brian.cameron@sun.com, 1.5.3 Marketing Manager: dan.robert@sun.com 1.5.4. Interest List: desktop-discuss@opensolaris.org 2. Project Summary 2.1. Project Description: This project updates Twisted from 2.5 to 8.1 and adds the Twisted.Web2 module. 4. Technical Description: 4.1. Details: Twisted is a popular, open-source networking engine written in Python and supporting numerous protocols. It contains a web server, chat clients, chat servers, mail servers, and more. Twisted is made up of a number of related sub-projects. This proposal integrates version 8.1 which includes the following 9 main modules: Twisted Conch SSH and SFTP protocol implementation. Twisted Core Asynchronous event loop. Twisted Lore Documentation generator with HTML and LaTeX support. Twisted Mail SMTP, IMAP and POP protocol implementation. Twisted Names DNS protocol implementation with client and server. Twisted News NNTP protocol implementation with client and server. Twisted Runner Process management, including an inetd server. Twisted Web HTTP protocol implementation with clients and servers. Twisted Words Chat and Instant Messaging. The Web2 module is a 10th module which is distributed separately. 4.1.1. Updating Python Twisted from 2.5 to 8.1 While this may seem like a fairly significant version bump, it is not. The Twisted community changed their version numbering system so that the first digit of the version is the last digit of the year. Release 2.5 was followed by release 8.0 and then 8.1. So, the Twisted community is not following normal major.minor.micro version numbering system used by most software. Python Twisted 2.5 was previously reviewed as PSARC 2008/121. Twisted version 8.1 contains many bug fixes. The following highlights the more significant interface-related features and deprecations. All features were relate to Twisted release 8.0 unless otherwise specified: - Twisted now uses setuptools and easy_install for better installation. - twisted.internet.defer contains a DeferredFilesystemLock which gives a Deferred interface to lock file acquisition. - twisted.python.modules is a new system for representing and manipulated module paths (i.e. sys.path). - twisted.internet.fdesc now contains a writeToFD function. - The reactor now has a blockingCallFromThread method for non-reactor threads to use to wait for a reactor-scheduled call to return a result. - twisted.internet.task.deferLater is a new API for scheduling calls and getting deferreds which are fired with their results. - twisted.python.log now contains a Twisted log observer which can forward messages to the Python logging system. - Log files now include seconds in the timestamps. - Int8Receiver was added. - The FTP client can be told what port range to bind passive transfer ports. - twisted.spread.jelly now supports decimal objects and all forms of sets. - Twisted Conch adds DEC private mode manipulation methods to ITerminalTransport. - Twisted Web twisted.web.client.getPage has support for the HTTP HEAD method. - Twisted Words provides a function for creating XMPP response stanzas. - Twisted Words adds 'and' and 'or' operators for Xish Xpath expressions. - Twisted Mail supports CAPABILITY responses that include atoms of the form "FOO" and "FOO=BAR" in IMAP4 - In 8.1, twisted.internet.error.ConnectionClosed is a new exception which is the superclass of ConnectionLost and ConnectionDone. - In 8.1, extending XMPP protocols is easier. Deprecations: - twisted.python.timeoutqueue. - twisted.entrprise.row, twisted.enterprise.reflector, and twisted.enterprise.util. - The dispatch and dispatchWithCallback methods of ThreadPool. - Starting the same reactor multiple times. - The visit method of various test classes in trial. - The --report-profile option to twistd and twisted.python.dxprofile. - The upDownError method of Trail reporters. - In 8.1, mktap is now deprecated. 4.1.2. Twisted.Web2 This case also adds the Twisted.Web2 module. This is an asynchronous HTTP 1.1 Web Server Framework built with Twisted. It provides a RFC 2616 compliant HTTP 1.1 protocol implementation, with pipelined and persistent request support, in a non-blocking threadless manner. It also includes a simple web framework with request and response objects, static file support, error handling, form upload support, HTTP range support, pre-built parsers for all standard headers, and so on. It is deployable as a standalone HTTP or HTTPS server, as a HTTP[S] server proxied behind another server, or as a SCGI, FastCGI, or CGI script. In addition to running native twisted.web2 applications, it can also run any WSGI or CGI application, or, via compatibility wrappers, most applications written for the older twisted.web API. Currently, twisted.web2 does not include a HTTP client or proxy. Twisted.web2 is not a templating framework. It provides mechanisms for locating and running code associated with a URL, but does not provide any means for separating code and data or to ease the task of generating HTML. According to its documentation, its interfaces should not be considered stable. Note that Python-Twisted contains a module named Web, which is a separate project and there is no compatibility between them. The Twisted.Web2 module has the following improvements over Twisted.Web: * Streaming upload data * Support for multiple headers of the same name * Separation of low level HTTP and high level request handling which allows it to run under other transports such as SCGI and CGI * IResource API improvements from Nevow * More versatile outgoing data streaming API * Correct header parsing * Full HTTP/1.1 support * Output filters (HTTP range support in a generic fashion, and gzip support) * Significantly better URI Rewriting when used behind a reverse proxy such as Apache's mod_pxy 4.2. Interfaces: Exported interfaces: Interface Stability Comment -------------------------------------- -------------- --------------------- twisted.flow Uncommitted Python Twisted module twisted.pair Uncommitted Python Twisted module twisted.web2 Uncommitted Python Twisted Web2 module Imported Interfaces Interface Stability Comments -------------------------------------- -------------- --------------------- Python 2.4.x External PSARC 2005/532 Python Setuptools Uncommitted PSARC 2008/084 zope-interface Uncommitted PSARC 2008/122 4.3. Doc Impact: Man page is needed. 4.4. Packaging & Delivery: SUNWpython-twisted - package for Python-Twisted SUNWpython-twisted-web2 - package for Python-Twisted-Web2 4.5. Dependencies: None. 4.6. L10N Impact: The Desktop team and the G11N are working together to evaluate and provide I18N/L10N support. 4.7 Security Impact: None. 5. Reference Documents: PSARC 2008/121 Python Twisted. Python Twisted Home Page http://twistedmatrix.com/trac/ Twisted Documentation http://twistedmatrix.com/projects/core/documentation/howto/index.html http://twistedmatrix.com/trac/wiki/Documentation