Deliver Lighttpd 1.4.x into OpenSolaris 18 March 2008 1. Summary and motivation 1.1. Introduction This project delivers Lighttpd 1.4.x into OpenSolaris Lighttpd is an Open Source Web Server that currently enjoys 4th position in the Netcraft rankings of Market Share for Top Servers across all domains. Lighttpd became popular on the back of The Ruby on Rails take-off in 2006. Much of it's popularity stems from it being extremely lightweight, hence the name. Lighttpd supports both PHP and Ruby on Rails through FastCGI For more information about Lighttpd see: http://lighttpd.net Lighttpd is a key component of an integrated Ruby on Rails stack which would include Ruby, Ruby on Rails, FastCGI, the fcgi Ruby Gem and an Open Source Database (MySQL, SQLlite and Postgres are all options here) This case seeks Minor Release Binding. 2. Technical issues 2.1. Key objects /usr/lighttpd/1.4/sbin/lighttpd /usr/lighttpd/1.4/bin/spawn-fcgi /usr/lighttpd/1.4/lib/mod_*.so /etc/lighttpd/1.4/lighttpd.conf /var/lighttpd/1.4/docroot 2.2. Versioning The versioning model for Lighttpd is ... The compatibility expectations for Lighttpd are as follows: (1) Version x.y.z is compatible for different values of z, such that z >= 0. (2) Version x.y.z is not expected to be compatible for different values of x or y At the time of writing the latest version of Lighttpd is 1.4.18, a lot of work is going into 1.5.x but given that most Lighttpd deployments are on 1.4.x, we are not considering 1.5.x at this time. We are however making provision for 1.5.x by versioning this integration as lighttpd14. 2.3 Loadable Modules Lighttpd ships with a number of loadable modules which are enabled through directives in the config file. Some of these modules have library dependencies that must be satisfied through build time package options (supplied to the configure script). An example is mod_magnet.so which has a dependency on the libraries for the Lua programming language. The module gets built even if the external library dependencies are not satisfied and is effectively a "stub" library. If such a module is enabled at runtime through the lighttpd config file, lighttpd will dlopen() the stub library but will fail when it calls dlsym() on that library. It's not clear why this was done but as the behavior and errors generated by such a scenario are well documented it's proposed that any such stub libraries be left in the distribution. Currently two modules are affected. One is mod_cml and the other is mod_magnet. mod_magnet is the replacement for mod_cml. Attempts to include LDAP support with the authentication module mod_auth.so failed due to Lighttpd's use of OpenLDAP extensions not documented in the internet draft LDAP C-API document. This case defers inclusion of the LDAP support to a future project. Support for loadable modules and other features is enabled at build time through flags to the configure script. The following features are enabled by this integration: --with-ipv6 - Allows for handling of IPv6 connection requests --with-bzip - Allows for the use of compression --with-pcre= - Allows use of Regular Expressions in the config file --with-mysql= - allows a vhost's docroot to be stored in a MySQL database --with-openssl= - Adds support for SSL 2.4 Directory Naming and Structure The proposed directory layout for Lighttpd is: /usr/lighttpd/1.4 /sbin /bin /lib /man /etc/lighttpd/1.4 /conf.d /var/lighttpd/1.4 /docroot /logs 2.5 Users and Groups Binary distributions of Lighttpd generally include a default config file. In this file, as with the likes of Apache, the user and group name under which to run Lighttpd are specified. The most common practice for user and group in the various binary distros is to use generic web/www related names such as www-run, www-data or webservd. It's understood at this time that it is not possible to deliver a new user or group into Solaris via a package install so the proposed user name and group is webservd:webservd 2.6 Configuration File This project will use the standard convention and deliver configuration files under /etc/lighttpd/1.4 The default configuration file provided with this integration should work out of the box and should therefore avoid relying on any dependent packages, particularly FastCGI. In addition, a directory conf.d will be created in /etc/lighttpd/1.4 This directory will be used for module/directive-specific configuration files and configuration files for virtual hosts. These are added to the main configuration file using the include directive. For this integration conf.d will contain configuration files for SSL and fastcgi. 2.7 Docroot and Log Files The Document Root for Lighttpd will be located in /var/lighttpd/1.4/docroot and will include a welcome file. Lighttpd source distributions do not include such a file so one will need to be created (index.html). This would be similar to the one supplied with Apache. Log Files will be located in /var/lighttpd/1.4/logs and will be named access.log and error.log. This directory will also include the run.pid file. /var/lighttpd/1.4 and it's files and sub directories will be owned by webservd:webservd 2.8 SMF and RBAC support On OpenSolaris, the public interface to Lighttpd will be SMF and the Lighttpd SMF service for this versions will be named: svc:/network/http:lighttpd14 This was chosen to avoid confusion should multiple versions of Lighttpd be installed on the system and is consistent with SMF being the public interface to Lighttpd. Having Lighttpd be an instance of svc:/network/http follows the convention set by the Apache integration. The appropriate manifest and method files will be provided in the package. In order to allow non-root users to manage the Lighttpd SMF service the following RBAC authorisations (auths) will be added to /etc/security/auth_attr: solaris.smf.value.http/lighttpd14 solaris.smf.manage.http/lighttpd14 A profile named "Lighttpd 14 Adminstrator" will be added to /etc/security/prof_attr. This Profile will only include the two authorisations specified above, no additions will be made to /etc/security/exec_attr. The Profile will be supported by the addition to the SMF manifest for svc:/network/http:lighttpd14 of the following properties: NAME TYPE VALUE action_authorization astring solaris.smf.manage.http/lighttpd14 value_authorization astring solaris.smf.value.http/lighttpd14 2.9 Build features See Appendix 2 for a list of features that Lighttpd can be built with. Build time package selection determines which Lighttpd loadable modules are available to the user at runtime. A complete list of Lighttpd modules is available on the Lighttpd Wiki[1] The core packages recommended by the Lighttpd developers are pcre, ssl and IPv6. Lua is also highly recommended as it is required by the mod_magnet module but currently there is no support for Lua in Solaris/OpenSolaris. 3. Lighttpd documentation Lighttpd provides man pages for the lighttpd and spawn-fcgi executables and the source distribution includes module documentation. It is not common practice to include the module documentation and will not be included in this integration The Lighttpd man pages that are provided with the sources don't cover all of command line flags. These man pages will be added to as part of this integration in order to include all of the command line flags and provide additional usage information to the user. The files affected by these additions are lighttpd.1 and spawn-fcgi.1 A "lighttpd" man page will be created and added to /usr/share/man/man1m This man page will provide the user with useful information for getting started with Lighttpd on OpenSolaris. 4. Packaging and Delivery We propose to package Lighttpd under the following packages: SUNWlighttpd14u - [usr] Server package (including server daemon, utility daemons, loadable modules, man pages) SUNWlighttpd14r - [root] (config files, smf files and data directory) Multiple versions can coexist, and are distinguished by the naming scheme (lighttpd14 for version 1.4.*, lighttpd15 for version 1.5.*). 5. Lighttpd Interfaces 5.1. Interface Stability Lighttpd has no obvious history of any interface instability and the expectation is that dot-dot releases of Lighttpd will be compatible. This is not completely guaranteed by the community and it is possible that interfaces may change due to the fixing of bugs. 5.2. Imported Interfaces OpenSSL External/Volatile PSARC/2003/500 PCRE Uncommitted PSARC/2007/164 libmysqlclient Committed LSARC 2007/608 5.3. Exported Interfaces NAME STABILITY NOTES SUNWlighttpd14r committed package SUNWlighttpd14u committed package lighttpd command line options uncommitted command line options lighttpd config file uncommitted config file syntax lighttpd log file not an interface log file syntax lighttpd pid file project private daemon pid file lighttpd umbrella Man Page committed man page spawn-fcgi command line options uncommitted command line options /usr/lighttpd/1.4/bin/spawn-fcgi uncommitted executable /usr/lighttpd/1.4/sbin/lighttpd uncommitted executable /usr/lighttpd/1.4/sbin/lighttpd-angel uncommitted executable /usr/lighttpd/1.4/man/man1/lighttpd.1 uncommitted man page /usr/lighttpd/1.4/man/man1/spawn-fcgi.1 uncommitted man page /var/lighttpd/1.4/docroot uncommitted document root /var/lighttpd/1.4/docroot/index.html uncommitted welcome page /var/lighttpd/1.4/docroot/logo.png uncommitted image /var/lighttpd/1.4/logs uncommitted log file dir /etc/lighttpd/1.4/lighttpd.conf uncommitted default conf file /etc/lighttpd/1.4/lighttpd.conf-fcgi uncommitted sample conf file /usr/lighttpd/1.4/lib/mod_*.so project private shared library svc:/network/http:lighttpd14 committed FMRI /lib/svc/method/lighttpd14-svc project private SMF svc method script /var/svc/manifest/network/lighttpd14.xml project private SMF Manifest 6. References [1] http://trac.lighttpd.net/trac/wiki/Docs ================================================================ Appendix 1: Lighttpd Integration Directory and File Structure. 1. The following files are included in the Lighttpd integration: /usr/lighttpd/1.4 /bin /spawn-fcgi /sbin /lighttpd /lighttpd-angel /man /man1 /lighttpd.1 /spawn-fcgi.1 /var/lighttpd/1.4 /docroot /index.html /logo.png /logs /error.log /access.log /lighttpd.pid /etc/lighttpd/1.4 /lighttpd.conf /lighttpd.conf-fcgi /usr/lighttpd/1.4/lib /mod_flv_streaming.so /mod_evasive.so /mod_webdav.so /mod_magnet.so /mod_cml.so /mod_trigger_b4_dl.so /mod_mysql_vhost.so /mod_cgi.so /mod_scgi.so /mod_staticfile.so /mod_dirlisting.so /mod_indexfile.so /mod_setenv.so /mod_alias.so /mod_userdir.so /mod_rrdtool.so /mod_usertrack.so /mod_proxy.so /mod_ssi.so /mod_secdownload.so /mod_expire.so /mod_evhost.so /mod_simple_vhost.so /mod_fastcgi.so /mod_extforward.so /mod_access.so /mod_compress.so /mod_auth.so /mod_rewrite.so /mod_redirect.so /mod_status.so /mod_accesslog.so /usr/share/man/man1m /lighttpd.1m /lib/svc/method/lighttpd14-svc /var/svc/manifest/network/lighttpd14.xml Appendix 2: Lighttpd Features and Packages The following options are provided by the configure script for enabling or disabling features and for adding/removing packages Optional Features: --disable-FEATURE do not include FEATURE (same as --enable-FEATURE=no) --enable-FEATURE[=ARG] include FEATURE [ARG=yes] --enable-maintainer-mode enable make rules and dependencies not useful (and sometimes confusing) to the casual installer --disable-dependency-tracking speeds up one-time build --enable-dependency-tracking do not reject slow dependency extractors --enable-static[=PKGS] build static libraries [default=no] --enable-shared[=PKGS] build shared libraries [default=yes] --enable-fast-install[=PKGS] optimize for fast installation [default=yes] --disable-libtool-lock avoid locking (might break parallel builds) --enable-lfs Turn on Large File System (default) --disable-ipv6 disable IPv6 support Optional Packages: --with-PACKAGE[=ARG] use PACKAGE [ARG=yes] --without-PACKAGE do not use PACKAGE (same as --with-PACKAGE=no) --with-gnu-ld assume the C compiler uses GNU ld [default=no] --with-pic try to use only PIC/non-PIC objects [default=use both] --with-tags[=TAGS] include additional configurations [automatic] --with-mysql[=PATH] Include MySQL support. PATH is the path to 'mysql_config' --with-ldap enable LDAP support --with-attr enable extended attribute support --with-valgrind enable internal support for valgrind --with-openssl[=DIR] Include openssl support (default no) --with-openssl-includes=DIR OpenSSL includes --with-openssl-libs=DIR OpenSSL libraries --with-kerberos5 use Kerberos5 support with OpenSSL --with-pcre Enable pcre support (default yes) --with-bzip2 Enable bzip2 support for mod_compress --with-fam fam/gamin for reducing number of stat() calls --with-webdav-props properties in mod_webdav --with-webdav-locks locks in mod_webdav --with-gdbm gdbm storage for mod_trigger_b4_dl --with-memcache memcached storage for mod_trigger_b4_dl --with-lua lua engine for mod_cml Appendix 3: Lighttpd man pages Lighttpd ======== User Commands LIGHTTPD(1) NAME lighttpd - a fast, secure and flexible webserver SYNOPSIS lighttpd -D -f DESCRIPTION FILES /etc/lighttpd/lighttpd.conf CONFORMING TO HTTP/1.0 HTTP/1.0 HTTP-Authentification - Basic, Digest FastCGI CGI/1.1 SEE ALSO spawn-fcgi(1) AUTHOR spawn-fcgi ========== User Commands SPAWNFCGI(1) NAME spawn-fcgi - spawning FastCGI processes SYNOPSIS spawn-fcgi -f [-p | -s ] [-C ] [-c ] [-u ] [-g ] spawn-fcgi -v spawn-fcgi -h DESCRIPTION spawn-fcgi is used to spawn remote FastCGI processes. SEE ALSO lighttpd(1) AUTHOR