Introducing XCB [ The X Protocol C-Language Bindings ] with X11 in Solaris Stefan Teleman 29 March 2010 1. Summary and motivation XCB [ The X Protocol C-Language Bindings ] is intended as an API and ABI incompatible replacement for Xlib. It is neither designed, nor intended as a compatible plug for Xlib. Further details about the XCB ABI and its intended [ and unintended ] consequences on libX11 are discussed below. XCB provides several advantages over the existing Xlib implementation: [a] small footprint [b] protocol direct access [c] latency hiding [d] improved thread support [e] extensibility XCB is delivered as a self-contained collection of shared library objects, header files, XML schema definition files, Python precompiled object files, and pkg-config component discovery files. A full and complete installation of XCB must be constructed from three different components: - xcb-proto - libxcb - xcb-util Constructing XCB is dependent on libXau, libXdmcp and libXtrans. The existing libX11.so.4 [ currently libX11 Version 1.3.2 ] provides an optional software construction dependency on XCB: the libX11 library can be XCB-enabled at software construction time [ ./configure ]. Enabling XCB in Xlib is an ABI compatible change: it alters neither the programmatic facilities, nor the public interfaces exposed by libX11. Further details of XCB's design, and the implications resulting from enabling XCB in libX11 are discussed below. This Case proposes the integration of the latest version of XCB compatible with libX11: - xcb-proto version 1.5 - libxcb version 1.4 - xcb-util version 0.3.6 For the purposes of this Document, ${PYTHON_VERSION} refers to Python 2.6. [6] This Case seeks Micro/Patch release binding. 2. Technical issues 2.1. Key Objects 2.1.1. Shared Libraries: /usr/lib/libX11-xcb.so.1 /usr/lib/libxcb.so.1 /usr/lib/libxcb-composite.so.0 /usr/lib/libxcb-damage.so.0 /usr/lib/libxcb-dpms.so.0 /usr/lib/libxcb-dri2.so.0 /usr/lib/libxcb-glx.so.0 /usr/lib/libxcb-randr.so.0 /usr/lib/libxcb-record.so.0 /usr/lib/libxcb-render.so.0 /usr/lib/libxcb-res.so.0 /usr/lib/libxcb-screensaver.so.0 /usr/lib/libxcb-shape.so.0 /usr/lib/libxcb-shm.so.0 /usr/lib/libxcb-sync.so.0 /usr/lib/libxcb-xevie.so.0 /usr/lib/libxcb-xf86dri.so.0 /usr/lib/libxcb-xfixes.so.0 /usr/lib/libxcb-xinerama.so.0 /usr/lib/libxcb-xinput.so.0 /usr/lib/libxcb-xtest.so.0 /usr/lib/libxcb-xv.so.0 /usr/lib/libxcb-xvmc.so.0 /usr/lib/libxcb-atom.so.1 /usr/lib/libxcb-aux.so.0 /usr/lib/libxcb-event.so.1 /usr/lib/libxcb-icccm.so.1 /usr/lib/libxcb-image.so.0 /usr/lib/libxcb-keysyms.so.1 /usr/lib/libxcb-property.so.1 /usr/lib/libxcb-render-util.so.0 /usr/lib/libxcb-reply.so.1 [ plus the corresponding 64-bit objects ] 2.1.2. Header files: /usr/include/xcb/xcb.h /usr/include/xcb/*.h [ exclusive of xcb.h ] 2.1.3. Metaprogramming XML Objects: /usr/share/xcb/*.xml 2.1.4. Component Discovery [ pkg-config ] Objects: /usr/lib/pkgconfig/xcb.pc /usr/lib/pkgconfig/xcb-*.pc [ plus the corresponding 64-bit pkg-config objects ] 2.1.5. [ xcbgen ] Python utility components: /usr/lib/python${PYTHON_VERSION}/vendor-packages/xcbgen/*.py There are no 64-bit Python objects for [ xcbgen ]. A complete listing of all the XCB shared libraries, header files, metaprograming XML files, pkg-config component discovery files, and [ xcbgen ] Python utility files is available in Appendix 1 of the ARC Case Materials. 2.2. XCB Construction, [ xcbgen ] and XCB Extensibility XCB's shared library objects provide interface wrappers around existing X11 Extensions [ and their corresponding shared libraries ]. This implementation encapsulation mechanism is consistent with one of XCB's stated primary goals: direct protocol access. XCB's software construction mechanism is based on meta-programming. The XML metaprogramming objects enumerated in [ 2.1.3 ] are used to generate translation units and associated header files for XCB's shared library objects via [ xcbgen ]. [ xcbgen ] is a Python utility delivered by the xcbproto component. XCB follows the standardized X component delivery design, by providing an xcbproto component and a separate, but xcbproto dependent XCB Component Collection. [ xcbgen ] enforces a relatively liberal dependency constraint on Python: it requires Python >= 2.5. Python objects are delivered by XCB in the standard Solaris Python package collection location: ${PREFIX}/lib/python${PYTHON_VERSION}/vendor-packages/xcbgen/ The [ xcbgen ] Python utiliy is of interest to XCB's construction mechanism, and to XCB developers writing new XCB extensions. It is not used outside of these two cases. The XCB XML Interface Definition Objects expose elements which define the interfaces, interface dependencies and formal relations between interface definitions for the XCB functions implemented in each translation unit. It is implied, and expected, that the XCB protocol meta-descriptions are formally verified against the existing, published specifications for the X protocol and protocol extensions they intend to replace. This reliance on meta-programmed interface definitions provides developers with an implicitly formalized [ and relatively easy ] mechanism for extending XCB: new XCB interfaces can be generated simply by creating XML Metadata Interface Definition Objects, and by generating their corresponding translation unit[s] and the resulting Shared Library. 2.3. Programmatic Facilities An example of constructing an XCB application is provided in [2]. If an application relies exclusively on Xlib's Committed Interfaces, XCB introduces no ABI or API incompatible changes. Applications will continue to work as expected, unchanged. Applications can import from, and bind to, XCB's Interfaces directly, bypassing Xlib and all the other X Extensions Libraries [ examples: GStreamer and libxine ]. This creates a direct depenedency on XCB's Interfaces. 2.4. Interface Stability The Shared Library Objects delivered by XCB do not contract, or imply, at this time, an interface stability commitment. Furthermore, there appears to be little guarantee that the physical presence of the XCB objects will be maintained across XCB releases. For example, libxcb-xlib.so.0 and its corresponding xcb-xlib.pc pkg-config file have been physically removed in a previous release version of XCB [ 1.1.92 ]. It is unspecified, at this time, if XCB will maintain its current physical object structure, or will undergo incompatible changes in the future. In spite of this apparent volatility, consumers of XCB should be allowed to test for XCB's physical object structure, and for explicit vesions of XCB, via standard pkg-config mechanisms. These consumers may also choose to bind directly to XCB's public interfaces. The potential instability of XCB's Interfaces creates a direct explicit binding between a given XCB release and its interface consumers. Applications binding directly to XCB's Interfaces must recognize, and adjust to, the potentially incompatible ABI and API changes which may occur in future releases of XCB. The responsibility for resolving any API or ABI incompatibilities between different release versions of XCB is explicitly delegated to the application consuming XCB's Interfaces. Existing pkg-config mechanisms are necessary and sufficient for discovering and enforcing XCB's interfaces particulars. 2.5. API, ABI and Programming Considerations The canonical Xlib Interfaces rely on the availability of a well-known, standard libX11 data structure: the [ X11 ] Display. Passing a Display pointer as an argument to a libX11 function is a well-known libX11 programming and interface design idiom. XCB introduces a different and incompatible data structure, similar in use to the X11 Display: the xcb_connection_t. There exists a striking similarity between the libX11 interface design idiom described above, and the XCB interface design idiom: many XCB functions take an xcb_connection_t pointer as an argument, akin to the Display pointer argument in libX11. XCB does not require the use, or availability, of a X11 Display structure in its API. Also, XCB does not provide a mechanism for obtaining a Display structure pointer from an existing, and valid, xcb_connection_t. It is entirely possible to construct an XCB application without using a libX11 Display structure. In fact, the concept of a libX11 Display is entirely absent from XCB's design: XCB provides a connection to a X server [ xcb_connection_t ], and [ optionally ] a XCB Screen [ identified by an integer screen enumerator ]. Enabling XCB in Xlib creates the possibility of mixing Xlib and XCB calls [ with documented restrictions ]. This feature enables a gradual application migration from Xlib to XCB [ and is currently required for applications using the GLX Extension ]. However, only one of either Xlib or XCB can manage the event queue at any discrete point in time. Programmatic facilities for switching the event queue handlers between Xlib and XCB exist. These facilities, and their restrictions are documented in the XCB API Manual. [1] XCB has been tested and verified to work on recent builds of Nevada, and on recent Update Releases of Solaris 10, going back to Solaris 10 Update 4. Interface Stability warnings outlined in 2.4 apply. XCB exports C and [ identical ] C++ Interfaces. 2.6. Known Limitations and Enhancements The current implementation of XCB does not provide full feature parity with the existing GLX Extension implementation. Usage of the GLX Extension with XCB still requires the usage of an X11 Display pointer. A future ARC Case will discuss a full XCB-based implementation of the GLX Extension. A different, future ARC Case will discuss a full XCB-based implementation of the Trusted Solaris Extensions for X [ XTSol ]. It is of interest to note that canonical XCB releases provide a XCB implementation of SELinux [ libxcb-selinux.so.0 ]. The XSUNSHMEM and SUN-DGA Extensions will not be re-implemented in XCB. 2.7. Documentation XCB is documented in HTML format at [1]. This integration will deliver the complete XCB HTML Manual, as provided in [1]. 3. Interfaces 3.1. Interface Stability This Case proposes the following Interface Stability Classification for XCB's components: Component Name Interface Stability -------------- ------------------- libX11-xcb.so.1 Uncommitted libxcb.so.1 Uncommitted libxcb-composite.so.0 Uncommitted libxcb-damage.so.0 Uncommitted libxcb-dpms.so.0 Uncommitted libxcb-dri2.so.0 Uncommitted libxcb-glx.so.0 Uncommitted libxcb-randr.so.0 Uncommitted libxcb-record.so.0 Uncommitted libxcb-render.so.0 Uncommitted libxcb-res.so.0 Uncommitted libxcb-screensaver.so.0 Uncommitted libxcb-shape.so.0 Uncommitted libxcb-shm.so.0 Uncommitted libxcb-sync.so.0 Uncommitted libxcb-xevie.so.0 Uncommitted libxcb-xf86dri.so.0 Uncommitted libxcb-xfixes.so.0 Uncommitted libxcb-xinerama.so.0 Uncommitted libxcb-xinput.so.0 Uncommitted libxcb-xlib.so.0 Uncommitted libxcb-xtest.so.0 Uncommitted libxcb-xv.so.0 Uncommitted libxcb-xvmc.so.0 Uncommitted libxcb-atom.so.1 Uncommitted libxcb-aux.so.0 Uncommitted libxcb-event.so.1 Uncommitted libxcb-icccm.so.1 Uncommitted libxcb-image.so.0 Uncommitted libxcb-keysyms.so.1 Uncommitted libxcb-property.so.1 Uncommitted libxcb-render-util.so.0 Uncommitted libxcb-reply.so.1 Uncommitted 3.2. Interface Dependencies and Compatibility XCB's Interface Table is provided in the ARC Case Materials directory [ xcb-interface-table.txt ]. Introducing XCB creates no additional run-time dependencies beyond those already present in the X Consolidation. libX11 with XCB imports interfaces from: - libxcb.so.1 libxcb.so.1 imports interfaces from: - libXau.so.6 - libXdmcp.so.6 Existing applications binding to Xlib's Committed Public Interfaces [ libX11 ] are unaffected by the private implementations changes created by XCB, and will continue to work as expected. Applications consuming Xlib Project Private interfaces may break. XCB removes a number of Xlib Project Private Interfaces from libX11 [ specifically from Xtrans ]. Details are provided in the Interface Table. XCB creates a compile-time dependency on Python 2.6 [ SUNWPython26 ]. 4. Packaging This Case proposes the following packaging structure for XCB: 4.1. IPS Packaging: - The libraries, headers, and pkg-config discovery files for the main xcb libraries, and the xcbgen developer tool will all be delivered in pkg:x11/library/libxcb. - The libraries, headers, and pkg-config discovery files for the additional libraries from the xcb-util collection will be delivered in pkg:x11/library/xcb-util. In both packages, the headers, pkg-config files & xcbgen files will all be part of the developer facet to avoid installation when minimizing for non-development systems. 4.2. SVR4 Packaging [ Solaris 10 ]: - XCB shared library objects, and pkg-config discovery files will be delivered in the existing SUNWxwplt package. This is required because of libX11's acquired explicit dependency on XCB, and avoids creating a new package dependency for SUNWxwplt. - XCB header files will be deliverd in the existing SUNWxwinc package. This will maintain consistency with the existing SUNWxwplt package object delivery design. - [ xcbgen ] objects will be delivered in a new package named SUNWxcbgen. As explained above, [ xcbgen ] is not a run-time dependency of XCB, and is of interest only to XCB developers. Application developers relying on XCB's canonical Public Interfaces, and who do not intend to extend the XCB API, do not need access to xcbgen, and may choose to omit its installation alltogether. 5. References [0] http://www.x.org/ [1] http://xcb.freedesktop.org/manual/index.html [2] http://en.wikibooks.org/wiki/X_Window_Programming/XCB [3] http://www.linuxshowcase.org/2001/full_papers/massey/massey.pdf [4] http://www.usenix.org/events/usenix02/tech/freenix/full_papers/sharp/sharp_html/index.html [5] http://www.usenix.org/events/usenix02/tech/freenix/full_papers/massey/massey_html/index.html [6] PSARC/2009/043 [ Python 2.6 ]