Functional Specification for JMX Management and Monitoring of Derby

• Libraries
• Default values

Overview

Derby is being used in a variety of applications, ranging from short-lived unit-tests to long-running complex multi-user applications and server frameworks. Although Derby (especially in its embedded mode) is known as a "zero-admin" database, many Derby users may have a need to monitor and manage their Derby system, for example the Derby Network Server or a particular database, at runtime.

This specification describes a management extention to Derby based on Java Management Extensions (JMX).

JMX technology is widely used and is well known as a flexible and useful way to manage Java applications. A JMX management interface is composed of a Management Bean (MBean), which is registered in an MBeanServer with a name (ObjectName). The MBean may "instrument" the application, or a part of it, by accessing Java objects or other resources available in the application.

This specification describes a set of initial JMX-related management and monitoring features for Derby.

The Jira-issue tracking the initial implementation of this specification is DERBY-1387. The Apache Derby wiki is also being used to track status and present further details; see http://wiki.apache.org/db-derby/DerbyJMX

Terms

For the remainder of this spec, we use the following terms:

• Target release - The release which carries these changes. This would be 10.4 or later.
• JVM - Java Virtual Machine.
• MBean - Managed Bean: Part of the JMX API representing a resource running in the JVM. The MBean represents the management interface of the resource.
• Managed application/system - an application (Derby) with JMX Management and Monitoring features enabled.

Dependencies

Libraries

Derby's Management Service uses the platform MBeanServer from java.lang.management.ManagementFactory which was introduced in Java^TM 2 Platform Standard Edition 5.0 (J2SE 5.0). Starting with J2SE 5.0, JMX technology is included in the Java SE platform. This means no additional libraries are needed when using a JVM supporting J2SE 5.0 or newer.

The platform MBeanServer is not available in J2SE 1.4.2, and is not part of the JMX 1.2 specification nor the JMX Remote API, meaning that Derby's management features will not be available with J2SE 1.4.2 or Java ME unless support for a replacement MBeanServer is implemented. More details may be available in the Limitations section and Appendix A.

Default property values

There has been a discussion on the derby-dev list about adding a "default.property"-file to Derby. This would give access to Derby properties through Java Properties whether they are set to default values or not. Such a configuration would benefit management and monitoring greatly, because currently there is no defined way of accessing default properties, nor is there a standard way of persisting changes to system properties at runtime.

Management and Monitoring Features

These features will give a Derby administrator the opportunity to manage and monitor a Derby system dynamically through a management application, for example represented by a Graphical User Interface (GUI), without writing any code or configuration files, and without requiring direct access to Derby libraries (jar files) from the management application.

The Derby administrator may also provide JMX-based access to other users at will, thus giving individual database administrators the opportunity to easily manage their databases.

Derby's JMX features will be implemented on two levels:

• Agent level - A management module in Derby which establishes an MBean server and suitable agent services (such as a protocol connector).
• Instrumentation level - MBeans instrumenting Derby resources

The instrumentation level instruments Derby resources with MBeans. Developers of Derby MBeans are free to choose the granularity of the Derby resources. This means that an MBean can represent the whole Derby system, a single service or module, a single class or object, etc.

The agent level is described as the Management Service.

Management Service (Agent Level)

The Management Service will be the heart of Derby's management extentions. The service provides an internal API for registering and exposing MBeans. Through this API, the service will provide access to an MBean Server. Derby code should use the Management Service API to register MBeans with an MBean server.

Implementation:

Derby's Management Service will be implemented as a Derby module. The Derby Monitor will start the module automatically at system startup, and the module will remain active until the Derby system shuts down.

The Management Service itself is part of Derby's engine architecture, and is thus only available in a running Derby instance. Derby code which creates and registers MBeans must therefore have direct access to the Derby engine code (derby.jar).

If the runtime environment (JVM) does not support the required JMX features, the Management Service will provide no functionality.

The Management Service is defined by an internal (non-public) interface

which is available to Derby code, if the module is running, by requesting an instance from the Derby monitor:

ManagementService ms = (ManagementService) Monitor.getSystemModule(Module.JMX);

Note that implementation details are subject to change at any time. The above was valid at the time of writing this document.

MBean Server:

The choice of MBean server affects Limitations. This spec assumes usage of the "platform MBeanserver", which uses the JVM's built-in JMX instrumentation and connectors (see the J2SE 5.0 Platform Monitoring and Management guide available from Sun Microsystems). This allows seamless interaction with generic JMX-compliant client applications such as "JConsole" or "VisualVM", and simplifies the implementation of the Management Service.

When Derby's Management Service has been started, a valid JMX user may be able to access the Derby-specific MBeans registered by the Management Service, potentially as well as any other MBeans or MXBeans made available by the managed JVM (if this functionality is supported and the JMX user is authorized to access such resources).

In this context, the term "valid JMX user" denotes a user who is able to connect to the managed JVM's platform MBean server, which means that the user must pass any enabled JMX security checks such as user authentication.

Enabling Management Features

Derby's Management Service will be enabled automatically when the Derby system starts up. The Derby system may be started by the user by loading Derby's embedded JDBC driver, directly or indirectly. For example, the embedded driver is loaded implicitly when starting the Network Server.

Different implementations of the Management Service may be booted depending on the JVM's JMX support level (see Dependencies).

The JMX based management features (primarily MBeans) described in this specification will not be available if the running JVM does not support the required JMX features (see Dependencies).

In order to enable platform JMX management in a given JVM, the user (JVM administrator) may have to configure special settings, for example system properties. A JVM normally distinguishes between local (same host, same user as the JVM administrator) and remote access (different user and/or host) in this context.

Such settings may include:

Detailed information on configuring the JMX agent can be found in the Platform Monitoring and Management guide and the Java SE Monitoring and Management Guide, Chapter 2.

Note that the com.sun.management.jmxremote* system properties are not part of any official standard, meaning that they are not guaranteed to work with any JVM vendor implementing J2SE 5.0 or Java SE 6. However, these properties are widely used, by various JVM vendors, and may be considered de facto standard for most practical purposes. It is possible to programmatically enable JMX features in a JVM without requiring the use of these properties, but this requires setting up a separate MBeanServer (that is not the platform MBeanServer) and RMI server connector; see Appendix B resources for details.

The following is an example where the Derby Network Server is started listening on the default address with the default security manager, configured for local and remote JMX management with JMX security disabled. Remote JMX clients may connect to port 1528:

java -Dcom.sun.management.jmxremote.port=1528
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false 
org.apache.derby.drda.NetworkServerControl start

(Line breaks should not be included).
Note that it is NOT recommended to disable JMX security in a production environment.

The following is an example where Derby's ij tool is started in a JVM supporting the Java SE 6 platform and Sun Microsystem's extension Attach API. In this case, local JMX management is automatically enabled by the JVM. Here, ij acts as an embedding application, and the Management Service will be started when a valid connect request is issued using the embedded driver:

$ java -jar lib/derbyrun.jar ij
ij version 10.4
ij> connect 'jdbc:derby:exampleDatabase';
ij>

Note that if the platform MBean server is not available to Derby, or for some reason is not functioning (e.g. due to missing security permissions if a Java Security Manager is installed), then Derby's MBeans will not be registered with an MBean server and will thus be inaccessible.

Note:
Derby's Management Service and related features are enabled by default. In order to restrict access to Derby's management features the user may install a Security Manager and tune the security policy accordingly (see the permissions described below). It is also possible to disable and otherwise manage parts of Derby's management features dynamically (see below).

Dynamic Management

It is possible to let a JMX user manage a subset the management features dynamically (at runtime). There are plans to provide a special MBean to handle this. When management is disabled dynamically, then none of Derby's MBeans (that are registered with Derby's Management Service) apart from the ManagementMBean will be visible (accesible) to JMX clients. If management is re-enabled after being dynamically disabled, then the MBeans that are registered with Derby's Management Service should once again become visible to valid JMX users.

This specification includes a description of an MBean for managing Derby's management service, called ManagementMBean. The reader should note that this description was up to date at the time when this specification document was written, but is subject to change at any time.

Utilizing Management Features

When Derby's Management Service is running and able to access the JVM's platform MBean server (see above), a valid JMX user may connect to this MBean server and access MBeans that are registered with the MBeanServer. The Derby administrator (and the JVM administrator) may restrict access to management features by utilizing the security mechanisms described elsewhere in this document and its references.

Any JMX client that supports the platform JMX agent (RMI connector) may request access to the JVM's and Derby's management features. This document does not describe any such client, but there are several general purpose JMX client tools available that can be used for this purpose.

JConsole, which is shipped with Sun's JDK 5.0 and JDK 6, is one such example. Refer to Using JConsole for more information. VisualVM is another such example.

It is also possible to access Derby's JMX features programmatically. Refer to documents such as the Java SE Monitoring and Management Guide, and the functional tests for details and examples.

MBeans (Instrumentation Level)

Each MBean is described by a public Java interface (see the Public API section). Each MBean interface is implemented by one or more classes, providing access to functionality and/or information relating to the managed Derby resource.

In the first target release, Derby's MBean interfaces will be statically defined and mainly follow the design pattern used by Standard MBeans. However, Derby MBeans may extend javax.management.StandardMBean or the Management Service may use this class as a wrapper in order to utilize advantages associated with this special class. Thus, the MBeans are technically being defined as dynamic MBeans. This approach (using either the javax.management.StandardMBean class or strictly following the design pattern of Standard MBeans), is considered to be a solution which is likely to provide good compatibility with current and future versions of generic JMX based tools, while staying compatible with J2SE 5.0 and having sufficient implementation flexibility.

A Derby MBean is instantiated and registered with the Management Service by a relevant part of the Derby system. This may happen at system startup or at a later point in time if that is more suitable. For example, some MBeans may be registered when the JDBC driver is first booted, while others may be registered when the Network Server starts or when a database is booted.

Similarly, MBeans may be removed (unregistered) if the managed resource (for example the Network Server, or a database) is shutting down. The creation and termination time of a specific MBean is implementation defined. Note that for Derby to be able to unregister MBeans from the MBean server, it must be allowed to do so by the security policy if a security manager is installed. The default Network Server security policy includes such permissions.

MBean Naming Conventions

MBean interface and class names in Derby should indicate which part of Derby the MBean is designed to manage or monitor. For example, it is obvious that the NetworkServerMBean is designed to manage and/or monitor the Network Server.

Every JMX MBean instance is associated with an ObjectName (javax.management.ObjectName), which is specified to the MBean server when the MBean is being registered with this server. Derby MBeans should have consistent and usable ObjectNames, as recommended by best practices. A String representation of each MBean's possible ObjectName(s) must appear in the class Javadoc of the MBean's public API.

An ObjectName (also referred to as object name) consists of a domain and a comma-separated list of key/property pairs. A JMX client may access an MBean by specifying its ObjectName (it is also possible to search for MBeans using an ObjectName pattern, potentially matching several MBeans).

The domain for Derby's MBeans is org.apache.derby. This is to avoid conflicts with other MBeans that may be available in the JVM.

An unquoted key (property) may not contain any of the characters comma (,), equals (=), colon (:), asterisk (*), or question mark (?). See the ObjectName API Javadoc for details.

There may be multiple instances of an MBean available, each with a different object name (since all Derby MBeans will have the same domain, this means that different instances of the same MBean must have at least one different key property). Thus, an MBean interface may be associated with more than one object name. Variable parts of the ObjectName must be clearly defined in the MBean interface's class Javadoc, so that the MBean may be easily accessed by JMX clients.

The following conventions are defined for the key properties of the ObjectName of an MBean that is created and registered by Derby:

Additional key properties may be defined by each MBean. However, all key properties and their possible values must be documented in the MBean interface's class Javadoc. Best practices should be followed whenever possible.

MBean Features

All MBeans may provide attributes and/or operations via JMX. MBeans may also emit notifications, and receive notifications from other MBeans.

The user can read and/or modify attribute values via a JMX-compliant client tool. Attributes can be readable, writeable, or both. This is defined by get, set and is methods in the MBean definition (interface).

Operations correspond to other methods in the MBean definition, and may be invoked by a valid JMX user. Getter and setter methods which include parameters in the method signature define operations, not attributes.

Some operations require user-supplied arguments. The user's primary source of reference in this regard is Derby's public API, with the inclusion of the new MBean interfaces as described below. However, usability may be improved by using MBean Metadata classes, special usage of the javax.management.StandardMBean class, or special annotations. This is outside the scope of this document.

MBeans planned for target release 1

A set of MBeans are planned for inclusion in the first target release. This will most likely be the first Apache Derby 10.4 release.

Note: Details regarding specific MBeans and exactly which MBeans will be part of the first target release is subject to change, as MBean specifications may be developed independently of the basic management features. However, basic descriptions of planned MBeans are inlcuded in this specification for the reader's convenience. The reader should refer to the Overview for references to sources of more detailed and up-to-date information.

Refer to the public API section for details about the availablity and naming of public (non-internal) MBean interfaces.

In the below descriptions, the value of the system key property is represented by the String <sysID>, since the actual value will vary.

ManagementMBean

ManagementMBean exposes status information about Derby's management service, and provides an API for managing Derby's JMX management and monitoring features.

Note that although this MBean description was up to date at the time of writing this document, it is subject to change.

This MBean provides the following read-only attributes:

The following table describes planned Operations of this MBean.

VersionMBean

VersionMBean exposes read-only information about the version of Derby, or a part of Derby, that is running. The user may use this as a diagnostic tool.

Note that although this MBean description was up to date at the time of writing this document, it is subject to change.

The jar key property's value is variant, distinguishing MBean instances from each other.

This MBean provides the following read-only attributes:

This MBean provides no operations.

JDBCMBean

JDBCMBean exposes read-only information about the JDBC Driver embedded with the running Derby system (the Derby Embedded JDBC driver).

Note that although this MBean description was up to date at the time of writing this document, it is subject to change.

This MBean provides the following read-only attributes:

The following table describes the planned Operations of this MBean.

NetworkServerMBean

Note that although this MBean description was up to date at the time of writing this document, it is subject to change.

The current public Network Server API (org.apache.derby.drda.NetworkServerControl) can only be successfully invoked from the machine on which the server is running, with the exception of the ping command. There are, however, use cases where configuring or retrieving diagnostic information in a remote setting is useful. The NetworkServerMBean is potentially able to provide the entire NetworkServerControl API to both local and remote JMX clients.

Detailed information about the NetworkServerControl API can be found in the NetworkServerControl javadoc or the Derby Server and Administration Guide.

This specifiation assumes that additional security mechanisms (for instance system authorization) needs to be in place before allowing fully featured management of the Network Server. For example, it is expected that some integration with the new System Privileges features (DERBY-2109) is needed. This is not yet part of this specification. Hence, for the first target release it is likely that only a limited subset of the desired functionality will be available, which is what is described below.

All attributes are defined as being read-only for the first target release.

The following table describes the planned Operations of this MBean.

Security

Java Security

If the Derby engine runs under a SecurityManager, the policy file should grant the Derby engine permissions to create an MBeanServer, register MBeans, read Derby properties and perform JMX operations. Such permissions will be included in the default security policy of the Network Server, as well as in the template security policy file included with some Derby distributions.

When using a custom policy file, in order to fully enable JMX management and monitoring the following permissions may need to be granted to derby.jar (the Derby engine code base):

• javax.management.MBeanServerPermission - permission to perform actions related to managing MBeanServers.
• javax.management.MBeanPermission - permission controlling access to MBeanServer operations, mainly for managing the MBeans.
• javax.management.MBeanTrustPermission - represents "trust" in a signer or codebase, which is needed to register MBeans.

Some MBeans may be designed to perform specific actions that may require additional security permissions. For example, a NetworkServerMBean may require that additional permissions related to network connections are granted to the Network Server code base. Otherwise, such an MBean may not be able to connect to/from itself when required. Another example of a permission that may be required by a future MBean is the permission to write certain system properties.

In addition, most MBean actions require a Derby SystemPermission, see each MBean's public Javadoc API and the mbean package descriptions for details.

Sample policy file grants follow. Only the permissions related to JMX management and monitoring are included to increase readability. Users should refer to Derby's template policy file for a full and up to date example.

grant codeBase "file:/home/user/Derby/lib/derby.jar"{
  // other required permissions...

  // management permissions:

  // allows Derby to create an MBean server
  permission javax.management.MBeanServerPermission "createMBeanServer";

  // allows registration and unregistration of MBeans which object name domain is 
  // "org.apache.derby" and which fully qualified class name starts with "org.apache.derby.".
  // It is possible to allow access only to certain MBeans by being more specific in this regard.
  permission javax.management.MBeanPermission "org.apache.derby.*#[org.apache.derby:*]","registerMBean,unregisterMBean";

  // trusts Derby code to be the source of MBeans
  permission javax.management.MBeanTrustPermission "register";

  // Gives permission for jmx to be used against Derby, but
  // only if JMX authentication is not being used.
  // In that case the application would need to create
  // a whole set of fine-grained permissions to allow specific
  // users access to MBeans and actions they perform.
  permission org.apache.derby.security.SystemPermission "jmx", "control";
  permission org.apache.derby.security.SystemPermission "engine", "monitor";
  permission org.apache.derby.security.SystemPermission "server", "monitor";  
};

grant codeBase "file:/home/user/Derby/lib/derbynet.jar"{
  // other required permissions...

  // connection permissions for a NetworkServerMBean, e.g. ping()
  permission java.net.SocketPermission "*", "connect";
};

Authentication and Authorization

Authentication of connections to the management service are handled by the built-in instrumentation of the JVM, as long as the platform MBean server is being used. The default security level with regards to JMX client authentication may depend upon the JVM used to run Derby, see the section on Enabling Management Features. Additional details are available in public documents such as Platform Monitoring and Management Using JMX (J2SE 5.0).

A valid JMX user (a user able to connect via JMX to the MBeanServer used by Derby's Management Service) must not be able to access information or perform operations that would otherwise be restricted by Derby's existing security mechanisms (authentication, authorization, Security Manager, etc.).

Specific soultions which will enable MBean developers to expose more security sensitive information or functionality than what is described in this specification may be provided at a later point in time. It is likely that Java security will be essential.

Limitations

The management features are independent of operating system, but may have limitations depending on the JVM used to run Derby.

JVM

The management features described in this document are only available in JVMs supporting the platform JMX Agent / MBeanServer. This out-of-the-box platform support is available in J2SE 5.0 or newer (see also Dependencies and Management Service).

Support for other JVMs, for example a JVM supporting J2SE 1.4.2 but no newer Java platforms, will not be pursued unless there is significant demand for this in the Derby community. Details from early experiments going in this direction are available in Appendix B.

Error messages

MBean interfaces may throw Exceptions when an operation fails, or if access is denied to a certain attribute, etc. Generic JMX-clients such as JConsole often use reflection to work their way through the Exception chain in order to get to the root cause of the error, and display the original error message.

Some MBean activities may result in Exceptions being trown by Derby and eventually by the MBean being accessed. Since such Exception chains often contain instances of Derby-internal exception types such as org.apache.derby.impl.jdbc.EmbedSQLException, the JMX client often fails to supply the user with a useful error message (this may be avoided by removing or masking internal Exception classes from the Exception chain sent to the JMX client).

A work-around is to include the required libraries in the classpath. For JConsole, this could look like this:

jconsole -J-Djava.class.path=$JAVA_HOME/lib/jconsole.jar:$JAVA_HOME/tools.jar:$DERBY_HOME/lib/derby.jar

Upgrade

No upgrade implications have been identified.

Public API

Derby's public API is directly available to non-Derby applications, and is documented by class- and interface Javadoc which is included with some Derby distributions and available on Derby's web site.

Note that although MBean specific details were up to date when this document was written, this is subject to change. For example, an actual Derby release may include fewer or more MBeans that those described by this document.

Note that all MBean interfaces planned so far are to be included on the engine- or server side of a Derby system. Since Derby's management service is part of the Derby engine architecture, Derby code managing MBeans will depend on the Derby engine, that is derby.jar.

Additional subpackages may be added later, for example org.apache.derby.mbeans.tools for tools-related MBeans.

Documentation

The new features will require changes to the user guides/manuals. It is assumed that the first target release will include little or no such documentation due to time contraints.

The following suggested changes should be considered for inclusion after target release 1. Note that some of the suggestions assume that specific MBean features are available, such as the ability to modify settings and manage the Network Server. These features are not currently planned for inclusion in the first target release (Derby 10.4.1).

• Derby Server and Administration Guide
• (Part 1) Obtaining system information - new task "Obtaining system information by using JMX"
• (Part 1) Managing the Network Server - Overview - new task "Using JMX Management and Monitoring"
• (Part 1) "Managing the Network Server remotely by using JMX" (New topic)
• (Part 2) "Monitoring and Management of Derby by using JMX" (New topic)
• Tuning Derby
• Ways of setting Derby properties - new subtopic "Using JMX to view or change properties"
• Verifying system properties - update + mention JMX
• Derby Developer's Guide
• "Running Derby under a security manager" - the examples should include all of the new permissions required by this spec and listed above under Java Security
• Consider mentioning JMX in the documentation of some or all Derby features that are available through JMX

The documentation should strongly discourage running Derby with JMX Management and Monitoring enabled and with JMX security features disabled in production environments.

All MBeans must be described by valid Javadoc, which must be included as part of Derby's public API. A String representation of each MBean's ObjectName must appear in the class Javadoc of the MBean's public API. Parts of the ObjectName that may vary between MBean instances of the same type must be documented as such (being variable).

Testing

New functional JUnit tests will be added for these new features, in a new test package:

These tests will be included in the org.apache.derbyTesting.functionTests.suites.All JUnit test suite and the junit-all and related ant testing targets.

The new tests will only run in environments supported by the implementation of these features, see Dependencies and Limitations for details.

Test development will be done incrementally. In the first increment, it is expected that the tests will:

• excercise all the public interfaces of all MBeans that are available by default when jmx is enabled
• be runnable in both embedded and client/server mode, with the exception of Network Server-specific functionality, which will only be tested in client/server mode
• be positive, that is testing that features are available and work
• only run with default security settings
• test using a single JMX client only
• test local management only (localhost)

In subsequent increments, tests may, for instance

• include negative testing (testing bad user input, error handling, security)
• connect to the JMX service using multiple JMX clients concurrently, testing performance, security and reliability in that context
• perform testing of remote management (requires a framework for running tests or a test-related JVM (e.g. the Network Server) on a remote host)
• test more combinations of pre-set properties, restarting Derby, various security settings, etc.

Appendix A: Rejected Alternatives

The following table summarizes why various alternatives were rejected:

Appendix B: Enabling support for J2SE 1.4.2

If the dependency on the platform MBeanServer is lifted, it may be possible to enable Derby's JMX management features even on JVMs older than J2SE 5.0. Below are some notes which may be helpful in that regard:

• compile JMX module with JDK 5.0, but with -source 1.4 -target 1.4
• modify ..impl.services.mbeans.BasicManagementService.java to use

      MBeanServerFactory.createMBeanServer("org.apache.derby");
      

  instead of

      ManagementFactory.getPlatformMBeanServer();
      

• implement an alternative to the JDK's Out-of-the-Box management. This should include features such as secure authentication and an RMI server connector.
• run Derby with the JMX and JMX Remote API Reference Implementations in the classpath (download)

Resources:

1. "running jconsole with jdk 1.4.2" (Java Forum thread)
2. "Getting Started with Java Management Extensions (JMX): Developing Management and Monitoring Solutions"
3. "Mimicking the out-of-the-box JMX management agent in J2SE 5.0"
4. "Mimicking Out-of-the-Box Management Using the JMX Remote API"