1) Summary and Motivation In 2003, a Sun Sigma team tasked with producing a pan-SMI strategy for file systems published a document [1] that made several recommendations with regard to storing, accessing, and managing data in a network. Among those recommendations was the creation of new mechanisms and tools to observe the behavior of data servers and their clients. One of the outcomes of that strategy paper was that a development team was tasked with improving the observability of data clients and servers. The team decided that one of its objectives was to provide a general observability tool for file systems that would display statistics for any file system in a consistent but file system independent way. The ZFS team has also expressed interest in this tool and has asked that its priority be raised so that it could be included in an upcoming Solaris 10 update. 2) Overview This fast-track proposes four things: - A general observability tool for file system: fsstat - A new structure for collecting statistics for mounted file systems and file system types: vopstats - A proposed scheme for naming kstats used with vopstats structures - An additional member (v_vsanchor) in the vnode structure used to reference an intermediate bookkeeping structure that references the kstat and vopstats structures. 2.1) fsstat 'fsstat' is a tool that displays kstat data which is collected at the VOP layer on a per mount-point and per filesystem-type basis. 'fsstat' uses existing user-level interfaces such as statvfs(2), sysfs(2), and the kstat library routines to extract data from the kernel. Because the data is collected at the file system independent VOP layer, this tool can be used for all file systems including unbundled Sun-supplied file systems and 3rd party file systems with no additional changes to the system. See Appendix A below for the draft man page for fsstat(1M). 2.2) struct vopstats The kernel changes to support 'fsstat' include a structure made up of kstat_named_t structures. Each vnode operation has at least one associated kstat_named_t structure: n{opname} Keeps count of the number of times this operation has been called. Additionally, I/O-related operations (read, write, readdir) have the following associated kstat_named_t structure: {opname}_bytes Holds the cumulative number of bytes that have been transferred in all calls to this operation. Where "opname" is the name of a vnode operation such as "open", "close", "read", "write", "ioctl", etc. The kstat_named_t structures associated with "read" would then be: nread Number of read calls read_bytes Cumulative bytes transferred for all read calls See Appendix B for a full listing of the vopstats structure. 2.3) Kstat name for vopstats In order to retrieve the vopstats structures, 'fsstat' relies on the kstat(3KSTAT) facility. Optimally, kstat names should be unique for easy retrieval, however, the length of a kstat name is limited to 30 characters plus a NUL character. One of the challenges of this project was to find a naming scheme that relied on existing interfaces and provided consistent naming between the kernel and user levels. There are two cases to consider in terms of kstat naming for vopstats: fstype and mount-point. The fstype is straightforward. Each file system type has a unique entry in the vfssw[] table. That entry contains a unique name (e.g., "ufs", "nfs", "zfs"). That information is available in the kernel as well as in user-space via sysfs(2). Mount-points are more problematic. Using the full path could easily overflow the 30 character limit. The one property that is unique for a mounted file system and is available in a consistent format in both the kernel and at user-level is the fsid. In the kernel, this can be obtained using VFS_STATVFS(). At user-level, it is obtained using statvfs(2). Since we have a unique identifier for FS-types and mount-points, we simply prepend a known string, "vopstats_", to each unique identifier to create a unique kstat name. The following are examples taken from a live test system: Mounted file systems: / ==> fsid = 0x1d80068 ==> vopstats_1d80068 /tmp ==> fsid = 0x49c0002 ==> vopstats_49c0002 /rsbpool/test1 ==> fsid = 0x3f50003 ==> vopstats_3f50003 File system types: ufs ==> vopstats_ufs zfs ==> vopstats_zfs 2.4) New member in struct vnode: struct vopstats_anchor *v_vsanchor The new v_vsanchor member is a pointer to a struct vopstats_anchor structure. This is a bookkeeping structure that includes the vopstats structure which will be updated on every VOP call. 3) Interfaces ________________________________________________________________ | Exported Interface | Classification | Comments | |________________________|___________________|__________________| | fsstat(1M) (CLI) | Evolving | See Appendix A | | | | | | fsstat(1M) (output) | Unstable | See Appendix A | | | | | | struct vopstats | Consolidation | See Appendix B | | | Private | | | | | | | vopstats_{fstype|fsid} | Consolidation | kstat/vopstats | | | Private | naming convention| | | | | | VOPSTATS_STR | Consolidation | vnode.h: #define | | | Private | for kstat naming | | | | prefix: | | | | "vopstats_" | | | | | | v_vsanchor | Consolidation | vnode.h: vnode_t | | | Private | structure member | |________________________|___________________|__________________| 4) References [1] "A Strategy for Storing, Accessing and Managing Data in a Network: Recommendations of the File System Sigma Team" (Morrison, O'Conner, Ross, Weinberg, Wong), November, 2003 http://fs.central/fssg/insight/docs/FSS-plan-v401.pdf Appendix A fsstat(1M) Appendix B struct vopstats Last Updated: 01/20/06 13:13:40 SCCS INFO: @(#) fast-track.txt 1.6@(#)