[I am working on a white paper version of the presentation I just gave at the STC Summit in Dallas. This is an excerpt. If you didn't get a chance to see the session, I'm doing it as a webcast in mid-June (event details) and also (presumably updated) at the tekom conference in November.]

flickr: law_keven

Creating a useful measurement system for document quality requires you to go deeper than just pages per hour. (For software developers, the equivalent sloppy metric is “lines of code per month.”)

We recommend developing a measurement system based on the following ducky components:

• Quality: This measures the correct application of the grammar, mechanics, style guide, consistency, and similar properties. Writing quality is more important for an audience of low literacy users, English as a second language users (assuming the content is in English), and picky users, such as English teachers. Writing quality is generally less important for an audience of highly motivated specialists; for example, software developers reading very technical API documentation.
• Usability: Writing quality measures the ease of comprehension of the text, graphics, or other content by the intended audience. Usability measures the ease of access to the information. To measure usability, you look at factors such as the document navigation system (headers and footers for print; breadcrumbs and the like for online). Did the author employ the proper medium for a particular piece of content? For example, are illustrations provided instead of—or in addition to—lengthy textual explanations? Is the content presented in an attractive, appealing way? Are simulations and video available? High usability is especially important if users can simply choose not to use the product. For example, for a consumer product, such as a cell phone, high usability is important because consumers have lots of options. For products that people must use as part of their job, usability is important to ensure that people can get the job done.
• Accuracy: Does the content describe the product’s features correctly? This factor is especially important for high-stakes documents, such as how to use a machine that delivers radioactive isotopes for nuclear medicine. A mistake in casual game–playing instructions is not of much concern.
• Completeness: Are all of the product features documented? Game documentation often includes only the bare minimum and allows players to discover features for themselves as they play the game. On the other hand, regulated products, such as medical devices, are required to have complete documentation.
• Conciseness: Documents should have as much content as required, and no more. Verbose documents are more difficult to understand, and they increase the cost of localization and printing. This principle is closely related to minimalism.

The overall equation is as follows:

(Q+U+A+C+K)/Cost

Consider your specific environment in refining the calculation for your environment. For
example, you might divided 100 total quality points among the five measurements in
different ways depending on your industry.

Your biggest cost factor is probably the cost of technical communicators. You may also want to factor in other costs, such as software and hardware. Unless you are printing actual paper books, your production costs are probably minimal. (Web servers are cheap!)

What are your thoughts? Do these factors give you a way to calculate your overall document quality?