USACERL Interim Report FF-94/27

June 1994

Evaluating Knowledge Worker Productivity: Literature Review

by

Beverly E. Thomas and John P. Baron

The U.S. Army is increasingly staffed with knowledge workers-professionals who use information as their main input and whose major products are distillations of that information. The U.S. Army Construction Engineering Research Laboratories (USACERL) is developing a computer-based performance support environment intended to improve the productivity of Army knowledge workers. This product, the Knowledge Worker System (KWS), is designed to help work groups enhance their performance while documenting and distributing business process information.

As part of this research, USACERL has identified the need to measure productivity among knowledge workers to recognize any gains that can be attributed to implementation of KWS. The objectives of this interim report are to compile from the literature the most promising approaches to measuring knowledge worker productivity, and to discuss which methodologies may work best in specific knowledge-work environments.

Quantifying knowledge work tasks is difficult. The literature suggests organizations categorize work by content, then select the most appropriate measurement technique based on implementation costs. Inaccuracies in productivity measurement are acceptable if the level of inaccuracy remains constant over time. The measures are most important for tracking trends, not quantifying empirical data.

Approved for public release; distribution is unlimited.

Back to Table of Contents

Back to KWS Home Page

Back to Table of Contents

Back to KWS Home Page

Background

The U.S. Army, like the private-sector American work force, is increasingly staffed with knowledge workers-professionals who use information as their main input and whose major products are distillations of that information. Knowledge workers have outnumbered "blue-collar" employees in the United States since the mid-1950s, and the gap is widening.

The U.S. Army Construction Engineering Research Laboratories (USACERL) is developing a computer-based performance support environment intended to improve the productivity of Army knowledge workers. This product, the Knowledge Worker System (KWS), is designed to help work groups improve their effectiveness while documenting and distributing business process information. The system will not only enhance the performance of individuals and groups, it will offer the organization a medium for process improvement.

As part of the development of KWS, USACERL has identified the need to measure productivity gains among knowledge workers. Specifically, there is a need to determine productivity gains that can be attributed to implementation of KWS. While productivity has been studied for decades and knowledge work has always existed, it is only recently that researchers have tried to measure knowledge worker productivity.

The concept of productivity has existed for a long time, and the idea has many different applications. This discussion addresses the meanings that refer to work and economics.

One basic way of defining productivity is "output divided by input" (O/I). If Company X uses 100 units of input to produce 100 units of output, their productivity ratio is 1. To interpret this formula in economic terms, one can substitute dollars for the input and output units, i.e., $100 of output divided by $100 of input produces the same productivity ratio of 1. Using money as a measure of value makes it possible to compare dissimilar inputs and outputs.

Productivity change-the measure of productivity this research addresses-refers to the change in the productivity ratio over time. If in the above example the ratio of outputs to inputs was measured at a later date and was found to be $200/$100, the new ratio would be 2. The change in productivity would be (2-1)/1 or 100 percent. A problem with this formula is that if Company X achieved this improvement in productivity and responded by cutting the price of its output in half, the measured productivity change would be zero even though there was a real improvement.

Productivity, defined by O/I, requires that the units of input be measured in some manner. The early applications of productivity measurement addressed simple, repetitive jobs of short duration. Several measurement techniques were developed for this purpose, including the well-known time-motion and stopwatch studies. Such techniques were designed to measure frequent actions that are easily observed and counted.

As long as the workforce consisted largely of manufacturing jobs, these techniques were adequate. The early measurement techniques, however, are not well suited to "white-collar" work because such work is not repetitive or simple. White-collar workers have become a large fraction of the workforce, and their number will continue to grow. Therefore, the productivity of an increasingly large part of the workforce cannot be measured by traditional methods.

Although it has only recently been given a special name, knowledge work has been around for centuries. Throughout history there have been managers who were paid not for what they produced, but for what others produced. This is an example of knowledge work in a very basic form.

Today the variety of knowledge workers ranges from managers to analysts to programmers to lawyers. The common denominator of these professions is their use of knowledge in their work.

Back to Table of Contents

Back to KWS Home Page

Approach

An extensive search of work measurement literature was conducted. More than 100 journal articles, papers, and books were reviewed. Topical areas reviewed included work measurement, productivity, organizations, psychology, decision theory, and quality improvement. Several methodologies were examined for applicability to the kinds of environments in which Army knowledge workers operate, and the most promising were identified.

A glossary of productivity-related terms (such as "blue collar" and "white collar") is included in Appendix A.

Scope

This report does not address the topic of activity-based costing, which is too extensive to be covered here. That topic will be discussed in a separate report.

Mode of Technology Transfer

The findings of this study will be incorporated into a final USACERL technical report addressing productivity measurement for knowledge workers. This research will feed into continuing USACERL work units whose objective is to develop an integrated performance support environment for Army knowledge workers.

Back to Table of Contents

Back to KWS Home Page

Productivity and Business Objectives

Many people would define a business in terms of making profits, but such a definition is too narrow. In a broader sense, the first valid business purpose is to create a customer (Drucker 1974). He says every business must satisfy its customers or it will fail.

It is the customer who determines what a business is. It is the customer alone whose willingness to pay for a good or service converts economic resources into wealth, things into goods. What the business thinks it produces is not of first importance-especially not to the future of the business and to its success. The typical engineering definition of quality is something that is hard to do, is complicated, and costs a lot of money! But it isn't quality; it's incompetence. What the customer thinks he is buying, what he considers value, is decisive-it determines what a business is, what it produces, and whether it will prosper. And what the customer buys and considers value is never a product. It is always utility, that is, what a product or service does for him.

A business converts economic resources into something else. It may do so well or poorly. At this level, productivity is the balance between all production factors that will give the greatest return for the least effort (Drucker 1974). Productivity at the organizational level is considered separately from productivity at lower levels.

The customer buys utility (Jury 1992), and productivity associates outputs with inputs. Productivity, at the organization level, may be considered a measure of how well the company satisfies the customers' utility. Therefore, productivity measurement shows how well a company is doing. This does not, however, tell anything about why the company is performing the way it is. To discover why, productivity must first be examined at lower levels such as the work group, which are best suited for using productivity measures as an indication of change (Rittenhouse 92).

The concept of productivity is often vaguely defined and poorly understood, although it is a widely discussed topic. Different meanings, definitions, interpretations and concepts have emerged as experts working in various areas of operations have looked at it from their own perspectives (Sardana 1987). But a different view is that the terms `performance' and `productivity' are used incorrectly. People who claim to be discussing productivity are actually looking at the more general issue of performance. Productivity is a fairly specific concept while performance includes many more attributes.

The white-collar sector, which is primarily composed of knowledge workers, represents 64 percent of the U.S. workforce. The blue-collar sector, which includes only a very small number of knowledge workers, represents only 33 percent of the U.S. workforce. (The remaining 3 percent is attributed to farm workers.) The white-collar workforce is 36 percent clerical, 12 percent sales, 31 percent professional, and 21 percent managerial (Anthony 1984). The managerial and professional sectors of the white-collar workforce increased by 25 percent from 1972 to 1986 (Davis 1990).

Knowledge work is the area that offers the greatest opportunities to increase productivity (Drucker 1974). In the past, the production line received a lot of attention because it was relatively easy to analyze and measure. On the other hand, management does not clearly understand what goes on in white-collar work areas, or how to match white-collar personnel needs to future business needs (Strassman 1985, Shackney 1989).

The production environment has been measured heavily and continues to dominate productivity efforts in spite of evidence that the returns on further refinements do not equal those possible in the white-collar environment.

Back to Table of Contents

Back to KWS Home Page

In a paper presented to the Center for Economic Policy Research at Stanford University (Lau 1983), Lawrence J. Lau commented on productivity as follows:

By comparing the sets of production possibilities of an economy at two or more different points in time, we infer whether there has been a change in the productive potential, that is, whether there is any input-output combination that is feasible at the later date but not feasible at the earlier date or vice versa.... What is interesting, in a world of scarcity, is whether we can obtain the same output with less resources, or a higher output with the same resources. This is where improvement in productivity or technological progress becomes important. The principal reason for our interest in the measurement of productivity is to identify and quantify technological progress.

Using the simplest theoretical example-one input and one output-if input increases, a corresponding output increase is expected (if inputs are not squandered and the system is rational). Figure 1 shows this relationship. When technology changes, so does the relationship between input and output. Figure 2 demonstrates this change as it affects the example in Figure 1.

Figure 1. One Input and One Output.


Figure 2. O/I With Improved Technology.

The output/input lines shown on these graphs depict the maximum output achievable for a given level of input. Lau (1983) labels this the "production possibility frontier." For any amount of input, this line shows the level of output the economy must produce to be considered efficient.

These examples may seem to imply that a change in productivity is easily quantified-but it is not so simple. The function that determines the production possibility frontier is normally unknown, and even experts do not always know all factors that affect it. Figure 3 shows an example: more than one line "explains" the increased productivity. In comparing Figure 2 with Figure 3 an alternative explanation for the change in output can be seen. Technology improvement is not necessarily the cause because the shift in production, from point A to point B, also shows on the third (curved) O/I line. The change from Ob to Oa may be the result of new technology, increased inputs, or both. And this is a simplified example, with only one input and one output; complex relationships are much harder to analyze.

Figure 3. An Alternative Example.

Lau (1983) mentions seven such difficulties with this economic analysis. The three most relevant to measuring productivity are inefficiency, input quality changes, and nonconstant returns.

Inefficiency

Figure 4 shows another simple O/I line. The area under the line represents all the points of inefficient input utilization. The line indicates the points of maximum output for the given input.

Figure 4. O/I Graph Showing Inefficiency.

Input/Output Changes

Output may drop into inefficiency if input quality is lowered. This is not necessarily true inefficiency, however. When input quality degrades, the lines must be redrawn accordingly. Only then can one tell whether output efficiency has declined.

Nonconstant Returns

The present assumption in the O/I line has been that each new unit of input will produce the same amount of output. This is not always true, particularly in more complex models. Nonconstant returns can change the straight O/I line into a curve, a stepped line, or even a discontinuous line or curve.

These three problems arise in even the simplest of models. As the number of inputs and outputs increases, so does the number and complexity of the problems.

Back to Table of Contents

Back to KWS Home Page

G. D. Sardina and Prem Vrat (1987) have compiled 20 definitions of productivity relevant to business. They say :

A large number of concepts consider productivity as an output-input relationship relevant mostly to a production system, implying that an organization works as a physical system with variables and their inter-relationships amenable to precise definitions. The basic reliance is on the acceptance of a stimulus-response model of causality that an input causes an output. This conceptualization apparently creates a bias towards production function or allied activities to the exclusion of other economic as well as non-economic performance outputs, such as achieving a share of the markets, new product introduction, completion of schedules, societal goals etc. These and several other non-economic performance consume the input resources and as such should get fully projected in a model to measure productivity. Similarly, factorial productivity measures connected with input factors such as labor, capital, etc., are misleading and inadequate. Firstly, the input factors cannot be studied in isolation to one another. Improvement in one factorial productivity is generally at the cost of the other. Besides, an input factor like labor is present everywhere. Secondly, an important input like managerial resource finds no place as an input factor in such measures.

Sardina and Vrat say those who measure productivity should have three objectives: (1) to identify potential improvements; (2) to decide how to reallocate resources; and (3) to determine how well previously established goals have been met. Sardina and Vrat use a broad definition of productivity that tells the observer how the measured organization is doing as a whole.

Productivity can be separated into two factors: performance and financial (Moore 1978). Performance productivity is based on the number of outputs produced. For example, if Company A produces 100 units one week and 120 the next, its performance productivity has increased by 20 percent. By contrast, financial productivity focuses on the value of the output. If Company A had produced 100 units in both weeks, but raised the price from $1.00 per unit to $1.20 per unit in the second week, its financial productivity would have increased by 20 percent with no increase in output.

Both measures can be misleading. Figure 5 shows these relationships. If Company A sold $100 worth of units in both Week 1 and Week 2 what is the change in productivity? From both a financial and a performance viewpoint, there appears to be no change. Suppose, though, that in Week 1, 100 widgets were produced and sold at $1.20 each. Then, in Week 2, 120 widgets were produced-an increase of 20 percent-and the price was dropped by 16.7 percent. The result is 120 times $1.00, or $120 in sales (see Figure 6). From a financial viewpoint there is no change, but from a performance viewpoint there has been change. Which viewpoint is correct?

Figure 5. Performance and Financial Productivity.


Figure 6. Combined Productivity.

Sink (1984) confines productivity to its simplest form-O/I. He states, "Productivity, as mentioned, is strictly a relationship between resources that come into an organizational system over a given period of time and outputs generated with those resources over the same period of time. It is most simply Output divided by Input." He also states that managers create confusion about productivity because they do not distinguish between productivity's definitions, measurement, and improvement on the one hand, and performance's concepts, measurement, and improvement, on the other. This failure to distinguish between productivity and performance, can make communicating about productivity difficult.

In the private sector, productivity is typically seen in terms of profit or sales. But how can productivity be understood in the public sector? The Bureau of Labor Statistics has collected productivity information from 304 organizations in 62 agencies, which represents 64 percent of the Federal executive branch civilian workforce (Forte 1992). In the Federal government, productivity measures focus on defining output and determining resource requirements, establishing accountability, and helping in the estimation of production goals. Outputs must be countable, similar over time, significant-their absence would be a cause for change-and the end result of some process. These criteria define high-level outputs and may not relate directly to the work in a specific work group. But from the higher-level output, some lower-level outputs can be established, based on their contribution to the final output. Separate measures can be developed for support groups, for use as supplemental management analysis tools. These measures are not related directly to the organizational outputs, which demonstrates that there can be a number of views of productivity-and therefore a number of different measures of productivity. This flexibility is not unique to the public sector.

Back to Table of Contents

Back to KWS Home Page

There is a distinct difference in the productivity of an organization and the productivity of a single work unit of that organization. Sardina and Vrat (1987) indicate this difference by use of their third objective-to establish measures that reflect an organization's degree of success in meeting its established goals. The goals for each level of the organization should differ to represent the contribution that specific level expects to make toward overall organizational goals. Therefore, each level's productivity evaluation should be different, reflecting its unique goals.

Economic theory differs when applied at the national level from when applied to an individual business. One is called macroeconomics and the other microeconomics. Productivity may be viewed in a parallel manner, with macroproductivity referring to productivity at the national level, microproductivity referring to productivity at the business level, and nanoproductivity referring to productivity at suborganizational levels. A general definition of productivity is possible, but to use it one must indicate the intended level of use, i.e., the national economy, firm, plant, department, or the individual (Thor 1988).

At the level of nanoproductivity, more detail is involved. Individual work units and workers are observed. At this level, productivity evaluation must take into account different types of work (as discussed in Chapter 3). Historically, work has been separated into blue-collar and white-collar categories. This view can be expanded to include knowledge work as a third category (Beruvides and Sumanth 1987). Knowledge work is all work whose output is mainly intangible, whose input is not clearly definable, and that allows a high degree of individual discretion in the task. This difference in work content requires different approaches to productivity evaluation.

This idea of additional classes of work has been talked about before, but were not so precisely defined (Drucker 1974). The difficulty of measuring something that is not clearly defined has been noted. An expanded definition of work that includes a category for knowledge work is a first step in the evaluation of knowledge worker productivity.

There is a great need to evaluate the productivity of knowledge work-and the need grows greater each year. Under the old classification of work, in which there were essentially only two categories, white-collar workers represented two-thirds of the workforce in the early 1980s, with the managerial and professional subgroups representing one-quarter of the workforce. Yet, productivity in knowledge work has shown little improvement over the past decades (Davis 1990).

Back to Table of Contents

Back to KWS Home Page

Evaluating productivity is never more difficult than when evaluating knowledge work. Consequently, this type of productivity evaluation is poorly understood (Drucker 1974; Salemme 1986). There are several reasons why knowledge work is so hard to evaluate.

First is the problem of inertia. If work is being measured and rewarded, those reaping the rewards will want it to stay the same. The areas and the types of work that have been measured in the past continue to get attention today. Problems associated with measuring new areas of work are seen as roadblocks rather than challenges. Planning and work measurement in the knowledge worker areas is not conducted as scientifically as it has been in other areas (Magliola-Zoch 1984). However, this inertia is diminishing as increasing numbers of studies show how to evaluate knowledge work, and as the potential benefits continue to grow.

A related problem is that individual productivity increases do not transfer to the productivity of higher levels of organization (Rittenhouse 1992). This is often the case for knowledge work, as the work flow is not tightly linked, and change in the productivity of one worker may not affect anyone else. This makes it seem as if measuring the productivity of knowledge workers will not change anything. But this does not mean knowledge workers should not be measured at all. Both Rittenhouse (1992) and Sassone (1991) correctly point out that the work group is the proper level at which to evaluate knowledge worker productivity.

Most of the remaining problems in applying productivity measures to knowledge work result either from the intrinsic complexity of the work or from disagreements about what to evaluate. The complexity of knowledge work arises from several factors. It is not routine, involves much independent judgment, and requires several people to work together. Furthermore, a considerable amount of knowledge is required to do the work.

The nonroutine nature of knowledge work means that it is very difficult to measure a norm. There is no obvious average to observe and record, so any measure will be somewhat inaccurate. The degree of independent judgment involved in knowledge work means that the "norm" may vary from individual to individual. Each person can accomplish the work in his or her own way, further complicating measurement of a norm. The dependency of one worker on another can mean that, although one worker is performing very well, the problems of another worker determine the overall performance.

The question of what to evaluate also stems in part from the complexity of knowledge work. Productivity measures applied to white-collar workers often concentrate on the countable results of the work rather than the work itself, which is information (Wilson 1988; Salamme 1986). The work is so complex that an artificial indicator is evaluated rather than the actual work. Often, the indicator is chosen because it is easily quantified. This approach ignores potentially important aspects of the output, such as quality (Rittenhouse 92). The value of the output, which includes its quality, is very important in knowledge work. This value is the primary output.

Back to Table of Contents

Back to KWS Home Page

Appendix B lists performance measures compiled (from Key Criteria for Performance Measurement, memorandum from Comptroller, DoD [25 October 1992]) by the Department of Defense (DOD). The discussions that follow repeatedly refer to these terms and measures.

Bridges (1992) gives one fundamental reason for measuring productivity: "Some type of benchmark (standard, average, mean) should be determined, if none exists. How can you be sure of how much is being saved if you do not have a baseline?" Peter Drucker (1974) has put it in a more general way: "Without productivity objectives, a business does not have direction. Without productivity measurement, it does not have control."

Measurement requires collecting data. Sink (1985) categorizes three basic ways to collect data about a given phenomenon or organizational system: inquiry, observation, and collecting system data or documentation. This data gathering is the essential part of measurement. It is the process by which productivity benchmarks are established. In the simplest form, the outputs are evaluated against the inputs, but even at this simple level terminology may be a problem. Some writers include nonquantitative indicators such as quality in their definition of "output," but others confine the discussion of productivity to O/I. The definition affects the type and amount of data gathered.

Productivity measurement is an indicator of how well the goals of a work group are being met. Whether a tight or loose definition of productivity is used, the validity of the results will depend on the validity of the input.

Back to Table of Contents

Back to KWS Home Page

Many measurement techniques and packages are available. Mundel (1989) presents a computer software package that evaluates productivity. Direct adjustments for quality by the package are excluded, but quality indicators may be implicit because the package considers only good output. The program does not consider raw materials, because the end product is knowledge.

In this and other computer programs, simple O/I algorithms are used to calculate productivity. The programs facilitate the calculation of productivity at the organization level. Mundel presents eight levels of work units, starting from the lowest-motion-up to the highest-results achieved because of outputs.

Sassone (1991) presents a technique that is relatively simple to implement. He classifies work by the lowest level of employee who could reasonably do it. Work is then recorded for each participant by the type he or she is doing. This record is then analyzed and compiled in a matrix format that shows the amount of effort expended by each type of employee, and whether employees are working at, above, or below their level. This information can indicate the mix of workers is needed in a work group. It can be used to explore the consequences of common assumptions, such as whether cutting support staff will actually reduce costs.

Sink (1985) presents several techniques of evaluating productivity. His three main methodologies are Multi-Factor Productivity Measurement Model (MFPMM), Normative Productivity Measurement Methodology (NPMM), and Multi-Criteria Performance/Productivity Measurement Technique (MCP/PMT). MFPMM is a computerized methodology for measuring productivity, based strictly on O/I. NPMM uses structured group processes to formulate appropriate productivity measures for white-collar or knowledge workers. It uses the group technique to establish consensus about what the productivity measures are and how they should be measured. MCP/PMT is designed to allow the user to evaluate the various productivity measures and decide which are the most important. It also allows the user to aggregate dissimilar productivity measures.

A number of other researchers use the group technique. Bernard (1986) discusses project teams and stresses maximizing their diversity, warning that it cannot be assumed that the manager knows what is going on. Thor (1990) talks about Normative Group Techniques (NGT), what they do, and how to use them. He strongly recommends the participatory approach of NGT for knowledge workers. The groups should be planned to get the most out of the available personnel. To avoid partisanship, each group should have a facilitator who is familiar with the technique but is relatively unknown by the group.

Kristakis (1984) describes a methodology that depends on estimation. The manager lists the types of work processes performed in the group, then breaks them down into detailed operations. He or she identifies who does what and estimates how long each process takes. This is a very simple technique, but it may not be accurate because it relies on the judgments of only one person.

Anthony (1984) discusses the use of time diaries, estimates, work sampling, and direct observation. He used these techniques on professional and technical staff. The data were analyzed by computer, then reviewed to eliminate insignificant tasks. Anthony concluded; "Although many people think that professional activities are nonroutine and nonrepetitive, we have found that if the scale of reference is expanded, they reoccur on a predictable basis." Overby (1984) discusses work sampling at predetermined periods rather than at random times during the day.

Back to Table of Contents

Back to KWS Home Page

A common theme among researchers is that knowledge worker's productivity can be measured (Bernard 1986; Sink 1984; Anthony 1984; Magliola-Zoch 1984). These writers offer several suggestions to make measurement simpler and acceptable to the workers. First, the workers must participate in the establishment and evaluation of the measures of productivity. The more they are involved, the less likely they will feel threatened. Second, any process that seems too complex to measure is likely to have less complex subprocesses, which are more practical to measure. Third, always use the best measure for the job, even if several different measures must be pursued for different processes. Fourth, do not expect absolute accuracy, but try for the best that is economical. Finally, regardless of the shortcomings, measuring is better than not measuring.

Seeking a Unified Concept of Productivity Management

The literature review shows that productivity measurement is discussed from a wide variety of viewpoints. A variety of implementation methodologies have been developed for different applications. What is lacking is a concept that unifies these diverse views. This section discusses several aspects of such a unifying concept.

In discussing productivity, the terms "measurement," "evaluation," "performance," and "improvement" are used in different ways by different authors.

The strictest interpretation of productivity is outputs divided by inputs (O/I). A number of people use this interpretation because it is easily defined, calculated, and implemented. "Performance" is a broader term than "productivity." It includes factors that are not easily quantified, such as quality, customer satisfaction, and worker morale. The inclusion of these fuzzy terms into the mix reduces the crispness of the measure and makes the calculation more difficult. However, these terms more fully describe what actually occurs in production. The difficulty in applying productivity measures frequently can be attributed to an overlapping of these two subjects.

"Productivity measurement" refers to the way in which productivity is indexed. In the strictest sense, a measurement is a numerical index. Consequently, the same inputs should produce the same outputs-that is, the same index number-each time the output is calculated. The advantage of this is that the index does not depend on who collects the data or when it is collected. "Measurement" also has a meaning by itself. It is the methodology of establishing the amount of work involved in a work function.

"Evaluation," a term used in this report, is similar to "measurement." Evaluation allows the use of measurements that are not strictly quantitative. Rather than being restricted to measures that are quantifiable, one may use qualitative measures such as "good," "bad," "poor," "superior," "fast," etc. This makes manipulation of the measures difficult, but allows previously unmeasured aspects of work to be measured. The application of fuzzy mathematics to such terms may someday make them more useful.

"Productivity improvement" refers to the change sought, noted, or measured in productivity. Productivity improvement can refer to the designed change in an operation to produce a positive change in the measured productivity of that operation. The term can also refer to the change in productivity that results from such a design change.

Bridges (1992) states, "The keystone to implementing productivity improvements is putting everything in measurable terms." Frazelle (1992) says "productivity must be understood before it is effectively measured." Productivity improvement is tied to productivity measurement, which is tied to the measurement of the work. The beginning step is measuring work.

Back to Table of Contents

Back to KWS Home Page

Historically, knowledge work has been exempt from productivity evaluation because of its complex nature and its minor contribution to the total cost. It has long been thought that more could be accomplished in the structured work of the production line and similar jobs. Managers have dismissed productivity measurement in the knowledge work areas because they assume that it is of low importance and that, if productivity cannot be measured with the same accuracy as in a production area, it is a useless measure (Chew 1988).

Productivity measurement systems are often unwelcome to both managers and workers (Sink 1987). A number of authors have written about the need to prepare the work area to be analyzed (Helton 1991; Salamme 1986; Sink 1987). Such preparation ranges from discussion to group participation to self-evaluation. Preparing the area in some manner makes it possible to implement a productivity program. However, a bad program will produce bad results.

Worker expectations are another barrier to implementing productivity measurement in the knowledge work area. This is partly due to the history of productivity and partly due to human nature. Historically, productivity efforts have produced detailed and highly organized results. The approach has been very structured and well documented. People are highly reluctant to accept anything that is less structured, less well documented, less detailed, and less accurate. Yet that is the nature of knowledge work, so productivity measures of knowledge work are inherently more loosely structured and less accurate than measures of other types of work.

Perhaps the strongest objection to measurement of knowledge worker productivity is that its results are inaccurate (Chew 1988). Still it is better to measure inaccurately than not at all. In addition, productivity measurement is most valuable as a dynamic measure, not as a static measure. This means that as long as measurement inaccuracies are consistently inaccurate, the dynamic measure will be an accurate indicator of the relative change.

Back to Table of Contents

Back to KWS Home Page

There was a time when "blue collar" and "white collar" were considered as opposite as black and white (Beruvides 1987). Today, this distinction is not accurate. The important issue is how these terms relate to the work content? These terms do not really say anything about the work being done. Work must be categorized by its content (Helton 1991; Strassman 1985), and work content is not one-dimensional, as implied by the old white-collar/blue-collar distinction.

The authors propose categorizing work by eight components, as detailed in Table 1. Figures 7-10 show the components of work arrayed on a horizontal scale. Each characteristic is represented by a horizontal line, and is scaled from high to low.

Figure 7. Two Examples of Knowledge Work


Figure 8. Two Examples of Blue-Collar Work.


Figure 9. Expected Graph Area of Knowledge Work.


Figure 10. Expected Graph Area of Blue-Collar Work.

The graph is set up so inversely related components are at opposite ends and strongly related components are grouped together. For example, "Decisionmaking" and "Knowledge Use" are directly related to "Complexity" by definition. "Structured" is inversely related to "Complexity," so these two components are at opposite ends of the graph. There is not a lot of "Complexity," as defined, in a very structured job-the amount of decisionmaking and the knowledge used is low. This means that "Structured" is also inversely related to "Knowledge Use" and "Decisionmaking." "Volume" is directly related to "Time per Job" and partially related to "Repetitive," "Structured" and "Complexity." Table 1 defines all eight components in more detail.

Figure 7 shows two examples of knowledge-intensive work plotted on the graph. Figure 8 shows two examples of what typically has been called "blue-collar" work. Figure 9 demonstrates the area into which very knowledge-intensive work would plot and Figure 10 does the same for "blue-collar" work.

There are many possible graphical ways to represent work. Figures 7 and 8 are examples using USACERL's proposed methodology. Regardless of the representation methodology used, there are some constant relationships among the components. These relationships account for the general slope of the lines in Figures 7 and 8. One would expect knowledge-intensive work to have a negative slope, i.e., the value of the components of work will decrease down the list. One also would expect that the skilled work, or blue-collar work, would have a positive slope. Neither of these slopes is expected to be perfect; rather, they are expected to indicate the knowledge or skill level of the work being examined.

The graph is set up so inversely related components are at opposite ends and strongly related components are grouped together. For example, "Decisionmaking" and "Knowledge Use" are directly related to "Complexity" by definition. "Structured" is inversely related to "Complexity," so these two components are at opposite ends of the graph. There is not a lot of "Complexity," as defined, in a very structured job-the amount of decisionmaking and the knowledge used is low. This means that "Structured" is also inversely related to "Knowledge Use" and "Decisionmaking." "Volume" is directly related to "Time per Job" and partially related to "Repetitive," "Structured" and "Complexity." Table 1 defines all eight components in more detail.

This proposed methodology is expected to be refined over time. The authors suggest that this approach demonstrates that the best way to describe work is by its component content. Using this approach gives a true picture of the work structure, which will allow a match of measurement techniques to the actual work.

Back to Table of Contents

Back to KWS Home Page

The measures discussed here are for measuring the amount of work done. Indirectly, these affect productivity. Where knowledge work is involved, work becomes more important than outputs in calculating productivity (Sink 1987). While knowledge workers may be using expensive equipment, the budgets for their areas usually consist primarily of salaries and benefits. As difficult as it may be to directly link knowledge work to outputs, it is even more difficult to link the knowledge worker's equipment to the same outputs. The knowledge work itself is often used to tie equipment use to outputs. This further increases the importance of the work in calculating productivity. Measurement, as discussed here, is a determination of the labor involved in the tasks performed by the work group.

Evaluation measures are not all alike. They differ in complexity, accuracy, adaptability, and applicability. There are many specific methods, but this discussion will focus on the categories that can be constructed to classify measuring techniques.

Many tags might be used for classification. These range from who performs the measurement, to how it is done, to how long it will take. The purpose here is to categorize the measurement techniques in a way that allows matching them to work based on content. The complexity of the measurement technique is often a good indicator of the type of work it is best suited to measure.

Note that these are not absolute matches. Sometimes the best method is different than expected. This does not invalidate the general approach to categorization because it is intended primarily as a guideline.

Complex measures often produce the most accurate results, but they are the most difficult to implement and often the most time-consuming. They are justifiable in situations where the return warrants the expense. Such measures are usually best used for work that is very repetitive, of short duration, unchanging, easily counted, and high-volume.

Very simple measures usually produce less accurate results, but are simple to implement and require less time. Their use is justifiable where it is impossible or not cost-effective to use more precise measures. These techniques can be used for any type of work, but are best reserved for complex tasks that occur infrequently, at random times, and at different levels of complexity.

Simple measures are the most generally applicable, and can be used with any type of work, but they are not the best technique for all types of work. Complex measures apply to fewer types of work, but when they are applicable, they produce better results than simple measures.

Table 2 groups the techniques, starting with the most complex and ending with the simplest. The groupings are based on the complexity of setting up the analysis and conducting the evaluation.

The more complex techniques require more expertise to design and implement. The techniques in Category 1 usually require extensive preparation. The work has to be analyzed and described. Data must be gathered on frequencies and volumes. A measurement plan has to be devised-one that fairly represents the work being evaluated. The implementation for Category 1 usually requires an analyst with a high level of expertise in the techniques being used. Techniques in Category 2 can be simpler to implement because setup involves simple measures designed to be performed by those involved in the normal workflow. But the preparation in Category 2 is difficult: the work must be understood so valid measures can be designed.

Category 3 is simpler to set up because the process is a continuous one, and much of the setup difficulty in Categories 1 and 2 can be spread over time. Implementation is only moderately complex because it is a continuation of the initial setup process. Recall that the inclusion of the workers and management in the design of any work analysis project in a knowledge work area is essential to the project's acceptance and correct result (Salemme 1986; Bernard 1986).

In the knowledge work environment it is important to understand that an individual's performance can vary over time, and that the difference in performance of the same work by two different individuals can be substantial (Davis 1990). It is also important to remember that the apparent inability to apply measurement techniques can often be attributed to the perception that the job is simply too large or complex to measure (Anthony 1984). Sometimes looking at individual parts of the job can make measurement easier (Magliola-Zoch 1984). Some people suggest starting with a definition of the group's products and working backwards to the lowest logical division of the work (Mundel 1989; Magliola-Zoch 1984). Others recommend examining the responsibilities for the work performed (Sassone 1991; Helton 1987).

Several of the work components discussed previously have a direct relationship to the types of measurements that should be employed. Highly repetitive work, for example, is best measured by techniques based on norms, such as time-motion studies. Nonrepetitive work is not suitable for such techniques. The time required per job is another component that directly affects the methodology used. If each job takes a long time, it does not make sense to time the work on a stopwatch. A work log is more applicable in such a situation. The higher the volume of the work, the more cost-effective any measurement technique will be. Techniques that are costly-those requiring a lot of effort and expertise-are best applied to high-volume work. Work with a high level of decisionmaking or complexity are poor candidates for stopwatch or time-motion techniques. These require a less structured technique.

It can be seen from the above discussion that any proposed productivity measurement technique should be examined to determine what it requires to function well. The work to be analyzed should be classified by its components so the measurement technique's applicability can be evaluated. Measurement techniques often must be tailored to fit the organization using them. Rather than exhaustively listing measurement techniques, the authors have provided some broad classes of techniques for the reader's reference (Anthony 1984; Bernard 1986; Davis 1990; Helton 1987; Magliola-Zoch 1984; Mundel 1989; Salemme 1986; Sassone 1991; Sink 1987).

Back to Table of Contents

Back to KWS Home Page

It appears clear that work should be evaluated by its own content, not on the basis of the old white collar/blue collar model. The approach to measurement has been shown to fall into several categories, and each category can be associated with the various elements of work content. Therefore, it is clear that any measurement techniques used to evaluate a work group should match the content of the work being performed there. In some units, each different type of work being performed may require different types of measurements (Thor 1987; Drucker 1974).

An example of the need for mixed measures can be illustrated by a work group that processes paperwork. In a complaint processing department, for example, a large section of the department may be devoted to routine processing of paperwork. This group may be thought of as the input section. This processing is very structured, repetitive, and high-volume, which would indicate that one of the more complex measurement techniques is applicable. The remainder of the department-called the investigation section in the example-may be involved in complex paperwork processing in which the work varies with each assignment, and the process is not easily counted. Based on the logic developed here, it seems that a simple measurement should be used. Choosing either one technique or the other may not be the optional solution.

Using the simple technique throughout the complaint processing department would mean losing some accuracy for the input section. Another option would be to measure only the input section using the more accurate technique, but in that case no data would be collected on the investigation section. A third possibility is to measure each section with the technique most applicable to it. The task would be more difficult, but no accuracy would be lost.

The choice of action in the example above would depend on the relative importance of the two work sections to the overall productivity measurement of the area. If the investigation section consisted of one person and the input section had 30 people, measuring the investigation section would have only a minimal effect on the productivity measure, but would incur substantial costs. The same principle would hold if the sizes of the sections were reversed and, in this case, the input section could be ignored due to its relative insignificance. If, however, the investigation and input sections were of similar size, then both should be included in the measure.

How to measure and what to measure is a complex decision. As demonstrated in the example, taking a single measure is not necessarily the best solution. The best way to measure depends on the cost, effort, and need. Lower levels in an organization require more detail than higher levels in the same organization (Rittenhouse 1992). At a departmental or work group level, detail is needed, but cost and available resources may dictate the use of a less-than-perfect measurement mix.

The total productivity measure is usually synthetic: it is derived from any number of other productivity measures and has no direct relationship to any specific activity. This type of measure is most often used at higher organizational levels to reduce the complexity and proliferation of productivity measures to be analyzed. At the higher levels in an organization, productivity is not directly related to any single workgroup, and there is frequently no need to explain why productivity changed from the previous measurement. Lower levels of the organization can also use a total productivity measure, but it will simply indicate how well the group is performing from period to period without providing insight into why.

Back to KWS Home Page

Back to KWS Home Page

Extensive review of the literature indicates that the possibility of measuring the productivity of knowledge work environments is acknowledged, but practical implementation lags far behind. The causes of this lag are based on the perception that knowledge work is unmeasurable and of little significance. The authors have shown that knowledge work is by far the area where measurement offers the greatest potential benefits. It is more difficult to measure knowledge worker productivity than it is to measure blue-collar worker productivity. This does not mean that knowledge work cannot be measured, but that more innovative measurement techniques are needed.

Categorizing work by its content components-decisionmaking, complexity, knowledge use, structure, repetition, volume, time, and skill level-facilitates understanding the work and how to best measure it. Picking a measurement technique appropriate to work content is only part of the job. Cost and accuracy must also be factored into the decision.

Measurement techniques vary in implementation costs and accuracy. Costly, accurate techniques are most appropriate for work whose content allows for accurate measurement and justifies the costs. Less-expensive, less-accurate techniques are available for work whose content prohibits a high degree of accuracy and where cost is a major deterrent to measurement. Table 3 recaps this information.

Productivity measurement is absolutely necessary for understanding knowledge worker productivity. The complexity of the work should not be a roadblock to measurement, but should only indicate which measuring technique is most appropriate. Some measure is better than no measure, but any proposed technique should be examined to evaluate its applicability to a specific type of knowledge work. Measurement techniques can be customized to fit a specific organization and work group.

Back to Table of Contents

Back to KWS Home Page

Back to Table of Contents

Back to KWS Home Page

Terms			Definitions 
Blue-collar		Work of a manual or physical nature. Its end result should be									
			tangible and identifiable, and it should be directly related									
			to the product being produced. It is also highly structured.
Crispness		The lack of ambiguity in the representation being discussed.									
			Formulas and measures are crisp when they can be defined and									
			applied with no ambiguity.
Effectiveness		Refers to the quality of the output produced considering the									
			inputs used. In comparing effectiveness to efficiency,									
			effectiveness is referred to as "doing the right thing" while									
			efficiency is referred to as "doing the thing right."
Efficiency		Efficiency is defined by the use of inputs in relation to the									
			production of outputs. Efficiency is used in defining									
			productivity, which is a broader term.
Evaluation		A means of classifying something. The quantifier used need not									
			be numerical. If it is numerical, it does not need to be highly									
			structured. This is the feature that differentiates evaluation									
			from measurement.
Fuzzy			Mathematics uses fuzziness to deal with information that cannot									
			be represented as a binary concept, such as on/off, black/white,									
			etc. Fuzzy measures allow the capture of uncertain data and make									
			it possible to quantify or process the information.
Goal			In productivity, a level of productivity that is anticipated. It									
			may also refer to a level of quality that is anticipated.
Input			The beginning element of a process. Additional input may be added									
			during the process. Input is normally a physical, quantifiable item,									
			but may also be intangible-knowledge, for instance. To quantify									
			intangible input, work hours are often used for a variety of									
			nonphysical inputs.
Knowledge		Relational information about objects or groups of objects. Knowledge									
			allows the worker to use data in performing an activity.
Knowledge work		A process that requires knowledge from both internal and									
			external sources to generate a product which is distinguished									
			by its specific information content.
Macroproductivity	Parallels the scope covered by the term "macroeconomics."									
			Refers to productivity at the national or industry level.									
			Compare with microproductivity, which refers to the business,									
			division, or department; and nanoproductivity, which refers to									
			the department, work group, or an organizational unit in between.									
			Different productivity measures are required at each different									
			structural level.
Measurement		Several categories of measurement techniques apply to measuring									
			productivity:												
				Predetermined-a specified set of functions is used												
				along with a map of the work process to calculate the												
				time required to complete a task.												
				Timed-a stopwatch is used to record actual times to												
				complete a task on several repetitions.												
				Log-individuals maintain a log of their own activities												
				to establish average times to complete tasks.												
				Short-interval scheduling-a variant of logging in which												
				an analyst records what a number of people are doing												
				at short intervals.									
			While these are the major categories of measurement techniques,									
			each category has a subset of techniques that vary in their									
			implementation, level of detail, and objective.
Microproductivity	Parallels the scope covered by the term "microeconomics."									
			Refers to the productivity of the organizational unit size									
			being examined, such as a business, division, or department.									
			Covers larger units than the term "nanoproductivity."
Nanoproductivity	Refers to productivity of the work unit. The term does not									
			refer to individual productivity. Productivity at the									
			individual level is not typically a goal of productivity									
			measurement.
Output			The result of performing a process. A physical quantifiable									
			output is easiest to measure, but many outputs are intangible									
			(such as an idea). Quantifying nonphysical output is more									
			difficult than quantifying nonphysical input, but it is									
			essential when measuring the relative output of knowledge workers.
Productivity		According to the classic definition, the ratio of inputs to									
			outputs (P=I/O). Straight quantities can be used, but									
			weighting factors (such as costs) are generally used.									
			Efficiency and effectiveness are related to productivity.									
			Efficiency is defined by the relationship between the inputs									
			and outputs. Effectiveness, however, relates to the quality									
			of the output.
Productivity		Can refer to the act of measuring an organization's				
measurement		productivity, or it can refer to the quantifier that													
			results from the measurement of the productivity.
Proficiency		A broader usage of the term "effectiveness." It addresses how									
			well a process allocates its resources.
Process			The activity involved in accomplishing a goal. A task, job,									
			assignment, function, etc., may all be a process or part of									
			a process.
Quality			A measure of how well an item meets expectations. In									
			manufacturing it is possible to quantify some measures of									
			quality because expectations are expressed in numerics. For									
			example, a sheet of plastic specified to be 6 cm by 10 cm									
			plus or minus 0.1 cm would not be of adequate quality if it									
			measured 6.2 cm or 9.8 cm. In knowledge work it is often									
			impossible to define quality in such absolute terms. A letter									
			with one typographical error might be accepted or rejected,									
			depending on the purpose of the letter.
Time			Expressed in hours, minutes, and seconds, it is a constant.									
			Time as a work input may not always have a linear									
			relationship to the quantity of output. A linear									
			relationship is most common, but a nonlinear relationship									
			may result from economies of scale, for example. The larger										
			the volume of production, the less time each unit takes, so									
			the increase in time input is not constant (or the average									
			time per unit decreases).
White-collar		Historically, workers who wore `white collars' performed									
			office work. White-collar work has lately been redefined by a									
			number of people. These definitions commonly refer to white-									
			collar work as unstructured, knowledge intensive, nonmanual,									
			and nonroutine.
Work			The human processes and subprocesses involved in changing									
			inputs into outputs.

Back to Table of Contents

Back to KWS Home Page

Note: The material in the Appendix is summarized from Key Criteria for Performance Measurement, memorandum from Comptroller, DoD (22 October 1992).

Performance Measures

Performance measures typically fall under three major categories: (1) factor of production indicators, (2) outcome indicators, and (3) work process indicators. Most measures are quantitative, but selected effective measures such as customer satisfaction may be qualitative.

Factor of Production Performance Measures

Factor of production performance indicators typically describe the relationship of resources to output.

Input measures

These describe the resources-time and staff-used for a program.

Output measures

These describe the goods or services produced.

Efficiency

The relationship of outputs to inputs.

Effectiveness

Output as it conforms to specified characteristics.

Outcome Measures

Outcome Measures

These measures assess the effect of output against specified objective standards.

Impact measures

How the outcome affects the organization.

Work Process Measures

Indicators of the way work gets done in producing output at a given level of resources, efficiency, and effectiveness.

Cost-effectiveness

Measures the change in the relationship of resources to output, or some other measurement.

Efficiency review

A process where the overall work process is analyzed. The inputs, outputs, and workflows are identified and studied. The result is an analysis of the existing process versus a standardized model of the process. This analysis is used to make recommendations for improvements and enhancements. Many methods are employed in performing these analyses.

Flow Charting

A graphical mapping of the activities of the work process. It is often used in conjunction with other techniques to produce a complete analysis of a work area.

Cost-Based Activity Modeling System

Is currently advocated by the Director of the Defense Information Systems Agency. Cost-based activity modeling charts work processes and subprocesses; identifies and eliminates nonvalue tasks; identifies costs of remaining tasks; and focuses on process changes, including identification of automation opportunities, to accomplish necessary tasks at reduced costs.

Theory of Constraints

Focuses on maximizing throughput, reducing inventory, and reducing turnaround time.

Macro Management Analysis Reviews

Typically uses economic analysis techniques to analyze the work process.

Benchmarking

Compares performance indicators of some part of an organization to indicators of another similar part of the organization, or to a similar entity outside of the organization.

Statistical Process Control

Statistical techniques used to evaluate the performance of a process.

Status of Conditions Indicators

Indirect measures of the work environment. They can include rates of absenteeism, accidents, and turnover. They give an indirect indication of the conditions of a work area that may affect efficiency and effectiveness.

Organizational Assessment Tools

Tools used to determine and evaluate an organization's culture and environment. The outcome is an analysis of an organization's potential.

Innovation

The rate of the introduction of innovation into the work process.

Quality

The measurement and assignment of cost to the level of quality in the work process.

Back to Table of Contents

Back to KWS Home Page