Empirical Process Control

Empirical Process Control

Complex problems are those that behave unpredictably. Not only are these problems unpredictable, but even the ways in which they will prove unpredictable are impossible to predict. To put that another way, a statistical sample of the operation of these processes will never yield meaningful insight into their underlying mathematical model, and attempts to create a sample can only be made by summarizing their operation to such a degree of coarseness as to be irrelevant to those trying to understand or manage these processes.

Much of our society is based on processes that work only because their degree of imprecision is acceptable. Wheels wobble, cylinders shake, and brakes jitter, but this all occurs at a level that doesn’t meaningfully impede our use of a car. When we build cars, we fit parts together with a degree of precision fit for their intended purpose. We can manage many processes because the accuracy of the results is limited by our physical perceptions. For example, when I build a cabinet, I need only cut and join the materials with enough precision to make them acceptable to the human eye; if I were aiming only for functionality, I could be far less precise.

What happens when we are building something that requires a degree of precision higher than that obtainable through averaging? What happens if any process that we devise for building cars is too imprecise for our customers, and we need to increase the level of precision? In those cases, we have to guide the process step by step, ensuring that the process converges on an acceptable degree of precision. In cases where convergence doesn’t occur, we have to make adaptations to bring the process back into the range of acceptable precision levels. Laying out a process that repeatably will produce acceptable quality output is called defined process control. When defined process control cannot be achieved because of the complexity of the intermediate activities, something called empirical process control has to be employed.

It is typical to adopt the defined (theoretical) modeling approach when the underlying mechanisms by which a process operates are reasonably well understood. When the process is too complicated for the defined approach, the empirical approach is the appropriate choice.

—B. A. Ogunnaike and W. H. Ray,
Process Dynamics, Modeling, and Control^[1]

We use defined processes whenever possible because with them we can crank up unattended production to such a quantity that the output can be priced as a commodity. However, if the commodity is of such unacceptable quality as to be unusable, the rework is too great to make the price acceptable, or the cost of unacceptably low yields is too high, we have to turn to and accept the higher costs of empirical process control. In the long run, making successful products the first time using empirical process control turns out to be much cheaper than reworking unsuccessful products using defined process control. There are three legs that hold up every implementation of empirical process control: visibility, inspection, and adaptation. Visibility means that those aspects of the process that affect the outcome must be visible to those controlling the process. Not only must these aspects be visible, but what is visible must also be true. There is no room for deceiving appearances in empirical process control. What does it mean, for example, when someone says that certain functionality is labeled “done”? In software development, asserting that functionality is done might lead someone to assume that it is cleanly coded, refactored, unittested, built, and acceptance-tested. Someone else might assume that the code has only been built. It doesn’t matter whether it is visible that this functionality is done if no one can agree what the word “done” means.

The second leg is inspection. The various aspects of the process must be inspected frequently enough that unacceptable variances in the process can be detected. The frequency of inspection has to take into consideration that processes are changed by the very act of inspection. Interestingly, the required frequency of inspection often exceeds the tolerance to inspection of the process. Fortunately, this isn’t usually true in software development. The other factor in inspection is the inspector, who must possess the skills to assess what he or she is inspecting.

The third leg of empirical process control is adaptation. If the inspector determines from the inspection that one or more aspects of the process are outside acceptable limits and that the resulting product will be unacceptable, the inspector must adjust the process or the material being processed. The adjustment must be made as quickly as possible to minimize further deviation.

Let’s take code review as an example of an empirical process control. The code is reviewed against coding standards and industry best practices. Everyone involved in the review fully and mutually understands these standards and best practices. The code review occurs whenever someone feels that a section of code or code representing a piece of functionality is complete. The most experienced developers review the code, and their comments and suggestions lead to the developer adjusting his or her code.