Policy Games for Strategic Management

Chapter 8 - Designing the Policy Exercise

Policy Games for Strategic Management

by Richard D. Duke and Jac L.A. Geurts

Rozenberg Publishers © 2004

Chapter 8: Designing the Policy Exercise

Note: Chapter 7 and 8 are envisioned as being used independently of the rest of the book. As a consequence, there is some redundancy to ensure clarity for the reader. You may also find the Detailed Design Tips located in the Appendix to be helpful.

8.1 Introduction

The purpose of this chapter is to describe a professional process for the design and use of the policy exercise. It draws heavily from Gaming: The Future’s Language (Duke, 1974). The methodology for the design of tailor-made exercises has been developed and tested over a period of 40 years. Many public and private organizations have used this methodology to assist management in handling complex problems (Duke, 1974; Greenblat & Duke, 1975, 1981; Geurts et al., 2000).

For many, the process for designing a policy exercise has been somewhat ad hoc and has not been well documented. This chapter explains the design process for policy games as it has developed over the years; it consists of deriving a theory of real-world behavior, constructing a theoretical model and converting it into a policy exercise. Each of these activities can and will increase the overall accuracy and validity of the model relative to the actual system under study. This has been an evolutionary process – it remains a work in progress. The approach described here is the result of stimuli and feedback from many tests in practice and accordingly it reflects the wisdom of many of our clients and colleagues. There are five phases to the process we employ; each of the phases will be described later in this chapter:

Phase I – Setting the Stage for the Project

Phase II – Clarifying the Problem

Phase III – Designing the Policy Exercise Phase IV – Developing the Exercise

Phase V – Implementation

In recent years we have witnessed a proliferation of decision techniques to aid planning and management in large organizations. Two major styles of decision aids have developed. One set of techniques has its history in applied mathematics, econometrics, operations research, and systems analysis, focusing on the use of formal models and algorithms for policy development; these are most effective when dealing with Quadrant I situations (see Figure 8.1).

Image from book

Figure 8.1: Macro-problems are Quadrant IV Situations

Although many applications of these techniques have been successful, their limitations, when applied to strategic level problems, have also been well documented. Criticisms of these formal strategic policy tools include over-simplification, over-emphasis on easy-to-quantify aspects, too much dependency on incomplete scientific theory, problems with data, that they are overly time-consuming, and most importantly, that they require that the users accept the results of a “black box” (the underlying model).

A recent example of the debate on formal modeling shows Alan Greenspan, Chairman of the Federal Reserve, answering his critics who accuse him of blending formal analysis too much with intuition. While many experts advocate the use of increasingly sophisticated models for managing the economy, Greenspan relies on a more hybrid type of “risk management.” In a speech quoted in the Asian Wall Street Journal, September 1, 2003, Greenspan is quoted as saying “Every model … is a vastly simplified representation of the world that we experience with all its intricacies on a day-to-day basis.” Greenspan explains how the Federal Reserve also uses what we have called scenarios. “Our knowledge base is barely able to keep pace with the ever increasing complexity of our global economy”. This means, he said, “The Fed doesn’t just set interest rates according to the most likely economic forecast but also considers less probable outcomes with potentially dire consequences”.

This example shows that a second set of decision techniques that have received much attention can be labeled “judgmental;” these are most effective when dealing with Quadrant IV situations (see Figure 8.1). Originating from disciplines like cognitive and social psychology, these techniques focus on intuition, creativity, discussion and communication as stepping stones to strategic policy formulation. They have proven to be useful in many policy applications; however, the criticism of these techniques is also undeniable. Their limitations can be characterized by concepts such as: subjectivity; reliance on incomplete mental models; neglect of data, theory and expert opinion; and too little capacity to deal with the complicated causal processes which affect the future outcomes of planned policy.

In our discussion of participatory policy analysis in Chapter 2 and through the cases in Chapter 3, we have made our position clear that an optimal approach to strategic policy formulation should try to combine the best of these two approaches. A basic characteristic of a design process is that it should facilitate a form of participatory policy analysis that combines the rigor of systems analytical techniques with the creativity and communicative power of structured group techniques. This process has to result in an operational model of a very complex system. Reality is simulated through the interaction of role players using non-formal symbols with formal, computerized sub-models (where necessary). We see the technique of the policy exercise as one important contribution to the creation of a family of hybrid decision techniques.

From a methodological point of view, the intended end product of the design process is an operating model of a real-life system. In the game model, people in different but interrelated roles create, at least partly, the dynamic behavior of the model. Gaming/simulation can generally be distinguished from free role-playing; it is true that roles form a part of the simulation, but a policy exercise also has other components. In addition to role descriptions, a designer also uses scenarios, a series of prescribed actions for operator and players as well as carefully selected symbols and paraphernalia: game boards, cards, etc. The policy exercise is more formalized than role playing and, in the design, there is more emphasis on portraying reality.

Games are also different from computer simulations as used in operations research. In a computer model, all aspects of a system are described with mathematical or logical symbols. It is the computer that works out the dynamic behavior of the programmed model structure. In a game, reality is simulated through the interaction of role players, using non-standardized symbols. Computer models are quite often a part of a game in the so-called interactive or man-machine games.

The design process presented here can be clearly distinguished from that employed by many past applications of the policy exercise by its flexibility and sensitivity to the problem at hand. The dichotomy of tailor-made versus off-the-shelf games may help to clarify the distinction. The discipline of gaming/simulation, as it emerged from the military and moved into fields like management and urban planning, has produced a library of “off-the-shelf” exercises which are used frequently and successfully in professional training programs and university courses worldwide (Horn and Cleaves, 1980; Elgood, 1984). Because most of these games describe generalized or ideal-typical situations in non-existing organizations, they have limited value as policy development tools. The macro-problems that we described in Chapter 3 are so unique that the policy exercises we build for them had to be client specific.

The design process has to guarantee the on-time delivery of tailor-made products under severe time pressure. It has to be very transparent and trustworthy. Many people are involved, all of whom have a high economic and emotional stake in the process and who usually are going through the process for the first time, sometimes with a degree of insecurity. The method has to be reliable and flexible at the same time; it has to help in creating the in-between and end products as planned but it also has to be able to allow for step-by-step learning. The design process has to be the reliable compass for the project team. Otherwise, they may get lost in the struggle with the different complexities of the macro-problem. The design process and its list of preplanned in-between deliverables helps the designers find their way back if they go astray. Usually they discover that they have skipped one of the steps in the process and must go back and follow the process in the intended order.

8.1.1 Key Design Features

There is a correlation among principles, communication, complexity, knowledge, and the policy exercise that avoids the partition of the solution process. The policy exercise approach uses a sequence that combines many management elements: issues, structure, content, policy, planning, decision making, problem solving, enabling, implementing, managing, process, elements, relations, dynamic conclusions, and frequent feedback loops to ensure validity. This approach is very effective for investigating the complexities of important, non-reversible decisions (Gordon, 1985). The technique provides an understanding of the complex structure of decision-making processes of the real world.

Well-designed gaming/simulations are transient in character; they permit the restructuring of the game during play. Evidence of this can be obtained by taking any given game and running it with different participants. The more thoughtful games deliberately structure mechanisms that encourage and facilitate player or operator modification during play.

Game design is a combination of a disciplined design approach with a mimicry of existing game formats and styles; it is an elusive but real “art.” Some principles of design are reasonably well articulated; others are just beginning to emerge. The game designer should have an express and coherent purpose or message to guide the construction of a game. Only the clear articulation of this purpose permits the rational selection of gaming as the appropriate communication mode. Skilled professionals have learned the art of gaming from experience; over time, using inspiration and imagination, they have built repertoires that permit the establishment of meaningful reference systems for policy discussions.

Gaming is best understood as a communication form: an analysis of the various communications media indicates that each game is very specific to some particular communications purpose (see Chapter 5). This specificity of purpose, as well as the high cost of the technique, become convincing arguments for precision in the design and parsimony in the construction and use of the game. When considering resources for game design, both cost and time are significant factors. The resources required to design, construct, test, and disseminate the game must be considered, as well as the cost encountered each time the game is used.

In determining the primary communications purpose to be achieved by the game, it is useful to review the primary purposes on which the exercise should focus. In establishing these, consider the cone of abstraction (Figure 8.2); this is useful in clarifying what can realistically be achieved with a specific technique (See Section 3.8 for an example). The product should be developed at an appropriate level to meet specific objectives. The exercise could be designed to:

* Transmit information to an audience;

* Serve as a questionnaire to extract information or opinions from the players;

* Establish dialogue among players;

* Motivate participants;

* Provide an environment in which creative ideas will spontaneously develop;

* Create awareness of a complex environment;

* Assist in decision making; and

* Achieve consensus.

Image from book

Figure 8.2: Cone of Abstraction (with references to the IJC case)

8.1.2 Basic Characteristics of the Discipline

The creation of a game requires the successful interaction of several different groups (representatives of the various stakeholders, content specialists, structural modelers, etc.). All games are basically iterative in their structure; repetition through many cycles of play facilitates learning. They are best thought of as an environment for improved learning, achieved by successively defining the totality of the problem in increasing detail. Players commit themselves, and their decisions receive prompt feedback. Additional iterations introduce more detail; this progression helps to establish a context that is broad and structural. Games attempt to reproduce the appearance of reality but the security of the artificial is always present. If their preparation is thoughtfully designed, the style will naturally evolve.

A central concept for the policy exercise is that of a safe environment. This technique is employed for group model building and the clarification of alternatives. The function of the design team is to help the participants establish a future-oriented “vision” that permits the evaluation of alternatives.

The policy exercise is designed using the concept of “neutrality of design”; it is rational within the bounds of the human condition. The emphasis of the design team will be on defining opportunities, impediments, and alternatives. The technique is used under the presumption of “people of good will” motivated to engage in productive confrontation of the issue at hand. Because the policy exercise is problem specific, each instrument should fit the particulars of a unique problem environment. The game design process is explicit; each step addresses a specific client need. All methodologies have limits, and the proper use of gaming requires precision for effective results.

8.1.3 Why a Deliberate Process is Required

The design phase (resulting in a Concept Report) is the most crucial stage in the development of a policy exercise. Often, designers skip this step and begin with construction. This is comparable to starting to build a house without a blueprint. The steps described below derive from both theory and practice; they have the redeeming quality of ensuring a successful project. In practice, they may sometimes be truncated to meet the needs of a given client. However, a sequential process is demanded, and all the concerns implied by the full sequence of steps should be addressed.

Games require a melding of differing perceptions of reality; as a consequence, they are best if designed by groups rather than individuals. Working with a group generates a richer game. Given that the design and use of a successful game is somewhat akin to putting an organization through psychological analysis, resistance can be expected from those who feel threatened by the process. Integrity and clarity of the design process ensure that a consistent framework is evident to all those who may be involved (design team, client, and final participants); the process also creates an environment in which “truth” is forced to surface. Viewpoints at variance are exposed and confronted; a sharper perspective is reached through negotiation among the respondents. As the process proceeds, step by step, distracting issues can be set aside and a sharp focus maintained on the macro-problem. The process, employed relentlessly, will not only achieve agreement on the problem, but on the alternatives available as well; in most cases, consensus for action can be expected.

* If a deliberate process is used:

* All staff time and energy are used to create an evolving prototype. Substantive knowledge may be

initially flawed; however, each of the rule-of-ten runs (Section 8.2.4) provides an opportunity for

specific content to be challenged and modified. Components are increasingly made to fit together

using a trial and error process;

* Staff meetings achieve one of two objectives: either they reach agreement on some aspect of the

exercise, or they identify the specific problem(s) that can be resolved by staff work. Each team

member is obliged to raise concerns in a tangible, explicit way. Discussion is controlled toward this

objective. The “complex system” is teased out, one aspect at a time. Progress is slow, but

deliberate; the team can judge where they are in the process;

* Because artifacts (e.g. the schematic) are used to capture ideas, the current state of knowledge is

captured; as a consequence, it can be challenged in a systematic way;

* There is self-evident project memory. The accumulating artifacts (game materials) serve not only to

remind the team of where their thinking was at the last gathering, but the prototype serves as an

excellent way to get feedback from outsiders in both impromptu and structured ways; and

* The focus is on the product (game) rather than a diffuse problem; this is central to the task; many

safeguards are built in to ensure that content of the final product is valid.

* If a deliberate process is not used:

* Staff time and energy may be diverted to pursue substantive knowledge; because this is infinite,

the task can never be completed (literally). The more they learn, the more they must learn to make

every thing fit together. There is no imperative to focus attention on that which is most relevant;

* Meetings tend to be cyclical, ending more or less where they began. This derives from the fact that

the subject is a truly complex system – each speaker struggles not only to articulate his or her

thoughts but also to understand the comments of others. The words employed are sequential, but

the system under discussion is a web. This results in a paradox – the more sophisticated the

participants, the more convoluted becomes the discussion! Progress is hard to judge; the team has

no established guideposts;

* Because artifacts are not used to capture ideas, content cannot be challenged in a systematic way;

* There is no project memory. There is no process that permits insight gained from a discussion to

be deliberately and efficiently re-articulated on the occasion of the next meeting; and

* The focus is on content rather than on the product (game).

8.2 The Design Sequence

The process of design falls into five broad phases; 21 specific steps are nested therein. The steps will be presented in detail in the remainder of this chapter. They are listed in Figure 8.3 in abbreviated fashion to provide the reader an overview of the process.

Image from book

Figure 8.3: The Design Sequence

As stated, the process of game design is a mix of adherence to the discipline, the development of logic for each case, proper reference to a repertoire of techniques, intuition and inspiration. The design process combines logic and serendipity; creating the simulated environment is an artistic challenge. Selecting the correct stage props and keeping them at an appropriate level of detail requires skill. The challenge is to retain verisimilitude while creating an interesting model that will mutate and evolve. The design process described below fulfills this function.

The policy exercise, first and foremost, is an abstraction of reality. It should serve as a valid and efficient model of a complex environment for communication purposes. It should capture those aspects of the problem that are central to the discussion at hand – but, equally important, it should be parsimonious. Too much detail, and/or a confusing presentation can obfuscate rather than clarify. The process described here, if faithfully followed, provides a clear, logical basis for capturing the essence of the problem environment and translating it into an appropriate exercise.

Experience shows that this is a replicable, disciplined process to guide the development and use of policy games. Twenty-one distinct steps allow the design team to proceed systematically; this permits the client to follow the progress of the work and to judge, explicitly, if the product meets the initial specifications. One of the strengths of the step-by-step process is that it supports stakeholder communication throughout the design effort. The process is designed to direct a sequence of activities that involves formulation of the initial situation, articulation of a conceptual model, design, construction, and running a gaming exercise, and capturing the insights that emerge.

In the abstract, the game design process is viewed as sequential; the game designer proceeds logically from point to point. In practice, the designer may attempt a somewhat more simultaneous solution and may adapt the order of progression if required to meet the client’s needs.

8.2.1 Phase I – Setting the Stage for the Project

Phase I has five steps that must be carefully addressed at the outset of a project. Steps 1 through 5 should be viewed as an iterative process. For example, as the effort to clarify the problem becomes more precise, the specific objectives to be achieved can be more clearly articulated. The initial steps serve to guide the remaining development process. Those who ignore these five activities do so at their peril; the remaining activities rest on this bedrock.

Step 1. Administrative Set-up

The purpose of this step is to ensure that the project is properly authorized and that the chain of command, review of progress, and administrative details are appropriately organized. Attention to administrative detail is required for any project; the standard concerns are well known; however, there are special concerns that need to be addressed, unique to the development of a policy exercise.

A clear definition of the client is essential; there are many situations in which this is not immediately evident. Someone within an organization may identify a problem of concern, fund a policy exercise to address the issue, and authorize it to begin. Experience shows that this may be initiated at any of several administrative levels (the CEO, a vice president, a department head, etc.). However, given the murky nature of these problems, the true locus for a policy decision to resolve the matter may ultimately prove to rest elsewhere within the organization. The study team may quickly realize that the problem is set in too restricted a fashion. For reasonable success, it may be necessary to negotiate the focus to a higher level. Needless to say, this requires tact; more importantly, it requires strict attention to the sequential (but iterative) completion of the design steps. Problems of this kind will become evident in Step 2 (Problem Definition); they will be resolved by the completion of Step 8 (Negotiating the Focus of the Exercise).

Proper authorization for the project is a must! It is common for these types of projects to be launched under great time pressure; the initial client representative may attempt to bypass internal procedures to get underway quickly. This can prove to be a fatal mistake for the project. It is necessary to arrange for the orderly conduct of business at three levels: the client, the organizational unit controlling the project, and the project team. This includes all the activities of the client and the consultant receiving proper authorization for the project. This initial step will vary significantly among organizations.

The successful development of a policy exercise requires, as does any project, a careful delineation of responsibilities and lines of authority. Care should be taken to follow standard contractual arrangements (e.g. fees, budget, specifications, expected results, procedural agreements, contractual obligations required of the client, deadlines, go/no-go decision dates, expected deliverables, sign-off required, etc.). These, and other appropriate administrative matters, should be fully resolved before the substantive material is addressed.

A “designated agent” must be identified to represent the client as a key contact or project manager. Responsibility for different aspects of the project may be delegated among several people (e.g. financial, access to data and information, graphics assistance, etc.), but it is crucial to specify one individual who is to be the central contact person for the design team. This agent should have sign-off authority or clear and quick access to someone who does. This agent is the one responsible for approving any major budget or technical changes as well as the finished product. Through this agent, the client will formally sign off by approving each major phase of the project (this protects against conflict at later phases of the project).

A project of this kind typically requires several types of personnel: various experts (content specialists, computer specialists, facilitator, etc.), game designers, and administrative staff. This may require negotiation to obtain an appropriate project mix. Typically, the client has the ability to provide both content experts and administrative staff; however, it will be the rare situation in which in-house game-building skills are available. The client must choose between three alternatives: completing the project internally, using a consultant to assist client staff, or contracting the project to an outside group. Specific concerns to be addressed include the need for objectivity (difficult for client personnel to achieve!), gaming expertise, confidentiality (hence the need to limit information to in-house staff), cost (it may be more cost-effective to contract the project out) and other client-specific concerns.

The utility and perception of the final product will be a function of good communication as the project proceeds from beginning to end. The urgency so typical of these situations demands proper procedures to facilitate quick and uninhibited communication by the policy exercise design team. This demands a clear chain of communication both within and beyond the organization. Within the organization, both vertical and horizontal lines of communication should be cleared to permit the team to cast a very broad net, particularly during Phase II (Problem Clarification). Communication that extends beyond the organization may be essential; this may require special procedures for clearance. Because egos are involved, many participants may feel threatened during the design phase; as a consequence, negotiation procedures should be worked through in advance.

Step 2. Define the Macro-Problem

The purpose of this step is to establish a sharply focused problem statement that defines what the game is to address. It is important to clarify the problem with precision at the outset of the design process. This should capture an overview of the problem that reflects the perspectives of the various stakeholders who must be interviewed to find the perceived problem boundaries. The initial problem statement has to be acceptable to all participants in the process. This statement should articulate both the technical aspects of the subject as well as any specific communications objectives (participatory decision making, communication of policy to staff, corporate culture, etc.) that are to be achieved by the exercise. The problem statement (accompanied by the specifications) should be sufficiently detailed to permit evaluation of the success of the game when completed. This statement will be redefined and clarified several times as consensus is established regarding the definition of the problem and the objectives of the exercise.

Only a clear problem statement will guide the selection and the active involvement of concerned parties in understanding the complexity of the issues involved; identifying, creating, analyzing, and selecting alternative solutions; and assessing the impact of proposed alternatives. A problem definition consists of three distinct statements: background information, the problem statement, and further elaboration as may be required.

Background of the Problem This statement should provide a summary of the characteristics of the problem environment (e.g. stakeholders, organizational structures, issues, etc.). This description usually elaborates on several components of the problem: its history, characteristics (scope, cost, size, etc.), technical aspects, how it affects the organization, how solving the problem will benefit the organization, etc. The need, conditions, or circumstance that prompted consideration of a game should be stated; it should be convincing to a neutral observer and as concise as possible. This background statement will lead into a summary statement of the problem that will be central to the creation and evaluation of the exercise. It will support a detailed analysis of actors (needs, objectives, and responsibilities) as well as of the problem environment (internal and external) that will take place during the schematic development phase of the design process (Steps 6, 7 and 8).

The Problem Statement

The problem statement is not a description of the amorphous big picture, but rather a narrowly defined concern that the exercise is to address. If properly achieved, it enables the active involvement of concerned parties to focus on:

* Understanding the complexity of the issues involved;

* Identifying, creating, analyzing, and selecting alternative solutions; and

* Assessing impacts of proposed alternatives

The problem should be reduced to one succinct statement that is acceptable to both the design team and the client; it should be no more than a few sentences long. This task is difficult to achieve, but if successful, it makes it possible to bring the goals of the project into sharp focus (Chapter 3 provides eight examples of problem statements.) The final product will be evaluated against this problem statement and the specifications that accompany it. Typically, there will be a need to redefine the problem statement a number of times.

Further Elaboration

In some cases, it is appropriate to extend the problem statement with further detail; this should be added at this point. It would include an elaboration of subject matter (substantive content), specific alternatives to be considered, and the themes, issues, and/or problems that are to be stressed. The substantive material that is to be dealt with by the game should be defined as explicitly as possible.

Step 3. Establish Goals and Objectives for the Project

The purpose of the exercise must be clarified at the outset! Failure to resolve this will result in confusion during construction. In turn, this leads to time and cost overruns as well as a poor quality process and product. The purpose of Step 3 is to define exactly what the intended contribution of the project is to strategic problem solving. Gaming/simulation can fulfill three classical functions of strategic management when used in a consulting project:

* Pre-decision Analysis – Discovering problems, formulating and ordering problems, developing and

clarifying alternatives, evaluating alternatives and developing plans of action;

* Decision making – Negotiating, building consensus, reaching decisions; and

* Implementation – Motivating and building awareness, educating, training and synthesizing

perspectives.

If the problem has been clearly established, goals can be articulated and the steps through the design process can then be optimally adapted to the project goals. Chapter 4 – Five Key Criteria, suggests many ways to formulate the contribution of gaming/simulation to strategic management. For a successful design process, a single primary purpose should be specified. If more than one of these purposes applies, each should be stated explicitly, and they should be clearly placed in order of priority. The policy exercise can be used as a structured meeting environment to gain understanding of the big picture, envision desirable long-term futures, identify related issues, facilitate discussion of issues, develop evaluation criteria, provide a structure for documentation of results, and facilitate the distribution of results. Typical objectives that have been central to past policy exercises can be found in Chapter 3. Here are a few more that relate to strategy making in the “sustainable corporation”:

* Encouraging innovation in technology and behavioral change;

* Teaching the impact of manufacturing on the ecosystem;

* Creating awareness of the concept of sustainable production;

* Demonstrating the interconnectedness of sustainable development issues;

* Forcing consideration of environmental issues;

* Requiring organizational views to interlock with a worldview;

* Focusing on long-term consequences of current business practices;

* Catalyzing ideas for new programs of strategic initiatives;

* Motivating participants to achieve a sustainable corporation;

* Promulgating a new corporate culture; and

* Gaining consensus for action on a major decision.

Goals should be broken down into a specific set of objectives to serve as progress markers during construction and evaluation. The objectives of the exercise should be realistic and appropriate, based on the problem being addressed. Typically, a multiple set of objectives will be listed; some will be clearer than others, and some may be in conflict. These need to be winnowed down and prioritized into the few that are most central to the concerns reflected in the problem statement. Objectives need to be presented in a sufficiently flexible form to accommodate the concerns of participants that may arise during the exercise.

Step 4. Project Objectives/Methods Employed Matrix

The purpose of Step 4 is to review a broad range of methodologies to ensure that gaming is appropriate. A basic ground rule to apply when making the decision to use a policy exercise is to consider other alternatives. Clients should be effectively engaged in making this choice to ensure their full knowledge of the policy exercise technique vis-à-vis other approaches that might be used. Games and/or simulations are only one of several options available when addressing the client’s needs. Because of the client’s unique characteristics, it is important to carefully weigh the pros and cons of each situation to substantiate the validity of the policy exercise approach. This forces consideration as to which form of communication is most effective in meeting the client’s objectives. Ultimately, effective evaluation of the project should start with an honest response to the question: “Is a policy exercise appropriate in this situation?”

The project-objectives/methods-employed matrix is used to define what devices are to be used when addressing a macro-problem. Handbooks on policy and strategy document a wide variety of research designs, decision aids, and communication procedures relevant for strategic problem solving. They include competitor analysis, market research, modeling techniques, computer-assisted group sessions and even outward-bound team motivation sessions. Because a policy game is hybrid and occasion specific, it is important to select and/or combine the appropriate methods for this client.

It is necessary to ascertain that the client’s needs cannot be met by some less cumbersome mode. Some questions need to be carefully addressed: Why was a policy exercise chosen in the first place? What benefits are to be derived exclusively from this approach? Will the game completely satisfy all the client’s objectives? What areas may not be fully covered? Note that the exercise is expected to meet only specific, limited, readily evaluated objectives; it is a mistake to expect a broad array of objectives to be met.

Step 5. Specifications for Game Design

The purpose of this step is to establish clear technical specifications; this is a natural extension of the development of the problem statement. They provide an opportunity to make objectives and constraints explicit during the phases of design, construction and use. The specifications become a pragmatic list of concerns that the game designer should address during the construction of the game. The objective is to delineate constraints and expectations for the project and also to raise and address specific questions that anticipate the conditions that will govern the design and use of the exercise. The responses, if carefully delineated, provide detailed specifications at the outset of construction against which the final product can be evaluated. It is important to consider each of the design steps and to anticipate what specifications will be required to ensure that the design process progresses smoothly. When writing specifications, it is a good idea to review all of the 21 design steps; pay particular attention to Steps 1, 6 and 11 through 21.

The specifications should spell out the chain of command, project management procedures, budget and financial arrangements, deadlines, critical dates, personnel arrangements, procedure for reporting changes, deliverables, sign-off authority, and similar concerns. Ground rules should be established for the circulation of reports (who receives them, confidentiality concerns, deadlines for response, etc.). Clarity and completeness here will contribute to a smoothly run project.

Particular care should be given to the financial resources available for the development and use of the exercise. The budget for the project should be specified separately for each of the major phases: setting the stage, clarifying the problem, designing the policy exercise, developing the exercise, and the implementation activities. Final game costs often exceed initial estimates, particularly when precise goals are not stated and approved by the client before construction begins. Similarly, the cost of using a game may vary greatly. With costs so difficult to estimate, it is incumbent on the client and designer to agree on what resources will be made available during construction and for normal use of the game. A product may have reduced utility because the unit cost per run is beyond the capacity of the organization to sustain. If each use of the game has a new set of participants and/or a new game facilitator, the cost will be high. In those instances where the game will be used frequently by the same highly trained facilitator and where audience characteristics are consistent, the cost of each run will drop markedly.

Allocation of time is no less important than dollars both in the development and use of a game. The time required for the development of a game will be the product of the clarity with which the problem has been stated, the appropriateness of gaming for the problem, the clear specification of goals in the concept report, and the range of skill and experience of the game designers. Careful task scheduling is required (timeline for the project – design, construction, testing); this demands the identification of critical due dates. The design of a serious game is an evolutionary process; as a consequence, procedures for reporting changes in the direction of the project to management should be established at the outset.

These notions are intended to suggest a reasonably comprehensive set of questions that the client and designer should specify before a game is commissioned. Since each game is to fill a specific need, these thoughts can only be used to prompt a careful search of conditions appropriate in a particular context. Specifications should be signed off by the client before beginning construction of a game.

8.2.2 Phase II – Clarify the Problem

The purpose of Phase II is to complete the cognitive mapping process (interviews, literature review, snow cards, schematic).

Games attempt to assist with decision making under uncertainty. Participants understand the need to improve the exchange of information among themselves about differing perceptions of the future as well as the need to develop and evaluate alternative scenarios. Games can be very powerful for this purpose. People find games memorable and valuable (memories from games have proven to be vivid after 25 years). They provide data as information that is structural and logical; this serves as a framework that permits the storage of information. The artifacts employed make imagery more memorable.

A lucid problem statement will contribute to the process of creating a format for the game (form follows function). If the problem is defined with care, the game design follows readily. Because this is so important, we recommend that no less than a third of the available resources (time and money) are devoted to clarifying objectives, developing the problem statement and developing the concept report (prepared in Step 13, it summarizes the findings of Steps 1-12).

Step 6. Defining the System

The purpose of this step is to investigate the underlying substantive nature of the macro-problem. Only a rigorous systems analytic procedure will be able to capture the problem environment in its entirety. It is essential to grasp an integrative (gestalt) perspective of the problem as it is reflected in the mental models of the various stakeholders. Information can be collected and organized using a wide variety of procedures: interviews, a literature review and workshops. All the major stakeholders are to be interviewed to discover their perception of problem boundaries. The literature review is to determine problem boundaries as reported by prior research. Care must be taken that not only the relevant internal executives are interviewed but also that external experts and data are explored.

The subject of a game must be specified in a precise way before construction begins. The specification of substantive content may take the form of written reports, abstracts and/or a detailed subject outline. Without such documentation, the final product may not be relevant. This document should capture and present logically that which is known about the problem environment; of equal importance, the document should identify that which is unknown – specific concerns for which no data, theory, and/or past experience are available.

The findings of the research should be captured in a model of the organization; this should be described in a short text. Identify the major actors: their goals, activities, and resources, and the interactions among them. The model should be simple, consisting of a limited number of key statements, a thumbnail statistical sketch and the major constraints affecting the organization. This is required both to guide the development of the exercise as well as to assist participants during play and in the critique. However abstract, this model should be defensible.

A systems analysis ensures a rational and thorough approach to game design. Without this step, assumptions may go undetected, variables and relationships may be unnoticed, and the resulting simulation may be unrealistic.

Information about the system and elements of the problem can be collected and organized using interviews and a structured workshop during which participants create “snow cards.” Additional snow cards may be generated from a literature review. (One client had a large number of these cards arranged on a tabletop. On return from lunch, team members opened several windows simultaneously. Wind blew the cards about the room and someone called out: It’s snowing! Thus the name “snow card” was born).

Snow cards are ideas, variables, problems, flows, relationships, models, laws, stakeholders, decisions, etc. that appear significant. Only one thought is committed to a single card, and this thought is kept as cryptic as possible (without loss of meaning). Each card captures a brief statement or a word that represents an issue or variable uncovered during the analysis. This process tries to find all the problem variables and how they interact; it uncovers the boundaries, actors, and the inputs and outputs of the problem. The snow card process is described in more detail in the Appendix.

The system may be defined from one or more conceptual referent systems (group dynamics, resource allocation, geographic, demographic, political, economic, social, psychological, anthropological, historical, etc.). It is common to have several hundred cards when analyzing a complex problem environment. The development of a set of snow cards is best done through a team effort using a freewheeling brainstorming approach. This inductive form of analysis is intended to be inclusive – if anyone thinks it might be important, it should be included. These cards are converted into a working schematic that should convey all the concerns expressed by the snow cards. The schematic is for staff use at this stage, not for the client. This is the most critical part of the process – if the boundaries are too tightly delineated, significant alternatives may be ignored.

Often, we divide the team into two groups. The first group works with members of the organization and their stakeholders using interviews and workshops. They concentrate on the “logic in use,” i.e. the diverse mental maps of the persons who live in the system. The second group seeks to find conceptual models and generalizations from multidisciplinary, academic and professional literature, statistical data, and if relevant, outside experts; they look for potential frames of “reconstructed logic.” In team meetings, the results of the “top down” and “bottom up” analyses are contrasted and one perspective provides the homework for the other group (see the concept of robustness in Chapter 4). Saturation is the criterion used to stop this process. When no new ideas, frames and/or perceptions emerge, it is time to go to the next step: the consolidation of these insights into a schematic. Of course, the criteria of saturation is only valid if one can guarantee that the teams have put forth a maximum effort in approaching the system from as many sources and perspectives as possible.

The teams must select sources (books, articles or people) with care. In all cases, they look for pithy statements that reveal a significant aspect of the problem. For example, if there is a journal article by an expert that addresses the problem, this should be searched for the major areas of concern – e.g. finance, geography, actors, science, fact, interrelationships, etc. Remember – you are trying to play detective and piece together clues that let you discover the system in its full flavor. Avoid the trap of taking too much from one source (as this may give you a distorted view). When exploring the system, there may be bewildering amounts of data, much of which is irrelevant or misleading. Take time to understand and identify the key elements in the system, otherwise you cannot successfully simulate it, even in highly abstract form. What one should look for is:

* Action triggers and their mode of operation;

* Major flows, stocks and delays (input, throughput, output) in the system;

* Areas of clarity and areas of ambiguity;

* Congruency and conflict in organizational perception;

* Decision-making levels;

* Key elements in control systems (feedback);

* Major decision points and factors influencing them;

* Major players;

* Patterns of interrelationships and points of interaction;

* The pattern of organizational politics; and

* Theoretical evidence (natural, man-made).

Step 7. Displaying the System

The next objective is to develop a graphic that contains an overview of all the considerations that are significant to the policy issue being addressed by this process. This should proceed through two primary steps: the working schematic and the final schematic. Schematic presentations of complex problems exploit the advantages of visual communication. They do so by simultaneously expressing the “big picture” as well as the significant issues required for understanding the problem environment. For the schematic to be able to convey an overview of the problem quickly and simply, it should be organized in such a way that it is clear not only to those building the game, but also to those participating in the exercise.

Every stakeholder will view the problem environment differently. These differences should be confronted and negotiated to the point that a single explicit visual conceptualization is agreed upon. Inevitably, this will be a complex rendition of the relationships of several hundred (possibly in excess of a thousand) variables (previously developed as snow cards, see Step 6) that are of interest to various stakeholders. All stakeholders should be able to locate the central features of the world as they view it within this schematic. The schematic is improved using repeated iterations; saturation is our operational criterion for having reached a state of robustness. Once we can no longer elicit new ideas and even the most hesitant opinion makers can agree with the schematic, we go to the next phase of game design. The more highly articulated and accepted the knowledge household is, the more robust or less controversial it will be. The knowledge should be organized in a format that provides a clear and stimulating path for the non-specialist, emphasizing not only what is known, but also what is not known.

Definition of a schematic

A schematic, as used in the development of a policy exercise, is a drawing that represents a model or overview of the system being investigated. It calls on the results of Step 6 (Defining the System). The drawing will show major system components (e.g. actors, processes, data files, etc.) and the primary relationships among them. The drawing will show any important sequence(s) of activities; however, this is not its primary purpose. Each system component should be described by a few primary attributes that help to define the processing sequence of the completed exercise. A well-designed schematic is an effective way to:

* Engage a group of professionals in the discussion of central issues;

* Provide the group with the big picture; and

* Help the group develop solutions that reflect the various interdependencies.

A schematic illustrates a much richer interpretation of the problem than one person can conceive alone. Most clients find the process of creating a schematic to be very useful for clarifying assumptions and perceptions of key decision makers and actors. The resulting schematic, therefore, takes on its own value and serves other purposes in addition to guiding the creation of the simulation. Many clients have volunteered enthusiastic endorsements of the final schematic. In more than one case, the client has indicated that this drawing alone was worth the price of the entire project!

The Working Schematic (Map of the snow cards)

The initial description of the situation usually comes forth in discrete, partially organized, and sometimes conflicting pieces. It is the task of the research team and the client to synthesize these elements of the system into an integrated and explicit model that can be easily described and discussed. The snow cards and other results from Step 6 are arranged into a working schematic. This should identify and schematically describe the major system characteristics and linkages (endogenous and exogenous factors, relationships, flows, tables, etc.). This initial effort should be presented on a single large sheet of paper, of any convenient size (typically very large). The result of this early trial schematic is an inclusive, but rough, approximation of the problem environment that should convey the bulk of the concerns expressed by the snow cards. It is a working document used by the staff to capture ideas and to organize them into a preliminary sketch of what the game might look like; as it progresses, it must be reviewed by the client.

Final Schematic

Eventually, the necessary abstractions, omissions, and re-definitions are undertaken and the entire configuration is committed to a final format (such as a colored graphic). This is a condensation of the original working schematic into a formal document for delivery to the client. As such, it will have been reduced to those factors considered most relevant within the constraints of the problem statement. This iterative process of selection will be done in the context of an ongoing dialogue with the client; it should evoke discussion resulting in consensus about the final drawing. This discussion should clarify variables and uncover gaps, assumptions, and disagreements about how the problem is perceived. The final document should retain enough detail to adequately represent the problem environment; it should communicate the central aspects of the problem visually and quickly. The schematic ensures that the team includes key components and excludes components that are not central to the simulation; it will be changed repeatedly as the team works towards consensus. Abbreviated examples of final schematics can be found in Chapter 3. An example with more detail is included in the Appendix.

Step 8. Negotiating Focus and Scope with the Client

The purpose of this step is to select the systems components from the schematic that are to be included in the systems components/gaming elements matrix (Step 9). Because the system has been diagrammed in its entirety, the team can make rational choices about what to include or exclude. The complete problem set (as initially perceived by the significant actors) is often too broad and inclusive. As a consequence, the schematic, when reviewed and accepted by the client, will normally contain more factors than need to be incorporated into the final game. Through dialogue with the client, the design team should interpret the original objectives, problem statement and specifications against the final schematic and select those factors of primary concern that must be represented in the gaming vehicle. The process guarantees that the game meets the specifications developed in Step 5. It is also imperative to ascertain the appropriate level of abstraction at this stage. If this step is done well, the final exercise will be situation specific, lucid, and relevant. The client has a serious obligation of review at this juncture; the objective is to set a clear target for the exercise.

8.2.3 Phase III – Designing the Policy Exercise

Many professionals have had the experience of developing a “great” game (enjoyable for participants) that did not dovetail well with the client’s actual need. In these cases, a retrospective view typically reveals that the problem was inadequately defined at the outset. Another mistake is to start with a preconceived game format instead of following a deliberate process (and allowing form to follow function). This results in wasted resources as the team attempts to force a fit between the problem and a preconceived gaming format. Phase III entails the creation of a blueprint for the exercise; this takes the form of a concept report. To achieve this document, four steps are required: the systems components/gaming elements matrix, a definition of gaming elements, a review of a repertoire of techniques, and the selection of a format for the exercise. This process is described below.

Step 9. Systems Components/Gaming Elements Matrix

The purpose of this step is to transform systematic thought of the substantive system into systematic thought of the policy exercise, component by component. The system (as represented in the schematic) must be transformed from a model of the problem to a model of the game. This demands an orderly, replicable process. The task here is one of capturing creativity, identifying and incorporating the best ideas into the game. Because the process is demanding of the design team, adequate time should be scheduled at the outset and appropriate facilities must be provided.

To avoid a nonproductive state (indicated by iterative comment, hostility, or disinterest), the process should be allowed to emerge as a “happening” around the central theme (ideas can be inserted in any cell at any time). A display area is organized with clusters of descriptors arranged on a large tack board surface (see Figure 8.4). The columns reflect the systems components (selected in Step 8) and row headings reflecting the game elements (from Chapter 7 – roles, rules, scenario, format, step of play, etc.). The cells are completed in a matrix conversion process that shifts the analysis from a schematic model of the problem to a procedural and symbolic model of the gaming exercise. This matrix fosters a creative process; it provides an effective tool to accomplish this task. The process followed is a series of brainstorming sessions by the design team, where the team addresses each cell of the matrix, making conversions where appropriate.

Image from book

Figure 8.4: Systems Components/Game Elements Matrix

This approach permits the logic to develop from the standpoint of the system (down), from the standpoint of the game (across), or at random (brainstorming any cell at any time), dependent on the insight and motivation of the staff. No attempt is made to correlate or rationalize the various cells one with the other at this time. The notes in a given cell may well contradict those of another; this will be resolved in the next step. Ideas for each gaming element should be captured in the matrix; the team must determine how each systems component will be addressed. For example, if an R&D process were being simulated, a research team might be a component of the system to include in the game, which in turn, could be represented by roles and further addressed in the scenario.

An iterative process is followed until all the gaming elements are described. This process resembles the “weaving” of a story line. Step by step, the team transforms components of the system into elements of the artificial world. The variables are all interrelated; without this matrix, it would be easy to “get lost in the story” and create a game that does not address the necessary systems components. As with all the previous steps, this step forces logic and clarity into the design process and allows the team to routinely check their effectiveness and evaluate their progress.

Even when well done, the results of this process will have two flaws: the ideas tend to be “bits and pieces” that need to be integrated, and these need to be winnowed down until only the best ideas are incorporated into the game. This refinement is achieved by selecting all those content cards (ideas) from a given row and rationalizing these into an initial document that gives a preliminary description of that particular gaming element. This iterative process is followed until all the gaming elements have an initial description. No attempt is made at this time to correlate or rationalize the differing perceptions that will exist between game elements.

Step 10. Definition of Gaming Elements

The purpose of this step is to transform the initial description of the gaming elements as developed from the systems Components/Gaming Elements Matrix into a more complete definition. These elements are the building blocks that translate the conceptual model represented by the schematic into a working policy exercise. This phase can be compared with the formalization phase in the construction of a mathematical model when concepts are translated into mathematical symbols. In this case, the concepts are translated into proper gaming “language.” The building blocks available to a game designer include the elements: scenario, events, theme, analogy/metaphor, roles, decisions, format, basic referent systems, policies, rules, scoring, steps of play, model, data and information, accounting system, indicators, visuals and paraphernalia (Duke, 1981). Chapter 7 describes the gaming elements in detail.

The theoretical basis for each element should be carefully described, with particular attention paid to the quality of empirical data. Finally, the output to be generated by each component should be described (both its character and purpose). The object is to create an explicit model of reality; in the process of creating the simulated environment, it is important to select the correct stage props and keep them at an appropriate level of detail. The central problem is the elaboration of a game “environment” wherein flexibility and realism are maximized. Usually, many modifications are necessary to bring a good game together, this often necessitates compromise with the original specifications or objectives. (The concept report (Step 13) should include provisions for reporting such changes and obtaining client approval).

After each gaming element has been described, the next step is to integrate ideas vertically to ensure logical consistency among the various elements. As these elements are developed, no matter how inadequate they may seem, they are committed to a report that gives a description of an early prototype of the full exercise. A test run of the prototype cannot be completed until these are available at an initial level of development. This draft document will be changed during the test runs; it provides an opportunity to give an initial description of the exercise to the client.

A preliminary mockup of all game materials is very useful at this stage; it should include diagrams of graphics, decision forms for player use, etc. These initial materials may be sketchy; however, if successfully presented at this stage, the client can be more certain that the level of detail is appropriate. Attention should be focused on these materials during the review process because they reveal a great deal to the client about the final character of the game.

Step 11. Repertoire of Techniques

The purpose of this step is to review a broad range of experience and select techniques appropriate for this client. Repertoire is defined as known gaming techniques (paraphernalia, basic styles or referent systems employed, various graphics, interpersonal dynamic mechanisms, etc.) which might be incorporated into the exercise. The admonition here is: Don’t re-invent the wheel! The design team should have familiarity with a broad range of techniques. These “tricks of the trade” can be derived from a review of the literature and professional practice. A review of previous gaming efforts can be undertaken in order to obtain a set of ideas that might be used in the development of this exercise. A major difficulty facing the team will be ready access to a complete library of related games.

Artistry comes into play at this point. Schon has made the point that artistry consists of the ability to invent, as needed, a method suited to the difficulties being experienced (Schon, 1986). It is clear, in the design of a policy exercise, that form follows function. If the process has been followed with care until this point, the selection and/or invention of technique will come readily to the design team. The techniques employed need to be reviewed with the client to gain a sense of those that will be most appropriate in this situation.

Step 12. Select a Format for the Game

The purpose of this step is to determine an appropriate format for this particular client. The format (style) can be thought of as the physical environment and the processes through which the exercise will be presented to the participants.

There are many styles of presentation that may be used to present a complex model; the selection requires a melding of various techniques to accomplish the client’s objective. The object is to try and find group process techniques that facilitate the communication objectives of the game. These exercises range from very volatile, fast-moving, emotional kinds of exercises, to very scientific, thoughtful man-machine interactive types. The object in each case is to develop a format that will be palatable to the people who will be participating and to create a format that is logically related to the problem at hand. The primary objective is to identify an analogy or a “model of a model” that can be used to improve communication among the participants during the actual use of the game. This is a critical element: the exercise must seem an acceptable environment to the stakeholders.

The selection of a format or style for the game is a creative process. Because designers use differing frames of reference, the framing of problem situations into a game format varies widely. Some common formats include role playing, board games (edge, grid, patterned), constraint cards, flowcharts, and computer simulations. Several client-specific variables influence game format; they should be examined for each situation. They will govern decisions about format and style of play as the exercise is developed: group size, purpose (goals and objectives), organizational context, time available for play, character of the participants (age, homogeneity, etc.), function of the exercise (policy decision, project evaluation, training, etc.), and substantive content.

The format can be thought of as the core of the model upon which the game will be built. The design team should wait until Step 12 to choose a format; this ensures that the team does not become overly committed to an idea before they have completed a thorough and rational research process. A review of existing formats is also helpful. The choice should further be based on personal preference, production constraints, and simplicity with respect to the objectives.

Step 13. Concept Report

The purpose of this step is to document design agreements that will govern the exercise, consolidate ideas into a workable blueprint and obtain client sign-off. No large-scale project should be undertaken without an agreed-upon plan. Just as an engineering team will formalize their blueprints before beginning construction, the policy exercise design team must capture their plans in a concept report. This ensures that they have communicated clearly with one another as well as with the client. Unless amended with the agreement of the client and the designer, the final game should conform to this document.

This report is a detailed statement that will guide the development of the exercise. As in the previous steps, the report is developed using an interactive process in which the client and the design team exchange ideas and negotiate a document that describes the nature of the desired product. After submission of the preliminary report to the client for critique and modification, the concept report becomes final. Any changes past this point require the joint approval (negotiation) of the client and the game design team as changes may have serious consequences in timing, cost, and/or effectiveness of the final product.

The concept report should describe the problem environment (schematic) and the proposed method of game presentation. It should address the initial specifications (e.g. characteristics of the participant group that would have a bearing on game design – age, level of sophistication, homogeneity, prior training, prior shared experiences required before being permitted to play the game; etc.). In addition to the synthesis described above, the concept report should outline the procedures for construction, including:

* Individual components, including their purpose;

* Data to be employed and its specific use;

* Order of sequence of any processing that will take place, (with or without a computer);

* Macro and micro cycles of play;

* Flowcharts of sufficient detail to adequately guide the building of individual components; and

* Detailed descriptions of each gaming element.

If the concept report is carefully prepared and reviewed, construction should be routine and uneventful; a thorough report will enhance the quality of the final game. The completion of the report can be thought of as the end of the planning/programming stage of the project. As with most planning efforts, the more energy given to this stage, the higher will be the quality of the resulting product and the more efficient will be the design process.

It is tempting to bypass the first three phases of game design (setting the stage, clarifying the problem, and the concept report) and to jump headlong into construction. Yielding to this approach will result in a failure to achieve precision of design, the careful engineering of the construct to meet a communication need, and almost certainly, a loss of parsimony in the construction activities themselves. There is a tendency for teams to begin in the middle of the design process and to avoid the hard work required in answering the questions that must be raised to complete the concept report. The difficulty is compounded by the element of risk inherent in making a written commitment that might later serve as an indictment of the team. The process of constructing a game inevitably forces the designer to confront these questions anyway, although in a less systematic way. Part of the art of good game construction lies in the ability to achieve a solution while juggling many variables; an orderly and sequential concept of game design will benefit the design team.

8.2.4 Phase IV – Developing the Exercise

After a concept report has been authorized, the team goes through three stages of design (building a prototype, technical evaluation and graphics design). The game elements must be fashioned appropriately, one by one. These initial modules must be fitted together and further developed during the rule-of-ten runs (see Step 14). The client should recognize that many different runs are required before the game itself is completely calibrated. Depending on the primary purpose of the game and the character of the system involved, there may never be a time when a final game exists. If an exercise is used in an ongoing context, there may be continuous modifications after an analysis of each game run in order to adjust the construct to meet the clients’ needs. Metro/Apex (Duke, 1966), a game for training students to cope with urban systems (municipal budgeting, air pollution, etc.), is still in use at a number of universities after 35 years; users maintain a monthly newsletter to report on evolutionary changes (additional models that have been integrated, new technology, etc.).

Step 14. Build, Test, and Modify the Prototype

The purpose of Step 14 is to construct an initial prototype that contains all gaming elements in rudimentary form. This trial and error method progresses logically, one game element at a time. The prototype evolves through ten test runs, each being progressively more rigorous. The “rule of ten” argues that a series of ten increasingly more precise rehearsals should be undertaken before the product is presented to the client. This requires about five test runs undertaken with staff (constructing and assembling the various components, and rough calibration) and a second set of about five runs with professional participants (final calibration). This iterative process ensures that the initial objectives are achieved; the procedure may be truncated, but this entails the risk of an immature product.

This initial prototype should be built quickly and with flexible materials to enable easy modification. In games of any complexity, there are numerous components that should be separately designed, constructed and assembled. These should contain rudimentary versions of each game element (scenario, events, roles, decisions, format, policies, rules, scoring, steps of play, model, data and information, accounting system, indicators, visuals, paraphernalia). These individual components should be created before attempting to assemble a prototype. All elements should be identified and any linkage with another game element should be described. A mockup of all elements at this stage will permit the designers to be more certain that the level of detail is appropriate and that it will provide the client with a preview of the final exercise.

The prototype should be carefully subjected to calibration before release to the client. If the game has much complexity, the final assembly and trials will reveal a variety of omissions, overlaps, inadequacies, trivia, redundancies and possible errors. Special attention should be addressed to the formulation of the final five test groups. The client may have a responsibility for participation during testing as spelled out in the specifications. Incremental changes in the construct reflect the learning obtained through these tests. Participant time can be squandered with immature products; this can result in the unnecessary aggravation of a captive audience. This phase requires that attention be directed to the original game objectives in order to maximize the final fit of the game.

Step 15. Technical Evaluation

The purpose of this step is to systematically compare project specifications and objectives with the proposed game structure using a technical matrix to ensure parsimony of design. A “game objectives/game elements matrix” is completed to cross check that the product under development conforms to the original specifications (as developed in Step 5). This matrix compares the carefully delineated objectives of the exercise on one axis against the gaming mechanisms employed on the other. The cells have language indicating how the specifics of the project have been met by one or more gaming elements. To the extent that a discrepancy exists, further refinement is required. This matrix not only ensures that objectives remain valid and are met by the product, it also assures that parsimony has served as the watchword and that no gimmicks remain in the exercise. The goals of the evaluation include checking on the generic criteria of validity, verisimilitude, playability, operability, and the specific objectives in terms of the five Cs (see Chapter 4) as well as the game content as described in the specifications. If a replicable, systematic process is used during construction, the rule-of-ten runs should establish validity.

Because the policy exercise approach is still in its infancy, clients have difficulty knowing what to expect from the final product and how to evaluate it when it is put into use. For this reason, it is important at the outset of a project to define the criteria that will later serve as the measure for the technical evaluation of the product. A set of descriptive criteria will help the clients and the participants understand the exercise. These would include the circumstances that induced the game, the availability of game kits, the character of the expected participants, the type of player involvement; the time required, the steps of play and plot outline, and the main dynamic of play.

Step 16. Graphic Design and Printing

The purpose of this step is to focus attention on the need to develop a professional presentation when transforming the prototype materials into a finished exercise. Graphic materials (visuals, paraphernalia, etc.) are used throughout the entire process of developing a policy exercise. The schematic (Step 7), the Concept Report (Step 13) and the final prototype require finished materials of a professional quality (in contrast to the prototype materials that tend to be expedient). Three issues come into play when considering graphic design and printing.

First, these materials will be evolving throughout the process; several people will have a hand in developing the prototypes. This can result in a lack of visual coordination among the materials. Unless a clear policy is established, materials can be prematurely brought to a final graphic stage resulting in unnecessary cost and delays. No commitment should be made to final graphics/paraphernalia until Step 16. Another concern is that these materials might best be contracted out to another division of the organization or to an independent contractor; this may be the most efficient way to obtain materials of professional quality. There also may be a requirement of the client to ensure that company policy is met concerning logos, copyright, etc. Finally, time delay at this stage is common, particularly if the materials are contracted out. If Step 16 is acknowledged as part of the total process from the outset, an appropriate time allocation can be made and retained in the schedule. This is particularly true if some software product has to be written and debugged.

8.2.5 Phase V – Implementation

The purpose of Phase V is to ensure proper use of the exercise by the client. At this interface, the game leaves the designer’s world and makes the transition to the client’s world. The game is no longer the responsibility of the designer (except in those cases where responsibility for facilitation rests with the design team); however, the team should oversee this stage by means of participation and staff training until the client feels comfortable with the product. If the initial objectives are to be fully achieved, the exercise should become vested; that is, the stakeholders should view the product as legitimate, non-threatening (within reasonable limits), and of immediate pragmatic value.

The successful use of the game by the client is crucial. Steps 17 through 21 are designed to systematically guide the design team and the client through this final stage. A number of concerns should be addressed here, including integrating the game into the client’s environment, facilitation, dissemination, ethical and legal concerns, and appropriate reports to the client. The design team should be present to the extent required and within the limits of the budget.

Step 17. Integrate the Game into the Client’s Environment

The purpose of this step is to ensure the proper integration of the exercise by the client. The objective is to make the exercise “fit” into the client’s environment. This requires that the client and the designer cooperate in anticipating the conditions under which the game will normally be used. This stage includes the tasks necessary to transfer the game to the people responsible for its use, facilitation, and maintenance. Organizations facing crisis will be preoccupied; the game should fit readily into their environment if it is to be used effectively. This is achieved by completing any refinements to the game and providing appropriate training to members of the client organization.

When defining the Specifications for Design (Step 5), i.e. its objectives and characteristics, careful thought should be given to the context of use of the final product (see Section 4.7 on contingency and the strategic process). Most games will be of greatest value when used in a format integrated into some larger context (e.g. a seminar). Specifications should determine the conditions that govern game use. A number of questions should be addressed: Under what conditions will the game normally be used? What follow-up circumstances are anticipated? Will the same group run the same exercise repeatedly? What considerations (social, political, technical, etc.) are anticipated relative to its use?

Even when the game is primarily designed for a one-time use, the client will often want the final product to be maintained and facilitated by a specific division (e.g. strategic planning and training). The process of transferring the game should ensure that those responsible for the re-use of the product have: a knowledge of the nature of the theory behind the exercise; familiarity with the concept report; and intimate knowledge of the particulars as may be required to facilitate the exercise.

A cultural translation towards the end of testing and before the game is given to the client may be necessary to achieve a better fit within the client’s corporate culture. Cultural translation refers to the adjustments required to fit the client’s environment (e.g. the inclusion of company or industry-specific jargon, adopting corporate procedures, the application of a company logo and artwork, etc.). It may also mean the removal of political red flags or problem areas identified during testing. In cases where the client’s language is other than English (or, as in many cases, the client intends to use the product in several countries), a language translation should be done in conjunction with the cultural translation.

Step 18. Facilitating the Exercise

The purpose of this step is to ensure that the client has access to proper facilitation skills for the exercise. A game of this kind is very demanding in several ways. Even if designed with care, a game requires a great deal of time commitment from participants as well as the cadre that facilitates it.

It is necessary at this point to distinguish between the designer and the facilitator (in some cases, one team serves both functions). During the final use of the game, the designer can no longer be held accountable; the game facilitator becomes largely responsible for its use. In many strategic implementation projects, the facilitators will not have been privy to the thought that went into the development of the concept report; as a consequence, the facilitators need to be familiar with this document before attempting to run the exercise. The specifications (Step 5) should have addressed the questions of facilitation (who will facilitate, what skills are needed, what training is required, and whether there are any debriefing considerations to be made mandatory for participant protection). In strategic projects, we strongly advise that the intended facilitators participate in the whole sequence of the design activities.

The facilitator assumes a considerable responsibility in undertaking a run of the game. He or she is responsible for the accurate completion of all accounts, communication exchanges and other interactions that occur. Facilitation also includes the activities required to ensure appropriate processing and evaluation. The process should ensure that participants are appropriately informed of content, structure, process, and the mechanics of the exercise. All the work that goes into an exercise of this type can be in vain if care is not taken to ensure that it is properly used. Facilitation is an art unto itself; but no matter how skilled a facilitation team may be, they should be provided with cogent materials to assist them in interpreting and using the exercise. The facilitator should use the exercise for the purposes and in the manner for which it was designed. A novice facilitator may resort to gimmickry to make the game more fun or to establish the game as more relevant to their particular ends. Arbitrary changes without full knowledge of their implications can do serious damage and may violate basic ethics of game use.

The facilitator should let the game proceed with a minimum of intervention. The various roles that have been established are part of a carefully coordinated system to permit an audience to address a problem. An overly strong facilitator, failing to understand the need to let communication occur spontaneously, can stifle interaction. The basic concept underlying the use of the game is to facilitate communication. Because the players are well informed, they do not need to be taught, but rather to confront each other and the issues in a productive and interactive style. Never attempt to defend the game as a predictive exercise or as a true replication of the world! It is neither. The game is to serve only one purpose – to prompt players to think about reality in new ways.

The participants need to be introduced to the role and function of the facilitator, with an emphasis on the point that a successful game run requires that procedures need to be followed. It can be damaging to engage in public disputes about particulars of the game and/or to let an ambitious player become too dominant. It is important to have one clearly defined central figure as the facilitator of the game whose word is beyond dispute. The facilitator must be the boss! This does not mean that the facilitator is not subject to challenge and interrogation by the players during the critiques and the final debriefing. Rather, it means that during the normal operation of a cycle, the players should submit to the instructions of the facilitator.

In addition to the facilitator and the roles embedded in the exercise, there are often other individuals present during the administration of the game; these might include role adviser(s), a bookkeeper, and/or other specialists. Generally, role advisors are used in games that are so complex that the start-up time would be prohibitive without them; they can assist players in learning both the function and the mechanics of the role. Subject matter specialists may be introduced at any time to aid the facilitator in transmitting factual information about the system.

No visitors should be permitted in the room during any part of the game. The presence of casual observers during the course of a game will be disruptive to the facilitator as well as the players. There is a tendency for observers to become enmeshed in the activities of the game, but in highly tangential and sporadic fashion. The result is that bona fide players, rather than being engaged in the primary dialogue that has been carefully established, are likely to find themselves in a secondary conversation. Casual observers may have neither an intense interest in gaming as a technique nor in the subject matter at hand, but simply a general interest in the proceedings; they are a deterrent to good communication and should be excluded. Casual observers in a game are invariably negative factors.

In some cases, scientific observers may be present to make observations of player behavior during the game relative to information processing, group dynamics, or other experimental activities. The presence of such observers raises ethical questions. Prior arrangements should be worked out that ensure no ethical codes are violated by the experiment or by the presence of the observer. If permitted in the game room, severe constraints should be imposed: absolutely no talking to the facilitator or any players, no interference with the activities, and a minimum of activity signifying their presence. Another type of legitimate observer may either be qualified on the basis of their interest in games or their interest in the subject matter; they can be included in the run of the game as assistants to the facilitator. These observers need specific tasks, materials and instructions that let them do their work without interfering with the game; communication with the facilitator or the players should be minimal.

Step 19. Dissemination

The purpose of this step is to deliver the policy exercise to the client and to ensure continued use of the exercise (if appropriate). The client’s successful exploitation of the product (if intended) requires that the original specifications anticipate the dissemination method to be employed as well as how facilitator training will be achieved. The concept report should detail appropriate techniques to minimize cost and complications. The non-commercial nature of policy games creates difficulty when preparing packages of the completed game for distribution; this is rarely a profit-making venture. Some designers have transformed a successful game into a product capable of commercial dissemination, and some publishing houses are making efforts to produce game kits. The task of reproduction and distribution of game paraphernalia is as important as that of training operators. There is both a strong ethical and professional need to reserve sufficient funds in the project budget to meet both functions.

A properly devised exercise can be quite revealing and can confront management with options not previously considered. As a result, the client may wish the exercise to be made available to a wider audience than originally specified. For example, the strategy game used in the Pharmaceutical case was transferred to the training division and used for internal courses. If the exercise is to be accessible, it should be transferred to a specific division to be properly maintained and facilitated. A basic information set should be made available to interested parties; this would include: title, date, client, designer, and description (content or subject matter); purpose (intended use); dynamics of play (participant, information, player involvement, time considerations); kit availability; and a basis for evaluating the exercise.

Step 20. Ethical and Legal Concerns

When developing a policy exercise, it is necessary to protect the participants, the client, and the design team against both legal and ethical pitfalls; there is a serious ethical responsibility for the product, both in its design and non-manipulative use. The policy exercise should be a safe environment for learning. If ideas are to flow freely and new insights to be gained, old shibboleths should be vigorously challenged and participants must be protected from retaliation.

Ethical Concerns

The designer and the operator share responsibility for any ethical problems that emerge during the use of the game. Several of these concerns have been addressed in our description of the design steps, we re-emphasize a few here. The design team has an obligation to provide the client with a clear conceptual statement of the characteristics of the process and the product. A game should only be designed and conducted if this seems to be the best and most appropriate way to address the problem. Often, the designer will not be the facilitator; therefore, the designer has an obligation to provide the facilitator with information that will allow the game to be used intelligently. This requires an accurate portrayal of the game (concept report, literature accurately describing the game objective and particulars, instructions for a mandatory debriefing for player protection, evaluation form, etc.) as well as ensuring that all materials are fully operational. A policy exercise is not a sufficient substitute for reality to permit an employer to use it as the exclusive measure of evaluation of an employee’s performance. This approach would be quite unfair, since people who perform very well in a real-world context (where personalities and the specifics of the situation are familiar) might behave differently in the game.

Inevitably, the policy exercise deals with sensitive issues within an organization. Because major policy issues will be involved, it is important that agreement be reached at the outset as to how sensitive materials will be handled. Procedures should be put into place permitting freedom of thought while providing adequate protection. All materials need to be treated with confidence; the process, and any products resulting from the process, are typically for internal use only. These internal reports are prepared for each aspect of the game; they should not be released without the express approval of the client. The benefit of these internal reports is that a written record is formulated as the game takes shape; they become invaluable when questions arise about the character of work undertaken.

Legal Concerns

If appropriate, it is important to consider the exploitation of a successful product as the specifications are developed. Legal precedent is probably adequate for any commercial versions that may be involved. Design specifications should address the issue of the ownership of the exercise (client, designer, or public domain), leasing arrangements, logo, patents, copyright, royalties, and the rules governing the retention, release or transmission of results. In the case of games that are essentially in the private domain, legal questions seem to be resolved by mimicking those procedures associated with the production of books and similar materials. Copyright, patent, trademark laws, and general procedures for royalties already employed for books can readily be transferred for use with games. Royalties and financial arrangements yield readily to standard convention.

Step 21. Reporting to the Client

The purpose of this step is to ensure proper closure for the project. The exercise is not complete until appropriate documentation of the results has been prepared reflecting the several stages of the design process and the results of subsequent use. As a minimum, there should be the original proposal (if contracted) or a prospectus (if in-house), a concept report and a final report to the client. An early agreement should be reached as to how these should be circulated (To whom will they be routed? How much time allowed for response? How to resolve differences that emerge?). In addition, the final product should be documented as appropriate; this may take the form of manuals for the game facilitator and participants, final graphics and artifacts, etc.

In many cases, a final “white paper” will need to be developed. This text should capture the results of the game for any reporting requirement that may exist. It is often useful to provide the participants with materials that summarize the happenings of the game. This document can also be circulated to the participants to confirm impressions derived during the exercise. Proper follow-up can serve as a further commitment of the participant to the process, by reinforcing the “message” from the experience. In some cases, a post-exercise evaluation may be required.

Games are notoriously hard to evaluate (Chapter 6 and the Appendix discuss evaluation in more detail). The criteria and methodology to measure the effectiveness of the exercise should be established in the concept report. The selection of an appropriate validity test is imperative; this should be done in accordance with the original goals, objectives, and specifications. Evaluation undertaken by a subject matter specialist who knows little about gaming as a disciplined activity may present a danger. When a formal evaluation of the exercise is undertaken, the initial specifications should determine whether the exercise will be measured against the judgment of experts in the context of use, the judgment of professional gamers and/or the reaction and evaluation of the participants.

Whatever method is chosen, after some experience with the final product, an evaluation is prepared. The goal is to develop a written description of the objectives of the exercise, the character of the product, the nature of the intended use, and a summary of the results of the game. This report must capture the main arguments, majority and minority opinion on these issues, and any agreements or decisions that may have been reached. If gaming/simulation is going to fulfill its promise, our profession should place a stronger emphasis on the need for improved evaluation methodologies (see Chapter 6). Failure to clearly define our methodology places the discipline at risk.

As stated before, a successful exercise often reveals opportunities not previously addressed (e.g. the client may find the product of potential value as a public relations tool). In many cases, it has been useful to devise extensions of the initial product to focus on particular situations and/or to be aimed at a particular subset of decision makers who are attempting to achieve change within the organization. Insofar as the client finds it helpful, it may be important in this phase to make presentations within the company to explain the policy exercise approach so that the results are fully understood beyond those who actually participated. In the case of the pharmaceutical company, the original version of the exercise was