Kahn and Mann's "Ten Common Pitfalls" (1957)
 |
| Herman Kahn |
At the RAND Corporation of the 1950s, perhaps no figure was larger (quite literally) than that of Herman Kahn (1922-1983).
Weighing in at over 350 lbs., Kahn joined RAND in the late 1940s. Educated in physics at UCLA and Caltech, and having served in the Army during World War Two, he initially participated in RAND research revolving around the physics and design of nuclear warheads. But in following decade, the scope of his work would come to include issues more directly related to nuclear strategy.
When I've interviewed people for my research on Albert Wohlstetter (1913-1997), a strategist who was also at RAND in the 1950s, the subject of Herman Kahn has often come up. RAND alumni would often say that Kahn was worth learning about, not just to compare and contrast his research with that of Wohlstetter, or Bernard Brodie, or William W. Kaufmann, or Andrew Marshall, but also because Kahn himself was such a strange, singular personality. Indeed, descriptions of Kahn were typically peppered with words and phrases like "brilliant," "hilarious," "grotesque," "gregarious," "sometimes exasperating" and "larger than life."
To explain Kahn's personality better, many of his RAND-era colleagues would draw attention to his late-1950s lectures on what were at the time very unpopular -- even inflammatory -- subjects, such as nuclear war-fighting, and civil defense in the face of nuclear war. During these marathon-like lectures, which were sometimes spread out over several days, Kahn would use his mesmerizing, rapid-fire oratory and almost comical gesticulations to punctuate his controversial claims, often to the amusement, amazement and utter astonishment of the crowd. The success of these lectures led Kahn, with no small amount of prodding from his assistants, to publish On Thermonuclear War (1960) 
. This mammoth 400-page bestseller -- the title of which consciously alludes to Clausewitz's classic On War -- is in part a word-for-word anthology of his many sensational talks.
Kahn and Mann's Ten Common Pitfalls
Less known, though, are Kahn's reports and memos, many of which are now available on the RAND Corporation's website. In my mind, one stands out:
Herman Kahn and Irwin Mann, Ten Common Pitfalls, RM-1937 (Santa Monica, CA: RAND Corporation, July 17, 1957).
This research memorandum (hence its designation as "RM-1937") was intended to be a very preliminary draft of a chapter in Kahn and Mann's planned -- but never completed -- book,
Military Planning in an Uncertain World. For those interested, RM-1937 offers the book's
provisional table of contents.
Ten Common Pitalls examines a series of methodological problems that often hamper or distort the work of policy analysts. Kahn and Mann's examination, however, is intended to be descriptive rather than analytical. In the introduction, they write that they hope RM-1937 will serve "as a sort of checklist" for analysts, or at least alert them "to the things to look for in an analysis." As a bonus, the research memorandum offered a hand-drawn illustration of each pitfall; I'm told these were drawn by Kahn, but I'm not certain. These humorous drawings, reproduced below, give a sparkling sense of Kahn's own wit and personality.
Pitfall #1: Modelism. Kahn and Mann write: "We shall start by considering what is to many people, the heart and soul of Systems Analysis--the use and abuse of models. We have already explained that it is necessary to use idealized models which abstract essentials and make assumptions explicit. It is, however, a frequent pitfall to abuse this modeling by being more interested in the model than in the real world...." (To read more,
click here.)
Pitfall #2: Statistical Uncertainty. Kahn and Mann write: "The second pitfall is illustrated by two pair of dice, one pair indicating good luck and the other bad luck. This is the kind of uncertainty about which books on probability theory are written. One can write equations and formulas when discussing it, and generally everybody involved will agree with the
technical discussions.
"In many practical problems, the only way to analyze the effects of Statistical Uncertainty is to do Monte Carlo calculations. While these are often convenient and useful, there seems to be a definite tendency to exaggerate their importance or necessity. In many cases simpler expected-value calculations would be satisfactory. The work that is saved might be better used in other problems of the analysis... Where Statistical Uncertainty is important, it usually is because it affects Low and High Confidence measures...." (To read more, click here.)
Pitfall #3: Real Uncertainty. Kahn and Mann write: "To avoid possible confusion, we should start this section by mentioning that usually a measure is classified as being High Confidence or Low Confidence, not because of Statistical Uncertainties as implied in the last section, but rather because of a more fundamental kind of uncertainty which we have called Real Uncertainty. This is the kind of uncertainty to which one might possibly assign
subjective probabilities, but for which it is impossible to obtain general agreement on the numerical value of these probabilities...." (To read more,
click here.)
Pitfall #4: Enemy Reaction. Kahn and Mann write: "It is obvious that our problem would be simplified if the enemy were not trying his hardest to thwart us. Our planning is complex mainly because his reaction and its effect on our design must be considered. In spite of this it is very common to treat the enemy as being supine or inert. Much less common but almost as bad is to treat him as being omnipotent...." (To read more,
click here.)
Pitfall #5: Over-Concentration. Kahn and Mann write: "It is usually necessary when studying a component of a larger system to decide what the real problems are so that one can concentrate on them. Unless this is done, the study may be hopelessly complicated. However, there is a real danger that the factoring out of a suitable area will be done carelessly or unskillfully and an overly-narrow viewpoint adopted. One can then end up by working a wrong or irrelevant problem. We have no objections to the driver in the illustration looking at the blonde--she is worth looking at--but not exclusively...." (To read more,
click here.)
Pitfall #6: [Time] Phasing. Kahn and Mann write: "Our picture shows an Air Force officer glancing coyly at the past (Civil War) and the future (Buck Rogers) but not really interested. This is, of course, a fairly serious mistake. One does not buy a military organization from a department store as a unit...." (To read more,
click here.)
Pitfall #7: Over-Ambition. Kahn and Mann write: "If one is at all conscientious it is easy to fall into the pitfall of trying to do too big a job. Many of the points raised in discussing Real Uncertainty, Enemy Reaction, Over Concentration and Time Phasing tend to lead the analyst in the direction of increasing the amount of work he has to do. It is, however, immediately apparent that
one of the main tricks in turning out a good analysis is to spend a lot of time inventing questions which can be usefully answered within the capabilities and time available...." (To read more,
click here.)
Pitfall #8: Fanaticism. Kahn and Mann write: "Fanatacism [sic] is a peculiarly easy and insidious sin. What we are worried about here is not the man with wild hair and rolling eyes and incoherent speech, who has gone wild about some gadget or technique and expects to solve all problems with it. It is not that this kind of guy doesn't exist--he does--but he is easy to recognize. His mannerisms and intensity are themselves sufficient to remove him from serious consideration.
"We are thinking here of the completely different kind of problem illustrated by the 'zoot suiter.' The curious thing about the 'zoot suiter' is that in his circles he is considered well dressed; his friends like his clothes. In fact, what we are stressing is not the fanatic individual but the fanatic organization...." (To read more, click here.)
Pitfall #9: Hermitism. Kahn and Mann write: "The problems of communication and persuasion are often ignored though they are central to getting recommendations translated into policy. It is really a question of the proper type of interaction between the analyst and the other parts of the analyst's organization and also between the analyst and the world of policy. A good analysis does four things simultaneously...." (To read more,
click here.)
Pitfall #10: Butch. Kahn and Mann write: "The ever-present possibility of a Butch is rarely discussed but is very important. There are two types--the first and obvious one is the classical mistake in arithmetic. This possibility gets worse with the advent of high-speed computing (only now it takes the form of a coding error). This kind of mistake can usually be avoided by careful checking and equally careful
qualitative evaluation of the results to see how reasonable they are.
"The second and more insidious kind of Butch is the completely mistaken technical notion or fact. The Systems Analyst, who is doing a broad context study, may have to work with a large number of experts drawn from many fields. It is crucial in dealing with these experts not to accept their statements uncritically, no matter how scholarly and distinguished they are...." (To read more, click here.)
Conclusion
Although Kahn and Mann did not end up writing their planned book, Ten Common Pitfalls remains as a suggestion of what might have been. More important, RM-1937 contains some helpful reminders of the foibles of analysis (and the analyst) that can make problem-solving efforts, well, even more problematic.
And, if nothing else, Herman Kahn's illustrations are priceless.
Posted by Robert at January 24, 2008 11:42 AM
