Object: [1]application/x-shockwave-flash
   "The art of asking questions"

  PRINCETON UNIVERSITY PRESS 999     The  Art of   Asking  Questions   BY   STANLEY L. PAYNE     PRINCETON, NEW JERSEY  1-951     9??  99?   ???  999     Copyright 1951 by Princeton university Press     Second printing, 1954  Third Printing, 1957   Fourth Printing, i960  Fifth Printing, 1963  Sixth Printing, 1965  Seventh printing, 1965     Printed in the United States of America     TO   Claude Robinson   WHO INSISTS THAT COMMUNICATION  IS OUR GREATEST PROBLEM     Foreword     Although a number of books have already appeared in  the relatively new field of public opinion and market research,  there is no book like this one. It is important and timely as  well as unique. It deals with the warp and woof on which all  surveys depend * the use of words.   "Spoken language," wrote Whitehead, "is merely a series  of squeaks." And anyone who reads what Mr. Payne has  written here will get a concrete understanding of what  Whitehead meant when he said that "Language ... is always  ambiguous as to the exact proposit
ion which it indicates."   There has been a long-standing and wide recognition of  the care needed in asking the right question in the right way,  as the author points out. But all too often this recognition  seems to remain on the intellectual and verbal level of lip  service with the importance of constructing the right ques-  tion neglected in practice both in the survey operation itself  and in the research conducted to improve methods. -The  apparent reasons for the relative neglect of the area Mr.  Payne has probed reflect the temper of the times.   For one thing, those of us engaged in "research" like to  think of ourselves as "scientists." We like to think that there  are certain "rules" which we can discover and follow in  order to be "objective." And we tend to think that if we  can only quantify our material and manipulate it statis-  tically, then, and only then, are we being "scientific." Hence,  much of our research deals with technical problems of  measurement.   But the author quite bluntly s
ays he is dealing with an  art, not a science, and the title of this book reflects, I think,  a correct understanding of scientific procedure itself. For  nearly any scientist whom we or history would label as  "great" agrees that it is quite mythical to think of the scien-   [ vii ]     tist as being "objective." The scientist is involved in making  personal value judgments at nearly every stage of his work *  in the setting of his problem, the selection of the aspects of  the problem which he feels are crucial and should be investi-  gated, the selection of the method he will use for his in-  vestigation, as well as the interpretation of his results. Real  scientific research of any kind is rooted in value judgments.  The list of outstanding scientists in any field would show  that these men and women were essentially great artists  in the sense that they had the intuitive capacity to ask  themselves the right questions in the right way at the right  time.   The intuitive hunches or hypotheses which have g
iven us  giant strides in our understanding of nature or of human  nature have seldom been created by following any rule of  thumb or any one method. And especially in. the area of  inquiry with which the author deals here, the number of  variables that must be implicitly taken into account are  legion. They are also subtle and they defy quantification.  They cannot be unwoven and analyzed independently for  each depends on the others for its own existence and func-  tion. Survey, questions tap an individual's motives, his  expectancies, his unique experiences, his whole range of  identifications and loyalties. In short, they are trying to  discover certain aspects of what we might call an individual's  assumptive world, a world which he himself has constructed  during the course of life as he has attempted to work out a  set of conditions within which he can satisfy the urges that  characterize him as a human being.   Since the framing of the right question in the right way  does involve so many subtle aspe
cts, an investigator's in-  tuitive ability to devise the right questions will be propor-  tional to the number of cues and signposts he takes account  of consciously and unconsciously as he goes about his task.  The insights and cautions the author points out in the present   [ viii ]     volume should go far to increase the range of inclusiveness  of the cues which investigators who study this volume can  bring to bear in the process of question construction. Yet  even if an investigator does test his question against the  sample one hundred considerations the author cites at the  end of the volume, he must remember that he is still being  an artist, that he still has the intuitive value judgment to  make of how to weigh and integrate these one hundred  considerations in relation to the concrete situation he is try-  ing to get at.   In addition to our bias of wanting to keep research "scien-  tific" in an artificially restricted way, there may be another  reason why no book like this has so far appeared i
n spite of  the crucial nature of the problem. This is the tendency for  those persons who reach high degrees of skill as practitioners  to hesitate or neglect to try to put down in print some ver-  balization describing the skills they have attained. Many  times such practitioners have little interest in doing this;  usually they are too busy; some have no particular facility  to generalize from rich backgrounds of / experience. As a  consequence, those of us who do not have the opportunity,  the inclination, the ability, or the stamina to put ourselves  in the position of participating in the actual occasions that  constitute a certain area of life, may be forced to operate in  a comparative darkness which we should like to have reduced.   In so far as; an investigator is not constantly forced to  test his assumptions as to what constitutes a good question  in actual situations, he may tend to create in his mind, for  reasons of expediency, an image of a non-existent "average"  respondent. And he may do th
is even though, intellectually,  he may realize fully that there is a long and devious route  from the central office where questionnaires are designed on  the basis of a certain amount of pretesting to the final re-  porting of the answers interviewers send in.   Mr. Payne is one of those rare individuals who has taken   [ ix ]     time off to try to find out what this experience adds up to  and how the art of asking questions may be described. A  book as rich in insight as this could only have been written  by someone who has had long and varied experience and who  has sensed the difficulties encountered in real situations. The  problems he raises and the answers he gives are derived  directly from concreteness. And it is these problems derived  from concrete experience that the academician or the in-  vestigator whose job confines him primarily to a desk are apt  to be unaware of. Perhaps the author's major contribution is  his own formulation of what questions to ask about asking  questions.   While this
 book is written chiefly with the practical,  everyday problems of question wording in mind, it is by no  means confined in its usefulness to those people whose job  it is to construct or to ask questions. For the problems the  author raises and the illustrations and data he brings to bear  on these problems pose a number of questions of theoretical  interest for specialists in a variety of areas. The sociologist  will see in Mr. Payne's material problems relating to con-  cepts such as class, status, and social change; psychologists  will see problems related to concepts such as frame of ref-  erence, ego involvement, and the attributes of opinion;  semanticists will find documentation for many of their  generalizations dealing with sources of misunderstanding as  men try to communicate with one another.   The reader should be forewarned that the easy and light  style of Mr. Payne's writing should not obscure the difficult  and serious problems he writes about. The author has de-  liberately tried to write 
an interesting, highly readable' book.  I believe he has succeeded. And I believe that even the most  experienced investigator, if he has a sense of humility at  all, will learn a great deal from this little volume.   Hadley Cantril   Princeton University  Princeton, NJ.     Preface     In the first place, this little book was not written by an  expert in semantics, not even by a specialist in question  wording. The author is just a general practitioner in research.  Having made more than my share of mistakes in phrasing  issues for public consumption and feeling the need for a  book on the subject, I found that it was necessary to write it  myself. In the process, my respect for the semanticist has  increased beyond words. He is smart enough to use symbols  to represent his ideas, but I have been so foolish as to try to  use words in talking about words.   In the second place, this book is very limited in its subject  matter. It discusses the wording of single questions almost  exclusively. It hardly touche
s upon problems of question  sequence or the overall matter of questionnaire design. It  seemed difficult enough to deal with wording alone. Perhaps  another book and another writer will cover these other  subjects. /   Third, the reader will be disappointed if he expects to  find here a set of definite rules or explicit directions. The  art of asking questions is not likely ever to be reduced to  easy formulas. As it stands, this book consists of some  observations of human behavior, a few principles of wording,  many exceptions to these principles, several unexplained odd-  ities, and numerous unsolved dilemmas. It is undoubtedly   richer on the how-not-to side than on the how-to side. For   i   want of a better description, it might be thought of as a  collection of possible considerations for question wording.   Fourth, I happen to think that even a serious subject can  be treated too seriously. Consequently, I have included some  of the amusingland perhaps irrelevant ideas which occurred  to me as I wro
te. In other words, I have enjoyed writing this  book. I hope you will enjoy reading it.   [ *i ]     Last but not least, my apologies and thanks to all the  people who graciously consented to read the original manu-  script. I have not always acted on their suggestions, but I  do appreciate their aid and counsel. Among those friends and  colleagues whose criticisms have been especially stimulating  are: Joseph Bevis, George Caldes, Frank Chokel, George  Cole, Thomas Crawford, W. Phillips Davison, Richard Ditt-  mer, Kendrick Few, Kenneth Fink, LeBaron Foster, Joseph  Goeke, Joseph Hochstim, Arthur Holland, Roger Lloyd,  Robert Mayer, Raymond Nasssimbene, Benjamin Phillips,  Donald Rugg, Esther Schwartzstein, J. Stevens Stock, Knute  Warren, and Albert Westefeld. My strongest plea for for-  giveness and deepest appreciation go to Lucy Leigh, who. so  obligingly labored over every word with me.   Stanley L. Payne     PROPERTY OF  7h" Kansas S.ate University of  Agriculture (J Applied Science  TC M India.     
 [ xii ]     Contents     Foreword by Hadley Cantril  Preface   1. Why concern yourself?   A plea for the importance of asking good questions   2. May we presume'?   A lecture on taking too much for granted   3. Who left it open?   A description of the free-answer question and its  demerits   4. Boy or girl ?   A discussion of the two-way question and its  duplicities   5. Win, place, or show? /   A discourse on the intermediate nature of the mul-  tiple-choice question and its misconstructions   6. How else?   Descriptions of special types of questions and  their special faults   1   7. Still beat your wife?   A sermon! on the care and treatment of re-  spondents J   i   8. Can you make it brief?   An illustrated lecture on the virtues of brevity  and simplicity   9. What's thejgood word?   A futile search for a list of perfect words, sup-  plemented by a list of 1,000 well-known words   [ xiii ]     10. What's wrong with "you"? 158   A rogue's gallery of problem words, with case  histories   1 1 . Isn't th
at loaded ? 177   An admission of guilt, with extenuating par-  ticulars .   12. How does it read? 203   A short lesson in punctuation, phonetics, abbre-  viations, etc.   13. Is it possible? *>' 214   A visuaLdemonstration of the development of a  passable ''question   or   14. How's that again? 228   A concise check list of 100 considerations   References 239  General Index and Index of Examples 243     [ xiv ]     ? <ijfrt of Asking Questions     f   1 . Why concern yourself?   A PLEA FOR THE IMPORTANCE OF ASKING  GOOD QUESTIONS   What is the need for this first book on question wording 4 ?  No one else has considered it necessary to devote a whole  book to the subject. A chapter or two has always seemed  enough before. Articles of several pages frequently appear  in professional journals here and there. What more can a  book do for question wording? If it all boils down to the  familiar platitudes about using simple, understandable, bias-  free, non-irritating wordings, all of us recognize these ob-  vio
us requirements anyway. Why say more ?   Oblivious of the obvious   One reason for elaborating on the subject is that all of  us, from time to time, forget these requirements. Like some  church-goers, we appear to worship the great truths only one  day a week and to ignore them on working days. Or we  remember a certain example, but fail to see how it applies  to other situations. In combatting our very human frailty, a  more provocative set of examples and a detailed list of  points to consider may be more helpful than the isolated  examples and the broad generalities which we now so often  disregard.   The import IS important _   Another reason for separate treatment is to help us realize  how basic the phrasing of questions is to worthwhile re-  search. As it is, we may look upon the few available examples  of differences wrought by wording as mere freak occurrences.  There is no danger of our making such weird mistakes, we  think. Besides,j if the question "works," it must be a good  question.   [ 3 ]   
  By now, we should realize that the fact that something  "works" does not mean that it works correctly. The Literary  Digest poll seemed to "work" all right until 1936. The  methods of subsequent election polls seemed to "work" all  right until 1948. In both of these cases, attention has con-  centrated on sampling difficulties as causes of the wide mar-  gins of error. In the case of the Digest the faulty sample  was no doubt the prime contributor to the error. At least one  expert had predicted how and why this faulty sample would  lead to error (1). In the more recent case, sampling is only  one of the many possible causes, according to the Social  Science Research Council committee (2).   Be that as it may, people * laymen and qualified experts  alike * are impressed that sampling is an important feature  of the survey method. Even if we agree that they are right,  however, let us not forget that the Gallup Poll in 1948  overestimated the Dewey vote by only^ 4.4 percentage points,  and the Digest poll i
n its 1936 prediction of the Landon vote  was only 19 percentage points too high.   I use the word "only" advisedly because, as we shall see,  survey differences resulting from changes in words or phrases  sometimes amount to considerably more than a 19 point  error ! Perhaps, if we grant that by emphasizing the impor-  tance of sampling the election upsets have contributed to  advances in sampling techniques, we should deplore the (fact  that nothing similar has happened to concentrate attention  on the importance of question wording.   Tens versus tenths   At the present stage of development of the survey method,  improvements in question wording and in other phases can  contribute far more to accuracy than further improvements  in sampling methods can. I don't mean that the sampling  experts should stop seeking further improvements; trying to  knock a few more tenths of a percent off the statistical error.   [ 4 ]     But, while they are laboring with tenths of a per cent, the  rest of us are letting tens
 of per cents slip through our fingers.  As Frederick Stephan of Princeton University has remarked,  "It's like using a surgeon's scalpel in a butcher shop."   The experts said so   Even as far back as 1936, however, when the Literary  Digest was riding to its fall, a group of experts called ques-  tion wording the Number One problem. Howard T. Hovde  asked a sample of researchers what they saw as the principal  defects of commercial research (3). Here are their most  frequently mentioned criticisms :   Improperly worded questionnaires 74%   Faulty interpretations 58   Inadequacy of samples 52   Improper statistical methods 44  Presentation of results without   supporting data 41   Since three experts in every four pointed their fingers at  question wording, it seems that the subject^hould have been  worthy of concentrated attention. The specialists in statistics  and sample theory certainly didn't let these expert opinions  dampen their efforts. It's too bad that question worders  weren't more stimulated by
 Hovde's report.   Samuel A. Stouffer and his collaborators in their recent  monumental work on the American soldier (4) came to a  similar conclusion: "To many who worked in the Research  Branch it soon became evident that error or bias attributable  to sampling andjto methods of questionnaire administration  were relatively small as compared with other types of vari-  ation * especially variation attributable to different ways of  wording questions." \     [ 5 ]     Jack-of -all-trades   Probably the reason that the question worder hasn't done  more to advance his phase of research is that he just doesn't  exist, at least not as a specialist. The statistician is the only  one among us who has a specialty. All the rest of the work  comes under the jurisdiction of a jack-of-all-trades. This  man's job is to develop the questionnaire, pretest and revise  it, have it printed, select, train, and supervise the inter-  viewers, conduct the survey, analyze the results, write the  report, and present the findings. 
His attention is necessarily  divided. Question phrasing is but one part of the complex  machinery which he must put together and operate. Small  wonder that this all-around type of researcher has not had  much time to formulate detailed statements on question  wording.   Not special pleading   Having said that sample theory has moved ahead through  specialization, I do not mean to imply that question wording  needs also to be treated as a specialty. In actual practice,  wording cannot be thought of as being in a vacuum apart  from other phases of the survey method. It would be ex-  tremely naive to expect a single question to provide "some  magic way of reducing a complex matter of people's atti-  tudes, wishes, and aspirations to some simple wording which  will not bias the returns." Most researchers will agree with  Daniel Katz (5) that the solution usually requires "an  integrated questionnaire which explores the problem compre-  hensively from many angles."   The fact that this book deals with the subje
ct of question  wording by itself and usually in terms of improving a single  question does not mean that the other phases' of the work  can be slighted. The steel square is a useful tool to the  carpenter and much can be said about its proper use without  denying his need for jack-plane, saw, hammer, nails, and   [ 6 ]     so forth. Several books are available on the use of the steel  square. It is in much the same way that I advocate paying  serious attention to question framing. For people who are  also interested in the more general features of the question-  naire technique, I might suggest Albert Blankenship's first  book, Consumer and Opinion Research (6).   All types of surveys   The importance of wording is not restricted to any single  type of survey. The mail questionnaire presents much the  same problems of wording as the personal interview. The  so-called "factual" survey is in need of careful wording just  as the attitudinal or opinion survey is. Although opinion  surveys present the greatest v
ariety of examples, experience  in factual or census-type enumerations also furnishes enough  examples to show that facts as reported in answer to ques-  tions are not always the facts that exist. We shall draw upon  experience with all these types of surveys here, but most of  the discussion will be based upon opinion surveys conducted  by personal interview. First, then, let us lqok at some opinion  questions.   Implied alternatives   Sometimes the questioner assumes that the negative side  of the question is so obvious that it need not be stated. He  may simply ask :   Do you think most manufacturing companies that lay off  workers during, slack periods could arrange things to avoid  layoffs and give steady work right through the year?   63% 'said companies could avoid layoffs,  22% said they couldn't, and  15% lhad no opinion. N   The alternative here seems to be so implicit in the question  that it need not be stated. Either companies could avoid   [ 7 ]     layoffs * or they couldn't. No other interpre
tation seems  possible. But what happens when we take the trouble to  state an alternative to another carefully matched cross sec-  tion of respondents?   Do you think most manufacturing companies that lay off  workers in slack periods could avoid layoffs and provide  steady work right through the year, or do you think layoffs  are unavoidable?   35% said companies could avoid layoffs,  41% said layoffs are unavoidable, and  24% expressed no choice.   So, a few words changed here and there and explicit state-  ment of the other side of the picture results in a 28 percent  falling off from the affirmative side of this question !* This -  suggests the need for definite expression of the alternatives.  Most of the discussion about alternatives, however, can be  postponed to later chapters where we will not be so intent  upon demonstrating the basic importance of wording.   Three little words   When we see the three words * might, could and should *  together, we realize that they have somewhat different con-  n
otations. Yet when it comes to stating questions, we may  sometimes use these words as synonyms. The trouble with  this assumption is" that the public actually does see distinc-  tions among these words and changes its replies to fit. Here  are the results of an experiment in which all things, other  than these words were kept equal. A casual reader might  not detect the differences among these three questions :   Do you think anything should be done to make it easier for  people to pay doctor or hospital bills?   *This example, as well as many others throughout these pages, came from  the files of The Public Opinion Index for Industry, a service of Opinion  Research Corporation, Princeton, N.J.   [ 8 ]     Do you think anything could be done to make it easier for  *people to pay doctor or hospital bills?   Do you think anything might be done to make it easier for  people to pay doctor or hospital bills?   But enough respondents understand the. distinctive feature  of each connotation that significant differ
ences show up in  the replies of three matched samples of people: 82% said  something should be done, 77% said something could be  done, and 63% said something might be done. The. two  extremes, should and might, come out 19 per cent apart.  This is the same amount by which the Literary Digest poll  missed the 1936 election. Again we see that wording is as  important as sampling, at least in opinion polls.   The issue at question   Question wording involves more than toying around with  this word and that to see what may happen, however. It is  more than a mere matter of manipulation of words to pro-  duce surprising illusions. The most critical need for attention  to wording is to make sure that the particular issue which  the questioner has in mind is the particular issue on which  the respondent gives, his answers.   If we refer again to the three questions using might, could,  and should, we observe that they pose different issues. The  should wording brings up the moral issue of need in the  sense of, "
It's ai crying shame ! Something should be done  about it." The could wording poses the issue of possibility,  "Yes, but could\ anything be done?" The might wording  moves to the issjue o{ probability, "Maybe it could, but it  might or might not be done." The results show that more  people see the moral need for an easier payment method than  grant the possibility of doing anything about it and that even  fewer people trunk such a method is likely to be put into  practice. These are three distinct sets of basic content. Yet   [ 9 ]     any one of them might inadvertently have been used alone  to cover any or all of the three contents.   To. assure that the intended issue is understood, then, is  a fundamental function of question wording. A large share  of our discussion will be given over to ways and means of  reproducing in the minds of respondents the same issues that  are in our thinking.   Census "facts"   Now, for some examples of the importance of question  phrasing in factual surveys. The U.S. Bureau
 of the Census  no doubt produces more facts than any other organization  anywhere. Its experts consider question wording an impor-  tant feature of securing this wealth of information. They  know that such facts as the number of employed workers,  the average number of bedrooms in homes, and even the  size of the population itself are susceptible to different in-  terpretation according to the skill used in devising the ques-  tioning techniques.   Baby counting   Would you believe that in the decennial Census there is  danger of undercounting the population because some par-  ents neglect to report their infant children? This very real  problem arises not so much because of any lack of" pride in  the new offspring, but apparently because the parents are  not yet used to thinking of the new little "it" as a person.  Consequently, the question was asked at every dwelling:  Have tve missed anyone away traveling? Babies? Lodgers?  Other persons staying here who have no home anywhere  else? The enumerator was a
lso instructed to "be sure to  include ... all children, even the very youngest." In addi-  tion, he received an extra 10 cents for each Infant Card,  which was to be filled for every child born in January,   [ 10 ]     February, or March 1950. These precautions probably  helped in the recording of thousands of infants who might  otherwise have been overlooked (7).   Found: 1,400,000 workers   Take the problem of estimating the millions of persons at  work in the nation, as done in the Monthly Report on the  Labor Force of the Census Bureau. Gertrude Bancroft and  Emmett Welch have explained how a change in the ques-  tioning technique affected these estimates (8). Prior to July  1945 a single question was used. It asked, Was this person  at work on a private or government job last week? Begin-  ning in that month, two questions were substituted. The first  of these merely asked what the person's major activity was  during the preceding week. If the major activity was some-  thing other than working, the enu
merator then asked whether  in addition the person did any work for pay or profit during  that week.   The upshot of this change in questions was that in the trial  when both versions were used, the new questioning showed  an increase of 1 ,400,000 employed persons over the old word-  ing. About half of these additional workers had worked 35  or more hours during the week under consideration !   Found : 500,000 unused bedrooms   This next example is more one of change of definition  than of the wording of the actual question, but in factual  surveys the two so often go hand-in-hand-that they may well  be taken together. During World War II the Census Bureau  conducted hundreds of surveys for the National Housing  Agency. Among these were some for the Homes Use Service,  which was interested in inducing s home occupants in critical  areas of housing shortage to make rooms available to war  workers. The delicate problem posed to the Census Bureau   [ n ]     was to determine how many bedrooms were not in use a
s  sleeping rooms in these areas.   This required asking a representative sample of home  dwellers how many bedrooms they had and how many they  were using for sleeping. It was necessary to explain that for  the purpose a bedroom was any room except the living room,  dining room, kitchen or bath, which contained a bed or which  with the addition of a bed could be used as a sleeping room.  When it came to asking how many of these bedrooms were  in use, it was necessary to emphasize use by the occupants for  sleeping. Excluded from this use were guest rooms; bedrooms  used only as studies, sewing rooms, storage rooms, or play-  rooms ; and bedrooms reserved for the use of absent members  of the family such as students away at school.   Surveys in 83 critical areas showed a total of 500,000  bedrooms not in current use for sleeping. Only a sixth of  these had previously been offered for rent. Publication of  these results, area by area, helped greatly to spur room regis-  trations (9).   This was a case, often 
duplicated in other surveys, where  the respondent was asked to consider a situation in some-  what unfamiliar terms. That playroom or sewing room had  never been used as a bedroom! Yet for this survey it was  called a bedroom. This survey was different from many in  that the researchers recognized that they were inverting  normal thinking" and went out of their way to define just  what was wanted. In other studies we sometimes seem to  forget that we are dealing with-an unfamiliar situation or in  unusual terms.   A whiff of ammonia   We can leave the Census type of enumeration to speak for  a moment of the facts developed in marketing studies, such  as surveys of consumer habits. The example we shall use is  illustrative of what may happen when we phrase our ques-   [ 12 ]     tions with unfamiliar terms, or with familiar terms used in  unfamiliar ways.   Recently a questionnaire on the subject of household dis-  infectants was mailed to a national sample of housewives.  Now, it happens that almost every m
anufacturer of a dis-  infecting product is anxious that his particular product be  accepted as a general household disinfectant. You can verify  this by looking at the label on almost any bottle whether it  was introduced to your home as an antiseptic, germicide,  laundry bleach, or for other uses. It seemed entirely natural,  therefore, that the questioning should be done in terms of  "household disinfectants."   Despite the manufacturers' wishful thinking, however,  housewives tend to think of such products in terms of their  specific uses. Consequently, only l per cent of the respondents  reported using ammonia among their "household disinfect-  ants." This made it painfully obvious that the reference to  household disinfectants did not conjure up the thought of  household ammonia.   A later survey asked about "bottle

References

   Visible links
   1. https://cdncache-a.akamaihd.net/items/e6a00/storage.swf?r=1