Question format and response elicitation

Open ended questions
Closed questions
Alternatives

There are two main types of question format:

Open ended in which the format of the response is free, with the user phrasing their own replies
Closed ended in which the respondent selects one or more response from a set of possible answers

Closed ended questions are easier for the respondent to answer, and also to code and analyse, than open ended questions which require content as opposed to statistical analysis. They should therefore be used sparingly in questionnaires, if at all.

For a discussion of these issues, see 'Open versus closed questions - an open issue' by Gerald Vinten.

Open ended questions

According to Peterson (2000)1 there are a number of instances when open ended questions may be appropriate:

  • when it is not easy to foresee the answers
  • when the responses may be influences by the choices
  • when the variables measured dictate unaided recall
  • where flexibility is called for by unanticipated events
  • where initial responses necessitate more in-depth follow-up questions.

Open questions are often used when carrying out research on issues prior to developing a questionnaire, in order to find out more background information and develop hypotheses on which to base questions.

Closed questions

These questions require some skill in writing, but are easier to code and analyse. The main types are as follows:

Multiple choice

The most straightforward format, this involves selecting from a range of options, as in the following example, taken from 'Libraries and desktop storage options: results of a Web-based survey':

Note the way in which the author has in both instances included an 'Other (please specify)' field for options which are not in the categories provided.

Free choice

Here, the respondent can choose one or more option from a list, which should include all alternatives, and not be mutually exclusive.

Free choice and multiple choice collect nominal data, in other words, it is possible to assign a number to the respondent's choice.

Ranked

Here, the respondent is required to list alternatives according to an order, for example the desirable attributes of a holiday, ranked according to importance. The data collected are described as ordinal.

Rating

This type of answer rates items according to a scale, as in the following example ('Is your TQM Programme successful? A self-assessment tool for managers'):

It is one of the most commonly used methods in management research, and is particularly useful for measuring affective issues such as attitude.

Rating scales are particularly useful for dealing with affective measures - i.e. those that relate to a person's perceptions, beliefs, feelings, attitudes and values towards themselves, individuals or organizations. Their use in management research is common. Because they measure interval data, which is a higher order than nominal or ordinal data, it is possible to apply a wider range of statistical procedures.

There are a number of different ways of measuring affective issues, of which some of the most commonly used in questionnaires are:

Numerical rating scales Respondent is asked to select a particular numbered response on a scale with either all points 'anchored' (i.e. strongly agree - strongly disagee) or just the beginning and end points
Likert rating scales Use a numbered scale as above but provide respondent with statements with which to agree/disagree
Semantic differential Use a numbered scale that is measured by bipolar adjectives
Thurstone scales Assign ratings to a series of statements which respondents are asked to check those which apply to them
Guttman scales A similar approach to Thurstone, but here the checking of statements can be cross-checked to see whether or not attitudes are uni-dimensional

An interesting example of the use of rating scales is the use of the Likert scale in research on library use and library anxiety as developed by Sharon L. Bostick and used in subsequent research by Qun G. Jiao, Anthony J. Onwuegbuzie in their research on the relationship between anxiety about library use and social interdependence, as published in a number of articles, for example 'Dimensions of library anxiety and social interdependence: implications for library services', in Library Review
Volume 51 Number 2 2002

An interesting example of use of the Likert scale
You are being asked to respond to statements concerning your feelings about college and university libraries. Please mark the number which most closely matches your feelings about the statement. The number ranges from:
1 = Strongly Disagree   2 = Disagree   3 = Undecided   4 = Agree   5 = Strongly Agree
  1. I'm embarrassed that I don't know how to use the library.
  2. A lot of the university is confusing to me.
  3. The librarians are unapproachable
  4. The reference librarians are unhelpful

1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5

Numerical rating scales are used in 'Libraries and desktop storage options: results of a Web-based survey', as in the following example from their questionnaire:

The choice of question format will be determined by the type of data you seek to elicit, but also by the level of sophistication of the statistical analysis which you hope to perform - some formats elicit more exact data than others, as we shall explore in the next section.

Alternatives

Many of the points raised earlier about writing questions apply about writing answers:

  • ensure that all items listed are strictly necessary to the question objectives
  • ensure that all items are clearly worded, and mean the same thing to the respondent as they do to you
  • avoid items which make use of jargon, technical terms or words which are outside the sample's knowledge range
  • the respondent should not have to look outside the survey - in other words, try and recall forgotten events

In addition:

  • response items should be exhaustive and mutually exclusive
  • do not provide too many alternatives - 15 is considered a maximum by some
  • decide what order to present the alternatives - this could be random, alphabetical, or with an order dictated by the research objective
  • if including very specific questions (e.g. How many cigarettes do you smoke a day? 1-5 etc.) preface it by a screening question (Do you smoke? Yes/no)
  • it may be appropriate in some instances to give range variables (0-5, 6-10) etc. in stead of precise measures, for example if asking a question such as 'How many emails do you receive in a day' when precise recall may be a problem
  • when using the same question type (for example, multiple choice, Likert scale etc.) use the same response format, as in the examples above.

1Peterson, R.A. (2000) Constructing Effective Questionnaires, Sage, Thousand Oaks, CA

The analysis and writing up of questionnaires