Know your enemy - what the peer review process entails

What is the peer review process?
What are the stages of the peer review process?
What are the criteria by which a paper is judged?

What is the peer review process?

The internet has in theory made it possible for anyone to publish anything at any time. What gives being published in a journal especial credibility is that other experts have read the work and deemed it acceptable. The peer review process is therefore what gives the journal in which you have chosen to place your article, and your paper if you have done the necessary revisions, a quality control stamp.

To enable the peer review process to be as objective as possible, both the author, and reviewers, will remain anonymous to one another, although this may not be possible in the case of a very small field.

The anonymity is the reason for referring to the process as blind. An article will be sent to at least two reviewers (double blind), in some cases three (triple blind). The reviewers will be experts in the same field who have probably been through the same process themselves. They will read the article very carefully, making notes, and should make detailed and constructive comments.

At all times a good editor will remain in control of the process, initially deciding if the paper is sufficiently in tune with the journal's objectives to warrant review, then selecting the reviewers and corresponding with them, ensuring that their advice is constructive and not contradictory, and seeing the paper through several revision stages.

More about the peer review process can be found by clicking on the following links:

How to Write Publishable Papers by Abby Day
The peer review process

What are the stages of the peer review process?

  1. Once you have sent in your paper, you should expect to receive an acknowledgement within a couple of weeks to say that the paper has been received. This does not mean that it has been read.
  2. The editor will read the paper and will at this stage be concerned:
    • to establish whether it meets the journal's overall aims and objectives
    • to gain a broad overview of the paper to establish whether it is of sufficient quality to be sent for peer review
  3. If the paper meets the journal's overall objectives and is of a sufficient quality, then it will be either sent to peer review directly or be returned to the author with a request for some revisions.
  4. The reviewers will read the paper in much greater detail than the editor and take detailed notes. They will return their comments, along with their recommendation to reject, revise, or (rarely) publish as is to the editor.
  5. A good editor will at this stage read the reviews, checking them for any contradictions (in which case a third review may be sought), ensuring that they are clear and either filtering out any negative or critical comments or reminding the author that they are not personal!
  6. The editor will notify the author of the decision, including any revisions required. (This whole process may take several weeks or even months; it is reasonable to enquire if you have not heard back within 12 weeks.)
  7. The author will then need to decide if he/she wants to revise the paper. The revisions may be relatively minor in nature, but may be quite substantial entailing a radical reappraisal of the research strategy. (See How to revise your paper for more about this stage.)
  8. Once the paper has been revised and returned to the editor, it will be sent for review again to check that the points raised by the first review have been covered, or that the reasons given for not covering them are acceptable. At this stage, the paper may be accepted for publication, rejected or further revisions requested.

The following graphic illustrates the stages illustrates the process, together with some statistics, for Emerald's journal International Journal of Service Industry Management (kindly supplied by the editor, Robert Johnston). Click on the graphic to see an animated and generalised view of the review process.

What are the criteria by which the paper will be judged?

This will vary from journal to journal (e.g. a journal with a strong research focus will put more emphasis on research methodology), as will the amount of guidance an editor gives his or her reviewers. Some editors just ask for a recommendation and comments, whilst others provide a structure in the form of set questions, e.g. 'Is the content of the article within the editorial aims and scope of the journal?' The following criteria, taken from a random collection of Emerald journals, should therefore be taken as guidance only and not as a checklist.

  • Is the subject appropriate to the editorial aims and scope of the journal?
  • Originality: does the article say something original, does it add to the body of knowledge, etc.? If a case study, is this its first use?
  • Research methodology: most journals are concerned about this, as would be expected for an academic publisher. Is the research design, methodology, theoretical approach, critical review, etc. sound? Are the results well presented, do they correlate to the theory, and have they been correctly interpreted? Is the analysis sufficiently rigourous?
  • Is the paper set in the context of the wider literature, are there sufficient relevant citings, are these well referenced and are other people's views credited?
  • Is the paper accurate, is any information missing or wrong?
  • Is the structure logical, is the sequence of the material appropriate, is there a good introduction and are the summary and conclusions adequate?
  • Does the title of the article accurately reflect its content?
  • How useful would the article be to a practitioner, is it a useful example of 'good practice'? Could the study be replicated in other situations?
  • Is the material clearly presented, readable? Are graphs and tables used to good effect? Is the level of detail appropriate? Is the use of terminology appropriate to the readership?
  • Is the perspective appropriate for an international audience?
  • Questions of format: are the abstract, keywords etc. appropriate?
  • Is it an appropriate length (note: many journals will stipulate length requirements in their Notes for Editors)?
Tips for getting your paper right in the first place