You are here: Home / How to ... / Peer-review content

Peer-review content

A brief description of how to peer-review works in this course.

Peer review in two words: "Constructive Criticism". The goal of peer-review is to help the author in the next step of improving her content by pointing out problems and providing potential solutions.

The process assumes that any written content can always be improved. When content is new, it can be improved a lot. After several rounds of improvements, the improvements get smaller.

Content can be improved on many levels and you will develop an awareness of this by engaging with peer review yourself. The most important ones are:

  • Form: goes much beyond presentation and writing style; it also includes appropriate references to earlier and related work, proper citations, rigorous statistical tests, the use of appropriate tools and good quality controls.
  • Ideas: how important and how reliable is the main idea; what assumptions does it make and how rigorous are the conclusions; is clear would have to happen for changing the conclusions.

As is clear from these lists, both can be broken into many sub-components. The criteria that are used to evaluate scientific texts can be very complex. For texts with significant problems it is often only possible to point out the first few larger problems without using more words than the text itself. A full analysis of all problems associated with some texts is often not possible in the given time.

Thus we use 'focused peer review' here: As a peer reviewer, you will be asked to look for an example of a particular type of problem and propose a solution that might help the author address that problem. This can give you as a peer reviewer the room to focus on actual progress in particular areas instead of attempting to cover everything at the same depth (often resulting in evenly distributed shallow reviews). Your task in this course is to learn how to engage with this process. It can be daunting, but it is also rewarding, because as a result the quality of work improves substantially. 

What reviews are not: the type of stuff you find in many print or online magazines, where the "reviewers" rarely find any serious problems with products and basically try to sell you what they are "reviewing". These reviews are infomercials without much substance. Reviews at that level are bordering worthless for research: they don't help improve anything. 

 

Evaluation criteria for review and editing

All peer review and editing evaluates these categories as appropriate:

  1. Quality of questions asked
  2. Importance of conclusions
  3. Innovative ideas
  4. Links to existing research literature 
  5. Quality of biological models, data and methods 
  6. Quality of technical, computational, statistical and math aspects 
  7. Clarity and elegance of writing 
  8. Educational quality 
  9. Broader impact

 

How to write a good review in a nutshell 

Good reviews are often written in rounds, especially if the type of content is new. Consider the following rounds:

  1. Review criteria. What is the type of content you are reviewing? What does it need to deliver? You need to know the criteria before you read. 
  2. Read the content with the review criteria in mind. Make marks or notes to help you remember what you think while reading. Don't try to order at this stage.
  3. Order problems you saw after reading. Here you evaluate what is worth telling the authors. Check the facts where needed to see if some of your guesses can be substantiated.
  4. Read the review criteria again to see gaps you hadn't noticed before. 
  5. Write your review. Stop before your review becomes longer than what is being reviewed. In such cases, learn to focus on the most important points. Finalize your review by checking it critically. Before submission read your review one more time, now imagining you had written the content and you would receive this review. Make sure your criticism is constructive, specific, professional and such that you would like to receive such a review. Then submit.
If you have to review content that is written so badly it would need a super long review, then pick a few major points that you think will help the authors. 
 

Practically, how to get started with a review

If you don't take notes on paper or scribble on printouts, then you may want to start setting up your review document on the course website right away:

  1. Go to the "Feedback" folder of your course and in there to the folder of the person you want to provide feedback to.
  2. In that folder, go to the menu: "Add new ..." then select "Focused Feedback".
  3. Fill in the initial details like what you are reviewing etc. 
  4. Keep the review page open during reading, as it contains the list of review criteria. You can take notes, but don't forget to save repeatedly.
  5. Use only the categories of feedback that apply to the given content. All others just leave empty.
  6. Before you submit your review, make sure that you would like to receive this review as a help for improving your content: Professional, friendly, constructive criticism, without down-playing real problems in the analysis and without any personal attacks on authors. 
  7. Submit as explained in "How to ... Submit by a deadline".

 

Hone your review writing skills as the course develops!

If you have any questions about how to write reviews, do not hesitate to ask the course organizer.