A Review of the Review Process

Greetings! We hope this post finds you well and that you have enjoyed the holiday seasons. In Sweden, it is (still) fairly common to send postcards for Christmas to your nearest and dearest friends and family. I always appreciate the nicely decorated cards and the warm wishes you get from people spanning from your mother to your old primary school teacher.

This year, a very special message reached my Ph.D. student colleague Yixuan and me in the days between Christmas and New Year’s Eve. Our paper, Pole-Transition Control of Variable-Pole Machines Using Harmonic-Plane Decomposition, was approved for publication in IEEE Transactions on Industrial Electronics and the associate editor and reviewers congratulated us for having successfully passed the scientific needle’s eye known as the scientific review process. %Indeed, this marked a cornerstone in our ongoing research.%

At this point, you may wonder what grants this process such an uncanny metaphoric epithet. This blog will try to give a glimpse of what Yixuan and I experienced in the past year and describe the scientific review process from a practical view. In other words, we’ll set out to review a review. Feel that meta-discussion!

All of us have personal experiences with reviews. As a matter of fact, we were reviewed every time we got an exam back at school. And most of us have gotten stuck in front of a product, film, or book review on Youtube or in a newspaper. The scientific review has similarities with these types of reviews, but also some distinguishing differences.

Let’s start with the similarities. Indeed, one part of the review is a score (humans love numbers – see the book Sifferdjur by prof. Micael Dahlén), or in other words, how well you have performed. Some journals or conferences hand out a grade based on the average of the reviewers’ opinions regarding e.g. technical content, novelty, and language. Others, like the review we just went through, give a summarizing statement on the opinion of the reviewers and a pass/fail type of judgment. Either you get a pass (accepted) or fail (declined), or you are informed that the paper has to go through a minor or major review to be accepted. In our case, we went through five agonizing rounds of revisions before the manuscript was accepted after more than a year.

That brings us to a major difference to, for example, a film review. Whereas the film review is carried out on the final product, a scientific paper review happens before the potential publication. Moreover, the reviewers are incognito to ensure their integrity. The reason for these differences is dumbfounding simple and reasonable. In the world of science, quality is less of a taste thing and more about complying with specific standards on how to conduct experiments, display results, and analyze their validity. In order to cross these bars, the reviewers’ jobs are to find flaws in your argumentation and iron out any uncertainties that may be caused by the way you present the results. The review is in a way also a coarse filter whose mesh density each journal can decide on, on their own. Perhaps, the journal has a very defined theme and a high citation score, which puts even higher demands on the scope, standard, and novelty of your paper. For those of you unfamiliar with the term citation score, you can think of it as a reputation rank. Being published in the journal with the highest citation score is for the researcher the equivalent of performing at Carnegie Hall for a violinist.

A third interesting aspect of a scientific review is the correspondence of ideas and arguments that go back and forth between the reviewers and the manuscript authors. Questions from the reviewers are not always answered directly by making a change in the manuscript. Rather, longer answers, arguments, opinions, and explanations may be attached in what is called a review letter.  This is a sophisticated way of settling disputes, although I would have liked to try the French combat way at some point.

An example of such an argument in our case was the presence of a small but evil ~100 Hz harmonic on the torque measurement. One reviewer highlighted that it was visible in one of our plots and shouldn’t be there if our proposed control was correct. However, Yixuan and I noticed that the ripple was caused by a resonance frequency in the mechanical structure of the test bench (it is very big and heavy). The first thing we did to try our hypothesis was to jump outside of our cell. When we did that, there seemed to be an amplification of the 100 Hz signal of the vibration we were causing. Then, we moved on to use a commercial drive and motor on our bench and we still observed the same ripple, independent of speed. So we conveyed these measurements to the reviewer and could settle this issue. After the review rounds, I was performing locked rotor tests at varying frequencies on our machine and it was peculiar to hear and feel the humming due to the resonance when I passed 100 Hz. At least, I know where it comes from. Otherwise, I would probably have become a bit bewildered.

In the end, did our paper get better from all the review rounds? The answer is undeniably yes, although the incremental improvements became smaller the further into the process we got. The system appears to work. Kudos to the reviewers!