Best Practice—Writing a Peer-Review Report
The importance of peer review has, if anything, increased in recent times. The enthrallment of current academia with “objective” quantitative measures in the processes of selection, promotion and evaluation of academic performance has put a premium on publication in “peer-reviewed” journals. Instead of a faculty reading carefully the work and making up its own mind as to its quality, they will outsource such to two anonymous peer reviewers. Also, in the face of the avalanche of self-publication in outlets such as SSRN (valuable in and of itself) and the like, peer review may help the discerning reader navigate these channels, thereby providing some guarantee of excellence.
Yet this importance is often not matched by the practice of peer review. The rate of refusal to peer review is as high as 50 percent—oftentimes by authors who themselves have published in, and benefited from, peer-reviewed journals. Authors who publish in I.CON and EJIL undertake to peer review for our journals, an undertaking not always honored. Of course, there is only so much peer reviewing that one can do and we understand when we receive a request to beg off with a promise to do it on some other occasion.
Then there is the problem of tardiness. Four to six weeks is a reasonable time to expect a peer-review report to come in. Frequently, to our and our authors’ frustration it can be as long as twenty-four weeks, after a slew of “gentle” and somewhat less gentle reminders.
And then there is the question of the quality of the review, oftentimes perfunctory and hardly helpful.
So here are some guidelines to this act of high academic citizenship.
There is a common misconception that the most important thing the Editors want is a judgment: publish or do not publish. Of course we are interested in that final judgment. We rarely, if ever, will publish an article where both peer reviewers have recommended rejection. But it should be remembered that all journals engage in initial screening, picking out the articles that will be sent to peer review. Articles may be screened out for a variety of reasons, such as subject matter interest, pipeline management (too many articles on the same topic) but also, of course, for quality. The editorial team will not send out to peer review articles that they expect will be rejected—that is not a clever way to manage this scarce resource. The result is that articles sent to peer review are those considered potentially publishable which, in turn, means that the most common outcome of the process will be a double “revise and resubmit” (R&R), in borderline cases one R&R and one rejection (R) and rarely two Rs.
Even when the recommendation is a straight A (acceptance, excuse the pun) good peer reviewing will still provide the Editors with a reasonably detailed evaluation of the piece, explaining their view of its quality and the original contribution it makes and, operating on the principle that there is nothing that is so good that it cannot be improved, providing abundant suggestions and recommendations to the author. The evaluation for the Editors is particularly valuable in those cases where there is a divergence of opinion among the peer reviewers. Explaining the originality and importance of the articles is key in a positive peer review because the Editors, knowledgeable as they may be, cannot be masters of all specialties in the general field and, whereas they can discern good or bad writing, powerful or weak reasoning and the like, will often be less confident in assessing originality and importance in areas not their own.
When the evaluation is negative it is even more imperative to invest in the rejection report, both for the benefit of the Editors as well as for the benefit of the author. If, for example, the claim is that the piece is not original, it should be accompanied by a couple of references to work that substantiates that opinion. If the negative judgment goes to quality rather than, or in addition to originality, for instance if it is poorly reasoned, contains lapses in argument and the like, once again this should be spelled out in the report.
The single-paragraph peer-review report which one sees from time to time—“This is an unoriginal, poorly reasoned and badly written article: Reject!”— is singularly unhelpful as well as lacking credibility. It is unlikely that the Editors would send out such a piece to peer review, and imagine yourself, as an author, receiving such an evaluation.
Here are some pitfalls in the judgmental function of peer reviewing.
One of the most common pitfalls is the confusion between “I don’t agree, the author is wrong” and “this is a bad article.” I will grant you that the line between the two can be fuzzy, but you grant me that there is, nonetheless, a difference between these two categories of judgment. And since we always seek specialists in the field covered by the article submitted, this inadvertent danger is enhanced since the peer reviewer has a stake in the field, in positions taken and the like. The only remedy is awareness of this distinction and self-awareness in the process of peer reviewing.
Another pitfall is ideological bias in the peer review. We notice this from time to time when we receive divergent evaluations of quality which track the ideological disposition of the peer reviewers. The remedy in this case is similar: awareness of the potential problem, self-awareness, and a sense of intellectual integrity.
The most common pitfall is … being perfunctory. A quick read, a quick report. A careful read, some reflection and a thoughtful non-hurried report is the Gold Standard.
For the reasons explained above Revise and Resubmit is the most common judgment and the report requires a little extra work. In the first place, the articulation of the defects that prevent a straight A should, somewhat paradoxically, be more detailed and expansive than those in a Rejection report, since here one wants to list not only those defects that are lethal to publication, but also non-lethal defects correction of which would improve and enhance the eventual published article.
Critical in an R&R report is the “road map” approach. The author should not only understand the weaknesses but should understand perfectly what needs to be done so that, if performed to an adequate standard, the revised article is likely to be accepted on resubmission. In drafting the R&R report, the peer reviewer should keep in mind this road map approach. Editors will often send the R&R report to the author, asking for reactions to the recommendations, an indication whether they are in agreement with the peer review and an indication of how they plan to revise the piece. The clearer the road map, the better this process unfolds.
Peer reviewers are typically asked whether they would be willing to review the revised piece. It always shocks me a bit when the No box is ticked. It is like a job left half done. Who better than the peer reviewer herself or himself to evaluate whether the revision is satisfactory? But, be that as it may, if the “not willing to review the revision” box is ticked, all the more important to have a very clear and elaborate road map so that the Editors themselves can better evaluate the revision.
Good peer reviewing probably requires somewhat more effort than preparing to comment on a paper at a conference. Yet it does not give the same exposure and, scandalously, is not taken into consideration in many places when evaluating the file of academics. These days people list in their CV their blog entries. I believe that listing peer reviewing should become standard practice. If deans and the profession place, as they do, so much weight on publication in peer-reviewed journals, surely it should mean something as regards reputation in the field that someone has been trusted to undertake peer reviews by those very journals. This thankless act of high academic citizenship should be valued.
JHHW
In this Issue
This issue opens with I•CON’s second Foreword, authored by Ayelet Shachar and Ran Hirschl, who engage with the spatial dimensions of public law and the resilience of state sovereignty, elaborating examples such as the immobility of cities and the mobility of people, and identifying a counter-narrative to the globalization account.
The issue continues with several essays authored by members of the journal’s Editorial Board, who offer reflections on the theme: “Thirty Years from the Fall of the Berlin Wall: The World after 1989.” Cora Chan reflects on the balance between Chinese authoritarianism and the Hong Kong liberal values, and on what needs to be done before China’s commitment to respect Hong Kong’s autonomy expires. Wen-Chen Chang focuses on the transitional and transnational dimensions of constitutionalism and argues that counter-democratic trends are a reprisal against the privileges of legal and judicial elites. Sujit Choudhry elaborates on how to extend the “post-sovereign” constitution-making project to contemporary self-determination challenges. Erika de Wet argues that democratic legitimacy is a relevant factor that has been considered in recognizing governments in Africa but that it has not evolved into a required condition for governmental recognition. José Manuel Díaz de Valdés and Sergio Verdugo examine how the constitutional strategies of the Bolivarian Alliance for the Peoples of Our America (ALBA) in Latin America have resorted to notions of political representation to weaken the constraining purpose of constitutionalism. Rosalind Dixon and David Landau explore how new autocrats or would-be authoritarians import foreign constitutional arrangements to serve their authoritarian agenda. James Fowkes and Michaela Hailbronner look at post-1989 Africa and Eastern Europe, arguing that a comparison between those regions suggests applying a decolonizing perspective not only to Africa, but also Eastern Europe. Finally, Tom Ginsburg reflects on the interdisciplinary turn in comparative constitutional studies, starting with the dominant academic paradigms of the Cold War era.
The next section features an I•CON Symposium on “Public Law and the New Populism.” The symposium opens with a contribution by Neil Walker, who claims that populists use a space between popular and authoritarian versions of constitutionalism and identify how the unstable balance among some constitutional goods expands the tension of modern constitutionalism. Paul Blokker focuses on East-Central Europe, and identifies four dimensions of populism, arguing that although populist constitutionalism rejects some of the problems associated with liberal democracy, it fails to achieve its democratic promise. Ming-Sung Kuo claims that the new populism suffers from a “pathology of instantaneous democracy” that harms democratic deliberation and representation. Tamar Hostovsky Brandes argues that the rise of populism may affect domestic courts’ willingness to use international law, harming the way international law can be used as a counter-populism tool. Bojan Bugarič reflects on the capacity of constitutional democracy to constrain majorities and examines how the populist governments of Hungary and Poland have dismantled critical democratic elements of those constitutional systems. John Morijn analyses the regulation on European Political Parties and European Political Foundations and discusses whether this regulation offers a comprehensive or limited response to illiberal European political forces. Finally, Robert Howse offers a counter-narrative against widespread academic criticisms of the new populism, presenting the case for disruptive democracy.
This issue also includes a special section entitled “The I•CONnect-Clough Center 2018 Global Review of Constitutional Law,” with two reports that summarize relevant constitutional developments in Colombia and Chile during the year 2018.
In the Review section, Chien-Chih Lin examines recent scholarship on Asian constitutionalism (Albert H.Y. Chen (ed.). Constitutionalism in Asia in the Early Twenty-First Century (2014) and Albert H.Y. Chen & Andrew Hardin (eds.), Constitutional Courts in Asia: A Comparative Perspective (2018)) with an eye to ideas of judicial dialogue. Finally, Arun Thiruvengadam, Diletta Tega and Tomasz Tadeusz Koncewicz engage in a review symposium on Bruce Ackerman’s new book Revolutionary Constitutions and its challenge to current debates on populism.
JHHW and GdeB
Comments