Substantive Dimensions of the Deliberations

Forum rules

We encourage contributors to the Discussion Board to publicly identify by registering and logging in prior to posting. However, if you prefer, you may post anonymously (i.e. without having your post be attributed to you) by posting without logging in. Anonymous posts will display only after a delay to allow for administrator review. Contributors agree to the QTD Terms of Use.

To participate, you may either post a contribution to an existing discussion by selecting the thread for that topic (and then click on "Post Reply") or start a new thread by clicking on "New Topic" below.

The transition to Stage 2 of the deliberations is currently underway but will take some time to complete. In the meantime, we very much welcome additional contributions to the existing threads in this forum.

For instructions on how to follow a discussion thread by email, click here.


Who are the gatekeepers? Editors or reviewers?

PostThu Apr 07, 2016 12:25 pm

As has been raised in prior discussions, JETS gives journal editors great discretion in deciding whether, and how, to implement transparency and data access requirements. I think we should take a step back and debate whether this role should be assigned to journal editors. Or should other individuals in the profession--like reviewers--serve as gatekeepers and DART-enforcers?

Journal editors do not necessarily have personal experience with particular research methods, not can they be expected to have expert knowledge of all research settings. Reviewers, on the other hand, have presumably done similar types of research, and they should be familiar with the standards and best practices of different subfields and epistemic communities. They should also be much more familiar with the case under study. So why not leave the question of transparency and data access to the reviewers, not editors? When an article goes out for review, either the main text or appendices would address transparency and data access. If the author does not plan to make the data publicly available, he/she would explain and justify that decision. In their reviews, the reviewers could agree or disagree with the author's rationale. This could be further discussed back-and-forth in R&R if necessary, until the reviewers, the editor, and the author settle on a course of action.

Finally, when data will be shared, it would seem to make a good deal of sense for the full data to be shared with the reviewers, not submitted to the journal only after an acceptance. The reviewers are best positioned to evaluate the quality of the data and the reasonableness of the author's interpretations. If we want transparency to lead to increased rigor and accountability, sharing data BEFORE publication seems crucial, because that is when a manuscript faces its most intense evaluation from other experts.

Post Reply


Re: Who are the gatekeepers? Editors or reviewers?

PostSun Apr 10, 2016 8:14 pm

I agree that editors are not necessarily in the best position to be the gatekeepers, but I'm not sure reviewers are either. off top of my head: this scenario could lead to 1) lead to massive variation in how otherwise similar articles are treated even by same journal depending on who the reviewers are and their particular beliefs and expectations; 2) lead to a potential stalemate when reviewers differ and the author has to answer to many different audiences; 3) involve a coordination problem among reviewers, who for a single submission will differ in their views of the article and what they expect in transparency.

Post Reply


Re: Who are the gatekeepers? Editors or reviewers?

PostSat Apr 16, 2016 10:14 am

Generally, I subscribe to what could be called epistemological subsidarity, that is, research transparency should be assessed as the most local level possible. Practically, this would mean the scholars most intimately familiar with the subject matter and methodology of the reviewed article. So, this would mean shifting the responsibility of assessing research transparency from editors to reviewers which is what we do currently anyway.
I would like to make one suggestion to strengthen this practice and propose one caveat.
The practice could be strengthened by adding research transparency to the check-list of criteria used to evaluate a submission. Many journals currently ask us to evaluate contributions for their originality, rigor, breadth etc. Why not add research transparency as another criteria and have use rate it. It might encourage reviewers to pay more or more explicit attention to it.
Let suggest a caveat to the broader epistemological subsidarity principle. What if journal editors would require that each submitted article have at least 50% of citations that contain actual page numbers. Page numbers arguably are the less onerous, least controversial research transparency benchmark, yet all too frequently, it is rarely met. Many of us probably have been frustrated by the drive by literature reviews that reduce an entire theoretical debate to five pageless citations or by empirical implications that are supported again by pageless citations. At the risk of over-generalizing, this benchmark might pose a slightly more formidable challenge for qualitative than qualitative submissions.

Post Reply