Who are the gatekeepers? Editors or reviewers?
Posted: Thu Apr 07, 2016 12:25 pm
As has been raised in prior discussions, JETS gives journal editors great discretion in deciding whether, and how, to implement transparency and data access requirements. I think we should take a step back and debate whether this role should be assigned to journal editors. Or should other individuals in the profession--like reviewers--serve as gatekeepers and DART-enforcers?
Journal editors do not necessarily have personal experience with particular research methods, not can they be expected to have expert knowledge of all research settings. Reviewers, on the other hand, have presumably done similar types of research, and they should be familiar with the standards and best practices of different subfields and epistemic communities. They should also be much more familiar with the case under study. So why not leave the question of transparency and data access to the reviewers, not editors? When an article goes out for review, either the main text or appendices would address transparency and data access. If the author does not plan to make the data publicly available, he/she would explain and justify that decision. In their reviews, the reviewers could agree or disagree with the author's rationale. This could be further discussed back-and-forth in R&R if necessary, until the reviewers, the editor, and the author settle on a course of action.
Finally, when data will be shared, it would seem to make a good deal of sense for the full data to be shared with the reviewers, not submitted to the journal only after an acceptance. The reviewers are best positioned to evaluate the quality of the data and the reasonableness of the author's interpretations. If we want transparency to lead to increased rigor and accountability, sharing data BEFORE publication seems crucial, because that is when a manuscript faces its most intense evaluation from other experts.
Journal editors do not necessarily have personal experience with particular research methods, not can they be expected to have expert knowledge of all research settings. Reviewers, on the other hand, have presumably done similar types of research, and they should be familiar with the standards and best practices of different subfields and epistemic communities. They should also be much more familiar with the case under study. So why not leave the question of transparency and data access to the reviewers, not editors? When an article goes out for review, either the main text or appendices would address transparency and data access. If the author does not plan to make the data publicly available, he/she would explain and justify that decision. In their reviews, the reviewers could agree or disagree with the author's rationale. This could be further discussed back-and-forth in R&R if necessary, until the reviewers, the editor, and the author settle on a course of action.
Finally, when data will be shared, it would seem to make a good deal of sense for the full data to be shared with the reviewers, not submitted to the journal only after an acceptance. The reviewers are best positioned to evaluate the quality of the data and the reasonableness of the author's interpretations. If we want transparency to lead to increased rigor and accountability, sharing data BEFORE publication seems crucial, because that is when a manuscript faces its most intense evaluation from other experts.