Qualitative Transparency Deliberations

[Rudra Sil University of Pennsylvania]

It is not surprising that many of the concerns expressed here are from scholars in comparative politics who invest in qualitative research in quite varied regions and political environments. In fact, the latest Comparative Politics Newsletter (see link below) offers essays that underscore how the imagined benefits of a uniform cross-subfield, cross-method DA-RT policy would not come close to balancing out the very real costs that would be inflicted upon qualitative scholars in comparative politics. The latter would have to adapt their modus operandi to an extent far beyond what, say, quantitative researchers studying US voting behavior would have to do to be in full compliance with DA-RT. And to what end? As others have already pointed out, DA-RT seems to be a solution in search of a problem. Those few who are inclined to violate norms of ethical, responsible, transparent research will find ways to do so even with DA-RT procedures in place. For the rest of us, as the experience of Economics suggests, even with data-sharing and replicability rules in place, most of the field will likely not end up actually devoting the time and energy to replicate each and every finding presented across the field of comparative politics.

In the meantime, the vast majority of us already aspire to construct arguments based on evidence that we hope peer reviewers and critical readers will find to be sufficiently reliable. The marginal utility that DA-RT may bring for some types of research communities simply does not justify the hugely disproportionate costs (in terms of time, money and energy) that many other types of research communities - and particularly qualitatively oriented comparativists studying non-Western societies - will have to bear, costs that will seem prohibitive to younger scholars at institutions with fewer resources. The result? Initially, senior researchers will likely do what Professor Keck has said she would do: simply submit research to journals other than the ones committed to DA-RT and JETS. And, more of us will begin to devote more of our attention to these other journals than to those "flagship" journals of the discipline that apparently see DA-RT as necessary for their reputations and rankings. Looking ahead, the more serious and unfortunate implications for the field will be a significant narrowing of the range of approaches, substantive questions, and countries/locales to be explored by the next generation of scholars, for whom the path to jobs and tenure would be through publishing the kind of work that can be more easily made to conform to DA-RT policies. That would be a truly depressing and dysfunctional outcome for comparative politics and for the discipline writ large. The questionable benefits of DA-RT simply do not justify taking even the smallest risk that any of these scenarios would come to pass.

http://comparativenewsletter.com/files/ ... ng2016.pdf

[Stephanie McNulty Franklin and Marshall College]

This post raises important points and I agree!

[Karen Alter Northwestern University]

When I publish in law journals, which often use active citation, I do so with a law professor co-author. I will only publish in law journals with this co-author, because he has access to many assistants who do the grunt work. By grunt work, I mean that my co-author’s assistants follow our pincites, adding the sentences when journals want the author to provide the quote (since not all journals do want this replication of the text from which the inference is drawn), double checking that the links are live, creating PDF copies of links that might go dead, and creating a Dropbox file with the original copies of every cited source for the copy-editors to consult, etc. For the work-study law students, this sort of work is pre-professional practice. I can tell you from experience, most of our political science graduate assistants-- and all of our undergraduate assistants-- won't be able to replicate what law assistants are doing for law faculty. Law students are also supported with professional and highly skilled law librarians, who are on-sight sleuths that will find and scan esoteric citations. And law faculty have the back up of professional assistants who can answer a request during vacations, exam times and when a quick answer is needed for publication.

I am based at a wealthy research institution, and I have access to law librarians who can find obscure sources (such as the legal document that someone handed me during research, but that must be traced to an accessible source). The challenges will be much harder for faculty who are not at deep pocket institutions with lots of library and student support.

We will thus be introducing an additional publication inequality-- beyond time to read and write, we are adding the need for research assistants to prepare the materials for publication. Those who lack the time and resources will publish in journals that do not require DA-RT-- so we will reinforce a publication/prestige divide that is not based on the quality of the mind or the work.

[Alan Jacobs University of British Columbia]

Thank you, Karen Alter, for these very useful comments on the costliness of a form of data-sharing. It's especially helpful to have this level of concrete detail from someone who has engaged in extensive data-sharing practices. It would be great to hear from others who have had experience with providing data access (beyond traditional quotations/citations). What specific tasks/efforts were required to share the data? How great were the costs? How were these costs paid or managed? What were the benefits of sharing these data?

We might also think about forms of data access that are less extensive than law-journal citations but that might nonetheless enhance the interpretability and evaluation of findings. We have posted a query out about this issue here, and there's been some discussion of this issue here.

[Guest]

As a junior faculty member at a public university with little to no research support (i.e. no research funding, no research leave, no graduate assistants, and only a small group of reference librarians), I completely agree with what Rudra and Karen have mentioned above. DA-RT stands to exacerbate the inequalities that already exist in the publishing process.

[Guest]

I agree with the Original Post. (Associate Prof at a non-R1)

[Guest]

As others have said, this definitely costs many of us far more than it costs R1 scholars.

DA-RT will be problematic for young scholars who collect their own quantitative data. I created a huge data set for my dissertation/subsequent research. I then took a job at a teaching school. That's what I wanted, but a 4-3 teaching load does limit research time. Had I been forced to publish my data early on, I wouldn't have been able to get more articles out of it before someone beat me to the punch. Scholars with research resources (I have none) and lower teaching loads would be far better positioned to use my data than I would be.