Substantive Dimensions of the Deliberations

Forum rules

We encourage contributors to the Discussion Board to publicly identify by registering and logging in prior to posting. However, if you prefer, you may post anonymously (i.e. without having your post be attributed to you) by posting without logging in. Anonymous posts will display only after a delay to allow for administrator review. Contributors agree to the QTD Terms of Use.

Instructions
To participate, you may either post a contribution to an existing discussion by selecting the thread for that topic (and then click on "Post Reply") or start a new thread by clicking on "New Topic" below.

The transition to Stage 2 of the deliberations is currently underway but will take some time to complete. In the meantime, we very much welcome additional contributions to the existing threads in this forum.

For instructions on how to follow a discussion thread by email, click here.

Daragh Grant
Harvard University
Posts: 1
Joined: Thu Apr 07, 2016 5:50 pm

A pointless DA-RT

PostFri Apr 08, 2016 12:39 am

Previous contributors to this forum have noted the practical and ethical problems with the proposal to apply the DA-RT and JET statements to the full spectrum of qualitative political science research. There are obvious ethical problems with requiring ethnographic researchers to provide their field notes for review, to say nothing of the fact that this would place a costly and onerous demand on researchers while furnishing editors and reviewers with little additional useful information for weighing the quality of a piece of research. As Nancy Hirschmann has noted, DA-RT is redundant for scholarship in the field of political theory. In my own area of specialization, archival research, it seems unnecessary to furnish copies of publicly available material to a journal during the review process (a point Hirschmann also raised). Even if this were not a redundant measure, however, the assumption that it would help an editor to assess the quality of a piece of research is tenuous at best. First, it would place a most unrealistic burden on journal editors to expect them to decipher the poor penmanship of a variety of authors, whether of hand-written documents or of marginalia, across hundreds of pages of material. Even if they had the patience to engage in this fact-checking, it is not clear that they would then be in a position of adjudicate the quality of the inferences that a historical researcher had advanced on the basis of cited materials alone. Without reading the hundreds or thousands of documents that form the context for these interpretations, the editor is condemned to the status of a dilettante. It seems much more sensible to maintain some confidence in the system of peer-review, to take seriously the idea that experts in a particular field-site or archive are best placed to assess the quality of a scholar’s work. If there is a concern that political science journals have difficulty in finding those with the expertise necessary to adjudicate the quality of qualitative research (in the absence of something like DA-RT), it would make more sense to embrace a more capacious and interdisciplinary understanding of who our peers are than to assume that all one needs to do to become an expert is to follow the footnotes in a particular essay.

There is a more fundamental question here: What is the problem in qualitative research that DA-RT is supposed to solve? Even if one were to grant that the discipline is awash with willful con-artists trying to pull the wool over everybody else’s eyes, DA-RT would only offer the veneer of rigor. People could still cherry pick their evidence and gloss it in tendentious ways. This has always been true of scholarly research, and always will be. Moreover, there is no easy way for the academy to resolve the problem of outright deception (any more than there is a way for us to root out intellectual dishonesty). What there is—all that there ever has been—is a commitment to the idea that just as we seek to challenge the entrenched orthodoxies of earlier generations, so too are our own intellectual contributions submitted for the consideration of those scholars who share our areas of expertise, whether today or at some point in the future. Asserting that one can offer such a contribution to knowledge depends, of course, on a kind of intellectual arrogance. But, when done well, this arrogance should be tempered by the humility of knowing that if one is saying anything even remotely interesting, then that thing will itself eventually be the subject of revision. The central problem with DA-RT might be that it lacks this kind of humility. It rests on the assumption that we can uncover the truth and assumes that if we can pile evidence upon evidence that will allow us to have more confidence in such a finding. Lisa Wedeen has already highlighted the narrowness of this scholarly imaginary. I would only add that any claim to truth of which we could all be convinced through the alchemy of DA-RT would probably be sufficiently obvious as to not merit the time and effort of proving it in the first place.

Post Reply

Erica Simmons
University of Wisconsin–Madison
Posts: 5
Joined: Mon Apr 25, 2016 1:23 pm

Re: A pointless DA-RT

PostWed Apr 27, 2016 3:19 pm

You make a number of excellent points, Daragh. My own understanding of one of the motivations for DA-RT (though I may be wrong) was a desire to increase the stature of qualitative methods in the discipline through addressing questions of transparency and rigor. As you point out the current proposal is unlikely to actually get us closer to either goal.

Post Reply

Guest

Re: A pointless DA-RT

PostTue May 17, 2016 2:30 am

very nicely put!

Catherine Boone
LSE

Post Reply

Tim Buthe
HfP/Technical Univ of Munich & Duke University
Posts: 32
Joined: Fri Feb 26, 2016 11:39 pm

Re: A pointless DA-RT

PostTue May 17, 2016 10:45 am

Thank you, Daragh, Erica, and Catherine, for your posts. Some of the issues you raise have actually been addressed in the January 2014 symposium in PS, where both Arthur Lupia and Colin Elman in their introduction and Colin Elman and Diana Kapiszewski in their article on "Data Access and Research Transparency in the Qualitative Tradition" note several reasons for why they had developed the DART and JETS proposals.

Even more pertinent to the current discussion, the Spring 2015 QMMR symposium on "Transparency in Qualitative and Multi-Method Research" featured an extended, more differentiated discussion of the pros and cons of greater research transparency for a variety of different qualitative and multi-methods research--which we defined broadly as not per se statistical or experimental ways of gathering and analyzing empirical information (or, if you prefer: "generating data," processing it, and drawing inferences). The introduction and conclusion to the symposium in particular discuss many reasons why greater research transparency may be desirable (most of which has nothing to do with detecting or safeguarding against fabrication of data or falsification of findings). And Marc Trachtenberg notes that much archival research falls far short of what would be required to fully understand what the author did, use it to train graduate students, and to generally facilitate follow-up research. Note that the QMMR symposium did not take any particular position on DART, but rather focused on a set of questions that also have motivated the proposal for these deliberations:

(A) What does research transparency mean for you/for the kind of research you do? (Please specify the "kind of research you do.")

(B) About which elements of your research do you try to be particularly transparent (or, as Craig Parsons has suggested, particularly "explicit")? What kind of transparency do you most value in the work you read?

(C) What do you see as the most important costs, risks, and other specific concerns that keep you and other scholars who do the kind of work you do from being as transparent as you might otherwise wish to be?

(D) Are there specific tools or practices that you have found to be particularly useful (in your own work or the work of others) to achieve transparency without undue costs or risks?

Maybe for some of us the answer to all of these questions is ultimately "None" but we can't arrive at that conclusion with addressing these questions explicitly and with specific reference to the kind of work you do (or are concerned with). I would like to encourage all of you to try to address them. Many thanks in advance.

Post Reply