A pointless DA-RT
Posted: Fri Apr 08, 2016 12:39 am
Previous contributors to this forum have noted the practical and ethical problems with the proposal to apply the DA-RT and JET statements to the full spectrum of qualitative political science research. There are obvious ethical problems with requiring ethnographic researchers to provide their field notes for review, to say nothing of the fact that this would place a costly and onerous demand on researchers while furnishing editors and reviewers with little additional useful information for weighing the quality of a piece of research. As Nancy Hirschmann has noted, DA-RT is redundant for scholarship in the field of political theory. In my own area of specialization, archival research, it seems unnecessary to furnish copies of publicly available material to a journal during the review process (a point Hirschmann also raised). Even if this were not a redundant measure, however, the assumption that it would help an editor to assess the quality of a piece of research is tenuous at best. First, it would place a most unrealistic burden on journal editors to expect them to decipher the poor penmanship of a variety of authors, whether of hand-written documents or of marginalia, across hundreds of pages of material. Even if they had the patience to engage in this fact-checking, it is not clear that they would then be in a position of adjudicate the quality of the inferences that a historical researcher had advanced on the basis of cited materials alone. Without reading the hundreds or thousands of documents that form the context for these interpretations, the editor is condemned to the status of a dilettante. It seems much more sensible to maintain some confidence in the system of peer-review, to take seriously the idea that experts in a particular field-site or archive are best placed to assess the quality of a scholar’s work. If there is a concern that political science journals have difficulty in finding those with the expertise necessary to adjudicate the quality of qualitative research (in the absence of something like DA-RT), it would make more sense to embrace a more capacious and interdisciplinary understanding of who our peers are than to assume that all one needs to do to become an expert is to follow the footnotes in a particular essay.
There is a more fundamental question here: What is the problem in qualitative research that DA-RT is supposed to solve? Even if one were to grant that the discipline is awash with willful con-artists trying to pull the wool over everybody else’s eyes, DA-RT would only offer the veneer of rigor. People could still cherry pick their evidence and gloss it in tendentious ways. This has always been true of scholarly research, and always will be. Moreover, there is no easy way for the academy to resolve the problem of outright deception (any more than there is a way for us to root out intellectual dishonesty). What there is—all that there ever has been—is a commitment to the idea that just as we seek to challenge the entrenched orthodoxies of earlier generations, so too are our own intellectual contributions submitted for the consideration of those scholars who share our areas of expertise, whether today or at some point in the future. Asserting that one can offer such a contribution to knowledge depends, of course, on a kind of intellectual arrogance. But, when done well, this arrogance should be tempered by the humility of knowing that if one is saying anything even remotely interesting, then that thing will itself eventually be the subject of revision. The central problem with DA-RT might be that it lacks this kind of humility. It rests on the assumption that we can uncover the truth and assumes that if we can pile evidence upon evidence that will allow us to have more confidence in such a finding. Lisa Wedeen has already highlighted the narrowness of this scholarly imaginary. I would only add that any claim to truth of which we could all be convinced through the alchemy of DA-RT would probably be sufficiently obvious as to not merit the time and effort of proving it in the first place.
There is a more fundamental question here: What is the problem in qualitative research that DA-RT is supposed to solve? Even if one were to grant that the discipline is awash with willful con-artists trying to pull the wool over everybody else’s eyes, DA-RT would only offer the veneer of rigor. People could still cherry pick their evidence and gloss it in tendentious ways. This has always been true of scholarly research, and always will be. Moreover, there is no easy way for the academy to resolve the problem of outright deception (any more than there is a way for us to root out intellectual dishonesty). What there is—all that there ever has been—is a commitment to the idea that just as we seek to challenge the entrenched orthodoxies of earlier generations, so too are our own intellectual contributions submitted for the consideration of those scholars who share our areas of expertise, whether today or at some point in the future. Asserting that one can offer such a contribution to knowledge depends, of course, on a kind of intellectual arrogance. But, when done well, this arrogance should be tempered by the humility of knowing that if one is saying anything even remotely interesting, then that thing will itself eventually be the subject of revision. The central problem with DA-RT might be that it lacks this kind of humility. It rests on the assumption that we can uncover the truth and assumes that if we can pile evidence upon evidence that will allow us to have more confidence in such a finding. Lisa Wedeen has already highlighted the narrowness of this scholarly imaginary. I would only add that any claim to truth of which we could all be convinced through the alchemy of DA-RT would probably be sufficiently obvious as to not merit the time and effort of proving it in the first place.