Page 1 of 1

DA-RT Approach Misses Important Aspects of Qualitative Research

Posted: Mon Apr 25, 2016 3:43 am
by winters
What is the essence of interpretation and analysis in Comparative Politics? A big part of it is not just having "data" or "evidence." It is also the investment of years building the capacity even to know what you are observing and to recognize its significance in context. This is what makes people authorities in a field. It is the accumulation of experience and expertise that allows for meaningful interpretation in the first place. And it is rooted in a mastery of languages, cultures, histories, and other nuances of places, populations, groups, and communities. Colleagues working on the US (who set DA-RT in motion) take these skills and insights for granted because they only focus on "home-America" where everything is relatively familiar.

Someone in an Indonesian, Vietnamese, or Filipino legislature could say or do something that has an apparent meaning to US-focused observers, but a very different (or possibly even opposite) meaning to someone who has a subtle understanding of these places. Speaking just for myself, it has taken decades to develop the knowledge needed for interpreting the not-always-evident evidence. And this didn't just come from reading books and articles over the years (which someone could presumably "cram" to replicate). It came also from hundreds of conversations, experiences, observations, people telling their stories, and listening to people from these places trying to interpret as well. And it came from lots of mistakes. You learn to triangulate and piece disparate elements together to make sense of things that are very hard to figure out.

This is related to a second point already touched on by Mimi Keck. Part of the problem with the whole survey-dataset-IRB image of research is that many of us have made scores of visits to the places we study that are not necessarily part of a formal research plan or episode. And yet during those visits, you had a chance to meet a labor leader, an NGO activist, or whomever. And during those conversations, interviews, lunches, or wedding receptions (even funerals and circumcisions!), you took out your notebook and wrote down what was said -- possibly as summary or maybe scribbling direct quotations. You knew what you were hearing was important, but you knew you might not need it or use it immediately. By my last count, I have 18 such notebooks filled with material, 49 (long) recordings of people I talked to from presidents and prime ministers to newspaper editors and oligarchs to peasant protesters and garbage pickers. And I have many more recordings I dictated to myself after various meetings, just to capture what was said while it was still fresh. It's an ordered hodge-podge of things that I have reviewed several times and drawn from for my work.

To "replicate" this, someone would have to replicate the entire range of my encounters with the places and people I've spent decades studying. Indeed, some people are fairly close approximations -- because they are themselves experts and authorities on these same places. Thankfully, a key part of the process of publishing includes editors sending our writings to such very informed people for their critical assessments.

The DA-RT initiative represents a hopelessly narrow slice of what good research is, how useful information is gathered, and how illuminating debate is advanced. Sometimes research involves IRB-cleared survey instruments and planned research episodes. But many times research happens when it does and looks nothing like what is in the minds of the people pushing this agenda. I take the honesty and integrity of colleagues in the academy as high unless proven otherwise. I can question their research and conclusions based on many dimensions of the work (what it asks, what it fails to ask, what evidence is present or lacking) without having to say "prove to me the evidence is actually real or accurate." For some limited kinds of research or analysis, appending datasets or other materials may make sense. But for others, it simply does not. If you want to replicate Ben Anderson's insights about the Javanese concept of power or reproduce his analyses in Imagined Communities, I invite you read what he read, go where he went, talk to the people he engaged with, and retrace the pilgrimages he made.

Jeffrey Winters
Northwestern University