QTD Process

Jump to:

The Qualitative Transparency Deliberations will proceed through a two-stage process, guided by a Steering Committee appointed by the QMMR Section President, Peter Hall. The first stage of the process will focus on identifying the dimensions along which we must differentiate when discussing research transparency. The second stage will involve substantive deliberations over research transparency, differentiated along the identified dimensions. The second stage will be driven forward by a set of working groups, constituted based on the input received during the first stage, each dedicated to considering transparency issues as they arise for a particular form of research. Each working group will be comprised of political scientists with experience in the aspects of qualitative and multi-method research under discussion, and each group will invite input from, and consult with, a broad range of scholars.

Preliminary Stage: Appointment of the Steering Committee

The members of the Steering Committee, appointed for 1-year terms by the QMMR Section President in consultation with other section officers, between them have research experience using a diverse set of qualitative methods—such as process tracing, ethnography, interpretive methods, comparative historical analysis, interview-based research, archival work, and algorithmic forms of qualitative analysis. Individual members, however, will not serve as representatives for particular constituencies; instead, the Steering Committee will endeavor to be broadly representative of QMMR scholarship. The Steering Committee includes at least one member who is junior (pre-tenure) and one who is teaching at an institution that is not a Ph.D.-granting research university. The Steering Committee will seek to make decisions by consensus but if necessary can make decisions with a vote of a majority of its members.

The Steering Committee's responsibilities will be to:

  • structure and moderate the at-large consultations;
  • select, delineate, and coordinate the substantive foci of the Working Groups, based on the input received during the at-large consultation;
  • appoint Working Group members;
  • provide an initial set of common structuring questions and thus a common framework to help guide and coordinate Working Group deliberations;
  • oversee the timely progress of the Working Groups;
  • facilitate conversation, where helpful, across Working Groups on issues of common interest; and
  • provide feedback on interim drafts of Qualitative Transparency Statements developed by the Working Groups.

Stage 1: At-Large Consultations

In Stage 1, QMMR scholars will be invited to participate in an at-large, online consultation over how to structure the substantive deliberations. While comments submitted during this stage may in principle address any aspect of research transparency in political science, the Steering Committee will at this stage primarily seek comments about the forms and aspects of qualitative research that require separate substantive discussions. The categories used to structure the symposium about research transparency in the Spring 2015 issue of Qualitative and Multi-Method Research—types of evidence, forms of analysis, and social settings in which empirical engagement unfolds—might serve as initial categories for this consultation, but the process during this stage will be open to any and all suggestions for structuring the substantive discussion. Comments at this stage might also articulate dilemmas or challenges that qualitative researchers face in pursuing transparency and that warrant substantive discussion.

The QTD consultation platform allows anyone to submit a comment, and we encourage participation by anyone interested in QMM research, regardless of section membership. Comments by registered users who are logged in post on the QTD web portal without delay; anonymous posts are also possible for users who prefer not to have their identify displayed with the forum post; they will post after a brief delay for review but without edits, provided that comments observe norms of civility and professionalism.

Interim Stage: Selection of Working-Group Foci

The Steering Committee will review the input received through the at-large consultations and, informed by these comments, will select a set of foci for the s substantive deliberation. These foci are likely to reflect the different dimensions along which qualitative research projects vary. For each identified focus, the Steering Committee will appoint a Working Group (WG) to consult and deliberate about research transparency issues as they relate to the kind or aspect of research that is the focus of that WG's work. In total, approximately 15 WGs will be formed, each comprised of 3 scholars who regularly engage in the kind of research that is the WG’s focus. In appointing the members of the WGs, the Steering Committee will aim to achieve diversity in scholars' professional circumstances, including career stage and type of institution. Liaison representatives from other relevant APSA organized sections will be asked to advise the Steering Committee on the appointment of WG members and to offer input and advice along the way.

The Steering Committee will provide the Working Groups with an initial set of common guiding questions. These questions are not intended to be exhaustive of the issues that a WG might consider, but will be designed to ensure a degree of commonality and comparability across the deliberations, while respecting the WGs' differentiated mandates. Guiding questions might, for instance, encourage the WGs to identify the features of research (of the kind that a given WG is examining) about which it is most important to be transparent; to identify current prevailing or "best" practices; to point to possible ways in which transparency practices might be improved; and to identify important benefits, costs, risks, and tradeoffs confronted by researchers conducting this form of research in pursuing transparency.

Stage 2: Working Group-Based Differentiated Deliberations

In addressing the issues that they have been tasked with considering, the WGs will widely with scholars in their research communities. For a period of at least two months, the WGs will gather broad input from relevant research communities on the guiding questions as well as other issues identified by the WG as warranting discussion.

Working Group consultations with interested research communities will take two forms: open online deliberation via the QTD platform, similar in form to the consultations carried out during the first stage (with separate discussion boards for each Working Group); and the active solicitation of input from colleagues with experience in the use of the qualitative methods or in dealing with the particular issues that are the focus of a particular Working Group. All scholars are invited to participate in substantive discussions of research transparency within differentiated research communities.

Throughout the deliberations, Working Groups will review the input received. Working remotely, they will begin to identify areas of agreement, areas of disagreement, key considerations, tradeoffs, and open questions that have surfaced during their consultations. Funding permitting, the Steering Committee and all Working Groups will meet and seek to identify and discuss areas of agreement, areas of tension, and common themes emerging across WG deliberations.

Output: Community Transparency Statements

Toward the end of these deliberations, each Working Group will write a text that summarizes the understandings of research transparency of the researchers who have participated in the deliberations over the aspects or kinds of qualitative research within the Working Group's focus. While the structure and content of these Community Transparency Statements will be decided upon as the process unfolds, we expect that each CTS will identify the perceived needs and benefits of transparency for a particular form or aspect of qualitative research; point to prevailing and "best" transparency practices within the relevant research communities; and discuss the costs, risks, tradeoffs, and practicalities that the pursuit of transparency in this research domain entails. Each WG will post an interim draft of its Community Transparency Statement online and allow a period of at least 2 weeks for online comment from their respective research communities.

Each Working Group will have ultimate discretion over the content of its Community Transparency Statement. Reflecting extended and inclusive deliberations among diverse qualitative research communities, these texts will likely vary in their degree of prescriptiveness, depending on the nature of the working groups' deliberations and degree of consensus: Some CTSs may take the form of specific transparency standards or guidelines; others may raise considerations or questions that those undertaking or assessing a given type of research should ask, including costs and benefits of being transparent about specific aspects of the kind of research under consideration, or of being transparent in particular ways; others might delineate tradeoffs, ambiguities, and lines of disagreement over the meaning and practicalities of transparency for a given mode of inquiry. Each Community Transparency Statement will also report the degree of heterogeneity of views among the participants in a given Working Group's deliberations. What the Community Transparency Statements will have in common, however, is that each will seek to articulate the (possibly multiple) understandings of research transparency that are current among scholars undertaking a given form of qualitative research. Each WG will also strive to produce a text that will be a useful resource for designing and conducting research, for teaching graduate students, and for evaluating research outputs. We also imagine that the Community Transparency Statements produced through this process will, ultimately, be tentative in nature, and may be revised as understandings, norms, practices, and debates evolve.