Expert Working Groups FAQ
Expert working groups (EWGs) are a core KGI strategy to evaluate evidence and strengthen policy consensus on specific issues.
What is a KGI expert working group?
Expert working groups bring together relevant experts for a time-bound project to summarize expert knowledge and articulate policy options. KGI builds EWGs by inviting selected volunteer contributors from across academia, industry, civil society, journalism, and practitioner communities to engage in a focused collaboration on a specific policy topic over the course of 6-12 months or more.
What is the goal of an expert working group?
The goal of each EWG is to produce non-partisan resources of broad utility to policymakers and/or industry decision makers. In producing resources we prioritize knowledge that is grounded in independent research, information that is accessible to those who may need to use it, and recommendations that are immediately actionable for relevant decision makers.
How do we decide who is an expert?
We consider experts to be those who have comprehensive or authoritative knowledge on the topic. They may be from academia, civil society, industry, journalism, or practitioner communities.Their contributions are related to the question(s) under consideration. Expertise emerges through a range of perspectives, including academic, professional, and lived experiences.
While we generally disclose the experts that are part of our working groups, in exceptional circumstances, some members may wish to participate without being publicly identified, either for personal or professional reasons, and we accommodate this preference in order to ensure more diverse representation in our EWGs.
In assembling an EWG we work to ensure that we have experts from multiple disciplines and perspectives, including social science, computer science, law, economics, and journalism, but acknowledge the inherent tradeoffs between group cohesion and diversity of perspective.
How do we identify relevant research on a question?
We identify relevant research on a question through an interactive process of seeking input from experts through the EWG. We often begin with a literature review or set of core questions and ask for input, feedback, and critique from experts in the EWG. By ensuring diverse perspectives in those groups, we broaden the types and scope of research that is brought to bear on the specific technology policy question.
How do we assess the quality of the research and data on which we rely?
Given the complicated and ever changing nature of technology, decisions about assessing research quality generally have to be done on a case-by-case basis. When we assess research quality, we consider the strength of the research design, its reliability and validity, and credibility, specifically related to the motivations of the research and its funders. Because our mission emphasizes the role of independent research, we focus on research that we perceive to be credible, non-partisan, and free from funder influence.
What is independent research?
Independent research means that the research occurs in an unbiased and objective manner, consistent with our mission and our core value of non-partisanship. We may analyze research from a range of sources, including companies, governments, civil society, or advocacy organizations. Our mission, however, is to use the EWG to assess the strengths of diverse research and identity recommended policy options. Many platforms have placed restrictions on independent research and therefore existing evidence may not always be conducted in an unbiased and objective manner.
Does research have to come from academia?
No, research that is both independent and high quality often comes from multiple sources, including but not limited to academia. Through EWGs we will consider research from a range of perspectives including civil society, think tanks, journalists, governments, and, in some circumstances, research from companies.
What do we do when research disagrees?
Sometimes research can be high quality and independent, but multiple sources can come to different conclusions. We are clear and transparent about where the research disagrees, and sometimes include a possible explanation for why different approaches may have come to different conclusions. We understand that effective policy often involves short and longer term trade offs and we work to enable ongoing learning and iteration.
How can you check our work?
We always include links and citations to all the research upon which we rely, so readers and policymakers can navigate for themselves to read the full research, not just our summarized version.
What if there isn’t enough research to answer a question?
This is an ever-present challenge in technology research, where the research is inhibited by the availability of data closely guarded by platforms. In particular, researchers’ confidence in answering some questions is sometimes lower because the data that has been used to answer the question is further divorced from the everyday usage scenario, relevant platform data is often unavailable, and societal dynamics are complex. This reality informs our work with researchers to develop a Gold Standard for publicly available platform data, in order to help clarify what data needs to be available and in what form in order to better inform questions related to technology policy and design.
What if I think you missed relevant research or perspectives?
Please share it with us! You can send it to us at knightgeorgetown@georgetown.edu.