Overview of Breakout Sessions

The purpose of this workshop is to develop ideas that will further define the problem space, the key problems and the critical questions that need to be answered to make progress toward understanding, developing, and evaluating of Trustworthy Algorithmic Decision-Making.

There will be 4-5 parallel sessions during each time set aside for breakouts on the agenda. Each parallel session will focus on one of the high-level themes that emerge from the note-taking and affinity diagramming during the first half of Day 1. Workshop participants will rotate through the different themes, working on a different theme during each scheduled breakout session. Each breakout session has a specific focus: Brainstorm, Synthesize, “How might we…”, and Problem Statement, such that a small group will be working on each theme during each stage, and then hand off their work to the group working on that theme during the next stage. At each stage, the groups will be randomized so that everyone gets to meet, work with and bounce ideas off of new people.

Each parallel breakout session is 1 hour and 15 minutes long; the last 15 minutes should be spent capturing and documenting for the next group. Don’t forget to do quick introductions first thing during each breakout session!

Roles and Responsibilities

• Theme Champion: one volunteer who stays with a theme through all four stages. Provides continuity by answering questions about earlier conversations. Responsible for keeping the discussions focused and on track, overseeing documentation of the work during each breakout session so the next group can build on what the previous group did, and collecting any files or photos that were taken and storing them in the location provided for each theme. (Ignore the room assignments on the back of your badge if you are a Theme Champion!)
• Note-Taker(s): PhD student participant who stays with a theme through all four stages of the parallel breakouts. Any additional participants who want to help with note-taking can also do so. Responsible for documenting the work during each breakout session, including notes and photos of post-its and anything that is written up on the whiteboard; this will all be invaluable for writing the report.
• Time-Keeper: Chosen at the beginning of each breakout session. Responsible for making sure the group stays on task, and stops ~15 minutes before the end of the breakout session to wrap up and document the work.
• Participants: Responsibilities include… participating! Keep an open mind, be inclusive, remember that the group is diverse and ask for questions and clarifications when necessary. If you have a question someone else probably does too! Be creative and patient, and have fun!

Parallel Breakouts | Stage 1: Brainstorm

The goal of this activity is to creatively generate ideas and background information to add content and context and further develop the theme. This is an expansion phase, not a reduction phase. The main output of this phase is the documented ideas that the group generates.

• Start by developing a question or prompt for the brainstorming that characterizes the theme, based on the group affinity diagram.
• Then do three short rounds of brainstorming, 10-15 minutes each, in response to the question or prompt, writing each idea on a post-it. Write first, then share later.
• After each round of brainstorming, each person sticks their post-its on the wall/whiteboard and reads/describes it. Listen to each other, and in the next round build on each others’ ideas!
• Aim for quantity! Come up with as many ideas as possible. Encourage weird and wacky ideas.
• Stay in a generative mindset, not a critical one. Keep an open mind, and be positive. One way to do this is to encourage “and” statements, not “but” statements.

Parallel Breakouts | Stage 2: Synthesize

The goal of this activity is to build on the idea generation in the previous phase, and identify the big ideas and key concepts related to the overarching theme. The main output of this phase is at least 3-5 “insight statements” about problems that need to be understood better and/or solved, along with text to describe each insight.

• Start by walking the wall for 10-15 minutes, and reading the ideas that the previous group generated. Add post-its if you have new ideas, observations, reactions, etc.
• Then discuss with the group what’s on the wall, and identify at least 3-5 “big ideas”.
• Write an “insight statement” for each big idea. To do this, discuss each idea, and rephrase it as a short statement that captures an understanding that sheds light on some important aspect of the theme. This doesn’t need to be perfect; it is just a building block for the next stage.
• Write some text to describe each insight, and refine. Borrow heavily on the output of the brainstorming and ideas from the group affinity diagram.

Parallel Breakouts | Stage 3: “How Might We…”

The goal of this activity is to expand on the insight statements, and rephrase them as questions that need to be answered. This transforms the thinking about the insights into opportunities for future research and design activities. The main output of this phase is one question per insight statement, along with notes captured from the discussion.

• Start by reading the insight statements and supporting text generated by the previous group.
• Rephrase the insight statements as questions that need to be answered, starting with “How might we…”. The questions should be broad enough to allow for a variety of possible approaches and answered, but narrow enough that they are not overly restrictive
• For each “How might we…” question, discuss and take notes on: examples in the world (domains?), approaches/methodologies, constraints, stakeholders, leverage points, etc.
• Write each idea up on the whiteboard or a giant post-it to hand off to the next stage.

Parallel Breakouts | Stage 4: Problem Statement

The goal of this activity is to select one problem statement from the candidates produced in the previous session, and further describe it. The main output is a presentation about it that you will deliver to all of the workshop participants after lunch on Day 2.

• Start by reading and discussing the output of the previous stage, which should consist of ideas written on the whiteboard or giant post-its, capturing new ideas as they come up.
• Choose one that you will focus your presentation on. We recommend using dot-voting (aka sticker voting, multi-voting, etc.) so that the choice is not dominated by individual voices in the room. 3-4 votes each should be enough although you may need more if there are a large number of candidates. Vote by putting one or more dots next to your favorite idea.
• Once you have selected an idea, write a problem statement that is ambitious, but still actionable, and in line with the goals of the workshop.
• Then work on your presentation. Remember to avoid jargon, and define discipline-specific terms.

The presentation should cover the following:

1. What is the problem and why is it important?
   • define key terms and identify stakeholders
   • provide a scenario/example that illustrates the problem
   • what are the best sources of information about the problem?
2. Why is this a difficult problem?
   • describe the scope of the problem
   • what are the important unsolved/poorly specified aspects?
3. Why is progress possible?
   • describe what progress would look like; how would we recognize it?
   • approaches likely to make progress
4. What are the barriers for success, and how might we mitigate them?
   • ideas, training, incentives, resources (time, funding, data, etc.)…