Performing Data Engagement Interviews

Onboarding Participants

Once we were prepared for conducting our interviews, we reached out to remind our participants about their deployment, and provide a prompt for them to consider ahead of their data engagement interview:

“Imagine you have access to your home’s indoor air quality data. What would you want to know about it? What would you want to do with it?”

We revisited this prompt at the start of participants’ interviews and had them write down their response on a worksheet while we set up our equipment¹. If participants struggled to come up with a specific goal on their own, we alternatively referred them to a list of prepared questions they could choose from.

Conducting data engagement interviews²

The interview team came prepared with an interview kit, and we conducted our data engagement interviews according to a prepared protocol that helped us step through a series of prompts and stages³. We provide a list of sample engagement prompts as examples to help motivate prompts for your own project.

We used Jupyter notebooks for live coding and presenting information to participants. This worked well for us, but researchers should consider whichever environments they are most comfortable with, and that afford live-coding and presentation capabilities (Power BI, Observable, etc.). Depending on the project, researchers may also consider environments where the analysis output can be decoupled from the code so that participants can focus on their data, and not the work required to process it.

Our original goal for conducting these interviews was to identify and understand what participants knew to do with their data, and how they went about operationalizing their goals. We therefore tailored data engagement interview prompts to capture observations on how they worked with data. After working through a question with the help of an analyst, we reserved a final prompt to have participants re-imagine how they would create the same solution on their own with the data they had available.

Analyzing data engagement interviews⁴

With participants’ permission, we audio-recorded each data engagement interview. We added these files along with participants completed interview worksheets, individual Jupyter notebooks for each participant interview, and any other artifacts they created during their interviews (design sketches, notes, or other mark-ups). We referred to each of these materials while performing our subsequent interview analyses.

The first author wrote up a reflexive summary on the interview experience after each participant interview as a way to capture their reactions to and impression of the overall interview process. This involved capturing information on whatever was interesting or surprising in the interview, as well as reflections on which aspects of the interview went well (or not). We also made sure to summarize outcomes related to our specific research goals which for us involved: participants stated goals, questions or interests; emerging themes on their motivations for working with data; observations on the data analysis process in general; participant operationalization steps; and any significant quotable moments we imagined being important for our write-up.

The interview audio was also professionally transcribed, then further summarized by the first author. Afterwards, a group of three researchers performed seven reflexive meetings, one for each participant, where they reviewed the interview transcripts and summary, and met to discuss their thoughts and reactions to the interview. These meetings were also recorded.

The first author then summarized these reflexive meetings and met once more with the other two researchers. These meetings used the reflexive meeting notes to construct an affinity map for important topics and ideas encountered through the participant interview transcripts, as well as the framework and outcomes presented in this work.