As described in the paper, we used this project as an opportunity to investigate new ways
of approaching, recording, and reporting design study research. In support of this learning,
we decided early on to capture notes and insights as frequently as possible both for our own
reflective analysis and for auditing by others for validation. We adopted a practice of taking
field notes following meaningful interactions, providing a log through which we could trace
the development of insights and ideas. While we are limited in what we can share due to the nature
of the project domain, we offer an interactive timeline of high-level summaries as a first step toward these efforts.
By doing this we hope to contribute to an ongoing dialogue within the visualization community
surrounding the recording and reporting of design study research process and findings
The timeline was built using the D3 Javascript Library. Hovering over or clicking nodes in the timeline
reveals the associated field note summary. Brush filters support filtering across time periods. Hovering over the colored rectangles above the timeline reveals overview details about the various stages of the study.
The timeline may also be downloaded and ran locally: download
Date: Mon Nov 07 2016 09:03:12 GMT+0000 (Coordinated Universal Time)keywords: Start of field study
Fieldnote: Field study day 1: orientation
Date: Tue Nov 08 2016 08:03:25 GMT+0000 (Coordinated Universal Time)keywords: First day of field study
Fieldnote: met with members of division through which the field study is based (home-division), met with communications team about possible outreach tool
Date: Wed Nov 09 2016 09:03:27 GMT+0000 (Coordinated Universal Time)keywords: weekly check-in with home-division advisor
Fieldnote: begin weekly check-ins with home-division advisor.
Date: Thu Nov 10 2016 08:03:29 GMT+0000 (Coordinated Universal Time)keywords: meetings with Zika Program stakeholder, comms team
Fieldnote: - meeting with communications team to discuss outreach tool
- meeting with Zika Program stakeholder to discuss potential project.
Date: Thu Nov 10 2016 09:03:43 GMT+0000 (Coordinated Universal Time)keywords: meeting with [group#1]
Fieldnote: Met with [group#1] team to discuss possible collaboration. Highly compelling project, but unclear if there is a vis research problem.
Date: Mon Nov 14 2016 08:03:08 GMT+0000 (Coordinated Universal Time)keywords: meeting with [group#2]
Fieldnote: meeting with [group#2] to discuss potential vis projects - discussed vis platform for analysis by GH experts.
Date: Mon Nov 14 2016 09:03:19 GMT+0000 (Coordinated Universal Time)keywords: weekly check-in with home-division advisor
Fieldnote: discussed possible projects
Date: Tue Nov 15 2016 08:03:34 GMT+0000 (Coordinated Universal Time)keywords: meeting with [group#3](outreach)
Fieldnote: meeting with [group#3] to discuss visualization for outreach, storytelling
Date: Tue Nov 15 2016 08:03:53 GMT+0000 (Coordinated Universal Time)keywords: Presentation to home-division
Fieldnote: presented slides on data visualization and applied vis research home-division. The goal of the talk was to give a better sense of the research that we do and how our work could contribute to GH efforts
Date: Thu Nov 17 2016 08:03:44 GMT+0000 (Coordinated Universal Time)keywords: meeting with [group#2](analysis)
Fieldnote: Another meeting with [group#2] to discuss vis platform for GH experts
Date: Fri Nov 18 2016 09:03:32 GMT+0000 (Coordinated Universal Time)keywords: begin work on outreach tool
Fieldnote: Beginning implementation of outreach tool for communications team. "Map of GH investments bolstering American companies, universities and research orgs" Tool will support exploration of GH investments by congressional district.
Date: Mon Nov 21 2016 08:03:21 GMT+0000 (Coordinated Universal Time)keywords: Present slides to internal group
Fieldnote: presented slides on data visualization and applied visualization research to GH experts working with monitoring and evaluation
Led to great discussion and interest in collaboration.
Date: Mon Nov 21 2016 09:03:03 GMT+0000 (Coordinated Universal Time)keywords: weekly check-in with home-division advisor
Fieldnote: discuss projects, map out timeline.
Date: Tue Nov 22 2016 10:03:41 GMT+0000 (Coordinated Universal Time)keywords: meeting with Zika stakeholders + fellow tool builders
Fieldnote: meeting with Zika stakeholders and fellow tool builders to discuss Zika vis project, progress, challenges, and to begin to map out the data - two types: data about response efforts, data about spread of Zika.
Date: Tue Nov 22 2016 14:03:15 GMT+0000 (Coordinated Universal Time)keywords: initial meeting with [group#4]
Fieldnote: meeting with [group#4] team. A lot of great ideas for projects. Also discussed challenges surrounding data.
Date: Mon Nov 28 2016 09:03:31 GMT+0000 (Coordinated Universal Time)keywords: prioritizing design study with Zika stakeholders
Fieldnote: Primary project: design study for Zika program - mapping GH response efforts to combat the Zika Virus.
Possible secondary projects (time permitting): [group#3] outreach project; [group#4] mapping project; other short term projects
Date: Mon Nov 28 2016 09:03:35 GMT+0000 (Coordinated Universal Time)keywords: weekly check-in with home-division advisor
Fieldnote: finalize project plan - discuss projects, gather feedback
Date: Tue Nov 29 2016 08:03:16 GMT+0000 (Coordinated Universal Time)keywords: initial meeting with [group#5]
Fieldnote: Meeting with [group#5] to discuss vis/potential collaboration
Date: Tue Nov 29 2016 10:03:57 GMT+0000 (Coordinated Universal Time)keywords: meeting with [group#3](outreach)
Fieldnote: preliminary interview with [group#3] - narrow in on data and tasks for vis for outreach. Could be a great secondary collaboration if there's time.
Date: Wed Nov 30 2016 09:03:51 GMT+0000 (Coordinated Universal Time)keywords: meeting with [group#2](analysis)
Fieldnote: continued discussion with [group#2] re vis for analysis
Date: Mon Dec 05 2016 09:03:11 GMT+0000 (Coordinated Universal Time)keywords: weekly check-in with home-division advisor
Fieldnote: general updates on progress
Date: Mon Dec 05 2016 12:03:33 GMT+0000 (Coordinated Universal Time)keywords: meeting with [group#2](analysis)
Fieldnote: review current system, sketch ideas for possible vis extensions. Acting as a vis consultant from here on out - another member of home-division is taking on the project
Date: Tue Dec 06 2016 09:03:59 GMT+0000 (Coordinated Universal Time)keywords: meeting with [group#3](outreach)
Date: Mon Dec 12 2016 09:03:11 GMT+0000 (Coordinated Universal Time)keywords: weekly check-in with home-division advisor
Fieldnote: Discussed plans moving forward. Feedback on outreach tool.
Date: Mon Dec 12 2016 09:03:37 GMT+0000 (Coordinated Universal Time)keywords: meeting with Zika primary collaborator + fellow tool builders
Fieldnote: met with fellow tool builders and Zika primary collaborator to discuss data - strategize on data wrangling
Date: Tue Dec 20 2016 09:03:00 GMT+0000 (Coordinated Universal Time)keywords: weekly check-in with home-division advisor
Fieldnote: Feedback on outreach tool.
Date: Wed Jan 04 2017 10:03:18 GMT+0000 (Coordinated Universal Time)keywords: preliminary interview with Zika primary collaborator
Fieldnote: First official preliminary interview with Zika primary collaborator.
Discussed current understanding of data and tasks.
Conducted think-aloud with existing ArcGIS system. This helped clarify tasks and revealed low-level data vis challenges
Date: Wed Jan 04 2017 15:03:39 GMT+0000 (Coordinated Universal Time)keywords: screenshots from think-aloud
Fieldnote: Think-aloud revealed issues of occlusion resulting from multiple layered textures (exhibited in ArcGIS).
Date: Mon Jan 09 2017 10:03:35 GMT+0000 (Coordinated Universal Time)keywords: weekly check-in with home-division advisor
Fieldnote: feedback on preliminary Zika research. Triangulation of initial ideas and hypotheses.
Date: Fri Jan 20 2017 10:03:23 GMT+0000 (Coordinated Universal Time)keywords: weekly check-in with home-division advisor
Fieldnote: update on progress, general feedback
Date: Wed Jan 25 2017 10:03:45 GMT+0000 (Coordinated Universal Time)keywords: meeting with fellow tool builder
Fieldnote: met to brainstorm ideas, gather feedback on sketches, discuss data, map out timeline
Date: Mon Jan 30 2017 09:03:22 GMT+0000 (Coordinated Universal Time)keywords: weekly check-in with home-division advisor
Fieldnote: general update, feedback
Date: Tue Jan 31 2017 09:03:35 GMT+0000 (Coordinated Universal Time)keywords: presentation of outreach tool to senior personnel
Fieldnote: presented outreach tool to senior personnel - next step is to publish to GH website.
Date: Tue Feb 07 2017 09:03:11 GMT+0000 (Coordinated Universal Time)keywords: weekly check-in with home-division advisor
Fieldnote: Feedback on prototypes; discussed broader challenges surrounding inconsistencies in GH data
Date: Wed Feb 08 2017 09:03:46 GMT+0000 (Coordinated Universal Time)keywords: AI prototypes
Fieldnote: Adobe illustrator prototypes
Date: Wed Feb 08 2017 10:03:39 GMT+0000 (Coordinated Universal Time)keywords: meeting with home-division Data expert
Fieldnote: met with a member of the home-division is an expert on collecting, processing, and presenting global health data.
Discussed sources of discrepancies in global health data.
Confirmed some initial hypotheses surrounding discrepancy.
Date: Wed Feb 08 2017 12:03:00 GMT+0000 (Coordinated Universal Time)keywords: call with primary collaborator
Fieldnote: really interesting discussion. Expert knowledge includes a mental database of country specific inconsistencies.
-discussed issues of data availability, strength of surveillance systems.
-users will interpret data differently based on level of expert knowledge
Date: Tue Feb 14 2017 14:03:16 GMT+0000 (Coordinated Universal Time)keywords: weekly check-in with home-division advisor
Fieldnote: general feedback, update on progress
Date: Tue Feb 21 2017 10:03:38 GMT+0000 (Coordinated Universal Time)keywords: meeting with primary collaborators
Fieldnote: discussed ideas, gathered feedback on prototypes, did some brainstorming.
- need to be able to support variation in data parameters/availability across countries
- small multiples with reference visualizations seems to be an effective approach
Date: Tue Feb 21 2017 14:30:23 GMT+0000 (Coordinated Universal Time)keywords: weekly check-in with home-division advisor
Fieldnote: tech probe experiments
- small multiples vs. time slider, perhaps a combination of both
- bar charts vs. heat maps
- grouped bar charts for regional view
- hierarchical approach to layering multiple categorical attributes
Date: Wed Mar 01 2017 19:03:29 GMT+0000 (Coordinated Universal Time)keywords: meeting with GH experts
Fieldnote: met again with GH experts from initial meetings. Great discussion about official vs. unofficial data. Could be very useful programmatically to highlight data inconsistencies, etc.
Date: Tue Mar 07 2017 09:03:49 GMT+0000 (Coordinated Universal Time)keywords: weekly check-in with home-division advisor
Fieldnote: screenshot from lo-fi D3 prototypes testing small multiples
Date: Fri Mar 17 2017 12:03:20 GMT+0000 (Coordinated Universal Time)keywords: meeting with primary collaborator
Fieldnote: feedback on prototypes, would be helpful to be able to toggle choropleth by indicator. Discussed sharing the tool with the larger group of Zika stakeholders. Need to wrangle more data.
Date: Mon Mar 20 2017 18:03:31 GMT+0000 (Coordinated Universal Time)keywords: lo-fi D3 prototype country view
Fieldnote: Discussed the issue of disparity with a GH expert working outside of Zika efforts. Asked how she and her team handle disparities -- they add footnotes explaining the situation.
Date: Wed Apr 26 2017 09:03:39 GMT+0000 (Coordinated Universal Time)keywords: Visit from vis researcher
Fieldnote: Visit from vis researcher. Met with primary collaborators, fellow tool builders, and home-division collaborators. Established plans moving forward post-field-study
Fieldnote: Screenshots of TP1 at the end of the field study
Date: Wed May 03 2017 10:03:51 GMT+0000 (Coordinated Universal Time)keywords: Brown bag presentation
Fieldnote: presented the final iteration of the technology probe in a "brown bag" series. Received very positive feedback. Got the impression that this work applies more broadly. Received questions about sustainability, and about outreach vs. analysis.
Date: Fri May 05 2017 10:03:10 GMT+0000 (Coordinated Universal Time)keywords: presentation to senior personnel
Fieldnote: presented the technology probe to senior personnel. Very encouraging response.
Date: Sat May 06 2017 10:03:10 GMT+0000 (Coordinated Universal Time)keywords: End of field study
Fieldnote: Wrapping up at the field study, heading back to home institution to continue research
Date: Wed May 10 2017 05:03:22 GMT+0000 (Coordinated Universal Time)keywords: webinar with Zika Stakeholders
Fieldnote: presented the technology probe to the larger Zika stakeholders via a webinar. Clear from this meeting that inconsistencies in that data present a major barrier for analysis.
Also, need to provide info surrounding where each chunk of data is coming from.
Did received multiple very encouraging emails from stakeholders interested in collaborating further.
Fieldnote: Resuming research on the project. Reflecting on what we've learned so far. Pursuing the idea of using annotation to capture contextual knowledge surrounding the nuances of epidemiological data. Starting a new iteration of the technology probe, based on feedback and with full support for annotation.
Fieldnote: adding a message board/feed to allow users to make more general comments at the global and country levels.
Date: Tue Oct 10 2017 09:03:28 GMT+0000 (Coordinated Universal Time)keywords: meeting with new primary collaborator
Fieldnote: Pre meeting: preparing for meeting with new primary collaborator to discuss the complexities of working with GH data. Want to explore the possibility of continuing the project with a new focus on annotation.
Post meeting: excellent meeting. Collaborator is interested in the idea of incorporating contextual knowledge into the vis pipeline.
Gathered feedback on the new iteration of the technology probe
Mentioned the possibility of doing a presentation/workshop with larger group of Zika stakeholders at the end of October.
Date: Sun Oct 29 2017 09:00:00 GMT+0000 (Coordinated Universal Time)keywords: tech probe 2
Artifact notes: Screenshots of the iteration of the technology probes used during the workshop
Date: Mon Oct 30 2017 21:03:53 GMT+0000 (Coordinated Universal Time)keywords: meeting with primary collaborator and in-country Zika expert
Fieldnote: great discussions with primary collaborator and with a Zika expert working in-country. Seems obvious in retrospect but I'm gathering that the nature of the annotation will depend on the specific questions being asked of the data. Some sort of tagging plus filtering system could be valuable.
Fieldnote: Workshop went well, aside from the inevitable technical issues.
- collected a lot of sample annotations.
- collected usability feedback and feedback on features
- participants were engaged. great discussion.
- from discussion and survey feedback, sounds like annotation is a very valuable direction to head in.
Fieldnote: collected the last of the poll results. 10/13. not everyone but enough to give us a sense of things.
poll results results:
q1: On a scale of 0 to 10, how useful do you think the annotation (i.e. dropping pins) feature could be?
7.9 (avg.)
q2: Is this feature/concept worth pursuing? (Y/N)
Y (100%)
q3: On a scale of 0 to 10, how useful do you think the commenting feature could be?
7.7 (avg.)
q4: Is this feature/concept worth pursuing? (Y/N)
Y (100%)
Fieldnote: news flash! PAHO already publishes footnotes along with their weekly epidemiological reports. This will provide a great dataset to experiment and prototype with.
Date: Sun Dec 10 2017 09:03:24 GMT+0000 (Coordinated Universal Time)keywords: early stage writing
Fieldnote: beginning to write things up. reflecting on problem characterization, data abstraction, research process
Date: Tue Jan 09 2018 09:03:48 GMT+0000 (Coordinated Universal Time)keywords: a more general model
Fieldnote: beginning to see a more general model emerging - incorporating contextual information into the vis analysis pipeline
Date: Wed Jan 10 2018 14:03:40 GMT+0000 (Coordinated Universal Time)keywords: formalizing data and tasks
Fieldnote: formalizing data and tasks for global health response coverage assessment
Date: Thu Jan 11 2018 09:03:54 GMT+0000 (Coordinated Universal Time)keywords: epi data generation pipeline diagram
Fieldnote: starting to model the epidemiological data generation pipeline.
Date: Thu Jan 11 2018 14:03:25 GMT+0000 (Coordinated Universal Time)keywords: source of discrepancy diagram#2
Fieldnote: zoom in on source of discrepancies at the country level
Fieldnote: The DIKW pyramid from information sciences and knowledge management really resonates with what we've been seeing in our research. And, there's a whole body of literature from VAST (knowledge assisted visualization and visual analytics) surrounding the translation of the DIKW pyramid into computational space
Date: Mon Feb 05 2018 14:10:16 GMT+0000 (Coordinated Universal Time)keywords: pre-paper talk to research group
Fieldnote: delivered pre-paper talk to research group. Very encouraged by the response. very useful feedback.
Date: Sat Feb 10 2018 09:03:35 GMT+0000 (Coordinated Universal Time)keywords: system prototype
Fieldnote: prototypical system designed to support the stages of the process model.
Date: Fri Feb 23 2018 15:03:31 GMT+0000 (Coordinated Universal Time)keywords: meeting with primary collaborator
Fieldnote: Walked through system, gathered feedback, brainstormed ideas. Discussed annotation template, need to use language that will resonate with GH experts.
Date: Sat Mar 03 2018 15:03:17 GMT+0000 (Coordinated Universal Time)keywords: meeting with vis researchers and primary collaborator
Fieldnote: phone meeting between both vis researchers and primary collaborator to discuss and triangulate vis researcher's formalization of learning (the notion of implicit error/discrepancy)
Date: Tue Mar 13 2018 10:03:52 GMT+0000 (Coordinated Universal Time)keywords: mapping research process
Fieldnote: Work session between vis researchers - mapping out the stages of the research process