Abstract
Quantifying user performance with metrics such as time and accuracy does not show the whole picture when researchers evaluate complex, interactive visualization tools. In such systems, performance is often influenced by different analysis strategies that statistical analysis methods cannot account for. To remedy this lack of nuance, we propose a novel analysis methodology for evaluating complex interactive visualizations at scale. We implement our analysis methods in reVISit, which enables analysts to explore participant interaction performance metrics and responses in the context of users' analysis strategies. Replays of participant sessions can aid in identifying usability problems during pilot studies and make individual analysis processes salient. To demonstrate the applicability of reVISit to visualization studies, we analyze participant data from two published crowdsourced studies. Our findings show that reVISit can be used to reveal and describe novel interaction patterns, to analyze performance differences between different analysis strategies, and to validate or challenge design decisions.Citation
Carolina Nobre,
Dylan Wootton,
Zach Cutler,
Lane Harrison,
Hanspeter Pfister,
Alexander Lex
reVISit: Looking Under the Hood of Interactive Visualization Studies
SIGCHI Conference on Human Factors in Computing Systems (CHI), 1-13, doi:10.1145/3411764.3445382, 2021.
BibTeX
@inproceedings{2021_chi_revisit,
title = {reVISit: Looking Under the Hood of Interactive Visualization Studies},
author = {Carolina Nobre and Dylan Wootton and Zach Cutler and Lane Harrison and Hanspeter Pfister and Alexander Lex},
booktitle = {SIGCHI Conference on Human Factors in Computing Systems (CHI)},
publisher = {ACM},
doi = {10.1145/3411764.3445382},
pages = {1-13},
year = {2021}
}
Acknowledgements
We want to thank Kiran Gadhave for making his study data available and for his help with the provenance tracking library. We gratefully acknowledge funding by the National Science Foundation (IIS 1751238, IIS 1815587, and OAC 1835904).
Images
These images are not part of the original paper and licensed using CC BY 4.0. If you use these images, please cite the paper. Click on the images for full resolution.
