Criteria for Rigor in Visualization Design Study

Criteria screenshot

Abstract

We develop a new perspective on research conducted through visualization design study that emphasizes design as a method of inquiry and the broad range of knowledge-contributions achieved through it as multiple, subjective, and socially constructed. From this interpretivist position we explore the nature of visualization design study and develop six criteria for rigor. We propose that rigor is established and judged according to the extent to which visualization design study research and its reporting are INFORMED, REFLEXIVE, ABUNDANT, PLAUSIBLE, RESONANT, and TRANSPARENT. This perspective and the criteria were constructed through a four-year engagement with the discourse around rigor and the nature of knowledge in social science, information systems, and design. We suggest methods from cognate disciplines that can support visualization researchers in meeting these criteria during the planning, execution, and reporting of design study. Through a series of deliberately provocative questions, we explore implications of this new perspective for design study research in visualization, concluding that as a discipline, visualization is not yet well positioned to embrace, nurture, and fully benefit from a rigorous, interpretivist approach to design study. The perspective and criteria we present are intended to stimulate dialogue and debate around the nature of visualization design study and the broader underpinnings of the discipline.

Citation

Miriah Meyer, Jason Dykes
Criteria for Rigor in Visualization Design Study
IEEE Transactions on Visualization and Computer Graphics (InfoVis), 26(1): 87--97, doi:10.1109/TVCG.2019.2934539, 2019.

BibTeX

@article{2019_infovis_criteria,
  title = {Criteria for Rigor in Visualization Design Study},
  author = {Miriah Meyer and Jason Dykes},
  journal = {IEEE Transactions on Visualization and Computer Graphics (InfoVis)},
  publisher = {IEEE},
  doi = {10.1109/TVCG.2019.2934539},
  volume = {26},
  number = {1},
  pages = {87--97},
  year = {2019}
}

Acknowledgements

This work has been informed by lengthy and active discussions with many colleagues in numerous places over a long period. In particular we would like to thank Nina McCurdy and Jonas Löwgren for deeply influential contributions to our thinking. We also thank Geraldine Fitzpatrick, Ethan Kerzner, Alex Lex, Julienne Meyer, Torsten Möller, Michael Sedlmair, Brett Smith, Jason Wiese, Jo Wood and four engaged and widely informed reviewers for provocative critical conversations.