Comprehension of Complex Visual Displays

It has been claimed that new technologies, such as multimedia, provide ways of augmenting human cognition by presenting information that is difficult to imagine or comprehend from traditional instructional methods. For example, one can now present diagrams in static or dynamic form, and produce powerful visualizations of scientific phenomena and more abstract information. In the Spatial Thinking Lab, we are interested in how people make sense of these complex visual-spatial displays and how they use them in learning, reasoning and problem solving. For example, recent projects have studied how people learn about mechanical systems from animations, and how experts and novices view weather maps when forecasting the weather. One of the primary techniques that we use to study these topics is the measurement of eye fixations. Eye tracking provides an online measure of the information that a viewer is accessing while reasoning and solving problems with complex visual displays. Correlating eye fixation data with measures such as performance and response time allows for a more complete understanding of the cognitive processes required for interpreting complex visualizations and the outcomes of these cognitive processes.

In the C2MD project (Cognition and Comprehension of Meteorological Displays), currently funded by the Office of Naval Research, we are examining people's intuitions about the effectiveness of different complex displays, and how these intuitions match or mismatch the actual effectiveness of these displays. For example, the goal of a display designer might be to present as much information as possible on a single display. But how much information is too much? We are examining whether factors such as the spatial abilities and expertise of the user affect both their preferences for different displays and their performance with these displays. This research is in collaboration with Harvey Smallman, Pacific Science & Engineering Group, Inc.

Publications

Cohen, C. A. & Hegarty, M. (in press). Individual differences in use of an external visualization while performing an internal visualization task. Applied Cognitive Psychology.

Hegarty, M. & Kriz, S. (in press). Effects of knowledge and spatial ability on learning from animation. In R. Lowe & W. Schnotz. Learning from Animation. Cambridge University Press.

Hegarty, M. (2005). Multimedia learning about physical systems. In R. E. Mayer (Ed). Handbook of Multimedia. Cambridge University Press.

Mayer, R.E., Hegarty, M., Mayer, S. Y Campbell, J. (2005). When passive media promote active learning: Static diagrams versus animation in multimedia instruction. Journal of Experimental Psychology: Applied, 11, 256-265.

Hegarty, M. (2004). Dynamic visualizations and learning: Getting to the difficult questions. Learning and Instruction, 343-351.

Kriz, S. & Hegarty, M. (2004). Constructing and revising mental models of a mechanical system: The role of domain knowledge in understanding external visualizations. In K. Forbus, D. Gentner & T Regier (Eds.) Proceedings of the 26th Annual Conference of the Cognitive Science Society. Mahwah, NJ: Lawrence Erlbaum Associates.

Hegarty M. (2004). Diagrams in the mind and in the world: Relations between internal and external visualizations. In A. Blackwell, K. Mariott and A. Shimojima (Eds.). Diagrammatic Representation and Inference. Lecture Notes in Artificial Intelligence 2980 (pp. 1-13). Berlin: Springer-Verlag.

Hegarty, M., Kriz, S. & Cate, C. (2003). The roles of mental animations and external animations in understanding mechanical systems. Cognition & Instruction, 21, 325-360. [PDF]

Narayanan, N. H. & Hegarty, M. (2002). Multimedia design for communication of dynamic information. International Journal of Human-Computer Studies, 57, 279-315.

Hegarty, M. Narayanan, N. H. & Freitas, P. (2002). Understanding Machines from Multimedia and Hypermedia Presentations. In J. Otero, A. C. Graesser & J. Leon (Eds.). The Psychology of Science Text Comprehension. Lawrence Erlbaum Associates.

Narayanan, N. H. & Hegarty, M. (2000). Communicating dynamic behaviors: Are interactive multimedia presentations better than static mixed-mode presentations? In M. Anderson, P. Cheng & V. Harslev (Eds.) Theory and application of diagrams. Lecture Notes in Artificial Intelligence 1889. Berlin: Springer.

Tendick, F., Downes, M., Gogtekin, T. Cavusoglu, M.C., Feygin, D. Wu, X., Eyal, R., Hegarty, M. & Way, L.W. (2000). A virtual testbed for training laparoscopic surgical skills. Presence, 9, 236-255.

Shah, P., Mayer, R. E. & Hegarty, M. (1999).  Graphs as aids to knowledge construction.  Journal of Educational Psychology, 91, 690-702.

Hegarty, M., Quillici, J., Narayanan, N. H., Holmquist, S., & Moreno, R. (1999).  Multimedia Instruction: Lessons from Evaluation of a Theory-based Design. Journal of Educational Multimedia and Hypermedia, 8, 119-150.

Narayanan, N. H. & Hegarty, M. (1998). On designing comprehensible hypermedia manuals. International Journal of Human-Computer Studies, 48, 267-301.

Hegarty, M. & Just, M.A. (1993).  Constructing mental models of machines from text and diagrams.  Journal of Memory and Language, 32, 717-742.

Hegarty, M., Carpenter, P. A. & Just, M. A. (1990).  Diagrams in the comprehension of scientific text. In R. Barr, M. L. Kamil, P. Mosenthal, & P. D. Pearson (Eds.) Handbook of reading research. New York: Longman.

Hegarty, M. & Just, M. A. (1989).  Understanding machines from text and diagrams. In H. Mandl & J. Levin (Eds.). Knowledge acquisition from text and picture. Amsterdam: North Holland (Elsevier Science Publishers).

Department of Psychology • University of California, Santa Barbara
Contact Mary Hegarty with questions. Created and maintained by Drew Dara-Abrams.