Lecture: Interactive-Visual Data Analysis (L&E)

As of 2022, but also applies to 2024.

General Description 

This course introduces fundamental concepts and techniques of interactive-visual data analysis (IVDA). The main focus is on the combination of automatic data analysis methods with interactive visual interfaces, as well as on their interplay to facilitate data analysis goals. As such, IVDA is particularly suited to leverage the strengths of both humans and machines in a human-in-the-loop data analysis process. Associated research fields are Information Visualization, Visual Analytics, Interactive Data Science, Interactive Machine Learning, and Human-Centered Artificial Intelligence.

Learning Outcome

Students will learn basic characteristics of data types and data attributes (WHAT), as well as data analysis tasks (WHY). Further, students will learn basic design skills about HOW data can be transformed into visual structures and which types of visualization techniques are meaningful design choices for given data types and analysis at hand. Students will also learn fundamental interaction techniques, as well as concepts for the composition of views in data analysis systems for different data types and their individual complexities.

Further, students will gain a deep understanding about how data analysis can benefit from both having a human and a (machine learning) model in the loop, following the goal to gain knowledge from data.  Along these lines, students will learn about the strengths (and weaknesses) of human and machines, as well as about combining these complementary strengths effectively, as described in Visual Analytics methodology. In detail, students will learn examples for interactive data preprocessing, human-centered unsupervised machine learning, as well as for human-centered semi-supervised and supervised machine learning. Finally, the course introduces approaches that allow training personalized machine learning models and conduct personal and human-centered data analytics and artificial intelligence.

Target Groups

This module is designed for MA students (POC, DS, AI). It would be possible for students in other disciplines to take this course with a programming background. There are no enforced prerequisites, but the willingness to participate actively and to work in groups is recommended.  

Prior Knowledge

The course requires some degree of  programming expertise. Student groups should be comfortable with frontend and backend development. Basic knowledge in data science, machine learning, and data analysis are useful but not mandatory.

Recommended Reading

Visualization Analysis and Design, Tamara Munzner (A K Peters Visualization Series, CRC Press, 2014) is the course textbook. Recommended reading also includes selected papers, as outlined below.

Recommended reading (pre-class)

• VAD Book Chapter 1. What's Vis, and Why Do It?

In-class Agenda

• Welcome to the class!
• Introduction to Interactive Visual Data Analysis

• Interactive Demo: Exploration of Funds Data

• Course logistics - organizational stuff

Further Reading

• [VIS Design Principles]: Semiology of Graphics, Jacques Bertin, Gauthier-Villars 1967, EHESS 1998
• [VIS Design Principles]: The Visual Display of Quantitative Information. Edward R. Tufte. Graphics Press, 1983.

Recommended reading (pre-class)

• VAD Book Chapter 2: What: Data Abstraction

In-class Agenda

• What to analyze? Introduction to the data perspective

• What to analyze? Dataset Types, the between-objects perspective

• What to analyze? Data Attributes: nominal, ordinal, and numerical

Further Reading

• [Dataset Types]: Information Visualization and Visual Data Mining - Daniel Keim - 2002

• [Time Series Data]: Visualization of Time-Oriented Data - Wolfgang Aigner, Silvia Miksch, Heidrun Schumann, Christian Tominski. Revised and Expanded Second Edition. Springer, 2023

Recommended reading (pre-class)

• VAD Book Chapter 3: Why: Task Abstraction

In-class Agenda

• Why analyze? Analysis Tasks - actions and targets

• Why analyze? Data and Task Abstraction using a four-level analysis framework for design and validation

Further Reading

• [Paper]: (The Nested Model) A Nested Model for Visualization Design and ValidationTamara Munzner. IEEE TVCG. 2009

• [Paper]: (Data and Task) The eyes have it: a task by data type taxonomy for information visualizations Ben Shneiderman. IEEE Symposium on Visual Languages. 1996

• [Paper]: (Data and Task) Information Visualization and Visual Data MiningDaniel Keim. IEEE TVCG. 2002

Recommended reading (pre-class)

• VAD Book Chapter 5: Marks and Channels
• VAD Book Chapter 6: Rules of Thumb

In-class Agenda

• How analyze? Marks - Basic Graphical Elements

• How analyze? Channels - Visual Variables

• How analyze? Visualization Guidelines - Perception, Color, and Rules of Thumb

• How analyze? Decoding of Visualizations - Chart Decomposition

• How analyze? Exercise

Further Reading

• [VIS Design Principles]: Semiology of Graphics, Jacques Bertin, Gauthier-Villars 1967, EHESS 1998
• [VIS Design Principles]: The Visual Display of Quantitative Information. Edward R. Tufte. Graphics Press, 1983.

Recommended reading (pre-class)

• VAD Book Chapter 11. Manipulate View
• VAD Book Chapter 12. Facet into Multiple Views
• VAD Book Chapter 14. Embed: Focus+Context

In-class Agenda

• How analyze? Interaction Design - Engaging in a dialog with the data

• How analyze? Interaction Techniques - Overview of atomic Interactions

• How analyze? View Composition - Leveraging interaction techniques

Further Reading

• Norman, D., The design of everyday things: Revised and expanded edition. Basic books. 2013
• Interactive Visual Data Analysis. Christian Tominski and Heidrun Schumann. AK Peters Visualization Series. CRC Press. 2020

Recommended reading (pre-class)

• VAD Book Chapter 7: Arrange Tables
• VAD Book Chapter 8: Arrange Spatial Data
• VAD Book Chapter 9: Arrange Networks and Trees
• Polaris: A System for Query, Analysis and Visualization of Multidimensional Relational Databases. Chris Stolte and Pat Hanrahan. Proceedings of IEEE InfoVis 2000. [research paper, intellectual foundation of the Tableau software]

In-class Agenda

Advanced visualization techniques for...

• Multivariate Data
• Networks & Graphs
• Trees & Hierarchies
• Time Series
• Geographical Data
• Other Data Types

Further Reading

Web-based overviews of techniques for...

• Multivariate Data
• Networks & Graphs
• Trees & Hierarchies
• Time Series
• Geographical Data
• Text Data

Required reading (pre-class)

• Research directions in data wrangling: Visualizations and transformations for usable and credible data Kandel S, Heer J, Plaisant C, Kennedy J, Van Ham F, Riche NH, Weaver C, Lee B, Brodbeck D, Buono P. (2011)

In-class Agenda

• Aspect of Dirty Data - Identification and Curation
• Data Transformations - Making data usable and useful
• Visual Preprocessing - Examples of Vis tool usage in applications

Recommended reading (pre-class)

• no entry

In-class Agenda

• Introduction - About Patterns and Models
• Knowledge Generation - Patterns, Models, and Analytical Reasoning
• Data Transform. Processes - The InfoVis and the KDD Process
• Humans and Machines - Strengths and Weaknesses
• Visual Analytics - Synthesis

Further Reading

• Book: Illuminating the Path: The Research and Development Agenda for Visual Analytics. Thomas, J. and Cook, K. National Visualization and Analytics Center, 2005
• Paper:Visual Analytics: Definition, Process, and Challenges. D Keim, G Andrienko, JD Fekete, C Gorg, J Kohlhammer, G Melancon, 2008

Recommended reading (pre-class)

• VAD Book Chapter 13. Reduce Items and Attributes

In-class Agenda

• Unsupervised ML - The two "Work Horses"
• Clustering - Finding Groups in Datasets
• Dimensionality Reduction - Reducing the Number of Attributes
• Visual Data Exploration - Using Self-Organizing Maps (SOM)

Further Reading

• Wang Q, Chen Z, Wang Y, Qu H. A Survey on ML4VIS: Applying Machine Learning Advances to Data Vis. TVCG. 2021
• Sedlmair, Aupetit: Data-driven Evaluation of Visual Quality Measures. EuroVis. 2015
• Bernard, Hutter, Zeppelzauer, Sedlmair, Munzner: ProSeCo: Visual Analysis of Class Separation Measures and Dataset Characteristics. G&C (2021)

Recommended reading (pre-class)

• Hohman F, Kahng M, Pienta R, Chau DH. Visual analytics in deep learning: An interrogative survey for the next frontiers. IEEE Transactions on Visualization and Computer Graphics (TVCG). 2018.

In-class Agenda

• Supervised ML - The two "Work Horses"
• Explainable AI Special - Why something happens in ML Models
• Interactive ML Application - VIAL: Visual Interactive Data Labeling

Further Reading

• Bernard J, Zeppelzauer M, Lehmann M, Müller M, Sedlmair M. Towards User‐Centered Active Learning Algorithms. Computer Graphics Forum. 2018.
• Sacha D, Kraus M, Keim DA, Chen M. Vis4ml: An ontology for visual analytics assisted machine learning. IEEE transactions on visualization and computer graphics. 2018
• Amershi S, Cakmak M, Knox WB, Kulesza T. Power to the people: The role of humans in interactive machine learning. AI Magazine. 2014

In-class Agenda

• Introduction to HC-AI
• Ensuring human control while increasing automation
• Human-AI collaboration challenges
• "IVDA Supertools": Data Exploration, Model Explanation, and Knowledge Externalization
• Live Demo: Real-world application on sustainability research: SDG Research Scout

Further Reading

• B. Shneiderman: Human-centered artificial intelligence: Reliable, safe & trustworthy. International Journal of Human–Computer Interaction. 2020

• J. Bernard, M. Zeppelzauer, M. Sedlmair, and W. Aigner: VIAL – A Unified Process for Visual-Interactive Labeling. The Visual Computer Journal (TVCJ), 2018

In-class Agenda

• Basics of human knowledge externalization
• Formalization of (explicit) human feedback types
• Human-centered data labeling
• Live Application: Interactive Visual Labeling of Handwritten Digits
• LLM-based "Knowledge Externalization

Further Reading

• J Bernard, M Hutter, M Zeppelzauer, D Fellner, M Sedlmair: Comparing Visual-Interactive Labeling with Active Learning: An Experimental Study. IEEE Transactions on Visualization and Computer Graphics, 2018

• J Bernard, M Zeppelzauer, M Lehmann, M Müller, M Sedlmair: Towards User-Centered Active Learning Algorithms. Computer Graphics Forum (CGF), 2018

• A Visual Active Learning System for the Assessment of Patient Well-Being in Prostate Cancer Research. Bernard, J., Sessler, D., Bannach, A., May, T., Kohlhammer, J. IEEE VIS Workshop on Visual Analytics in Healthcare (VAHC), 2015

In-class Agenda

• Introduction to Preference-Based and Personalized Analytics
• Application & Example: Creation of a personalized music classifier

• Application & Example: Creation of a preference-based item ranking

• Application & Example: Creation of a similarity metric for countries

• Application & Example: Personalized analytics of Type-1-diabetes

Further Reading

• The human is the loop: new directions for visual analytics. Endert, A., Hossain, M. S., Ramakrishnan, N., North, C., Fiaux, P., Andrews, C. Journal of Intelligent Information Systems, 2014.

• Brown ET, Liu J, Brodley CE, Chang R. Dis-function: Learning distance functions interactively. IEEE conference on visual analytics science and technology (VAST). 2012

• User-Based Visual-Interactive Similarity Definition for Mixed Data Objects-Concept and First Implementation. Bernard, J., Sessler, D., Ruppert, T., Davey, J., Kuijper, A., Kohlhammer, J. WSCG. 2014

• Personalized Visual-Interactive Music Classification. Christian Ritter, Christian Altenhofen, Matthias Zeppelzauer, Arjan Kuijper, Tobias Schreck, and Jürgen Bernard, EuroVA @ EuroVis (EuroGraphics), 2018