PhD Seminar: Qualitative Methods in Software Engineering, FS 11

Many important phenomena in the practice of software engineering resist reliable quantification. Human processes are particularly adverse to traditional measurement. What motivates developers to make certain design decisions? How do developers learn to integrate in a new environment? Why do developers use tools a certain way?

Software engineering researchers are increasingly turning to qualitative methods to answer questions about complex human behavior in software engineering.

This two-part seminar will provide a hands-on introduction to qualitative methods in software engineering.

The seminar will address the following topics, interleaving short presentations, discussions, and exercises.

• Research questions suited for qualitative methods.
• Types of qualitative methods.
• Qualitative data collection.
• Qualitative data analysis techniques.
• Quality and reliability in qualitative studies.
• Reporting on qualitative data.

The seminar will be split in 2 parts of 3 hours each.

Part I-A: - Overview

Presentation: Introduction to Qualitative Methods and Research Questions
Exercise: Writing Research Questions
Discussion: Research Questions

Break

Presentation: Data collection and analysis techniques
Discussion: Choosing a type of qualitative Study
Exercise: Designing a qualitative study
Discussion: Critique of qualitative studies. Validity and reliability.

Part II - Case Studies

Students will complete, in groups, the critique of a qualitative study chosen from a list of choices. The critique will include a description of the questions, methods, and data analysis techniques involved in the study. Groups will then present their report. A guided discussion of the trade-offs of each study design will follow.

There is a wealth of references of empirical methods for software engineering. For example, see Steve Easterbrook's collection. This seminar will mainly draw on two overview texts:

• Easterbrook et al. Selecting Empirical Methods for Software Engineering Research.
• Seaman, Qualitative Methods in Empirical Studies of Software Engineering, IEEE TSE 25(4):557-572, Jul/Aug 1999.

The following are examples of qualitative studies, for use in part II of the seminar.

1. Ko et al. Information Needs in Collocated Software Development Teams
2. Cherubini et al. Let's go to the whiteboard: how and why software developers use drawings
3. Dagenais et al. Moving into a New Software Project Landscape
4. Robillard and DeLine. A Field Study of API Learning Obstacles
5. Lutters and Seaman. Revealing actual documentation usage in software maintenance through war stories
6. de Souza and Redmiles. An Empirical Study of Software Developers’ Management of Dependencies and Changes