Seminar in Advanced Software Engineering, FS 18

• The report has to be 9 to 11 pages long in the Lecture Notes in Computer Science format, which is a one column format (not counting cover sheet, table of contents, reference list and word of honour).
• You can now upload your reports to the submission system.
• Please make sure that you fill the questionnaire for the quality report of the lecture to help us to further improve the seminar in the future.
• Please note the updated schedule (final submission & presentations day)

Software requires constant evolution due to changing customer needs, bugs that have to be fixed or changes in the environment. This has been formulated in Lehman’s first law of software evolution, which states that a software system must be continuously adapted, or it becomes less and less useful. This constant change poses many challenges, for instance, on the reliability of the software as well as on the software developers that continuously have adapt. Both researchers and practitioners have recognized the importance to study and support software evolution and the humans involved in the process. In this seminar, we will cover some the most relevant studies, approaches and techniques that researchers have looked at in this context.

This course will be a combination of the traditional writing and presenting of a report on a chosen topic, as well as three sessions in the beginning of the term to discuss some research undertaken on each of the seminar topics. The three sessions will already cover research articles that can be used in the seminar report as well and should provide you a good start for writing the report. Short response papers for each of these sessions will also be required by each student to ensure the papers were read and stimulate an interesting discussion in class.

Learning objectives:

At the end of this course, students should be able to:

• find and identify relevant related work,
• reflect on a topic and discuss it,
• explain the selected subtopic and area of research,
• critically analyze and reflect on research articles (especially on the selected subtopic),
• review and summarize the current state of the art of the selected subtopic,
• reflect on the impact on the broader topic of the seminar and on software engineering in general,
• apply techniques and ideas found in the surveyed research to novel cases,
• reflect on possible future directions,
• provide constructive feedback on a research report

Part 1: Group Discussions

For the first three weeks, we will meet and discuss two topics per week. For each week, students have to read the main paper of each topic (i.e. two papers for the two topics per class) and find additional papers that is related to the topics. Students will have to read the papers and write a short and concise response paper on the three papers (less than a page long!). In class, we will then discuss the research, opinions and reflections on the topics. This will provide students a good introduction to their selected topic and help to understand what is important in a paper, what others think about the papers, and how to find relevant related work.

Part 2: Report on one of the topics, reviews and presentation of final report

Starting from the listed published research articles and the ones discussed in class, the students have to undertake a critical review of the topic assigned and write a report on it. The structure and content of this report is left open-ended (see details below). The report will then reviewed by the teaching assistant and 2-3 other students. Based on the feedback, a final submission will be created which will then be a part of the final grade.

Please note: All meetings and deadlines in this list are mandatory!

The extra papers are only listed to give you some context. Don't take one of these as the additional paper for the response papers.

Please note: To access the papers which are hosted on ACM or IEEE, you need a subscription, which is provided by the university. Download the papers when you are connected to the university Wi-Fi (UZH, eduroam doesn't work) or via VPN.

Topic 1: Sensing Developers

• Müller et al. Using Bio(Metrics) to Predict Code Quality Online, ICSE'16
• Extra: Siegmund et al. Measuring Neural Efficiency of Program Comprehension, FSE'17

Topic 2: Testing

• Inozemtseva et al. Coverage is not strongly correlated with test suite effectiveness, ICSE'14
• Extra: Xuan et al. Test Case Purification for Improving Fault Localization, FSE'14

Topic 3: Recommendation Systems for Software Engineering

• Ponzanelli et al. Supporting Software Developers with a Holistic Recommender System, ICSE'17
• Extra: Nguyen et al. API code recommendation using statistical learning from fine-grained changes, ASE'16

Topic 4: Source-Code Evolution

• Negara et al. Is it dangerous to use version control histories to study source code evolution?, ECOOP'12
• Extra: Barik et al. Commit Bubbles, ICSE'15

Topic 5: Continuous Integration

• Vassallo et al. Continuous Delivery Practices in a Large Financial Organization, ICSME'16
• Extra: Kerzazi et al. Why Do Automated Builds Break? An Empirical Study, ICSME'14

Topic 6: Development Work in Practice

• Sanchez et al. An empirical study of work fragmentation in software evolution tasks, SANER'15
• Extra: Meyer et al. The work life of developers: Activities, switches and perceived productivity, TSE'17

Each student writes three response papers in this course. The response paper is a short and concise (no more than 1 page) reflection of the main paper for each topic and one additional paper, which fits the given context of research. The response paper should be send as a PDF via email to Thomas Fritz and Sebastian Proksch. You will find the deadlines in the course schedule.

Please read the following three excellent response papers from previous years: Example 1, Example 2, Example 3

The ACM Digital Library, IEEE Digital Library, Citeseer and GoogleScholar are very good online catalogues for technical literature search and can be useful for finding related work. Use them to find the additional paper for your three response papers in part 1, as well as finding relevant papers for part 2. Both the ACM and the IEEE publications can be downloaded for free, as long as you are connected to the UZH network.

A good starting point are the proceedings of major SE conferences, such as:

• International Conference in Software Engineering (ICSE)
• Symposium on the Foundations of Software Engineering (FSE)
• Conference on Human Factors in Computing Systems (CHI)
• International Conference on Automated Software Engineering (ASE)
• Working Conference on Mining Software Repositories (MSR)
• International Conference on Program Comprehension (ICPC)
• International Conference on Software Maintenance and Evolution (ICSME)

Finally, the provided papers often cite relevant related work in the references section.

Starting from the listed published research articles and the ones discussed in class, the students have to undertake a critical review of the topic assigned and write a report on it. The structure and content of this report is left open-ended, however the students, need to make sure they:

• clearly explain the topic selected,
• review and summarize the current state of the art,
• critically analyze and reflect on research articles they present, also reporting on their commonalities and differences,
• reflect on the impact of the topic they studied on the broader theme of the seminar and on software engineering in general,
• elaborate on the application of the techniques and ideas found in the surveyed research to novel cases, topics, etc.,
• reflect on possible future directions.

The report has then to be submitted for reviewing through the seminar Easy Chair page (for more information please refer to the `Delivery' subsection of this page). The report will then go through a first review phase (blind review), done by the teaching assistant and 2-3 other students. Every participant has to review two to three other participants' reports. The goal of this first review is to provide and gather some useful feedback on the report, which should then be used to improve and modify the report accordingly and submit it for the second and final time. At the end, the participants will have to present their work on the presentation day.

The written report represents the second part of the seminar. It has to be 9-11 pages long (not counting the cover sheet, the table of contents, the reference list and the word of honor) and in the Lecture Notes in Computer Science format. Both the Microsoft Word ("word.zip") and LaTeX format ("llncs2e.zip") are available here for downloading, even though we strongly suggest anyone to use the LaTeX format. Eventually the report will have to be delivered as PDF on the submission site.

Each student should investigate and cite at least 7-11 papers from related work in the report in addition to the 2 papers provided by us.

Please pay attention to the tips in the format template, in particular:

• the correct structure of a scientific research article,
• the right use of figures and tables,
• the correct citation of other works.

Below you can find two very good reports from a previous seminar that you can use as blueprints while writing your report:

Report 1 (PDF, 543 KB)

Report 2 (PDF, 225 KB)

Do not forget the word of honor, declaring that you worked independently and did not plagiarize.

For any other question or if in doubt please contact the organizers.

The report of each student goes through a first review phase, done by 2-3 other students. The goal of this first reviews is to give some useful feedback on the report, which should then be improved and modified accordingly.

The reviews take the following criteria into account:

• Technical quality (critical analysis of the topic)
• Logical structure (the structuring of the work)
• Presentation (the use of graphics and tables)
• Style (orthography and typos)
• References (significant selection of relevant literature)

Grade each report according to one of the following options:

• accept
• weak accept
• weak reject
• reject

Every participant has to review two to three other participants' reports. The whole reviewing process (the reports and their subsequent reviews submission) will be done through the EasyChair online platform. An email with all the necessary instructions will be sent after the seminar kick-off.

Below you can find some examples of good reviews from previous years: Review 1, Review 2

• All the participants will present their work on a special presentation day (see schedule).
• Every presentation consists of a 10-13 minutes talk (enforced!) followed by a 5-10 minutes discussion (so 15-20min per student, max!).
• All participants need to have the presentation in electronic format (Microsoft PowerPoint, Apple Keynote or slides in PDF). We provide both a beamer and a laptop to be used (in case you don't want to use your own laptop). Test the presentation on the beamer in advance!
• Attendance to all the presentations is mandatory. Exceptions are made only if a proper, official excuse (e.g. doctor's excuse note) is given.

Presentation Guidelines

• Make sure that your talk includes enough background material and motivation so that it can be understood by those who are not specialists in what is being presented.
• Design your slides using large text and diagrams such that they can be read by anyone in the room. The person far away in the corner may be the one with central insights to your work, so make sure he/she gets the message, too.
• Do not place text on the slides that you will speak aloud anyway. The audience will read the text faster than you speak, and continuously experience lags. Slides and spoken word should support each other, and not be redundant representations of each other.
• Modern presentation programs can switch the laptop screen in presenter mode, showing the upcoming slides or presenter notes. Make use of these tools to ensure fluid transitions between slides and a smooth information flow.
• Spend at least one slide to clearly motivate your work, explicitly stating the problem you are addressing.
• Spend at least one slide on related work, such that the audience can place your work in a greater context.
• Do not end with a slide saying nothing but "Thank you!" or "Questions?" Instead, on the last slide, summarize your talk (problem, approach, results). Have this slide stay on the screen during the discussion; people will be able to recall your talk and provide insightful questions and comments.
• Presentations are allotted 25 minutes: 15 minutes for talk, 10 minutes for questions & answers. These limits will be strictly enforced.

For the final grade we will take the following aspects into account:

• The response papers and the active participation in the 3 discussions and the questions/discussions on the presentation day (20% of the grade)
• The reviewing of other students' reports (10% of the grade)
• The written report (50% of the grade)
• The presentation of the report (20% of the grade)

The following are grading criteria/guidelines for the final report and the presentation (this is not a complete list, rather a list of hints):

5.50 - 6.0 : An excellent work

• The report represents works of an exceptional standard. It is a highly articulate and professional document and includes critical comments with extended justification. The author demonstrates initiative and originality in analysis or interpretation.
• The presentation is comprehensive and highly professional. The discussion is integrated into a logical, coherent whole. It tells a story and leads logically into the research presented.

5.0 - 5.50 : A high quality work

• The report is of a superior standard. It is well written, free of errors and includes coherent critical comments with substantial justification. The author demonstrates complex, deep understanding of the subject.
• The presentation is effective and comprehensive.

4.0 - 5.0 : A good work with just a couple of small weaknesses

• The report is complete, well structured and well presented. It is written in a clear style that communicates points effectively on first reading. It includes coherent critical comments with justification based on evidence. The author has some tendency to summarize literature then develop an integrativ and logical argument.
• The presentation identifies and defines the main issues in a clear yet somewhat simplistic way.

3.0 - 4.0 : An average work with clear weaknesses

• The report addresses the main themes but does not integrate or relate key ideas and issues efficiently. It is presented in an organized manner but may have style and expression irregularities that do not intefere with the meaning. It provides some basic critical comments that reflect a basic understanding of key issues. However, the the minimal critical analysis attempted lacks in depth and/or is confused. The author demonstrates the the literature in the field has been checked.
• The presentation is essentially descriptive and features minimal critical analysis. The main themes are presented but not well integrated. The author fails to tell a story.

0 - 3.0 : Insufficient work with many substantial weaknesses

• The report does not address the main themes, is disorganized, incoherent or reveals evidence of plagiarism. It provides little or no critical comments.
• The presentation is disorganized, incoherent and fails to effectively describe the topic.