Seminar in Advanced Software Engineering, Fall 2018

Software requires constant evolution due to changing customer needs, bugs that have to be fixed or changes in the environment. This has been formulated in Lehman’s first law of software evolution, which states that a software system must be continuously adapted, or it becomes less and less useful. This constant change poses many challenges, for instance, on the reliability of the software as well as on the software developers that continuously have adapt. Both researchers and practitioners have recognized the importance to study and support software evolution and the humans involved in the process. In this seminar, we will cover some the most relevant studies, approaches and techniques that researchers have looked at in this context.

This course will be a combination of the traditional writing and presenting of a report on a chosen topic, as well as three sessions in the beginning of the term to discuss some research undertaken on each of the seminar topics. The three sessions will already cover research articles that can be used in the seminar report as well and should provide you a good start for writing the report. Short response papers for each of these sessions will also be required by each student to ensure the papers were read and stimulate an interesting discussion in class.

Learning objectives:

At the end of this course, students should be able to:

• find and identify relevant related work,
• reflect on a topic and discuss it,
• explain the selected subtopic and area of research,
• critically analyze and reflect on research articles (especially on the selected subtopic),
• review and summarize the current state of the art of the selected subtopic,
• reflect on the impact on the broader topic of the seminar and on software engineering in general,
• apply techniques and ideas found in the surveyed research to novel cases,
• reflect on possible future directions,
• provide constructive feedback on a research report

News:

(Sep 20) We were suffering from an extended downtime of our content management system and it was not possible to update the website. The system is back now and I just included the papers. Sorry for the delay and any inconvenience caused!

(Sep 20) Due to the high number of participants, we will change the format of the presentation day. Instead of an all-day event, we will do smaller sessions by topic and do a group discussion. You will find details in the kick-off slides.

Part 1: Group Discussions

For the first three weeks, we will meet and discuss two topics per week. For each week, students have to read the main paper of each topic (i.e. two papers for the two topics per class). Students will have to read the papers and write a short and concise response paper on the three papers (less than a page long!). In class, we will then discuss the research, opinions and reflections on the topics. This will provide students a good introduction to their selected topic, help to understand what is important in a paper, and what others think about the papers.

To practice literature research, students have to find a related paper for both topics and briefly elaborate in the response paper, why the suggested related work could be interesting in the context of the class. It is not necessary though to read that paper.

Part 2: Report on one of the topics, reviews and presentation of final report

Starting from the listed published research articles and the ones discussed in class, the students have to undertake a critical review of the topic assigned and write a report on it. The structure and content of this report is left open-ended (see details below). The report will then reviewed by the teaching assistant and 2-3 other students. Based on the feedback, a final submission will be created which will then be a part of the final grade.

Please note: All meetings and deadlines in this list are mandatory!

The extra papers are only listed to give you some context. Don't take one of these as the additional paper for the response papers.

Please note: To access the papers which are hosted on ACM or IEEE, you need a subscription, which is provided by the university. Download the papers when you are connected to the university Wi-Fi (UZH, eduroam doesn't work) or via VPN.

Topic 1: Recommendation Systems for Software Engineering

• Robbes and Lanza “How program history can improve code completion”, ASE, 2008.
• Extra: Liu et al. "Effective API Recommendation without Historical Software Repositories", ASE, 2018.

Topic 2: Naturalness of Software

• Hindle et al. On the naturalness of software", ICSE, 2012.
• Extra: Allamanis et al. Suggesting accurate method and class names", FSE, 2015

Topic 3: Search Based Software Engineering

• Harman and Jones "Search Based Software Engineering", IST, 2001.
• Extra: Fraser and Arcuri "Whole Test Suite Generation", TSE, 2012.

Topic 4: Software-Performance Measurements

• Kalibera and Jones "Rigorous Benchmarking in Reasonable Time", ISMM, 2013.
• Extra: Georges et al. "Statistically Rigorous Java Performance Evaluation", OOPSLA, 2007.

Topic 5: Computer-Supported Cooperative Work

• Meyer et al. "Retrospecting on Work and Productivity: A Study on Self-Monitoring Software Developers' Work", CSCW, 2017.
• Extra: Biehl et al. "FASTDash: a visual dashboard for fostering awareness in software teams", CHI, 2007.

Topic 6: Continuous Integration

• Hilton et al. "Trade-offs in continuous integration: assurance, security, and flexibility", FSE, 2017.
• Extra: Zhao et al. "The impact of continuous integration on other software development practices", ASE, 2017.

Additional topics

Due to the high number of participants, we extended the list of available topics. These topics are not going to be discussed in class, but can be picked for the report.

Topic 7: Code Review

• Bacchelli and Bird, "Expectations, Outcomes, and Challenges of Modern Code Review", ICSE, 2013.
• Extra: Thongtanunam et al. "Revisiting Code Ownership and its Relationship with Software Quality in the Scope of Modern Code Review", ICSE, 2016.

Topic 8: Release Engineering

• Zhang et al. "One Size Does Not Fit All: An Empirical Study of Containerized Continuous Deployment Workflows", FSE, 2018.
• Extra: Kevic et al. "Characterizing experimentation in continuous deployment: a case study on bing", ICSE, 2017.

Starting from the listed published research articles and the ones discussed in class, the students have to undertake a critical review of the topic assigned and write a report on it. The structure and content of this report is left open-ended, but should consider the points discussed in the kick-off slides.

The report has then to be submitted for reviewing through the seminar Easy Chair page (for more information please refer to the `Delivery' subsection of this page). The report will then go through a first review phase (blind review), done by the teaching assistant and 2-3 other students. Every participant has to review two to three other participants' reports. The goal of this first review is to provide and gather some useful feedback on the report, which should then be used to improve and modify the report accordingly and submit it for the second and final time. At the end, the participants will have to present their work on the presentation day.

The written report represents the second part of the seminar. It has to be 9-11 pages long (not counting the cover sheet, the table of contents, the reference list and the word of honor) and in the Lecture Notes in Computer Science format. Both the Microsoft Word ("word.zip") and LaTeX format ("llncs2e.zip") are available here for downloading, even though we strongly suggest anyone to use the LaTeX format. Eventually the report will have to be delivered as PDF on the submission site (not available, yet).

Each student should investigate and cite at least 7-11 papers from related work in the report in addition to the 2 papers provided by us.

Below you can find two very good reports from a previous seminar that you can use as blueprints while writing your report:

Report 1 (PDF, 543 KB)

Report 2 (PDF, 225 KB)

Do not forget the word of honor, declaring that you worked independently and did not plagiarize.

For any other question or if in doubt please contact the organizers.

The report of each student goes through a first review phase, done by 2-3 other students. The goal of this first reviews is to give some useful feedback on the report, which should then be improved and modified accordingly.

The reviews take the following criteria into account:

• Technical quality (critical analysis of the topic)
• Logical structure (the structuring of the work)
• Presentation (the use of graphics and tables)
• Style (orthography and typos)
• References (significant selection of relevant literature)

Grade each report according to one of the following options:

• accept
• weak accept
• weak reject
• reject

Every participant has to review two to three other participants' reports. The whole reviewing process (the reports and their subsequent reviews submission) will be done through the EasyChair online platform. An email with all the necessary instructions will be sent after the seminar kick-off.

Below you can find some examples of good reviews from previous years: Review 1, Review 2

For the final grade we will take the following aspects into account:

• The response papers and the active participation in the 3 discussions and the questions/discussions on the presentation day (20% of the grade)
• The reviewing of other students' reports (10% of the grade)
• The written report (50% of the grade)
• The presentation of the report (20% of the grade)

The following are grading criteria/guidelines for the final report and the presentation (this is not a complete list, rather a list of hints):

5.50 - 6.0 : An excellent work

• The report represents works of an exceptional standard. It is a highly articulate and professional document and includes critical comments with extended justification. The author demonstrates initiative and originality in analysis or interpretation.
• The presentation is comprehensive and highly professional. The discussion is integrated into a logical, coherent whole. It tells a story and leads logically into the research presented.

5.0 - 5.50 : A high quality work

• The report is of a superior standard. It is well written, free of errors and includes coherent critical comments with substantial justification. The author demonstrates complex, deep understanding of the subject.
• The presentation is effective and comprehensive.

4.0 - 5.0 : A good work with just a couple of small weaknesses

• The report is complete, well structured and well presented. It is written in a clear style that communicates points effectively on first reading. It includes coherent critical comments with justification based on evidence. The author has some tendency to summarize literature then develop an integrativ and logical argument.
• The presentation identifies and defines the main issues in a clear yet somewhat simplistic way.

3.0 - 4.0 : An average work with clear weaknesses

• The report addresses the main themes but does not integrate or relate key ideas and issues efficiently. It is presented in an organized manner but may have style and expression irregularities that do not intefere with the meaning. It provides some basic critical comments that reflect a basic understanding of key issues. However, the the minimal critical analysis attempted lacks in depth and/or is confused. The author demonstrates the the literature in the field has been checked.
• The presentation is essentially descriptive and features minimal critical analysis. The main themes are presented but not well integrated. The author fails to tell a story.

0 - 3.0 : Insufficient work with many substantial weaknesses

• The report does not address the main themes, is disorganized, incoherent or reveals evidence of plagiarism. It provides little or no critical comments.
• The presentation is disorganized, incoherent and fails to effectively describe the topic.