Module 1: Generative Language Models (3 days)

Day 1: Foundations of Language Models  

Instructor: Prof. Dr. Rico Sennrich

• Introduction to Theoretical Concepts

  • Purpose of text completion as a learning objective
  • Overview of neural networks and inference methods

• Text-Specific considerations

  • Approaches to tokenization and multilinguality

• Technical limitations and ethical issues

  • Factuality, bias, and detection of language-model-generated text

Day 2: Prompt Engineering and the LM Ecosystem 

Instructors: Dr. Jannis Vamvas & Dr. Simon Clematide

• Prompt Design Techniques

  • Strategies for zero-shot, few-shot, and complex workflow

• Hands-On Session

  • Engagement with open-source tools like Huggingface and Google Colab notebooks.
  • Develop ideas for open-source demo app

Day 3: Building LM-based Applications for Specialized Tasks 

Instructors: Dr. Jannis Vamvas & Dr. Simon Clematide

• Application Implementation

  • Building and presenting a demo app using the gradio framework.

• Model Optimization

  • Aligning and fine-tuning language models for specific tasks.

• Annotation and Generation

  • Semi-automatic annotation and explanation generation with generative AI.