Annotation of screencasts: Distinguishing Between Relevant and Irrelevant Sections

Tabea Ulm

Research output: ThesisMaster's Thesis

44 Downloads (Pure)


This thesis proposes a method to annotate screencasts, in order to identify sections of significance. The proposed approach quantifies the relevance frame by frame over the duration of the recording, making it easier for an external observer to navigate to sections of interest. Within this work, we implemented an approach for annotating screencasts of programming activities. Given a recording of screencasts only, the proposed method measures the amount of written code between each pair of subsequent frames. The approach is divided into three steps: extracting the code editor of a development environment, separating individual characters within those regions, and finally analyzing changes of those characters between subsequent frames. The detection of code editors is performed using computer vision methods that detect features characteristic for those regions. Character segmentation algorithms are then applied to the detected regions, in order to decide whether it contains a monospaced font, as this is a distinct attribute for fonts used in code editors. Changes in those characters are then analyzed, taking into account possible disturbances. The results were evaluated using 56 screencasts. The recordings originated from three different programming exercises, completed by 20 different students, each student using one of two development environments. The evaluation of those recordings result in a median accuracy of 83.4% with a median F2 score of 81.5%.
Translated title of the contributionAnnotation von Screencasts: Wie man in einer Computer-Aufzeichnung zwischen relevanten und irrelevanten Abschnitten unterscheidet
Original languageEnglish
Awarding Institution
  • Montanuniversität
  • Auer, Peter, Supervisor (internal)
Award date1 Jul 2022
Publication statusPublished - 2022

Bibliographical note

no embargo


  • Event Detection
  • Screencasts
  • Programming Activities
  • Computer Vision

Cite this