A DSMS approach to support surveillance data based services in U-space

Daniel Pfisterer

Research output: ThesisMaster's Thesis

89 Downloads (Pure)

Abstract

The market for UAS (Unmanned Aerial System) holds a lot of potential for growth in the near future, both in industrial and consumer applications. With an increasing rate of adoption, standardized regulations and technical developments are crucial to enable safe UAS operations. One initiative that blends both is the definition of a U-space that provides digital services for a safe access to airspace. In order to provide these services in real-time, with 1Hz as reference value, the implementation of a prototype of a DSMS (Data Stream Management System) is proposed.
The developed DSMS is specified based on a combination of regulatory and problemspecific requirements. For defining the technical specifications the system is divided into three parts (ingestion, processing, and storing). According to the specifications a DSMS should be fault-tolerant and scalable while providing tools for stateful computations and SQL-compatibility. These specifications serve as basis for selecting the frameworks for the implementation of the prototype. For the data ingestion Apache Kafka is utilized, the stream processing is done with Kafka Streams, and all data is stored in a PostgreSQL database. Based on these frameworks the architecture of the whole system is designed. As Kafka is responsible for most of the data handling, designing the data flow within, from and to Kafka proved to be crucial for a successful implementation. To simplify testing and deployment of the prototype, all frameworks are implemented as containerized applications using Docker. Yet, even as containerized applications, applying these frameworks is not trivial. A smooth data exchange between the di!erent components, is only possible with consistent schema definitions. Coordinating the partitioning logic from Kafka with the stream processors for scalability requires careful adjustment of all parameters. Overcoming these challenges demands a deep understanding of the whole system with all its components and the interactions between them.
The finished DSMS prototype implements a selection of U-space services which are tested with a customized simulator. Although results have to be viewed with care, as all tests are conducted in a controlled environment, the results demonstrate the feasibility of using a DSMS to provide U-space services.
Translated title of the contributionEin DSMS-Konzept zur Unterstützung datengestützter Überwachungsdienste im U-Space
Original languageEnglish
QualificationDipl.-Ing.
Awarding Institution
  • Montanuniversität
Supervisors/Advisors
  • Ramos Gonzalez, Juan José , Supervisor (external), External person
  • Ortner, Ronald, Supervisor (internal)
Award date22 Mar 2024
DOIs
Publication statusPublished - 2024

Bibliographical note

no embargo

Keywords

  • U-space
  • Apache Kafka
  • DSMS
  • SQL
  • stream processing
  • Kafka Streams
  • data engineering
  • Docker
  • unbounded data
  • log-based message broker
  • stateful processing
  • exactly once
  • idempotence
  • PostgreSQL
  • Kafka Connect
  • Java
  • real-time processing

Cite this