This is Vera. She is currently living at Hannover Zoo. She is also the namesake of our project, as she was the first polar bear we tested our software on.
Student Theses
Over the course of the project, our students have contributed to investigating a range of research questions. Below is a list of all the theses completed as part of the VERA project. Interested in a more in-depth insight? We can provide access to individual theses on request and in consultation with the respective students. Please contact Matthias Zürl.
Lynx Re-Identification from Camera Trap Images in the Wild (2024)
Master’s Thesis in Medical Engineering
Submitted by: Sara Zarif
Advisors: Matthias Zürl, René Raab, Prof. Dr. Bjoern Eskofier
Monitoring of animals is one of the key aspects of making decisions in wildlife management. The ability to differentiate individual animals within the same species is important for accurate population analysis, tracking movements, and studying their behavior. Traditional monitoring methods, however, are time-consuming, often impractical for large-scale analysis, and can be invasive. The use of advanced and non-invasive technologies like camera traps has become a popular approach for animal monitoring. However, the manual labeling of captured images by ecologists and experts is a labor-intensive task. With the recent success of deep learning algorithms, there is potential to automate the re-identification (re-ID) of animals, making it applicable for largescale monitoring and behavioral analysis. In this thesis, the focus is on the re-identifying of lynx individuals. Due to the absence of publicly available lynx datasets, a dataset containing 53 lynx individuals has been created from raw camera trap images for re-ID. Given the distinct patterns on the left and right flanks of the lynx, four models were proposed to consider the viewpoint variations in the re-ID task. The models were also tested on a public dataset to assess their generalizability. The outcomes revealed a rank_1 accuracy of 74.6% for lynx re-identification. The results also demonstrated the models’ capability to re-identify new lynx individuals with a rank_5 accuracy of 83.8%. Furthermore, the impact of gallery size on the models’ performances was investigated. The findings of this thesis highlight the potential of automated animal re-identification, providing valuable support for ecologists in wildlife monitoring and analysis.
Creation and evaluation of a video-based polar bear Re-Identification (Re-ID) dataset (2023)
Master’s Thesis in Computer Science
Submitted by: Nils Steinlein
Advisors: Matthias Zürl, Richard Dirauf, Prof. Dr. Björn Eskofier
Zoological institutions face considerable challenges in monitoring the well-being of animals. Traditional observation methods are labour-intensive and do not allow for continuous observation. Video-based monitoring offers a potential solution by automating the observation of animals. However, algorithms face challenges in the Re-identification (Re-ID) of species that lack distinctive visual landmarks. Those animals would benefit from additional extracted features provided by temporal information from videos. Because the research on animal Re-ID currently focuses exclusively on image-based approaches, the Re-ID dataset PolarBearVidID was created in this thesis. We hope to encourage the development of video-based species-general animal Re-ID methods by providing a benchmark dataset for newly developed algorithms. The dataset includes 13 polar bear individuals from six zoos. With nearly 140,000 images, this represents, to the best of our knowledge, the first video-based and the most extensive animal Re-ID dataset in terms of image count. To prove the application, we evaluated the state-of-the-art transformer-based person Re-ID model Pyramid in Transformer (PiT) on the created dataset. With a rank-1 performance of 95.88 ± 1.45 % and a mean average precision (mAP) score of 81.62 ± 7.23 %, the adapted PiT model outperformed the image-based baseline approach while almost reaching the performance of the traditional Convolutional Neural Network (CNN)-based method Global-Local Temporal Representation (GLTR). The developed dataset PolarBearVidID and results from this thesis should provide a foundation for future work on animal Re-ID aiming to enhance animal welfare and conservation efforts.
Wing Print: Automated Bat Re-Identification Based On Distinct Wing Membrane Patterns (2023)
Master’s Thesis in Computer Science
Submitted by: Julian Deyerler
Advisors: Matthias Zürl, Philipp Schlieper, Dr. Ralph Simon, Prof. Dr. Björn Eskofier
Bats are essential to many ecosystems but hard to track individually. Traditional methods of marking individuals like forearm bands are highly invasive, inefficient, and affect the animal’s behavior. These issues could be solved by a re-identification (Re-ID) approach based on neural networks. In recent years, convolutional neural networks (CNNs) have already achieved very good results performing Re-ID in other animal species. A prerequisite is the presence of a unique biometric feature. In bats, candidates are the visible blood vessels and elastin bundle patterns inside and on the wing membranes, forming a potentially unique Wing Print. The external and internal structure of the wings can be captured by a camera in front of a backlight. The aim of this thesis was to test if a CNN-based machine learning model could successfully perform Re-ID on bats, based on their wing visuals. We examined three different scenarios: Re-ID within closed datasets, Re-ID of individuals across several datasets recorded on different nights, and prediction of individuals in unknown data. In cooperation with Nuremberg Zoo which houses a population of nectar-feeding bats, we developed a camera trap capable of capturing images of the bats’ wings and recorded three datasets that were then used to train and test multiple instances of a ResNet-50. Within the closed datasets, the model achieved Re-ID accuracies of 99.9 % (4 individuals), 99 % (15 individuals), and 99.75 % (23 individuals). The Re-ID of four reappearing individuals across all three datasets worked for 96.12 % of the input images. In predicting the number of individuals in unknown data, the model achieved over 91.25 % congruence with the ground truth, confirming that the Re-ID of bats based on their Wing Print is possible.
Video-based Deep Learning Approaches for Animal Behavior Classification (2023)
Master's Thesis in Computer Science
Submitted by: Jonas Süskind
Advisors: Matthias Zürl, René Raab, Prof. Dr. Björn Eskofier
Studying animals through scientific observation, termed ethology, is fundamental to biological
research. Through this, researchers can unravel biological processes and understand evolutionary
relations. Furthermore, scientific observation is crucial in ensuring animal welfare and
conservation efforts in enclosures and the wild. Animal observation is usually conducted
manually, severely limiting the extent of such studies. With the recent success of deep learning
systems, these tasks could be automated, leading to scalable, real-time, cheap opportunities for
improving animal welfare. Currently, research on automating behavior detection is impeded, as
appropriate datasets are expensive to create.
The contribution of this thesis is two-fold. First, a dataset containing video sequences of polar
bears, annotated with nine primary and 19 secondary behaviors, was created. On this and two
further datasets from the literature, one of mice and one of meerkats, four models are evaluated
for their classification performance. Each model focuses on different modalities of extracting
information from the video sequences.
The results show that the dataset quality is more important than the actual choice of model.
Furthermore, the best results could be achieved when additionally incorporating optical flow,
although only by a slight margin. Overall, a mean F1 score of up to 0.89 and a mean accuracy of
up to 0.94 is achieved. Without making significant concessions concerning accuracy, this thesis
demonstrates that such systems can perform real-time analysis even on devices with restricted
computational resources.
Video-based Re-Identification of Captive Polar Bears (2022)
Master’s Thesis in Computer Science
Submitted by: Richard Dirauf
Advisors: Matthias Zürl, Dr. Dario Zanca, Prof. Dr. Björn Eskofier
A scientific observational study that analyses an animal’s behavior is a commonly used method in animal behavioral research. Unfortunately, the process to gather data is a very time-consuming task for biologists. Therefore, automating data acquisition with non-invasive monitoring methods like video surveillance is an emerging research field. The main challenge is the identification of individual animals in the recorded videos, which can be tackled by re-identification (Re- ID) methods. The few existing research projects on animal Re-ID are limited to image-based approaches that mostly use species-specific body features. Also, there are not many animal Re-ID datasets and those that do exist are image-based and only from a few animal species. As a result, the scope of this thesis is to identify polar bears using a suitable video-based Re-ID method initially proposed for identifying persons. To accomplish this research goal, a video-based polar bear Re-ID dataset with 618 sequences is created that includes eight polar bears recorded at four zoos. The selected Re-ID approach called Global-Local Temporal Representation (GLTR) is trained and tested on this dataset. The results show that the polar bears from the dataset can be identified with a rank-1 accuracy of 81 % which is comparable to the performance GLTR achieves on the person Motion Analysis and Re-ID Set (MARS). Additionally, the Re-ID method can also identify new polar bears it was not trained on with a rank-1 accuracy of 71 %. These results should encourage biologists and computer scientists to further automate the behavioral analysis of animals.
Long-Term Automated Behaviour Monitoring of Captive Polar Bears (2022)
Master’s Thesis in Medical Engineering
Submitted by: Philip Stoll
Advisors: Matthias Zürl, René Raab, Prof. Dr. Björn Eskofier
One of the key responsibilities of any animal-keeping institution is to ensure the welfare
of their kept individuals. Therefore, indicators for poor animal welfare should be assessed
regularly and ideally monitored continuously. Although this may be possible with manual
observation methods on a small scale, it is very time-consuming for larger institutions like
zoos with hundreds of individuals.
This thesis proposes an automated pipeline for analysing the behaviour of the two polar bears
in the Nuremberg Zoo based on their movement patterns. The focus is on the quantification
of the occurrence of stereotypical behaviour (SB) long-term, which can give indications
toward the animal’s wellness. The pipeline is divided into three stages, the detection and
identification stage, trajectory processing with a custom Particle Filter (PF) implementation,
and behaviour classification with frequency-based features. Every stage is evaluated in
detail on bounding-box-annotated images of the two polar bears, synthetic trajectory data
with augmented errors and expert-annotated temporal behaviour data. The detection and
identification stages reach a mean Intersection over Union (IoU) of 0.762 and an F1@0.5 of
0.80 and the PF for trajectory processing a Mean Abolute Error (MAE) of 0.672 m and an
IoU of 97.1 %. The behaviour recognition stage achieves a micro IoU of 0.778 and a micro
F1-score of 0.875.
As a proof-of-concept for the processing time, the analysis of 477.8 h of recorded video data is
performed in ∼65.3 h. Compared to the approximated time of 477.8 h it would take a human
expert, the pipeline offers a speed-up of ∼730 % and also provides precise positions over
time, which has not been possible to this point.
Unsupervised Polar Bear Re-Identification (2022)
Master’s Thesis in Computer Science
Submitted by: Jonas Beyer
Advisors: Prof. Dr. Björn Eskofier, Matthias Zürl, Franz Köferl
The welfare of wild animals in zoos, such as polar bears, in an important topic. To understand what factors lead to living conditions that are appropriate for a given species, biologists would like to conduct observational studies over long periods of time. These are performed by hand and are labor-intensive. It would be ideal to be able to automate the observation process. Current approaches of doing so are based on deep learning techniques that require large amounts of labeled data. We have data of this form from four different polar bear enclosures. We present an alternative approach, based on ideas from Person Re-Identification, that reduces the amount of data that has to be collected and annotated for new enclosures.