ESR 13 – Nayee Muddin Khan Dousai (FER, Croatia)
“Deep-learning based detection of humans for search and rescue operations” (FER, Croatia)
About the project
The main objective of this research topic is to develop methods for detection of humans in search and rescue operations from aerial datasets. Investigate the integration of information obtained from images invisible, near-infrared and thermal spectral ranges sensed by multi-spectral cameras. Develop approaches for detecting humans to improve visualization quality under various harsh conditions such as low visibility, dust, snow, camouflage environment and other scenarios. Identify and exploit novel use scenarios of UAV imaging.
The expected outcomes will be to detect humans in aerial datatsets. A novel method is implemented for detecting humans on HERIDAL dataset based on deep-learning object detectors.
Objectives
The first objective of this project is to obtain datasets of images and videos acquired from unmanned aerial vehicles (UAV) in different scenarios related to rescue operations. The desired scenarios can be earthquakes,
fire accidents or harsh weather conditions.
The second objective is research and development of object detection methods and perform method validation using the UAV image datasets.
First Milestone
The first milestone, “Data Acquisition System using UAV” has been achieved by using the VisDrone dataset for testing the developed algorithms.
Second Milestone
The second milestone, “Preliminary algorithms for tracking objects in mixed visual data ready” has been successfully completed by implementing the novel object detection models based on EfficientDET architecture using the HERIDAL dataset.
Published Work
The work in progress has been presented at several conferences, workshops, and scientific journals:
1. Dousai, Nayee Muddin Khan, and Sven Lončarić, “Object Detection Challenges in Drone Images for safety – critical Applications”, ImmerSAFE special session at the 11th International Symposium on Image and Signal Processing and Analysis (ISPA 2019)
2. Dousai, Nayee Muddin Khan, and Sven Loncaric. “Detection of humans in drone images for search and rescue operations.” 2021 3rd Asia Pacific Information Technology Conference. pp. 69–75, 2021
3. Dousai, Nayee Muddin Khan, and Sven Lončarić. “Detecting Humans in Search and Rescue Operations Based on Ensemble Learning.” IEEE Access 10 (2022): 26481-26492
4. Dousai, Nayee Muddin Khan, and Sven Lončarić, “Object Detection Challenges in Drone Images for safety – critical Applications “, 4th International Workshop on Data Science (IWDS 2019)
5. Dousai, Nayee Muddin Khan, and Sven Lončarić, “Human detection in aerial images for search and rescue operations”, 6th International Workshop on Data Science (IWDS 2021)
6. Dousai, Nayee Muddin Khan, and Sven Lončarić, , “Human detection in aerial images for search and rescue operations”, 9th Croatian Computer Vision Workshop (CCVW 2021)
About the ESR
Mr. Nayeem holds a Bachelor of Technology (B.Tech.) in Electronics and Communication Engineering from India, as well as a Master of Science (MSc) in Computer Vision from the Universite de Bourgogne in France. He has experience working for international companies and has also been the co-founder and manager of an educational robotics company in India.
Watch my science slam video explaining the basics of object detection
Contact Information
Address: The University of Zagreb, Faculty of Electrical Engineering and Computing
Unska ul. 3, 10000 Zagreb, Croatia
Email: khan.nayeem78@gmail.com
This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 764951.