Skip to main content
Springer Nature Link
Log in

Markov Decision Processes and Reinforcement Learning for Timely UAV-IoT Data Collection Applications

  • Book
  • Aug 2025

Overview

Authors:
  1. Oluwatosin Ahmed Amodu

    1. Wireless Lab, Department of Electrical and Electronics Engineering, Faculty of Engineering and Built Environment, Universiti Kebangsaan Malaysia, Bangi Selangor, Malaysia

  2. Raja Azlina Raja Mahmood

    1. Department of Communication Technology and Network, Faculty of Computer Science and Information Technology, Universiti Putra Malaysia, Selangor , Malaysia

  3. Huda Althumali

    1. Computer Science Department, College of Science and Humanities, Imam Abdulrahman bin Faisal University, Jubail, Saudi Arabia

  4. Umar Ali Bukar

    1. Faculty of Information Science & Technology, Multimedia University, Jalan Ayer Keroh lama, Beruang, Malaysia

  5. Nor Fadzilah Abdullah

    1. Department of Electrical, Electronic & Systems Engineering, Faculty of Engineering & Built Environment, Universiti Kebangsaan Malaysia, Selangor, Malaysia

  6. Chedia Jarray

    1. Bimatech, Medenine , Tunisia

  • Introduces the concept of the Age of Information (AoI)
  • Presents the MDP framework for various problems and offers their mathematical representations
  • Discusses several algorithms designed to solve issues related to fresh data collection

Part of the book series: Studies in Computational Intelligence (SCI, volume 1220)

Buy print copy

Hardcover Book USD 169.99
Price excludes VAT (USA)
This title has not yet been released.You may pre-order it now and we will ship your order when it is published on 5 Sep 2025.
  • Durable hardcover edition
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

About this book

This book offers a structured exploration of how Markov Decision Processes (MDPs) and Deep Reinforcement Learning (DRL) can be used to model and optimize UAV-assisted Internet of Things (IoT) networks, with a focus on minimizing the Age of Information (AoI) during data collection. Adopting a tutorial-style approach, it bridges theoretical models and practical algorithms for real-time decision-making in tasks like UAV trajectory planning, sensor transmission scheduling, and energy-efficient data gathering. Applications span precision agriculture, environmental monitoring, smart cities, and emergency response, showcasing the adaptability of DRL in UAV-based IoT systems. Designed as a foundational reference, it is ideal for researchers and engineers aiming to deepen their understanding of adaptive UAV planning across diverse IoT applications.  

Keywords

  • Computational Intelligence
  • Age of information (AoI)
  • data acquisition
  • Deep Reinforcement Learning (DRL)
  • drones
  • energy-efficiency
  • Internet of Things (IoT)
  • Markov Decision Process
  • scheduling
  • trajectory
  • Unmanned Aerial Vehicles (UAVs)
  • wireless sensor networks (WSN)

Authors and Affiliations

  • Wireless Lab, Department of Electrical and Electronics Engineering, Faculty of Engineering and Built Environment, Universiti Kebangsaan Malaysia, Bangi Selangor, Malaysia

    Oluwatosin Ahmed Amodu

  • Department of Communication Technology and Network, Faculty of Computer Science and Information Technology, Universiti Putra Malaysia, Selangor , Malaysia

    Raja Azlina Raja Mahmood

  • Computer Science Department, College of Science and Humanities, Imam Abdulrahman bin Faisal University, Jubail, Saudi Arabia

    Huda Althumali

  • Faculty of Information Science & Technology, Multimedia University, Jalan Ayer Keroh lama, Beruang, Malaysia

    Umar Ali Bukar

  • Department of Electrical, Electronic & Systems Engineering, Faculty of Engineering & Built Environment, Universiti Kebangsaan Malaysia, Selangor, Malaysia

    Nor Fadzilah Abdullah

  • Bimatech, Medenine , Tunisia

    Chedia Jarray

Accessibility Information

PDF accessibility summary

This PDF has been created in accordance with the PDF/UA-1 standard to enhance accessibility, including screen reader support, described non-text content (images, graphs), bookmarks for easy navigation, keyboard-friendly links and forms and searchable, selectable text. We recognize the importance of accessibility, and we welcome queries about accessibility for any of our products. If you have a question or an access need, please get in touch with us at accessibilitysupport@springernature.com. Please note that a more accessible version of this eBook is available as ePub.

EPUB accessibility summary

This ebook is designed with accessibility in mind, aiming to meet the ePub Accessibility 1.0 AA and WCAG 2.2 Level AA standards. It features a navigable table of contents, structured headings, and alternative text for images, ensuring smooth, intuitive navigation and comprehension. The text is reflowable and resizable, with sufficient contrast. We recognize the importance of accessibility, and we welcome queries about accessibility for any of our products. If you have a question or an access need, please get in touch with us at accessibilitysupport@springernature.com.

Bibliographic Information

  • Book Title: Markov Decision Processes and Reinforcement Learning for Timely UAV-IoT Data Collection Applications

  • Authors: Oluwatosin Ahmed Amodu, Raja Azlina Raja Mahmood, Huda Althumali, Umar Ali Bukar, Nor Fadzilah Abdullah, Chedia Jarray

  • Series Title: Studies in Computational Intelligence

  • Publisher: Springer Cham

  • eBook Packages: Intelligent Technologies and Robotics, Intelligent Technologies and Robotics (R0)

  • Copyright Information: The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2025

  • Hardcover ISBN: 978-3-031-97010-8Due: 05 September 2025

  • Softcover ISBN: 978-3-031-97013-9Due: 05 September 2026

  • eBook ISBN: 978-3-031-97011-5Due: 05 September 2025

  • Series ISSN: 1860-949X

  • Series E-ISSN: 1860-9503

  • Edition Number: 1

  • Number of Pages: XI, 107

  • Number of Illustrations: 34 illustrations in colour

Publish with us

Policies and ethics

Back to top