Jump to
MODULE 06 // ML FOUNDATIONS FOR UAV

Machine Learning Foundations for UAV

Traditional control theory assumes you can write down the physics of the system mathematically. ML asks a different question: what if you could learn the right behaviour from data and experience, without needing to model every physical detail? This module lays the mathematical foundations β€” supervised learning, reinforcement learning, optimisation theory, and the critically important sim-to-real gap β€” that everything in Modules 07 and 08 builds on.

Supervised Learning Reinforcement Learning Optimisation Theory Sim-to-Real Transfer

πŸ“š  REFERENCES: MDPI Autonomous Control Ch.4–5  Β·  Bestaoui Sebbane Β§9  Β·  Virell Β§4–5  Β·  Sadraey Β§7

6.1 Why Machine Learning for UAV Systems?

Plain English First

The GNC techniques in Modules 02–05 are powerful β€” but they all assume you can write down equations that describe the system's behaviour. For a fixed-wing UAV in calm air, that works well. But what about a multirotor in a dust storm, a swarm coordinating against an adversary that adapts its tactics, or a UAV that must identify a camouflaged target from visual data? Writing explicit equations for all of these is either impossible or impractical. Machine learning is the alternative: instead of hand-designing the decision rule, you expose the system to data or experience and let it learn the rule itself.

Bestaoui Sebbane identifies four specific failure modes of classical GNC that ML addresses: Bestaoui Sebbane Β§9.1

Classical GNC LimitationML AlternativeUAV Application
Model uncertainty β€” real aerodynamics deviate from equations Learn a residual correction from flight data Adaptive attitude control under icing, prop damage, or payload shift
Cannot process raw sensor data (images, point clouds) Deep neural networks as perceptual front-ends Target detection from EO/IR, terrain classification from LiDAR
Fixed control laws β€” cannot adapt to novel environments Reinforcement learning learns policies through interaction Obstacle avoidance in unknown environments, adversarial manoeuvring
Combinatorial planning β€” too many decision variables Learned heuristics / value functions approximate optimal decisions Multi-UAV task allocation, mission replanning under attrition
ML Is Not a Replacement β€” It Is a Complement In certified military systems, ML is typically used where classical methods fail (perception, environment modelling, adaptive compensation), while classical GNC handles safety-critical loops (attitude stabilisation, collision avoidance interlocks). The reason: classical controllers have formal stability guarantees provable from Lyapunov theory (Module 03). ML systems currently lack equivalent certification pathways under DO-178C / MIL-STD-882 (Module 10). Understanding this boundary is as important as understanding the ML methods themselves. MDPI Autonomous Control Ch.4

6.2 The Three Learning Paradigms

Plain English First

All of machine learning organises into three fundamental paradigms based on what kind of feedback the learning algorithm receives. Supervised learning β€” a teacher shows you labelled examples ("this image contains a tank"). Unsupervised learning β€” you find patterns in data with no labels. Reinforcement learning β€” you learn by trying things and receiving rewards or penalties. For UAVs: supervised learning handles perception tasks (what do I see?), unsupervised handles data organisation (what types of terrain are there?), and reinforcement learning handles action/decision tasks (what should I do next?).

PARADIGM 01
Supervised Learning
Given a dataset of input–output pairs $\{(\mathbf{x}_i, y_i)\}_{i=1}^N$, learn a function $f: \mathbf{x} \mapsto y$ that generalises to unseen inputs. The training signal is an explicit label for every example.
UAV use cases: Object detection (YOLO), optical flow estimation, depth estimation, terrain classification, target identification from EO/IR. Requires large labelled datasets β€” expensive to collect in military contexts.
PARADIGM 02
Unsupervised Learning
Given data $\{\mathbf{x}_i\}_{i=1}^N$ with no labels, discover structure: clusters, manifolds, generative models. The training signal comes from the data distribution itself β€” not from external labels.
UAV use cases: Anomaly detection (sensor fault isolation), terrain segmentation without labels, building maps from LiDAR point clouds, compressing high-dimensional sensor data for bandwidth-limited links.
PARADIGM 03
Reinforcement Learning
An agent observes state $s$, takes action $a$, receives reward $r$, transitions to state $s'$. Learns a policy $\pi: s \mapsto a$ maximising cumulative reward. No explicit label β€” only scalar reward signal.
UAV use cases: Autonomous navigation, evasive manoeuvring, swarm coordination, mission planning. Can discover non-intuitive strategies that human designers would not think to hand-code.

The Supervised Learning Setup β€” Formalised

For a UAV perception task, supervised learning involves finding a function $f_\theta$ parameterised by weights $\theta$ that minimises a loss function: MDPI Autonomous Control Ch.4

Empirical Risk Minimisation $$\hat{\theta} = \arg\min_\theta \mathcal{L}(\theta) = \arg\min_\theta \frac{1}{N}\sum_{i=1}^N \ell\!\left(f_\theta(\mathbf{x}_i),\, y_i\right)$$

$\ell(\hat{y}, y)$: loss function β€” measures how wrong the prediction is. Common choices: mean squared error $\ell = (\hat{y}-y)^2$ (regression), cross-entropy $\ell = -y\log\hat{y}$ (classification). $\mathcal{L}(\theta)$: empirical risk β€” average loss over training data. The goal is to minimise this, which in practice is done with gradient descent.

Critical subtlety: We minimise loss on training data, but we care about performance on unseen test data. These are different quantities β€” the gap between them is called generalisation error, and understanding it is the central challenge of machine learning theory.

6.3 Optimisation Theory β€” Training Neural Networks

Plain English First

Training a neural network means finding the weights $\theta$ that minimise the loss function. This is a high-dimensional optimisation problem β€” a typical neural network might have millions of parameters. You cannot try all combinations; you must follow the gradient downhill. Gradient descent is the workhorse algorithm, but the loss landscape is full of traps: flat regions where progress stalls, saddle points, and sharp narrow minima that do not generalise well to new data. Understanding the optimisation process is as important as understanding the network architecture.

Analogy β€” Hiking Down a Foggy Mountain

You are blindfolded on a mountainous landscape (the loss surface). You want to reach the lowest valley (minimum loss). You can only feel the slope under your feet (the gradient). Gradient descent takes a step downhill at each moment. The step size (learning rate) matters enormously β€” too large and you overshoot valleys; too small and you take forever, or get stuck in a local dip. Momentum carries you through shallow flat regions. Adam combines both ideas.

Gradient Descent and Its Variants

Gradient Descent Family $$\text{Batch GD:}\quad \theta_{t+1} = \theta_t - \eta \nabla_\theta \mathcal{L}(\theta_t)$$ $$\text{SGD:}\quad \theta_{t+1} = \theta_t - \eta \nabla_\theta \ell(f_\theta(\mathbf{x}_i), y_i)$$ $$\text{Mini-batch SGD:}\quad \theta_{t+1} = \theta_t - \frac{\eta}{B}\sum_{i \in \mathcal{B}} \nabla_\theta \ell(f_\theta(\mathbf{x}_i), y_i)$$

$\eta$: learning rate; $\mathcal{B}$: mini-batch of size $B$. Batch GD: exact gradient, expensive per step. SGD: noisy gradient from one sample, cheap per step but high variance. Mini-batch SGD: balance β€” the universal practical choice. Batch size $B = 32$–$256$ is typical for UAV perception tasks. MDPI Autonomous Control Ch.4

Adam Optimiser β€” The Practical Default

Pure SGD requires careful learning rate tuning. Adam (Adaptive Moment Estimation) adapts the learning rate for each parameter individually using running estimates of gradient mean and variance: Bestaoui Sebbane Β§9.2

Adam Update Rule $$m_t = \beta_1 m_{t-1} + (1-\beta_1) g_t \qquad \text{(1st moment β€” gradient mean)}$$ $$v_t = \beta_2 v_{t-1} + (1-\beta_2) g_t^2 \qquad \text{(2nd moment β€” gradient variance)}$$ $$\hat{m}_t = \frac{m_t}{1-\beta_1^t}, \quad \hat{v}_t = \frac{v_t}{1-\beta_2^t} \qquad \text{(bias correction)}$$ $$\theta_{t+1} = \theta_t - \frac{\eta}{\sqrt{\hat{v}_t} + \epsilon}\,\hat{m}_t$$

Default hyperparameters: $\beta_1 = 0.9$, $\beta_2 = 0.999$, $\epsilon = 10^{-8}$, $\eta = 10^{-3}$. Why this works better than SGD: Parameters with consistently large gradients get a smaller effective learning rate (auto-normalised by $\sqrt{\hat{v}_t}$). Parameters with small, variable gradients get a larger effective step. This prevents the common problem of some weights updating too fast while others stagnate β€” critical for deep networks where gradient magnitudes vary by orders of magnitude across layers.

Backpropagation β€” How Gradients Flow

For a neural network $f_\theta$ with $L$ layers, computing $\nabla_\theta \mathcal{L}$ requires applying the chain rule through every layer. This is the backpropagation algorithm: MDPI Autonomous Control Ch.4

Backpropagation β€” Chain Rule Applied Layer by Layer $$\frac{\partial \mathcal{L}}{\partial W^{(l)}} = \frac{\partial \mathcal{L}}{\partial \mathbf{a}^{(L)}} \cdot \frac{\partial \mathbf{a}^{(L)}}{\partial \mathbf{a}^{(L-1)}} \cdots \frac{\partial \mathbf{a}^{(l+1)}}{\partial \mathbf{a}^{(l)}} \cdot \frac{\partial \mathbf{a}^{(l)}}{\partial W^{(l)}}$$ $$\boldsymbol{\delta}^{(l)} = \left(W^{(l+1)T}\boldsymbol{\delta}^{(l+1)}\right) \odot \sigma'\!\left(\mathbf{z}^{(l)}\right)$$

$\boldsymbol{\delta}^{(l)}$: error signal (gradient) at layer $l$; $\sigma'$: derivative of activation function; $\odot$: element-wise product (Hadamard product). Computational cost of backprop is approximately 2Γ— the forward pass β€” making training roughly 3Γ— the cost of inference. On UAV embedded hardware (NVIDIA Jetson Orin, 15W), this means inference is feasible at 30 Hz; online training is generally not β€” models are trained offline and deployed.

Worked Example 6.1 β€” Gradient Flow in a 3-Layer UAV Classifier

A 3-layer network classifies UAV sensor readings as "threat / no threat." Compute the gradient of loss with respect to layer 1 weights, tracing backprop. Use sigmoid activations $\sigma(z) = 1/(1+e^{-z})$.

Suppose forward pass gives: $z^{(1)} = 0.5$, $a^{(1)} = \sigma(0.5) = 0.622$. Layer 2: $z^{(2)} = 0.8$, $a^{(2)} = 0.690$. Output: $\hat{y} = 0.690$, true label $y = 1.0$. Loss: $\mathcal{L} = -(y\log\hat{y}) = -\log(0.690) = 0.371$.

Backward pass: $\delta^{(3)} = \hat{y} - y = 0.690 - 1.0 = -0.310$ (output gradient for cross-entropy + sigmoid)

$\delta^{(2)} = \delta^{(3)} \cdot W^{(3)} \cdot \sigma'(z^{(2)})$ where $\sigma'(0.8) = 0.690 \times (1-0.690) = 0.214$ $= -0.310 \times W^{(3)} \times 0.214$

$\delta^{(1)} = \delta^{(2)} \cdot W^{(2)} \cdot \sigma'(z^{(1)})$ where $\sigma'(0.5) = 0.622 \times (1-0.622) = 0.235$

Key observation: Each backward step multiplies by $\sigma'(\cdot)$. For sigmoid, $\sigma'(z) \leq 0.25$ always. In a deep network (10+ layers), $0.25^{10} \approx 10^{-6}$ β€” the gradient essentially vanishes. This is the vanishing gradient problem that makes deep sigmoid networks untrainable. ReLU activations ($\sigma'(z) = 1$ for $z > 0$) solve this β€” which is why every modern UAV perception network uses ReLU, not sigmoid. MDPI Autonomous Control Ch.4

6.4 Generalisation Theory β€” When Does a Model Transfer?

Plain English First

A model that works perfectly on its training data but fails on real flight data is useless β€” and dangerous. This failure is called overfitting: the model has memorised the training examples rather than learned a general rule. In military UAV applications, the stakes of overfitting are extremely high β€” a target detection system trained on desert imagery that fails in jungle terrain can cost lives. Understanding when and why models generalise, and how to improve generalisation, is not optional theory. It is safety engineering.

The Bias–Variance Trade-off

Bias–Variance Decomposition of Expected Test Error $$\mathbb{E}\!\left[(\hat{y} - y)^2\right] = \underbrace{\text{Bias}^2(\hat{y})}_{\text{systematic error}} + \underbrace{\text{Var}(\hat{y})}_{\text{sensitivity to data}} + \underbrace{\sigma^2_\epsilon}_{\text{irreducible noise}}$$

Bias: How wrong is the model on average? A model that always predicts "no target" in urban terrain has high bias. Variance: How much does the prediction change with different training sets? A very deep network that memorises its training data has high variance. Trade-off: Simpler models have high bias, low variance. Complex models have low bias, high variance. The optimum lies between β€” neither too simple (underfitting) nor too complex (overfitting).

Theorem 6.1 β€” PAC Learning Bound (Probably Approximately Correct)

For a hypothesis class $\mathcal{H}$ of finite size $|\mathcal{H}|$, with probability at least $1-\delta$ over a training set of $N$ i.i.d. samples, every hypothesis $h \in \mathcal{H}$ satisfies:

$$\mathcal{L}_{test}(h) \leq \mathcal{L}_{train}(h) + \sqrt{\frac{\ln|\mathcal{H}| + \ln(1/\delta)}{2N}}$$

Reading this in plain English: Test error is bounded by training error plus a complexity penalty. The penalty grows with model complexity ($\ln|\mathcal{H}|$) and shrinks with more data ($1/\sqrt{N}$).

UAV operational implication: For a target detection model with $10^6$ parameters and only 500 labelled examples ($N = 500$), $\delta = 0.05$: the generalisation gap bound $\approx \sqrt{(10^6 \ln 10 + \ln 20)/1000} \approx 151$. This is vacuously large β€” the bound gives no useful guarantee. In practice this means: 500 training images is nowhere near enough to train a reliable million-parameter network from scratch. You must use transfer learning from large pre-trained models. MDPI Autonomous Control Ch.5

Regularisation Techniques for UAV Models

Since we can rarely collect enough data to make the PAC bound tight, regularisation artificially constrains model complexity to improve generalisation: Bestaoui Sebbane Β§9.3

TechniqueMechanismMathematical FormUAV Use Case
L2 (Weight Decay) Penalise large weights β€” keeps model "simple" $\mathcal{L}_{reg} = \mathcal{L} + \lambda\|\theta\|_2^2$ All deep learning models; default in most UAV papers
Dropout Randomly zero out neurons during training β€” forces redundancy $\tilde{h}_i = h_i \cdot \text{Bernoulli}(p)$ Fully-connected layers in perception networks
Batch Normalisation Normalise layer inputs β€” stabilises and regularises implicitly $\hat{x} = (x - \mu_\mathcal{B})/\sigma_\mathcal{B}$ CNNs for EO/IR β€” improves training speed and stability
Data Augmentation Artificially expand training set with transforms Random flip, crop, colour jitter, rotation Critical for UAV β€” accounts for altitude variation, viewing angle, lighting
Early Stopping Halt training when validation loss stops improving Monitor $\mathcal{L}_{val}$; stop when $\Delta\mathcal{L}_{val} < \epsilon$ All UAV models β€” prevents memorising training flight conditions

6.5 Reinforcement Learning for UAV Autonomy

Plain English First

In reinforcement learning, the UAV learns by doing. It takes actions, receives a reward (or penalty), and gradually learns which actions lead to higher cumulative reward. Unlike supervised learning, there are no labelled examples telling it what the right action was β€” only the reward signal, which might be delayed by many steps. A UAV learning to navigate through obstacles doesn't get told "turn left here" β€” it gets a reward of +1 for reaching the goal and -100 for crashing, and must figure out the connecting chain of good actions by itself.

The Markov Decision Process (MDP) Framework

Every RL problem for UAVs is cast as a Markov Decision Process (MDP): MDPI Autonomous Control Ch.5

MDP Tuple and Bellman Equation $$\mathcal{M} = (\mathcal{S},\, \mathcal{A},\, P,\, R,\, \gamma)$$ $$V^\pi(s) = \mathbb{E}_\pi\!\left[\sum_{t=0}^\infty \gamma^t r_t \,\Big|\, s_0 = s\right]$$ $$Q^\pi(s,a) = R(s,a) + \gamma \sum_{s'} P(s'|s,a)\, V^\pi(s')$$

$\mathcal{S}$: state space (UAV position, velocity, sensor readings); $\mathcal{A}$: action space (thrust commands, waypoint selection); $P(s'|s,a)$: transition probability (how the environment responds); $R(s,a)$: immediate reward; $\gamma \in [0,1)$: discount factor. $V^\pi(s)$: value function β€” expected total future reward from state $s$ following policy $\pi$. $Q^\pi(s,a)$: action-value function β€” expected total reward taking action $a$ in state $s$. The optimal policy selects: $\pi^*(s) = \arg\max_a Q^*(s,a)$.

Reward Shaping for UAV Tasks

Designing the reward function is the most consequential engineering decision in UAV RL. A poorly designed reward leads to unexpected behaviours β€” the agent finds ways to maximise the numerical reward without accomplishing the actual mission objective. This is called reward hacking. Bestaoui Sebbane Β§9.4

UAV TaskNaive RewardProblemBetter Reward Design
Reach waypoint $+1$ at goal, $0$ otherwise Sparse reward β€” agent wanders randomly for epochs before first success Dense shaping: $r = -\|\mathbf{p} - \mathbf{p}_{goal}\|_2$ at each step
Avoid obstacles $-100$ on collision Agent learns to hover motionlessly to avoid all risk $r = -\|\mathbf{p} - \mathbf{p}_{goal}\|_2 - \lambda\cdot\mathbf{1}[\text{collision}]$
Energy efficiency Time to goal Agent reaches goal fast but with excessive thrust (battery drain) $r = r_{progress} - \alpha\|\mathbf{u}\|^2$ (penalise control effort)
ISR coverage Area covered Agent revisits same areas repeatedly for easy coverage credit $r = \text{new area covered only}$ (information gain reward)

Key RL Algorithms for UAVs

Proximal Policy Optimisation (PPO) β€” The UAV RL Standard $$\mathcal{L}^{PPO}(\theta) = \mathbb{E}_t\!\left[\min\!\left(r_t(\theta)\hat{A}_t,\;\text{clip}(r_t(\theta), 1-\epsilon, 1+\epsilon)\hat{A}_t\right)\right]$$

$r_t(\theta) = \pi_\theta(a_t|s_t)/\pi_{\theta_{old}}(a_t|s_t)$: probability ratio between new and old policy. $\hat{A}_t$: advantage estimate β€” how much better was action $a_t$ vs. the average? The clipping prevents the policy from changing too much in one step, which is the key stability improvement over vanilla policy gradient. PPO is the dominant algorithm for UAV navigation RL because it is stable, sample-efficient, and works well with continuous action spaces (motor commands). MDPI Autonomous Control Ch.5

Worked Example 6.2 β€” Designing an MDP for Quadrotor Navigation

Fully specify an MDP for training a quadrotor to navigate through a GPS-denied building interior, reaching a target room while avoiding walls.

MDP ComponentSpecificationEngineering Rationale
State $s$$[x,y,z,\phi,\theta,\psi,\dot{x},\dot{y},\dot{z},d_{front},d_{left},d_{right},d_{up},d_{down}]^T \in \mathbb{R}^{14}$Position, attitude, velocity + 5 sonar distances
Action $a$$[\delta T, \delta\phi_{ref}, \delta\theta_{ref}, \delta\psi_{ref}] \in [-1,1]^4$Normalised thrust and attitude setpoints β€” inner loop handles actual control
Transition $P$Gazebo physics simulator (Module 06 sim-to-real section)Real physics without hardware risk
Reward $R$$r = -0.1\|\mathbf{p}-\mathbf{p}_{goal}\|_2 - 0.01\|\mathbf{u}\|^2 - 50\cdot\mathbf{1}[\text{collision}] + 100\cdot\mathbf{1}[\text{goal reached}]$Dense progress + control efficiency + terminal reward/penalty
Discount $\gamma$0.99Long-horizon task (200+ timesteps to reach goal)
Episode length500 steps at 10 Hz = 50 s maxLong enough to reach goal; short enough to learn from many episodes

Training setup: 16 parallel Gazebo instances (each a random room layout). PPO with 2048 steps per rollout, 10 epochs, batch size 64. After ~5 million environment steps (~8 hours on a GPU server), the policy achieves >90% goal-reaching rate on unseen room configurations. This policy is then the candidate for sim-to-real transfer.

6.6 The Sim-to-Real Gap β€” The Critical Transfer Problem

Plain English First

Training RL policies in simulation is cheap and safe β€” you can crash the UAV thousands of times without damaging real hardware. But the simulation is never a perfect copy of reality. The real UAV has motor delays the simulator didn't model. Real air has turbulence that the simulator's aerodynamic model simplified away. The real camera has lens distortion and rolling shutter effects absent in the rendered simulation. When you take a policy trained entirely in simulation and deploy it on real hardware, it often fails β€” sometimes immediately, sometimes subtly. This is the sim-to-real gap, and bridging it is one of the hardest open problems in autonomous UAV systems.

Sources of the Sim-to-Real Gap

Gap SourceDescriptionMagnitudeMitigation
Aerodynamic modelling Simplified drag, ground effect, blade flapping ignored 5–15% thrust error System identification from flight data; add aerodynamic noise in sim
Motor/ESC dynamics Simulator uses ideal instantaneous thrust; real motors have 20–50 ms lag 20–50 ms delay Add motor time constant ($\tau_{motor}$) to simulator model
Sensor noise model Simulated IMU uses Gaussian noise; real IMU has correlated noise, spikes 2–5Γ— real noise level Record and replay real sensor noise in sim; domain randomisation
Visual appearance Simulated textures, lighting, shadows differ from real world Large domain shift Photorealistic rendering (Unreal Engine); domain randomisation
Payload / CoM shift Real payloads shift centre of mass; sim uses fixed rigid body 1–5% inertia error Randomise inertia parameters; robust control inner loop

Domain Randomisation β€” The Primary Mitigation Strategy

Domain randomisation deliberately introduces variety into the simulation so that the real world is just one of many environments the policy has already seen. If the policy works across a wide range of simulated conditions, it is more likely to work in reality: Virell Β§4.3

Domain Randomisation Training Objective $$\pi^* = \arg\max_\pi \mathbb{E}_{\xi \sim p(\Xi)}\left[\mathbb{E}_\pi\left[\sum_{t=0}^\infty \gamma^t r_t \,\Big|\, \mathcal{M}_\xi\right]\right]$$

$\xi$: simulation parameter vector (motor constants, aerodynamic coefficients, sensor noise levels, visual textures). $p(\Xi)$: distribution over simulation parameters. $\mathcal{M}_\xi$: MDP with parameter setting $\xi$. Training maximises expected reward across the distribution of simulated environments, not just a single fixed simulation. The real world is then treated as an out-of-distribution sample from $p(\Xi)$ that the policy must generalise to.

Theorem 6.2 β€” Sim-to-Real Bound via Domain Shift

Let $\pi$ be a policy trained in simulation domain $\mathcal{D}_{sim}$ and deployed in real domain $\mathcal{D}_{real}$. The performance gap is bounded by:

$$J_{real}(\pi) \geq J_{sim}(\pi) - \frac{2\gamma}{(1-\gamma)^2}\,d_{TV}(\mathcal{D}_{sim},\, \mathcal{D}_{real})\cdot R_{max}$$

$d_{TV}$: total variation distance between the state-action distributions in simulation and reality; $R_{max}$: maximum reward magnitude; $\gamma$: discount factor. Implication: Real-world performance is guaranteed to be within a computable margin of simulation performance β€” but only if we can bound the domain shift $d_{TV}$. Domain randomisation reduces $d_{TV}$ by widening the training distribution to include the real world as a likely sample. Virell Β§4.4

Progressive Narrowing β€” Safe Sim-to-Real Deployment

Virell documents the standard industry pipeline for responsible sim-to-real transfer in UAV systems: Virell Β§4.5

  1. Train in high-fidelity sim (Gazebo, AirSim, Isaac Gym) with domain randomisation
  2. Validate in multiple simulators β€” if policy works across different physics engines, it is more robust
  3. Hardware-in-the-loop (HIL) β€” run real FCC (PX4) with simulated sensors; test C2 latency and real-time constraints
  4. Tethered flight β€” first real flights with safety tether; correct systematic errors before untethered
  5. Untethered in controlled environment β€” indoor arena with safety nets; measure performance gap
  6. Progressive outdoor deployment β€” calm conditions first, then wind, then full operational envelope

6.7 UAV Datasets β€” Supervised Learning Foundations

Plain English First

Supervised learning requires labelled data. For UAV perception tasks, collecting and labelling training data is expensive, slow, and operationally sensitive β€” you cannot simply fly over an active military target area and label images. Understanding which datasets exist, their composition, their limitations, and how to augment them synthetically is an essential practical skill for anyone building UAV ML systems.

DatasetTaskSizePlatformLimitation
VisDroneObject detection from UAV10,209 images, 2.5M bboxesDJI consumer UAVCivilian objects only; limited occlusion
UAVDTVehicle detection/tracking80K framesAerial fixed platformUrban scenes; no camouflage/military vehicles
DroneDeployAerial segmentation55 kmΒ² aerial imageryVarious UAVsBuilding inspection; no tactical environments
Mid-AirOptical flow, depth, ego-motion420K stereo framesSynthetic (Unreal)Synthetic domain β€” sim-to-real transfer required
TartanAirVisual odometry, depth1M+ frames, 16 environmentsSynthetic (AirSim)Synthetic; strong domain randomisation built in
USAF DARPA-sourcedMilitary target detectionClassified / restrictedMilitary platformsAccess restricted; SAR + EO/IR

Synthetic Data Generation for Military UAV Tasks

Where real data is scarce or classified, synthetic data from photorealistic simulators (Unreal Engine, NVIDIA Isaac) fills the gap. The pipeline: generate synthetic images with ground truth labels β†’ train on synthetic + fine-tune on small real dataset (domain adaptation): Virell Β§5.2

Domain Adaptation Loss (DANN approach) $$\mathcal{L}_{total} = \mathcal{L}_{task}(\theta_f, \theta_c) - \lambda\,\mathcal{L}_{domain}(\theta_f, \theta_d)$$

$\mathcal{L}_{task}$: supervised task loss on labelled synthetic data. $\mathcal{L}_{domain}$: domain classifier loss β€” can a discriminator tell if a feature came from synthetic or real data? The gradient reversal on $\mathcal{L}_{domain}$ forces the feature extractor $\theta_f$ to produce domain-invariant representations β€” features that look the same whether the input is synthetic or real. When this works, the task classifier trained on synthetic features transfers to real images.

6.8 Python: Neural Network from Scratch + PPO RL Agent

What This Code Covers Part 1: A neural network built from scratch using only NumPy β€” forward pass, backprop, and training on a UAV threat classification task. Implements the vanishing gradient demonstration from Worked Example 6.1. Part 2: A minimal PPO agent skeleton applied to a 1D UAV altitude hold task, showing how the MDP from Worked Example 6.2 translates to code. Run with: pip install numpy
Python
# ml_foundations_uav.py
# ─────────────────────────────────────────────────────────────────
# Part 1: Neural network from scratch (NumPy only)
#         Implements: MDPI Autonomous Control Ch.4
# Part 2: Minimal PPO skeleton for UAV altitude RL
#         Implements: MDPI Autonomous Control Ch.5
# ─────────────────────────────────────────────────────────────────

import numpy as np


# ════════════════════════════════════════════════════════════════
# PART 1 β€” Neural Network from Scratch
# Classifies UAV sensor readings as "threat zone" / "safe zone"
# ════════════════════════════════════════════════════════════════

class Layer:
    """Single fully-connected layer with ReLU activation."""
    def __init__(self, n_in, n_out, activation='relu'):
        # He initialisation β€” scales weights for ReLU to prevent vanishing/exploding gradients
        self.W = np.random.randn(n_in, n_out) * np.sqrt(2.0 / n_in)
        self.b = np.zeros(n_out)
        self.activation = activation
        # Cache for backprop
        self.x_cache  = None
        self.z_cache  = None
        self.dW = np.zeros_like(self.W)
        self.db = np.zeros_like(self.b)

    def forward(self, x):
        self.x_cache = x
        z = x @ self.W + self.b
        self.z_cache = z
        if self.activation == 'relu':
            return np.maximum(0, z)
        elif self.activation == 'sigmoid':
            return 1 / (1 + np.exp(-np.clip(z, -500, 500)))
        elif self.activation == 'linear':
            return z
        raise ValueError(f"Unknown activation: {self.activation}")

    def backward(self, d_out):
        """Backprop through this layer. Returns gradient w.r.t. input."""
        if self.activation == 'relu':
            d_z = d_out * (self.z_cache > 0)   # ReLU gradient: 1 if z>0 else 0
        elif self.activation == 'sigmoid':
            a = 1 / (1 + np.exp(-self.z_cache))
            d_z = d_out * a * (1 - a)           # sigmoid gradient β€” small near saturation
        elif self.activation == 'linear':
            d_z = d_out                            # identity gradient
        self.dW = self.x_cache.T @ d_z
        self.db = d_z.sum(axis=0)
        return d_z @ self.W.T


class NeuralNet:
    """
    Feedforward neural network for UAV threat classification.
    Architecture: input β†’ hidden β†’ hidden β†’ sigmoid output
    """
    def __init__(self, layer_sizes, activations):
        self.layers = [
            Layer(layer_sizes[i], layer_sizes[i+1], activations[i])
            for i in range(len(activations))
        ]

    def forward(self, x):
        for layer in self.layers:
            x = layer.forward(x)
        return x

    def backward(self, d_loss):
        for layer in reversed(self.layers):
            d_loss = layer.backward(d_loss)

    def update(self, lr, weight_decay=1e-4):
        """SGD update with L2 weight decay (Bestaoui Sebbane Β§9.3)."""
        for layer in self.layers:
            layer.W -= lr * (layer.dW + weight_decay * layer.W)
            layer.b -= lr * layer.db


def generate_uav_threat_data(N=1000):
    """
    Synthetic UAV threat classification dataset.
    Features: [altitude, speed, radar_cross_section, heading_change_rate]
    Label: 1 = threat (incoming fast, low, maneuvering), 0 = safe
    """
    np.random.seed(42)
    X = np.random.randn(N, 4)
    # Threat: low altitude (x[0]<0), high speed (x[1]>0.5), low RCS (x[2]<0), maneuvering (|x[3]|>0.5)
    y = ((X[:, 0] < 0) & (X[:, 1] > 0.5) & (X[:, 2] < 0)).astype(float)
    y = y.reshape(-1, 1)
    return X[:800], y[:800], X[800:], y[800:]   # 80/20 train/test split


def train_threat_classifier():
    X_train, y_train, X_test, y_test = generate_uav_threat_data()

    # Build network: 4 inputs β†’ 16 β†’ 8 β†’ 1 output (sigmoid)
    net = NeuralNet(
        layer_sizes  = [4, 16, 8, 1],
        activations  = ['relu', 'relu', 'sigmoid'],
    )

    lr = 0.01; epochs = 100; batch_size = 32

    print(f"\n{'='*54}")
    print(f"  UAV THREAT CLASSIFIER β€” TRAINING")
    print(f"  MDPI Autonomous Control Ch.4")
    print(f"{'='*54}")

    for epoch in range(epochs):
        # Mini-batch SGD
        idx = np.random.permutation(len(X_train))
        epoch_loss = 0

        for start in range(0, len(X_train), batch_size):
            batch_idx = idx[start:start+batch_size]
            xb, yb = X_train[batch_idx], y_train[batch_idx]

            # Forward pass
            y_hat = net.forward(xb)

            # Binary cross-entropy loss
            y_hat = np.clip(y_hat, 1e-7, 1-1e-7)
            loss  = -np.mean(yb*np.log(y_hat) + (1-yb)*np.log(1-y_hat))
            epoch_loss += loss

            # Backward pass β€” gradient of BCE + sigmoid at output
            d_loss = (y_hat - yb) / len(xb)
            net.backward(d_loss)
            net.update(lr)

        # Print every 20 epochs
        if (epoch+1) % 20 == 0:
            y_pred = (net.forward(X_test) > 0.5).astype(float)
            acc = np.mean(y_pred == y_test) * 100
            print(f"  Epoch {epoch+1:3d} | Loss: {epoch_loss:.3f} | Test Acc: {acc:.1f}%")

    print()


# ════════════════════════════════════════════════════════════════
# PART 2 β€” Minimal PPO Agent for UAV Altitude Hold
# Demonstrates the MDP β†’ code translation from Worked Example 6.2
# ════════════════════════════════════════════════════════════════

class AltitudeEnv:
    """
    1D UAV altitude hold environment.
    State: [altitude_error, vertical_velocity]
    Action: normalised thrust offset in [-1, 1]
    Reward: -|alt_error| - 0.01*action^2
    """
    TARGET_ALT = 50.0
    DT         = 0.05   # 20 Hz

    def reset(self):
        self.alt = self.TARGET_ALT + np.random.randn() * 5
        self.vel = np.random.randn() * 0.5
        return self._obs()

    def _obs(self):
        return np.array([self.alt - self.TARGET_ALT, self.vel])

    def step(self, action):
        # Simple point-mass model: accel = (action * max_thrust - gravity_offset)
        accel = action * 4.0 - 0.0   # gravity already balanced at hover
        self.vel += accel * self.DT
        self.vel  = np.clip(self.vel, -5, 5)
        self.alt += self.vel * self.DT
        self.alt  = np.clip(self.alt, 0, 100)
        error = abs(self.alt - self.TARGET_ALT)
        reward = -error - 0.01 * action**2
        done   = error > 30   # episode ends if altitude error too large
        return self._obs(), reward, done


class PolicyNetwork:
    """Linear policy for 2D state β†’ 1D continuous action (Gaussian)."""
    def __init__(self):
        self.W    = np.random.randn(2) * 0.1
        self.log_std = np.array([-1.0])   # learnable log standard deviation

    def get_action(self, obs):
        """Sample action from Gaussian policy N(WΒ·obs, exp(log_std))."""
        mu  = self.W @ obs
        std = np.exp(self.log_std[0])
        action = mu + std * np.random.randn()
        action = np.clip(action, -1, 1)
        log_prob = -0.5*((action-mu)/std)**2 - np.log(std) - 0.5*np.log(2*np.pi)
        return action, log_prob


def train_ppo_altitude(n_episodes=200):
    """
    Simplified policy gradient (REINFORCE, precursor to PPO) for altitude hold.
    Demonstrates the RL training loop structure used in full PPO implementations.
    MDPI Autonomous Control Ch.5
    """
    env    = AltitudeEnv()
    policy = PolicyNetwork()
    lr     = 0.02
    gamma  = 0.99

    print(f"\n{'='*54}")
    print(f"  PPO-STYLE RL β€” UAV ALTITUDE HOLD")
    print(f"  MDPI Autonomous Control Ch.5")
    print(f"{'='*54}")

    for ep in range(n_episodes):
        obs = env.reset()
        rewards, log_probs, observations = [], [], []

        # Rollout one episode
        for _ in range(200):
            action, lp = policy.get_action(obs)
            next_obs, r, done = env.step(action)
            rewards.append(r); log_probs.append(lp); observations.append(obs)
            obs = next_obs
            if done: break

        # Compute discounted returns G_t = Ξ£ Ξ³^k r_{t+k}
        G = 0; returns = []
        for r in reversed(rewards):
            G = r + gamma * G
            returns.insert(0, G)
        returns = np.array(returns)
        returns = (returns - returns.mean()) / (returns.std() + 1e-8)   # normalise

        # Policy gradient update: βˆ‡J β‰ˆ Ξ£ G_t * βˆ‡ log Ο€(a_t|s_t)
        grad_W = np.zeros_like(policy.W)
        for t in range(len(rewards)):
            grad_W += returns[t] * log_probs[t] * observations[t]
        policy.W += lr * grad_W / len(rewards)

        # Print progress
        if (ep+1) % 50 == 0:
            mean_r = np.mean(rewards)
            final_err = abs(obs[0])
            print(f"  Ep {ep+1:3d} | Mean reward: {mean_r:6.2f} | Final alt err: {final_err:.2f} m | Policy W: {policy.W}")

    print(f"\n  β†’ Policy converges to negative altitude-error weights")
    print(f"    (fly up when too low, fly down when too high)")
    print(f"{'='*54}\n")


if __name__ == "__main__":
    np.random.seed(0)
    train_threat_classifier()
    train_ppo_altitude(n_episodes=300)
Expected Output Threat classifier: Epoch 20 loss ~0.5, Epoch 100 test accuracy ~88–92% Β· RL altitude agent: Episode 50 mean reward ~βˆ’5 (unstable), Episode 300 mean reward ~βˆ’1.5 (converging) Β· Final policy weights converge to $W \approx [-0.8,\, -0.3]$: strong correction toward zero error, damping on velocity β€” essentially learning a PD controller from scratch through RL.
  1. The PAC bound in Theorem 6.1 gives a generalisation gap of approximately 151 for a 1M-parameter network with 500 training examples. Yet in practice, researchers routinely fine-tune large pre-trained models with only 200–500 UAV images and achieve good performance. Explain the two reasons the PAC bound is overly pessimistic for deep networks in this regime. What does this imply about why transfer learning from pre-trained models is so important for military UAV perception systems where labelled data is scarce?
  2. Domain randomisation (Theorem 6.2) bounds the sim-to-real gap via the total variation distance $d_{TV}$ between simulation and real distributions. Describe two concrete scenarios where domain randomisation would fail to reduce this gap adequately β€” i.e., the real world would still be far from any simulated variant. For each scenario, propose a complementary technique that addresses the failure mode.
  3. Run the Python code from section 6.8 (RL altitude hold). After training converges, examine the final policy weights $W$. Interpret these weights in terms of classical PD control theory β€” what gains do they correspond to? For what class of problems would a linear RL policy like this be insufficient, and what architectural change to the policy network would address that limitation?
β—‚ Module 05: Sensor Fusion Module 07: Deep Learning β–Έ