A UAV that cannot interpret what its sensors see is blind. Deep learning gives UAVs the ability to extract meaning from raw images, point clouds, and video streams โ identifying targets, estimating depth, tracking motion, and understanding the 3D structure of the environment. This module covers CNNs, object detection, Vision Transformers, optical flow, and depth estimation โ with a sharp focus on what works in the constrained compute and harsh conditions of military platforms.
๐ REFERENCES: MDPI Autonomous Control Ch.5โ6 ยท Bestaoui Sebbane ยง9โ10 ยท Virell ยง5โ6 ยท Sadraey ยง7
A camera mounted on a UAV produces a stream of numbers โ pixel brightness values. Nothing in those numbers directly says "this is a vehicle" or "that is a person." Turning raw pixel data into useful information โ locations of targets, distances to obstacles, velocity of moving objects โ is the perception problem. It is the bridge between sensing the world and acting in it. Before deep learning, this required hand-crafting features (edge detectors, histogram gradients) that were brittle, slow, and needed re-engineering for every new environment. Deep learning learns those features automatically from data, and the resulting systems are both more accurate and more adaptable.
The perception stack for a military UAV involves several distinct tasks, each addressed by a different deep learning architecture: MDPI Autonomous Control Ch.5
| Perception Task | Input | Output | Architecture | UAV Mission Use |
|---|---|---|---|---|
| Image classification | Single image | Class label | CNN / ViT | Scene type (urban/forest/water) |
| Object detection | Image | Bounding boxes + class | YOLO, DETR | Target/vehicle/person identification |
| Semantic segmentation | Image | Per-pixel class label | U-Net, DeepLab | Terrain mapping, landing zone assessment |
| Optical flow | Image pair | Per-pixel velocity field | FlowNet, RAFT | Ego-motion estimation, moving target detection |
| Monocular depth | Single image | Per-pixel depth map | MonoDepth2, DPT | Obstacle distance, terrain reconstruction |
| Pose estimation | Image | 6-DOF camera pose | PoseNet, SuperGlue | Visual localisation, landing on moving platforms |
A Convolutional Neural Network (CNN) processes images by sliding small filters (kernels) across the image and detecting local patterns โ edges, corners, textures, then shapes, then objects at increasing abstraction. Think of it like looking at an image through increasingly complex magnifying glasses: the first glass sees edges, the second sees shapes made of edges, the third sees objects made of shapes. This hierarchical feature extraction is what makes CNNs so powerful for vision โ and it is the reason every modern UAV perception system is built on a CNN backbone or its attention-based successor, the Vision Transformer.
A 2D convolution slides a kernel $\mathbf{k}$ across an input feature map $\mathbf{x}$ to produce an output feature map: MDPI Autonomous Control Ch.5
$K$: kernel size (typically 3 or 5); $[i,j]$: output pixel location. For an input of size $H \times W$ with a $K \times K$ kernel, stride $s$, and padding $p$, the output size is:
$$H_{out} = \left\lfloor\frac{H - K + 2p}{s}\right\rfloor + 1, \qquad W_{out} = \left\lfloor\frac{W - K + 2p}{s}\right\rfloor + 1$$With padding $p = (K-1)/2$ ("same" padding), output size equals input size. With stride $s = 2$ (strided convolution), output halves in size โ used for downsampling instead of max pooling in modern architectures. Number of learnable parameters per conv layer: $(K^2 \cdot C_{in} + 1) \times C_{out}$, where $C_{in}, C_{out}$ are input/output channel counts and $+1$ is the bias term.
Compare ResNet-50 vs MobileNetV3-Small as backbones for a UAV real-time detection system. Target hardware: NVIDIA Jetson Orin Nano (15W, ~40 TOPS INT8).
| Backbone | Parameters | GFLOPs (224ร224) | ImageNet Top-1 | Jetson Orin Nano FPS | Verdict |
|---|---|---|---|---|---|
| ResNet-50 | 25.6 M | 4.1 | 76.1% | ~18 FPS (FP16) | Accurate, below 30 Hz requirement |
| ResNet-18 | 11.7 M | 1.8 | 69.8% | ~40 FPS (FP16) | Fast enough, accuracy borderline |
| MobileNetV3-Small | 2.5 M | 0.06 | 67.4% | ~150 FPS (FP16) | Fast, moderate accuracy โ good for real-time |
| EfficientNet-B0 | 5.3 M | 0.39 | 77.1% | ~65 FPS (FP16) | Best accuracy/speed trade-off for UAV use |
| YOLOv8n (full detector) | 3.2 M | 8.7 | โ | ~45 FPS (FP16) | Recommended: end-to-end detection pipeline |
Engineering decision: For a 30 Hz real-time detection requirement at 640ร640 input resolution, YOLOv8n or EfficientDet-D0 are the practical choices on a Jetson Orin Nano. ResNet-50 is the right choice only if the platform has a full Jetson AGX Orin (275 TOPS) and accuracy is paramount (e.g., ISR target identification with low false-alarm requirement). Always profile on target hardware โ benchmark numbers in papers often assume A100 GPUs. Virell ยง5.3
Object detection solves two problems at once: where is the object in the image (a bounding box), and what is it (a class label). For a military UAV, this might mean detecting vehicles, personnel, weapons systems, or infrastructure in real-time aerial imagery. The challenge is that objects of interest can be tiny (10โ20 pixels from altitude), partially occluded, camouflaged, or appear at any orientation โ none of which standard ground-level detectors handle well.
| Paradigm | Mechanism | Speed | Accuracy | UAV Use |
|---|---|---|---|---|
| Two-stage (R-CNN family) | Stage 1: propose regions. Stage 2: classify + refine each region | ~5โ15 FPS | High mAP | Offline ISR analysis; not real-time |
| One-stage (YOLO family) | Single network predicts all boxes and classes simultaneously | 30โ150+ FPS | Competitive mAP | Real-time onboard detection |
| Transformer (DETR family) | Attention-based end-to-end detection; no NMS post-processing | ~15โ30 FPS | State of the art | High-accuracy ISR on capable hardware |
YOLO frames detection as a regression problem: a single CNN produces a grid of predictions directly. MDPI Autonomous Control Ch.5
$G \times G$: grid size (e.g., 20ร20 at the smallest feature map scale); $c_x, c_y$: grid cell offset; $p_w, p_h$: anchor box prior dimensions. $\sigma(t_x), \sigma(t_y)$: sigmoid constrains box centre within cell. $p_{obj}$: objectness score โ is there an object here at all? $p_c$: class probabilities. Final confidence = $p_{obj} \times p_c$. Multi-scale prediction (3 grid scales in YOLOv8) handles objects at different sizes.
$\mathcal{L}_{IoU}$: Intersection-over-Union loss โ directly optimises box overlap quality. $\mathcal{L}_{BCE,cls}$: binary cross-entropy for each class independently (allows multi-label). $\mathcal{L}_{DFL}$: Distribution Focal Loss โ predicts a distribution over box edge locations rather than a single value, improving localisation precision for small targets. Default weights: $\lambda_{box} = 7.5$, $\lambda_{cls} = 0.5$, $\lambda_{dfl} = 1.5$. Bestaoui Sebbane ยง9.6
A detection is a True Positive if IoU with a ground-truth box exceeds 0.5 (COCO uses 0.5โ0.95). mAP@0.5:0.95 is the standard COCO benchmark โ average AP over 10 IoU thresholds. For military UAV applications, a high-recall / moderate-precision operating point is often preferred (miss nothing, accept some false alarms) vs. the reverse โ this is controlled by the confidence threshold.
An MQ-9 carries an EO sensor at 7,620 m altitude with GSD = 24 cm/pixel (from Module 01). A standard military vehicle (Toyota Hilux technical) is 5.3 m long ร 1.8 m wide. How many pixels does it occupy? What does this imply for detector design?
Vehicle dimensions in pixels: Length: $5.3 / 0.24 \approx 22$ px. Width: $1.8 / 0.24 \approx 8$ px. Bounding box area: $22 \times 8 = 176$ pxยฒ.
COCO "small object" threshold: area $< 32^2 = 1024$ pxยฒ. At 176 pxยฒ, this vehicle falls into the "small" category โ the hardest regime for all detectors. YOLOv8 achieves mAP@0.5 of ~54% on COCO small objects vs ~82% on large objects.
Mitigations for small-object detection: (1) Higher input resolution (1280ร1280 instead of 640ร640) โ doubles compute but improves small AP by ~8%. (2) SAHI (Sliced Aided Hyper Inference) โ run detector on overlapping tiles, merge results. (3) Feature Pyramid Network with higher-resolution P2 feature map. (4) Fine-tune on aerial vehicle dataset (VisDrone, UAVDT) โ domain adaptation from COCO. At lower altitudes (2,000 m), GSD โ 6 cm/pixel, vehicle = 88ร30 px โ "medium" object, standard YOLO achieves ~72% mAP. Mission altitude is a key perception design parameter. Bestaoui Sebbane ยง9.7
CNNs process images locally โ each filter sees a small patch at a time. For many tasks this works well, but sometimes context far away in the image is important: a camouflaged vehicle is more recognisable when you also see the nearby road and tyre tracks. Vision Transformers address this by splitting the image into patches and letting every patch "attend" to every other patch โ enabling global context from the very first layer. They have surpassed CNNs on many benchmarks but are computationally heavier, so hardware context matters when deciding which to use.
Self-attention is the core operation in a Transformer. For an image split into $N$ patches of size $P \times P$: MDPI Autonomous Control Ch.6
$X \in \mathbb{R}^{N \times d}$: sequence of $N$ patch embeddings of dimension $d$. $W_Q, W_K, W_V \in \mathbb{R}^{d \times d_k}$: learned projection matrices. $Q$: queries ("what am I looking for?"); $K$: keys ("what do I contain?"); $V$: values ("what do I send?"). The softmax score $\frac{QK^T}{\sqrt{d_k}}$ for patch $i$ vs patch $j$ is how much patch $i$ should "pay attention" to the content of patch $j$. Dividing by $\sqrt{d_k}$ prevents vanishing gradients in the softmax from very large dot products.
Standard self-attention has quadratic complexity in the sequence length:
$$\text{Complexity of full self-attention:} \quad O(N^2 d)$$ $$\text{For a 640ร640 image with P=16 patches: } N = (640/16)^2 = 1600 \text{ patches}$$ $$\Rightarrow 1600^2 = 2.56 \times 10^6 \text{ attention weight pairs per head per layer}$$With 12 attention heads and 12 layers, this is $\sim 360M$ operations per forward pass โ approximately $8\times$ more expensive than a ResNet-50. Practical consequence for UAVs: Full ViT (ViT-B/16) runs at ~5โ8 FPS on a Jetson Orin Nano โ unsuitable for real-time detection. Solutions: (1) Hierarchical ViTs (Swin Transformer) โ attention only within local windows, $O(N \cdot w^2 d)$ where $w$ is window size; (2) Hybrid architectures โ CNN backbone + Transformer neck; (3) Distillation โ compress ViT into a smaller CNN for deployment. Bestaoui Sebbane ยง10.1
The Swin Transformer (Shifted Window Transformer) reduces attention complexity from $O(N^2)$ to $O(N)$ by restricting attention to local windows, then shifting the windows between layers to allow cross-window communication:
$w$: window size (typically 7ร7 patches = 49 tokens per window). For $N = 1600$ patches, $w=7$: complexity โ $1600 \times 49 \times d = 78400d$ vs. $2.56 \times 10^6 d$ for full attention โ a 33ร reduction. Swin-T achieves 81.3% ImageNet top-1 at ~4.5 GFLOPs (comparable to ResNet-50), and runs at ~35 FPS on Jetson Orin Nano โ borderline real-time for UAV use.
Optical flow is the apparent motion of pixels between two consecutive video frames. It tells you: every pixel moved this many pixels left/right/up/down between frame $t$ and frame $t+1$. For a UAV flying forward, the entire image flows backward โ that flow pattern tells you speed and direction. For a vehicle on the ground moving while the UAV is stationary, only the vehicle's pixels flow โ that's how you detect moving targets. Optical flow is a bridge between raw video and motion understanding, and is fundamental to both visual ego-motion estimation (VIO, Module 04) and moving target detection.
Classical optical flow is derived from the brightness constancy assumption: Bestaoui Sebbane ยง10.2
$I_x, I_y$: spatial image gradients (edges); $I_t$: temporal gradient (change over time); $u, v$: unknown flow at pixel $(x,y)$. This is one equation with two unknowns โ the aperture problem: without additional constraints, only motion perpendicular to edges is observable. Lucas-Kanade solves this by assuming constant flow within a local window $\Omega$: $$\min_{u,v} \sum_{(x,y)\in\Omega} (I_x u + I_y v + I_t)^2 \quad \Rightarrow \quad \underbrace{\begin{bmatrix}\sum I_x^2 & \sum I_x I_y \\ \sum I_x I_y & \sum I_y^2\end{bmatrix}}_{M} \begin{bmatrix}u\\v\end{bmatrix} = -\begin{bmatrix}\sum I_x I_t \\ \sum I_y I_t\end{bmatrix}$$ The matrix $M$ is the structure tensor. Its eigenvalues determine whether optical flow is reliably estimable at this point.
Deep learning methods dramatically outperform classical methods. RAFT (Recurrent All-Pairs Field Transforms) is the current benchmark leader for dense optical flow: MDPI Autonomous Control Ch.6
$\mathbf{f}_1, \mathbf{f}_2$: feature maps of frame 1 and frame 2 (from shared CNN encoder). The 4D correlation volume measures how similar every pixel in frame 1 is to every pixel in frame 2. A GRU (recurrent unit) iteratively refines the flow estimate by querying this correlation volume, achieving state-of-the-art accuracy (1.5 EPE on Sintel clean) at moderate compute cost (~10 FPS on Jetson Orin). For UAV ego-motion, a lightweight variant (RAFT-Small, 0.9M params) achieves ~30 FPS.
A single camera image contains depth cues โ perspective foreshortening, known object sizes, texture gradients โ but extracting metric depth from monocular imagery is an ill-posed problem: infinitely many 3D scenes can produce the same 2D image. Deep learning partially solves this by learning statistical priors about scene geometry from training data. The result is a dense depth map โ every pixel gets an estimated distance. For UAVs, this enables terrain reconstruction, obstacle detection, and landing zone assessment without requiring a stereo camera or LiDAR.
Supervised depth estimation requires LiDAR ground truth โ expensive to collect from UAVs. Self-supervised methods use consecutive video frames as supervision: Bestaoui Sebbane ยง10.3
$\hat{I}_{t\to t'}$: frame $t'$ warped to match frame $t$ using predicted depth + estimated camera pose. If depth is correct, warping should perfectly reconstruct frame $t$ from frame $t'$ โ the photometric error should be zero. The $\min$ over multiple source frames handles occlusions. $\mathcal{L}_{smooth}$: edge-aware depth smoothness โ penalises depth discontinuities unless there is also an image edge there (objects should have smooth depth surfaces). $\alpha = 0.85$ typically.
Self-supervised monocular depth is scale-ambiguous: the network cannot distinguish between a small object close up and a large object far away from a single image. The predicted depth is up to an unknown scale factor $s$: $\hat{d} = s \cdot d_{metric}$. For UAV navigation, this is critical โ obstacle avoidance requires metric distances. Solutions: MDPI Autonomous Control Ch.6
A model that achieves state-of-the-art accuracy on a GPU server is useless if it cannot run in real time on the 15โ50 W compute budget of a UAV payload. Edge deployment is the engineering discipline of making models fast enough, small enough, and energy-efficient enough to run onboard. The tools: quantisation (use 8-bit integers instead of 32-bit floats), pruning (remove unnecessary connections), knowledge distillation (train a small network to mimic a large one), and hardware-optimised inference engines (TensorRT, ONNX Runtime).
$s$: scale factor (maps float range to int range); $z$: zero point. INT8 quantisation reduces model size by 4ร and memory bandwidth by 4ร vs FP32, enabling ~2โ4ร inference speedup on hardware with INT8 tensor cores (Jetson Orin, Neural Processing Units). Accuracy impact: typically <1% mAP drop with calibration-aware quantisation. FP16 (half precision) halves size with negligible accuracy loss โ easier to apply, less speedup than INT8. Virell ยง6.2
| Hardware | Power | FP16 TOPS | INT8 TOPS | YOLOv8n FPS | Platform Class |
|---|---|---|---|---|---|
| NVIDIA Jetson Orin Nano | 7โ15 W | 20 | 40 | ~45 | Group 2โ3 UAV payload |
| NVIDIA Jetson AGX Orin | 15โ60 W | 275 (INT8) | 275 | ~200 | Group 3โ4 payload |
| Qualcomm QRB5165 (RB5) | ~5 W | 15 (DSP) | 15 | ~30 | Nano UAV |
| Hailo-8 NPU | 2.5 W | โ | 26 | ~60 | Group 1 edge AI |
| Coral Edge TPU | ~2 W | โ | 4 | ~15 | Micro UAV (limited) |
torch.onnx.export) โ
(3) Optimise with TensorRT (trtexec --fp16 or --int8) โ
(4) Deploy with DeepStream or custom inference code on Jetson โ
(5) Validate accuracy on representative aerial test set before field use.
TensorRT optimises layer fusion, memory layout, and kernel selection for the specific
GPU architecture โ typically 2โ5ร faster than native PyTorch on Jetson hardware.
Virell ยง6.3
This visualisation shows a neural network propagating activation through its layers โ analogous to a forward pass through a CNN feature extractor. Green nodes are the input layer (representing raw sensor data โ pixels or feature vectors). Cyan nodes are hidden layers (progressively more abstract feature representations). Red nodes are the output layer (class probabilities or bounding box coordinates). The pulsing activation wave shows information flowing from input to output. Drag to orbit; active layer is highlighted in the telemetry strip.
pip install numpy
# dl_perception_uav.py # โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ # Part 1: CNN building blocks (NumPy) โ MDPI Autonomous Control Ch.5 # Part 2: YOLO box decoder โ Bestaoui Sebbane ยง9.6 # Part 3: NMS post-processing โ standard detection pipeline # โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ import numpy as np # โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ # PART 1 โ CNN Building Blocks in NumPy # โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ def conv2d_naive(x, kernel, stride=1, padding=0): """ 2D convolution โ naive implementation for clarity. x: (H, W, C_in) input feature map kernel: (kH, kW, C_in, C_out) filter weights Returns: (H_out, W_out, C_out) output feature map MDPI Autonomous Control Ch.5 """ H, W, C_in = x.shape kH, kW, _, C_out = kernel.shape # Add padding if padding > 0: x = np.pad(x, [(padding, padding), (padding, padding), (0, 0)]) H_out = (H + 2*padding - kH) // stride + 1 W_out = (W + 2*padding - kW) // stride + 1 out = np.zeros((H_out, W_out, C_out)) for i in range(H_out): for j in range(W_out): h_start = i * stride; h_end = h_start + kH w_start = j * stride; w_end = w_start + kW patch = x[h_start:h_end, w_start:w_end, :] # (kH, kW, C_in) # Vectorised over output channels out[i, j, :] = np.tensordot(patch, kernel, axes=([0,1,2], [0,1,2])) return out def relu(x): return np.maximum(0, x) def maxpool2d(x, pool_size=2, stride=2): """2ร2 max pooling โ halves spatial dimensions.""" H, W, C = x.shape H_out = (H - pool_size) // stride + 1 W_out = (W - pool_size) // stride + 1 out = np.zeros((H_out, W_out, C)) for i in range(H_out): for j in range(W_out): out[i, j, :] = x[i*stride:i*stride+pool_size, j*stride:j*stride+pool_size, :].max(axis=(0,1)) return out def batch_norm(x, gamma=None, beta=None, eps=1e-5): """ Batch normalisation โ normalises per channel. x: (H, W, C) feature map Returns normalised map with learned scale gamma and shift beta. """ mu = x.mean(axis=(0,1), keepdims=True) # mean per channel sigma = x.std(axis=(0,1), keepdims=True) + eps # std per channel x_hat = (x - mu) / sigma if gamma is None: gamma = np.ones(x.shape[-1]) if beta is None: beta = np.zeros(x.shape[-1]) return gamma * x_hat + beta def count_conv_params(C_in, C_out, K, bias=True): """ Count learnable parameters in a conv layer. Formula: (K^2 * C_in + bias) * C_out """ return (K**2 * C_in + 1 if bias else 0) * C_out def demo_cnn_building_blocks(): """Forward pass through a minimal CNN block.""" print(f"\n{'='*54}\n CNN BUILDING BLOCKS DEMO\n{'='*54}") # Simulate a 16ร16 aerial image patch, 3 channels (RGB) x = np.random.randn(16, 16, 3) print(f" Input: {x.shape} (HรWรC)") # Conv layer 1: 3โ16 channels, 3ร3 kernel, same padding k1 = np.random.randn(3, 3, 3, 16) * 0.1 out1 = relu(conv2d_naive(x, k1, stride=1, padding=1)) print(f" After Conv1+ReLU: {out1.shape} params={count_conv_params(3,16,3):,}") # Batch norm out1_bn = batch_norm(out1) print(f" After BatchNorm: {out1_bn.shape} (same shape, normalised)") # Max pool 2ร2: halves spatial size out1_pool = maxpool2d(out1_bn, pool_size=2, stride=2) print(f" After MaxPool: {out1_pool.shape} (halved)") # Conv layer 2: 16โ32 channels, 3ร3 kernel k2 = np.random.randn(3, 3, 16, 32) * 0.1 out2 = relu(conv2d_naive(out1_pool, k2, stride=1, padding=1)) print(f" After Conv2+ReLU: {out2.shape} params={count_conv_params(16,32,3):,}") total_params = count_conv_params(3,16,3) + 16*2 + count_conv_params(16,32,3) + 32*2 print(f"\n Total params: {total_params:,} (tiny โ real YOLOv8n has ~3.2M)") # โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ # PART 2 โ YOLO Bounding Box Decoder # Converts raw YOLO output tensor to (x1,y1,x2,y2,conf,class_id) # Bestaoui Sebbane ยง9.6 # โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ def decode_yolo_output(raw_output, anchors, n_classes, img_size, conf_thresh=0.5): """ Decode YOLO raw prediction tensor to bounding boxes. raw_output: (G, G, A*(5+C)) โ grid predictions anchors: list of (pw, ph) anchor dimensions (in pixels, relative to img_size) n_classes: number of object classes img_size: input image size (square assumed) conf_thresh: minimum objectness confidence to keep Returns: list of [x1, y1, x2, y2, confidence, class_id] """ G = raw_output.shape[0] # grid size A = len(anchors) # number of anchors stride = img_size / G # pixels per grid cell raw = raw_output.reshape(G, G, A, 5 + n_classes) detections = [] for cy in range(G): for cx in range(G): for a, (pw, ph) in enumerate(anchors): pred = raw[cy, cx, a] # Decode box centre and size (YOLO parameterisation) bx = (1 / (1 + np.exp(-pred[0])) + cx) * stride # ฯ(tx) + cx by = (1 / (1 + np.exp(-pred[1])) + cy) * stride # ฯ(ty) + cy bw = pw * np.exp(pred[2]) # pw * e^tw bh = ph * np.exp(pred[3]) # ph * e^th # Objectness + class probability obj_conf = 1 / (1 + np.exp(-pred[4])) # sigmoid cls_logits = pred[5:] cls_probs = np.exp(cls_logits) / np.exp(cls_logits).sum() # softmax cls_id = np.argmax(cls_probs) confidence = obj_conf * cls_probs[cls_id] if confidence > conf_thresh: # Convert to corner format (x1,y1,x2,y2) x1 = bx - bw/2; y1 = by - bh/2 x2 = bx + bw/2; y2 = by + bh/2 detections.append([x1, y1, x2, y2, confidence, cls_id]) return detections # โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ # PART 3 โ Non-Maximum Suppression (NMS) # Removes duplicate detections of the same object # โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ def compute_iou(boxA, boxB): """ Intersection-over-Union between two boxes [x1,y1,x2,y2]. IoU = intersection_area / union_area """ xA = max(boxA[0], boxB[0]); yA = max(boxA[1], boxB[1]) xB = min(boxA[2], boxB[2]); yB = min(boxA[3], boxB[3]) inter = max(0, xB - xA) * max(0, yB - yA) areaA = (boxA[2]-boxA[0]) * (boxA[3]-boxA[1]) areaB = (boxB[2]-boxB[0]) * (boxB[3]-boxB[1]) union = areaA + areaB - inter return inter / (union + 1e-6) def non_maximum_suppression(detections, iou_thresh=0.45): """ NMS: suppress overlapping boxes, keep highest confidence. detections: list of [x1,y1,x2,y2,conf,class_id] Returns: filtered detections """ if not detections: return [] dets = sorted(detections, key=lambda d: d[4], reverse=True) # sort by confidence kept = [] while dets: best = dets.pop(0) kept.append(best) # Remove boxes that overlap too much with best dets = [d for d in dets if d[5] != best[5] or # different class โ always keep compute_iou(best[:4], d[:4]) < iou_thresh] return kept def demo_detection_pipeline(): """Demonstrate YOLO decode + NMS pipeline.""" print(f"\n{'='*54}\n YOLO DETECTION PIPELINE DEMO\n Bestaoui Sebbane ยง9.6\n{'='*54}") # Simulate a 20ร20 YOLO grid with 3 anchors, 4 classes (UAV mission classes) # Classes: 0=vehicle, 1=person, 2=weapon_system, 3=building G, A, C = 20, 3, 4 raw = np.random.randn(G, G, A * (5 + C)) * 0.5 # Inject a confident vehicle detection at grid cell (10, 8) # anchor 0, conf=2.5 (sigmoidโ0.924), class 0 (vehicle) dominant raw[10, 8, : (5+C)] = np.array([0.2, 0.2, 0.3, 0.2, 2.5, 3.0, -1.0, -1.0, -1.0]) # Add a near-duplicate detection for NMS demo raw[10, 9, : (5+C)] = np.array([0.1, 0.1, 0.3, 0.2, 2.0, 2.8, -1.0, -1.0, -1.0]) anchors = [(32, 16), (24, 24), (16, 32)] # anchor sizes in pixels img_size = 640 # Decode dets = decode_yolo_output(raw, anchors, C, img_size, conf_thresh=0.5) print(f"\n After decode: {len(dets)} detections above threshold") for d in dets[:3]: cls_names = ['vehicle', 'person', 'weapon_sys', 'building'] print(f" [{cls_names[int(d[5])]:10s}] conf={d[4]:.3f} box=({d[0]:.0f},{d[1]:.0f},{d[2]:.0f},{d[3]:.0f})") # NMS kept = non_maximum_suppression(dets, iou_thresh=0.45) print(f"\n After NMS: {len(kept)} detections remaining") for d in kept[:3]: print(f" [{cls_names[int(d[5])]:10s}] conf={d[4]:.3f} box=({d[0]:.0f},{d[1]:.0f},{d[2]:.0f},{d[3]:.0f})") # IoU sanity check b1 = [100, 100, 200, 200] # box 1 b2 = [150, 150, 250, 250] # 50% overlap b3 = [300, 300, 400, 400] # no overlap print(f"\n IoU sanity: 50% overlap = {compute_iou(b1,b2):.3f} (expect ~0.143)") print(f" IoU sanity: no overlap = {compute_iou(b1,b3):.3f} (expect 0.000)") print(f" IoU sanity: identical = {compute_iou(b1,b1):.3f} (expect 1.000)") print(f"\n{'='*54}\n") if __name__ == "__main__": np.random.seed(7) demo_cnn_building_blocks() demo_detection_pipeline()