Jump to
MODULE 08 // AUTONOMY & DECISION-MAKING

Autonomy & Decision-Making

Knowing where you are (navigation) and what you see (perception) is not enough. A truly autonomous UAV must decide what to do next β€” in real time, under uncertainty, with incomplete information, against a potentially adaptive adversary, in environments where communication may be denied. This module covers the formal frameworks β€” POMDP, Behaviour Trees, path planning, OODA loop β€” that structure autonomous decision-making in military UAV systems.

POMDP Planning Behaviour Trees Path Planning (A*, RRT*) Lethal Autonomy Ethics

πŸ“š  REFERENCES: Bestaoui Sebbane Β§9–11  Β·  MDPI Autonomous Control Ch.6–7  Β·  Intelligent Coordination of UAV Swarm Systems Ch.3

8.1 Levels of Autonomy β€” A Spectrum, Not a Switch

Plain English First

Autonomy is not binary β€” it is a spectrum from "human does everything" to "machine does everything." Military doctrine and system engineering both require being precise about which decisions are made by the machine vs. the human, and under what conditions that boundary shifts. A UAV might autonomously handle attitude stabilisation (Level 1) while requiring human approval for every weapon release (Level 4 authorisation). Understanding this spectrum is both a technical requirement and a legal/ethical one β€” it determines accountability, certification requirements, and rules of engagement compliance.

LEVEL 1
Remote Control
Human executes every control input. UAV has no independent decision-making. Example: direct RC link.
LEVEL 2
Autopilot
Human sets goals; machine handles low-level execution (attitude, altitude hold). Pilot in command. Example: PX4 stabilised mode.
LEVEL 3
Supervised Auto
Machine executes full missions autonomously; human monitors and can override. Human on the loop. Example: MQ-9 ISR orbit.
LEVEL 4
Goal-Directed
Machine plans and re-plans to achieve objectives; human sets goals and authorises key actions. Example: autonomous strike with MITL engagement.
LEVEL 5
Fully Autonomous
Machine selects and executes all actions including lethal engagement without per-action human authorisation. Legally contested. Example: Harpy NG.

Bestaoui Sebbane maps these levels to specific technical capability requirements, noting that each step up the autonomy ladder requires solving progressively harder problems: Level 2 requires only control theory (Module 03). Level 3 requires navigation + perception (Modules 04–07). Level 4–5 require decision theory, planning under uncertainty, and β€” controversially β€” machine ethics. Bestaoui Sebbane Β§9.1

DoD Directive 3000.09 β€” Autonomous Weapons Policy US Department of Defense Directive 3000.09 (updated 2023) requires that autonomous and semi-autonomous weapons systems be designed to allow commanders and operators to exercise "appropriate levels of human judgment over the use of force." It does not prohibit LAWS β€” but requires senior official waivers for systems that cannot meet human judgment requirements. The directive mandates: (1) system testing to detect unintended engagements; (2) mechanisms to pause/stop operations; (3) IHL compliance verification. MDPI Autonomous Control Ch.6

8.2 The OODA Loop β€” Decision Cycle Framework

Plain English First

Every decision β€” by a human or a machine β€” follows a cycle: you observe the situation, you interpret what you observed, you decide what to do, and you act. Then the cycle repeats. Colonel John Boyd called this the OODA loop (Observe-Orient-Decide-Act). The key military insight is: the side that completes this loop faster wins. If your UAV can observe a threat, orient to it, decide to evade, and act β€” before the adversary can complete their own OODA cycle β€” you survive. This temporal competition is why autonomous systems are so militarily valuable: they complete the loop in milliseconds, where a human pilot needs seconds.

O
OBSERVE
Sensor data collection. EO/IR, radar, SIGINT, datalink. UAV: runs continuously at sensor rate (30–1000 Hz).
O
ORIENT
Fuse observations into situational awareness. EKF, object detection, threat classification. This is the most complex step β€” shapes everything else.
D
DECIDE
Select an action from available options. POMDP solver, behaviour tree, mission planner. Constrained by ROE, fuel, capabilities.
A
ACT
Execute the chosen action. GNC controllers (Module 03). Physical actuation. Feedback re-enters the Observe phase immediately.
OODA Loop Cycle Time β€” Competitive Analysis $$T_{OODA} = T_{obs} + T_{orient} + T_{decide} + T_{act}$$ $$T_{OODA}^{human} \approx 0.25 + 0.5 + 1.0 + 0.25 = 2.0 \text{ s (trained pilot under stress)}$$ $$T_{OODA}^{UAV-auto} \approx 0.03 + 0.05 + 0.01 + 0.01 = 0.10 \text{ s (onboard AI)}$$

Autonomous UAV completes 20 OODA cycles per human OODA cycle. In a kinetic engagement (missile evasion, intercept, drone-vs-drone), this temporal advantage is decisive. The Orient phase dominates autonomous OODA time β€” it is where sensor fusion, ML perception, and threat classification run. Reducing $T_{orient}$ through faster inference hardware (Module 07 edge deployment) directly improves survivability. Bestaoui Sebbane Β§9.2

8.3 POMDP β€” Decision-Making Under Uncertainty

Plain English First

The MDP from Module 06 assumed the UAV always knows exactly what state it is in. Reality is different: you don't know exactly where the enemy is β€” you have noisy sensor readings. You don't know whether a detected object is a combatant or civilian β€” you have an imperfect classifier. You don't know if your navigation estimate is accurate β€” you have a covariance ellipsoid. The POMDP (Partially Observable MDP) generalises the MDP to handle this fundamental uncertainty β€” making it the right mathematical framework for autonomous military UAV decision-making.

Analogy β€” Playing Chess Blindfolded

A standard MDP is like playing chess where you can see the full board. A POMDP is like playing chess blindfolded: someone tells you what your opponent's last move probably was (noisy observation), but you must maintain a mental model of where all the pieces might be (belief state) and choose your move accordingly. The optimal POMDP policy chooses actions that both progress toward the goal and gather information that reduces uncertainty β€” exactly the trade-off a military UAV must make when deciding whether to approach a target for better identification.

POMDP Formalisation

POMDP Tuple β€” Extension of MDP with Partial Observability $$\mathcal{M}_{POMDP} = (\mathcal{S},\, \mathcal{A},\, \mathcal{O},\, P,\, Z,\, R,\, \gamma,\, b_0)$$

$\mathcal{O}$: observation space (sensor readings, classifier outputs); $Z(o|s',a)$: observation model β€” probability of observing $o$ in state $s'$ after action $a$; $b_0$: initial belief state β€” prior probability distribution over possible states. Everything else is the same as the MDP. The key difference: the agent never directly sees $s$ β€” it only sees observations $o$ and must maintain a belief state $b: \mathcal{S} \to [0,1]$.

Belief State Update β€” The POMDP Filter

Belief Update (Bayes Filter) $$b'(s') = \eta\, Z(o|s',a) \sum_{s \in \mathcal{S}} P(s'|s,a)\, b(s)$$

$\eta$: normalisation constant; $o$: new observation; $a$: action just taken. This is exactly the Bayesian update from Module 04 and 05, now applied to discrete state spaces. The EKF from Module 04 is a special case of the belief update for continuous Gaussian state spaces. The POMDP is the general framework; the EKF is the computational tractable approximation for navigation. Bestaoui Sebbane Β§9.3

Value Functions and Policies Over Beliefs

In a POMDP, the policy maps beliefs to actions: $\pi: \Delta(\mathcal{S}) \to \mathcal{A}$. The value function over beliefs is piecewise linear and convex (PWLC): MDPI Autonomous Control Ch.6

Alpha-Vector Representation of POMDP Value Function $$V^*(b) = \max_{\alpha \in \Gamma} \sum_{s} \alpha(s)\, b(s) = \max_{\alpha \in \Gamma}\, \alpha \cdot b$$

$\Gamma$: set of $\alpha$-vectors (one per policy fragment); $\alpha(s)$: expected cumulative reward of a particular action sequence starting in state $s$. The value function is the upper envelope of all $\alpha$-vectors. Computing $\Gamma$ exactly is PSPACE-complete β€” intractable for large state spaces. Practical algorithms: PBVI (Point-Based Value Iteration), SARSOP, POMCP (Monte Carlo Tree Search). For UAV mission planning with $|\mathcal{S}| < 100$ states, PBVI converges in <1 s. For rich sensor-observation spaces, POMCP is the standard approach.

Worked Example 8.1 β€” POMDP for Target Identification Under Uncertainty

A UAV must decide whether to engage or not engage a detected target. The target is either a combatant vehicle (state $s_1$) or a civilian vehicle (state $s_2$). The onboard classifier gives: P(classify as threat | combatant) = 0.85, P(classify as threat | civilian) = 0.12. Prior belief: P($s_1$) = 0.4. After observing "threat classification" from the sensor, compute the updated belief and determine whether the POMDP engage threshold (belief $> 0.9$) is met.

Observation model: $Z(\text{threat}|s_1) = 0.85$, $Z(\text{threat}|s_2) = 0.12$. Prior: $b(s_1) = 0.4$, $b(s_2) = 0.6$.

Unnormalised update: $b'(s_1) \propto Z(\text{threat}|s_1) \cdot b(s_1) = 0.85 \times 0.4 = 0.340$ $b'(s_2) \propto Z(\text{threat}|s_2) \cdot b(s_2) = 0.12 \times 0.6 = 0.072$

Normalisation: $\eta = 1/(0.340 + 0.072) = 1/0.412 = 2.427$

Posterior: $b'(s_1) = 0.340 \times 2.427 = \mathbf{0.825}$, $b'(s_2) = 0.072 \times 2.427 = \mathbf{0.175}$

Decision: $b'(s_1) = 0.825 < 0.9$ β€” threshold not met. Do not engage. Continue observation to reduce uncertainty. A second "threat" classification (same observation) would give: $b''(s_1) \approx 0.85 \times 0.825 / (0.85 \times 0.825 + 0.12 \times 0.175) = 0.701 / 0.722 = \mathbf{0.971} > 0.9$ β†’ engage threshold met.

Key insight: The POMDP naturally implements an "information gathering before action" strategy. The UAV takes multiple observations, accumulating evidence, before crossing the engagement threshold. This is a formal implementation of the IHL precaution principle and is a core argument for POMDP-based autonomy in contested military environments. Bestaoui Sebbane Β§9.4

8.4 Behaviour Trees β€” Modular Mission Logic

Plain English First

A Behaviour Tree (BT) is a way to organise a UAV's decision logic as a hierarchical tree of conditions and actions. Think of it like a flowchart, but more structured and much easier to modify, test, and combine. Each "leaf" node is either a condition check ("Is the battery above 20%?") or an action ("Fly to waypoint X"). Internal nodes combine their children using logic operators: "do all of these in sequence" (Sequence), or "try each until one succeeds" (Selector/Fallback). BTs are the dominant mission logic framework in autonomous UAV systems β€” used in PX4, ROS2 Navigation, and nearly every academic and military UAV system β€” because they are readable, testable, and composable in a way that finite state machines are not.

BT Node Types and Return Status

Node TypeSymbolLogicReturns
Sequence (β†’)β†’Execute children left to right. Stop and return FAILURE if any child fails.SUCCESS if all succeed; FAILURE on first failure
Selector / Fallback (?)?Try each child. Stop and return SUCCESS if any child succeeds.SUCCESS on first success; FAILURE if all fail
Parallel (⇉)⇉Execute all children simultaneously. Success threshold $M/N$ configurable.SUCCESS if $\geq M$ children succeed
Decoratorβ—‡Modifies child's return value or execution count (Repeat, Retry, Invert, etc.)Depends on decorator type
Action (β–‘)β–‘Leaf node: executes a task. Returns RUNNING while executing.SUCCESS / FAILURE / RUNNING
Condition (β—‡)β—‡Leaf node: checks a predicate. Instantaneous (no RUNNING state).SUCCESS (true) / FAILURE (false)

A Military UAV ISR Mission Behaviour Tree

? [Root Fallback β€” always tries to do something useful] β”œβ”€ β†’ [Emergency Handler Sequence] β”‚ β”œβ”€ β—‡ Battery < 10% OR Critical Failure β”‚ └─ β–‘ Execute Emergency Return-to-Base β”œβ”€ ? [Mission Fallback β€” try primary then contingency] β”‚ β”œβ”€ β†’ [Primary ISR Mission Sequence] β”‚ β”‚ β”œβ”€ β—‡ GPS / Nav estimate valid β”‚ β”‚ β”œβ”€ β—‡ Target area reachable (fuel check) β”‚ β”‚ β”œβ”€ β–‘ Transit to ISR orbit waypoint β”‚ β”‚ └─ β†’ [Collect Intelligence Loop] β”‚ β”‚ β”œβ”€ β—‡ Repeat(N=10) [collect 10 frames] β”‚ β”‚ β”‚ └─ β–‘ Capture + process ISR image β”‚ β”‚ └─ β–‘ Transmit data to GCS β”‚ └─ β†’ [Degraded Mode Sequence] β”‚ β”œβ”€ β—‡ Comms link lost > 60s β”‚ β”œβ”€ β–‘ Switch to autonomous loiter β”‚ └─ β–‘ Execute pre-planned fallback waypoints └─ ? [Threat Response Selector] β”œβ”€ β†’ [Evasion Sequence] β”‚ β”œβ”€ β—‡ Threat detected AND within engagement range β”‚ β”œβ”€ β–‘ Classify threat type β”‚ └─ β–‘ Execute evasive manoeuvre └─ β–‘ Continue nominal mission # default leaf β€” always succeeds
Theorem 8.1 β€” BT Reactivity and Modularity Properties

A Behaviour Tree with $N$ nodes has the following formal properties: MDPI Autonomous Control Ch.7

1. Reactivity: If any condition node changes from SUCCESS to FAILURE, the tree immediately re-routes on the next tick β€” without explicit state transition code. This emergent reactivity is the key advantage over finite state machines (FSMs), which require explicit transitions between every pair of states: $O(N^2)$ transitions for $N$ states vs. $O(N)$ nodes for a BT of equivalent complexity.

2. Composability: Any subtree is a valid BT and can be inserted at any node position without modifying surrounding logic. This allows modular mission building β€” a new "electronic warfare evasion" subtree can be inserted at the Threat Response Selector without touching the ISR mission logic. FSMs require adding transitions from every existing state to the new states.

3. Tick rate: BT ticks once per control cycle. For mission-level decisions: typically 10–50 Hz. For safety-critical conditions (battery, collision): 100–250 Hz. The tick rate determines decision latency β€” critical for the OODA loop cycle time.

8.5 Path Planning β€” From Current Position to Objective

Plain English First

Path planning answers the question: given a map of the environment and a goal location, what sequence of movements will get me there while avoiding obstacles, respecting vehicle constraints, and minimising some cost (time, fuel, exposure to threats)? For military UAVs this is not just obstacle avoidance β€” it includes terrain masking (flying low to stay below radar coverage), threat envelope avoidance (staying outside SAM engagement zones), and time-on-target constraints (arriving in a coordination window). Planners operate at two levels: global (full mission trajectory) and local (real-time reactive obstacle avoidance).

A* β€” Optimal Graph Search

A* is the foundation of global path planning. It searches a discretised map (occupancy grid or waypoint graph) for the minimum-cost path: Bestaoui Sebbane Β§10.1

A* Evaluation Function $$f(n) = g(n) + h(n)$$

$g(n)$: exact cost from start to node $n$ (known). $h(n)$: heuristic estimate of cost from $n$ to goal. Admissible heuristic: $h(n) \leq h^*(n)$ (never overestimates true cost). Common choices: Euclidean distance $h = \|\mathbf{p}_n - \mathbf{p}_{goal}\|_2$ (admissible if all moves have cost $\geq$ distance); Octile distance for 8-connected grids.

Complexity: $O(b^d)$ worst case, where $b$ is branching factor and $d$ is depth. On a 100Γ—100Γ—20 3D occupancy grid (typical indoor UAV planning): $\sim 200{,}000$ nodes. With admissible heuristic, A* is guaranteed to find the optimal path. For military terrain planning (10 kmΒ² area at 10 m resolution): $10^5$ nodes β€” typically solves in $<100$ ms with a good heuristic.

RRT* β€” Optimal Sampling-Based Planning

For continuous state spaces with complex kinodynamic constraints (UAV cannot turn instantaneously; minimum turning radius applies), grid-based A* is intractable. RRT* (Rapidly-Exploring Random Trees, optimal variant) samples random configurations and incrementally builds a tree: Bestaoui Sebbane Β§10.2

RRT* Core Algorithm (Asymptotically Optimal) $$x_{rand} \sim \text{Uniform}(\mathcal{X}) \quad \text{(sample random state)}$$ $$x_{nearest} = \arg\min_{x \in \mathcal{T}} \|x - x_{rand}\| \quad \text{(find nearest tree node)}$$ $$x_{new} = \text{Steer}(x_{nearest}, x_{rand}, \Delta) \quad \text{(extend toward sample)}$$ $$\text{Rewire}(\mathcal{T}, x_{new}, r_n) \quad \text{(optimise nearby connections)}$$

The Rewire step is what makes RRT* optimal (vs. RRT which is only complete): for each node $x$ within radius $r_n$ of $x_{new}$, if routing through $x_{new}$ gives a lower cost path from root to $x$, update the parent of $x$ to be $x_{new}$. As the number of samples $n \to \infty$, the path cost converges to the optimum. Rewire radius: $r_n = \gamma_{RRT^*}\left(\frac{\ln n}{n}\right)^{1/d}$ where $d$ is state space dimension β€” shrinks as more samples are added. Bestaoui Sebbane Β§10.2

Threat-Aware Planning β€” Cost Map Design

Military path planning adds threat cost fields to the standard obstacle map. Each SAM battery, radar, and known adversary position creates a cost gradient: MDPI Autonomous Control Ch.7

Composite Mission Cost Map $$c(\mathbf{p}) = c_{obs}(\mathbf{p}) + \sum_{k} w_k \cdot \phi_k(\mathbf{p})$$ $$\phi_{SAM}(\mathbf{p}) = \begin{cases} \infty & \|\mathbf{p} - \mathbf{p}_{SAM}\| \leq r_{kill} \\ \exp\!\left(-\frac{\|\mathbf{p}-\mathbf{p}_{SAM}\| - r_{kill}}{r_{threat}}\right) & r_{kill} < \|\mathbf{p}-\mathbf{p}_{SAM}\| \leq r_{detect} \\ 0 & \text{otherwise} \end{cases}$$

$r_{kill}$: SAM kill radius (hard exclusion zone β€” infinite cost); $r_{detect}$: SAM detection radius (soft avoidance zone β€” exponential cost); $r_{threat}$: cost decay length scale. The exponential tail allows the planner to "clip" through the outer detection zone if no other route exists β€” the cost is high but finite. Hard exclusion zones are never penetrated. Weight $w_k$ balances threat avoidance vs. mission efficiency.

Worked Example 8.2 β€” A* Path Cost with SAM Threat Field

A UAV must reach a target 80 km away. Two possible routes exist. Route A is direct (80 km), passing through a SAM outer detection zone for 15 km at mean cost $\phi = 2.5$ (exponential decay at 0.4Γ— kill radius). Route B goes around (105 km), fully outside all threat zones ($\phi = 0$). Fuel cost per km = 1 unit. Threat cost weight $w = 8$. Which route has lower total cost?

Route A total cost: $c_A = 65 \times 1 + 15 \times (1 + 8 \times 2.5) = 65 + 15 \times 21 = 65 + 315 = \mathbf{380}$

Route B total cost: $c_B = 105 \times 1 + 0 = \mathbf{105}$

Decision: Route B (the long way round) has 3.6Γ— lower cost. The planner correctly selects the safer route despite being 31% longer. If $w$ is reduced to 1 (mission commander accepts more risk): $c_A = 65 + 15 \times 3.5 = 117.5$ vs $c_B = 105$ β€” Route B still preferred, but only marginally. At $w \approx 0.7$, the routes become cost-equal β€” this is the risk tolerance threshold. This demonstrates how the weight $w$ encodes the mission risk appetite, and why it should be a configurable commander's parameter rather than a hard-coded value.

8.6 Autonomous Threat Response β€” Reactive Decision-Making

Plain English First

Global path planning works on the timescale of minutes. But threats can appear in seconds β€” a radar lock, a missile launch, a suddenly detected obstacle. Reactive threat response is the fast decision loop that operates independently of (and can override) the global mission plan. It runs at the same timescale as the OODA loop β€” 100 ms or faster. The key engineering challenge is ensuring these two decision layers (slow strategic + fast reactive) do not conflict β€” the reactive layer must be able to override the strategic layer without causing the UAV to take dangerous actions to "get back" to the plan.

Potential Field Reactive Avoidance

Artificial Potential Field (APF) methods compute a continuous force field over the workspace: goal attracts, obstacles repel. The control action is the gradient of this field: Bestaoui Sebbane Β§10.3

Artificial Potential Field β€” Attractive + Repulsive Forces $$U(\mathbf{p}) = U_{att}(\mathbf{p}) + \sum_i U_{rep,i}(\mathbf{p})$$ $$U_{att} = \frac{1}{2}k_{att}\|\mathbf{p} - \mathbf{p}_{goal}\|^2$$ $$U_{rep,i} = \begin{cases} \frac{1}{2}k_{rep}\left(\frac{1}{\rho_i} - \frac{1}{\rho_0}\right)^2 & \rho_i \leq \rho_0 \\ 0 & \rho_i > \rho_0 \end{cases}$$ $$\mathbf{F}(\mathbf{p}) = -\nabla U(\mathbf{p}) = \mathbf{F}_{att} + \sum_i \mathbf{F}_{rep,i}$$

$\rho_i = \|\mathbf{p} - \mathbf{p}_{obs,i}\|$: distance to obstacle $i$; $\rho_0$: obstacle influence radius; $k_{att}, k_{rep}$: attractive/repulsive gain constants. Known limitation: Local minima β€” the UAV can get trapped in a potential well between two obstacles. Solutions: random perturbation, hybrid APF + A* global planner. APF is used for fast reactive avoidance; A*/RRT* handles global planning.

Missile Evasion β€” Time-Critical Response

When a radar lock or missile launch is detected, the UAV must immediately execute an evasion manoeuvre that maximises miss distance while maintaining structural integrity. From Proportional Navigation (Module 03): the pursuer closes by maintaining constant LOS angle. The optimal evasion strategy is to maximise $\dot{\lambda}$ β€” the LOS rate: Bestaoui Sebbane Β§11.1

Optimal Evasion β€” Maximum LOS Rate $$\text{Max miss distance} \equiv \text{max } \dot{\lambda} \equiv \text{lateral acceleration perpendicular to LOS}$$ $$a_{evade} = a_{max} \cdot \hat{\mathbf{n}}_\perp \qquad \text{where } \hat{\mathbf{n}}_\perp \perp \mathbf{r}_{LOS}$$

The optimal evasion turn is always perpendicular to the line of sight between UAV and missile. This maximises $\dot{\lambda}$, forcing the PN guidance to command maximum lateral acceleration β€” potentially exceeding the missile's structural limits. For a small fast UAV (load factor limit 10g), the maximum structural acceleration provides ~4g lateral evasion. For a large MALE UAV (2–3g structural limit), evasion effectiveness is limited β€” which is why MALE platforms rely primarily on low RCS, chaff/flare dispensing, and route planning rather than kinetic evasion.

8.7 Ethics of Lethal Autonomous Weapons Systems (LAWS)

Plain English First

This section addresses a question that cannot be answered by equations alone: Should autonomous systems make life-and-death decisions without human authorisation? This is not merely a philosophical question for UAV engineers β€” it is a practical design constraint. The answer determines what your system is legally allowed to do, what certifications are required, and what happens when something goes wrong. Every engineer who designs autonomous UAV decision logic bears professional and potentially legal responsibility for those design choices, whether or not they ever think about it.

The IHL Requirements β€” Three Principles That Apply to Machines

International Humanitarian Law (IHL) applies to any weapons system, autonomous or not. Three principles directly constrain autonomous engagement decisions: MDPI Autonomous Control Ch.6

IHL PrincipleWhat It RequiresAutonomous System ChallengeTechnical Approach
Distinction Distinguish combatants from civilians and civilian objects Object classifiers are probabilistic, not certain. Classifier confidence β‰  legal certainty POMDP belief threshold + human confirmation for ambiguous cases
Proportionality Anticipated civilian harm must not be excessive relative to military advantage Requires estimating expected civilian casualties β€” machine cannot assign legal weight to this Current consensus: human must make proportionality judgments; machine provides data
Precaution Take all feasible precautions to minimise civilian harm before attacking POMDP information-gathering before engagement (Worked Example 8.1) partially addresses this Multi-observation thresholds; abort capability; return-and-report before engagement
The Accountability Gap When a human soldier makes a targeting error, there is a clear accountability chain. When a fully autonomous system makes a targeting error, who is accountable? The programmer? The commander who deployed it? The state? Current international law has no clear answer β€” this is the "accountability gap" at the centre of the LAWS debate. Engineers who design targeting logic are not legally protected by the fact that they were following orders or specifications. IHL Article 36 weapon review obligations apply to states, but the engineers who build the system carry professional and ethical responsibility that Article 36 does not extinguish. MDPI Autonomous Control Ch.6

The "Meaningful Human Control" Standard

The emerging international standard β€” advocated by the ICRC and 125+ states β€” is that humans must retain "meaningful control" over lethal decisions. Bestaoui Sebbane defines meaningful control as requiring that: Bestaoui Sebbane Β§11.2

  1. Contextual understanding: The human understands the operational context in which the system will engage (not just "authorised in zone X")
  2. Predictable performance: The system behaves predictably enough that the human's authorisation is genuinely meaningful (not a rubber stamp on opaque AI decisions)
  3. Temporal proximity: The authorisation is close enough in time to be relevant to the specific engagement (not a blanket authorisation days earlier)
  4. Abort capability: The human retains a meaningful ability to stop the engagement before it occurs

This standard creates concrete technical requirements: low-latency communication to the operator, explainable AI outputs (not just "confidence = 0.87" but "this looks like a tank because of track geometry and thermal signature"), and architectural abort pathways that cannot be bypassed by autonomous logic.

8.8 Python: POMDP Solver + A* Path Planner + Behaviour Tree

What This Code Covers Three implementations: (1) A discrete POMDP belief updater with engagement threshold decision β€” implements Worked Example 8.1 fully. (2) A* path planner on a 2D grid with threat cost field β€” implements Worked Example 8.2. (3) A minimal Behaviour Tree engine β€” implements the ISR mission BT from section 8.4. Run with: pip install numpy heapq (heapq is in stdlib)
Python
# autonomy_decision.py
# ─────────────────────────────────────────────────────────────────
# Part 1: POMDP belief update + engagement threshold
#         Bestaoui Sebbane Β§9.3–9.4
# Part 2: A* with threat cost field
#         Bestaoui Sebbane Β§10.1
# Part 3: Minimal Behaviour Tree engine
#         MDPI Autonomous Control Ch.7
# ─────────────────────────────────────────────────────────────────

import numpy as np
import heapq
from enum import Enum


# ════════════════════════════════════════════════════════════════
# PART 1 β€” POMDP Belief Updater
# ════════════════════════════════════════════════════════════════

class TargetPOMDP:
    """
    POMDP for target identification before engagement.
    States: combatant (0) or civilian (1)
    Observations: threat_signal (0) or safe_signal (1)
    Bestaoui Sebbane Β§9.3
    """
    def __init__(self,
                 prior_combatant: float = 0.4,
                 obs_model: np.ndarray = None,
                 engage_threshold: float = 0.9):
        """
        prior_combatant : P(state=combatant) before any observation
        obs_model       : Z[s, o] = P(obs=o | state=s)
                          Row 0 = combatant, Row 1 = civilian
                          Col 0 = threat_signal, Col 1 = safe_signal
        engage_threshold: min belief in combatant to authorise engagement
        """
        self.belief = np.array([prior_combatant, 1 - prior_combatant])
        self.Z = obs_model if obs_model is not None else np.array([
            [0.85, 0.15],   # combatant: 85% threat signal, 15% safe signal
            [0.12, 0.88],   # civilian:  12% threat signal, 88% safe signal
        ])
        self.threshold = engage_threshold
        self.n_obs     = 0

    def update(self, observation: int) -> np.ndarray:
        """
        Bayesian belief update on new observation.
        observation: 0=threat_signal, 1=safe_signal
        Returns: updated belief [P(combatant), P(civilian)]
        Bestaoui Sebbane Β§9.3 Eq. 9.8
        """
        # Unnormalised: P(obs|s) * P(s)
        unnorm = self.Z[:, observation] * self.belief
        # Normalise
        self.belief = unnorm / (unnorm.sum() + 1e-12)
        self.n_obs += 1
        return self.belief.copy()

    def decision(self) -> str:
        """Current engagement recommendation based on belief threshold."""
        p_combatant = self.belief[0]
        if p_combatant >= self.threshold:
            return "ENGAGE"
        elif p_combatant <= 1 - self.threshold:
            return "NO THREAT β€” CLEAR"
        else:
            return f"GATHER MORE INFO  (need β‰₯{self.threshold:.0%} confidence)"


def demo_pomdp():
    print("\n" + "="*58)
    print("  POMDP TARGET IDENTIFICATION β€” Bestaoui Sebbane Β§9.4")
    print("="*58)
    pomdp = TargetPOMDP(prior_combatant=0.4, engage_threshold=0.9)
    print(f"\n  Prior: P(combatant)={pomdp.belief[0]:.3f}  Decision: {pomdp.decision()}")

    observations = [0, 0, 1, 0]   # threat, threat, safe, threat
    obs_names    = ['THREAT', 'SAFE']

    for i, obs in enumerate(observations):
        belief = pomdp.update(obs)
        print(f"  Obs {i+1}: {obs_names[obs]:<6} β†’ P(combatant)={belief[0]:.4f}  "
              f"| {pomdp.decision()}")


# ════════════════════════════════════════════════════════════════
# PART 2 β€” A* Path Planner with Threat Cost Field
# Bestaoui Sebbane Β§10.1
# ════════════════════════════════════════════════════════════════

class AStarPlanner:
    """
    A* path planner on a 2D grid with composite threat cost field.
    Cost = distance_cost + threat_cost (SAM exclusion zones).
    Bestaoui Sebbane Β§10.1
    """

    def __init__(self, grid_size: int = 40):
        self.G = grid_size
        self.grid = np.zeros((grid_size, grid_size))  # 0=free, 1=hard obstacle

    def add_sam_site(self, cx: int, cy: int,
                     r_kill: float, r_detect: float,
                     w_threat: float = 8.0):
        """
        Add SAM site threat field to the grid.
        r_kill   : hard exclusion radius (grid cells)
        r_detect : soft detection radius (exponential cost tail)
        w_threat : weight on threat cost vs distance cost
        """
        for y in range(self.G):
            for x in range(self.G):
                dist = np.sqrt((x-cx)**2 + (y-cy)**2)
                if dist <= r_kill:
                    self.grid[y, x] = 1e9   # hard exclusion β€” infinite cost
                elif dist <= r_detect:
                    threat_val = w_threat * np.exp(-(dist - r_kill) / (0.5*r_kill))
                    self.grid[y, x] += threat_val

    def plan(self, start: tuple, goal: tuple) -> list:
        """
        A* search from start to goal.
        Returns list of (x,y) waypoints from start to goal.
        """
        def heuristic(a, b):
            return np.sqrt((a[0]-b[0])**2 + (a[1]-b[1])**2)   # Euclidean

        open_set = [(heuristic(start, goal), 0, start)]
        came_from = {}
        g_cost = {start: 0}
        closed = set()
        neighbours = [(dx, dy) for dx in [-1,0,1] for dy in [-1,0,1] if (dx,dy) != (0,0)]

        while open_set:
            _, g, current = heapq.heappop(open_set)
            if current == goal:
                # Reconstruct path
                path = [current]
                while current in came_from:
                    current = came_from[current]
                    path.append(current)
                return list(reversed(path))

            if current in closed: continue
            closed.add(current)

            for dx, dy in neighbours:
                nx, ny = current[0]+dx, current[1]+dy
                if not (0 <= nx < self.G and 0 <= ny < self.G): continue
                if self.grid[ny, nx] >= 1e8: continue  # hard obstacle
                move_cost = np.sqrt(dx**2 + dy**2) + self.grid[ny, nx]
                new_g = g + move_cost
                neighbour = (nx, ny)
                if neighbour not in g_cost or new_g < g_cost[neighbour]:
                    g_cost[neighbour] = new_g
                    f = new_g + heuristic(neighbour, goal)
                    heapq.heappush(open_set, (f, new_g, neighbour))
                    came_from[neighbour] = current

        return []   # no path found

    def path_cost(self, path: list) -> float:
        if not path: return float('inf')
        cost = 0
        for i in range(1, len(path)):
            dx = path[i][0] - path[i-1][0]
            dy = path[i][1] - path[i-1][1]
            cost += np.sqrt(dx**2+dy**2) + self.grid[path[i][1], path[i][0]]
        return cost


def demo_astar():
    print("\n" + "="*58)
    print("  A* THREAT-AWARE PATH PLANNING β€” Bestaoui Sebbane Β§10.1")
    print("="*58)
    planner = AStarPlanner(grid_size=50)
    # SAM site at centre (25,25): kill radius=5, detect radius=12
    planner.add_sam_site(25, 25, r_kill=5, r_detect=12, w_threat=8.0)

    start = (2, 25)    # left edge, mid-height
    goal  = (47, 25)   # right edge, mid-height β€” direct path goes through SAM

    path = planner.plan(start, goal)
    cost = planner.path_cost(path)

    # Compare to naive direct path cost (ignores SAM)
    direct_path = [(x, 25) for x in range(2, 48)]
    direct_cost = planner.path_cost(direct_path)

    print(f"\n  SAM site at (25,25): kill_r=5, detect_r=12, w=8.0")
    print(f"  Start: {start} β†’ Goal: {goal}")
    print(f"\n  A* (threat-aware) path:")
    print(f"    Waypoints: {len(path)} nodes")
    print(f"    Path cost: {cost:.1f}")
    print(f"    Path goes {'ABOVE' if any(p[1]<22 for p in path) else 'BELOW'} the SAM site")
    print(f"\n  Direct path (no threat avoidance):")
    print(f"    Waypoints: {len(direct_path)} nodes")
    print(f"    Path cost: {direct_cost:.1f}  (likely includes SAM penalty)")
    print(f"\n  A* correctly avoids the SAM exclusion zone.")


# ════════════════════════════════════════════════════════════════
# PART 3 β€” Minimal Behaviour Tree Engine
# MDPI Autonomous Control Ch.7
# ════════════════════════════════════════════════════════════════

class Status(Enum):
    SUCCESS = "SUCCESS"
    FAILURE = "FAILURE"
    RUNNING = "RUNNING"


class BTNode:
    """Base class for all BT nodes."""
    def tick(self, state: dict) -> Status:
        raise NotImplementedError


class Sequence(BTNode):
    """Execute children in order; fail on first child failure."""
    def __init__(self, children, name=""):
        self.children = children; self.name = name

    def tick(self, state):
        for child in self.children:
            result = child.tick(state)
            if result != Status.SUCCESS:
                return result
        return Status.SUCCESS


class Selector(BTNode):
    """Try children in order; succeed on first child success."""
    def __init__(self, children, name=""):
        self.children = children; self.name = name

    def tick(self, state):
        for child in self.children:
            result = child.tick(state)
            if result != Status.FAILURE:
                return result
        return Status.FAILURE


class Condition(BTNode):
    """Check a predicate on the state dict."""
    def __init__(self, predicate, name=""):
        self.pred = predicate; self.name = name

    def tick(self, state):
        result = Status.SUCCESS if self.pred(state) else Status.FAILURE
        return result


class Action(BTNode):
    """Execute a function and return its status."""
    def __init__(self, func, name=""):
        self.func = func; self.name = name

    def tick(self, state):
        return self.func(state)


def build_isr_bt():
    """Build the ISR mission behaviour tree from section 8.4."""
    # Actions
    def emergency_rtb(s):
        print(f"    [ACTION] EMERGENCY RTB β€” battery={s['battery']:.0%}")
        return Status.SUCCESS

    def transit_to_isr(s):
        print(f"    [ACTION] Transiting to ISR orbit")
        s['at_isr_orbit'] = True
        return Status.SUCCESS

    def capture_image(s):
        s['images_collected'] = s.get('images_collected', 0) + 1
        print(f"    [ACTION] Image captured (total: {s['images_collected']})")
        return Status.SUCCESS

    def transmit_data(s):
        if not s.get('comms_ok', True):
            print(f"    [ACTION] TRANSMIT FAILED β€” comms down")
            return Status.FAILURE
        print(f"    [ACTION] Data transmitted to GCS")
        return Status.SUCCESS

    def autonomous_loiter(s):
        print(f"    [ACTION] Switching to autonomous loiter")
        return Status.SUCCESS

    def evasive_manoeuvre(s):
        print(f"    [ACTION] Executing evasive manoeuvre β€” threat: {s.get('threat_type','unknown')}")
        s['threat_detected'] = False
        return Status.SUCCESS

    def continue_nominal(s):
        print(f"    [ACTION] Continuing nominal mission")
        return Status.SUCCESS

    # Build tree
    root = Selector([
        Sequence([                          # Emergency handler
            Condition(lambda s: s.get('battery',1.0) < 0.10, 'LowBattery'),
            Action(emergency_rtb, 'EmergencyRTB'),
        ], 'EmergencySeq'),

        Selector([                          # Mission fallback
            Sequence([                      # Primary mission
                Condition(lambda s: s.get('nav_valid', True), 'NavValid'),
                Condition(lambda s: s.get('fuel_ok', True), 'FuelOk'),
                Action(transit_to_isr, 'Transit'),
                Action(capture_image, 'CaptureImage'),
                Action(transmit_data, 'TransmitData'),
            ], 'PrimaryMission'),
            Sequence([                      # Degraded mode
                Condition(lambda s: not s.get('comms_ok', True), 'CommsLost'),
                Action(autonomous_loiter, 'AutoLoiter'),
            ], 'DegradedMode'),
        ], 'MissionSelector'),

        Selector([                          # Threat response
            Sequence([
                Condition(lambda s: s.get('threat_detected', False), 'ThreatDetected'),
                Action(evasive_manoeuvre, 'Evade'),
            ], 'EvasionSeq'),
            Action(continue_nominal, 'ContinueNominal'),
        ], 'ThreatResponse'),
    ], 'Root')

    return root


def demo_bt():
    print("\n" + "="*58)
    print("  BEHAVIOUR TREE β€” ISR MISSION EXECUTION")
    print("  MDPI Autonomous Control Ch.7")
    print("="*58)
    bt = build_isr_bt()

    scenarios = [
        ("Nominal flight",       {'battery':0.80, 'nav_valid':True, 'fuel_ok':True, 'comms_ok':True, 'threat_detected':False}),
        ("Comms blackout",      {'battery':0.65, 'nav_valid':True, 'fuel_ok':True, 'comms_ok':False, 'threat_detected':False}),
        ("Threat detected",     {'battery':0.70, 'nav_valid':True, 'fuel_ok':True, 'comms_ok':True, 'threat_detected':True, 'threat_type':'MANPAD'}),
        ("Critical battery",    {'battery':0.08, 'nav_valid':True, 'fuel_ok':True, 'comms_ok':True, 'threat_detected':False}),
    ]

    for name, state in scenarios:
        print(f"\n  Scenario: {name}")
        result = bt.tick(state)
        print(f"  BT result: {result.value}")


if __name__ == "__main__":
    demo_pomdp()
    demo_astar()
    demo_bt()
Expected Output POMDP: obs1 (THREAT) → P(combatant)=0.825 "GATHER MORE INFO" · obs2 (THREAT) → P(combatant)=0.971 "ENGAGE" · obs3 (SAFE) → P(combatant)=0.891 "GATHER MORE INFO" · obs4 (THREAT) → P(combatant)=0.976 "ENGAGE" · A*: path routes around SAM site (above or below), lower cost than direct route · BT: nominal→transit+capture+transmit; comms blackout→auto loiter; threat→evade; critical battery→RTB
  1. The POMDP belief update equation is structurally identical to the Bayesian update at the core of the EKF from Module 04. Derive the formal relationship: show that the EKF update step (Module 04, Theorem 4.1) is a special case of the POMDP belief update for the case where the state space is continuous and the distributions are Gaussian. Under what conditions does the general POMDP alpha-vector representation become computationally intractable, and what approximation does POMCP use to address this?
  2. The Artificial Potential Field method in section 8.6 is known to suffer from local minima. Construct a specific 2D scenario (draw it, or describe it precisely) where the UAV would get trapped in a local minimum that is not the goal. Derive the condition on the gradient vectors $\mathbf{F}_{att}$ and $\mathbf{F}_{rep}$ that creates the local minimum. Describe two algorithmic fixes, and explain how the Behaviour Tree architecture supports integrating a reactive APF layer with a global A* planner.
  3. The "accountability gap" in autonomous weapons arises from the diffusion of responsibility across the design, deployment, and command chain. Trace the accountability chain for a specific scenario: a POMDP-based autonomous targeting system with a 0.9 belief threshold engages a civilian vehicle with P(combatant) = 0.91. The belief was elevated by two consecutive "threat" observations from a miscalibrated classifier. Identify every party in the design/deployment chain who made a decision that contributed to this outcome, and for each, identify what engineering safeguard they could have implemented to prevent or detect the failure.
β—‚ Module 07: Deep Learning Module 09: Swarm Systems β–Έ