Model Predictive Control

Receding Horizon, Constraints, and Stability


Instructor: Hasan A. Poonawala

Mechanical and Aerospace Engineering
University of Kentucky, Lexington, KY, USA

Topics:
Receding horizon idea
From iLQR/LQR to MPC
Constraints
Asymptotic stability

The Core Idea

One-Shot vs. Receding Horizon

iLQR / LQR: solve once offline for a fixed initial state x0x_0.

What if the real state drifts from the plan? Disturbances, model mismatch, new information.

The feedback gain KkK_k handles small deviations — but what about large ones, or problems with constraints (other than dynamics)?

Model Predictive Control (MPC): re-solve the finite-horizon optimal control problem at every timestep, from the current measured state.

At time t:
  1. Measure current state  x_t
  2. Solve OCP over horizon [t, t+T]:  min J(x_t, u_{t:t+T-1})
  3. Apply only the first control:  u_t = u_0*
  4. Advance:  t ← t + 1,  repeat

Receding Horizon Diagram

At each timestep, the planning window shifts forward by one step.

t=0:  [●━━━━━━━━━━━━━━━━━━━━●]         solve, apply u_0*
t=1:    [●━━━━━━━━━━━━━━━━━━━━●]       solve, apply u_1*
t=2:      [●━━━━━━━━━━━━━━━━━━━━●]     solve, apply u_2*
           ↑ current state              ↑ horizon end

Key properties:

  • The horizon always looks TT steps into the future (same length each time)
  • Only u0*u_0^* is ever applied — the rest of the plan is discarded and recomputed
  • The re-solve at t+1t+1 can be warm-started from the shifted solution at tt

Discarding and recomputing seems wasteful. The payoff: the plan always reflects the current state, handles constraints, and adapts to disturbances.

From iLQR/LQR to MPC

Linear MPC

If dynamics are linear and cost is quadratic — no constraints:

xk+1=Axk+Buk,J=xTTPfxT+∑k=0T−1(xkTQxk+ukTRuk)x_{k+1} = Ax_k + Bu_k, \qquad J = x_T^T P_f x_T + \sum_{k=0}^{T-1}(x_k^T Q x_k + u_k^T R u_k)

Offline LQR: compute gains K0,K1,…,KT−1K_0, K_1, \dots, K_{T-1} once. At runtime: ut*=−K0xtu_t^* = -K_0 x_t (time-varying) or ut*=−K∞xtu_t^* = -K_\infty x_t (infinite horizon). Cost: 𝒪(1)\mathcal{O}(1) per timestep.

Linear MPC: solve the same problem online at each tt from xtx_t.

Without constraints: LQR and MPC give the identical control — MPC adds no benefit, just cost.

MPC is used when

  1. Constraints on uu or xx must be enforced
  2. Time-varying reference (track a trajectory, not just regulate to origin)
  3. Nonlinear dynamics (re-linearize online → Nonlinear MPC)
  4. Model mismatch (re-solve corrects for real-world drift)

Nonlinear MPC via iLQR

At each timestep tt:

  1. Warm-start: shift the previous solution forward by one step, append a guess for the new last control
  2. Run iLQR (a few iterations, not necessarily to convergence)
  3. Apply û0\hat{u}_0 to the real system; measure xt+1x_{t+1}
previous solution:  ū₀, ū₁, ū₂, ..., ū_{T-1}   (from time t)
warm-start at t+1:  ū₁, ū₂, ..., ū_{T-1}, ū_{T-1}   (shifted + hold last)
                     ↑ already near-optimal for new problem

Real-Time Iteration (RTI): one backward pass + one forward pass per timestep, regardless of convergence. Works well when the problem changes slowly between steps.

Solve time budget governs the number of iLQR iterations, not convergence. Horizon length TT is a design parameter: longer TT gives better plans but costs more to solve.

Constraints

Why Constraints Matter

Unconstrained iLQR / LQR ignores physical limits. Real systems have:

Constraint type Example
Input limits Motor torque saturation, voltage bounds: |uk|≤umax|u_k| \leq u_{\max}
State limits Joint angle limits, speed limits: xk∈𝒳x_k \in \mathcal{X}
Safety / obstacle Keep state outside a forbidden region
Terminal Reach a target set at k=Tk = T

Ignoring an active constraint (e.g., clipping the control post-hoc) invalidates the optimality of the LQR/iLQR solution. The optimal unconstrained u*u^* can violate limits by large margins.

Handling Constraints in MPC

Input constraints uk∈𝒰u_k \in \mathcal{U} (convex, e.g., box |u|≤umax|u| \leq u_{\max}):

  • Linear MPC: the constrained problem is a Quadratic Program (QP) — efficient solvers (OSQP, qpOASES) solve it in milliseconds
  • Nonlinear MPC with iLQR: project the feedforward onto 𝒰\mathcal{U} during the backward pass, or add a barrier/penalty term

State constraints xk∈𝒳x_k \in \mathcal{X} (harder — couples across timesteps):

  • Augmented Lagrangian (AL-iLQR): add λTc(xk)+ρ2∥c(xk)∥2\lambda^T c(x_k) + \frac{\rho}{2}\|c(x_k)\|^2 to the cost; alternate between iLQR and Lagrange multiplier updates
  • Barrier / interior-point: add −μlog(−c(xk))-\mu \log(-c(x_k)) to keep iterates feasible; anneal μ→0\mu \to 0

Handling constraints is the primary engineering challenge in MPC. Constraint satisfaction is exact at convergence; the number of iLQR iterations needed may increase.

Stability

The Finite-Horizon Trap

A finite horizon TT creates a problem: the optimizer doesn’t care what happens after step TT.

Pathological example: to minimize cost over [0,T][0, T], the optimizer might drive the state to a large value at k=Tk = T — exploiting the fact that the cost is zero beyond the horizon.

horizon:   [x₀ ――――――――――――→ x_T]  |  (nothing after here)
optimizer: reaches goal cheaply ...  |  then blows up

Consequence: MPC with a naive terminal cost Vf=0V_f = 0 may be unstable even when each individual solve is “optimal.”

Stability of MPC is not automatic. It requires careful design of the terminal cost and/or terminal constraint.

Fix 1: Terminal Cost

Choose Vf(xT)=xTTP∞xTV_f(x_T) = x_T^T P_\infty x_T, where P∞P_\infty is the infinite-horizon LQR solution:

P∞=Q+ATP∞A−ATP∞B(R+BTP∞B)−1BTP∞AP_\infty = Q + A^T P_\infty A - A^T P_\infty B(R + B^T P_\infty B)^{-1} B^T P_\infty A

Intuition: xTTP∞xTx_T^T P_\infty x_T is the optimal cost-to-go for the unconstrained infinite-horizon problem from xTx_T.

The finite-horizon cost plus this terminal cost approximates the full infinite-horizon cost:

JT(x0,𝐮)+Vf(xT)≈J∞(x0,𝐮∞*)J_T(x_0, \mathbf{u}) + V_f(x_T) \approx J_\infty(x_0, \mathbf{u}^*_\infty)

Effect: the optimizer is penalized for leaving xTx_T far from the origin — it accounts for the “cost tail” beyond the horizon, rather than ignoring it.

For linear systems without constraints, this makes MPC exactly equivalent to infinite-horizon LQR, regardless of horizon length TT. The terminal cost absorbs the truncation error completely.

Fix 2: Terminal Constraint

Require xT∈𝒳fx_T \in \mathcal{X}_f, where 𝒳f\mathcal{X}_f is a positively invariant set under a local controller u=Kxu = Kx:

x∈𝒳f⟹Ax+BKx∈𝒳fx \in \mathcal{X}_f \implies Ax + BKx \in \mathcal{X}_f

Common choice: 𝒳f={x:xTP∞x≤c}\mathcal{X}_f = \{x : x^T P_\infty x \leq c\} (an ellipsoid where the LQR gain K∞K_\infty keeps the state inside).

  • Inside 𝒳f\mathcal{X}_f: the LQR controller is feasible and stabilizing
  • The MPC trajectory must reach 𝒳f\mathcal{X}_f by step TT

Strictest case: 𝒳f={0}\mathcal{X}_f = \{0\}, i.e., xT=0x_T = 0 (equality constraint). Guarantees stability but drastically reduces the feasible set — requires large TT.

Terminal cost and terminal constraint are often used together: the terminal set 𝒳f\mathcal{X}_f ensures that VfV_f is a valid Lyapunov function within the set, and the constraint ensures the state reaches the set.

Practical MPC

Design Choices

Parameter Effect Typical guidance
Horizon TT Longer → better plans, better stability margins Start with T≈10T \approx 10–5050; increase until performance plateaus
Terminal cost VfV_f P∞P_\infty from LQR is the principled choice Always include it; Vf=0V_f = 0 is asking for trouble
Terminal constraint Guarantees stability with shorter TT Use when TT is short and stability matters
Sample time Δt\Delta t Shorter → more accurate, more solves needed Match to fastest relevant dynamics
Iterations per step More → closer to optimal; limited by solve time RTI (1 iteration) often works well warm-started

Warm-Starting

The previous solution is nearly optimal for the next problem (state has moved only one step).

Shift and hold: u‾t+1=[u1*,u2*,…,uT−1*,uT−1*]\bar{u}_{t+1} = [u_1^*, u_2^*, \dots, u_{T-1}^*, u_{T-1}^*]

The last control is repeated (held) as a placeholder for the new horizon end.

Why it works: the optimal solution is a continuous function of xtx_t (in smooth problems). A small change in xtx_t → small change in the optimal plan → warm-start is close to the new optimum → very few iLQR iterations needed.

Real-Time Iteration (RTI): one iLQR backward pass + one forward pass per MPC step, no convergence check. Widely used in fast MPC applications (automotive, drone control). Stability analysis is more involved but tractable.

Summary

LQR → iLQR → MPC

Method Dynamics Constraints When solved Stability
LQR Linear None Once, offline Guaranteed (infinite horizon)
iLQR Nonlinear None Once, offline Trajectory-local
Linear MPC Linear Yes Online, every step Guaranteed with terminal cost
Nonlinear MPC Nonlinear Yes Online, every step Guaranteed with terminal cost/constraint

MPC

The receding horizon principle:

  • Re-solve from the current state → robust to disturbances and model error
  • Apply only u0*u_0^* → plan adapts at every step
  • Discard and recompute → no commitment to a fixed trajectory

Stability requires design effort:

Terminal cost Vf=xTP∞x+Terminal constraint xT∈𝒳f\text{Terminal cost } V_f = x^T P_\infty x \quad + \quad \text{Terminal constraint } x_T \in \mathcal{X}_f

⟹\implies MPC value function J*(xt)J^*(x_t) is a Lyapunov function ⟹\implies asymptotic stability