Safe Online Control-Informed Learning
1Purdue University · IEEE Control Systems Letters · arXiv:2512.13868
Abstract
This paper proposes a Safe Online Control-Informed Learning framework for safety-critical autonomous systems. The framework unifies optimal control, parameter estimation, and safety constraints into an online learning process. It employs an extended Kalman filter to incrementally update system parameters in real time, enabling robust and data-efficient adaptation under uncertainty. A softplus barrier function enforces constraint satisfaction during learning and control while eliminating the dependence on high-quality initial guesses. Theoretical analysis establishes convergence and safety guarantees, and the framework's effectiveness is demonstrated on cart-pole and robot-arm systems.
Method
Learning Inside the Limits
The robot is an optimal control system whose dynamics, objective and safety limits are all unknown. Safe OCIL unifies optimal control, parameter estimation and safety constraints into one online learning process: the parameters are updated in real time, one measurement at a time, and the constraints hold during learning as well as control.
The constrained system is turned into an unconstrained one: every inequality is folded into the objective as a softplus barrier, a differentiable approximation of ReLU. Four things follow from that choice.
- It is finite outside the safe set. An initial guess that violates the constraints still returns a trajectory, and the optimisation is guided back toward feasibility. A logarithmic barrier is undefined there, so it needs a feasible starting point and can fail outright.
- It is differentiable everywhere. ReLU is not. Differentiability is what makes the gradient of the trajectory computable at all, and the filter cannot run without it.
- Its sharpness is a dial. Shrinking α and β brings the relaxed problem closer to the constrained one, so accuracy can be traded against the stability of the solve.
- The limits can be learned like anything else. Being ordinary terms in the objective, they carry gradients too — so the safe set itself is estimated rather than assumed.
Interactive
Wrong Guess, Safe Finish
Two learners start from the same wrong parameters and see the same demonstration. The difference is that one of them carries safety constraints and the other does not. Drag the slider to feed them measurements, one at a time, and watch both estimates improve.
Safety does not have to be learned first. At the very first update, before either has seen a measurement, the barrier already keeps its trajectory inside the limits.
Results
Fewer Data, Lower Loss
One hundred random initial guesses per system. The line is the mean over those runs and the band is ±3 standard deviations. Safe OCIL updates after every measurement; Safe PDP updates once per full pass over the demonstration, which is why its markers sit one horizon apart.
Two things the repository’s own scripts do, worth knowing before reading the gap. Safe PDP’s initial guesses are drawn from a spread four times narrower than Safe OCIL’s, so it starts nearer the truth. And eleven of its hundred cart-pole runs diverge; the paper’s plotting script drops them, and so does this chart.
Where each method leaves the safe set
These are the trajectories at the end of learning. The tick on each bar is the true limit, and the bar is how far the worst point of the trajectory actually went. A bar that stops at the tick stayed inside. One scale for both systems, so the lengths compare.
OCIL has no limits to respect at all. Safe OCIL respects the ones it has estimated, which are unknown parameters like everything else — so how well it holds the true limit depends on how well it has learned that limit. On the cart-pole its estimate lands just inside the truth, 4.97 against 5 and 0.77 against 0.8, and nothing crosses. On the robot arm, learning from noisy measurements, the estimated torque limit settles at 1.07 — so the trajectory does go past the true 1.0, but only by about as much as the estimate itself is loose.
How noisy can the measurements be?
The same cart-pole run five times over, with more and more measurement noise. At the hardest level the noise is larger than the whole margin the cart has — σ = 1.0 against ±0.8 m of travel. Every curve still falls.
Timing
Milliseconds to Spare
Wall-clock time for a complete update, averaged over every measurement of all 100 trials. Each bar is one control period, and the coloured part is what learning costs inside it.
Measured on the saved runs in the repository. The optimal control solve is the balance of the update, and it is where most of the time goes — around 70 % on both systems, while the filter itself is under a millisecond. Table II of the paper reports the same figures alongside plain OCIL, which is not meaningfully cheaper despite carrying no constraints at all.
Citation
@article{zhou2025safe,
title = {Safe Online Control-Informed Learning},
author = {Zhou, Tianyu and Liang, Zihao and Lu, Zehui and Mou, Shaoshuai},
journal = {IEEE Control Systems Letters},
volume = {9},
pages = {3083--3088},
year = {2025},
doi = {10.1109/LCSYS.2025.3648637}
}