Iterative Learning Control: Enhancing Performance Through Repetition

Iterative Learning Control (ILC) is a powerful control strategy designed to improve system performance in repetitive tasks. This article provides a comprehensive overview of ILC, exploring its principles, applications, and various design considerations.

Introduction to Iterative Learning Control

Iterative Learning Control (ILC) is an open-loop control approach of tracking control for systems that work in a repetitive mode. It's an improvement in run-to-run control. The core idea behind ILC is to leverage the repetitive nature of a task to iteratively refine the control signal, ultimately achieving high precision. Unlike traditional feedback control, which reacts to errors in real-time, ILC learns from past iterations to anticipate and compensate for errors proactively.

Applications of ILC

ILC finds applications in various fields where systems operate repetitively, including:

Robot arm manipulators: Performing the same pick-and-place operations repeatedly.
Chemical batch processes: Executing the same sequence of steps for each batch.
Reliability testing rigs: Subjecting components to repeated stress cycles.
Manufacturing robots: Welding the same parts in the same place each time.
Antenna pointing: Tracing the same path through the sky.
Medical robots: Performing pre-defined surgical procedures.
Subway trains: Accelerating and decelerating to ensure smooth operation.

In each of these tasks, the system is required to perform the same action over and over again with high precision. Repetition allows the system to sequentially improve tracking accuracy, in effect learning the required input needed to track the reference as closely as possible.

The ILC Learning Process

The learning process uses information from previous repetitions to improve the control signal, ultimately enabling a suitable control action to be found iteratively. The system sequentially improves tracking accuracy, in effect learning the required input needed to track the reference as closely as possible.

Read also: Understanding PLCs

Mathematical Representation

The ILC update law can be expressed as:

u_{p+1} = u_p + f(e_p)

where:

u_p is the input to the system during the pth repetition.
e_p is the tracking error during the pth repetition.
f is a design parameter representing operations on e_p.

Achieving perfect tracking through iteration is represented by the mathematical requirement of convergence of the input signals as p becomes large, whilst the rate of this convergence represents the desirable practical need for the learning process to be rapid. There is also the need to ensure good algorithm performance even in the presence of uncertainty about the details of process dynamics. The operation f is crucial to achieving design objectives (i.e. robustness).

Convergence and Learning Rate

Two key aspects of ILC algorithm performance are:

Convergence: Ensuring that the input signals converge as the number of iterations increases.
Learning Rate: Achieving a rapid convergence rate for practical applications.

Robustness

Good algorithm performance must be maintained even with uncertainty about the details of process dynamics.

Read also: Learning Resources Near You

ILC Algorithm Design

The design of the control algorithm still leaves many decisions to be made to suit the application.

Filter

In many cases a low-pass filter is added to the input to improve performance:

u_{p+1} = u_p + L * f(e_p)

where L is a low-pass filtering matrix.

Types of ILC Algorithms

There are several variations of ILC algorithms, each with its own strengths and weaknesses:

Model-Based ILC

Model-based ILC algorithms rely on a mathematical model of the system to design the learning function.

Read also: Learning Civil Procedure

Inverting the Dynamics

One approach is to invert the dynamics of the system, where the learning function L = G inverse. However, this can be risky due to model inaccuracies.

Gradient Descent

Another model-based approach is based on gradient descent, where a cost function J is minimized with respect to the input U.

Model-Free ILC

Model-free ILC algorithms do not require a mathematical model of the system.

D-type or PD-type ILC

Model-free ILC update law, specified as a nonnegative scalar. D-type, or PD-type ILC. applicable only to SISO systems. respectively.

This approach is suitable for single-input single-output (SISO) systems.

Implementation of ILC

Simulink Implementation

To implement ILC in Simulink®, use the Iterative Learning Control block.

ILC Block Parameters

Duration of one ILC iteration: Specifies the duration of one ILC iteration as a positive scalar.
Ts: the model-based ILC algorithm.
ILC mode signal: Specify the ILC mode signal at this port. switches between two modes: control and reset. to use in the next iteration. In the reset mode, the block output is 0. and from 1 to 0 to switch to the reset mode. to its initial condition by the nominal controller.
Data Types: single | double
Output: Updated control signal for the kth iteration.

Advantages of ILC

High precision: ILC can achieve very high tracking accuracy in repetitive tasks.
Handles unknown disturbances: ILC can learn to compensate for repetitive disturbances that are difficult to model.
Simple implementation: ILC algorithms can be relatively simple to implement.

Disadvantages of ILC

Requires repetitive tasks: ILC is only suitable for systems that perform the same task repeatedly.
Sensitive to changes in the system: If the system dynamics change, the ILC algorithm may need to be retrained.
Can be slow to converge: The learning process can be slow, especially for complex systems.
Doesn't work very well if stochastic disturbances dictate the control performance or a task is to be performed only a small number of times.
Each ILC iteration may imply performing destructive machining on expensive material with suboptimal result before convergence. This may only be cost effective if the task is to be performed many times after an initial "tuning" by means of ILC.

Combining ILC with Feedback Control

ILC can be effectively combined with traditional feedback control to achieve even better performance. The feedback controller provides initial stabilization and disturbance rejection, while the ILC algorithm learns to compensate for repetitive errors.

Example: Iterative-Learning Control (ILC) iteration scheme

In this example, we will design an Iterative-Learning Control (ILC) iteration scheme. ILC can be thought of as a simple reinforcement-learning strategy that is suitable in situations where a repetitive task is to be performed multiple times, and disturbances acting on the system are also repetitive and predictable but unknown. Multiple versions of ILC exists, in this tutorial we will consider a heuristic scheme as well as a model-based scheme.

a_{k+1}(q) = Q(q) * (a_k(q) + L(q) * e_k(q))

where $q$ is the time-shift operator, $G(q)$ is the transfer function from the reference $r$ to the output $y$, i.e, typically a closed-loop transfer function, $ek$ is the control error and $ak$ is the ILC adjustment signal, an additive correction to the reference that is learned throughout the ILC iterations in order to minimize the control error. $Q(q)$ and $L(q)$ are stable filters that control the learning dynamics. Interestingly, these filters does not have to be causal since they operate on the signals $e$ and $a$ between ILC iterations, when the whole signals are available at once for acausal filtering. In simulation (the rollout $yk = G(q) (r + ak)$ is simulated), this scheme is nothing other than an open-loop optimal-control strategy, while if $yk = G(q) (r + ak)$ amounts to performing an actual experiment on a process, ILC turns into episode-based reinforcement learning or adaptive control.

System to Control

The system to control in this example is a double-mass system with a spring and damper in between. This system is a common model of a servo system where one mass represents the motor and the other represents the load. The spring and damper represents a flexible transmission between them. We will create two instances of the system model. $G$ represents the nominal model, whereas $G_{act}$ represents the actual (unknown) dynamics. This simulates a model-based approach where there is a slight error in the model. The error will lie in the mass of the load, simulating, e.g., that the motor is driving a heavier load than specified.

Non-causal filtering

For ILC to work well, we define two helper functions. One that applies a zero-phase filter by filtering both forwards and backwards (filtfilt). This is possible since ILC operates on signals offline, between iterations in the ILC scheme. We also define a special lsim that handles non-causal systems to allow "lookahead" into the future. This typically improves the performance of ILC by quite a lot, and is once again possible since ILC operates on prerecorded signals.

Choosing filters

The next step is to define the ILC filters $Q(q)$ and $L(q)$. The filter $L(q)$ acts as a frequency-dependent step size. To make the procedure take smaller steps, simply scale $L$ by a constant < 1. Scaling down $L$ makes the learning process slower but more robust. A heuristic choice of $L$ is some form of scaled lookahead, such as $0.5z^l$ where $l \geq 0$ is the number of samples lookahead. A model-based approach may use some form of inverse of the system model, which is what we will use here. [nonlinear]

The filter $Q(q)$ acts to make the procedure robust w.r.t. noise and modeling errors. $Q$ has a final say over what frequencies appear in $a$ and it's good to choose $Q$ with low-pass properties.

Conclusion

Iterative Learning Control is a powerful technique for improving the performance of systems that operate in a repetitive manner. By learning from past iterations, ILC can achieve high precision and compensate for unknown disturbances. While ILC has its limitations, it can be a valuable tool in a variety of applications, especially when combined with traditional feedback control.

tags: #iterative #learning #control #overview