Filtering Sensor Data: From Moving Averages to Complementary Filters

Why filtering is needed (and what it can and cannot do)

Filtering makes sensor data more usable by reducing unwanted variations while preserving the information your controller or estimator needs. It does not create “perfect” measurements: it trades off noise reduction against delay (lag) and loss of fast dynamics. In robotics, filtering is typically needed for three practical reasons:

• Noise reduction: Raw readings often jitter. If you feed jitter directly into control loops, you can excite oscillations or cause unnecessary actuator wear.
• Differentiation stabilization: Velocity from position or angular rate from angle requires differentiation, which amplifies high-frequency noise. A small amount of noise in position can become large noise in velocity.
• Outlier suppression: Occasional spikes (e.g., electrical glitches, missed edges, transient vibrations) can cause large, brief errors that should not dominate your estimate.

A useful mental model: filtering is a controlled “forgetting” of rapid changes. The key is to forget noise faster than you forget real motion.

Core filters and intuition

1) Moving average (boxcar) filter

Intuition: Average the last N samples. Random noise tends to cancel out; persistent changes remain. This is easy to implement and works well for reducing white-ish noise, but it introduces delay roughly proportional to the window length.

Definition:

y[k] = (1/N) * sum_{i=0..N-1} x[k-i]

Step-by-step implementation (efficient running sum):

Continue in our app.

• Listen to the audio with the screen off.
• Earn a certificate upon completion.
• Over 5000 courses for you to explore!

Or continue reading below...

Download the app

• Keep a circular buffer of length N.
• Maintain a running sum S.
• On each new sample x: subtract the oldest sample from S, add x, store x into the buffer, output y = S/N.

// Moving average with running sum (O(1) per sample)  (C-like pseudocode)  const int N = 8;  float buf[N] = {0};  int idx = 0;  float S = 0;  float moving_average(float x) {    S -= buf[idx];    buf[idx] = x;    S += x;    idx = (idx + 1) % N;    return S / N;  }

Practical notes: A moving average has a frequency response with “notches” and can distort some periodic signals. If you see unexpected attenuation at certain frequencies (e.g., wheel speed ripple), consider exponential smoothing instead.

2) Exponential smoothing (first-order low-pass / IIR)

Intuition: Blend the new sample with the previous filtered value. This behaves like a simple low-pass filter with less memory and smoother behavior than a boxcar average.

Update equation:

y[k] = y[k-1] + α * (x[k] - y[k-1])   where 0 < α < 1

Step-by-step tuning from a time constant:

• Choose a time constant τ (seconds) that represents how quickly you want the filter to respond.
• Given sample period dt, compute α = dt / (τ + dt).
• Initialize y[0] to the first sample to avoid a long startup transient.

// Exponential smoothing (low-pass)  float y = 0;  bool initialized = false;  float lowpass(float x, float dt, float tau) {    float alpha = dt / (tau + dt);    if (!initialized) { y = x; initialized = true; }    y = y + alpha * (x - y);    return y;  }

How to think about α: Larger α tracks changes faster (less smoothing). Smaller α smooths more but increases lag.

3) Median filter for spikes (outlier suppression)

Intuition: Replace the current value with the median of the last N samples. A single spike does not affect the median much, so this is excellent for removing impulsive noise while preserving edges better than averaging.

Typical use: When you see occasional large spikes that are clearly not real motion (e.g., a one-sample glitch). Median filtering is common before further processing.

Step-by-step (small window):

• Choose a small odd window size (e.g., N=3, 5, or 7).
• Collect the last N samples.
• Sort them (for small N, insertion sort is fine).
• Output the middle value.

// Median of 5 samples (simple approach)  float median5(float a[5]) {    float b[5];    for (int i=0;i<5;i++) b[i]=a[i];    // insertion sort    for (int i=1;i<5;i++){      float key=b[i]; int j=i-1;      while (j>=0 && b[j]>key){ b[j+1]=b[j]; j--; }      b[j+1]=key;    }    return b[2];  }

Practical notes: Median filters can introduce “stair-step” behavior on smooth signals if the window is too large. Keep the window minimal for spike removal, then apply a low-pass if needed.

4) Derivative estimation with smoothing (avoiding noise amplification)

Problem: The naive discrete derivative, (x[k]-x[k-1])/dt, amplifies high-frequency noise. If x is noisy, the derivative becomes very noisy.

Two practical patterns:

• Low-pass then differentiate: Smooth x first, then compute the derivative.
• Differentiate then low-pass: Compute the raw derivative, then smooth it (often easier to retrofit).

Step-by-step example (low-pass then differentiate):

• Apply exponential smoothing to position/angle: x_f.
• Compute derivative: v = (x_f[k] - x_f[k-1]) / dt.
• Optionally low-pass v again if the derivative is still too noisy.

// Smoothed derivative  float xf_prev = 0;  bool init = false;  float smoothed_derivative(float x, float dt, float tau_x) {    static float xf = 0;    float alpha = dt / (tau_x + dt);    if (!init) { xf = x; xf_prev = x; init = true; }    xf = xf + alpha * (x - xf);    float v = (xf - xf_prev) / dt;    xf_prev = xf;    return v;  }

Rule of thumb: If you need a derivative for control (e.g., damping), it is usually better to accept some lag than to inject high-frequency noise into actuators.

Complementary filter: combining fast gyro behavior with long-term reference

Some signals are best estimated by combining two sources with different strengths. A classic case is estimating tilt (roll/pitch) by combining:

• Gyro integration: Good short-term smoothness and responsiveness, but it drifts over time.
• Accelerometer tilt from gravity: Provides a long-term reference (gravity direction), but is noisy and corrupted by linear acceleration and vibration.

Concept: Use a high-pass behavior on the gyro-derived angle (keep fast changes) and a low-pass behavior on the accelerometer-derived angle (keep slow, drift-correcting information). The two parts are “complementary” because they sum to approximately 1 across frequencies.

Discrete-time complementary filter (common form):

// angle estimate  angle[k] = (1-β) * (angle[k-1] + gyro_rate[k]*dt) + β * accel_angle[k]  where 0 < β < 1

Here, (angle[k-1] + gyro_rate*dt) is the integrated gyro prediction, and accel_angle is the tilt estimate from the accelerometer. The parameter β controls how strongly you pull the estimate back toward the accelerometer reference.

Step-by-step implementation outline:

• Compute dt from your real sampling interval (do not assume it is constant unless you guarantee it).
• Integrate gyro rate to get a predicted angle: angle_pred = angle + gyro*dt.
• Compute accelerometer tilt angle (e.g., using atan2 of axes consistent with your frame convention).
• Blend: angle = (1-β)*angle_pred + β*accel_angle.
• Optionally gate or reduce β when you detect strong linear acceleration (accelerometer magnitude far from expected gravity), because accel tilt is less trustworthy then.

// Complementary filter (single axis tilt)  float angle = 0;  bool init = false;  float comp_filter(float gyro_rate, float accel_angle, float dt, float tau) {    // tau sets how quickly accel corrects drift (larger tau => trust gyro longer)    float beta = dt / (tau + dt);  // small beta for large tau    if (!init) { angle = accel_angle; init = true; }    float angle_pred = angle + gyro_rate * dt;    angle = (1.0f - beta) * angle_pred + beta * accel_angle;    return angle;  }

Interpreting the tuning parameter: Using beta = dt/(tau+dt) makes tau a time constant for drift correction. If tau is 1–2 seconds, the estimate will slowly correct toward the accelerometer over that time scale; if tau is 0.1 seconds, it will correct much more aggressively (and will be more sensitive to accel disturbances).

Practical tuning guidance

Choose cutoff/time constant based on robot dynamics

Filtering should be tied to the fastest real motion you care about, not the noisiest part of the signal.

• Identify the bandwidth of interest: For a balancing robot, tilt changes quickly; for a slow mobile base, heading/velocity may change more slowly.
• Pick a time constant τ (or cutoff) that passes that motion: If your robot needs to respond to changes around a few Hz, a very large τ will cause sluggish behavior.
• Start conservative: Begin with moderate smoothing, then reduce smoothing until noise becomes problematic.

Helpful mapping: for a first-order low-pass, cutoff frequency relates to time constant approximately by f_c ≈ 1/(2π τ). You can think in either domain: “I want changes faster than X Hz to be attenuated” or “I want the filter to settle in about Y seconds.”

Recognize over-filtering vs under-filtering

Test with step and sinusoidal inputs

Use simple test signals to see the trade-offs clearly. You can do this with logged data and offline replay, or in real time with a controlled motion.

Step test (sudden change):

• Apply a sudden change in the measured quantity (e.g., rotate the robot quickly to a new tilt angle and hold).
• Observe rise time and overshoot (filters should not overshoot by themselves, but combined with other processing they may).
• Measure lag: how long until the filtered signal reaches, say, 90% of the final value.
• If lag is too large, reduce smoothing (smaller N, larger α, smaller τ).

Sinusoidal test (periodic motion):

• Move the robot with an approximately sinusoidal motion at a frequency relevant to operation (e.g., 0.5 Hz, 1 Hz, 2 Hz).
• Compare amplitude and phase between raw and filtered signals.
• If the filtered signal’s amplitude is too attenuated or phase lag is too large at that frequency, your cutoff is too low (too much filtering).

Tip: When tuning a complementary filter, do the sinusoidal test both with gentle motion (accelerometer mostly measuring gravity) and with aggressive motion (accelerometer disturbed). You should see that aggressive motion benefits from trusting the gyro more (smaller β / larger τ), possibly with gating.

Implementation considerations (real-time, numeric format, cost)

Floating-point vs fixed-point

• Floating-point: Simplifies implementation of α, β, and trigonometric functions. On many modern MCUs, single-precision float is fast enough for common sensor rates.
• Fixed-point: Useful when hardware floating-point is slow or when you need deterministic performance. Represent α and β as Q-format constants (e.g., Q15). Be careful with overflow in running sums (moving average) and with scaling of dt.

For fixed-point exponential smoothing, a common pattern is:

// Q15 example: y += (alpha * (x - y)) >> 15

Computational cost and memory

• Moving average: O(1) per sample with running sum, but needs a buffer of size N.
• Exponential smoothing: O(1) per sample, minimal memory (just previous output).
• Median filter: Needs a buffer and sorting; keep N small to control cost.
• Complementary filter: Very cheap (a few multiplies/adds), but computing accel_angle may require atan2. If atan2 is expensive, consider computing it at a lower rate or using an approximation, while still running the blend each cycle.

Maintaining real-time behavior

• Use the actual dt: Filters depend on timing. If your loop jitter is nontrivial, compute dt from timestamps and update α/β accordingly.
• Avoid blocking operations: Sorting for median filters or heavy math should not cause missed deadlines. If needed, run expensive parts at a lower rate.
• Initialize safely: Set filtered state to the first measurement (or a known reasonable value) to avoid long transients.
• Log and replay: Record raw sensor streams and run filters offline to tune parameters without risking unstable on-robot behavior.