View previous topic :: View next topic 
Author 
Message 
SteveS
Joined: 27 Oct 2003 Posts: 126

Filters (long) 
Posted: Thu Jun 03, 2004 8:31 am 


It seems that filtering comes up pretty often, so I thought I'd write up some real basics on digital filtering (please remember this is basic  there are exceptions to everything I say here):
Filters are generally used to remove unwanted frequencies from a signal and can be low pass (LPFremove high frequencies  usually hf noise), high pass (HPFremove low frequencies  usually DC), bandpass (BPFremove high and low  you are looking for a particular frequency), or notch (remove a small band of frequencies  like 50/60 Hz line noise). I'm going to use LPF here as they are the most commonly needed.
Digital filters take discrete samples of a signal and use the new and past inputs and past outputs, each multiplied by various constants, and summed to produce a new output.
Digital filters can be further subdivided into two major groups: Infinite Impulse Response (IIR also known as recursive) and Finite Impulse Response (FIR). IIR filters are usually designed from an analog filter with the desired characteristics. Theoretically they have infinite memory of past inputs (hence the name); in practice it isn't infinite, but they do recover slower from an impulse (spike) on the input than an FIR filter. However, they generally require less calculations to achieve a particular filtering level (order). FIR filters only use the new input and some number of past inputs to calculate the new output, so a spike is 'forgotten' once it is old enough to not be included in the calculation. FIR filters are very versatile but generally require more calculations than an equivalent IIR filter.
There are many software packages available to generate the equations for a filter based on your requirements. However, the PIC doesn't really have the ability to do very complex filters, so don't go overboard! I've listed a few basic filters below that are easy to implement and use and require little knowledge about filtering.
Be warned that many filters may cause overflow since they rely on adding up several values then dividing. So if you use an eight bit converter be wary of using eight bit math. 10 or 12 bit converters and 16 bit math should work in most situations.
==== Note: I use the following abreviations: ===
Out0 = new output
Out1 = last ouput
Out2 = next to last output, etc
In0 = newest (current) input
In1 = last input, etc
a, b, c etc, are constants
===== Low Pass Filter =====
A really simple low pass filter that approximates a single pole (RC) analog filter:
Out0 = a * In0 + b * Out1
where a + b = 1.0 (a and b are fractions > 0 and < 1)
This is an IIR filter since it uses past outputs. Basically it averages the new input with last output. The smaller the 'a' term is, the slower the response of the filter. If you use constants divisible by a power of two it is a very fast calculation. A good starting point is to set a = 1/4 and b = 3/4:
Out0 = (In0 + Out1 + Out1 + Out1) / 4;
===== Averaging Filter =====
A simple FIR filter. This takes the average of several inputs to produce the output. In theory you would need past and future inputs but you can't, so you get a fixed time lag in this filter of half the number of samples used.
Again, if you use a poweroftwo number of samples, the calculations are faster:
Out0 = (In0 + In1 + In2 + In3) / 4;
You will find you will probably need more than just four samples to get much filtering. You can also shape the filter by multiplying each input by different coefficents, but it complicates the calculations and can introduce worse truncation and/or overflow problems.
===== Median Filter =====
A type of FIR filter with very interesting characteristics. A median filter takes (generally) an odd number of inputs (5  7 is a good starting point), sorts them by value and takes the median (middle) value as the new output. This filter is extremely effective at removing noise spikes that last only a few samples. At the same time the filter will faithfully pass step changes in the signal. Other filters round off steps. The median is not as easy to code as other filters. The data must be sorted each time;, consequently it doesn't have the flow of the other filters. On the positive side, the median filter will not overflow.
===== Other filters =====
Just for completeness sake, let me add one of my favorites: the LockIn Amplifier, or Demodulation filter (or other names). This is a special bandpass filter for modulated signals. If you can modulate the sensor at a fixed frequency, then you can use that same frequency to effectively create a very narrow bandpass filter. These filters can pick out signals that are completely lost in noise. Do a Google search for more info.
So, which to use?
 The IIR LPF is simple and effective and is good at removing general noise from a signal.
 The FIR can be tailored to most anything, but requires more code. It's great if you need true averages. It would also be good for something like a weigh scale, where the input data jumps around (with each new weighing), but needs to averaged during each weighing.
 The median filter is when the data gets a lot of short duration spike noise. Some sensors have outputs that behave this way.
Feel free to add to (or correct) this thread.
 SteveS
Last edited by SteveS on Thu Jul 01, 2004 9:45 am; edited 2 times in total 


SherpaDoug
Joined: 07 Sep 2003 Posts: 1633 Location: Cape Cod Mass USA

Re: Filters (long) 
Posted: Thu Jun 03, 2004 9:37 am 


SteveS wrote:  A median filter takes (generally) an odd number of inputs (5  7 is a good starting point), sorts them by value and takes the median (middle) value as the new input.
 SteveS 
Make that "and takes the median value as the new OUTPUT. 


SteveS
Joined: 27 Oct 2003 Posts: 126


Posted: Thu Jun 03, 2004 1:12 pm 


corrected thanks! 


SimpleAsPossible
Joined: 19 Jun 2004 Posts: 21

Minor correction 
Posted: Sat Jun 19, 2004 6:13 pm 


Out0 = (In0 + In1 + In2 + In3) << 2;
and
Out0 = (In0 + Out1 + Out1 + Out1) << 2;
should be
Out0 = (In0 + In1 + In2 + In3) >> 2;
and
Out0 = (In0 + Out1 + Out1 + Out1) >> 2;
to divide the sums by 4 rather than multiply.
Or, if you prefer a slightly clearer (to the nonC minded) version:
Out0 = (In0 + Out1 + Out1 + Out1) / 4; 


SteveS
Joined: 27 Oct 2003 Posts: 126


Posted: Thu Jul 01, 2004 9:44 am 


DOH!
I do that all the time. I also do this:
x>>2;
thinking
x= x>>2;
Fixed. 


Neutone
Joined: 08 Sep 2003 Posts: 839 Location: Houston

Variations on a theme 
Posted: Mon Mar 14, 2005 10:55 am 


In attempting to filter a 10 bit ADC and produce a 12 bit output signal I found I could get a good result using a different version of the "Low Pass" filter SteveS mentioned. With the elements of time and oversampeling the signal output really is more than 10 bits for a steady input. Even with oversampeling the output is not going to get much more than 2 added bits of resolution. Using a variable voltage input you can actually adjust the voltage to produce almost every 12bit value. For the values of zero and full scale input the filter produces an error of 2 bits.
A simple version that runs very fast is this.
Code:  Filtered_Result=Filtered_Result(Filtered_Result>>2)+read_adc(); 
A version that takes more time to process but gives better filtering. The 2bit error at end of scale is removed.
Code:  Intermediate_Result=Intermediate_Result(Intermediate_Result>>6)+read_adc();
Filtered_Result=Intermediate_Result>>4; 



Storic
Joined: 03 Dec 2005 Posts: 182 Location: Australia SA

Re: Variations on a theme 
Posted: Sun Mar 19, 2006 6:10 am 


Neutone wrote:  In attempting to filter a 10 bit ADC and produce a 12 bit output signal I found I could get a good result using a different version of the "Low Pass" filter SteveS mentioned. With the elements of time and oversampeling the signal output really is more than 10 bits for a steady input. Even with oversampeling the output is not going to get much more than 2 added bits of resolution. Using a variable voltage input you can actually adjust the voltage to produce almost every 12bit value. For the values of zero and full scale input the filter produces an error of 2 bits.
A simple version that runs very fast is this.
Code:  Filtered_Result=Filtered_Result(Filtered_Result>>2)+read_adc(); 
A version that takes more time to process but gives better filtering. The 2bit error at end of scale is removed.
Code:  Intermediate_Result=Intermediate_Result(Intermediate_Result>>6)+read_adc();
Filtered_Result=Intermediate_Result>>4; 

I would like to thank Neutone for the above analog filter and SteveS who started this thread, I was stuck for days tring to stop the Analog from hunting, (I was measuring a 10K thermistor and getting a variance of up to 1'C ie 25'C would range from 24'C to 26'C, now with the filter
This is my final little piece of code for a 10K thermister, in order to slow down any rapid change, I read the value 10 times and take an average, I then used 2 x old results to average out reads. before I would get the peaks of up to 1deg, now with the filter + average 10 reads and further average of 2 reads from Old_result1 and Old_result2, I see any peaks of .1 deg
Code:  /***********************************************************
* Read Tempature 10K thermister filtered code
***********************************************************/
void Set_Temp ( ) {
int16 Ref_10K_25deg = 715; // 20=262(748) 30=237(677)
int16 Intermediate_Result = 0;
signed int16 Final_Result = 0;
int16 min=65535;
int16 max=0;
int8 i;
set_adc_channel(0);
delay_ms(100);
for(i=0; i<=10; ++i) {
delay_ms(100);
Intermediate_Result=Intermediate_Result(Intermediate_Result>>6)+read_adc();
if(Intermediate_Result<min)
min=Intermediate_Result;
if(Intermediate_Result>max)
max=Intermediate_Result;
}
//convert Average read to value, deduct from ref to reverse the negative action
Final_Result=(((min + max)/2)>>2)Ref_10K_25deg;
//convert to readable temp(Final_Result + 25'C), add to old convert, round to 0.1 value
Temp_Result =(((final_Result*13)+2500)+old_result1)/20*10;
old_result1 = Temp_Result; // get 1st result value
//convert new Temp result from Old result 1 and 2 and round to 0.1 value
Temp_Result =(old_result1 + old_result2)/20*10;
old_result2 = Temp_Result; // get 2st result value
......// aditional error check for open/short ciruits
}

Andrew _________________ What has been learnt if you make the same mistake? 


Ken Johnson
Joined: 23 Mar 2006 Posts: 197 Location: Lewisburg, WV


Posted: Wed Sep 20, 2006 7:03 am 


Can't take credit  I read this somewhere  but I use this digital exponential filter a lot:
Value += Filter * (newValue  Value);
All variables are floats (calculation is still pretty fast). You can adapt this to "int" types if time is of the essence.
Filter <= 1.0 (smaller Filter gives more filtering).
Works great!
Ken 


mthornton
Joined: 22 Jan 2008 Posts: 4 Location: Canal Flats BC

integer version of above averaging method 
Posted: Thu Feb 21, 2008 5:26 pm 


Value += FilterK * (newValue  Value);
where FilterK <1
this works great, but requires floating point variables & math

so do this instead
multiply FilterK by 10 (I often use .9, so make =9)
Value = (10*Value+(9*(newValue  Value)))/10;
works fine on a 10 bit value using unsigned int16 variables
Also... Try playing with multipliers > 10
(works good x50). Not much higher or you overflow 16 bit unsigned values when working with 10 bit raw a/d newValue 


alan69
Joined: 09 Jun 2010 Posts: 1 Location: London



