AD Digital Filter

[asmboy]

though your circuit is scary -if you must use it-

\then add a rail to rail op amp -and share your 5V pic supply with the opamp
use it to BUFFER the ADC input pin with said PICADC .

the opamp could also be SWEETLY configured as a sallen -key second order LOW PASS filter with a break freq of about 800hz
(remember your sampled input, due to full wave rectifier has a 100/120 hz FFT 'bin' center )
and to do a good job of it you want a DUAL op amp to buffer the input of the SK filter as well, if you so decide - ;-))
see:

http://sim.okawa-denshi.jp/en/OPseikiLowkeisan.htm

also ADD a 1n4005 diode between R1-R2 with cathode DOWN!!!!

all this will reduce the impact of spikes and have minimal phase interference on your reading line rectified samples.

[Mike Walne]

Can we assume from your schematic that the D2/D3 junction connects to VSS?

You're also insisting, without reason, that you cannot use lower values than 10M in your voltage measuring pot chain.
This does not make sense (to me) when you have significantly lower values in your ZC network.
I have not double checked the data sheet, a >30k resistor feeding the A2D seems to be seriously on the high side for a reliable design.

Mike

[viki2000]

A brainstorming is always better than a single thinking head.
Regarding the mains signal and distortion I only can say next for the moment:
- I have for the application a coil and a capacitor for power factor correction at the mains and hopefully some noise and harmonics are rejected.
- I cannot install additional components and therefore I have to rely only on digital filter inside the PIC. With a bigger PIC, better subroutines may be implemented to analyze the signal and extract the useful samples out of it. But I must use the small PIC10F due to its size and there is not too much space inside and not too much computation possibilities. In case of real troubles I must analyze the sinus wave where the application has troubles, but it is also not critical.
- In the past I worked several years in industrial sector, power distribution, automation with network of PLCs, big motors up to hundreds of KW, medium voltage switching lines up to 20KV...and of course I watched the mains with the oscilloscope and I know what it means in terms of noise, harmonics and bad power factor for example. Indeed, when big motors start then the waveform of the sinus is affected. There are different procedures with filters, soft-start panels, variable speed drives which can help, but even then the start-up moment is critical. Additional are all kind of solenoids from air valves, hydraulic system which affect the sinus. But generally that is local.
- I did not expect that in private utility sector to be so bad in some countries, even if in the past 2 months I received a report it a measurement of the lines from a restaurant here in Germany, measured by the local city hall, for a period of over 2 weeks. Some lights were flashing during dimming…well, the stability of frequency was not very great but within acceptable limits and the harmonics were OK. I see only 2 possible problems for which I impose 2 limits:
- If the waveform itself is affected badly then my reading can be out of the expected range. Then I just limit the timing result to the given range. For example, if the result of timing is out of 1.6s-3.5s then I make the result one of the value 1.6s or 3.5s depending if it is lower or higher value. It is not critical, but is good that we discussed about it.
- If due to the noise, the rectified wave does not have a good zero, then the external interrupt will fail the detection. If I use my last subroutine with difference between 2 points (following the sinus) and see when the signal is changing the sign, then is better for offset, but if a variation, a glitch is in the signal at the middle of the wave then the point of detection is wrong. Here I can play with the time between 2 readings used to follow the wave and make the difference, but a too long time equals with an error in the real ZC and a too short time gives the possibility to detect ZC wrong somewhere in the middle of the wave if a glitch, a small valley is there.
- Well, as conclusion, I cannot solve all the situations with a simple circuit and I have to live with the given hardware. In case the ZC is not detected for longer time, then the Watchdog timer will reset the PIC. Actually is an extended Watchdog timer. It is also not very critical.
I think is good that people point each time when is possible warnings about the mains, touching the live circuit, especially when not isolated and in the present case. I am really thankful for that.
In the past I have seen accidents that 6KV and I have seen destroyed such equipment, 2 cubic meter of fire and sparks…Then errors done by other professional guys, after many years of experience, at 20KV.
The 400V was our daily job and there are funny stories there too…
In fact I can tell you many stories with such problems that I have seen, but is not here the place for that.
Important is to remember: do not play even with 110VAC or 230VAC. It is enough to make one mistake, once and maybe you are no more on the earth. So pay attention. That is valid for me and everyone else who test similar circuits.
In my case now, I have on the table a circuit which has 10KHz short period pulses at almost 1500V and I use minimum 3 double switches to start-stop. The circuit-breaker is differential 30mA, but sometimes I have to use isolation transformer for oscilloscope measurements….

[viki2000]

Thank you for hardware suggestion to improve the schematic, but I cannot add more components (it is almost forbidden ) and the circuit works without problem for the moment.
Mike, the 10M must like that. If I lower the value of the 10M , then the high-voltage pulses value is lowered and some other oscillations do not work properly. You know, initially I made a lot of calculations, but then during “trial and error” with different values for all resistors I arrived to 10M as a must. Don’t worry about it, almost everything in my circuit does not work as expected…there are components specially designed by semiconductor companies for this application.

I would like to come to the code itself, because I have new ideas which worth to be discussed.
1) @clarence:
Could you explain a bit more about your suggestion?
AV=AVold + K*(Avold - reading);
AVold =AV;
For me is just not working. What have I done wrong? I just get in the end a big number far away of average, mode or median value as expected.
Or is your idea wrong?
2) I have more comments on olympic code written by asmboy. Let’s say variations, not necessarily improvements. When I have time I will come back with clear explanations.

[Ttelmah]

I think you need to get your head round the basic nature of the PIC's ADC input circuit, and understand 'why' this is going to give problems.

Electrically, the voltage being read, goes through a resistor (the resistance of the multiplexer), to an internal node. Then on through another resistor (switching circuit for the actual ADC), to the top of a capacitor, which is used to store the voltage to be read. This point then has an internal current sink, which discharges the node, caused by the internal gates used for the ADC itself. This current varies a little from 'chip to chip', and varies significantly from batch to batch.
When you set the ADC channel, it turns on the multiplexer to the pin you want to read. It then takes time for the internal capacitor to charge to within a margin of the 'real' voltage.
Now the specified maximum source impedance for the ADC, is designed to ensure that the voltage on the capacitor will be within 0.5bit, in a reasonable time, and that the internal leakage will not cause a significant error in the results. Only using 8bits, allows a higher source impedance to be used, but with increasing phase lag on the voltage being read, and increasing error 'chip to chip'.
The internal capacitance is disconnected from the input during the reading, and also during this time, the current drawn on Vref (in your case the supply pin) rises, which can cause further errors.
The most accurate way to do things with your high source impedance, would be to calculate an approximation to the phase lag, and measure the voltage once a short interval after the peak voltage, designed to approximate the lag at this point. This is the point where the waveform is flattest, and as such an error in the timing will give the least effect.

[temtronic]

The problem I see is the 'hardware handcuffs' this project has imposed.
First you say you can't add/change 'sensor' hardware even though you could get far better results at very little cost and minor PCB changes.
Then your design is limited to one PIC type with limited resources.There are probably a dozen or so , similar PICs with more power and features that would make the project easier to accomplish,again at little cost and minor PCB changes.
You have and will continue to spend a LOT of $$ in R&D costs trying to 'tweak' the software when a simpler, cost effective hardware solution will get the product 'up and running, 'out the door' faster.These days hardware is very,very cheap compared to labour and I've found 'off the shelf' solutions that were far more cost effective to days of code cutting.

hth
jay

[viki2000]

Thank you very much for spending time in writing your good recommendations about the hardware. If in the future I will notice any critical problems I will dare to disturb with additional questions.
FYI: the PIC must be the smallest on the market and as far as I know SOT-23-6 is the smallest. The PIC10F322 is best from its class and I do not think ATtiniy10 is better.

Now, let's go back to the code.

First is better if we explain it one more time on short, also for future readers, in simple words.
That will give the chance to visualize it and hopefully to tweak it.
Let’s take the original code with 10-12 bits ADC and 16 bits array.
The code samples first the signal, each 8us one sample, and then save the AD number in an array b[16].
Next part of the code is arranging, sorting the array in an ascending order from b[0] to b[15].
I consider that the most important part of the code.
After the array is sorted, then we make an average with only 8 samples from b[4] to b[11]. The first 4 samples b[0] to b[3] are considered possible noise low values and b[12] to b[16] are possible noise, or if you want out of the expected AD result too high and therefore are neglected.

The above thinking assumes or relies on the fact that at least 8 samples in the middle are acceptable as good signal, not noise.

1) First question:
- Why do you bother to make the average instead of just choosing as end result the middle of the array?
- Once the array is sorted just choose b[8] or b [9]. It is not the average, is more like a median value, but in many situations the average converges to the middle, center value of the sorted array.
The code assumes that the signal must be sampled very fast; one sample after another and the AD numbers are stored in a 16 bit array.
What if that is not the case, as for instance my situation?
I sample 1 time each 10ms and in mean time the PIC is boring itself 
With 16 bits array the tiny PIC10F322 starts to feel claustrophobic in its RAM 
Then we can make a compromise between reducing the RAM space and increasing a bit the flash memory used and a bit more timing for the code to be processed.
Here is the idea: if from 16 samples we choose and consider good, golden samples 8 then the rest of 8 is “bad”, noise. That implies from 9 samples, at least one is good.
Generally, the rule would be then 50% +1 samples to be used as array, RAM memory.
Lets’s use an array with only 9 bits from b[0] to b[8].
We sample and store the sample in that 9 bit array.
Then we sort it as usual and we choose the middle value b[5].
We save then that b[5] in b[7] and we make b[8]=0.
Then we sample the rest of 7 bits from 10 up to 16 and we save them in b[0] to b[6].
Then we sort again the 9 bit array, by calling, repeating the same part of the sorting code.
Then, if you like the average make it with bits b[1] to b[8]. If you like only the median value then just choose b[4] or b[5].
Here I assume that you have only positive values and the b[8]=0 will become the smallest b[0].

One more observation: in case the 8 gold samples from middle still have too high or too low values among them due to noisy signal and they are not equal distributed, then the average is worse than median, middle value.
Choosing the median value gives the advantage of not using a 16 bit variable in my case for accumulator used for average.

In my case with 8 samples would be only a 5 bit array, which is very important for gaining RAM space.
And I will try for test purpose the median value, without average.

2) Second question:
- How do you see this approach with regards to advantages and disadvantages?
Please let me have your comments.

[asmboy]

[Ttelmah]

The offset from the zero cross one, is what has me the most worried.

Key thing is that with your rectifier circuit, assuming 5v peak, you are going to get near 0v, for about +/- 0.4 mSec around the zero point, then the slope is going to be rising at perhaps 0.8V/mSec. The capacitive nature of the ADC, and the high impedance you are feeding it with, limits the effective frequency response of the circuit. The voltage on the capacitor will lag the actual incoming voltage curve massively, and won't ever catch up. The point where it'll be _closest_, is perhaps 6mSec after the zero cross, just a little after the peak. Everywhere else, you will be reading a value that will change massively if the timing is even a tiny fraction of a second off....

[viki2000]

I read the zero-cross on Zener Diode, not on 10M line.
There is no problem there, especially if I follow the curve instead of external interrupt, then the offset has no effect.

[asmboy]

[viki2000]

I cannot add op-amp or other components.
The application is not critical.
The initial and primary purpose of AD from PIC was to detect when the AC line goes from 230V RMS to somewhere under half. For that "precision" is more than enough the present circuit. The PIC is doing of course also other functions.
There is no problem with safety at all. Nobody will touch the circuit. And other type of safety required by standards are fulfilled for sure.
Here we do not speak about instrumentation and high precision as you mentioned it above.
I even know the signal or what I expect to be, the sinus,eventually with the mentioned distortions.
In fact I was more concerned about the voltage variation. Standard we have +/-10%, meaning 207V-253V. But even that is covered with calculations and many tests in the range 180V-260V and a bit under and over these limits.

Just imagine you have the mains 180V-260V and you need to detect when the line goes somewhere between 90-140V, generally speaking under 150V. I have seen no problem so far.
It came only as a second function and opportunity to include, without being critical for random situations, a function which transpose the mains variation 180V-260V into a timing range of 1.5s-3.5s with an acceptable precision of 0.1s. And if sometime the error is 0.2s or even 0.5s it is no problem, no safety to concern, and also no performance to concern if it is random. I would concern only for performance IF is a systematic error. But that can be then corrected in another way.
Too much concerns about the present circuit and its precision.

With high precision for 33.2K as 0.2% or 0.1% and the digital filter I get enough precision.

I am aware also about the differences between PICs. In the same circuit I replaced up to 20 PICs and I noticed the difference. The difference was observable for the high impedance at AD input when by mistake I forgot about weak pull-up resistors. The difference between PICs gave errors, but not very critical if a digital filter was used. When the pull-up resistors were disabled and the digital filter was used, then the error was more than acceptable as +/- 1 digit in reading the AD result.

Now regarding the code, for my situation makes no sense to read the signal 8 or 16 times one after another very fast in one point. If the wave is distorted there I just read the same error or close to it many times. Makes more sense to read it each half period of the sinus, each 10ms in the same point. The chances to have different waveform, a better sinus, are higher later in time than in very close moments as you do it.

I have reasons why I do not do it on the top of the wave, where is more flat and dV/dt is lower. There I do something else. I must do it at 3ms.
What else I could do to improve the measurements, if the flash memory will allow me, is to use the expected symmetry of the sinus and to read one more time the sinus at 7ms. So far was not necessary, but more tests will be done.

[viki2000]

Mains frequency influence.

If you look at your link:
http://wwwhome.cs.utwente.nl/~ptdeboer/misc/mains.html
and also here:
http://www.mainsfrequency.com/verlauf_en.htm
http://www.mainsfrequency.com/index.htm

you may notice that there is in general no variation out of 49-51Hz.
Here are some calculations which show that there no big influence of the frequency variation, (even if is lower or higher than calculated range below):

https://drive.google.com/file/d/0BwXmKaSw75eKejZVYkE3WktNUlU/edit?usp=sharing

How worse than that is the variation in your area?

"The maximum error of the frequency measurement is defined by the European Network of Transmission System Operators for Electricity (ENTSO-E) 10 mHz. The error of the measurement on this website is below 1 mHz. At a frequency of 50 Hz this corresponds to an accuracy of 0,002 % or 20 ppm."

[Ttelmah]

Have you got something like Spice (Psice, Isis etc...).

_Model_ the analog behaviour inside the PIC, with the external circuitry you are using to drive the pin. Microchip give the capacitance, the range of internal leakage currents, the internal resistances, and when the signal is attached/disconnected for the readings. Feed a synthetic 50Hz waveform to the input, and look at what happens inside the PIC. Look at what is happening on the capacitor inside the PIC, with the high impedance connection, and when it is changing fast, and when it is reasonably stable. Look at how small timing changes alter the voltages you would measure.....

[viki2000]

Zero-cross detection influence.

Here is a simple calculation of the ZC influence:
https://drive.google.com/file/d/0BwXmKaSw75eKa3RBQ255WlZ3ZWc/edit?usp=sharing

If I follow the wave, I get rid of offset problem. With 10us speed between samples, there is theoretically no error for what I expect.

The same with frequency stability. If the mains is kept in the range of 49.7-50.3Hz, then here is theoretically no error. The European standard impose 49.90-50.10 Hz stability.

Then theoretically, the resistor tolerance influence is also close to zero if the 33.2K is 0.1% and 4.99M are 1%.

If it is something that I "worry" is just the differences between PICs and the distortion of the mains waveform.
The first problem I will find out during the next months only statistical, and the second is a problem which will imply an external solution later on and will be considered special case.

You cannot "save the world" with such a small PIC and such simple circuit. Everybody is aware of that, but the purpose is to make the best of it for a non-critical application.