Webcontrol - timing errors?

[rossw]

So I've been waiting for 10 days for my new boards to arrive... they got here about 20 minutes ago.

I've set up a test for my application - what I got these things for - and am initially quite disappointed with the results, so am asking here for any tips, hints or suggestions.

I need to measure the frequency "fairly accurately" of a source. It's a TTL signal of variable duty cycle and variable frequency. I'm only interested in the frequency.

So I connected the board - no temperature sensors, no A/D inputs. The ONLY thing connected is my signal source.

I wrote a small piece of code, as follows:

start

set ram1 cs
secwait:
tsteq ram1 cs
goto secwait


set var1 counter
set counter 0

tsteq ram1 59
set var2 var1
tsteq ram1 59
set var3 var1

tstlt var1 var2
set var2 var1

tstgt var1 var3
set var3 var1

end

The code is intended to do nothing apart from sit in a very tight loop waiting for a second transition. The moment it detects the transition it sets var1 to the current counter value and immediately clears the counter. There are no branches, decisions, calculations etc that may introduce timing variabilities here, intentionally to keep the loop times as near identical as possible.

Once it has the counter safely stored in var1, it checks to see if we've just transitioned to the first second of a new minute (if seconds WERE 59, we must NOW be 0) - and will reset the "minimum" (var2) and "maximum" (var3) counters to the current value. Each other pass, it simply compares and updates the minimum and maximum count.

The program them drops out the bottom and restarts - where it goes back into the loop to catch the second change again.

I'm feeding it a constant signal of around 120KHz at the moment, and here's the output:

32 Bit Signed Vars 121641 106485 128908 0 0 0 0

A variability from 106,485 to 128,908 is HUGE - unusably so.

How can I improve this, because quite honestly - it's useless like this!

 

[fwd03]

how many seconds do you plan to measure the pulses?
CS is current second in a minute. It could loop around back to zero.

 

[rossw]

how many seconds do you plan to measure the pulses?
CS is current second in a minute. It could loop around back to zero.

Yes, it could, will and does. Indeed, I need it to! That's how I know its time to update the "over the last minute" registers.

I would like to measure the pulses for exactly one second, and do so for exactly 300 times every five minutes.
In that time I will want to calculate the average, minimum and maximum frequencies observed over that 5 minute period.
The average should be a pretty reasonable estimate of the frequency over the sample period.
The maximum and minimum will give me some "confidence" in the signal I am monitoring. The closer min/max are, the higher my confidence will be with the answers. (This is a very sensitive instrument capable of measuring tiny fractions of a nanotesla). The jitter I'm getting at the moment is totally outside the limits of use. (I've been using a duinomite for the prototype for 6 months, and it's been *FAR* more stable. Orders of magnitude less variation. Just that its interface to the rest of the world sucks)

 

[CAI_Support]

Ross,
measuring 1 second in that way is not accurate. It is a lot more accurate is to accumulate pulses for 10, 20, 30 seconds, or even longer, then divide that by the time. Because PLC code is not always running, the processor must handle everything in turn. By the time, the CS is updated, the time is already passed 1 second.

I would suggest you to change your logic to start count (empty counter), then after certain time, copy the counter value. Divided by the time.

 

[rossw]

measuring 1 second in that way is not accurate
..... the processor must handle everything in turn.

what else is it doing? There are no temperature sensors, no ADC reading, no email sending.... that's why I asked how else I can improve the accuracy of the events.

It is a lot more accurate is to accumulate pulses for 10, 20, 30 seconds, or even longer, then divide that by the time.

Yes, it might be. But it's not PRACTICAL in my application. Events can start, build and finish in 10 seconds.

I would suggest you to change your logic to start count (empty counter), then after certain time, copy the counter value. Divided by the time.

Thats like saying "Sorry the speedometer in the car we sold you doesn't really indicate the speed. We suggest you time how long it takes you to drive between mile markers and calculate it yourself".

Come on guys, its not rocket science.... what else can I do to alleviate the poor overworked processor so it can see the change of second a little closer to the event? (Or are you telling me I just totally wasted my time, effort and money on a product that doesn't do what you said it would?)

 

[apostolakisl]

Would adding the "real time clock" DS2417 improve things? I am asking, I don't really know how that particular feature works, but I understand that its purpose is to improve the clock accuracy.

 

[CAI_Support]

what else is it doing? There are no temperature sensors, no ADC reading, no email sending.... that's why I asked how else I can improve the accuracy of the events.




Yes, it might be. But it's not PRACTICAL in my application. Events can start, build and finish in 10 seconds.



Thats like saying "Sorry the speedometer in the car we sold you doesn't really indicate the speed. We suggest you time how long it takes you to drive between mile markers and calculate it yourself".

Come on guys, its not rocket science.... what else can I do to alleviate the poor overworked processor so it can see the change of second a little closer to the event? (Or are you telling me I just totally wasted my time, effort and money on a product that doesn't do what you said it would?)

Ross,

It is not rocket science that this processor must handle network tasks and check other I/Os, even there is no other I/O going on. So PLC must share the CPU with other tasks. When the time PLC starts running, the CS is not exactly on the second. If you want to have accurate count result, you will need to have the time long enough. Or simply uses the frequency counter, that is better than your PLC logic.

Realtime clock has nothing to do with task management. CPU must divide the CPU cycles in chunks to different tasks to be able to work properly.

 

[rossw]

When the time PLC starts running, the CS is not exactly on the second.

No, which is why each cycle I had a tight loop trying to synchronise on the transition - expecting that even if it wasn't exactly "on time" it would be a fairly constant offset.

If you want to have accurate count result, you will need to have the time long enough.

Even if I count over 10 seconds, its only going to reduce the error by one order of magnitude, which still isn't enough.

Or simply uses the frequency counter, that is better than your PLC logic.

I tried the frequency counter and it was even wilder!

Realtime clock has nothing to do with task management. CPU must divide the CPU cycles in chunks to different tasks to be able to work properly.

OK, so now we're getting to the crux of it.... the granularity of the timer and scheduling simply precludes accurate sampling intervals?

 

[CAI_Support]

No, which is why each cycle I had a tight loop trying to synchronise on the transition - expecting that even if it wasn't exactly "on time" it would be a fairly constant offset.


Even if I count over 10 seconds, its only going to reduce the error by one order of magnitude, which still isn't enough.



I tried the frequency counter and it was even wilder!


OK, so now we're getting to the crux of it.... the granularity of the timer and scheduling simply precludes accurate sampling intervals?

Ross,

timer is a seperate circuit in the processor that is running by itself. We get time from the timer. Even display is zero second, the actual timer is probably hundres micro seconds passed. In another word, try to get second from timer will not have the accuracy of exactly whole seconds. From your original post, you were trying to count pulses from zero to 60 seconds. If you let it count all the way to 60 seconds then divide by 60, the result is much better. If you use frequency counter, you can read it multiple times, the lowerest value is the most accurate one.

Because the Microchip TCP stack must be called multiple times in each second, we don't always have CPU to do anyting else during that time. If network is so busy, the internal timer could even lose a tick. That is why real time clock chip or internet time service is important to keep the clock accurate. In some so called real time OS, they just slice the CPU time to smaller chuncks, so that the jitter is less obvious. If you have any better idea to best work with Microchip TCP stack, we will love to hear from you.

 

[rossw]

From your original post, you were trying to count pulses from zero to 60 seconds. If you let it count all the way to 60 seconds then divide by 60, the result is much better.

The result might be closer, but it isn't what I need.
I actually NEED the maximum and minimum frequencies over fairly short timespans, because this will tell me how much the near field is fluctuating.
Knowing only that the average over 60 seconds was (x) is of almost no use whatsoever - because fluctuations are above and below the baseline.

The original code I posted was FAR from any finished code - but there's no point writing the full thing only to discover the underlying hardware is incapable of doing the job - which is basically where I am now.

If you use frequency counter, you can read it multiple times, the lowerest value is the most accurate one.

Can you please tell me EXACTLY your method to calculate frequency? I need this to understand if it is doing what I need.

Because the Microchip TCP stack must be called multiple times in each second, we don't always have CPU to do anyting else during that time.
If network is so busy, the internal timer could even lose a tick.

OK, I just presumed that was done under interrupt, or hardware into a buffer.

That is why real time clock chip or internet time service is important to keep the clock accurate.

Yes, I am using NTP. On that - it would be great if you could remove the NTP server from being hard-coded and put it in the network setup page.
Give up one of the "Access limit" addresses if you like. 7 IPs plus settable NTP is far more use than 8 access addresses!

If you have any better idea to best work with Microchip TCP stack, we will love to hear from you.

I've been involved in real-time systems since the late 1970's and have some ideas. Would you like to take this discussion to email?

 

[CAI_Support]

The result might be closer, but it isn't what I need.
I actually NEED the maximum and minimum frequencies over fairly short timespans, because this will tell me how much the near field is fluctuating.
Knowing only that the average over 60 seconds was (x) is of almost no use whatsoever - because fluctuations are above and below the baseline.

The original code I posted was FAR from any finished code - but there's no point writing the full thing only to discover the underlying hardware is incapable of doing the job - which is basically where I am now.



Can you please tell me EXACTLY your method to calculate frequency? I need this to understand if it is doing what I need.



OK, I just presumed that was done under interrupt, or hardware into a buffer.


Yes, I am using NTP. On that - it would be great if you could remove the NTP server from being hard-coded and put it in the network setup page.
Give up one of the "Access limit" addresses if you like. 7 IPs plus settable NTP is far more use than 8 access addresses!


I've been involved in real-time systems since the late 1970's and have some ideas. Would you like to take this discussion to email?

I am trying to help you based on your posted code, which showing you did 60 times measurement in a minute.
Our way to calculate frequency is at certain time interval, we go check to see how many counts, the counter has. We know for sure counter is accurate.
However the time is not as accurate as we like, because TCP stack may not return, and even we count ticks internally, we may from time to time lose ticks due to TCP stack.
TCP stack is not always that busy, so there are chances that the time is accurate. That is why I say the lower frequency number is more accurate amoung multiple readings .

NTP only being checked every 4 seconds, that reasult is only being used to update the second counts.

Surely you are welcome to write to us at [email protected]. We love to hear from anyone with any suggestion, include any criticize. We appreciate each and everyone using our product. If you use our product, you are king, we will listen to you. Hopefully, in this process, we can improve our products, too.

 

[rossw]

I am trying to help you based on your posted code, which showing you did 60 times measurement in a minute.

Yes, but thats not ALL the original tiny code did. Look again, you will see each second it got a value and compared to stored max and min and updated the maximum and minimum as a new high or low was detected. It was nothing like merely counting for 60 seconds.


I've been trying the "frequency" method you suggested - and the error appears almost identical. Here's the relevant section of code:

    SET RAM1 CS  
    SET VAR1 0  
SECWAIT:
    TSTEQ VAR1 0 	  # If no value already
    SET VAR1 FCOUNTER    # initialise with current freq
    TSTLT FCOUNTER VAR1   # is frequency lower than last?
    SET VAR1 FCOUNTER 	 # use lowest
    TSTEQ RAM1 CS 	 # still the same second?
    GOTO SECWAIT      # yes, loop

    TSTLT VAR1 VAR2   # is this seconds lowest lower than the stored minimum
    SET VAR2 VAR1     # yes, then update
    TSTGT VAR1 VAR3   # is this seconds lowest higher than the stored maximum?
    SET VAR3 VAR1   # yes, then update

My only doubt - perhaps you can clarify - is if there's a problem here:
TSTLT FCOUNTER VAR1 # is frequency lower than last?
SET VAR1 FCOUNTER # use lowest

Is the value of fcounter likely to change between the test and the set??
Do I need to read it into another variable first so I can test and guarantee I update with the same value?

 

[CAI_Support]

FCOUNTER is being updated every second by the scheduler. How frequent does your input freqency change? If we know your situation better, we can help you by changing some the logic to accomdate you. Right now, we know the counter is accurate in its count. However, time is not accurate for short period of time due to network stack may not return the CPU. We thought about doing comparison inside our firmware. However, if user having variable frequency, it will be hard for our internal logic to realize that case. That is why we decide to let user see this inaccuracy, allow user to adopt logic in PLC to select the better value.


If you have any suggestion to make it working better, we are more than happy to change it for you.

 

[rossw]

FCOUNTER is being updated every second by the scheduler.

Only every second? Hmm, that makes my looping for a second and taking the lowest value pretty pointless!

How frequent does your input freqency change?

At the moment, I have it in a null box so it will change only by a few PPM at most. That is under 1Hz variation @ 150KHz
In use, the instrument is capable of constant and rapid change - several KHz change in under a millisecond.
In my application, I'm only really interested in events around 3 seconds duration and slower - many will be only a few tens of PPM over hours, but the interesting ones will be hundreds to thousands of PPM in minutes or less.

I don't know if the chip you're using has interrupts or not, but a precise (even if not accurate) interrupt to collect count information asynchronously would help. Medium term stability is critical too of course.

 

[CAI_Support]

There are interrupts on this chip, but what event to trigger the interrupt is to consider. From original communication, I thought you would count pulses for 10 seconds or longer then divide them to get a frequency. However, if you want to measure 3 seconds, that is not going to be very accurate, due to time is not precise one second based on CS value -- the friction of a second is not showing in CS value. Using FCOUNTER then picking the smaller value over few readings might be better. We will have to think how to make this work better for you.