(resolved) Acontis DC Sync failing over a period of time

[daviddein]

Hey, so...

I am looking to keep various EtherCAT slaves in sync and the default settings in PowerPMAC IDE for an Acontis CK3E appear to work for about 10-20 minutes.

Within this goldy locks zone of operation, I can time I/O slice input and outputs, watch system times from slave devices, and get a maximum offset or jitter of 0.8us; this is on par with the DC sync characteristics I get when utilizing the same hardware setup in TwinCAT.

After 10-20 minutes of operation of the CK3E master, the timing of my I/O slices immediately becomes indeterminate and I start seeing jitters as high as 65us. This appears to correspond with the task manager reported "EtherCAT Tasks" CPU percentage time (4khz rate * 27% utilization = 67.5us). To me, this alludes to a timing conflict between the EtherCAT servo tick and the DC Sync pulse; but I can't be certain since I don't know how to monitor either of these processes. A vast majority of the ECAT[].Slave[].DCxxxx variables have no context for Acontis, only for Etherlab stack so I don't even know what dial to turn or button to push to see whats going on under the hood of the Acontis master.

CK3E Firmware Version 2.5.4.0
PowerPMAC IDE 4.4.1.7

Slave Hardware:
Beckhoff EK1100 Bus coupler
Beckhoff EL1252 Input Module
Beckhoff EL2252 Output Module
Copley BEL-090-06

Troubleshooting steps that have not provided any gains:
1. clear config, start from scratch.
2. changing servo rate (4khz to 2khz)
3. changing ethercat rate (4khz to 2khz)
4. setting slave distributed clocks to different cycle rates with an appropriate offset
5. power cycling the slaves (not the master)
6. ecat reset
7. adjusting DC timing variables (plus,minus,kp,ki,refband)
8. change dc to master shift or bus shift.


Any help is greatly appreciated,
David

[DaveBarnett]

What you describe has the telltale signs of Distributed Clocks not working.
Some useful techniques I've used to debug this type of thing include the following:

Try making the Copley drive the sync master
(Distrib Clocks tab)
And, put Acontis master in "Master Shift" so Pmac will sync to the drive

Sometimes the position of the drive in the network chain will make a difference. Put the drive first.

In a watch window, include:
Sys.ServoPeriod
Ecat[0].MasterState
Ecat[0].Enable
Ecat[0].DCClockDiff



Make sure *.DCClockDiff is not drifting over time...(it should just dither around a constant value)

[Sanjay]

There's an app note that DT had sent me that helped me tune EtherCat DC loop using the Hystresis method.
It seems to be more stable on my Ethercat loops using the Hystresis instead of the PID loop.

Can't find a link in the file depot to it but here's a Google Drive link that you can try:
https://drive.google.com/open?id=1VyeOB1...w8oe0r01PL

[daviddein]

Thank you for the help.

The ultimate problem was the math I was using. I was utilizing IOBuffer to pump data over to the slave device. Not aware that the local variable trigger_time was in fact a double, truncating of the data seems to be occuring as I attempt to slice it back into pieces to fit in the IOBuffer array. Code segment below.

//transfer toggle time to output byte
index=0;
while(index<8){
buffidx = byte_shift_time_trigger + index;
ECAT[0].IOBuffer[buffidx] = trigger_time>>(index*8); //this line must be wrong
index++;
}

I saw the problem when inspecting "watch ethercat mapped variables", the 64 bit word over time had its LSB's go to FF.

Now to fix it...

[daviddein]

The primary issue was that I was updating my trigger time on my Beckhoff 2252 I/O slice with a delta tau local variable, which is a double data type.

After 18.23 minutes of time from start of the controller, a double doesn't have the precision to represent the last 64us of time. This gave me the false indication that timing was not working properly, when in fact it was working the entire time I was just uploading a start time that was incorrect.

To fix, I used two local variables representing the upper and lower of my 64-bit nanosecond time when I want the I/O slice to trigger, and copied that number over to the ethercat bus.

[Eric Hotchkiss]

If you use the clock_gettime function, you will have access to the time in two variables to get past these limitations. Here is a CfromScript example.

Code:

double CfromScript(double cfrom_type, double arg2, double arg3, double arg4, double arg5, double arg6, double arg7, struct LocalData *Ldata)
{
    double Nano_Time;
    struct timespec start;
    clock_gettime( CLOCK_REALTIME, &start);
    Nano_Time=start.tv_sec+start.tv_nsec/1000000000.0;
    return Nano_Time;
}