Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
DesVel hangs to zero before end of move
#1
I have a problem on a BrickLV with firmware 2.3.2.5, managing 16 axes, 8 + 8 on a "slave" through macro communication.

Our application launches via gpasccii a series very small moves ("J=XXX" commands) , executing in parrallel, and once complete it launches a motion program with PVT moves on 5 motors, and cycling that again and again with the same commands and same moves.

Under above conditions I have a probability around 10% to never end the move during processing of the jog commands sent by gpascii. The main symptoms are: one motor among all (the problematic motor number is changing all the time), is blocked with InPos=false but DesVel=0. It often blocks at the begining of the move.

Hints:
- When I change default Sys.PreCalc to Sys.PreCalc=10 (and keep SegMoveTime=0), I can reduce the failure frequency by a factor of 10 to 50. But none of the parameter values can solve the problem completely.
- I can see in Task Manager that regularly I get Bsy/Err in RTI. I tried to change RtIntPeriod and to stop all my PLC, without any improvements.
- I tried to group "parallel" jogs in a single gpascii command, but the result is the same.

When this freeze happens, I dump the CS and motors status. Here is below one dump result example, where Motor[3] is OK (arrived) and Motor[4] is blocked forever.

Motor[3].Status[0] = f900;
Motor[3].Status[1] = 80000000;
Motor[3].DesPos = -44657.05590625014;
Motor[3].ActPos = -44656.50390625;
Motor[3].JogPos = -44657.05590625014;
Motor[3].DesVel = 0.0;
Motor[3].ActVel = -1.0;
Motor[3].JogSpeed = 121.2416;
Motor[3].InPos = 1;
Motor[3].AmpEna = 1;

Motor[4].Status[0] = f100;
Motor[4].Status[1] = 80000000;
Motor[4].DesPos = 5420.312093749992;
Motor[4].ActPos = 5420.49609375;
Motor[4].JogPos = -860.0027862499992;
Motor[4].DesVel = 0.0;
Motor[4].ActVel = 0.0;
Motor[4].JogSpeed = 6.0398998;
Motor[4].InPos = 0;
Motor[4].AmpEna = 1;

Coord[1].Status[0] = 1f000;
Coord[1].Status[1] = 80800001;

This non-reliability is critical, so I have been looking for a solution over a long period (before it took hours to get a failure but now I have the script to get it within 1 min).
Could you please tell me what might be going on. Is it normal that PreCalc has an such impact on parallel Jog moves ? What kind of solution/configuration could be used to get 100% of the moves finishing ?
Thanks,
Bernard Lavault
Reply
#2
Motor[x].DesVelZero goes true when the trajectory (or desired position) stops moving. Motor[x].InPos should go true some time after once the following error stays below a limit value. If this were the case the suggestions would be:
  • Increase Motor[x].InPosBand (following error limit for In Pos)
  • Increase Motor[x].InPosTime (number of consecutive Servo Cycles)
  • Improve tuning.

This could only delay motion is a PLC or motion program was checking the value of Motor[x].InPos. There are no built in checks of this variable that will stop motion or program execution.

However, it sounds like you may be getting a runtime error which is causing the program to abort.
When failure occurs, what are the values of the following?
  • Coord[x].RunTimeError
  • Coord[x].ErrorStatus

You might try making some space by limiting the values of Sys.MotorsPerRtInt, Sys.MaxMotors or Sys.MaxCoords.

You may need to adjust frequencies to make enough room for RtInt to finish more reliably. Can you post a picture of the tasks tab of your IDE task manager?
Reply
#3
Thank you for the suggestions.
The issue is that when this happen we see zero in the following error (we have InPosTime=100 meaning 10 ms). The tuning is not the issue since the motor doesn't even start its move.
Concerning the runtime error , there is actually no program running, it is a gpasii command that issue the problematic move. Anyway I used to check Coord status from the IDE and there is no error. This is confirmed by the dump above ( Coord[1].Status[0] = 1f000)

At the moment I have Sys.MotorsPerRtInt=0, Sys.MaxMotors=17, Sys.MaxCoords=7. I will try to reduce to Sys.MotorsPerRtInt=8, Sys.MaxMotors=15, Sys.MaxCoords=2.
Reply
#4
I doubt that this is a CPU loading issue. So, I would not mess with Sys.MotorsPerRtInt. Be aware that changing this element requires numerous changes to other elements, including I2T settings.

One factor to keep in mind is the delays between jog moves and status bit checking. Can you share your timing scheme? Issuing a JOG command to a motor in the middle of another moves' acceleration or deceleration is BAD. This could accumulate following error. Otherwise, you probably want to make sure that you are not sending redundant JOG command(s) repeatedly.

I would verify the consistency of the cycle and motor jogging outside of the application. The best way is to do it from a background PLC. Assuming all 5 motors are in coordinate system 1, below is an example. You can increase the CycleCtr value to loop more than once.

If this PLC works then the issue is in the the application timing.

Code:
OPEN PLC CycleTestPLC
LOCAL CycleCtr
CycleCtr = 0
WHILE(CycleCtr < 1)
{
JOG 1..5:10
CALL DelayTimer.msec(50)
WHILE(Coord[1].InPos == 0){}

START 1:1
CALL DelayTimer.msec(50)
WHILE(Coord[1].ProgActive == 1){}
WHILE(Coord[1].InPos == 0){}
CycleCtr++
}
DISABLE PLC CycleTestPLC
CLOSE
Reply
#5
Thanks for the comment. Indeed my various tests on the CPU load brings no improvement. The only consistent behaviour is that increasing Sys.PreCalc reduce the frequency of the freezes.

In addition to previous described issue, I can find now sometimes a Coord[1] runtime error raised during execution of the PVT motion program that helps to induce the freezes in subsequent "J=XXX" calls. I don't know if this is a clue or completely not correlated. It happens only if Sys.PreCalc=0. The parameters of the program are always the same.

As far as I can see the application never request two moves to the same motor in parallel. We do simultneous moves of different motors (either in PVT motion programs or in Jog gpascci commands). I can see in the logs that motors are inPos before accepting a new command that freezes. Most of the time there is at least 10 seconds between the end of the previous move and the subsequent one on the same motor that freezes.

I could imagine that if a motor has a bounce (it goes out of InPosBand ) between a check and the command (separated by env 5ms) , it might be bad. I would be surprised that it is the case because it happens randomly with all the motors and they are all tuned carefully.

When I have access to the machine next Monday I will measure the cycle thanks to your PLC
Reply
#6
Setting Sys.PreCalc = 0 is strongly discouraged. As you have seen, it does result in runtime errors. For calculation intensive applications (possibly like yours), users have had to increase it from the default of 1 to ~5-10 to alleviate runtime errors. Generally, this has minimal to no affect to the cycle time or positioning speed. I am not sure why are you interested in setting it to 0.

Motor[x].InPos bit going in and out does not cause or accumulate errors.
Reply
#7
It works best with Sys.PreCalc=10 but I had the impression that the freeze occurs more often when the value is small (it might be a false conclusion), which is interesting to reproduce more quickly the problem. I believed that the default value was zero, but It is 1 instead, and if Sys.PreCalc=0 is not good : it make the PVT program fail in RunTimeError often. Anyway I don't see the link between the jogs and the PVT program. I even see some freeze on motors of Coord[2] after we ran the PVT program on other motors in Coord[1].

In your sample program you suggest to get rid of the application and call both the jog and the PVT motion program , right ? Whyis it better in a background PLC than in motion program ? No issue if there are loops of several seconds ?

I can see in the dumps that the following error of motors freezed with Inpos=0 is always below 2 cts. Sometimes it happens when ActPos=DesPos=JogPos and most often with ActPos = DesPos != JogPos
Reply
#8
For default variable values you can check the software reference manual. It's currently available from the forum file depot.
http://forums.deltatau.com/filedepot/dow...Manual.pdf

Quote:In your sample program you suggest to get rid of the application and call both the jog and the PVT motion program , right ? Whyis it better in a background PLC than in motion program ? No issue if there are loops of several seconds ?
I believe the point of the PLC was just to make sure there was a pause before each jog move or motion program run and check if the timing of consecutive jog moves is the issue. Is there a check for InPos between Jog moves? PMAC doesn't finish a jog move before moving to the next line.

Quote:I can see in the dumps that the following error of motors freezed with Inpos=0 is always below 2 cts. Sometimes it happens when ActPos=DesPos=JogPos and most often with ActPos = DesPos != JogPos
InPos does not go true while the motor is moving, regardless of following error.

From the Software Reference Manual.
Quote:The Motor[x].InPos status bit is set to 1 when all of the conditions for “in position” are satisified: the motor is closed-loop, the desired velocity is zero, the move timer is not active (no move, dwell, or delay being executed, the magnitude of the following error is less than or equal to Motor[x].InPosBand, and all of these conditions have been true for (Motor[x].InPosTime - 1) consecutive servo cycles. It is 0 otherwise. It is bit 11 of 32-bit element Motor[x].Status[0].
Reply
#9
In our application we check carefully that the previous move is terminated and this can happen 10 sec after end of previous move.

Anyway, without any application involved, the CycleTestPLC mentionned above leads to one motors waiting indefinitetly after a several cycles. I just changed specified this move command: cmd "#1J:-100 #3J:-100 #4J:-100 #5J:-10 #6J:-10". I didn't reproduce the issue with all moves having the same distance.

Each cycle duration was pretty stable (+- 5 servo cycles) until it blocked suddenly waiting end of the Jog. From IDE 4.5 terminal I dumped motor 6 statuses after few minutes blocked. For me all the conditions for InPos are met. In the following dump the move is almost finished (but most of the time it is almost not started when it happens):

Motor[6].InPos
Motor[6].InPos=0
Motor[6].JogPos
Motor[6].JogPos=121345495
Motor[6].ActPos
Motor[6].ActPos=121345506
Motor[6].DesPos
Motor[6].DesPos=121345505
Motor[6].DesVel
Motor[6].DesVel=0
Motor[6].ActVel
Motor[6].ActVel=0
#6F
2
Motor[6].Status[0]
Motor[6].Status[0]=$f100
Motor[6].Status[1]
Motor[6].Status[1]=$80000000
Coord[1].Status[0]
Coord[1].Status[0]=$1f000
Coord[1].Status[1]
Coord[1].Status[1]=$80800001
Motor[6].InPosBand
Motor[6].InPosBand=200
Motor[6].InPosTime
Motor[6].InPosTime=100

Any suggestion what to do to find the cause, or is there any additionnal information that I should provide ?
Reply
#10
Try making sure all other PLCs are disabled and closing any watch windows. They could be sending other commands interfering with your motion.

If that doesn't help. Can you share the project so I can take a look? You can send it to eric.hotchkiss@omron.com
Reply
#11
Hello,
I am working with Bernard on that issue.
I notice something with the new IDE we just installed :
during normal operations of the machine we very often have the
CoordExecStatus[x] alarm being raised,
I don't find any reference of it in the User Manual and in the SRM.
would anyone know what causes these alarms ? And if they can be linked to the problem described by Bernard ?
Have a good day,
Jeremy
Reply
#12
I believe this a normal condition being misinterpreted as an error and it is solved in IDE 4.5.1.3 available here:
https://automation.omron.com/en/us/produ...PMAC%20IDE

The release notes show:
Quote:Alarm pop-up incorrectly showing the status. Alarms are for error only.
For example Plc[1].Ldata.Status = Stopped on Quit or CoordExecStatus[1] = Stopped on Quit, is not an Alarm but status.
Reply
#13
Hi ,
OK thank you, I'll try the new IDE, but I understand it will probably make the false error dissapear without solving our jog issue.

cheers,
Jeremy
Reply
#14
A command "#1J:100 #2J:100 #3J:100 #4J:100" preceeded by a PVT move is freezing after less than 100 cycles for us (and same for series of CMD). Our supplier reported that he can reproduce it after 29000 cycles.
Good news: on our PowerBrick "&1 cpx Inc Rapid A100X100Y100Z100" preceeded by a PVT move is passing 50000 cycles with no problem so far.
Reply


Forum Jump:


Users browsing this thread: 1 Guest(s)