Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Power PMAC appears to "go to sleep"
#1
A Power PMAC appears to be freezing up or going to sleep. PMAC was purchased in spring of 2016.
Symptoms:
Symptom 1: Loss of Ethernet communications. Restarting the computer or the Windows app that is communicating with the PMAC does not reestablish communications. Cycling power to the PMAC does reestablish communications.
Symptom 2: We've had two instances whereby the PMAC appears to have stopped running and has left its outputs in their last state. IE; in one case, the PMAC kept a digital output on despite the fact that the condition that caused the output to turn on having been reset. In another case, when the same input should have caused the PMAC to turn on an output, it did not do so. As soon as we cycled power to the PMAC the unit operated as expected. When we try to recreate this ourselves, of course, we cannot. Everything works as it should.

Any advice on how to troubleshoot this? I've searched this forum for similar systems but the communications failures discussed herein don't seem relevant to what I'm seeing. Have there been any similar problems reported in the last few years? Are there any issues with firmware such that I need to update firmware? When we bought the unit in the spring of 2016 we used it "as is" without updating firmware.

Thanks;
MShaver
Reply
#2
The main problem is you can not reproduce. When we troubleshoot this type of problem we do the following
1. Connect an RS232 line to the unit and run a terminal program such as puTTy in logging to a file mode. We just let is run and when the problem happens we see if anything came out the RS232 port
2. After the problem happens we also check if we still have RS232 comms.
3. In preparation for the problem and hoping that after the problem happens we still have RS232 comms after the Pmac boots we send the Linux command dmesg -c to clear and messages. Then we occasional check the buffer with the command dmesg while it runs OK. After the problem we also try the command dmesg and if the RS232 is still working maybe we get something from it that point 1 did not get.

We have had some good successes with this method.
Reply
#3
Thanks Brad:

One more question. You didn't mention anything about firmware so it sounds like there are not any known firmware bugs that would explain this and I don't need to upgrade firmware or anything like that?
Thanks;
mshaver
Reply
#4
Not that I know about. But you should tell which type of PPmac and the current firmware version you are using as that could make a difference in debugging technique. For instance, Motion Machine has no RS232 port so the above tests are harder to do. In any case, it is good to first know what is happening.
Reply
#5
(09-15-2016, 11:28 PM)bradp Wrote: Not that I know about. But you should tell which type of PPmac and the current firmware version you are using as that could make a difference in debugging technique. For instance, Motion Machine has no RS232 port so the above tests are harder to do. In any case, it is good to first know what is happening.

We have a PPMAC system here in a lab that is used to control a rotary table and read encoder position along with some A/D input values. The symptom of this PPMAC is that too appears to "go to sleep" in that network communications are lost; it will no longer even respond to a ping request. It is however running its PLCs, etc. and maintaining the rotary table rotation and position. It is currently configured with a STATIC IP address. Two things that we have found: 1.) the serial port is still available and accepts GPASCII commands; 2.) you can use "ifdown eth0" followed by "ifup eth0" to cause it to reconnect onto the network (for a while). We have never actually tracked down what is happening (we've tried a few Linux TCP programs) but recently I moved the ethernet connection to a router in the lab that is active with a number of other devices and Windows machines - it has never disconnected since. There is a suspicion that the original network drop is configured with/for VOIP phones on it. Any further ideas/suggestions are welcome.
- Joe
Reply
#6
We've had the same issue with our Power PMAC controllers. If we try to put them on our corporate network (that also has VOIP, btw...) the Ethernet port will eventually stop communicating.

We've tried to troubleshoot the problem but our limited knowledge of Linux does not do us any favors. As Ibisjoe indicated, ifup/ifdown commands issued through the working eth or serial port get it connected for 10 or 20 minutes, but it will eventually disconnect.
Reply
#7
We have also seen the same issue at a customer's site. In the end it was tracked down to the customer's server. Their IT department was using a server tool that would randomly scan for ports it doesn't recognize and close them to prevent unauthorized network traffic. We could also use ifdown+ifup to temporarily recover.

I never got a technical answer from the IT guys about what they did to fix the issue.
Reply
#8
In my case,
Disconnected PPMAC ISSUE was clear after IP address change.
for example
LAN CARD 1) 192.168.0.2 connected PPMAC
LAN CARD 2) before ->192.168.0.5 after-> 192.168.1.5 connected vision
Thanks
Reply


Forum Jump:


Users browsing this thread: 1 Guest(s)