Author |
Message
|
ankurlodhi |
Posted: Tue May 31, 2011 6:33 am Post subject: NEED help with FDC file with CO051000 |
|
|
Master
Joined: 19 Oct 2010 Posts: 266
|
Recently we were doing an activity of a server move when i started the channel the channel didn't started later i found an FDC file AMQ16031.0.FDC with probe id CO051000
the FDC file is as followes:
MQSeries First Failure Symptom Report |
| ===================================== |
| |
| Date/Time :- Saturday May 21 09:56:00 EDT 2011 |
| Host Name :- crpls01 |
| PIDS :- 5765B75 |
| LVLS :- 510 |
| Product Long Name :- MQSeries for Sun Solaris 2 (Sparc) |
| Vendor :- IBM |
| Probe Id :- CO051000 |
| Application Name :- MQM |
| Component :- cciTcpReceive |
| Build Date :- Mar 1 2000 |
| UserID :- 00001001 (mqm) |
| Program Name :- amqcrsta_nd |
| Process :- 00016031 |
| Thread :- 00000001 |
| Major Errorcode :- rrcE_BAD_DATA_RECEIVED |
| Minor Errorcode :- OK |
| Probe Type :- MSGAMQ9207 |
| Probe Severity :- 2 |
| Probe Description :- AMQ9207: The data received from host 'crpls01 |
| (IP address )' is not valid. |
| Comment1 :- crpls01 (IP address) |
| |
| Comment2 :- TCP/IP |
| |
| Comment3 :- [/b] |
|
Back to top |
|
 |
Vitor |
Posted: Tue May 31, 2011 7:01 am Post subject: Re: NEED help with FDC file with CO051000 |
|
|
 Grand High Poobah
Joined: 11 Nov 2005 Posts: 26093 Location: Texas, USA
|
ankurlodhi wrote: |
i started the channel the channel didn't started later i found an FDC file AMQ16031.0.FDC with probe id CO051000 |
Found where? On the sender or receiver side? I put it to you it's the receiving queue manager.
The error seems quite clear - something is sending bad data, where bad here means "not WMQ".
Is this the receiving side? Are there any errors in the other queue manager logs (presumably the sending side)?
What steps have you already taken to find and resolve the problem? _________________ Honesty is the best policy.
Insanity is the best defence. |
|
Back to top |
|
 |
mqjeff |
Posted: Tue May 31, 2011 9:58 am Post subject: Re: NEED help with FDC file with CO051000 |
|
|
Grand Master
Joined: 25 Jun 2008 Posts: 17447
|
ankurlodhi wrote: |
| LVLS :- 510 | |
I hope that doesn't mean what I think that means?
If it does, there's more help you need than with an FDC. |
|
Back to top |
|
 |
ankurlodhi |
Posted: Tue May 31, 2011 10:54 am Post subject: |
|
|
Master
Joined: 19 Oct 2010 Posts: 266
|
yes this error is on the receiver side,
i have checked all the error logs, and services if there is something else accessing the same port,
i checked the sender side and there is also the same FDC file. am adding that FDC below
MQSeries First Failure Symptom Report |
| ===================================== |
| |
| Date/Time :- Monday May 23 09:52:58 EDT 2011 |
| Host Name :- sabls09 |
| PIDS :- 5765B75 |
| LVLS :- 510 |
| Product Long Name :- MQSeries for Sun Solaris 2 (Sparc) |
| Vendor :- IBM |
| Probe Id :- CO051000 |
| Application Name :- MQM |
| Component :- cciTcpReceive |
| Build Date :- Mar 1 2000 |
| UserID :- 00001001 (mqm) |
| Program Name :- amqcrsta_nd |
| Process :- 00013421 |
| Thread :- 00000001 |
| Major Errorcode :- rrcE_BAD_DATA_RECEIVED |
| Minor Errorcode :- OK |
| Probe Type :- MSGAMQ9207 |
| Probe Severity :- 2 |
| Probe Description :- AMQ9207: The data received from host 'localhost |
| (127.0.0.1)' is not valid. |
| Comment1 :- localhost (127.0.0.1) |
| |
| Comment2 :- TCP/IP |
| |
| Comment3 :- |
|
this system is currently in production and other channel are working fine so i didn't tried anything much. |
|
Back to top |
|
 |
mqjeff |
Posted: Tue May 31, 2011 10:59 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008 Posts: 17447
|
That LVLS stanza suggests you are using MQ version 5.1.0.
Is that correct?
If so, that's the source of the problem, and the other channels are not actually working fine. They are only working fine "right now", and are at significant risk of falling over for no reason and leaving you without any recourse.
Again, someone is providing bad data to the MQ listener - either a bad client or someone is perhaps doing something fun like using telnet or ping to talk to the MQ listener.
But you really shouldn't be spending this much effort on troubleshooting a three or four times obsolete version of MQ on a who knows how old version of Solaris. |
|
Back to top |
|
 |
ankurlodhi |
Posted: Tue May 31, 2011 11:01 am Post subject: |
|
|
Master
Joined: 19 Oct 2010 Posts: 266
|
A senior member of my team is saying this issue can be because of wrongly copied file system of MQ or the shared memory "But I dont think so cause if it would have been because any of these then how come other channels are working fine"
but still let me know if they are correct, is this possible because of any kind of shared memory problem. |
|
Back to top |
|
 |
Vitor |
Posted: Tue May 31, 2011 11:01 am Post subject: |
|
|
 Grand High Poobah
Joined: 11 Nov 2005 Posts: 26093 Location: Texas, USA
|
ankurlodhi wrote: |
i have checked all the error logs, and services if there is something else accessing the same port, |
Well it wouldn't need to be running on the same box would it?
But given this:
ankurlodhi wrote: |
| Probe Description :- AMQ9207: The data received from host 'localhost |
| (127.0.0.1)' is not valid. |
| Comment1 :- localhost (127.0.0.1) |
|
maybe it is
ankurlodhi wrote: |
this system is currently in production and other channel are working fine so i didn't tried anything much. |
If it's in production I would have thought you'd have been trying a lot of things.
Why did you move servers? Is the new server on the same OS version as the old one?
(This echos the comment of my most worthy associate regarding software levels) _________________ Honesty is the best policy.
Insanity is the best defence. |
|
Back to top |
|
 |
Vitor |
Posted: Tue May 31, 2011 11:05 am Post subject: |
|
|
 Grand High Poobah
Joined: 11 Nov 2005 Posts: 26093 Location: Texas, USA
|
ankurlodhi wrote: |
A senior member of my team is saying this issue can be because of wrongly copied file system of MQ or the shared memory |
So when you say "server move" you mean you copied everything you thought might be relevant from one server to another, rather than installing WMQ on the new server and copying the WMQ objects from one machine to another?
So you're now trying to run what sounds a lot like a seriously back level of WMQ on a new server on which WMQ is not properly installed, for which the kernel settings will not have been properly changed, the group ids will be different, and you're surprised you're having problems? It sounds like this is just the poor object that got unlucky and has gone down first. _________________ Honesty is the best policy.
Insanity is the best defence.
Last edited by Vitor on Tue May 31, 2011 11:06 am; edited 1 time in total |
|
Back to top |
|
 |
ankurlodhi |
Posted: Tue May 31, 2011 11:05 am Post subject: |
|
|
Master
Joined: 19 Oct 2010 Posts: 266
|
yes I agree with you, it is a significant risk and i have put this thing before management, many times but they don't listen. perhaps if we solve this problem then we will able to talk to them about version upgrade, and yes the solaris platform is also old.
but we need to solve this problem ASAP. |
|
Back to top |
|
 |
Vitor |
Posted: Tue May 31, 2011 11:09 am Post subject: |
|
|
 Grand High Poobah
Joined: 11 Nov 2005 Posts: 26093 Location: Texas, USA
|
ankurlodhi wrote: |
if we solve this problem then we will able to talk to them about version upgrade, and yes the solaris platform is also old. |
a) tell them this is what they get for not listening
b) it's more likely you'll get a version upgrade if you can't fix it
c) what level of Solaris?
d) why did you move servers?
ankurlodhi wrote: |
we need to solve this problem ASAP |
You might mention to management that if you were on a supported version of WMQ you'd have access to IBM's support. Instead of relying on a bunch of volunteers on the Internet who all have other things to do. _________________ Honesty is the best policy.
Insanity is the best defence. |
|
Back to top |
|
 |
ankurlodhi |
Posted: Tue May 31, 2011 11:12 am Post subject: |
|
|
Master
Joined: 19 Oct 2010 Posts: 266
|
both the sender are receiver are on different boxes, there was an error with old box because of what this move took, place the SA copied all the data rather then installing the MQ. well i know its odd and version which is obsolete is being used, but it right now we are helpless with the version kind of thing,
so what can we probably try this new box was before in use can it be that there might be some other instance stuck which is sending data to the Listener. |
|
Back to top |
|
 |
ankurlodhi |
Posted: Tue May 31, 2011 11:16 am Post subject: |
|
|
Master
Joined: 19 Oct 2010 Posts: 266
|
the level of solaris is 5.1
the box was moved due to some issue with the app server installed on the old box the application server was tuxedo. |
|
Back to top |
|
 |
Vitor |
Posted: Tue May 31, 2011 11:19 am Post subject: |
|
|
 Grand High Poobah
Joined: 11 Nov 2005 Posts: 26093 Location: Texas, USA
|
ankurlodhi wrote: |
well i know its odd and version which is obsolete is being used, but it right now we are helpless with the version kind of thing, |
You're also helpless with the fact that WMQ is not properly installed on the box & is trying to run with non-matching group ids & kernel settings. Basically you (in the site sense of the word you) have tried a quick and contact admin rather than doing things properly and it's blown up in your face.
The fact that you've tried a quick and contact admin on a seriously back version of the software that's even less tollerant of such tactics is simply icing on the cake.
ankurlodhi wrote: |
so what can we probably try this new box was before in use can it be that there might be some other instance stuck which is sending data to the Listener. |
There's clearly something somewhere sending bad data to this port but since no-one in this forum knows what this box was used for before we can't really comment on if it's something stuck on it. Presumably you've tried restarting the box (in case the Solaris is so old it's gone funny in the head & thinks it's Windows)? _________________ Honesty is the best policy.
Insanity is the best defence. |
|
Back to top |
|
 |
ankurlodhi |
Posted: Tue May 31, 2011 11:41 am Post subject: |
|
|
Master
Joined: 19 Oct 2010 Posts: 266
|
I will ask the System Administrator about these details and get back to you guys if he had configured the kernal and group id's
and I agree that it is a contact admin way but it's good that this thing has blown off on there faces cause now they will take it seriously, atlease consider what we say, cause they really need it ..
but is there any way by which i can check what is sending data garbage data on the port.
as the ps -ef |grep port number |
|
Back to top |
|
 |
mqjeff |
Posted: Tue May 31, 2011 11:55 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008 Posts: 17447
|
ps shows you processes.
netstat shows you net connections.
but that might not be sufficient. |
|
Back to top |
|
 |
|