| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Strange Issue with COM and MQ Series |
« View previous topic :: View next topic » |
| Author |
Message
|
| PisgahMan |
 Posted: Tue Aug 09, 2005 8:35 am Post subject: Strange Issue with COM and MQ Series |
 |
|
Voyager
Joined: 27 Jul 2004
Posts: 93
|
I posted this on the MQWF Forum as well. Could this be an issue with apartment threading and COM+?
******************************
We have a .NET Windows Service with a (COM component/facsade) that we use to get messages from the MQWF request queues, route them to the appropriate web service/upes and then send the response back to the workflow XML input queue(EXEXMLINPUTQ). When we need to add new queues to be monitored, we just add them to the config file of this .NET Windows Service. This way the individual application don't have to know how to pull messages from queue.
The way this web service works, it opens connections to all of the request q's and listens continuously.
(getOptions.Options = MQC.MQWI_UNLIMITED + MQC.MQGMO_WAIT;)
for the response queues, it opens a new connection to EXEXMLINPUTQ, send the message and then closes the connection.
**********
Now to the problem. We recently added new request queues to the configuration and we are seeing some strange problems. 1st I see a message in the event logs like this
Remote host 'myservername (123.456.78.90) (5010)' not available, retry later.
The attempt to allocate a conversation using TCP/IP to host 'myservername (123.456.78.90) (5010)' was not successful. However the error may be a transitory one and it may be possible to successfully allocate a TCP/IP conversation later.
Try the connection again later. If the failure persists, record the error values and contact your systems administrator. The return code from TCP/IP is 10060 (X'274C'). The reason for the failure may be that this host cannot reach the destination host. It may also be possible that the listening program at host 'myservername (123.456.78.90) (5010)' was not running. If this is the case, perform the relevant operations to start the TCP/IP listening program, and try again.
...after that no responses for that particular request will work again. They all get this error
[color=#444444]Unable to process a message to workflow!!! Using this Reply Queue:
MY.MYGRP.MYSYS.EXE.XML
Reply Queue Manager:MYWFQM
Error!!!
Reason:
QueueManager creation failed. Reason 2059 [/color]
This causes expirations to happen in Workflow on the UPES call. The strange part is that only 1 out of 13 REQUEST queues has this issue. There should not be a relationship between the request and response connections, but it seems like there is. I would expect to see failures across all of them. Also the UPES service run on 2 servers and only 1 is throwing the error. In DEV and TEST enviroments no such error happens.
..Here is my best guess. Some TCP/IP hicupp happens, the connectivity between the queue manager and Windows service is lost, and the app is unable to reconnect. But that does not explain why it just started recently.
Any insight at all will be appreciated. |
|
| Back to top |
|
 |
| jefflowrey |
| Posted: Tue Aug 09, 2005 8:45 am Post subject: Re: Strange Issue with COM and MQ Series |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| PisgahMan wrote: |
| I posted this on the MQWF Forum as well. |
Because you haven't seen all of the hundreds of times we've told people not to double-post?
| PisgahMan wrote: |
| Could this be an issue with apartment threading and COM+? |
It could be. What have you done to troubleshoot that?
| PisgahMan wrote: |
| We have a .NET Windows Service with a (COM component/facsade) that we use to get messages from the MQWF request queues, route them to the appropriate web service/upes and then send the response back to the workflow XML input queue(EXEXMLINPUTQ). When we need to add new queues to be monitored, we just add them to the config file of this .NET Windows Service. This way the individual application don't have to know how to pull messages from queue. |
Nor do those applications gain any the benefits of using MQ.
| PisgahMan wrote: |
| The way this web service works, it opens connections to all of the request q's and listens continuously. |
Server connections?
Client connections? What does it do when one of those connections goes away?
| PisgahMan wrote: |
| (getOptions.Options = MQC.MQWI_UNLIMITED + MQC.MQGMO_WAIT;) |
And what do you do when you want to shut down the queue manager? Force kill the service?
for the response queues, it opens a new connection to EXEXMLINPUTQ, send the message and then closes the connection.
**********
| PisgahMan wrote: |
| ..Here is my best guess. Some TCP/IP hicupp happens, the connectivity between the queue manager and Windows service is lost, and the app is unable to reconnect. |
Unable? Or written so that it doesn't even try?
| PisgahMan wrote: |
| But that does not explain why it just started recently. |
Have you done any recent networking maintenance?
Have you examined the queue manager connections to see if they are being used up due to increased load?
Have you done things to troubleshoot this problem, other than post here?
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| PisgahMan |
| Posted: Tue Aug 09, 2005 9:39 am Post subject: |
|
|
Voyager
Joined: 27 Jul 2004
Posts: 93
|
Sorry I was not aware of the double posting rule, I don't post here very often. I saw on another thread someone having issues with apartment threading, maybe someone has had a similar issue before.
The connections are from MQ Client on an Intel Box to MQ Server on AIX. The app has reconnect logic built in which was tested and worked on the response queues. This request queue creates a new queue manager each time, so I don't understand how it stops working for that response altogether after 1 error.
As far as I know there have been no changes to the network, or anything else other than adding new queues to this app to be monitored. I am not sure if that is related to the issue.
The queue manager connections are around 40ish generaly, if I understand correctly problems will not start to occur until 100.
I didn't write this application or design the architecture. Sure there are some improvements that can be made both, but with what we have it was working until a couple of weeks ago. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Tue Aug 09, 2005 9:49 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Check the MQ server error logs to see if it shows any thing. Similarly see if there is anything in the MQ client error logs.
When you see a problem occur, see if you can connect with amqsputc or amqscnx. Also check the number of open queue manager connections at that point, to see how many are active.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| PisgahMan |
| Posted: Tue Aug 09, 2005 10:05 am Post subject: |
|
|
Voyager
Joined: 27 Jul 2004
Posts: 93
|
Thank you. I checked the server logs and didn't see anything unusual, esp around the time that the 1st tcp/ip error occured.
The client logs had the same tcp/ip error that the windows event log had.
My guess/gut feel is that somehow the COM itself somehow has a limit to its connections, but the code does not match the behavior I am seeing.
We have recycled the service and re-register the COM a couple of times and everything is fine for a few days.
Our next step is to turn on some additional logging in the .NET Service and see if we get any meaningful info from that. |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
