| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Simple flow - MQin=>TCPClntOut=>TCPClntRcv=>MQOut - |
« View previous topic :: View next topic » |
| Author |
Message
|
| DSPS |
 Posted: Mon Apr 22, 2013 1:23 pm Post subject: Simple flow - MQin=>TCPClntOut=>TCPClntRcv=>MQOut - |
 |
|
Apprentice
Joined: 11 Sep 2008
Posts: 44
Location: Southern England
|
We have a simple flow in Broker 8.0.0.1 on MQ 7.5 running on Red Hat Enterprise Linux 6.4.
MQ Input node takes a message off a REQUEST_IN queue.
The output node goes to the input node of a TCP Client Output node, which just talks to a TCP socket application to deliver an XML payload.
We then move on to a TCP Client Receiver, which should receive an XML response from the socket application and put that response as a message into a REQUEST_OUT queue using an MQ Output node.
When we try this, it seems to get to the first TCP node and then stop. I am remote, and the firewalls allow me to contact the MQ listener port, and to the execution group's debug port, but when I try to run it in debug, the deployment takes over 6 minutes and then gives up (and that timeout has increased from the 2 minutes originally configured)
When we run the flow normally, the input message disappears and nothing shows up for a long time. If I stop the broker though, a message appears in the backout queue.
The connection to the back-end socket does get established - nestat -an|grep <socket> shows ESTABLISHED. After a message goes through, it goes to FIN_WAIT2 - suggesting that something hasn't finished a close handshake. We've tried some of the TCP advanced settings about closing and releasing connections, but to no noticeable effect.
Any thoughts you're willing to share about why we're not getting the XML payload through would be gratefully received, as would any hints about why it's remaining in FIN_WAIT2 until the broker is stopped.
Oh, and the XML we submit, if I just paste that into a telnet session to the socket application's port, I do get some valid XML back...
_________________
Distributed Systems Professional Services
WebSphere and MQ Consulting and Training |
|
| Back to top |
|
 |
| Vitor |
| Posted: Mon Apr 22, 2013 1:38 pm Post subject: Re: Simple flow - MQin=>TCPClntOut=>TCPClntRcv=>MQO |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| DSPS wrote: |
| it seems to get to the first TCP node and then stop. I am remote, and the firewalls allow me to contact the MQ listener port, and to the execution group's debug port, but when I try to run it in debug, the deployment takes over 6 minutes and then gives up (and that timeout has increased from the 2 minutes originally configured) |
This normally indicates that there's a dead/zombie/unexpected instance hanging onto the debug port which will prevent the deployment of anything new, and may be eating your responses which will prevent them turning up in your "real" flow.
| DSPS wrote: |
| Any thoughts you're willing to share about why we're not getting the XML payload through would be gratefully received, as would any hints about why it's remaining in FIN_WAIT2 until the broker is stopped. |
See above. I would stop the broker, restart, delete all instances of your flow then deploy again so you know what version you have (rather than any version you think you have). I would then take a user trace of your flow reading a message off the queue which will, as has been said before here, give you a lot more information on what's happening.
Given that all the debugger is giving you is that "it stops" that's not hard but this is a classic example of the debugger just telling you something bad is happening. The user trace will tell you what exactly is happening (or not happening) when the data hits the socket. We call this "a clue" and may indicate something probative. At a bare minimum, you'll be no worse off....
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| McueMart |
| Posted: Tue Apr 23, 2013 12:20 am Post subject: |
|
|
Chevalier
Joined: 29 Nov 2011
Posts: 490
Location: UK...somewhere
|
Another thing worth trying is using some kind of TCP proxy to see exactly what data is crossing the socket. This will enable you to figure out what is going wrong far more easily.
You can use the one build into the toolkit (Preferences -> Run/Debug -> TCP/IP Monitor) , or you could use something like WireShark.
Also on your TCPIPClientReceive Node, what do you have the Records & Elements -> Record Detection set to? |
|
| Back to top |
|
|
| DSPS |
| Posted: Tue Apr 23, 2013 1:41 am Post subject: RE: Simple flow.. or so I thought. |
|
|
Apprentice
Joined: 11 Sep 2008
Posts: 44
Location: Southern England
|
The record detection on the TCP Receive Node is 'Connection Closed'
That in accordance with the InfoCenter page at: -
http://localInfoCenter/help/topic/com.ibm.etools.mft.doc/ac67600_.htm
I might be able to use IPTrace or similar, but the broker's user trace shows activity after the MQ Input Node, activity around the TCP Client Output node, then just one line about the TCP Client Receive node: -
-<UserTrace line="366" file="/build/slot1/S800_P/src/DataFlowEngine/TemplateNodes/ImbOutputTemplateNode.cpp" number="3903" catalog="BIPmsgs" text="'Propagating message to destination terminal'" function="ImbOutputTemplateNode::processMessageAssembly" thread="6" timestamp="2013-04-23 07:27:44.464056"><Insert type="string">'ComIbmTCPIPClientOutputNode'</Insert><Insert type="string">'TCPIP Client Output'</Insert><Insert type="string">'out'</Insert></UserTrace>
<UserTrace line="175" file="/build/slot1/S800_P/src/DataFlowEngine/NativeTrace/ImbNativeTrace.cpp" number="3563" catalog="BIPmsgs" text="'3563'" function="com.ibm.broker.nodes.tcpipnodes.TCPIPClientReceiveNode@2fe02fe.requestData" thread="6" timestamp="2013-04-23 07:27:44.533508"><Insert type="string">'X.X.X.X'</Insert><Insert type="string">'Port'</Insert><Insert type="string">'TCPIP Client Receive'</Insert><Insert type="string">'createOrderFlow'</Insert></UserTrace>
And I've obscured the IP address and port number - those aren't the values in the log.... The next entry in the user trace comes up 12 minutes later.
_________________
Distributed Systems Professional Services
WebSphere and MQ Consulting and Training |
|
| Back to top |
|
|
| McueMart |
| Posted: Tue Apr 23, 2013 2:05 am Post subject: |
|
|
Chevalier
Joined: 29 Nov 2011
Posts: 490
Location: UK...somewhere
|
So does your back-end TCP application correctly close the socket connection after sending the response message? Can you check the code to 100% check this?
Else due to you having 'Connection Closed' as the record detection, your TCP Receive Node might just be sitting there waiting for more data.... |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Tue Apr 23, 2013 2:27 am Post subject: Re: RE: Simple flow.. or so I thought. |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| DSPS wrote: |
-<UserTrace line="366" file="/build/slot1/S800_P/src/DataFlowEngine/TemplateNodes/ImbOutputTemplateNode.cpp" number="3903" catalog="BIPmsgs" text="'Propagating message to destination terminal'" function="ImbOutputTemplateNode::processMessageAssembly" thread="6" timestamp="2013-04-23 07:27:44.464056"><Insert type="string">'ComIbmTCPIPClientOutputNode'</Insert><Insert type="string">'TCPIP Client Output'</Insert><Insert type="string">'out'</Insert></UserTrace>
<UserTrace line="175" file="/build/slot1/S800_P/src/DataFlowEngine/NativeTrace/ImbNativeTrace.cpp" number="3563" catalog="BIPmsgs" text="'3563'" function="com.ibm.broker.nodes.tcpipnodes.TCPIPClientReceiveNode@2fe02fe.requestData" thread="6" timestamp="2013-04-23 07:27:44.533508"><Insert type="string">'X.X.X.X'</Insert><Insert type="string">'Port'</Insert><Insert type="string">'TCPIP Client Receive'</Insert><Insert type="string">'createOrderFlow'</Insert></UserTrace>
|
you should really format the XML to make it more human readable with the
command
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| DSPS |
| Posted: Tue Apr 23, 2013 7:07 am Post subject: |
|
|
Apprentice
Joined: 11 Sep 2008
Posts: 44
Location: Southern England
|
Interesting, the formatted log actually suggests that the TCP Client Receive node did get something: -
2013-04-23 08:27:44.533508 6 UserTrace BIP3563I: A record has been received from Hostname ''X.X.X.X'' on Port ''YYYY'' in TCPIP receive node ''TCPIP Client Receive'' in message flow ''createOrderFlow''.
Which is promising, but that's the last entry from that flow invocation, the next entry is 12 minutes later when I tested again, it didn't seem to send what ever it got back onwards to the MQ Output node.
The question of whether or not the socket application closes the communication cleanly is moot, the vendor doesn't want to delve in to it, and doesn't know it, which is a(nother) challenge.
_________________
Distributed Systems Professional Services
WebSphere and MQ Consulting and Training |
|
| Back to top |
|
|
| DSPS |
| Posted: Wed Apr 24, 2013 4:29 am Post subject: |
|
|
Apprentice
Joined: 11 Sep 2008
Posts: 44
Location: Southern England
|
Right, I've tried keeping the connection open and reserved (we're likely to put quite a few messages into this TCP connection, if it ever works)
We get back: -
2013-04-24 11:42:24.626918 7 RecoverableException BIP3552E: Timeout occurred on connection (Hostname ''X.X.X.X'', Port ''yyyy'', Id ''0e19c2:13e3ba22e00:-8000'') while waiting for all the data required to form a record. The timeout period was ''0'' and the last data received was ''72 3E 3C 2F 69 58 4D 4C 4D 65 73 73 61 67 65 3E ''.
A TCP/IP node attempted to read a record from an input stream, but the time limit was reached before the full record was received. The timeout period is measured from when the first byte of data has arrived until the last byte of data has arrived. The timeout period is not reset until the full record has been received. Check that the whole record was sent in the specified length of time. Review the end application to make sure it is sending full records, and, if it is, increase the timeout period to a value that is sufficient to allow all the data to arrive.
Now, I was not expecting this to be trying to interpret the data coming back, or waiting for any record to complete, I had told it just to expect a BLOB.
The data should actually be XML, so I tried setting the TCP Client Receiver to expect XMLNS data and I get the same error back.
The problem appears to be in getting data back via the TCP Client Receiver, which I've also changed to reserve the stream, both input and output, since we might then avoid the overhead of reconnection for each message.
_________________
Distributed Systems Professional Services
WebSphere and MQ Consulting and Training |
|
| Back to top |
|
|
| McueMart |
| Posted: Wed Apr 24, 2013 4:50 am Post subject: |
|
|
Chevalier
Joined: 29 Nov 2011
Posts: 490
Location: UK...somewhere
|
So lets recap:
1) You have Record detection set to 'Connection Closed' on your TCPIP Receive node.
2) you claim that "The question of whether or not the socket application closes the communication cleanly is moot, the vendor doesn't want to delve in to it, and doesn't know it",
3) You have a log message showing you that the TCPIPRecive node is hitting a timeout exception after receiving the data "r></iXMLMessage>" (I converted the bytes for you).
I REALLY think you need to understand what 'Record Detection' is. The TCPIP Receive node is clearly getting the response data back, but it doesnt magically know that that is the end of the data - and it is therefore just sitting there waiting for more data.
You need to figure out some way for the TCPIPReceive node to work out the end of data is reached, which will result in it propagating the message further down your flow. |
|
| Back to top |
|
|
| DSPS |
| Posted: Wed Apr 24, 2013 7:31 am Post subject: |
|
|
Apprentice
Joined: 11 Sep 2008
Posts: 44
Location: Southern England
|
Thank you very much. I had also translated the HEX into ASCII.
I altered the record detection to be delimited and used the HEX as the custom delimiter, since that should close each XML string we get back from this socket.
We actually got some data and timing back.
With the other things going on, I think I'll go for a celebratory lie down in a dark room.
Thanks again.
_________________
Distributed Systems Professional Services
WebSphere and MQ Consulting and Training |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
