Author |
Message
|
Mattmanx |
Posted: Tue Sep 23, 2008 7:09 am Post subject: IO error transmitting message buffer on basic PUT |
|
|
Newbie
Joined: 23 Sep 2008 Posts: 4
|
All -
We started encountering this issue about one year into our product's lifecycle. Basically, we are suddenly unable to put messages to queue. We continually see the MQ 2009 error and the following written to SysErr - MQJE003: IO error transmitting message buffer on PUT.
We are using the Java MQ API to perform a direct PUT of a String message to queue. Strangely, the error occurs on the QM.commit, not the put.
The only thing changed before this started occurring was a decrease in the size of each message (about a tenth the size of the messages that had been working fine for a year).
I was actually able to recreate this in a test environment. Our messages are about 32,500 characters in length. I tried altering the contents, assuming we had an issue with a special character or something like that. No dice - still received the same error. The only way I have been able to avoid the exception is to increase or decrease the size of the message by about 25-30 characters.
We've placed a temporary fix in our production system as we had a huge backup of messages waiting to be sent. We are basically pairing two functional messages into a single MQ message to get around what seems to be an MQ issue with a size around 32 KB. This is not a potential long-term fix, though, as some of our single messages may approach our queue manager's max message length.
One important piece of information is that these same messages that fail with the MQ API send successfully from a different application using JMS. We may be able to switch our implementation to JMS in the future, but that was not an immediate option when this started occurring a few weeks ago.
Does anyone have any experience with a similar issue? Any idea what we're doing wrong (the MQ code is almost identical to the IBM RedBook examples for a basic PUT)? I haven't been able to find anything on the net... Thanks in advance |
|
Back to top |
|
 |
mqjeff |
Posted: Tue Sep 23, 2008 7:31 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008 Posts: 17447
|
I'd look at your network switches.
When you're changing your message size, you're likely changing the packet size as well. Your network switches could be deciding to do funny things at smaller sizes, that are hosing up the MQ connection such that you can't do a commit. |
|
Back to top |
|
 |
PeterPotkay |
Posted: Tue Sep 23, 2008 7:32 am Post subject: |
|
|
 Poobah
Joined: 15 May 2001 Posts: 7722
|
What version of MQ? MQ 5.3 CSD8, 9, 10 and 11 had plenty of flaky issues with 2009 error codes for Java apps. Particulary CSD 8 and 9. _________________ Peter Potkay
Keep Calm and MQ On |
|
Back to top |
|
 |
Mattmanx |
Posted: Tue Sep 23, 2008 1:41 pm Post subject: |
|
|
Newbie
Joined: 23 Sep 2008 Posts: 4
|
We are on MQ 6.0.0... Not sure if it's a network switch issue. I'm able to recreate this in a local environment, hitting MQ running on my local dev machine. Any rec's on what to do / try with this? Is it possible to change the size of the underlying network buffer? |
|
Back to top |
|
 |
gbaddeley |
Posted: Tue Sep 23, 2008 7:42 pm Post subject: |
|
|
 Jedi Knight
Joined: 25 Mar 2003 Posts: 2538 Location: Melbourne, Australia
|
Are there diagnostic messages written at the time to the errors/AMQERR01.LOG file, or are there .FDC files created at the time in the errors directory? (under /var/mqm on Unix, or C:/Program Files/IBM/WebSphere MQ on Windows) _________________ Glenn |
|
Back to top |
|
 |
Mattmanx |
Posted: Wed Sep 24, 2008 6:06 am Post subject: |
|
|
Newbie
Joined: 23 Sep 2008 Posts: 4
|
Yeah, I took a look at this last night... From testing in my local environment, here's the error I see. We've confirmed with our middleware folks that this same error is occurring in production.
I've found a few posts asking about this same error - one person made a change to their journaling, the other made a fix to their TCP/IP stack. We are in the process of running some tests, but if anyone has any feedback, it'd be much appreciated. Thank you for the help so far.
I find it disconcerting that the error only seems to occur at specific byte-lengths in our messages, and we see no problem with the longer messages. Please see MQ error below:
9/22/2008 17:19:05 - Process(3940.36) User(SYSTEM) Program(amqrmppa.exe)
AMQ9208: Error on receive from host localhost (127.0.0.1).
EXPLANATION:
An error occurred receiving data from localhost (127.0.0.1) over TCP/IP. This may be due to a communications failure.
ACTION:
The return code from the TCP/IP (recv) call was 10053 (X'2745'). Record these values and tell the systems administrator. |
|
Back to top |
|
 |
gbaddeley |
Posted: Wed Sep 24, 2008 9:09 pm Post subject: |
|
|
 Jedi Knight
Joined: 25 Mar 2003 Posts: 2538 Location: Melbourne, Australia
|
According to http://msdn.microsoft.com/en-us/library/ms740668.aspx :
WSAECONNABORTED 10053
Software caused connection abort.
An established connection was aborted by the software in your host computer, possibly due to a data transmission time-out or protocol error.
Not sure if the abort orginates from MQ, the winsock stack or Windows itself. _________________ Glenn |
|
Back to top |
|
 |
|