| Author |
Message
|
| kwelch |
 Posted: Fri Nov 08, 2002 10:01 am Post subject: wmqi get failing with truncated message |
 |
|
Master
Joined: 16 May 2001
Posts: 255
|
Hi,
We have been doing some performance testing on our WMQI messageflows at our company and in some of our tests have discovered that occaisionally a GET will fail with truncated message. Once it fails it attempts a second GET and is successful. The number it seems to start out with for the Buffer length is 3732. First question does anyone know how this number is derived? We noticed then that if a 5000 byte message comes in, we get the warning truncated message, then it sets the buffer length to 5000 and works. Then if a bigger message comes in, it gets the truncated message warning and ups the buffer length. It seems to use the largest message it has gotten to date and stays with that. All this happens behind the scenes and is transparent to us.....except for this is an application where miliseconds count and sometimes the retry on the get has taken up to 2 seconds, causing a timeout on the reply. Can someone explain how this is set, what causes it to be reset back to 3732(we tried stopping and starting the flow that didn't cause it) and if there is a way we can set it?
Thanks in advance for any explanation. We are seeing this trough a monitoring tool called Transaction Vision.
Karen |
|
| Back to top |
|
 |
| kirani |
| Posted: Fri Nov 08, 2002 10:58 pm Post subject: |
|
|
Jedi Knight
Joined: 05 Sep 2001
Posts: 3779
Location: Torrance, CA, USA
|
Hi karen,
This is a strange problem!! we've worked with large message (as big as 10-15 MB) in MQSI, but I never had this problem. What is your max msg size on the input queue?
From Where did you get 3732 number?! Is this error logged into Event viewer? I don't think there is any option in MQInput node where you can specify input buffer length. Could you please explain more about this error?
_________________
Kiran
IBM Cert. Solution Designer & System Administrator - WBIMB V5
IBM Cert. Solutions Expert - WMQI
IBM Cert. Specialist - WMQI, MQSeries
IBM Cert. Developer - MQSeries
|
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Mon Nov 11, 2002 5:43 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7716
|
Kirani,
We are piloting a new product called Transaction Vision, which captures all MQ API calls on the platforms it is installed on. This product is actually very interesting to use. One can see how various MQ applications are actually coded and what they do behind the scenes.
What we have seen watching MQSI is that when a new Flow is turned on, the buffer it initially specifies for the MQGET is 3732. The MQGMO Options MQSI is using are:
MQGMO_WAIT|MQGMO_SYNCPOINT|MQGMO_FAIL_IF_QUIESCING|MQGMO_LOGICAL_ORDER|MQGMO_COMPLETE_MSG
It uses this buffer size until it gets a message bigger than 3732, at which point the MQGET fails with a 2080. MQSi then redoes the MQGET with a buffer upped to the size of the message it just tried to MQGET. If that message was 5040 bytes for instance, this flow is now using 5040 bytes for all subsequent GETs, until something bigger than 5040 comes in.
The problem is, the time between the first MQGET on the to big message which fails to the time the second MQGET with the larger buffer is sometimes up to 1.5 seconds. This kills our front end app, which has a 2000 millisecond wait time.
Granted, this is not a big problem, because MQSI keeps the larger buffer, so the longer this flow runs, the less often we have the 2080 errors.
But where did it get that 3732 size in the first place? And when does the buffer get dropped back to this #? When the broker is stopped maybe? We have noticed that this 3732 # is used for all flows in all execution groups.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| maxis |
| Posted: Mon Nov 11, 2002 6:41 am Post subject: |
|
|
Centurion
Joined: 25 Jun 2002
Posts: 144
|
| PeterPotkay wrote: |
We are piloting a new product called Transaction Vision, which captures all MQ API calls on the platforms it is installed on. This product is actually very interesting to use. One can see how various MQ applications are actually coded and what they do behind the scenes.
|
hey peter .. product u mentioned seems to be very interesting. Can u provide web-site address where in can have peek at it ?
M |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Mon Nov 11, 2002 6:47 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7716
|
|
| Back to top |
|
|
| kwelch |
| Posted: Mon Nov 11, 2002 6:57 am Post subject: |
|
|
Master
Joined: 16 May 2001
Posts: 255
|
Hi Kirani,
No error is logged anywhere. Everything works ok as WMQI recovers by then resetting the bufferlength and doing the GET again. It's just that some messages seem to come back slower than our expected wait time. When we started looking into performance issues and using this tool Transaction Vision, we saw this was happening. I was just wondering how WMQI came up with the 3732 number and if there was a way that we could override it to something closer to our true buffer size to avoid the periodic timeouts we receive.
Karen |
|
| Back to top |
|
|
| vmcgloin |
| Posted: Mon Nov 11, 2002 7:31 am Post subject: |
|
|
Knight
Joined: 04 Apr 2002
Posts: 560
Location: Scotland
|
Karen,
You might already have this information but the WMQI readme has a section that mentions this in passing...
http://www-3.ibm.com/software/ts/mqseries/support/readme/all21_read.html
| Quote: |
30. MQInput node limitation: Data Conversion and expanding messages
c. The size of message involved and the size of the current internal buffer. If the message is less than 512 bytes in size, this problem should not occur. This is because the starting default buffer size that MQSeries Integrator uses for an MQInput node is 1K. For bigger messages the problem will only occur if the size of the converted message is greater than the size of the current buffer held by WMQI. This buffer generally increases as larger and larger messages are processed. However it can decrease if a series of smaller messages are encountered - hence an identical message to one that previously left the MQInput node down the OUT terminal may on another occasion go down the FAILURE terminal. (defect 22918) |
Maybe there is some kind of internal message, e.g. on broker startup, using the deployed message flow that has a length of 3732?
But, it sounds like the buffer size can go down as well as up...
Regards,
Vicky |
|
| Back to top |
|
|
| kwelch |
| Posted: Mon Nov 11, 2002 7:57 am Post subject: |
|
|
Master
Joined: 16 May 2001
Posts: 255
|
Hi Vicky,
No I had not seen that readme. Thanks! It makes it sound as though you must choose the convert option on the MQInput Node, which we do not. My understanding though is that WMQI will automatically convert to Unicode for processing your messageflow and back to the original on the way out, so I would guess that this would still apply to our situation even though we are not selecting the convert option.
I am not disagreeing that the buffer could decrease but we have not seen this happen yet. Our reply messages are only a few hundred bytes and they seem to consistently be at the 3732 buffer size.
Karen |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Sun Nov 24, 2002 5:00 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7716
|
We are contacting IBM to see if there is a way to disable this "feature". I will let you know of the results.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| kirani |
| Posted: Sun Nov 24, 2002 7:47 pm Post subject: |
|
|
Jedi Knight
Joined: 05 Sep 2001
Posts: 3779
Location: Torrance, CA, USA
|
Peter & Karen,
Currently MQGET plug-in node does something similar, it first allocates 4097 bytes of memory before making MQGET call, if it fails to get the message from queue because of smaller buffer size (by checking the reason code MQRC_TRUNCATED_MSG_FAILED), it then reallocates the memory with new length, which is returned by the failed MQGet call.
One has to come up with some number to allocate the memory for MQGET call first, so I guess they came up with 3272 number! It would be a great idea to make this number configurable. This might become a problem if you are working with small messages (100 bytes).
Thanks for sharing this info with us.
_________________
Kiran
IBM Cert. Solution Designer & System Administrator - WBIMB V5
IBM Cert. Solutions Expert - WMQI
IBM Cert. Specialist - WMQI, MQSeries
IBM Cert. Developer - MQSeries
|
|
| Back to top |
|
|
|
|
