| Author |
Message
|
| nzr1949 |
 Posted: Mon Jul 25, 2005 7:44 am Post subject: Segmentation problem: message too large |
 |
|
Novice
Joined: 16 Nov 2004
Posts: 15
|
Hi,
I am using MQSeries Java and connecting to a remote queue. I am attempting to use segmentation to post a large file onto a queue as several messages. I am encountering an error with reason code 2030 (MQRC_MSG_TOO_BIG_FOR_Q) when my code tries to put the messages on the queue. The code is below:
| Code: |
qMgr = new MQQueueManager(qManager);
int openOptions = MQC.MQOO_INPUT_AS_Q_DEF |
MQC.MQOO_OUTPUT | MQC.MQOO_INQUIRE |
MQC.MQOO_BROWSE |
MQC.MQPMO_SET_IDENTITY_CONTEXT;
myQueue = qMgr.accessQueue(queue, openOptions,null,null,null);
File f = new File("myFile.zip");
FileInputStream fileInput = new FileInputStream(f);
MQPutMessageOptions pmo = new MQPutMessageOptions();
pmo.options = MQC.MQPMO_LOGICAL_ORDER |
MQC.MQPMO_SET_IDENTITY_CONTEXT |
MQC.MQPMO_SYNCPOINT;
MQMessage m = new MQMessage();
m.format = MQC.MQFMT_STRING;
m.messageFlags = MQC.MQMF_SEGMENTATION_ALLOWED;
int i = 0;
while ((i = fileInput.read()) != -1)
{
m.write(i);
}
myQueue.put(m,pmo);
|
The above code fails when the last statement is executed (the put). I've looked into the error message and found the following advice:
1. Increase the value of the queue's MaxMsgLength attribute; the queue-manager's MaxMsgLength attribute may also need increasing
2. Break the message into several smaller messages.
3. Specify MQMF_SEGMENTATION_ALLOWED in the MsgFlags field in MQMD; this will allow the queue manager to break the message into segments.
It's obviously no. 3 from the above that I'm interested in since I'm attempting to do segmentation. Looking at my code you can see that I have specified MQMF_SEGMENTATION_ALLOWED as suggested but still the problem persists.
I've checked the MaxMessageLength value for both the queue and queue manager and they both return a value of 4Mb. The file I'm attempting to put onto the queue is 7Mb in size. I've also checked for threads on this topic, the best match so far is
http://www.mqseries.net/phpBB/viewtopic.php?t=20654&highlight=segmentation
which didn't solve my problem. I have the feeling that there is a problem with the combination of options I have in my code. I am completely stuck, can anybody please help?
Thanks |
|
| Back to top |
|
 |
| fschofer |
| Posted: Mon Jul 25, 2005 7:58 am Post subject: |
|
|
Knight
Joined: 02 Jul 2001
Posts: 524
Location: Mainz, Germany
|
Hi,
| Code: |
int openOptions = MQC.MQOO_INPUT_AS_Q_DEF |
MQC.MQOO_OUTPUT | MQC.MQOO_INQUIRE |
MQC.MQOO_BROWSE |
MQC.MQPMO_SET_IDENTITY_CONTEXT; |
Why do you use a PMO option within the openOptions ?
Greetings
Frank |
|
| Back to top |
|
|
| nzr1949 |
| Posted: Mon Jul 25, 2005 8:19 am Post subject: |
|
|
Novice
Joined: 16 Nov 2004
Posts: 15
|
Yep, that MQC.MQPMO_SET_IDENTITY_CONTEXT shouldn't be in there. I now have these options as:
| Code: |
int openOptions = MQC.MQOO_OUTPUT | MQC.MQOO_INQUIRE |
MQC.MQOO_FAIL_IF_QUIESCING;
|
The problem still persists. Any ideas guys (or gals  )? |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Mon Jul 25, 2005 10:47 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Are you sure it doesn't have anything to do with the syncpoint option?
Do you have 10 MB in log space ?
Enjoy  |
|
| Back to top |
|
|
| colincrist |
| Posted: Mon Jul 25, 2005 11:42 am Post subject: |
|
|
Novice
Joined: 24 Feb 2004
Posts: 22
|
You may be able to crib code from this:
http://activemq.org/JMS+Streams
Its Apache 2.0 licensed so no problems with reuse and the code is not dependent on ActiveMQ so should work with WMQ (if you' don't mind using JMS that is). Note you cannot currently have concurrent streamed messages on the queue, I beleive they're gonna fix this.
Simple but effective.
Colin. |
|
| Back to top |
|
|
| fschofer |
| Posted: Mon Jul 25, 2005 1:12 pm Post subject: |
|
|
Knight
Joined: 02 Jul 2001
Posts: 524
Location: Mainz, Germany
|
Hi, this code is working fine for me
| Code: |
import com.ibm.mq.*;
import java.io.*;
public class Seg extends Object {
private static String qManager = "WBRK";
private static String queue = "SEG";
private static MQQueueManager qMgr;
private static MQQueue myQueue;
public Seg() {
}
public static void main(String args[]) {
try {
qMgr = new MQQueueManager(qManager);
int openOptions =
MQC.MQOO_INPUT_AS_Q_DEF
| MQC.MQOO_OUTPUT
| MQC.MQOO_INQUIRE
| MQC.MQOO_SET_IDENTITY_CONTEXT;
myQueue = qMgr.accessQueue(queue, openOptions, null, null, null);
File f = new File("myFile.zip");
FileInputStream fileInput = new FileInputStream(f);
MQPutMessageOptions pmo = new MQPutMessageOptions();
pmo.options =
MQC.MQPMO_LOGICAL_ORDER
| MQC.MQPMO_SET_IDENTITY_CONTEXT
| MQC.MQPMO_SYNCPOINT;
MQMessage m = new MQMessage();
m.format = MQC.MQFMT_STRING;
m.messageFlags = MQC.MQMF_SEGMENTATION_ALLOWED;
int i = 0;
while ((i = fileInput.read()) != -1) {
m.write(i);
}
myQueue.put(m, pmo);
qMgr.commit();
} catch (Exception e) {
e.printStackTrace();
}
}
} |
Greetings Frank |
|
| Back to top |
|
|
| nzr1949 |
| Posted: Tue Jul 26, 2005 1:42 am Post subject: |
|
|
Novice
Joined: 16 Nov 2004
Posts: 15
|
fschofer:
I tried using the source code you posted but still have the same problem. You said that the code works fine for you, then this could mean that my problem isn't the code but something else, perhaps the queue? How can I check?
fjb_saper:
I tried removing the syncpoint option but it didn't solve the problem. Also, fschofer says that the code works with the syncpoint option intact so this definitely isn't the cause. I'm not sure what you meant by the log space of 10Mb, but if it's related to the syncpoint option I don't think it has a bearing on the problem.
One of the suggestions I've come across is to "...check whether the BufferLength parameter is specified correctly...". I've gone through all the relevant redbooks I can get my hands on and the best info I've found on the topic of bufferLength with regards to segmentation is the following:
| Quote: |
The length of the message in Buffer. Zero is valid, and indicates that the message contains no application data. The upper limit for BufferLength depends on various factors:
* If the destination is a local queue or resolves to a local queue, the upper limit depends on whether:
– The local queue manager supports segmentation.
– The sending application specifies the flag that allows the queue manager to segment the message. This flag is MQMF_SEGMENTATION_ALLOWED, and can be specified either in a version-2 MQMD, or in an MQMDE used with a version-1 MQMD.
If both of these conditions are satisfied, BufferLength cannot exceed 999 999 999 minus the value of the Offset field in MQMD. The longest logical message that can be put is therefore 999 999 999 bytes (when Offset is zero). However, resource constraints imposed by the operating system or environment in which the application is running might result in a lower limit. If one or both of the above conditions is not satisfied, BufferLength cannot exceed the smaller of the queue’s MaxMsgLength attribute and queue-manager’s MaxMsgLength attribute.
* If the destination is a remote queue or resolves to a remote queue, the conditions for local queues apply, but at each queue manager through which the message must pass in order to reach the destination queue; in particular:
1. The local transmission queue used to store the message temporarily at the local queue manager
2. Intermediate transmission queues (if any) used to store the message at queue managers on the route between the local and destination queue managers
3. The destination queue at the destination queue manager
The longest message that can be put is therefore governed by the most
restrictive of these queues and queue managers. When a message is on a transmission queue, additional information resides with the message data, and this reduces the amount of application data that can be
carried. In this situation, subtract MQ_MSG_HEADER_LENGTH bytes from the MaxMsgLength values of the transmission queues when determining the limit for BufferLength.
Note: Only failure to comply with condition 1 can be diagnosed synchronously (with reason code MQRC_MSG_TOO_BIG_FOR_Q or
MQRC_MSG_TOO_BIG_FOR_Q_MGR) when the message is put. If
conditions 2 or 3 are not satisfied, the message is redirected to a
dead-letter (undelivered-message) queue, either at an intermediate queue
manager or at the destination queue manager. If this happens, a report
message is generated if one was requested by the sender.
MQPUT – BufferLength parameter
|
From the above:
| Quote: |
"... The local queue manager supports segmentation ..."
|
Could this be the problem, perhaps the queue manager I'm connecting to doesn't support segmentation? I doubt it, but how can I check? This could be a problem considering that it isn't on my local machine and I'm connecting remotely.
Also from the above:
| Quote: |
The sending application specifies the flag that allows the queue manager to segment the message. This flag is MQMF_SEGMENTATION_ALLOWED, and can be specified either in a version-2 MQMD, or in an MQMDE used with a version-1 MQMD.
|
I've checked this by placing
| Code: |
m.setVersion(MQC.MQMD_VERSION_2);
|
into my code but it doesn't solve the problem.
Thanks for all the responses. Any other ideas? |
|
| Back to top |
|
|
| Nigelg |
| Posted: Tue Jul 26, 2005 5:33 am Post subject: |
|
|
Grand Master
Joined: 02 Aug 2004
Posts: 1046
|
You do not mention the platform you are on.
In the APR it says
| Quote: |
On z/OS, the queue manager does not support the segmentation of messages. If a message is too big for the queue, the MQPUT or MQPUT1 call fails with reason code MQRC_MSG_TOO_BIG_FOR_Q. However, the MQMF_SEGMENTATION_ALLOWED option can still be specified, and allows the message to be segmented at a remote queue manager.
|
Are you on zOS? |
|
| Back to top |
|
|
| nzr1949 |
| Posted: Tue Jul 26, 2005 6:23 am Post subject: |
|
|
Novice
Joined: 16 Nov 2004
Posts: 15
|
| I'm on the Windows platform |
|
| Back to top |
|
|
| fschofer |
| Posted: Tue Jul 26, 2005 7:34 am Post subject: |
|
|
Knight
Joined: 02 Jul 2001
Posts: 524
Location: Mainz, Germany
|
Hi,
| Quote: |
| This could be a problem considering that it isn't on my local machine and I'm connecting remotely. |
If you use a client connection you may have to increase the MAXMSGL parameter of your server connection channel.
| Quote: |
MAXMSGL(integer)
Specifies the maximum message length that can be transmitted on the
channel. This is compared with the value for the partner and the actual
maximum used is the lower of the two values.
The value zero means the maximum message length for the queue
manager.
On AIX, Compaq OpenVMS, HP-UX, Linux, OS/2 Warp, OS/400, Solaris,
and Windows, specify a value in the range zero through to the maximum
message length for the queue manager.
See the MAXMSGL parameter of the ALTER QMGR command for more
information.
On z/OS, specify a value in the range zero through 100 MB.
On other platforms, specify a value in the range zero through 4 194 304
bytes (4 MB). |
Greetings
Frank |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Tue Jul 26, 2005 12:49 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
What is the max qdepth of your xmitq ?
Will it hold a transaction of the size of your file (before/after segmentation) ?
Enjoy |
|
| Back to top |
|
|
| nzr1949 |
| Posted: Wed Jul 27, 2005 12:33 am Post subject: |
|
|
Novice
Joined: 16 Nov 2004
Posts: 15
|
Hi,
Thanks for everyone's replies. I've put automatic segmentation on hold for now since it's been giving me so trouble. I've decided to "manually" do it by splitting up the file into segments in my code and placing each of these onto the queue. So far it seems to work quite well. If I have time to revisit automatic segmentation at the end I'll give it another shot.
A question: at the moment I'm having to write the name of the file into the first line of the first message so that when I get the group of messages off of the queue and reassemble the file I know what it was originally called and can therefore give it the exact same name. Is there any automatic way of doing this, maybe an MQ variable that can be set which would allow the recipient system to access this info?
Thanks |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Jul 27, 2005 11:39 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
If you are using JMS you can set specific user defined properties.
like
| Code: |
| msg.setProperty("File Name","my file.pdf"); |
Enjoy |
|
| Back to top |
|
|
| pawanwatt_008 |
| Posted: Thu Oct 27, 2005 4:23 am Post subject: Segmentation problem: message too large |
|
|
Newbie
Joined: 27 Oct 2005
Posts: 1
|
| Just change the value (Increase to some upper limit) of MAXMSGL on Server connection channel. Then it works like a breeze. |
|
| Back to top |
|
|
| hopsala |
| Posted: Thu Oct 27, 2005 8:16 am Post subject: Re: Segmentation problem: message too large |
|
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
| pawanwatt_008 wrote: |
| Just change the value (Increase to some upper limit) of MAXMSGL on Server connection channel. Then it works like a breeze. |
?!
Dude/t, did you look at the date? This thread is dead and buried. Not only that, but your answer was off the mark - he specifically stated "since I'm attempting to do segmentation". Please don't post unnecessarily  |
|
| Back to top |
|
|
|
|
