| Author |
Message
|
| reddy_kal |
 Posted: Mon Mar 01, 2010 7:50 am Post subject: Getting Reason Code 2010 when putting 130MB file to a queue |
 |
|
Acolyte
Joined: 04 Nov 2002
Posts: 73
|
Hello All,
I have a C Program (we modified "amqsputc.c" based on our requirements) to put a message to queue. I am using QueueManager Segmentation for splitting the messages into mulitple pieces. It is working fine when I run this program on machine where I have Queuemanager. When I try to run this similar program (Compiled with mqic32.lib) from a MQClient box, I am getting 2010 error. I have Queue, Channel and QueueManager all set to default size (4MB).
Here is the snap shot of my code...
memcpy(md.Format, MQFMT_STRING, (size_t)MQ_FORMAT_LENGTH);
md.MsgFlags=MQMF_SEGMENTATION_ALLOWED;
md.Version=MQMD_VERSION_2;
//pmo.Options = MQPMO_LOGICAL_ORDER + MQPMO_SYNCPOINT;
pmo.Version = MQPMO_VERSION_2;
buffer = malloc(Filesize);
pos = 0;
while ((c = fgetc(fp)) != EOF) // Get 1 character
{
buffer[pos++] = c; // append the character to the message
continue;
}
MQPUT(Hcon, /* connection handle */
Hobj, /* object handle */
&md, /* message descriptor */
&pmo, /* default options (datagram) */
messlen, /* message length */
buffer, /* message buffer */
&CompCode, /* completion code */
&Reason); /* reason code */
if (Reason == MQRC_NONE)
{
}
else
{
fprintf(stderr, "MQPUT ended with reason code %ld\n", Reason);
exit( (int)Reason );
}
Can we do MQSegmentation from a MQClient machine?
Thanks in Advance.
Thanks
-Reddy |
|
| Back to top |
|
 |
| exerk |
| Posted: Mon Mar 01, 2010 8:43 am Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
| Code: |
Reason Code 2010 x'7DA'
MQRC_DATA_LENGTH_ERROR
The DataLength parameter is not valid. Either the parameter pointer is not valid, or it points to read-only storage. (It is not always possible to detect parameter pointers that are not valid; if not detected, unpredictable results occur.)
This reason can also be returned to an MQ client program on the MQGET, MQPUT, or MQPUT1 call, if the BufferLength parameter exceeds the maximum message size that was negotiated for the client channel.
Completion Code: MQCC_FAILED
Programmer response: Correct the parameter.
If the error occurs for an MQ client program, also check that the maximum message size for the channel is big enough to accommodate the message being sent; if it is not big enough, increase the maximum message size for the channel. |
The fact you are segmenting implies that the messages are greater than 4MB, and 100MB is the maximum, so what is the buffer size and operational reason for segmenting?
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Mar 01, 2010 8:57 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| exerk wrote: |
| The fact you are segmenting implies that the messages are greater than 4MB, and 100MB is the maximum, so what is the buffer size and operational reason for segmenting? |
To be fair, the subject line does say the "message" is 130Mb and over the max length for a single message.
Though I shudder to think how long it takes to put 130Mb into a message buffer 1 character at a time!
Aside from this code problem, there does seem to be a buffer issue with the code. There is also a question about why you're trying to stuff a large file into a single mesage & why you're reinventing a wheel rather than using FTE.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| exerk |
| Posted: Mon Mar 01, 2010 9:00 am Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
| Vitor wrote: |
| ...To be fair, the subject line does say the "message" is 130Mb and over the max length for a single message... |
I'm having one of those b****y doh! days agin 
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| reddy_kal |
| Posted: Mon Mar 01, 2010 10:47 am Post subject: Getting Reason Code 2010 when putting 130MB file to a queue |
|
|
Acolyte
Joined: 04 Nov 2002
Posts: 73
|
MQFTE is licensed copy and we will have to pay for using that product.
We are already using these C Programs for many years and we never had any problems.
We got a new requirement to transfer a file (which is more than 100MB) from one machine to another (where we have MQClient installed).
After adding the MQ segmentation to my existing C program I was able to make it working (Segmenting this 130MB into multiple 4MB files) but I am able to use on the machine where I have MQServer installed. So if I have to transfer a 130MB file from one machine to another machine, I will have to install MQServer and use this C program for transfering bigger files. But in my case I have MQClient installed. I am using MQSERVER environment variable with Server Connection Channel set to 4MB.
Since I cannot increase SVRCONN to more than 100MB I am not able to put this message to the queue.
"buffer" in this program is size of the file. So if the filesize is 130MB, buffer will be allocated with 130MB. I also tried to hardcoding the buffer to 4MB but it did not work.
-Reddy |
|
| Back to top |
|
|
| reddy_kal |
| Posted: Mon Mar 01, 2010 10:55 am Post subject: Getting Reason Code 2010 when putting 130MB file to a queue |
|
|
Acolyte
Joined: 04 Nov 2002
Posts: 73
|
Hello,
If I try to transfer 90MB file with Queue and QueueManager set to 4MB (MaxMessageLength). I was able to transfer it successfully.
The only change I had to do is change the SVRCONN MaxMessage Length Property to 100MB (104857600). After I change that I was able to transfer 90MB file using this Client program with segmentation. It divided 90MB file into multiple messages.
My goal is to transfer message bigger than 100MB from one queue to another queue using MQClient libraries.
-Reddy |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Mar 01, 2010 10:59 am Post subject: Re: Getting Reason Code 2010 when putting 130MB file to a qu |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| reddy_kal wrote: |
| MQFTE is licensed copy and we will have to pay for using that product. |
Fair point. I wonder out loud if the amount you've spent developing this code and maintaining it all these years is less or more than the license fee for FTE..... 
| reddy_kal wrote: |
| We are already using these C Programs for many years and we never had any problems. |
Aside from poor response times as it moves the file into the buffer one byte at a time? 
| reddy_kal wrote: |
| "buffer" in this program is size of the file. So if the filesize is 130MB, buffer will be allocated with 130MB. I also tried to hardcoding the buffer to 4MB but it did not work. |
Well if you hardcode the buffer to 4Mb and then put 130Mb in it I'm not surprised it didn't work!
I also have to ask if a 32 bit signed binary can take a number as large as 130Mb. Anyone?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| reddy_kal |
| Posted: Tue Mar 02, 2010 7:09 am Post subject: |
|
|
Acolyte
Joined: 04 Nov 2002
Posts: 73
|
Actually we are using these C Programs for many year.
MQFTE is farily new product.
Most of our application team have already embeded these "C" Programs in their application for sending and receiving messages from the queues.
Since this is the new requirement for transfering large files(bigger than 100MB) from one machine to another, trying to find modify this existing C program instead of migrating the whole environment to new product.
Thanks
-Reddy |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Tue Mar 02, 2010 7:40 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| Quote: |
| I also have to ask if a 32 bit signed binary can take a number as large as 130Mb. Anyone? |
I quickly ran out of fingers... but 31 bits allows for 2Gig (2147483647).
But MQ only allows for a 100meg max for each physical message (segment or not).
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| RogerLacroix |
| Posted: Wed Mar 03, 2010 10:52 am Post subject: Re: Getting Reason Code 2010 when putting 130MB file to a qu |
|
|
Jedi Knight
Joined: 15 May 2001
Posts: 3264
Location: London, ON Canada
|
Hi,
| reddy_kal wrote: |
| I have a C Program (we modified "amqsputc.c" based on our requirements) to put a message to queue. |
You should have used the free open source File2Msg and/or Msg2File programs from:
http://www.capitalware.biz/mq_code_c.html
| reddy_kal wrote: |
| MQFTE is licensed copy and we will have to pay for using that product. |
| Vitorl wrote: |
| I wonder out loud if the amount you've spent developing this code and maintaining it all these years is less or more than the license fee for FTE |
What about the FREE open source project MQ File Mover ? It is a complete package for moving files in/out of MQ. It can compress message data before sending the message so that 130MB is more like 40MB (or whatever the compressed size may be).
Also, in the next release of MQ File Mover, it will support file streaming so that the file size will no longer be an issue. For more info on MQ File Mover, go to:
http://www.capitalware.biz/mqfm_overview.html
Regards,
Roger Lacroix
Capitalware Inc.
_________________
Capitalware: Transforming tomorrow into today.
Connected to MQ!
Twitter |
|
| Back to top |
|
|
|
|
