| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
| Messages lost when application putting very fast. |
« View previous topic :: View next topic » |
| Author |
Message
|
| betterchoices |
 Posted: Sat Jun 20, 2009 4:20 am Post subject: Messages lost when application putting very fast. |
 |
|
Novice
Joined: 08 Jun 2009
Posts: 18
|
I have an application which puts messages on the MQ Queue.
Below is the code to put messages on the MQ Queue:
for(int i=0; i<5000;i++){
MQMessage msg = new MQMessage();
msg.format = MQC.MQFMT_STRING ;
msg.writeString(message); // message is my message
MQPutMessageOptions pmo = new MQPutMessageOptions();
queue.put(msg, pmo); // queue is my MQ Queue.
}
now when I check my Queue in MQ Explorer, there are never 5000 messages. It can be any Current Queue Depth is any random number between 2000 to 5000. No one is listeneing to this Queue. Where are rest of messages gone.
Same Code works if I put one extra line( i.e. simple System.out.println)
i.e.
for(int i=0; i<5000;i++){
MQMessage msg = new MQMessage();
msg.format = MQC.MQFMT_STRING ;
msg.writeString(message); // message is my message
System.out.println("Putting Message: " + message);MQPutMessageOptions pmo = new MQPutMessageOptions();
queue.put(msg, pmo); // queue is my MQ Queue.
}then evrything works fine. Means I can 5000 messages in the Queue(using MQ explorer).
I can;t understand this, as only difference between two codes is the :
System.out.println("Putting Message: " + message);
And so only difference is difference in the speed of putting messages, so is the very fast speed of putting messages on the Queue is issue.
Because first code takes around 1-3 seconds and second one takes 10-15 seconds.
Please help. |
|
| Back to top |
|
 |
| bruce2359 |
| Posted: Sat Jun 20, 2009 7:00 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| Quote: |
| now when I check my Queue in MQ Explorer... |
The MQExplorer does not continuously update. Did you refresh the MQExplorer? There's a button at the top - two circular arrows pointing at each other, if memory serves. Click it.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| betterchoices |
| Posted: Sat Jun 20, 2009 8:30 am Post subject: |
|
|
Novice
Joined: 08 Jun 2009
Posts: 18
|
yes i refresh it several times...i cant even guess what can be issue. only thing .. only diff. in two codes is the time in putting all messages...
if 5000 messages comes in the queue in 1-2 secs.. then behavior is wired.. ramdom no of messages come on queue... queue depth is ramdomly..now if queue depth shows soemnumber(<no Of Messages sent).. no matter how many tiimes i refresh.. it remains the same... and i m sure not all messages come to queue becauz... by consuming the messages i mk sure how many messages a re there in the queue.. |
|
| Back to top |
|
|
| zpat |
| Posted: Sat Jun 20, 2009 9:17 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
Try checking the return code from the MQPUT.
MQ does not lose messages that it has accepted. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Sat Jun 20, 2009 9:31 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| Quote: |
| diff. in two codes is the time in putting all messages... |
Two codes?? What two codes?
What application are you using to put the messages? For ever mq call, a CompletionCode and ReasonCode are returned? Are these the codes that you recieved? What were the values of these two codes? Do a search on the codes to see what the values mean.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| betterchoices |
| Posted: Sat Jun 20, 2009 4:40 pm Post subject: |
|
|
Novice
Joined: 08 Jun 2009
Posts: 18
|
BY codes I mean peice of java code
i.e.
Code 1:
for(int i=0; i<5000;i++){
MQMessage msg = new MQMessage();
msg.format = MQC.MQFMT_STRING ;
msg.writeString(message); // message is my message
MQPutMessageOptions pmo = new MQPutMessageOptions();
queue.put(msg, pmo); // queue is my MQ Queue.
}
And
Code 2:
for(int i=0; i<5000;i++){
MQMessage msg = new MQMessage();
msg.format = MQC.MQFMT_STRING ;
msg.writeString(message); // message is my message
System.out.println("Putting Message: " + message);
MQPutMessageOptions pmo = new MQPutMessageOptions();
queue.put(msg, pmo); // queue is my MQ Queue.
}
By code I didn't mean any error code.
Code 1 executes very fast. Code 2 is lightly slower.
If I execute Code 1, I see any random number as Queue Depth(<5000).
If I execute Code 2, I see 5000 as Queue Depth. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Sat Jun 20, 2009 6:12 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| Quote: |
I execute Code 1, I see any random number as Queue Depth(<5000).
If I execute Code 2, I see 5000 as Queue Depth. |
Are these two code samples really two separate executable programs? Or are code1 and 2 executed in order, with no lag between them?
What is maximum depth for the queue?
What do you do with the messages put to the queue between test executions?
Your code doesn't appear to catch any errors. In both of your code1 and 2, what is the last ReasonCode and CompletionCode?
| Quote: |
| only difference is difference in the speed of putting messages |
Are you saying that execution time for code1 and 2 are different? If so, by how much? One second, one minute, one hour? Why the difference?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| sumit |
| Posted: Sun Jun 21, 2009 1:38 am Post subject: |
|
|
Partisan
Joined: 19 Jan 2006
Posts: 398
|
| bruce2359 wrote: |
| Are you saying that execution time for code1 and 2 are different? If so, by how much? One second, one minute, one hour? Why the difference? |
From poster's first post.
| Quote: |
| Because first code takes around 1-3 seconds and second one takes 10-15 seconds. |
I think betterchoices compiled the first code and when didn't find all messages in queue, tried to debug the code and added one extra line (s.o.p). The code took some extra time but results were good.
I actually had a similar encounter with a java code. Because of lack of time java developer just placed some debug codes like the one poster used and all messages arrived to queue.
betterchoices, can you compare the memory usage while you run code1 and code2? What if there is not enough memory available to your 'put' call. It can only get some when earlier instance of 'put' operation finished. What's the size of message?
And as bruce2359 pointed, you need to catch CompletionCode and ReasonCode and analyse it.
_________________
Regards
Sumit |
|
| Back to top |
|
|
| Vitor |
| Posted: Sun Jun 21, 2009 1:40 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
I add my voice to the chorus of others - there's no error trapping in that code. I will believe that your Code 1 causes WMQ to lose messages at the point when you demonstrate that it executes 5000 MQPut statements (or the Javamagic equivalent) without receiving any error codes from the queue manager yet less than 5000 messages have been received in all the possible destinations (receiving less than 5000 message where you expect that is not proof of loss!  )
Also you'll need to demonstrate all these messages are persistent. Non-persistent messages going into The Great Bit Bucket In The Sky is just one of the tragic things about being a non-persistent message.
Once you get to that point you will be entitled to punch the air & post this fact proudly on the forum as you head off, evidence in hand, to raise a PMR with IBM.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| betterchoices |
| Posted: Sun Jun 21, 2009 4:31 am Post subject: |
|
|
Novice
Joined: 08 Jun 2009
Posts: 18
|
| Quote: |
I think betterchoices compiled the first code and when didn't find all messages in queue, tried to debug the code and added one extra line (s.o.p). The code took some extra time but results were good.
I actually had a similar encounter with a java code. Because of lack of time java developer just placed some debug codes like the one poster used and all messages arrived to queue. |
Thats absolutely true..
| Quote: |
| What is maximum depth for the queue? |
its 5000, I even tried to increase and test but results were same.
| Quote: |
| Are these two code samples really two separate executable programs? Or are code1 and 2 executed in order, with no lag between them? |
No Intially there was only Code 1. When it did'nt worked as expected, then I placed one extra line(System.out.println) and then executed again(And i am refereing this new code i.e Code 1+ S.O.P. as Code 2).
| Quote: |
| What do you do with the messages put to the queue between test executions? |
I am doing nothing with the messagess Code1 and Code 2 executed separately.
First I ran Code 1, and when found message loss, then I modified and prepared Code 2, and executed Code 2, and result was rihgt(i.e. 5000 messages reached Queue).
| Quote: |
| our code doesn't appear to catch any errors. In both of your code1 and 2, what is the last ReasonCode and CompletionCode? |
There is a catch block, but i have not shown in the sample java codes(i.e. Code 1 & Code 2). but I never received and exception, no error, or any error code.
| Quote: |
betterchoices, can you compare the memory usage while you run code1 and code2? What if there is not enough memory available to your 'put' call. It can only get some when earlier instance of 'put' operation finished. What's the size of message? |
Message size is very small, well behaior is same, no matter message size, i tried with "test", "message", "t" i.e. simple string messages.
Regarding memory, I would check and confirm. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Sun Jun 21, 2009 5:30 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| Quote: |
| Because first code takes around 1-3 seconds and second one takes 10-15 seconds. |
Betterchoices: do you understand that a queue with maxdepth set to 5000 cannot contain more than 5000 messages. If your code had trapped errors, you would have received a queue-full on the second and subsequent executions.
After execution of your code is complete, can you display the contents of the queue with another utility - like amqsbcg?
What other source code is missing from your post? Does the queue seem to empty itself? Did you set message expiry, for example?
And one last question: are you at all familiar with the WMQ product? Have you read the WMQ Fundamentals manual?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| Vitor |
| Posted: Sun Jun 21, 2009 1:17 pm Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| betterchoices wrote: |
| I never received and exception, no error, or any error code. |
Then you have apparently satisfied the criteria I set - 5000 successful message puts, no errors returned, less than 5000 messages delivered. I offer you congratulations, and would ask that you post the results of the PMR you're about to raise, reporting this serious bug in the software.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Sun Jun 21, 2009 1:44 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| betterchoices wrote: |
There is a catch block, but i have not shown in the sample java codes(i.e. Code 1 & Code 2). but I never received and exception, no error, or any error code.
|
Lets see it.
Post your code that includes your connection to the QM, your open of the q, your puts, your close of the q, your disconnect from the QM and the try/catch/finally block.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Sun Jun 21, 2009 3:47 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| Quote: |
| I never received and exception, no error, or any error code. |
I repeat: For each and every mq call a ReasonCode and CompletionCode are returned to the application. These are issued for both successful and unsuccessful calls.
Apparently, you did not write the appropriate application source code to successfully capture, display and take appropriate action, when these codes were returned to your application.
I'd pass on Vitor's PMR suggestion at this point. Vitor's cynnical side is showing. (I'm right behind you.)
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| Vitor |
| Posted: Sun Jun 21, 2009 9:55 pm Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| bruce2359 wrote: |
| Vitor's cynnical side is showing. (I'm right behind you.) |
I'd feel hurt and slighted by that comment if my cynical side didn't run the width of my body.... 
But I stand by my comment. If betterchoices is as confident as he sounds in his code and the results (and we must defer to his observations in this matter) then the software has exhibited an inappropriate response. It should either:
- accept and process all 5000 messages
- reject 1-n messages with a relevant code.
(For the pedantic among the readership, I repeat that "process" can mean "delete" or "route to an unexpected place")
Thus if the situation is as the situation appears, it's a software fault and only IBM can deal with that. No advice we give will be especially relevant.
I can't even duplicate the problem. But I'm using C; I did try the Java sample code posted (wretched Java - might as well have a YouHaveNoClue error.....   )
Nor have I seen this problem in lo these many years of watching applications putting messages onto and off of queues. Often at very high speed as people get rich by making money wizz round and round in circles. But I have what I have in my sig for a reason, and maybe this is a bug in the new version.
Or a very old version. We don't know.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
|
|
|
  |
Goto page 1, 2 Next |
Page 1 of 2 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
