ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » Clustering » Messages missing intermittently in cluster

Post new topic  Reply to topic Goto page 1, 2  Next
 Messages missing intermittently in cluster « View previous topic :: View next topic » 
Author Message
cra1gl1
PostPosted: Tue Jun 05, 2018 11:42 am    Post subject: Messages missing intermittently in cluster Reply with quote

Apprentice

Joined: 25 Apr 2018
Posts: 30

I have a cluster of 8 qmgrs with 2 full repos. Manually defined xmit queues with DEFINE QL(CLUSTER1.XMITQ) USAGE(XMITQ) CLCHNAME(*.CLUSTER1)

QM5 and QM6 has QL(TESTQ) CLUSTER(CLUSTER1) DEFBIND(NOTFIXED)

Trying to put msgs from QM3(also tried from QM2), "amqsput TESTQ QM3" and messages are being load balanced as expected, BUT intermittently, I'm missing messages. if I put 6 msgs, 3 go to 1 qmgr and 2 go to the other and 1 is missing. Noticed this multiple times. I did check the xmitq on QM3 and deadq on the target qmgrs and the msgs aren't there.

dspmqrte -m QM3 -q TESTQ also shows no issues(puts msg on QM5 and QM6 alternaively).

Where else should i be checking??
Back to top
View user's profile Send private message
mvic
PostPosted: Tue Jun 05, 2018 12:23 pm    Post subject: Re: Messages missing intermittently in cluster Reply with quote

Jedi

Joined: 09 Mar 2004
Posts: 2080

This is a common thing to see if your getter app is getting outside syncpoint, then you end it with Ctrl+C, and then a message is put. The amqsget sample program gets outside syncpoint, so is susceptible to this. It trips a few people up when they are trying out MQ. How are you consuming the messages that arrive on their destination queues?
Back to top
View user's profile Send private message
cra1gl1
PostPosted: Tue Jun 05, 2018 12:37 pm    Post subject: Re: Messages missing intermittently in cluster Reply with quote

Apprentice

Joined: 25 Apr 2018
Posts: 30

[quote="mvic"]This is a common thing to see if your getter app is getting outside syncpoint, then you end it with Ctrl+C, and then a message is put. The amqsget sample program gets outside syncpoint, so is susceptible to this. It trips a few people up when they are trying out MQ. How are you consuming the messages that arrive on their destination queues?[/quote]

Damn.. that is what I’m using to get the msgs. I’ll test it again..I’ll make sure the ipprocs is 0 before I put msgs again. Thanks for your time.
Back to top
View user's profile Send private message
tczielke
PostPosted: Tue Jun 05, 2018 12:37 pm    Post subject: Reply with quote

Guardian

Joined: 08 Jul 2010
Posts: 939
Location: Illinois, USA

Is your TESTQ defined with DEFPSIST(YES)? From a testing stand point, that could help rule out the message was dropped in flight for some reason because it was non-persistent. amqsput does not set the persistence, and relies on the default setting of the queue.
_________________
Working with MQ since 2010.
Back to top
View user's profile Send private message
bruce2359
PostPosted: Tue Jun 05, 2018 12:49 pm    Post subject: Re: Messages missing intermittently in cluster Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

cra1gl1 wrote:
Trying to put msgs from QM3(also tried from QM2), "amqsput TESTQ QM3" and messages are being load balanced as expected, BUT intermittently, I'm missing messages. if I put 6 msgs, 3 go to 1 qmgr and 2 go to the other and 1 is missing.

How did you end the amqsput program?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
cra1gl1
PostPosted: Tue Jun 05, 2018 1:12 pm    Post subject: Re: Messages missing intermittently in cluster Reply with quote

Apprentice

Joined: 25 Apr 2018
Posts: 30

bruce2359 wrote:
How did you end the amqsput program?


By pressing enter twice,
And ending the amqsget with ctrl+c
Back to top
View user's profile Send private message
cra1gl1
PostPosted: Tue Jun 05, 2018 1:14 pm    Post subject: Reply with quote

Apprentice

Joined: 25 Apr 2018
Posts: 30

I can't reproduce the issue with "ALTER QL(TESTQ) DEFPSIST(YES)"

not sure if amqsget is causing the issue or something else
Back to top
View user's profile Send private message
mvic
PostPosted: Tue Jun 05, 2018 1:38 pm    Post subject: Reply with quote

Jedi

Joined: 09 Mar 2004
Posts: 2080

Maybe use something other than amqsput/amqsget.
For example, amqsblst.
To put 5 messages using syncpoint: amqsblst QM Q -W -u 1 -c 5
and
To get 5 messages using syncpoint: amqsblst QM Q -R -u 1 -c 5
Back to top
View user's profile Send private message
tczielke
PostPosted: Tue Jun 05, 2018 4:23 pm    Post subject: Reply with quote

Guardian

Joined: 08 Jul 2010
Posts: 939
Location: Illinois, USA

This might have to do with NPMSPEED(FAST) on your clusrcvr channels. Try going back to non-persistent messages and a NPMSPEED setting of normal, and see if the issue goes away.
_________________
Working with MQ since 2010.
Back to top
View user's profile Send private message
cra1gl1
PostPosted: Wed Jun 06, 2018 7:18 am    Post subject: Reply with quote

Apprentice

Joined: 25 Apr 2018
Posts: 30

with multiple tests with amgsget and amqsblst, I noticed that

the issue doesn't occur with NPMSPEED(NORMAL) or with DEFPSIST(YES)

it happens with NPMSPEED(FAST) or with DEFPSIST(NO), with amqsget and with amqsblst- but onnly when i have to close amqsblst -R (read) with ctrl+C


Does this mean the tools are getting the msgs even after exiting (with ctrl+c) and cluster is working fine?
or is there something else i need to worry about?
Back to top
View user's profile Send private message
Vitor
PostPosted: Wed Jun 06, 2018 7:53 am    Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

cra1gl1 wrote:
or is there something else i need to worry about?


DEFPSIST(NO) means messages added to the queue will not be persistent unless the putting application deliberately sets it (which the amqsput sample does not)

NPMSPEED(FAST) means increase the message transmission rate at the cost of message safety, and the description of this option states that non-persistent messages are at risk in the event of problems.

Using cntl+C is a forced (problematic) close of the application. Hence the software will examine it's options for cleaning up the problem, will discover that the configuration indicates the messages are not important, and take the easy route of flushing messages and starting clean.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
mvic
PostPosted: Wed Jun 06, 2018 7:56 am    Post subject: Reply with quote

Jedi

Joined: 09 Mar 2004
Posts: 2080

Hearing that the NPMSPEED and/or DEFPSIST setting is important, then the implication would be that the channel is losing a non-persistent message, but I find this surprising.
I would only expect a channel to lose a non-persistent message in channel abend scenarios, no matter the setting of NPMSPEED (maybe someone can correct me, though).
If you used -R -u 1 on amqsblst then a reading of the source code of that sample app suggests that MQGMO_SYNCPOINT would be used, hence no opportunity for the application message loss that I first referred to.
So, were you using -u 1 ? If you were, then that's the 2nd time I am surprised today.
Back to top
View user's profile Send private message
mvic
PostPosted: Wed Jun 06, 2018 8:02 am    Post subject: Reply with quote

Jedi

Joined: 09 Mar 2004
Posts: 2080

Vitor wrote:
Using cntl+C is a forced (problematic) close of the application. Hence the software will examine it's options for cleaning up the problem, will discover that the configuration indicates the messages are not important, and take the easy route of flushing messages and starting clean.

If the MQGET is done within syncpoint then you can "safely" abend the app without risk of losing a message inflight between qmgr and app.
The same thing affects NonPersistent and Persistent messages, though.
Conclusion: use syncpoint if your app ever abends (which most apps do) and you care about your messages. Or, another alternative is syncpoint-if-persistent if the app is handling both types, and wants to handle NonPersistent msgs faster with lower QOS.
Back to top
View user's profile Send private message
tczielke
PostPosted: Wed Jun 06, 2018 8:45 am    Post subject: Reply with quote

Guardian

Joined: 08 Jul 2010
Posts: 939
Location: Illinois, USA

When I saw this post, I had a case of deja vu. I remember seeing the same issue that cra1gl1 is reporting, when also doing just simple cluster PUT tests a few years ago. NPMSPEED(FAST) was dropping non-persistent messages for even these simple test cases with amqsput. There probably is a "bug" that NPMSPEED(FAST) would do this for such a trivial test case, but it is working as designed, according to the specifications of NPMSPEED(FAST).

Personally, I plan on changing all our cluster receiver channels to run with the default of NPMSPEED(NORMAL). I prefer that as the default, as quality of service should trump speed for a default, in my opinion. We also don't have speed requirements that would warrant this trade off.
_________________
Working with MQ since 2010.
Back to top
View user's profile Send private message
cra1gl1
PostPosted: Wed Jun 06, 2018 9:06 am    Post subject: Reply with quote

Apprentice

Joined: 25 Apr 2018
Posts: 30

Thanks for your time guys.

@mvic, you're right. The issue doesn't occur with "-R -u 1" (I missed -u 1 in earlier tests.)

CONCLUSION:
The issue doesn't occur with NPMSPEED(NORMAL) or with DEFPSIST(YES) (no matter what tools i use to GET the msgs)
Issue occurs with NPMSPEED(FAST) or with DEFPSIST(NO) when the getting app abends( amqsget).
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Goto page 1, 2  Next Page 1 of 2

MQSeries.net Forum Index » Clustering » Messages missing intermittently in cluster
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.