| Author |
Message
|
| avla |
 Posted: Tue May 09, 2006 6:27 am Post subject: Topic performance and unsubscribe durable subscriptions |
 |
|
Newbie
Joined: 09 May 2006
Posts: 4
|
I am finding that large volume through a Topic is causing severe performance degradation, often with messages being delivered to subscribers with an hour in between, but only after 50-60% of the message flow has been processed. My first question is has anyone else seen this issue?
Second is that I am theorizing that part of my problem might be a durable subscription being left out there although the subscriber is no longer active. However, I seem to run into the same issue that some other people have expressed regarding not being able to get dumpbroker and unsubscribe examples from the IBM site to successfully kill this subscription, getting a NOT_REGISTERED error.
Has anyone got this to work successfully? Is there something else besides the standard example parameters that must be passed when the subscription has a clientId assigned or MQREGO_VARIABLE_USER_ID set? (This last one is shown on amqspsd but not dumpbroker)
Running 530.10 CSD10.
Any help would be appreciated.
Ouput from Dumpbroker--->
IY73579 - fix installed
IY72849 - fix installed
Publishers
--------------
Stream Name: SYSTEM.BROKER.PUBLISH.PTSREPLY.TOPIC.STREAM
Subscribers
--------------
Stream Name: SYSTEM.BROKER.PUBLISH.PTSREPLY.TOPIC.STREAM
Topic: postingReplyTopic
BrokerCount: 0
ApplCount: 1
AnonymousCount: 0
RegistrationQName: SYSTEM.JMS.D.PTSREPLY.SUB.2.LQ
RegistrationCorrellId: 414D51204E59545343534D3120202020443D2AC22000AC07
RegistrationUserIdentifier: a_was1
RegistrationOptions: 1048577 : MQREGO_CORREL_ID_AS_IDENTITY
RegistrationTime: 2006041218154788
Topic: MQ/S/NYTSCSM1 /Subscribers/Identities/*
BrokerCount: 0
ApplCount: 1
AnonymousCount: 1
Ouput from unsubscribe--->
Reading deregister data from file: ./subsc.out
IY73579 - fix installed
Deregister topic: postingReplyTopic
Deregister stream: SYSTEM.BROKER.PUBLISH.PTSREPLY.TOPIC.STREAM
Deregister queue: SYSTEM.JMS.D.PTSREPLY.SUB.2.LQ
Deregister correlid: AMQ NYTSCSM1 D=*? ?
Unsub string is MQPSCommand DeregSub MQPSTopic postingReplyTopic MQPSRegOpts CorrelAsId MQPSStreamName SYSTEM.BROKER.PUBLISH.PTSREPLY.TOPIC.STREAM MQPSQName SYSTEM.JMS.D.PTSREPLY.SUB.2.LQ MQPSQMgrName "NYTSCSM1 "
IY72849 - fix installed
Unsubscribe failed for reason NOT_REGISTERED
1 Deregistrations attempted. 0 successfully removed. |
|
| Back to top |
|
 |
| jefflowrey |
| Posted: Tue May 09, 2006 6:32 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
dumpbroker is deprecated for amqspsd, which is a sample program.
What's the qdepth of SYSTEM.BROKER.PUBLISH.PTSREPLY.TOPIC.STREAM?
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| avla |
| Posted: Tue May 09, 2006 6:37 am Post subject: |
|
|
Newbie
Joined: 09 May 2006
Posts: 4
|
| qdepth is zero for that queue. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Tue May 09, 2006 7:52 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Oh, sorry. I meant for SYSTEM.JMS.D.PTSREPLY.SUB.2.LQ
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| avla |
| Posted: Tue May 09, 2006 7:59 am Post subject: |
|
|
Newbie
Joined: 09 May 2006
Posts: 4
|
| When trying to do the unsubscribe, it has about 8500 in it. I have noticed it will have somewhere between a few thousand to 15k when the performance hits a wall. I am thinking these are my extra messages hanging around for that subscriber. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Tue May 09, 2006 8:29 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Yes, they probably are.
So first use amqsbcg to verify the CorrelID of all the messages in that queue.
Then clear the queue of anything you don't need.
Then try the unregister.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| avla |
| Posted: Tue May 09, 2006 4:22 pm Post subject: |
|
|
Newbie
Joined: 09 May 2006
Posts: 4
|
Thanks for the help Jeff. I got the unsubscribe to work but it doesn't seem to have solved my performance issues much.
Any ideas about what might cause a severe lag in message delivery after a large amount of messages and/or a large amount of messages on the topic queue? We have 6 subscribers with varying message selectors on the same topic.
I am pretty convinced it is a tuning issue of some sort but my boss is ready to pull the plug and write our own routing mechanism. Are there any good basic tuning measures that might help? Is it better to have each subscriber with their own durable queue rather than have them all share the one? Is there a way to increase the memory allocated to the broker?
Andrew |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Tue May 09, 2006 4:34 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
If there are messages on the queue, it takes longer to identify each subscription.
If there are uncommited messages on the queue, it takes longer to process each new message being added to an existing transaction.
That is, it's actually likely an application issue and not a tuning issue.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Tue May 09, 2006 4:36 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Oh. It's definitely better to have each subscriber with their own queue.
Which broker are you using?
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| briancop |
| Posted: Wed May 10, 2006 4:29 am Post subject: |
|
|
Apprentice
Joined: 04 Apr 2005
Posts: 28
Location: Hursley, UK
|
And what do the selectors look like? Unless they are simple selection criteria using MsgId and CorrelId, then the JMS implementation will be matching messages to the selectors within the client process, i.e. browsing the queue, looking for messages of interest and getting these. If you have a relatively deep queue, this will not be efficient.
Is it possible that you can do something slightly different that doesn't require selectors? Regardless of how the JMS implementation works, selectors are always going to be inherently inefficient by their nature of having to parse message headers and properties and do string comparisons against the parsed selector strings.
Perhaps you can do something with topic hierarchies such that you can be more specific in your publishing application(s) about which subscribers the messages get routed to?
Or if you are using Message Broker, perhaps you can set up a flow such that you publish to a master topic, pass the messages through one or more filter nodes (including content-based filtering if necessary) and route them on the basis of the filters to further topics to which your consuming applications subscribe.
I don't know for sure, but I would guess that a lot of your performance issues may be a result of your use of selector strings. At least that is where I would start looking for answers. |
|
| Back to top |
|
|
| briancop |
| Posted: Wed May 10, 2006 4:30 am Post subject: |
|
|
Apprentice
Joined: 04 Apr 2005
Posts: 28
Location: Hursley, UK
|
| Oh, by the way, Jeff's comment about using a separate queue for each subscriber is even more important when using selectors, since it will greatly reduce the queue depths involved. |
|
| Back to top |
|
|
|
|
