| Author |
Message
|
| eputsato |
 Posted: Thu Apr 16, 2009 12:45 am Post subject: |
 |
|
Novice
Joined: 14 Apr 2009
Posts: 20
|
| sumit wrote: |
Follow this scenario- you have 2 local queues ql1 and ql2 at subscriber's end. The end site application will interact with the ql2 only. ql1 is responsible to receive messages from publisher.
May be you can write a code for this or use standard utility (let's call it put-get utility). The way how it can work is:
1. A message comes to ql1 and triggeres put-get utility.
2. put-get checks if a message is there in ql2.
3. If NO, get the message from ql1 and put the same in ql2.
4. If YES, delete the message from ql2.
5. then, get message from ql1 and put the same in ql2.
In this way, you can always get the latest message for your end side users. |
Which looks like rewriting the whole PubSub pattern...
Where can I write this code managing q1 and q2? Server side? Client side?
It's so ridiculous that one has to bother doing this, when using the most well known MQ in the world... |
|
| Back to top |
|
 |
| eputsato |
| Posted: Thu Apr 16, 2009 12:48 am Post subject: |
|
|
Novice
Joined: 14 Apr 2009
Posts: 20
|
| mqjeff wrote: |
To reiterate the scenario if you enable expiry, here's what will happen.
You publish message A, B, and C.
A slow subscriber reads A and starts to process.
Time passes.
You publish message D,E, F and G.
Time passes.
Message B, C, D and E expire. The queue now has only F and G on it.
The subscriber finishes processing A and goes to get the next message. It's F. |
If i put expiry equal to 1 ms, then after subscriber connects he gets MQRC_EXPIRY_ERROR |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Apr 16, 2009 1:01 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| eputsato wrote: |
| Which looks like rewriting the whole PubSub pattern... |
It is a rather odd solution, and not one I'd have suggested.
| eputsato wrote: |
| It's so ridiculous that one has to bother doing this, when using the most well known MQ in the world... |
It's a miracle the rest of us manage with it.
One possible solution (which I've not tested or really worked through in my head) is for the subscribing app to read it's subscription queue until it's empty or until the put time of the message is within a tollerance of it's deal time. In this way you get the most recent or the most appropriate state.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| WMBDEV1 |
| Posted: Thu Apr 16, 2009 1:08 am Post subject: |
|
|
Sentinel
Joined: 05 Mar 2009
Posts: 888
Location: UK
|
| Vitor wrote: |
One possible solution (which I've not tested or really worked through in my head) is for the subscribing app to read it's subscription queue until it's empty or until the put time of the message is within a tollerance of it's deal time. In this way you get the most recent or the most appropriate state. |
I also thought of a solution along these lines. the spanner in the works that I could see is if the different messages contain prices for different stocks. If this was the case you would still need to inspect the message to see which stocks were in the message and so you may be back to square 1. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Apr 16, 2009 1:10 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Okay.
If you really want to skew the proper functioning of pub/sub such that at every point that your application gets a subscription, the only one it has available to get is "the last and only last" message, then you can do this:
1) Use retained publications
2) DO NOT use durable subscriptions
3) DO NOT leave your subscribers connected or registered for subscriptions
4) EVERY TIME that they are ready to get a subscription, they should REOPEN the subscription and then CLOSE IT ENTIRELY after ONE receive.
In this manner, you will meet your requirement.
Or you could do something sensible and have the subscriber app add additional logic to only process the message it received just before it gets a 2033. Then every time it finishes processing a message, it will consume all the queue without processing most of the messages. |
|
| Back to top |
|
|
| eputsato |
| Posted: Thu Apr 16, 2009 1:18 am Post subject: |
|
|
Novice
Joined: 14 Apr 2009
Posts: 20
|
| Vitor wrote: |
One possible solution (which I've not tested or really worked through in my head) is for the subscribing app to read it's subscription queue until it's empty or until the put time of the message is within a tollerance of it's deal time. In this way you get the most recent or the most appropriate state. |
But if the subscriber is slow - it takes him ages to receive all the 100 messages pending |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Apr 16, 2009 1:19 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| eputsato wrote: |
| But if the subscriber is slow - it takes him ages to receive all the 100 messages pending |
Does it take ages for the subscriber to READ 100 messages pending, or does it take ages for the subscriber to PROCESS 100 messages pending.
MQGET operations tend to be very fast. Doing anything with the message thus read tends to be slower. |
|
| Back to top |
|
|
| eputsato |
| Posted: Thu Apr 16, 2009 1:21 am Post subject: |
|
|
Novice
Joined: 14 Apr 2009
Posts: 20
|
| mqjeff wrote: |
4) EVERY TIME that they are ready to get a subscription, they should REOPEN the subscription and then CLOSE IT ENTIRELY after ONE receive.
In this manner, you will meet your requirement.
|
Could you tell me, please, is it a heavy operation to reopen subscription?
And also even if there were no publications, every time i reopen sub i'll have to get all the retained messages.
| mqjeff wrote: |
Or you could do something sensible and have the subscriber app add additional logic to only process the message it received just before it gets a 2033. Then every time it finishes processing a message, it will consume all the queue without processing most of the messages. |
There's gonna be no heavy processing. It's mostly a matter of slow internet speed that makes the subscriber to be really slow |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Apr 16, 2009 1:25 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| eputsato wrote: |
| mqjeff wrote: |
4) EVERY TIME that they are ready to get a subscription, they should REOPEN the subscription and then CLOSE IT ENTIRELY after ONE receive.
In this manner, you will meet your requirement.
|
Could you tell me, please, is it a heavy operation to reopen subscription?
And also even if there were no publications, every time i reopen sub i'll have to get all the retained messages. |
Yes, it's heavy weight.
No, you won't get ALL retained messages. Retained publications only keeps the LAST one. As I said earlier, the difference between a retained publication and an unretained publication is what happens when a new subscriber registers.
| mqjeff wrote: |
Or you could do something sensible and have the subscriber app add additional logic to only process the message it received just before it gets a 2033. Then every time it finishes processing a message, it will consume all the queue without processing most of the messages. |
There's gonna be no heavy processing. It's mostly a matter of slow internet speed that makes the subscriber to be really slow[/quote]
The sensible approach here has always been to improve the performance of the subscribers. You keep not wanting to do that, so there we have been making slightly less sensible solutions. The more you reject these solutions, the more you work yourself into a corner where nothing sensible can be done. |
|
| Back to top |
|
|
| eputsato |
| Posted: Thu Apr 16, 2009 1:26 am Post subject: |
|
|
Novice
Joined: 14 Apr 2009
Posts: 20
|
| mqjeff wrote: |
Does it take ages for the subscriber to READ 100 messages pending, or does it take ages for the subscriber to PROCESS 100 messages pending.
MQGET operations tend to be very fast. Doing anything with the message thus read tends to be slower. |
It's all due to poor conenction, but not to the subsciber's speed.
And the idea is simply not to receive all those unwanted and uout-of-date pubications. I need to skip the on server, not on client. |
|
| Back to top |
|
|
| eputsato |
| Posted: Thu Apr 16, 2009 1:33 am Post subject: |
|
|
Novice
Joined: 14 Apr 2009
Posts: 20
|
| mqjeff wrote: |
No, you won't get ALL retained messages. Retained publications only keeps the LAST one. As I said earlier, the difference between a retained publication and an unretained publication is what happens when a new subscriber registers. |
I have 140 topics
| mqjeff wrote: |
The sensible approach here has always been to improve the performance of the subscribers. You keep not wanting to do that, so there we have been making slightly less sensible solutions. The more you reject these solutions, the more you work yourself into a corner where nothing sensible can be done. |
I'm not simply rejecting what you suggest. But this is a "non-systematic" approach to things. This is why windows requires nowadays 4Gigas of RAM instead of 128 megs.
Why to perform unnecessary processing if you can simply avoid this traffic (in this case) |
|
| Back to top |
|
|
| WMBDEV1 |
| Posted: Thu Apr 16, 2009 1:35 am Post subject: |
|
|
Sentinel
Joined: 05 Mar 2009
Posts: 888
Location: UK
|
| mqjeff wrote: |
The more you reject these solutions, the more you work yourself into a corner where nothing sensible can be done. |
and heres my latest wacky suggestion....
Create a new subscription point on the topic and have your publisher send one in ten (or whatever) messages to it. You will have to create some sort of state on the flow but I did say it was wacky!
The slow subscriber will then subscribe to this topic and sub point. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Apr 16, 2009 1:43 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
MQ based pub/sub does not work the way you would like it to.
I don't believe that *any* pub/sub works the way you want it to.
The closest way that I know of that you can get the behavior you want is to "continually" unregister and reregister your subscriptions and use retained publications. There are performance implications to doing this. You need to evaluate those against the performance implications you already have.
If you want to change your existing setup, you are going to have to make additional MQ calls to the server - either to make additional registrations, or to make inquiries to find out queue depth and then make additional MQ calls in response to that value. |
|
| Back to top |
|
|
| eputsato |
| Posted: Thu Apr 16, 2009 1:45 am Post subject: |
|
|
Novice
Joined: 14 Apr 2009
Posts: 20
|
| mqjeff wrote: |
MQ based pub/sub does not work the way you would like it to.
I don't believe that *any* pub/sub works the way you want it to.
The closest way that I know of that you can get the behavior you want is to "continually" unregister and reregister your subscriptions and use retained publications. There are performance implications to doing this. You need to evaluate those against the performance implications you already have.
If you want to change your existing setup, you are going to have to make additional MQ calls to the server - either to make additional registrations, or to make inquiries to find out queue depth and then make additional MQ calls in response to that value. |
Sun JMS does as I mentioned, but they don't have hierarchical topics and neither the retained publications |
|
| Back to top |
|
|
| eputsato |
| Posted: Thu Apr 16, 2009 1:49 am Post subject: |
|
|
Novice
Joined: 14 Apr 2009
Posts: 20
|
My idea is VERY simple:
1)I need the client to have the latest price at every moment even if he's slow. Which can be thought of as what he sees the next second the publisher stops the feed
2)Retained publication. So when he connects he gets the last value
3)Hierarchical topics
No ready-on JMS support all the three 8( |
|
| Back to top |
|
|
|
|
