| Author |
Message
|
| jefflowrey |
 Posted: Tue Dec 19, 2006 4:35 am Post subject: |
 |
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
If this is not a technical requirement, then you should not be trying to solve it technically. The requirement is *not* that you read the queue in LIFO order.
The requirement seems to be that a) every application is notified of the most recent "event" (this is pub/sub), b) every application can decide if it needs to know anything other than the most recent event.
I would have the sender keep a datastore, and initiate a request/reply process from the readers. The sender sends the most recent event, and sticks it into a datastore of some kind (this can be a queue if you really want to make yourself crazy). The reader application, when it decides that it needs to know something other than the "most recent event", can send a request for the backlog that it needs to know to the sender. The sender will then send back all messages in that backlog.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
 |
| Vitor |
| Posted: Tue Dec 19, 2006 4:53 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| jefflowrey wrote: |
| I would have the sender keep a datastore |
| GLudgate wrote: |
| I cannot read the messages into a central database as all inter-application messaging is via MQ. |
If GLudgate can store the data, would it not make more sense to dequeue it at the receiving end? Otherwise you'll get a network pulse every time someone requests the backlog?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Tue Dec 19, 2006 4:57 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
And?
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Dec 19, 2006 5:09 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
And maybe nothing. Depends on your architecture. 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| xxx |
| Posted: Tue Dec 19, 2006 10:56 am Post subject: |
|
|
Centurion
Joined: 13 Oct 2003
Posts: 137
|
If you need only the last message , send it as a priority message and rest as non priority , Trigger the Queue on Priority message,
This will pick the Last message First ( as it is set to priority ) and rest FIFO, |
|
| Back to top |
|
|
| elvis_gn |
| Posted: Tue Dec 19, 2006 11:06 am Post subject: |
|
|
Padawan
Joined: 08 Oct 2004
Posts: 1905
Location: Dubai
|
Hi,
| xxx wrote: |
If you need only the last message , send it as a priority message and rest as non priority , Trigger the Queue on Priority message,
This will pick the Last message First ( as it is set to priority ) and rest FIFO, |
And when the next "latest message arrives" which priority message will be picked 
| GLudgate wrote: |
| I cannot read the messages into a central database as all inter-application messaging is via MQ. |
All inter application messaging should be on MQ not the data storage....
Jeff's post on pub-sub seems like a great idea...
Other than that, I would only suggest a database solution....and if this is a "small" requirement, then I don't see why ur company is so worried about its implications...
Regards. |
|
| Back to top |
|
|
| tleichen |
| Posted: Tue Dec 19, 2006 12:20 pm Post subject: |
|
|
Yatiri
Joined: 11 Apr 2005
Posts: 663
Location: Center of the USA
|
| xxx wrote: |
If you need only the last message , send it as a priority message and rest as non priority , Trigger the Queue on Priority message,
This will pick the Last message First ( as it is set to priority ) and rest FIFO, |
That won't work, because his putting application does not know ahead of time which message is the "last one". Besides, this is a relative term, as it keeps changing as more records keep coming. 
_________________
IBM Certified MQSeries Specialist
IBM Certified MQSeries Developer |
|
| Back to top |
|
|
| GLudgate |
| Posted: Wed Dec 20, 2006 1:26 am Post subject: |
|
|
Newbie
Joined: 18 Dec 2006
Posts: 6
Location: Vancouver BC
|
Sorry for the delay.
fjp_saper - good idea. Gets me to the most current few messages (good enough to be considered equivalent for processing purposes). But does not solve the problem of quickly processing ALL other messages in LIFO order. It is however, a good start.
jefflowrey - it is a business requirement and an automated solution was also specified by the business as humans are too slow. So a technical solution is needed. The requirement is that the most recent message must be process first. Which application processes it does not matter. Any application becoming free MUST therefore take the most recent message to process ie LIFO order. Currently they are processed in FIFO order. But the business realized that the most current message was of greatest value and had to be done before all others.
Vitor - there are many receivers spread over a very wide area, not one.
elvis_gn - The requirement of changing "FIFO" to "LIFO" was seen as a "small" change as only 2 letters changed in the requirement sentence. Other, Java based applications in a related system, made this change over-night and are now running. This system is MQ based and does not seem to be able to change at all. The implications of processing the most current message first are huge and the gain very substantial.
All - thanks for your help. I do appreciate it. Let us not flog this topic to death. I will have to think harder on how to simulate LIFO reads. I will check back if I find a "close" solution.
Bye Bye.
_________________
George Ludgate |
|
| Back to top |
|
|
| EmirGarza |
| Posted: Wed Dec 20, 2006 2:57 am Post subject: |
|
|
Newbie
Joined: 19 Dec 2006
Posts: 1
|
The following may do the trick:
1. Put the messages with an expiry interval, and set "report on expiration" so that it puts the whole original message on the reply-to-queue (MQRO_EXPIRATION_WITH_FULL_DATA).
2. Have a utility program that gets the expired messages from the reply-to-queue, and puts them back to the original queue slowly.
Messages that stay on the queue for too long expire and are moved away, so that they don't obstruct new messages. But they are not lost; they trickle back to the queue (slowly enough not to interfere with new messages).
_________________
Emir Garza, Hursley Lab Services |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Wed Dec 20, 2006 5:54 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
GLudgate - You are desiging yourself into a big deep hole that leads to performance pain, application problems, and problem identification disaster.
First off, sharing one queue between many different applications is not a great idea.
Second off, leaving messages on a queue is not a great idea - in particular because it makes it difficult to get the "newest" message.
At a MINIMUM, you should redesign your applications so that they DO NOT leave messages on this queue. The only message that should ever be sitting on it is the newest message. Then implement SOME kind of "catchup" mechanism, for those applications that need to deal with slightly older data.
If you do not do these things, you are always going to be suffering from unnecessary performance hits as your M applications all try to wade through the N messages on the queue (causing N*M gets every interval, which will impact the performance of your queue manager). And if at some point, an application process what it THINKS is the "newest" message, but actually isn't, you won't know why.
I repeat, you are heading for disaster.
You do NOT need to read the queue in LIFO order. Stop trying to solve the problem this way.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| markt |
| Posted: Wed Dec 20, 2006 5:59 am Post subject: |
|
|
Knight
Joined: 14 May 2002
Posts: 508
|
| look at pub/sub RETAINED publications - always hold the most recent info on a topic |
|
| Back to top |
|
|
| rtsujimoto |
| Posted: Wed Dec 20, 2006 9:29 am Post subject: |
|
|
Centurion
Joined: 16 Jun 2004
Posts: 119
Location: Lake Success, NY
|
| What platform are you using? |
|
| Back to top |
|
|
| tleichen |
| Posted: Wed Dec 20, 2006 11:55 am Post subject: |
|
|
Yatiri
Joined: 11 Apr 2005
Posts: 663
Location: Center of the USA
|
| GLudgate wrote: |
...The requirement of changing "FIFO" to "LIFO" was seen as a "small" change as only 2 letters changed in the requirement sentence...
|
I do hope he's joking with that remark! 
| markt wrote: |
| look at pub/sub RETAINED publications - always hold the most recent info on a topic |
I thought of this too. It would seem a viable solution, if they would stop going against the grain.
_________________
IBM Certified MQSeries Specialist
IBM Certified MQSeries Developer |
|
| Back to top |
|
|
| mvarghese |
| Posted: Thu Dec 28, 2006 6:44 pm Post subject: LIFO--problem |
|
|
Centurion
Joined: 27 Sep 2006
Posts: 141
|
Is possible an application can read based on the MQPUT time...so application can search the MQput time and then do a search which is latest PUT message based on current time...so read that message MSGID and pick that message..am not sure wheather it will work in practical cases...may be am throwing wrong info:)
_________________
Jain Varghese |
|
| Back to top |
|
|
| pathipati |
| Posted: Thu Dec 28, 2006 6:52 pm Post subject: |
|
|
Master
Joined: 03 Mar 2006
Posts: 296
|
|
| Back to top |
|
|
|
|
