| Author |
Message
|
| J C Hill |
 Posted: Thu Jan 26, 2006 4:15 am Post subject: How is FIFO decided when all messages have same Priority? |
 |
|
Newbie
Joined: 26 Jan 2006
Posts: 5
|
Could someone please explain how: -
When a queue is configured with a 'Message Delivery Sequence' of 'Priority' and messages have same 'Priority' how is FIFO established/calculated?
The reason I ask is that I've recently had a problem, similar to 'Messages delivered out of sync' problem from member 'thindk00' where messages are delivered in sequence, but are read out of sequence.
The scenario I have is WebSphere Broker v5 extracting tens of thousands of messages (from DB2) using an ORDER BY clause to establish sequence.
The messages are instantly dropped onto a Remote Queue, which is read by IBM's JText adaptor, and creates a flat file of the DB2 data.
99.99% of the time this works perfectly and the sequence is correct.
However, in rare situations, the sequence gets muddled for 1-2 records.
Because the 'Priority' value is actually the same for all messages, I've fixed the problem by changing 'Message Delivery Sequence' to FIFO.
But, in theory, I shouldn't need to do this.
Is 'FIFO' and 'Priority -> FIFO' different?
Cheers
J.
p.s. This is not a clustered/load balanced environment.
p.p.s The 'NumConcurrentRequests' on the adapter is set to '1'. |
|
| Back to top |
|
 |
| csmith28 |
| Posted: Thu Jan 26, 2006 7:41 am Post subject: |
|
|
Grand Master
Joined: 15 Jul 2003
Posts: 1196
Location: Arizona
|
You mena FIFO as in first in first out right?
_________________
Yes, I am an agent of Satan but my duties are largely ceremonial. |
|
| Back to top |
|
|
| J C Hill |
| Posted: Thu Jan 26, 2006 7:47 am Post subject: |
|
|
Newbie
Joined: 26 Jan 2006
Posts: 5
|
|
| Back to top |
|
|
| JT |
| Posted: Thu Jan 26, 2006 8:01 pm Post subject: |
|
|
Padawan
Joined: 27 Mar 2003
Posts: 1564
Location: Hartford, CT.
|
How do you know the messages were delivered in sequence ?
Can you confirm that they arrived on the queue in sequence ? |
|
| Back to top |
|
|
| J C Hill |
| Posted: Fri Jan 27, 2006 2:11 am Post subject: |
|
|
Newbie
Joined: 26 Jan 2006
Posts: 5
|
Within each message on the queue, which is exported to the flat file via JText, there's a sequence number field (which is 'ordered by' when I extract the data from DB2).
I'm as sure as I can be that this is sound (unless there's some unknown problem with the ORDER BY clause in DB2....which I doubt) and each record is delivered to the remote queue in this order.
As for arriving on the JText queue in sequence, I believe this is the problem and that it doesn't...but the only evidence I really have is the file that JText builds (due to the large volumes of data involved and the rarity of the problem) which can be out of sequence on rare occasions.
What appears to be happening is that, when 'Message Delivery Sequence' is FIFO, the message order is maintained by a method such as an MQ sequence counter (that's a guess).
However, if FIFO is used because 'Message Delivery Sequence' is Priority, but all messages have same priority, then a different method of maintaining message order is used such as Date/Time stamp (again a guess).
The only evidence I have of the problem is that 'Message Delivery Sequence' FIFO works fine...but FIFO when Priority is the same doesn't appear to be 100% reliable. |
|
| Back to top |
|
|
| wschutz |
| Posted: Fri Jan 27, 2006 6:53 am Post subject: |
|
|
Jedi Knight
Joined: 02 Jun 2005
Posts: 3316
Location: IBM (retired)
|
MQ has twenty "sub-queues", one for each priority (0-9) and one set for persistent and one set for non-persistent messages.
When you specify MSGDLVSQ(FIFO), the "default priority" of the queue is used to determine which sub-queue the message will go into. The message goes to the end of the sub-queue.
When you specify MSGDLVSQ(PRIORITY), the priority on the MQMD is used to determine which sub-queue the message will go into. The message goes to the end of the sub-queue.
Of course, MQPUT syncpoint considerations and uncommitted MQGETs by other applications could make the messages appear out of order ....
So, in theory, it shouldn't make a difference.
What is the platform and the MQ levels / CSDs?
_________________
-wayne |
|
| Back to top |
|
|
| J C Hill |
| Posted: Mon Jan 30, 2006 2:20 am Post subject: |
|
|
Newbie
Joined: 26 Jan 2006
Posts: 5
|
Cheers Wayne.
Can we pick this up again w/c 6th February?
I'm not on site at the moment so can't provide accurate version details.
Cheers
J. |
|
| Back to top |
|
|
| Cliff |
| Posted: Mon Jan 30, 2006 9:17 am Post subject: |
|
|
Centurion
Joined: 27 Jun 2001
Posts: 145
Location: Wiltshire
|
The answer to your question is in your post - "The messages are instantly dropped onto a Remote Queue".
MQ is an asynchronous protocol - for message sequence to be guaranteed there are very specific criteria to be met. From memory these include: all messages put in a single unit of work, target queue to be local, only one instance of the putting program. As soon as you deliver messages remotely, there is scope for MQ (quite legitimately) to get things out of sequence. There might be a temporary network glitch, for example, resulting in messages being rolled back onto the xmitq and then sent again.
If sequence must be maintained for your application to work, you will have to design a programmatic mechanism to handle this.
HTH -
Cliff |
|
| Back to top |
|
|
| J C Hill |
| Posted: Wed Feb 08, 2006 1:53 am Post subject: |
|
|
Newbie
Joined: 26 Jan 2006
Posts: 5
|
Hi Cliff
I completely agree that the transmission of messages to remote queues has the potential for the problems you've described, as does multiple instances of a 'put' or 'get' program. However, all things being equal, the problem has cured itself by changing the order from 'Priority' to 'FIFO'...which is the reason for my original question regarding if there's a difference.
Cheers
J. |
|
| Back to top |
|
|
|
|
