| Author |
Message
|
| DeadHead |
 Posted: Tue Oct 10, 2006 1:47 pm Post subject: MQ Commit question |
 |
|
Novice
Joined: 28 Sep 2006
Posts: 12
|
'lo all,
I'm new at Java & MQ and this place helped me to get my first Java/MQ program running...thanks to all. I have a question however in regards to the MQ commit command. I'm reading an ISeries MQ queue from a Java program located on another ISeries box as a MQ client. It's writing to a DDM DataQ located on yet another ISeries box. I do my MQ commit after reading the MQ and writing to the DataQ. Everything is happy as far as I know. Is this correct? Is this commit needed just because of the one read? I mean, the ISeries DataQ doesn't know what a "Commit" is. This leads to another question...If you are reading from multiple MQ queues and writing to multiple other MQ queues, do you need a commit for each separate write or will one suffice after the end of the logic cycle? Thanks in advance for any help or direction. 
_________________
"One man gathers what another man spills" GD |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Tue Oct 10, 2006 2:54 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
You can have multiple messages in one commit... but remember the size of the logs and the size of the messages and the max uncommitted messages and the fact that you are probably not the only user of the qmgr.... So yes there are limits and reasons.
I would suggest using the commit to have restart recovery points and to have them fairly close (no more than 1000) in one go...
If the messages are large a UOW size of 1 message may be totally justified...
Enjoy 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Tue Oct 10, 2006 3:18 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
If performance of UOW size=1 has been proven to be a performance bottleneck, then I'd consider increasing it.
Otherwise, I prefer to keep units of work as logically connected as possible, and except in cases of grouping or message aggregation/segregation, that should be 1 message = one logical action = 1 UOW.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Tue Oct 10, 2006 3:37 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Jeff I agree with you 100% in a discreet message environment. My understanding is, he is talking mainframe/AS400 and my assumption however misplaced it may be, was that he was trying to do some batch processing here...
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Tue Oct 10, 2006 5:51 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
well, I do iSeries even less than I do zSeries... So I'll take your word for it if it makes sense to do commits in larger batches.
On the other hand, I've had to put in hacked-up throttling mechanisms to deal with channels suddenly popping 10,000 messages onto a distributed processor input queue because someone's committed their batch all at once...
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| DeadHead |
| Posted: Wed Oct 11, 2006 5:51 am Post subject: MQ Commits... |
|
|
Novice
Joined: 28 Sep 2006
Posts: 12
|
'lo again dudes...
first off, thanks for the advice. I think you've answered my question plus I've learned the term UOW which is very descriptive on what I was asking about. My pgm is a "N"ever "E"nding "P"program (NEP) which will be constantly running on an AS400 monitoring an MQ Q and writing to a DataQ on another AS400. It's just passing along information, but it's business critical information and will usually handle a heavy volume. My initial question was just wondering how COMMITS could be set up when only reading and not writing to other MQ Q's plus the efficiency involved for a heavy volume. Can you be more explicit on how to tune a READ/COMMIT for heavy volume? The record length I'm reading is 125 bytes long. I'm not parsing it out because the receiving DATAQ is 125 also, so I just READ/WRITE, etc.... Thanks again, this is an AWESOME MQ site for learning.... 
_________________
"One man gathers what another man spills" GD |
|
| Back to top |
|
|
| JLRowe |
| Posted: Wed Oct 11, 2006 6:43 am Post subject: |
|
|
Yatiri
Joined: 25 May 2002
Posts: 664
Location: South East London
|
Every time you do a commit, a certain amount of synchronous (blocking) IO occurs that hardens the transaction to disk. The idea being that if the system fails just after the commit completes then everything is updated on disk and the system will be in the correct post-commit state.
You can do a commit after every message, or after a number of messages to improve performance - this is called batching or blocking. All the messages are dequeued during a single commit which gives better performance. But, there are other concerns such as database locks (two messages try to update the same row - the first message has it locked), or timeouts where perhaps you are waiting for 10 messages before the commit, but only 9 arrive - you need to timeout and commit.
Also, you should commit MQ *after* you write to the dataqueue, if the dataqueue write fails then you can rollback the MQ message and try again later with the same MQ message. If you commit before writing to the dataqueue, and the write fails, it is too late to rollback MQ and the message has been effectively lost. |
|
| Back to top |
|
|
|
|
