| Author |
Message
|
| hughson |
 Posted: Tue Apr 12, 2022 5:44 pm Post subject: Determining longest running transaction in seconds |
 |
|
Padawan
Joined: 09 May 2013
Posts: 1914
Location: Bay of Plenty, New Zealand
|
I have been asked (on twitter!) this question which my MQ expertise does not extend to.
They appear to be trying to size log files with a formula 2* (N+1)* B where B is "number of bytes they will send per second in peak time" and N is "longest running transaction in seconds". They have managed to get their application team to tell them B but they can't get N from their application team.
I believe this question is about a distributed platform queue manager.
I naively imagine that this might be something you can monitor MQ for, but when I looked through all the MQ Accounting and Statistics details I don't see any details about the length of transactions. You can get connection durations out of these but not transaction duration. You can get a count of commits for the connection, and so could get an average length of transaction that way, but that would presumably include the time the application sat in a get-wait until a message first showed up and that would skew the transaction length to be much longer than it really is.
So, how does one monitor an MQ system to determine the longest running transaction?
Cheers,
Morag
_________________
Morag Hughson @MoragHughson
IBM MQ Technical Education Specialist
Get your IBM MQ training here!
MQGem Software |
|
| Back to top |
|
 |
| bruce2359 |
| Posted: Tue Apr 12, 2022 7:38 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9397
Location: US: west coast, almost. Otherwise, enroute.
|
Some initial thoughts...
If this was z/os, I'd send you to SMF 115 and 116 records, but ...
So, you are pondering that message arrival will be unpredictable.
I'd suggest modeling what you can - a transaction with a message already in queue - no wait - as start time, then capture the end time. MQ trace should provide time-stamps. You can display log growth RBA from log time stamps during the transaction. MQGET wait time uses no log resources.
If msg arrival is really unpredictable, SLA's should reflect that, and stated in terms like 80% of transactions less than 2 seconds, 20% less than 6 seconds - something like that.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| markt |
| Posted: Wed Apr 13, 2022 12:45 am Post subject: |
|
|
Knight
Joined: 14 May 2002
Posts: 504
|
DIS CONN shows two values for a UOW - the "start" time and the "first log write" time to assist in distinguishing a long-running MQGET-WAIT. But of course that doesn't give historic information and you'd have to poll (possibly rapidly) to get meaningful data.
This sounds like a good opportunity for activity trace analysis. You would collect the data in a test environment to get the numbers but not need that trace running in prod. |
|
| Back to top |
|
|
| hughson |
| Posted: Wed Apr 13, 2022 3:11 am Post subject: |
|
|
Padawan
Joined: 09 May 2013
Posts: 1914
Location: Bay of Plenty, New Zealand
|
| markt wrote: |
| DIS CONN shows two values for a UOW - the "start" time and the "first log write" time to assist in distinguishing a long-running MQGET-WAIT. But of course that doesn't give historic information and you'd have to poll (possibly rapidly) to get meaningful data. |
Yeah - I know about the two UOW date/times in DISPLAY CONN. Summarised it here. I didn't consider hammering the command server as a good way to "monitor" for this. Hoping for something more historic.
| markt wrote: |
| This sounds like a good opportunity for activity trace analysis. You would collect the data in a test environment to get the numbers but not need that trace running in prod. |
This might turn out to be the only way. Perhaps I should raise an RFE to request some more numbers in Acct&Stat?
_________________
Morag Hughson @MoragHughson
IBM MQ Technical Education Specialist
Get your IBM MQ training here!
MQGem Software |
|
| Back to top |
|
|
| Andyh |
| Posted: Wed Apr 13, 2022 8:21 am Post subject: |
|
|
Master
Joined: 29 Jul 2010
Posts: 237
|
The queue manager doesn't care how long (in time) a transaction runs for, and the time N (in the equation below) can simply be the longest acceptable time from a customer perspective. If the some transaction exceeds this duration, at a time of peak log throughput, and thus causes the log to approach capacity then this transaction will be rolled back and the corresponding app will be notified appropriately.
In the event that a transaction is rolled back in this manner then the queue manager will write messages to AMQERR01.LOG to help the customer identify the offending transaction, and the customer action recommended at this point would typically be to fix the application, rather than to increase the size of the recovery log (the opposite of what usually happens in practice).
A modern distributed queue manager can publish stats showing the log usage, giving the customer an idea as to how much of the configured log space is being used. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Wed Apr 13, 2022 10:56 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9397
Location: US: west coast, almost. Otherwise, enroute.
|
Disk space is cheap and plentiful - use linear logs. Exceeding log capacity - must be referring to circular logs - where it is possible to exceed log capacity and force transaction rollbacks.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Apr 14, 2022 1:01 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY
|
| bruce2359 wrote: |
| Disk space is cheap and plentiful - use linear logs. Exceeding log capacity - must be referring to circular logs - where it is possible to exceed log capacity and force transaction rollbacks. |
Wrong!. If your transaction spans the defined log space (primary + secondary) in linear logs it can also be rolled back... By definition that would be a long running transaction... 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Thu Apr 14, 2022 3:23 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9397
Location: US: west coast, almost. Otherwise, enroute.
|
I had intended to refer to exceeding file system capacity, the usual warning about circular log limitation. Mea culpa.
Back to an 80-20 type SLA. An errant (read: badly-designed and coded) endlessly looping transaction would yield a “Longest-running transaction” vs. “long-running transaction” of a set of transactions.
One of my clients misunderstood the requirement to update a 330,000 record database as a single transaction (Unit of Work, syncpoint), and soon (and predictably) filled all primary and secondary logs - should have been 330,000 individual transactions. I didn’t include this type of errant transaction in “how to calculate” discussion.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| gbaddeley |
| Posted: Mon Apr 18, 2022 4:12 pm Post subject: |
|
|
Jedi
Joined: 25 Mar 2003
Posts: 2494
Location: Melbourne, Australia
|
It can be hard to predict log usage without intimate knowledge of transaction behavior and edge cases on run times. We size all our production queue managers the same and then observe the write time stamps on the log files to see how long it takes for them to roll over, and if any secondary logs are being used. Anything less than 10 - 30 minutes is probably cause for concern. Some queue managers take many days. Some churn through 500GB many times a day.
_________________
Glenn |
|
| Back to top |
|
|
|
|
