| Author |
Message
|
| MQ281 |
 Posted: Tue Jun 15, 2010 9:24 pm Post subject: Number of Primary Vs Secondary Log files |
 |
|
Newbie
Joined: 05 Aug 2009
Posts: 6
|
Hi, the requirement is to have the maximum Log size allowed by MQ. On Unix systems, the maximum LogFilePages is 65535 and max total number of Primary and Secondary Log files is 511
Environment details: MQ 6.0.2.4 on Linux
In the current environment, LogFilePages could be 16834 or 65535(current value is 16384 and decision is not made whether this parameter is required to be changed).
What is the recommended split up for Primary and Secondary in this case and what are the performance consideration one should keep in mind? does the LogFilePages being 16834 or 65535 have any kind of impact? Also, we are talking of persisting huge data(max 127GB), is there any performance impact on the queue manager operation?
Your quick response would be highly appreciated.
Thanks,
Deepika |
|
| Back to top |
|
 |
| Mr Butcher |
| Posted: Tue Jun 15, 2010 11:01 pm Post subject: |
|
|
Padawan
Joined: 23 May 2005
Posts: 1716
|
if you use that "big" data then yes, you will have performance impacts (whatever that means in your case), no matter how you fiddle around with the logs.
MQ is not a database, maybe you should consider using one......
_________________
Regards, Butcher |
|
| Back to top |
|
|
| zpat |
| Posted: Tue Jun 15, 2010 11:19 pm Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
Are you looking at linear or circular logs?
You do realise that messages are stored on queues, as well as logs?
How much history do you want to keep? It might be better to use other means of keeping message data (e.g. a database using WMB or one of the commercial products that can keep an archive). |
|
| Back to top |
|
|
| MQ281 |
| Posted: Wed Jun 16, 2010 12:04 am Post subject: |
|
|
Newbie
Joined: 05 Aug 2009
Posts: 6
|
Hi zpat,
We are using circular logs.
Its not about history, we are expecting around 100GB data a day which would eventually be transferred to a Database. We are looking for log sizing of above 100GB incase there is a issue with DB/message flows/ application and all messages have to stay on a queue manager for say, 1 day or 2.
@Mr. Butcher:
By performance issue, I meant queue manager not responding i.e., channels retrying etc. Do you think few queues on a queue manager holding 100GB data cumulatively would impact queue manager operation?
Thanks,
mq281 |
|
| Back to top |
|
|
| zpat |
| Posted: Wed Jun 16, 2010 12:11 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
OK - circular logs only need to hold messages which are currently in an uncommitted unit of work.
This should not need to be a very large amount of disk space unless your applications are not committing their queue updates in a timely manner.
I think you are confusing queue storage with log usage. Your messages will be on queues until they are processed. Size your queue space accordingly.
Persistent message performance is largely limited by the speed of the disk subsystem (or SAN) used. |
|
| Back to top |
|
|
| MQ281 |
| Posted: Wed Jun 16, 2010 12:19 am Post subject: |
|
|
Newbie
Joined: 05 Aug 2009
Posts: 6
|
Thanks zpat.
Yes, I thought circular logs would hold all persistent messages too along with 'q' file. Are the uncommitted messages stored on 'q' file?
Do the linear logs hold all persistent messages, committed and uncommitted?
Thanks a tonn for you time  |
|
| Back to top |
|
|
| MQ281 |
| Posted: Wed Jun 16, 2010 12:26 am Post subject: |
|
|
Newbie
Joined: 05 Aug 2009
Posts: 6
|
Hi zpat,
The current message flow is as below:
QMA and QMB are clustered
QAlias on QMA -> QL on QMB (QL is input queue for message flow A and these messages are inserted to Database)
Case1:
If QMB is down and all messages(persistent) are on transmit queue on QMA, would they be in committed or uncommitted state?
Case2:
If DB is in stopped state and all messages are on a dead letter or backout queue or any other queue on QMB, would they be in committed or uncommitted state?
Thanks,
mq281 |
|
| Back to top |
|
|
| zpat |
| Posted: Wed Jun 16, 2010 12:35 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
You need to read the manual about units of work & uncommitted messages.
Uncommitted means that the active application has not yet completed the MQGET or MQPUT operation.
Messages on queues (and in both your cases) which are not actively being currently processed by an application will be committed messages.
This has nothing to do with whether the messages are in transit or not.
Think queues, not logs for storing messages. |
|
| Back to top |
|
|
| MQ281 |
| Posted: Wed Jun 16, 2010 2:21 am Post subject: |
|
|
Newbie
Joined: 05 Aug 2009
Posts: 6
|
"Think queues, not logs for storing messages."
Is this true for Linear logs too? |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Jun 16, 2010 5:10 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| MQ281 wrote: |
"Think queues, not logs for storing messages."
Is this true for Linear logs too? |
No, linear logs hold persistent messages for media recovery.
To answer your originial question, with that amount of data you will see an impact on queue manager performance as it tries to juggle that amount of data across log files. This will lead to the queue manager being unresponsive for periods of time as it tries to stamp the next huge message in.
You're also going to find the queue manager massively I/O bound by all this. I echo the comments of my most worthy associates about using WMQ as a database.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Wed Jun 16, 2010 7:15 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Moving 100GB a day through MQ is not the same as trying to keep 100GB in MQ all at once.
100GB through MQ over the course of a day is nothing at all.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Jun 16, 2010 7:42 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| PeterPotkay wrote: |
Moving 100GB a day through MQ is not the same as trying to keep 100GB in MQ all at once.
100GB through MQ over the course of a day is nothing at all. |
Valid distinction there.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| MQ281 |
| Posted: Thu Jun 17, 2010 8:19 pm Post subject: |
|
|
Newbie
Joined: 05 Aug 2009
Posts: 6
|
It is 100GB a day through and not all at once and it was never that high so far, it is just high mark.
Thanks for all the valuable info from all of you. |
|
| Back to top |
|
|
|
|
