| Author |
Message
|
| sebastia |
 Posted: Tue Oct 30, 2007 6:17 am Post subject: max size of MQ log in windows |
 |
|
Grand Master
Joined: 07 Oct 2004
Posts: 1003
|
Whe need to create a Queue Manager with a big LOG.
We need to store 3.000 persistent messages,
each of them being of 500 KBytes.
This gives us a 1,5 GB of LOG size.
The "-lf" parameter of CRTMQM command goes up to 65535,
in 4 KB chunks. Thah gives us a maximum log size
of 262 MB. Lets say 256, so we have 4 in a GB.
So, the only solution is to define "-lp 6" to be able to store the 3000 messages.
Am I wrong in some point ?
S. |
|
| Back to top |
|
 |
| Vitor |
| Posted: Tue Oct 30, 2007 6:25 am Post subject: Re: max size of MQ log in windows |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| sebastia wrote: |
| Am I wrong in some point ? |
A design that stores 3000 persistent messages is slightly suspect.... 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Oct 30, 2007 6:31 am Post subject: Re: max size of MQ log in windows |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| sebastia wrote: |
So, the only solution is to define "-lp 6" to be able to store the 3000 messages.
|
There have been a number of discussions in here about log files recently which the search button will turn up and you may find useful.
There is a school of thought that holds you should declare as many primary log files as you have space for up front. It's certainly generally held that the default values are not much use in the average production system, being too small for real use.
You also don't mention if you're using circular or linear logging, which will have a bearing on your decision. Not perhaps in the sizing of the log, but the efficientcy with which you must housekeep it!
Don't forget you'll need space for the queue file(s) as well.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Tue Oct 30, 2007 6:32 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
As long as the messages are PUT and GET within individual units of work - not all 3,000 in one transaction - you can configure it to only have enough active log space for one message.
Also, it's almost never the case that the default -lp and -ls are sufficient for any real use. The prevailing wisdom these days is that disk space is cheaper than downtime, so it's not a bad idea to max -lp and -ls and set -lf as high as you can stand.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| sebastia |
| Posted: Tue Oct 30, 2007 9:24 am Post subject: |
|
|
Grand Master
Joined: 07 Oct 2004
Posts: 1003
|
Vitor - I can not question that part : there HAVE TO BE between 3.000 (end of 2007) and 5.000 (end of 2008) permanent messages in that queue.
Jeff - disk space is no problem, so the queues shall have no problem.
Thanks. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Tue Oct 30, 2007 9:59 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
I didn't mention the q files at all.
I just said what Vitor said. Max -lp and -ls. Default values are not sufficient! if you're going to change -lf, you might as well change -lp and -ls.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| sebastia |
| Posted: Tue Oct 30, 2007 11:39 am Post subject: |
|
|
Grand Master
Joined: 07 Oct 2004
Posts: 1003
|
Ok jeff - to your suggestion, I have to say all 3.000 messages have to stay in the queue in the central office in our design, and the 3.000 MQ clients will come and get each one of them their specific message, each in their office-opening time, which can be very variable all along european time zones ...
Thanks all for your opinions. S. |
|
| Back to top |
|
|
| mvic |
| Posted: Tue Oct 30, 2007 1:23 pm Post subject: |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| sebastia wrote: |
Ok jeff - to your suggestion, I have to say all 3.000 messages have to stay in the queue in the central office in our design, and the 3.000 MQ clients will come and get each one of them their specific message, each in their office-opening time, which can be very variable all along european time zones ...
Thanks all for your opinions. S. |
IMHO, a relational database meets these requirements better than MQ. |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Oct 31, 2007 1:10 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mvic wrote: |
| sebastia wrote: |
Ok jeff - to your suggestion, I have to say all 3.000 messages have to stay in the queue in the central office in our design, and the 3.000 MQ clients will come and get each one of them their specific message, each in their office-opening time, which can be very variable all along european time zones ...
Thanks all for your opinions. S. |
IMHO, a relational database meets these requirements better than MQ. |
Absolutely yes. Much easier to control, administer and report on. Using a queue is going to cause pain in the medium to long term.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Wed Oct 31, 2007 4:53 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
He mentioned the clients are spread across the continent. What if the clients are running on various CCSIDs. Wouldn't MQ's built in conversion (I know that's not translation) be a reason to use MQ in this scenario over a DB?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Wed Oct 31, 2007 4:56 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| PeterPotkay wrote: |
| He mentioned the clients are spread across the continent. What if the clients are running on various CCSIDs. Wouldn't MQ's built in conversion (I know that's not translation) be a reason to use MQ in this scenario over a DB? |
No. Databases handle conversion as well.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Wed Oct 31, 2007 5:05 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
So is the reason a DB is being recomended here because this is a "pull" type transaction? The 3000 clients initiate activity by going and pulling some info. Versus a more typical MQ scenario where a client will push up a request message to get a reply?
Seems if this is the case a DB could be recomended over MQ for any transaction other than a psuedo-synchronous request / reply. One way datagrams? Just write to a DB. Batch job that reads from a q once day? Have the puuters write to a DB and the batch job reads from the DB. 
3000 messages sitting in a queue for a few hours does not seem like a horrible thing to me.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Wed Oct 31, 2007 5:28 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
To be clear, a database is being recommended for long term storage of data. Particularly data that needs to be retrieved based on variable criteria. That is, generally, any situation that has a program that browses messages to read the data and then leaves them on the queue.
It's not clear from the requirements that this is the case - i.e, it's not clear that the messages are left on the queue after they've been processed.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Wed Oct 31, 2007 5:39 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
OK, that makes more sense. DBs are better for infrequent writes and then that data is read lots and lots of times, often chosen by multiple criteria.
But for put it once / get it once with a simple selector (like Correl ID) MQ makes more sense.
(2 days in a row for this padiwan picking something up from the master  )
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Wed Oct 31, 2007 6:02 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| PeterPotkay wrote: |
| But for put it once / get it once with a simple selector (like Correl ID) MQ makes more sense. |
Yes. A DBA would have a fit if you wrote an application that did lots of insert/select/DELETEs.
| PeterPotkay wrote: |
| (2 days in a row for this padiwan picking something up from the master ) |
At 3275+ messages, you're a Jedi Knight, not a padawan... 
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
|
|
