| Author |
Message
|
| mvic |
 Posted: Thu Oct 11, 2007 3:37 am Post subject: |
 |
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| sebastia wrote: |
| 3k queues |
Did you try 3000 queues in test... were there any problems? I don't know a reason why it would cause any problems. |
|
| Back to top |
|
 |
| jefflowrey |
| Posted: Thu Oct 11, 2007 3:39 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
It's still three thousand (+1) lines of mqsc, and three thousand (+1) objects in the qmgr, and three thousand (+1) directory entries in the qmgr queues/ folder...
But the only real actual PROBLEM with this is that it's a lot to administer. And if the business needs require it, there's no getting past that.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| mvic |
| Posted: Thu Oct 11, 2007 3:44 am Post subject: |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| jefflowrey wrote: |
| But the only real actual PROBLEM with this is that it's a lot to administer. And if the business needs require it, there's no getting past that. |
Agreed. Sure, so you need to script your Defines that are run into runmqsc (this is a Good Thing even when creating one queue, IMHO) and if looking at qdepths etc, parse outputs and aggregate reports from runmqsc using perl or whatever. Yes, a bit more work in that way. But I wonder (genuinely - interested to know) whether that's a big deal for folks in general. |
|
| Back to top |
|
|
| sebastia |
| Posted: Thu Oct 11, 2007 4:11 am Post subject: |
|
|
Grand Master
Joined: 07 Oct 2004
Posts: 1003
|
Excuse my english, Jeff - I dont understand your phrase
""And if the business needs require it, there's no getting past that.""
"NO GETTING PAST THAT"
means
"IF YOU HAVE TO DO IT, LETS HAVE IT DONE" ?
or similar ?? |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Thu Oct 11, 2007 4:19 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Um, yes. "Get past" like what I do when I drive on the highway, and there's a big truck in front of me... I move in front, so that I have passed it and it is now also in my past....
I mean if the business requirements say "do X", then you can't avoid them... move past them in line ... for IT requirements.
So if your business needs require these messages to be persistent, then you can't use dynamic queues and there's nothing else to be done but create 3000 objects.
Business requirements come first - the job of IT is to implement the business needs in the best possible way that the technology allows. And to suggest ways that business processes can be transformed to create or take advantage of new opportunities.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| sebastia |
| Posted: Thu Oct 11, 2007 5:14 am Post subject: |
|
|
Grand Master
Joined: 07 Oct 2004
Posts: 1003
|
| Thanks,, Jeff - excellent, as always. |
|
| Back to top |
|
|
| zpat |
| Posted: Thu Oct 11, 2007 5:54 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
If it was me, I would design the application to retrieve messages selectively, then it will work with any number of queues.
I would then use the minimum number of queues that worked without performance problems.
One option is to send the message to one of 26 queues - depending which letter of the alphabet the location name started with.
It's easier to increase the number of queues later, than it is to reduce them.
The "business" does not want or should not ask for XXXX number of queues, they simply need a solution that works. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Thu Oct 11, 2007 6:23 am Post subject: |
|
|
Guest
|
If 3000 statically (administratively) defined queues are in a script, that's easy.
If names of queues that applications are to open depend on some outside, real-time influence, and are found in a table, that's easy.
If 3000 queues are dynamically defined as needed by applications that's easy.
If the question remains: "which is better?"; then I'm going to stick with
| Quote: |
| the answer to your question is: it depends. |
Application architecture should answer the business requirement. Us propellor-heads will continue to have fun with these esoteric issues. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Oct 11, 2007 3:16 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Don't do a single queue setup write a program around it.
Your message store is a DB.
You have an input queue selected for the app
You have a model queue for persistent queues setup.
The requesting app connects to the qmgr creates it's perstistent queue from the model queue, logs the time and queue name, and sends a message to the service app including the persistent queue it just created (request/reply model)...
The serving app gets the message, reads the dB for what needs to be sent and puts it all on the reply queue. When it is finished it adds a "done" message and stops servicing this thread.
The requesting app reads all its messages, finally gets to the "done" message, logs time and qname, deletes the queue and quits...
There you have a nice pull mechanism valid for 3000 + and you don't define more than a handful of queues...
You just need to ensure that your service app is SCALABLE...
Enjoy 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| sebastia |
| Posted: Thu Oct 11, 2007 3:31 pm Post subject: |
|
|
Grand Master
Joined: 07 Oct 2004
Posts: 1003
|
mr Saper - your design is just nice
But as far as I understand it, it does not fit our situation.
Our deal is to distribute a file with some data to 3000 stores.
Each store gets its own data, diferent from the others.
So, they do not "request" that data to be sent.
It can be sent from the central site to the stores ...
... once a day (for sure)
... 3 times a day (special seasons)
... once a week (lazy months of year)
But it is always the sender that decides when to do it.
In your design, it is the Requesting (consumer) application that drives all the data exchange - our design is "from center to terminals", I would say.
But I shall keep your lines in my mind : I liked it, real !
Thanks and Enjoy ! |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Oct 11, 2007 3:40 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
There is really no difference...
Your sender just pushes the data to the DB with the reference to the addressee.
When the addressee is ready it queries the server and says -- send me my data and here is the persistent queue I created dynamically for it.
When the server is done it puts a --done-- message on the queue.
When the requester reads the done message it can delete the queue.
Logging the queue name allows for exception operations... (connection broken before all the messages were read). Just reconnect and drain the queue for which you logged the name.
When you are done you can send a pcf message to delete that queue.
Enjoy
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| sebastia |
| Posted: Thu Oct 11, 2007 3:42 pm Post subject: |
|
|
Grand Master
Joined: 07 Oct 2004
Posts: 1003
|
Yes, I got it ! You are absolutely right ...
I am just not used to use a DB to store such things as you do ...
... but I like it (and will try to use it, of course)
( ) |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Oct 11, 2007 3:51 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Now using the lowest priority for the done message you can play along with priorities and create flash urgent messages and push them out and the client would receive it before its less urgent traffic...
Enjoy
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| sebastia |
| Posted: Thu Oct 11, 2007 3:58 pm Post subject: |
|
|
Grand Master
Joined: 07 Oct 2004
Posts: 1003
|
Man, you are effervescent tonite ... and I am in Europe, so quite tired at the moment ....
Have a nice weekend ! |
|
| Back to top |
|
|
| ma.eyal |
| Posted: Mon Oct 15, 2007 1:21 am Post subject: |
|
|
Novice
Joined: 13 Sep 2005
Posts: 15
Location: Israel
|
Just thinking about managing 3000 queues gives me nightmares 
Another option I would suggest for a "mixed" solution is using regional queues (alias or physical). I would suggest alias queues as well. |
|
| Back to top |
|
|
|
|
