| Author |
Message
|
| mhubbard |
 Posted: Fri Feb 04, 2005 2:28 pm Post subject: Creating cluster queues programatically |
 |
|
Acolyte
Joined: 25 Aug 2004
Posts: 54
|
Hello
How do I create a cluster queues programatically? Specifically, I am using the C++ API.
Thanks
Mike Hubbard
_________________
Michael J. Hubbard |
|
| Back to top |
|
 |
| csmith28 |
| Posted: Fri Feb 04, 2005 3:25 pm Post subject: |
|
|
Grand Master
Joined: 15 Jul 2003
Posts: 1196
Location: Arizona
|
Could you be more vague?
Thanks
Chris
_________________
Yes, I am an agent of Satan but my duties are largely ceremonial. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Fri Feb 04, 2005 4:11 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
|
| Back to top |
|
|
| Nigelg |
| Posted: Sat Feb 05, 2005 10:46 pm Post subject: |
|
|
Grand Master
Joined: 02 Aug 2004
Posts: 1046
|
There is no direct API from any programming language to create a queue or any other WMQ object.
You can use PCF commands to create queues if the command server is running and the userID has sufficient permissions to the command queues, and has sufficient admin permissions.
See the PCF manual for copious details. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Sun Feb 06, 2005 7:48 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| Nigelg wrote: |
| There is no direct API from any programming language to create a queue or any other WMQ object. |
You can use MQOpen to create a dynamic queue.... 
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| mhubbard |
| Posted: Mon Feb 07, 2005 5:27 am Post subject: |
|
|
Acolyte
Joined: 25 Aug 2004
Posts: 54
|
Thank you for the mocking.
I guess I have become confused by the instructions in the basic cluster set-up documentation of the Queue Manager Clustering book.
It explains how to set up a simple 2 queue manager cluster and then, as a last step, has you define a queue using the CLUSTER key word. It states that, by using the cluster key word, the queue will be advertised and thus become known to the entire cluster.
I want to use dynamic queues and Jeff indicated. I guess I should have just tried it out, but there did not seem to be a way to use the CLUSTER attribute with defining a dynamic queue. There is not a Cluster attribute for a model queue, and there is only read access to the ClusterName member of the queue object in the C++ API.
I will just try the dynamic queue as I have it written and hope the magic happens.
_________________
Michael J. Hubbard |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Mon Feb 07, 2005 7:25 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
You can't use dynamic queues in a cluster setup, I think.
If you describe, in a reasonable level of detail, what you are actually trying to do, then we can help.
In other words - why do you want to use dynamic queues? What configuration are you trying to create/what business problem are you trying to solve?
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| mhubbard |
| Posted: Mon Feb 07, 2005 8:11 am Post subject: |
|
|
Acolyte
Joined: 25 Aug 2004
Posts: 54
|
I posted earlier on this list under the title "Advice on load balancing for clients that do both GET and PUT.
In that thread, I described by client devices (up to 20000) in the field that attach to a queue to obtain commands. I was having trouble, because I wanted to spread the incoming connections across several servers in a cluster, but did not understand that I could not do a GET from anything besides a local queue - that is, I can not just attach to any old server in the cluster and do a GET from my queue. In that thread, you suggested that I try either dynamic queues or a Pub/Sub methodology.
The Pub/Sub method seemed more complicated then I had time for, so I decided to go with dynamic queues. So I modified the client code to create a dynamic queue when it attached and the commands would be placed there. But if I cannot really use dynamic queues in a cluster, then this doesn't really help either - unless I change the "command putter" to search for the queue in every queue manager in the cluster.
_________________
Michael J. Hubbard |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Mon Feb 07, 2005 8:20 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Ah.
Okay. It's a follow up to that.
I'm not sure if clustering can be used for this or not - I'm not up on my name resolution stuff. But I *think* it can be used. But it wouldn't involve sharing the dynamic queue in the cluster.
It would merely involve using the cluster to resolve the remote queue manager name when the command putter did it's put. I think you would create queue manager aliases for each of your QMS and share those in the cluster. Then the command putter would use the dynamic queue name and the queue manager name registered by the client app, and name resolution would route the message through the cluster channels.
I *think*.
Maybe PeterPotkay or someone else with more cluster experience can add something here.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Mon Feb 07, 2005 10:31 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Before we even get into the clustering aspect of this....which I suspect is irrelavent anyway....
1)How is this command putter going to know the names of these dynamic queues so that it can address messages to them?
2)Are 20,000 clients going to create 20,000 dynamic queues, each with a unique name?
3)If yes, I assume you want these queues somewhat evenly divided across multiple QMs, and these QMs happen to be in a MQ cluster?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| mhubbard |
| Posted: Mon Feb 07, 2005 10:38 am Post subject: |
|
|
Acolyte
Joined: 25 Aug 2004
Posts: 54
|
Yes, each device has a queue of a unique name. The name contains our standard name of the device as part of the name. To be exact, each clients queue name is MYNAMECOMMAND.QUEUE.
The putter is driven by user input. Sometimes this input might be 1 command to 1 device. Sometimes it might be 1 command to all devices. In any case, the putter checks the destination machine name with a DB2 table where information on all active machines are stored. It confirms each machine name before attempting to place a command in its queue.
I should note that there are only about 1200 devices at this time. It is required to be scalable up to 20000.
_________________
Michael J. Hubbard |
|
| Back to top |
|
|
| mhubbard |
| Posted: Mon Feb 07, 2005 11:16 am Post subject: |
|
|
Acolyte
Joined: 25 Aug 2004
Posts: 54
|
I failed to answer question 3 very well.
I would like to have the queues spread around, but that was not my main concern in trying to make a cluster work. My main concern was trying to spread around the incoming client connections.
The tool is working beautifully in the test lab. I only began tinkering with clusters in an effort to make sure things would scale down the road.
_________________
Michael J. Hubbard |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Mon Feb 07, 2005 11:35 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Alriiiiiiighty then.
It is true that a model queue cannot spawn dynamic cluster queues, although it can create regular dynamic queues that happen to be defined on QMs who themsleves are in a cluster.
Unfortunatly, this does not help you unless the command putter knows both the q name (looks like u got that covered) AND the hosting QM name. If the comand putter was able to know both the destination q name and the destination QM name, then it could MQOPEN the queue with both peices of info, and the message would get to where it needs to. But you would need XMITQs and SNDR/RCVR channels on all the QMs (kinda ugly if the QMs are already in a cluster.)
So the problem now is how to get the command putter to know the destination QM, if the destination queue might be on any one of X QMs that you have. Do you think your naming standards could include this some how, so that the command putter would know the hosting QM of any particular dynamic queue?
If you cross that hurdle, it will work. But, this is definitly a scenario for pub/sub I think. Never having done pub/sub myself, I understand your hesitation. We tend to stick with what we know.
If it were me, and I did not have the pub/sub option, I would just bite the bullet, and create 100 or so permanent local queues on each QM in the cluster, and have 20 or 30 clients share each one, doing gets by CorrelID. Unless you need incredibly fast performance, this will work just fine. If everything is working, the queues will be empty, so the MQGETs should be very quick. You can then cluster these queues. True, there would only be one of each, so no load balancing, but at least you wopuld be able to advertise these to the cluster, and name resolution would work where the cluster channels would move your messages around. No need for muddying up the cluster with regular channels and XMITQs.
As yopu need more capacity, you add another q for another 30 (or 50, or 100) client apps, you cluster that queue, and wha-la, everything continues to work.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Mon Feb 07, 2005 11:55 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| PeterPotkay wrote: |
| So the problem now is how to get the command putter to know the destination QM, if the destination queue might be on any one of X QMs that you have. Do you think your naming standards could include this some how, so that the command putter would know the hosting QM of any particular dynamic queue? |
Peter, in the lightweight faux "pub/sub" model I suggested in the previous discussion... the client app would create the dynamic queue and then send a request message to the server app. This could very clearly hold the destination QM as well as the Q name of the dynamic queue.
If he created QM aliases on his "server" queue manager for all of his client QMs (not client Q's), would that allow the server program to use an MQOPEN with dynamic queue name and requested QM name, and have the resolution work and use the CLUSTER channels to reach the right QM?
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| mhubbard |
| Posted: Mon Feb 07, 2005 11:57 am Post subject: |
|
|
Acolyte
Joined: 25 Aug 2004
Posts: 54
|
Peter and Jeff - Thanks very much for spending the time to think about my problem.
There is obviously not going to be a sweet easy answer.
If I use Peter's last idea of a queue for x number of clients, isn't it true that each client must always connect to 1 and only 1 Queue manager? I think that is true and could be done, but does create a bit of a maintenance (or machine building) headache.
Am I understanding correctly that with a connection to, say, QM1, I can open a queue on QM2 (another queue manager in the cluster) even if it is not a cluster queue - provided I provide QM2 on the MQOPEN?
If that is true, and, if I expect to start with 2 Servers in the cluster and slowly grow from there, would it be a ridiculous performance hit to attempt the open on each QM looking for success?
(as to performance) - when 1 user is entering 1 command, I hope to achieve realtime like performance, but, in general, when many commands are going out, performance is not so much a concern.
**Jeff slipped his message in while I was still compiling mine. I would be interested in this possibility if it can be done.
_________________
Michael J. Hubbard |
|
| Back to top |
|
|
|
|
