| Author |
Message
|
| jconway |
 Posted: Thu Nov 18, 2004 5:46 am Post subject: MDB question - Browsing a queue. |
 |
|
Novice
Joined: 28 Apr 2003
Posts: 17
Location: Paris, France
|
Hello,
Firstly I have to admit to being pretty ignorant about JMS, EJB, MDB, and the Java world in general. My role is more Websphere MQ administration and not application development.
Here is my question:
With the standard MQ API, and also when using JMS, the possibility exists to just "browse" a message on a queue. This leaves the message "visible" to other applications which have this queue open.
When using an MDB listener is there any way of having this 'browse' functionality?
That is to say does an MDB listener always have to "consume" a message or is there a way to create an MDB so that the QueueBrowser JMS functionality is used instead of the QueueReceiver.
Again, please excuse me if my question is not clear or properly/precisely expressed. I don't know how to speak Java properly!!
The reason I ask is because we want to implement a solution where many application servers listen on the same Queue. The message sender does not know which application server the message is destined for. We would like an MDB listener in each application server to be able to "browse" the message and, using information in the message body, to decide whether it's for him or not. The application server who wants the message then consumes it. The others ignore it.
Thank you.
(ps. Application Server being used is Weblogic.) |
|
| Back to top |
|
 |
| jefflowrey |
| Posted: Thu Nov 18, 2004 5:58 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
This is a poor design.
You should use a simple Filter pattern instead - have a single MDB receive messages off a queue, and put them into different queues for each App Server.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| wmq_guy |
| Posted: Thu Nov 18, 2004 6:50 am Post subject: |
|
|
Acolyte
Joined: 21 Oct 2004
Posts: 50
|
| short, sweet and nicely put Jeff. that is exactly what it's for. |
|
| Back to top |
|
|
| jconway |
| Posted: Thu Nov 18, 2004 7:13 am Post subject: |
|
|
Novice
Joined: 28 Apr 2003
Posts: 17
Location: Paris, France
|
Thank you very much for your prompt replies. I agree that it is a poor design but unfortunately sometimes necessity and constraints dictate design. The idea of having a queue for each application server is not attractive when the number of application servers we have is taken into account. Also this would mean intervention at the MQ level (i.e. the addition of a new queue) each time a new application server instance is added to our architecture.
Also, it is only the app server instance which has the "infomation" to decide whether the message is for him or not. The idea of having a single MDB "somewhere" which reads all messages from the queue and then dispatches these message to the relevent app server means creating (and constantly maintaining) a table relating this "infomation" to queue names to be used by this "master" MDB (which is then, in effect, a mini-broker).
There is nothing inherently wrong with the design of many MQ applications having the same queue Open and browsing the message contents to see whether it is for them or not (particularly if the relevent information is in the first few bytes and this is all that is read by the MQGET with BROWSE option). This is quite a common solution when using the basic MQ Api. Apparently judging by your replies, this is not the case when using MDBs.
My question still stands however (if only as a point of interest); is a "browse" functionality possible when using MDB's or does the usage of an MDB constrict one to "consume"?
Thanks again all. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Thu Nov 18, 2004 8:18 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
I disagree with most of your points. I will not refute them, because as you have said, this is a question about MDB functionality and not about MOM architecture and design. However, in my opinion, the design you have lain out is an AntiPattern rather than a Pattern.
I do see a property on the queue connection factory definition for a WebSphere MQ Queue Connection Factory in WebSphere App Server,
| InfoCenter wrote: |
Message retention
Select this check box to specify that unwanted messages are to be left on the queue. Otherwise, unwanted messages are dealt with according to their disposition options.
Data type Enum
Units Not applicable
Default Cleared
Range
Selected
Unwanted messages are left on the queue.
Cleared
Unwanted messages are dealt with according to their disposition options. |
I do not know if this will help.
I also do not know if you are using WAS or not. I also do not know, off the top of my head, if Browse is allowable under the EJB 2.0 spec for MDBs.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Thu Nov 18, 2004 8:22 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
You might also consider a pub/sub solution rather than a browse solution.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| jconway |
| Posted: Thu Nov 18, 2004 9:33 am Post subject: |
|
|
Novice
Joined: 28 Apr 2003
Posts: 17
Location: Paris, France
|
Hi again,
I really do appreciate your replies. I have searched everywhere and can find no reference to MDBs and message browsing ... only message consuming. I think I shall move on from my previous idea and consider another approach.
Unfortunately pub/sub is not an option for us as we currently do not have the budget to purchase the Event Broker product and our company is not open to using the freely availble pub/sub support pac as its support will be discontinued shortly.
I am very interested in your opinions regarding my "anti pattern". You obviously have a lot more MQ experience than I do and I would very much like to hear your ideas for a better pattern for my current scenario if you have the time.
Here's the deal:
* We have many Weblogic app servers.
* None of these app servers can intercommunicate directly.
* Each app server hosts a number of "clients" which have a unique clientId.
* A distant application (not in one of the app servers) produces a message to update information for a particular client.
* This message has to be consumed by the app server which hosts this client.
* The message producer has no way of knowing which app server hosts this client.
The options I have, as far as I can see, are the following:
(1) One Queue where the "client update" mesage is placed. All app servers have this queue open and browse the message to see what client it is for. If the message is for a client that is hosted by the app server the message is consumed by the app server. If not, then the message is left on the queue.
(2) One Queue for each app server. The producing application uses a Distribution List to "late fan out" the message to each queue. Each app server gets the message and either discards it or actions it depending on whether they host the client concerned. We have many app servers which would mean having many queues.
(3) One Queue for each app server. Externalise the clientId information somewhere (an LDAP for example) and associate a Queue name with each clientId. The PUTting application references this "table" and uses it to decide which queue to put the message on. This table would risk being very large as we have many clientIds and would be constantly changing due to addition/removal of clients.
(4) Your suggestion - producer puts the message to a queue where it is consumed by an application which references a "table" as above and PUTs the message to the relevent queue.
The "One Queue" option does, I agree, means that every app server except the one concerned performs a "wasted browse". This is the only disadvantage I can see and I am trying to weigh this disadvantage against the advantages that this option provides compared to the "one queue per app server" options:
* Only one Queue to define and monitor.
* Addition of a new app server requires no other intervention (no new queue definition or any update to an external "clientId->Queue name" table).
* No changes to this external table are required if the "location" of a client is moved from one app server to another as the server always knows which clients it owns.
* No usage of "Distribution Lists" required (which I have heard can be a pain to program) by the message producing application.
* No need to write/maintain an extra message consuming program which then dispatches the message to the correct app server queue.
Now that I have explained our situation more deeply would you still recommend the multi-queue approach with an external table? This approach is, for sure, more elegant but on balance do you think the advantages in the one-Queue method might outweigh the "wasted browses"? I suppose the message throughput would be an important consideration in making this choice.
Do you have any other suggestions that I have not considered?
Am I considering somthing completely stupid here that just won't work?
I would really appreciate your opinions on what I have written as I am trying to learn more about MOM architecture and design. In a perfect world I would not go with the "Browse one queue" approach but considering our constraints (no budget to buy a broker, a strong desire to limit inhouse software development ... sounds like I work in hell eh??) would you regard the "one queue" approach as a valid compromise or as a disaster in the making?
Maybe I should move this post to General Discussion forum?
Again, thanks a million for your assistance; opinions and interest.
Jim. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Thu Nov 18, 2004 10:19 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
I do not see any significant additional burden in creating an additional queue for each App Server instance, as you have to take steps to add a new app server already. Including an additional step in that procedure to create a queue is not significant, in my opinion.
Having multiple applications "cherry-pick" from the same queue is significantly resource intensive for MQ, as they each have to essentially walk through all messages on the queue to find messages they are interested in, and then walk through again to actually FETCH the message.
In addition, this design actually ends up HIDING a lot of the complexity of the situation. It also makes it very difficult to find out where a misrouted message went. You know that someone picked it up... but you may have to take a lot of steps to find out who.
By putting in a Filter or by putting in a Router (the difference is essentially the complexity and dynamic nature of the decision making), then you know that a) the message will only get to the right service queue, b) if the message didn't get there you know who failed to put it there.
The advantage of a Filter/Router over a DestinationList approach is that you can centralize the decision making, instead of having to spread it out to every producing application.
It's still not clear if you are using WAS or WebLogic or some other J2EE App Server. I believe the current JMS spec requires a JMS Provider to include a pub/sub broker of some sort, and I'm fairly certain that WAS has one, that just needs to be configured (and it's not quite the standalone support pack).
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| JLRowe |
| Posted: Thu Nov 18, 2004 1:27 pm Post subject: |
|
|
Yatiri
Joined: 25 May 2002
Posts: 664
Location: South East London
|
| Why not just have one Queue? You can use selectors on each MDB to only pick off certain messages with a property value. I believe IBM optimised MQ support for selectors some time ago and I would not expect a performance problem. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Thu Nov 18, 2004 1:34 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| timna wrote: |
| Why not just have one Queue? You can use selectors on each MDB to only pick off certain messages with a property value. I believe IBM optimised MQ support for selectors some time ago and I would not expect a performance problem. |
No. Actually, JMS Selectors are highly unoptimized in the WebSphere MQ JMS provider. If you think about this, you will realize why. JMS Selectors require you to parse the MQRFH2 header to retrieve the properties, interrogate those properties, and then either accept or reject the message. This must be done for each and every message, each and everytime.
This is slow, and is not optimizable without hardcore changes to the internal data structures that describe MQ messages.
In addition, it is STILL an ANTI-PATTERN for applications to share a queue, in my opinion.
Why go to the trouble of writing multiple applications? Why not just have one big huge lump of code that does everything?
Queues should contain messages of a single logical message type. Multiple applications should not process messages of the same logical message type. You should not mix Orders with Inventory Updates in the same queue. You don't mix them in the database, why would you mix them in the queue?
Adding and maintaining additional queues is not a significant burden on the infrastructure, the operational procedures, or the application.
And again, sharing queues HIDES complexity that is much better made EXPLICIT.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Nov 18, 2004 4:45 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Hi Guys,
This is a design question:
I cannot imagine that each app server would need its own queue. What I do question though is the need for sending all the messages to the same queue. The sending application should be able to determine one queue per message type. Then any app server interested in the message type could have an MDB listening to the corresponding queue and process on a first come first served basis. (Load balancing at the app server level). (2 phase commit only if on the MQ Server or if transactional client)
One other option would be to have each app server have the same queue defined but on a different qmgr. Then an MDB can read from the queue
map the message and post to the corresponding destination all in the same transaction. A property file can govern the mapping. Now the receiving qmgr needs to have a default way to communicate with each of the apps server's qmgr. (hub spoke design or if you prefer broker on the cheap).
Now if you want to use only 1 qmgr this is still feasable but you will need one queue per destination (app server).
This is not a problem in a J2EE environment as you can map a different queue to the same jndi queue name in each app server. They all think its the same queue whereas behind the scenes each has its own queue. Again 2 phase commit only if qmgr on same machine as app server or transactional client.
But as Jeff said, what you propose to do is a bad design and a performance killer up front. Bring it up with your managers and see what they think of it.
Enjoy  |
|
| Back to top |
|
|
| jconway |
| Posted: Fri Nov 19, 2004 2:28 am Post subject: |
|
|
Novice
Joined: 28 Apr 2003
Posts: 17
Location: Paris, France
|
Hello everybody,
I'd like to thank everyone for their tremendous advice and interest. I understand better the "anti-pattern" comment and am now in total agreement with the dangers and inefficiencies inherent in my original "one-queue, everyone browse" idea.
I'm definitely going to push for a "one queue per application server" idea with a "mini-broker/router type" application that knows which app server hosts which client, and is then responsible for dispatching the correct message to the correct queue/app server.
Thanks again for showing me a proper pattern. I'll update this post with information concerning how well I succeed in pushing this idea for those who are interested .... hopfully this will be a "happily ever after" story, and this better MOM architecture design will be accepted. If accepted, I'll also update this post with technical details of exactly how we implementated this type of solution.
Many, many thanks.
Jim. |
|
| Back to top |
|
|
|
|
