| Author |
Message
|
| poonchikit |
 Posted: Wed Oct 04, 2006 1:28 am Post subject: Retrieving Msg of the same Msg ID from Different QMs |
 |
|
Novice
Joined: 04 Oct 2006
Posts: 10
|
Hello.
Do you know if there is any tool that can retrieve (in browsing mode) all messages of the same message id from different queue managers?
I found that in using MQ there is always the problem of identifying which message is stored/stuck in which queue. It is quite common for troubleshooting - when the client application did not receive any response, it is necessary to ascertain in which QM the whole processing failed.
Thanks a lot. |
|
| Back to top |
|
 |
| jefflowrey |
| Posted: Wed Oct 04, 2006 1:37 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Any tool that can browse a queue and use MsgId to select messages for browsing can do what you want. Manually - you'll have to connect to each queue manager, browse each queue and ask for the message id specifically.
There isn't anything, that I know of, that will do this automatically. It wouldn't be hard to modify the Browse sample code to do, though. You'd just have to give it a list of queue managers.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| poonchikit |
| Posted: Wed Oct 04, 2006 5:55 pm Post subject: |
|
|
Novice
Joined: 04 Oct 2006
Posts: 10
|
Thanks.
Actually, after browsing IBM's MQ support pacs, I found me02 is exactly the one for this purpose. |
|
| Back to top |
|
|
| poonchikit |
| Posted: Wed Oct 04, 2006 9:51 pm Post subject: |
|
|
Novice
Joined: 04 Oct 2006
Posts: 10
|
After further examination, this SupportPac can only be used for searching MQ objects instead of MQ messages.
I think I have to write my own application to do this.
Thanks. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Oct 05, 2006 3:03 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| poonchikit wrote: |
After further examination, this SupportPac can only be used for searching MQ objects instead of MQ messages.
I think I have to write my own application to do this.
Thanks. |
And pardon me for asking but what distinction do you make between MQ object and MQ message?
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| poonchikit |
| Posted: Thu Oct 05, 2006 6:07 pm Post subject: |
|
|
Novice
Joined: 04 Oct 2006
Posts: 10
|
What MQ objects to MQ messages is like DB objects (tables, triggers, index, etc.) to the data stored in DB.
It's quite strange that after 2 decades (to the best of my knowledge) of evoluation, MQSeries do not have this kind of functionality built into the MQ explorer. The only reason that I can come up with is performance. However, this is not insurmountable. |
|
| Back to top |
|
|
| kevinf2349 |
| Posted: Fri Oct 06, 2006 4:42 am Post subject: |
|
|
Grand Master
Joined: 28 Feb 2003
Posts: 1311
Location: USA
|
| Quote: |
| It's quite strange that after 2 decades..... |
Not really, MQ is not supposed to be used to store data, MQ is a messaging transport. Messages should be consumed.
Of course MQ uses MQ to store data, but that is mostly internal use only.  |
|
| Back to top |
|
|
| poonchikit |
| Posted: Sat Oct 07, 2006 7:44 am Post subject: |
|
|
Novice
Joined: 04 Oct 2006
Posts: 10
|
Actually, my point is not using MQ as a data store. It's more on diagnosis.
If you have many QMs and you don't know where your message got stuck, you have to go thru all the queue managers and the relevant message queues to see up to which step the transaction failed. It's really an excruciating task.
Also, the search functionality is not just frosting on the cake. Considering you've got a dead letter queue with many messages stuck in it and you have to locate a specific message for further analysis, you certainly want to have a search function by criteria like message id, message content, etc.
What would you suggest in these 2 cases? Thanks. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Sat Oct 07, 2006 8:37 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
MQ networking is pretty deterministic.
If you know what queue manager the message was PUT to, there's usually only a few places a message could have gone.
And if you know what QUEUE name the message was PUT to, there's usually even fewer.
I'd consider it a sign of poor network or appplication management if you routinely have to search large numbers of queue managers to find errant messages.
If faced with that kind of problem myself, I'd start putting in channel exits that log message ids.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| poonchikit |
| Posted: Sun Oct 08, 2006 5:27 am Post subject: |
|
|
Novice
Joined: 04 Oct 2006
Posts: 10
|
Thank you very much for your advice. Maybe I can give more details and see if you can provide further insights based on your experience.
I am doing an EAI project connecting around 20 sales channel applications with the mainframe in retail banking environment. Every sales channel application has a QM of its own and the QM joins a cluster with 3 QMs used by MQSI. Also the mainframe has its own QM. In this scenario there are 24 QMs. For the MQSI part, there are around 20 queues in used by different nodes and every node may encounter problems. Of course by properly designing the message flows with exception handling and dead letter queues joined to the failure exits of the nodes, the message location problem can somewhat be minimized. However, in the real world scenario, it is never be perfect. Especially, when it was done by your predecessor who did MQSI design and programming for the first time.
Having said that, the interesting thing happens when 1 of the sales application owner calls in and said there was no response for his applicatin's request. Since his application only talls to our EAI layer and is transparent to the mainframe, he only knows that his request message sent to a "black whole". The responsibility then lies on us to locate where the problem happened. Considering the many queues used by MQSI and the chances that many sales channel applications may have problems at the same time. Even if we carefully designed the message flows and put all failed messages in the dead letter queue, we would still require a facilitiy to search the failed message by its content, msg id, etc. for fast turnaround of support request for one specific sales application. I have to stress that the user would not tell us the msg id. They usually tell us some application level information like transaction id, which is stored in the msg payload. That's why search by msg id is not enough. Search by content is even more important.
Added to this, we are the first layer of support to locate the problem. We have to conclusively confirm that the message got stuck in which layer before we can forward our request to the mainframe team or the applicatin team themselves for further investigation if it is their responsibility.
Being deterministic is a reason we use queuing model instead of RPC for the design. But being deterministic is one thing and locating a specific message in a deterministic systems consisting of many shared components is quite another. This is my second enterprise level MQSI project but still I have not found an effective way to tackle this problem without relying manual means. I think this area can actually be automated and the information should be made available by a run of a simple tool.
Did you have a simliar kind of experience? How did you tackle it? Thanks. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Sun Oct 08, 2006 6:00 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
This kind of thing should be built into the ESB, which is your MQSI hub.
Every message recieved by the ESB should be stored in a database, exactly as it arrives. This database should include indexes on the most commonly used meta-data about the message - for example your transaction id. This is usually implemented by a single message flow that receives copies of every message from a queue - or occasionally by a subflow.
Every ESB subsystem should report what it does to a message, including where it sends it. This is basic application logging. This is usually implemented in a ESB/SOA by a single process/flow that recieves logging information from all the other applications and writes that again to a database.
If you are using WMB v6, at least at the FP1 level, you can write a user exit (in C only) to monitor all of your flows and extract all the necessary information. This would be a non-invasive solution and wouldn't require changing your existing WMB solutions/applications. You would have to restart your production Broker environment to enable this exit.
Or, again, you could do this at the MQ level and implement a channel or API exit that would log the relevant information about messages that cross that boundary.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| poonchikit |
| Posted: Sun Oct 08, 2006 6:41 am Post subject: |
|
|
Novice
Joined: 04 Oct 2006
Posts: 10
|
Thank you very much for your insights. It's very helpful.
Could you please provide me further information on WMB user exits? What's it usually for? Is its functionality duplicate with that of MQ exits? Pardon my ignorance, I have never heard of this. Thanks.
I get your point. And our current design has logged all messages in and out of WMB. It's just that we sometimes need to locate in which step (queue) that the message transformation process fails when the message was being processed by WMB. Installing channel exits can provide valuable infomration at the entry and exit points of WMB but locating where the message failed and hence got stuck within the QM of a WMB broker is still a chore, at least for my team. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Sun Oct 08, 2006 6:51 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| poonchikit wrote: |
| Could you please provide me further information on WMB user exits? What's it usually for? Is its functionality duplicate with that of MQ exits? Pardon my ignorance, I have never heard of this. Thanks. |
Well, like I said, it's brand new for v6, and only in the FP1 level of v6, too.
http://publib.boulder.ibm.com/infocenter/wmbhelp/v6r0m0/index.jsp?topic=/com.ibm.etools.mft.doc/ac02140_.htm
I get your point. And our current design has logged all messages in and out of WMB. It's just that we sometimes need to locate in which step (queue) that the message transformation process fails when the message was being processed by WMB. Installing channel exits can provide valuable infomration at the entry and exit points of WMB but locating where the message failed and hence got stuck within the QM of a WMB broker is still a chore, at least for my team.[/quote]
Well, then you need to add some level of logging into each of your flows. User exits would let you do this without changing your flows - but are C only and probably a little complicated for the average WMB user. A logging subflow to produce messages and a standalone logger flow to record messages might be easier - then you just insert the subflow into all of your other flows and implement the logger flow in one of your brokers somewhere.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
|
|
