| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
| Log MQ Messages (possible?) |
« View previous topic :: View next topic » |
| Author |
Message
|
| Vitor |
 Posted: Wed Feb 04, 2009 3:44 am Post subject: |
 |
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| exerk wrote: |
| Don't Cressida make something that will do it? |
The ReQuest product I mentioned in my first post.
What I was attempting to convey is that it's not as simple as just keeping the linear logs, even if they had all the information you wanted.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
 |
| yogeshi12 |
| Posted: Wed Feb 04, 2009 7:21 am Post subject: Some clarifications |
|
|
Newbie
Joined: 03 Feb 2009
Posts: 4
|
Hi Vitor, Thanks.
In essence I don't have any ability to change the applications, and the health of the qmanager is only somewhat important:
Some more answers:
What use will the logged messages be put to?
None of my business. When using Mirrorq I would have a listener read off the queue and send it into the ether of my shop
How long will they be kept for?
On the mirrorq, not long -- seconds.
What process will delete obsolete logs?
once I read the message from the mirrorq, and send it into the ether, someone else's app will deal with that.
Is mirrorq still not a great option? What else can I do? I don't have much budget for any enterprise product.
Thank you. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Wed Feb 04, 2009 7:25 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| Quote: |
| In essence I don't have any ability to change the applications |
But the application developers now want message replication (logging) for some secret or undefined reason? If this is an application requirement, the developers should pay for it.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Feb 04, 2009 7:39 am Post subject: Re: Some clarifications |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| yogeshi12 wrote: |
| the health of the qmanager is only somewhat important: |
That statement will come back to bite you one day.... 
| yogeshi12 wrote: |
| None of my business. When using Mirrorq I would have a listener read off the queue and send it into the ether of my shop |
Yes it is. If the listener fails or the ether fills up / stops working then it will become your business very quickly. The exit won't like not being able to put messages, and the tantrum will be spectacular.
It's also stupid in the extreme for you to be given a requirement to produce message copies that go "someplace, for some reason". This is management at it's purest. If there's no concrete reason for doing this it should not be done.
| yogeshi12 wrote: |
How long will they be kept for?
On the mirrorq, not long -- seconds. |
Not what I meant. If they're kept for a long time, or more likely kept forever because no-one's defined a strategy for deleting old ones (and if you don't know why you're keeping them how can you possibly know when you won't need them anymore?) then the ether will fill up.
Bang. 
| yogeshi12 wrote: |
What process will delete obsolete logs?
once I read the message from the mirrorq, and send it into the ether, someone else's app will deal with that. |
You hope. See above.
| yogeshi12 wrote: |
| Is mirrorq still not a great option? What else can I do? I don't have much budget for any enterprise product. |
No exit is ever a great option, because when they go wrong they ruin your day.
I would also put it to you that an exit will have a greater TCO than the other options mentioned here. Plus your budget can be combined with that for the app someone else is writing to deal with these messages..... 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Wed Feb 04, 2009 7:43 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
HP's Transaction Vision can log messages for you.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| yogeshi12 |
| Posted: Wed Feb 04, 2009 9:28 am Post subject: Such is life |
|
|
Newbie
Joined: 03 Feb 2009
Posts: 4
|
Thanks all -- I was a bit short on the response. The fact is that there is a very good reason and method to handle all of the logs that are generated -- and the reality is that this is an auditing requirement so that we can't rely on the applications teams, especially since we don't own most of the applications. Also, the qmgr health is already being monitored and, surprisingly healthy almost all of the time. The laws of a contact admin IT shop are fully enforced here.. If I spend $ these days it has to come out of headcount, don't want to do that...I'm also only privy to a few of the reasons some of these decisions are being made...so I can only focus on the feasible technical solutions.
So I guess my questions are more technical, for example:
>>The exit won't like not being able to put messages, and the tantrum will be spectacular.
Is there a way to change that in the mirrorq? try catch or something?
>>No exit is ever a great option, because when they go wrong they ruin your day.
HP TV, how does it monitor Queues? Does it read the transaction log? Do any commercial "loggers" require exits? Is log reading (via vendor apps, I get it that i don't want to be in that business) the only great option?
Thanks |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Wed Feb 04, 2009 9:37 am Post subject: Re: Such is life |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| yogeshi12 wrote: |
So I guess my questions are more technical, for example:
>>The exit won't like not being able to put messages, and the tantrum will be spectacular.
Is there a way to change that in the mirrorq? try catch or something?
|
mirrorq is sample code. Change it to do anything you like.
| yogeshi12 wrote: |
>>No exit is ever a great option, because when they go wrong they ruin your day.
HP TV, how does it monitor Queues? Does it read the transaction log? Do any commercial "loggers" require exits? Is log reading (via vendor apps, I get it that i don't want to be in that business) the only great option?
Thanks |
HP TV is an API Exit on Windows and UNIX. You can configure it to capture part of or the entire payload of the message. TV does not have anything to do with the MQ logs.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Feb 04, 2009 10:40 am Post subject: Re: Such is life |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| yogeshi12 wrote: |
| Is there a way to change that in the mirrorq? try catch or something? |
As my most worthy associate points out, mirrorq is a sample and you can do what you like. However, you need to remember the rules of an exit. Say that (in an exceptional but plausable scenario) whoever is reading your copy lets you down and your put fails with a "queue full". You catch it with a try/catch. Then what? Sit in the exit and wait for 5 seconds to try again? That'll affect throughput. Suppose he lets you down big time and the queue is full for minutes? What then?
Here's another scenario. The exit code (which you now own) has a crisis and throws an abend. Next thing you know the queue manager's on the bottom of the server, twitching gently and everyone's looking at you.
| yogeshi12 wrote: |
>>No exit is ever a great option, because when they go wrong they ruin your day.
HP TV, how does it monitor Queues? Does it read the transaction log? Do any commercial "loggers" require exits? |
Allow me to clarify my overly simplistic statement. Exits are never a great option unless they're owned by a commercial organisation that's tested them to the point of destruction to beyond, who you can call when they fail and who you can blame when they fail. 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Wed Feb 04, 2009 10:50 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| Quote: |
| It's also stupid in the extreme for you to be given a requirement to produce message copies that go "someplace, for some reason". |
Add to this the requirement that it cost little or nothing today (acquisition, development), and have little or no long-term cost (exits, disk, tape).
If it's a requirement, it is (or should be) part of some project that is funded to accomplish its stated goal.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| yogeshi12 |
| Posted: Wed Feb 04, 2009 11:19 am Post subject: Test Test and retest, got it |
|
|
Newbie
Joined: 03 Feb 2009
Posts: 4
|
First off -- Man, what a great board..thanks so much for your help!!!
Hey, life at the bottom of an IT Shop, gotta love it...
Is there a difference between an "API" exit and the exit that mirrorq does?
And besides loosing my "audit" message if I can't put onto mirrorq, the performance overhead on the exit is a factor of the extra put? Is there a way to thread that (in general, not necessarily in the mirrorq sample). |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Wed Feb 04, 2009 11:33 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| Quote: |
| the performance overhead on the exit |
Keep in mind (in managements mind) that the exit will be invoked for every message sent across the channel.
If many different application types use the same channel, they will all endure the execution of the exit.
More of a concern (for me) is the possibility of a failure in exit code could kill off the channel, cause messages to be lost, allow errant exit code to change $4.00 to $4,000,000.00.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Feb 04, 2009 12:20 pm Post subject: Re: Test Test and retest, got it |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| yogeshi12 wrote: |
| Is there a difference between an "API" exit and the exit that mirrorq does? |
The mirrorq sample is an API exit.
| yogeshi12 wrote: |
| besides loosing my "audit" message if I can't put onto mirrorq, the performance overhead on the exit is a factor of the extra put? |
It is.
| yogeshi12 wrote: |
| Is there a way to thread that (in general, not necessarily in the mirrorq sample). |
One of the reasons exits are so potentially dangerous is they run as part of the queue manager process. So not only are they in the same thread, if they fail they get to kill it. Hence my standard rant (you can search for the full version) that a badly written exit will kill your queue manager's throughput, a really badly written exit will kill your queue manager.
So if you take the sample mirrorq code (intended only to demonstrate a technique and not intended for use in a production environment with decent load) make sure you change the code so it's utterly bullet proof, can never fail in any circumstances, runs like a rocket, can be demonstrated to the auditors never alters or corrupts data and is fully maintainable for when WMQv8/9/n changes the exit block.
Knowing that some software companies employ an entire team to do this.
And find a way to get a good night's sleep while it's in use.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
