| Author |
Message
|
| Muhammad Haider |
 Posted: Tue Nov 01, 2016 10:59 am Post subject: How to track MQ transactions ? |
 |
|
Apprentice
Joined: 14 Mar 2011
Posts: 43
|
| How can we make sure that we are maintaining MQ data for a period of 60-90 days so that we can track a transaction later i.e when a particular message was exchanged with a particular client. Is linear logging suitable here or circular logging is required here. Plus in which directory MQ is maintaining all this information and how can we access that information in a human readable form ? Do we require some support pac for this ? Do we need to implement exits (if yes is there any built in exit functioanlity for this purpose or do we require some programming for it) ? So what will be the best solution ? |
|
| Back to top |
|
 |
| Vitor |
| Posted: Tue Nov 01, 2016 11:08 am Post subject: Re: How to track MQ transactions ? |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| Muhammad Haider wrote: |
| How can we make sure that we are maintaining MQ data for a period of 60-90 days so that we can track a transaction later i.e when a particular message was exchanged with a particular client. |
Add code to the application.
| Muhammad Haider wrote: |
| Is linear logging suitable here or circular logging is required here. |
Neither is suitable.
| Muhammad Haider wrote: |
| Plus in which directory MQ is maintaining all this information |
Wherever you told it to when the queue manager was build.
| Muhammad Haider wrote: |
| how can we access that information in a human readable form ? |
You can't - the format is proprietary to IBM and they don't document it.
| Muhammad Haider wrote: |
| Do we require some support pac for this ? |
No such support pac exists - see above.
| Muhammad Haider wrote: |
| Do we need to implement exits (if yes is there any built in exit functioanlity for this purpose or do we require some programming for it) |
MQ doesn't come with any built in exits, so you have to build your own. If the answer is "we need to build our own exits" then you need a different question. Exits are an advanced topic with the capability to slow your queue manager to a crawl (if badly written) or bring your queue manager down in a smoking heap (if very badly written)
| Muhammad Haider wrote: |
| So what will be the best solution ? |
Buy the solution from people who have very well written exits. Who also sponsor this forum.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Nov 01, 2016 11:15 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
I am not now nor have I ever been employed or associated with Cressida Technology, all opinions are my own and no endorsement express or implied should be read, inferred or determined from these opinions. Other solutions are undoubtedly available and may be a better fit for any specific use case.
But they're nice people.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Muhammad Haider |
| Posted: Tue Nov 01, 2016 11:20 am Post subject: |
|
|
Apprentice
Joined: 14 Mar 2011
Posts: 43
|
Hi Vitor,
Thank you very much for each reply.
What do you say about this link:
Where's my message? Tool and instructions to use the MQ recovery log to find out what happened to your persistent MQ messages on distributed platforms
http://www-01.ibm.com/support/docview.wss?uid=swg21660642
They suggest to use dmpmqlog, and then use the provided jar utility to convert logs into human readable form. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Nov 01, 2016 11:32 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| Muhammad Haider wrote: |
| What do you say about this link: |
I'd say it's almost 2 years old, is filled with caveats (the one about it not always tracking persistent messages put out of syncpoint should especially give you pause) and is unsupported. And not even tested on v8. So if it fails (or doesn't have the message the client swears should be there) you're on your own.
If you're serious about doing that, buy this which does the same thing but is supported by the nice people at Cressida.
I've always preferred InQuest to ReQuest as there's less chance IBM will try and pull the run out from under it by changing the log format. Again.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| tczielke |
| Posted: Tue Nov 01, 2016 12:04 pm Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 941
Location: Illinois, USA
|
You may want to also look at the Application Activity Trace. It does provide the ability to record application activity including the message body. There is also some selection criteria you can use on what applications to trace.
_________________
Working with MQ since 2010. |
|
| Back to top |
|
|
| zpat |
| Posted: Wed Nov 02, 2016 4:00 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5856
Location: UK
|
You could use pub/sub to duplicate messages sent to a queue.
Then you can housekeep the duplicates. However high volumes really need to be logged to a database.
A message broker is ideal for this sort of thing.
_________________
Well, I don't think there is any question about it. It can only be attributable to human error. This sort of thing has cropped up before, and it has always been due to human error. |
|
| Back to top |
|
|
| Muhammad Haider |
| Posted: Thu Nov 03, 2016 1:47 am Post subject: |
|
|
Apprentice
Joined: 14 Mar 2011
Posts: 43
|
I was looking into Application Activity Trace. When I put message using rfhutil and generate Application Activity Records using MS0P support pac or the command: amqsact present in MQ's bin folder, I can see that messages put/get through rfhutil can be found in the trace file. Sample is given below. I want to know how can I read/format the data under Message Data tag, as currently its in Hex or some other form
Message sent:
<type1obj><member1>m1</member1><member3>2</member3></type1obj>
Excerpt from trace file:
001 2016-11-03 02:05:10 Put Ok 0000 (NONE) 4 SYSTEM.ADMIN.COMMAND.QUEUE
Highres Time 1478163910088952
Qmgr Op Duration 128
Put Options 0x00000044 [nosync newmsgid ]
Message Data
00000000 1000 0000 2400 0000 0300 0000 0d00 0000 |....$...........|
00000010 0100 0000 0100 0000 0000 0000 0000 0000 |................|
00000020 0300 0000 0400 0000 1800 0000 e007 0000 |............Ã ...|
00000030 0000 0000 0100 0000 2a00 0000 0300 0000 |........*.......|
00000040 1000 0000 1400 0000 0100 0000 0500 0000 |................|
00000050 1800 0000 ea03 0000 0200 0000 e007 0000 |....ê.......à ...|
00000060 1400 0000 |.... |
Msg Length 100
Known Dest Count 1
Unknown Dest Count 0
Invalid Dest Count 0
Object Type Queue
Object Queue Manager Name
Resolved Queue Name SYSTEM.ADMIN.COMMAND.QUEUE
Resolved Queue Manager QMR01
Resolved Local Queue Name SYSTEM.ADMIN.COMMAND.QUEUE
Resolved Local Queue Manager QMR01
Resolved Type Queue
Report 0
Msg Type 1
Expiry 10000
Format Name MQADMIN
Priority 2147483646
Persistence Non-persistent
Msg Id
414d5120514d52303120202020202020abfd1a5820002203 |AMQ QMR01 «ý.X .".|
Correl Id
000000000000000000000000000000000000000000000000 |........................|
Reply To Queue RFHUTIL.REPLY.QUEUE581AFDAB02220020
Reply To Queue Manager
Coded Char Set Id/mqiamo Monitor Delta 0
Encoding 546
Put Date 20161103
Put Time 09051007 |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Thu Nov 03, 2016 2:36 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| Muhammad Haider wrote: |
Sample is given below. I want to know how can I read/format the data under Message Data tag, as currently its in Hex or some other form
|
MQ does not know or care what format your data is in. It is a payload pure and simple.
IF you want to know the format then 'BLOB' comes closest.
Because of this 'format agnosticity', MQ can and does transmit message in XML, JSON, flat file (Cobol Copybook), My shopping list format, Vitors Coffee bill format and so on.
It just does not care and as someone who has written a Queue Manager (long before MQ was born), nor should it.
So to get a utility that decoded the HEX (aka BLOB) then you will have to roll your own.
That said, there are some that do a good job on the standard formats (XML, Cobol etc). One of these is Rfhutil.
It looks like you are going to have to face up that there is more than linekly nothing off the shelf that will satisfy your requirement. I'd take a step back and think how you woul meet the requirement in any other situation. How will the accuracy of your monitoring be audited and validated? etc etc etc
Then think about designing a solution that works from that point of view.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| exerk |
| Posted: Thu Nov 03, 2016 2:53 am Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
| smdavies99 wrote: |
| ...Because of this 'format agnosticity', MQ can and does transmit message in...Vitors Coffee bill format... |
I think that the tax alone on that bill would probably pay for socialised health care in the country within which he operates 
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| zpat |
| Posted: Thu Nov 03, 2016 3:23 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5856
Location: UK
|
A coffee tax would be a good idea, since the tax on tea worked out so well... 
_________________
Well, I don't think there is any question about it. It can only be attributable to human error. This sort of thing has cropped up before, and it has always been due to human error. |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Nov 03, 2016 5:03 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| zpat wrote: |
| A coffee tax would be a good idea, since the tax on tea worked out so well... |
No taxation without refills
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Nov 03, 2016 5:06 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| exerk wrote: |
| smdavies99 wrote: |
| ...Because of this 'format agnosticity', MQ can and does transmit message in...Vitors Coffee bill format... |
I think that the tax alone on that bill would probably pay for socialised health care in the country within which he operates |
One of the reasons I work where I do is that they provide free coffee.
The rise in operating costs over the last 5 years is a coincidence.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| exerk |
| Posted: Thu Nov 03, 2016 5:11 am Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
| Vitor wrote: |
| exerk wrote: |
| smdavies99 wrote: |
| ...Because of this 'format agnosticity', MQ can and does transmit message in...Vitors Coffee bill format... |
I think that the tax alone on that bill would probably pay for socialised health care in the country within which he operates |
One of the reasons I work where I do is that they provide free coffee.
The rise in operating costs over the last 5 years is a coincidence. |
For 'them' it's a tax write-off...
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| tczielke |
| Posted: Thu Nov 03, 2016 6:31 am Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 941
Location: Illinois, USA
|
If you are looking for an audit of the message, I would think the "Message Data" in amqsact would be enough. It is giving you the hex dump of the message with bytes translated in printable output to the right. If you want to act on that data in some way (e.g. reload it from the amqsact output onto a queue), the MH06 supportpac provides tools like mqtrcfrmt and MQFile2Msg that can do that.
_________________
Working with MQ since 2010. |
|
| Back to top |
|
|
|
|
