ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » General IBM MQ Support » Using dmpmqmsg to dump messages from a queue to a file

Post new topic  Reply to topic
 Using dmpmqmsg to dump messages from a queue to a file « View previous topic :: View next topic » 
Author Message
lchiu7
PostPosted: Wed Apr 27, 2022 2:26 pm    Post subject: Using dmpmqmsg to dump messages from a queue to a file Reply with quote

Novice

Joined: 27 Apr 2022
Posts: 10

We want to be able to send a file(s) out to a remote location. Source for the data comes from various sources but one must use MQ as they don't have other file transfer mechanisms available. So we have decided to standardise on using MQ to send files to a server and then read that queue and send the output via SFTP since the remote location does not support MQ.

We don't have MQ MFT installed but reading docs it seems we could use use dmpmqmsg to read a queue and create a file.

I also find fteCreateTransfer but that seems to require MFT is installed.

Is that possible and/or are there any gotchas? The file would be created daily.

Thanks
Back to top
View user's profile Send private message
gbaddeley
PostPosted: Wed Apr 27, 2022 3:13 pm    Post subject: Reply with quote

Jedi

Joined: 25 Mar 2003
Posts: 2492
Location: Melbourne, Australia

dmpmqmsg can get messages from a queue and write to a file, but the file is in a special format that is designed to be used by dmpmqmsg to put messages back onto a queue. I don't think this will help your requirement.

If you don't want to use MFT (requires an MQ Advanced license), you will need an app that can read files and put to queues, and the converse. You may need to take into account file size, character set, checksumming, security and how you are going to identify the file name in the message, etc.

There are ISV products that do this.
_________________
Glenn


Last edited by gbaddeley on Wed Apr 27, 2022 4:54 pm; edited 1 time in total
Back to top
View user's profile Send private message
lchiu7
PostPosted: Wed Apr 27, 2022 3:19 pm    Post subject: Reply with quote

Novice

Joined: 27 Apr 2022
Posts: 10

gbaddeley wrote:
dmpmqmsg can get messages from a queue and write to a file, but the file is in a special format that is designed to be used by dmpmqmsg to put messages back onto a queue. I don't think this will help your requirement.

If you don't want to use MFT (requires an MQ Advanced license), you will need an app that can read files and put to queues, and the converse. You may need to take into account file size, character set, checksumming, security and how you are going to identify the file name in the message, etc.


Thanks that is very helpful. So I guess I either look at the MFT licence and then use fteCreateTransfer or find some developers. I see that MQ can be called from Python which might open up the pool of developers for me.
Back to top
View user's profile Send private message
zpat
PostPosted: Fri Apr 29, 2022 12:47 am    Post subject: Reply with quote

Jedi Council

Joined: 19 May 2001
Posts: 5849
Location: UK

The q program (support pac MA01 originally) does what you need.

Can be obtained from Github

https://www.ibm.com/support/pages/node/6409580

Or a more recent version from the author at MQGEM.

https://www.mqgem.com/q.html
_________________
Well, I don't think there is any question about it. It can only be attributable to human error. This sort of thing has cropped up before, and it has always been due to human error.
Back to top
View user's profile Send private message
zpat
PostPosted: Sat Apr 30, 2022 5:57 am    Post subject: Reply with quote

Jedi Council

Joined: 19 May 2001
Posts: 5849
Location: UK

lchiu7 wrote:


Thanks that is very helpful. So I guess I either look at the MFT licence and then use fteCreateTransfer or find some developers. I see that MQ can be called from Python which might open up the pool of developers for me.


You can simply code your own M2F or F2M logic in any programming language that the MQI or JMS is available for.

Capitalware has plenty of sample source code.
_________________
Well, I don't think there is any question about it. It can only be attributable to human error. This sort of thing has cropped up before, and it has always been due to human error.
Back to top
View user's profile Send private message
RogerLacroix
PostPosted: Tue May 03, 2022 11:12 am    Post subject: Reply with quote

Jedi Knight

Joined: 15 May 2001
Posts: 3252
Location: London, ON Canada

You should have a look at the free open source project called Universal File Mover.

Regards,
Roger Lacroix
Capitalware Inc.
_________________
Capitalware: Transforming tomorrow into today.
Connected to MQ!
Twitter
Back to top
View user's profile Send private message Visit poster's website
lchiu7
PostPosted: Tue May 10, 2022 1:14 pm    Post subject: Reply with quote

Novice

Joined: 27 Apr 2022
Posts: 10

The problem with all of these solutions is that some development is required and it's not easy get our developers to put things into their already full queue. Our parent organisation also reads MQ messages (which we send them) and they said they use a utility called MBGateway to read the queue and create files. I have tried to find this (waiting on a clarification email from them) but Google shows nothing remotely like a utility that can do this so it might be a typo or something. Anybody heard of this utility?
Back to top
View user's profile Send private message
lchiu7
PostPosted: Tue May 10, 2022 7:27 pm    Post subject: Reply with quote

Novice

Joined: 27 Apr 2022
Posts: 10

lchiu7 wrote:
The problem with all of these solutions is that some development is required and it's not easy get our developers to put things into their already full queue. Our parent organisation also reads MQ messages (which we send them) and they said they use a utility called MBGateway to read the queue and create files. I have tried to find this (waiting on a clarification email from them) but Google shows nothing remotely like a utility that can do this so it might be a typo or something. Anybody heard of this utility?


Just discovered we have the z/OS version of Q installed. I had no idea. So even if the z/OS licence cannot be transferred to Windows or Linux, at least we already have the product so vendor qualification is not going to be too hard I hope. That will solve our problems
Back to top
View user's profile Send private message
gbaddeley
PostPosted: Wed May 11, 2022 3:46 pm    Post subject: Reply with quote

Jedi

Joined: 25 Mar 2003
Posts: 2492
Location: Melbourne, Australia

lchiu7 wrote:
Just discovered we have the z/OS version of Q installed. I had no idea. So even if the z/OS licence cannot be transferred to Windows or Linux, at least we already have the product so vendor qualification is not going to be too hard I hope. That will solve our problems

You may wish to confirm that Q meets your requirements and there is support for Q if it fails in production, before plowing ahead with vendor qualification.
_________________
Glenn
Back to top
View user's profile Send private message
lchiu7
PostPosted: Wed May 11, 2022 7:56 pm    Post subject: Reply with quote

Novice

Joined: 27 Apr 2022
Posts: 10

Well on further research it looks like I can use dmpmqmsg

https://www.ibm.com/docs/en/ibm-mq/8.0?topic=commands-dmpmqmsg

Formerly the IBM® MQ queue load and unload utility.

Purpose

Use the dmpmqmsg utility to copy or move the contents of a queue, or its messages, to a file.

The docs don't say the output format is particularly special - just a file

Going to ask one of our MQ admin folks to try it out.
Back to top
View user's profile Send private message
hughson
PostPosted: Wed May 11, 2022 9:34 pm    Post subject: Reply with quote

Padawan

Joined: 09 May 2013
Posts: 1914
Location: Bay of Plenty, New Zealand

lchiu7 wrote:
The docs don't say the output format is particularly special - just a file

Then the docs are truly lacking. The output format of dmpmqmsg is the same as the output format of QLOAD. They are designed to be understood by the same program to load the messages up again.

Here is an example of the output just to give you some idea.

Code:
* QLOAD Version:9.1.1 Created:Fri Aug 14 18:52:17 2020
* Qmgr  = MQG1
* Queue = ADMIN.TOOL.INFO

A VER 2
A RPT 0
A MST 8
A EXP -1
A FDB 0
A ENC 546
A CCS 437
A FMT MQSTR   
A PRI 0
A PER 1
A MSI 414D51204D5147312020202020202020DF0F7F5E24F8CA07
A COI 000000000000000000000000000000000000000000000000
A BOC 0
A RTQ                                                 
A RTM MQG1                                           
A USR mqgemusr   
A ACC 160105150000001AFA5FFE70975006C3A1C633E903000000000000000000000B
A AIX 2020202020202020202020202020202020202020202020202020202020202020
A PAT 11
A PAN d:\nttools\mqedit.exe       
A PTD 20200402
A PTT 05294052
A AOX 20202020
A GRP 000000000000000000000000000000000000000000000000
A MSQ 1
A OFF 0
A MSF 0
A ORL -1
X 3C50726F647563743E3C4E616D653E4D515343583C2F4E616D
X 653E3C576562706167653E68747470733A2F2F7777772E6D71
X 67656D2E636F6D2F6D717363782E68746D6C3C2F5765627061
X 67653E3C506C6174666F726D733E3C506C6174666F726D3E57
X 696E646F77733C2F506C6174666F726D3E3C506C6174666F72
X 6D3E4C696E7578207838362036346269743C2F506C6174666F
X 726D3E3C506C6174666F726D3E506F7765724C696E75782042
X 452036346269743C2F506C6174666F726D3E3C506C6174666F
X 726D3E4149582036346269743C2F506C6174666F726D3E3C2F
X 506C6174666F726D733E3C4C6963656E6365733E3C4C696365
X 6E63653E3C4C6963547970653E456D6572616C643C2F4C6963
X 547970653E3C50726963653E4E5A44203131303C2F50726963
X 653E3C2F4C6963656E63653E3C4C6963656E63653E3C4C6963
X 547970653E527562793C2F4C6963547970653E3C5072696365
X 3E4E5A44203331303C2F50726963653E3C2F4C6963656E6365
X 3E3C4C6963656E63653E3C4C6963547970653E536170706869
X 72653C2F4C6963547970653E3C50726963653E4E5A44203331
X 303C2F50726963653E3C2F4C6963656E63653E3C4C6963656E
X 63653E3C4C6963547970653E4469616D6F6E643C2F4C696354
X 7970653E3C50726963653E4E5A44203935303C2F5072696365
X 3E3C2F4C6963656E63653E3C4C6963656E63653E3C4C696354
X 7970653E456E74657270726973653C2F4C6963547970653E3C
X 50726963653E4E5A4420323835303C2F50726963653E3C2F4C
X 6963656E63653E3C2F4C6963656E6365733E3C2F50726F6475
X 63743E

_________________
Morag Hughson @MoragHughson
IBM MQ Technical Education Specialist
Get your IBM MQ training here!
MQGem Software
Back to top
View user's profile Send private message Visit poster's website
Display posts from previous:   
Post new topic  Reply to topic Page 1 of 1

MQSeries.net Forum Index » General IBM MQ Support » Using dmpmqmsg to dump messages from a queue to a file
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.