| Author |
Message
|
| abtimo |
 Posted: Thu Apr 16, 2020 9:27 pm Post subject: Simplest possible design loading unloading to remote qmgr |
 |
|
Novice
Joined: 24 Mar 2015
Posts: 23
|
Hello
I would like to set up a configuration on Win 10 with MQ v9.0 client installed, allowing to have 2 folders, C:\input and C:\output
The input folder needs to be watched by MQ and upon arrival of a file, this file must be picked up and inserted in the queue INPUTQ on remote QMGR
There is another requirement to watch the output queue OUTPUTQ on the same QMGR and to move the messages arrived to OUTPUTQ to C:\output
Now naively, I think the easiest and simplest way to do it is to use at scheduler and dmpmqmsg to run load and unload .bat files to post incoming messages to INPUTQ then backup them to a backup folder and retrieve acknowledgement from OUTPUTQ to c:\output
I wanted just to ask whether there is even easier and simpler way involving even less components and coding to fulfill the requirement?
Maybe I can trigger dmpmqmsg using some other way upon arrival of the input file to the input folder or upon arrival of the acknowledgement to the output queue? Looks like a use case for IIB, MQ and a file adapter, however  Thank you in advance. |
|
| Back to top |
|
 |
| hughson |
| Posted: Thu Apr 16, 2020 11:11 pm Post subject: |
|
|
Padawan
Joined: 09 May 2013
Posts: 1948
Location: Bay of Plenty, New Zealand
|
Putting aside for a moment the starting up of an application when something arrives in a folder. Do you know what format the files are going to show up in the folder with? And what format the messages on the queues need to be for whatever it is that is reading them? Same in the other direction. What format do the files need to be when read from the queue and written into the output directory. Using dmpmqmsg or QLOAD could certainly do the file to message and message to file, but only in certain formats.
Cheers,
Morag
_________________
Morag Hughson @MoragHughson
IBM MQ Technical Education Specialist
Get your IBM MQ training here!
MQGem Software |
|
| Back to top |
|
|
| abtimo |
| Posted: Thu Apr 16, 2020 11:59 pm Post subject: |
|
|
Novice
Joined: 24 Mar 2015
Posts: 23
|
How do you do Morag. Thank you for your observation. Never been to New Zealand, looks like very nice place. Afraid of earthquakes 
The format of both messages is ANSI-X12, it's in fact the connection to/from https://www.cbp.gov/
On the other hand now i think wether it is better to connect directly from client or putting a local QMGR will be an advantage...
| hughson wrote: |
Putting aside for a moment the starting up of an application when something arrives in a folder. Do you know what format the files are going to show up in the folder with? And what format the messages on the queues need to be for whatever it is that is reading them? Same in the other direction. What format do the files need to be when read from the queue and written into the output directory. Using dmpmqmsg or QLOAD could certainly do the file to message and message to file, but only in certain formats.
Cheers,
Morag |
|
|
| Back to top |
|
|
| hughson |
| Posted: Fri Apr 17, 2020 1:50 am Post subject: |
|
|
Padawan
Joined: 09 May 2013
Posts: 1948
Location: Bay of Plenty, New Zealand
|
| abtimo wrote: |
| Never been to New Zealand, looks like very nice place. Afraid of earthquakes |
It is a lovely place. I've lived here 5 years, and although there have been a number of big earthquakes dring that time, I have yet to feel one!
| abtimo wrote: |
| The format of both messages is ANSI-X12 |
What about the files - how will you load messages from a file to a message if the tool doing so is not written to understand this format. Are the files just a hex representation of the message for example?
| abtimo wrote: |
| On the other hand now i think wether it is better to connect directly from client or putting a local QMGR will be an advantage... |
If you can avoid the file to queue, and go directly from the application generating the data to a queue, by client or with a local queue manager, that will certainly be advantageous.
Cheers,
Morag
_________________
Morag Hughson @MoragHughson
IBM MQ Technical Education Specialist
Get your IBM MQ training here!
MQGem Software |
|
| Back to top |
|
|
| abtimo |
| Posted: Fri Apr 17, 2020 2:40 am Post subject: |
|
|
Novice
Joined: 24 Mar 2015
Posts: 23
|
As far as i understand the requirement now, the file will arrive to c:\input as a text file. I am supposed to just send it as text payload, no need to manage any MQ headers...
Then i am supposed to receive the acknowlegement and put it to c:\output without any header manipulation/correlation management either
in addition since the communication is supposed to be trough MQIPT i am curious whether it will be easy to configure connection for MQ Explorer to remote QMGR trough MQIPT to be able to debug the setup...
Alas it is requirement to go trough files in any case. The app that generates /consume these files can not do MQ directly
| hughson wrote: |
| abtimo wrote: |
| The format of both messages is ANSI-X12 |
What about the files - how will you load messages from a file to a message if the tool doing so is not written to understand this format. Are the files just a hex representation of the message for example?
| abtimo wrote: |
| On the other hand now i think wether it is better to connect directly from client or putting a local QMGR will be an advantage... |
If you can avoid the file to queue, and go directly from the application generating the data to a queue, by client or with a local queue manager, that will certainly be advantageous.
Cheers,
Morag |
|
|
| Back to top |
|
|
| hughson |
| Posted: Fri Apr 17, 2020 4:16 am Post subject: |
|
|
Padawan
Joined: 09 May 2013
Posts: 1948
Location: Bay of Plenty, New Zealand
|
If the file is just text, and the message is just to be the whole contents of that text file, dmpmqmsg can't do that as it expects the message to be in a particular format (unload a message and view it to see the format).
QLOAD (from MQGem) has added additional features that allow you to load up messages that are not in QLOAD format with the -n1 attribute. That would be one option for you. There will be other tools that will write a whole text file as a message too, just not dmpmqmsg.
Cheers,
Morag
_________________
Morag Hughson @MoragHughson
IBM MQ Technical Education Specialist
Get your IBM MQ training here!
MQGem Software |
|
| Back to top |
|
|
| abtimo |
| Posted: Fri Apr 17, 2020 5:04 am Post subject: |
|
|
Novice
Joined: 24 Mar 2015
Posts: 23
|
Indeed thank you for the reminder i see that i must plan for headers that mq requires, so mqdmpmsg must insert already enhanced ANSI-X12 with all necessary headers.
Most probably i can request people to send me these messages already in rtight format. (text + headers)
Otherwise as we are on windows maybe command line of rfhutil will be good enough ? Theoretically i can write java or even C code to process messages and put them to MQ but it will be less easy to maintain afterwards
| hughson wrote: |
If the file is just text, and the message is just to be the whole contents of that text file, dmpmqmsg can't do that as it expects the message to be in a particular format (unload a message and view it to see the format).
QLOAD (from MQGem) has added additional features that allow you to load up messages that are not in QLOAD format with the -n1 attribute. That would be one option for you. There will be other tools that will write a whole text file as a message too, just not dmpmqmsg.
Cheers,
Morag |
|
|
| Back to top |
|
|
| RogerLacroix |
| Posted: Fri Apr 17, 2020 12:49 pm Post subject: |
|
|
Jedi Knight
Joined: 15 May 2001
Posts: 3258
Location: London, ON Canada
|
You should have a look at the open source project called Universal File Mover (UFM). Its whole purpose is to move files to and from MQ (and FTP, HTTP, SCP, SFTP). It can even watch directories for files.
Regards,
Roger Lacroix
Capitalware Inc.
_________________
Capitalware: Transforming tomorrow into today.
Connected to MQ!
Twitter |
|
| Back to top |
|
|
| hughson |
| Posted: Fri Apr 17, 2020 5:30 pm Post subject: |
|
|
Padawan
Joined: 09 May 2013
Posts: 1948
Location: Bay of Plenty, New Zealand
|
| abtimo wrote: |
Indeed thank you for the reminder i see that i must plan for headers that mq requires, so mqdmpmsg must insert already enhanced ANSI-X12 with all necessary headers.
Most probably i can request people to send me these messages already in rtight format. (text + headers)
Otherwise as we are on windows maybe command line of rfhutil will be good enough ? Theoretically i can write java or even C code to process messages and put them to MQ but it will be less easy to maintain afterwards
| hughson wrote: |
If the file is just text, and the message is just to be the whole contents of that text file, dmpmqmsg can't do that as it expects the message to be in a particular format (unload a message and view it to see the format).
QLOAD (from MQGem) has added additional features that allow you to load up messages that are not in QLOAD format with the -n1 attribute. That would be one option for you. There will be other tools that will write a whole text file as a message too, just not dmpmqmsg.
Cheers,
Morag |
|
Please don't misunderstand my point. I was making the point that the format of dmpmqmsg files (input and output) are not plain message text.
If the only header you need is the MQMD, then any file to queue application can cope with that.
Just be aware the dmpmqmsg is not a tool for loading a plain text file to a queue. It works with its own specific text format.
QLOAD (an extension to dmpmqmsg) has recently added the ability to load a plain text file. And Roger has also given another example of a tool that can do the same.
Cheers,
Morag
_________________
Morag Hughson @MoragHughson
IBM MQ Technical Education Specialist
Get your IBM MQ training here!
MQGem Software |
|
| Back to top |
|
|
| abtimo |
| Posted: Sun Apr 19, 2020 5:32 am Post subject: |
|
|
Novice
Joined: 24 Mar 2015
Posts: 23
|
Thank You very much. Fantastic tool does just what i need. The only question whether i will be able to connect it trough MQIPT, otherwise it does all we need and even more. It has the functionality we required, and replaces a lot of custom coding, batches, scheduler or possible IIB/Mule community edition workflow. Very economical solution btw and aparently with the source code in case of fork/end of support etc.
| RogerLacroix wrote: |
You should have a look at the open source project called Universal File Mover (UFM). Its whole purpose is to move files to and from MQ (and FTP, HTTP, SCP, SFTP). It can even watch directories for files.
Regards,
Roger Lacroix
Capitalware Inc. |
|
|
| Back to top |
|
|
| abtimo |
| Posted: Sun Apr 19, 2020 5:38 am Post subject: |
|
|
Novice
Joined: 24 Mar 2015
Posts: 23
|
Yes indeed i used to work with mqdmpmsg while doing production support / troubleshooting doing unload of files to folders for analysis and small modification in case of payload error and accustomed to its custom text format and modifications using sed in case of. It is attractive because installed with mq already.
| hughson wrote: |
| abtimo wrote: |
Indeed thank you for the reminder i see that i must plan for headers that mq requires, so mqdmpmsg must insert already enhanced ANSI-X12 with all necessary headers.
Most probably i can request people to send me these messages already in rtight format. (text + headers)
Otherwise as we are on windows maybe command line of rfhutil will be good enough ? Theoretically i can write java or even C code to process messages and put them to MQ but it will be less easy to maintain afterwards
| hughson wrote: |
If the file is just text, and the message is just to be the whole contents of that text file, dmpmqmsg can't do that as it expects the message to be in a particular format (unload a message and view it to see the format).
QLOAD (from MQGem) has added additional features that allow you to load up messages that are not in QLOAD format with the -n1 attribute. That would be one option for you. There will be other tools that will write a whole text file as a message too, just not dmpmqmsg.
Cheers,
Morag |
|
Please don't misunderstand my point. I was making the point that the format of dmpmqmsg files (input and output) are not plain message text.
If the only header you need is the MQMD, then any file to queue application can cope with that.
Just be aware the dmpmqmsg is not a tool for loading a plain text file to a queue. It works with its own specific text format.
QLOAD (an extension to dmpmqmsg) has recently added the ability to load a plain text file. And Roger has also given another example of a tool that can do the same.
Cheers,
Morag |
|
|
| Back to top |
|
|
| RogerLacroix |
| Posted: Mon Apr 20, 2020 3:00 pm Post subject: |
|
|
Jedi Knight
Joined: 15 May 2001
Posts: 3258
Location: London, ON Canada
|
| abtimo wrote: |
| Thank You very much. Fantastic tool does just what i need. The only question whether i will be able to connect it trough MQIPT |
I'm not an IPT expert but I thought IPT was for queue manager to queue manager communication.
Regards,
Roger Lacroix
Capitalware Inc.
_________________
Capitalware: Transforming tomorrow into today.
Connected to MQ!
Twitter |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Mon Apr 20, 2020 6:03 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20729
Location: LI,NY
|
| RogerLacroix wrote: |
| abtimo wrote: |
| Thank You very much. Fantastic tool does just what i need. The only question whether i will be able to connect it trough MQIPT |
I'm not an IPT expert but I thought IPT was for queue manager to queue manager communication.
Regards,
Roger Lacroix
Capitalware Inc. |
Hi Roger, It is just as good for client to Qmgr communication... 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
|
|
