ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » General IBM MQ Support » Simplest possible design loading unloading to remote qmgr

Post new topic  Reply to topic
 Simplest possible design loading unloading to remote qmgr « View previous topic :: View next topic » 
Author Message
abtimo
PostPosted: Thu Apr 16, 2020 9:27 pm    Post subject: Simplest possible design loading unloading to remote qmgr Reply with quote

Novice

Joined: 24 Mar 2015
Posts: 23

Hello
I would like to set up a configuration on Win 10 with MQ v9.0 client installed, allowing to have 2 folders, C:\input and C:\output
The input folder needs to be watched by MQ and upon arrival of a file, this file must be picked up and inserted in the queue INPUTQ on remote QMGR
There is another requirement to watch the output queue OUTPUTQ on the same QMGR and to move the messages arrived to OUTPUTQ to C:\output

Now naively, I think the easiest and simplest way to do it is to use at scheduler and dmpmqmsg to run load and unload .bat files to post incoming messages to INPUTQ then backup them to a backup folder and retrieve acknowledgement from OUTPUTQ to c:\output
I wanted just to ask whether there is even easier and simpler way involving even less components and coding to fulfill the requirement?

Maybe I can trigger dmpmqmsg using some other way upon arrival of the input file to the input folder or upon arrival of the acknowledgement to the output queue? Looks like a use case for IIB, MQ and a file adapter, however Thank you in advance.
Back to top
View user's profile Send private message
hughson
PostPosted: Thu Apr 16, 2020 11:11 pm    Post subject: Reply with quote

Padawan

Joined: 09 May 2013
Posts: 1914
Location: Bay of Plenty, New Zealand

Putting aside for a moment the starting up of an application when something arrives in a folder. Do you know what format the files are going to show up in the folder with? And what format the messages on the queues need to be for whatever it is that is reading them? Same in the other direction. What format do the files need to be when read from the queue and written into the output directory. Using dmpmqmsg or QLOAD could certainly do the file to message and message to file, but only in certain formats.

Cheers,
Morag
_________________
Morag Hughson @MoragHughson
IBM MQ Technical Education Specialist
Get your IBM MQ training here!
MQGem Software
Back to top
View user's profile Send private message Visit poster's website
abtimo
PostPosted: Thu Apr 16, 2020 11:59 pm    Post subject: Reply with quote

Novice

Joined: 24 Mar 2015
Posts: 23

How do you do Morag. Thank you for your observation. Never been to New Zealand, looks like very nice place. Afraid of earthquakes
The format of both messages is ANSI-X12, it's in fact the connection to/from https://www.cbp.gov/

On the other hand now i think wether it is better to connect directly from client or putting a local QMGR will be an advantage...

hughson wrote:
Putting aside for a moment the starting up of an application when something arrives in a folder. Do you know what format the files are going to show up in the folder with? And what format the messages on the queues need to be for whatever it is that is reading them? Same in the other direction. What format do the files need to be when read from the queue and written into the output directory. Using dmpmqmsg or QLOAD could certainly do the file to message and message to file, but only in certain formats.

Cheers,
Morag
Back to top
View user's profile Send private message
hughson
PostPosted: Fri Apr 17, 2020 1:50 am    Post subject: Reply with quote

Padawan

Joined: 09 May 2013
Posts: 1914
Location: Bay of Plenty, New Zealand

abtimo wrote:
Never been to New Zealand, looks like very nice place. Afraid of earthquakes

It is a lovely place. I've lived here 5 years, and although there have been a number of big earthquakes dring that time, I have yet to feel one!

abtimo wrote:
The format of both messages is ANSI-X12

What about the files - how will you load messages from a file to a message if the tool doing so is not written to understand this format. Are the files just a hex representation of the message for example?

abtimo wrote:
On the other hand now i think wether it is better to connect directly from client or putting a local QMGR will be an advantage...

If you can avoid the file to queue, and go directly from the application generating the data to a queue, by client or with a local queue manager, that will certainly be advantageous.

Cheers,
Morag
_________________
Morag Hughson @MoragHughson
IBM MQ Technical Education Specialist
Get your IBM MQ training here!
MQGem Software
Back to top
View user's profile Send private message Visit poster's website
abtimo
PostPosted: Fri Apr 17, 2020 2:40 am    Post subject: Reply with quote

Novice

Joined: 24 Mar 2015
Posts: 23

As far as i understand the requirement now, the file will arrive to c:\input as a text file. I am supposed to just send it as text payload, no need to manage any MQ headers...
Then i am supposed to receive the acknowlegement and put it to c:\output without any header manipulation/correlation management either

in addition since the communication is supposed to be trough MQIPT i am curious whether it will be easy to configure connection for MQ Explorer to remote QMGR trough MQIPT to be able to debug the setup...

Alas it is requirement to go trough files in any case. The app that generates /consume these files can not do MQ directly

hughson wrote:


abtimo wrote:
The format of both messages is ANSI-X12

What about the files - how will you load messages from a file to a message if the tool doing so is not written to understand this format. Are the files just a hex representation of the message for example?

abtimo wrote:
On the other hand now i think wether it is better to connect directly from client or putting a local QMGR will be an advantage...

If you can avoid the file to queue, and go directly from the application generating the data to a queue, by client or with a local queue manager, that will certainly be advantageous.

Cheers,
Morag
Back to top
View user's profile Send private message
hughson
PostPosted: Fri Apr 17, 2020 4:16 am    Post subject: Reply with quote

Padawan

Joined: 09 May 2013
Posts: 1914
Location: Bay of Plenty, New Zealand

If the file is just text, and the message is just to be the whole contents of that text file, dmpmqmsg can't do that as it expects the message to be in a particular format (unload a message and view it to see the format).

QLOAD (from MQGem) has added additional features that allow you to load up messages that are not in QLOAD format with the -n1 attribute. That would be one option for you. There will be other tools that will write a whole text file as a message too, just not dmpmqmsg.

Cheers,
Morag
_________________
Morag Hughson @MoragHughson
IBM MQ Technical Education Specialist
Get your IBM MQ training here!
MQGem Software
Back to top
View user's profile Send private message Visit poster's website
abtimo
PostPosted: Fri Apr 17, 2020 5:04 am    Post subject: Reply with quote

Novice

Joined: 24 Mar 2015
Posts: 23

Indeed thank you for the reminder i see that i must plan for headers that mq requires, so mqdmpmsg must insert already enhanced ANSI-X12 with all necessary headers.
Most probably i can request people to send me these messages already in rtight format. (text + headers)
Otherwise as we are on windows maybe command line of rfhutil will be good enough ? Theoretically i can write java or even C code to process messages and put them to MQ but it will be less easy to maintain afterwards

hughson wrote:
If the file is just text, and the message is just to be the whole contents of that text file, dmpmqmsg can't do that as it expects the message to be in a particular format (unload a message and view it to see the format).

QLOAD (from MQGem) has added additional features that allow you to load up messages that are not in QLOAD format with the -n1 attribute. That would be one option for you. There will be other tools that will write a whole text file as a message too, just not dmpmqmsg.

Cheers,
Morag
Back to top
View user's profile Send private message
RogerLacroix
PostPosted: Fri Apr 17, 2020 12:49 pm    Post subject: Reply with quote

Jedi Knight

Joined: 15 May 2001
Posts: 3252
Location: London, ON Canada

You should have a look at the open source project called Universal File Mover (UFM). Its whole purpose is to move files to and from MQ (and FTP, HTTP, SCP, SFTP). It can even watch directories for files.

Regards,
Roger Lacroix
Capitalware Inc.
_________________
Capitalware: Transforming tomorrow into today.
Connected to MQ!
Twitter
Back to top
View user's profile Send private message Visit poster's website
hughson
PostPosted: Fri Apr 17, 2020 5:30 pm    Post subject: Reply with quote

Padawan

Joined: 09 May 2013
Posts: 1914
Location: Bay of Plenty, New Zealand

abtimo wrote:
Indeed thank you for the reminder i see that i must plan for headers that mq requires, so mqdmpmsg must insert already enhanced ANSI-X12 with all necessary headers.
Most probably i can request people to send me these messages already in rtight format. (text + headers)
Otherwise as we are on windows maybe command line of rfhutil will be good enough ? Theoretically i can write java or even C code to process messages and put them to MQ but it will be less easy to maintain afterwards

hughson wrote:
If the file is just text, and the message is just to be the whole contents of that text file, dmpmqmsg can't do that as it expects the message to be in a particular format (unload a message and view it to see the format).

QLOAD (from MQGem) has added additional features that allow you to load up messages that are not in QLOAD format with the -n1 attribute. That would be one option for you. There will be other tools that will write a whole text file as a message too, just not dmpmqmsg.

Cheers,
Morag

Please don't misunderstand my point. I was making the point that the format of dmpmqmsg files (input and output) are not plain message text.

If the only header you need is the MQMD, then any file to queue application can cope with that.

Just be aware the dmpmqmsg is not a tool for loading a plain text file to a queue. It works with its own specific text format.

QLOAD (an extension to dmpmqmsg) has recently added the ability to load a plain text file. And Roger has also given another example of a tool that can do the same.

Cheers,
Morag
_________________
Morag Hughson @MoragHughson
IBM MQ Technical Education Specialist
Get your IBM MQ training here!
MQGem Software
Back to top
View user's profile Send private message Visit poster's website
abtimo
PostPosted: Sun Apr 19, 2020 5:32 am    Post subject: Reply with quote

Novice

Joined: 24 Mar 2015
Posts: 23

Thank You very much. Fantastic tool does just what i need. The only question whether i will be able to connect it trough MQIPT, otherwise it does all we need and even more. It has the functionality we required, and replaces a lot of custom coding, batches, scheduler or possible IIB/Mule community edition workflow. Very economical solution btw and aparently with the source code in case of fork/end of support etc.

RogerLacroix wrote:
You should have a look at the open source project called Universal File Mover (UFM). Its whole purpose is to move files to and from MQ (and FTP, HTTP, SCP, SFTP). It can even watch directories for files.

Regards,
Roger Lacroix
Capitalware Inc.
Back to top
View user's profile Send private message
abtimo
PostPosted: Sun Apr 19, 2020 5:38 am    Post subject: Reply with quote

Novice

Joined: 24 Mar 2015
Posts: 23

Yes indeed i used to work with mqdmpmsg while doing production support / troubleshooting doing unload of files to folders for analysis and small modification in case of payload error and accustomed to its custom text format and modifications using sed in case of. It is attractive because installed with mq already.

hughson wrote:
abtimo wrote:
Indeed thank you for the reminder i see that i must plan for headers that mq requires, so mqdmpmsg must insert already enhanced ANSI-X12 with all necessary headers.
Most probably i can request people to send me these messages already in rtight format. (text + headers)
Otherwise as we are on windows maybe command line of rfhutil will be good enough ? Theoretically i can write java or even C code to process messages and put them to MQ but it will be less easy to maintain afterwards

hughson wrote:
If the file is just text, and the message is just to be the whole contents of that text file, dmpmqmsg can't do that as it expects the message to be in a particular format (unload a message and view it to see the format).

QLOAD (from MQGem) has added additional features that allow you to load up messages that are not in QLOAD format with the -n1 attribute. That would be one option for you. There will be other tools that will write a whole text file as a message too, just not dmpmqmsg.

Cheers,
Morag

Please don't misunderstand my point. I was making the point that the format of dmpmqmsg files (input and output) are not plain message text.

If the only header you need is the MQMD, then any file to queue application can cope with that.

Just be aware the dmpmqmsg is not a tool for loading a plain text file to a queue. It works with its own specific text format.

QLOAD (an extension to dmpmqmsg) has recently added the ability to load a plain text file. And Roger has also given another example of a tool that can do the same.

Cheers,
Morag
Back to top
View user's profile Send private message
RogerLacroix
PostPosted: Mon Apr 20, 2020 3:00 pm    Post subject: Reply with quote

Jedi Knight

Joined: 15 May 2001
Posts: 3252
Location: London, ON Canada

abtimo wrote:
Thank You very much. Fantastic tool does just what i need. The only question whether i will be able to connect it trough MQIPT

I'm not an IPT expert but I thought IPT was for queue manager to queue manager communication.

Regards,
Roger Lacroix
Capitalware Inc.
_________________
Capitalware: Transforming tomorrow into today.
Connected to MQ!
Twitter
Back to top
View user's profile Send private message Visit poster's website
fjb_saper
PostPosted: Mon Apr 20, 2020 6:03 pm    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY

RogerLacroix wrote:
abtimo wrote:
Thank You very much. Fantastic tool does just what i need. The only question whether i will be able to connect it trough MQIPT

I'm not an IPT expert but I thought IPT was for queue manager to queue manager communication.

Regards,
Roger Lacroix
Capitalware Inc.

Hi Roger, It is just as good for client to Qmgr communication...
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
Display posts from previous:   
Post new topic  Reply to topic Page 1 of 1

MQSeries.net Forum Index » General IBM MQ Support » Simplest possible design loading unloading to remote qmgr
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.