| Author |
Message
|
| fswoc |
 Posted: Thu Nov 23, 2006 1:51 pm Post subject: How to convert messages with a blank message format |
 |
|
Newbie
Joined: 23 Nov 2006
Posts: 2
Location: Toronto, Ontario
|
Hi all.
Here's where I'm stuck: ex. using q.exe from ma01 to copy messages from source to dest....
| Code: |
q.exe -iSOURCE.QUEUE -oDEST.QUEUE -s
MQSeries Q Program by Paul Clarke [ V4.5 Build:Sep 15 2006 ]
Connecting ...connected to 'MYQUEUEMGR'.
MQGET on object 'SOURCE.QUEUE' returned 2110 Format error.. |
Other posts have suggested that the cause of my "2110 Format error" is due to the message format being blank... and browsing the messages via Websphere Explorer, I can see that indeed that is the case.
Another post suggested IH03 as a investigation tool - this utility helped, allowing me to confirm the message content, etc.
The task I'm trying to complete is to set up a basic message duplication process. Initially, I've used q.exe to send the same data to 2 internal queues. Source queue is EBCDIC, internal queues need to be in ASCII. I've successfully done this on several other queues like this however, they have all had the format set to MQSTR by the sender app, and I was using the -c437 option for q.exe to convert to ASCII when copying to my other local queues.
So my question is... what are some options to fix this?
1) a different -c option for q.exe (tried this, get the same error, however this seems to be the expected behaviour per info in other posts when dealing with a blank message format)
2) can I use another utility to set the format to MQSTR prior to copying / converting the messages with q.exe?
3) change the sender code to add MQSTR as the MQMD format ... but I don't have access to the code that creates the message on the sender queue;
Sender platform is main-frame, I'm on Windows w/Websphere 5.3
Suggestions?
Thanks! |
|
| Back to top |
|
 |
| zpat |
| Posted: Thu Nov 23, 2006 2:10 pm Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
Format defaults to NONE (aka BINARY) - (Yeah thanks a lot IBM!).
So fix the original program - if it's from a vendor make them fix it.
Anything else is a kludge that will cause more problems long term.
The other ways that you could change the MQMD.format would be to pass it through a message broker, pass it through your own program, or write an exit of some sort. |
|
| Back to top |
|
|
| Nigelg |
| Posted: Fri Nov 24, 2006 1:05 am Post subject: |
|
|
Grand Master
Joined: 02 Aug 2004
Posts: 1046
|
| Quote: |
| Yeah thanks a lot IBM! |
A somewhat odd comment...
It is the responsibility of the programmer to set the MD format to match the msg format, not for IBM to default a format which may or may not match the msg data.
A blank format ensures that the msg data cannot be converted inadvertently.
_________________
MQSeries.net helps those who help themselves.. |
|
| Back to top |
|
|
| zpat |
| Posted: Fri Nov 24, 2006 1:11 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
I would rather the default was format STRING and GMO_CONVERT - they are so many examples (on here and in my experience) where the cross-platform nature of MQ is compromised because of the IBM defaults and the fact that developers don't realise they have to override them.
This is compounded when MQ applications are delivered by a third-party who have only ever tested them on one platform and fail to understand why format (and convert) are vital to cross-platform sites, especially with the dreaded EBCDIC on the mainframe.
When a particular option is needed 99.999% of the time (certainly in my experience which goes back to when MQ was first released) then it helps to make it the default. Or allow the customer to set the MQI default at the queue manager level. Most modern messages are XML (STRING!) and even if we send binary data we BASE64 encode it as a string.
The real world is different from the inside of an IBM lab! I have to spend a lot of my time chasing developers and vendors and explaining that the defaults are dangerous and they have to specify MQSTR and GMO_CONVERT explicitly despite the main purpose of MQ being cross-platform messaging. |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Nov 24, 2006 1:18 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| zpat wrote: |
When a particular option is needed 99.999% of the time (certainly in my experience which goes back to when MQ was first released) then it helps to make it the default. Or allow the customer to set the MQI default at the queue manager level.
|
Throwing my 2 cents into the ring, over the last 10 years or so of my MQ use I don't remember using a message that wasn't string format, even within the same platform. I can see the rational of having no format as a default, thereby pushing the responsibility from the software to the user, but it is something of a pain.
I like the idea of being able to set it at queue manager level. Is there a suggestion box at IBM?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Nigelg |
| Posted: Fri Nov 24, 2006 3:24 am Post subject: |
|
|
Grand Master
Joined: 02 Aug 2004
Posts: 1046
|
Whoa, seem to have touched a nerve here!!!
| Quote: |
| The real world is different from the inside of an IBM lab! |
Isn't it MORE important to take responsibility for correct coding in the 'real world' than in a lab environment?
| Quote: |
| I don't remember using a message that wasn't string format |
I suppose you have used the command server, or Explorer, or any number of commercially available or home-grown admin tools?
The format of msgs used by 99.9999% of such apps is MQFMT_ADMIN.[/i][/quote]
_________________
MQSeries.net helps those who help themselves.. |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Nov 24, 2006 3:39 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| Nigelg wrote: |
I suppose you have used the command server, or Explorer, or any number of commercially available or home-grown admin tools?
The format of msgs used by 99.9999% of such apps is MQFMT_ADMIN. |
Point taken, but I thought the discussion was around the use of MQMD in an application environment? Certainly that was the point I was making & to extend it to include your point, 99% of the messages going through a queue manager (by volume) will be application (and probably string based) rather than admin messages.
Also the default setting of the MQMD under discussion here is irrelevant to such tools as they clearly set the Format and are therefore immune to whatever default is set. These tools by their nature are written by more experienced people, versed in MQ and the proper use of the MQMD fields. I therefore stand by my previous assertion that string format would be a more widely applicable default, at least as a queue manager setting.
To pick up your comment:
| Nigelg wrote: |
| It is the responsibility of the programmer to set the MD format to match the msg format |
Indeed, but we don't currently expect them to know the CCSID they should be using either. This is allowed to default to a queue manager setting; it's not such a leap to allow format to similarly default. There will be occassions where the format is incompatable or inappropriate for the message content; these will occur with the same frequency as messages are currently sent with the wrong CCSID information.
The question "The Real World is not like an IBM Lab - Discuss" sounds like something from an A-Level Socialology exam & I don't propose to go near it except to say that in my world programmers are recruited to a budget not a quality, have little interest or training in the concept of messaging and the more message functions I can move into the admin layer the better!
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Fri Nov 24, 2006 8:55 am Post subject: |
|
|
Guest
|
The MQ product comes with structures that have inital values, not defaults. From the Application Programmers Reference: The initial value of this field is MQFMT_NONE.
It is up to the programmer to ensure (set) values in MQ structure fields meet business requirements. This is consistent with all data fields in all programming languages.
Although not a best practice (more like a worst practice), if your site doesn't like the initial value supplied by IBM (in the copybook/include-file), then change it. |
|
| Back to top |
|
|
| wschutz |
| Posted: Fri Nov 24, 2006 10:34 am Post subject: |
|
|
Jedi Knight
Joined: 02 Jun 2005
Posts: 3316
Location: IBM (retired)
|
| bruce2359 wrote: |
| Although not a best practice (more like a worst practice), if your site doesn't like the initial value supplied by IBM (in the copybook/include-file), then change it. |
Yeah, I would never recommend changing the WMQ supplied copybook/header files ....
_________________
-wayne |
|
| Back to top |
|
|
| zpat |
| Posted: Fri Nov 24, 2006 11:29 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
Like I said, leaving all fields to default will work fine on a single platform so the program will appear to work initially, as far as the developer is concerned.
It's only when you later want to process the same messages on a Java client or a mainframe that ASCII messages can't be converted (for example).
If developers understood the issue, of course they would set the format.
You can see from the postings here that some developers are attempting to use MQ without training and are often located far from their more experienced colleagues.
This BB has lots of posting asking "How do I convert my messages?".
The best solution would be to make it a queue manager property that can be set - just like the default CCSID value is. |
|
| Back to top |
|
|
| mvic |
| Posted: Fri Nov 24, 2006 3:21 pm Post subject: |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| zpat wrote: |
If developers understood the issue, of course they would set the format.
You can see from the postings here that some developers are attempting to use MQ without training and are often located far from their more experienced colleagues. |
Given that a default has to be chosen, MQ can't please all people all the time. FWIW I think binary is the safest default, given the very important MQI principle that the message data is the application's responsibility. As for MQGMO_CONVERT, this implies additional processing cost on an MQGET call. Again, it's impossible to please everyone when choosing a default, sorry.
| Quote: |
| This BB has lots of posting asking "How do I convert my messages?". |
It's in the documentation, I believe [1]. If the documentation needs to be improved, please do contribute comments via the route that IBM provides (there are "Feedback" links at the bottom of some (most/all?) pages of the Information Center).
| Quote: |
| The best solution would be to make it a queue manager property that can be set - just like the default CCSID value is. |
One difficulty with this: the contents of a message are the responsibility of the application. It would lead to very strange effects if developers allowed a queue manager default to be used for this, and later an administrator changed the default.
[1] http://publib.boulder.ibm.com/infocenter/wmqv6/v6r0/topic/com.ibm.mq.csqzal.doc/chmsgt.htm |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Fri Nov 24, 2006 4:19 pm Post subject: |
|
|
Guest
|
The MQ Application Programmers Reference and MQ Application Programmers Guide are very clear about the main points touched on in this post; namely:
1. structures have 'initial values,' not defaults. As such, these values are not reset (do not revert) to a value after each use (or before the next use); and
2. the programmers is responsible for setting values before use of the structure; or leaving the initial values unchanged (which is like setting the values, isn't it?).
What isn't documented all that well is that it is our responsibility (to the organization, to stockholders, etc.) to protect our systems from errant* applications, application programmers, vendors, managers and users. Sounds like we are doing just that.
*errant defined: straying from the right course or from accepted standards; "errant youngsters." Source: Wordnet.princeton.edu/perl/webwn
As a side note: programmers are trainable. |
|
| Back to top |
|
|
| zpat |
| Posted: Sat Nov 25, 2006 1:51 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
| You are ignoring reality. A not uncommon attitude in IBM. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Sat Nov 25, 2006 7:51 am Post subject: |
|
|
Guest
|
| zpat wrote: |
| You are ignoring reality. A not uncommon attitude in IBM. |
Recognizing reality is different from ignoring reality. I agree (recognize) that programmers misuse MQ and other tools.
As discussed in this post, sysadmins should not take the wrong action (worst practice) to compensate for poor programming.
Assume you have only two MQ applications: one written by a programmer who understands MQ, and one programmer MQ-unaware. Should you (or IBM) change the product to satisfy the ignorant programmer?
As designed, MQ delivers programmer-created messages unchanged UNLESS the application program takes some kind of action (like changing the message format field of the MD; or coding MQGET with CONVERT). If this doesn't work because your programmers don't understand MQ, sysadmins can't fix this.
I am sympathetic. I recognize your reality. I understand that your reality is uncomfortable. |
|
| Back to top |
|
|
| zpat |
| Posted: Sat Nov 25, 2006 8:42 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
To give another example. I had a vendor whose programmers were actually very good and even read the IBM MQ manuals when coding their adapter, but they only tested on Windows using local queues and didn't pick up on quite a few points like not opening output queues for input and output (try it on a remote queue and find out why!).
However those IBM manuals don't make a point of recommending using format MQSTR for cross-platform text messages. Most people assume the default that IBM have chosen is actually the best option.
So how about IBM explaining clearly in the manuals that text message applications should in 99.999% of cases use MQSTR even though it isn't the default, and should code MQGMO_CONVERT even though it isn't the default? |
|
| Back to top |
|
|
|
|
