ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum IndexGeneral IBM MQ SupportMQ data structure alignment

Post new topicReply to topic Goto page 1, 2  Next
MQ data structure alignment View previous topic :: View next topic
Would this be useful to you?
greatly
25%
25% [ 1 ]
some
0%
0% [ 0 ]
not a lot
0%
0% [ 0 ]
not at all
75%
75% [ 3 ]
Total Votes : 4
Author Message
jojo2020
PostPosted: Tue Jun 02, 2020 8:17 am Post subject: MQ data structure alignment Reply with quote

Newbie

Joined: 02 Jun 2020
Posts: 5

Hi,

I would like to know how people here maintain and keep aligned their MQ data structures to their published IBM counterparts?

The documentation is mostly flat and it takes investigation and effort to arrange the published types into objects for correct use within your code.

The initial grouping of types into objects and additional maintenance to keep them aligned with future changes can introduce defects down the line if care and attention is not taken.

Wouldn't it be easier to the customer if IBM were to supply a model in some format (e.g. YAML) that describe the objects and types?

Doesn't IBM model MQ in one of their tools (e.g. Rhapsody) and from there have the ability to export the data into some useful format easily?

Your feedback is appreciated.

UPDATE: 06/06/2020

IBM doesn't ship a model (or table) of fields that maps which constants are valid for each field I am told.

Example:
Some MQCOMPRESS_ constants are valid for the CurHdrCompression and HdrCompList fields and others for the CurMsgCompression and MsgCompList fields of the MQ_CHANNEL_EXIT and MQCD structures.

Which constants are valid for which fields is documented in...
https://www.ibm.com/support/knowledgecenter/SSFKSJ_9.0.0/com.ibm.mq.ref.dev.doc/q108440_.htm
https://www.ibm.com/support/knowledgecenter/SSFKSJ_9.0.0/com.ibm.mq.ref.dev.doc/q108620_.htm
https://www.ibm.com/support/knowledgecenter/SSFKSJ_9.0.0/com.ibm.mq.ref.dev.doc/q109400_.htm
https://www.ibm.com/support/knowledgecenter/SSFKSJ_9.0.0/com.ibm.mq.ref.dev.doc/q109410_.htm

Couldn't the above structure and type relationship be modeled in YAML ?
Please forgive my ignorance as I am entirely new to MQ.


Last edited by jojo2020 on Sat Jun 06, 2020 2:26 am; edited 1 time in total
Back to top
View user's profile Send private message
bruce2359
PostPosted: Tue Jun 02, 2020 8:47 am Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 8715
Location: US: west coast, almost. Otherwise, enroute.

I'm not sure I understand your point here. An MQ message or publication can be of any data type. MQ is agnostic to the content of your messages/publications.

If you seek structure to the data , perhaps XML is a possibility.
_________________
My life flows on in endless song;
How can I keep from singing?
Back to top
View user's profile Send private message
jojo2020
PostPosted: Tue Jun 02, 2020 9:10 am Post subject: Reply with quote

Newbie

Joined: 02 Jun 2020
Posts: 5

I agree, an MQ message or publication can be of any data type, and yes I do wish for them to structure the published data which is why I suggested YAML or some format.
You understood correctly.
Would this be of benefit to you?
Back to top
View user's profile Send private message
Vitor
PostPosted: Tue Jun 02, 2020 9:24 am Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 25995
Location: Texas, USA

I likewise don't understand your point.

IBM supplies machine readable formats for the parts of the message they control (MQMD and so forth). I'm not sure how much value these portions would have in XML or YAML but if enough people agree with the RFE you'd raise it would get done.

As my worthy associate points out, once you get into the payload all bets are off and the content is entirely at the discretion of the application designer. IBM could not and should not provide any model (and implicit restriction) for this portion.

Perhaps if you provided a use case for the kind of additional description you're describing?
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
RogerLacroix
PostPosted: Tue Jun 02, 2020 11:32 am Post subject: Re: MQ data structure alignment Reply with quote

Jedi Knight

Joined: 15 May 2001
Posts: 3163
Location: London, ON Canada

jojo2020 wrote:
I would like to know how people here maintain and keep aligned their MQ data structures to their published IBM counterparts?

What? I'm confused. You want IBM to know about the format/structure of your application's payload? Why? IBM doesn't care how you format your data.

jojo2020 wrote:
The documentation is mostly flat and it takes investigation and effort to arrange the published types into objects for correct use within your code.

The initial grouping of types into objects and additional maintenance to keep them aligned with future changes can introduce defects down the line if care and attention is not taken.


I've been programming for a very long time and updating the documentation for an application sucks but it is part of your job. The job of a developer/programmer is to (1) write code, (2) test code, (3) update docs and (4) deploy program. There is no magic wand. And if you work in the middleware space providing middleware services to other applications that it is an ABSOLUTE must that the documentation be updated and be available before the latest release of the program is deployed.

jojo2020 wrote:
Wouldn't it be easier to the customer if IBM were to supply a model in some format (e.g. YAML) that describe the objects and types?

I prefer XML or JSON.

jojo2020 wrote:
Doesn't IBM model MQ in one of their tools (e.g. Rhapsody) and from there have the ability to export the data into some useful format easily?

Why don't you use JavaDoc or Doxygen.

Regards,
Roger Lacroix
Capitalware Inc.
_________________
Capitalware: Transforming tomorrow into today.
Connected to MQ!
Twitter
Back to top
View user's profile Send private message Visit poster's website
hughson
PostPosted: Tue Jun 02, 2020 2:51 pm Post subject: Reply with quote

Grand Master

Joined: 09 May 2013
Posts: 1433
Location: Bay of Plenty, New Zealand

I have to also add a confused expression to this thread. Perhaps it would be helpful to us all to understand what you mean if you gave at least one small example of a problematic issue you have had?

Why do you have to arrange the IBM published types for use in your code? Don't you just use the IBM-supplied include files?

All IBM supplied structures for the MQ API are forward compliant. No future change would break an existing application.

Please, provide an example. I imagine your post is as a result of some experience you have had. Share it with us, and we will understand you much better.

Cheers,
Morag
_________________
Morag Hughson @MoragHughson
IBM MQ Technical Education Specialist
Get your IBM MQ training here!
MQGem Software
Back to top
View user's profile Send private message Visit poster's website
gbaddeley
PostPosted: Tue Jun 02, 2020 3:38 pm Post subject: Reply with quote

Jedi

Joined: 25 Mar 2003
Posts: 2124
Location: Melbourne, Australia

Quote:
I would like to know how people here maintain and keep aligned their MQ data structures

We don't. Apps that use MQ messages for data integration maintain data structure definitions at the application level, such as XML schema, fixed column and CSV record layouts. It is irrelevant to MQ. Other middleware products could be used for transporting the same data, and it would make no difference.

If you are referring to app usage of MQ APIs, queue objects and their myriad of options and properties, there are normally only a few that need to be documented and agreed. Using a modelling tool or a descriptive language is probably not necessary.
_________________
Glenn
Back to top
View user's profile Send private message
bruce2359
PostPosted: Tue Jun 02, 2020 3:58 pm Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 8715
Location: US: west coast, almost. Otherwise, enroute.

Perhaps you've misunderstood the function and purpose of MQ. MQ doesn't launch applications - MQ is not an app server. MQ can do simple code-page (CCSID) conversion at channel end points, from EBCDIC to ASCII, English to Spanish, and so on.

MQ is like a pipe through which messages flow. An oil pipe is agnostic to the type of oil. A water pipe is agnostic to the type of water.

Message source/content/context/format/conversion can be managed (documented) with IIB - IBM's message broker.
_________________
My life flows on in endless song;
How can I keep from singing?
Back to top
View user's profile Send private message
jojo2020
PostPosted: Tue Jun 09, 2020 1:47 am Post subject: Example added Reply with quote

Newbie

Joined: 02 Jun 2020
Posts: 5

I have added an example of what I require. Maybe people can respond? @MoragHughson
Back to top
View user's profile Send private message
hughson
PostPosted: Tue Jun 09, 2020 1:52 am Post subject: Reply with quote

Grand Master

Joined: 09 May 2013
Posts: 1433
Location: Bay of Plenty, New Zealand

Hi there,

I have seen your update, and you describe where to find what you need.

I had an impression earlier that this question was about applications, but this is a configuration question. Why do you need/want a YAML model for configuration? Are you writing an administration application for MQ?

Could you elaborate on the situation that led you to be concerned about the compression fields that you mention in your update?

Cheers,
Morag
_________________
Morag Hughson @MoragHughson
IBM MQ Technical Education Specialist
Get your IBM MQ training here!
MQGem Software
Back to top
View user's profile Send private message Visit poster's website
RogerLacroix
PostPosted: Tue Jun 09, 2020 9:07 am Post subject: Reply with quote

Jedi Knight

Joined: 15 May 2001
Posts: 3163
Location: London, ON Canada

Those links point to fields in both MQCD and MQCXP structures. I'm going to guess that you are writing a channel exit.

- MQCD structure contains the values of how the channel was defined. A channel exit can change any field in the MQCD to reflect a new setting.

- MQCXP structure contains the run-time values of the channel.

What values are used in each field are all listed in the MQ Knowledge Center (which you have already discovered). So, if you want to know what a particular field supports, scroll to the field name and click on it.

The simple solution (what I do), is to have Eclipse running on 1 monitor and my browser open on a different monitor and that way, I can code particular fields and know what values they support.

Regards,
Roger Lacroix
Capitalware Inc.
_________________
Capitalware: Transforming tomorrow into today.
Connected to MQ!
Twitter
Back to top
View user's profile Send private message Visit poster's website
jojo2020
PostPosted: Wed Jun 10, 2020 2:24 pm Post subject: Further explanation... Reply with quote

Newbie

Joined: 02 Jun 2020
Posts: 5

Hi, yes it is an administration tool being written but not by me so I cannot say much more on it. I am still learning about it which explains the confusing original question, so I apologize.

It was mentioned that coding and maintaining these types was time consuming and it would be easier if these were somehow provided for us. We are not asking for header files but something independent of language choice, my preference would be YAML.

The plumbing around these types would of course be manual.


Last edited by jojo2020 on Thu Jun 11, 2020 12:31 am; edited 1 time in total
Back to top
View user's profile Send private message
gbaddeley
PostPosted: Wed Jun 10, 2020 3:48 pm Post subject: Re: Further explanation... Reply with quote

Jedi

Joined: 25 Mar 2003
Posts: 2124
Location: Melbourne, Australia

jojo2020 wrote:
Hi, yes it is an administration tool being written but not by me so I cannot say much more on it. I am still learning about it which explains the confusing original question, so I apologize.
It was mentioned that coding and maintaining these types was time consuming and it would be easier if these were somehow provided for us (my preference would be YAML).
The plumbing around these types would of course be manual.

You said that you were new to MQ. The structures, data types and all the constants are defined in the MQ header files (eg. cmqc.h + few others). I have never had problems with using these. IBM provides this information in various formats to suit the supported development languages, but there is no universal format.

Developing an adminstration tool requires a deep technical understanding of MQ. Not many folks write such tools and there are many intricacies and traps for the unwary. Have you considered using existing administration tools?
_________________
Glenn
Back to top
View user's profile Send private message
jojo2020
PostPosted: Thu Jun 11, 2020 12:47 am Post subject: Reply with quote

Newbie

Joined: 02 Jun 2020
Posts: 5

These guys have the MQ experience. Whilst learning about the code I saw that the model could be represented in a human friendly way.

I converted the cmqc* header files into YAML and a few of the structures that use them. The question then arose, if IBM presented the data structures as YAML then that would separate the model from the code. This would speed up development and reduce model tests & bugs.
Back to top
View user's profile Send private message
Vitor
PostPosted: Thu Jun 11, 2020 4:57 am Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 25995
Location: Texas, USA

jojo2020 wrote:
The question then arose, if IBM presented the data structures as YAML then that would separate the model from the code. This would speed up development and reduce model tests & bugs.


Again, my 2 cents:

These MQ structures (especially the examples you've quoted) are low level and are inextricably linked to the code, to the point where byte alignment can matter. With this kind of structure (and this counts double for exits) you're right down on the metal, the model is the code and there's nothing human friendly about the environment.

As anyone who's written a dodgy exit and crashed a queue manager will confirm.

I don't see the value of using YAML for this kind of data description as it's another, additional, level of abstraction with no added benefits. That's not to say there isn't value, just that I'm not seeing it.

If IBM starts shipping YAML I won't use it, but I won't complain either. I would prefer IBM to devote their efforts to the laundry list of other things we've been complaining about for years.




Like I said, my 2 cents, other opinions are equally valid and may be better.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
Display posts from previous:
Post new topicReply to topic Goto page 1, 2  Next Page 1 of 2

MQSeries.net Forum IndexGeneral IBM MQ SupportMQ data structure alignment
Jump to:



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP


Theme by Dustin Baccetti
Powered by phpBB 2001, 2002 phpBB Group

Copyright MQSeries.net. All rights reserved.