ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum IndexIBM MQ Performance MonitoringMessage Properties

Post new topicReply to topic Goto page 1, 2  Next
Message Properties View previous topic :: View next topic
Author Message
RogerLacroix
PostPosted: Tue Mar 29, 2016 8:57 am Post subject: Message Properties Reply with quote

Jedi Knight

Joined: 15 May 2001
Posts: 3074
Location: London, ON Canada

All,

I have a structure (about 25 fields) that is meta-data for a message. Since the meta-data structure is fixed length, I could put it in front of the message data as I have done over the years for other applications but I thought this would be a good use of message properties.

Questions (assume the messages per day will be over 1,000,000):

- Has anyone done any performance testing using message properties?

- Is creating 25 message properties per message too much? (when the application is doing million plus messages per day)

- Most of the fields of the meta-data are strings, so I could convert the non-strings to strings then put 1 message property which contains the entire mega-data structure. i.e. It would be about 400 bytes long. This means that the receiving application would need to parse the single message property to get the 25 fields.

I'm just curious how much of a performance hit it would be to use a large number (25) message properties per message when the sending application is putting a million plus messages per day.

Regards,
Roger Lacroix
Capitalware Inc.
_________________
Capitalware: Transforming tomorrow into today.
Connected to MQ!
Twitter
Back to top
View user's profile Send private message Visit poster's website
PaulClarke
PostPosted: Tue Mar 29, 2016 1:36 pm Post subject: Reply with quote

Guardian

Joined: 17 Nov 2005
Posts: 903
Location: New Zealand

Asking things like 'too much' is a little hard to answer. Wouldn't it be best to just try it and measure it and then you'll know just what the cost delta is?

Cheers,
Paul.
_________________
Paul Clarke
MQGem Software
www.mqgem.com
Back to top
View user's profile Send private message Visit poster's website
gbaddeley
PostPosted: Tue Mar 29, 2016 2:34 pm Post subject: Reply with quote

Padawan

Joined: 25 Mar 2003
Posts: 1888
Location: Melbourne, Australia

Hi Roger,
Most of the time penalty for a large number of properties would be CPU spent by MQ in marshalling all the property names and values into and out of the message.

If the fields are fixed length, it would be slightly more efficient to store them in one message property, and the app program directly accesses each fixed length field.

I would also be tempted to not use MQ message properties in this case, and just store the data structure in front of the message data. After all, fixed width string fields are the norm for MQ message formats, as it is easy coding and efficient for apps to process, and works well in cross-platform situations. Of course it lacks the versatility of XML, JSON, SOAP etc.

HTH, G.
_________________
Glenn
Back to top
View user's profile Send private message
PaulClarke
PostPosted: Tue Mar 29, 2016 3:00 pm Post subject: Reply with quote

Guardian

Joined: 17 Nov 2005
Posts: 903
Location: New Zealand

Of course another consideration is forward compatibility. You say that there is a fixed format of 25 fields but is this true for all time ? A fixed format header would undoubtedly be the fastest but is also the least flexible and extensible. However, sometimes that is all you need....look at all the MQ headers One of the downsides with headers though is data conversion - distributing a data conversion exit for all your data formats can be a bit of a pain.

Message Properties are a good way of allowing you to add extra properties if and when they are dreamt up. the other advantage of properties is that is lets you use selectors. Sometime it is useful to be able the get/browse only a subset of the messages efficiently.

Of course another way to have multiple elements to a message is to put everything in a PCF message. To be honest PCF messages are what I tend to do if I need to send a message containing lots of different pieces. They have lots of advantages. It is very easy to add/remove the individual pieces. PCF messages allow me to send lots of different formats of data and it is pretty quick to manipulate. Data conversion is handled automatically by MQ. There are also a number of tools which allow me to format out the contents fairly readily.

It all depends on the trade of betweenf Performance vs Usability vs Extensibility

Cheers,

Paul.
_________________
Paul Clarke
MQGem Software
www.mqgem.com
Back to top
View user's profile Send private message Visit poster's website
RogerLacroix
PostPosted: Wed Mar 30, 2016 3:46 pm Post subject: Reply with quote

Jedi Knight

Joined: 15 May 2001
Posts: 3074
Location: London, ON Canada

Hi,

Both of you point out all of the reasons I asked the question. I'm leaning towards using 25 message properties but I thought I would ask before going too far down the path and then having a Homer moment - D'oh!

I know the MQSETMP call is a local call in the application's address space, so it should be very fast. It is just doing it 25 times that makes me wonder. Although, message properties allows for the greatest flexibility and I like that.

Regards,
Roger Lacroix
Capitalware Inc.
_________________
Capitalware: Transforming tomorrow into today.
Connected to MQ!
Twitter
Back to top
View user's profile Send private message Visit poster's website
bruce2359
PostPosted: Wed Mar 30, 2016 3:59 pm Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 8159
Location: US: west coast, almost. Otherwise, enroute.

RogerLacroix wrote:
Hi,

Both of you point out all of the reasons I asked the question. I'm leaning towards using 25 message properties but I thought I would ask before going too far down the path and then having a Homer moment - D'oh!

I know the MQSETMP call is a local call in the application's address space, so it should be very fast. It is just doing it 25 times that makes me wonder. Although, message properties allows for the greatest flexibility and I like that.

Regards,
Roger Lacroix
Capitalware Inc.

Will this be a client-bindings app? If so, there will be MQI network flows.
_________________
I would tell you a UDP joke, but you might not get it.
Back to top
View user's profile Send private message
PaulClarke
PostPosted: Wed Mar 30, 2016 4:46 pm Post subject: Reply with quote

Guardian

Joined: 17 Nov 2005
Posts: 903
Location: New Zealand

No there won't. Message property verbs are all local to the application address space. The properties are only transported to the server when the usual MQPUT or MQGET call is made.

As an indication of this you can see that you can create a message handle, MQCRTMH, without actually giving it a valid connection handle at all but instead the value contact admin. M Q H C _ U N A S S O C I A T E D _ H C O N N

Cheers,

Paul
_________________
Paul Clarke
MQGem Software
www.mqgem.com
Back to top
View user's profile Send private message Visit poster's website
bruce2359
PostPosted: Wed Mar 30, 2016 8:25 pm Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 8159
Location: US: west coast, almost. Otherwise, enroute.

Ah. Got it.
_________________
I would tell you a UDP joke, but you might not get it.
Back to top
View user's profile Send private message
mqjeff
PostPosted: Thu Mar 31, 2016 4:51 am Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

... I haven't checked, and I'm sure Roger has. But.

Is there anything relevant in the performance reports?
_________________
chmod -R ugo-wx /
Back to top
View user's profile Send private message
RogerLacroix
PostPosted: Thu Mar 31, 2016 3:50 pm Post subject: Reply with quote

Jedi Knight

Joined: 15 May 2001
Posts: 3074
Location: London, ON Canada

mqjeff wrote:
Is there anything relevant in the performance reports?

I didn't see anything related to using a large number (25) of message properties.

Different subject: There are 10 types for a message property:
    - MQTYPE_BOOLEAN
    - MQTYPE_BYTE_STRING
    - MQTYPE_INT8
    - MQTYPE_INT16
    - MQTYPE_INT32
    - MQTYPE_INT64
    - MQTYPE_FLOAT32
    - MQTYPE_FLOAT64
    - MQTYPE_STRING
    - MQTYPE_NULL

For MQTYPE_STRING, MQ Knowledge Center says 'A character string. ValueLength must be zero or greater, or the special value MQVL_NULL_TERMINATED'.

General question: does anyone know what the type MQTYPE_BYTE_STRING is? All the MQ Knowledge Center says is 'A byte string. ValueLength must be zero or greater'. Not exactly useful.

Does it mean a hexadecimal string like: 5468697320697320612068657820737472696E67

Regards,
Roger Lacroix
Capitalware Inc.
_________________
Capitalware: Transforming tomorrow into today.
Connected to MQ!
Twitter
Back to top
View user's profile Send private message Visit poster's website
PaulClarke
PostPosted: Thu Mar 31, 2016 5:02 pm Post subject: Reply with quote

Guardian

Joined: 17 Nov 2005
Posts: 903
Location: New Zealand

Roger,

A MQTYPE_BYTE_STRING is just that, a string of bytes. You say that the manual is not exactly useful but Im not sure what else there would be to say. You can put anything you like in this string of bytes and MQ will not do anything with them except transport them.

So, no it is not data like 54686973...., those are just hexadecimal characters which describe bytes. Rather the actual bytes of 0x54686973. Needless to say sending bytes is twice as efficient, in storage, as sending the bytes as a string.

It is similar in principle to the MQCFBS PCF structure.

Cheers,

Paul
_________________
Paul Clarke
MQGem Software
www.mqgem.com
Back to top
View user's profile Send private message Visit poster's website
tczielke
PostPosted: Fri Apr 01, 2016 6:57 am Post subject: Reply with quote

Shaman

Joined: 08 Jul 2010
Posts: 739
Location: Illinois, USA

This manual doc provides a little more detail on what would constitue a MQTYPE_BYTE_STRING -> http://www.ibm.com/support/knowledgecenter/SSFKSJ_8.0.0/com.ibm.mq.tro.doc/q044840_.htm?lang=en

"For MQBYTEn fields use MQTYPE_BYTE_STRING."

So examples would be:

MQBYTE24 MsgId; /* Message identifier */
MQBYTE24 CorrelId; /* Correlation identifier */

I agree with Roger that describing a MQTYPE_BYTE_STRING as "a byte string" is not exactly very helpful. Fundamentally, every data type (integers, character strings, etc.) is a string of bytes. However, I do understand it is kind of hard to succinctly describe "a string of bytes where MQ treats it as nothing more than raw bytes and does not assign any meaning to it like a string of characters, or an integer, etc.". Perhaps, providing some examples of what would be an MQTYPE_BYTE_STRING (e.g. MsgId) in the manual doc would be more helpful.
_________________
MQ administrator since 2010.
Back to top
View user's profile Send private message
mqjeff
PostPosted: Fri Apr 01, 2016 7:05 am Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

I think "byte string" is used consistently across the MQ knowledge center.

Granted, that might be hard to find if one is looking at a single page or set of pages.

A byte array might be a better term.
_________________
chmod -R ugo-wx /
Back to top
View user's profile Send private message
bruce2359
PostPosted: Fri Apr 01, 2016 7:39 am Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 8159
Location: US: west coast, almost. Otherwise, enroute.

From the APR:
Quote:
MQBYTEn - string of n bytes
MQCHARn - string of n characters


I'd have preferred 'array,' as it implies an ordered arrangement.

Then, there's the Java 'string array.'
_________________
I would tell you a UDP joke, but you might not get it.
Back to top
View user's profile Send private message
PaulClarke
PostPosted: Fri Apr 01, 2016 11:45 am Post subject: Reply with quote

Guardian

Joined: 17 Nov 2005
Posts: 903
Location: New Zealand

An array doesn't imply an 'ordered arrangement' to me. An array to me implies that it is a series of the same object types which can be accessed directly by index. While you could of course do that with a byte string or character string or whatever it is not it's main purpose. So, for me personally, calling it an array would have been misleading. It is a string of bytes which should be treated as a string, not as lots of independently addressable units. It is a string that implies to me an 'ordered arrangement'.

As for MQ not naming it the same way as Java that would clearly not be possible since MQ is older than Java. Having said that, I'm no Java expert by any means, but doesn't Java also have the concept of ByteBuffers ? So Java doesn't always call them arrays either.

I guess it is a common problem in Computing generally that the nomenclature is not standard. Programming languages, command scripts, protocols and Development environments frequently use different names for the same things. That's why we are all paid the big bucks to sort it all out

Cheers,

Paul.
_________________
Paul Clarke
MQGem Software
www.mqgem.com
Back to top
View user's profile Send private message Visit poster's website
Display posts from previous:
Post new topicReply to topic Goto page 1, 2  Next Page 1 of 2

MQSeries.net Forum IndexIBM MQ Performance MonitoringMessage Properties
Jump to:



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP


Theme by Dustin Baccetti
Powered by phpBB 2001, 2002 phpBB Group

Copyright MQSeries.net. All rights reserved.