ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum IndexIBM MQ Performance MonitoringHow can I check if read-ahead is actually enabled?

Post new topicReply to topic Goto page Previous  1, 2, 3
How can I check if read-ahead is actually enabled? View previous topic :: View next topic
Author Message
chmd
PostPosted: Wed Dec 16, 2015 6:26 am Post subject: Reply with quote

Novice

Joined: 11 Dec 2015
Posts: 18

bruce2359 wrote:
chmd wrote:

This code is still not faster when used with the production queue, though. If anything, the option makes the code slower.

What option? Please be a bit more precise when you post.


Sorry. I am talking about the option CMQC.MQOO_READ_AHEAD. You can see in my run that I get a very slightly better performance (13 TPS vs 15 TPS) on my production queue when I am *not* using it, which is counter intuitive. On the other hand, when I use the same program on another queue, I get a very good performance with CMQC.MQOO_READ_AHEAD (300 TPS vs 20 TPS).

fjb_saper wrote:
Don't build the MQMD and the GMO each time around in the loop.
Build it once outside, and assign the value to the actual field in each loop iteration...


No, this is actually needed. If I reuse md and gmo, then I get this bug that I can pull only one message from the queue, and I exit with the error code 2033 while the queue is in fact not empty. You do need to use a different message descriptor every time if you don't want to face this bug.
Back to top
View user's profile Send private message
mqjeff
PostPosted: Wed Dec 16, 2015 6:29 am Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

You need to clear the msgid and correlid on the mqmd... or make sure you don't match on either.

The code object/structure is reusable.
_________________
chmod -R ugo-wx /
Back to top
View user's profile Send private message
bruce2359
PostPosted: Wed Dec 16, 2015 7:35 am Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 8032
Location: US: west coast, almost. Otherwise, enroute.

chmd wrote:

Sorry. I am talking about the option CMQC.MQOO_READ_AHEAD. You can see in my run that I get a very slightly better performance (13 TPS vs 15 TPS) on my production queue when I am *not* using it, which is counter intuitive. On the other hand, when I use the same program on another queue, I get a very good performance with CMQC.MQOO_READ_AHEAD (300 TPS vs 20 TPS).

Are the message that are put to the two queues the exact same messages? Exact same message length? Same data content?

Something must be different that causes the different timing. If the queue attributes are identical, and the exact same application is consuming the messages, what is different?

Does the same application produce the messages that end up in the two queues?

Have you embedded a wait or timer of any kind in your application?

Is there a database involved in your application?
_________________
I would tell you a UDP joke, but you might not get it.
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Wed Dec 16, 2015 11:05 am Post subject: Reply with quote

Grand Poobah

Joined: 18 Nov 2003
Posts: 19661
Location: LI,NY

chmd wrote:

fjb_saper wrote:
Don't build the MQMD and the GMO each time around in the loop.
Build it once outside, and assign the value to the actual field in each loop iteration...


No, this is actually needed. If I reuse md and gmo, then I get this bug that I can pull only one message from the queue, and I exit with the error code 2033 while the queue is in fact not empty. You do need to use a different message descriptor every time if you don't want to face this bug.

mqjeff wrote:

You need to clear the msgid and correlid on the mqmd... or make sure you don't match on either.

The code object/structure is reusable.

So you need to declare / define the structures twice.
Once for initialization (think of them as constants) and once for use inside the loop. Then at each iteration you initialize the value of the loop field with that of the corresponding constant field. It may shave off a few cycles off your loop.
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
bruce2359
PostPosted: Wed Dec 16, 2015 11:54 am Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 8032
Location: US: west coast, almost. Otherwise, enroute.

fjb_saper wrote:

So you need to declare / define the structures twice.
Once for initialization (think of them as constants) and once for use inside the loop. Then at each iteration you initialize the value of the loop field with that of the corresponding constant field. It may shave off a few cycles off your loop.

This is a best programming practice - instantiate an MQMD to use as a template for resetting its field values to factory (initial) values, and another MQMD to use for MQGETs and MQPUTs.

Values in the MQMD are NOT defaults - they are not automatically reset to factory values after use. The Application Programming Ref. manual correctly describes fields in MQMD and other structures as initial values - and not defaults.

Before an MQGET or MQPUT, copy the MQMD template over the MQMD your app used for MQGET/MQPUT.

The size of the MQMD is a few hundred bytes. The number of processor cycles to copy the MQMD template over the recently-used MQMD is insignificant.
_________________
I would tell you a UDP joke, but you might not get it.
Back to top
View user's profile Send private message
chmd
PostPosted: Fri Dec 18, 2015 3:05 am Post subject: Reply with quote

Novice

Joined: 11 Dec 2015
Posts: 18

All right, I just did an interesting test: I copied about 26000 messages from my production queue to a file, then to my test queue. Now I get the same problem on the test queue!

The thing is, I checked all of them, and they are all non persistent!
Code:

     1 : dis qstatus(TEST.Q) all
AMQ8450: Display queue status details.
   QUEUE(TEST.Q)                           TYPE(QUEUE)
   CURDEPTH(26770)                         IPPROCS(0)
   LGETDATE( )                             LGETTIME( )
   LPUTDATE( )                             LPUTTIME( )
   MEDIALOG( )                             MONQ(OFF)
   MSGAGE( )                               OPPROCS(0)
   QTIME( , )                              UNCOM(NO)


Code:

$ amqsbcg TEST.Q QMGR|grep "Persistence : 0" | wc -l
26770


However, my code is super slow when dealing with those:
Code:

time QUEUE_NAME=TEST.Q READ_AHEAD=1 ./read_ahead_test
Starting.
Using read_ahead
TEST.Q: Got 41 messages (8.2 tps, 122.3 KiB/s, 14.9 KiB/msg)
TEST.Q: Got 47 messages (9.4 tps, 155.6 KiB/s, 16.6 KiB/msg)
TEST.Q: Got 44 messages (8.8 tps, 144.8 KiB/s, 16.5 KiB/msg)


The same code gets 230 TPS when dealing with test messages on the same size, loaded with the q utility like this:

Code:

q -af -oTEST.Q -lmqic -M'#!50000/15360'


It is definitely not a problem of the queue, but really with the messages. This is weird, because the ony thing that is required should be non-persistence...
Back to top
View user's profile Send private message
chmd
PostPosted: Fri Dec 18, 2015 5:21 am Post subject: Reply with quote

Novice

Joined: 11 Dec 2015
Posts: 18

Ok, I compared some test messages with what I get from my prod queue. The only difference that seems significant is the encoding!

Production message:
Code:

  Encoding : 273  CodedCharSetId : 819


Test message:
Code:

  Encoding : 546  CodedCharSetId : 819


Maybe read ahead was working all along? Maybe there is some conversion that I did not explicitly request that takes place and screws up the performance?
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Fri Dec 18, 2015 5:47 am Post subject: Reply with quote

Grand Poobah

Joined: 18 Nov 2003
Posts: 19661
Location: LI,NY

So you have the same CCSID but a different encoding... What is the FORMAT on the MQMD of your messages as shown by the browse tool?
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
chmd
PostPosted: Fri Dec 18, 2015 7:07 am Post subject: Reply with quote

Novice

Joined: 11 Dec 2015
Posts: 18

fjb_saper wrote:
So you have the same CCSID but a different encoding... What is the FORMAT on the MQMD of your messages as shown by the browse tool?


I get
Code:

  Format : 'MQSTR   '


Now, what I did: In my dump file, I modified this format to
Code:

 Format : '        '

(8 spaces)with a search and replace, and reloaded the messages in the queue. Then I used my program again to pull the messages.

The program does not run any faster, which means that changing the Format to None will not help
Back to top
View user's profile Send private message
mqjeff
PostPosted: Fri Dec 18, 2015 7:25 am Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

Are you reading messages at the same time as you are writing them - from whatever you are using instead of qload?
_________________
chmod -R ugo-wx /
Back to top
View user's profile Send private message
bruce2359
PostPosted: Fri Dec 18, 2015 8:14 am Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 8032
Location: US: west coast, almost. Otherwise, enroute.

Exactly what platform (o/s, hardware) is the message created on?

Exactly what platform (o/s, hardware) is the qmgr?

Exactly what platform (o/s, hardware) is the message consumer?

Are any exits involved?
_________________
I would tell you a UDP joke, but you might not get it.
Back to top
View user's profile Send private message
chmd
PostPosted: Mon Dec 21, 2015 1:56 am Post subject: Reply with quote

Novice

Joined: 11 Dec 2015
Posts: 18

mqjeff wrote:
Are you reading messages at the same time as you are writing them - from whatever you are using instead of qload?


No, just reading. I do not touch the production queue anymore now that I can reproduce the problem on the test queue. When loaded with test messages (15KiB-30KiB messages generated with the 'q' utility), the queue reads at 130-200 TPS. When reading messages from my production batch (about 15KiB on average), it reads at 8-10TPS.

bruce2359 wrote:
Exactly what platform (o/s, hardware) is the message created on?

Exactly what platform (o/s, hardware) is the qmgr?

Exactly what platform (o/s, hardware) is the message consumer?

Are any exits involved?


- The qmgr is running on a Red Hat Enterprise Linux Server release 6.6.
- The consumer is running on a Debian 7.9. It runs exactly this code.
- I don't know what you mean by 'exits'. As you can see if you read the gist I linked, the code drops the messages in order to avoid IO related problems and focus on MQ performance.

As for the hardware: Both are virtual machines, with plenty of resources. I don't have more relevant information, but I guarantee that there is no issue with it: we extensively checked this aspect and made sure no other VM was stealing resources.
Back to top
View user's profile Send private message
chmd
PostPosted: Mon Dec 21, 2015 2:08 am Post subject: Reply with quote

Novice

Joined: 11 Dec 2015
Posts: 18

For the record, I am adding all the meta info I can have about the messages.

Production (slow)
Code:

****Message descriptor****

  StrucId  : 'MD  '  Version : 2
  Report   : 0  MsgType : 8
  Expiry   : -1  Feedback : 0
  Encoding : 273  CodedCharSetId : 819
  Format : 'MQSTR   '
  Priority : 0  Persistence : 0
  MsgId : X'(removed)'
  CorrelId : X'(removed)'
  BackoutCount : 0
  ReplyToQ       : '(removed)'
  ReplyToQMgr    : '(removed)'
  ** Identity Context
  UserIdentifier : '(removed)'
  AccountingToken :
   X'0000000000000000000000000000000000000000000000000000000000000000'
  ApplIdentityData : '                                '
  ** Origin Context
  PutApplType    : '6'
  PutApplName    : '(removed)                   '
  PutDate  : '20151218'    PutTime  : '10292976'
  ApplOriginData : '    '

  GroupId : X'000000000000000000000000000000000000000000000000'
  MsgSeqNumber   : '1'
  Offset         : '0'
  MsgFlags       : '0'
  OriginalLength : '-1'

****   Message      ****
[...]


Test (fast)
Code:

****Message descriptor****

  StrucId  : 'MD  '  Version : 2
  Report   : 0  MsgType : 8
  Expiry   : -1  Feedback : 0
  Encoding : 546  CodedCharSetId : 819
  Format : 'MQSTR   '
  Priority : 0  Persistence : 0
  MsgId : X'(removed)'
  CorrelId : X'(removed)'
  BackoutCount : 0
  ReplyToQ       : '(removed) '
  ReplyToQMgr    : '(removed)'
  ** Identity Context
  UserIdentifier : '(removed)    '
  AccountingToken :
   X'0000000000000000000000000000000000000000000000000000000000000000'
  ApplIdentityData : '                                '
  ** Origin Context
  PutApplType    : '6'
  PutApplName    : 'q'
  PutDate  : '20151218'    PutTime  : '12375004'
  ApplOriginData : '    '

  GroupId : X'000000000000000000000000000000000000000000000000'
  MsgSeqNumber   : '1'
  Offset         : '0'
  MsgFlags       : '0'
  OriginalLength : '-1'

****   Message      ****
[...]
Back to top
View user's profile Send private message
chmd
PostPosted: Mon Dec 21, 2015 7:18 am Post subject: Reply with quote

Novice

Joined: 11 Dec 2015
Posts: 18

All right, seems like the issue is unrelated to encoding in the end.

I tried 2 more things:
- Changing the Encoding metadata in the dumpfile to 546 and retrying: still slow
- Downloading the messages with q instead of my own program: It takes only 1m18 to dowload all the 26919 messages -> 340 TPS!

So there is a bug in my own code that makes fetching those messages super slow, and I have no idea what it is.
Back to top
View user's profile Send private message
Display posts from previous:
Post new topicReply to topic Goto page Previous  1, 2, 3 Page 3 of 3

MQSeries.net Forum IndexIBM MQ Performance MonitoringHow can I check if read-ahead is actually enabled?
Jump to:



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP


Theme by Dustin Baccetti
Powered by phpBB 2001, 2002 phpBB Group

Copyright MQSeries.net. All rights reserved.