ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum IndexIBM MQ API Supportpcf 25x slower than runmqsc for inquire_q

Post new topicReply to topic
pcf 25x slower than runmqsc for inquire_q View previous topic :: View next topic
Author Message
scottybro
PostPosted: Wed May 22, 2019 7:54 am Post subject: pcf 25x slower than runmqsc for inquire_q Reply with quote

Newbie

Joined: 22 May 2019
Posts: 3

Hi:

We noticed a big perf penalty when using pcf for admin commands-- specifically inquire_q. Anyone aware of a way to bring pcf operations closer to mqsc performance? Our tests are showing inquire_q is 25x slower than getting the same information via runmqsc. Strange as well-- inquire_q_names returns very fast using pcf < 100ms.

Scenario:
1. Server Q: Remote qmgr running 9.1 on Linux w/ 98 local queues
2. Server A: execute runmqsc -c $qmgr < DIS QL(*) ALL
3. Server A: execute pcf inquire q with attrs set to only MQIA_Q_TYPE
4. Server A: execute pcf inquire_q_names

Both client and server are running v9.1 on Linux
pcf via Java PCFMessageAgent

runqmsc dis ql(*) all: ~ 200 (ms)
pcf inquire_q_names: 96 (ms)
pcf inquire_q name(*) type (MQQT_LOCAL) no attrs specified: 4500 (ms)
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Wed May 22, 2019 8:46 am Post subject: Re: pcf 25x slower than runmqsc for inquire_q Reply with quote

Grand Poobah

Joined: 18 Nov 2003
Posts: 19981
Location: LI,NY

scottybro wrote:
Hi:

We noticed a big perf penalty when using pcf for admin commands-- specifically inquire_q. Anyone aware of a way to bring pcf operations closer to mqsc performance? Our tests are showing inquire_q is 25x slower than getting the same information via runmqsc. Strange as well-- inquire_q_names returns very fast using pcf < 100ms.

Scenario:
1. Server Q: Remote qmgr running 9.1 on Linux w/ 98 local queues
2. Server A: execute runmqsc -c $qmgr < DIS QL(*) ALL
3. Server A: execute pcf inquire q with attrs set to only MQIA_Q_TYPE
4. Server A: execute pcf inquire_q_names

Both client and server are running v9.1 on Linux
pcf via Java PCFMessageAgent

runqmsc dis ql(*) all: ~ 200 (ms)
pcf inquire_q_names: 96 (ms)
pcf inquire_q name(*) type (MQQT_LOCAL) no attrs specified: 4500 (ms)

Did you try running a pcf inquire_q_names with attribute q_type and then sort out the non local ones? Or using the exact same runmqsc command in a pcf escape message?
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
scottybro
PostPosted: Wed May 22, 2019 9:16 am Post subject: Reply with quote

Newbie

Joined: 22 May 2019
Posts: 3

I created a pcf escape message using: "DIS QL('*') ALL". For comparison, tested against a 8.x qmgr as well. Results were roughly equivalent to the pcf_inquire_q times-- note the pcf escape returned a response msg per queue, not packed into a single response (as it appears happens in runqmsc).

Results of pcf+escape command match that of pcf+inquire_q perf. This points to something in the PCF processing is significantly slower than how runmqsc processes admin commands.

I noticed that runmqsc displays that only 2 responses are returned. It appears that maybe the qmgr is backing responses more efficiently than w/ pcf. The numbers roughly bear out that:

(total response time / response count) =~ response time for inquire_q_names

inquire_q_names packs all the data into a single response.

I'm wondering if the internals of the PCFAgent are receiving and ack each response message in serial fashion.. causing the pcf for inquire_q to suffer higher latency.. and therefore higher total throughput times.

runqmsc output ends with: 2 command responses received
pcf escape+dis ql(*) all command receives: 1 response msg per queue
pcf inquire_q command receives: 1 response msg per queue
pcf inquire_q_names command receives: 1 response msg for ALL
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Thu May 23, 2019 4:52 am Post subject: Reply with quote

Grand Poobah

Joined: 18 Nov 2003
Posts: 19981
Location: LI,NY

scottybro wrote:
I created a pcf escape message using: "DIS QL('*') ALL". For comparison, tested against a 8.x qmgr as well. Results were roughly equivalent to the pcf_inquire_q times-- note the pcf escape returned a response msg per queue, not packed into a single response (as it appears happens in runqmsc).

Results of pcf+escape command match that of pcf+inquire_q perf. This points to something in the PCF processing is significantly slower than how runmqsc processes admin commands.

I noticed that runmqsc displays that only 2 responses are returned. It appears that maybe the qmgr is backing responses more efficiently than w/ pcf. The numbers roughly bear out that:

(total response time / response count) =~ response time for inquire_q_names

inquire_q_names packs all the data into a single response.

I'm wondering if the internals of the PCFAgent are receiving and ack each response message in serial fashion.. causing the pcf for inquire_q to suffer higher latency.. and therefore higher total throughput times.

runqmsc output ends with: 2 command responses received
pcf escape+dis ql(*) all command receives: 1 response msg per queue
pcf inquire_q command receives: 1 response msg per queue
pcf inquire_q_names command receives: 1 response msg for ALL


Also you should not confuse the time needed to generate the responses vs the time needed to evaluate / display them. If getting multiple responses slows you down so much you need to find a way for pcf to send you back a single response with the same information.

Have fun
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
Andyh
PostPosted: Thu May 23, 2019 5:54 am Post subject: Reply with quote

Disciple

Joined: 29 Jul 2010
Posts: 161

Just a quick sanity check that you're using non-persistent messages and have NPMSPEED(FAST) on your Q<->A channels.
100 ms sounds like an extraordinary amount of time to exchange a single PCF message, yet alone the 4500 ms to exchange ~100 messages.
Performance this bad might typically suggest either an underlying network or IO issue.

If the java program making these requests is running as an MQ client program then ideally you'd also have read ahead enabled.
Back to top
View user's profile Send private message
RogerLacroix
PostPosted: Thu May 23, 2019 1:04 pm Post subject: Reply with quote

Jedi Knight

Joined: 15 May 2001
Posts: 3117
Location: London, ON Canada

You didn't show us any code or even mention the programming language, so it really difficult for anyone to know if you have a code error or it is an MQ issue.

On my blog, I have published a fully functioning Java PCF program to do a PCF "inquire queue" request message for all ("*") local queues of a remote queue manager:
https://www.capitalware.com/rl_blog/?p=5463

Try it out and see if there is a performance difference.

Regards,
Roger Lacroix
Capitalware Inc.
_________________
Capitalware: Transforming tomorrow into today.
Connected to MQ!
Twitter
Back to top
View user's profile Send private message Visit poster's website
scottybro
PostPosted: Thu May 23, 2019 5:23 pm Post subject: Reply with quote

Newbie

Joined: 22 May 2019
Posts: 3

@Andyh: The client -> qmgr connection is over a svrconn channel, so npmspeed is not available.

I'm trying a few other things. Thank you for the suggestions.
Back to top
View user's profile Send private message
PaulClarke
PostPosted: Thu May 23, 2019 11:40 pm Post subject: Reply with quote

Guardian

Joined: 17 Nov 2005
Posts: 938
Location: New Zealand

Unfortunately issuing PCF or MQSC commands across a client link is always going to be fairly slow because 'sadly' each reply comes back as a separate messages. So, if you have 500 Queue Definitions then a DIS Q(*) will result in 500 round trips to the server. This takes a considerable time. You can reduce the effect somewhat by using read- ahead but it's always going to be slow. When I worked for MQ I argued many times for a more 'sensible' command server which sent sets of messages in a single response (and also allowed for some sort of page limit) but never quite managed to win the argtument.

However, what makes you think it is MQSC vs PCF. Could it be C vs Java ? If I understand you correctly You are comparing the whole stack here. Or indeed one program vs another ? Try another program that generates PCF messages (and processes), such as MO71 and see whether you get consistent results

Cheers,

Paul.
_________________
Paul Clarke
MQGem Software
www.mqgem.com
Back to top
View user's profile Send private message Visit poster's website
fjb_saper
PostPosted: Fri May 24, 2019 4:26 am Post subject: Reply with quote

Grand Poobah

Joined: 18 Nov 2003
Posts: 19981
Location: LI,NY

And as long as we're talking about the elephant in the room, make sure that reverse DNS lookup is turned off, as this could quite slow down the initial connection from the pcf agent.
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
Andyh
PostPosted: Sat May 25, 2019 10:15 am Post subject: Reply with quote

Disciple

Joined: 29 Jul 2010
Posts: 161

Sorry I hadn't appreciated there was no QMgr running on A.

Using non-persisent message, sending all (~100) of the inquire queues with fire and forget before then getting all (~100) of the response messages with read ahead would minimize the number of line turn arounds and the number of IO's and would be likely to effect the best performance from this scenario.

Using MQIBindType=FASTPATH might help with CPU consumption on server Q, but at ~50ms per message it's unlikely you have a CPU bound measurement.
Back to top
View user's profile Send private message
Display posts from previous:
Post new topicReply to topic Page 1 of 1

MQSeries.net Forum IndexIBM MQ API Supportpcf 25x slower than runmqsc for inquire_q
Jump to:



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP


Theme by Dustin Baccetti
Powered by phpBB 2001, 2002 phpBB Group

Copyright MQSeries.net. All rights reserved.