| Author |
Message
|
| scottybro |
 Posted: Wed May 22, 2019 7:54 am Post subject: pcf 25x slower than runmqsc for inquire_q |
 |
|
Newbie
Joined: 22 May 2019
Posts: 3
|
Hi:
We noticed a big perf penalty when using pcf for admin commands-- specifically inquire_q. Anyone aware of a way to bring pcf operations closer to mqsc performance? Our tests are showing inquire_q is 25x slower than getting the same information via runmqsc. Strange as well-- inquire_q_names returns very fast using pcf < 100ms.
Scenario:
1. Server Q: Remote qmgr running 9.1 on Linux w/ 98 local queues
2. Server A: execute runmqsc -c $qmgr < DIS QL(*) ALL
3. Server A: execute pcf inquire q with attrs set to only MQIA_Q_TYPE
4. Server A: execute pcf inquire_q_names
Both client and server are running v9.1 on Linux
pcf via Java PCFMessageAgent
runqmsc dis ql(*) all: ~ 200 (ms)
pcf inquire_q_names: 96 (ms)
pcf inquire_q name(*) type (MQQT_LOCAL) no attrs specified: 4500 (ms) |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Wed May 22, 2019 8:46 am Post subject: Re: pcf 25x slower than runmqsc for inquire_q |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| scottybro wrote: |
Hi:
We noticed a big perf penalty when using pcf for admin commands-- specifically inquire_q. Anyone aware of a way to bring pcf operations closer to mqsc performance? Our tests are showing inquire_q is 25x slower than getting the same information via runmqsc. Strange as well-- inquire_q_names returns very fast using pcf < 100ms.
Scenario:
1. Server Q: Remote qmgr running 9.1 on Linux w/ 98 local queues
2. Server A: execute runmqsc -c $qmgr < DIS QL(*) ALL
3. Server A: execute pcf inquire q with attrs set to only MQIA_Q_TYPE
4. Server A: execute pcf inquire_q_names
Both client and server are running v9.1 on Linux
pcf via Java PCFMessageAgent
runqmsc dis ql(*) all: ~ 200 (ms)
pcf inquire_q_names: 96 (ms)
pcf inquire_q name(*) type (MQQT_LOCAL) no attrs specified: 4500 (ms) |
Did you try running a pcf inquire_q_names with attribute q_type and then sort out the non local ones? Or using the exact same runmqsc command in a pcf escape message? 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| scottybro |
| Posted: Wed May 22, 2019 9:16 am Post subject: |
|
|
Newbie
Joined: 22 May 2019
Posts: 3
|
I created a pcf escape message using: "DIS QL('*') ALL". For comparison, tested against a 8.x qmgr as well. Results were roughly equivalent to the pcf_inquire_q times-- note the pcf escape returned a response msg per queue, not packed into a single response (as it appears happens in runqmsc).
Results of pcf+escape command match that of pcf+inquire_q perf. This points to something in the PCF processing is significantly slower than how runmqsc processes admin commands.
I noticed that runmqsc displays that only 2 responses are returned. It appears that maybe the qmgr is backing responses more efficiently than w/ pcf. The numbers roughly bear out that:
(total response time / response count) =~ response time for inquire_q_names
inquire_q_names packs all the data into a single response.
I'm wondering if the internals of the PCFAgent are receiving and ack each response message in serial fashion.. causing the pcf for inquire_q to suffer higher latency.. and therefore higher total throughput times.
runqmsc output ends with: 2 command responses received
pcf escape+dis ql(*) all command receives: 1 response msg per queue
pcf inquire_q command receives: 1 response msg per queue
pcf inquire_q_names command receives: 1 response msg for ALL |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu May 23, 2019 4:52 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| scottybro wrote: |
I created a pcf escape message using: "DIS QL('*') ALL". For comparison, tested against a 8.x qmgr as well. Results were roughly equivalent to the pcf_inquire_q times-- note the pcf escape returned a response msg per queue, not packed into a single response (as it appears happens in runqmsc).
Results of pcf+escape command match that of pcf+inquire_q perf. This points to something in the PCF processing is significantly slower than how runmqsc processes admin commands.
I noticed that runmqsc displays that only 2 responses are returned. It appears that maybe the qmgr is backing responses more efficiently than w/ pcf. The numbers roughly bear out that:
(total response time / response count) =~ response time for inquire_q_names
inquire_q_names packs all the data into a single response.
I'm wondering if the internals of the PCFAgent are receiving and ack each response message in serial fashion.. causing the pcf for inquire_q to suffer higher latency.. and therefore higher total throughput times.
runqmsc output ends with: 2 command responses received
pcf escape+dis ql(*) all command receives: 1 response msg per queue
pcf inquire_q command receives: 1 response msg per queue
pcf inquire_q_names command receives: 1 response msg for ALL |
Also you should not confuse the time needed to generate the responses vs the time needed to evaluate / display them. If getting multiple responses slows you down so much you need to find a way for pcf to send you back a single response with the same information.
Have fun
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| Andyh |
| Posted: Thu May 23, 2019 5:54 am Post subject: |
|
|
Master
Joined: 29 Jul 2010
Posts: 239
|
Just a quick sanity check that you're using non-persistent messages and have NPMSPEED(FAST) on your Q<->A channels.
100 ms sounds like an extraordinary amount of time to exchange a single PCF message, yet alone the 4500 ms to exchange ~100 messages.
Performance this bad might typically suggest either an underlying network or IO issue.
If the java program making these requests is running as an MQ client program then ideally you'd also have read ahead enabled. |
|
| Back to top |
|
|
| RogerLacroix |
| Posted: Thu May 23, 2019 1:04 pm Post subject: |
|
|
Jedi Knight
Joined: 15 May 2001
Posts: 3264
Location: London, ON Canada
|
You didn't show us any code or even mention the programming language, so it really difficult for anyone to know if you have a code error or it is an MQ issue.
On my blog, I have published a fully functioning Java PCF program to do a PCF "inquire queue" request message for all ("*") local queues of a remote queue manager:
https://www.capitalware.com/rl_blog/?p=5463
Try it out and see if there is a performance difference.
Regards,
Roger Lacroix
Capitalware Inc.
_________________
Capitalware: Transforming tomorrow into today.
Connected to MQ!
Twitter |
|
| Back to top |
|
|
| scottybro |
| Posted: Thu May 23, 2019 5:23 pm Post subject: |
|
|
Newbie
Joined: 22 May 2019
Posts: 3
|
@Andyh: The client -> qmgr connection is over a svrconn channel, so npmspeed is not available.
I'm trying a few other things. Thank you for the suggestions. |
|
| Back to top |
|
|
| PaulClarke |
| Posted: Thu May 23, 2019 11:40 pm Post subject: |
|
|
Grand Master
Joined: 17 Nov 2005
Posts: 1002
Location: New Zealand
|
Unfortunately issuing PCF or MQSC commands across a client link is always going to be fairly slow because 'sadly' each reply comes back as a separate messages. So, if you have 500 Queue Definitions then a DIS Q(*) will result in 500 round trips to the server. This takes a considerable time. You can reduce the effect somewhat by using read- ahead but it's always going to be slow. When I worked for MQ I argued many times for a more 'sensible' command server which sent sets of messages in a single response (and also allowed for some sort of page limit) but never quite managed to win the argtument.
However, what makes you think it is MQSC vs PCF. Could it be C vs Java ? If I understand you correctly You are comparing the whole stack here. Or indeed one program vs another ? Try another program that generates PCF messages (and processes), such as MO71 and see whether you get consistent results 
Cheers,
Paul.
_________________
Paul Clarke
MQGem Software
www.mqgem.com |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Fri May 24, 2019 4:26 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
And as long as we're talking about the elephant in the room, make sure that reverse DNS lookup is turned off, as this could quite slow down the initial connection from the pcf agent.
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| Andyh |
| Posted: Sat May 25, 2019 10:15 am Post subject: |
|
|
Master
Joined: 29 Jul 2010
Posts: 239
|
Sorry I hadn't appreciated there was no QMgr running on A.
Using non-persisent message, sending all (~100) of the inquire queues with fire and forget before then getting all (~100) of the response messages with read ahead would minimize the number of line turn arounds and the number of IO's and would be likely to effect the best performance from this scenario.
Using MQIBindType=FASTPATH might help with CPU consumption on server Q, but at ~50ms per message it's unlikely you have a CPU bound measurement. |
|
| Back to top |
|
|
|
|
