| Author |
Message
|
| mago |
 Posted: Wed Apr 02, 2025 6:04 am Post subject: Topic/subscribe - MQ z/OS |
 |
|
Newbie
Joined: 14 Mar 2025
Posts: 3
|
Hello, everyone.
We are currently dealing with a performance issue related to a high-traffic topic on MQ for z/OS, processing millions of messages per day. The environment also includes around 30 subscriptions, each targeting a different queue.
The application is experiencing significant latency during PUT operations to the topic.
I've been searching for any IBM documentation or best practices specifically related to performance tuning and configuration of TOPICs and SUBSCRIPTIONS on MQ for z/OS, but so far haven't found anything relevant. Notably, the MP16 redbook (IBM MQ for z/OS ? Capacity Planning and Tuning) doesn't seem to cover this scenario.
If anyone has references, insights, or experience with similar setups, your input would be greatly appreciated. |
|
| Back to top |
|
 |
| bruce2359 |
| Posted: Wed Apr 02, 2025 7:08 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| mago wrote: |
Hello, everyone.
In this same scenario we have a topic on MQ z/OS with high message traffic (millions per day).
|
Millions of messages per second would be high message traffic on z/OS
| mago wrote: |
| There are also 30 subscriptions pointing to different queues. |
Are the 30 subscriptions in a single application?
| mago wrote: |
The application is experiencing very high latency at the time of the PUT.
|
How do you know that the app is experiencing very high latency at the time of the PUT?
Does the app do database SQL calls?
Is this a new app? An existing app with recent changes?
| mago wrote: |
Is there any IBM document with configuration and tuning recommendations for TOPICs and SUBSCRIPTIONS? I haven?t found anything so far (for example, there?s nothing in MP16: IBM MQ for z/OS - Capacity planning & tuning).
-- |
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| mago |
| Posted: Wed Apr 02, 2025 8:35 am Post subject: |
|
|
Newbie
Joined: 14 Mar 2025
Posts: 3
|
| bruce2359 wrote: |
Millions of messages per second would be high message traffic on z/OS
|
The applications (RabbitMQ and CICS) performs the put on the TOPIC located on Z/OS. Approximately 11,000 messages per second.
| bruce2359 wrote: |
Are the 30 subscriptions in a single application?
|
No. RabbitMQ and CICS. But all subscriptions subscribe to the same TOPIC STRING.
| bruce2359 wrote: |
How do you know that the app is experiencing very high latency at the time of the PUT?
|
We observed high latency across several monitoring applications, including IBM Z IntelliMagic Vision for z/OS.
| bruce2359 wrote: |
Does the app do database SQL calls?
|
No. And all of them use the MQ API.
| bruce2359 wrote: |
Is this a new app? An existing app with recent changes?
|
The application has been running since February 2024. However, starting in November, a growing increase in latency began to be noticed. There are only three differences in the current scenario:
1) the volume of messages has increased over the months;
2) more subscriptions have been created;
3) more queue managers have been created. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Wed Apr 02, 2025 6:21 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
It would help us help you if we knew more about your TOPIC OBJECT. Please enter these MQRC commands and post results here:
DISPLAY TOPIC(topic-name) ALL
DISPLAY TPSTATUS(topic-string) ALL
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| mago |
| Posted: Thu Apr 03, 2025 6:43 am Post subject: |
|
|
Newbie
Joined: 14 Mar 2025
Posts: 3
|
DISPLAY TOPIC(TP.ACD.CDC) ALL
TOPIC(TP.ACD.CDC)
TYPE(LOCAL)
QSGDISP(QMGR)
CLUSTER()
TOPICSTR(ACD/CDC)
DEFPRTY(ASPARENT)
DEFPSIST(ASPARENT)
DURSUB(ASPARENT)
NPMSGDLV(ASPARENT)
PMSGDLV(ASPARENT)
MDURMDL()
MNDURMDL()
PUB(ASPARENT)
SUB(ASPARENT)
PUBSCOPE(ASPARENT)
SUBSCOPE(ASPARENT)
PROXYSUB(FIRSTUSE)
WILDCARD(PASSTHRU)
CLROUTE(DIRECT)
DESCR()
DEFPRESP(ASPARENT)
USEDLQ(ASPARENT)
CUSTOM()
ALTDATE(2024-11-03)
ALTTIME(01.14.05)
END TOPIC DETAILS
DISPLAY TPSTATUS(ACD/CDC) ALL
TPSTATUS(ACD/CDC)
TYPE(TOPIC)
DEFPRESP(SYNC)
DEFPSIST(NO)
DEFPRTY(0)
DURSUB(YES)
PUB(ENABLED)
SUB(ENABLED)
ADMIN(TP.SPI.SED)
MDURMDL(SYSTEM.DURABLE.MODEL.QUEUE)
MNDURMDL(SYSTEM.NDURABLE.MODEL.QUEUE)
NPMSGDLV(ALLAVAIL)
PMSGDLV(ALLDUR)
RETAINED(NO)
PUBCOUNT( 8 )
SUBCOUNT(29)
PUBSCOPE(ALL)
SUBSCOPE(ALL)
CLUSTER()
USEDLQ(YES)
CLROUTE(NONE)
END TPSTATUS DETAILS |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Thu Apr 03, 2025 8:17 am Post subject: Re: Topic/subscribe - MQ |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| mago wrote: |
| The application is experiencing significant latency during PUT operations to the topic. |
What does this mean exactly?
What is the SLA for this task?
What was the number of tasks per second before?
What is the number of tasks per second now?
Was the ithe workload projected to grow significantly?
What other symptoms? CPU increase? Paging/swapping increase? Other apps missing SLA's? Or just this one app?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| mago |
| Posted: Thu Apr 03, 2025 10:06 am Post subject: |
|
|
Newbie
Joined: 14 Mar 2025
Posts: 3
|
What does this mean exactly?
| bruce2359 wrote: |
It would help us help you if we knew more about your TOPIC OBJECT. Please enter these MQRC commands and post results here:
DISPLAY TOPIC(topic-name) ALL
DISPLAY TPSTATUS(topic-string) ALL |
|
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sun Apr 06, 2025 7:11 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| Code: |
NPMSGDLV(ALLAVAIL)
PMSGDLV(ALLDUR) |
You need to make sure you understand what those 2 parameters mean and determine if they have the right value for your use case.
Hope it helps 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Wed Apr 16, 2025 3:14 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
Moved to mainframe forum.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Wed Apr 16, 2025 3:26 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
Is all of the software you mentioned running on z/OS? Or, some on midrange?
Is all of the work done in SYNCPOINT?
Have you opened a PMR with IBM on this issue?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| hughson |
| Posted: Thu Apr 17, 2025 2:28 am Post subject: Re: Topic/subscribe - MQ z/OS |
|
|
Padawan
Joined: 09 May 2013
Posts: 1959
Location: Bay of Plenty, New Zealand
|
| mago wrote: |
We are currently dealing with a performance issue related to a high-traffic topic on MQ for z/OS, processing millions of messages per day. The environment also includes around 30 subscriptions, each targeting a different queue.
The application is experiencing significant latency during PUT operations to the topic. |
The MQPUT of the publishing application cannot return until the puts to all the subscriber queues is done. This suggests that there are issues putting to the subscriber queues.
| mago wrote: |
| I've been searching for any IBM documentation or best practices specifically related to performance tuning and configuration of TOPICs and SUBSCRIPTIONS on MQ for z/OS, but so far haven't found anything relevant. Notably, the MP16 redbook (IBM MQ for z/OS ? Capacity Planning and Tuning) doesn't seem to cover this scenario. |
From a performance perspective, treat the scenario as you would an application that is putting to 30 different queues. Where are the queues located? Are the queues being consumed from in a timely manner? Are they getting deep and therefore message retrieval is doing real I/O? Are your buffer pools big enough? Are any other applications sharing the page set and/or buffer pool. Are you using managed subscriber queues or pre-defined local queues (which would allow you to separate them more)? Are your messages persistent or non-persistent? Are messages going to the DLQ? What is supposed to happen when failures to put to subscriber queues occur? Review your [N]PMSGDLV settings on the topic and the USEDLQ setting on the topic. Are the topics shared in a cluster? Are there QM-QM channels involved? If so, are transmission queues backing up with publication messages? Have you reviewed the settings on these?
Are you can see, most of these questions are the same things you need to think about with a queued scenario. Publish/Subscribe is still, at the end of the day, putting messages on queues. It just has the ability to have a lot more messages and a lot more queues, more easily. Treat the performance of it the same as you would if you were just thinking about queues and you will be 95% of the way there.
Cheers,
Morag
_________________
Morag Hughson @MoragHughson
IBM MQ Technical Education Specialist
Get your IBM MQ training here!
MQGem Software |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Thu Apr 17, 2025 6:24 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
|
| Back to top |
|
|
|
|
