| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| DataFlow Engines claiming more and more memory & PROD |
« View previous topic :: View next topic » |
| Author |
Message
|
| mqsiuser |
 Posted: Mon Oct 06, 2014 5:21 am Post subject: DataFlow Engines claiming more and more memory & PROD |
 |
|
Yatiri
Joined: 15 Apr 2008
Posts: 637
Location: Germany
|
Dear MQ Series Experts,
we have a Broker production environment and some flows run seldomly and create (a) huge file(s)/document(s) (so they deal with big (output-)messages).
As of my knowledge the Execution Group does not release memory (once claimed).
Now we are running out of memory (overall memory & SWAP-space on AIX).
I guess we are mis-using flows: Creating one / 'a couple' huge message (a report) once a day (or a couple of times a day), while Broker is designed to deal with a lot of messages (small or large)?
Is there any possibility to release the memory that an exec group claims / holds on to, except restarting the exec group ?
Currently we are planning to restart all exec groups nightly (2 am)... in production 
Not sure if this is the best approach.
looking forward to hear your thoughts,
mqsiuser
_________________
Just use REFERENCEs |
|
| Back to top |
|
 |
| Vitor |
| Posted: Mon Oct 06, 2014 5:33 am Post subject: Re: DataFlow Engines claiming more and more memory & PRO |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mqsiuser wrote: |
Currently we are planning to restart all exec groups nightly (2 am)... in production
Not sure if this is the best approach. |
So you have a couple of flows which, due to their processing, require a large amount of memory which (as you correctly say) the execution group doesn't release.
So why not put them all in a single execution group called BIGREPORTFLOWS and get them to reuse the memory that execution group is holding onto? That's what broker expects you to do, and why it's holding onto the memory - it expects another whonking workload to turn up.
This means that instead of (for example) 1 report flow in 3 execution groups holding (for example) 1 Gb of SWAP for 3Gb out of your memory pool, you have 3 flows in 1 EG that holds 1Gb and you're 2 Gb better off.
You may need a little ingenuity to ensure the reports are produced single threaded, but I've never seem a report with a sub-second SLA.....
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
