| Author |
Message
|
| Sam Uppu |
 Posted: Fri Mar 16, 2012 1:37 pm Post subject: memory usage |
 |
|
Yatiri
Joined: 11 Nov 2008
Posts: 610
|
Hi Guys,
I found this Technote on IBM.
| Quote: |
| http://www-01.ibm.com/support/docview.wss?uid=swg21497847 |
It says "A DataFlowEngine process cannot release allocated memory until it is stopped (or restarted). "
Question:
Will execution group does not release the memory until EG is restarted?
My version of WMB is 6.1 and OS is AIX.
I ran a script to test the memory usage on my local and and could see pagespace going up.
Then stopped the script and ran again, I could see pagespace usage started going up from my last testscript run.
When I reloaded EG it released all the space.
Can you guys pls throw some light.
Thanks |
|
| Back to top |
|
 |
| smdavies99 |
| Posted: Fri Mar 16, 2012 10:26 pm Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
Are you using the
mqsireload
command?
Check the process ID of the EG before and after. I'd expect that it would be different. that would meant that....
It appears to be working as advertised.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| Sam Uppu |
| Posted: Sat Mar 17, 2012 10:31 am Post subject: |
|
|
Yatiri
Joined: 11 Nov 2008
Posts: 610
|
| smdavies99 wrote: |
Are you using the
mqsireload
command?
Check the process ID of the EG before and after. I'd expect that it would be different. that would meant that....
It appears to be working as advertised. |
I used mqsireload command to reload the EG. This releases the memory.
When I run the testcases the pagespace increases slowly. But after stopping the test script will Broker release the pagespace, as it is not processing any messages?
will it only release when the EG is restared? |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Sat Mar 17, 2012 11:23 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
So did you check the process ID before and after the 'mqsireload'?
If it is different then that will explain the release of resources.... 
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| Sam Uppu |
| Posted: Mon Mar 19, 2012 6:36 am Post subject: |
|
|
Yatiri
Joined: 11 Nov 2008
Posts: 610
|
The PID before and after reload are different. Also noticed that pagespace got released after the reload.
Is it reqd to reload everytime the memory consumed increases? Will broker not release space after it processes messages? |
|
| Back to top |
|
|
| mqjeff |
| Posted: Mon Mar 19, 2012 6:40 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| Sam Uppu wrote: |
| Will broker not release space after it processes messages? |
It depends.
In newer levels of Broker, generally, it won't.
It will, however, *reuse* the memory. So the memory in use by the EG will grow to a stable maximum based on the necessary usage of the flow. |
|
| Back to top |
|
|
| Sam Uppu |
| Posted: Mon Mar 19, 2012 7:13 am Post subject: |
|
|
Yatiri
Joined: 11 Nov 2008
Posts: 610
|
| mqjeff wrote: |
It depends.
In newer levels of Broker, generally, it won't.
It will, however, *reuse* the memory. So the memory in use by the EG will grow to a stable maximum based on the necessary usage of the flow. |
We have lot of transactions coming in. The memory is slowly keep on increasing. How frequently do we need to reload the EG? |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Mar 19, 2012 7:21 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| Sam Uppu wrote: |
| How frequently do we need to reload the EG? |
How frequently do you exceed the amount of available memory?
An alternative way of looking it at it is this: if the memory allocated to an EG increases until it reaches a stable maximum for the workload, do you need more memory for the workload or redeisgn the flow to use less memory? It can't be right to be routinely reloading the EG just to free up memory; I've never done it just for that in all my years. It also occurs to me that there's a danger whatever's causing the memory to increase will one day spike between reloads & catch you out.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| zpat |
| Posted: Mon Mar 19, 2012 7:42 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
I had an EG using 2 GB of memory. When I reloaded it - the usage did not go down. It went down when I stopped and started the EG.
Why would this be? |
|
| Back to top |
|
|
| mqjeff |
| Posted: Mon Mar 19, 2012 7:48 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| zpat wrote: |
I had an EG using 2 GB of memory. When I reloaded it - the usage did not go down. It went down when I stopped and started the EG.
Why would this be? |
Because that's the way it's written to work? Because the relaod immediately reallocated all of the same amount of memory due to work-in-progress needing it?
You're thinking about the issue incorrectly, still.
It only represents a problem that an EG is holding onto 2GB of memory if the system does not have 2GB of memory to spare. In which case, you need to add more memory to the system.
It only represents a problem that an EG is holding onto 2GB of memory if you are reasonably sure that this represents a leak rather than necessary usage for accomplishing the business requirements.
It does not sound like you have done sufficient analysis to determine that there is a leak. Maybe, however, you have, and simply haven't explained this.
2GB is not a reason to panic. |
|
| Back to top |
|
|
| zpat |
| Posted: Mon Mar 19, 2012 7:53 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
It uses up paging space and if every EG did this, we would run out.
This EG was not processing anything, but the mqsireload still did not free up the memory. A peak load might not occur again for a month.
I think a periodic release of unneeded memory (when the EG has been idle for a configurable period of time) would be a good thing.
You can say "this is how it works" all you like, but it does not make it the best way to run an operational service, with finite resources.
Do know how much hassle it is to add (real) memory to a production 24x7 server? |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Mar 19, 2012 8:10 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| zpat wrote: |
| I think a periodic release of unneeded memory (when the EG has been idle for a configurable period of time) would be a good thing. |
I'm sure your enhancement request will be reviewed favourably. Once implemented, I look forward to the first post from someone who's set the idle time to 30 seconds and complains about poor performance.
| zpat wrote: |
| You can say "this is how it works" all you like, but it does not make it the best way to run an operational service, with finite resources. |
You still haven't explained why it's a problem.
| zpat wrote: |
| Do know how much hassle it is to add (real) memory to a production 24x7 server? |
Yes I do know. What I'm having trouble with is a 24x7 production server with so little resource that 2Gb being held is an issue and "if every EG did that we would run out".
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| sridhsri |
| Posted: Mon Mar 19, 2012 8:36 am Post subject: |
|
|
Master
Joined: 19 Jun 2008
Posts: 297
|
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Mon Mar 19, 2012 2:53 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
zpat, what O/S and WMB version? If you do a search for the word "leak" on the Issues Fixed page on IBM's WMB support page you'll see both WMB 7 and WMB 6 have several fixes for memory leaks.
We are grappling with this now. The memory profile on the server shows it having less and less available memory over several weeks until it locks up and needs a reboot. We are at RHEL 5 with WMB 6.1.0.9. What exactly is leaking the memory is TBD, but these servers are dedicated to WMB.
Does your EG's memory profile, or the overall server's, continue to slowly rise over time, or does it stop at 2 GB and stabalize, and there is plenty to spare? And/or do you reboot often enough that you are not able to see if it would rise until the server had issues?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| zpat |
| Posted: Tue Mar 20, 2012 10:00 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
The thing is - we have finite paging space (24 GB) - it wouldn't take many big EGs to use this up if their peak memory is never being released. We have a new WMB project which seems to use a lot of memory (maybe due to Java?).
I view this as very similar to MQ using disk space. When messages are removed from the queue, the actual disk space is not immediately released - but the scavenger tasks runs now and then (and on certain events) to do so.
Looks like we need something similar for WMB memory. Just because it might be possible to set the scavenge interval to a stupidly low value is not a reason to refuse to provide a feature at all.
They are many MQ and WMB settings that can be set to stupid values - but as long as the default is sensible (e.g. one hour) then it should be fine and the documentation can advise on the side effect of a smaller value.
I don't see a need for a small value, but the current value is effectively infinite which is also undesirable, and it can't be changed, other than using AIX disclaim memory, which does have a major performance impact. |
|
| Back to top |
|
|
|
|
