| Author |
Message
|
| shashivarungupta |
 Posted: Tue Mar 24, 2009 9:56 pm Post subject: Default size of an EG ? |
 |
|
Grand Master
Joined: 24 Feb 2009
Posts: 1343
Location: Floating in space on a round rock.
|
Hi Folk,
What is the default size being allocated to an Execution Group when an EG is created?
Thanks  |
|
| Back to top |
|
 |
| Vitor |
| Posted: Tue Mar 24, 2009 10:00 pm Post subject: Re: Default size of an EG ? |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| shashivarungupta wrote: |
| What is the default size being allocated to an Execution Group when an EG is created? |
Size of what? Disk used? Memory? Name? Queue depth? 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| shashivarungupta |
| Posted: Tue Mar 24, 2009 10:04 pm Post subject: Re: Default size of an EG ? |
|
|
Grand Master
Joined: 24 Feb 2009
Posts: 1343
Location: Floating in space on a round rock.
|
| Vitor wrote: |
| shashivarungupta wrote: |
| What is the default size being allocated to an Execution Group when an EG is created? |
Size of what? Disk used? Memory? Name? Queue depth? |
"Memory and Disk space". |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Tue Mar 24, 2009 10:39 pm Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
AFAIK, an Execution Group is a Memory only construct.
By that I mean that the broker will create the EG according to the data held in the Broker Database. It will then add/load the message flows, message sets and other deployable objects to the running process again as defined in the broker database.
On some platforms you will need some Swap/pagefile space but again this is dynamically created.
The size of the above is again very dependant upon the flows, messages sets etc that are deployed into that EG.
Again, AFAIK there is no hard & fast rule as to the amount of memory used by an EG
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| shashivarungupta |
| Posted: Wed Mar 25, 2009 1:54 am Post subject: |
|
|
Grand Master
Joined: 24 Feb 2009
Posts: 1343
Location: Floating in space on a round rock.
|
| smdavies99 wrote: |
AFAIK, an Execution Group is a Memory only construct.
By that I mean that the broker will create the EG according to the data held in the Broker Database. It will then add/load the message flows, message sets and other deployable objects to the running process again as defined in the broker database.
On some platforms you will need some Swap/pagefile space but again this is dynamically created.
The size of the above is again very dependant upon the flows, messages sets etc that are deployed into that EG.
Again, AFAIK there is no hard & fast rule as to the amount of memory used by an EG |
Thanks for the quick reply.
| Quote: |
The size of the above is again very dependant upon the flows, messages sets etc that are deployed into that EG.
|
I have a question again, if the size/memory/disk space of EG depends on the mflow/mset/deployable elements then do you mean to say we can deploy any number of mflows to that EG or a limited to them.
I think, as each Mflow occupies its 'x' amount of memory. And that 'x' can very in terms of memory space depending upon the mflow. So there has to be some limit of memory space defined on EG so that we can deploy a limited set of deployable elements.
For example: 'x', 'y', 'z' are the memory spaces occupied by mflows and 'O' is the total memory space occupied by EG after deployment of mflows.
'T' is the total memory space of EG (default)
'x' + 'y' + 'z' = 'O'
Available memory = 'T' - 'O'
What do you say ? |
|
| Back to top |
|
|
| WMBDEV1 |
| Posted: Wed Mar 25, 2009 2:38 am Post subject: |
|
|
Sentinel
Joined: 05 Mar 2009
Posts: 888
Location: UK
|
| smdavies99 wrote: |
AFAIK, an Execution Group is a Memory only construct.
By that I mean that the broker will create the EG according to the data held in the Broker Database. It will then add/load the message flows, message sets and other deployable objects to the running process again as defined in the broker database.
On some platforms you will need some Swap/pagefile space but again this is dynamically created.
The size of the above is again very dependant upon the flows, messages sets etc that are deployed into that EG.
Again, AFAIK there is no hard & fast rule as to the amount of memory used by an EG |
,
On Windows and AIX (and I believe other *NIX platforms) the default memory (RAM) limit is 1GB per execution group. This can be upped to 2 GB on said platforms and 3GB on Solaris. Assuming a 32 bit architecture. |
|
| Back to top |
|
|
| WMBDEV1 |
| Posted: Wed Mar 25, 2009 2:43 am Post subject: |
|
|
Sentinel
Joined: 05 Mar 2009
Posts: 888
Location: UK
|
| shashivarungupta wrote: |
Thanks for the quick reply.
| Quote: |
The size of the above is again very dependant upon the flows, messages sets etc that are deployed into that EG.
|
I have a question again, if the size/memory/disk space of EG depends on the mflow/mset/deployable elements then do you mean to say we can deploy any number of mflows to that EG or a limited to them.
|
The baseline footprint of an EG is dependant upon the components deployed to it. However, the actually memory required during runtime is also dependant upon the size of the input message(s) and the type of manipulations / domains you work in. |
|
| Back to top |
|
|
| shashivarungupta |
| Posted: Wed Mar 25, 2009 3:19 am Post subject: |
|
|
Grand Master
Joined: 24 Feb 2009
Posts: 1343
Location: Floating in space on a round rock.
|
| Quote: |
On Windows and AIX (and I believe other *NIX platforms) the default memory (RAM) limit is 1GB per execution group. This can be upped to 2 GB on said platforms and 3GB on Solaris. Assuming a 32 bit architecture. |
Could you please provide me some reference to this statement.
That would be nice !!
Appreciate that.
Thanks |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Mar 25, 2009 3:28 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
There are at least *two* separate memory containers in a single Broker EG - the C++Runtime and the Java Runtime.
The Java Runtime is tunable and limited by normal Java mechanisms. This is thoroughly documented in the InfoCenter, although it may take just a little bit of searching to find it.
The C++ Runtime is managable by OS level process limits, but should generally be assumed to be capable of consuming the entire available memory on the machine in cases where things have gone wrong or where messages are larger than you have capacity for (A 1 GB message read from FileInput could consume up to five or six times that in RAM depending on how your flow works and which parser you use and etc. etc. etc.) |
|
| Back to top |
|
|
| ramires |
| Posted: Wed Mar 25, 2009 3:35 am Post subject: |
|
|
Knight
Joined: 24 Jun 2001
Posts: 523
Location: Portugal - Lisboa
|
|
| Back to top |
|
|
| ramires |
| Posted: Wed Mar 25, 2009 3:40 am Post subject: |
|
|
Knight
Joined: 24 Jun 2001
Posts: 523
Location: Portugal - Lisboa
|
| mqjeff wrote: |
| There are at least *two* separate memory containers in a single Broker EG - the C++Runtime and the Java Runtime |
When I do a ps -elf | grep DataflowEngine32 I see the memory the process has allocated. This amount displayed is the sum of those two containers?
Thanks
ramires |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Mar 25, 2009 3:47 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| ramires wrote: |
| mqjeff wrote: |
| There are at least *two* separate memory containers in a single Broker EG - the C++Runtime and the Java Runtime |
When I do a ps -elf | grep DataflowEngine32 I see the memory the process has allocated. This amount displayed is the sum of those two containers |
Both containers exist only inside the same process. There is not a separate process for the JVM. So, yes, the amount of memory that the OS reports for the DFE process should be the total memory in use at the time by the EG as a whole. |
|
| Back to top |
|
|
| WMBDEV1 |
| Posted: Wed Mar 25, 2009 3:56 am Post subject: |
|
|
Sentinel
Joined: 05 Mar 2009
Posts: 888
Location: UK
|
I have a document I got from raising a PMR with IBM that says:
"Before discussing the storage taken by a message flow to process a large message it is sensible to go through the maximum amount of storage such a
message flow has available to it. A message flow is assigned to an execution group which exists as a DataFlowEngine process in the operating system. Like
any other operating system process, it has an address space that contains the executable program code, stack storage and heap storage etc. For the V5
broker product and below, the DataFlowEngine processes are 32-bit applications, and even when executed in a 64 bit operating system, they still run as
32-bit applications. A 32 bit application has 4 byte pointers and hence the maximum address possible is 2 to the power of 32 = 4294967296 (4GB). Therefore
a 32 bit application has the capability to address up to a maximum address of 4GB and hence such an applications address space could never grow past
4GB. Although this is the maximum that the architecture will allow this does not mean that a DataFlowEngine process has this much storage available to it. In
fact if you read the documentation, particularly page au16640_, you will see that the following maximum DataFlowEngine sizes are stated :
1. Windows : 2GB
2. Solaris : 3.75GB
3. HP-UX : About 3GB dependent on Kernel settings.
4. AIX : 2GB
5. zOS : 2GB"
The document is called 2783prb_largemessages.pdf. Hope that helps |
|
| Back to top |
|
|
| shashivarungupta |
| Posted: Thu Mar 26, 2009 4:35 am Post subject: |
|
|
Grand Master
Joined: 24 Feb 2009
Posts: 1343
Location: Floating in space on a round rock.
|
| ramires wrote: |
| mqjeff wrote: |
| There are at least *two* separate memory containers in a single Broker EG - the C++Runtime and the Java Runtime |
When I do a ps -elf | grep DataflowEngine32 I see the memory the process has allocated. This amount displayed is the sum of those two containers?
Thanks
ramires |
$ ps -elf | grep DataFlowEngine32
F S UID PID PPID C PRI NI ADDR SZ WCHAN STIME TTY TIME CMD
202001 A mqsiprd 483554 303562 0 60 20 1d283e400 224 f10001100ff34308 08:29:55 pts/4 0:00 grep DataFlowEngine32
224 is the figure that I got.
My O.S. is AIX. |
|
| Back to top |
|
|
| ramires |
| Posted: Thu Mar 26, 2009 4:58 am Post subject: |
|
|
Knight
Joined: 24 Jun 2001
Posts: 523
Location: Portugal - Lisboa
|
I you do that you may have more than one line as output, look for the one with the name of the EG, and you are using 64 bits EG try a ps -elf | DataFlowEngine, in my example was for a 32bit EG.
regards
ramires |
|
| Back to top |
|
|
|
|
