| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Multiple execution groups and monitoring (record and replay) |
« View previous topic :: View next topic » |
| Author |
Message
|
| ruimadaleno |
 Posted: Fri Feb 06, 2015 3:53 am Post subject: Multiple execution groups and monitoring (record and replay) |
 |
|
Master
Joined: 08 May 2014
Posts: 274
|
Hi all,
our broker environment was installed with a single execution group (named "default"). The broker was configured to store events emitted by message flows in a database. This is done by "record and replay" (monitoring), that uses a MQ where message flow events are published, and configurable services to consume (through topic subscription) , a datacapturestore and datacapturesource.
Right now we are expanding the number of execution groups. We want that all message flows events that occur in the new message flows are stored in the record and replay database. So, our question here now is: what is the best (maybe the only) way to achieve this requirements ? Should we create a new datacapturesource + a new topic for each new execution group ? can the existing capturesouces + capturestores be configured to work with the new execution groups ?
my environment
WMB 8.0.0.4 running on windows 2008 R2
_________________
Best regards
Rui Madaleno |
|
| Back to top |
|
 |
| sumit |
| Posted: Fri Feb 06, 2015 11:36 am Post subject: |
|
|
Partisan
Joined: 19 Jan 2006
Posts: 398
|
In my opinion, the 'best way' varies from case to case. You would need these points (not limited to) for your analysis -
1. number of transactions happening in a min/an hour/a day, etc.
2. what all details are you capturing in your monitoring events.
3. Is payload captured or not.
4...
While creating datacapturesource, you specify a subscription string. You can use wildcard so that all events emitted under a broker goes to one queue and from there, to the DB.
You can also set it to an execution group level or a flow level. Depending upon the load, you can select an approprite load.
In our case, we utilized a dedicated exection group for datacapturestore.
_________________
Regards
Sumit |
|
| Back to top |
|
|
| ruimadaleno |
| Posted: Mon Feb 09, 2015 4:27 am Post subject: |
|
|
Master
Joined: 08 May 2014
Posts: 274
|
i'll try to ilustrate the environment, i think it will help us to gather better ideias 
Important info: in the past we had only one execution group (default). We have upgraded the broker license and now we are able to create several execution groups.
So, we are reorganizing execution groups (eg) , we plan to distribute message flows (mf) throughout this egs by class. We have found that our mf fit essentially in 3 classes:
1) Apps: mf executing business functions (providing info for user screens)
2) Utility: integration mf. This mf provide intregration between distinct core information systems like enterprise content management, core (as400), email system, fax system etc
3) Batch: as the name says, this mf provide integration between information systems in a "bacth" fashion
so, these are the eg created in broker , as you can see, we created a couple (two or three) eg for each of the mf classes:
Broker
--- default
--- eg_utility_01
--- eg_utility_02
--- eg_apps_02
--- eg_apps_03
--- eg_apps_04
--- eg_batch_01
for simplicity let's use the following abreviations:
Execution group - eg
message flow - mf
DatacaptureStore - DcStore
DatacaptureSource - DCsource
Our policy is to monitor every mf deployed in every eg. The main reason for that is because we use the monitoring to gather info about mf performance and usage (every mf developed and deployed in our environment emmits events on firts node - mainly soap inputs, and the last node)
So, we have on DCSource configured with "$Sys/Broker/<broker name>/Monitoring/#" , and this DCSource is using a DCStore to keep this events in a database.
The problem here is: the DCStore is configured with "egforRecord" = default and "egForView"=default (this was the only eg in our previous license). Querying the database we can only see records from events emitted by mf in the "default" eg.
Reading from documentation, my understanding is that we should create a DCSource + DCstore for every execution group:
Broker
| --- default
|------- DcSource ($sys/Broker/<broker name>/Monitoring/default/#)
|------- DcStore
|--- eg_utility_01
|------- DcSource ($sys/Broker/<broker name>/Monitoring/eg_utility_01/#)
|------- DcStore
|--- eg_utility_02
|------- DcSource ($sys/Broker/<broker name>/Monitoring/eg_utility_02/#)
|------- DcStore
|--- eg_apps_02
|------- DcSource ($sys/Broker/<broker name>/Monitoring/eg_apps_02/#)
|------- DcStore
|--- eg_apps_03
|------- DcSource ($sys/Broker/<broker name>/Monitoring/eg_apps_03/#)
|------- DcStore
| --- eg_apps_04
| ------- DcSource ($sys/Broker/<broker name>/Monitoring/eg_apps_04/#)
|------- DcStore
|--- eg_batch_01
|------- DcSource ($sys/Broker/<broker name>/Monitoring/eg_batch_01/#)
|------- DcStore
is this the right direction ? what are the pros/cons you seen in this approach ?
_________________
Best regards
Rui Madaleno |
|
| Back to top |
|
|
| mqjeff |
| Posted: Mon Feb 09, 2015 5:40 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| ruimadaleno wrote: |
| Reading from documentation, my understanding is that we should create a DCSource + DCstore for every execution group |
Which piece of the documentation? |
|
| Back to top |
|
|
| ruimadaleno |
| Posted: Mon Feb 09, 2015 6:36 am Post subject: |
|
|
Master
Joined: 08 May 2014
Posts: 274
|
|
| Back to top |
|
|
| mqjeff |
| Posted: Mon Feb 09, 2015 6:55 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
You need to create one or more DataCaptureSource configurable services that identify the topics to subscribe to and record.
The example given on your page shows '$SYS/Broker/myBroker/Monitoring/executionGroupName/msgFlowName' but this could just as easily be '$SYS/Broker/myBroker/Monitoring/*' or other wildcarded topic.
You need to create one or more DataCaptureStore configurable services to identify databases to store events in.
You need to nominate one EG to process events received from the subscription identified in each DataCaptureSource, and insert data into the DataCaptureStore. This EG is identified in the DataCaptureStore. It's the EG that records the events, not the EG that events are recorded from.
You can have a single EG that is used for nothing else, that processes all event messages from every EG in a broker. |
|
| Back to top |
|
|
| ruimadaleno |
| Posted: Mon Feb 09, 2015 7:36 am Post subject: |
|
|
Master
Joined: 08 May 2014
Posts: 274
|
Hi MQJeff, thank you for pointing another direction 
i might misunderstand documentation on this subject 
So, i've redesigned the approach:
Broker
| --- default
|--- eg_utility_01
|--- eg_utility_02
|--- eg_apps_02
|--- eg_apps_03
| --- eg_apps_04
|--- eg_batch_01
now, i'll need to configure a single DCSource + DCStore
DCSource ($sys/Broker/<broker name>/Monitoring/#)
DCStore (egforRecord = egforview= default)
Is this approach correct ? I have a single DCSource that get's all events from all eg on the broker. Then a single DCStore records this events to database.
Pros: simple approach, mininal config/change.
no changes if we create another eg
Cons: if we get heavy event traffic, the eg default can hit performance problems, because it's this execution group that is responsible for recording all events from topic to database.
_________________
Best regards
Rui Madaleno |
|
| Back to top |
|
|
| mqjeff |
| Posted: Mon Feb 09, 2015 8:03 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
You could, for example, set up two datacapture topics, one that records eg_util* and one that records eg_app*.
You otherwise seem to understand the pros and cons. |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
