| Author |
Message
|
| Mercator |
 Posted: Tue Aug 24, 2010 7:27 am Post subject: Common practices for subscribing to Monitoring Messages |
 |
|
Apprentice
Joined: 21 Jul 2009
Posts: 34
|
Pub/Sub is not my strong point so I apologize in advance. I haven't been able to find a good answer to my following question.
How do people handle subscriptions in an environment where the topic they are interested in never changes?
Ex. We have deployed business flows generating monitoring events. We have one utility flow that processes monitoring events. This utility flow needs a subscription to be set up once. The subscribed topic uses wildcards so even if new business flows are deployed, the original subscription will handle it. This is the only place in the environment where we are using pub/sub.
In this scenario I assume I have three options:
1. I can reate the subscription manually.
2. I can create the subscription within the utility flow.
3. I can create another flow that handles subscription creation.
I understand why flows / applications should handle their own subscriptions (option #2 & #3) in dynamic environments, but it seems to be unnecessary overhead in the described scenario.
Thoughts?
Our admins do not like #1 since they are the ones who would have to do it in production, albeit once.
Thanks in advance. |
|
| Back to top |
|
 |
| smdavies99 |
| Posted: Tue Aug 24, 2010 11:04 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
I've just looked at this very thing.
In order to minimise the possibe errors, Ive done the following.
1) In my flow that is receiving the publications, I have added an MQInput node.
This is set to receive a BLOB as I don't care about the format.
This is connected to a Compute Node. Here I add the RFH2 Header and setup the psc folder with the subscription information.
The out terminal is connected to an MQOutput Node that send the subscription to SYSTEM.BROKER.CONTROL.QUEUE.
So when the flow is deployed the sysadmins send something to the InputQ and the flow then makes sure that the subscription is sent. This solves your 1) problem.
It also provides a mechanisn if by chance some Admin cancels the subscription it can be renewed.
That is not the end of the story if you are actually processing the FlowMonitoring events with a message flow.
Ask yourself what happens if someone does a...
| Code: |
mqsichangeflowmonitoring <brokername> -g -j
|
Then you have to code for this eventuality.
Then, how are you matching the events? TransactionStart with TransactionEnd/TransactionRollback
You can use a collector node but then you have to worry about unmatched events. (These do happen especially if the EG abends...)
etc etc etc.(error processing connected to the Collector node Catch terminal)
As I said at the beginning, this is something that I am currently familiar with...
(lets out a big sigh)
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| Mercator |
| Posted: Tue Aug 24, 2010 11:33 am Post subject: |
|
|
Apprentice
Joined: 21 Jul 2009
Posts: 34
|
Thanks for the response.
| smdavies99 wrote: |
1) In my flow that is receiving the publications, I have added an MQInput node.
This is set to receive a BLOB as I don't care about the format.
This is connected to a Compute Node. Here I add the RFH2 Header and setup the psc folder with the subscription information.
The out terminal is connected to an MQOutput Node that send the subscription to SYSTEM.BROKER.CONTROL.QUEUE.
So when the flow is deployed the sysadmins send something to the InputQ and the flow then makes sure that the subscription is sent. This solves your 1) problem.
It also provides a mechanisn if by chance some Admin cancels the subscription it can be renewed. |
We thought about doing something similar, but its still hardcoding the topic information unless you put it in your input message and parse it with the flow. That is ultimately what we will probably do long term. But for now we create the subscription with RFHUtil from a saved message so there is no chance of any typos.
I am not sure what you mean by:
| smdavies99 wrote: |
Ask yourself what happens if someone does a...
| Code: |
mqsichangeflowmonitoring <brokername> -g -j
|
Then you have to code for this eventuality. |
That command would be used to set monitoring control to either 'active' or 'inactive'. I can see how it would affect the publication of the events, but how would it affect subscriptions? I can set monitoring to 'inactive' and the subscription remains. What am I missing?
| smdavies99 wrote: |
Then, how are you matching the events? TransactionStart with TransactionEnd/TransactionRollback
You can use a collector node but then you have to worry about unmatched events. (These do happen especially if the EG abends...)
|
We are lucky in that we don't have to match events in a flow. We simply load information extracted from the events in to a database. One of the fields is a transaction ID from the parsed message that we include as a simpleContent field within the event message. That transaction ID is then used to match database records.
Thanks again. |
|
| Back to top |
|
|
| kirank |
| Posted: Tue Aug 24, 2010 2:45 pm Post subject: |
|
|
Centurion
Joined: 10 Oct 2002
Posts: 136
Location: California
|
I would recommend having a Message flow to register subscribers. I have been using a flow to do that for last 3 years and it works well with the admins as well. I created a XML message for registering subscribers. Every time we have a new subscriber either for an existing Publication or for a new publication, we just create an XML message and give it to Admins for Deployment. The XML file needs queue name and topic name to changed based on the publication. Admins run a script which drops the XML message to input queue of the flow and the flow registers the subscribers.
Regards
Kiran |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Tue Aug 24, 2010 10:52 pm Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| Mercator wrote: |
I am not sure what you mean by:
| smdavies99 wrote: |
Ask yourself what happens if someone does a...
| Code: |
mqsichangeflowmonitoring <brokername> -g -j
|
Then you have to code for this eventuality. |
|
Well,
if you were to actually do
| Code: |
mqsichangeflowmonitoring <brokername> -g -j -c active
|
For every event message you processed broker would emit TWO more relating to the transactionStart & transactionEnd events for the flow that processed that event message.
Then for the next Single event TWO more.
A gives you a case of positive feedback which I am sure it is something you want to avoid.
Onto the subscriptions.
Unless you want to get complicated IMHO, it is better to control the emission of events by the mqsichangeflowmonitoring command and have a single wildcard subscription.
We are in the process of moving to a Maven based system of Barfile building & deployment(along with migration to V7). This system is setup to automatically enable flowmonitoring after the deployment of a message flow unless it is to a specific Execution Group. This is where the EvenMonitoring flow runs but as I indicated, even it has the ability to turn off event monitoring for itself.
I call this 'defensive programming'.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| Mercator |
| Posted: Wed Aug 25, 2010 6:03 am Post subject: |
|
|
Apprentice
Joined: 21 Jul 2009
Posts: 34
|
| smdavies99 wrote: |
Well,
if you were to actually do
| Code: |
mqsichangeflowmonitoring <brokername> -g -j -c active
|
For every event message you processed broker would emit TWO more relating to the transactionStart & transactionEnd events for the flow that processed that event message.
Then for the next Single event TWO more.
A gives you a case of positive feedback which I am sure it is something you want to avoid. |
Ok, I understand your point now. But it was my understanding that for events to be emitted from a flow, two things had to occur. First, you had to configure the events within the flow or a monitoring profile. Second, activate monitoring via mqsichangeflowmonitoring. In your scenario, the events would never have been configured on the flow processing the event messages, therefore it should never emit TransactionStart or TransactionEnd event regardless if it has been set to 'active'. Is my understanding incorrect? Have you experienced a case where just activating monitoring has caused a flow to emit TransactionStart / TransactionEnd events even though they have not been configured within the flow/profile?
| smdavies99 wrote: |
Onto the subscriptions.
Unless you want to get complicated IMHO, it is better to control the emission of events by the mqsichangeflowmonitoring command and have a single wildcard subscription. |
I agree. Thats basically how we have our environment set up. We have a single subscription for events generated by all flows in the "business" EG.
Thanks. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Wed Aug 25, 2010 10:07 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
We aren't using moitor profiles.
So, given that caveat....
Once you have explicitly enabled flow monitoring for a flow or used a wildcard (-g -j) approach,
Any message process by that flow will emit a minimum of TWO Events.
These are transactionStart, AND one of either transactionEnd or TranactionRollback.
If you add specific node events there are in addition to the standard ones.
What I'm saying is that if you (or an illinformed Admin) accidentally do
| Code: |
mqsichangeflowmonitoring <broker> -g -j -c active
|
you have by default enabled the emission of transactionStart etc events from the flow that is receiving these events.
That is what I meant by 'positive feedback'. For every one consumed, two more will appear on the input queue.
Opps.
Thus the monitoring flow much check that the MessageFlowLabel contained in an event message is not the same as its own (ie you are consuming you own dog food).
If this is the case then you can resiove the issue by uaing the JobControl Support Pack node and execute (in an asynchronous mode by the way)
| Code: |
mqsichangeflowmonitoring <broker> -e <my EG> -f <my flowLabel> -c inactive
|
I had a post about this last thing a week or so ago. It explaines the reasoning why you can't wait for the changeflow... to complete.
So you have to have some caching mechanism in your flow to stop say 50 such requests being issued before the async nature of the request takes effect.
And I here I was thinking that making a flow consume other flow's monitoring events was going to be simple....
 
[edited to correct my silly typos]
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions.
Last edited by smdavies99 on Wed Aug 25, 2010 11:53 am; edited 1 time in total |
|
| Back to top |
|
|
| Mercator |
| Posted: Wed Aug 25, 2010 10:20 am Post subject: |
|
|
Apprentice
Joined: 21 Jul 2009
Posts: 34
|
| Thanks very much. Thats some excellent information. |
|
| Back to top |
|
|
|
|
