| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
| determine last record in the Queue based on input message |
« View previous topic :: View next topic » |
| Author |
Message
|
| smdavies99 |
 Posted: Thu Dec 05, 2013 2:59 am Post subject: |
 |
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| christopher j wrote: |
Read the messages one by one and store it in a shared variable untill you read the 'Last Record Indicator' message.
Once you received the 'Last Record Indicator' message, propagate the shared variable data(if u want to send the group data as a separate message) or store it in a separate variable and start a next collection by using the 'Last Record Indicator' message as a first record. |
Personally, I'd store this in a DB table. What happens if you failover right in the middle of a group of messages? If the data is stored in memory then you gave lost everything... 
Also you have to make sure that there are no groups of message 'in process' when you stop the flow.
What happens if you get groups of messages all mixed up?
etc
etc
etc
etc
There is more to this than at first meets the eye.
I implemented a solution like this when we had problems with open ended collections using the collector node. It has been running like for over a year without problem and has processed in excess of 1,250,000 collections.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
 |
| rampriya |
| Posted: Mon Dec 09, 2013 2:59 pm Post subject: |
|
|
Newbie
Joined: 27 Jul 2009
Posts: 9
|
| smdavies99 wrote: |
| christopher j wrote: |
Read the messages one by one and store it in a shared variable untill you read the 'Last Record Indicator' message.
Once you received the 'Last Record Indicator' message, propagate the shared variable data(if u want to send the group data as a separate message) or store it in a separate variable and start a next collection by using the 'Last Record Indicator' message as a first record. |
Personally, I'd store this in a DB table. What happens if you failover right in the middle of a group of messages? If the data is stored in memory then you gave lost everything...
Also you have to make sure that there are no groups of message 'in process' when you stop the flow.
What happens if you get groups of messages all mixed up?
etc
etc
etc
etc
There is more to this than at first meets the eye.
I implemented a solution like this when we had problems with open ended collections using the collector node. It has been running like for over a year without problem and has processed in excess of 1,250,000 collections. |
Would you be able to share simple project which will explain this, that would be really helpful for me.
Thanks.
Ramanujan |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Mon Dec 09, 2013 11:58 pm Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
It is not that difficult to put something together yourself. I did the implementation I spoke about while snowed in, in a hotel. It took me less than a day to put the whole thing together.
The key is to identify the element(s) that you need to make a collection unique. I was lucky in that this was easy to get in the data I was using.
So you make this bit the primary key. Then you cast the InputRoot as a blob (assuming that you have to parse it to get the key) and add that to the table along with the key and a timestamp.
There is also an index table that has the key and the time of the last update in it.
Then a timernode runes periodically(in my case ever 30 seconds) and does a select on the index table for any 'key' that has not has an update in the last 30 seconds. This is passed to the next stage that extracts all the data from the main table and builds the same message tree (apart from the MQMD and properties folders) that would come out of a collector node.
That is basically how it works. I can't share any code with you because of IP requirements (it would be too hard to obfuscate the client data).
So have a go and see what happens. Come back with any specific problems and I'll try to help.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
