| Author |
Message
|
| flahunter |
 Posted: Thu Apr 22, 2010 8:51 pm Post subject: Help on this situation |
 |
|
Acolyte
Joined: 30 Oct 2008
Posts: 62
|
Hi,
I got a requirement that if a lot of input messages come at the same time, we need to process them one by one at a time interval 1 second.
Is there anyone who can give me some advice how to achieve this? |
|
| Back to top |
|
 |
| Vitor |
| Posted: Thu Apr 22, 2010 9:09 pm Post subject: Re: Help on this situation |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| flahunter wrote: |
| I got a requirement that if a lot of input messages come at the same time |
Define "a lot" and "the same time". Are you saying that if you get 10 messages over 10 seconds you process them as they arrive but if 10 messages arrive in 1 second you need to space them out? What if "a lot" is 1000 messages but you only get 999 is that still a lot?
| flahunter wrote: |
| we need to process them one by one at a time interval 1 second |
Process them through what? Given that broker can handle large message volumes easily, what external component requires this throttling?
| flahunter wrote: |
| Is there anyone who can give me some advice how to achieve this? |
Ask yourself why IBM has spent years perfecting a product that can handle large data volumes easily, then ask why you're fighting that.
Seriously, why this requirement? Surely it makes more sense to work on removing a bottleneck rather than bending WMB against it's will to pander to it.
As you're likely to come back with "this is the client's requirement and I can't change it", my suggestion is a Java Compute Node with a sleep statement in it. This will cause the EG to suspend for the length of the sleep and hence prevent the next message being picked up.
How you inform the JCN that "too many" messages have arrive in "too short a time" I've no clue. You could of course leave this rather odd mechanism in place permanently but that's going to adversly affect WMB performance.
I'm assuming you have other requirements that have messages processed more expidiously.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| flahunter |
| Posted: Thu Apr 22, 2010 9:29 pm Post subject: |
|
|
Acolyte
Joined: 30 Oct 2008
Posts: 62
|
Hi, Vitor
Thank you for your reply.
For question1:
Just as you said, if I get 10 messages over 10 seconds I process them as they arrive but if 10 messages arrive in 1 second I need to space them out
For question2:
I have to use MB because it's a minor new requirement from a project which has already been done.
For question3:
Besides JAVACOMPUTE node,is there any other choice for this situation. I just took a look at Timeout and TimeNotification node, seems it's not fit for this requirement which is used for trigger a flow. |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Apr 22, 2010 9:39 pm Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| flahunter wrote: |
| Just as you said, if I get 10 messages over 10 seconds I process them as they arrive but if 10 messages arrive in 1 second I need to space them out |
So what happens if you get 9 messages in a second, or 11 messages in 1.2 seconds? I'm trying to get a handle on how you're measuring this, and from that how you could signal the flow.
| flahunter wrote: |
| I have to use MB because it's a minor new requirement from a project which has already been done. |
That's not what I asked. I asked why, given that WMB can easily process 10 messages a second & more, you're trying to introduce a delay in broker rather than fixing this bottleneck (which you've still not explained).
| flahunter wrote: |
| Besides JAVACOMPUTE node,is there any other choice for this situation. I just took a look at Timeout and TimeNotification node, seems it's not fit for this requirement which is used for trigger a flow. |
Not really. WMB (as I indicated) is designed to process high volumes quickly. You're working against this and hence there are no easy ways to do this. The JCN method has serious drawbacks & I agree the Timer nodes are unsuitable; they're intended for other uses.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| flahunter |
| Posted: Thu Apr 22, 2010 9:57 pm Post subject: |
|
|
Acolyte
Joined: 30 Oct 2008
Posts: 62
|
Hi, Vitor
Thank you. Seems JAVACompute is the best way to achieve this so far.
Yes, I totally agree with you that WMB can process messages quickly. But I have to do the oppsite thing. Because we are trying to generate a file with a value of YYYYMMDDHHmmss (Based on client's requirement and it should be unique) but sometimes as you know several messages may be processed by WMB at one second interval, which may generate the same value.And it's not acceptable. And then comes a requirement that we need to process them one by one at one second interval.
I'm trying to check if MQGet node can do this job |
|
| Back to top |
|
|
| mqjeff |
| Posted: Fri Apr 23, 2010 3:31 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Use a TimeoutNotification node.
Wire it to an MQGet node. get one message, and output it.
If you need to control the rate that messages are sent out, you can use something to calculate what the rate should be and then use a TimeoutControl. |
|
| Back to top |
|
|
| flahunter |
| Posted: Mon Apr 26, 2010 1:05 am Post subject: |
|
|
Acolyte
Joined: 30 Oct 2008
Posts: 62
|
| mqjeff wrote: |
Use a TimeoutNotification node.
Wire it to an MQGet node. get one message, and output it.
If you need to control the rate that messages are sent out, you can use something to calculate what the rate should be and then use a TimeoutControl. |
That's a good idea. But I'm considering this situation: Only 100 input messages comes at 8:00 every day. There is no need to notify the flow to get it at the designed rate the whole day. So in this situation, I prefer to add a JAVACOMPUTE node to let the thread sleep for a while. Anyway, thank you so much for your kind advice:) |
|
| Back to top |
|
|
|
|
