| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
| Delay a MQ processing flow with ESQL SLEEP |
« View previous topic :: View next topic » |
| Author |
Message
|
| hunghip95 |
 Posted: Tue Nov 24, 2020 7:16 pm Post subject: Re: Delay a MQ processing flow with ESQL SLEEP |
 |
|
Apprentice
Joined: 12 Oct 2017
Posts: 30
|
| gbaddeley wrote: |
| hunghip95 wrote: |
| gbaddeley wrote: |
| hunghip95 wrote: |
Hi all,
I want to share you all a process and expect that whether there is a better suggestion... |
I don't understand why a delay is required.
Why don't you use
MQInput(queueA.in)-> Business Process -> MQOutput(queueA.out)
and run multiple instances of the flow?
If the BP can't keep up, the messages will queue up on queueA.in until BP capacity is available. |
The delay time allows the "Business Process" sector to keep handling messages at a slow speed. For example: with delay time = 10s, Service 1 has 1 instance, so maximal speed of "Business Process" sector is just 0.1message/s ( 10s for 1 message).
And you can speed it up by decreasing this delay time to be lower or to be 0. |
Can you provide more information about how the Business Process interface works? Is it a web service call, and the message flow waits for a response? Is the web service single threaded (can only handle 1 request at a time) ?
We should try to find a solution using MQ queueing that does not require dodgy rate throttling or sleeping. |
Honestly, its my customer's requirement. We are develop team.
Exactly, "Business Process" sector is a REST calling out to Core Banking system. First of all, Core Banking is weak sometimes. Besides, this solution is just supposed for some low priority Business Services which allows handling later as an asynchronous process.
For example: ticket selling service, this service allows the Bank selling ticket immediately, though the real money is not computed at the same time. |
|
| Back to top |
|
 |
| gbaddeley |
| Posted: Wed Nov 25, 2020 2:26 pm Post subject: Re: Delay a MQ processing flow with ESQL SLEEP |
|
|
Jedi Knight
Joined: 25 Mar 2003
Posts: 2538
Location: Melbourne, Australia
|
| hunghip95 wrote: |
... Honestly, its my customer's requirement. We are develop team.
Exactly, "Business Process" sector is a REST calling out to Core Banking system. First of all, Core Banking is weak sometimes. Besides, this solution is just supposed for some low priority Business Services which allows handling later as an asynchronous process.
For example: ticket selling service, this service allows the Bank selling ticket immediately, though the real money is not computed at the same time. |
Is their requirement that the Core Banking REST only be called every N seconds, at most? N = 10, or any other setting.
Get a message
BeginTime = Get current time
REST call
EndTime = Get current time
If EndTime - BeginTime < N Then Sleep ( BeginTime + N - EndTime )
_________________
Glenn |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Wed Nov 25, 2020 3:36 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
Seems like over-engineering (look up Rube Goldberg)- delaying lower priority workload in anticipation of higher priority workload arriving.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| gbaddeley |
| Posted: Thu Nov 26, 2020 2:07 pm Post subject: |
|
|
Jedi Knight
Joined: 25 Mar 2003
Posts: 2538
Location: Melbourne, Australia
|
| bruce2359 wrote: |
| Seems like over-engineering (look up Rube Goldberg)- delaying lower priority workload in anticipation of higher priority workload arriving. |
Agree. If the incoming messages arrive at a faster average rate than what the REST service is allowed to process, the backlog will increase indefinitely. It's a losing game.
We have a lot of flows that call REST web services. Some can be slow to respond (1 - 2 minutes) due to intensive back end processing that they do. We don't sleep, we just let the request messages queue up in MQ, and they are processed when the response time allows.
_________________
Glenn |
|
| Back to top |
|
|
| hunghip95 |
| Posted: Mon Nov 30, 2020 3:17 am Post subject: |
|
|
Apprentice
Joined: 12 Oct 2017
Posts: 30
|
| gbaddeley wrote: |
| bruce2359 wrote: |
| Seems like over-engineering (look up Rube Goldberg)- delaying lower priority workload in anticipation of higher priority workload arriving. |
Agree. If the incoming messages arrive at a faster average rate than what the REST service is allowed to process, the backlog will increase indefinitely. It's a losing game.
We have a lot of flows that call REST web services. Some can be slow to respond (1 - 2 minutes) due to intensive back end processing that they do. We don't sleep, we just let the request messages queue up in MQ, and they are processed when the response time allows. |
Thank you, it seems that we should use the maximum rate in workload management.
Hmm, REST calling out is a tricky issue. |
|
| Back to top |
|
|
| gbaddeley |
| Posted: Mon Nov 30, 2020 2:41 pm Post subject: |
|
|
Jedi Knight
Joined: 25 Mar 2003
Posts: 2538
Location: Melbourne, Australia
|
| hunghip95 wrote: |
| Hmm, REST calling out is a tricky issue. |
REST calls are inherently unreliable. They can fail with TCP socket errors due to network glitches or outages, or http server issues at the destination (500 code), or calls that take much longer than expected, or time out, or TLS certificates expire, or credentials change, or the service is moved to another host or ip address without informing you, or the URI format has changed. Things over which you have no direct control.
Your message flows should not break in these situations. Incoming transactions should queue up until the issue has been resolved, and have a periodic retry mechanism.
_________________
Glenn |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Tue Dec 01, 2020 3:55 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| gbaddeley wrote: |
| hunghip95 wrote: |
| Hmm, REST calling out is a tricky issue. |
REST calls are inherently unreliable. They can fail with TCP socket errors due to network glitches or outages, or http server issues at the destination (500 code), or calls that take much longer than expected, or time out, or TLS certificates expire, or credentials change, or the service is moved to another host or ip address without informing you, or the URI format has changed. Things over which you have no direct control.
Your message flows should not break in these situations. Incoming transactions should queue up until the issue has been resolved, and have a periodic retry mechanism. |
I agree with you, I paraphrased this today more than once.
Is there anything more "official" regarding this line of thinking, a Technote or Conference presentation I can point people at? While I may trust gbaddeley, people that don't know him would just read that and go full Lebowski: "Yeah, well, you know, that's just, like, your opinion, man."
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| hunghip95 |
| Posted: Wed Dec 02, 2020 12:26 am Post subject: |
|
|
Apprentice
Joined: 12 Oct 2017
Posts: 30
|
To be honest, this topic is going far away from its purpose 
We need to keep this topic good enough for the similar problems. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Wed Dec 02, 2020 7:47 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| hunghip95 wrote: |
To be honest, this topic is going far away from its purpose
We need to keep this topic good enough for the similar problems. |
The OP pondered the wisdom and feasibility of imbedding a delay in a business process (that involves MQ). I’d say that the consensus here is that such a requirement is not a good practice.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| gbaddeley |
| Posted: Wed Dec 02, 2020 2:21 pm Post subject: |
|
|
Jedi Knight
Joined: 25 Mar 2003
Posts: 2538
Location: Melbourne, Australia
|
| PeterPotkay wrote: |
| "Yeah, well, you know, that's just, like, your opinion, man." |
There is plenty of opinion on this forum, that's for sure. Fortunately there is more consensus than bickering and disagreement. Practical MQ patterns are not always self evident. Poor design will soon lead to grief.
_________________
Glenn |
|
| Back to top |
|
|
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
