ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum IndexWebSphere Message Broker SupportDelay a MQ processing flow with ESQL SLEEP

Post new topicReply to topic Goto page Previous  1, 2
Delay a MQ processing flow with ESQL SLEEP View previous topic :: View next topic
Author Message
hunghip95
PostPosted: Tue Nov 24, 2020 7:16 pm Post subject: Re: Delay a MQ processing flow with ESQL SLEEP Reply with quote

Apprentice

Joined: 12 Oct 2017
Posts: 30

gbaddeley wrote:
hunghip95 wrote:
gbaddeley wrote:
hunghip95 wrote:
Hi all,
I want to share you all a process and expect that whether there is a better suggestion...

I don't understand why a delay is required.
Why don't you use
MQInput(queueA.in)-> Business Process -> MQOutput(queueA.out)
and run multiple instances of the flow?
If the BP can't keep up, the messages will queue up on queueA.in until BP capacity is available.


The delay time allows the "Business Process" sector to keep handling messages at a slow speed. For example: with delay time = 10s, Service 1 has 1 instance, so maximal speed of "Business Process" sector is just 0.1message/s ( 10s for 1 message).

And you can speed it up by decreasing this delay time to be lower or to be 0.

Can you provide more information about how the Business Process interface works? Is it a web service call, and the message flow waits for a response? Is the web service single threaded (can only handle 1 request at a time) ?

We should try to find a solution using MQ queueing that does not require dodgy rate throttling or sleeping.


Honestly, its my customer's requirement. We are develop team.
Exactly, "Business Process" sector is a REST calling out to Core Banking system. First of all, Core Banking is weak sometimes. Besides, this solution is just supposed for some low priority Business Services which allows handling later as an asynchronous process.
For example: ticket selling service, this service allows the Bank selling ticket immediately, though the real money is not computed at the same time.
Back to top
View user's profile Send private message
gbaddeley
PostPosted: Wed Nov 25, 2020 2:26 pm Post subject: Re: Delay a MQ processing flow with ESQL SLEEP Reply with quote

Jedi

Joined: 25 Mar 2003
Posts: 2199
Location: Melbourne, Australia

hunghip95 wrote:
... Honestly, its my customer's requirement. We are develop team.
Exactly, "Business Process" sector is a REST calling out to Core Banking system. First of all, Core Banking is weak sometimes. Besides, this solution is just supposed for some low priority Business Services which allows handling later as an asynchronous process.
For example: ticket selling service, this service allows the Bank selling ticket immediately, though the real money is not computed at the same time.

Is their requirement that the Core Banking REST only be called every N seconds, at most? N = 10, or any other setting.

Get a message
BeginTime = Get current time
REST call
EndTime = Get current time
If EndTime - BeginTime < N Then Sleep ( BeginTime + N - EndTime )
_________________
Glenn
Back to top
View user's profile Send private message
bruce2359
PostPosted: Wed Nov 25, 2020 3:36 pm Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 8840
Location: US: west coast, almost. Otherwise, enroute.

Seems like over-engineering (look up Rube Goldberg)- delaying lower priority workload in anticipation of higher priority workload arriving.
_________________
Five out of four people have trouble with fractions. - Steven Wright.
Back to top
View user's profile Send private message
gbaddeley
PostPosted: Thu Nov 26, 2020 2:07 pm Post subject: Reply with quote

Jedi

Joined: 25 Mar 2003
Posts: 2199
Location: Melbourne, Australia

bruce2359 wrote:
Seems like over-engineering (look up Rube Goldberg)- delaying lower priority workload in anticipation of higher priority workload arriving.

Agree. If the incoming messages arrive at a faster average rate than what the REST service is allowed to process, the backlog will increase indefinitely. It's a losing game.

We have a lot of flows that call REST web services. Some can be slow to respond (1 - 2 minutes) due to intensive back end processing that they do. We don't sleep, we just let the request messages queue up in MQ, and they are processed when the response time allows.
_________________
Glenn
Back to top
View user's profile Send private message
hunghip95
PostPosted: Mon Nov 30, 2020 3:17 am Post subject: Reply with quote

Apprentice

Joined: 12 Oct 2017
Posts: 30

gbaddeley wrote:
bruce2359 wrote:
Seems like over-engineering (look up Rube Goldberg)- delaying lower priority workload in anticipation of higher priority workload arriving.

Agree. If the incoming messages arrive at a faster average rate than what the REST service is allowed to process, the backlog will increase indefinitely. It's a losing game.

We have a lot of flows that call REST web services. Some can be slow to respond (1 - 2 minutes) due to intensive back end processing that they do. We don't sleep, we just let the request messages queue up in MQ, and they are processed when the response time allows.


Thank you, it seems that we should use the maximum rate in workload management.
Hmm, REST calling out is a tricky issue.
Back to top
View user's profile Send private message
gbaddeley
PostPosted: Mon Nov 30, 2020 2:41 pm Post subject: Reply with quote

Jedi

Joined: 25 Mar 2003
Posts: 2199
Location: Melbourne, Australia

hunghip95 wrote:
Hmm, REST calling out is a tricky issue.

REST calls are inherently unreliable. They can fail with TCP socket errors due to network glitches or outages, or http server issues at the destination (500 code), or calls that take much longer than expected, or time out, or TLS certificates expire, or credentials change, or the service is moved to another host or ip address without informing you, or the URI format has changed. Things over which you have no direct control.

Your message flows should not break in these situations. Incoming transactions should queue up until the issue has been resolved, and have a periodic retry mechanism.
_________________
Glenn
Back to top
View user's profile Send private message
PeterPotkay
PostPosted: Tue Dec 01, 2020 3:55 pm Post subject: Reply with quote

Poobah

Joined: 15 May 2001
Posts: 7641

gbaddeley wrote:
hunghip95 wrote:
Hmm, REST calling out is a tricky issue.

REST calls are inherently unreliable. They can fail with TCP socket errors due to network glitches or outages, or http server issues at the destination (500 code), or calls that take much longer than expected, or time out, or TLS certificates expire, or credentials change, or the service is moved to another host or ip address without informing you, or the URI format has changed. Things over which you have no direct control.

Your message flows should not break in these situations. Incoming transactions should queue up until the issue has been resolved, and have a periodic retry mechanism.

I agree with you, I paraphrased this today more than once.
Is there anything more "official" regarding this line of thinking, a Technote or Conference presentation I can point people at? While I may trust gbaddeley, people that don't know him would just read that and go full Lebowski: "Yeah, well, you know, that's just, like, your opinion, man."
_________________
Peter Potkay
Keep Calm and MQ On
Back to top
View user's profile Send private message
hunghip95
PostPosted: Wed Dec 02, 2020 12:26 am Post subject: Reply with quote

Apprentice

Joined: 12 Oct 2017
Posts: 30

To be honest, this topic is going far away from its purpose
We need to keep this topic good enough for the similar problems.
Back to top
View user's profile Send private message
bruce2359
PostPosted: Wed Dec 02, 2020 7:47 am Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 8840
Location: US: west coast, almost. Otherwise, enroute.

hunghip95 wrote:
To be honest, this topic is going far away from its purpose
We need to keep this topic good enough for the similar problems.

The OP pondered the wisdom and feasibility of imbedding a delay in a business process (that involves MQ). Id say that the consensus here is that such a requirement is not a good practice.
_________________
Five out of four people have trouble with fractions. - Steven Wright.
Back to top
View user's profile Send private message
gbaddeley
PostPosted: Wed Dec 02, 2020 2:21 pm Post subject: Reply with quote

Jedi

Joined: 25 Mar 2003
Posts: 2199
Location: Melbourne, Australia

PeterPotkay wrote:
"Yeah, well, you know, that's just, like, your opinion, man."

There is plenty of opinion on this forum, that's for sure. Fortunately there is more consensus than bickering and disagreement. Practical MQ patterns are not always self evident. Poor design will soon lead to grief.
_________________
Glenn
Back to top
View user's profile Send private message
Display posts from previous:
Post new topicReply to topic Goto page Previous  1, 2 Page 2 of 2

MQSeries.net Forum IndexWebSphere Message Broker SupportDelay a MQ processing flow with ESQL SLEEP
Jump to:



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP


Theme by Dustin Baccetti
Powered by phpBB 2001, 2002 phpBB Group

Copyright MQSeries.net. All rights reserved.