| Author |
Message
|
| ucbus1 |
 Posted: Thu Aug 02, 2018 8:07 am Post subject: Wait time question and delay in getting messages |
 |
|
Knight
Joined: 30 Jan 2002
Posts: 560
|
We have mainframe and datapower communicating via MQ interface. Datapower puts a message which is received by mainframe via Get. Based on the log time of the message Put and Get we are seeing a delay ranging from 29-35 sec. The queue is indexed on correlation id and the mainframe application uses correlation id to GET the message. Following are the mainframe options used:
wait interval and correlation id. The mainframe application issues a GET on a particular correlation id and waits for 2-seconds and times out. From DP logs we know that the message was put with in the 2-second interval, yet we are finding timeouts on mainframe side for not getting the messages in time. Any input is appreciated. If this question was already addressed I apologize and appreciate if you can point me in right direction. |
|
| Back to top |
|
 |
| exerk |
| Posted: Thu Aug 02, 2018 8:13 am Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
The message may have been PUT, but was it COMMITTED ?
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| belchman |
| Posted: Thu Aug 02, 2018 10:31 am Post subject: |
|
|
Partisan
Joined: 31 Mar 2006
Posts: 386
Location: Ohio, USA
|
Things that could slow the process down
Is the mainframe process triggered?
Are the channels SSL enabled?
Is Datapower connecting via svrconn?
Does the queue have depth?
_________________
Make three correct guesses consecutively and you will establish a reputation as an expert. ~ Laurence J. Peter |
|
| Back to top |
|
|
| gbaddeley |
| Posted: Thu Aug 02, 2018 7:09 pm Post subject: |
|
|
Jedi Knight
Joined: 25 Mar 2003
Posts: 2538
Location: Melbourne, Australia
|
2 secs is quite a short WaitInterval, for any MQ interface. Try increasing it to 60 secs to make it easier to diagnose. How often does the issue happen? Do you observe non-zero UNCOM counts on DIS QSTATUS(xx) TYPE(QUEUE) ALL? Do the messages have ExpiryInterval set? Are they persistent or non-persistent?
_________________
Glenn |
|
| Back to top |
|
|
| ucbus1 |
| Posted: Thu Aug 02, 2018 7:26 pm Post subject: |
|
|
Knight
Joined: 30 Jan 2002
Posts: 560
|
few more updates on the application.
1.mainframe puts a request on a queue.
2. Datapower's front side handler takes the message and hands it over to a back end process via Multi protocol gateway (MPGW)
3. The response from back end process is received by the Datapower and the response is put on the mainframe response queue .The response queue is indexed on correlation id. Mainframe picks up the message using correlation id
4. The front side handlers of the datapower use "units of work"="1" on MQ queue manager object. I believe this setting should make messages put by datapower on COMMIT.
While getting the message from mainframe, datapower does not encounter any timeout issues. However when the response is sent by datapower, mainframe is timing out on Getting the message.
@belchman, I will try to get more info and provide the input |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Fri Aug 03, 2018 4:54 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| ucbus1 wrote: |
While getting the message from mainframe, datapower does not encounter any timeout issues. However when the response is sent by datapower, mainframe is timing out on Getting the message.
@belchman, I will try to get more info and provide the input |
Please explain what you mean by mainframe is timing out on Getting the message.
Timing out?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| ucbus1 |
| Posted: Fri Aug 03, 2018 5:12 am Post subject: |
|
|
Knight
Joined: 30 Jan 2002
Posts: 560
|
[quote="bruce2359"]
| ucbus1 wrote: |
Please explain what you mean by mainframe is timing out on Getting the message.
Timing out? |
Sure let me try. Mainframe issues a GET with 2 sec wait (WAITINTERVAL option) for a message with specific correlation id. If the message could not be located then it gets a timeout. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Fri Aug 03, 2018 5:51 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
[quote="ucbus1"]
| bruce2359 wrote: |
| ucbus1 wrote: |
Please explain what you mean by mainframe is timing out on Getting the message.
Timing out? |
Sure let me try. Mainframe issues a GET with 2 sec wait (WAITINTERVAL option) for a message with specific correlation id. If the message could not be located then it gets a timeout. |
So, what does the app do next? Give up, end itself? Or, does it do another MQGET with 2-second WAITINTERVAL?
When it times out are there any messages (other) in the queue?
Does the MQGET call return a 2033 ReasonCode? Something else?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Aug 03, 2018 6:43 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| ucbus1 wrote: |
| Sure let me try. Mainframe issues a GET with 2 sec wait (WAITINTERVAL option) for a message with specific correlation id. If the message could not be located then it gets a timeout. |
As others have said, that's a very short interval to wait. Why so short and why not increase it? What does that do to the application / business process?
To me (again, as others have said) this sounds a lot like DataPower not committing the message. Perhaps (and I therorize here) the message is put but the unit of work isn't closed until it reaches a 30 second idle threshold
at which point (29 - 35 seconds after the put) the mainframe finds it.
(I'm also theorizing that however you got that timing, there's a +/- 1 second variance in the system clocks)
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| ucbus1 |
| Posted: Fri Aug 03, 2018 7:04 am Post subject: |
|
|
Knight
Joined: 30 Jan 2002
Posts: 560
|
@bruce:
No. the mainframe application does not issue a MQGET again. The messages left unpicked are processed through a batch job. Ideally we want the retry processing to be as small as possible.
@Victor: Yes we increased the wait interval to 6 sec and we find the same behavior.
Back to the comment, datapower not committing the message. Its my understanding setting unit of work=1 ensures auto committing of the message in case of "no errors". In case of errors, we have set up a backout queue and the errored messages are moved to the backout queue.
Let me know if this is not the expected behavior from setting units of work=1 on datapower. |
|
| Back to top |
|
|
| exerk |
| Posted: Fri Aug 03, 2018 7:09 am Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
| ucbus1 wrote: |
...Its my understanding setting unit of work=1 ensures auto committing of the message in case of "no errors"...
...Let me know if this is not the expected behavior from setting units of work=1 on datapower. |
I don't know Datapower so can't comment on the veracity of that, but it still doesn't answer the question in regard to the length of time between the PUT and COMMIT.
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Aug 03, 2018 7:13 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| ucbus1 wrote: |
| Its my understanding setting unit of work=1 ensures auto committing of the message in case of "no errors". |
Yes, automatically committed when? At the end of the unit of work? What ends the unit of work? An explicit command or no activity within the UoW until the idle timeout?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| ucbus1 |
| Posted: Fri Aug 03, 2018 7:33 am Post subject: |
|
|
Knight
Joined: 30 Jan 2002
Posts: 560
|
| Vitor wrote: |
Yes, automatically committed when? At the end of the unit of work? What ends the unit of work? An explicit command or no activity within the UoW until the idle timeout? |
Thanks Victor. What will it be in the case of datapower? Not hands-on with datapower. In the case of datapower it is not explicitly coded but configure hence asking. |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Aug 03, 2018 7:55 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| ucbus1 wrote: |
| Vitor wrote: |
Yes, automatically committed when? At the end of the unit of work? What ends the unit of work? An explicit command or no activity within the UoW until the idle timeout? |
Thanks Victor. What will it be in the case of datapower? Not hands-on with datapower. In the case of datapower it is not explicitly coded but configure hence asking. |
I'm only slightly better with datapower than you, but I do know that it's configurable within the map (Datapower isn't generating this message, it's the gateway to whatever your mainframe is talking to).
So, given that you're not hands on with datapower, how did you set "units of work = 1" on the front handler? If you didn't, who did and how did they arrive at that value.
Bottom line: the next stage of your problem investigation is to determine at what point datapower is actually committing the message (which, as with all software, may vary from when you think or have configured it to do so). For this you will need the team that's hands on with DataPower. I would start with the activity logs and verify that the put and the commit are shown together, because if they are, and that happens 30ish seconds before the MF gets the message, this commit thing is a red herring.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| belchman |
| Posted: Fri Aug 03, 2018 10:55 am Post subject: |
|
|
Partisan
Joined: 31 Mar 2006
Posts: 386
Location: Ohio, USA
|
|
| Back to top |
|
|
|
|
