| Author |
Message
|
| sunny_30 |
 Posted: Mon Nov 19, 2007 8:41 pm Post subject: Timer nodes problem |
 |
|
Master
Joined: 03 Oct 2005
Posts: 258
|
Hi,
This is the Msg-flow design:
mqinput->compute->mqoutput->compute->mqget->timeoutcontrolnode
timeoutnotificationnode->mqoutput(same mqoutput from above)
I process request messages out to the mqoutput node & collect the responses using the mqget node.
For few scenarios, looking at the response collected from the mqget I will have to retry the same message again.
The transaction in mqoutput is set to 'NO', orelse the mqoutput isnt releasing the request message & the mqget always times out.
Im able to reprocess the same message again passing it out of the timeoutcontrol node. The timeoutnotification node retries the same message after 30 seconds directly connected to the mqoutput node, sends the same earlier message again.
The problem Im having now is that in the span of this 30 seconds(timer node interval) the mqinput is picking up several other messages from the mqinput.
I want the flow to continue retrying only one message. I dont want any other messages to sneak in while Im already in the middle of processing the earlier message using timer nodes.
How can I achieve what I need to do?
Is there a solution for my problem?
please help.
Thanks,
Sunny. |
|
| Back to top |
|
 |
| AkankshA |
| Posted: Mon Nov 19, 2007 8:57 pm Post subject: |
|
|
Grand Master
Joined: 12 Jan 2006
Posts: 1494
Location: Singapore
|
time out nodes are behaving the expected way  they won't block the thread....
from your description, as i understand you need to keep trying few messages for processing....
why is this 30 sec delay in between????
may be u need to give your requirements and design a second look....
_________________
Cheers |
|
| Back to top |
|
|
| sunny_30 |
| Posted: Mon Nov 19, 2007 9:12 pm Post subject: |
|
|
Master
Joined: 03 Oct 2005
Posts: 258
|
Thanks for your reply.
The 30 second interval is for testing purposes only. I will reduce the timer down to the least(1 sec) later. Im inducing an interval of a second so as to retry the same message to the SAP adapter(max 5 times) in case of the communication error from SAP. This is almost rare, but I have seen situations of this happening.
Do you mean to say that the timer nodes will not block the processing-thread during the WaitInterval & the mqinput node can pick up the other messages from the InputQ? If so, there is certainly a possibility that 100s of other messages might get processed in a second gap, before I process the failed one. My requirement is also to maintain the FIFO sequence to the adapter.
Im thinking if I cant be able to achieve the above functionality, then I would stop the flow (using PCF messages to get Inhibit the queue) during the reprocessing interval. Please suggest your thoughts.
Thanks,
Sunny. |
|
| Back to top |
|
|
| AkankshA |
| Posted: Mon Nov 19, 2007 9:19 pm Post subject: |
|
|
Grand Master
Joined: 12 Jan 2006
Posts: 1494
Location: Singapore
|
| sunny_30 wrote: |
Do you mean to say that the timer nodes will not block the processing-thread during the WaitInterval & the mqinput node can pick up the other messages from the InputQ?
|
Yes. absolutely
| sunny_30 wrote: |
If so, there is certainly a possibility that 100s of other messages might get processed in a second gap, before I process the failed one.
|
yes..
| sunny_30 wrote: |
My requirement is also to maintain the FIFO sequence to the adapter.
Im thinking if I cant be able to achieve the above functionality, then I would stop the flow (using PCF messages to get Inhibit the queue) during the reprocessing interval. Please suggest your thoughts.
|
it all depends on how often you get such messages?? what does the SLA say about the delivery time etc etc etc...
keep all the factors in mind and design
all the best 
_________________
Cheers |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Tue Nov 20, 2007 7:46 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
I thouth I read somewhere that you could not do an MQget from the same q as your MQInput node... Anyways I would expect you to do a get with msgid or some such if you want to reprocess the same msg. 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Nov 20, 2007 7:53 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| fjb_saper wrote: |
| I thouth I read somewhere that you could not do an MQget from the same q as your MQInput node... |
Quite true under v6. Relaxed under v6.1 AFAIK (but prepared to be corrected)
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| sunny_30 |
| Posted: Tue Nov 20, 2007 8:46 am Post subject: |
|
|
Master
Joined: 03 Oct 2005
Posts: 258
|
Yes Saper, Im picking up the response using mqGet matching the correlId. I pass the MsgId as the correlId of the request message.
The flow picks up the request messages using mqInput node. The adapter replies to the ReplyToQ field of the MQMD request sent to it. The ReplyToQ where the flow picks up the response messages using mqGet is different from the mqInput node Q.
If I set the transaction property in the mqOutput node to 'YES', the mqGet node always times-out. As I connected the mqOutput node & the mqGet of the flow, the request message sent out of the mqOutput is NOT released by the flow & the adapter wont pick up until the whole flow is complete i.e mqGet time-out. When I set the Transaction to 'NO' it works fine. I get the response while Im still processing the flow.
The problem Im having is with the Timer nodes. I think I got the needed info from Akanksha. Thanks. I will look into changing the flow design a bit. I certainly cant let the flow to process the following messages when Im 'sleeping/pausing' the flow processing an earlier message using Timer nodes. I cant be losing the sequence of the messages I feed to the mySap adapter. That is the requirement I have. When I dont receive any response from the adapter, Im stopping the flow. I probably will do the same when I get a communication-error too.
Thanks,
Sunny |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Tue Nov 20, 2007 10:11 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
That sounds to me like you are trying to transform what is essentially an asynchronous flow into a synchronous flow.
The assumption here is that it is one system talking to another and not some people interface...
Here is my proposition:
Change your request / reply with get node to a request / reply with aggregation. Your messages to the SAP system are persistent. If the connector goes down they'll wait in aggregation, and on the connector input queue.
You need to monitor the connector input queue and allow it sufficient depth for maintenance intervals.
When you get the reply the aggregation will take care of any matching that needed to be made and send the information back to the requester system.
The requester system can now match up (DB wise) or just process any answers.
That gives you a totally asynchronous model...
If you have truely synchronous needs (messages that inform but do not change the data) you should possibly use messages with expiry and a different flow...
Enjoy
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| sunny_30 |
| Posted: Wed Nov 21, 2007 6:27 am Post subject: |
|
|
Master
Joined: 03 Oct 2005
Posts: 258
|
Yes, Saper. The model Im trying to build with Broker & Adapter is totally synchronous.
Broker needs to have total control on the adapter for each processed message.
This is the requirement:
Broker gets success/failure status as Reply from Adapter for each IDOC processed in SAP via ALE.
Success has the value 0 in Reply.RFH2.usr.Status
Failure has the value -1 in Reply.RF2.usr.Status
SAP is set to process all messages immediately.
The sequence should always remain the same for same shipment.
If the broker doesnt recieve the 0 status as reply, and if the reply is -1 & communication-error, retry 5 times with a second gap. If not successful still, stop the flow.
Any other -1 errors will load the DB with that shipment#, save original in failQ.
All the following same shipment IDOCS that were errored out earlier will also land in errorQ using DB.
If no response, stop the flow. Continue once the adapter is up & processing.
When no response is received from adapter, our design wont let the messages pile up on the adapter queue. In such case, broker will not be able to process replies for those messages & there are chances of losing the same batch IDOC order when something fails in the adapter to post in SAP.
I found it easy using MQGET, so I can save & reprocess the same earlier 'requested' message.
But what is the real difference in my scenario if I use MQGET or Aggregate?
Both match CorrelId to pick up the response message. And, I have only 1 response for 1 request, not multiple responses ideal for Aggregate scenario.
Thanks,
Sunny. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Nov 21, 2007 11:27 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
The problem I see is that you have a technical requirement that makes very little sense.
What you need is the business scenario.
What you've been told is:
you are processing a shipment and as a consequence sending an IDOC to SAP. You want to immediately know whether the IDOC processed successfully or not.
This makes little business sense as the driver remains the SAP system.
Even if the IDOC is processed immediately in SAP some internal SAP parameters may force it to go into an error status. SAP has workflow to move those IDOC's into somebody's inbox and reprocess manually or even change (some data values) before reprocessing...
You should only be responsible for sending the IDOC to SAP. This is in fact the adapter's responsibility. So monitoring the queue feeding the Adapter is the only sensible thing here. If your adapter is well written any communications problem with SAP will roll back the message to the queue.
Anything else you should not be responsible for.
Error resolution needs to happen in the SAP system. They will eventually tell you if you need to change your map or anything else pertaining to the message.
If they have throughput problems (repeat 5 times) they should look into the RFC parms of the app server / appserver group, you really can't do anything about it...
Enjoy 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| sunny_30 |
| Posted: Wed Nov 28, 2007 6:31 am Post subject: |
|
|
Master
Joined: 03 Oct 2005
Posts: 258
|
Thanks Saper for you expert suggestions.
I agree that we have a requirement thats kind of wierd. I dont have much control over the team that works with the mysap.com adapter. They laid out few requirements which I have to follow..
Looks like the problem they are having on communication error with SAP is that the original perisistent message is NOT rolled back & there is a status(-1) message to the replyToQ. The Idoc is failed to get submitted to the SAP.
So the requirement I have on the Broker level is to read this response & resend if necessary. As currently Im having problem with the Timer nodes to resend, I have decided to divert all the same shipment Idocs to the FailQ to manually reprocess later. The retrying logic is removed.
On No-response from adapter, I stop the flow.
On negative(-1) response from adapter, I save this message to FailQ & I use the DB to divert the same shipment Idocs that follow to the failQ.
If the adapter rolls back the original message, will it be able to retry it 'n' number of times. If it still fails, will it divert the failed message to the FAULT queue & reprocess the later ones?
If yes, I wouldnt want this behavior as I have to strictly preserve the sequence for same shipment Idocs!
I wanted to keep the whole flow logic asynchronous, but for the requirements I have I dont see any other way out.
The response I receive from the adapter only says status 0 for success & -1 for failure. This (I guess) just says the status of the Idoc if 'submitted' to SAP or not, but not abt if it failed when SAP runs their job.
If the SAP side fails to 'process' the Idoc, we have a design that we can use to reprocess the Idocs from archived files.
Thanks,
Sunny. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Nov 28, 2007 6:39 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| sunny_30 wrote: |
Thanks Saper for you expert suggestions.
I agree that we have a requirement thats kind of wierd. I dont have much control over the team that works with the mysap.com adapter. They laid out few requirements which I have to follow..
Looks like the problem they are having on communication error with SAP is that the original perisistent message is NOT rolled back & there is a status(-1) message to the replyToQ. The Idoc is failed to get submitted to the SAP.
So the requirement I have on the Broker level is to read this response & resend if necessary. As currently Im having problem with the Timer nodes to resend, I have decided to divert all the same shipment Idocs to the FailQ to manually reprocess later. The retrying logic is removed.
On No-response from adapter, I stop the flow.
On negative(-1) response from adapter, I save this message to FailQ & I use the DB to divert the same shipment Idocs that follow to the failQ.
If the adapter rolls back the original message, will it be able to retry it 'n' number of times. If it still fails, will it divert the failed message to the FAULT queue & reprocess the later ones?
If yes, I wouldnt want this behavior as I have to strictly preserve the sequence for same shipment Idocs!
I wanted to keep the whole flow logic asynchronous, but for the requirements I have I dont see any other way out.
The response I receive from the adapter only says status 0 for success & -1 for failure. This (I guess) just says the status of the Idoc if 'submitted' to SAP or not, but not abt if it failed when SAP runs their job.
If the SAP side fails to 'process' the Idoc, we have a design that we can use to reprocess the Idocs from archived files.
Thanks,
Sunny. |
Their problem not yours.
The SAP adapter has a transactional mode. MQ has a transactional mode.
If the transaction to SAP fails it IS THE ADAPTER's responsibility to roll back the MQ message... or take appropriate action
You should not have to code for this.
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| francoisvdm |
| Posted: Wed Nov 28, 2007 6:48 am Post subject: |
|
|
Partisan
Joined: 09 Aug 2001
Posts: 332
|
I agree, adapter's problem. The SAP adapter do have some replay capabilties. But, if they depend on those "failed" msgs to come in before you send new ones..... eish, you need to do something with the msgs in the failed queue.
Also, they must just set the adapter to roll back the message.
_________________
If you do not know the answer or you get the urge to answer with "RTFM" or "Search better in this forum", please refrain from doing so, just move on to the next question. Much appreciated.
Francois van der Merwe |
|
| Back to top |
|
|
|
|
