| Author |
Message
|
| jrsetters |
 Posted: Wed Oct 10, 2012 5:50 am Post subject: Returning a message to its source queue |
 |
|
Acolyte
Joined: 24 Aug 2011
Posts: 72
Location: Cincinnati, OH
|
The giant subflow I have been working on for several months has now been deployed to 5 of our production systems and is doing very well. They have asked me to add one more piece of functionality to it, in the case of a specific type of message event, they want me to return the message to its source queue.
The event is basically when the receiving system does not Acknowledge our message within 30 seconds, currently we send it along with any NAKs to an error queue where they can be individually handled. There is a specific variable in the message tree that will read 'TIMEOUT' when this occurs.
Our input nodes are all set to Transaction Mode, although frankly we have done nothing with it at all. I am a little concerned about using it because everything else works exactly as we want it to now (99.9% of messages at least) .
I was reading this article on message flow transactions http://publib.boulder.ibm.com/infocenter/wmbhelp/v7r0m0/index.jsp and although the concept is pretty easy I have no experience implementing it.
Is there any other way to do it? I know explicitly what the source queue was from the environment tree. I also store an exact copy of the original message in the environment at the start of my subflow.
Basically what they want is to put the message in the back of the line in the source queue rather than treating it as an error to work because most external systems will resolve ACK failures on their own in short order and a re-submission will be sufficient. |
|
| Back to top |
|
 |
| lancelotlinc |
| Posted: Wed Oct 10, 2012 5:53 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
1. Break up the giant monolithic monster.
2. Do not use Transaction mode yes. Use Transaction mode no.
3. Store the original message in Environment, a queue, or a database.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| zpat |
| Posted: Wed Oct 10, 2012 5:56 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
If you allow transactional backout - the message will not be at the "back" of the queue, it is simply made available again for getting.
You can put a message to the same queue as you received it from and commit both the get and put at the same time when the flow completes.
In all cases where you return messages to the same queue, you need to guard against looping around repeatedly doing this (unless you want to).
On the transactional backout - the MQMD.Back count is incremented which allows detection of how often the message has been processed. You might want to implement something similar (perhaps in a spare MQMD.field) to keep a note of how often the flow has seen the same message.
PS - I have to disagree with the above comment about transactional mode - it is a critical part of ACID to use transaction mode. Otherwise you can get failures which lose messages. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Oct 10, 2012 6:06 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
You should simply propagate an exception out of your subflow.
The main flow that includes your subflow should then allow that exception (or a different one) to get passed back to the MQInput node that started the main flow.
The MQinput node will then call the catch terminal. which should again make sure that an exception is thrown back to the MQInput node (rather than completing successfully, i.e. end the catch terminal with a throw node).
This will cause the MQInput node to roll back all transactions, including the one that read the message off the queue.
The message will then be available for getting again and be immediately read into a new instance of the main flow. |
|
| Back to top |
|
|
| jrsetters |
| Posted: Wed Oct 10, 2012 6:35 am Post subject: |
|
|
Acolyte
Joined: 24 Aug 2011
Posts: 72
Location: Cincinnati, OH
|
I appreciate Lancelothic's advice, esp regarding my sublfow which started off so simple and elegant and now looks like a plate of spaghetti.
In to more specifics, I am storing my whole message in a usable state right at the beginning of the subflow. When the HL7Output node generates certain conditions I route this message on an outbound terminal that ties to an error queue. It looks like this:
| Code: |
CREATE FILTER MODULE AddTimeStamp_CatchAENaks
CREATE FUNCTION Main() RETURNS BOOLEAN
BEGIN
DECLARE hl7 NAMESPACE 'urn:hl7-org:v2xml';
DECLARE Processchk BOOLEAN UNKNOWN;
DECLARE flag, flag1 CHAR;
SET flag = COALESCE(LocalEnvironment.HL7."FlowMilestoneReached", '');
SET flag1 = COALESCE(ExceptionList.RecoverableException.RecoverableException."Text", '');
-- In certain events we want to interrupt the retry count
IF flag IN ('ACKPARSEERROR', 'ACKERROR', 'ACKAE', 'TIMEOUT') THEN
SET Processchk = TRUE;
END IF;
IF flag1 = 'hostname unresolved' THEN
SET Processchk = FALSE;
SET Environment.Variables.StopFlow = 'YES';
END IF;
RETURN Processchk;
END;
END MODULE; |
The next step in the flow when 'true' will then write the original message out to a subflow terminal called ERR that is tied to the message flow error queue.
| Code: |
CREATE COMPUTE MODULE AddTimeStamp_ExtractHL7
CREATE FUNCTION Main() RETURNS BOOLEAN
BEGIN
SET OutputRoot.Properties = Environment.Input.Properties;
SET OutputRoot.MQMD = Environment.Input.MQMD;
SET OutputRoot.MRM = Environment.Input.MRM;
PROPAGATE;
RETURN FALSE;
END;
END MODULE; |
So now what I want to do is take the 'TIMEOUT' condition and instead of routing it to the error queue, just throw it back into the source queue. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Oct 10, 2012 6:38 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| jrsetters wrote: |
| Code: |
IF flag IN ('TIMEOUT') THEN
THROW USER EXCEPTION;
END IF;
|
|
|
|
| Back to top |
|
|
| jrsetters |
| Posted: Wed Oct 10, 2012 6:41 am Post subject: |
|
|
Acolyte
Joined: 24 Aug 2011
Posts: 72
Location: Cincinnati, OH
|
zpat,
We actually do want it to loop, although we'd prefer it not hold up additional messages while doing so, which is why I was hoping for a way to put it to the back of the queue. I really can't think of a scenario where a vendor will fail to ACK a message in 30 seconds unless the are having an itinerant issue of some sort, but anything is possible. HL7 standard says that if they actually have a problem with the message they should be NAKing it not just failing to ACK it. But healthcare messaging is still a cottage industry in many ways. |
|
| Back to top |
|
|
| jrsetters |
| Posted: Wed Oct 10, 2012 6:44 am Post subject: |
|
|
Acolyte
Joined: 24 Aug 2011
Posts: 72
Location: Cincinnati, OH
|
| mqjeff wrote: |
| jrsetters wrote: |
| Code: |
IF flag IN ('TIMEOUT') THEN
THROW USER EXCEPTION;
END IF;
|
|
|
Sorry, I was actually looking at your response and researching the MQInput node. I am assuming I work this somehow through the monitoring or validation settings in the MQInput node. |
|
| Back to top |
|
|
| McueMart |
| Posted: Wed Oct 10, 2012 6:45 am Post subject: |
|
|
Chevalier
Joined: 29 Nov 2011
Posts: 490
Location: UK...somewhere
|
| If you want to put it to the back of the input queue, why not just use the MQMD.SourceQueue field to dynamically put the message back to where it came from (using an MQOutput node which uses a DestinationList) ? |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Oct 10, 2012 6:50 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| McueMart wrote: |
| If you want to put it to the back of the input queue, why not just use the MQMD.SourceQueue field to dynamically put the message back to where it came from (using an MQOutput node which uses a DestinationList) ? |
That puts it to the end of the queue, probably with a new msgid and correlid, and potentially with other new headers.
Using built-in rollback is the better choice. |
|
| Back to top |
|
|
| zpat |
| Posted: Wed Oct 10, 2012 7:04 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
I don't see a problem with putting it to the source queue, and committing both the get and put in the same transaction. That's safe.
It would go to the back of the queue that way.
However no-one wants indefinite re-processing, so think about what happens when it simply does not ever succeed - at some point it needs to go to an exception queue.
Guard against a tight CPU failure loop - maybe code a small ESQL sleep before putting back to the source queue. |
|
| Back to top |
|
|
| NealM |
| Posted: Wed Oct 10, 2012 3:13 pm Post subject: |
|
|
Master
Joined: 22 Feb 2011
Posts: 230
Location: NC or Utah (depends)
|
I was going to chime in with suggestions on using the input Q's backout count, etc., but I feel I don't really know enough. For instance, why wouldn't you get an acknowledgement from the receiving system? Is it down sometimes? Is a particular message you send causing an exception at that end sometimes? How long or how many times should you retry? Do you have to process messages in a particular order? Does/can the main flow have multiple instances? Do any Nodes in the path between the main flow's MQInput and your subflow's no-ack decision point have a Failure terminal wired to something? Do you have any control over that?
The only thing that seems clear is that, whatever you do, you don't have to worry about a tight loop as you evidently have a 30 second timeout on an MQGet in your subflow. |
|
| Back to top |
|
|
|
|
