ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » General IBM MQ Support » Messages going to DLQ

Post new topic  Reply to topic
 Messages going to DLQ « View previous topic :: View next topic » 
Author Message
vmanel
PostPosted: Mon Aug 19, 2019 8:51 am    Post subject: Messages going to DLQ Reply with quote

Acolyte

Joined: 12 Mar 2019
Posts: 57

Hi,

We are facing an issue in production. We have a standalone QM which has the all the input and output queues of the IIB flows and an IIB QM which is used just for publish/subscribe, the subscriptions in this QM are pointed to Standalone QM through remote queues. IIB uses MQ policy to connect to Standalone QM

First application publishes the messages and second application subscribes and sends the message to the output queue. The messages in output queue is consumed by IBM sterling agents(which uses JMS).

To handle failure we have MQ output node connected to the catch of the mq input node, which acts as the backout queue for the flow

Some messages failed as there was some intermittent issue with intermediate web service that IIB publication flow calls. But once the issue was resolved, we reprocessed the failed messages by saving the messages from backout queue along with headers using mq visual edit, and then load them to the input queue of the publication flow. But some messages went to DLQ of standalone QM mutiple times with error code 2362, as we dont have a backout requeue defined for the output queue(input queue for sterling). When we browsed the messages of DLQ in MQ explorer, it showed put application as our IIB QM. But in DLQ header it shows
Put application as MQ JMS connection consumer. And the destination queue is the output queue of our flow(input queue for sterling)

So we are not sure if the issue is at our end, because we are not seeing any errors at our end. Or is the issue at sterling end. And after multiple tries processing was successful. And since this is occurring in production, we are not able to reproduce the issue.

Thanks
Back to top
View user's profile Send private message
exerk
PostPosted: Mon Aug 19, 2019 10:53 am    Post subject: Reply with quote

Jedi Council

Joined: 02 Nov 2006
Posts: 6339

Quote:
"...it showed put application as our IIB QM. But in DLQ header it shows
Put application as MQ JMS connection consumer..."

When looking in the DLQ Header you are seeing that the originating queue manager is your IIB node? Because I'd expect the Standalone queue manager would put those messages to its DLQ with a different reason, e.g. MQRC 2085.

From what you've written I interpret it to be that Sterling is having trouble processing the messages, and in good old (bad in other words) JMS fashion when they can't be processed is dumping them on the DLQ.

I'm not sure what you mean by "...To handle failure we have MQ output node connected to the catch of the mq input node, which acts as the backout queue for the flow...", so would you please clarify that, thank you.
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys.
Back to top
View user's profile Send private message
vmanel
PostPosted: Mon Aug 19, 2019 10:54 pm    Post subject: Reply with quote

Acolyte

Joined: 12 Mar 2019
Posts: 57

exerk wrote:
When looking in the DLQ Header you are seeing that the originating queue manager is your IIB node? Because I'd expect the Standalone queue manager would put those messages to its DLQ with a different reason, e.g. MQRC 2085.


Yeah actually its showing IIB QM, for all messages when we do browse messages from MQ explorer. But when we check individual message DLH, it shows put application name as MQ JMS connection consumer

exerk wrote:
From what you've written I interpret it to be that Sterling is having trouble processing the messages, and in good old (bad in other words) JMS fashion when they can't be processed is dumping them on the DLQ.


But why nothing is shown in the AMQ error logs at our MQ end? Messages are sent to DLQ, so something should be logged at our side as well right? Or is there any other logs where it is getting logged?

exerk wrote:
I'm not sure what you mean by "...To handle failure we have MQ output node connected to the catch of the mq input node, which acts as the backout queue for the flow...", so would you please clarify that, thank you


We have flows starting with MQ input node as input, whose catch terminal is connected to a error handling library which will send an error mail when a failure occurs and then a mq output node is connected which will store the failed message
Back to top
View user's profile Send private message
exerk
PostPosted: Mon Aug 19, 2019 11:34 pm    Post subject: Reply with quote

Jedi Council

Joined: 02 Nov 2006
Posts: 6339

vmanel wrote:
But why nothing is shown in the AMQ error logs at our MQ end? Messages are sent to DLQ, so something should be logged at our side as well right?

An application is getting a message from one queue, is unable to process that message so puts it to another queue. As far as the queue manager is concerned an application is just doing GET and PUT operations, so why would it log anything?

vmanel wrote:
Or is there any other logs where it is getting logged?

I'd expect to see any exceptions logged by Sterling as that's the application doing the "back-out to exception queue".

vmanel wrote:
We have flows starting with MQ input node as input, whose catch terminal is connected to a error handling library which will send an error mail when a failure occurs and then a mq output node is connected which will store the failed message

I'm still a bit hazy on this - is this in the IIB queue manager?
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys.
Back to top
View user's profile Send private message
vmanel
PostPosted: Tue Aug 20, 2019 7:23 am    Post subject: Reply with quote

Acolyte

Joined: 12 Mar 2019
Posts: 57

exerk wrote:
I'm still a bit hazy on this - is this in the IIB queue manager?


No, the queues are present in Standalone QM, the flows use MQ policy to connect to QM. IIB QM is used only for publication node to publish and subscriptions with remote queues to Standalone QM are created in that.
Back to top
View user's profile Send private message
HubertKleinmanns
PostPosted: Wed Aug 21, 2019 12:37 am    Post subject: Reply with quote

Shaman

Joined: 24 Feb 2004
Posts: 732
Location: Germany

vmanel,

first you should create a backout queue and specify this one in the original queue definition. This would separate application errors from internal MQ errors.

You wrote, the initial reason code was 2362 and this means - as exerk already explained - this indicates that the application as trouble with the content of the data.

You also wrote, that on IIB QM publications are made to remote queues.

- How are these remote queues defined?

- Which MQ version do you have?

I guess, IIB and /or Sterling use Client channels.

- How are these Client channels defined?

You wrote, "some" Messages fail.

- Are theses message sent across different remote queues / channel?

- How big are the Messages?
_________________
Regards
Hubert
Back to top
View user's profile Send private message Visit poster's website
vmanel
PostPosted: Wed Aug 21, 2019 1:55 am    Post subject: Reply with quote

Acolyte

Joined: 12 Mar 2019
Posts: 57

HubertKleinmanns wrote:

- How are these remote queues defined?

- Which MQ version do you have?

I guess, IIB and /or Sterling use Client channels.

- How are these Client channels defined?

You wrote, "some" Messages fail.

- Are theses message sent across different remote queues / channel?

- How big are the Messages?


IIB QM will have subcriptions which will have destination as an alias queue. Alias queue will in turn have base as remote queue pointing to Standalone QM. Sender-Receiver channel is used here to send messages from IIB QM to Standalone QM.

MQ version is 9.1.0.2

Sterling is using Server Connection Channel to read from the queues present in Standalone QM.

Not sure what is meant by how are these channels defined

Messages which we tried to reprocess failed and after trying multiple times, same messages processed successfully

Yes, the messages are first put in the publication flow input queue present in Standalone QM by an application, which publishes the transformed message. Then the subscription in IIB QM subscribes to the message and sends it to Subscription flow input queue present in Standalone QM through remote queue. Then subscription processes the message and puts in the output queue to which Sterling is listening

Length of messages are around 3KB
Back to top
View user's profile Send private message
HubertKleinmanns
PostPosted: Wed Aug 21, 2019 2:50 am    Post subject: Reply with quote

Shaman

Joined: 24 Feb 2004
Posts: 732
Location: Germany

Hi vmanel,

vmanel wrote:
Sterling is using Server Connection Channel to read from the queues present in Standalone QM.

Not sure what is meant by how are these channels defined


Could you post the channel definition?

vmanel wrote:
Messages which we tried to reprocess failed and after trying multiple times, same messages processed successfully


Hmm, strange - did something - outside MQ - change in the meantime? Does "Sterling" mean "Sterling Connect:Direct"?

vmanel wrote:
Yes, the messages are first put in the publication flow input queue present in Standalone QM by an application, which publishes the transformed message. Then the subscription in IIB QM subscribes to the message and sends it to Subscription flow input queue present in Standalone QM through remote queue. Then subscription processes the message and puts in the output queue to which Sterling is listening


Could you post the definitions of SUBs, queues and topics?
_________________
Regards
Hubert
Back to top
View user's profile Send private message Visit poster's website
exerk
PostPosted: Thu Aug 22, 2019 7:39 am    Post subject: Reply with quote

Jedi Council

Joined: 02 Nov 2006
Posts: 6339

Head-check me on this please, to ensure I have it correct:

App (PUT) -> S-AQM Input QLOCAL <- (GET) Transform & Publish App (PUT) -> ? S-AQM QREMOTE ? <SDR/RCVR> IIB QM QLOCAL <- (GET) IIB Flow (PUT) -> IIB QM QREMOTE <SDR/RCVR> S-AQM QLOCAL <- (GET) Subscribe App (PUT) -> S-AQM QLOCAL <- (GET) Sterling

Because that's what picking apart "...Yes, the messages are first put in the publication flow input queue present in Standalone QM by an application, which publishes the transformed message. Then the subscription in IIB QM subscribes to the message and sends it to Subscription flow input queue present in Standalone QM through remote queue. Then subscription processes the message and puts in the output queue to which Sterling is listening..." gives me.

Unless you have a flow connecting from IIB to the Stand-Alone queue manager? So:

App (PUT) -> S-AQM Input QLOCAL <- (GET) Transform & Publish App (PUT) -> S-AQM QLOCAL <- (GET) IIB Flow connected to S-AQM | hand off to | IIB Flow connected to IIB QM (PUT) -> QREMOTE <SDR/RCVR> S-AQM QLOCAL <- (GET) Subscribe App (PUT) -> S-AQM QLOCAL <- (GET) Sterling

I agree with HubertKleinmanns, if you are successfully processing the messages after multiple attempts, the issue lies not with MQ or IIB but with the processing application, or sub-system on which that application is reliant, e.g. a database.

And please try and be more clear and precise in describing the messaging sequence because it really does help a bear of small brain like me!
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Page 1 of 1

MQSeries.net Forum Index » General IBM MQ Support » Messages going to DLQ
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.