MQSeries.net :: View topic - MQ persistent messages go missing in WMQ v7 and WMB v7

Diwakar · Posted: Fri Feb 25, 2011 7:25 pm Post subject:

Hello,

We have MQ v7.0.1, and MB v7.0.0.1 on Win 2008 servers.
We just upgraded from MQv6.0.x, and MB v6.0.x, and JText Adapters.

We have a peculiar problem (we had this in version 6 too, and we thought upgrading to v7 would fix our problem but it didn't).

We have a series of msgflows like:

flow-A --> flow-B --> flow-C --> flow-D

All flows have just 1 instance running (additional instances in each of the 4 BAR files is set to zero).
Each BAR file has just 1 flow, and is deployed to a separate execution group.

Big xml files, each with a fixed xml-header and repeating structures inside the xml-body are dropped into a folder.
And a java program splits them into smaller xml files, each having the fixed xml-header and a fixed number of repeating structures.
And we verified that the java program works correctly all the time.

flow-A takes the smaller xml files(that were split by the java program) and converts each of them to a persistent MQ message and puts them into a queue, then

flow-B takes the persistent messages and checks the MQ headers, and puts them into a different queue, then

flow-C takes the messages and processes them, and inserts/updates an Oracle database on a remote unix server, and puts them into a queue, then

flow-D takes them and does further processing, and inserts/updates an Oracle database on a remote unix server, and sends them across the network to a different queue manager on a different box.

We verified that flow-A worked fine all the time. All the smaller xml files were properly becoming MQ messages in the destination queue.
We also verified that flow-B too worked fine all the time.
When the MQ messages go to flow-C some of them just disappear, leaving no clue as to what happened.No errors anywhere. They seem to disappear in flow-C (after flow-B outputs the messages successfully).
Sometimes the messages disappear in flow-D, but this mostly happens in flow-C.

When the MQ messages go onto the second queue manager, a J2EE application server on the box takes the messages and further processes them, and updates the same oracle database that the MB does.

When we check the database we see some missing MQ messages (as the initial updates to the database from flow-C in MB didn't happen).

But when we re-drop the missing split files into the folder, they get processed successfully by flows-A,B,C, and D and then update the database properly.

We see the issue of missing-messages occasionally, especially when the original xml file is huge (when the split-files and the resulting MQ messages are more).

As mentioned already we saw the issue with both version 6 and 7 in production.
FYI - When we upgraded to version 7, we replaced the JText Adapter with flow-A.

And whenever we saw the issue, we just re-dropped the missing files (that became MQ messages) and the issue got resolved for the short-term.

But so far we couldn't figure out why the missing-messages issue has been happening.
As mentioned already, all of them are persistent messages (flow-A sets the persistence first, and then it remains the same on all the other flows)

Any clues / ideas where we can look ? I tend to think that the issue has something to do with the ODBC connection but not sure about it.

Any help is greatly appreciated.

Thanks.
_________________
Diwakar.