ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » General IBM MQ Support » How to get the uncommited messages

Post new topic  Reply to topic Goto page 1, 2  Next
 How to get the uncommited messages « View previous topic :: View next topic » 
Author Message
bartek_n
PostPosted: Wed Sep 28, 2005 9:56 am    Post subject: How to get the uncommited messages Reply with quote

Novice

Joined: 28 Sep 2005
Posts: 10

Hio all!

Currently we've got a problem with tracking down the reasons and the point of failure in 'information flow' which bases on MQSeries queue. We have big java applications which communicate through the queue. And somehow, one message has dissapeared... I can only guess what happened and which system should be blamed...
Anyway, I can see that (basing on the current depth of the queue) there are few old uncommited messages there. I suppose there might be the one.

So, the question is - how can I get this messages from the queue?
What tool should be used - as far as I know there is no way to read them using Java API. But maybe some administration tools?

And sesond question - is it possible to configure queue manager that way, that it would log every message incoming/outgoing from the queue? Just msgId and time would do the trick...

Thanks in advance!
Bartek
Back to top
View user's profile Send private message
wschutz
PostPosted: Wed Sep 28, 2005 10:31 am    Post subject: Reply with quote

Jedi Knight

Joined: 02 Jun 2005
Posts: 3316
Location: IBM (retired)

There is no way to read uncommitted messages (unless you're reading them from the same process that created them).

Did these messages come across a channel or local puts? Are you saying that the "putting" big java application still has the unit of work open? Or has that application gone away? If it crashed or disconnected from mq the queue manager will handle the uncommitted messages properly.

The only way to log all messages is to use an API crossing exit to somehow log the messages yourself.
_________________
-wayne
Back to top
View user's profile Send private message Send e-mail AIM Address
bartek_n
PostPosted: Wed Sep 28, 2005 11:31 am    Post subject: Reply with quote

Novice

Joined: 28 Sep 2005
Posts: 10

Quote:
Did these messages come across a channel or local puts? Are you saying that the "putting" big java application still has the unit of work open? Or has that application gone away? If it crashed or disconnected from mq the queue manager will handle the uncommitted messages properly.


The application has been restarted few times and these uncommited msgs are still there. I am not sure what is the exact reason such messages are left in the queue. We are using XA transaction spanning MQ and Oracle access, so there are many possiblities something goes wrong. Usually, such msgs are removed when transaction finishes, but in some cases (I guess when no commit nor rollback has been invoked and process dies for some reason) they stay.
So, I don't know why queue manager didn't handle this...

Quote:
The only way to log all messages is to use an API crossing exit to somehow log the messages yourself.

Yeah My logs say that msg has been sent, guys from the other side say that they never received it, because they don't see any statement in their logs...
Back to top
View user's profile Send private message
wschutz
PostPosted: Wed Sep 28, 2005 3:44 pm    Post subject: Reply with quote

Jedi Knight

Joined: 02 Jun 2005
Posts: 3316
Location: IBM (retired)

Quote:
We are using XA transaction spanning MQ and Oracle access


Is MQ the transaction manager (or is something else, like websphere?) There's XA protocols about resource managers determining the outcome of indoubt transactions when they can reconnect to the transaction manager.
_________________
-wayne
Back to top
View user's profile Send private message Send e-mail AIM Address
bartek_n
PostPosted: Wed Sep 28, 2005 3:54 pm    Post subject: Reply with quote

Novice

Joined: 28 Sep 2005
Posts: 10

Quote:
Is MQ the transaction manager (or is something else, like websphere?) There's XA protocols about resource managers determining the outcome of indoubt transactions when they can reconnect to the transaction manager.


Our app is running on ATG Dynamo application server and we use its transaction manager. What do you mean - that TM should be configured specially to handle such situations? Because so far, it doesn't...
Maybe also important is that it happends from time to time that we find the lock in Oracle db held by in-doubt transaction, which seems not to be resolved automatically by Oracle's RECO process. And also, there are no serious problems with network, unclean servers' restarts etc. which would result in some unnormal behaviour.
Back to top
View user's profile Send private message
wschutz
PostPosted: Wed Sep 28, 2005 4:31 pm    Post subject: Reply with quote

Jedi Knight

Joined: 02 Jun 2005
Posts: 3316
Location: IBM (retired)

So ATG is the transaction manager and MQ and Oracle are resource managers. What I'm saying is, that if there is an indoubt MQ transaction, MQ askes ATG what the outcome of the transaction was. It sounds like there might be some protocol error, leaving indoubt messages.

I don't know anything about configuring ATG. afaik, ATG isn't listed as a supported TM.

But in any case, do you see any FDC files in the errors directory?
_________________
-wayne
Back to top
View user's profile Send private message Send e-mail AIM Address
fjb_saper
PostPosted: Wed Sep 28, 2005 8:10 pm    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY

We have the same problem.
You can't trust the ATG ecommerce transaction manager to be fully XA Compliant specifically if you are not using the jta transaction manager and are using the "FakeXADriver" which I have been told does the commit to the DB on the "prepare commit".

We are getting 2 types of uncommitted messages
a) "ghost" messages. They are uncommitted and will disappear as soon as the qmgr is recycled.

b) uncommitted messages. They have received a "prepare commit" but never got the commit or rollback. You can force commit them using dspmqtrn and rsvmqtrn.

You should have a number of FDC files with a specific Probe id (from memory 4xxxx010 ... XAE_PROTO_ERR)

At the same time you may have long running transactions getting rolled back (see qmgr logs) even though you do not have any long running transaction. This is a side effect of the messages on MQ still waiting on a definitive commit or rollback even though the session is done.(~ above 1% of the messages)

At least I know we are no longer suffering alone.

In our case we have cases where we get a close.XA being sent before an end.XA is being sent. (XA Protocol violation). We are still checking our different options and no the 10g driver against the 9i Db did not do it for us....

We have an open ticket with ATG about this problem...

Enjoy the pain
Back to top
View user's profile Send private message Send e-mail
bartek_n
PostPosted: Thu Sep 29, 2005 1:30 am    Post subject: Reply with quote

Novice

Joined: 28 Sep 2005
Posts: 10

Quote:
We have the same problem.
You can't trust the ATG ecommerce transaction manager to be fully XA Compliant specifically if you are not using the jta transaction manager and are using the "FakeXADriver" which I have been told does the commit to the DB on the "prepare commit".


I managed not to use the "FakeXADriver" - we got separate datasource using oracle.jdbc.xa.client.OracleXADataSource. However, currently (this app is 'inherited' from the previous developer/maintainer) we got 2 repositories taking part in XA transactions - one (more important) is using real XA drivers and second is still using "FakeXADriver". I got feeling this can be source of problems, but switching whole huge app (already running in live env.) right now is a bit scaring - but it will have to be done, anyway.

Quote:

We are getting 2 types of uncommitted messages
a) "ghost" messages. They are uncommitted and will disappear as soon as the qmgr is recycled.

b) uncommitted messages. They have received a "prepare commit" but never got the commit or rollback. You can force commit them using dspmqtrn and rsvmqtrn.

May you please give more light how to use these tools? is it possible then to somehow peek into these msgs header/body and see what are they from? As I understand, if manuall commit is forced, they will show up as a normal msgs in the queue?

Quote:

You should have a number of FDC files with a specific Probe id (from memory 4xxxx010 ... XAE_PROTO_ERR)

There are plenty of files in 'errors' folder, but none of them has info about such error. However, I'm not the admin of these MQSeries instance and I have no clue how it is configured...

Quote:

At the same time you may have long running transactions getting rolled back (see qmgr logs) even though you do not have any long running transaction. This is a side effect of the messages on MQ still waiting on a definitive commit or rollback even though the session is done.(~ above 1% of the messages)

I will try to verify logs (as soon as I find them...) - but it seems that this is the problem here...

Quote:

At least I know we are no longer suffering alone.

I'm glad I could help!

Quote:

In our case we have cases where we get a close.XA being sent before an end.XA is being sent. (XA Protocol violation). We are still checking our different options and no the 10g driver against the 9i Db did not do it for us....


Quote:

We have an open ticket with ATG about this problem...

Enjoy the pain

Yeah that's a lot of joy, indeed
You have reported this problem to ATG - so I guess that you've some sample code when this problem has been isolated, right? Would it be possible to give a hint how to construct such minimal snippet?

Bit thanks for reply!
Bartek
Back to top
View user's profile Send private message
wschutz
PostPosted: Thu Sep 29, 2005 2:22 am    Post subject: Reply with quote

Jedi Knight

Joined: 02 Jun 2005
Posts: 3316
Location: IBM (retired)

Quote:
You can force commit them using dspmqtrn and rsvmqtrn.


dspmqtrn is used to display in-doubt transactions. It gives you a list of transactions IDs. You can use rsvmqtrn to commit or to backout those transactions.

If the indoubt messages are a result of MQPUTs, then the backout makes those msessages go-away. If indobuts are a result og MQGETs, then the backout makes them messages re-appear.
_________________
-wayne
Back to top
View user's profile Send private message Send e-mail AIM Address
bartek_n
PostPosted: Thu Sep 29, 2005 2:48 am    Post subject: Reply with quote

Novice

Joined: 28 Sep 2005
Posts: 10

Quote:
dspmqtrn is used to display in-doubt transactions. It gives you a list of transactions IDs. You can use rsvmqtrn to commit or to backout those transactions.

If the indoubt messages are a result of MQPUTs, then the backout makes those msessages go-away. If indobuts are a result og MQGETs, then the backout makes them messages re-appear


Thanks! I am just digging into docs about these tools.

Bartek
Back to top
View user's profile Send private message
bartek_n
PostPosted: Thu Sep 29, 2005 3:05 am    Post subject: Reply with quote

Novice

Joined: 28 Sep 2005
Posts: 10

I've tried to use the dspmqtrn command and I got output like that:

Code:
AMQ7056: Transaction number 0,925213.

    XID: formatID 4871251, gtrid_length 52, bqual_length 46
         gtrid [FF5F5700FFFF57FF6461622D746573742D6170703230332E696E742E6469726162612E64652C5431313237343638323139333430]
         bqual [6461622D746573742D6170703230332E696E742E6469726162612E64652C54313132373436383231393334302C02]

AMQ7056: Transaction number 0,0.

    XID: formatID 4871251, gtrid_length 52, bqual_length 46
         gtrid [7FFF1700FFFF4BFF6461622D746573742D6170703230332E696E742E6469726162612E64652C5431313234393834333935373934]
         bqual [6461622D746573742D6170703230332E696E742E6469726162612E64652C54313132343938343339353739342C02]

AMQ7056: Transaction number 0,1.

    XID: formatID 4871251, gtrid_length 52, bqual_length 46
         gtrid [FFFF1700FFFF4BFF6461622D746573742D6170703230332E696E742E6469726162612E64652C5431313234393834333935373934]
         bqual [6461622D746573742D6170703230332E696E742E6469726162612E64652C54313132343938343339353739342C02]



and according to documentation it should be like:

Code:

AMQ7107: Resource manager 0 is MQSeries.
AMQ7107: Resource manager 1 is Oracle MQBankDB
AMQ7107: Resource manager 2 is Oracle MQFeeDB
 
AMQ7056: Transaction number 0,1.
    XID: formatID 5067085, gtrid_length 12, bqual_length 4
         gtrid [3291A5060000201374657374]
         bqual [00000001]
AMQ7105: Resource manager 0 has committed.
AMQ7104: Resource manager 1 has prepared.
AMQ7104: Resource manager 2 has prepared.



Why there is no info displayed about other resource managers? Is it different situtation because MQSeries is not a transaction manager but resource manager?

And more - is it possible to match such entry with the lock in Oracle (held by in-doubt transaction)?
Back to top
View user's profile Send private message
wschutz
PostPosted: Thu Sep 29, 2005 6:06 am    Post subject: Reply with quote

Jedi Knight

Joined: 02 Jun 2005
Posts: 3316
Location: IBM (retired)

Quote:
Is it different situtation because MQSeries is not a transaction manager but resource manager?
Yes, if MQ isn't the TM, it doesn't know about the RM's. Is this causing you a problem? Can't you still use rsvmqtrn to correct the indoubt uow's?
_________________
-wayne
Back to top
View user's profile Send private message Send e-mail AIM Address
bartek_n
PostPosted: Thu Sep 29, 2005 6:26 am    Post subject: Reply with quote

Novice

Joined: 28 Sep 2005
Posts: 10

Quote:
Yes, if MQ isn't the TM, it doesn't know about the RM's. Is this causing you a problem?

I hope not, just wandering why mine otput was different.

Quote:
Can't you still use rsvmqtrn to correct the indoubt uow's?

I guess I could, but I have to wait before trying this until the system scanning this queue will be down and when some admin will invoke this command (I don't have rights to do this).
As I understood, it is possible that after running it some of the 'ghosts' could show up and I don't want the listening system to pick it up, before I get to this msgs.

Big thanks for help!
Bartek
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Thu Sep 29, 2005 12:56 pm    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY

Thats the right approach.

We create a .sh file using
Code:
dspmtrn | grep AMQ | sed |sed > forcecommit.sh


The output is right.
rsvmqtrm will need to have something like:
Code:
rsvmqtrn -m QMGR -c 0,925213


Have you tried yet using the JTA transaction manager supplied by ATG ?

Quote:
You have reported this problem to ATG - so I guess that you've some sample code when this problem has been isolated, right? Would it be possible to give a hint how to construct such minimal snippet?


To reproduce the problem we need to stress test in our qual environment.
The problem appears in between 1 to 2 % of the messages we shot through. There does not seem to be a specific pattern to it and when you try to isolate it transaction by transaction it seems to always work as designed ....


Enjoy
Back to top
View user's profile Send private message Send e-mail
bartek_n
PostPosted: Thu Oct 06, 2005 3:02 am    Post subject: Reply with quote

Novice

Joined: 28 Sep 2005
Posts: 10

Quote:
Have you tried yet using the JTA transaction manager supplied by ATG ?


Basing on information from the page:
https://www.atg.com/portal/myatg/developer/support?paf_dm=full&paf_gear_id=2200004&page=supportTechBulletin.jsp&id=9406#topic1
seems to me that there is no point in switching from the default (since ATG 6.0.0) com.sun.jts.jta.TransactionManagerImpl to using class atg.dtm.TransactionManagerImpl, when we are using XA transactions.

So, I will not try to use the old one, specially that we don't have any separate environment/sample code to reproduce the error at will...

Still struggling with the problem...
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Goto page 1, 2  Next Page 1 of 2

MQSeries.net Forum Index » General IBM MQ Support » How to get the uncommited messages
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.