| Author |
Message
|
| TheAndroid |
 Posted: Fri Feb 22, 2008 1:41 pm Post subject: MQ as a Resource Manager in "Thread of control" mo |
 |
|
Novice
Joined: 04 Dec 2007
Posts: 23
|
All,
I'm seeing something odd with MQ V6 on AIX 5.3. I have built an executable which includes my "lite" transaction manager. Everything appears to be working per the XA/Open spec. However, whenever I call prepare on the MQ session, it always returns RD_ONLY.
By the spec, this means that MQ considers the current UOW as read only and thus shouldn't have commit called (the RD_ONLY implies it has already committed).
Meanwhile, my main program has done several SYNCPOINT MQGET and MQPUT calls. The messages appear to be on (or off, depending on the activity) the queues. This confuses me as clearly MQ thinks there is nothing to commit.
However, if I let the process run long enough, I will get the dreaded MQRC_BACKED_OUT error. When that happens, ALL of the messages received with MQGET are placed back on the queue.
This tells me that the activity on from the main program is not being tracked correctly within the MQ Extended Client. I believe am getting the RD_ONLY because of this. I am linking with the appropriate libraries (libmqcxa and libmqz) as the Websphere site states.
Any ideas? |
|
| Back to top |
|
 |
| jefflowrey |
| Posted: Fri Feb 22, 2008 3:48 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
And you're not actually failing to COMMIT?
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| TheAndroid |
| Posted: Fri Feb 22, 2008 4:03 pm Post subject: |
|
|
Novice
Joined: 04 Dec 2007
Posts: 23
|
Yes and no. I don't call MQCMIT. I shouldn't have to. In fact, the manual says I should not call MQCMIT or MQBEGIN because I am using "external" syncpoint management.
The main application calls my transaction manager's commit which ends the current branch, prepares (and gets RD_ONLY for it's trouble) and then commits (not MQ though because of the RD_ONLY).
This sequence works perfectly for UDB. I'm beginning to suspect that MQ does not handle transaction managers using it in static mode very well. I am currently testing with dynamic registration to see if I get called back by MQ when I do the first MQGET. I didn't use dynamic registration in the first place because I didn't want to do the extra work! |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Fri Feb 22, 2008 4:11 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY
|
Can you please clarify where your transaction manager is.
From what you are saying it is not clear to me whether your TM is MQ or an outside TM?
What are all the other parties participating in the transaction?
Thanks
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| TheAndroid |
| Posted: Fri Feb 22, 2008 6:08 pm Post subject: |
|
|
Novice
Joined: 04 Dec 2007
Posts: 23
|
Okay..... there are a few players in this so I'll have to name each one.
I have a C application, which has embedded MQ and SQL calls within it, that calls a TransactionManager I wrote which is contained within a .so. We'll call the C Application CApp and the Transaction Manager TM. Using this model, they basically combine into one thing, but they have different responsibilities.
When CApp begins, it establishes an MQI connection by calling MQCONN and MQOPEN. It then tells TM that it wants to begin using coordinated transactions. TM reads it's configuration file and finds the an XASwitch entry for MQ and UDB. It uses this information to dynamically load the XASwitch .so's for each. Let's call them RM/MQ and RM/UDB as each is in the role of a Resource Manager in the XA scheme of things.
Once the switch structures are obtained, TM does the appropriate calls to initialize and start a transaction thread with RM/MQ and RM/UDB. Most importantly, TM calls xa_start on RM/MQ and RM/UDB which are successful. In the XA world, this means they are able (and responsible) for determining if any of thier own data resources have changes.
Because of the way the Extended MQ client is written, RM/MQ is sharing the MQI connection that CApp already established. RM/UDB is doing the same with CApp's UDB connection.
Now, CApp does a series of MQGET calls setting the Get Message Options to include SYNCPOINT. After each MQGET, it takes the message and inserts a record into UDB. Additionally, CApp does an MQPUT1 (also with SYNCPOINT in the options) onto a different queue acknowledging the message was received. This continues for 40 messages or when the input queue is drained.
At that point, CApp calls TM requesting a commit. TM, using the XA switch obtained earlier calls xa_end (which closes the UOW) and xa_prepare (to determine if RM/MQ will fail the transaction) on RM/MQ (which returns RD_ONLY which is not a failure) and RM/UDB (which returns XA_OK). At this point, TM sets a flag for RM/MQ signifying that it should not have xa_commit called on it. This is because RM/MQ returned RD_ONLY which by the XA spec means the transaction was effectively committed as no changes were noted.
And that's wrong. CApp has read 40 messages and put 40 more. Moreover, all of this activity happened within the UOW TM created. RM/MQ acknowledged this UOW because the xa_start call to it was successful. By the XA spec, it is supposed to notice the changes to it's resources! All of this activity happened on the same thread.
TM honors the flag and does not call xa_commit on RM/MQ. It does call xa_commit on RM/UDB which does successfully file down the new rows in the table.
CApp continues processing in a loop taking messages off, putting messages on and inserting rows. About 4000 messages into this cycle, CApp receives a 2013 from MQGET (the ROLLED_BACK error). In an effort to end gracefully, CApp calls TM requesting a rollback. TM calls end and then prepare on RM/MQ which returns RD_ONLY (again!). RM/UDB on the other hand does the rollback and returns cleanly.
At this point, I now have 4000 some odd messages back on my original input queue. I also have 4000 less messages on my output queue. RM/MQ never, ever committed.
The IBM MQ website states that when using an external transaction coordinator, you are never to call MQBEGIN, MQCMIT or MQROLLBACK. This is because TM is presumably handling the coordination of several Resource Managers which are participating within the same UOW. Calling MQCMIT would only commit MQ thus negating any advantage having 2 phase commit would offer.
XA is forcing these gyrations on me. I have to insure that the database content matches the queue content. The only way to do this is with 2 Phase Commit.
So you are aware, I am using both UDB and MQ in static mode. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Sat Feb 23, 2008 3:13 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
You may be trying to do things with the ETC that are beyond it's scope. I'm not positive that it's designed to allow you to use any TM. It may only be designed and licensed to use with some particular TMs.
Either way, it seems you're running into behavior that is far outside most people's scopes and uses of the product. It seems a good idea to open a PMR on the RD_ONLY.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sat Feb 23, 2008 8:22 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY
|
| TheAndroid wrote: |
| At this point, TM sets a flag for RM/MQ signifying that it should not have xa_commit called on it. This is because RM/MQ returned RD_ONLY which by the XA spec means the transaction was effectively committed as no changes were noted. |
I am not sure that your interpretation of this and MQ's interpretation of this coincide. What happens if you nevertheless call xa_commit on MQ?
| jefflowrey wrote: |
You may be trying to do things with the ETC that are beyond it's scope. I'm not positive that it's designed to allow you to use any TM. It may only be designed and licensed to use with some particular TMs.
Either way, it seems you're running into behavior that is far outside most people's scopes and uses of the product. It seems a good idea to open a PMR on the RD_ONLY. |
I thought the ETC especially in its java form was geared towards a J2EE TM only... So I have no idea how the other forms would work.
Perhaps geared towards MSTC?
As Jeff said, open a PMR!
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| TheAndroid |
| Posted: Mon Feb 25, 2008 3:18 pm Post subject: |
|
|
Novice
Joined: 04 Dec 2007
Posts: 23
|
Here's an update on the saga:
I have the XA stuff working. To use the extended client with XA under AIX, one has to link BOTH libmqic and libmqcxa. Apparently, if you just use libmqic, the xa portion of application does not have any visibility to the normal MQ calls. The manual sort of says this, but it isn't abundantly clear.
So now I'm on to another problem. When I was using the server version of the libraries, I did not require having a channel defined to access qmgrs on the same instance. For example, if I had one channel, defined to go to QM1, and I had that channel defined via the MQSERIES environment variable, I could open QM2 (which happens to reside on the same server instance as QM1) without incident. Both QM1 and QM2 have listeners running and I think this is what makes this happen (IMBW).
With the extended client, this looks like it is no longer true. I can only connect to QM1 because it has a channel defined. Is there some snazzy way I can get around this? What would be ideal is to have QM1 forward my connection to the listener for QM2. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Mon Feb 25, 2008 3:31 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY
|
I believe what you are looking for is a channel table. See clients manual.
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Mon Feb 25, 2008 4:43 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
You probably don't understand how you were actually connecting to QM2.
You likely were making an indirect connection - by way of QM1 - rather than the direct connection you think you were making.
MQSERVER only allows you to connect to THE queue manager living on THE channelname/TCP/hostname(port) that you specify.
Using a client channel gives you more flexibility.
Using MQCONNX gives you even more flexibility - at the cost of increasing the amount of information you may need to include in your application's configuration file.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| TheAndroid |
| Posted: Mon Feb 25, 2008 5:52 pm Post subject: |
|
|
Novice
Joined: 04 Dec 2007
Posts: 23
|
| jefflowrey wrote: |
| You probably don't understand how you were actually connecting to QM2. |
I completely agree. Sifting through the manuals is making my head spin!
| jefflowrey wrote: |
You likely were making an indirect connection - by way of QM1 - rather than the direct connection you think you were making.
|
Schweeet! That's what I want. It would be great to have it back.
| jefflowrey wrote: |
MQSERVER only allows you to connect to THE queue manager living on THE channelname/TCP/hostname(port) that you specify.
|
MQSERVER or MQCLIENT only allows it? I used to link with the server library (libmqm). Now I'm using the client one (libmqic)
Please tell me I don't have this backwards in my head! The server library did what I wanted. The client one doesn't appear to.
| jefflowrey wrote: |
Using a client channel gives you more flexibility.
Using MQCONNX gives you even more flexibility - at the cost of increasing the amount of information you may need to include in your application's configuration file. |
I'm porting code from z/OS. I really, really, really don't want to touch it if I don't have to.
| fjb_saper wrote: |
I believe what you are looking for is a channel table. See clients manual.
|
I did try the channel table. I set the environment variables per the manual. I got no joy. I'm quite sure I missed something.
I don't understand how putting the IP address alone in the CONNNAME gets the traffic to the listener (this is what the example in the manual does). The listeners for the qmgrs are each on a specific port and at least by what the manual says that's the right configuration.
Moveover, how does the MQI library know which channel I want? Would I have to declare that somewhere else?
Thanks! |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Tue Feb 26, 2008 2:17 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Aha.
You meant something different by MQSERVER than I thought you did.
Yes, if you're using a bindings connection, then you can connect to any qmgr on the same host. And you weren't making an indirect connection.
I thought you were using the MQSERVER environment variable to establish a client connection.
Regardless, you can connect to any qmgr you want, directly, from your code. You just need to know how to point to it.
To establish a client connection to a queue manager, you need three pieces of information: the network address (hostname/IP) of the machine, the Port # of the MQ listener, and the channel name. This forms a unique destination.
There are three main ways to specify this information: 1) set the environment variable MQSERVER to the value "Channel Name/TCP/hostname(port)", 2) set the environment variables for the MQ Client Channel Table, 3) use MQCONNX.
In the case of a z/OS queue manager, you may need an additional piece of software installed and running on zOS called the "Client Attachment Facility".
And remember that you can only GET from qlocals on the qmgr you're connected to.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Thu Apr 10, 2008 1:09 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9405
Location: US: west coast, almost. Otherwise, enroute.
|
| Quote: |
| a TransactionManager I wrote |
IBM doc refers to a transaction manager or external syncpoint manager as an "XA-compliant transaction manager," like BEA Tux, TXeries, RRS. I don't believe that implies a TM that you wrote.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
|
|
