ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » User Exits » API exit - when are threads called for init and term?

Post new topic  Reply to topic
 API exit - when are threads called for init and term? « View previous topic :: View next topic » 
Author Message
NomadAU
PostPosted: Thu Aug 24, 2017 7:50 pm    Post subject: API exit - when are threads called for init and term? Reply with quote

Novice

Joined: 06 Feb 2017
Posts: 15

For some time now I have been working on an API exit. Until recently having run the exit in a simple MQ environment (QM and some java apps reading/writing) I was very comfortable that I knew how/when MQ invoked the exit, but after deploying it into its target environment and doing some basic testing, I am now very much confused.

Up until now, I have thought that
- the EntryPoint function would be called ONCE ONLY for each unique MQ thread (i.e. a unique combination of process id and thread id).
- the Terminate function(if registered) would also be called ONLY ONCE per thread.
- apart from EntryPoint, none of the other callback functions would be called without the calling thread first having called EntryPoint

All this seems reasonable, and provides a sound programming model for me to allocate and manage resources on a per process, and/or per thread basis.

However, my recent discovery has thrown all this into complete confusion, so I'm hoping there's an API exit guru or two out there who can shed some light.

To prove what I'm about to say, all I have done is created a 'bare-bones' exit that registers all callbacks and then does some simple logging in each of the call back functions when they are called.
What is apparent is that
- the exact same thread can call EntryPoint multiple times, prior to calling Terminate
- the MQ CallBack Before (MQXR_BEFORE, MQXF_CALLBACK) and CallBack After (MQXR_AFTER, MQXF_CALLBACK) is invoked without ever calling EntryPoint

In the first case (multiple EntryPoint calls) I sometimes see the exact same number of Terminate calls made for the thread, but not always.

Understandably, this has thrown my implementation into complete disarray, as I can no longer rely on having 'seen' a particular thread and done initialisation for it, prior to one of the exit call back functions being called.
Furthermore, even if I keep a count of the number of initialisation calls for a thread, I have no guarantee that the same number of terminate calls will be made, so this makes freeing of resources almost impossible.

Has anyone else seen this behaviour? It certainly doesn't seem normal to me, but I'm open to suggestions.

The environment is
RHEL 7
MQ 8.0.0.6, level LAIT17234-175072
Application running is IIB v10
Back to top
View user's profile Send private message
hughson
PostPosted: Thu Aug 24, 2017 9:44 pm    Post subject: Reply with quote

Padawan

Joined: 09 May 2013
Posts: 1914
Location: Bay of Plenty, New Zealand

I thought INIT was all about a connection initialising not a thread. If a connection handle was created on one thread and later used on another thread (SHARED hConns) then that thread may not have seen an INIT call, but the connection certainly had.

wrt CallBack invocations, they may well be on a different thread, because the thread is managed by the queue manager rather than the application, so that's perhaps just another example of the above.

You mention obtaining and freeing resources. The usual way is to save yourself off a little area in UserExitArea with what you need on INIT, and then on TERM, test for existence of, and then free off said resources.

Are you reliant in some way on the thread location? There are restrictions you could make the applications use to ensure that the connection is always tied to the thread, even for call back invocations. Can you say a little about the resources you are obtaining at INIT?
_________________
Morag Hughson @MoragHughson
IBM MQ Technical Education Specialist
Get your IBM MQ training here!
MQGem Software
Back to top
View user's profile Send private message Visit poster's website
tczielke
PostPosted: Fri Aug 25, 2017 7:40 am    Post subject: Reply with quote

Guardian

Joined: 08 Jul 2010
Posts: 939
Location: Illinois, USA

Note that there is a bug with MQ where the incorrect thread id number can be passed into the API exit.

http://www.mqseries.net/phpBB2/viewtopic.php?t=69770

AndyH who is on the MQ development team confirmed this for me back in 2015 when I posted that thread. I am not sure if the bug was fixed or when. Perhaps AndyH can comment.
_________________
Working with MQ since 2010.
Back to top
View user's profile Send private message
Andyh
PostPosted: Fri Aug 25, 2017 8:26 am    Post subject: Reply with quote

Master

Joined: 29 Jul 2010
Posts: 237

Tim,
I'm afraid I don't seem to have copy of the e-mail that I apparently sent to you ! Please would you forward that e-mail back to me so that I've hopefully got a little more context.
Thanks
Andy.
Back to top
View user's profile Send private message
Andyh
PostPosted: Fri Aug 25, 2017 9:01 am    Post subject: Reply with quote

Master

Joined: 29 Jul 2010
Posts: 237

Thanks Tim for forwarding me the information.
I'm afraid that this issue has NOT yet been resolved, it looks like it should be a trivial fix but if you want it in a current release then a PMR/APAR will need to be raised.
Back to top
View user's profile Send private message
tczielke
PostPosted: Fri Aug 25, 2017 9:50 am    Post subject: Reply with quote

Guardian

Joined: 08 Jul 2010
Posts: 939
Location: Illinois, USA

Thanks, Andy!

NomadAU - For some more context, the issue is when you have multiple threads that are using the same connection. There is a bug with MQ where it will keep sending in the thread id for the first thread that created the connection into the API exit. So if thread id #1 creates the connection and thread id #2 later uses that same connection, the API exit incorrectly gets sent thread #1, eventhough it is really thread id #2 that is running with this connection. It sounds like you will need to raise a PMR, if you want this fixed. It sounds like your API exit is probably being impacted by this bug, too.
_________________
Working with MQ since 2010.
Back to top
View user's profile Send private message
hughson
PostPosted: Fri Aug 25, 2017 2:53 pm    Post subject: Reply with quote

Padawan

Joined: 09 May 2013
Posts: 1914
Location: Bay of Plenty, New Zealand

If this is a bug, then it's also in Activity Trace. That will continue to show the thread that created the hConn in all subsequent api calls even if they are done on another thread. Essentially the thread ID shown on DISPLAY CONN, is always used to record all Activity Trace.

If this bug in API exits is fixed, would the Activity Trace also get fixed as a matter of course, or is that different code showing the same behaviour?

Cheers
Morag
_________________
Morag Hughson @MoragHughson
IBM MQ Technical Education Specialist
Get your IBM MQ training here!
MQGem Software


Last edited by hughson on Sun Aug 27, 2017 3:34 am; edited 1 time in total
Back to top
View user's profile Send private message Visit poster's website
Andyh
PostPosted: Fri Aug 25, 2017 11:54 pm    Post subject: Reply with quote

Master

Joined: 29 Jul 2010
Posts: 237

While I don't believe this is currently working as designed (and expect to fix it), I'd have expected the MQ thread id to be of relatively limited interest/use to an API exit.

As indicated earlier, in a shared hConn environment there is no correlation between the thread and the hConn, and as Morag pointed out earlier an hConn scoped context field is provided via the ExitUserArea.

If you were doing something that relied upon OS thread context, then it might be better to use an explicit OS thread context mechanism, for example pthread_key_create and friends (on Unix).
Back to top
View user's profile Send private message
tczielke
PostPosted: Sat Aug 26, 2017 4:25 am    Post subject: Reply with quote

Guardian

Joined: 08 Jul 2010
Posts: 939
Location: Illinois, USA

hughson wrote:
If this is a bug, then it's also in Activity Trace. That will continue to show the thread that created the hConn in all subsequent api calls even if they are done on another thread. Essentially the thread ID shown on DISPLAY CONN, is always used to record all Activity Trace.

If this but in API exits is fixed, would the Activity Trace also get fixed as a matter of course, or is that different code showing the same behaviour?

Cheers
Morag


Yes, I do see the same issue with Activity Trace (tracing amqscbf). strmqtrc is recording the different threads correctly, however. I will open a PMR for both the API exit and Activity Trace, so this can at least get tracked and looked at properly.
_________________
Working with MQ since 2010.
Back to top
View user's profile Send private message
NomadAU
PostPosted: Sun Aug 27, 2017 3:42 am    Post subject: Reply with quote

Novice

Joined: 06 Feb 2017
Posts: 15

Thankyou to all who replied, particularly Morag, who put me right in her first sentence. It does seem pretty obvious...once it's been pointed out
I've now restructured my design and am pleased to say it is now behaving in a consistent (and understandable) manner. And yes, I was trying to tie the OS pid/tid combinations to MQ handles, and that would never have worked...although there is more often than not, a strong correlation between the 2 resources types. This was what caught me out, and lead me to my incorrect assumption on the programming model. BTW, I had already seen the odd (incorrect) TID value being passed by MQ but had worked around this - it might also be of interest to know that the TID passed via the PMQAXC is different to the underlying OS TID.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Page 1 of 1

MQSeries.net Forum Index » User Exits » API exit - when are threads called for init and term?
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.