ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » WebSphere DataPower » MQI client connections

Post new topic  Reply to topic
 MQI client connections « View previous topic :: View next topic » 
Author Message
George Carey
PostPosted: Tue Jan 31, 2012 4:01 pm    Post subject: MQI client connections Reply with quote

Knight

Joined: 29 Jan 2007
Posts: 500
Location: DC

Looking for someone who has used MQ client connection functionality on Data Powers ... specifically an MPGW, MQ FSH component.
(if you don't know what those acronyms are you likely can't help)

Here is the scenario that I am looking for a MQI connection behavior explanation.

I put 100+ XML messages on an MQ queue that serves as the input queue for an MPGW component's MQ FSH. The MPGW processes the message (does a digital signature) and puts it out to a back end MQ queue on a second QMGR via an MQ BSH. Here is what happens.

When the MQ FSH is started (after) the messages are placed on the queue a sizable number of MQ Client channel connections get started (~10-12) and the messages are consumed and placed on the backside MQ queue where also another ~10-12 channels get started.

Questions:
1.) Why do all the freaking connections/channels get started ?
2.) They don't self terminate they remain 'Running' state but are actually stale and not active (last message date and time not updated) on all but 1 channel.
3.) If MQ FSH is started before messages sent through queue via 'q' utility or the like ... no extra channels started up and all messages processed.

Looking for a plausible connection behavior explanation?

TIA,
GTC
_________________
"Truth is ... grasping the virtually unconditioned",
Bernard F. Lonergan S.J.
(from book titled "Insight" subtitled "A Study of Human Understanding")
Back to top
View user's profile Send private message Visit poster's website AIM Address
Esa
PostPosted: Wed Feb 01, 2012 12:32 am    Post subject: Re: MQI client connections Reply with quote

Grand Master

Joined: 22 May 2008
Posts: 1387
Location: Finland

George Carey wrote:
1.) Why do all the freaking connections/channels get started ?


The MQ Client on DP may not support all client features and I think it does not support sharing connection handles between threads (MQCONNX). If you have pre-loaded the queue with messages the service obvously scales automatically upwards as it notices that it does not have to wait for new messages. It does it by starting new parallel threads.

Check the number of concurrent MQ connections in your MQ FSH configuration. The default is 1 but I think your setting is higher, probably 12. I think you can test this by setting the value to 1. You should not get more than one client connection even if you had the queue pre-loaded before you start the service (I guess).

George Carey wrote:
2.) They don't self terminate they remain 'Running' state but are actually stale and not active (last message date and time not updated) on all but 1 channel.


The remaining active channel is the one the FSH uses for listening the input queue. That the FSH does not close the other channels sounds a bit strange. Have you checked open input count for the input queue? That would tell if the FSH is caching only the connection handles or the queue handles as well.

George Carey wrote:
3.) If MQ FSH is started before messages sent through queue via 'q' utility or the like ... no extra channels started up and all messages processed.


This is because DP is much faster than q. It does not have to scale up with the tiny rate of messages q provides.

I hopes this helps. I don't actually qualify to aswer your question as I don't have any practical experience on MQ clients connections with DP.
Back to top
View user's profile Send private message
George Carey
PostPosted: Wed Feb 01, 2012 8:23 am    Post subject: will get back Reply with quote

Knight

Joined: 29 Jan 2007
Posts: 500
Location: DC

Looks like my response qualifications are bogus ... as your responses were just fine ... I will test again and check what the input/output counts and have some more input shortly I hope.

Quote:
"If you have pre-loaded the queue with messages the service obvously scales automatically upwards as it notices that it does not have to wait for new messages..."


Just exactly how does the service/code determine it does not have to wait for new messages and spawn a new thread ... trying to get that logic straight in my mind. And do those thread connections get reused were I to send new messages after a pause or would new connections be created ... what is the connection behavior logic ?
_________________
"Truth is ... grasping the virtually unconditioned",
Bernard F. Lonergan S.J.
(from book titled "Insight" subtitled "A Study of Human Understanding")
Back to top
View user's profile Send private message Visit poster's website AIM Address
George Carey
PostPosted: Wed Feb 01, 2012 4:28 pm    Post subject: open i/o counts Reply with quote

Knight

Joined: 29 Jan 2007
Posts: 500
Location: DC

Ok, here is something I don't recall seeing before ... all client connections to the MQ Server were terminated (from an app server and a Data Power). These are the only connecting devices to the MQ server and yet there were two queues that are accessed by the app server only and they both still had double digit open input an output counts on them?

I refreshed the MQ Explorer a couple of time to make sure I had latest status info but still showed double digit i/o counts with no connection channels even running. Is this the cached open handles you were referring to ?

Any way to get rid of these other than rebooting QMGR ? The queue status command showed IPs from the app server and thread 0 for all of them.
_________________
"Truth is ... grasping the virtually unconditioned",
Bernard F. Lonergan S.J.
(from book titled "Insight" subtitled "A Study of Human Understanding")
Back to top
View user's profile Send private message Visit poster's website AIM Address
fjb_saper
PostPosted: Wed Feb 01, 2012 9:04 pm    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY

Did you look at the connections where the object is the queue?
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
Esa
PostPosted: Wed Feb 01, 2012 11:39 pm    Post subject: Re: open i/o counts Reply with quote

Grand Master

Joined: 22 May 2008
Posts: 1387
Location: Finland

George Carey wrote:
Any way to get rid of these other than rebooting QMGR ? The queue status command showed IPs from the app server and thread 0 for all of them.


Depends on your MQ version. You can right-click the QM in MQExplorer and select 'Application Connections...' . From there you can close client connections manually.
Back to top
View user's profile Send private message
George Carey
PostPosted: Thu Feb 02, 2012 9:17 am    Post subject: object is queue Reply with quote

Knight

Joined: 29 Jan 2007
Posts: 500
Location: DC

Quote:
"Did you look at the connections where the object is the queue?"


I don't follow? I terminated all channels except the MQ Explorer's and yet had open handles on two queues. No application can connect except via client channels. How does one look at the connections where the object is the queue ... unless you are talking about the queue status display which I mentioned?
_________________
"Truth is ... grasping the virtually unconditioned",
Bernard F. Lonergan S.J.
(from book titled "Insight" subtitled "A Study of Human Understanding")
Back to top
View user's profile Send private message Visit poster's website AIM Address
George Carey
PostPosted: Thu Feb 02, 2012 9:22 am    Post subject: cool Reply with quote

Knight

Joined: 29 Jan 2007
Posts: 500
Location: DC

"
Quote:
You can right-click the QM in MQExplorer and select 'Application Connections"


Was not aware of this MQ Explorer option ... totally cool!

What if the UOW says NONE likely a dead connection ?
Only three of 19 say Active!

This looks like the elements of the 'dis conn(*)' command output!

Any thoughts on how/why the handles didn't clear after killing the channel connections?

GTC
_________________
"Truth is ... grasping the virtually unconditioned",
Bernard F. Lonergan S.J.
(from book titled "Insight" subtitled "A Study of Human Understanding")
Back to top
View user's profile Send private message Visit poster's website AIM Address
Esa
PostPosted: Fri Feb 03, 2012 12:13 am    Post subject: Re: cool Reply with quote

Grand Master

Joined: 22 May 2008
Posts: 1387
Location: Finland

George Carey wrote:
Any thoughts on how/why the handles didn't clear after killing the channel connections?


Sounds like you are running MQ server V6 ?

For client connections the MCA starts a client stub that receives serialised MQI commands from the client and runs them locally using bindings. In version 6 that stub didn't terminate if the channel stopped because of a network failure, for example. That caused problems if a queue had been opened in exclusive mode. When the client reconnected, it could not reopen the queue because the stub was still keeping the handle exclusively.

In MQ V7 the client protocol is full duplex and this problem should not exist. However, it may be possible that DP does not make use of all MQ V7 client connection features even if it supports V7. I'm sure there are mqseries.net members that know the client better and can verify if I remember this correctly.

You didn't tell exactly how you terminated the client channels. Stopping the channel with MQ Explorer or mqsc commands should terminate the client stub, i think. but
Code:
kill -9
might not.

Re-run your test, and this time close the client connections from Application Connections...
Then check the open input and output counts.

Anyhow, you faced the problem only when you pre-loaded the queue with messages. In real life you probably do not have so many messages, unless messages get accumulated in a queue because of a network failure, for example. Adjust the number of concurrent connections in the MQ FSH to a number you can tolerate, 5 for example, and start planning for the weekend
Back to top
View user's profile Send private message
George Carey
PostPosted: Fri Feb 03, 2012 11:50 am    Post subject: mq version 7 Reply with quote

Knight

Joined: 29 Jan 2007
Posts: 500
Location: DC

Actually the server is an MQv 7.0.0.2 version but the clients are v6 and v7
But all channel sharecnv values set to zero(0), to behave compatible to v6 clients.

(an aside what is best/easiest/most reliable way to check the client version)

Quote:
"For client connections the MCA starts a client stub that receives serialised MQI commands from the client and runs them locally using bindings",
that's the info I am looking for ... 'client stub' you mean as in an IPC, RPC type stub code component?

Actually queue build up happens not frequently but (not) infrequently and sizable number of channels connections get started. Along the lines of describing the MCA behavior what would cause that to happen? I mean why are new channel connections kicked off instead of just reading the next messages from the queue on the same channel?

Also initial client connections is set to 1. I used the MQ Explorer to stop channel for designated IP to kill the channels.

GTC
_________________
"Truth is ... grasping the virtually unconditioned",
Bernard F. Lonergan S.J.
(from book titled "Insight" subtitled "A Study of Human Understanding")
Back to top
View user's profile Send private message Visit poster's website AIM Address
fjb_saper
PostPosted: Fri Feb 03, 2012 7:14 pm    Post subject: Re: object is queue Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY

George Carey wrote:
Quote:
"Did you look at the connections where the object is the queue?"


I don't follow? I terminated all channels except the MQ Explorer's and yet had open handles on two queues. No application can connect except via client channels. How does one look at the connections where the object is the queue ... unless you are talking about the queue status display which I mentioned?

MQSC: dis conn(*) where (objname eq myqname)
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
George Carey
PostPosted: Mon Feb 06, 2012 12:30 pm    Post subject: where clause ! Reply with quote

Knight

Joined: 29 Jan 2007
Posts: 500
Location: DC

Ah, I always did a dis conn(*) and did not ever see any queue names in output field and so did not think I could use it as a where selector!

My mistake/misunderstanding/ignorance !

Looking at them now ... not seeing anything that I can tell more about, however.
_________________
"Truth is ... grasping the virtually unconditioned",
Bernard F. Lonergan S.J.
(from book titled "Insight" subtitled "A Study of Human Understanding")
Back to top
View user's profile Send private message Visit poster's website AIM Address
George Carey
PostPosted: Mon Feb 06, 2012 12:33 pm    Post subject: original question Reply with quote

Knight

Joined: 29 Jan 2007
Posts: 500
Location: DC

Anybody got more ideas on my original question?

Quote:
Questions:
1.) Why do all the freaking connections/channels get started ?
2.) They don't self terminate they remain 'Running' state but are actually stale and not active (last message date and time not updated) on all but 1 channel.
3.) If MQ FSH is started before messages sent through queue via 'q' utility or the like ... no extra channels started up and all messages processed.

Looking for a plausible connection behavior explanation?


New connections get started (in some cases not all) instead of just the next message being read on the same channel connection ... why ?

GTC
_________________
"Truth is ... grasping the virtually unconditioned",
Bernard F. Lonergan S.J.
(from book titled "Insight" subtitled "A Study of Human Understanding")
Back to top
View user's profile Send private message Visit poster's website AIM Address
Display posts from previous:   
Post new topic  Reply to topic Page 1 of 1

MQSeries.net Forum Index » WebSphere DataPower » MQI client connections
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.