ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » IBM MQ Java / JMS » Setting MQ exclusive-read client properties in JMS

Post new topic  Reply to topic Goto page Previous  1, 2
 Setting MQ exclusive-read client properties in JMS « View previous topic :: View next topic » 
Author Message
RGFanta
PostPosted: Thu Sep 23, 2010 6:26 am    Post subject: Our use case and design is pretty solid Reply with quote

Novice

Joined: 03 Sep 2010
Posts: 12

Thanks for your reply.

Quote:
Going the controlling route you'd need to set the attribute on the destination.
This is where there is some difficulty as the attribute is not readily available when you cast the Queue to an MQQueue. However you do have the capability to do a setProperty(name, value) on the Queue / Destination.


Yes, but finding the equivalent MQ JMS extensions properties (i.e. a simple mapping of MQ properties Exclusive Read: On, Shareability: Off) is not straight-forward, and combinatoric. Crazy.


Quote:
Finding the right values for this, or whether controlling this attribute from JMS is even possible is a matter for a PMR.


That such a simple matter can't be explained through documentation and needs paid advice is a non-positive indictment of IBM.


Quote:
Be at the same time aware that you are severely restricting JMS scalability because at that point you can only have at best 1 MDB instance.
......
My advice: It looks like your design has a serious message affinity problem (you cannot have multiple instances of the consumer receiving messages concurrently). Solve your message affinity problem and not only will this requirement be moot, but you will greatly gain in throughput and scalability.


Our use case is pretty solid. The application that needs to be the sole consumer is a scheduler, which allocates work to a pool of downstream consumer applications. In some cases it needs to hold up various types of units of work based on differing criterion. Since it keeps state and does different actions based on the outcomes of the downstream processing, it needs to be a solo instance.

Restricting to one exclusive reader in the client application is safer to do from the client side than administratively.
Back to top
View user's profile Send private message
Vitor
PostPosted: Thu Sep 23, 2010 6:42 am    Post subject: Re: Our use case and design is pretty solid Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

RGFanta wrote:
Yes, but finding the equivalent MQ JMS extensions properties (i.e. a simple mapping of MQ properties Exclusive Read: On, Shareability: Off) is not straight-forward, and combinatoric. Crazy.


I stand by my previous comments. Parts of the JMS preclude access to native WMQ functions. JMS applications (because they're intended to work without modification across multiple vendors) can't use vendor specific functions. So accessing those functions "behind the back" of JMS is naturally problematic.

RGFanta wrote:
That such a simple matter can't be explained through documentation and needs paid advice is a non-positive indictment of IBM.


No, it's an indication that you're trying to bend the JMS standard and you need vendor advice on how far it can be bent & how.

Remember also this is why IBM provides 2 Java implementations; one conforming to the JMS (with all the limitations and restrictions that implies) and one exposing the native interface.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
mqjeff
PostPosted: Thu Sep 23, 2010 7:06 am    Post subject: Re: Our use case and design is pretty solid Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

Vitor wrote:
No, it's an indication that you're trying to bend the JMS standard and you need vendor advice on how far it can be bent & how.

Remember also this is why IBM provides 2 Java implementations; one conforming to the JMS (with all the limitations and restrictions that implies) and one exposing the native interface.




RGFanta wrote:
Our use case is pretty solid. The application that needs to be the sole consumer is a scheduler, which allocates work to a pool of downstream consumer applications. In some cases it needs to hold up various types of units of work based on differing criterion. Since it keeps state and does different actions based on the outcomes of the downstream processing, it needs to be a solo instance.


No, this merely means that you think you need to solve this with a single instance and with an exclusive consumer.

If you externalize the state to a database, then you can update multiple sets of state from multiple instances. Work that is held can also be externalized to the database and forwarded for processing when the criteria is met - independent of the source.

Remember that every time you build a single-instance single-threaded component, you have done nothing more than put a neck on the bottle.
Back to top
View user's profile Send private message
RGFanta
PostPosted: Thu Sep 23, 2010 3:57 pm    Post subject: Disagree, Vitor Reply with quote

Novice

Joined: 03 Sep 2010
Posts: 12

Vitor, I can't agree with you.

MQ JMS is built on top of (non-JMS) MQ, so "bending" should be trivial with good documentation on what is a very straight-forward issue. Sadly, reality is otherwise.

Also, as you know, JMS is just an interface specification for standard features common to all JMS implementations.

By your logic, why even have any MQ JMS Extensions at all?? Clearly they provide value in some cases (like this one). In the few cases that they're needed, those extensions should be easy to use.

If given the choice, I'd much rather do little small coding for a mostly JMS compliant application (that provides messages to JMS consumer using an application built on a vendor product) than to have to rewrite a good bit to use native non-JMS MQ calls. The MQ JMS Extensions were intended to help with that.


Last edited by RGFanta on Thu Sep 23, 2010 4:30 pm; edited 1 time in total
Back to top
View user's profile Send private message
RGFanta
PostPosted: Thu Sep 23, 2010 4:16 pm    Post subject: Are you forgetting that distributed transactions don't work? Reply with quote

Novice

Joined: 03 Sep 2010
Posts: 12

Quote:
No, this merely means that you think you need to solve this with a single instance and with an exclusive consumer.

If you externalize the state to a database, then you can update multiple sets of state from multiple instances. Work that is held can also be externalized to the database and forwarded for processing when the criteria is met - independent of the source.


That's another way to try to solve the problem, but in event/message driven systems with databases, it forces you to rely on distributed transactions which don't fully work.

1) Message arrives and triggers processing (start GET transaction on MQ)
2) Update database (2nd transaction on Database)
3) Publish async message to next processing stage (using a PUT on MQ)

1) and 3) can happen atomically using MQ sync points if both queues are on the same queue manager. Transaction managers try to do both the MQ GET/PUT commit (transaction #1) and the database commit (transaction #2) at the same time, but that's on a best effort basis. There are no guarantees, and any implementation has to do one commit before doing the next.

That leaves you with retry logic having to check the backout count and whether the db transaction succeeded .... and compensating if it didn't.... (for every single kind of db transaction that your application does)...

I know that having a single scheduler instance can be a bottle neck, but as long as you're not trying to do too much in that component, it's fine, fast, and foolproof.

I understand the tradeoffs between these two approaches.
Back to top
View user's profile Send private message
mqjeff
PostPosted: Thu Sep 23, 2010 5:09 pm    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

I think you misunderstand the role of XA or the meaning of "best effort" in this case.

Or you are working under reliability constraints that are significantly higher than one would normally expect.

Given that you appear to be writing and using a standalone program and not a program that runs inside a Java EE container, it is less clear why you continue to insist on using a Java EE standard - JMS - in ways that are outside the scope of what is allowed or designed for in Java EE and JMS.

All of the advantages of using JMS that you get in terms of abstracting connectivity information from the application logic are directly achievable in the MQ Java interface using a CCDT. Almost all of the other "advantages" of JMS over MQ Java are also not significantly lost. It is a slightly more complicated API to implement, but 95% of it is SMOP (especially for someone who routinely deals with XA environments and fully understands performance tradeoffs vs. complexity tradeoffs of single-threading state machines rather than using distributed state).

And you can not really think of the MQ JMS interface as being "built on top of" the MQ Java interface. They are built to implement the JMS interface. This is a very different use case than "extending the MQ Java interface to provide JMS functions" and very fully explains why it is not trivial to do what you're trying to do. At least, supposing that the JMS standard doesn't allow for exclusive consumers, which I don't know either way.
Back to top
View user's profile Send private message
RGFanta
PostPosted: Thu Sep 23, 2010 5:36 pm    Post subject: Reply to mqjeff Reply with quote

Novice

Joined: 03 Sep 2010
Posts: 12

Quote:
I think you misunderstand the role of XA or the meaning of "best effort" in this case.


I've used XA and similar "best effort, first commit" systems for years, and am told that I understand the trade-offs of using them well.

Quote:
Or you are working under reliability constraints that are significantly higher than one would normally expect.


Expected reliability is very high, yes. That's pretty common in the environments I've worked in.

Quote:
Given that you appear to be writing and using a standalone program and not a program that runs inside a Java EE container, it is less clear why you continue to insist on using a Java EE standard - JMS


Sorry, but a great many applications use JMS outside of J2EE. In this case our application is built on top of a product called Spring Integration which has JMS connections out of the box, but no MQ connectors. J2EE is pretty bloated and many people find that they can get most of the benefit and far fewer drawbacks by just using frameworks like Spring.

Quote:
All of the advantages of using JMS that you get in terms of abstracting connectivity information from the application logic are directly achievable in the MQ Java interface using a CCDT.


Yes, if you want to fully roll your own transports and maintain them....

Quote:
And you can not really think of the MQ JMS interface as being "built on top of" the MQ Java interface. They are built to implement the JMS interface. This is a very different use case than "extending the MQ Java interface to provide JMS functions" and very fully explains why it is not trivial to do what you're trying to do. .


This almost seems backwards. The MQ JMS interface is indeed built on top of MQ, no? What are MQ JMS Extensions intended for if not to allow (mostly) JMS applications to use more of the (non-JMS) MQ functionality?

I'm not asking for anything collossal. I just don't understand why getting at 2 basic MQ properties should be this difficult and require this much effort.

Do you work for IBM? I appreciate the thoughtfulness to challenge our design (many people fail to think things through), but there seems to be more focus here on poking holes in our pretty sound use case than in the premise of the original question.
Back to top
View user's profile Send private message
RGFanta
PostPosted: Thu Sep 23, 2010 5:55 pm    Post subject: Direct quote Reply with quote

Novice

Joined: 03 Sep 2010
Posts: 12

http://publib.boulder.ibm.com/infocenter/wmqv7/v7r0/index.jsp?topic=/com.ibm.mq.csqzaw.doc/jm35150_.htm

WebSphere® MQ classes for JMS contains a set of extensions to the JMS API called the IBM® JMS extensions. An application can use these extensions to create connection factories and destinations dynamically at run time, and to set the properties of WebSphere MQ classes for JMS objects.

[Emphasis mine.]
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Thu Sep 23, 2010 7:22 pm    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY

RGFanta wrote:
I'm not asking for anything collossal. I just don't understand why getting at 2 basic MQ properties should be this difficult and require this much effort.


That may really look like this to you, because you discard completely the use purpose and pattern of the MDB JMS Standard and try to bend to your own need. MQ JMS is not just a bunch of classes on top of MQ. It is also an implementation of a design pattern and a standard.

Like I said the standard caters for having a class implement the MessageDrivenBean interface, implement the MessageListener interface (public void onMessage(javax.jms.Message msg) and be able to scale this in the J2EE server by using configuration.

Frameworks like Spring build upon this standard and the patterns it follows, possibly with even a stricter enforcement of the pattern than called for by the spec.

The Connection and Destination are defined by configuration
The number of instances (vertical scaling) are defined by the configuration.
You also may get to use a helper class (ActivationBean) that allows you some more flexibility with the J2EE framework.


So by the intended use pattern you should not be able to use the destination as exclusive read.

Stop grabbing for an anti-pattern and start designing your application in a correct scalable way.

Note that this does not invalidate your use case in any way. However you will not be successful trying to implement it using anti-patterns!

Have fun

NO I Don't work for IBM...
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
RGFanta
PostPosted: Thu Sep 23, 2010 7:39 pm    Post subject: Reply to fjb Reply with quote

Novice

Joined: 03 Sep 2010
Posts: 12

Fjb, there are no anti-patterns here.

Did you see the previous postings with a) the (verbatim quote) from the IBM® JMS extensions documentation? b) the fact that we're not using the full J2EE framework much like so many other people/projects out there?

Some applications benefit from full J2EE, while others (e.g. those using Spring) take the parts they need and do very well w/o the full spec. [I've used both.]

I understand the tradeoffs, and have made a good choice for our needs.

If you have any further insight into the original question (which almost seems to have gotten lost in this thread), I'd be delighted to hear it. Thanks for your previous suggestions.
Back to top
View user's profile Send private message
joebuckeye
PostPosted: Fri Sep 24, 2010 5:59 am    Post subject: Re: Direct quote Reply with quote

Partisan

Joined: 24 Aug 2007
Posts: 365
Location: Columbus, OH

RGFanta wrote:
http://publib.boulder.ibm.com/infocenter/wmqv7/v7r0/index.jsp?topic=/com.ibm.mq.csqzaw.doc/jm35150_.htm

WebSphere® MQ classes for JMS contains a set of extensions to the JMS API called the IBM® JMS extensions. An application can use these extensions to create connection factories and destinations dynamically at run time, and to set the properties of WebSphere MQ classes for JMS objects.

[Emphasis mine.]


I would bet they mean some properties, not all.

As an example, there is no method in the MQ JMS API to get the depth of a queue but it's a simple property of the queue in the base API (getCurrentDepth()).
Back to top
View user's profile Send private message
mqjeff
PostPosted: Fri Sep 24, 2010 6:20 am    Post subject: Re: Direct quote Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

joebuckeye wrote:
RGFanta wrote:
http://publib.boulder.ibm.com/infocenter/wmqv7/v7r0/index.jsp?topic=/com.ibm.mq.csqzaw.doc/jm35150_.htm

WebSphere® MQ classes for JMS contains a set of extensions to the JMS API called the IBM® JMS extensions. An application can use these extensions to create connection factories and destinations dynamically at run time, and to set the properties of WebSphere MQ classes for JMS objects.

[Emphasis mine.]


I would bet they mean some properties, not all.

As an example, there is no method in the MQ JMS API to get the depth of a queue but it's a simple property of the queue in the base API (getCurrentDepth()).


Yes, they do mean "some" properties and not all. And an important word in that above quote is "contains".

Presumably, however, given the diligence apparent in RGFanta's comments (Said diligence being notable for it's presence rather than the usual absence of said shown), there is already a PMR in place to actually get real answers, rather than trust the word of random strangers on the internet.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Goto page Previous  1, 2 Page 2 of 2

MQSeries.net Forum Index » IBM MQ Java / JMS » Setting MQ exclusive-read client properties in JMS
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.