|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
Troubles Implementing the Service Gateway Pattern in WMB 6.1 |
« View previous topic :: View next topic » |
Author |
Message
|
ccrandall |
Posted: Mon Jun 29, 2009 1:41 pm Post subject: Troubles Implementing the Service Gateway Pattern in WMB 6.1 |
|
|
Acolyte
Joined: 23 Oct 2008 Posts: 52
|
I'm trying to modify our Message Broker framework so we can use it to front legacy services with a SOAP-based web service. In the beginning, we created 1 Message Set/Message Definition for each WSDL and a corresponding SOAP Input Node. We quickly realized that in a short amount of time, this wouldn't scale very well.
Recently, we've added DataPower to our set up and we are using it primarily for providing security. However, it's also providing us with validation of the SOAP requests. Since our MB framework no longer needs to validate the request, and each request is processed the same way, it appeared that there'd be an easier way to handle N number of web services.
We worked for a few days with a consultant who recommended we try using the Service Gateway pattern; basically use a generic WSDL whose only type is a message of anyType. Below is the WSDL I've created based off our discussions:
<?xml version="1.0" encoding="UTF-8"?>
<wsdl:definitions name="ServiceGateway" xmlns:wsi="http://ws-i.org/profiles/basic/1.1/xsd" xmlns:wsdlsoap="http://schemas.xmlsoap.org/wsdl/soap/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:tns="http://com.abc..broker.service.ServiceGatewayOperation" targetNamespace="http://com.abc.broker.service.ServiceGatewayOperation" xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/">
<wsdl:types>
<xsd:schema xmlns="http://www.w3.org/2001/XMLSchema" targetNamespace="http://com.abc.broker.service.ServiceGatewayOperation">
<xsd:element name="message" type="xsd:anyType" />
</xsd:schema>
</wsdl:types>
<wsdl:message name="gatewayMessage">
<wsdl:part name="message" element="tns:message" />
</wsdl:message>
<wsdl:portType name="ServiceGateway">
<wsdl:operation name="ServiceGatewayOperation">
<wsdl:input name="gatewayRequest" message="tns:gatewayMessage" />
<wsdl:output name="gatewayResponse" message="tns:gatewayMessage" />
<wsdl:fault name="fault" message="tns:gatewayMessage" />
</wsdl:operation>
</wsdl:portType>
<wsdl:binding name="ServiceGatewayBinding" type="tns:ServiceGateway">
<wsdlsoap:binding style="document" transport="http://schemas.xmlsoap.org/soap/http" />
<wsdl:operation name="ServiceGatewayOperation">
<wsdlsoap:operation soapAction="" />
<wsdl:input name="gatewayRequest">
<wsdlsoap:body use="literal" />
</wsdl:input>
<wsdl:output name="gatewayResponse">
<wsdlsoap:body use="literal" />
</wsdl:output>
<wsdl:fault name="fault">
<wsdlsoap:fault name="fault" use="literal" />
</wsdl:fault>
</wsdl:operation>
</wsdl:binding>
<wsdl:service name="ServiceGatewayService">
<wsdl:port name="ServiceGateway" binding="tns:ServiceGatewayBinding">
<wsdlsoap:address location="http://localhost:7800/service/ServiceGateway" />
</wsdl:port>
</wsdl:service>
</wsdl:definitions>
Using this WSDL, I created a Message Set/Message Definition and a corresponding SOAP Input Node and wired that node to our request flow.
I then created another WSDL that represents one of the services we want to support and then used SOAPUI to send SOAP requests to our MB frameowrk. Below is a sample request:
Host: localhost:7800
Content-Length: 525
SOAPAction: "ServiceGatewayOperation"
User-Agent: Jakarta Commons-HttpClient/3.0.1
Content-Type: text/xml;charset=UTF-8
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:m="http://com.abc.broker.service.ServiceGatewayOperation">
<soapenv:Header/>
<soapenv:Body>
<m:ServiceGatewayOperation>
<request>
<KeyType>JobFunctionCde</KeyType>
<KeyValue>5</KeyValue>
<JobFuncStatusInd>A</JobFuncStatusInd>
</request>
</m:ServiceGatewayOperation>
</soapenv:Body>
</soapenv:Envelope>
(I removed some proprietary info, so the content length is incorrect.)
When sending this message, I get the following error in the MB framework:
ParserException BIP3602E: The Web service payload ''{http://com.abc.broker.service.ServiceGatewayOperation}ServiceGatewayOp
eration'' does not match an operation described by WSDL binding ''ServiceGatewayBinding'' in file ''/var/mqsi/components/APS61BA3/27400fc2-1b01-0000-0080-f7b
fd499757a/config/XSD/ServiceGatewayMessageSet/servicegatewayoperation/service/broker/abc/com/ServiceGateway.wsdl''.
I also tried editing the Service Gateway WSDL so that the soapAction="ServiceGatewayOperation", but I get the same error.
I'm relatively new to web services and I've spent a lot of time the last several days trying to read up SOAP/WSDL definitions, etc. to make sure I'm crafting my WSDL correctly. I'm still not seeing where my error is, though.
Furthermore, we really need our MB framework to accept any SOAPAction/Operation w/o it being hardcoded as "ServiceGatewayOperation". This is a requirement to support our Java-based web service clients... they need to use unique method names to call the operation and apparently cannot overload using the same operation name (not clear on the specifics here, it's another set of folks working on that stuff).
Is what I'm trying to do possible? If so, if someone could point me in the right direction I would be very appreciative.
Thank you!
Curt |
|
Back to top |
|
 |
marnick |
Posted: Wed Oct 14, 2009 11:20 am Post subject: |
|
|
Novice
Joined: 26 Oct 2006 Posts: 10
|
Hi CCrandall:
I´m facing the same requirement, and I´m wondering if you could finally make it work?
Many thanks in advance
MartÃn |
|
Back to top |
|
 |
ccrandall |
Posted: Fri Feb 26, 2010 2:47 pm Post subject: |
|
|
Acolyte
Joined: 23 Oct 2008 Posts: 52
|
@marnick
Sorry, this is still an outstanding issue for me.
We do have DataPower fronting our broker. DP provides validation and security for us. So, we've considered just using a plain ol' HTTP Input node instead of a SOAP node and that let's us avoid having a ton of Message Sets and SOAP Input nodes. |
|
Back to top |
|
 |
goffinf |
Posted: Sat Oct 09, 2010 12:10 pm Post subject: |
|
|
Chevalier
Joined: 05 Nov 2005 Posts: 401
|
Don't think that just using xs:anyType is going to be much use as a specification for arbitrary messages with any content model. I think you really meant the xs:any wildcard construct probably with ##any as the namespace and skip or lax for processContents. Even with this approach you will need a containing element for the xs:any since your message must be defined by at least one global element or type.
Fraser. |
|
Back to top |
|
 |
lancelotlinc |
Posted: Mon Oct 11, 2010 9:30 am Post subject: |
|
|
 Jedi Knight
Joined: 22 Mar 2010 Posts: 4941 Location: Bloomington, IL USA
|
Being creative here, and thinking out of the box, one could front-end the SOAP requests with an AMP stack which would then send an MQ message to Broker.
Separation of concerns could be well managed with this architecture and would scale very well under strain. _________________ http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
Back to top |
|
 |
fjb_saper |
Posted: Mon Oct 11, 2010 9:51 am Post subject: |
|
|
 Grand High Poobah
Joined: 18 Nov 2003 Posts: 20756 Location: LI,NY
|
lancelotlinc wrote: |
Being creative here, and thinking out of the box, one could front-end the SOAP requests with an AMP stack which would then send an MQ message to Broker.
Separation of concerns could be well managed with this architecture and would scale very well under strain. |
Seems I have heard that the SOAP connection from PHP is not yet enterprise ready... and there is still some trouble installing the MQ JMS PHP modules correctly... our developers did not want to trouble themselves with mq base.
Yes the AMP or LAMP stack is nice, but not necessarily suited for all purposes...  _________________ MQ & Broker admin |
|
Back to top |
|
 |
mqjeff |
Posted: Mon Oct 11, 2010 12:44 pm Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008 Posts: 17447
|
SOAP defines a tight coupling between two endpoints. (but to be clear, it defines a tight coupling to an INTERFACE, not to an IMPLEMENTATION).
If you have moved one of those two endpoints to a different implementation, you have then essentially introduced a second pair of endpoints, and can put whatever interface you want between them.
This is all to say there's no reason not to use HTTPInput and even the BLOB parser at the broker level, if you're making no decisions or doing no augmentation or transformation or etc. of the message based on the actual SOAP interface defined in a specific WSDL.
If you do need at least some of the SOAP message structure, without binding tightly to a specific WSDL, you can of course use XMLNSC instead of BLOB.
It's all well and good to think outside the box, but there's also no reason to add moving parts that require additional skill sets and additional maintenance and change control. (And, no, Broker is not a hammer. It's a toolbox.) |
|
Back to top |
|
 |
lancelotlinc |
Posted: Tue Oct 12, 2010 5:12 am Post subject: |
|
|
 Jedi Knight
Joined: 22 Mar 2010 Posts: 4941 Location: Bloomington, IL USA
|
mqjeff wrote: |
It's all well and good to think outside the box, but there's also no reason to add moving parts that require additional skill sets and additional maintenance and change control. |
I believe the Original Poster stated this was the reason:
Quote: |
We quickly realized that in a short amount of time, this wouldn't scale very well.
|
_________________ http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
Back to top |
|
 |
mqjeff |
Posted: Tue Oct 12, 2010 5:28 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008 Posts: 17447
|
lancelotlinc wrote: |
mqjeff wrote: |
It's all well and good to think outside the box, but there's also no reason to add moving parts that require additional skill sets and additional maintenance and change control. |
I believe the Original Poster stated this was the reason:
Quote: |
We quickly realized that in a short amount of time, this wouldn't scale very well.
|
|
Yes, that's a reason to use a different approach within Broker.
It is also based on a misunderstanding of WebServices generally. There's nothing inherently different how a deployment of services scales in Broker than in WAS or etc. Every service needs a provider, that provider needs to be specific to the service and needs a copy of the WSDL. |
|
Back to top |
|
 |
lancelotlinc |
Posted: Tue Oct 12, 2010 5:32 am Post subject: |
|
|
 Jedi Knight
Joined: 22 Mar 2010 Posts: 4941 Location: Bloomington, IL USA
|
@mqjeff
You have excellent points sir. I wonder what the solution finally was for the OP?
The OP originally posted on June 29, 2009. As of Feb 26, 2010, the OP had not yet found a solution.
I wonder if the OP could update us on what the final solution was. _________________ http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
Back to top |
|
 |
ccrandall |
Posted: Tue Oct 12, 2010 8:03 am Post subject: |
|
|
Acolyte
Joined: 23 Oct 2008 Posts: 52
|
I'm surprised to see this thread resurrected. But that's a good thing since it's still something we never really solved.
Currently, we are still using 1 Message Set + 1 SOAP Node per service. This is fine for our needs, but once we start having to support 50, 60, 70, ... etc. of these services it will not scale.
This is something we've brought up with IBM and apparently other customers have too. The web services gateway is the right pattern to use, unfortunately, it's just not something we've been able to get right.
We did consider using a regular HTTP Input node, however, I ran into some correlation issues. Our use case is to give legacy MQ services a SOAP facade. With SOAP Input nodes, we were able to use the SOAP Reply ID, with some additional logic, to create a MQ correlation ID. However, with HTTP Input nodes, I only have a Request ID and using that value with similar logic that we used with the SOAP Reply ID gave us errors.
So, hoping that the web services gateway would be supported in the near future in broker, we just went back to SOAP Input nodes. Since the only project on our radar requires the support of about 7-10 services, this is adequate for now. I'm more focused on moving our broker apps and framework from MB 6 to 7 at this time. After that's completed, we may relook at the solution using HTTP nodes. |
|
Back to top |
|
 |
lancelotlinc |
Posted: Tue Oct 12, 2010 9:16 am Post subject: |
|
|
 Jedi Knight
Joined: 22 Mar 2010 Posts: 4941 Location: Bloomington, IL USA
|
With all respect to my colleagues mqmatt, mqjeff, vitor, and others, and with great admiration for the IBM WebSphere Message Broker product, it has been my experience at many client sites in many diverse industries that containing a full solution inside the WMB is wonderful for maintainability in a low TPS environment, low variance patterns.
If either of these two attributes are higher than the average bear, then a successful implementation must resort to using more moving parts, usually outside of the ESB box.
Low TPS: Insurance industry application peaking around 5 to 10 TPS.
High TPS: Banking application peaking around 1,200 TPS, with 800 sustained.
Low variance: Insurance industry application with three or four different patterns.
High variance: banking application with more than a dozen patterns.
For source code maintenance and simplicity, I agree with mqjeff's comments if both Low TPS and Low variance attributes hold true.
I like ccrandall's suggestion that the SOAP requests could be front-ended with something else, like Web Services Gateway. One could build a Web Services Gateway using Apache Tomcat or similar. By the way, mod_jk version 1.2.30 was recently released this year.
If someone would like to disagree, I welcome the discussion. If the disagreeable party is in the software development team of the product, I may say that not enough field experience with real life customer challenges would have been considered. Its nice when the product works as designed for all purposes. But the reality is, real life challenges and customizations are necessary, and sometimes these customizations cannot be fully contained in the WMB ESB implementation. _________________ http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
Back to top |
|
 |
vmcgloin |
Posted: Tue Oct 12, 2010 2:18 pm Post subject: |
|
|
Knight
Joined: 04 Apr 2002 Posts: 560 Location: Scotland
|
Hi,
I am not qualified to comment on the previous post, but I am curious about your correlation problems with the Http Input node.
I use the HTTP.RequestIdentifier in a similar scenario. When you say that you perform some additional logic on the SOAP ReplyIdentifier (that does not work in the Http case) I get the impression that you are making things more complicated than they need to be. What 'additional logic' and how do you use it for correlation?
You should be able to store and reinstate the value (in a header/utility field or if necessary out to a queue). to get this working with an Http node.
Vicky
Edited to add: Actually, I do disagree with the previous post - I have and do use broker in high TPS, high variance conditions (by your definitions) and think that broker is more than capable of being used as a web services gateway itself... I am just waiting for someone with better skills to reason with you  |
|
Back to top |
|
 |
fjb_saper |
Posted: Tue Oct 12, 2010 7:36 pm Post subject: |
|
|
 Grand High Poobah
Joined: 18 Nov 2003 Posts: 20756 Location: LI,NY
|
Web Services Gateway... and web services provider?
I'd like to separate that into 2 different execution groups.
The Web Services Gateway specifically should scale completely differently from the Services provider...
Also worth thinking about... best performance is achieved if the request and the response are in 2 different flows... So the Web Services gateway would only have a routing and maybe a protocol translation function...
Have fun  _________________ MQ & Broker admin |
|
Back to top |
|
 |
ccrandall |
Posted: Wed Oct 13, 2010 6:44 am Post subject: |
|
|
Acolyte
Joined: 23 Oct 2008 Posts: 52
|
Yes, we do separate our request and reply into 2 separate flows. In our case, we are just providing a SOAP facade for existing MQ services, and the SOAP payload is identical to the payload we get when using MQ.
So, we just read it in with the SOAP Input node, grab the payload, and the rest of the logic is the same as if the message was MQ to begin with. On the reply, then we just go through the extra step of taking the MQ reply and changing it to SOAP.
Since DataPower does all our validation and security, all our broker app really needs to do is make sure the SOAP message is well-formed.
To get our MQ correlation ID from the SOAP Reply ID, we don't do a whole lot of additional logic. We basically take that SOAP Reply ID, determine it's length, which is 4 bytes, and then pad it with spaces to get a length of 24 (i.e. X'2020202020202020202020202020202020202020fe4567ab).
I tried the same thing with the HTTP Request ID, but I received errors. Since this exercise was timeboxed to just 2 days, I didn't do a lot of debugging and unfortunately I do not have the exact error anymore to share.
If the WSDL-less SOAP Input node isn't added in the near future to MB, then we'll probably revisit this issue next year before our teams that support hundreds of service decide they want this SOAP facade. |
|
Back to top |
|
 |
|
|
 |
Goto page 1, 2 Next |
Page 1 of 2 |
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
|
|
|