| Author |
Message
|
| chris boehnke |
 Posted: Mon Jul 27, 2015 4:58 am Post subject: DB connectivity |
 |
|
Partisan
Joined: 25 Jul 2006
Posts: 369
|
Hi All,
I have a very basic question.
Why most of the enterprises restrict Message Broker connectivity to Databases?
Thanks |
|
| Back to top |
|
 |
| Vitor |
| Posted: Mon Jul 27, 2015 5:05 am Post subject: Re: DB connectivity |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| chris boehnke wrote: |
| Why most of the enterprises restrict Message Broker connectivity to Databases? |
Do most of the enterprises so restrict? 
Or restrict more than a more typical application?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| chris boehnke |
| Posted: Mon Jul 27, 2015 5:19 am Post subject: |
|
|
Partisan
Joined: 25 Jul 2006
Posts: 369
|
| Yes. They do restrict message broker directly connecting to database. |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Jul 27, 2015 5:21 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| chris boehnke wrote: |
| Yes. They do restrict message broker directly connecting to database. |
Ok. This is not my experience over the last 14 years & more sites than I like to think of, nor is it the policy at my current site, so I don't think I'm going to have much to add to this. 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Mon Jul 27, 2015 5:45 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
DB connectivity is part and parcel of 95% of all the Broker Systems I've worked on since the days of V2.0.1/2
Not all Message flows are solely transformation. on my current project 30% of flow recieve messages and from that do DB updates.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Jul 27, 2015 6:22 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| smdavies99 wrote: |
| DB connectivity is part and parcel of 95% of all the Broker Systems I've worked on since the days of V2.0.1/2 |
Another common use case is where the "transformation" of a flow includes enrichment of the message contents with data selected from 1-n tables which the application that originated the message can't or won't obtain.
If most sites are restricting DB access (as the OP asserts), IBM must be bummed about that specific drawer of nodes.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| joebuckeye |
| Posted: Mon Jul 27, 2015 8:39 am Post subject: |
|
|
Partisan
Joined: 24 Aug 2007
Posts: 364
Location: Columbus, OH
|
| Where I work broker access to databases is the preferred method. We don't allow database access from Datapower, for instance, so any solution where a Datapower hosted service (which is almost all of them) need DB access they call out to a message broker flow which handles it. |
|
| Back to top |
|
|
| chris boehnke |
| Posted: Mon Jul 27, 2015 6:23 pm Post subject: |
|
|
Partisan
Joined: 25 Jul 2006
Posts: 369
|
I worked with two clients and both of them do not allow message broker to connect to database.
Instead, We call java based web service for any database related activities. This webservice connects to database and provides the required data.
Thanks |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Mon Jul 27, 2015 10:55 pm Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| chris boehnke wrote: |
I worked with two clients and both of them do not allow message broker to connect to database.
Instead, We call java based web service for any database related activities. This webservice connects to database and provides the required data.
Thanks |
Well, that's one way to throttle your performance.
does this Java WebService also log everything using log4j by any chance?  
If I had to do work that way on my current systems then they just wouldn't work in a timely manner. Some of the enrichment we do is quite involved. One flow uses nine different selects.
Oh well, each to their own but I think your way of using Db's is in the minority, a very small minority.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Tue Jul 28, 2015 4:21 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20697
Location: LI,NY
|
| chris boehnke wrote: |
I worked with two clients and both of them do not allow message broker to connect to database.
Instead, We call java based web service for any database related activities. This webservice connects to database and provides the required data.
Thanks |
Quite the slower path, I would think. Having a WAS based MDB read the message and reply with the DB info could go faster... and may handle concurrency better than the flow...
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Jul 28, 2015 4:49 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| chris boehnke wrote: |
| I worked with two clients and both of them do not allow message broker to connect to database. |
With respect, I think calling 2 clients "most of the enterprises" a bit of a stretch. It's certainly not a statistically significant sample...
| chris boehnke wrote: |
| Instead, We call java based web service for any database related activities. This webservice connects to database and provides the required data. |
It's a valid design; from most of the comments on this thread the consensus seems to be it's uncommon. I do wonder at the design imperative at work here; it introduces an extra hop through the network and couples the broker to whatever runs these web services, both of which add possible points of failure. I would assume (but only assume) that this is someone's way of "fixing" the fact broker doesn't pool DB connections the way WAS does but I'd sooner live without a connection pool .
| smdavies99 wrote: |
| If I had to do work that way on my current systems then they just wouldn't work in a timely manner. Some of the enrichment we do is quite involved. One flow uses nine different selects. |
Likewise. It also adds a degree of complexity to the code that I'd prefer to keep out of my production system (some of the chimps allowed to deploy code to my beautiful brokers are a bit clumsy), over and above the risk of being on the sharp end of a Sev 1 ticket because "broker's failing with an HTTP 500 error". I hate having to explain, again, that broker is reporting that it received an HTTP 500 from a system it was trying to call and didn't throw it itself.
(I hate it because someone always makes the "but an HTTP 500 is an internal error; we see it from WAS all the time" comment and then I have to slap them)
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| joebuckeye |
| Posted: Tue Jul 28, 2015 8:12 am Post subject: |
|
|
Partisan
Joined: 24 Aug 2007
Posts: 364
Location: Columbus, OH
|
| Vitor wrote: |
Likewise. It also adds a degree of complexity to the code that I'd prefer to keep out of my production system (some of the chimps allowed to deploy code to my beautiful brokers are a bit clumsy), over and above the risk of being on the sharp end of a Sev 1 ticket because "broker's failing with an HTTP 500 error". I hate having to explain, again, that broker is reporting that it received an HTTP 500 from a system it was trying to call and didn't throw it itself.
(I hate it because someone always makes the "but an HTTP 500 is an internal error; we see it from WAS all the time" comment and then I have to slap them) |
Ah yes, our WMB/Datapower run team are very good at doing triage work on WMB/Datapower "issues" and getting them routed to the correct failure point. You would think over time these other areas would learn but everyone always blames the messenger for the error and not the originator of it. |
|
| Back to top |
|
|
| nelson |
| Posted: Wed Jul 29, 2015 5:23 am Post subject: |
|
|
Partisan
Joined: 02 Oct 2012
Posts: 313
|
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Jul 29, 2015 5:43 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| nelson wrote: |
| Definitively I'm not a WAS guru, but reading the WAS Connection Pool settings I see that broker also handles (internally) a similar ODBC/JDBC connection pooling settings... |
Nor am I, but people who are and people who are database gurus have claimed in my hearing that the WAS connection pooling is "better" because there's a specifically configurable connection pool in WAS where you can set number of connections, time to live and other stuff. As I think this thread has indicated, there's a broad groundswell of opinion that this is splendidly irrelevant and we continue to perform db operations from inside broker.
My comment was intended to convey that the OP appears to have experienced sites where this argument has carried the day, and the lack of a WAS-style connection pool is a problem that needs to be "fixed". In much the same way it's been suggested to me that it would be "better" if IIB exported it's Java operations and used WAS rather than the JVM attached to the EG.
I invented a whole new facial expression to respond to that.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Jul 29, 2015 6:05 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20697
Location: LI,NY
|
I believe the main argument in favor of "exporting" the DB connections to a J2EE environment is transaction handling and concurrency. Of course this should be done over WMQ.
If your flow is transactional it will hold the connection until done, but your DB operation might not need a transaction. Thus exporting the DB request may get you a better throughput with fewer connections used... and let you specify the max wait time (MQGET wait time) for the DB response...
_________________
MQ & Broker admin |
|
| Back to top |
|
|
|
|
