| Author |
Message
|
| dbjohnsson |
 Posted: Thu Aug 12, 2010 12:28 am Post subject: Message Broker vs DataStage. What are the limitations of WMB |
 |
|
Newbie
Joined: 19 Apr 2007
Posts: 8
|
Hi,
I need to know what the limitations are of Message Broker? When would a company decide to use DataStage over Message Broker? I know that DataStage is generally used for ETL but when would Message Broker not be able to do what DataStage can do?
Are there any limitations in Message Broker when querying large datasets?
Can anyone point me to IBM site or documentation that gives guidelines around these questions?
Cheers
Dave |
|
| Back to top |
|
 |
| Vitor |
| Posted: Thu Aug 12, 2010 4:21 am Post subject: Re: Message Broker vs DataStage. What are the limitations of |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| dbjohnsson wrote: |
| I need to know what the limitations are of Message Broker? |
None. Well the skill of the people writing transformations.... 
| dbjohnsson wrote: |
| When would a company decide to use DataStage over Message Broker? |
When they have specific message and transformation requirements that fit with DataStage.
| dbjohnsson wrote: |
| I know that DataStage is generally used for ETL but when would Message Broker not be able to do what DataStage can do? |
I would disagree that's what you'd use DataStage for. I would also assert that you'd use Message Broker when DataStage couldn't meet the requirement.
| dbjohnsson wrote: |
| Are there any limitations in Message Broker when querying large datasets? |
The resources on your system. There's a section in the WMB product documentation giving best practice for large message handling (parsing tips to save memory, that sort of thing).
| dbjohnsson wrote: |
| Can anyone point me to IBM site or documentation that gives guidelines around these questions? |
I'd have started with the product whitepapers (from the IBM site) myself. There may also be some redbooks you'd find useful. There also used to be some benchmarks but those might have been for WMQ rather than WMB. Been a while.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Thu Aug 12, 2010 5:59 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
Hi DB -
I've worked with both products, and even commented on a recent post related to the same question.
To consider DataStage as an Apple and WMB as an Apple, so as to make comparison between the two products Apples to Apples, is an incorrect assumption.
All of DataStage functions are folded into WMB through the WTX node. Buy WMB and WTX and you have DataStage.
The value DataStage/WTX brings to the table is the ability to have a ready-made library of translation artifacts available so the development effort is easier. Notice I did not say faster.
WMB/WTX is where the future is. The legacy DataStage product is where the past is. If you want your effort this year to be around 20 years from now, WMB/WTX is the right choice IMHO. The customer base of the legacy DataStage product will keep it around also. But do you want to write SOA code or not? Do you understand the value of SOA? Please tell the difference between an ESB and a product that ETLs data.
Tell me the value of an ESB. That is why you want WMB/WTX. What does an ESB give you that a point-solution does not?
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| Gaya3 |
| Posted: Thu Aug 12, 2010 6:03 am Post subject: |
|
|
Jedi
Joined: 12 Sep 2006
Posts: 2493
Location: Boston, US
|
i will take this in to a different way, in my own words 
ESB i call it as a Business Integrator Tool [say MB ]
ETL i call it as a Data Integrator Tool [say Informatica/DataStage etc]
A business integrator [Super Set] can have a Data Integrator inside [Sub Set]
_________________
Regards
Gayathri
-----------------------------------------------
Do Something Before you Die |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Thu Aug 12, 2010 6:10 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
Well stated Gaya. Exactly the point I was trying to develop Dave's thought on.
WMB is a superset of all Dave wants to do. While the legacy DataStage product might work in the initial project Dave has in mind, the value proposition of implementing in the Message Broker provides 1+1=3 value, you get a bonus value point through the use of WMB.
Event-driven SOA http://en.wikipedia.org/wiki/Event-driven_SOA boosts that value even further. It's a different way of thinking of things.
Remember the olden days when we were experts and very proficient writing RPG code or C code inthe 1960s? Then object-oriented concepts came along, and we had to re-learn all of our strengths all over again so we could use the new way. The same metamorphic change is taking place by using an ESB in an Event-driven architecture.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| Gaya3 |
| Posted: Thu Aug 12, 2010 6:15 am Post subject: |
|
|
Jedi
Joined: 12 Sep 2006
Posts: 2493
Location: Boston, US
|
I really agree with this, this is going to be a break through in design patterns. 
Complex Event Processing is a vast topic, WMB 7 has come up with more additional features on this, like where you can filter the events on some states, or levels. This helps to get the proper information and post it as a pie chart for example.
_________________
Regards
Gayathri
-----------------------------------------------
Do Something Before you Die |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Aug 12, 2010 6:37 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| Gaya3 wrote: |
I really agree with this, this is going to be a break through in design patterns.
|
Another one? I remember when data warehousing was going to be a breakthrough in design patterns.
Which is not to say data warehousing isn't important, or that SOA 2.0 won't be useful and important. Simply that a number of sites still struggle with data warehousing and SOA 1.0, and each new technology always causes a smaller initial impact than people think.
Which still leaves all the points in this post valid.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Thu Aug 12, 2010 7:00 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
@Vitor
Very keen observation, sir. The strugglers, or shall we say, stragglers are numerous. It all depends on what you invest into something.
If corporations under-fund any intiative, it will struggle and flounder, not keep up with advancements in thought leadership or product patches for that matter.
Patterns are not the silver bullet, but they help map out a commonly accepted way of implementation for those who are clueless or don't invest the thinking time to come up with a solution on their own.
As we see most days in this and other forums, people want their work done for them rather than take a leadership position and blaze a trail in the Wild Wild West!
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Aug 12, 2010 7:18 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| lancelotlinc wrote: |
| people want their work done for them rather than take a leadership position and blaze a trail in the Wild Wild West! |
A more charitable view is that a number of people want others to blaze a trail & see exactly how hot it is.
I think Gartner did a normal distribution chart sometime that showed the %age of early adopters against % laggers? The think I found interesting was the very small %age of Fortune 500 in the early adopters.
I refrain from any comments regarding letting others spend money finding the problems as we're already a long way off topic & should start heading back to the trail.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| inMo |
| Posted: Thu Aug 12, 2010 7:28 am Post subject: |
|
|
Master
Joined: 27 Jun 2009
Posts: 216
Location: NY
|
| Just to be nit-picky, shouldn't we differentiate between WTX & Datastage. Datastage is an ETL with roots in Ascential, and WTX has it's original roots in Mercator. To my knowledge, the WTX node does not provide access to Datastage. Let me know if I am missing something. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Thu Aug 12, 2010 7:28 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
Here is a telling press release that reinforces the concepts presented above:
http://it.toolbox.com/wiki/index.php/WebSphere_DataStage_TX
| Quote: |
| Important Note: IBM is merging this offering into a combined edition with installations for deployment with WebSphere Process Server, WebSphere ESB, or WebSphere Message Broker. This offering is called WebSphere Transformation Extender for Integration Servers v8.2. |
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Thu Aug 12, 2010 7:31 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
@inMo
The WTX node's logic is produced by WebSphere Transformation Extender Design Studio, which is based on the DataStage/Mercator product.
WTX node provides the same functionality as the DataStage legacy product. It does not connect to the legacy product, but provides the same functions.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| inMo |
| Posted: Thu Aug 12, 2010 7:42 am Post subject: |
|
|
Master
Joined: 27 Jun 2009
Posts: 216
Location: NY
|
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Thu Aug 12, 2010 7:54 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
inMo, You are correct. These are two product offerings.
The original poster (OP) wanted to compare DataStage with WMB, making an (incorrect) assertion that WMB had limitations regarding ETL when compared to DataStage.
The InfoSphere DataStage product focus is on bulk data loads, aligning functionality to DBAs.
The WebSphere TX product focus is on record-by-record transformation on-the-fly as data streams in from (usually) third-party business partners.
So one product deals with bulk data sitting someplace that you want to mold and shape into a different normalization and place it some other place using the skills of a DBA. The other product deals with singular data (ie. not bulk) and is like the catcher in baseball, catching the data, molding and shaping it to your normalized data model, and placing it into your enterprise using the skills of a Middleware Analyst. It is possible that Middleware people can work on the first and DBAs can work on the second. It's just that the aim of the product's core market are those skillsets.
Does this help? Please ask more questions.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
|
|
