| Author |
Message
|
| smdavies99 |
 Posted: Thu Apr 17, 2014 6:09 am Post subject: WMQ/IIB comparison with MS- Biztalk |
 |
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
Does anyone know of a reference where there is a comparison of WMQ Clustering/Load Balancing vs Biztalk clusters.
I'm dealing with a load of time dependent messages. i.e. it is imperative that message A is processed and delivered to the target system before message B.
The subjects of each message could be the same but one could be for a create order and the second for a cancellation of that order. If the cancellation was processed before the create order then the end system will get itself into a mess.
If someone could explain how Biztalk might handle this I would be grateful.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
 |
| zpat |
| Posted: Thu Apr 17, 2014 7:09 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
The thing to remember is that Biztalk is polling based.
When a MQ message arrives on a queue, it will sit there until Biztalk polls the queue. It will then move it to the in message box database.
This database is polled occasionally and it will be picked up and processed by the pipeline or orchestration etc and placed in the out message box.
This will be polled after a while and the message put on a queue (assuming MQ in, MQ out for simplicity).
The net result is LATENCY. Biztalk is for leisurely workflow, not real-time transactional processing like the event driven WMB.
If you try to tune down the database polling intervals - you will hammer the database - there is no escaping this.
As for sequence order preservation, I guess if you have only one Biztalk pipeline it might preserve the order - I would ask my local Biztalk person but he's on vacation.
If you're not in a hurry it might be OK. Personally I think it's pretty slow and of course Windows is PITA operationally with patching, reboots etc.
Incidentally the default values set in Biztalk for MQ ports are not good - here is some free advice from me
| Quote: |
Biztalk ports have a MQ queue "polling" time value and a "wait" time value. The default values are far from ideal and cause excessive, repeated, connections to MQ.
Poll time = is the interval that Biztalk itself waits between making each MQ connection (MQCONN, MQOPEN, MQGET). (default 3 seconds)
Wait time = is the MQGET wait time value (MQGMO), in other words how long MQ is allowed to wait for a message (if the queue is empty), (default 0 seconds).
If a message is already on the queue, or arrives on the queue, MQ will immediately return it and end the MQGET. The MQGET wait time is only a maximum time that MQ can keep the MQGET request open before returning control. In many applications this can wait time can actually be indefinite, or a very high value without problems.
So the Biztalk poll time is less important to increase than the MQGET wait time (as the default is zero). The max MQGET wait biztalk allows is 60 seconds.
All Biztalk MQ ports should therefore have their "wait" time set to 60 seconds, this will not cause any increase in the time delay when receiving messages. It just allows MQ to work more efficiently, and in fact I believe this option was added to Biztalk, after I discussed this with Microsoft development some years ago.
So, to ensure Biztalk does not needlessly connect and disconnect to MQ every three seconds for each queue, please advise Biztalk developers to set 60 seconds for the MQGET wait time. I would suggest 15 seconds for the poll time. This means that they will get any available message within 15 seconds maximum.
|
_________________
Well, I don't think there is any question about it. It can only be attributable to human error. This sort of thing has cropped up before, and it has always been due to human error. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Apr 17, 2014 7:21 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
actually, it's probably a good idea to increase the poll time, as well, at least if the biztalk server is making a client connection.
channel instances have a habit of sticking around for a while waiting to be reused. if you mqconn too quickly, this might not happen correctly.
biztalk presumably has some notion of deferring a given event based on timestamp or etc.
For order-dependent messages, it's best to make sure that the messages themselves include a sequence number, and take steps to process messages in sequence. With GlobalCache, one can easily share a 'current sequence number' across lots of instances, and use ESQL sleep or etc to delay processing a given message until it's sequence number is ready to be processed.
Of course, one could also use resequence nodes to ensure that they come out 'in order'.
It's generally a harder problem than it seems, but if you look for info on Lamport timestamps, you can find a lovely rabbit hole full of deep computer science and some nice basic algorithms. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Thu Apr 17, 2014 8:40 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
Thanks for the hints Zpat & mqjeff.
My data flows are whiles time dependent totally separate units of work. Whilst I could add a sequence number not every input message will result in a correcposning output message due to the filtering in the flows. The output are sent to multiple destinations via Pub/Sub. For this reason sequence number in the message won't really work.
Currently we use Active-passive MSCS clustering. The competition is saying that this is a weakness and that their clustered Biztalk solution is the answer to 'life, the universe and everything' and that because we don't do load balancing our solution is deficient. The customer is currently agreeing with them on this matter hence the need to compete against BizTalk.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Apr 17, 2014 9:56 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
One thing to take into consideration there is the real throughput numbers.
I doubt that biztalk will favorably show against IIB.
On the other hand you can load balance using multiple IIBs on top of a clustered MQ...
As well you can use Multi-instance instead of MSCS and reduce costs that way... 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Apr 17, 2014 10:59 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Yeah, if the only claim is that biztalk runs in an active-active cluster, that's easy to replicate with IIB.
And the performance *will* not be better with Biztalk.
I'm waiting for some wayward ohioans to pop in with real facts. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Thu Apr 17, 2014 11:36 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| fjb_saper wrote: |
As well you can use Multi-instance instead of MSCS and reduce costs that way... |
I don't quite follow how this will reduce the costs? We will need two full broker licenses.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Apr 17, 2014 11:58 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Typical cost of a multi-instance license is x0% of the full license cost...
Of course only your IBM rep can specify what that % is...
This is in an active / passive scenario...
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Apr 17, 2014 12:37 pm Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
the only way switching from MSCS cluster to multi-instance saves any license is it removes the cost of MSCS itself.
It doesn't provide active-active, in any way. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Apr 17, 2014 8:37 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| mqjeff wrote: |
the only way switching from MSCS cluster to multi-instance saves any license is it removes the cost of MSCS itself.
It doesn't provide active-active, in any way. |
Sorry but AFAIK biztalk does not provide active - active either.
What it provides is multiple instances sharing the same config / DB? i.e. some kind of load balancing?
But then what do I know? I'm no biztalk expert...
_________________
MQ & Broker admin |
|
| Back to top |
|
|
|
|
