| Author |
Message
|
| Toko60 |
 Posted: Tue Mar 24, 2009 6:34 am Post subject: MQ6 Virtual cluster / fall back |
 |
|
Newbie
Joined: 24 Mar 2009
Posts: 8
|
I am trying to create a process that will send every message (xml) coming into a queue
to a virtual queue on the Internet ( OnlineMQ ).
It will act as fallback in case My internal LAN is not accessible or when my MQ is down or upgraded.
At the moment, i am using a C program to get the message out of the queue and then send it to OnlineMQ via WebService (Soap via Wsdl).
Is there an MQ V6 feature to forward a every incoming message via webservice? |
|
| Back to top |
|
 |
| sebastianhirt |
| Posted: Tue Mar 24, 2009 6:43 am Post subject: Re: MQ6 Virtual cluster / fall back |
|
|
Yatiri
Joined: 07 Jun 2004
Posts: 620
Location: Germany
|
| Toko60 wrote: |
| Is there an MQ V6 feature to forward a every incoming message via webservice? |
AFAIK, No. Writing your own exit might do the trick. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Mar 24, 2009 6:55 am Post subject: Re: MQ6 Virtual cluster / fall back |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| Toko60 wrote: |
It will act as fallback in case My internal LAN is not accessible or when my MQ is down or upgraded.
|
Why are you not using WMQ's inbuilt abilities to deal with these situation?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Tue Mar 24, 2009 7:00 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
I'm curious.
Will this pass an audit? How will you ensure that your data is not intercepted, altered, deleted? What will be your backup of messages sent to OnlineMQ should it fail?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| Toko60 |
| Posted: Tue Mar 24, 2009 8:07 am Post subject: |
|
|
Newbie
Joined: 24 Mar 2009
Posts: 8
|
Answer to Vitor:
Well, we are using a cluster for MQ, no problem there, the system is fully redundant.
Now, Our customer are connected to us via the Internet, the next points convinced us to check for the mentioned solution.
1 - Our datacenter had several malfunction lately that prevented customers connect to our MQ (one time for almost 18 hours).
2 - We needed to have an MQ upgrade and customers complained about loosing messages (although they were informed not to send at this time), not ewe have about 400 customers connected.
As a result we decided to have fall back Queue that will be located on the internet in order to prevent this kind of malfunction to reoccur.
So now once a customer is not able to connect to MQ, it will try to read the message from OnlineMQ.com
Answer to bruce2359
Well, you have a point, this is still issue we are figuring out, i assume using transaction will do the work.
So far we are testing the solution and i am sure more questions will arise as a result. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Tue Mar 24, 2009 8:21 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| Quote: |
| 1 - Our datacenter had several malfunction lately that prevented customers connect to our MQ (one time for almost 18 hours). |
If your datacenter is out of service, it will be difficult/impossible for your messages to arrive and be rerouted elsewhere. Having the messages routed to a virtual queue for 18 hours doesn't seem to help much, does it? Perhaps a second datacenter site is now apporpriate.
| Quote: |
| 2 - We needed to have an MQ upgrade and customers complained about loosing messages (although they were informed not to send at this time), not ewe have about 400 customers connected. |
With MQ Clusters, you can take one qmgr at a time out of service for upgrade, and leave the other(s) to continue processing messages.
You are not the first IT shop to manage outages and upgrades while continuing normal operations.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| Toko60 |
| Posted: Tue Mar 24, 2009 8:34 am Post subject: |
|
|
Newbie
Joined: 24 Mar 2009
Posts: 8
|
Answer:
Of course when datacenter is down forwarding is not possible, we are not going to use forwarding, the solution is on the client side.
We supply to clients a piece of software to send messages, this piece of software will know how to shift messages should MQ is not reachable.
As for the cluster upgrade, well we know the concept....but human error created malfunctions that resulted in outage.
And, we are a German national phone company, not a shop  |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Mar 24, 2009 8:35 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| Toko60 wrote: |
| 1 - Our datacenter had several malfunction lately that prevented customers connect to our MQ (one time for almost 18 hours). |
If your customers can't connect to WMQ, how are you going to redirect these messages using any WMQ function?
| Toko60 wrote: |
| 2 - We needed to have an MQ upgrade and customers complained about loosing messages (although they were informed not to send at this time), not ewe have about 400 customers connected. |
That's just poor upgrade planning and/or configuration. Every WMQ customer upgrades at some point or another, doing it without losing messages is straightforward.
Following from the comments about auditing & security, what will you do if one of your 400 customers doesn't like confidential data being bandied round the Internet? Or a regulator?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Tue Mar 24, 2009 8:56 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| Quote: |
| we are a German national phone company, not a shop |
I intended no insult. IT shop is an idiomatic expression meaning IT organization or IT department. I appologize.
IBM WMQ offers a client-side solution, namely: a client channel table that tries to connect to each of the queue managers (and ipaddreses) named in the table until one attempt succeeds.
Thus, a single qmgr, or several qmgrs, or an entire site can be out of service; and (if you have a second site) your applications can continue.
One of my clients (customers) is a banking funds transfer organization, and has a second site for this exact reason.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| Toko60 |
| Posted: Wed Mar 25, 2009 1:40 am Post subject: |
|
|
Newbie
Joined: 24 Mar 2009
Posts: 8
|
Thank you all for your comments, although my question was purely technical, i like your intelligent questions and the fact that you debate what we choose as technological architecture.
I will try to elaborate and present our thinking.
Security
=======
Most of our customers are WSO (whole sales), we did a survey and many of them already use services on the Internet (mainly CRM of salesforce) and some cloud computing application over Amazon & GoGrid. Therefor security of messages was not a real problem for them. We plan to use the SSL option of the vendor for sending messages. I sensed that this extra measure for security was good for them.
In addition, you can set IP addresses in the onlinemq setting that will give only specific authorized IP to connect (list of 400 IP's). That was sufficient for our customers.
Never the less, when you have your MQ exposed to the Internet, it draw a lot of hackers. Just this months we had over 2000 hack attempt on our MQ. The mentioned service will solve it as our system will get messages only from one allowed IP (onlinemq).
Upgrade malfunction
======================
Indeed Vitor you are right, that was poor migration plan...and we lost some money as a result.
I know also that most of our problems result from human error. Even the great Google had a serious problem with human error. we must improve, but a fall back is essential in any case.
Now, Not Everything is technology !!!
==============================
- Lets assume i want to convince my big organization to choose for a technical solution, lets assume that the cycle of decision is short although it never like that, ususaly monthsssss.
- Lets say that i use the suggestion to build another MQ in another location.
The first question a manager will ask :"what do you need, and how much it costs ?"
The answer in my case is:
1. We need a NEW server and a location in a data center
2. We need NEW MQ Licenses
3. We need support contract
4. We need maintenance contract.
Can i compare it to 45$ a month ?
And bruce2359, i was not offended, but thank you for being sensetive |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Mar 25, 2009 1:53 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
There's more to the calculation. Yes, the cost of a second MQ server is more than $45 a month. But the risk associated with it may be significantly smaller than the risk associated with using onlinemq.
What is the value in your messages, such that there is that much hacking attempts? What is the business loss if one of those messages that you sent to onlinemq gets exposed? What is the business loss if *all* of those messages get exposed? What is the business loss if someone is able to insert new "malicious" messages into onlinemq? What is the business loss if some of those messages sent to onlinemq are lost without notification to you?
How much *more* or *less* likely are those things to happen with onlinemq than with another MQ server? |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Wed Mar 25, 2009 6:19 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
How risk-averse is your organization? Do they insure other assets? Did the last outage cost them more than $45? What will the next one cost? Does your organization have a DR/BC (disaster recovery/business continuation) plan? If not, why not?
This process is called cost-benefit analysis.
Most organizations undervalue their data, mistakenly believing that their physical property (cars, buildings, equipment) are the most valuable assets.
Generally speaking, if the next outage (and there will be one) is expected to cost your organization $100,000, then it is probably worthwhile to spend $100,000 to avoid the possibility of loss.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| Toko60 |
| Posted: Thu Mar 26, 2009 2:55 am Post subject: |
|
|
Newbie
Joined: 24 Mar 2009
Posts: 8
|
As a Integration team leader i have the privilege to work with MQ, Tibco WebLogic and many other application, some are SaaS (Software as a service).
I will open another post for discussing if SaaS MQ is a good solution.
Personally, I think you guys have good point in asking about reliability and security.
In our case this system is a BACKUP system only, so price was a factor.
Our TCO for a year (total cost of ownership) calculation was
For backup IBMMQ implementation 130000 Euro in appose to onlinemq backup system 10000 Euro.
This is why we choose for this solution. |
|
| Back to top |
|
|
| zpat |
| Posted: Thu Mar 26, 2009 3:06 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
| This would be an ideal application for Websphere Message Broker 6.1 using the http or soap nodes. |
|
| Back to top |
|
|
|
|
