| Author |
Message
|
| RAN001 |
 Posted: Tue Aug 15, 2017 8:52 am Post subject: How do you do 24 X 7 X 365? |
 |
|
Novice
Joined: 14 Feb 2017
Posts: 11
|
I work for a large retailer, not Amazon, so maybe I work for a small retailer 
Anyway, we are trying to figure out how to setup MQ preferably on linux so that we can stay up 24 X 7 X 365, but still patch and upgrade everything, and not lose access to any messages. No matter what we read, Multi Instance, Clustering, etc we cannot see a way that we keep access to all the messages all the time.
What do you all use for these kinds of requirements?
How do you perform patching, of OS, MQ, IIB and never go down?
Any insight or suggestions would be greatly appreciated.
Thanks,
Andy |
|
| Back to top |
|
 |
| Vitor |
| Posted: Tue Aug 15, 2017 8:55 am Post subject: Re: How do you do 24 X 7 X 365? |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| RAN001 wrote: |
| What do you all use for these kinds of requirements? |
Red Hat Cluster Services.
(Other active/passive OS level solutions are available)
Or a network load balancer.
(This site uses both - don't ask......  )
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Aug 15, 2017 8:56 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
For clarification - we don't use them both for the same solution.
We use both to provide the kind of uptime you're looking for.
We could have picked one and just used that, but why would they make my life simple? 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| RAN001 |
| Posted: Tue Aug 15, 2017 9:20 am Post subject: |
|
|
Novice
Joined: 14 Feb 2017
Posts: 11
|
Hi Vitor,
Thanks for the quick response.
Can you go into more detail please about how you would apply an upgrade say from MQ 7.5 to 8 while not going down? These are the kind of things we are working around. Again we don't want to lose access to any messages at any point either. With outages we see how to do it, with OS patching and failing back and forth it works. But once we need to modify the structure of the Message DataStore sitting on NFS, we would need an outage. This is the thing we am trying to work around in particular.
Thanks,
Andy |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Aug 15, 2017 9:31 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| RAN001 wrote: |
| But once we need to modify the structure of the Message DataStore sitting on NFS, we would need an outage. This is the thing we am trying to work around in particular. |
Explain exactly what you mean by "modify the structure of the Message DataStore" and why you need to do it often enough that it affects availability.
Whatever that is, it sounds like the load balancer is more like the solution you need.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| RAN001 |
| Posted: Tue Aug 15, 2017 9:41 am Post subject: |
|
|
Novice
Joined: 14 Feb 2017
Posts: 11
|
| Honestly I don't know which MQ patches/ upgrades would require a modification of the "MQ Datastore" that is on NFS. But I don't understand how I could apply a patch that needs to modify the structure of it, while still being up with access to all messages. I am trying to run and patch a system without outages. Literally zero down time. It is one thing if a catastrophic event pushes us into another data center, but normal planned maintenance should not cause this. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Aug 15, 2017 9:58 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| RAN001 wrote: |
| Honestly I don't know which MQ patches/ upgrades would require a modification of the "MQ Datastore" that is on NFS. |
Why do you think any of them would?
| RAN001 wrote: |
| But I don't understand how I could apply a patch that needs to modify the structure of it, while still being up with access to all messages. I am trying to run and patch a system without outages. |
Ok, you're talking about 2 different things here. Do you want access to all messages or no outages / Literally zero down time? These are 2 different things with 2 different solutions. A system can be available without all of the messages being available.
Aside from talking about a situation you don't know you'll ever face. Are you worried about flying monkeys nesting on the building as well?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| RAN001 |
| Posted: Tue Aug 15, 2017 11:22 am Post subject: |
|
|
Novice
Joined: 14 Feb 2017
Posts: 11
|
| Vitor wrote: |
| Do you want access to all messages or no outages / Literally zero down time? |
My answer is Yes and Yes and Yes. If this is impossible, I think IBM has been slacking. For what they are charging for these products this should not be an issue. Users and systems are unwilling to tolerate maintenance outages.
We are processing orders across these systems and we are competing with the likes of Amazon. You will never get to 1 hour delivery with 4 hour maintenance windows. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Aug 15, 2017 12:05 pm Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| RAN001 wrote: |
| My answer is Yes and Yes and Yes. |
Ok you are, possibly deliberately, misunderstanding my question. I'll assume you're not trolling...... 
It's very easy and very possible to design an MQ based system with no outages and zero down time. It's very easy and very possible to design a system where you have access to messages under all circumstances. These are 2 separate requirements.
If you want all of that, then you'll need a blend of solutions and not all of them technical. Sort out your use cases.
| RAN001 wrote: |
| We are processing orders across these systems and we are competing with the likes of Amazon. You will never get to 1 hour delivery with 4 hour maintenance windows. |
Good luck with that; please post when you're ready to destroy Amazon (who use messaging) so I can buy things with my Amazon points
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Aug 15, 2017 12:09 pm Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| RAN001 wrote: |
| You will never get to 1 hour delivery with 4 hour maintenance windows. |
And I get grief if our banking customers can't access their money for any length of time.
Purely for the record, we have 6 hour weekly maintenance windows. With no outages. Zero down time.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Tue Aug 15, 2017 3:45 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
I think the only way you are going to come close to being able to say you will "always" have access to that specific message on the specific queue, even if I have to take a queue manager down for patching, is if you are using Queue Sharing Groups on z/OS.
Notice I put "always" in quotes. Never say never. I don't have first hand experience with Queue Sharing Groups, but I imagine once every blue moon the MVS guys may need to do some kind of maintenance on the Coupling Facility or some other stuff deep down in the bowels of z/OS that may mean a brief outage to the coupling facility meaning a brief outage to the shared queue(s). Not sure though.
But any other platform? That queue manager has to come offline to apply a patch, and if the only instance of that message is on a queue on that queue manager, you have outage, even if only for a minute or two. Not an outage to the overall system and its ability to process new messages but an outage to access that one specific message.
Always.
Instant.
Never.
Forever.
These words do not work in the real world of IT.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| gbaddeley |
| Posted: Tue Aug 15, 2017 4:06 pm Post subject: |
|
|
Jedi Knight
Joined: 25 Mar 2003
Posts: 2538
Location: Melbourne, Australia
|
MQ Clustering goes a long way towards your goal. Spread the load across many app hosts, so that at least one app host can be unavailable at a time for maintenance, and the others will take up all the load.
HA & MIQM solutions do not provide true 24X7. There will still be a short outage when a fail-over occurs (2-10 minutes or longer is not uncommon, if app restart is included).
There may also be a full outage to the HA cluster if the underlying HA s/w updated. We recently had to endure full 2 hour outages on AIX HA clusters due to a PowerHA s/w upgrade.
If the app needs a common back-end data store that is fully available, even a HA DB server is not perfect. You may need to consider a distributed DB solution, or be able to stage / cache the DB queries and updates on the local server while the DB is unavailable.
Don't forget you are also at the mercy of the network and data center infrastructure (power, air conditioning, external n/w links, routers etc). Network reliability does not seem to be improving much over the years.
_________________
Glenn |
|
| Back to top |
|
|
| zpat |
| Posted: Tue Aug 15, 2017 10:49 pm Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
24 x 7 x 365 is easy, just do the upgrade on Feb 29th.
_________________
Well, I don't think there is any question about it. It can only be attributable to human error. This sort of thing has cropped up before, and it has always been due to human error. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Aug 16, 2017 4:38 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
And don't forget These days you have also the option of using an MQ Appliance... 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Aug 16, 2017 4:56 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| PeterPotkay wrote: |
| I think the only way you are going to come close to being able to say you will "always" have access to that specific message on the specific queue, even if I have to take a queue manager down for patching, is if you are using Queue Sharing Groups on z/OS. |
Hence my banging on about needing access to the messages or access to the messaging facility. If you don't care about the messages (because you're in a low SLA request/reply scenario) then you focus on having at least 1 queue manager up and available. If you need the messages then I refer to your post.
Please also assume I've done the whole thing about storing large numbers of messages on a queue like it's a database and how that stinks as a design.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
|
|
