| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Time for new MQ Servers |
« View previous topic :: View next topic » |
| Author |
Message
|
| PeterPotkay |
 Posted: Wed May 11, 2005 12:04 pm Post subject: Time for new MQ Servers |
 |
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
As my servers get grey and old, its time to start planning for new MQ
servers in 2006.
How is my money best spent for a typical MQ server (no WB-IMB)? One type of
QM would be a Gateway into my clusters, the other type would be an MQClient
Concentrator, where 2-3 thousand MQClients would connect in. With logs in MQ
6.0 now able to go to 64GB on Windows and 128GB on Solaris, I already know
my hard drive space needs to go way up.
Am I better off with 4 CPUs, or only 2 CPUs, but those 2 or much bigger? How
does MQ take advantage of 2,4,8, etc CPUs? Or should I spend the $$$ on gobs
of memory?
The performance support packs do not talk about this. They just give you the
specs on what machine they did use, and then tried to ram as much as
possible through it.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
 |
| JLRowe |
| Posted: Wed May 11, 2005 12:58 pm Post subject: |
|
|
Yatiri
Joined: 25 May 2002
Posts: 664
Location: South East London
|
| Buy opteron based kit with single core processors, not only do they trounce intel xeon kit, but you can upgrade to dual core without changing the motherboard. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Wed May 11, 2005 1:04 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
What OS do you want to run?
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Wed May 11, 2005 3:32 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Does it make a differance to the answer?
But that is another decision: go with Windows, or fight for Solaris (more $$$).
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| malammik |
| Posted: Wed May 11, 2005 4:42 pm Post subject: |
|
|
Partisan
Joined: 27 Jan 2005
Posts: 397
Location: Philadelphia, PA
|
CPUs.
Depending on the number of clients you might not need
more than 1 CPU, however from personal observations I
would recommend 1 processor for every 200 concurrent
connections. IBM pSeries at least 670 and 690 models,
allows you to add cpus on demand which is a nice
feature not only from performance reasons but also
because of loopholes in licensing. I would invest into
64 bit hardware only as MQ 6.0 will be able to take
full advantage of them and that's where the future is.
Memory. I dont know how much you would need based on
your loads but once again IBM pSeries servers allow to
add/enable additional memory on the fly.
Disks.
Because MQ does so much IO, I would say your disks are
most important and will have most impact on the
server. If your company already has a SAN solution
from EMC and likes, go with that. Those can be very
fast and reliable. Best thing about managed storage
solution is that you no longer have to worry about
separating log filesystem from data filesystem. If EMC
or the like notices a congestion on a particular
storage device that data is automatically moved to
another device to avoid disk congestions.
THEORETICALLY. If you dont have a SAN solution in
place, I would recommend RAID 1+0 which does stripping
and mirroring at the same time. Striping significantly
improves MQPUT performace as data is written across
multiple disks simultaneously.
I was at one client site where they were using EMC
storage for mq data but their fibre channel cards were
misconfigured and they were utilizing only half of
available bandwith. Dont be greedy, get someone who is
a professional to evaluate/advice on your set up.
Last and most important point. Picking the right
hardware is not really an MQ Administrator's role. In
the past five years, hardware technologies have
reached space age level complexity and power. I would
say that you will be better of coming with a thorough
profile of your application (MQ) in this case and its
usage. Put together a sample document that will help
and hardware specialist pick the most appropriate
harware for you. This document should contain at least
following informations.
max and average of:
concurrent client connected via tcp
server connections
messages put
average data per message
linear logging vs circular
number of transactions
size of transactions
listener vs inetd
number of queues
number of queue managers per server
back up requirements
HA? yes no?
availability requirements and business impact
queue manager to queue manager commuications.
For example if you are specing out a queue manager hub
server that talks to 100 other queue managers, and
volumes are huge, you better get several gigabit
ethernet adapters to handle that volume. All in all
information like that will provide the hardware guy
with necessary info to pick the right hardware for
you.
HTH,
Mikhail.
http://www.netflexity.com
_________________
Mikhail Malamud
http://www.netflexity.com
http://groups.google.com/group/qflex |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Wed May 11, 2005 4:52 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
I hear what you are saying, but what is a hardware guy going to do with that info?
Me:"I will have circular not linear logs."
Hardware guy:"What, no trapezoidial logging?"
"I will have x Clients running."
"How much CPU does a client connection use?"
"I will send 1,000,000 NP messages a day, and they will be under 1K in size"
"That's nice. Your laptop can do that, right?"
"I will have 2 QMs on this server"
"OK, you need 32 CPUs, because a QM needs 16 CPUs, or so I once heard."
You see where I am going? An MQAdmin needs to tell them what an MQAdmin needs. There is nothing in the manuals to help figure this out. The only direction we have is for sizing our storage, as the logging info has lots of numbers.
But when it comes to size and # of CPU, amount of memory, all we get is the bare minimum, which you couldn't go that small if you wanted to.
What the performance pacs and/or Quick Beginnings need is a list like:
Open Queues use this much RAM:
Each channel type uses this much RAM per:
1 fat CPU will work, but 2 will work better because:
1 fat CPU will work, but 4 will work better because:
The 64Bit comment is a good thought.
We have the EMC / SAN option. Storage I am OK with. Its justifying the CPU and RAM that I need help with.
"They" will want me to spend as little as possible.
I will want to get the most appropriate machine for what I expect to push thru.
I need #s to justify what I ask for.
Maybe I am needlessly concerned, and whatever I get for hardware as long as it is at least average will be fine, but given the fact that I have a clean slate for new machines, I would feel better if I have at least something to go on.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| malammik |
| Posted: Wed May 11, 2005 5:02 pm Post subject: |
|
|
Partisan
Joined: 27 Jan 2005
Posts: 397
Location: Philadelphia, PA
|
I see your point too. Here is my final word on CPUs. Consider a scenario where a webserver farm hitting queue manager directly. One of my customers had a pretty large web farm, 100+ servers all connection to 4 queue managers. During sale spikes, I would get a large influx of incomming connections. This is the situation where more cpu's are better. Once the client is connected, the number of cpu's does not really matter because new cpu's, network cards, and diskdrives can talk to each bypassing cpu, called Direct Memory Access DMA for short.
RAM. I would say go with 2MB for every connected client. So if you have 100 clients connecting get at least 2GB of ram and so on.
_________________
Mikhail Malamud
http://www.netflexity.com
http://groups.google.com/group/qflex |
|
| Back to top |
|
|
| Michael Dag |
| Posted: Thu May 12, 2005 1:43 am Post subject: |
|
|
Jedi Knight
Joined: 13 Jun 2002
Posts: 2607
Location: The Netherlands (Amsterdam)
|
| PeterPotkay wrote: |
Does it make a differance to the answer?
But that is another decision: go with Windows, or fight for Solaris (more $$$). |
Peter is the choice in your shop Windows or Solaris?
What about Linux or AIX, is that an option?
_________________
Michael
MQSystems Facebook page |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Thu May 12, 2005 3:11 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
OS does have some impact on what you buy. Personally, I would spec a windows box about 30% bigger than a unix box for the same purposes.
And I wouldn't get a single cpu windows box for anything other than a desktop.
I think the manuals may help some. If you go back and look into the details of runmqlsr and how that relates to process instances - a new process for every X number of active channels - then you can make some basic guesses from your existing hardware about how much cpu/ram each of those individual processes use.
Also, you need to look at your message traffic. If you're generally throwing around a LOT of <1 meg messages, then RAM and CPU are going to be important - particularly if these are NP messages. If you're generally throwing around >10 meg messages, then fast disk is more important than CPU - but RAM is even more important.
Also, haven't you been running into some windows limitations on number of active channels? Or is that someone else? Or is it why you're buying new boxes?
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Thu May 12, 2005 4:38 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
The vast majority of messages are NP and under 100K.
On the list serve, one respondent swears that a single CPU is the way to go, just make it a huge CPU.
We get new servers because these are 4 years old, and are up for refresh, even though nothing is wrong with them. Since I gotta get new ones, might as well ask for something appropriate. My current ones are Windows, 2 CPU (1.2ghz), 4 GB of RAM. They seem to perform OK, but how do I know if I am near some limit, and if I add x more messages or clients connected, things start to go wrong. Maybe I should go to 4 CPUs. Maybe 1 CPU is all I need, although the Sys Admin made me a good point - with 2+ CPUs, one can die, and the box still functions.
No AIX here. Linux...eh...these are very important boxes, Linux is still a little to cutting edge here. I don't want to be breaking new ground in terms of the learning curve on such critical boxes.
Windows - well, I am familiar with it, and it is easier to work in it. But I am sick of all the monthly scheduled outages for Microsoft Security Patches. The few Solaris QMs I have seem so stable. They just work. That's why I would like to maybe switch to Solaris. But, maybe we don't have a lot of Solaris MQ problems just because we don't have a lot of Solaris MQ servers. I know Solaris has HACMP for HA / DR, but to my team that would be a whole new learning curve. Could it be any worse than MSCS on windows? Plus, the Windows Sys Admins assigned to my MQ servers are the best. What if my Solaris Admin is not in the same respective league? Imagine someone's perspective on MQ if their MQ admin was just OK...
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| Michael Dag |
| Posted: Thu May 12, 2005 11:19 pm Post subject: |
|
|
Jedi Knight
Joined: 13 Jun 2002
Posts: 2607
Location: The Netherlands (Amsterdam)
|
Peter,
I understand your dilemma and it is not an easy task to decide.
Windows - does it all, supports availability and failover through MSCS, but has it's quirks (registry stuff) etc, low hardware cost
Linux - the new kid on the block, looks like the other unixes (AIX/Solaris) but runs on cheaper hardware, the risk is in the unknown and lack of knowledge
Solaris - you know this platform already, works, but also has it's quirks in some areas, always when there is a tough situation you get into a blame the other discussion between MQ (IBM) and the OS (Sun), been there (threading model for example with MQSI). Another point you mention Solaris support HACMP / HA, yes and no, HACMP is an IBM term. on Solaris you would use Veritas clustering (another vendor involved) or Sun Clustering (the latest version needs seperate MQ Agents from Sun which cost additional $$$)
AIX - the problem is you/your organisation has no experience with this platform, on the other hand it comes from the same vendor as MQ so no discussion there (although that does not mean you will never have a problem). pSeries hardware is more expensive then comparatively (CPU wise) Sun boxes with the same number of CPU's, that said with AIX you need less CPU's to do the same work. (Just ask you solaris guys to run a CPU usage test and you will see your CPU's idling a lot of the time).
- I know at least one customer that is switching from Solaris to AIX as they had both and saw the 'smaller' AIX boxes perform much better then their 'bigger' Solaris boxes.
Like I said it is not an easy decision, I bet someone will come into this discussion and suggest to move to z/OS 
There is no right or wrong answer it all depends on what you know, and what you are willing and can afford to acquire in the future.
My first choice would be AIX, then Linux, then Solaris and then Windows, for backbone type environments with High availability needs, for application environments Windows is acceptable.
I hope this helped...
_________________
Michael
MQSystems Facebook page |
|
| Back to top |
|
|
| PGoodhart |
| Posted: Fri May 13, 2005 3:15 am Post subject: |
|
|
Master
Joined: 17 Jun 2004
Posts: 278
Location: Harrisburg PA
|
Just a note, while your team is upgrading servers, you might want to make sure you are taking advantage of the available bandwidth on your network. I've seen a lot of servers with 10/100 network cards when the network will do gig. I've seen the 10/100 cards misconfigured so they ran at 10 all the time too. Actually on your client connection box you might want to go to multiple and redundent on the network cards/connections.
_________________
Patrick Goodhart
MQ Admin/Web Developer/Consultant
WebSphere Application Server Admin |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Fri May 13, 2005 4:12 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
z/OS...the problem is that it gets IPLed about every other week here on Saturday nights. That means MQ is down for a couple of hours every 2 weeks...not acceptable as a solution for a gateway QM into a cluster housing 24x7 apps, or for an MQ Client Concentrator supporting 24x7 clients.
Thanks for the feedback, any more is appreciated. I will take a print of this post with me when I meet with the server engineers.
What abot this 32 bit versus 64 bit stuff. Looks like MQ 6.0 will take advantage of 64 bit on some UNIX platforms, but not Windows. How much of an advantage is that? What makes 64 bit so good? And snce Windows won't handle 64 bit in MQ in 6.0, is that a big plus for going Solaris?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| malammik |
| Posted: Fri May 13, 2005 5:25 pm Post subject: |
|
|
Partisan
Joined: 27 Jan 2005
Posts: 397
Location: Philadelphia, PA
|
Benefit of 64 bit computing is that you can fit more data into a single instruction to the cpu. Comparing to large files for example will take almost 80% faster on a 64 bit processor if of course the application was compiled to create 64 bit instructions. This particularly handy in bio computing applications such as dna matching where u litterally match string by string gigs of data, cryptography, parsing, compression, etc. MQ wont significantly benefit from 64bit because it does not care about the data a great deal. Although you will be able to select messages by message id a lot faster.
Some windows servers already support Itanium not sure 100 if in 64 bit mode or not. XP although not a server OS does come in 64 bit mode. Odds are by the time u get ur servers and ready to migrate MS will catch to 64 bit race by I dont think it is worth it for switching to another os. The only significant benefit of buying 64 bit hardware for mq is that the hardware will depreciate slower than 32bit and then you can use it for something else.
_________________
Mikhail Malamud
http://www.netflexity.com
http://groups.google.com/group/qflex |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
