| Author |
Message
|
| sharonv |
 Posted: Sun Jul 27, 2014 12:50 am Post subject: mq 7.0.1.11 performance differences between windows and linu |
 |
|
Apprentice
Joined: 03 Feb 2011
Posts: 27
|
Hi
Till recently we used a window server 2003 to run mq 7.0.1.11
last week we put into action our linux environemnt
after doing some basic performance test
for example connecting to a QMGR and writing a small message
we noticed that the linux system in about 3-6 times slower then the windows regarding MQPUT
this is puzzeling as the linux machine is storonger then the windows machine
i would appriciate if anyone can point me where to look for the differences
more details
1- we use persistent messages
2 - both environemnt linux and windows uses the same network
3- linux redhut 2.6.3 ( maybe higher i ;ll need to check)
4- run the ibm test script for the linux and it past ok , also adjuxted limits accroding to ibm docs
5- both system linux and windows have the same configuration qmgrs,logs etc
thanks |
|
| Back to top |
|
 |
| smdavies99 |
| Posted: Sun Jul 27, 2014 4:12 am Post subject: Re: mq 7.0.1.11 performance differences between windows and |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| sharonv wrote: |
3- linux redhut 2.6.3 ( maybe higher i ;ll need to check)
|
you probably are on RHEL 6.<something> that uses a 2.6.3 kernel.
I'd look at the storage that you are using for your logs & data.
Are you using a SAN connected over the network or are you using a native adapter and Fibre etc
Does the network connection (if used) run in Auto-negociate mode or in Full duplex?
There are a whole range of things that can and will affect the performance. If you don't have the resources of someone who really understands Linux and Linux systems then you may not get the results you think you should.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| sharonv |
| Posted: Sun Jul 27, 2014 4:49 am Post subject: |
|
|
Apprentice
Joined: 03 Feb 2011
Posts: 27
|
HI
As you figured out i'm the mq administrator , and it is REDHUT 6 wit 2.6.3. kernel
i do have a professional linux and storage team .
according to them the linux and widnows machine using the same storage
SAN with fiber ( no network connection )
Not sure if i mentioned before , the windows MQ runs with MSCS with 2 shared disk one of the log and other for the configuration
The Linux one runs with veritas cluster with 2 shared disks as well for the log and configuration
IF you have any more suggestion please rasie them
Thanks
Sharon |
|
| Back to top |
|
|
| tczielke |
| Posted: Sun Jul 27, 2014 6:21 am Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 939
Location: Illinois, USA
|
If you are at Red Hat 6, your kernel is probably 2.6.32. 2.6.3 would be a very old Linux kernel, for sure.
I would recommend working with your SAN support to see if there is any noticeable I/O issues. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Sun Jul 27, 2014 7:33 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9403
Location: US: west coast, almost. Otherwise, enroute.
|
Are the images provisioned with the same resources?
Are either or both VMware images?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| tczielke |
| Posted: Sun Jul 27, 2014 11:43 am Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 939
Location: Illinois, USA
|
| Quote: |
| we noticed that the linux system in about 3-6 times slower then the windows regarding MQPUT |
Can you elaborate more what specifically you are measuring here? For example, are you doing a gettimeofday function call before and after the MQPUT API and comparing timestamps? Or is this the overall time for the application when it does MQPUTs? Also, when you say 3-6 times slower, can your provide more timing specifics? For example, an MQPUT was 0.000010 seconds on average on the Windows server and 0.000030 seconds on average on the Linux server. Also, how many tests are your running to get your baseline?
I know these are a lot of questions, but when you start dealing with things like microsecond performance, especially on the x86 processor (if this is Linux on x86), you have to be a little careful. If this is a multi processor server, you have to start to take into things like TSC drift. The TSC is the Time Stamp Counter register that would be used to get to microsecond accuracy. The Linux kernel itself can usually only measure time at the milli-second accuracy, since the system timer is normally set to interrupt the processor in the 1/1000 of a second range. In order to get micro-second timings, the TSC register is usually leveraged. However, the TSC registers can start to drift apart between different processors. If your Linux task is dispatched onto different CPUs you can get the phenomenon where you actually could get a negative number when you subtract the two timestamps due to TSC drift! The Linux kernel will definitely prefer that your task runs on the same CPU, but these are some of the "anomalies" you have to be aware of when dealing with micro-second performance on an x86 processor. |
|
| Back to top |
|
|
| sharonv |
| Posted: Mon Jul 28, 2014 11:19 pm Post subject: |
|
|
Apprentice
Joined: 03 Feb 2011
Posts: 27
|
Hi Thanks for the response
First we are running on physical machines no VM .
the test is performed using a simple client program that runs on a seperate windows station ( same network for win and linux and win station )
connect to the qmgr
then do MQPUT and commit
and i calculate the time diff between the begining and the end of the operation ( only the mqput not the connect)
The different i see is as follows
on Windows machine about 1-2 miliseconds
on linux machine 6-8 milliseconds
the difference is reduced when i use larger bulk of messages but not by a lot 5-10 %
and again the win and linux running the same MQ configuration and uses the same storage. |
|
| Back to top |
|
|
| Andyh |
| Posted: Tue Jul 29, 2014 1:54 am Post subject: |
|
|
Master
Joined: 29 Jul 2010
Posts: 237
|
The latency when putting a persistent message will be dominated by the cost of synchronously writing the data to the recovery log.
6-8ms sounds like rotational delay, which suggests that either /var/mqm/log is not located on the SAN, or that the SAN isn't using any form of reliable write cache.
It would be prudent to run a local version of the application on the linux machine, to eliminate the possibility of any network delay (alternatively measure a non-persistent put from a client to eliminate the synchronous disk write). |
|
| Back to top |
|
|
| tczielke |
| Posted: Tue Jul 29, 2014 4:13 am Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 939
Location: Illinois, USA
|
I would consider doing the following.
First get your baseline of your PUT on the Linux server with several tests. Then go onto the Linux queue manager server and turn on an API trace for just the amqrmppa process (the process that your SVRCONN channel is more than likely running on). You would do this like follows on the Linux queue manager server:
strmqtrc -m qmgr -t api -p amqrmppa
Now run your client PUT tests again. If the timings did not change much, the overhead of the trace is very minimal. It probably will be, since this is a very targeted trace.
Now go back to your Linux queue manager machine and format the trace in the trace directory (usually in /var/mqm/trace):
dspmqtrc AMQ12345.0.TRC > AMQ12345.0.FMT (assuming 12345 is your amqrmppa pid number)
Edit the AMQ12345.0.FMT file and look for your channel name. Note that tid that your channel is running on. Then shortly after you should see a line that has this:
MQPUT >> (the entry point into the PUT API)
and a subsequent line that has this:
MQPUT << (the exit point of the PUT API)
If you look at the formatted data on the right after these MQPUT lines, you should be able to see your queue name, which will help identify that you have the right MQPUT.
The MQPUT lines will have a microsecond timestamp on the left which will give you a sense of how long the MQPUT is actually taking.
You can do the same test on the Windows queue manager server. The main difference is that the Windows trace does not need to be formatted and appears in a different location (i.e. \Program Files\IBM\WebSphere MQ\trace). Besides that, it is basically the same process to run the trace.
If you find that the traces are not adding much overhead in your tests, then these MQPUT trace timings should help you rule out or confirm if the delay is really inside the MQPUT. It is possible you are experiencing some type of network delay between the Windows client and Linux server. |
|
| Back to top |
|
|
| JosephGramig |
| Posted: Tue Jul 29, 2014 5:04 am Post subject: |
|
|
Grand Master
Joined: 09 Feb 2006
Posts: 1231
Location: Gold Coast of Florida, USA
|
|
| Back to top |
|
|
|
|
