| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
| Performance tuning of sending a lot of small message to MQ |
« View previous topic :: View next topic » |
| Author |
Message
|
| marcin.kasinski |
 Posted: Thu Jun 28, 2007 11:26 am Post subject: Re: Performance tuning of sending a lot of small message to |
 |
|
Sentinel
Joined: 21 Dec 2004
Posts: 850
Location: Poland / Warsaw
|
Hi,
I think it will be very difficult to help you with this problem.
I think you have no MQ background and ask us about application which as you red here don't follow "best practises".
Maybe first start from talking to your MQ dev people.
Then post question(s) here and we will try to help you.
No offense.
_________________
Marcin |
|
| Back to top |
|
 |
| carfield |
| Posted: Thu Jun 28, 2007 6:07 pm Post subject: Re: Performance tuning of sending a lot of small message to |
|
|
Novice
Joined: 30 May 2007
Posts: 13
Location: Hong Kong
|
| marcin.kasinski wrote: |
I think you have no MQ background and ask us about application which as you red here don't follow "best practises".
|
Sure I am, would you recommend some document for me about the best practises of using MQ?? |
|
| Back to top |
|
|
| carfield |
| Posted: Thu Jun 28, 2007 6:41 pm Post subject: Re: Performance tuning of sending a lot of small message to |
|
|
Novice
Joined: 30 May 2007
Posts: 13
Location: Hong Kong
|
| carfield wrote: |
>>
I use
Vector<Integer> headerComp = new Vector<Integer>();
headerComp.add(new Integer(MQC.MQCOMPRESS_SYSTEM));
MQEnvironment.hdrCompList = headerComp;
Vector<Integer> msgComp = new Vector<Integer>();
msgComp.add(new Integer(MQC.MQCOMPRESS_RLE));
msgComp.add(new Integer(MQC.MQCOMPRESS_ZLIBHIGH));
MQEnvironment.msgCompList = msgComp;
But it doesn't help
|
Is this the correct way to compress message? I don't need to do anything at server side?? |
|
| Back to top |
|
|
| marcin.kasinski |
| Posted: Thu Jun 28, 2007 10:45 pm Post subject: Re: Performance tuning of sending a lot of small message to |
|
|
Sentinel
Joined: 21 Dec 2004
Posts: 850
Location: Poland / Warsaw
|
| carfield wrote: |
| carfield wrote: |
>>
I use
Vector<Integer> headerComp = new Vector<Integer>();
headerComp.add(new Integer(MQC.MQCOMPRESS_SYSTEM));
MQEnvironment.hdrCompList = headerComp;
Vector<Integer> msgComp = new Vector<Integer>();
msgComp.add(new Integer(MQC.MQCOMPRESS_RLE));
msgComp.add(new Integer(MQC.MQCOMPRESS_ZLIBHIGH));
MQEnvironment.msgCompList = msgComp;
But it doesn't help
|
Is this the correct way to compress message? I don't need to do anything at server side?? |
How your application connects to QM client or binding mode ?
To make it working you have to check channel definition. It shoud accept compression.
| carfield wrote: |
>> You are connecting ONCE to the QM, opening the queue ONCE, and
>> then looping on just the 5000 puts, right? You're not connecting
>> openeing putting closing and disconnecting for each and every
>> message are you?
This is critical, I think may be we are setting up connection for every message. How can we set MQ JMS library to make sure connection will reuse for all message??
|
This is the most important thing in your problem.
If you don't check / fix it you will not achieve good performance.
You can use compression, increase batch size, ... move code to C, everyting, but firs fix it.
If you don't know how your framework works check it or change it.
3 seconds for one message shows your design is very, very weak.
_________________
Marcin |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Jun 29, 2007 12:23 am Post subject: Re: Performance tuning of sending a lot of small message to |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| carfield wrote: |
| marcin.kasinski wrote: |
I think you have no MQ background and ask us about application which as you red here don't follow "best practises".
|
Sure I am, would you recommend some document for me about the best practises of using MQ?? |
You could do worse that look at some of the redbooks around performance, also some of the Java sample code.
To mitigate some of the previous comments, your design is solid but the execution has some issues. You need to resolve the connection problem - any savings you get from message compression will be trivial with messages the size of yours, and the processor cost of compression/decompression may well make things worse!
Work out why your code does not use a single connection for all of it's puts - this is likely to be a coding problem. If you're unwilling or unable to change the code then you need to investiagte alternative solutions, most of which (that I can think of anyway) involve significant changes to your architecture and additional cost.
These may be desireable anyway, rather than using a client connection halfway round the world, but that's a different class of decision entirely. 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| carfield |
| Posted: Fri Jun 29, 2007 8:48 am Post subject: Re: Performance tuning of sending a lot of small message to |
|
|
Novice
Joined: 30 May 2007
Posts: 13
Location: Hong Kong
|
| marcin.kasinski wrote: |
How your application connects to QM client or binding mode ?
To make it working you have to check channel definition. It shoud accept compression.
|
I connect as TCP client , like the follow
| Code: |
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:lang="http://www.springframework.org/schema/lang"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
http://www.springframework.org/schema/lang http://www.springframework.org/schema/lang/spring-lang-2.0.xsd">
<bean id="mqConnectionFactory" class="com.ibm.mq.jms.MQQueueConnectionFactory">
<property name="hostName" value="localhost" />
<property name="port" value="1414" />
<property name="channel" value="default" />
<!-- JMSC.MQJMS_TP_CLIENT_MQ_TCPIP = 1-->
<property name="transportType" value="1" />
</bean>
<bean id="connectionFactory" class="org.springframework.jms.connection.SingleConnectionFactory">
<property name="targetConnectionFactory" ref="mqConnectionFactory" />
</bean>
<bean id="jmsTemplate" class="org.springframework.jms.core.JmsTemplate">
<property name="connectionFactory" ref="connectionFactory" />
</bean>
</beans>
<property name="queue">
<bean class="com.ibm.mq.jms.MQQueue">
<property name="baseQueueName" value="${mq_queue_name}"/>
<property name="targetClient" value="1"/>
</bean>
</property>
|
Do you think it accept compression??
| marcin.kasinski wrote: |
This is the most important thing in your problem.
If you don't check / fix it you will not achieve good performance.
You can use compression, increase batch size, ... move code to C, everyting, but firs fix it.
If you don't know how your framework works check it or change it.
3 seconds for one message shows your design is very, very weak. |
I changed to SingleConnectionFactory which will reuse connection for all message, now it is faster, but still slow than expect... so I still welcome any comment... I just send message simple use
| Code: |
jmsTemplate.send(queue, new MessageCreator()
{
public Message createMessage(Session session) throws JMSException
{
return session.createTextMessage(input);
}
});
|
[/code] |
|
| Back to top |
|
|
| carfield |
| Posted: Fri Jun 29, 2007 8:51 am Post subject: Re: Performance tuning of sending a lot of small message to |
|
|
Novice
Joined: 30 May 2007
Posts: 13
Location: Hong Kong
|
| Vitor wrote: |
You could do worse that look at some of the redbooks around performance, also some of the Java sample code.
|
Could you give me link about that?
| Vitor wrote: |
To mitigate some of the previous comments, your design is solid but the execution has some issues. You need to resolve the connection problem - any savings you get from message compression will be trivial with messages the size of yours, and the processor cost of compression/decompression may well make things worse!
|
Yes, try that, it just don't make any difference if my compress setup really work
| Vitor wrote: |
Work out why your code does not use a single connection for all of it's puts - this is likely to be a coding problem. If you're unwilling or unable to change the code then you need to investiagte alternative solutions, most of which (that I can think of anyway) involve significant changes to your architecture and additional cost.
These may be desireable anyway, rather than using a client connection halfway round the world, but that's a different class of decision entirely. |
According to the document I should reuse my connection now... However, I will try to not use spring jmstemplate to see if thing get better |
|
| Back to top |
|
|
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
