| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Design Question - MQ Input Node |
« View previous topic :: View next topic » |
| Author |
Message
|
| sieijish |
 Posted: Fri May 19, 2006 4:50 am Post subject: Design Question - MQ Input Node |
 |
|
Acolyte
Joined: 29 Nov 2004
Posts: 67
Location: London
|
I have a situation where I have to receive messages from about 50,000 end points and publish them to appropirate topics. Each message will be around 1KB and persistant.
The challenge here is to support so many incoming connections. There could be flood gate scenarios where all clients try to connect.
My question is how would we manage such large incoming traffic on broker? If I use MQInputNode to receive the message, then QueueManager will handle the client connections and clients can use JMS API to connect to a Server Connection channel.
Is this feasible? Also what is the rate at whcih the clients can push in data? Can someone shed some light into this.
Regards,
SD |
|
| Back to top |
|
 |
| jefflowrey |
| Posted: Sat May 20, 2006 5:20 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
If you use an MQInput node, you can farm the server connections out to several queue managers, all of which have regular channels that connect to broker queue manager.
What is the expected average time from when one of thiese clients produces a message to the time that it is published by broker? What is the maximum allowable time?
What hardware is the broker running on? What hardware are the clients running on? What are the clients written in?
What else do the clients do? How fast could the clients reasonably PRODUCE data, assuming everything else is instantaneous? How fast is the network between the client and the broker?
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| sieijish |
| Posted: Thu May 25, 2006 7:38 am Post subject: |
|
|
Acolyte
Joined: 29 Nov 2004
Posts: 67
Location: London
|
During peak time, Clients can generate message at 4msgs/min. You can take upto 24hrs to tranfer the messages from the clients to the Server. On the server the message is published and the subscribers populate it to a Datawarehouse to generate reports.
The Message broker is running on IBM P650, AIX 5.2. The clients are normal PCs with enough memory & harddisk. Memory, disk space & CPU on the client terminal is not a limitation.
The key requirements are
1) message integrity - once & once only delivery.
2) Scalablity - 50,000 - 100,000+ terminals sending message at a peak rate of 4msgs/min per terminal
3) Flood gate stability
4) Pub-Sub is a must
The clients are on an ADSL line and but bandwidth is costly and hence usage should be as min as possible. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Thu May 25, 2006 8:07 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Okay. So if the time from "client sends message" to "subscribers insert data into datawarehouse" is 24 hours, then there are no problems here.
Create one or more "client concentrator" queue managers. All client applications will connect to these, and these can be scaled horizontally or vertically to meet the load of the client connections. All clients will write to either qclusters or qremotes on these queue managers - and MQ networking will be setup to move messages to broker. Likewise, any data that the clients need to receive will be moved by MQ to qlocals on the qmgr that a particular client is connected to. These client connectrators can be as big or small as you want.
This will isolate the broker from failures caused by client connection volume resource usage.
You can use build one big client concentrator qmgr machine - perhaps another p650. Or you can build several smaller client concentrator machines - perhaps xSeries servers running linux.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
