| Author |
Message
|
| Monk |
 Posted: Fri Nov 02, 2007 10:10 pm Post subject: MQ Vs FTP |
 |
|
Master
Joined: 21 Apr 2007
Posts: 282
|
Hi All,
This is a general design question .
I have a client/server scenario , MQ client is installed on many of the client m/c and I have one MQ server.
Now I need to implement file transfer over MQ.
My deliemma is , whether FTP can do the job , since we have a client server architecture.
Can MQ provide any benefit in this client server architecture.
I mean to say...some people claim that FTP is not reliable .
Can MQ be reliable to send files in a Client / server scenario or just plain simple FTP will do?
Any ideas/inputs on this will be much appreciated.
Thanks.
_________________
Thimk |
|
| Back to top |
|
 |
| Gaya3 |
| Posted: Fri Nov 02, 2007 10:25 pm Post subject: |
|
|
Jedi
Joined: 12 Sep 2006
Posts: 2493
Location: Boston, US
|
Hi
MQ V/s FTP, both are on different poles.
1. Depends upon whats your requirements
2. How important is your data (Critical persistant or non-persistant)
3. Do you require any Acknowledge back (synchronous / asynchronous)
4. Do you want it in Delivery Once.
5. Do you want to secure it while transfering (ssl etc )
There are much more, you will get it soon, try to get these answers.....
IMHO if you are already having MQ infrastructure set up, then whats the need of transfering it through some other way
Note: IBM MQ Series is an award winning middleware tool
Regards
Gayathri
_________________
Regards
Gayathri
-----------------------------------------------
Do Something Before you Die |
|
| Back to top |
|
|
| Monk |
| Posted: Fri Nov 02, 2007 11:01 pm Post subject: |
|
|
Master
Joined: 21 Apr 2007
Posts: 282
|
my requirement is simple.
Only that the file transfer should be reliable.
Can MQ in a Client/server enviroment guarantee that?.
that means MQ client transfers files to MQ server. Is this method reliable.
What if the connection breaks..
what will happen to the file..
Do i need to resend it.., if so then a simple FTP can do the job..
just need an answer to this one Q only..
_________________
Thimk |
|
| Back to top |
|
|
| elvis_gn |
| Posted: Sat Nov 03, 2007 1:27 am Post subject: |
|
|
Padawan
Joined: 08 Oct 2004
Posts: 1905
Location: Dubai
|
Hi Monk,
| Monk wrote: |
Only that the file transfer should be reliable.
Can MQ in a Client/server enviroment guarantee that?.
that means MQ client transfers files to MQ server. Is this method reliable.
What if the connection breaks..
what will happen to the file..
Do i need to resend it.., if so then a simple FTP can do the job..
just need an answer to this one Q only.. |
The file transfer is definitely reliable as MQ Messaging is reliable...when the connection breaks the msgs will definitely be somewhere in the local/transmission queues.
But then I don't think the MQ File transfer is a good option for production...it does not have APIs to program the file pick and tranfer...
Rather I would suggest you look at some FTP app or Adapters with some tool like Message Broker...
Regards. |
|
| Back to top |
|
|
| Monk |
| Posted: Sat Nov 03, 2007 3:50 am Post subject: |
|
|
Master
Joined: 21 Apr 2007
Posts: 282
|
Ok so the point is MQ is reliable w transfering files in a Server-Server environment and not reliable or just as reliable in a Client-server environment , when transfering files.
Someone come out and say *yes* to the above statement.
Thats all I ask...
or someone prove me wrong.
_________________
Thimk |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sat Nov 03, 2007 7:19 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| Monk wrote: |
Ok so the point is MQ is reliable w transfering files in a Server-Server environment and not reliable or just as reliable in a Client-server environment , when transfering files.
Someone come out and say *yes* to the above statement.
Thats all I ask...
or someone prove me wrong. |
Have a look at industrial strength ftp over MQ like PM4DATA...
Issue a RFP and check against your requirements...
Measure the cost and worth of a solution for YOU.
Check against cost of the proposals that meet your requirements...
Great now you can go shopping for a product... 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Sat Nov 03, 2007 7:25 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
MQ doesn't know about "files".
MQ is better than FTP, because you know that a message is "ready". You don't know that an FTP file transfer is complete, because you don't know how big the file is. You don't know that an FTP transfer is complete, because FTP servers vary quite a lot in how they handle files - they may not keep any files open for reading or writing, even though they are still "in process". They may or may not lock the file for other people to read or write. Even if they do, you may or may not be able to see the lock depending on how you are viewing the file (NFS, for example, doesn't support locks).
Files are significantly more complicated to deal with than MQ messages.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| Monk |
| Posted: Sun Nov 04, 2007 7:42 pm Post subject: |
|
|
Master
Joined: 21 Apr 2007
Posts: 282
|
my requirement is that i need to move just one .zip file from a client machine to a server.
Do I really need to use MQ for this?.
but some of the ppl are forcing me to use MQ...
coz for some reason they think that MQ is a better approach than FTP for this requirement..
I do not agree to this..
What do u ppl think?
_________________
Thimk |
|
| Back to top |
|
|
| RogerLacroix |
| Posted: Sun Nov 04, 2007 9:25 pm Post subject: |
|
|
Jedi Knight
Joined: 15 May 2001
Posts: 3264
Location: London, ON Canada
|
| Monk wrote: |
coz for some reason they think that MQ is a better approach than FTP for this requirement..
I do not agree to this.. |
Seriously!?!? 
So, you want to transfer your file successfully about 98.5% of the time using FTP. You don't mind that it is not 100%. Seriously!?!?
I've never found that doing something half-ass is a good career move.
Anyway, that's my 2 cents.
Regards,
Roger Lacroix
_________________
Capitalware: Transforming tomorrow into today.
Connected to MQ!
Twitter |
|
| Back to top |
|
|
| Monk |
| Posted: Sun Nov 04, 2007 9:34 pm Post subject: |
|
|
Master
Joined: 21 Apr 2007
Posts: 282
|
what you forget my friend is that end of the day..its still client server ..and if MQ client is transfering the files..and connection breaks the file still has to be transfered again..so there is a failure rate..
so how can contact admin me here..
and the fact that FTP is W3C approved and still used as the best solution for this sort of requirement in most places..
only reason for that 2 % failure rate is bad design and use of stream mode for file transfer instead of block mode.
So i really don't see how MQ can be better than FTP in this scenario(client server).
_________________
Thimk |
|
| Back to top |
|
|
| Monk |
| Posted: Sun Nov 04, 2007 9:37 pm Post subject: |
|
|
Master
Joined: 21 Apr 2007
Posts: 282
|
Also if IBM has envisioned that MQ was better than FTP ..then they would have put in this functionality in V 5.3 and not the late V6 , which is still being experimented.
_________________
Thimk |
|
| Back to top |
|
|
| RogerLacroix |
| Posted: Sun Nov 04, 2007 9:54 pm Post subject: |
|
|
Jedi Knight
Joined: 15 May 2001
Posts: 3264
Location: London, ON Canada
|
| Monk wrote: |
| Also if IBM has envisioned that MQ was better than FTP ..then they would have put in this functionality in V 5.3 and not the late V6 , which is still being experimented. |
IBM did not write an FTP replacement product. They created an assured message delivery product. This product can be used for most ypes of applications including FTP.
Regards,
Roger Lacroix
_________________
Capitalware: Transforming tomorrow into today.
Connected to MQ!
Twitter |
|
| Back to top |
|
|
| Monk |
| Posted: Sun Nov 04, 2007 10:21 pm Post subject: |
|
|
Master
Joined: 21 Apr 2007
Posts: 282
|
The point is what benefits do I get if I use MQ to transfer files in a client - Server environement.
_________________
Thimk |
|
| Back to top |
|
|
| ashoon |
| Posted: Mon Nov 05, 2007 4:49 am Post subject: |
|
|
Master
Joined: 26 Oct 2004
Posts: 235
|
here's a few reasons... some of them don't apply to your mq client-server environment however I don't see why a COA can't alleviate that issue...
FTP is synchronous - it requires both ends to be available at the time of transmission.
MQ is asynchronous.
FTP has no connection or data retry logic built-in.
MQ can guarantee that a particular message arrives exactly once, no more no less.
FTP does not support SSL, although it can be run through an SSL tunnel or otherwise make use of SSL.
MQ has support for SSL built in.
FTP can send a single packet of data that is as large as you want it to be.
MQ can only send a single packet that is less than or equal to 104857600 bytes in length.
FTP is fairly sensitive to network speed and usage.
MQ is fairly insensitve to network speed and current usage.
With FTP the following problems could occur:
the sending system might delete a file (message) when the sender thinks the receiver has received the file
the sending system might send multiple copies of a file to the receiver (over writing the originals)
the sending system might send messages out of order
FTP the userid and password is sent in clear text with obvious security risks.
FTP access to files from programs is not transactional and needs to be serialised (lock out).
FTP is not CLUSTER aware and so cannot be managed with MSCS etc.
FTP the password cannot be changed using the FTP protocol.
FTP does not provide any restart/resume capability for interrupted transfer (no checkpoints).
FTP has no in-built ability to re-retry a file transfer if the other end is down.
FTP cannot ensure that the entire file has been sucessfully transfered.
FTP cannot perform conditional post-transfer processing automatically.
FTP does not work in a transactional mode (which would ensure data is sent once only).
FTP does not provide assured delivery, or confirmation of arrival.
FTP cannot ensure the integrity of the data being transmitted.
FTP in get mode allows unauthorised access attempts to the data.
FTP does not ensure record/message order (over more than one file).
FTP does not allow overlapping - eg processing records before they have finished being produced - MQ can reduce elapsed time.
FTP requires disk space to store a complete copy of data before and after transmission
FTP only has basic data conversion if any - no support for UNICODE etc.
FTP sends all the data in clear text, therefore introducing privacy issues.
FTP forces the use of batch windows and file based procedures.
FTP cannot ensure that the other end of the link is who they claim to be.
FTP only one program/thread can be reading a file, whereas multiple threads can be processing messages from a queue at the same time, exploiting multiple processors.
FTP does not have even the most basic built-in scheduling features.
FTP has no system management, tracking or control features for operations.
FTP usage often requires that online systems have downtime periods.
FTP in progress can't be cleanly cancelled in the event that the need arises.
FTP limitations usually result in considerable manual intervention and costs.
FTP does not offer compression of data which can be important for large volumes.
FTP inherently has major data latency whereas a messaging design might have almost none.
FTP has no central point of administration or control, so that operational and management effort increases with every deployment of an additional FTP transfer.
FTP embeds meta-data information such as user name, password, host name, file name making them awkward to change and, in the case of password, violating security policies when not changed.
FTP accidental or deliberate use of the wrong FTP password will cause failures in production by revoking ids.
_________________
IBM Certified - SOA Solution Designer & WebSphere Datapower SOA Appliances |
|
| Back to top |
|
|
| AkankshA |
| Posted: Mon Nov 05, 2007 8:41 pm Post subject: |
|
|
Grand Master
Joined: 12 Jan 2006
Posts: 1494
Location: Singapore
|
ashoon,
that was impressive   
_________________
Cheers |
|
| Back to top |
|
|
|
|
