| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
| MQSeries vs. FTP |
« View previous topic :: View next topic » |
| Author |
Message
|
| rtsujimoto |
 Posted: Wed Apr 19, 2006 6:53 am Post subject: |
 |
|
Centurion
Joined: 16 Jun 2004
Posts: 119
Location: Lake Success, NY
|
| Hmm, what is the speed of the comm line you're going over? |
|
| Back to top |
|
 |
| HubertKleinmanns |
| Posted: Wed Apr 19, 2006 10:42 pm Post subject: |
|
|
Shaman
Joined: 24 Feb 2004
Posts: 732
Location: Germany
|
native FTP
- reads the data from disk (out of the source file)
- puts it onto the network
- writes it to the destination disk (to the destination file)
FTP over MQ - reads the data from disk (out of the source file)
- maybe writes the data to the log (if it uses persistent messages)
- maybe writes the data to the queue file (if messages are big or the reserved memory is too small)
- puts it onto the network
- maybe writes the data to the log of the destination QMgr (if it uses persistent messages)
- maybe writes the data to the queue file of the destination QMgr (if messages are big or the reserved memory is too small)
- writes it to the destination disk (to the destination file)
When you use fast disks (e. g. SAN) and have only a poor network, the additional disk accesses may be not observable and MQ may outrun native FTP due to better network performance.
When you have slow disks but a very fast network, or persistent messages in combination with small (circular) logs (which often have to perform rollbacks), or badly settings of the log attributes, or big messages in combination with poor memory, or ..., native FTP would be faster than FTP over MQ.
So the ultimative answer is: it depends on 
_________________
Regards
Hubert |
|
| Back to top |
|
|
| rtsujimoto |
| Posted: Thu Apr 20, 2006 5:52 am Post subject: |
|
|
Centurion
Joined: 16 Jun 2004
Posts: 119
Location: Lake Success, NY
|
| Hubert, that's a lot of *ifs*. I'm still waiting for an answer on the comm line speed...I have questions about the throughput the poster claims to have achieved, e.g. 4.5GB in about 90 sec. |
|
| Back to top |
|
|
| zpat |
| Posted: Thu Apr 20, 2006 6:43 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5867
Location: UK
|
Neither MQ nor FTP is a network protocol. They are network applications. The network protocol is TCP/IP.
Network performance will vary based on many factors, including the application. For example if FTP is cancelled at 99% complete then all the data will be have to be re-sent, whereas MQ would only need to re-send the current message.
MQ file transfer performance will depend on how many data records are in each MQ message, 1-1 will be slower than using larger (but not huge) messages.
MQ file transfer could use non-persistent messages (for higher performance), if the re-send logic was in the MQ application. You can only really compare actual implementations. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Thu Apr 20, 2006 6:51 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
The File Transfer Protocol standard provides a definition of the data exchange protocol that implementations will use to transfer files over a network protocol.
The MQ channel protocol (FAP?) is likewise a definition of a data exchange protocol over various different forms of network protocols. It happens to be a proprietary protocol that is not defined in a public standard, unlike FTP.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| rtsujimoto |
| Posted: Thu Apr 20, 2006 7:35 am Post subject: |
|
|
Centurion
Joined: 16 Jun 2004
Posts: 119
Location: Lake Success, NY
|
| Both MQ and FTP are in the 7th layer of the OSI model, e.g. applications. So, essentially the network is irrelevent since neither application has control over it. The essential question is the application design. FTP is point-to-point, whereas MQ is a store-and-forward system. The overhead and intricacies of a store-and-forward system, e.g. potential logging, queuing of data, etc., does not bode well for speed. Forget about MQ features, e.g. resending at the message level, UOWs, etc. The MQ FTP behaviour should be as close to FTP as possible, e.g. the entire file is sent, or the whole file has to be resent again, and eliminate logging as much as possible by using non-persistent messages. That would be as close to an apples-to-apples comparison as possible given the dissimilar architectures. |
|
| Back to top |
|
|
| ashoon |
| Posted: Thu Apr 20, 2006 10:11 am Post subject: comm line = 100 MBit LAN |
|
|
Master
Joined: 26 Oct 2004
Posts: 235
|
The test between the two servers were connected to each other via my LAN which I believe is 100MBit.
| rtsujimoto wrote: |
| Hmm, what is the speed of the comm line you're going over? |
|
|
| Back to top |
|
|
| rtsujimoto |
| Posted: Thu Apr 20, 2006 10:41 am Post subject: |
|
|
Centurion
Joined: 16 Jun 2004
Posts: 119
Location: Lake Success, NY
|
Ok. 100mb lan = 100,000,000 bits/sec
2.5 GB = 2,500,000,000 bytes = 20,000,000,000 bits
R/T = 1.5 min = 90 sec
20,000,000,000 / 90 = 222,222,222.22 bits/sec
How did you do that? |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Apr 20, 2006 2:50 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20763
Location: LI,NY
|
| rtsujimoto wrote: |
Ok. 100mb lan = 100,000,000 bits/sec
2.5 GB = 2,500,000,000 bytes = 20,000,000,000 bits
R/T = 1.5 min = 90 sec
20,000,000,000 / 90 = 222,222,222.22 bits/sec
How did you do that? |
100 MB = 100*1024*1024*8 bits. = 838,860,800 bits
1 Byte = 8 bits.
2.5GB @ 100MB/s = 2,560 MB@ 100 MB/s ==> 2,560 /100 = 25.6 s well within the 90 s limit even allowing for a great number of extra bits for protocoll overhead...
Enjoy 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| rtsujimoto |
| Posted: Thu Apr 20, 2006 2:58 pm Post subject: |
|
|
Centurion
Joined: 16 Jun 2004
Posts: 119
Location: Lake Success, NY
|
| The lan speed is mega-bits, not mega-bytes. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Apr 20, 2006 3:03 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20763
Location: LI,NY
|
In that case looks like he has a mighty powerfull compression algorythm on top of it. 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Thu Apr 20, 2006 3:07 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| rtsujimoto wrote: |
| The lan speed is mega-bits, not mega-bytes. |
Yes, but I think it's bigger than 100,000,000 bits.
I think it's 1000*1024*1024 bits. Or possibly even 1024*1024*1024 bits.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| rtsujimoto |
| Posted: Thu Apr 20, 2006 4:06 pm Post subject: |
|
|
Centurion
Joined: 16 Jun 2004
Posts: 119
Location: Lake Success, NY
|
|
| Back to top |
|
|
| wschutz |
| Posted: Thu Apr 20, 2006 4:14 pm Post subject: |
|
|
Jedi Knight
Joined: 02 Jun 2005
Posts: 3316
Location: IBM (retired)
|
|
| Back to top |
|
|
| ashoon |
| Posted: Thu Apr 20, 2006 8:28 pm Post subject: hmmm... |
|
|
Master
Joined: 26 Oct 2004
Posts: 235
|
good point and my mistake... double checking the lan speeds in the test env. and it's a Gigabit Network. I believe though the point still stands where MQ was much faster than FTP.
| rtsujimoto wrote: |
Ok. 100mb lan = 100,000,000 bits/sec
2.5 GB = 2,500,000,000 bytes = 20,000,000,000 bits
R/T = 1.5 min = 90 sec
20,000,000,000 / 90 = 222,222,222.22 bits/sec
How did you do that? |
|
|
| Back to top |
|
|
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
