SFT - Smart File Transfer
by Markus Laakso
SFT, Smart File Transfer, is an application developed by VTT Information
Technology for automatic transfer and distribution of files over
mobile networks. The need for this emerged from the Finnish Maritime
Administration as the Finnish icebreaker fleet had problems in
transferring data to and from the icebreakers. Reliable data transfer
is essential for optimal fleet management. Using SFT, the nine
Finnish icebreakers get satellite images and ice charts daily
and use this information in combination with information about
the ship traffic to co-ordinate the assistance of ships through
ice. The most cost-effective network in the areas of operation
is still the NMT-450 analogue cellular network. In the Gulf of
Finland GSM-data has been used and the newest multi-purpose icebreakers
are equipped with satellite communication facilities.
Using SFT, the information is automatically distributed to the
icebreakers. Controlled by its parameters, SFT scans predefined
input directories and transfers any files found in these directories
to the remote nodes and vice versa. As the connection is unreliable,
and may break in the middle of a file transfer, SFT uses an acknowledging
protocol with resendings if needed, to guarantee that a file,
once queued for sending, is received once and only once at the
receiving node.
SFT uses standard transfer protocols on top of which additional
layers have been built. In the present implementation, RCP (Remote
Copy) has been used for file transfer, but could be substituted
by FTP or some secure transfer protocol if needed. In this way
standard routers, with dial-on-demand features, can be used for
establishing the connection.
The applications using the services provided by SFT, may have
different types of transfer requirements: small files related
to operational information should be transferred with as little
delay as possible, but maps and satellite images are large and
do not have as stringent delay requirements. SFT implements two
types of queues: a priority queue and an ordinary queue. Files
in the priority queue are sent before files in the ordinary queue.
In addition, new priority files suspend the transfer of an ordinary
file. This multiplexing makes it possible to send even large files
as background transfer without blocking the channel for time sensitive
priority files.
Automatic file compression
To save transfer time, the files are compressed before sending
using lossless compression (Lempel - Ziw) and expanded at the
reception. This enables simple formats to be used by different
applications without the need of separate compression prior to
sending. To guarantee that the file is correctly reassembled an
MD5 checksum algorithm is used to check that the original file
has been transferred unchanged.
Because SFT is used unattended 24 h/day 7days/week, special logfiles
are automatically created, which gives information about the performance
of the system as well as trace information in case of unexpected
errors.
To summarise, SFT atomises the information exchange between the
co-ordinating centre and the icebreakers providing reliable unattended
file transfer over unreliable connections. Although the use of
standard protocols does not necessarily maximise the net transfer
rate, the robustness and modularity of this approach are considered
to be more valuable for operational use than maximised channel
capacity utilisation.
Please contact:
Markus Laakso - VTT Information Technology
Tel: +358 9 456 4568
E-mail: Markus.Laakso@vtt.fi
