Monitoring and Displaying Traffic on the World Wide Web
by Evangelos P. Markatos and Athanasios E. Papathanasiou
Developed by the Computer Architecture and VLSI Group at the Institute
of Computer Science - FORTH, Palantir is an application that can
be used to display the origin, volume and type of the incoming
requests of a web server. A good knowledge of the geographic distribution
and access patterns of the clients creating these requests may
indicate a more efficient way of erving them.
World-Wide Web traffic continues to increase at impressive rates.
Busy web servers may get as many as several millions of hits (accesses)
in a day. Accesses may originate from all over the world and may
result in a ‘rush hour’ that lasts 24-hours per day. Web traffic
will probably continue to increase as more people gain access
and new applications (including commercial ones) are emerging.
To meet the demands of this ever-increasing traffic, webmasters
should design their web servers in such a way as to disseminate
information (and sell or advertise products) effectively and reliably.
A first step towards effective information dissemination is understanding
a web server’s client base, and reaching out to it. Palantir is
a web traffic monitor and visualization tool that can be used
to display the origin, volume and type of the incoming requests
of a web server. A good knowledge of the geographic distribution
and access patterns of the clients creating these requests may
indicate a more efficient way of serving them.
Palantir reads the log files of a web server and after categorizing
the requests into six major types (text, graphics, audio, video,
scientific, and other - that is any other kind of request) displays
the origin of the requests on a geographical landscape (in pictorial
form). The geographical origin of the request is found by comparing
its IP address to databases of information about geographical
locations of the address. In the case of addresses registered
in the US, the addresses are compared to registry databases and
then that information in compared against a zipcode database to
determine the geographic origination of the address. The type
and magnitude of requests that originate from each region are
shown in the map either stacked bars or as concentric circles.
Different types of requests are easily identified by being displayed
with different colors.
Palantir can animate the web traffic in static mode or in dynamic
mode. In the first case, the requests, which have occurred during
a specific period of time and are contained in the selected log
files, are animated in the viewer. Each request remains displayed
until the end of the simulation (it has an unlimited time life).
Thus, the stacked bars (or the concentric circles) present the
total amount of requests cumulatively (summary of traffic over
a specified period). In the Dynamic mode, Palantir’s viewer tries
to capture the instant traffic of requests. Each request, contained
in the log file, is considered to have a limited time life. As
time passes, new requests are displayed on the viewer, while those
that have exceeded their time life (old requests) are deleted.
In this way a visualization of the way that network traffic varies
with time is given! Moreover, Palantir checks continually the
log file to display new incoming requests at real-time.
To facilitate its use, Palantir provides an graphical interface
with several functions most important of which are:
- aggregation: requests originating from several predefined (or
user-defined) regions are aggregated into a single stacked bar
(or concentric cycle)
- zooming: Palantir may zoom in a specific location in order to
study more effectively the traffic that originates from a particular
geographic region.
- filtering: Palantir provides two kinds of filters: Domain Filter
and Request Filter. The Domain Filter is used to display only
requests that come from a specified domain, while the Request
Filter displays only those requests that ask for a specific kind
of files.
Palantir is completely written in Java, mainly for portability
reasons and may be accessed though its Home Page at ICS-FORTH:
http://archvlsi.ics.forth.gr/OS/www.html.
A Palantir server is currently running at http://sappho.ics.forth.gr:9000,
which may be freely used for downloading Log files and viewing
Network Traffic Visualizations.
Current plans for Palantir include visualizing a domain’s outgoing
requests. A simple way for achieving this is by monitoring outgoing
TCP packets that have as a destination port the 80 port (default
port for http requests) of a host.
Please contact
Evangelos P. Markatos - ICS-FORTH
Tel: +30 81 391655
E-mail: markatos@ics.forth.gr
Athanasios E. Papathanasiou - ICS-FORTH
Tel: +30 81 391437
E-mail: papathan@ics.forth.gr
