mirmon - monitor the state of mirrors
mirmon [ -v ] [ -q ] [ -t timeout ] [ -get opt ] [ -c conf ]
option v : be verbose
option q : be quiet
option t : set timeout [ default 300 ] ;
option get : 'all' : probe all sites
: 'update' : probe a selection of the sites (see doc)
option c : configuration file ; default list :
./mirmon.conf $HOME/.mirmon.conf /etc/mirmon.conf
-------------------------------------------------------------------
Documentation : the program contains 'pod' style documentation.
-------------------------------------------------------------------
The program is intended to be run by cron every hour.
42 * * * * perl /path/to/mirmon -q -get update
It quietly probes a subset of the sites in a given list, writes the results in the 'state' file and generates a web page with the results. The subset contains the sites that are new, bad and/or not probed for a specified time.
When no 'get' option is specified, the program just generates a new web page from the last known state.
The program checks the mirrors by running a (user specified) program on a pipe. A (user specified) number of probes is run in parallel using nonblocking IO. When something can be read from the pipe, it switches the pipe to blocking IO and reads one line from the pipe. Then it flushes and closes the pipe. No attempt is made to kill the probe.
The probe should return something that looks like
1043625600 ...
that is, a line of text starting with a timestamp. The exit status of the probe is ignored.
A config file can be specified with the -c option. If -c is not used, the program looks for a config file in
A config file looks like this :
+--------------------------------------------------
|# lines that start with '#' are comment
|# blank lines are ignored too
|# tabs are replaced by a space
|
|# the config entries are 'key' and 'value' pairs
|# a 'key' begins in column 1
|# the 'value' is the rest of the line
|somekey A_val B_val ...
|otherkey X_val Y_val ...
|
|# indented lines are glued
|# the next three lines mean 'somekey part1 part2 part3'
|somekey part1
| part2
| part3
|
|# lines starting with a '+' are concatenated
|# the next three lines mean 'somekey part1part2part3'
|somekey part1
|+ part2
|+ part3
|
|# lines starting with a '.' are glued too
|# don't use a '.' on a line by itself
|# 'somekey' gets the value "part1\n part2\n part3"
|somekey part1
|. part2
|. part3
+--------------------------------------------------
Specify a short plaintext name for the project.
project_name Apache
project_name CTAN
Specify an url pointing to the 'home' of the project.
project_url http://www.apache.org/
Specify the file containing the mirrors to probe.
Two formats are supported :
us http://www.tux.org/ [email] ...
nl http://apache.cs.uu.nl/dist/ [email] ...
nl rsync://archive.cs.uu.nl/apache-dist/ [email] ...
ftp us ftp://ftp.tux.org/pub/net/apache/dist/ user@tux.org ...
http nl http://apache.cs.uu.nl/dist/ user@cs.uu.nl ...
Note that in style 'plain' the third item is reserved for an optional email address : the site's contact address.
Specify the required format with 'list_style' (see below). The default style is 'plain'.
Specify where the html report page is written.
Specify the directory where the icons can be found.
Specify the program+args to probe the mirrors. Example:
probe /usr/bin/wget -q -O - -T %TIMEOUT% -t 1 %URL%TIME
Before the program is started, %TIMEOUT% and %URL% are substituted with the proper timeout and url values.
Here it is assumed that each hour the root server writes a timestamp in /path/to/archive/TIME, for instance with a crontab entry like
42 * * * * perl -e 'printf "%s\n", time' > /path/to/archive/TIME
Mirmon reads one line of output from the probe and interprets the first word on that line as a timestamp ; for example :
1043625600
1043625600 Mon Jan 27 00:00:00 2003
1043625600 www.apache.org Mon Jan 27 00:00:00 2003
Mirmon is distributed with a program probe that handles
ftp, http and rsync urls.
Specify where the file containing the state is written.
The program reads this file on startup and writes the file when mirrors are probed (-get is specified).
Specify the file containing the country codes; The file should contain lines like
us - united states
nl - netherlands
The mirmon package contains a recent ISO list.
Optionally specify the number of parallel probes (default 25).
Optionally specify the timeout for the probes (default 300).
After the last probe is started, the program waits for <timeout> + 10 seconds, cleans up and exits.
Optionally specify (the SRC of the IMG of) a logo to be placed top right on the page.
project_logo /icons/apache.gif
project_logo http://www.apache.org/icons/...
Optionally specify some HTML to be placed before </HEAD>.
htm_head
<link REL=StyleSheet HREF="/style.css" TYPE="text/css">
Optionally specify some HTML to be placed near the top of the page. The supplied text is placed between <P> and </P>.
htm_top testing 1, 2, 3
Optionally specify HTML to be placed near the bottom of the page.
htm_foot
<HR>
<A HREF="..."><IMG SRC="..." BORDER=0></A>
<HR>
Optionally specify where the age histogram must be placed. The default is 'top'.
For 'min_poll' see next item. A time-spec is a number followed by a unit 's' (seconds), or 'm' (minutes), or 'h' (hours), or 'd' (days). For example '3d' (three days) or '36h' (36 hours).
Optionally specify the maximum probe interval. When the program is called with option '-get update', all sites are probed which are :
Sites are not probed if the last probe was less than 'min_poll' ago. So, if you specify
min_poll 4h
max_poll 12h
the 'reachable' sites are probed twice daily and the 'unreachable' sites are probed at most six times a day.
The default 'min_poll' is '1h' (1 hour). The default 'max_poll' is '4h' (4 hours).
Optionally specify how often the mirrors are required to make an update.
The default 'min_sync' is '1d' (1 day).
Optionally specify the maximum allowable sync interval.
Sites exceeding the limit will be considered 'old'. The default 'max_sync' is '2d' (2 days).
With a low probablility, mirmon probes mirrors that would otherwise not be probed. In the long run, this balances the number of mirror probes over the hourly mirmon runs. Specifically, if there are N mirrors in the list and some mirmon run would probe K sites, on average (N-K)/N extra sites will be probed.
If you don't want this behaviour, use no_randomize.
If the url part of a line in the mirror_list doesn't end in a slash ('/'), mirmon adds a slash and issues a warning unless it is in quiet mode.
If you don't want this behaviour, use no_add_slash.
Optionally specify the format ('plain' or 'apache') of the mirror-list.
See the description of 'mirror_list' above. The default list_style is 'plain'.
Optionally specify a substitute url for a site.
When access to a site is restricted (in Australia, for instance), another (sometimes secret) url can be used to probe the site. The <site> of an url is the part between '://' and the first '/'.
Optionally specify an environment variable.
Optionally specify a file to include.
The specified file is processed 'in situ'. After the specified file is
read and processed, config processing is resumed in the file where the
include was encountered.
The include depth is unlimited. However, it is a fatal error to
include a file twice under the same name.
When the config processor encounters the 'show' command, it
dumps the content of the current config to standout, if option
-v is specified. This is intented for debugging.
When the config processor encounters the 'exit' command, it terminates the program. This is intented for debugging.
The state file consists of lines; one line per site. Each line consists of white space separated fields. The seven fields are :
Note: The (empty) state file must exist before mirmon runs.
The mirmon repository is here :
REPO = https://subversion.cs.uu.nl/repos/staff.henkp.mirmon/trunk/
To set up the software, create a mirmon webdir
WEBDIR = DOCUMENTROOT/mirmon
and install mirmon in some directory MIRMON, say
MIRMON = /usr/local/src/mirmon/
Do the following steps :
cd DOCUMENTROOT mkdir mirmon
cd /usr/local/src svn checkout REPO mirmon
or download the package and unpack it.
touch state.txt
mkdir DOCUMENTROOT/mirmon/icons cp icons/* DOCUMENTROOT/mirmon/icons
nl http://archive.cs.uu.nl/your-project/ contact@cs.uu.nl uk http://mirrors.this.org/your-project/ mirrors@this.org us http://mirrors.that.org/your-project/ mirrors@that.org
The email addresses are optional.
project_name .... project_url .... mirror_list mirror_list state state.txt countries countries.list web_page WEBDIR/index.html icons icons probe /usr/bin/wget -q -O - -T %TIMEOUT% -t 1 %URL%TIME
This assumes the project's timestamp is in file TIME.
probe perl /usr/local/src/mirmon/probe -t %TIMEOUT% %URL%TIME
perl mirmon -v -get all
The mirmon report should now be in 'DOCUMENTROOT/mirmon/index.html'
http://www.your.project.org/mirmon/
cd /usr/local/src/mirmon svn status -u svn up
© 2003 Henk P. Penning, Computer Science Department, Utrecht University
mirmon-1.39 - Tue Dec 22 10:19:05 2009 ; henkp