MIRMON(1) User Contributed Perl Documentation MIRMON(1) NNAAMMEE mirmon - monitor the state of mirrors SSYYNNOOPPSSIISS mirmon [ -v ] [ -q ] [ -t timeout ] [ -get opt ] [ -c conf ] OOPPTTIIOONNSS option v : be verbose option q : be quiet option t : set timeout [ default 300 ] ; option get : ’all’ : probe all sites : ’update’ : probe a selection of the sites (see doc) option c : configuration file ; default list : ./mirmon.conf $HOME/.mirmon.conf /etc/mirmon.conf ------------------------------------------------------------------- Mirmon normally only reports errors and changes in the mirror list. ------------------------------------------------------------------- UUSSAAGGEE The program is intended to be run by cron every hour. 42 * * * * perl /path/to/mirmon -get update It quietly probes a subset of the sites in a given list, writes the results in the ’state’ file and generates a web page with the results. The subset contains the sites that are new, bad and/or not probed for a specified time. When no ’get’ option is specified, the program just generates a new web page from the last known state. The program checks the mirrors by running a (user specified) program on a pipe. A (user specified) number of probes is run in parallel using nonblocking IO. When something can be read from the pipe, it switches the pipe to blocking IO and reads one line from the pipe. Then it flushes and closes the pipe. No attempt is made to kill the probe. The probe should return something that looks like 1043625600 ... that is, a line of text starting with a timestamp. The exit status of the probe is ignored. CCOONNFFIIGG FFIILLEE llooccaattiioonn A config file can be specified with the -c option. If -c is not used, the program looks for a config file in * ..//mmiirrmmoonn..ccoonnff * $$HHOOMMEE//..mmiirrmmoonn..ccoonnff * //eettcc//mmiirrmmoonn..ccoonnff ssyynnttaaxx A config file looks like this : +-------------------------------------------------- │# lines that start with ’#’ are comment │# blank lines are ignored too │# tabs are replaced by a space │ │# the config entries are ’key’ and ’value’ pairs │# a ’key’ begins in column 1 │# the ’value’ is the rest of the line │somekey A_val B_val ... │otherkey X_val Y_val ... │ │# indented lines are glued │# the next three lines mean ’somekey part1 part2 part3’ │somekey part1 │ part2 │ part3 │ │# lines starting with a ’+’ are concatenated │# the next three lines mean ’somekey part1part2part3’ │somekey part1 │+ part2 │+ part3 │ │# lines starting with a ’.’ are glued too │# don’t use a ’.’ on a line by itself │# ’somekey’ gets the value "part1\n part2\n part3" │somekey part1 │. part2 │. part3 +-------------------------------------------------- CCOONNFFIIGG FFIILLEE :: rreeqquuiirreedd eennttrriieess pprroojjeecctt__nnaammee _n_a_m_e Specify a short plaintext name for the project. project_name Apache project_name CTAN pprroojjeecctt__uurrll _u_r_l Specify an url pointing to the ’home’ of the project. project_url http://www.apache.org/ mmiirrrroorr__lliisstt _f_i_l_e_-_n_a_m_e Specify the file containing the mirrors to probe. mirror_list /path/to/mirror-list If your mirror list is generated by a program, use mirror_list /path/to/program arg1 ... │ Two formats are supported : * plain : lines like us http://www.tux.org/ [email] ... nl http://apache.cs.uu.nl/dist/ [email] ... nl rsync://archive.cs.uu.nl/apache-dist/ [email] ... * apache : lines like those in the apache mirrors.list ftp us ftp://ftp.tux.org/pub/net/apache/dist/ user@tux.org ... http nl http://apache.cs.uu.nl/dist/ user@cs.uu.nl ... Note that in style ’plain’ the third item is reserved for an optional email address : the site’s contact address. Specify the required format with ’list_style’ (see below). The default style is ’plain’. wweebb__ppaaggee _f_i_l_e_-_n_a_m_e Specify where the html report page is written. iiccoonnss _d_i_r_e_c_t_o_r_y_-_n_a_m_e Specify the directory where the icons can be found, relative to the _w_e_b___p_a_g_e, or relative to the DOCUMENTROOT of the web server. If/when the _w_e_b___p_a_g_e lives in directory ".../mirmon/" and the icons live in directory ".../mirmon/icons/", specify icons icons If/when the icons live in "/path/to/DOCUMENTROOT/icons/mirmon/", spec- ify icons /icons/mirmon pprroobbee _p_r_o_g_r_a_m _+ _a_r_g_u_m_e_n_t_s Specify the program+args to probe the mirrors. Example: probe /usr/bin/wget -q -O - -T %TIMEOUT% -t 1 %URL%TIME Before the program is started, %TIMEOUT% and %URL% are substituted with the proper timeout and url values. Here it is assumed that each hour the root server writes a timestamp in /path/to/archive/TIME, for instance with a crontab entry like 42 * * * * perl -e ’printf "%s\n", time’ > /path/to/archive/TIME Mirmon reads one line of output from the probe and interprets the first word on that line as a timestamp ; for example : 1043625600 1043625600 Mon Jan 27 00:00:00 2003 1043625600 www.apache.org Mon Jan 27 00:00:00 2003 Mirmon is distributed with a program "probe" that handles ftp, http and rsync urls. ssttaattee _f_i_l_e_-_n_a_m_e Specify where the file containing the state is written. The program reads this file on startup and writes the file when mir- rors are probed (-get is specified). ccoouunnttrriieess _f_i_l_e_-_n_a_m_e Specify the file containing the country codes; The file should contain lines like us - united states nl - netherlands The mirmon package contains a recent ISO list. CCOONNFFIIGG FFIILLEE :: ooppttiioonnaall eennttrriieess mmaaxx__pprroobbeess _n_u_m_b_e_r Optionally specify the number of parallel probes (default 25). ttiimmeeoouutt _s_e_c_o_n_d_s Optionally specify the timeout for the probes (default 300). After the last probe is started, the program waits for + 10 seconds, cleans up and exits. pprroojjeecctt__llooggoo _l_o_g_o Optionally specify (the SRC of the IMG of) a logo to be placed top right on the page. project_logo /icons/apache.gif project_logo http://www.apache.org/icons/... hhttmm__hheeaadd _h_t_m_l Optionally specify some HTML to be placed before . htm_head hhttmm__ttoopp _h_t_m_l Optionally specify some HTML to be placed near the top of the page. htm_top testing 1, 2, 3 hhttmm__ffoooott _h_t_m_l Optionally specify HTML to be placed near the bottom of the page. htm_foot
ppuutt__hhiissttoo ttoopp││bboottttoomm││nnoowwhheerree Optionally specify where the age histogram must be placed. The default is ’top’. mmiinn__ppoollll _t_i_m_e_-_s_p_e_c For ’min_poll’ see next item. A _t_i_m_e_-_s_p_e_c is a number followed by a unit ’s’ (seconds), or ’m’ (minutes), or ’h’ (hours), or ’d’ (days). For example ’3d’ (three days) or ’36h’ (36 hours). mmaaxx__ppoollll _t_i_m_e_-_s_p_e_c Optionally specify the maximum probe interval. When the program is called with option ’-get update’, all sites are probed which are : * new the site appears in the list, but there is no known state * bad the last probe of the site was unsuccessful * old the last probe was more than ’max_poll’ ago. Sites are not probed if the last probe was less than ’min_poll’ ago. So, if you specify min_poll 4h max_poll 12h the ’reachable’ sites are probed twice daily and the ’unreachable’ sites are probed at most six times a day. The default ’min_poll’ is ’1h’ (1 hour). The default ’max_poll’ is ’4h’ (4 hours). mmiinn__ssyynncc _t_i_m_e_-_s_p_e_c Optionally specify how often the mirrors are required to make an update. The default ’min_sync’ is ’1d’ (1 day). mmaaxx__ssyynncc _t_i_m_e_-_s_p_e_c Optionally specify the maximum allowable sync interval. Sites exceeding the limit will be considered ’old’. The default ’max_sync’ is ’2d’ (2 days). nnoo__rraannddoommiizzee To balance the probe load over the hourly mirmon runs, mirmon may probe a few extra randomly choosen mirrors : * only if the the number of mirrors to probe is below average, * at most 2% of the mirrors If you don’t want this behaviour, use nnoo__rraannddoommiizzee. nnoo__aadddd__ssllaasshh If the url part of a line in the mirror_list doesn’t end in a slash (’/’), mirmon adds a slash and issues a warning unless it is in quiet mode. If you don’t want this behaviour, use nnoo__aadddd__ssllaasshh. lliisstt__ssttyyllee ppllaaiinn││aappaacchhee Optionally specify the format (’plain’ or ’apache’) of the mir- ror-list. See the description of ’mirror_list’ above. The default list_style is ’plain’. ssiittee__uurrll _s_i_t_e _u_r_l Optionally specify a substitute url for a site. When access to a site is restricted (in Australia, for instance), another (sometimes secret) url can be used to probe the site. The of an url is the part between ’://’ and the first ’/’. eennvv _k_e_y _v_a_l_u_e Optionally specify an environment variable. iinncclluuddee _f_i_l_e_-_n_a_m_e Optionally specify a file to include. The specified file is processed ’in situ’. After the specified file is read and processed, config processing is resumed in the file where the "include" was encountered. The include depth is unlimited. However, it is a fatal error to include a file twice under the same name. sshhooww When the config processor encounters the ’show’ command, it dumps the content of the current config to standout, if option "-v" is speci- fied. This is intented for debugging. eexxiitt When the config processor encounters the ’exit’ command, it terminates the program. This is intented for debugging. SSTTAATTEE FFIILLEE FFOORRMMAATT The state file consists of lines; one line per site. Each line con- sists of white space separated fields. The seven fields are : * field 1 : url The url as given in the mirror list. * field 2 : age The mirror’s timestamp found by the last succesful probe, or ’undef’ if no probe was ever successful. * field 3 : status last probe The status of the last probe, or ’undef’ if the mirror was never probed. * field 4 : time last succesful probe The timestamp of the last succesful probe or ’undef’ if the mirror was never successfully probed. * field 5 : probe history The probe history is a list of ’s’ (for success) and ’f’ (for failure) characters indicating the result of the probe. New results are appended whenever the mirror is probed. * field 6 : state history The state history consists of a timestamp, a ’-’ char, and a list of chars indicating a past status: ’s’ (fresh), ’b’ (oldish), ’f’ (old), ’z’ (bad) or ’x’ (skip). The timestamp indicates when the state history was last updated. The current status of the mirror is determined by the mirror’s age and a few configuration parame- ters (min_sync, max_sync, max_poll). The state history is updated when the mirror is probed. If the last update of the history was less than 24 hours ago, the last status is replaced by the current status. If the last update of the history was more than 24 hours ago, the current status is appended to the history. One or more ’skip’s is inserted, if the timestamp is two or more days old (when mirmon hasn’t run for more than two days). * field 7 : last probe The timestamp of the last probe, or ’undef’ if the mirror was never probed. IINNSSTTAALLLLAATTIIOONN ggeenneerraall * Note: The (empty) state file must exist before mirmon runs. * The mirmon repository is here : https://subversion.cs.uu.nl/repos/staff.henkp.mirmon/trunk/ * The mirmon tarball is here : http://people.cs.uu.nl/henkp/mirmon/mirmon.tar.gz iinnssttaallllaattiioonn ssuuggggeessttiioonnss To install and configure mirmon, take the following steps : * First, make the webdir : cd DOCUMENTROOT mkdir mirmon For _D_O_C_U_M_E_N_T_R_O_O_T, substitute the full pathname of the document root of your webserver. * Check out the mirmon repository : cd /usr/local/src svn checkout REPO mirmon where REPO = https://subversion.cs.uu.nl/repos/staff.henkp.mirmon/trunk/ or download the package and unpack it. * Chdir to directory mirmon : cd mirmon * Create the (empty) state file : touch state.txt * Install the icons in the webdir : mkdir DOCUMENTROOT/mirmon/icons cp icons/* DOCUMENTROOT/mirmon/icons * Create a mirror list "mirror_list" ; Use your favorite editor, or genererate the list from an existing database. nl http://archive.cs.uu.nl/your-project/ contact@cs.uu.nl uk http://mirrors.this.org/your-project/ mirrors@this.org us http://mirrors.that.org/your-project/ mirrors@that.org The email addresses are optional. * Create a mirmon config file "mirmon.conf" with your favorite editor. # lines must start in the first column ; no leading white space project_name .... project_url .... mirror_list mirror_list state state.txt countries countries.list web_page DOCUMENTROOT/mirmon/index.html icons /mirmon/icons probe /usr/bin/wget -q -O - -T %TIMEOUT% -t 1 %URL%TIME This assumes the project’s timestamp is in file "TIME". * If you have rsync urls, change the probe line to : probe perl /usr/local/src/mirmon/probe -t %TIMEOUT% %URL%TIME * Run mirmon : perl mirmon -v -get all The mirmon report should now be in ’DOCUMENTROOT/mirmon/index.html’ http://www.your.project.org/mirmon/ * If/when, at a later date, you want to upgrade mirmon : cd /usr/local/src/mirmon svn status -u svn up SSEEEE AALLSSOO mirmon.pm(3) AAUUTTHHOORR (c) 2003-2010 Henk P. Penning Computer Science Department, Utrecht University http://people.cs.uu.nl/henkp/ -- penning@cs.uu.nl mirmon-2.1 - Thu Jan 28 18:42:13 2010 ; henkp perl v5.8.5 2010-01-28 MIRMON(1)