
	Digest delivery

------------------------------------------------------------
1	Overview
1.1	What fml administrator needs to do
1.2	What the user to do
1.3	Cron (FML Vixie cron compatible program, bin/cron.pl)
1.3	Cron (FML Vixie cron compatible program, bin/cron.pl)
1.4	Vixie Cron
1.5	How To Install cron.pl

2	Digest Delivery / Matome Okuri (Japanese term)
2.1	Mechanism Of Digest Delivery
2.2	Digest Delivery Server (msend.pl)
2.3	Example of crontab
2.4	Control file of msend.pl $MSEND_RC(/var/log/msendrc)
2.5	Command To Control Each User Digest Delivery Configuration
2.6	Subject of Digest
2.7	Available Hooks In Digest Delivery
2.8	Digest Delivery Default Configuration
2.9	"digest" command parameters
2.10	Notify that "no digest is delivered today" to members
2.11	To add or not add X-ML-Info:
2.12	How msend.pl works
2.13	Modify when msend.pl should send articles?
2.14	Digest only mailing list
2.15	About bin/cron.pl
2.16	Digest Once In A Week
2.17	Cron (FML Vixie cron compatible program, bin/cron.pl)
2.17	Cron (FML Vixie cron compatible program, bin/cron.pl)
2.18	Vixie Cron
2.19	How To Install cron.pl
------------------------------------------------------------


1	Overview

digest is "send articles once N hours" not in realtime.
/usr/local/fml/msend.pl doed this service.  Please run this program
periodically for example by cron(8).  cron is built-in in UNIX
usually.

	% crontab /var/spool/ml/elena/crontab

1.1	What fml administrator needs to do

inject /var/spool/ml/elena/crontab to crontab by crontab(1).
/var/spool/ml/elena/crontab is created by "makefml newml" command.

% makefml newml elena
% crontab /var/spool/ml/elena/crontab

1.2	What the user to do

	matome 3mp

Each user sends "matome option" command to elena-ctl@fml.org (in
"elena" ML case). Here "option" is "hour" + "fop".  "fop" is file
format in compression.

See /var/spool/ml/elena/help for more details.

To back again to real time delivery, send "matome 0" to control
address.

	matome 0

1.3	Cron (FML Vixie cron compatible program, bin/cron.pl)
1.3	Cron (FML Vixie cron compatible program, bin/cron.pl)

	man cron
	man crontab

The present OS may have a cron for users. Please check it in the
first.  e.g. man cron, man crontab If only root can set up OS's cron
so a general user cannot control cron, you can use FML cron (clone of
4.4BSD cron).

FML cron sleeps and runs once a minute.  If the crontab entry has a
job now, cron run it.  It is same as UNIX cron daemon.

You can run cron manually. fml.pl can run cron.pl automatically. If
fml.pl automatically runs cron.pl, it runs each cron.pl for each
ML. It is of no use. It is enough that one cron.pl for one user.

fml.pl runs only one cron.pl, not plural for one ML. fml.pl checks
$CRON_PIDFILE ($DIR/var/run/cron.pid) to ensure only one cron.pl
running.

cron.pl may be down. OS does not run cron.pl automatically, so 
digest delivery does not run until you are aware of it. 
For it, you can set up fml.pl runs cron.pl when mail comes in.

In default cron.pl itself exits, and new cron.pl runs to avoid strange
behavior when a process runs long long time.
However -n option enables infinite running.

SYNOPSIS
	cron.pl [-a] [-d] [-b43] [-fcrontab] [-bd] $DIR [$LIBDIR]

    -d                debug mode 
    -a                run eternally(default: 180sec. = 60sec. * 3times) 
    -n                working all times without RESTART
    -mtimes           run from now to (60 * times) sec. after 
    -fcrontab-file    alternative crontab 
    -h                show this help and exit 
    -bd		      daemon (Become Daemon) 
    -bOSTYPE          -b43(OSTYPE = 43): 4.3BSD Like. not mailed to you 

-b43 option set

	$CRON_NOTIFY    = 0; (default 1)

In default, cron.pl notify the cron job log to the owner of the
process by mail (like 4.4BSD cron).

Also in default, cron.pl does not detach the current tty, that is, not
become a daemon since becoming daemon does not work well on an OS.

	-onotty

cron.pl becomes daemon by itself. In default

	$NOT_USE_TIOCNOTTY = 1;		# no ioctl

1.4	Vixie Cron

4.4BSD has Vixie Cron, so you can use it for yourself. Please use it
on 4.4BSD. Also firstly you should check your OS's manuals. 

    cron (8) - daemon to execute scheduled commands (Vixie Cron)
    crontab (1) - maintain crontab files for individual users (V3)
    crontab (5) - tables for driving cron

An old OS has no such cron. However an administrator installed Vixie
cron. It is of value to trek your system:-)

cron.pl is a clone for such an old OS.

1.5	How To Install cron.pl

The system in the bootstrap time does not run cron.pl automatically.
We would like to run cron.pl anyway. What should we do?

To run cron.pl, the arguments are

	% cron.pl $DIR(config.ph location) $LIBDIR(lib*.pl location)

The problem is how to run cron!.  For example, you can set up the
following ways:

   1. 	set up your ~/.login which run cron.pl. When you log in the
	system, you run cron.pl. Cron.pl always checks other cron.pl 
	runs or not, so	only one cron.pl runs.

   2. runs cron.pl manually :)

   3. fml.pl runs cron.pl when mail comes in.

config.ph example for case 3 is 

	$CRON_PIDFILE = "$DIR/var/run/cron.pid";
	$CRONTAB      = "$DIR/etc/crontab";

	$FML_EXIT_PROG = "$LIBDIR/bin/cron.pl $DIR $LIBDIR -a";

cron.pid ($DIR/var/run/cron.pid) corresponds to /var/run/cron.pid but
exists for each ML under FML.

When you run "makefml newml", FML sets up each crontab for each ML.
It is an example for Vixie cron.  You can use it on 4.4BSD and under
Vixie cron.

[Example]
	% makefml newml elena
	  ...
	% cd /var/spool/ml/elena
	% crontab crontab

2	Digest Delivery / Matome Okuri (Japanese term)

2.1	Mechanism Of Digest Delivery

Digest delivery process works as follows:

   for (;;) {	# infinite loop
	wait for the digest delivery time

	check $ACTIVE_LIST and $MSEND_RC to determine
	"whom to send which range of articles" and "now"?
	If msend.pl has members to be sent a bundle of articles,
	msend.pl sends it to them.
   }

This periodic sending mechanism is digest delivery. This mechanism
clearly requires some help from OS. Cron on some systems (OS) kicks
digest delivery program periodically. Also you need to set up the
setting in your OS. We use "cron", standard service of UNIX.

Each user controls its digest delivery parameters, "when" and "how
articles are packed". FML provides commands for the control, "digest"
and "matome" (Japanese term), described below.

2.2	Digest Delivery Server (msend.pl)

SYNOPSIS
   msend.pl    [ options ] $DIR [ options ]  [ $LIBDIR [ options ] ]

   * fml.pl, msend.pl and fmlserv.pl have the same argument syntax.

   -d		debug mode
   -bt		test mode

   -bd		daemon mode (become daemon)
   -sOPT $OPT	$OPT = 1;
   -uOPT $OPT	$OPT = 0;
   -lfile	$LOAD_LIBRARY = "file";
   -q		quite mode
		if not quiet mode, msend.pl runs with showing 
		a lot of information.

2.3	Example of crontab

"makefml newml <ML>" command generates examples which include crontab
example. Let $DIR be the ML's home directory. You need to set up cron
kicks msend.pl periodically since system(OS) does not work for you in
default. To set up cron, do

   (shell prompt)% crontab $DIR/crontab 

An example of crontab (4.4BSD) is

0 * * * * /usr/libexec/fml/msend.pl /var/spool/ml/Elena /usr/libexec/fml

where msend.pl runs at 0 minute in each hour.

2.4	Control file of msend.pl $MSEND_RC(/var/log/msendrc)

You need to define $MSEND_RC (defined as $DIR/var/log/msendrc in
default). If it does not exist, msend.pl warns it and exits.  You
should know you can run msend.pl. This policy comes from against the
possibility that msend.pl may send a big bundle of articles if the
configuration has an error.

2.5	Command To Control Each User Digest Delivery Configuration

Each user can control each digest delivery configuration by a command.
e.g.
	digest 3u
	matome 3u
which sets up digest delivery is sent once 3 hours with UNIX FROM
format(PLAIN-TEXT). Please see help file for available options.

	matome 3u

2.6	Subject of Digest

	$MSEND_SUBJECT_TEMPLATE

is a template for Subject: of digest mails. It is defined as

    Digest _ARTICLE_RANGE_ _PART_ _ML_FN_

in default. Each variable (_\S+_ style) is converted to in the
following way

	   _ARTICLE_RANGE_  <=>    Article 768
	   _DOC_MODE_       <=>    [last:10 tar + gzip]
	   _PART_           <=>    (1/4)
	   _ML_FN_          <=>    $ML_FN (here is "(Elena ML)") 

These variables control Subject:. So you can customize Subject: for
you. In default the subject: after this conversion will be like

   Digest Article 768 (1/1) (Elena ML)

2.7	Available Hooks In Digest Delivery

$MSEND_START_HOOK is evaluated after initialization but before the
main routine.

$MSEND_HEADER_HOOK is evaluated in the main routine and just before
Subject: substitution described above.

2.8	Digest Delivery Default Configuration

The default file format is gzip-ed file format (for highly compression
efficiency). You can change the default by the variable

	$MSEND_MODE_DEFAULT

where a set of available parameters is the same as ones for
$MGET_MODE_DEFAULT (xref: internals 5.2).

* A few shortcuts 

(After config.ph CFVersion 3)

	$MSEND_MODE_DEFAULT = "rfc1153"    (default RFC1153)
	$MSEND_MODE_DEFAULT = "rfc934"	   (default RFC934)

Example: change the default to RFC934 style (e.g. You can split the
format mail up by MH burst).

	$MSEND_MODE_DEFAULT = "rfc934";

	------- Forwarded Message

	From: uja
	Subject: daa 

	Books by Takemoto Izumi are wonderful.

	------- Forwarded Message

	From: uja
	Subject: daa

	Aoichan Panic is good, but Seifukumono is also good :D

	--fukachan@ Kerochan is good

	------- Forwarded Message

To change the default to RFC1153 style.

	$MSEND_MODE_DEFAULT = "rfc1153";

FML does not check the sequence number since each user is each user
parameters, time, file format ... and the time parameter can change.
We cannot control them. Hence the sequence number is not maintained as
unique.
To hack RFC1153, see librfc1153.pl.

2.9	"digest" command parameters

FML checks modulus between the time and parameters.  "digest" command
fundamental syntax is

digest N 		If N is 0-23, a digest is delivered once N hours.
			If N is 13-36, a digest is delivered once a day
			at N hour. 

After N, file option may be added. If without a file format option,
the file format is "gzip", which is gzip-ped file containing articles.

	u	PLAIN-TEXT(UNIX FROM)
	uf	PLAIN-TEXT(UNIX FROM)
	i	lha + ish 
	ish	lha + ish 
	mp	MIME/Multipart	PLAIN-TEXT
	d	RFC1153 format 	PLAIN-TEXT
	b	RFC934 format 	PLAIN-TEXT(mh-burst)
	rfc934	
	ish	lha + ish 
	lhaish
	i
	li
	lu	lha + uuencode
	lhauu
	unpack	PLAIN-TEXT(UNIX FROM)

digest 0		real time delivery (stop digest)

   matome 1		once in an hour  (file format is gzipd)
   matome 2		once in two hours (file format is gzipd)
   matome 2u		once in two hours (file format is unix from)
   matome 2mp		once in two hours (file format is MIME/Multipart)
   matome 17		once in a day (send at 17:00)
   matome 29		once in a day (send at 05:00; 29 = 24 + 5 )

2.10	Notify that "no digest is delivered today" to members

	$MSEND_NOTIFICATION = 1;

FML sends "no digest today for no article" to members of the ML.

	$MSEND_NOTIFICATION_SUBJECT (default "Notification $ML_FN")

is Subject: of the notification mail.

2.11	To add or not add X-ML-Info:

	$MSEND_NOT_USE_X_ML_INFO = 1;

disables X-ML-Info: in the header of digest mails.

2.12	How msend.pl works

Let actives (delivery list) be like this.

	-------- actives --------
	katri
	pollyanna	m=6u
	-------- actives --------

	* cron kicks msend.pl once in an hour.

	* checks delivery list (here actives) and find 
	  "pollyanna" hopes FML sends a pack of articles once in 6 hours
	   with the file format "u"(UNIX FROM style plain text).

	* checks whether the present hour % 6 is 0 or not.
	  If the modulus is 0, msend.pl should sent articles to "pollyanna".
	  If not, skip this user entry.

	* logs the largest number of articles in $MSEND_RC for the next time.
	  The range of sending articles is 
	  from the number for "pollyanna" in $MSEND_RC
	  to the current sequence number ($SEQUENCE_FILE).  

Conclusion: FML does not send stored articles somewhere but generates
mail with a bundle of articles for a user and sends it to him/her each
time. It enables the customization for each user. The degree of
freedom is the same as that of "mget" command. 

2.13	Modify when msend.pl should send articles?

Please hack "sub MSendP" (msend-predicate) in msend.pl.

Example: current code(sending if return value is 1)

    # CHECK: Modulus 
    if (($ID - $Request{$who}) >= 0) {
	return 1 if (0 == ($Hour % $When{$who}));
	return 1 if ($When{$who} > 12 && ($Hour + 24 == $When{$who}));
    }

    return 0; # not send

where 
	$ID		sequence number ($SEQUENCE_FILE)
	$Hour		the present time (hour)
	$who		user mail address
	$When{$who}	user's parameter (e.g. 3, 6, 12, 17 ...)

2.14	Digest only mailing list

Please see contrib/msend-anytime patches.

Apply this patch and run "msend.pl -a -A other-options".

	$MSEND_MODE_DEFAULT 

$MSEND_MODE_DEFAULT is the default value (e.g. 3mp) of digest delivery
parameter.

2.15	About bin/cron.pl

(^^)

2.16	Digest Once In A Week

See bin/weekend-msend.sh for an example.

2.17	Cron (FML Vixie cron compatible program, bin/cron.pl)
2.17	Cron (FML Vixie cron compatible program, bin/cron.pl)

	man cron
	man crontab

The present OS may have a cron for users. Please check it in the
first.  e.g. man cron, man crontab If only root can set up OS's cron
so a general user cannot control cron, you can use FML cron (clone of
4.4BSD cron).

FML cron sleeps and runs once a minute.  If the crontab entry has a
job now, cron run it.  It is same as UNIX cron daemon.

You can run cron manually. fml.pl can run cron.pl automatically. If
fml.pl automatically runs cron.pl, it runs each cron.pl for each
ML. It is of no use. It is enough that one cron.pl for one user.

fml.pl runs only one cron.pl, not plural for one ML. fml.pl checks
$CRON_PIDFILE ($DIR/var/run/cron.pid) to ensure only one cron.pl
running.

cron.pl may be down. OS does not run cron.pl automatically, so 
digest delivery does not run until you are aware of it. 
For it, you can set up fml.pl runs cron.pl when mail comes in.

In default cron.pl itself exits, and new cron.pl runs to avoid strange
behavior when a process runs long long time.
However -n option enables infinite running.

SYNOPSIS
	cron.pl [-a] [-d] [-b43] [-fcrontab] [-bd] $DIR [$LIBDIR]

    -d                debug mode 
    -a                run eternally(default: 180sec. = 60sec. * 3times) 
    -n                working all times without RESTART
    -mtimes           run from now to (60 * times) sec. after 
    -fcrontab-file    alternative crontab 
    -h                show this help and exit 
    -bd		      daemon (Become Daemon) 
    -bOSTYPE          -b43(OSTYPE = 43): 4.3BSD Like. not mailed to you 

-b43 option set

	$CRON_NOTIFY    = 0; (default 1)

In default, cron.pl notify the cron job log to the owner of the
process by mail (like 4.4BSD cron).

Also in default, cron.pl does not detach the current tty, that is, not
become a daemon since becoming daemon does not work well on an OS.

	-onotty

cron.pl becomes daemon by itself. In default

	$NOT_USE_TIOCNOTTY = 1;		# no ioctl

2.18	Vixie Cron

4.4BSD has Vixie Cron, so you can use it for yourself. Please use it
on 4.4BSD. Also firstly you should check your OS's manuals. 

    cron (8) - daemon to execute scheduled commands (Vixie Cron)
    crontab (1) - maintain crontab files for individual users (V3)
    crontab (5) - tables for driving cron

An old OS has no such cron. However an administrator installed Vixie
cron. It is of value to trek your system:-)

cron.pl is a clone for such an old OS.

2.19	How To Install cron.pl

The system in the bootstrap time does not run cron.pl automatically.
We would like to run cron.pl anyway. What should we do?

To run cron.pl, the arguments are

	% cron.pl $DIR(config.ph location) $LIBDIR(lib*.pl location)

The problem is how to run cron!.  For example, you can set up the
following ways:

   1. 	set up your ~/.login which run cron.pl. When you log in the
	system, you run cron.pl. Cron.pl always checks other cron.pl 
	runs or not, so	only one cron.pl runs.

   2. runs cron.pl manually :)

   3. fml.pl runs cron.pl when mail comes in.

config.ph example for case 3 is 

	$CRON_PIDFILE = "$DIR/var/run/cron.pid";
	$CRONTAB      = "$DIR/etc/crontab";

	$FML_EXIT_PROG = "$LIBDIR/bin/cron.pl $DIR $LIBDIR -a";

cron.pid ($DIR/var/run/cron.pid) corresponds to /var/run/cron.pid but
exists for each ML under FML.

When you run "makefml newml", FML sets up each crontab for each ML.
It is an example for Vixie cron.  You can use it on 4.4BSD and under
Vixie cron.

[Example]
	% makefml newml elena
	  ...
	% cd /var/spool/ml/elena
	% crontab crontab


		INDEX

-b43                                       ...   1.3 2.17 
bin/cron.pl                                ...   1.3 2.17 
cron.pl                                    ...   1.3 2.17 
$CRON_NOTIFY                               ...   1.3 2.17 
$CRON_PIDFILE                              ...   1.5 2.19 
$CRONTAB                                   ...   1.5 2.19 
crontab (4.4BSD)                           ...   2.3 
digest                                     ...   2 
$DIR/crontab                               ...   1.3 2 2.3 2.17 
$DISTRIBUTE_CLOSE_HOOK                     ...   1.5 2.19 
etc/crontab                                ...   1.3 2.17 
ISC (Internet Software Consortium)         ...   1.4 2.18 
msend                                      ...   2 
msend.pl                                   ...   2.2 
$MSEND_DEFAULT_SUBJECT                     ...   2.6 
$MSEND_DEFAULT_SUBJECT (CFv2)              ...   2.6 
$MSEND_HEADER_HOOK                         ...   2.7 
$MSEND_MODE_DEFAULT                        ...   2.8 
$MSEND_NOT_USE_X_ML_INFO                   ...   2.11 
$MSEND_NOTIFICATION                        ...   2.10 
$MSEND_NOTIFICATION_SUBJECT                ...   2.10 
$MSEND_RC                                  ...   2.4 
$MSEND_START_HOOK                          ...   2.7 
$MSEND_SUBJECT_TEMPLATE                    ...   2.6 
MSendrc -> var/log/msendrc                 ...   2.4 
MSendv4.pl                                 ...   2.2 
$NO_ML_INFO                                ...   2.11 
$NOT_SHOW_DOCMODE                          ...   2.6 
$NOT_USE_TIOCNOTTY                         ...   1.3 2.17 
-onotty                                    ...   1.3 2.17 
P.Vixie                                    ...   1.4 2.18 
RFC1153                                    ...   2.8 
$USE_RFC1153                               ...   2.8 
$USE_RFC1153_DIGEST                        ...   2.8 
$USE_RFC934                                ...   2.8 
