
		utility programs

------------------------------------------------------------
1	Overview
1.1	fmlserv: listserve style interface
1.2	popfml: POP3 based ML emulator
1.3	ML via .forward
1.4	miscellaneous programs

2	libexec/fml_local.pl

2	libexec/fml_local.pl

3	Fmlserv - Listserv Style Interface -
3.1	Listserv (Or Majordomo) Style Interface (libexec/fmlserv.pl)
3.2	Install And Setup Fmlserv
3.3	Available commands
3.4	Variables

4	~/.forward and ML
4.1	Overview to use .forward 
4.2	Set Up A Virtual User
4.3	Runs A ML Under Your Account

5	Executing FML based on POP3
5.1	how to emulate elena ML based on POP3
5.4	Popfml.pl Reference 

6	Several Utilities (bin/, lib/Utilities/ ...)
6.1	analyze_mail_error.pl (replaced to libexec/mead.pl)
6.2	Archive Generator
6.3	base64decode.pl base64encode.pl
6.4	cron (bin/cron.pl)
6.5	Daemon-ize programs
6.6	daily, log analyzer 
6.7	Checking DNS
6.8	emumail.pl
6.9	Expire
6.10	WIX Language Formatter  (bin/fwix.pl)
6.11	gen_summary.pl (summary re-creator)
6.12	bin/msend_master.sh
6.13	bin/more_secure_cf.pl
6.14	Multipart Canceler Filter (bin/multipart.pl)
6.15	NEWSYSLOG SHELL VERSION (bin/newsyslog.sh)
6.16	A demo of SMTP library
6.17	initialize and change password  (bin/passwd.pl)
6.18	get mails via POP3
6.19	split and send files (bin/split_and_msend.pl)
6.20	SPOOL -> HTML Hierarchy (bin/spool2html.pl)
6.21	MH scan.format (lib/Utilities/scan.format)
6.22	to make articles by "mget" mbox style (lib/Utilities/packmbox.pl)
6.23	RMAIL -> fml.pl
6.24	Stardate (STAR TREK)
6.25	weekend-msend.sh
6.26	Append X-Stardate: (Emacs Lisp)
------------------------------------------------------------


1	Overview

1.1	fmlserv: listserve style interface

1.2	popfml: POP3 based ML emulator

1.3	ML via .forward

1.4	miscellaneous programs

2	libexec/fml_local.pl
2	libexec/fml_local.pl
*** NOT TRANSLATE THIS CHAPTER FOR AN ADDITIONAL BONUS PROGRAM ***
3	Fmlserv - Listserv Style Interface -

3.1	Listserv (Or Majordomo) Style Interface (libexec/fmlserv.pl)

Fmlserv provides listserv compatible mode. It is interface convertor.

Consider Elena ML (elena@baycity.asia) and Freekick ML
(freekick@baycity.asia).

To get articles from 1 to 10 in MIME/Multipart mode, you send "get
1-10 mp" to Elena-ctl@baycity.asia.

Fmlserv provides the same function if you send "get elena 1-10 mp" to
fmlserv@baycity.asia.  The command style for fmlserv is "command <ML>
options".  You can insert <ML> within command syntaxes. So you can
inject commands such as

	"get Freekick 1-10 mp" 	
	"get elena 1-10 mp".

   "get Freekick 1-10 mp" 	

3.2	Install And Setup Fmlserv

To set up listserv style interface for FML, please use

	/usr/local/fml/makefml fmlserv 

(path may be different). In default FML provides each command address
for each ML. But fmlserv provides listserv style. The difference is
just the interface. The internal routines are the same.

Consider fmlserv@baycity.asia serve two mailing lists.

	elena@baycity.asia	(/var/spool/elena)
	Freekick@baycity.asia	(/var/spool/Freekick)

When "makefml install", I suppose you select "fmlserv" in mode choice
(personal or group or fmlserv).
To set up "fmlserv",
	1	makefml fmlserv, which creates 
		/var/spool/fmlserv/include.
		It implies fmlserv is a kind of mailing lists.
	2	set up /etc/aliases and newaliases using
		/var/spool/fmlserv/include.

"/var/spool/fmlserv/include" is as follows:

	"|/usr/libexec/fml/libexec/fmlserv.pl /var/spool"

where the argument of fmlserv.pl is the top directory ("/var/spool")
which provides mailing list homes in "/var/spool" such as

	/var/spool/fmlserv
	/var/spool/elena
	/var/spool/Freekick
	...

There exist

	/var/spool/elena/config.ph
	/var/spool/elena/spool/
	...

	/var/spool/Freekick/config.ph
	/var/spool/Freekick/spool/
	...

	/var/spool/fmlserv/config.ph
	...

Configuration variables are that /var/spool/fmlserv is $FMLSERV_DIR,
/var/spool/fmlserv/log is $FMLSERV_LOGFILE.

3.3	Available commands

In default, fmlserv cannot provide "lists" for security.
To make "lists" available, please set

	$FMLSERV_PERMIT_LISTS_COMMAND  = 1; (default 0)

To make "which" available, please set

	$FMLSERV_PERMIT_WHICH_COMMAND  = 1; (default 0)

* fmlserv commands

	help		fmlserv help

	*** prohibited for security
	which
	lists

* all fml commands are available.

	command <ML> options

3.4	Variables

	$MAP_DB 

is a cache for 'lists' command.

4	~/.forward and ML

You can configure .forward to run ML in secret in the following
way:-).  It is same as :include: method except for that you need to
create accounts and write their ~/.forward for one another addresses
in set up ML's.

4.1	Overview to use .forward 

Sendmail evaluates ~/.forward as the same way as :include: of
/etc/aliases. You can write ~/.forward such as

"|/home/beth/rosy/ml/fml.pl /home/beth/rosy/ml /home/beth/rosy/ml" 

Sendmail injects the mail to fml.pl.

But in general .forward is for the owner, so mail to .forward ML is
delivered to the owner itself again. It creates a loop problem.
To avoid this, consider the following two ways.

*	Account to use ~/.forward is not yours.
	Set up a virtual user which has a ML name.
	This is an usual ML by .forward. The difference is only
	written in .forward or in /etc/aliases. 
	This situation is of no mean, but it may be a case that
	ISP do not permit /etc/aliases changes but you want to run ML
	on it.

* 	Account to use ~/.forward is yours.
	After a check of the header of an in-coming mail, a filtering
	program determines .forward executes ML or delivers locally to
	you.

4.2	Set Up A Virtual User

Set up a virtual user which has a ML name. This is an usual ML by
$HOME/.forward. The only difference is written in .forward or in
/etc/aliases.

$HOME/.forward is the same as $DIR/include generated by "makefml newml
<ML>".  It must be

	ML address ($MAIL_LIST)	==	virtual user
	$MAINTAINER address	==	your address

This requires your ISP provides, what is called, "shell service".

4.3	Runs A ML Under Your Account

			...

Your account receives mail. A filter run through .forward checks 
keywords in the mail to determine
	injects it to local spool ?
	runs ML1 ?
	runs ML2 ?
	...

The filtering process requires some keywords to determine which ML
this mail should be sent to. The keyword is such as "[Elena]" in
"Subject: [Elena]" and so on.

Example:

	Subject: [Elena]

If the Subject: field contains a keyword "[Elena]", this mail is for
Elena ML. The filter injects the mail to Elena ML process, which is
run by the filter.

5	Executing FML based on POP3

We implement libexec/popfml.pl for the case that

	* You read mails via POP3 only
	* Your mail server's administration policy prohibits programs
	  from run kicked by MTA.

The fundamental logic is simple. FML (libexec/popfml.pl) retrieves a
mail via POP and injects it to fml.pl. This technique is functional,
so applicable to all cases/os's where you can only read mails (of
course you need "perl" works well :-). This technique works well both
on UNIX and Windows NT (beta test phase).

A problem is you should store the POP3 password somewhere on your
machine. In default popfml.pl read it from ~/.netrc.

5.1	how to emulate elena ML based on POP3

% makefml newml popfml

	elena
	elena-ctl

% /usr/local/fml/libexec/popfml.pl /var/spool/ml/popfml /usr/local/fml \
	-user elena -pwfile /var/spool/ml/etc/netrc \
	-include_file /var/spool/ml/elena/include

% /usr/local/fml/libexec/popfml.pl /var/spool/ml/popfml /usr/local/fml \
	-user elena-ctl -pwfile /var/spool/ml/etc/netrc \
	-include_file /var/spool/ml/elena/include-ctl

    login    elena
    
    login    elena-ctl

  */5 * * * * 	/var/spool/ml/etc/popfml/master.sh

5.4	Popfml.pl Reference 

SYNOPSIS:

    popfml.pl [-hd] [-user username] [-host host] [-f config-file]

    -user username			
    -host host(pop server running)
    -f    config-file
    -h    this message
    -d    debug mode

Example:
Retrieve a user "fukachan" 's mails from POP server "hikari" and
injects them to programs defined in ~/.popfmlrc.

	popfml.pl -user fukachan -host hikari

Configuration files is ~/.popfmlrc. popfml.pl runs $POP_EXEC variable
in it as a filtering program.

Example: inject mails retrieved from POP servers into fml.pl.

	$POP_EXEC = "/usr/local/fml/fml.pl /var/spool/ml/Elena";
	1;

How to set up cron differs from OS to OS. An example crontab (4.4BSD)
is

* * * * * popfml.pl -user fukachan -host hikari

FML via POP3 technique may enable FML to run on several plat homes out
of UNIX and administration policy.

	Variables

	$POPFML_PROG
	$POP_EXEC

filtering program which we inject mail to. Example: 

   $POP_EXEC = "/usr/local/fml/fml.pl /var/spool/ml/Elena";

	$POP_SERVER

POP Server

	$POP_USER

The pop user name (account) corresponds to the mailing list address 

	$POP_PASSWORD

POP password with $POP_USER.

	$POP_TIMEOUT

timeout to close POP connection with POP server.

	$POP_QUEUE_DIR

temporary queue directory.

	$DIR/var/pop.queue

	$POP_LOGFILE

session log of POP3.

	$DIR/var/log/_poplog

	$POPFML_MAX_CHILDREN

the maximum number of programs running at the same time. 
If the value is large, OS is painful ;-)

	$POP_PORT

POP3 port number. The default is 110/tcp. Usually you need not to
concern it.

	$POPFML_QUEUE_EXPIRE_LIMIT

expire limit of queue spool.

6	Several Utilities (bin/, lib/Utilities/ ...)

In this chapter we describe utility programs under bin/ and
contrib/Utilities/.

6.1	analyze_mail_error.pl (replaced to libexec/mead.pl)

use libexec/mead.pl.

6.2	Archive Generator

Please see the chapter of "archive" (daily 3).

6.3	base64decode.pl base64encode.pl

BASE64 encoder and decoder. A part of library uses this.

6.4	cron (bin/cron.pl)

cron is a tiny program which is almost compatible of Vixie Cron.
Please see cron chapter (digest 2.17).

6.5	Daemon-ize programs

Example:

	% bin/daemon.pl cron.pl $PWD

to make cron.pl a daemon. perl version of the following C program.
This may work on BSD's.

    #include <stdio.h>
    #include <sys/ioctl.h>
    #include <sgtty.h>
    #include <sys/file.h>
    
    void usage()
    {
      char *string="USAGE: targetprogram";
      fputs(string,stderr);
      exit(0);
    }
    
    main(argc, argv)
         int argc;
         char **argv;
    {
      int f;
    
      f = open( "/dev/tty", O_RDWR, 0);
      if( -1 == ioctl(f ,TIOCNOTTY, NULL))
        exit(1);
      close(f);
    
      if( argc > 1 )
        execvp(*(++argv), argv);
      else
        usage();
    
      exit(0);
    }

6.6	daily, log analyzer 

	% bin/daily.pl -f etc/daily.ignore log

which shows ambiguous yesterday lines in the log file($LOGFILE).

	/usr/local/fml/etc/daily.ignore

	/usr/local/fml/etc/daily.ignore
	("FML installed directory"/etc/daily.ignore)

determines which lines daily.pl ignores. This file contains key
phrases regarded as valid in the context of FML operations. We
recommend you run daily.pl and check the output. If it has superfluous
lines, you add needless phrase patterns to "daily.ignore". It is good
to run daily.pl each morning by cron.

	daily.pl [-hd] [-f patfile] [-t trap_patfile] [-p days]

	-h        this message
	-d        debug mode

	-f file   pattern file to ignore
	-t file   pattern file to trap (evaluated before ignore list)
	-p days   scan at which day (default is 1, that is "yesterday")

Example: for elena ML (/var/spool/ml/elena)

[$HOME/sbin/daily.sh]

	#!/bin/sh
	/usr/local/fml/bin/daily.pl -f $HOME/etc/daily.ignore \
		-t $HOME/etc/daily.trap /var/spool/ml/elena/log

[crontab]
0 5 * * * /bin/sh $HOME/sbin/daily.sh

6.7	Checking DNS

A small program to check DNS values.

	% make dns_check

runs this program.

6.8	emumail.pl

Emulation program of output from sendmail into fml.pl which is for
debug only.

"makefml test <ML>" is the same function and more useful I believe.

6.9	Expire

6.10	WIX Language Formatter  (bin/fwix.pl)

Please see the chapter documentation 2

6.11	gen_summary.pl (summary re-creator)

When summary file is broken, you can re-create summary file with this
program from ML articles in the spool.

in fml-support ML's Count: 02393

   gen_summary.pl -I /usr/local/fml Mailing List's spool > summary

	-I lib*.pl locations (required)
	-m MIME decoding

6.12	bin/msend_master.sh

example of control over multiple msending processes.
See also INSTALL file.

6.13	bin/more_secure_cf.pl

a real program of "make secure" operation.

6.14	Multipart Canceler Filter (bin/multipart.pl)

Author: Yuao TANIGAWA <yuao@infonia.or.jp>

You can use "| multipart.pl | fml.pl" in include file.
This cancels out multipart mails to inject it to fml.pl since 
commands in multipart mails could fail.

    From:    Yuao TANIGAWA <yuao@infonia.or.jp>
    Subject: Multipart Canceler
    To:      fml-support@axion.phys.titech.ac.jp (fml supporting ML)
    Date:    Fri, 10 Jan 1997 21:09:03 +0200
    X-Mail-Count: 01937
    
6.15	NEWSYSLOG SHELL VERSION (bin/newsyslog.sh)

Inheritance in the age without libnewsyslog.pl

6.16	A demo of SMTP library

A demo.

6.17	initialize and change password  (bin/passwd.pl)

Please use "makefml passwd ML ...", which is more useful today.

FML saves crypted passwords for remote administration authentication
in $DIR/etc/passwd. So you should use a special program to encrypt
passwords for initialization and change.

 passwd.pl [-i] [-f password-file] user password

   -i initialize
   -f password file	 

Example:

	% passwd.pl -i fukachan@phys.titech.ac.jp 

shows

	Address: fukachan@phys.titech.ac.jp 
	Password: 

Please input your password. The password file contains

	fukachan@phys.titech.ac.jp 32ZrV/XvYj1kY

where this "32ZrV/XvYj1kY" is a crypted password.

6.18	get mails via POP3

This gets mails from a mail server via POP3 and injects them into
fml.pl and runs ML. The real program is libexec/popfml.pl, others is a
link to it.

6.19	split and send files (bin/split_and_msend.pl)

Obsolete.

6.20	SPOOL -> HTML Hierarchy (bin/spool2html.pl)

make HTML style articles of ML. In default this generates index.html,
thread.html (thread) and each article.html. Please run this program in
ML's HOME DIRECTORY ($DIR) or define -D $DIR.

SYNOPSIS:

  spool2html.pl [-h] [-I INC] [-f cf] [-d HTML_DIR] [-t TYPE] [-D DIR] SPOOL

    -h    this message
    -d    $HTML_DIR
    -t    number of day ($HTML_INDEX_UNIT)

    -h    this message
    -d    $HTML_DIR
    -f    config.ph location
    -t    number of day ($HTML_INDEX_UNIT)
    -D    $DIR
    -I    path; add path to @INC

    SPOOL $SPOOL_DIR

Example:
   % cd /var/spool/ml/elena
   % spool2html.pl -I /usr/local/fml -d /htdocs /var/spool/ml/elena/spool

Attention: Without -D, $DIR == $PWD.

6.21	MH scan.format (lib/Utilities/scan.format)

Please input lib/Utilities/scan.format to your ~/Mail, MH scan shows
the following

  65 -06/14 Horaki Hikari fml-su Subject uja  <<UJA
     Date:  From:         X-ML-Name: Subject:

It must be useful. You can write MH scan.format if you understand

	if	%<
	elif	%?
	else	%|
	fi	%>

Statement Examples:

%?{X-Mailinglist-Name}%6{X-Mailinglist-Name}\

%?(match NetBSD.ORG)NetBSD\

6.22	to make articles by "mget" mbox style (lib/Utilities/packmbox.pl)

packmbox.pl	(by ukai@hplj.hpl.hp.com)

By lib/Utilities/packmbox.pl, you do to make mbox style file. 

	(cd spool; packmbox.pl ) > ~/mbox

You can incorporate mails to your ~/Mail/inbox if with MH

	inc -file ~/mbox -truncate 

6.23	RMAIL -> fml.pl

Rmail2fml.pl	(yamane@ngi.co.jp)

	RMAIL file -> fml.pl

6.24	Stardate (STAR TREK)

Add a special header field Stardate which is STAR TREK's date:-).

	X-Stardate: [-31]8122.12

If you use MH, you can replace MH post by my-mh-post.pl (perl) or
my-mh-post(shell) to add Stardate in your mail when sending.

libstardate.pl is stardate library.

* This algorithm is one of stardate calculation. Several
interpretation are available for stardate, but I choice one of them.

6.25	weekend-msend.sh

FML drives Matomeokuri / Digest in the interval 24 hours, so FML does
not implement sending once in a week.  To send articles in a week once
in a week, you run weekend-msend.sh once in a week where you set m=1u
as a digest parameter. 

In default you cannot configure FML to control weekly digest
configurations for each address by remote (e.g. command).
# Of course, you can hack FML.

The heart of weekly-msend is 

        MSEND_RC=$DIR/weekend/msendrc
        ACTIVE_LIST=$DIR/weekend/actives
        MEMBER_LIST=$DIR/weekend/members

and "address option" in actives or members (depended on modes).  You
require "1" of "m=1" as an option since we require msend.pl works
always.

	address m=1u

6.26	Append X-Stardate: (Emacs Lisp)

Please load contrib/Utilities/stardate.el and add the following codes
to your ~/.emacs. mh-comp generates draft buffer with X-Stardate:.  I
check this code on mule-2.3.

(load-library "stardate")
(setq startrek-stardate-process "/some-directory-path/libstardate.pl")


		INDEX

.forward                                   ...   4 
analyze_mail_error.pl                      ...   6.1 
analyze_mail_error.sh                      ...   6.1 
archive.pl                                 ...   6.2 
archive.sh                                 ...   6.2 
base64decode.pl                            ...   6.3 
base64encode.pl                            ...   6.3 
bin/base64decode.pl                        ...   6.3 
bin/base64encode.pl                        ...   6.3 
bin/inc_via_pop.pl                         ...   6.18 
bin/pop2recv.pl                            ...   6.18 
bin/spool2html.pl                          ...   6.20 
cron.pl                                    ...   6.4 
daemon.pl                                  ...   6.5 
daily.ignore                               ...   6.6 
daily.pl                                   ...   6.6 
$DIR/var/log/_poplog                       ...   5.4 
$DIR/var/pop.queue                         ...   5.4 
dns_check.pl                               ...   6.7 
emumail.pl                                 ...   6.8 
FML based on POP3                          ...   5 
fmlserv                                    ...   3 
$FMLSERV_DIR                               ...   3.2 
$FMLSERV_LOGFILE                           ...   3.2 
$FMLSERV_PERMIT_LISTS_COMMAND              ...   3.3 
fwix.pl                                    ...   6.10 
gen_summary.pl                             ...   6.11 
$HOME/.forward                             ...   4 
inc_via_pop.pl                             ...   6.18 
lib/Utilities/scan.format                  ...   6.21 
listserv compatibility                     ...   3.1 
majordomo compatibility                    ...   3.1 
manual: fml_local.pl                       ...   2 
$MAP_DB                                    ...   3.4 
multipart.pl                               ...   6.14 
my-mh-post                                 ...   6.24 
my-mh-post.pl                              ...   6.24 
newsyslog.sh                               ...   6.15 
packmbox.pl                                ...   6.22 
passwd.pl                                  ...   6.17 
pmail.pl                                   ...   6.16 
pop2recv.pl                                ...   6.18 
POP3                                       ...   5 
$POP_EXEC                                  ...   5.4 
$POP_LOGFILE                               ...   5.4 
$POP_PASSWORD                              ...   5.4 
$POP_PORT                                  ...   5.4 
$POP_QUEUE_DIR                             ...   5.4 
$POP_SERVER                                ...   5.4 
$POP_TIMEOUT                               ...   5.4 
$POP_USER                                  ...   5.4 
popfml.pl                                  ...   5.4 
$POPFML_MAX_CHILDREN                       ...   5.4 
$POPFML_QUEUE_EXPIRE_LIMIT                 ...   5.4 
Rmail2fml.pl                               ...   6.23 
scan.format                                ...   6.21 
split_and_msend.pl                         ...   6.19 
spool2html.pl                              ...   6.20 
stardate.el                                ...   6.24 6.26 
STAR TREK                                  ...   6.26 
var/log/_poplog                            ...   5.4 
var/pop.queue                              ...   5.4 
weekend-msend.sh                           ...   6.25 
~/.forward                                 ...   4 
