		
		header rewriting

------------------------------------------------------------
1	Examples of header manipulation
1.1	rewrite subject
1.2	copy header fields
1.3	enforce the value for a header field
1.4	remove a header field
1.5	pass through a field
1.6	set up the value for command reply message 
1.8	order of header fields
2	other topics
2.1	filtering by header

3	Examples
3.1	Reply-To: in a message from portable phone is invalid?
3.2	hack subject of welcome message to be in Japanese
3.3	hack subject of confirmation message
3.4	Preserve Received: field in distribution articles
3.5	Date: Type ($DATE_TYPE)
3.6	Customizing Header Fields Of The Command Status Report Mail
3.7	Configure Reply-To: of a command mail reply
3.8	Subject Tags
3.9	Cut off [Elena:ID] form strings in a subject
3.10	Change the digit of the number of a subject tag
3.11	No padding of a subject tag e.g. [Elena:00001]
3.12	Pass the original Message-Id through
3.13	Original Message-ID for your mailing list
3.14	Star Trek Stardate: field
3.15	One ML has plural addresses for posting 
3.16	Configure X-ML-Info:
3.17	To: rewriting
3.18	Against NIFTYSERVE.OR.JP (obsolete)
3.19	Append Errors-To:
3.21	Set Reply-To: be From: adress + $MAIL_LIST
3.22	Set Reply-To: be From: adress + $MAIL_LIST (2)

4	Internals
4.1	Configure Header Fields

5	Customize Fields of the Mail Header 
5.1	Enforce the content for a header field
5.2	Enforce the value of a field to one of the in-coming mail header
5.3	Add or Remove @HdrFieldsOrder entries
5.4	COPY_FIELD

Appendix A	Examles
Appendix A.1	Example: set up Reply-To if required when Reply-To == $MAIL_LIST
------------------------------------------------------------


1	Examples of header manipulation

1.1	rewrite subject

	MENU -> HEADER_CONFIG -> SUBJECT_TAG_TYPE
	MENU -> HEADER_CONFIG -> SUBJECT_TAG_FIGURE

1.2	copy header fields

	&COPY_FIELD(src, dst);

	&COPY_FIELD('Sender', 'X-Sender');

1.3	enforce the value for a header field

	&DEFINE_FIELD_FORCED(fieldname, value);
  For example:

	&DEFINE_FIELD_FORCED("reply-to", "$From_address, $MAIL_LIST");

1.4	remove a header field

	&DELETE_FIELD(fieldname);
   For example:

	&DELETE_FIELD('X-Uja');

1.5	pass through a field

	&DEFINE_FIELD_ORIGINAL(fieldname);
   For example:

	&DEFINE_FIELD_ORIGINAL('reply-to');

1.6	set up the value for command reply message 

	&DEFINE_FIELD_OF_REPORT_MAIL(fieldname, value);

	&DEFINE_FIELD_OF_REPORT_MAIL('Reply-To', "Your Reply-To Address");

.	pass through the whole header (by default)
.	pass through the whole header (by default)

   $PASS_ALL_FIELDS_IN_HEADER = 1;

1.8	order of header fields

@HdrFieldsOrder defines the order of header fields.

2	other topics

2.1	filtering by header

   &DEFINE_FIELD_PAT_TO_REJECT('subject', '/FREE|SEX|ADULT|XXX/');

3	Examples

3.1	Reply-To: in a message from portable phone is invalid?

Q: Reply-To: in a message from portable phone is invalid?

A: "Reply-To: is enforced to ML" is a solution. It may resolve your
   annoyance. 

   &DEFINE_FIELD_FORCED('reply-to', $MAIL_LIST);

   % cd /var/spool/ml/elena/cf
   % makefml update-config.ph elena

3.2	hack subject of welcome message to be in Japanese

omitted since this is Japanese specific topic.

fml-support: 07219

$START_HOOK = q#

	&use('MIME');
	$WELCOME_STATEMENT  = &STR2EUC($WELCOME_STATEMENT);
	$WELCOME_STATEMENT  = &mimeencode($WELCOME_STATEMENT);

#;

make config.ph

3.3	hack subject of confirmation message

   "Subscribe confirmation request $ML_FN";

Hmm, in the present fml you need to customize internal function.

$CONFIRM_REPLAY_SUBJECT_FUNCTION = 'myGenConfirmReplySubject';

sub myGenConfirmReplySubject
{
    local(*e, *cf, $mode) = @_;
    local($s);

    if ($debug_confirm) {
	local(@c) = caller;
	&Log("GenConfirmReplySubject<$c[2]>: $mode") if $mode ne 'Default';
    }

    if ($mode eq 'Default') {
	$s = "Subscribe request result $ML_FN";
    }
    elsif ($mode eq 'Confirm::Confirmed') {
	$s = $CONFIRMATION_WELCOME_STATEMENT || $WELCOME_STATEMENT;
	# $s = "Newly added $From_address $ML_FN";
    }
    elsif ($mode eq 'Confirm::Error') {
	$s = "Subscribe with confirmation error $ML_FN";
    }
    elsif ($mode eq 'Confirm::GenPreamble') {
	$s = &STR2EUC($s);
	# $s = "Subscribe confirmation request $ML_FN";
    }
    elsif ($mode eq 'IdCheck::syntax_error') {
	$s = "Subscribe confirmation errror $ML_FN";
    }
    elsif ($mode eq 'Confirm::expired') {
	$s = "Subscribe confirmation expired $ML_FN";
    }
    elsif ($mode eq 'BufferSyntax::Error') {
	$s = "Subscribe confirmation errror $ML_FN";
    }
    elsif ($mode eq 'BufferSyntax::InvalidAddr') {
	$s = "Subscribe confirmation errror $ML_FN";
    }
    else {
	&Log("GenConfirmReplySubject: unknown mode [$mode]") if $debug_confirm;
	"Subscribe request result $ML_FN";
    }
}

3.4	Preserve Received: field in distribution articles

After FML 2.2, set

    $SKIP_FIELDS  = 'Return-Receipt-To';

to preserve Received: fields in the header. A capital is important.

Out of security problems, in default FML cuts off Received: fields to
avoid "too many hops" error and the concept that FML receives this
mail so that we have network connectivity insurance between the poster
and this host.

You may want to log whole input mails for security. To log all mails,
set (in the last of config.ph, but before "1;")

	&DEFINE_MODE("stdinlog");

which enables FML to log inputs from MTA to a file under
$DIR/var/log/. Caution: you should need to expire them by hand :-)

In FML, "Posted:" is when this mail is sent from the original poster
and "Date:" is the time when FML runs. The difference between Date:
and Posted: reflects network delays.

FYI: fml-support's ML Count: 01816, 01821 for the discussion

3.5	Date: Type ($DATE_TYPE)

$DATE_TYPE default is "original-date".

	not defined (default for compatibility), the same as "received-date"

	received-date
	received-date+posted
		Date:	When FML receives mail
		Posted:	Date: of the original mail

	received-date+x-posted
		Date:	When FML receives mail
		X-Posted:	Date: of the original mail

	received-date+x-original-date
		Date:	When FML receives mail
		X-Original-Date:	Date: of the original mail

	original-date
		Date:	
		Posted: not appears in the header 
			(remove the same information)

	distribute-date
	distribute-date+posted
		Date:	When FML start to deliver the article
		Posted:	Date: of the original mail

	distribute-date+x-posted
		Date:	When FML start to deliver the article
		X-Posted:	Date: of the original mail

	distribute-date+x-original-date
		Date:	When FML start to deliver the article
		X-Original-Date:	Date: of the original mail

"distribute-date" sets up both Date: and X-Mail-Count: has the same
order on time sequence. It is useful to sort mails on functionless
MUA.

3.6	Customizing Header Fields Of The Command Status Report Mail

You can customize the mail header of command replies by

	$Envelope{'GH:field:'} 

"field" of it is the name of a header field. The order of fields in
the header is defined by @HdrFieldsOrder.

Example:

Enforce "From: uja@aoi.chan.panic (AOICHAN PANIC)".

	$Envelope{'GH:From:'} = "uja@aoi.chan.panic (AOICHAN PANIC)";

$REPORT_HEADER_CONFIG_HOOK enables you to be more flexible.

3.7	Configure Reply-To: of a command mail reply

Described above, 

	&DEFINE_FIELD_OF_REPORT_MAIL('Reply-To', "Your Reply-To Address");

is the same as 

	$Envelope{'GH:Reply-To:'} = "Your Reply-To Address";

where the field is one of fields defined in @HdrFieldsOrder.

3.8	Subject Tags

In default FML does not append a subject tag to Subject:. 
FML appends always X-Mail-Count: and X-ML-Name: and so on.
The user client program (MUA) should show the useful screen using
them. IT SHOULD BE NOT THE FUNCTION OF MAILING LIST SERVERS.
However FML provides $SUBJECT_TAG_TYPE to append a subject tag.

  $SUBJECT_TAG_TYPE		Subject: tag

	TYPE    Subject Example

		Subject: 	 	(DEFAULT, FML recommends this)
	(:)	Subject: (Elena:100)
	[:]	Subject: [Elena:100]	(hml 1.6 compatibility)
	( )	Subject: (Elena 100)
	[ ]	Subject: [Elena 100]
	(,)	Subject: (Elena,100)
	[,]	Subject: [Elena,100]
	()	Subject: (Elena)
	[]	Subject: [Elena]
	(ID)	Subject: (100)
	[ID]	Subject: [100]

[DESCRIPTION]
FML interprets $SUBJECT_TAG_TYPE (e.g. "(:)") as a set of variables

	$SUBJECT_FREE_FORM = 1;
	$BEGIN_BRACKET     = '(';
	$BRACKET           = 'Elena';
	$BRACKET_SEPARATOR = ' ';
	$END_BRACKET       = ')';
	$SUBJECT_FREE_FORM_REGEXP = "\\($BRACKET \\d+\\)";

Another example: Subject: [Elena 100]

	$SUBJECT_FREE_FORM = 1;
	$BEGIN_BRACKET     = '[';
	$BRACKET           = 'Elena';
	$BRACKET_SEPARATOR = ' ';
	$END_BRACKET       = ']';
	$SUBJECT_FREE_FORM_REGEXP = "\\[$BRACKET \\d+\\]";

3.9	Cut off [Elena:ID] form strings in a subject

Historical, not used today.

3.10	Change the digit of the number of a subject tag

	$SUBJECT_FORM_LONG_ID (5; the number > 1)

is the digit of the number. Default is 5.

3.11	No padding of a subject tag e.g. [Elena:00001]

	$SUBJECT_FORM_LONG_ID = -1;

3.12	Pass the original Message-Id through

	$USE_ORIGINAL_MESSAGE_ID = 1; (default)

3.13	Original Message-ID for your mailing list

&DEFINE_FIELD_FORCED('Message-Id', "your favorite ID"); enables your
mailing list original ID, which should be unique through the whole
spacetime.

Let be "Message-Id: uja@aoi.chan.panic".

&DEFINE_FIELD_FORCED('Message-Id', "uja$$\@aoi.chan.panic");

is the same as

$HEADER_ADD_HOOK = q#
   $body .= "Message-ID: uja@aoi.chan.panic\n";
#;

Another Example:
Message-ID: <19950518.01905.Elena.Lolobrigita@Baycity.jp>

$HEADER_ADD_HOOK = q#
   $body .= "Message-ID: ".
	sprintf("<%4d%02d%02d.%05d.%s>\n", 1900 + $year, $mon + 1, $mday, $ID, 
	"Elena.Lolobrigita@Baycity.jp");
#;

3.14	Star Trek Stardate: field

$APPEND_STARDATE = 1; in config.ph enables 

	Date: Fri, 19 May 95 22:31:09  JST
	From: "No.6 Fukachan" <fukachan@phys.titech.ac.jp>
	Subject: Re: Nantonaku Leonard Nimoy 
	To: enterprise@phys.titech.ac.jp (Foreign TV Series ML)
	X-ML-Name: Prisoner

3.15	One ML has plural addresses for posting 

To enable the aliases of mailing list name e.g. both addresses 

	Elena@phys.titech.ac.jp
	Anna@phys.titech.ac.jp

write aliases in the array @MAIL_LIST_ALIASES in config.ph.  The first
element should be the original $MAIL_LIST. If this array defined, FML
changes To: (if rewriting To: also) and Reply-To: of distribution
articles.

  @MAIL_LIST_ALIASES = ('Elena@phys.titech.ac.jp', 
			'Anna@phys.titech.ac.jp'
		    );

In the past this technique is implemented by a hook. Today you need
not to set up a hook.

3.16	Configure X-ML-Info:

	&DEFINE_FIELD_FORCED('x-ml-info', "information you say");

$X_ML_INFO_MESSAGE causes the same effect. 

3.17	To: rewriting

For your eyes To: an Cc: should be appropriate order.
Posters should check its own mails. 
FML can enforce "To: $MAIL_LIST" for your eyes. To use it, please set

	$REWRITE_TO = 0; (FML 2.2 default)
		pass the original To: and Cc: through

	$REWRITE_TO = 1;
		If To: contains $MAIL_LIST, let it be.
		If not, FML formats
			To: $MAIL_LIST $ML_FN,
				the-original-To-content
	$REWRITE_TO = 2;
		Always To: $MAIL_LIST $ML_FN.
		The original To: content is lost.		

3.18	Against NIFTYSERVE.OR.JP (obsolete)

obsolete. Please ignore this.

3.19	Append Errors-To:

RFC1123 prohibits Errors-To: but terrible MTA's or SMTP convertor
cannot understand it.  Such terrible MTA returns error mails to From:
or To: but can understand Errors-To:. Sigh...
$USE_ERRORS_TO = 1; if you append Errors-To:.

	$SUBJECT_TAG_TYPE = "";
        $SUBJECT_FREE_FORM = 1;
        $BEGIN_BRACKET     = '(';
        $BRACKET           = 'Elena';
        $BRACKET_SEPARATOR = ' ';
        $END_BRACKET       = ')';
	$SUBJECT_FREE_FORM_REGEXP = $NULL;

3.21	Set Reply-To: be From: adress + $MAIL_LIST

$START_HOOK = q#
      &DEFINE_FIELD_FORCED("reply-to", "$From_address, $MAIL_LIST");
#;

You need to use a hook since From: address is dynamic variable, so
cannot define static setting in config.ph.

3.22	Set Reply-To: be From: adress + $MAIL_LIST (2)

fml-support: 06240

* What do you do ?

   else

   if post from not member
	Reply-To: From:'s address and mailing list
   else
	Reply-To: mailing list

* How do you config?

config.ph is a perl script. The design of fml is so that you can write
hooks in config.ph if the configuration you want is not just switch
(on/off) of some variables. So you maintain configuration apart from
fml upgrade.

$START_HOOK =q%
   if (&MailListMemberP($From_address)) {
	&DEFINE_FIELD_FORCED("reply-to", $MAIL_LIST);
   }
   else {
	&DEFINE_FIELD_FORCED("reply-to", "$From_address, $MAIL_LIST");
   }
%;

Q:

$SMTP_OPEN_HOOK = q#
    if ($Envelope{'h:Cc:'}) {
        &DEFINE_FIELD_FORCED('Cc', 
		$Envelope{'h:Cc:'}. ", ". $From_address );
    } else {
        &DEFINE_FIELD_FORCED('Cc', $From_address);
    }
#;

   #!/bin/sh

   ml=elena
   date=`date -u +%C%y%m%d`

   /usr/local/fml/makefml lock $ml 60 &

   cd /var/spool/ml/$ml || exit 1

   for file in seq spool summary log
   do
       if [ -f $file ];then
          # mv $file $file.$date
          cp  $file $file.$data
         cp /dev/null $file
       fi
   done

A:

	&COPY_FIELD('Received', 'X-Received');
	$SKIP_FIELDS = 'Received|Return-Receipt-To';

[Result]

 Received: by hikari.fml.org (8.9.3/3.4W6+Hikari-2.1) with ESMTP id XAA17996 for <fukachan@sapporo.iij.ad.jp>; Wed, 6 Oct 1999 23:29:52 +0900 (JST)
 Received: from beth.fml.org (localhost.sapporo.iij.ad.jp [127.0.0.1])
 	by beth.fml.org (Postfix) with ESMTP id 1CE0EA6837
 	for <fukachan@fml.org>; Wed,  6 Oct 1999 23:29:52 +0900 (JST)
 X-Received: by hikari.fml.org (8.9.3/3.4W6+Hikari-2.1) with ESMTP id XAA17981 for <fukachan@sapporo.iij.ad.jp>; Wed, 6 Oct 1999 23:29:30 +0900 (JST)
 X-Received: by beth.fml.org (Postfix)
 	id 465B4A6838; Wed,  6 Oct 1999 23:29:30 +0900 (JST)
 X-Received: by beth.fml.org (Postfix, from userid 1106)
 	id 0DB1EA6837; Wed,  6 Oct 1999 23:29:29 +0900 (JST)
 Date: Wed,  6 Oct 1999 23:29:29 +0900 (JST)
 From: fukachan@fml.org (Ken'ichi Fukamachi)
 Reply-To: elena@fml.org
 To: fukachan@fml.org
 Message-Id: <19991006142929.0DB1EA6837@beth.fml.org>
 X-ML-Name: elena
 X-Mail-Count: 00207
 X-MLServer: fml 2.2; post only (only members can post)
 X-ML-Info: If you have a question, send e-mail with the body
 	"help" (without quotes) to the address elena-ctl@fml.org;
 	help=<mailto:elena-ctl@fml.org?body=help>
 Precedence: bulk
 Lines: 2

 test

 -- fukachan

4	Internals

4.1	Configure Header Fields

fml.pl passes only defined fields through to cut off annoying fields,
e.g. X-Faces, and distributes the mail with a formatted header. It
orders the header fields of distribution following an array
@HdrFieldsOrder.

If $PASS_ALL_FIELDS_IN_HEADER is defined, fml.pl passed all fields in a
header except for Received: and Return-Receipt-To: (fields defined in
$SKIP_FIELDS).

	$PASS_ALL_FIELDS_IN_HEADER = 1; (path through, 1 is default)

* fml 2.1 is "0", fml 2.2 is "1".

If you consists on your customize, you can set up @HdrFieldsOrder
e.g. in config.ph. In default @HdrFieldsOrder (1998/04/26) is

    @HdrFieldsOrder =	# rfc822; fields = ...; Resent-* are ignored;
	('Return-Path', 'Received',
	 'Date', 'Posted', 'X-Posted', 'X-Original-Date',
	 'From', 'Reply-To', 'Subject', 'Sender', 
	 'To', 'Cc', 'Errors-To', 'Message-Id', 'In-Reply-To', 
	 'References', 'Keywords', 'Comments', 'Encrypted',
	 ':XMLNAME:', ':XMLCOUNT:', 'X-MLServer', 
	 'XRef', 'X-Stardate', 'X-ML-Info', 
	 'X-Mailer',
	 ':body:', ':any:', 
	 'X-Authentication-Warning',
	 'Mime-Version', 'Content-Type', 'Content-Transfer-Encoding',
	 'Content-ID', 'Content-Description', # RFC2045
	 'Precedence', 'Lines');

where FML passes X-Authentication-Warning through to show the security
warning.

In @HdrFieldsOrder

	:body:
	:any:
	:XMLNAME:
	:XMLCOUNT:

are defined for special purposes and expanded before distribution. The
meaning is

	:body:		$doby variable is expanded
			(obsolete, left for backward compatible)

	:any:		fields not defined in @HdrFieldsOrder are shown 
			when $PASS_ALL_FIELDS_IN_HEADER is set.

	:XMLNAME:	$XMLNAME (X-ML-Name: Field)

	:XMLCOUNT:	$XMLCOUNT (X-Mail-Count: field)

5	Customize Fields of the Mail Header 

5.1	Enforce the content for a header field

SYNOPSIS:
	&DEFINE_FIELD_FORCED('field', "field-contents");

Enforces the value of "field" to "field-contents". where the field is
one of fields defined in @HdrFieldsOrder.  This function sets up

	$Envelope{'fh:field:'} = "field-contents";

Example: set "X-ML-Info: ML Information"
	&DEFINE_FIELD_FORCED('x-ml-info', "ML Information");

5.2	Enforce the value of a field to one of the in-coming mail header

SYNOPSIS:
	&DEFINE_FIELD_ORIGINAL('field');

enforce the value of a field to one of the in-coming mail header.  The
field must be one of elements of @HdrFieldsOrder. This function sets
up

	$Envelope{'oh:field:'};

Example: Let the value of To:, Cc:, Reply-To: be original.

	&DEFINE_FIELD_ORIGINAL('to');
	&DEFINE_FIELD_ORIGINAL('cc');
	&DEFINE_FIELD_ORIGINAL('reply-to');

5.3	Add or Remove @HdrFieldsOrder entries

To add or remove header fields, set up @HdrFieldsOrder in config.ph.
Also FML provides macros, &ADD_FIELD, &DELETE_FIELD, for them. To add
a field "X-Uja" in the last of @HdrFieldsOrder,

	&ADD_FIELD('X-Uja');

If you always remove 'Cc:' field in distribution mail, you can set it
up by

	&DELETE_FIELD('Cc');

	unshift(@HdrFieldsOrder, 'X-Some-Thing');

5.4	COPY_FIELD

where the field is one of fields defined in @HdrFieldsOrder.

		&COPY_FIELD(field, new-field);
	Example:
		&COPY_FIELD('Sender', 'X-Sender');

Appendix A	Examles

Appendix A.1	Example: set up Reply-To if required when Reply-To == $MAIL_LIST

*** NOT TRANSLATE THIS CHAPTER FOR AN ADDITIONAL BONUS PROGRAM ***

$START_HOOK = q!
    local($pat) = '.*\s*reply-to:\s*';
    local($s);
    if ($Envelope{'Body'} =~ /^$pat(\S+)/i) {
	&DEFINE_FIELD_FORCED('reply-to', $s=$1);
        &Log("reply-to: <$s>");
	$Envelope{'Body'} =~ s/^$pat\S+\n//io;
    }
!;


		INDEX

$APPEND_STARDATE                           ...   3.14 
$DATE_TYPE                                 ...   3.5 
--fh:field                                 ...   5.1 
$FORCE_COMMAND_REPLY_TO                    ...   3.7 
&GenerateHeader()                          ...   3.6 
GenerateHeader()                           ...   3.6 
'GH:field:'                                ...   3.6 
'GH:Reply-To:'                             ...   3.7 
@HdrFieldsOrder                            ...   4.1 
$HML_FORM_LONG_ID                          ...   3.10 
libstardate.pl                             ...   3.14 
@MAIL_LIST_ALIASES                         ...   3.15 
$MAINTAINER                                ...   3.6 
$MAINTAINER_SIGNATURE                      ...   3.6 
Message-ID                                 ...   3.12 
--oh:field                                 ...   5.2 
$PASS_ALL_FIELDS_IN_HEADER                 ...   4.1 
@PLAY_TO                                   ...   3.15 
StarDate                                   ...   3.14 
STAR TREK                                  ...   3.14 
$STRIP_BRACKETS                            ...   3.9 
$SUBJECT_FORM_LONG_ID                      ...   3.10 
$SUBJECT_FORM_LONG_ID                      ...   3.11 
$SUBJECT_FREE_FORM                         ...   3.8 
$SUBJECT_TAG_TYPE                          ...   3.8 
$USE_ERRORS_TO                             ...   3.18 
$USE_ORIGINAL_MESSAGE_ID                   ...   3.12 
$X_ML_INFO_MESSAGE                         ...   3.16 
XMLCOUNT                                   ...   4.1 
$XMLNAME                                   ...   4.1 
