From jik@kamens.brookline.ma.us Thu Dec 09 19:10:02 1999
X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil]
	["3539" "Thu" "9" "December" "1999" "14:05:47" "-0500" "Jonathan Kamens" "jik@kamens.brookline.ma.us" nil "118" "Re: \"movemail.pl\" which works with maildirs" "^From:" nil nil "12" nil nil nil nil nil]
	nil)
Return-Path: <jik@kamens.brookline.ma.us>
Delivered-To: nelson@desk.crynwr.com
Received: (qmail 4055 invoked by uid 0); 9 Dec 1999 19:10:02 -0000
Received: from ns.crynwr.com (nelson@192.203.178.14)
  by desk.crynwr.com with SMTP; 9 Dec 1999 19:10:02 -0000
Received: (qmail 22443 invoked by uid 500); 9 Dec 1999 19:06:25 -0000
Delivered-To: nelson@qmail.org
Received: (qmail 22440 invoked from network); 9 Dec 1999 19:06:25 -0000
Received: from nautilus.shore.net (207.244.124.104)
  by www.qmail.org with SMTP; 9 Dec 1999 19:06:25 -0000
Received: from jik.shore.net [206.243.167.15] 
	by nautilus.shore.net with esmtp (Exim)
	id 11w8sx-0001Q1-00; Thu, 9 Dec 1999 14:05:51 -0500
Received: from jik2.kamens.brookline.ma.us (IDENT:root@jik2.kamens.brookline.ma.us [192.168.0.3])
	by jik.shore.net (8.9.0/8.9.0) with ESMTP id OAA23498;
	Thu, 9 Dec 1999 14:07:29 -0500
Received: (from jik@localhost)
	by jik2.kamens.brookline.ma.us (8.9.3/8.9.3) id OAA16893;
	Thu, 9 Dec 1999 14:05:47 -0500
Message-Id: <199912091905.OAA16893@jik2.kamens.brookline.ma.us>
In-reply-to: <19991207200616.7287.qmail@calisto2.curl.com> (jik@curl.com)
References:  <19991207200616.7287.qmail@calisto2.curl.com>
From: Jonathan Kamens <jik@kamens.brookline.ma.us>
To: jik@curl.com
CC: nelson@qmail.org
Subject: Re: "movemail.pl" which works with maildirs
Date: Thu, 9 Dec 1999 14:05:47 -0500

Here's an updated version of the maildir movemail script which I sent
you on Tuesday, which makes the following improvements:

1) Add the name of the script in the comment at the top.
2) Delete the stuff in the comment about using this script with
   nnmail, because the instructions I included don't actually work and
   I'm not sure exactly how to make them work.
3) Add functionality for stripping duplicate messages in the maildir,
   since qmail doesn't do that by default.
4) Sort the message file names when reading them from the maildir, so
   that the messages will appear in the output file in the order in
   which they were received.
5) Fix a typo in an error message.

  jik

#!/usr/bin/perl

# mdmovemail.pl - a replacement for the Emacs "movemail" program which
# understands how to read mail from a qmail-style maildir.
#
# Install it somewhere, make it executable, and then set
# rmail-movemail-program to point to it.  If you don't want the script
# to automatically strip out duplicate messages (defined as messages
# with the same message ID and the same body checksum), set the
# $strip_duplicates variable below to undef (it's enabled by default).
#
# You'll then need to modify rmail-inbox-list to point at your
# maildir.  For example:
# 
# (setq rmail-inbox-list (list (expand-file-name "~/Maildir")))
# 
# Written by Jonathan Kamens <jik@kamens.brookline.ma.us>.  Please
# send comments, corrections, etc. to me.

$strip_duplicates = 1;

($whoami = $0) =~ s,.*/,,;

$usage = "Usage: $whoami [-p] maildir destfile\n";

if ($ARGV[0] eq '-p') {
    shift;
    $preserve++;
}

die $usage if (! ($maildir = shift));
die $usage if (! ($destfile = shift));

die "$whoami: $maildir does not exist\n$usage" if (! -e $maildir);
die "$whoami: $maildir is not a directory\n$usage" if (! -d $maildir);
die "$whoami: $destfile already exists\n$usage" if (-e $destfile);

$msg_dir = "$maildir/new";
$message_start = "\f\n0, unseen,,\n";
$message_end = "\037";

die "$whoami: opening $msg_dir: $!\n" if (! opendir(DIR, $msg_dir));
exit 0 if (! (@dirfiles = sort readdir(DIR)));
closedir(DIR);

foreach $file (@dirfiles) {
    next if ($file eq '.');
    next if ($file eq '..');
    next if (! -f "$msg_dir/$file");
    &output_file($file);
}

die "$whoami: closing $destfile: $!\n" if ($dest_created && !close(OUTPUT));
unlink(@output_files) if (! $preserve);

sub output_file {
    my($basename) = @_;
    my($inpath) = "$msg_dir/$basename";
    local($/) = undef;
    my($contents);

    if (! $dest_created++) {
        open(OUTPUT, ">$destfile") || die "$whoami: opening $destfile: $!\n";
    }

    open(INPUT, $inpath) || die "$whoami: opening $inpath: $!\n";
    die "$whoami: reading $inpath: $!\n" if (! defined($contents = <INPUT>));
    close(INPUT);

    $contents =~ s/([^\n])$/$1\n/;

    if ($strip_duplicates && &is_duplicate($contents)) {
        push(@output_files, $inpath);
        return;
    }

    print(OUTPUT $message_start, $contents, $message_end) ||
        die "$whoami: writing to $destfile: $!\n";

    push(@output_files, $inpath);
}

sub is_duplicate {
    my($contents) = @_;
    my($head, $body, $checksum, $id);

    ($head, $body) = split(/\n\n/, $contents, 2);
    $head =~ s/\n+$//;
    $body =~ s/^\n+//;
    $body =~ s/\n+$//;

    $checksum = unpack("%16C*", $body);

    return undef if ($head !~ /(^|\n)\bmessage-id:\s*(\S+)\s*(\n|$)/i);
    $id = $2;

    return 1 if (defined($checksums{$id}) && $checksums{$id} == $checksum);

    $checksums{$id} = $checksum;

    return undef;
}


.