AWSTATS setup on OS X Tiger Server 

Setup AWSTATS on OS X Tiger Server is plain easy. Check this link for step by step setup
http://www.afp548.com/article.php?story ... 3205258972

But there is one thing bite me - the Apache log format. Apache's default combined log format is like this:

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined

So I followed the AWSTATS setup instruction to use the combined format for my virtual domain, and watch the log format while browsing the virtual website, the first column of log line always shows the virtual domain server IP instead of visiting IP. Used the same combined log format on OS X Tiger client, no problem at all. After digging around, I found out that Tiger server use "%{PC-Remote-Addr}i" to replace "%h" as the visiting IP. The combined log format on Tiger Server should be like this:

LogFormat "%{PC-Remote-Addr}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined

There is no documentation mentioned about this as I search around, Apparently, Tiger is trying to do good thing, but on the contrary bite us.

I did not run the awstats configure script to make changes to apache config file. just manually edit a httpd_awstats.conf and include it in httpd.conf. I also copied awstats.model.conf to awstats.virtualdomain.conf , changed the "LogFile", "SiteDomain", "HostAlias" parameter to match the virtual domain and run the update script:

/Library/WebServer/awstats/tools/awstats_updateall.pl -awstatsprog=/Library/WebServer/awstats/wwwroot/cgi-bin/awstats.pl now

To view the virtual domain awstats, the link should be like this:

http://myhost/awstats/awstats.pl?config=SiteDomain


[ add comment ] permalink ( 3 / 70 )
Use SpamAssassin Message.pm module to fetch email attachment 

Here is a little script I wrote to fetch email attachment. If user got spam and forward it as attachment to me, I could use this script to rip the attachment and feeds it to sa-learn, the bayes learning engine of SA.

#!/usr/bin/perl
use strict;
use warnings;

use Mail::SpamAssassin::Message;

my $fh;
open $fh, "<", shift or die "Could not open message file:$!";
my @message = <$fh>;

my $msg = Mail::SpamAssassin::Message->new(
{
'message' => \@message,
}
) || die "Message error?";

#my $msg = Mail::SpamAssassin::Message->new() || die "Message error?";

#foreach my $p ($msg->find_parts(qr/^(text|image|application)\b/i, 1)) {
foreach my $p ($msg->find_parts(qr/^message\b/i, 0)) {
eval {
no warnings ;
my $type = $p->{'type'};
my $attachname = $p->{'name'};
print "Content type is: $type\n";
print "write file name: $attachname\n";
open my $out, ">", "$attachname" || die "Can't write file $attachname:$!";
binmode $out;
print $out $p->decode();
};
# warn $@ if $@;
}
[ add comment ] permalink ( 3.1 / 81 )
Re-queueing quarantined spam 

I happen to know that one amavisd user misfiltered 10000 emails as spam, and want to release them. How that could be achieved. Here is the relevant discussion link:
http://marc.theaimsgroup.com/?l=amavis- ... 84&w=2

The following both solutions are that we get mail_id from the quarantined *.gz file itself, typical file like this : spam-*******.gz, the '*******' is the mail_id.

The config in amavisd.conf should be:

-------------
$policy_bank{'AM.PDP-SOCK'} = {
protocol => 'AM.PDP', # Amavis policy delegation protocol
auth_required_release => 0, # don't require secret_id for
amavisd-release
};

$interface_policy{'SOCK'} = 'AM.PDP-SOCK';
----------------------


Scenario 1:
If the mail log is in MySQL, and spam are quarantined as *.gz file, we just pipe the mail_id to amavisd-release

Use the on-liner perl:

perl -e 'opendir(my $dir, "/var/amavis/quarantine"); \
print "$_\n" for map { /spam-(.*?)\.gz/ } readdir($dir); ' |
amavisd-release -

Scenario 2:
If the mail log is not in MySQL and spam are quarantined in *.gz, amavisd-release needs the full quarantine file name, not just mail_id, so we use different one-liner perl

perl -e 'opendir(my $dir, "/var/amavis/quarantine");\
print "$_\n" for grep { /^spam/} readdir($dir); ' | amavisd-release -
[ add comment ] permalink ( 3 / 71 )
Amavisd-2.3.3 and SA's sa-update 

If system are running on Amavisd-new-2.3.3 and SA version older than 3.1.5. the rules updated by sa-update will not accessed by amavisd unless add one line code to Mail::SpamAssassin->new( LOCAL_STATE_DIR => '/var/lib',). The relevant discussion link:
http://www.mail-archive.com/amavis-user ... 05859.html

[ add comment ] permalink ( 3 / 81 )
P0f - Passive OS fingerprinting tool 

I normally use nmap to profile a remote system, but I found a lightweight sniffing tool p0f - passive os fingering tool http://lcamtuf.coredump.cx/p0f.shtml which gather all kinds of profiling information about a remote system. Here is how I run it on my email server to profile all the remote system connecting to my smtp port

p0f -d -o /var/log/p0f.log -t -U -i en1 -l '(dst host my emailserver and tcp dst port 25)'

change the network interface en1 to eth0 if you are on Linux.

It will capture some interesting information like below:

<Mon Sep 25 14:00:07 2006> 137.82.45.4:33785 - Solaris 9.1 (up: 645 hrs) -> my emailserver:25 (distance 4, link: ethernet/modem)

<Mon Sep 25 14:00:26 2006> 201.227.187.11:1296 - Windows XP SP1+, 2000 SP3 -> my emailserver:25 (distance 21, link: ethernet/modem)

<Mon Sep 25 14:00:55 2006> 65.17.242.10:3853 - FreeBSD 4.6-4.9 (up: 3512 hrs) -> my emailserver:25 (distance 14, link: ethernet/modem)

The you can make this information available to SpamAssassin through Amavisd's p0f feature and add spam score based on spammer's OS and hop distance. Following statistics quote from Amavisd-new-2.4.2 RELEASE_NOTES:

-----------------------
Some statistics collected from our logs in February 2006:
p0f OS guess ham : spam
-----------------------------
Windows-XP 0.7 % : 99.3 %
Windows-2000 5.8 % : 94.2 %
UNKNOWN 16.5 % : 83.5 %
Linux 58.8 % : 41.2 %
Unix 80.3 % : 19.7 %
(Unix+Linux 66.5 % : 33.5 %)
(ham: mail with score below 3, spam: score above 6)
-----------------------------------

[ add comment ] permalink ( 3 / 110 )

Back Next