Powered by Twitter Tools
We’ve started in the last couple of months distributing the (currently) weekly backups of customer VPS/VDS disk/storage devices to large scale (4TB+) data stores on our network in Sydney and Canberra.
This has given us a far better backups and storage options & capabilties, and now most (nearly all) backups of customer VPS or VDS “disks” are stored on physically separate storage, usually at the same POP within Network Presence, but sometimes distributed between Canberra and Sydney..
Very Fast VPS “Screamer” disks now available on all or #NetPres large storage VPS! http://t.co/DFyMlf2
I’m an “old school” Unix user, who grew up on sed, awk & Sendmail, but many people still use sed & its syntax from the Unix or Linux command-line daily, so this is a quick note on how to use ‘sed’ when you’re working with URLs.
The ‘trick’ here (if there is one) is that most sed examples are for using it to substitute characters in a text line or file.
eg: changing http://this-site/ to http://that-site/
But the usual delimiter of the text to be changed is the ‘/’ (slash or divide-sign) character, which simply doesn’t work when the text your substituting also contains the ‘/’ (ie: URLs).
Please you don’t have to use the ‘/’ to delimit search & replace text in sed command lines, you can use (just about/within reason) any single character you like (eg: the ‘%’ (percent-sign) character), just as long as you use it consistently in your sed command line.
So if you’re using sed to substitute text in a URL, use the % to delimit the text to search for and the text to replace it with.
Here’s some examples:
Eg 1: We want to change all references of http://google.com to https://google.com in an inputfile, so a sed command line for this may be:
sed 's%http://google.com%https://google.com%g' inputfile > outputfile
Eg 2: You have a shell variable containing a URL, say $thisurl which you want to change to the contents of another variable $thaturl :
sed "s%$thisurl%$thaturl%g"
Note: the use of the double-quotes (“) character to allow shell variable expansion within the sed command.
Don’t forget that “man sed” is your friend & there’s an O’Reilly “sed” book too on it for much “sed goodness”.
“Automatic Login Links”, with Rails code segments from @danielmorrison http://t.co/opZ2j8D Useful followup comments too..
“base your business around an idea. Products are just the manifestation of the idea.” http://t.co/WYlZNcs Good thoughts on the SF Bubble too
Powered by Twitter Tools
Listening to This Week in Cloud Computing podcast on my iPhone with @shiftyjelly‘s Pocketcasts app.. #twicc
Powered by Twitter Tools
I’ve previously described the install & setup process for spamass-milter on Sendmail for Anti-Spam processing.
This blog post describes a different interface to Sendmail for Anti-Spam & Anti-Virus (AS/AV), using the Amavis daemon (amavisd-new) along with the Milter interface for Amavis to Sendmail.
1. Software Installs
1.0 Pre-Requisites
This requires (at least during this install process) the C compiler, so to install all the pre-req’s run:
yum -y install gcc make sendmail-devel
1a. Amavisd-Milter
Please DO NOT install the ‘amavisd-new-milter’ package in RPMForge, it’s old & lacks functionality. Please download the source distribution for amavisd-milter from http://sourceforge.net/projects/amavisd-milter/
The latest amavisd-milter in early 2011 delivers the file ‘amavisd-milter-1.5.0.tar.gz’, so extract that file & install that software to its default install locations.
Then do the following in the extract amavisd-milter directory:
./configure
make install
Which yields the /usr/local/sbin/amavisd-milter exacutable & the ‘man amavisd-milter’ manual page.
1b. Amavisd itself and its pre-requisites
Run:
yum -y install amavisd-new
1c. Check that the SpamAssassin & ClamAV packages are installed with:
yum -y install clamav clamav-db clamav-milter clamd spamass-milter perl-Mail-SPF sendmail-cf perl-Mail-DKIM
1d. There are some scanners that aren’t installed with amavisd-new’s Yum install, so bring in these decoders:
yum -y install tnef unzip lrzip p7zip-plugins
(updated for CentOS 5.9)
2 Configuring components & Amavisd
2a. See our previously posted SpamAssassin/ClamAV & Sendmail page for the SpamAssassin & ClamAV setup, but the important difference here is that ClamAV (clamd) must be reconfigured to run as the same pid/user that runs the Amavisd software (amavis username & group).
We set the pid & gid of the clamav user in /etc/passwd to be the same pid/gid as the amavis user that’s installed when using Yum to install the amavisd-new package.
Then you need to chown the ClamAV run & log directories to the new pid/gid of the clamav user with:
chown -R clamav.amavis /var/run/clamav /var/log/clamav/ /var/clamav/
Then restart clamd with:
service clamd restart
And check that it comes up ok with no errors in /var/log/maillog and /var/log/messages
2b. Amavisd-Milter Sysconfig file isn’t provided, as we’ve installed it from source, not a pre-build Package, so we need to create the file & I’ve used the following, from the site http://users.on.net/~hilton/amavisd-milter-sysconfig.txt
### /etc/sysconfig/amavisd-milter
### Configuration options for amavisd-milter
### Suitable for Redhat & SuSE systems.
#
#
### Amavisd's homedir.
### This should match the '$MYHOME' directive in amavisd.conf
AMAVISD_HOME="/var/amavis"
### Location of milter binary.
MILTER="/usr/local/sbin/amavisd-milter"
### User that amavisd-milter will run as.
### For RH/CentOS/Fedora set to "amavis"
### For SuSE set to "vscan"
AMAVISD_MILTER_USER="amavis"
### This is the socket used for communication between sendmail <--> milter
### It must correspond to the "S=" variable of the milter definition in sendmail.cf
### Note the variable substitution!
MILTER_SOCKET="local:$AMAVISD_HOME/amavisd-milter.sock"
### This is the socket used for communication between amavisd <--> milter
### It must correspond to the value of "$unix_socketname" in amavisd.conf
### Note the variable substitution!
AMAVISD_SOCKET="$AMAVISD_HOME/amavisd.sock"
### Pid file
### Note the variable substitution!
MILTER_PID="$AMAVISD_HOME/amavisd-milter.pid"
### All the args to milter
MILTER_FLAGS="-s $MILTER_SOCKET -p $MILTER_PID -w $AMAVISD_HOME -S $AMAVISD_SOCKET"
2c. Amavisd-Milter init.d file isn’t provided, as we’ve installed it from source, not a pre-build Package, so we need to create the file & I’ve used the following, from the site http://users.on.net/~hilton/amavisd-milter-init.d.txt
#!/bin/bash
# Init script for Amavisd-Milter.
# Written by Ben Tisdall
# chkconfig: 2345 78 31
# description: Amavisd Milter Interface
# processname: amavisd-milter
### Read in the standard init functions
source /etc/rc.d/init.d/functions
### Default variables
AMAVIS_USER="amavis"
MILTER_SOCKET=""
MILTER_FLAGS=""
desc="Amavisd Milter Interface"
RETVAL=0
SYSCONFIG="/etc/sysconfig/amavisd-milter"
### Read configuration
[ -r "$SYSCONFIG" ] && source "$SYSCONFIG"
### MILTER set in /etc/sysconfig/amavisd
prog="${MILTER##*/}"
progdir="${MILTER%/*}"
### Check we have the milter
if ! [ -x $progdir/$prog ]; then
echo -e "\nFATAL ERROR: $progdir/$prog not found and/or not executable, please check your installation.\n"
exit 1
fi
### Functions
start() {
if [ "$MILTER_SOCKET" -a -x "$progdir/$prog" ]; then
echo -n $"Starting $desc ($prog): "
daemon --user "$AMAVIS_USER" $progdir/$prog "$MILTER_FLAGS"
RETVAL=$?
echo
if [ $RETVAL -eq 0 -a -n "$MILTER_PID" -a ! -L "/var/run/${MILTER_PID##*/}" ]; then
ln -s "$MILTER_PID" "/var/run/${MILTER_PID##*/}"
touch /var/lock/subsys/$prog
fi
fi
}
stop() {
if [ "$MILTER_SOCKET" -o -f /var/lock/subsys/$prog ]; then
echo -n $"Shutting down $desc ($prog): "
killproc $prog
RETVAL=$?
echo
[ $RETVAL -eq 0 ] && rm -f /var/lock/subsys/$prog
fi
return $RETVAL
}
reload() {
echo -n $"Reloading $desc ($prog): "
killproc -HUP $prog
RETVAL=$?
echo
return $RETVAL
}
restart() {
stop
start
}
case "$1" in
start)
start
;;
stop)
stop
;;
restart)
restart
;;
reload)
restart
;;
condrestart)
[ -e /var/lock/subsys/$prog ] && restart
RETVAL=$?
;;
status)
status $prog
status $prog
RETVAL=$?
;;
*)
echo $"Usage: $0 {start|stop|restart|reload|condrestart|status}"
RETVAL=1
esac
exit $RETVAL
Download those two text files & copy them to the relevant system files with the commands:
Install amavisd-milter sysconfig script:
wget http://users.on.net/~hilton/amavisd-milter-sysconfig.txt
mv amavisd-milter-sysconfig.txt /etc/sysconfig/amavisd-milter
Install amavisd-milter init.d script:
wget http://users.on.net/~hilton/amavisd-milter-init.d.txt
mv amavisd-milter-init.d.txt /etc/init.d/amavisd-milter
chmod u+x /etc/init.d/amavisd-milter
chkconfig --add amavisd-milter
2d. Configuring Amavis
Amavis config file is /etc/amavisd.conf and it’s a large text config file, but the relevant settings to check/set are:
$protocol = "AM.PDP"; # Use AM.PDP protocol.
$unix_socketname = "$MYHOME/amavisd.sock"; # uncomment when using milter.
#$inet_socket_port = 10024; #comment out with milter.
$notify_method = 'pipe:flags=q argv=/usr/sbin/sendmail -Ac -i -odd -f ${sender} -- ${recipient}';
$forward_method = undef; #must be set like this with sendmail milter.
$mydomain = "example.com" #Your domain
$myhostname = "mail.example.com"; #The FQDN of your Mail Server host
$virus_admin = "root\@$mydomain"; #NDR recipient if virus found
$mailfrom_notify_admin = "virusalert\@$mydomain"; #NDR --> admin sender
Then the following affect the inclusion of mail message headers about the Amavis activity:
$sa_tag_level_deflt = -9999; # add spam info headers if at, or above that level
$sa_tag2_level_deflt = 3.4; # add 'spam detected' headers at that level
#sa_kill_level_deflt = 6.31; # triggers spam evasive actions
#sa_dsn_cutoff_level = 9; # spam level beyond which a DSN is not sent
#$sa_quarantine_cutoff_level = 20; # spam level beyond which quarantine is off
$sa_spam_subject_tag = '[SPAM] '; # Prepended to the subject line if defined.
2e. Create the ‘virusalert@yourdomain’ email address or alias, enter the following to /etc/aliases & run ‘newaliases’
virusalert: root
spam-police: root
2f. Setting domains which will be passed to Scanners
Amavis (Amavisd-New) only hands of messages which are deemed able to be ‘locally delivered’ by Sendmail, but you can include domains for SA (SpamAssassin) processing by loading them to the @local_domains_maps variable in /etc/amavisd.conf, which by default is set to the value of $mydomain & its subdomains:
@local_domains_maps = ( [".$mydomain", ".foo.com"] );
You may want to list in @local_domains_maps all hosts & domain names that you have in /etc/mail/local-host-names & /etc/mail/relay-domains
2g. Further down in the amavisd.conf file you need to enable the ClamAV sections and set the /var/run/clamav/clamd.sock file (matching the value in /etc/clamd.conf)
['ClamAV-clamd',
\&ask_daemon, ["CONTSCAN {}\n", "/var/run/clamav/clamd.sock"],
qr/\bOK$/m, qr/\bFOUND$/m,
qr/^.*?: (?!Infected Archive)(.*) FOUND$/m ],
Now save all these edits to /etc/amavisd.conf
2h. Set relevant entry in /etc/sysconfig/amavisd
AMAVIS_SENDMAIL_MILTER="no"
2i. Sendmail configs to enable use of Amavis-Milter
Add the following MILTER definition to /etc/mail/sendmail.mc & remove any/all other Milter definitions (eg: remove clamav & spamass milter entries in sendmail.mc
define(`MILTER', 1)dnl
INPUT_MAIL_FILTER(`milter-amavis', `S=local:/var/amavis/amavisd-milter.sock, F=T, T=S:10m;R:10m;E:10m')dnl
If you want your Sendmail server to be available on port 25 (SMTP), then also ensure:
DAEMON_OPTIONS(`Port=smtp, Name=MTA')dnl
Then save that update & recreate the sendmail.cf & submit.cf files by running
make
in /etc/mail, then restart sendmail (when ready) with
service sendmail restart
3. Start daemons up & monitor /var/log/maillog for their logging
service sendmail restart
service amavisd start
service amavisd-milter start
service clamd restart
4. Test / Check
4a. Send an email through the server, either with manual “telnet your-mail-server 25” or such.
4b. Monitor /var/log/maillog
