Tuesday, April 29, 2014

Local DNS problems

To access my mail server I use a dynamic DNS service that points to the public ip of my router. This means that whenever I'm at home I will still be accessing my server through the public ip even though I'm on the same local subnet. This creates the following route:

Local ip -> router local ip -> router public ip -> router local ip -> mail server local ip. I'm not surprised my cheap ISP supplied router does not handle this very well.

The solution is to have a DNS server on the local network that will give the local ip for my mail server for all local requests of my dynamic hostname. For this I use DNSmasq.

Install DNSmasq

apt-get install dnsmasq

In /etc/hosts make sure that there is an entry for your servers local hostname using the local ip:  localhostname localhostname

In /etc/resolv.conf set your ISPs DNS servers or use Google public DNS or OpenDNS.

In /etc/dnsmasq.conf set something like:
#return addresses from hosts file with correct local address matching incoming interface
#proxy authentication flag
# Never forward plain names (without a dot or domain part)
# Never forward addresses in the non-routed address spaces.
# Add other name servers here, with domain specs if they are for
# non-public domains.
# Add domains which you want to force to an IP address.
# disable DHCP and TFTP

Now either set your devices to use your new DNS server or adjust your router to hand out the new DNS setting automatically.


Friday, April 25, 2014

Getting through spam filters

More and more mail servers employ stricter policies to reject spam. Being a legitimate domain you want to make sure your mail is not rejected.

If you've read my post Setting up my mail server you'll know I use my ISP's mail server as a relay because my dynamic IP is blacklisted (most dynamic IPs are).

Your ISP's mail server probably has a good reputation and is unlikely to be rejected, but to make sure you do everything you can to make it through spam filters and make sure your domain is not forged by spammers you want to set up SPF, DKIM and DMARC (all three are currently being checked by GMail <- good reference).


A description of how to set up DKIM on your own mail server running postfix, please read Setting up my mail server.


SPF is published as a TXT DNS record for your domain and simply lists which servers are permitted to send email for the domain.

For a standard domain hosted on a single server the SPF record would look something like this:
"v=spf1 a mx ~all"

v=spf1 specifies that this is a version 1 spf record.

'a' means that any server which has a valid a record on this domain can send mail.

'mx' means your mail servers are allowed to send mail.

'~all' means that any other server or IP should 'softfail'. I've you've tested your SPF record for a while you want to change this to '-all' which means hardfail.

Since I use my ISP's mail server as a relay, all mail is sent through their servers. If this would be only a single IP I could just add 'ip4:<ip address>'. But my ISP uses multiple servers and IPs and it's difficult to find out which. Also my ISP doesn't have an SPF record themselves so I can't do an 'include:example.com'.

One thing which is for sure is that their servers all identify as being from 'ispdomain.com' where the domain is (luckily) only used for the backbone network of the ISP. So allowing any mail server on ispdomain.com to send mail would be a safe option which doesn't require much maintenance (e.g. in case IP addresses are changed).

To permit this domain to send mail I add the following to my SPF record:

The receiving mail server will take this and expand the %{h2} to the domain of the mail server (ISPdomain.com) and %{d} as the domain of the sender (domain part of email address). Now it will query the DNS for this domain: ispdomain.com.senderdomain.com and if it finds a valid A record it will allow the email.

So I add a DNS record to my domain:
ispdomain.com IN A

Now my SPF record is ready to publish:
"v=spf1 a mx exists:%{h2}.%{d} ~all"


Dmarc specifies a policy in your DNS defining what the receiving mail server should do with any email that does not pass the SPF and DKIM validation.

We start by creating a TXT record for our domain:
_dmarc IN TXT "v=DMARC1; p=none; rua=mailto:postmaster@example.com"

Basically this does nothing but tell the receiver that there is no policy, but that we would like to receive a daily report by email for any email that did not pass the SPF and DKIM validations.

This is great as a start and will allow you to monitor what's going on without the risk of emails not being delivered.

When you are done monitoring and satisfied with your setup you can change the p=none; to p=quarantine; or p=reject;. Quarantine will instruct the mail server to mark the message as spam. Reject will instruct them to just discard the mail, similar to softfail and hardfail in SPF.


Ready? Then start by sending a message to test@allaboutspam.com. After about 15 minutes you'll receive a bounce message with a link included where you can see a detailed report about your SPF and DKIM. If any are red or yellow, please check if you've made any mistakes.

Setting up my mail server

In this guide I'll be installing a mail server to host virtual mail domains with imap acess using postfix and dovecot using flat files for user configuration (no mysql).

Guides used in this article:

Preparing the environment

After a fresh install of Debian I start with making sure the system is up to date.
apt-get update
apt-get upgrade

To keep the script kiddies out I install fail2ban.
apt-get install fail2ban

Out of the box it's configured to ban anyone who makes 6 failed login attempts through SSH for 5 minutes.

Then we prevent root from logging in through SSH and allow only a single user to login from one or two specific IPs by editing /etc/ssh/sshd_config
PermitRootLogin no
AllowUsers user@your-ip user@another-ip-if-any

On the local machine we copy the public RSA key and add it to ~/.ssh/authorized_keys
Then in /etc/ssh/sshd_config we set:
PasswordAuthentication no

Restart ssh:
service ssh restart

Install firewall and only allow port 22 from a specific ip:
apt-get install ufw
ufw allow from your-ip to any port 22
ufw enable

Enable automatic security upgrades.
apt-get install unattended-upgrades

Install logwatch to keep an eye on things.
apt-get install logwatch

In the file /etc/cron.daily/00logwatch change
/usr/sbin/logwatch --output mail

/usr/sbin/logwatch --output mail --mailto your@mail.com --detail high

Installing the MTA

apt-get install postfix dovecot-imapd dovecot-lmtpd

When asked choose either 'internet site' or 'internet site with smarthost'. I choose the smarthost because I'm on a consumer connection and use my ISP server for outgoing mail.

The system will now be able to send and receive mail for the Linux user accounts on this box using the specified mailname (during postfix setup).

To host any additional domains we use Virtual Mailboxes.

Copy /etc/postfix/main.cf to /etc/postfix/main.cf.old. Then in /etc/postfix/main.cf add:
virtual_transport = lmtp:unix:private/dovecot-lmtp
virtual_mailbox_domains = yourdomain.com
virtual_mailbox_maps = hash:/etc/postfix/vmailbox
virtual_alias_maps = hash:/etc/postfix/virtual

And also add the following line to disable the VRFY command that is frequently used by spammers to enumerate accounts on the server:
disable_vrfy_command = yes

In /etc/postfix/vmailbox:
info@example.com   whatever
sales@example.com whatever
# Comment out the entry below to implement a catch-all.
# @example.com example.com/catchall
#...virtual mailboxes for more domains...

in /etc/postfix/virtual:
postmaster@example.com postmaster

Execute the command "postmap /etc/postfix/virtual" after changing the virtual file, execute "postmap /etc/postfix/vmailbox" after changing the vmailbox file, and execute the command "postfix reload" after changing the main.cf file.

We have now set up postfix to deliver mail via lmtp to Dovecot, using the /etc/postfix/vmailbox file to check which addresses to accept mail for.

Now (after copying /etc/dovecot/dovecot.conf to /etc/dovecot/dovecot.conf.old) we add the following at the bottom of the main config file /etc/dovecot/dovecot.conf to enable the imap and lmtp protocols:
protocols = imap lmtp

In /etc/dovecot/conf.d/10-master.conf (make copy first) we setup the lmtp listener:
service lmtp {
# unix_listener lmtp {
unix_listener /var/spool/postfix/private/dovecot-lmtp {
mode = 0666
group = postfix
user = postfix
# Create inet listener only if you can't use the above UNIX socket
#inet_listener lmtp {
# Avoid making LMTP visible for the entire internet
#address =
#port =

In /etc/dovecot/conf.d/10-mail.conf we specify where to store mail:
mail_location = maildir:/var/mail/vhosts/%d/%n

Create the vhosts dir and make it writable to GID 5000.
mkdir /var/mail/vhosts
chgrp 5000 /var/mail/vhosts
chmod -R g+rw /var/mail/vhosts

Mail will be stored in /var/mail/vhosts/<domain>/<user>/.

Now in /etc/dovecot/conf.d/10-auth.conf uncomment the line:
!include auth-passwdfile.conf.ext

Now create a file /etc/dovecot/users using the format:
# email:pass:uid:gid
# generate pass hash with doveadm pw -s crypt

Enable the use of TLS in /etc/postfix/main.cf
# TLS parameters
smtpd_tls_session_cache_database =btree:${data_directory}/smtpd_scache
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
# send auth only after TLS has been set up
smtpd_tls_auth_only = yes
# request that TLS be used
smtpd_tls_security_level = may
smtp_tls_security_level = may
smtpd_tls_ask_ccert = yes
#enable some extra loggin on TLS
smtpd_tls_loglevel = 2
smtp_tls_loglevel = 2
smtpd_tls_received_header = yes

Open the firewall on port 25
ufw allow 25
ufw reload

You should now be able to receive email to the virtually hosted domain.

Enable IMAPS (and disable IMAP) in /etc/dovecot/conf.d/10-master.conf
service imap-login {
inet_listener imap {
port = 0
inet_listener imaps {
port = 993
ssl = yes

You should now be able to access your mail using IMAPS. Now let's configure SALS for authenticated mail sending.

In /etc/postfix/master.cf uncomment the submission line and just below it add the following:
-o syslog_name=postfix/submission
-o smtpd_tls_security_level=encrypt
-o smtpd_sasl_auth_enable=yes
-o smtpd_client_restrictions=
-o milter_macro_daemon_name=ORIGINATING
-o smtpd_sasl_type=dovecot
-o smtpd_sasl_path=private/auth
-o smtpd_recipient_restrictions=

The 'smtpd_tls_security_level=encrypt' ensures that encryption is always required. 'smtpd_sasl_auth_enable=yes' enables SASL authentications. The 'smtpd_recipient_restrictions=permis_sasl_authenticated,reject' makes sure only authenticated users can submit mail and everything else is rejected.

To block some obvious spam we add to /etc/postfix/main.cf:
smtpd_recipient_restrictions =
reject_rbl_client zen.spamhaus.org,

And in /etc/dovecot/conf.d/10-master.conf
service auth {
# Postfix smtp-auth
unix_listener /var/spool/postfix/private/auth {
mode = 0666
user = postfix
group = postfix

And in /etc/dovecot/conf.d/10-ssl.conf
ssl = required
ssl_cert = </etc/ssl/certs/ssl-cert-snakeoil.pem
ssl_key = </etc/ssl/private/ssl-cert-snakeoil.key

Reload postfix and dovecot.  This will setup port 587 with required TLS connection and authentication through Dovecot. Remember to use ufw to open port 587 for access.

Now configure fail2ban to also check Postfix and Dovecot. in /etc/fail2ban/jail.conf set:

enabled = false
port = smtp,ssmtp
filter = postfix
logpath = /var/log/mail.log


enabled = true
port = smtp,ssmtp,imap2,imap3,imaps,pop3,pop3s
filter = sasl
logpath = /var/log/mail.log


enabled = true
port = smtp,ssmtp,imap2,imap3,imaps,pop3,pop3s
filter = dovecot
logpath = /var/log/mail.log

service fail2ban restart


Greylisting prevents spam by first rejecting a message for 5 minutes after which the sending server will try again. Then it's accepted. Spammers most of the time don't bother trying a second time.

Install postgrey by:
apt-get install postgrey

And then in /etc/postfix/main.cf add check_policy_service inet: under smtpd_client_restrictions as the last option before permit.

UPDATE: I've decided to disable greylisting for now, because it just don't like the fact that people cannot email me 'now'.

Reverse DNS (PTR)

I'm on a dynamic IP so I use a dynamic DNS service to reach my server. To prevent mail being seen as spam it's important that the SMTP banner hostname matches the reverse DNS record (PTR record). Luckily my ISP sets PTR records for all IP's so I use the following script to set the correct parameter in postfix to use my PTR record as the SMTP banner.
IP=$(dig veenhoven.dtdns.net +short)
PTR=$(dig -x $IP +short)
sudo postconf -e myhostname=$HST
echo "Mail hostname set to" $HST

Domain Keys Identified Mail (DKIM)

To get trough spam filters more easily, I'm going to sign all my outgoing mail using DKIM.
apt-get install opendkim opendkim-tools
mkdir -pv /etc/opendkim/
chown -Rv opendkim:opendkim /etc/opendkim
cd /etc/opendkim/
opendkim-genkey -r -h rsa -d yourdomain -s mail
mv -v mail.private mail
chown opendkim:opendkim ./*

The contents of /etc/opendkim/mail.txt are your DNS record which needs to be set. You should also set a DNS record specifying the policy regarding DKIM.
_adsp._domainkey IN TXT " "dkim=discardable;"

Test the policy record of your DNS with:
opendkim-testadsp yourdomain.com

Tests the DNS public key with:
sudo opendkim-testkey -d yourdomain.com -k /etc/opendkim/mail -s mail -vvv

If all is OK we can continue the DKIM setup process with creating some config files.
nano /etc/opendkim/KeyTable
yourdomain.com yourdomain.com:mail:/etc/opendkim/mail

nano /etc/opendkim/SigningTable
*@yourdomain.com yourdomain.com

nano /etc/opendkim/TrustedHosts

nano /etc/opendkim.conf
## opendkim.conf -- configuration file for OpenDKIM filter
Canonicalization relaxed/relaxed
ExternalIgnoreList refile:/etc/opendkim/TrustedHosts
InternalHosts refile:/etc/opendkim/TrustedHosts
KeyTable refile:/etc/opendkim/KeyTable
LogWhy Yes
MinimumKeyBits 1024
Mode sv
PidFile /var/run/opendkim/opendkim.pid
SigningTable refile:/etc/opendkim/SigningTable
Socket inet:8891@localhost
Syslog Yes
SyslogSuccess Yes
TemporaryDirectory /var/tmp
UMask 022
UserID opendkim:opendkim

And in /etc/postfix/main.cf add:
smtpd_milters = inet:
non_smtpd_milters = $smtpd_milters
milter_default_action = accept

Restart postfix and opendkim and test it out.

Enable Push

For push we use d-push.
apt-get install d-push

Patch these in /etc/d-push/config.php:
define('IMAP_PORT', 993);
define('IMAP_OPTIONS', '/ssl/novalidate-cert');
define('IMAP_SENTFOLDER''Sent Messages');

To get the “Deleted Messages” folder working I needed to patch patch /usr/share/d-push/backend/imap.php:
else if($lid == "trash") {
$folder->parentid = "0";
$folder->displayname = "Trash";
$this->wasteID = $id;

change the first line to:
else if($lid == "trash" || $lid == "deleted messages") {

Enable https:
a2ensite default-ssl
a2enmod ssl
service apache2 restart
ufw allow 443

Now you can use the Exchange type of account on your iPhone or iPad and enjoy push. NOT!

So it seems d-push that comes with Debian Wheezy is broken. On sending an email I get an error that the server didn't accept the email and keeps the email in the out box. Even though it is actually sent!
Next to that, the iPad doesn't allow to set a sender name for exchange accounts.

And to be honest: Do I really need push? I only check my email a few times a day anyway. On my workstation I've even disabled autonotification because it is so distracting. Let's simplify life a bit and don't worry about reading email the minute it arrives.

So I remove d-push:
apt-get purge d-push
apt-get autoremove

Full text search

So I don't care about push. I do care about search. Because lately I've been very annoyed with searching mail in IMAP accounts on my iPad. It is terrible. I just won't find emails I know are there.
We start by installing dovecot-solr and solr-tomcat:
apt-get install solr-tomcat dovecot-solr

We add a reference to the plugin in /etc/dovecot/conf.d/20-imap.conf.
mail_plugins = $mail_plugins antispam fts fts_solr

And in /etc/dovecot/conf.d/90-plugin.conf we add:
plugin {
fts = solr
fts_solr = break-imap-search url=http://localhost:8080/solr/

Now restart dovecot and tomcat
service dovecot restart
service tomcat6 restart

VERY IMPORTANT. By default, tomcat6 is globally accessible, which means just anybody with a web browser can query your mail! We need to turn this off. Inside /etc/tomcat6/server.xml, there is a line called
Connector port="8080" protocol="HTTP/1.1" connectionTimeout="20000" URIEncoding="UTF-8" redirectPort="8443"

We need to add address="" onto that:
Connector address="" port="8080" protocol="HTTP/1.1"

Now we need to fix some stuff first. Wget the original tar.gz file for the dovecot-solr package from debian.org and copy the file doc/solr-schema.xml to /etc/solr/conf/schema.conf.

Next in the file /etc/solr/conf/solrconfig.xml we comment out the line with >text<:
<lst name="defaults">
   <str name="echoParams">explicit</str>
   <int name="rows">10</int>
<!--<str name="df">text</str>-->

Restart dovecot and tomcat6 and enjoy full text search!

Increasing max message size

To increase the maximum message size from the default 10M to 20M add the following line to /etc/postfix/main.cf
message_size_limit = 20480000


I have a mailbox with my ISP that uses POP3 and which I cannot set to forward to my own domain. To get this mail I use Fetchmail.

Install fetchmail:
apt-get install fetchmail

After installation edit /etc/default/fetchmail and set:

Then create /etc/fetchmailrc:
# /etc/fetchmailrc for system-wide daemon mode
# This file must be chmod 0600, owner fetchmail

set daemon 300 # Pool every 5 minutes
set syslog # log through syslog facility
set postmaster root

set no bouncemail # avoid loss on 4xx errors
# on the other hand, 5xx errors get
# more dangerous...

# Hosts to pool

# Defaults ===============================================================
# Set antispam to -1, since it is far safer to use that together with
# no bouncemail
timeout 300
antispam -1
batchlimit 100

poll pop.someprovider.tld protocol POP3 user "falko@someprovider.tld" there with password "secret" is falko here nokeep fetchall ssl

The nokeep will delete the messages on the server (we don't want any private email out there now do we?). Fetchall makes sure everything is fetched and ssl makes sure a secure connection is used.

Make sure you chown fetchmail /etc/fetchmailrc and chmod 600 /etc/fetchmail rc.

Then start fetchmail with:
service fetchmail start

That's about it.

To create a virtual mailbox you'll have to do the following:

  • add mailbox to /etc/postfix/vmailbox

  • run 'sudo postmap /etc/postfix/vmailbox'

  • add user to /etc/dovecot/users

  • if new domain, setup dkim for domain and add to virtual_mailbox_domains in /etc/postfix/main.cf

To create new virtual alias:

  • add alias to /etc/postfix/virtual

  • run 'sudo postmap /etc/postfix/virtual'

Forward local users using /etc/aliases file:


Before I forget, use a good backup solution! I've set up some cronjobs with tar.

  • Every week a new level 0 backup is created

  • Hourly an incremental backup

  • Daily 1 incremental backup is saved

These are rsynced to a second machine.

Thursday, April 24, 2014

Ban -- block an annoying machine

CentOS doesn't come with the handy ufw tool. So you have to configure iptables manually.

This blocks a specific IP address from reaching your server. This is useful if you are getting annoying traffic from another machine and you want to get rid of them. Replace with the IP address you want to drop.
iptables -I INPUT -j DROP -s

Set these aliases in your .bash_aliases file (sourced by .bashrc) for easy acess:
alias ban='iptables -I INPUT -j DROP -s'
alias unban='iptables -D INPUT -j DROP -s'

Now you can just do:

For more info: http://www.noah.org/wiki/Iptables#RedHat

Wednesday, April 23, 2014

Setting up a CentOS server: fail2ban and automatic security updates

I'm a big Debian fan, but sometimes I have to use CentOS. Here are 2 tips:

Install fail2ban on CentOS

Fail2ban is not in the default repositories on CentOS, so you have to install the EPEL repos first.

For CentOS 5 run:
rpm -Uvh http://mirror.1000mbps.com/fedora-epel/5/i386/epel-release-5-4.noarch.rpm

For CentOS 6 run:
rpm -Uvh http://mirror.proserve.nl/fedora-epel/6/i386/epel-release-6-8.noarch.rpm

Now you can do a simple yum install fail2ban and then service fail2ban restart to install and start fail2ban.

Enable automatic security updates

To automatically install security updates you need an extra plugin for yum:
yum install yum-security

Then in your crontab set it to daily run:
yum -y update --security

That's all....

Privacy has become a real concern (to me)

With the recent revalations of Edward Snowden about how governments worldwide are treating our privacy I finally realised that we live in a collective state od denial. Don't we all know that putting everything online about our personal life has to be a bad idea?

There are abuntant examples of how insecure our private data is stored by cloud services. Take for example the incident where DropBox had a bug in their authentication process that allowed login without a password to any account without a password. Next to that DropBox has admitted that they can decrypt your data and hand it to the government if requested to do so.

And DropBox is not the only cloud service with these kind of issues. Think about all the private data that's being stored by GMail? Do you want the government or NSA to 'just have access' to your email?

That's why I've recently embarked on a privacy mission myself. In the next few weeks I'll be implementing some measures to gain control of my online privacy again. These are the sub-projects that are on my list right now:

1) delete facebook account: if you like your privacy you should not use facebook (this will also simplify your life extensively because you don't have to keep up-to-date with all those shallow friends)
2) use a privacy aware search engine and plugins: this will minimise the amount of tracks we leave while browsing.
3) set up an in-home mailserver: if you're worried about cloud services handing over your data, don't let them: host your own email.
4) encrypt dropbox: DropBox is just too convenient to let go of. But in case someone does get access to my account or the stored data is handed to the government, at least give them encrypted data only so they have something to chew on.

Stay tuned for some followup posts about these projects and I'll explane exactly how to go through all the steps I'm going through.

Tuesday, April 22, 2014

Yield Thought, I swapped my MacBook for an iPad+Linode

How about this: work anywhere anytime. No more boring offices. Spend your days working in the park, exploring new places everyday and still being productive.

I can do that with my laptop you say? No way! Using this setup you can work an entire day without have to recharge. Try working wireless with a laptop for more than a few hours. You won't make it through the day.

Seriously; give the following idea a try. I know I will.

On September 19th, I said goodbye to my trusty MacBook Pro and started developing exclusively on an iPad + Linode 512. This is the surprising story of a month spent working in the cloud. It all started when I bought my first MacBook a couple of years ago. Frustrated by the inconsistent usage of c...

Link: Yield Thought, I swapped my MacBook for an iPad+Linode via yieldthought.com