Managing Bulk Email Delivery – Part 2 – Bounce Handling with Postfix and PHP

If you are unfamiliar with the basics of how email messages are sent and what happens with bounces, please read the previous article.

This article explains ONE way of dealing with bounces. It has its pros and its cons and does not account for ALL instances of bounces, but it does deal with the basics of implementing bounce handling and can be used as the foundation of a more sophisticated system.

The primary logic in this system is the ability to control the SMTP Envelope FROM address. We want to construct this so that when a message is returned, it will have encoded in it information that will tell us exactly who the message was originally sent to. The drawback to this method is that if we had originally a single message that was being blasted to 1000 recipients, we now have to create 1000 messages each with it’s own customized/encoded FROM address. (there can be only one FROM)

Once we can control the Envelope FROM, we need a domain for the bounces to be returned to. Whereas the messages may have been originally FROM list@mycompany.com, we want bounces to be returned to a special host name. This way we can segregate bounce handling to a different system (if so desired.) With that in mind the FROM address will be constructed like so: encodedrecipaddr@bounces.mycompany.com. Don’t forget to setup an MX record for this host name (which must point to an A record, CNAMEs or bare IPs are not allowed as MX data)

So, how do we encode the local part of the FROM address? It’s really up to you, but pick one way and stick with it. Our solution uses the following:

sentto-brian=networkjack.info@bounces.mycompany.com

You could get fancy like so:

sentto-2e64665495eab1fa4c276f73a610e054@bounces.mycompany.com

where 2e64665495eab1fa4c276f73a610e054 is an MD5 hash of the original email address.

Whatever method chosen, it’s necessary to track that particular encoding somehow as we will see it as the recipient on any possible bounces.
Here is the SQL table we used:

CREATE TABLE EmailAddressTracker (
EmailAddress varchar(255) NOT NULL,
EncodedFROM varchar(255) NOT NULL,
IgnoreBounces tinyint(1) unsigned NOT NULL default '0',
MsgCount int(10) unsigned NOT NULL default '0',
FirstEmail datetime NOT NULL,
LastEmail datetime NOT NULL,
FirstBounce datetime NOT NULL,
LastBounce datetime NOT NULL,
LastBounce2 datetime NOT NULL,
LastBounce3 datetime NOT NULL,
BounceCount smallint(5) unsigned NOT NULL default '0',
PRIMARY KEY (EmailAddress),
UNIQUE KEY EncodedFROM (EncodedFROM)
)

EncodedFROM holds the ENTIRE local part.

In the function that is ultimately responsible for sending the email out, we lookup/maintain entries in this table. This would be the place to apply policy and either let the message actually be sent or disable the email address somehow or ignore any policy if the IgnoreBounces flag were enabled for this particular email address.

---

If a message ultimately is rejected, we have to have some way of accessing this table. This is where Postfix and PHP come into play.We could simply have all messages for that domain fall into a mailbox which is accessed and read and parse the payload for undeliverable recipients, but we want direct access to the Envelope information. We could create a two tiered system that does do parsing as a fallback, but for now let’s keep it simple.

We are using Postfix on the server that is responsible for handling bounces. Two main additions to the postfix configuration are necessary.

1. add a transport to the master.cf file:
   mybh unix – n n – 10 pipe
   user=mailadmin argv=/usr/local/bouncehandler/mybh.php $sender $recipient
   This defines for postfix a transport that is of the pipe variety. Postfix will pipe any bounces we tell it to, to the executable script in question with the given parameters.
2. add a domain entry to the transport map so that messages that come in for our bounces.mycompany.com domain are sent to the newly defined transport:
   bounces.mycompany.com mybh:
   this can be a file called transport.map in /etc/postfix.
   don’t forget to the call postmap on the file so that it becomes a map hash file for fast access by postfix.

Once postfix is ready, the script defined can then do pretty much anything we want it to do.

Here is the relevant section of the PHP shell script that does the decoding and updating of the table.


#! /usr/bin/php -q
$sender = trim($argv[1]); // should be EMPTY
$recipient = trim($argv[2]);

$bounceProcd = FALSE;

$conn = ConnectToDB();
if (FALSE !== $conn)
{
list($encodedFrom, $bhDomain) = explode(‘@’, $recipient, 2);
//sentto-brian=networkjack.info@bounces.mycompany.com
$encodedFromSQL = mysql_real_escape_string($encodedFrom, $conn);
$query = “UPDATE EmailAddressTracker “.
“SET FirstBounce = IF(FirstBounce=0, NOW(), FirstBounce), BounceCount=BounceCount+1, “.
“LastBounce3=LastBounce2, LastBounce2=LastBounce, LastBounce=NOW() “.
“WHERE EncodedFROM = ‘$encodedFromSQL'”;
// We keep track of the datetime of the last three bounces to allow time based policy
// to be applied

$qResult = mysql_query($query, $conn);
$bounceProcd = mysql_affected_rows($conn) > 0;

// We have to read the data that postfix is sending to us in stdin
// we don’t have to necessarily do anything with the data, but we could store it into a table for later
// processing if we couldn’t determine the original recipient or wanted to double check our results

$dataLen = IgnoreMessageData();
}

// if we couldn’t connect to the db or there was not a record in the table that matched
// our clause for the specific encoded FROM, then exit back to postfix with a
// Temporary Failure. This will cause postfix to queue up the bounce
// message for later processing
$exitStatus = (TRUE == $bounceProcd) ? 0 : 75;
// 75 = EX_TEMPFAIL per sysexits

exit($exitStatus+0);

function IgnoreMessageData()
{
$msgLen = 0;
$fd = fopen(‘php://stdin’, ‘r’);
while (FALSE === feof($fd))
{
$dunsel = fread($fd, 1024);
$msgLen += strlen($dunsel);
}
fclose($fd);
return $msgLen;
}
return;
?>

Notice that this script does NOT apply policy. It merely is there for statistical tracking and that is all it should do.

Policy of whether to allow any future messages to be sent to the user are applied in the Sending function, since that is closer to where the emails are actually generated. The bounce handling system has it’s one job and can do it well without complications.

So there it is. A simple and effective way of catching bounces for your web application.
My favorite part of this solution is the extremely minimal configuration required inside postfix.

Happy bounce tracking.