How To Ensure Your Email Gets Delivered

It goes without saying that email is the de facto method by which online businesses communicate with users and clients; without a doubt, it’s a necessary and core function of business today. That being the case, how does a business know that its email is actually getting delivered?

Despite how significant email is, there’s confusion and a general lack of education about why email may or may not make it to its destination. So, I’ve spent some time putting together some helpful tips, and just as importantly, a list of things you want to be sure to avoid.

Things You Can Do

###1. Capture and Process Your Bounce Mail Bounce mail is mail that gets returned because it cannot be delivered. There are numerous reasons for this; the receiving email address does not exist, the mailbox is full (usually an indication that the mailbox is inactive), the message is too large, rdns failed, too many sessions opened, etc.

Bounce mail must be processed and the email addresses from the bounces removed from your mailing lists, because ignoring it and continuing to send to email service providers who monitor bounces is a sure way of getting flagged as a spammer and possibly put on their black list. The best way to access bounce mail is to poll the incoming email address using IMAP or POP3. Luke Francl details a few ways you can extract your bounced mail on his Rail Spike blog. ###2. Setup A Feedback Loop With Major Email Providers A feedback loop is the service where an ISP lets you know if it has received complaints about your email. Even if your opt-out process is clearly laid out, some users will still hit the spam button to stop receiving emails. This may affect your future email delivery with that email service provider, if they keep track of how many of their users report your email as spam.

To use the feedback loop service, send a request for an ISP feedback loop via email or through a web form.

The usual information you need to provide to sign up for a feedback loop is:

• The IP addresses for which you want to receive feedback loops
• The list of domains you want to subscribe
• A valid abuse@ and/or postmaster@ email address on the domains you want to subscribe
• Your contact information: name, contact email and phone number
• The email address a feedback loop will be sent from
• The number of email messages sent daily (weekly, monthly) to the domain of the feedback loop
• The type of email content you send
• The URL of the page with your opt-in form
• Single or double opt-in subscription process

###3. Generate Your Own Mailing Lists Be careful about using lists you didn’t put together yourself, like purchased lists. Obviously, if you’ve bought your list, you’ll be sending mail to folks who did not explicitly ask for your email; and this can lead to numerous hard bounces.

All of these will increase your so-called spamminess factor, possibly leading to local black listing. Sending lots of unsolicited mail will eventually lead to your IP being blacklisted on third party email blacklists out there, like spamcop or spamhaus, and some email service providers will reject all email originating from IPs on these third party lists.

When users sign up to receive email from you, use a closed loop opt-in (also known as double opt-in). This means that the user signs up on your site and an email is sent to their email address. They are not _actually _signed up to receive email from you until they reply back to that email or click on a link within it.

This is pretty standard email etiquette, and leads to clean lists with minimal bounces. Once your users are signed up, be sure you’re sending relevant email related to your services, to minimize spam reports.

###4. Don’t Use 100% Image-Based Content for Your Email Campaigns Minimize the use of images as much as possible, as most users have images blocked by default. If you use images for important content like headers, links or interactive content (like a “Buy Now” button) realize that many users won’t ever see it, and the email won’t be nearly as effective as it could be. Also, having many images or an all image email will increase its chances of ending up in the spam folder, which we now know can lead to the email service provider flagging you for spam. ###5. Test Your Email Content for Spamminess Receiving SMTP servers process incoming emails and scan the content, analyzing them heuristically for general spam patterns, with tools like Spam Assassin. Run test emails through a content analyser like brandonchecketts.com/emailtest.php for spamminess and adjust your content to lower your spamminess score. This will improve your chances of making it to the inbox, instead of being routed directly to the spam folder. ###6. Validate Your HTML If sending HTML, make sure your HTML email validates (http://validator.w3.org). Email clients are notoriously finicky; if your HTML doesn’t validate right, providers may not display the content the way you want it. A malformed HTML email is ineffective and has a higher likelihood of getting put in the junk or spam folder. ###7. Simulate the End-User Experience Test on multiple email clients; there are various tools that allow you to simulate the user experience. litmusapp.com is a good example of such a tool. Use it or some similar tool to preview your email campaign on the major email clients out there. Obviously, you want it to look the way your design intends it to on as many clients as possible, for maximum effectiveness.

Things Engine Yard Can Do for You

Engine Yard bundles an SMTP outbound service with its private cloud services (Slices, Fractional, Clusters), but recommends the use of a third party SMTP service (like authsmtp or sendgrid) for Engine Yard Cloud customers running on Amazon Web Services. These tips are for private cloud customers, or for customers running their own SMTP service. ###1. Setup a Dedicated Email IP By default, email generated by all clients in a cluster in our private cloud is sent from a shared IP. If one client is sending a lot of email, an IP could get flagged by an email service provider and mail for all clients going to that service provider may be throttled. This affects all other clients sending mail using that shared IP. Having your own dedicated email IP insulates you from issues like this; you will not be punished for others accidental or bad behavior. Check out this link to our knowledge base for more detail about getting setup with a dedicated email IP with Engine Yard. ###2. Setup SPF Validation SPF (Sender Policy Framework) is an e-mail protocol that fights return-path address forgery and makes it easier for ISPs to identify spoofed addresses. If widely adopted by senders and receiving SMTP servers, protocols like SPF would cut down on a majority of sent spam. This is because SPF validates the sender of the email (the Return Path: field) using a DNS record and delivers only if the sender is who they say there are.

You can read a bit more on SPF in the Engine Yard knowledge base as well. We recommend you have SPF set up to improve email delivery, especially to email service providers like hotmail (MSN) that use it as a factor when determining your spamminess. If you’re a private cloud customer and we host your DNS, all you need to do is open a ticket requesting SPF setup, and we’ll do the rest. ###3. Setup Domain Keys Verification Domain Keys verification is another e-mail protocol that attempts to deal with sender verification, like SPF, but using a different method. Essentially, mail servers generate a public/private key pair and sign outgoing messages with the private key, while publishing the public key as part of their DNS record. Because only they can publish to their public key in this way, the signature can be used to confirm that the sender of the email has not been spoofed.

The presence or lack of a signature can be used as part of the process of identifying spam by email service providers; it could also be a powerful tool for fighting phishing, if widely adopted by senders and receiving SMTP servers. Once again, if you’re a private cloud customer and Engine Yard hosts your DNS, simply open a ticket requesting domain keys verification. ###4. Reverse DNS Most email service providers require all sending IP addresses to have a valid rDNS. rDNS lookup failures lead to delays in sending or complete rejection of your email, depending on the service provider. Also, what is considered a valid email address by some providers is not necessarily a generic rDNS value, but either the same or a derivative of the forward DNS value. This latter check is not widely adopted, since it would much rejection of valid emails, but some do use it.

Getting Off Email Service Provider Blacklists

So you did all or most of what we’ve talked about so far, but you still got blacklisted by a provider or two (Gmail, Hotmail, Yahoo!, AOL, AT&T, etc). Take a breath: it happens even to the most legitimate senders. First, analyse the error that came back with the email. Your next action will depend on what type of an error it is. Even though email delivery codes are well defined in RFC 3463, the error code that you are likely to get from a bounced mail will be specific to the email service provider.

As an example, Comcast may send back an email with a 550 error and a block code of BL000001. This means that the email was rejected because it was listed in the Spamhaus Zen blacklist. The course of action in that case is to get the IP off the Spamhaus blacklist. If you’re a private cloud customer, we will help with that, of course, since we are responsible for the IP.

In your bounced mail notice, the email service providers will mention why they are returning the mail. Generally errors codes that are 4XX are transient errors which soft-bounced (the mail delivery can be tried later). Usually, receiving SMTP servers will soft bounce an email three times before it hard bounces. A 5XX error is permanent, and means that the mail has been outright rejected. Common reasons for rejection/defered mail and your course of action are:

1. __ Too many sessions open__: Usually results in deferred/throttled mail. Getting a dedicated IP or additional IP’s will spread the sessions. If you’re using the private cloud SMTP service, we have the number of max sessions set low so as not to trigger this error.

2. Reverse DNS failure: The email was probably permanently rejected. Get your provider to set up a valid rDNS for you. Engine Yard has a default rDNS entry for all IPs, so this is not an error you would normally get with our services.

3. Too many emails sent in time window: An IP sending too many emails within a time window will be throttled by email service providers so email delivery will be delayed. Each provider has a different threshold, so there is no hard and fast rule as to how many to send at a time. At Engine Yard our SMTP setup is set so that deferred email will be queued on special deferred SMTP servers, leaving the main SMTP servers free to send “fresh” email undeterred. If you’re sending lots of email using the default shared email, we monitor the queue and recommend getting a dedicated email IP, so as not to affect other clients.

4. Too many recipients in message: This will likely result in rejected mail. The solution is to send mail to recipients one at a time. If the email delivery fails it only affects that one email address.

5. Invalid sender domain: The email was permanently rejected. Ensure the sender domain is valid or the email is sure to be rejected.

6. Account not available: The email was permanently rejected. Remove this email address from your mailing lists.

7. Account mailbox full: The email was permanently rejected. Remove this email address from your mailing lists; most full mailboxes are an indication of an unused mailbox.

8. Mail has characteristics of spam: The email was permanently rejected. Analyze your content for spamminess and adjust to reduce your spam score.

9. Sender IP on third party email blacklist: The email was permanently rejected. Since the email is legitimate, have your provider help get the IP removed from the third party blacklist. We don’t experience this much at Engine Yard, which is encouraging. It means our clients are generally sending email that is relevant to their end users.

10. __Sender IP on local blacklis__t: The email was permanently rejected. Virtually all of the major email service providers have web forms that you can fill out to request removal of your IP from their blacklist (and possibly get whitelisted as well). To ensure prompt removal of yourself from a local blacklist the providers generally check if you have good email etiquette and follow the following rules:

• Have a valid rDNS
• Have SPF set up
• Have domain keys set up
• Have signed up for feedback loop
• Have a valid abuse@ and/or postmaster@ email address on your domain

Email can be simple, but it can also be frustrating if done incorrectly. Considering how essential it is though, you definitely want to get it right. May all your email be delivered!