One of the keys to mastering email and deliverability is knowing how to measure success or failure. However, deliverability is complicated and despite what marketers are told from popular third-party deliverability services, there is no cut-and-dry deliverability score that will accurately predict whether you will land in the inbox every time. Furthermore, mailbox providers (MBPs) and Email Service Providers (ESPs) don’t tell us what our deliverability rate actually is. Bummer.
That leaves us to know what metrics to monitor in order to gain a clearer picture of our deliverability rate and the health of our email program. In this episode of our Email Deliverability Unfiltered Series, featuring special guest expert Alyssa Dulin, Deliverability Lead for ConvertKit, we’ll break down the fundamental metrics that matter most for email.
Foundational Deliverability Metrics
So what metrics matter with deliverability? While performance-based metrics like Click-Through Rates (CTRs) and Conversions can help us understand our email programs’ performance, others are more foundational to our email program’s performance. So let’s start by reviewing the metrics that matter most for monitoring deliverability. These allow you to confidently know whether you are likely to land in the inbox of your recipients.
As we mentioned in the first episode, “What is Email Deliverability”, your ‘Delivery Rate’ is very different from your ‘Deliverability Rate’. The delivery rate confirms that your mail has been accepted into the MBP’s servers. In fact, the MBPs actually send a message back to you (a server response called a “250”) confirming the mail has been successfully received into their system.
The “deliverability” piece — what people refer to as ‘measuring inbox placement’ — is what happens after that point. MBPs do not send any kind of feedback that lets you know if your message landed in the inbox or the spam folder.
Bounced emails are addresses that could not be delivered successfully to recipients. In these cases, you’ll get a bounce response from the MBP telling you the message was not delivered, including some codes and other information that should help you understand “why” the message wasn’t delivered.
Bounces can be a bit cryptic, pointing to a transient failure, or pointing to your messages being flagged as spam in some way. But in general, they typically fall into two categories:
- A hard bounce indicates the address you are contacting either does not exist or is not accepting messages at the time that you tried to contact them. Welcome programs tend to have a higher hard bounce rate than others, but in general, a healthy bounce rate is one which is 2% or lower.
If you find that your bounces are exceeding 2-3%, you’ll want to focus on how you can improve data capture using tools like CAPTCHAs on all of your sign-up forms or a double opt-in process. Also, consider a solution like Kickbox’s real-time email verification on all of your points of signup (anywhere someone is giving you an email address), which can reduce the number of invalid addresses that are making their way onto your list at the point of signup. Not to mention, help ensure you have a way to start an email relationship with every lead that comes your way.
- A soft bounce usually stems from an issue with sender reputation. In most cases, the bounce message you’re seeing will mention spam or refer to being blocked due to policy reasons, content reasons, etc. Soft bounces can also occur when the domain you’re sending to doesn’t exist, or simply isn’t accepting messages at the time you attempted to hit ‘Send’.
In these cases, pay close attention to what the bounce is telling you. Is this just Yahoo having a bad day, and you should try sending again later? Or is this a soft bounce driven by reputation, in which case you should look to back off your sending and resolve the issue: whether it be with complaints, spam trap hits, blocklistings and/or something else.
A great example of when soft bounces are pointing to a reputation issue is with Gmail. This MBP doesn’t actually block legitimate mail very often…they typically just send more and more of it to the Spam folder over time, until your sender reputation has become very bad. If you’re seeing even a few soft bounces at Gmail, take a look at your Google Postmaster results and explore your metrics other than bounces to identify what might be causing the problem.
In general, people should care about bounces because sending to too many invalid addresses, or not backing off when a MBP tells you to, can negatively impact deliverability. The type of bounce you’re seeing, the message that’s provided, and the network you’re seeing it from all tell you something very specific. Work with your ESP, ask for help in the #emailgeeks Slack group, or book time with your favorite deliverability professional to understand the details in your bounces and how to deal with them most effectively.
Email opens are tracked using a tiny, invisible pixel at the bottom of an email. When an email gets opened, the idea is that a pixel should be loaded, and an open will be tracked.
As Alyssa shared with us back in a previous blog article about overrated deliverability metrics, open rates can be both overrated and misunderstood by marketers for many reasons. In fact, she goes into deeper detail about why legit opens might not be tracked, while in other cases, opens may be registered when a recipient hasn’t actually opened your email in an episode of her ConvertKit podcast, Deliverability Defined.
But when open rates are tracked over time, they can be used to tell more of a story than simply how effective your subject line and From Name combination is. Tracking the trends in your open rates can help you quickly identify sudden, unexpected drops that can be an early indicator of a deliverability issue.
Tracking open rates at the destination level can also help you identify if you’re potentially going to spam in certain places. Imagine, if your open rate is 30% across all of the top MBPs you send to, including Gmail, Yahoo and a few others, but it’s only 6 or 10% at Hotmail, you likely have a spam folder issue with Hotmail. And now, you can focus in on how to best solve an issue specifically with Hotmail.
Also referred to as “marked as spam” rates, complaint rates are typically the most heavily weighted metric by MBPs, because they are a signal directly from a recipient saying they don’t find value in the mail. Spam complaints can show up in your stats, even if you’re collecting opt-ins from your users, which is why it’s important to properly set expectations with recipients at the point of sign-up, target responsibly and also make it incredibly easy for them to opt-out.
Many ESPs will allow their customers to generate a complaint rate that is around 0.1% before intervening, but it’s important to note that this is not meant to be a target! It’s just the point at which your ESP will rate limit or suspend your account. And just because you’re not in hot water with your ESP, it doesn’t mean the MBPs will feel comfortable delivering your mail to the inbox.
If you’re aiming for consistent inbox placement, you’ll generally want to keep your complaint rates below the 0.02 – 0.03% range. This is what MBPs typically view as a “healthy” complaint rate, and considering how much weight this signal carries, the closer to zero, the better.
Also, keep in mind that a complaint rate of zero is also an indicator of a problem, as recipients can’t mark an email as spam if it’s already in their spam folder and the recipient’s MBP may not be providing you complaint data via feedback loop.
In general, our recommendation is to aim for an unsubscribe rate that is consistently below 0.6%. Unsubscribes tend to have a much smaller impact on deliverability than complaints, but in a very similar fashion, your subscriber is telling you they want off the ride. Take these seriously and keep them very low, because the next time, someone might decide to mark the mail as spam instead… that’s usually much easier to do.
At the same time, do not take a low unsubscribe rate as a sign that your recipients are engaged and loving your content, simply because they haven’t taken the time to opt-out. If your low unsubscribe rates are paired with open rates that are in the single-digits, it’s time to consider some changes in targeting to focus on more highly engaged users.
Other indicators to consider
In addition to the metrics we mentioned above, there are other data points that you can use to understand your deliverability even further. These include:
While the metrics we mentioned above are foundational to monitoring your deliverability and driving strong ROI with email, pairing them with more performance-based metrics such as click-through rates and conversions can help ensure your email marketing is truly optimized.
This is a great time to remind you to be sure that your performance KPIs match your goals with email. Because you surely aren’t sending emails just to trigger opens, right? Make sure people are converting!
If inbox placement and open rates are high but click-through rates or conversions are low, then it’s time to take another look at your content, your offer, and your lead sources to figure out where the disconnect might be.
Often referred to as “inbox placement testing”, this is a process where you can send your email to a list of seed addresses used for testing email delivery and inbox placement. Email marketers often create test accounts at the major MBPs (including Gmail, Microsoft, Verizon Media Group, etc), along with other test addresses at domains where they have a lot of recipients (such as corporate domains).
We’re likely going to have some upcoming content dedicated to this topic, so for now, we’ll just mention that seed testing results can vary greatly from actual delivery to your recipients since MBPs can very easily tell the difference between a seed address and one owned and managed by a real-life human.
Seed tests do give you more information and another area where you have the ability to monitor trends over time, so they are still a helpful tool in a marketer’s toolbox. You just need to take ‘inbox placement’ testing results with a grain of salt (particularly with Gmail).
Real-time blocklists, also known as RBLs, are used by MBPs to identify IP addresses and domains that are known to send spam. Blocklistings are usually triggered by spam trap hits and/or user complaints, indicating a problem with list collection or management.
There are 100’s of blocklists in operation today, and not all of them will doom a sender to email jail. Any listing with Spamhaus should be taken very seriously — like, drop-everything-else-in-your-day seriously — because a majority of MBPs will block your mail if Spamhaus says to. But other blocklists may only affect delivery to certain regions of the world or a specific types of destinations (for example, SORBS listings mainly only affect B2B senders).
MX Toolbox and multiRBL.valli.org are the go-to’s used by most deliverability pros. Bear in mind that each of these tools includes a multitude of blocklists in their search results and not all of them will be applicable to the destinations you send to. Each blocklist operates differently, so if you notice you’re listed somewhere, work to understand what’s driving the listing and how it might be affecting your inbox placement before losing your cool.
Many MBPs also manage internal blocklists that have been built over the years. There is no tool to search for these types of listings, so rely on your bounce messages to let you know if you might have triggered a listing with a MBP.
Lack of engagement
Although negative engagement is the most important to avoid, MBPs have also begun to weigh more heavily on a lack of subscriber engagement as something that indicates a problem and can therefore impact deliverability. If you have subscribers on your list who never open, never click, and also never engage on your website, it might be time to consider dropping them from your mailing list, or at least reducing your sending frequency.
Important things to remember
There are a few important things to call out about the deliverability metrics you monitor:
- Take inventory of all your metrics – Be sure that you’re focusing on ALL of your metrics, not just 1 or 2. This can help you identify anything that stands out as problematic and course-correct before it becomes a major issue. For example, a high open rate might seem like a success, until you notice that you also generated very high spam and unsubscribe rates as well. Consider the full picture before you start the celebration.
- Follow the trends in your data – reviewing the results for 1-off campaigns lacks proper context to know if your program is fully optimized, and doesn’t allow you to spot problematic trends. Taking a moment to step back and view the bigger picture will help you identify sudden spikes or drops in user engagement (indicating quick action might be needed), as well as note when certain metrics are trending in the wrong direction slowly over time (suggesting a change in targeting might be needed).
- Your past results are your best benchmark – The benchmark reports created by ESPs and other 3rd parties are a good starting point if you have no idea what a good open or click rate might be, but keep in mind that each ESP may be reporting on metrics a little differently. And your email program may work very differently from others, even those within the same industry. So it’s best to benchmark against your past self and track your performance over time instead of comparing one-off campaigns against competitors.
Want to make it easier to see all of your email metrics in one view? The Analytics dashboard within the new Kickbox Deliverability Suite provides you with a birds-eye view on all of your metrics, across all of your sending platforms, while also allowing you to drill-down into the nitty-gritty details if you need to investigate further.
Get your free consultation today to see if the Deliverability Suite is right for you.
Since MBPs and sending platforms don’t share inbox placement information with us as senders, it’s important to adopt a holistic view to monitoring our metrics, in addition to other relevant data points, so that we can paint a fuller picture of our deliverability. As much as we can, it’s critical to focus on ALL of these metrics instead of just 1 or 2… and to look for anything that stands out as problematic.
It can take a bit of time to become familiar with your email programs’ deliverability metrics, especially when managing several sending domains and high volumes, but it will allow you to significantly increase your ROI once you understand what to monitor and when to take action.
Stay tuned for the next episode
In the next installment of our series, special guest Matthew Vernhout will be joining us as we explore the ever-changing world of authentication and how you can use it to reach the inbox more consistently.
Not caught up on the series? Check out our last episode about understanding sender reputation, how it impacts email deliverability and what influences this multi-faceted score.
Get Your Free Deliverability Consultation
Speak with a deliverability expert to see if the Kickbox Deliverability Suite is the right fit for your organization.