Could someone spell out a use case for this? One that comes to mind would be validating that a mailing list doesn't have any outdated email addresses in it, but couldn't that be determined by just checking for a bounce when a message is sent to it?
I tried to sign up for SiriusXM the other day, and though I could create an account with my .pro email address, I couldn't actually sign up for service with that same address for some reason. It's frustrating that validating email addresses is still something that people get so wrong. Please just take whatever seeming garbage I've entered into your email address field and try to send a message to it.
(Their site also had stupid password generation rules such that I couldn't use the 21-character one my password manager auto-generated, but even after I made one that followed the rules on the page, it was still rejected because there were apparently rules on the back end that weren't spelled out in the front end. Please hire me, SiriusXM.)
I receive a lot of fake user signups on the web tool I publish for free. These users (bots?) sign up using fake email addresses, but putting a captcha does not help much. I send a confirmation email to these users to make sure they exist, but if the email address does not exist, sending that confirmation email hits my account's deliverability (spam) score as it generates a bounce which my email sender provider does not appreciate.
For example, I've received multiple warnings from Intercom that I need to improve deliverability of my email, or they will ban my account. Ironically one of the suggestions is to use confirmation emails - but that's exactly where the problem is for me.
A tool like this helps me to weed out a ton of these undeliverable email addresses to avoid sending emails that will hit my spam score.
> checking for a bounce
So in my case, generating that bounce is exactly what I need to avoid in order to make sure my account remains in good standing.
I think what this is telling you is that users don’t value giving you their email address for your free tools. They know why you want their email address, and they don’t want that, but they do want your tools. Maybe it’s time to charge money for those tools or change models? Maybe not give access until the email address is confirmed? Worth considering I guess. Idk the details. I do this all the time. Ads and malpractice have made giving out your email address risky business. If a user can give you a fake email and get what they want without giving you anything (subject to your ads or whatever you’re using their email for) that’s the logical thing for them to do.
Not judging or anything though I know my tone might seem that way.
You seem to be assuming there's no reason these tools need a login system at all. Assuming they do and they're storing some kind of state or data for the user, how do you handle forgotten passwords if the user can't receive an email? A second free tier for people who can't be bothered to add a recovery email is too much work for a tool that's free anyway.
To the parent: Just send a validation email during the onboarding process. Don't create the account until they validate. And DO rate limit the number of these sent to any given address and the number of signup attempts by IP / fingerprint. Captchas are also good, but just for slowing fakes down.
Is it that risky? Talking, at least, as a somewhat knowledgeable person, which is the target of such web tools.
I usually have no problem using my email address to sign up for things. And yes, I do know that emails will come.
Sometimes they are actually useful and I keep them. Often, I don't care and I just unsubscribe. I some not-so-common cases, it is spam, in which case I mark it as such and I am never bothered by it again.
Sure, it's a bit of extra work. But I would not classify it as risky business.
The cost for me to give a fake email is very close to 0, and the risk of getting spam emails or who knows what is much higher in comparison so why bother giving anyone my real email address? I mean I do give it out from time to time (you can easily find it if you want) I’m just speaking generally.
You seem to be assuming “a lot of fake user signups” == “few real user signups.” OP only said the first. For all we know they also get plenty of real ones.
Home grown email systems require monitoring (e.g. deliverability rates, IP reputation monitoring) and maintenance of the system itself which can be a time sink.
The reality is without using SES, Sendgrid, or a similar 3rd party provider, avoiding people's spam folder is very tricky.
We were onboarding a large new client to our SAAS product. This process involved creating accounts for all of their employees (tens of thousands) and sending emails with an activation link. (Where they'd be able to set up their password.)
Our system sends these emails in batches, and as soon as the first batch went out we got an alert from our monitoring system that our bounce rate was surging - high enough to risk a sending pause from Amazon SES. We stopped sending and investigated the issue, and it turned out that the email list we were given was a mess - it included all current employees, but also a huge number of former ones. Just under 1/10th of the emails in our first batch were invalid.
We asked the client to give us a better list, but due to internal issues they couldn't get that to us any time soon. Meanwhile they were breathing down our necks to get these emails out ASAP, and they were a large enough client that our management wanted to keep them happy, so we tried out one of these email validation services. Unfortunately, it didn't work. It turns out that this technique doesn't work for all mail servers. It was reporting every email as valid, even ones we knew were invalid since they'd already hard bounced.
(Edit: thinking back - this was several years ago - I think it wasn't saying that they were valid emails, just that it couldn't tell whether they were valid or not - the service was able to detect that the server wasn't rejecting non-existent addresses.)
We ended up unpausing the emails and just hoping for the best. Ended up with something like an 8% bounce rate that eventually fell off our record as our normal sending patterns resumed. Amazon's guidelines say they might cut you off when you hit 10%, so we cut it pretty close.
This is clearly trying to solve a non-technical problem with technical means. The root problem is that AWS cancels you with too high bounce rate. The obvious solution is to talk to some AWS representative to at least temporarily not cancel you after explaining the situation. If AWS does not let you talk to them, then that's where the problem lies, not in some not cleaned up email list.
It's terrible to spend a lot of effort on this kind of tech just because some business partner has shitty customer support.
Is it? In my dayjob I'm solving technical problems with technical means.
World hunger is not a technical problem. You won't solve it with technical means. If you think you can, you have already lost the fight.
Climate change is not a technical problem. You won't solve it with technical means. If you think you can, you have already lost the fight.
And so on, and so forth. Technical means can help solving certain components needed for the overall solution. These are then technical (sub)problems though. For example, how to store more energy in a battery, or how to grow certain crops with less water. But the overall problems are social in nature. People need to understand that world hunger is a distribution problem. That one is easier to solve with certain (technical) tools available, but that won't be enough. People need to understand that we can't use more natural resources than get replenished. Not a technical problem. If only the tools get better, people will find new ways to be wasteful. Etc etc.
Climate change is technical problem. Carbon sequestering coal plants can remove all carbon from the atmosphere. World hunger is undoubtedly not a technical problem though.
World hunger is caused by power imbalances. That's a social/political problem. Whatever technical solution you provide helps worsening the root cause and at best keeps the status quo.
We have observed dramatic technical advances in all fields related to food production in the last 3 decades. We've seen at best marginal improvements in world hunger in the same time, and even that only if you're beeing optimistic with your statistics.
If they're checking using SMTP's VRFY command then it's actually considered a best practice for the server to always reply with a 252 "cannot verify" since otherwise it can be used to fish for valid addresses.
Yes, and you could be eg: graylisted. Or server could accept all an silently drop. Or rate limit.
It's sad that VERIFY is basically dead due to spam. In the olden days, you might have been able to use finger - but it's also dead for (among other) similar reasons.
The spam cat and mouse game leads to quite silly situations for benign actors.
For some mx's you might be able to designate some ips as trusted, and do real verify for those.
I can't even imagine wanting to handle managing accounts and credentials for that many users at an enterprise! At that point SSO integration is well worth the money. How did you handle removing access when a user was no longer employed at the company?
Not OP, but also building a similar user system. I can totally understand the motivation to not use the internal SSO. With most companies I know, as soon as you actually connect to their private datasources, you have to do some extra steps to prove how you're securing your platform. This makes sense from the companies perspective, but also introduces a huge technical and organizational overhead for the startup which might be better spend elsewhere if your product does not absolutely rely on SSO
It seems like the most practical solution to that should be calling AWS, explaining it for 5 min, and getting an exception. Is that kind of reasonable solution no longer possible with the cloud providers being so huge?
Ok, so first pass gives me a TERRIBLE idea, that would "get the job done". I'm sure you thought of this and dismissed it:
Keep your "overall bounce rate" low, by ALSO sending out extra emails to confirmed email addresses. Like, for every "confirmation" email, also send a "thanks for joining us" email to someone that already confirmed their email.
I'd hope you'd at least be able to explain the situation to your account manager and get an exception(maybe for that single companies domain?), but I've never used AWS so I wouldn't know if thats possible.
This is a reason you need an escape hatch from SES.
In the past when I worked on a system that needed to notify via email we always had a way to change delivery process for certain emails, domains, etc for exactly this reason. This is one of those cases where we would “deliver directly” (i.e. send directly to their mail provider).
Have you looked into what SES or other email services provide? Sending emails is easy, while actually getting them delivered is harder. You have to make sure you're not getting flagged as spam, can handle bouncebacks, etc.
Not getting flagged as spam isn't actually that hard, though. Besides, if you're using SES or some other hosted SMTP service, you still have to set up SPF for your domain, so you haven't even really gained much comfort. The only really useful thing is to gain an ip address with a high reputation, but you can generally get those at any reputable hosting provider as well. Just don't try sending emails from your residential internet connection.
The use case is marketing email. Sending to non-existent addresses can radically affect deliverability.
If the address is to a large host, then they will use reaching invalid email addresses as evidence that you are not keeping to best practices. They will throttle deliverability, and possibly reject email.
If your sending to an invalid host, then your mail sending provider (if your using one) may consider you a bad customer and send you through a lower grade of outbound IP addresses.
Frequently new registrations are processed at once as batch imports from another system or from a partner. There is a need to remove these invalid email addresses pre-sending and hurting sending reputation.
Used to be in the same camp, before starting my own company. While the border is thin, it is clear, and there is a difference between marketing and spam.
If that was the case, all newsletter senders in the world are spammers too. And all spammers thinking they are marketers is not logically equivalent to all marketing is spamming
I have! Expensify. Not a customer, and never have been. Signed up because I was considering using the service. The emails are exceedingly infrequent and delightful to read.
That’s you (and the large part of HN who doesn’t know anything about normal users). When GDPR came into effect, we even had people write us that they were annoyed having to reconfirm, as obviously they still wanted our newsletter.
Some people want to receive marketing emails. The exact same emails are spam to the rest of us. If you default opt-in users and don't make blanket unsubscription from all marketing emails easy (and make the clear distinction [both internally and externally] between emails regarding the existing functionality of products/services that we've already paid for and emails to ask for more revenue from us, which includes copy supported by advertisers), then you're a spammer, simple as that.
Spammers aren't necessarily evil, but definitely annoying, and annoying customers is a good way to lose them. It's easy to drink the coolaid and eat the dogfood when you're paid to like it or have a passion for creating it but potential customers have roughly the opposite incentive and it pays to remember that.
I receive a lot of unread, unwanted email from companies I pay money to every month. It's ridiculous.
There's some things right on the edge, like Netflix as an example. They're never trying to sell me anything; they've got my $N/month. The emails (that I don't read until now to sample them) are suggestions of shows I might like. User retention mail and promoting the shows they own the rights to are spam, but not all of it is.
My credit union sends infrequent warnings about recent phishing and scam techniques. I never opted in to those emails but they're not useless for everyone and possibly do net good.
I think a good heuristic is to look at the value provided in emails; if the recipient stands to benefit significantly more than the sender (on average) then it's not spam. Sure, my credit union has to deal with less hassle reversing charges or resetting passwords but it's quite a lot more hassle to be a victim of phishing or other scams.
I have a common name first year/generation gmail account. The volume of mail I receive that I didn't solicit is mind-boggling. Not just spammers, but it's obvious that people have saved the wrong address in their browser suggestions, and that people give it out when asked for an email addresses at retail. Disney employees, California private school parents, iPhone receipts and apple IDs. Taxes, warrants, bail bonds, social security information. People are f'ing stupid when it comes to email.
Everything should require a confirmation before you assume it's valid. Not because it doesn't exist but because it might not be who you think it is.
I maintain a small fee tool for creating API endpoints to trigger email alerts for yourself (varmail.me). Since it's very basic and has no marketing, it doesn't get too many users. You have to click a link in an email to even log in, but I still get bots stuffing my login form with known bad email addresses (I've googled and some of them appear in honeypot lists). There is no chance they can get in, but this is bad because my login verification emails look like spam, since a large percentage of them are sent in response to these bots. So I definitely see some value in a way to pre-screen the emails there.
Yes, I've considered it. The thing is I really dislike captchas so I tried to avoid deploying one. I do have a hidden form field that gets populated with a secret value in JavaScript, so the bots must be running JS at least to function.
Personally, I do this kind of thing manually if I want to send some positive feedback to people who work at companies I like, but they don't have any obvious emails published.
A recent example was the CEO of Evernote for the work put in to their behind the scenes series although I don't expect anyone to read it of course. People are busy!
Now does this scale? Not at all and I haven't read the email spec or anything like that. It's also handy in a pinch if you wrote down an email but can't remember if it's spelled correctly or not.
The practice of email confirmation is still widely used, but the change in email deliverable rules has make it a pain to properly validate them.
Even if you are using a 3rd party provider like SES or mailgun, they have a email bounce limit. A considerable number of real world users give fake email address(which is even sometimes encouraged on HN) which triggers those bounce limits.
To fix it, there are paid services but they does not work very well. Fixing it yourself take a lot of engineering time, that is better spent elsewhere.
Providing an open solution to this problem (which is given in the github repo) is a double edge sword. As this gives a edge to spammers who created the problem in the first place.
all AI is plagiarizing something, though. Computers cannot reason, they can just jam stuff together that satisfies whatever the program says.
ML is matrix math (generally) - stuff that you can do by hand, computers are faster at it. As fancy as GPT and co-pilot are, it's all copied from somewhere.
Yeah basically, but it doesn't validate an email address (user@example.com), it validates say, the From: header of a MIME message, which could contain an email address, a phrase (like a first and last name) comments, and all sorts of dumb dumb ideas like having phrases that have embedded comments and phrases that look like email addresses but aren't and on and on and on. This RFC is a trainwreck and whoever wrote it should feel bad about themselves.
>but couldn't that be determined by just checking for a bounce when a message is sent to it
The fewer times you bounce, the better your chances of not being marked as a spammer.
This could also be useful for an ecommerce site, where you want to be able to easily contact the buyer if there's a delay, address correction needed, etc. People typo their own email at a rate that's surprising.
I use email validation via MailGun for exactly this purpose with a productized service business. If we don't have a good email, then we can't deliver the service once complete. That leads to angry customers, even if the issue was a typo when they created the account. Easier to try and catch it during signup rather than at the time of delivery when emails start bouncing and you have no way to get ahold of the person.
I bought a new car a few years ago and I was unable to enter my .life email address into the SiriusXM registration form in the car. Had to use a .com address :(
I have a hyphen in the domain (.com) I use for personal email (it's shorter, just my initial and surname) and I still have had problems in some places with that - it's so frustrating.
pipesandcigars.com let me sign up with a .miami but now I can't login because it says it's invalid. My ISP frantically called me up after starting service because they took all my info down and then the system didn't like the domain. My utility company kept sending me requests to go paperless and I had to call them to get an answer on why that happened since I had accepted the paperless prompt, the issue was the domain.
It's nice to have but I've had quite a few headaches with it
I had a similar experience with LG. You need a developer account in order to install applications on webOS TVs. I successfully registered on the LG Developer portal with something like myname+us.lgaccount.com+2021-07-01@mydomain.com, but I couldn't log in on the TV with that email address. Had to register again without the plus signs.
I had something worse(?) happen with one of my accounts for a hotel chain. I entered name+hotel@majoremail.com and they just stripped out the + leaving me with an account that I couldn't use unless I made another account at the email provider.
I am not sure there is any clear legitimate use for deep validation (beyond checking for syntax). If you are providing a valuable service to a customer, wether it's a catch-all or free mail shouldn't matter to you.
Now, if you are buying lists here and there to spam the hell out of it, the bounce rate would flag you very quick and you'd need to find another smtp provider every week. This service would be your life line.
Similar experiences here. I have [myfirstname]@[mylastname].party as my primary email. In meatspace it’s a curiosity, and people are sometimes incredulous. Online, I’m routinely given some variant of “enter a valid email address.”
Any kind of technical email validation is horribly unreliable at best and there is no chance that this GitHub repo is going to work any better.
As a result the only place such a service is useful is for someone who has a ton of low-value emails they don’t trust, and they don’t want a ton of bounces when they hit send (which risks losing your send privileges with pretty much high-volume email platform).
So they run all their emails through a service like this, and only send to the ones marked valid. This excludes a ton of emails that actually are valid, but failed the check (false negatives). But that’s ok because the emails were low-value to begin with.
If this sounds like a spammy operation… bingo. Technical email validation services are really only useful for people who are doing things like buying email lists from commercial providers, harvesting emails from sites like HN, or forcing people to enter an email address to do basic things with a free service.
Financial/fintech companies use services like these for fraud-detection on account opening. While validating an email is by no means and exhaustive and conclusive signal to classify a fraud/genuine user, verifying the validity of new customers's email addresses is a big help.
If you use Amazon SES and have a high bounce rate they will just block you. So, Eve could flood your registration with invalid emails that would cause bounces, that would stop all your emails from getting delivered. A DoS you can only recover from by switching to a different email provider
Given their questionable business practices, their customer service dark patterns, their dated and awful UX, and their inevitable demise to much more popular streaming services, you’d be best to stay far away.
The service I work on emails users(B) after their email address has been input by another user(A) of our service. This would allow us to check that the email is reachable before we attempt to send the email. If it is not we can ask user(A) to check before they proceed.
I sometimes use it for sales. Looking up a company on LinkedIn, then finding the name of a person I want to each our to but am not sure about the email. Then I could try the classic first name.last name @ domain to see if it’s worth sending an email
I use to have a active product DearEle.com an email reminder service. There was a feature to cc everyone in email, which is used by spammers to send junk products to everyone. The source emails were all fake. Email check prevent fake emails.
If you're asking for an email address in exchange for some piece of content (e.g. a soft paywall), I find that it's reasonably sensible to filter out the absolute spammiest-looking of email addresses in order to collect more real email addresses and prevent people just rolling their face across the keyboard and calling it a day. (It can also help with bots a little bit, although most bots are smart enough to use <random firstname>.<random lastname>@gmail.com or something). The best way to do this is obviously double opt-in, i.e. actually emailing the address in question. Most businesses believe (probably accurately; I don't have the stats) that this creates too much friction. I've used Sendgrid's email verification service in the past, and the actual scoring is basically garbage but if you set the threshold really low (e.g. reject all emails with a score between 0 and 0.1) then your precision is really good (almost every email address you reject is invalid, as measured by actually sending an email to them and the email hard-bouncing or being otherwise undeliverable). You let a decent number of bad emails through but it's an improvement over nothing.
I tried to sign up for SiriusXM the other day, and though I could create an account with my .pro email address, I couldn't actually sign up for service with that same address for some reason. It's frustrating that validating email addresses is still something that people get so wrong. Please just take whatever seeming garbage I've entered into your email address field and try to send a message to it.
(Their site also had stupid password generation rules such that I couldn't use the 21-character one my password manager auto-generated, but even after I made one that followed the rules on the page, it was still rejected because there were apparently rules on the back end that weren't spelled out in the front end. Please hire me, SiriusXM.)