💾 Archived View for dioskouroi.xyz › thread › 29448204 captured on 2021-12-05 at 23:47:19. Gemini links have been rewritten to link to archived content
-=-=-=-=-=-=-
________________________________________________________________________________
This is a low effort proposal and the author seems blissfully unaware of the long and varied history of proposed alternatives to email.
Alternate proposals have failed and will always fail due to the ingrained nature of email. Nobody is going to use a new email system that allows communication to zero or very few recipients, and the system can never bootstrap. And that's that, no alternative to email will succeed if its only claim to fame is that is "email done right", that's a much too weak feature to break SMTP's decades of first mover advantage.
To break this clinch, you need to either:
1. deliver a communication product that has much superior features, and incidentally does mail very very well; people will start to use it for the other features, play with its email subset functionally, and in many, many, many years, the email will wither and die. Up to now, despite many claims to the contrary, no one managed to do that. The average lifespan of instant messaging protocols is around two decades (anyone on ICQ or Yahoo anymore?), while email is going strong into its 50s.
2. backport those advanced features into SMTP so that they are interoperable with the existing infrastructure AND, at the same time, deliver a tangible benefit to the people using those features, AND deliver those benefits today, not into some distant future where most people start to use it (PGP, where are you). This is a very hard set of conditions and very few technologies have managed to fulfill them, as a consequence email has evolved very slowly.
So if you don't want to have the old "canned letter response making fun of your antispam system" rubbed in your face, you better start working on number 1. or number 2.
The success of a wide variety of messaging and discussion apps demonstrates that the network effects of email are not necessarily a high barrier.
What these products have lacked, which email has, is an open protocol. (XMPP had early success, but never addressed the main use cases of email, and has major design flaws.)
A handful of credible alternative protocols are now in development:
DIME -
https://en.wikipedia.org/wiki/Dark_Mail_Alliance
Math Mesh -
TMTP -
(my work)
Email was designed in the 1970s & 80s for an Internet whose population and topology have dramatically changed since.
We no longer need a messaging system that allows anyone, claiming any identity (i.e. "real name"), to send you any content, without limits. Federation was necessary for the topology of the early Internet, but today it is a drawback --
making email by far the most effective method to initiate cyberattacks.
Features can be added as long as they are backwards compatible, by adding another tag for it.
For example, the tag could be your public encryption key. Then, your email app, whenever it receives a public key from a sender, the app now knows their public key and can send them encrypted messages. Those without the public key, get the plaintext email.
This can all happen completely without any action on the part of the user.
Over time, it will gradually become ubiquitous, like https did.
Attempting to bolt public key onto encryption is one of the many things that’s already been attempted multiple times. We can see from those attempts that it’s not worked, for a variety of reasons.
Why does the variant I suggest not work?
How do I trust the public key I get via a tag in an email from somebody whose public key I don’t already have?
How does the person rotate keys if we only talk occasionally?
Or, if we’re looking for a bigger, overall flaw: e-mail is fundamentally fire-and-forget via SMTP, so there isn’t a way to use the users pubkey to creat a per-message session key, and reusing a persistent key for encrypting data is a massive pitfall.
> How do I trust the public key I get via a tag in an email from somebody whose public key I don’t already have?
You trust it as much as you trust getting getting unencrypted email from your friends. The point, though, is that from then on, nobody else can eavesdrop on your communications with the person who sent you their public key. You can still verify it by calling up the person and asking if he sent it. Or you can have him send the key via snail mail.
> How does the person rotate keys if we only talk occasionally?
The same way you got the original key.
> so there isn’t a way to use the users pubkey to creat a per-message session key, and reusing a persistent key for encrypting data is a massive pitfall.
Is it that massive? SSH uses keys that aren't changed every time it's used it, and it's still viable.
Besides, it's as secure as any public key cryptosystem is, and that's a heck of a lot better than plaintext.
SSH uses asymmetric crypto to authenticate, and then negotiates session keys. That’s how most modern PKI systems actually work. TLS is a great example: all actual traffic is encrypted symmetrically, and all modern ciphers use forward-secret session key negotiations.
And, to say the underlying part out loud: an encryption system can be a heck of a lot worse than plaintext, because plenty of ways of handling encryption are trivially broken, but give the user an illusion that what they’re saying is secure.
For rotation: my specific question was around long delays. Say, for example, you email me today. I get your public key from this new tag we’ve added. A month later, you rotate your key, but the next time we exchange emails is when I email you a year from now. I’m using your old key, the one I have.
Yes, but anyone with my SSH keys can connect with SSH.
> but give the user an illusion that what they’re saying is secure.
I don't trust any encryption system with my life or the nuclear launch code. But for casual privacy, the scheme I proposed is fine.
> I’m using your old key, the one I have
Then my system should bounce it back saying the key is invalid. Of course, if I change my key the email app should send a "new public key" email to my address book.
I’m not sure what you’re pitching at this point.
I said long-lived keys for data encryption is a pitfall, you pointed out SSH uses long-lived keys, I said it doesn’t use them for data encryption, and now you’ve just said an unrelated true thing about SSH?
There’s a huge valley of users between “casual privacy” and “nuclear launch codes”. Most humans live there. If your pitch is that your system is a casual privacy tool, just use email. Casual threats aren’t doing dragnet email surveillance or hacking email providers.
> If your pitch is that your system is a casual privacy tool, just use email
The lock on my front door is easily circumvented by anyone determined to get in. But we still use door locks, and locks are far more effective than otherwise.
My proposal is ridiculously simple. All it requires is to standardize a single tag in the email protocol which provides a public key and agrees on a particular public key method. That's it.
Email apps can support it, or not, it still works. Even if just gmail supported it, or Thunderbird, or Outlook, it will be viable.
Well, you could use opportunistic encryption: once the first (plain text) email is sent, it establishes an identity for the sender and from that point forward the exchange is encrypted. It's, of course, not authenticated, and anyone can MITM the exchange, but it kills dead all passive (eavesdrop only) attacks. Also it makes a past compromise easy to detect, without reusing any session key.
A more direct solution would be to leverage the DNSSec infrastructure and publish, say, an "MXK record" (MX Keyserver) for each compliant server. If such a record is present, then the mail domain vouches that you can find at that specific address a VKS keyserver that can resolve individual public keys for all its mailboxes. The whole exchange is encrypted and hierarchically authenticated, from the DNSSec root down to the individual mailbox, so it requires the cooperation/compromise of the specific level to break.
This way, when you type "user@host" in your email client's To: bar, the client can immediately resolve the public key of the recipient, and encrypt and sign the message accordingly with a high confidence that the encryption will be transparent and ensure the security of the exchange.
You’ve not addressed the overall problem of session vs long lived keys, but more so you’re pitching we tie this to DNSSEC? I’d think the fact that DANE is basically the same idea and has been jettisoned by browser vendors would be a good data point.
Long lived public keys work well for PGP, the trouble with that is the manual handling of keys and huge learning curve. With automated identities handled by the email client and countersigned by the provider, you can easily establish a key rotation schedule. Maybe not as effective as Signal but it stops all MITM attacks that don't compromise the providers, a high bar. Of course, the actual symmetric encryption key is unique for each message.
DANE was ditched not because of technical problems or political pushback, but simply due to lack of adoption. Massive network effects and existing investment in the broken CA model, sites don't care about interacting with two sets of PKI if customers will only reach them on one, therefore they don't publish DANE records, therefore the support is removed from major browsers after a few years essentially to reduce dead code and 1024 bit RSA keys. If you know otherwise please enlighten. So it's the same problem everywhere, lack of interoperability chains us to outdated protocols, a problem of course shared by my proposal too.
> the trouble with that is the manual handling of keys and huge learning curve
There's no learning curve with the scheme I proposed.
GPG doesn’t work well generally, and long lived keys are one of the many major flaws.
The network effect was what was wrong with Google Wave, which _was_ better than email in practically every way.
If it had been possible to send a Wave to an email address (some sort of embedding?) and receive email to your Wave address, I think it would have succeeded in replacing email.
Instead, for reasons I'll never understand, Google tried to replicate the limited number of invitations system that had worked for Gmail, despite the fact that only having a limited number of people you could communicate with made the product effectively useless.
It worked for Gmail because you could send email from Gmail to anywhere, and receive it from anywhere! Missing that Wave was fundamentally different to me is one of the worst tech marketing failures I've seen.
Only one of my Wave invites went out, and to a person who I saw nearly every day but didn't communicate with electronically. I invited my sister, classmates, and some friends who I regularly communicated and collaborated with electronically, none of them ever got in before it was shutdown. It was very frustrating because for one person it had limited utility (may as well just use Google Docs or Dropbox), and I had no community to interact with within Wave.
> Missing that Wave was fundamentally different to me is one of the worst tech marketing failures I've seen.
Should we tell smeej about Google's fifty OTHER failed IM marketing attempts?
That's like saying that no one's going to use a new social media, or IM, or chat, or ...
If it's happened for all of those things (and some of them going from a more useful platform to less useful), it can happen for email.
Social media are not primarily communication tools, they are content consumption and aggregation platforms. So if a new platform offers good content (say, pretty pictures - Instagram or funny videos -TikTok), they can build a critical mass of users that will attract more content providers etcetera.
For communication platforms the network effects are paramount, no one will move to Mail2 just to send plain old emails that almost no one can receive. If you can build a new email with much more attractive features, then you can disrupt email, which is exactly what I said.
To that i can just add: Text is cheap, show me your code. Any proposal is great, but if it doesn't turn itself in code, it is meaningless.
In my opinion, most of the problems with today's email systems could be solved if the recipient system would require an ε payment to the recipients' public wallet. I don't care what the value of ε is nor in which digital coin it is. But there should be a negligible cost to send email, and a benefit to the receiver.
I would even go further and alot email priorities, each high priority requiring a higher payment. So my bank notifying me of an attempted login might pay a little more, but that message goes to the top of my heap. Whereas aunt sally's digital birthday card cost her near nothing and I'll get to it when I'm done with all the attempted logins to my bank.
The best part of this system is that it really does not require actual money to be exchanged. If one prefers, he could just mine the coins he uses to send the mail with. That's a feature, not a bug, to keep the system inclusive for all low-volume senders but far less so for spammers. It also makes scamming far less lucrative.
As noted by other people, your idea, especially in its final version, has already been proposed.
The basic problem with the proposal is that it fundamentally misunderstands how the economics of spam works. Especially when it comes to the idea of using proof-of-CPU time to generate the cost to send mail. Spam generally comes from compromised machines, so the people who would actually be paying the cost of sending mail are not the spammers, but those who own the compromised systems. In effect, you'd be making everyone _but_ the spammers pay for their email. (And the idea of being able to pay extra to get higher priority is going to be a godsend to spammers--historically, spammers were the most eager to implement anti-spam technologies such as SPF because of the boost it gives the spam.)
The parent comment isn't talking about proof of work it's talking about actual payment. In practice payment system are much more rarely compromised (or at least it doesn't last long) and if you could compromise someone's payment system you'd not bother sending spam, you'd just take the money and run.
_>The parent comment isn't talking about proof of work it's talking about actual payment._
I think he was replying to this that the gp wrote: _"The best part of this system is that it really does not require actual money [...] , he could just _mine_ the coins he uses to send the mail with."_
Indeed I somehow missed that last paragraph >_<
I _did_ notice that bit, and it seemed bizarre and sort-of irrelevant to his point.
You've gone full circle and reinvented HashCash, the algorithmic predecessor to Bitcoin :)
In case readers aren't aware because the comment doesn't say, the Bitcoin proof of work algorithm was invented by the designers of Hash Cash as an e-mail anti-spam measure, more than a decade before anyone ever thought to use it for currency. They're effectively the e equivalent of postage stamps, with prices that dynamically adjust to whatever level is needed to deter spammers. Just as with stamps, someone eventually realized they could be traded for more than face value, and like with limited-edition collector's stamps, someone eventually realized you could make "non-fungible" versions and try to trade them for even more than face value.
Interesting, thank you.
Maybe new-email could be linked to a cryptocurrency?
Please read
https://craphound.com/spamsolutions.txt
and check all boxes that apply to your post yourself. Ideas such as yours have been around since the mid-1990’s and have been debated to death since then, so much so that the aforementioned template was created.
That template essentially shot down any discussion of fixing email in the open source space, and so companies like Slack, Google and MSFT were able to create walled gardens that can only be interacted with by their systems or by email senders they consider secure enough.
If, instead of creating that template, somebody had made Thunderbird send PoW headers and priotize those that had it, maybe we would have been in a freer world.
And yet, nobody who could have made that change in Thunderbird thought it would be a good idea to do that. This is how ideas are evaluated and either adopted or rejected.
Yea, there's about 0% chance that PoW would fix anything. Much like how bitcoin miners run in javascripts these days, victims would have born the brunt of the computing expense.
Does everything need to devolve into sudoku-solving coins that the makers can try to profit off of?
If we’re building all this tech for coins or digital stamps, it seems like the existing system could work fine by rolling per-contact email addresses to give out. Traceability without having to kill any polar bears. Apple is already offering this through their Apple Sign-In
_>, it seems like the existing system could work fine by rolling per-contact email addresses to give out._
The problem is you can't give out "per-contact email addresses" to future unknown people when you can't predict who will legitimately send you email that's not spam.
The random-generated-rolling email addresses that hides real addresses such as systems in Apple/craigslist/ebay/domainWhoISprivacy can function because there's _already a non-email type of _transaction_ or _activity_ to establish the first link for communication_ -- without requiring a reference to a person id. E.g. Consider that it's the _user_ that first enters the "fake" rolling email address into a shopping website. In that case, the activity is "shopping cart" and now the website can communicate back with that fake email.
That doesn't work for other real-life scenarios where the sender only has the _person_ in mind to send a message but _no transaction_ or other triggering activity to publish/discover a random email address. For that, you need _stable_ easy-to-type addresses such as johndoe@example.com which you can put on business cards. But the problem is that spammers also know that non-random email address. So the cpu-puzzles approach was (attempting to) solve a different problem: people can publicize their _stable email addresses_ but spammers also knowing it doesn't matter because they don't want to spend $$ on cpu.
This is getting a bit fanciful, but there’s a universe where you print unique addresses on business cards.
Of course you can use a QR code.
In the world where you can’t do this, you could have a request mechanism. It’s a link, you go there, and it requires one-time auth from _any site at all of note_. Some sort of social proof. And that gives you a link.
Though ultimately business cards are business cards are business cards. They’re pretty low-value and you probably want to put a less important email on there.
For what it's worth I have never used any of the cryptocurrencies. It just seemed to me that this is a viable solution to a difficult problem - a way to associate a cost with a behaviour that we'd like to limit, in a way that does not prevent the poor, disadvantaged, or those who wish to remain anonymous from participating.
If the sender is on the recipient's allow-list then it goes straight through. Otherwise the recipient has to solve a captcha. This allows humans to send small numbers of emails easily, but makes it harder for spammers.
Also, recipients could have deny-lists and automatically share them with others they trust. The deny-lists might be a list of senders' cryptographically generated IDs, in which case it should be reasonably hard to generate new ones (e.g. 10 minutes on an X86 cpu using an algorithm not conducive to parallelisation or special hardware).
Shouldn’t the sender be who needs to solve
the captcha?
Spam from hell: you don't know if the email is real and you have to solve a CAPTCHA to find out.
> Otherwise the recipient has to solve a captcha.
No way am I going to solve captchas for incoming emails! Not any.
> in a way that does not prevent the poor, disadvantaged,
Yeah mining coins totally doesn't exclude poor people. If it costs less than the profit spammers make they will use it. You can't just handwave the thresold away, and pretend you are solving the problem by just asking for money(thereby excluding large swaths of people). Just use your iMessage.
This is probably the worst part about any kind of payment based system, whether it be money or proof of work- these systems do reflect the disadvantages of the economies of the world.
That said, since this problem already exists in the world, it is often overstated.
If a stamp is a reasonable cost, and only a one time cost, it will be largely invisible. 25 cents (a price set by the recipient) to send a message from a stranger won't have a big impact on most people, and it's important to recognize it's only the first time.
It's further important to know it would even be possible to refund the cost of the stamp back to the sender once the message is sent. The recipient could have a button that says "Return stamp" which would then let the sender re-use or in some other way organize the stamp.
Now let's contrast to today... Let's say I want to send a message to a big movie star or a CEO. The chances that I will be able to get their address is relatively low. Even if I do, they likely have someone they pay to filter their messages before it gets to them.
I am sure such a scheme embedded into a protocol would work with certain group of people. Most people would just get the same amount of spam, because they can't afford to up their threshold in a way that would deter the "marketeers" to stop such activity.
I actually think it would dramatically cut down on spam, but not hate messages.
I've said this in another comment but it bears repeating, and perhaps repeating in different ways.
Stamps are just tokens, as in bearer tokens. How you get stamps is a different issue. Let me give you a couple of examples how one might do stamps without cryptocurrency:
1. Alice, "I'd like 100 Bob stamps please." Bob: "Please pay me $5"
2. Alice, "I'd like 100 Bob stamps please." Bob: "Can you please do some computational work on my behalf?"
3. Alice: "I'd like 100 Bob stamps please." Bob: "How will you pay for them?"
Alice: "I could trade in 30 Carol stamps, would that work?" Bob: "Yes, let's trade Carol stamps for Bob stamps."
As for Apple Sign-In, I don't know what it is, so I can't speak to it.
Apple generates a random email address for each individual subscription, and forwards it to your inbox. If you cancel the subscription, then the forward is changed to a blackhole.
Not only does this give you a nice way to really cancel, it also means you can track down if someone sold your address to a spam list. Or if it accidentally leaks, rotate it.
> if the recipient system would require an ε payment to the recipients' public wallet
And allow recipients to whitelist senders who don't need payment.
Why would you prefer and value your bank sending you some automated valueless security theater[1] over your relative sending you a human to human birthday greeting? And why would you build the system so that, for everyone, personal communication is expensive and burdensome, and for every company, mass advertising is cheap and easy? (Relatively cheap, because companies have vastly more money than individuals).
This applies whatever the cost values is; if it's low, it's low for spammers. If it's high, it's high for individuals. And wherever "low" or "high" is, someone you like will think it expensive enough to be in the way and someone you dislike will think it cheap enough to be no problem.
[1] if the bank stopped the attempted logon, I don't care. If the login was actually me messing up, I already know. If they didn't stop the logon, then they didn't detect it was fraudulent and wouldn't be emailing or calling at all.
Given how much SMS and phone spam there is I am not sure this would work.
This is a proposed email spam mitigation, not an SMS spam mitigation.
Yes, but SMS are way more expensive for the spammers to send than I assume this proposal for mail will be and there still is a ton of SMS spam. The issue with spam is that it is lucrative and I do not think a small fee will fix that.
This idea is not bad. I would just add that there should be an option for parties with mutual trust to exchange without cost.
Your bank is going to increase your fees to cover the cost of sending mail to you, so the process is a pointless waste of resources in that case. This illustrates the need for a free mechanism between parties that trust each other.
My bank just sent an email "A surprise for you!!".
Open it up ..it's a link to my app to login to see a useless budgeting report I've been suffering with for over a year.
That should cost them.
Like I said, they can just charge it back to you, and no law is going to completely prevent them from doing that one way or another. So I think it's a waste of time with a party you already have a paid service agreement with.
> _That should cost them._
It could, if you choose, cost them your business.
i suppose if your inbox auto-replied with an ACK you could just instantly repay the postage
Yeah, but as others have pointed out, I _know_ my bank, and I _know_ my aunt, so wouldn't a properly implemented signing mechanism just solve the problem? I know that PGP failed, but with a better implementation and greater support it just might work. This would also solve phishing, where the attacker would be willing to pay a few cents per attempt.
For institutional senders, this is what DKIM already does. Your bank has an e-mail domain registered with a CA somewhere that other e-mail domains, such as wherever you receive mail, trust.
Too bad that even if I setup DKIM correctly Google still black holes my email.
This is something that is missing from the OPs list of problems with e-mail. There is an "extended" version of it by a certain bigtech company that a lot of people use, which is not completely interoperable with e-mail. Just enough to confuse people into thinking that it is, but not more.
Yes! But that's not a solution for aunt Sally, though indeed it solves the phishing issue.
> If one prefers, he could
_one_ could
Email is fine. It's simple, it maps very well to peoples' intuitive real-world understanding of "mail" and it works. I've had the same email address for decades. I get some spam, but I don't drown in it. I get _more_ spam in the real (postal) mail, if you measure by the ratio of spam mail to legitmate mail. And I get real mail that is spam but looks legitimate on first inspection.
It's generally _very_ easy to identify spam email, and delete it. I can almost always do this by the sender and subject line alone, without opening the message. It takes less time to do this than it does to periodically peruse your spam folder for false postives.
Encryption/digital signatures that rely on any kind of PKI architecture won't work, because most people don't understand it or won't trust it or both.
What might work is ssh-like "trust on first contact." You get a signed email from a friend. You get a little notice "you haven't seen this signature before, do you want to trust it." You can turn it off if you don't want to bother with this, and you need to be able to easily revoke the trust if you make a mistake (so even this is getting complicated). The keys need to be generated with no more complexity than "ssh-keygen" (and I work with many people, college-educated, who can't manage even that).
I'm in tech, I do it for a living. I don't sign my email, and I don't verify signatures on the signed email I get. If I don't think it's worth the bother, my mom certainly won't.
> What might work is ssh-like "trust on first contact."
Deltachat is working like that and I can't wait for the day everyone switches to an alternative client that behaves like it is from this century, like deltachat, instead of the last one.
Who actually _needs_ the following ?
- open a different window (ie a different _context_) than the one you're currently in
- a greetings line
- quoting the entire mail you're replying to
- the choice of whether you include an attachment inline or as a real attachment
- threads that you can't have a glance at because messages are in different mailboxes (inbox, archive, Sent, ...)
- having to click on each message to read the conversation instead of having it all in one view that you can just scroll (thanks Gmail for doing this correctly)
- a custom signature no one really reads
Email's basis works, but we still use it like we wrote snail mail. No one writes snail mail anymore. It's time we use it in a more modern manner.
I use a lot of those in my work mail:
- open in different window: all the time. I am writing one email and searching/reading email archive for context. Also just keeping an individual message open in a separate window as a reference, while using main email window regularly.
- greetings line: I don't think it's a "feature" but I use it when starting a thread, especially when writing to people who I am not very familiar with.
- quoting entire mail: pretty frequent, for example when adding a new person to an email thread. Also handy for selective reply, it is easier to delete parts you don't need from quoted email than to start from blank and copy from original email.
- inline/attachment toggle: I don't think my email program has this
- threads split across mailboxes: not very helpful. But the multiple mailboxes is must-have.
- clickable message list: situational but occasionally helpful. Mostly when the is a thread with lots of messages, and you are looking for this rare message from the specific author.
- custom signature: I am don't have one personally, but some people put their position, office location and phone number there, and this is pretty helpful.
The email is different than instant messaging - it is harder to use, but is much more powerful if you learn to master it. The deltachat (judging by screenshots on their website) seems like a very basic and low-featured client, which unfortunately promotes bad habits in email, like writing short messages and expecting instant replies. I am glad it has not caught on.
> I get some spam, but I don't drown in it. I get more spam in the real (postal) mail, if you measure by the ratio of spam mail to legitmate mail.
And all it cost was IP reputation systems that undermine email’s claim to be decentralized, language heuristics that undermine its claim to be reliable, and greylisting that undermine its claim to be fast.
Don't forget the centralization that undermines its ability to be private!
I totally agree that email, and messaging in general, would be much better if we solved the identity issue. I don't agree that there is anything particularly broken about the two existing standards to do this. The last thing we need is another standard for signing email.
The root problem is that we treat anonymous email (unsigned) the same as email from people we know (signed by such a person). Treating anonymous email with normal suspicion pretty much solves the problems with email ... as pointed out in the article.
I think there is a tendency to want to solve a technical problem, even when no such problem exists. This will ultimately have to be solved at the people level.
> Treating anonymous email with normal suspicion pretty much solves the problems with email
I have a rule in Thunderbird that directs all email from senders not in my contacts list to a separate folder instead of the inbox.
The solutions are already there... there is a lack of will from the corporations to implement them. google checking gpg signature? nope.
The internet is unfortunately not user-centric enough, there is much more value in sending crap html emails "only 10 hours left 50% discount" and scanning your emails than there is to provide users with secure communications.
let's all sign our emails to raise awareness of google's & outlook shortcoming
P.S : we tend not to sign our emails since it is not the standard, standard is de-facto dictated by google and outlook now.
P.P.S : seeing an unsigned email would raise suspicion if the standard was the opposite, a filter could even classify between signed and unsigned
P.P.P.S : you would only bother to verify the identity for the emails you care for
P.P.P.P.S : if you think google does not dictate the standards, feel free to point to me the part of the rfc that mention that residential ips are not valid.
Google and Gmail account for a very small amount of the global mail-flow and have even less of an impact on standards. In fact they only appear as a contributor/author on RFC8617 (ARC).
No standard says, or even suggest, that “residential ips are not valid.” However, reputation is an important par of mail security. Most, if not _all_ residential IPs won’t be sending enough mail to even register a reputation (look up your own on
https://talosintelligence.com/
). _This_ is the main challenge for self-hosters. That and a significant majority of residential ISPs block SMTP traffic, for very obvious reasons. This isn’t some Google/Microsoft conspiracy. SMTP is fundamentally a plain text format that has been abused for the last 30 or so years - 30, because that’s when MIME became a thing.
The Gmail _web client_ is the 2nd largest email client, at about 30% market share. The number of people who use gmail but use an email client instead of the web client is going to be much, much higher. (Especially when you include people whose clients automatically block the tracking pixels used to generate these statistics, which is sadly few of the clients).
The vast majority of email routes through a few major email service providers, with gmail the largest, but MS (via O365 hosting) and Yahoo are also likely double-digit market share.
How much _traffic_ does that generate? On average, there are ~300 billion messages sent daily - the majority coming from automated services and/or bulk messaging providers. Sure, Gmail and Google Workspaces have a lot of "market share", but that is meaningless in the context of worldwide mail-flow.
In this case, the context of worldwide mail flow may be a meaningless statistic. Sending messages to real individuals for purposes of commerce is going to be much more important to a companies profit, than say a script sending out a message saying "This task completed successfully" to an email address.
Thank you for the link. I checked my duckdns.org subdomain and found that I am on a blocklist and my reputation is poor. I don’t even run SMTP. Comcast Xfinity, if that matters.
Fwd/Rev DNS Match Yes
Last Day Vol. 0.0
Last Month Vol. 0.0
Block Lists 1
BLOCKED IN:
pbl.spamhaus.org
Email Rep. Poor
I get weekly phishing SMSes from duckdns.org domains, probably why it’s on such a list
google block residential ips. even with a specific message. not based on reputation, based on ip block. I am well aware that smtp have been abused for decades now, I run my own mail server and have no problem sending to google outlook etc. my point was only regarding blocking residential ip. google definitely does that (on top of many isp's) and it is not part of how the smtp protocol works. no conspiracy here.
Yes, _because of low/no reputation_. It's a feature, not a bug.
just testing now, not every residential ip it seems (I moved country and now my email go through without the message : ...belongs to a residential ip...)
no, because they belong to a residential ip block. it is easy to test... and the error message is without ambiguity.
I would also challenge the "Google and Gmail account for a very small amount of the global mail-flow". 100% of the companies I worked at use office365 and Gsuite.
> google checking gpg signature? nope
Are you proposing that they mark every mail spam that's not on the contact list of user? Otherwise checking signatures doesn't do anything, spammers already use dkim.
One idea would be to treat an email from someone you don't know with the normal level of suspicion for that case. The HTML would be converted to something safe for the user's browser. Image loading would be blocked. You would probably throw on a few spam points. Anything other than an introduction message gets lots of spam points. Anything that was properly signed would skip the spam process entirely.
that makes sense, without breaking the actual mail flow
once you receive a message that you actually want, you could validate the signature and every subsequent communication from this sender is more likely a desired email than a spam
Receiving Email from strangers
Lets walk through the proposed solutions to this issue, because this is the biggest problem I see with the proposed new way of mailing:
In the simplest case, the server might never give out stamps
For me, that's not feasible. I hand out my mail addr. with cards, as signature, its on websites, in READMEs, etc. I knowingly invite people to contact me via mail.
The server might require the putative sender to solve a CAPTCHA of some kind.
How does that work if the mail is generated automatically, eg. a coworker makes me the recipient of a customers monitoring system? How does this work if the mail is generated without a GUI, eg. using a terminal mail-client?
The server might require the sender to write a short sentence of why they want to reach the recipient. If that contains keywords chosen by the recipient, the server issues the stamp.
How does this solve the problem of being able to be contacted by strangers? The would-be sender has to know the keywords. If these are publicly available, eg. in a README, then there is no security, if they are not, sending a mail becomes guesswork for the sender.
The server might require some sort of proof of work.
Let's forget for a moment how much trouble is caused by the energy consumption of cryptocurrencies, but how would that work on, say, a mobile device with already limited battery power?
The server could require a very small payment.
The article points out the problem with this already. This would mean companies that can afford the fees are now free to spam me with advertising, but someone from an underprivileged community, who found a bug in my project, may think twice about contacting me.
-----
Edit:
In my opinion, spam and phishing are not problems that should be solved by changing or replacing SMTP.
Don't get me wrong, I am not defending SMTP. It has a lot of shortcomings that need to be addressed, the biggest are an incabability to transmit binary data without resorting to base64 encoding, and the lack of built-in encryption support.
But spam and phising are issues to be solved at the application level, not the protocol level. Spam isn't that different from portscans, and we don't change TCP/IP to prevent these.
> The server might require some sort of proof of work.
One of the things that I think many people don't really understand about the proposal I made for Stamps is that POW is not required for stamps. Stamps can be given in any number of ways.
For example, someone could simply issue them, with money. For example, if Alice wants to contact Bob for the first time, Bob might say "It will cost one Stamp please", and then Alice will need to get a stamp, maybe it costs 25 cents. Then Alice gives the stamp to Bob, and Bob lets the message pass.
I think proof-of-work systems are incredibly wasteful, and would think that should be an absolute last resort.
>Spam isn't that different from portscans, and we don't change TCP/IP to prevent these.
Guess you've not heard of port knocking.
Portknocking is not part of TCP.
It's implemented at the application layer via a daemon listening for knocks and opening the firewall on demand.
DeltaChat has an interesting approach to extending email and open standards towards something that feels like "instant messaging", but incorporates a lot of cool features: Bots, multiparty chats, games, e2ee, p2p mode etc:
Today’s mail servers often don’t know who’s in the receiver’s address book or who they have received mail from before. That seems to be an untapped source for scoring on the server site. If the SMTP server also hosts the mailbox, and can access well known folders (Archive, All Mail, Junk) it could quickly decide after the RCPT TO how to react. Options: Slowing down to increase sender’s cost, requiring proof of work (hashcash style SMTP extension), adding scoring information for spam filters, issue a temporary unavailable (gray listing), you name it.
Some have contact lists and email from the same provider and synced to the cloud (e.g. gmail/gcontacts; e.g. iCloud+ email hosting and icloud sync for contacts). One would expect these data sources work together, but it seems that today at best one informs the spam filter of the other, and not necessarily server side. There is potential in moving the smartness to earlier stages of the transmission where the sender is also bearing some of the costs.
JMAP (sic!) is a good step to standardize syncing here. Still think there is untapped potential even there.
> Today’s mail servers often don’t know who’s in the receiver’s address book or who they have received mail from before.
One of the big revolutions in antispam was reputation-based, which is used in all large email providers. Effectively, every email domain gets a reputation based on their email history. Someone like @gmail.com that sends a lot of mail to the server that isn't spam is going to get through easily. Someone like @spammymcspamface.com that sends naught but spam is going to get black holed. Someone like @whoami.com who hasn't sent anything is going to get throttled until they can establish a reputation as not-spammy.
Reputation is, however, not weighted by data tied to the individual receiver's mailbox. Reputation is often tied to the sending domain. Reputation, as helpful as it is, doesn't tap into the data that currently remains largely untapped.
Since I'm explicitly mentioned in the article, let me say a few things:
a) I agree email is forked up.
b) I think a stamps-like solution could possibly work, but it's not the email part that makes it hard, it's the ecosystem around email that makes it hard. For example, once you send an attenuated capability to someone else, you've lost it, which means your system needs to be aware that what you're sending is a capability token and not an identity token.
If what I've said doesn't make sense- I'll explain a bit later on in this post.
c) I think it's nice to think creatively about message systems, including email, XMPP, ActivityPub
Now I'll say where I see the problems here being:
1. Overlaying on top of email
The problem with a protocol such as email is once it's in place, it's in place. Stamps (which is the idea the author references) could maybe be overlayed on ActivityPub- but I worry the time for that ship is either closing or it's sailed. I have many more thoughts on this point but it would be pages of thinking, so I'll leave it at that.
2. Cryptography not required
The author wants public keys, emails signed, etc. You don't need any of that for this to work.
What I proposed for AP (and still do) is that we can have a "public" address and then private, revocable addresses. You can then combine that with a bearer token exchange system. It's actually very simple, though I'm oversimplifying it too much here.
3. The author says I got it from Christine Webber, and "who knows where she got it from"- She got it from Mark Miller's Ph.D thesis, as well as the work done by Chip Morningstar. The theory behind all of this is called Object-capability Theory (
https://en.wikipedia.org/wiki/Object-capability_model
)
The email addresses are what the OCAP folks would call a capability token- the ability to send an email is a capability, and the special email I give someone is a capability token, a bearer token in this case (though there are other patterns in the OCAP world).
I can then make other attenuated capabilities, ie proxy capabilities that may act slightly different or serve a specific purpose- like a share link that only works for a week, or a share link that's upload only, etc.
Why is all this important? Because understanding the theory will help you understand the possibilities, and also the possible flaws.
Am happy to discuss further.
Capabilities don't solve double spending(or in this case, double sending) token, as they are intentionally transferrable.
So you'd need to have a way to bind a capability to a sender, which naturally leads to PK, or treat them as short lived tokens you revoke as soon as you see they've leaked, which is identical to throwaway email addresses we already have.
> Capabilities don't solve double spending(or in this case, double sending) token, as they are intentionally transferrable.
They do if the entity doing the transfer is the issuing entity.
> once you send an attenuated capability to someone else, you've lost it
I'm not sure what you mean here.
For example, if Alice sends a token to Bob giving Bob a capability, Alice can sign it with her private key and Bob's public key, such that if Bob (or anyone else) wants to use that capability, they have to have Bob's private key to do so.
What I mean is that if Bob hands his capability to Carol, she can give it to Dave and he can't stop it.
I don't see any reason we need keys here.
But Dave can only use the capability if he has Bob's private key. That's why we need keys.
Dave doesn't need a private key, the token just needs the ability to be traded.
One of the more complex aspects of OCAPy design vs the we do things now is that in a fully ocap world, the token wouldn't just be a string, but possibly a reference to a living object that Alice owns.
Then the trade (and any use of the capability) always goes through Alice.
That's why you don't need crypto in a real OCAP world.
Isn't this an obvious (and actually useful) product for a startup? You implement whatever solution you come up with handling payments, tokens or whatever and users can have it as a filter before they see their email.
Interesting article. I think the reasoning about multiple identities and adresses is spot on, as is the use of signature and stamps to limit spam.
I do think, however, that use case of receiving mail from strangers is a much bigger case and can't be handwaved away.
I was thinking of something similar lately, coming at it from the social networks angle. I was thinking about personas in social network setting (what google+ attempted with circles), ownership of your data and social links, and the thought process led me to something very similar to what OP is describing.
I like many of the idea's but the real work is thinking how you would use this in practice. For example take some typical scenarios and imaging how they would work in this post-SMTP world.
(1) I'm talking to someone on the phone and I want to give them my "email address". What would this look like?
(2) I sell canned soup in a store and want to put something on the can. What would this look like? (possible a QR code with an identity and a stamp?)
(3) How would you go setting up a "mail list" for a club with minimal friction, especially if everyone doesn't have their phone with them.
What we need is a trustless trust infrastructure on which identities can be verified by anyone. Governments around the globe can verify their citizen's and companies identities as well as each others, until we have a decentralized network of trust that includes all people and relevant corporal and organisational entities. Emails are signed by all of these entities, and people choose which emails they'd like to consider through choosing trust providers, a role that'd be taken by governments or other organisations. Hence we get a distinction of emails generated by people and corporate accounts; and corporate accounts can be blocked until their communication behavior is determined acceptable. Communication challenges between corps and people suddenly are worth being treated right because they can be closed any second. Communication channels between pairs of entities _exist_, that's the first innovation actually.
> a role that'd be taken by governments or other organisations
It sounds like it would be in governments best interest to control this, so they can inject their own fake identities, much like they do with spies now.
Also, how does one revoke ones identity if one loses control of it. I mean, identity theft is common currently?
I was working on something quite similar for the last year. It’s an email system based on digital identities, where every piece of outgoing mail is encrypted and signed. It works with the existing email infrastructure by having a gateway which does encryption for the incoming mail and decrypts the outgoing mail. There is also a IMAP/SMTP bridge to allow any existing email client to work with the system.
Eventually the identity registry will be decentralized, choosing the right blockchain is the tricky part.
Isn't there already a viable technical solution in use for all listed issues?
E.g. spam/scam can be avoided by SPF (
https://en.wikipedia.org/wiki/Sender_Policy_Framework
),
identity verification and encpryption is solved by
https://en.wikipedia.org/wiki/S/MIME
.
I think the article misses the mark. The #1 problem is spam, but spam is mostly a social issue, not a technical one.
Case in point: 90%+ of my spam is currently delivered by google. Either using fake accounts and/or hacked ones.
That spam passes _all_ the technical solutions we have put in place today: SPF/DKIM/DMARC. Sure, the SMTP protocol isn't that great in today's light, but you think changing the protocol would solve it?
Say hello to whatsapp spam, facebook's messenger spam, and so on...
My spam filter works very reliably for all such cases; my first and only manual rule I had to add recently was to stop "bitcoin" spam delivered by Jira accounts; and I also check the filtered messages from time to time. I think there will be spam with any protocol we might get in future.
You'd be surprised at how many sending domains do not have SPF configured correctly. When you look at DKIM and DMARC, it get worse...
For one, S/MIME certificates all require a non-trivial amount of yearly price.
I use a free self-signed one just for the purpose of encryption. But there are also providers where you can get identity verifiably certificates for free.
I would like some feedback on a possible solution to one of the problems: Spam.
And my proposed solution is blockchain. Hear me out :-)
What if sending a mail required a proof of work (the digital equivalent of a stamp)? Then sending mail in bulk would be costly enough to lessen the spate of spam. Unstamped mail would be spam unless the recipient agreed to accept it without the stamp. Legal bulk mail senders could obtain stamps in bulk providing they agree to abide by the rules (no tracking, easy unsubscribe etc.) and provide proof of doing so.
By allowing unstamped mail to be delivered, yet be contingent on the receipients choice, I think we can deal with older clients that can't be updated, noble causes etc.
Feedback welcome.
Mentioned in another comment here, but you're describing hashcash:
https://en.wikipedia.org/wiki/Hashcash
This is already proposed in the article. You probably haven't read it or may be I misunderstood what author meant by requiring proof of work to avoid spam.
Does not work against spambot networks on hacked computers.
Doesn't it, though? Wouldn't it mean you'd need more hacked computing resources for sending the same amount of mail?
It's an economics problem then. They'd send less spam but they'd be paid more for each mail sent.
Previously:
https://news.ycombinator.com/item?id=22847092
Wouldn't an SMTP that requires authentication and non forgeable sender addresses solve 90% of the problem of spam etc.
That's what we have now, SMTP open relays are not the out of the box configuration. The problem is who owns that SMTP server. If I self host then the big providers don't want to know me. Google goes straight to spam regardless of SPF/DKIM and Microsoft flat out 550 refuses to even accept a connection. A local ISP also pre-banned me because of DigitalOcean's past lax approach to stamping out spammers on their droplets.
There seems to be less spam for SMS. At least I does not seem to be a problem?
So maybe the idea of a small fee for the sender is not bad?
SMS spam is exploding for some.[1] A good 25% of my messages are from an address like 1a02341234e7@uyah23f.com.
[1]
https://www.dallasnews.com/business/technology/2021/09/28/do...
Oh - I had no idea. Since I have not even got one.
This is not true in India, at least. There is so much SMS spam here that everyone defaults to WhatsApp for any kind of actual communication with a human. SMS is reserved for bank OTPs, 2FA, and and and endless deluge of marketing messages.
To solve 'strangers can send me email' you don't need any fancy crypto. Randomly generated long aliases work great, as they can forward to your address with an "alias[.+]" prefix or "[.+]alias" postfix, and you can kill the aliases that receive spam. Killing an alias should send all such mail to spam, or result in a bounce back. If we can push that functionality into the MTA / MDA (along with a general spam button) the mail servers can learn and respond to spam that happens across all their user accounts and either start to deliver mail into Spam folders or push back on mail deliveries with either a delay or bounce.
I am concerned that charging even a penny per email would make it harder for people in developing nations to send mail (you need to understand how insanely poor some people are), and those are the countries that spammers might take advantage of. We should be very careful in today's world not to make technology that negatively affects the most vulnerable.
Isn’t it trivial to remove the alias from the address?
Actually I meant you make the "public" e-mail address totally random, like "ksdjfksjlfdjasljhflsk@whatever.com". That address then forwards to "yourname+ksdjfksjlfdjasljhflsk@whatever.com". Then both your client and mail server will know both the "public" part of the address, and who it's supposed to really be delivered to. This enables spam filtering on a specific alias, or killing a specific alias. Additional support would need to be added to handle these features, and also automatically translating replies from your mail client so the From: address continues to be hidden.
This idea mostly already exists in proprietary e-mail relays like Craigslist email addresses, but it would be nice if it were a standard.
Everything here assumes the existence of some sort of method to associate a random string with an identity and enough data about that identity to find a server to talk to, without having an initial server to talk to...
On the other hand, as much as I dislike Google, I use Google to filter spam on my email on my own domain and it seems to be extremely effective at that particular job.
Over past 6 or 7 years I have been using Google I remember only one case when valid mail was classified as spam. I receive about 10-20 spam a day and I get about one spam into my inbox per month.
In the past I would use catch all on my domain and give addresses like adobe.com@<my-domain> to companies.
It led to some fun stories like when I started getting a lot of spam to adobe.com@<my-domain> and Adobe support (I have been paying customer -- no more) claiming that they have absolutely not lost any of my information even though I could positively verify my adobe.com@<my-domain> address and password was available in one of publicly available databases.
Nowadays I am using more and more SSO + MFA and it kind of makes that pattern less useful.
The main problem with email is simple lack of privacy in my opinion.
Privacy is the primary attraction of instant messaging. End to end encryption is OK but most users don't really need it and it tends to complicate operations.
Fixing the privacy issue doesn't require any changes to protocol or email clients. The solution is really just a configurable email forwarding service.
Every user should have both a public email address used for sending and a private email address used for receiving. The mail server automatically replaces the private address with the public address on all outgoing mail.
Received mail gets forwarded from public to private only if it meets certain rules setup on the mail server. Otherwise, the address is rejected as unknown.
A simple rule set would be a list of allowed addresses. If an address is not on the list, it gets rejected. Addresses are automatically added to the allow list by sending an outbound message to the recipient's address _OR_ by sending a coded message to the mail server. Likewise, an address can be deleted from the list by sending a coded message to the mail server. If you actually want email from anyone, don't apply any rules.
What does a "coded message" look like? How about one addressed to the target but with an empty body and either "!ADD" or "!DEL" as the subject?
I have considered building this myself on occasion but all the reputation machinations currently in place that attempt but largely fail to address the spam problem act as a deterrent.
I'd call spam one of the biggest problems with email.
To solve it, I'd say a protocol extension is needed.
In the ideal case, you could have a certificate authority and everyone's email gets signed. Spam and get your cert revoked globally.
That would be hard to pull off though.
Spam is the lack of privacy I was referring to.
_To solve it, I'd say a protocol extension is needed._
Or maybe a protocol reduction.
The relay system is the source of lots of evil and was really designed for a bygone era. Eliminate "open" relays --- only accept email directly from the source server --- and most spam goes away with a simple privacy filter as I described above. It is easy to "spoof" an address, much harder to "spoof" a server.
This has been tried (several times). Email server operators are too lazy to configure it. Spammers subvert the servers that do have it configured.