💾 Archived View for gemini.circumlunar.space › users › kraileth › neunix › eerie › 2018 › intro_to_e… captured on 2024-08-18 at 18:06:37. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2021-12-05)
-=-=-=-=-=-=-
Here I'm republishing an old blog post of mine originally from March 2018. The article has been slightly improved.
The two updates included here were part of the original article and are not new.
[update: Seems like the maildir patch for the Alpine mail client does not currently work with the latest version... So we need to get an older version of the ports tree - this is not ideal, but fortunately this is only a test VM where we can do bad things]
[update2: More changes... A necessary patch is no longer available from its previous location]
Email - short for electronic mail - is one of the things of modern life that we all are familiar with... Or are we? We know how to use it and probably have a rough idea of what's going on when we press the send button. But email is actually a surprisingly complicated topic. You probably won't notice that until you think about setting up your own mail server for the first time. Once you do however, you're in for quite some reading.
I thought to write this mail mini series in late 2016 when I had to debug an issue with a customer's mail system and thus figured that I really, really could use a bit more knowledge when it comes to email. I quickly went on with other tasks, but I'm still interested in the topic - and maybe I'll find some time for it this time. This series of posts is intended as a summary of things that I found noteworthy about email.
When it comes to email, various daemons are involved. Daemons? Why of course! Email is native to Unix; it had precursors, yes, but it was on Unix that things really took off. That's why for understanding email it really helps to understand Unix.
The part of the mail system that's visible to the user is called *Mail User Agent* (_MUA_ or just _UA_) in mail terminology. That's what most people just call the "Email client": Programs like Thunderbird, Sylpheed or Outlook. It can be used to compose messages (that's the correct term - you're actually sending _messages_ not "emails") and hand them over to an email server or to retrieve messages from there.
The server-side of the mail system consists of multiple daemons. First there's the *Mail Transfer Agent* (_MTA_) that's responsible for accepting incoming messages (e.g. from the MUA), process it and either send it to a second MTA on a foreign server or to save it locally (if the message's recipient has an account on this server). To store a message locally, the MTA can pass it to a *Mail Delivery Agent* (_MDA_).
It's common that an MTA accepts email that's destined for another server and hands it over to that server's MTA. This is called _relaying_. To know where it needs to send the message to, it looks at the domain part of the recipients address (xyz@example.com). Then it does a lookup for the *MX* (_Mail eXchange_) records in the *DNS* (_Domain Name System_) for that _zone_ (DNS terminology; think domain for now if you're unfamiliar with it).
To enable the various daemons to interact with each other they need to follow standards in communicating. These standards come in form of so-called _RFCs_ - if you don't know what that is, do some quick research right now. No matter what, if it has anything to do with the Internet at all, RFCs are what defines the standards that every implementation is expected to follow. There are several _protocols_ which allow for various actions:
The MUA needs to speak *SMTP*, the _Simple Mail Transfer Protocol_, if it wants to pass a message to an MTA because that's what this daemon is expecting. The MTA can use make use of *MLTP*, the _Mail Local Transfer Protocol_, to talk to an MDA. And if the MUA is expected to retrieve messages from the mailbox it needs to implement the *POP* (_Post Office Protocol_) or *IMAP* (_Internet Mail Access Protocol_) so it can communicate with a POP or IMAP daemon which will then read messages stored on the server and send them to the MUA.
And there's more when it comes to mail. Today's internet is well known to be a hostile place. However when email was developed, people on the net were few and shared a common passion: Tech. Nobody meant to do the others any harm. This infantile period is far from what we experience today. But the basic principles of email haven't changed and in fact cannot be changed easily. So the problem arises how to retroactively secure something that was designed in a carefree way and therefore proved to be inherently insecure?
When you think email today, you automatically have to think _encryption_, too. Otherwise there might always be someone to eavesdrop on you. When you think email you have to think _authorisation_. If you don't protect your mail account somebody may break into it, steal your secrets or abuse it. You have to think _spam_. How do you avoid being flooded with all those useless messages that want you to buy blue pills or the like? And worse: How do you prevent spammers to use your mail server so you don't get blacklisted? You have to think _viruses_, _phishing_, _trojans_, etc. Have to consider security holes in your applications, newer protocol versions being established, and so on.
You can already see that mail is a rather complex topic that requires you to have a fair bit of knowledge on other topics like e.g. DNS, security and more. Fortunately all the knowledge that you need is somewhere out there. You just need some determination, a lot of free time and a bit of luck to find the relevant pieces. I cannot help you with the former two but I'll try to provide a source that could help you learn some important things about mail without having to search on for a tutorial (even though of course I cannot cover everything).
So much for a _tiny_ bit of theory. It's merely meant to be enough so we can start doing something and cover more of our topic along the way.
So let's build some kind of test system to play around with, shall we? Sure thing. I suggest using FreeBSD - it is an excellent choice of OS to get into the topic. No, not because I've come to like it quite a bit and base a lot of what I post on my blog on that system. For many things Linux would be more or less an equally good choice. When it comes to getting into mail however, it isn't.
If you already know the whole topic quite well, you can install all the needed programs on Linux without any problems. If however you're just starting out, FreeBSD makes the first steps so much easier as it already comes with a working mail solution by default! Setting up mail server software is a very complex and complicated thing to do and FreeBSD really makes your life so much easier in this regard by removing this obstacle for you.
In a previous tutorial on backups with Bacula I used _VirtualBox_ together with _Vagrant_ and it proved to be a very convenient solution. So I'm going to use it here as well. If you haven't used VirtualBox or don't even know what Vagrant is, I've written an article for you which explains things very detailed and with pictures:
Article on using Vagrant (2016)
For creating the base box that we need, you can refer to another post that explains everything step by step. Just make sure that you read this article completely before you build your base box! Because we're doing the customization we need for this VM differently from the recommendation in the old article:
This post assumes version we're using FreeBSD 11.1 - any version of 11 should be fine, though. If at the time you are reading this 12.x or even newer is out, you may want to check if fundamental things (like finally putting Sendmail to rest and importing a different MTA) have changed in between these versions. Starting with version 11.0, FreeBSD has a new installation dialog screen that lets you choose some system hardening options. While this is a great idea and I've recommended to disable Sendmail in the post about building the base box for Vagrant, it would ruin the mail functionality that we're going to use here. So make sure to _not_ disable Sendmail for this installation!
Selecting hardening options (PNG)
Also don't install Bacula as described in the customization section for the Vagrant base box - we're not going to use it here. Instead do the following (if you're not using a c-shell variant, leave out the "env" for the second command):
# svnlite co svn://svn.freebsd.org/ports/tags/RELEASE_11_0_0 /usr/ports
# env ASSUME_ALWAYS_YES=1 pkg bootstrap
# sed -i.bak -e 's|pkg$|pkg register|' /usr/share/mk/bsd.own.mk
This downloads an old version of the ports tree which holds a version of _Alpine_ that we're going to use. It also bootstraps the package manager and makes a change so that it will work with the old tree. Now we need to prepare Alpine; unfortunately one required file is no longer available from the original site. I chose to mirror it since it took me some time to find the right one...
# mkdir -p /usr/ports/distfiles/alpine-2.20
# cd /usr/ports/distfiles/alpine-2.20
# fetch http://elderlinux.org/files/maildir.patch.gz
# cd /usr/ports/mail/alpine
# make makesum
Now we should be good to configure the port:
# make config-recursive
Be sure to check _MAILDIR_ here as we're going to need it later. Unfortunately this option is not built into Alpine if we were to install it from a package. So we have to resort to ports. While we're at it, we can disable _IPv6_ (which we certainly don't need), _mouse support_, _NLS_ and check _NOSPELL_ since we do not require spelling correction in this tutorial either. Configure _pico-alpine_ accordingly. For all the other packages you can generally turn off _NLS_ and _DOCS_. Then fetch the source for all packages recursively:
# make fetch-recursive
When it's done change into OpenSMTPD's port directory and configure it:
# cd /usr/ports/mail/opensmtpd
# make config-recursive
Here you want to check the _MAILERCONF_ option. The rest is fine. You don't need to build the _EXAMPLES_ for m4 and again can generally deselect _NLS_. Then fetch the sources:
# make fetch-recursive
Next is configuring dovecot2 (you can keep the default options this time) and fetch the distfiles for it:
# cd /usr/ports/mail/dovecot2
# make config fetch
Then follow the rest of the base box build process.
You have your base box built and imported? Good. Create your Vagrant directories if you haven't done so already as well as a Vagrantfile in there:
% mkdir -p ~/vagrant/mail-vm
% cd ~/vagrant/mail-vm
% vi Vagrantfile
Put the following lines into it (assuming that your box has the name _fbsd11.1-template_):
Vagrant.configure("2") do |config|
config.vm.box = "fbsd11.1-template"
config.vm.network "private_network", ip: "192.168.0.10"
#config.ssh.username = "root"
config.vm.synced_folder ".", "/vagrant", disabled: true
end
Now fire up the VM, ssh into it and switch to root. Then edit _rc.conf_:
% vagrant up
% vagrant ssh
% sudo su -
# vi /etc/rc.conf
Change the line that defines the hostname to:
hostname="mail-vm.local"
Save the file and exit the editor. Let's do a _bad_ thing next: Let's configure the VM so that we can SSH into it directly as root. If this just sent a shiver down your spine then you're having the right feeling. You're not supposed to do this like... ever! But now and then you feel like doing forbidden things, right? And now's the perfect opportunity since this is just a private, non internet facing VM. Just remember to never do this on a production machine! Ok, ready? First we need to copy Vagrant's public key and allow it for root:
# mkdir .ssh
# touch .ssh/authorized_keys
# chmod 0700 .ssh
# chmod 0600 .ssh/authorized_keys
# cp /home/vagrant/.ssh/authorized_keys /root/.ssh/authorized_keys
Now edit the SSH daemon's config:
# vi /etc/ssh/sshd_config
Look for the line _#PermitRootLogin no_, remove the comment sign and change it to _yes_. Save and exit the editor. Then power down the VM:
# shutdown -p now
When you're back at your host machine's prompt, edit the Vagrantfile:
% vi Vagrantfile
Now uncomment
config.ssh.username = "root"
then save and exit.
Test the change you just made by starting the VM and ssh'ing into it:
% vagrant up
% vagrant ssh
If everything is correct, you should have entered the VM as root now. Power down the VM again:
# shutdown -p now
We've now prepared our test environment for trying out all things mail. Use vagrant to create a snapshot now to save your progress so that we can pick up right there next time. But before we do so, we need to make sure that the VM is already powered down:
% vagrant status
If it says _running (virtualbox)_ then wait a moment and issue the same command again. It's important that the status is _poweroff (virtualbox)_. If you snapshot the VM while it's powering down, your machine will always power down if you restore the previous state! As soon as it's off, snapshot it:
% vagrant snapshot save mail1
You now have an idea of what the whole topic _email_ is all about. And you have a test system to try out things and get familiar with mail from the very fundamentals to - perhaps - a fully working mail server. I hope that this was interesting for you and that you'll follow the next part, too, where we'll explore sending mail from the console and discuss using a text mode MUA.
Introduction to email (pt. 2): Mail dialog / the "mail" command
