Your email is the key to your online world; you can reset your password at most other sites you use if you have access to your email. When you reuse your FastMail password at other sites, you're making it much easier for attackers to potentially break in to your account. Other sites often don't have the same high security measures as FastMail (such as compulsory HTTPS, locked-down servers, etc.), which makes them much easier for criminals to break in to. If they hold your email address and the same password that you use for FastMail, the attacker can then access your email account and get into everything else you use online.
FastMail provides a lot of different services over many different protocols. To ensure that your password is compatible with all of them, and the many (often slightly buggy) clients out there, we only allow the characters
In case you ever forget your password, it's useful to set a backup email address. This is where account recovery information is sent if you forget your username or password.