@eitanc: I think you said you can send mail without authentication on the test form. Try sending the message to a different domain, so if you're using gmail, send to yahoo, or if you're not using gmail, send to gmail. If you send mail to yourself with your own mailserver, it'll probably always accept it, with or without authentication.
Maybe your config allows you to send email to yourself (so WordPress emails to you would work) but emails to other places won't.