Success! [1]
Basically, install mod_ssl [2] first, with the file locations you want for Apache [3] specified using the configuration script configure. Then, using config.status, add the other modules (like mod_perl [4] or mod_php [5]) and forget about adding them to the configuration file by hand.
[Don't assume this is actually usable advice. This is so I don't forget what to do. Your milage may vary.]