I got the feeling that the people keeping it a secret were doing so with the intent of preventing others from using it. It didn't seem to have any obvious negative effects. But I did get the impression they were "given" that magic. Likely by sacrificing something. So maybe it was only safe for them...