Gravar-mail: Habitable Planets Around White and Brown Dwarfs: The Perils of a Cooling Primary