Gravar-mail: Pump‐Color Selective Control of Ultrafast All‐Optical Switching Dynamics in Metaphotonic Devices