Electric field of an EM wave is an oscillating field and so is the electric force caused by it on a charged particle. This electric force averaged over an integral number of cycles is zero since its direction changes every half cycle. Hence, electric field is not responsible for radiation pressure.