The work aims to key factors affecting material removal mechanism and erosion resistance of CVD diamond film during sand erosion by analyzing morphology features and sand erosion resistance of CVD diamond film and several common infrared window materials during sand erosion. Jet-type erosion wear system was used to test erosion resistance of CVD diamond film and other common infrared optical materials. Morphology of materials after erosion was observed with SEM, sand erosion rate of infrared materials was measured with electronic balance. The infrared optical materials before and after sand erosion were measured with infrared spectrometer, and infrared performance variation before and after erosion was evaluated. The sand erosion resistance of CVD diamond film was much higher than that of Ge, ZnS, MgF2 and quartz glass. Only after 6 seconds of sand erosion test under set measurement conditions, infrared transmittance of Ge, ZnS, MgF2 and quartz glass decreased by 40%~46% except for CVD diamond film. While the transmittance of CVD diamond decreased by merely 9.5% even though CVD diamond has been eroded for 240 min under the same condition, and thereby exhibiting superior sand erosion resistance. Main cause of CVD diamond film erosion is microcrack formation and material loss led by expanded connection. The main cause of material erosion is repeated cutting and ploughing, for other materials, crack formation as well as cutting and ploughing.