The waviness sawing mark as well as the high surface roughness is a huge problem that hinders the production rate of the NdFeB magnet. The work aims to propose a new ultrasonic vibration assisted water lubricated sawing method to improve the surface quality and reduce the surface roughness and research the material removal mechanism under ultrasonic vibration assistance. The formation mechanism of the linear bow was analyzed, and the relationship between material removal form and chip removal lubrication was studied. The formation mechanism of line bow, line mark and surface morphology was analyzed comprehensively, and the removal mechanism of materials was revealed. Several diamond wire sawing experiments were conducted under a feed rate of 0.1 and 0.2 mm/min, while the sawing condition was oil lubricated, water lubricated and ultrasonic vibration assisted with water lubrication. The surface roughness of the sawed surface from all groups was obtained under white light interferometer and so was the surface morphology. It was discovered that the surface roughness of the sample under three conditions was almost the same when the feed rate was only 0.1 mm/min. When the feed rate increased to 0.2 mm/min, the surface roughness Ra for the water lubricated group can be controlled below 0.4 μm, which was half of the oil lubricated group. And its PV value was also half of the oil lubricated group, which can effectively reflect the influence from waviness mark, was much lower for water lubricated group and even lower for the ultrasonic vibration assisted group. But the surface roughness Ra along the sawing direction for ultrasonic vibration assisted group was higher than the water lubricated group. The wire bow was measured under both water lubricated sawing and ultrasonic vibration assisted sawing using a linear laser displacement sensor. The lateral movement of the wire at different spots were measured by a linear laser displacement sensor. The lateral displacement for the diamond wire were as high as 0.55 mm at the wire releasing spot, and gradually decreased to 0.05 mm and then 0.01 mm after going through the tension wheel and the guiding wheel. The sawing part of the wire still had a 5-10 μm lateral displacement due to the large lateral gap on the guiding wheel. At the same feed speed, the difference line between ultrasonic vibration assisted bow deformation and water cutting decreases after increasing, and reaches 0.5 mm at the feed speed of 0.2 mm/min. It was discovered that the surface waviness mark was formed due to the lateral displacement of the diamond wire and it was originated from the spool lateral feed. After that the wire bow were measured for both water lubricated groups. The wire bow for ultrasonic vibration assisted groups were higher and it was discovered that the micro-grinding effect was responsible for the wire bow increase and the reduced surface mark. The ultrasonic vibration induced the wire to vibrate in vertical direction, hence having an intense micro grinding effect on the sawed surface. Such micro grinding effect effectively removed the sawing mark but at the same time increased the cutting force, hence the larger wire bow. Using a self-developed Matlab script, the total area of fracture zones on the microscope image of sawed surface can be identified and calculated. It was found that the fracture rate of the surface under oil lubrication was much higher than water lubrication, which shows that the chip transportation of the cutting fluid is very important for the wire sawing process. In conclusion, the ultrasonic vibration assisted diamond wire sawing under water lubrication can not only improve the surface quality, but also eliminate the surface waviness mark.