Objective To investigate the influence of surface mechanical attrition treatment technique on the grain refinement behavior and mechanism of AZ91D magnesium alloys. Methods The AZ91D magnesium alloy in hexagonal-closet-packet structure was treated by surface mechanical attrition treatment (SMAT) with different ball diameters and handling time, and the microstructure of surface deformed layer was characterized and compared. Results The grain of AZ91D magnesium alloys was obviously refined by surface mechanical attrition treatment. In a certain range, as the balls diameter or handling time increased, the grain size became smaller, and could finally reach about 24 nm. Conclusion The grain refinement mechanism of AZ91D magnesium alloy by surface mechanical attrition treatment was the results of twinning and dislocation slip. When the diameter of balls was 16 mm and the handling time was 180 min, surface mechanical attrition treatment has the highest efficiency.