The work aims to investigate into the process and effectiveness of rust removal technique by shot blasting based on numerical simulations so as to direct the optimization of technological parameters. While low-carbon steel for building and its corrosion products were taken as the object of study, finite element (FE) models were established by utilizing the element removal technique and periodic cell algorithm, so as to simulate the rust removal process for the mild steel under the random impact of various shots. The simulations were validated by comparing the results with the experimental data in literatures. The effectiveness of the shot blasting in rust removal was examined in terms of rust removing rate, residual stress distribution and surface roughness. The effectiveness of shot blasting treatment was improved by increasing kinetic energy of the impinging shots, i.e. the incident velocity and shot diameter. Longer shot blasting treatment was required to remove the rust for thicker oxide layer. Shot blasting is an efficient technique to remove the rust coating, improve the surface roughness and introduce the compressive residual stress. Hence it has promising application prospect. FE simulation is a useful tool to provide flexible study approach for optimizing the process.