This paper aims to explore the physical properties of different materials of roller polishing blocks and their influence on the finishing effect of 7075 aluminum alloy. Under the same conditions, the 7075 aluminum alloy specimens were subjected to barrel polishing and finishing experiments and friction and wear experiments with different materials of rolling abrasive blocks to test the roughness value and metal removal amount of the aluminum alloy specimens before and after processing. Scanning electron microscope (SEM) and energy disperse spectroscopy (EDS) were used to characterize the changes of the surface and wear scar morphology and element composition of the specimen before and after processing. For aluminum alloy specimens with initial surface roughness Ra of 1.200 μm, the brown fused alumina grinding block had the highest processing efficiency, the lowest processing limit Ra of 0.262 μm, and the material removal of the specimens was the largest, showing an increasing trend. The surface brightness of the specimens processed by white ceramics was the best, followed by alumina, and brown corundum the worst. The surface of the aluminum oxide processed specimens showed accumulation of Fe elements. After the brown corundum abrasive block was processed, the surface pits and other defects of the specimens were more serious, with the content of C and O elements in the pits reaching 20.85% and 41.86%, respectively. For aluminum alloy specimens with rough initial surface, the processing efficiency and processing limit of white ceramic grinding block are moderate, and the surface of processed specimens has no defects and the surface brightness is the best. The rolling mill with small particle strength has better self-sharpness and high processing efficiency. However, due to its poor surface compactness and high abrasion, the grinding block not only has a low utilization rate, but also is more likely to adsorb and oxidize with the specimen surface, forming impurity particles and metal oxides on the surface of the specimen and reducing the surface brightness of the specimen.