The work aims to predict the wheel surface defects during the production by constructing a visual inspection system for the online production of the wheel. According to the definition and evaluation criteria of wheel surface defects, a calculation model of wheel surface defects was produced. The improved Faster-RCNN target detection algorithm was used, and the deep generative adversarial network was introduced to eliminate the ambiguity of the image. The limpid image of wheel surface was adopted for model training and the network parameters were optimized combined with the expert‘s discriminative criteria, thus constructing a wheel surface defect detection model. The deep learning Pytorch framework was used to perform the model training on the NVIDIA’S Tesla P100 image acceleration card, and the model results were analyzed comparatively to find the optimal prediction model. In the basic network, the performance of the ResNet101 network using the residual model improved the accuracy rate by 24% compared with the VGG16 model. The multi-channel fusion module was introduced into the target detection network model, and the accuracy rate was improved by 2%. Then, the FPN pyramid was introduced, and the low-level and high-level semantic information was incorporated, which made the output of multi-scale prediction map better. Finally, the ROI-Pooling algorithm of the residual network was changed to the ROI-Align algorithm, which improved the accuracy by 5%. With continuous improvement and optimization, the recognition rate of the wheel surface defects was improved continuously. The improved Faster-RCNN network can recognize the types and locations of wheel surface defects, thus meeting the requirements of the production environment, and providing great value in the engineering application.