The work aims to propose a magnetic tile defect detection and recognition algorithm based on the improved UNet model and a classification neural network for the limitation of extracting the surface defects of the magnetic tile by the traditional algorithm, and the lack of accuracy in the method for judging the defect type by artificially selecting the defect feature. The improved UNet model was used to extract defects, and the classification neural network was used to classify and identify the extracted defect regions. In order to improve the classification accuracy of the model, the cavity convolution was adopted to replace the partial convolution layer and the pooling layer of the UNet model to reduce the loss of detail caused by multiple pooling. At the same time, adding multiple jump connections enabled the UNet model to combine more convolution features. The experimental results showed that the improved UNet model could predict the defect area by 93%. According to the prediction results, classification neural network was used to classify defects. After verified by experiment, the accuracy of the classification could reach 94% and meet the industrial requirements. The improved UNet model improves the accuracy of magnetic tile defect extraction; the classified neural network has higher defect classification accuracy; and the combination of improved UNet model and classification neural network can realize the defect extraction and classification identification simultaneously and effectively, which lays a foundation for the quality detection and performance evaluation of the magnetic tile.