When plastic deformation of materials is caused by machining, the surface of the specimen will appear a degree of fiber structure distortion along the cutting speed direction of the tool, which will change the stress direction along the grain boundary inside the surface metal grains. Therefore, when the direction of metal fiber structure is parallel or perpendicular to the loading direction, its influence on the fatigue performance of the specimen is necessarily different. The work aims to explore the influence law and mechanism of the fiber structure direction in the plastic deformation layer of the turning and milling surface on the fatigue resistance of TC4 titanium alloy specimens, improve the surface integrity research system, and provide references for fatigue resistance manufacturing of actual aerospace structural parts. With TC4 titanium alloy as the test material, the surface integrity test of turning and milling processing and the fatigue performance test of the specimens were carried out. By selecting turning tools and milling tools with the same radius, the fatigue specimens with the two machining methods had similar surface morphology. Processing parameters were selected reasonably to control the surface roughness, surface microhardness and surface residual compressive stress of the fatigue specimens. Through the above measures, the relationship between the direction of the fiber structure in the plastic deformation layer and the fatigue life of the specimens can be analyzed more intuitively. In this experiment, surface roughness, surface microhardness and surface residual stress of the turned fatigue specimens were better than those of the milled specimens, and the surface morphologies of the two were very similar, however, the fatigue resistance of the latter was 12 to 48 times that of the former. The large difference in fatigue life cannot be explained by surface roughness, surface microhardness, surface residual stress and surface morphology. By observing the surface microstructure, it is found that the direction of the fiber structure in the plastic deformation layer on the surface of the two specimens is different, which is the dominant factor that the fatigue resistance of the milling specimens is much greater than that of the turning specimens. Turning and milling are different in the mechanisms for forming the machined surface. The plastic deformation direction and deformation degree of the surface fiber structure are also quite different. The fiber structure of the turning fatigue specimens is along the circumferential direction of the specimens, and the fiber structure of the milling fatigue specimens is along the axial direction of the specimens. Under the alternating load in the axial direction, the fiber structure along the axial direction can inhibit the initiation and propagation of fatigue cracks, especially the former period. Thus, the fatigue resistance of milling TC4 titanium alloy specimens is greatly improved. On the premise of meeting the design requirements of the machined surface quality, by selecting the machining method with the same cutting speed direction and the loading direction of the specimens, the direction of the fiber structure in the machined plastic deformation layer is the same as the direction of the alternating load thereby improving the fatigue resistance of the specimens.