The work aims to study the tribological properties of TiN/BN and AlN/BN nano-additives in oil lubrication and analyze the lubrication mechanism of nano-lubricating oil. Oleic acid was used as dispersant to improve the dispersibility of nano-additives in base oil, and the tribological properties of TiN/BN and AlN/BN nano-lubricants with different mixing ratios and concentrations were tested by MRS-10A four-ball friction and wear tester. SEM was used to observe the surface morphology of wear scar, and EDS and XPS were used to detect the types of elements on the surface of wear scar and corresponding valence states. When the mass ratio of nano-particles dispersed by oleic acid was 1︰1, the nano-lubricating oil showed the best anti-wear and anti-friction performance. With the application of 0.6wt% TiN/BN mixed additives, the wear scar diameter and friction coefficient were reduced by 34.97% and 16.75%, respectively and the maximum non-sticking bite load was increased by 65.96%. While the wear scar diameter and friction coefficient were reduced by 24.49% and 11.76% and the maximum non-sticking bite load was increased by 38.30%, with the application of 0.2wt% AlN/BN composite additives. The depth and width of the grooves and the surface roughness were reduced with the application of nano composite additives. AlN, BN and TiN nanoparticles dispersed in oil play the role of bearing support between friction pairs, changing sliding friction into sliding-rolling mixed friction and reducing friction and wear. AlN nanoparticles entering the friction pairs are deposited and adsorbed in the pits of the friction surface to repair the worn surface due to its high surface energy characteristics. TiN and BN nanoparticles react with the friction surface to form a lubricating self-repairing film composed of Fe-O, Ti-O, BOx and TiNxOy, which shows great anti-wear, anti-friction and self-repairing properties.