Solution-casting was used to create films of polyvinyl alcohol (PVA) nanocomposite containing different concentrations of Fe3O4@SiO2 nanoparticles (NPs). The co-precipitation technique was used to synthesize Fe3O4@SiO2 NPs. Fe3O4@SiO2 impact on the PVA structure was investigated via X-ray diffraction (XRD), optical microscope, and Fourier transform-infrared (FT-IR) techniques. XRD reveals the destruction of the PVA semi-crystallinity with the Fe3O4@SiO2 additive. FT-IR analysis supported hydrogen bond formation between PVA molecules and the Fe3O4@SiO2 surface. The UV–visible spectrophotometer was used to investigate the optical parameters. The optical bandgap decreased with increasing the Fe3O4@SiO2 concentration in the PVA matrix. Based on the optical bandgap, the theoretical linear refractive index (n) was deduced with theoretical models. The enhancement in the nonlinear refractive index and nonlinear optical susceptibility with the Fe3O4@SiO2 additive to the PVA matrix makes it a possible material for nonlinear optical devices. © The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2024.