We present resistivity, magnetization and specific heat measurements on flux grown single crystals of NiBi3. We find typical behavior of a type-II superconductor, with, however, a sizable magnetic signal in the superconducting phase. There is a hysteretic magnetization characteristic of a ferromagnetic compound. By following the magnetization as a function of temperature, we find a drop at temperatures corresponding to the Curie temperature of ferromagnetic amorphous Ni. Thus, we assign the magnetism in NiBi$_3$ crystals to amorphous Ni impurities.