Spin current is the key element for nanoscale spintronic devices. For ultrafast operation of such nano-devices, generation of spin current in picoseconds, a timescale that is difficult to achieve using electrical circuits, is highly desired. Here we show thermally driven ultrafast demagnetization of a perpendicular ferromagnet leads to spin accumulation in a normal metal and spin transfer torque in an in-plane ferromagnet. The data are well described by models of spin generation and transport based on differences and gradients of thermodynamic parameters. The temperature difference between electrons and magnons is the driving force for spin current generation by ultrafast demagnetization. On longer timescales, a few picoseconds following laser excitation, we also observe a small contribution to spin current by a temperature gradient and the spin-dependent Seebeck effect.