Abstract Owing to the higher aspect ratio and presence of negatively-charged carboxylate groups that would favorably interact with hydrophilic amide groups in poly(N-isopropylacrylamide) (PNIPAM) molecules, TEMPO-oxidized cellulose nanofibers (TO-CNF) is a kind of promising candidates to improve both swelling and mechanical properties of PNIPAM hydrogels. In this work, a series of TO-CNFs were isolated from inexpensive and abundant bamboo pulp to clarify effects of carboxylate contents (0.5–1.8 mmol/g) on the surfaces of TO-CNF on microstructure as well as compression and swelling properties of PNIPAM/TO-CNF composite hydrogels. An increase in the carboxylate content on the surfaces of TO-CNFs resulted in a higher degree of nanofibrillation. With increasing carboxylate content in the added TO-CNFs, the pore size of the composite hydrogels progressively became larger. It was found that the lower critical solution temperature (LCST) of PNIPAM hydrogels was independent of presence of TO-CNF and its carboxylate contents, while the equilibrium swelling ratio of PNIPAM/TO-CNF composite hydrogels below LCST were largely influenced by carboxylate contents of added TO-CNFs. The incorporation of rigid TO-CNF markedly strengthened the three-dimensional network structure of PNIPAM hydrogels, and compressive properties of the composite hydrogels were found to continuously increase with the carboxylate content in the TO-CNF. When carboxylate content of TO-CNF was 1.8 mmol/g, both compressive strength and modulus of the composite hydrogel reached 79.60 kPa and 0.97 MPa, respectively, which were 300% and 900% higher than those of pure PNIPAM one. Thus, TO-CNF appeared to be a “green” nanofiller that can simultaneously improve swelling and mechanical properties of PNIPAM hydrogels.