In this paper, we report a magnetic nanoparticle (MNP) detection scheme without the presence of any external magnetic field. The proposed magnetic sensor uses a patterned groove structure within the sensor so that no external magnetic field is needed to magnetize the MNPs. An example is given based on a giant magnetoresistance (GMR) sensing device with a spin valve structure. For this structure, the detection of MNPs located inside the groove and near the free layer is demonstrated under no external magnetic field. Micromagnetic simulations are performed to calculate the signal to noise level of this detection scheme. A maximum signal to noise ratio (SNR) of 18.6 dB from one iron oxide magnetic nanoparticle with 8 nm radius is achieved. As proof of concept, this external-field-free GMR sensor with groove structure of 200 nm × 200 nm is fabricated using a photo and an electron beam integrated lithography process. Using this sensor, the feasibility demonstration of the detection SNR of 9.3 dB is achieved for 30 μl magnetic nanoparticles suspension (30 nm iron oxide particles, 1 mg/ml). This proposed external-field-free sensor structure is not limited to GMR devices and could be applicable to other magnetic biosensing devices.