Lane detection is one of the most important task in autonomous driving. While the semantic segmentation based method is widely explored and recognized in recent decade, some post-processing are required to estimate the exact location of the predicted lane markings and can be easily failed in complex scenarios. To tackle these limitations, this paper proposes a novel lane detection network named PCRLaneNet. Firstly, we use a fully convolutional network to predict the coordinates of lane marking points directly, which can better meet with the requirements of autonomous driving. Secondly, to take the fully advantage of the correlation of these lane marking points, a point feature fusion strategy is designed to fuse feature maps of the points on the same lane marking, which makes our method capable of handling challenging scenarios. Lastly, the robustness, accuracy and latency of the proposed method are extensively verified in two datasets (CULane and TuSimple).