Rice field sidewalk (RIFIS) identification plays a crucial role in enhancing the performance of agricultural computer applications, especially for rice farming, by dividing the image into areas of rice fields to be ploughed and the areas outside of rice fields. This division isolates the desired area and reduces computational costs for processing RIFIS detection in the automation of ploughing fields using hand tractors. Testing and evaluating the performance of the RIFIS detection method requires a collection of image data that includes various features of the rice field environment. However, the available agricultural image datasets focus only on rice plants and their diseases; a dataset that explicitly provides RIFIS imagery has not been found. This study presents an RIFIS image dataset that addresses this deficiency by including specific linear characteristics. In Bali, Indonesia, two geographically separated rice fields were selected. The initial data collected were from several videos, which were then converted into image sequences. Manual RIFIS annotations were applied to the image. This research produced a dataset consisting of 970 high-definition RGB images (1920 × 1080 pixels) and corresponding annotations. This dataset has a combination of 19 different features. By utilizing our dataset for detection, it can be applied not only for the time of rice planting but also for the time of rice harvest, and our dataset can be used for a variety of applications throughout the entire year.