Optical computing presents a promising avenue to meet the escalating computational demands. However, optical analog computing is susceptible to environmental perturbations, relies heavily on digital-to-analog converters and analog-to-digital converters, and requires electronic or photonic nonlinear operations. While optical digital computing mitigates some issues, its reliance on manual, task-specific configuration hinders broader applications like inference. Here, we propose the concept of an optical logic convolutional neural network (OLCNN). We demonstrate a 1-by-3 optical logic convolutional operator (OLCO) for pattern generation and validate its high-speed computing capacity at 20 Gbit/s. A 2-by-2 OLCO is then implemented to perform three types of image edge extraction. By scaling up, a 3-by-3 OLCO is constructed for an OLCNN to achieve four-class classification on the MNIST dataset with an average test accuracy of 95.1%. By synergizing optical logic devices with neural networks, this work pioneers a logic-driven paradigm for high-speed, energy-efficient optical hardware in artificial intelligence.