Abstract As someone who had been attending conferences on pattern recognition and computer vision since 1978, I have watched with interest the ups and downs of pattern recognition and computer vision areas and how they have been presented at various conferences such as PRIP, CVPR, ECCV, ICCV, ACCV, ICPR, IJCAI, AAAI, NIPS, ICASSP and ICIP. Given the recent successes of deep learning networks, it appears that the scale is tilting towards pattern recognition as is commonly understood. A good number of papers in recent vision conferences seem to push data through one or other deep learning networks and report improvements over state of the art. While one cannot argue against the remarkable (magical?) performance improvements obtained by deep learning network-based approaches, I worry that five years from now, most students in computer vision will only be aware of software that implements some deep learning network. After all, 2-D based detection and recognition problems for which the deep learning networks have shown their mettle are only a subset of the computer vision field. While enjoying the ride, I would like to caution that understanding of scene and image formation, invariants, interaction between light and matter, human vision, 3D recovery, and emerging concepts like common sense reasoning are too important to ignore for the long-term viability of the computer vision field. It will be a dream come true if we manage to integrate these computer vision concepts into deep learning networks so that more robust performance can be obtained with less data and cheaper computers.