Deep feedforward neural network models of vision dominate in both computational neuroscience and engineering. However, the primate visual system contains abundant recurrent connections. Recurrent signal flow enables recycling of limited computational resources over time, and so might boost the performance of a physically finite brain or model. In particular, recurrence could improve performance in vision tasks. Here we find that recurrent convolutional networks outperform feedforward convolutional networks matched in their number of parameters in large-scale visual recognition tasks. By terminating recurrent computations once the output probability distribution has concentrated beyond a predefined entropy threshold, we show that recurrent networks can trade off speed for accuracy without employing additional parameters for deeper computations. This enables balancing the cost of error against the cost of a delayed response (and of greater energy consumption). In addition to better task performance, recurrent convolutional networks better predict human reaction times than parameter-matched and state-of-the-art feedforward control models. These results suggest that recurrent models are preferable to feedforward models of human vision in terms of their more realistic connectivity, improved performance and flexibility in vision tasks, and their ability to explain human behavioural responses. Author summary Deep neural networks provide the best current models of biological vision and achieve the highest performance in computer vision. Inspired by the primate brain, these models transform the image signals through a sequence of stages, leading to recognition. Unlike brains, however, these models do not process signals recurrently, with outputs of a given component computation being fed back into the same computation. The ability to recycle limited neural resources by processing information recurrently could explain the robustness and flexibility of biological visual systems, which is not yet matched by computer vision systems. Here we report that recurrent processing can improve recognition performance compared to similarly complex feedforward networks. Recurrent processing also enabled models to behave more flexibly and trade off speed for accuracy. Like humans, the recurrent network models can compute longer when an object is hard to recognise, which boosts their accuracy. The model’s recognition times correlated with human recognition times for the same images. The performance and flexibility of recurrent neural network models illustrates that modeling biological vision can help us improve computer vision.
- Downloaded 1,062 times
- Download rankings, all-time:
- Site-wide: 10,360 out of 89,036
- In neuroscience: 1,617 out of 15,842
- Year to date:
- Site-wide: 6,591 out of 89,036
- Since beginning of last month:
- Site-wide: 7,255 out of 89,036
Downloads over time
Distribution of downloads per paper, site-wide
- 18 Dec 2019: We're pleased to announce PanLingua, a new tool that enables you to search for machine-translated bioRxiv preprints using more than 100 different languages.
- 21 May 2019: PLOS Biology has published a community page about Rxivist.org and its design.
- 10 May 2019: The paper analyzing the Rxivist dataset has been published at eLife.
- 1 Mar 2019: We now have summary statistics about bioRxiv downloads and submissions.
- 8 Feb 2019: Data from Altmetric is now available on the Rxivist details page for every preprint. Look for the "donut" under the download metrics.
- 30 Jan 2019: preLights has featured the Rxivist preprint and written about our findings.
- 22 Jan 2019: Nature just published an article about Rxivist and our data.
- 13 Jan 2019: The Rxivist preprint is live!