• Publicationsclose
• Research softwareclose
• Preprint close
• Article close
• AE close
• arXiv.org e-Print Archive close

We propose a data-driven approach for deep convolutional neural network compression that achieves high accuracy with high throughput and low memory requirements. Current network compression methods either find a low-rank factorization of the features that requires more memory, or select only a subset of features by pruning entire filter channels. We propose the Cascaded Projection (CaP) compression method that projects the output and input filter channels of successive layers to a unified low dimensional space based on a low-rank projection. We optimize the projection to minimize classification loss and the difference between the next layer's features in the compressed and uncompressed networks. To solve this non-convex optimization problem we propose a new optimization method of a proxy matrix using backpropagation and Stochastic Gradient Descent (SGD) with geometric constraints. Our cascaded projection approach leads to improvements in all critical areas of network compression: high accuracy, low memory consumption, low parameter count and high processing speed. The proposed CaP method demonstrates state-of-the-art results compressing VGG16 and ResNet networks with over 4x reduction in the number of computations and excellent performance in top-5 accuracy on the ImageNet dataset before and after fine-tuning.

We compare the half-light circular velocities, V_{1/2}, of dwarf galaxies in the Local Group to the predicted circular velocity curves of galaxies in the NIHAO suite of LCDM simulations. We use a subset of 34 simulations in which the central galaxy has a stellar luminosity in the range 0.5 x 10^5 < L_V < 2 x 10^8 L_{sun}. The NIHAO galaxy simulations reproduce the relation between stellar mass and halo mass from abundance matching, as well as the observed half-light size vs luminosity relation. The corresponding dissipationless simulations over-predict the V_{1/2}, recovering the problem known as too big to fail (TBTF). By contrast, the NIHAO simulations have expanded dark matter haloes, and provide an excellent match to the distribution of V_{1/2} for galaxies with L_V > 2 x 10^6 L_{sun}. For lower luminosities our simulations predict very little halo response, and tend to over predict the observed circular velocities. In the context of LCDM, this could signal the increased stochasticity of star formation in haloes below M_{halo} \sim 10^{10} M_{sun}, or the role of environmental effects. Thus, haloes that are "too big to fail", do not fail LCDM, but haloes that are "too small to pass" (the galaxy formation threshold) provide a future test of LCDM. 6 pages, 3 figures, accepted to MNRAS letters

Prototype-based meta-learning has emerged as a powerful technique for addressing few-shot learning challenges. However, estimating a deterministic prototype using a simple average function from a limited number of examples remains a fragile process. To overcome this limitation, we introduce ProtoDiff, a novel framework that leverages a task-guided diffusion model during the meta-training phase to gradually generate prototypes, thereby providing efficient class representations. Specifically, a set of prototypes is optimized to achieve per-task prototype overfitting, enabling accurately obtaining the overfitted prototypes for individual tasks. Furthermore, we introduce a task-guided diffusion process within the prototype space, enabling the meta-learning of a generative process that transitions from a vanilla prototype to an overfitted prototype. ProtoDiff gradually generates task-specific prototypes from random noise during the meta-test stage, conditioned on the limited samples available for the new task. Furthermore, to expedite training and enhance ProtoDiff's performance, we propose the utilization of residual prototype learning, which leverages the sparsity of the residual prototype. We conduct thorough ablation studies to demonstrate its ability to accurately capture the underlying prototype distribution and enhance generalization. The new state-of-the-art performance on within-domain, cross-domain, and few-task few-shot classification further substantiates the benefit of ProtoDiff. Comment: Under review

Images captured by fisheye lenses violate the pinhole camera assumption and suffer from distortions. Rectification of fisheye images is therefore a crucial preprocessing step for many computer vision applications. In this paper, we propose an end-to-end multi-context collaborative deep network for removing distortions from single fisheye images. In contrast to conventional approaches, which focus on extracting hand-crafted features from input images, our method learns high-level semantics and low-level appearance features simultaneously to estimate the distortion parameters. To facilitate training, we construct a synthesized dataset that covers various scenes and distortion parameter settings. Experiments on both synthesized and real-world datasets show that the proposed model significantly outperforms current state of the art methods. Our code and synthesized dataset will be made publicly available. Comment: 16 pages, 5 figures

Person re-identification has achieved great progress with deep convolutional neural networks. However, most previous methods focus on learning individual appearance feature embedding, and it is hard for the models to handle difficult situations with different illumination, large pose variance and occlusion. In this work, we take a step further and consider employing context information for person search. For a probe-gallery pair, we first propose a contextual instance expansion module, which employs a relative attention module to search and filter useful context information in the scene. We also build a graph learning framework to effectively employ context pairs to update target similarity. These two modules are built on top of a joint detection and instance feature learning framework, which improves the discriminativeness of the learned features. The proposed framework achieves state-of-the-art performance on two widely used person search datasets. Comment: To appear in CVPR 2019

Neural memory enables fast adaptation to new tasks with just a few training samples. Existing memory models store features only from the single last layer, which does not generalize well in presence of a domain shift between training and test distributions. Rather than relying on a flat memory, we propose a hierarchical alternative that stores features at different semantic levels. We introduce a hierarchical prototype model, where each level of the prototype fetches corresponding information from the hierarchical memory. The model is endowed with the ability to flexibly rely on features at different semantic levels if the domain shift circumstances so demand. We meta-learn the model by a newly derived hierarchical variational inference framework, where hierarchical memory and prototypes are jointly optimized. To explore and exploit the importance of different semantic levels, we further propose to learn the weights associated with the prototype at each level in a data-driven way, which enables the model to adaptively choose the most generalizable features. We conduct thorough ablation studies to demonstrate the effectiveness of each component in our model. The new state-of-the-art performance on cross-domain and competitive performance on traditional few-shot classification further substantiates the benefit of hierarchical variational memory. Comment: 17 pages, 5 figures

A characteristic feature of the 3d plaquette Ising model is its planar subsystem symmetry. The quantum version of this model has been shown to be related via a duality to the X-Cube model, which has been paradigmatic in the new and rapidly developing field of fractons. The relation between the 3d plaquette Ising and the X-Cube model is similar to that between the 2d quantum transverse spin Ising model and the Toric Code. Gauging the global symmetry in the case of the 2d Ising model and considering the gauge invariant sector of the high temperature phase leads to the Toric Code, whereas gauging the subsystem symmetry of the 3d quantum transverse spin plaquette Ising model leads to the X-Cube model. A non-standard dual formulation of the 3d plaquette Ising model which utilises three flavours of spins has recently been discussed in the context of dualising the fracton-free sector of the X-Cube model. In this paper we investigate the classical spin version of this non-standard dual Hamiltonian and discuss its properties in relation to the more familiar Ashkin-Teller-like dual and further related dual formulations involving both link and vertex spins and non-Ising spins. Comment: Reviews results in arXiv:1106.0325 and arXiv:1106.4664 in light of more recent simulations and fracton literature. Published in special issue of Entropy dedicated to the memory of Professor Ian Campbell

In this paper, we present a novel data structure, called the Mixture Graph. This data structure allows us to compress, render, and query segmentation histograms. Such histograms arise when building a mipmap of a volume containing segmentation IDs. Each voxel in the histogram mipmap contains a convex combination (mixture) of segmentation IDs. Each mixture represents the distribution of IDs in the respective voxel's children. Our method factorizes these mixtures into a series of linear interpolations between exactly two segmentation IDs. The result is represented as a directed acyclic graph (DAG) whose nodes are topologically ordered. Pruning replicate nodes in the tree followed by compression allows us to store the resulting data structure efficiently. During rendering, transfer functions are propagated from sources (leafs) through the DAG to allow for efficient, pre-filtered rendering at interactive frame rates. Assembly of histogram contributions across the footprint of a given volume allows us to efficiently query partial histograms, achieving up to 178$\times$ speed-up over na$\mathrm{\"{i}}$ve parallelized range queries. Additionally, we apply the Mixture Graph to compute correctly pre-filtered volume lighting and to interactively explore segments based on shape, geometry, and orientation using multi-dimensional transfer functions. Comment: To appear in IEEE Transacations on Visualization and Computer Graphics (IEEE Vis 2020)