Shift-invariant convolution

1. What is shift-invariant in convolution?
2. How is CNN shift-invariant?
3. What is shift equivariance?
4. Is convolution shift equivariant?

What is shift-invariant in convolution?

Shift-invariance: this means that if we shift the input in time (or shift the entries in a vector) then the output is shifted by the same amount.

How is CNN shift-invariant?

Thanks to the use of convolution and pooling layers, convolutional neural networks were for a long time thought to be shift-invariant. However, recent works have shown that the output of a CNN can change significantly with small shifts in input: a problem caused by the presence of downsampling (stride) layers.

What is shift equivariance?

So in this case the output of the cat detector should shift exactly in the same way as the cat is shifted So basically shift equivariance means that (click) sfx produces the same result as fsx. In other words f and S commute. F applied to shifted S is the same as S applied to f applied to the output of f.

Is convolution shift equivariant?

Convolutional neural networks lack shift equivariance due to the presence of downsampling layers. In image classification, adaptive polyphase downsampling (APS-D) was recently proposed to make CNNs perfectly shift invariant.