Convolutional neural networks usually learn useful features from training data. The features learned by the first convolutional layer are often the basic elements of some training data depending on the task. For example, in image data, the learned features can reflect edges and spots. In the subsequent network layer, these learned features can represent more abstract and advanced features.

Visualizing the learned features and their changes over time can provide some effective information about how the network learns. In fact, the network structure is far more than that of a few-layer network. A large number of convolution kernels makes it difficult to intuitively explain and analyze the learned features.

However, we can prove by contrast experiments how the weights of the convolution kernel develop in real time as the network learns. Since the characteristics that the network should learn are already known in advance, that is, the data generation process and parameters are completely defined and completely under our control, the learning task can be easily determined. We can achieve the above process by constructing a very simple single-layer convolutional network and training it to use multiple kernels for linear filtering.

In the next experiment, we use a traditional edge detection method commonly used in image processing and computer vision on the dataset-Sobel edge filtering, and train our model to perform similar linear mapping. We also try to use some larger kernels than Sobel filters to learn some more general and arbitrary filters.

These can help us feel how the convolutional layer in the neural network operates on the input data, how the weight of the convolution kernel changes during training, and how the training of the neural network is regarded as a minimization problem.

First, we must use a linear filter to process the image data X to obtain the filtered result Y of the original image. Linear filter operation can be summarized as follows:

For any set of parameters (convolution kernel) or input data we can think of, linear filters have well-defined operations.

We can now construct a single-layer, single-core, approximately linear filtered convolutional neural network. The calculations that occur in the two ways of linear filter and convolutional neural network are exactly the same except for the convolution kernel parameters we want to learn from the data.