Keep the original intention and move forward

This not only records some nutritious materials consulted during the study, but also records a little bit of my opinion about the resource for the students of the same way to enjoy.

The resources are all fragmented and there is no complete routine or system. However, in addition to daily meals, some snacks will appear more colorful. and so,

There must be something to help you.

• Least squares
  This article not only clarifies what the least squares method is (this is not the purpose of this article I recommend), but also most students have never thought about it. Or a question that has never been noticed-why it is the least squares method (this is the focus), and the analysis is thorough.
  The focus of the reading is on the fourth part—Principle Exploration, and the link to that chapter.

• Wasserstein GAN
  This article is really good news for me who don’t like reading papers (in fact, I don’t understand). It explains the content of Wasserstein GAN paper in a simple and complete way, including not only the principle but also the formula derivation. And the formula derivation is easy to understand and very friendly to mathematics.

• GPU temperature is too high during deep learning training? A few commands to quickly cool down your GPU.
  On a hot summer day, although the air conditioner and the fan are flying together, my position is still several degrees higher than other people, and waves of heat are rushing from the leg box to the whole body. Maybe many colleagues in machine learning will have this experience when running the program, so this article is absolutely useful.

• How to determine the size of the batch when training the neural network?
  This article has some titles, so the content is general. But look at it still has a little benefit.

• If you only know GAN, you are OUT-the philosophy and mathematical principles behind VAE
  If you only know GAN, you will be OUT. Have you heard it yet?
  This article does not seem to me as an introduction to VAE. It requires some VAE foundation and some math skills, and then read a few more times, it will make a leap in the understanding of VAE.
  This article and the above"Amazing Wasserstein GAN"Similarly, they all need to be read over and over again in the practice of GAN and VAE, and then a deeper understanding will be produced.

• Interpretation of Batch Normalization

• Why does Batch Normalization work well in deep learning?
  For Batch Normalization, these two are enough.
  batch normalization is "batch normalization". Google's description in the ICML article is very clear, that is, in each SGD, the mini-batch is used to normalize the corresponding activation, so that the average value of the results (all dimensions of the output signal) is 0 and the variance is 1.
  When the neural network training encounters a slow convergence rate, or a gradient explosion and other untrainable conditions, you can try BN to solve it. In addition, BN can also be added to speed up the training speed and improve the accuracy of the model in general use.

• When using tensorflow, is there a way to limit the size of video memory occupied by each task?
  Running a model, the entire video memory is filled, which is really an inconvenient setting. But this can be modified.
  But why does Google have to set up to fill up the video memory by default?

• "SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient" paper notes
  seqGAN is the first work that combines reinforcement learning and GAN, it is very meaningful. And this idea can continue to dig. If the thesis is difficult, just look at this~

• GAN hidden space dimension selection
  A very interesting article. Have you ever thought about changing the dimension of the initial noise when running GAN? Still using the default 100? Have you ever wondered what this dimension can stand for, and what effect does larger or smaller have on the training results?
  After reading this article, you can try to write the code to implement the formulas in it, and see what is the minimum dimension required by your data set.

If you also like machine learning, and work hard on ML like me, please follow me, I will update it from time to time, there are always some that can help you.