How to make HW accelerators for Machine Learning

RNN has been increasingly used in recent years, and it is being used in various ways, such as language translation, automatic sentence completion, card fraud prevention, and prediction of time series data such as stocks/weather. Here, we will use a model that predicts time series data. Before RNN was used, these data were predicted using a statistical method called ARIMA. Using a kind of filter technology, relatively near future were predicted, and with the advent of RNNs, more accurate predictions were made possible by learning Big Data. In particular, in the case of periodicity, it can be predicted with ARIMA because it has a temporal autocorrelation structure.  Anyway, we'll implement this model for a relatively simple and easy-to-check example. Of course, there is a way to build this model using CNN, but it does not show optimal performance in terms of performance because it has no choice but to receive the input as a fixed. If there is an opportunity in the future, I will try ...

CNN already has too many tutorials and is composed of various layers such as convolution and max-pooling in the middle. When you think of RNN, you think of language translators, and then you think of LSTMs (Long Short Term Memory) or GRUs (Gated Recurrent Unit). Since it can also take considerable time and effort to implement in hardware, I selected an RNN example for Sinewave Prediction that is as simple as possible and can show the characteristics of RNN well. Through this example, I would like to explain the whole process of how an ML model made in Python can be implemented as a hardware accelerator.  The development outline is as below 1. Reference model development for Wave Prediction (Pytorch)  2. System Simulator development for RTL design from Pytorch reference model (Python)  3. RTL IP development (Modelsim Simulation)  4. Complete system design and FPGA bit file creation using Xilinx Vivado  5. Write overlay driver in Python on Pynq-Z2 board and verify...

T here are too many articles or blogs on this topic, so I think that explaining from the scratch would add unnecessary traffic to the Internet. Usually, the most referenced site is Google's Tensorflow tutorial or Pytorch tutorial, and those who want to look into it, they will meet Andrew Ng's Stanford course (Coursera provides the cool lectures of Andrew Ng, so I strongly recommend you take the course.) Most of the tutorials start with saying that anyone can use AI very easily without knowing the mathematical details. I think some are right and some are wrong. You should know at least basic linear algebra such as matrix addition/multiplication, derivertives, etc., so you can go one step further. Even though I completed tutorials (like MNIST) without any hurdle, I felt get so astrayed that I never catched what to do next. Most hardware engineers might feel the same feeling as mine. So, I decided to dig out what's inside of Machine Learning to turn it into hardware implementa...