RNN (3) - System Simulator

-

The Pytorch reference in the previous post uses the optimized library and floating point number, so the performance can be considered the most ideal. In the system simulator for Verilog design, some restrictions follow. 

- Fixed Point Number: Because the range of numbers is limited, performance restrictions follow. 

- Suboptimal Matrix Operation: The existing Pytorch model has been optimized for efficient and accurate operation in the Python environment. There is a difference in performance in the System Simulator that is actually implemented. 

- Non-linear activation function: Here, tanh (hyperbolic tangent) is used, and linear approximation is performed for convenience of implementation. 


Here, a 16-bit fixed point number is used. In fact, all 16 bits are not used and only 7 decimal places are used for easy calculation. Unused redundant bits should be optimized and removed in the future, but the truncation so that prevent overflow during every Addition/Multiplication is time-consuming. It can be simply hacked by multiplying the actual parameter by 128 and make it an 7-bit integer. The final result is obtained by dividing the final result by 128 again. By doing so, it is possible to reduce the truncate work for each operation. This part is difficult to understand because it is a bit complicated to explain in words, so let's take a simple example. 

A = 0.35, B = -0.20 -> AxB = ? 

Suppose we solve this problem. then, 

1) Convert A and B to fixed point 

A = 0.0101100110011001101 B = -0.00110011001100110011 

2) Multiply each by 128 to treat A and B. (Actually, it is the same as 7-bit shift left.) 

A = 16'b0000000000101100 (44), B = -16'b0000000000011001 (-25) 

3) AxB = -10001001100 (1100) 

4) Divide the above result by 128x128 (14 bit shift right)

AxB = -0.00010001001100 (-0.067)

The reason why such a complicated process is necessary is that floating operations are rarely used in Verilog considering the implementation area. Therefore, we get a result of -0.067, which is actually less than -0.07 that we expected. It can be seen as a kind of quantization error. The System Simulator defines a constant called "Precision" to determine how many decimal places can be cut off. 128, which is the line that the system performance is judged to be appropriate, is set, and if a value smaller than that is specified, many decimal points of the parameter are truncated and the performance deteriorates. When it was raised above that, there was no significant performance improvement, so it was defined as 128 (7-bit shift). And, if you specify the number like this, you can use Verilog's 2's complement, so you can calculate conveniently without worrying about sign extension one by one. (For the concept of 2's compliment, sign extension, please check the wiki.) Once the number system is established, it is enough to figure out which operations are performed sequentially. A summary of the parameters of the reference model is as follows. (If you run the last line of Reference, you can get each parameter and size.)


- rnn.weight_ih_l0 param.shape: torch.Size([9, 1])    => Wih
- rnn.weight_hh_l0 param.shape: torch.Size([9, 9])   => Whh
- rnn.bias_ih_l0 param.shape: torch.Size([9])           => bih
- rnn.bias_hh_l0 param.shape: torch.Size([9])          => bhh
- out.weight param.shape: torch.Size([1, 9])           => Who
- out.bias param.shape: torch.Size([1])                  => bho

Now, if we perform the operation in the following order, we can obtain the desired inference.

i)     Xo = Wih x Xi + bih
ii)    Hi = Whh x Hi-1 + bhh (H0 = 0)
iii)   Ho = tanh(Xo + Hi)
iv)   Ypred = Who x Ho + bho
If you execute the code in the link below, you can see the performance of the reference model and the system simulator as shown in the figure below. There is a clear performance difference due to the three factors mentioned above, and once the trend is following well, if the result is scaled, the prediction result can be compensated to some extent. So, the System Simulator is frozen here. It is also used to make a test vector necessary for Verilog development, and is also used to compare with the final RTL result. (* I am a hardware engineer, so Python code may be not nice from a SW engineer's point of view. Since it was written with an algorithm that is as close to Verilog as possible. It would be appreciated if someone could verify whether there is any other reason other than the quantization error. Smart truncation can be applied while monitor the performance not to degrade the performance. But it's out of the scope)

-          data = torch.sin(time+torch.sin(time))


-          data = torch.sin(time+torch.sin(time)+torch.cos(time))



source: https://github.com/bxk218/RNN_wave_predictor_verilog/blob/main/systemSimulator.ipynb

Comments

Post a Comment

Popular posts from this blog

RNN (5) - FPGA System Design

-
Image
This is the last post on hardware implementation. The RTL codes of the previous post were designed for HW Accelerator IP to perform RNN, and this post describes how to design the whole system for targetting FPGA. The FPGA board used here is an FPGA Board using Xilinx ZYNQ-7000 as open hardware called Pynq-z2. (For Pynq-z2, please refer to the link below.) -  http://www.pynq.io/ -  https://www.tulembedded.com/FPGA/ProductsPYNQ-Z2.html -  https://pynq.readthedocs.io/en/latest/ First, the RNN Accelerator must open a channel to communicate with the ARM-Cortex A9 where SW is running. IP Packaging is performed using Vivado of Xilinx, and I used the AXI interface template provided by Vivado. RTL codes  on the right of the figure below  in the code window  shows how to  combine RNN IP with AXI interface. IP Packaging is possible easily through a few steps in Vivado. (Details will be posted separately.) After IP Packaging, you need to create a bit file to downl...
Read more

RNN (4) - RTL Design

-
Image
I designed the RTL in the same way as the System Simulator designed in the previous post. The major considerations for RTL design are as follows.  (The detailed code description will be posted in a separate post later.  Also, there are many good tutorials on how to use Intel Quartus Modelsim, so you'd better search for it. I will post the explanation of simulating this code in a separate post. ) - I/O method: The most common method is to save the input in the memory, drive the RNN, save the predicted value to output memory, and then send the completion signal. At this time, the memory can be selected in various ways. Various methods such as FIFO, Single Port, Dual Port, etc. can be used. Here, Single Port Memory is selected as the simplest.  - Timing: RTL stands for Register Transfer Level. I don't know who put it on, but it seems to have been put together relatively well. Register is a component whose output is changed at the edge of the clock. Since there is some logic ...
Read more