Data partition issue, could anyone help me to explain it's reasons

[Charlie-ZHIJIE]

I try to deploy a customer NN by FINN, And I found a confusing issue.

1. my model is shown below:
   class CustomBrevitasModel(nn.Module):
   def init(self, weight_bit_width):
   super(CustomBrevitasModel, self).init()

    self.dense1_re1 = QuantLinear(num_cal, NUM_FEATS_1, bias=False, weight_bit_width=8)
    self.dense1_im1 = QuantLinear(num_cal, NUM_FEATS_1, bias=False, weight_bit_width=8)
    self.dense1_re2 = QuantLinear(num_cal, NUM_FEATS_1, bias=False, weight_bit_width=8)
    self.dense1_im2 = QuantLinear(num_cal, NUM_FEATS_1, bias=False, weight_bit_width=8)
    self.dense2_re1 = QuantLinear(NUM_FEATS_1, NUM_FEATS_2, bias=False, weight_bit_width=8)
    self.dense2_im1 = QuantLinear(NUM_FEATS_1, NUM_FEATS_2, bias=False, weight_bit_width=8)
    self.dense2_re2 = QuantLinear(NUM_FEATS_1, NUM_FEATS_2, bias=False, weight_bit_width=8)
    self.dense2_im2 = QuantLinear(NUM_FEATS_1, NUM_FEATS_2, bias=False, weight_bit_width=8)
    self.QuantReLU1 = QuantReLU(bit_width=16)
    self.QuantReLU2 = QuantReLU(bit_width=16)
    self.model = nn.Sequential(
        #QuantLinear(4, 32, bias=True, weight_bit_width=weight_bit_width),
        QuantReLU(bit_width=8),##key line
        QuantLinear(64, 64, bias=False, weight_bit_width=8),##key line
        nn.Sigmoid()
    )
   

   def forward(self, inputs):
   real, imag = torch.split(inputs, num_cal, dim=1)
   re_re_mult = self.dense1_re1(real)
   re_im_mult = self.dense1_im1(imag)
   im_re_mult = self.dense1_re2(real)
   im_im_mult = self.dense1_im2(imag)
   real = re_re_mult - re_im_mult
   imag = im_re_mult + im_im_mult
   mag = torch.sqrt(real ** 2 + imag ** 2)
   pha = torch.atan(real / (imag+1e-6))
   # Initialize the weights
   bias = nn.Parameter(torch.zeros(32)) # Initialize the bias
   add = mag + bias # Add the bias to the magnitude
   add_relu = self.QuantReLU1(add)
   real = add_relu * torch.cos(pha)
   imag = add_relu * torch.sin(pha)
   Vs BNN-PYNQ #2
   re_re_mult = self.dense2_re1(real)
   re_im_mult = self.dense2_im1(imag)
   im_re_mult = self.dense2_re2(real)
   im_im_mult = self.dense2_im2(imag)
   real = re_re_mult - re_im_mult
   imag = im_re_mult + im_im_mult
   mag = torch.sqrt(real ** 2 + imag ** 2)
   pha = torch.atan(real / (imag+1e-6))
   bias2 = nn.Parameter(torch.zeros(64)) # Initialize the bias
   add = mag + bias2 # Add the bias to the magnitude
   add_relu = self.QuantReLU2(add)
   real = add_relu * torch.cos(pha)
   imag = add_relu * torch.sin(pha)
   output = torch.sqrt(real ** 2 + real ** 2)
   output = self.model(output)

    return output
   

2. issue happening location:
   self.model = nn.Sequential(
   #QuantLinear(4, 32, bias=True, weight_bit_width=weight_bit_width),
   QuantReLU(bit_width=8),##key line
   QuantLinear(64, 64, bias=False, weight_bit_width=8),##key line
   nn.Sigmoid()
   )
   we can see two Key lines, If I add them, I can pass the step_create_dataflow_partition, if I delete them, I cannot pass it and tool told me the len(sdp_node), not 1, I print it, and it was [], empty. I guess it may due to the optimation strategy, I want to know more detail about it, could you let me know more things about it?

Thanks
Zhijie

[auphelia]

Hi @Charlie-ZHIJIE,

To determine what might be the problem I would recommend to look into the intermediate .onnx files that the FINN compiler produces. In the compiler step step_create_dataflow_partition, the compiler groups all HLS layers(*) together and excludes the non-HLS layers. In the following steps a hardware design can be generated from this group of layers, this group is also called Streaming Dataflow Partition (SDP). If there is no layer in your .onnx graph that can be converted into an HLS layer no SDP will be created and len(sdp_node) will be empty.

(*) HLS layer: Layer that corresponds to an HLS function call in finn-hlslib or another hardware implementation