I want export "GFPGANCleanv1-NoCE-C2.pth" to ONNX #181

New Issue

Closed

opened 2026-01-29 21:45:09 +00:00 by claunia · 13 comments

claunia commented 2026-01-29 21:45:09 +00:00

Owner

Copy Link

Originally created by @jjxxmiin on GitHub (Mar 23, 2022).

I want to use dynamic weights in F.conv2d in pytorch.

    def forward(self, x, style):
        b, c, h, w = x.shape  # c = c_in

        style = self.modulation(style).view(b, 1, c, 1, 1)
        weight = self.weight * style  # (b, c_out, c_in, k, k)
        weight = weight.view(b * self.out_channels, c, self.kernel_size, self.kernel_size)

        b, c, h, w = x.shape
        x = x.view(1, b * c, h, w)

        out = F.conv2d(x, weight, padding=self.padding, groups=b)
        out = out.view(b, self.out_channels, *out.shape[2:4])

        return out

However, only constant weights are supported in Barracuda.

How can I change F.conv2d?

Originally created by @jjxxmiin on GitHub (Mar 23, 2022). I want to use dynamic weights in F.conv2d in pytorch. ``` def forward(self, x, style): b, c, h, w = x.shape # c = c_in style = self.modulation(style).view(b, 1, c, 1, 1) weight = self.weight * style # (b, c_out, c_in, k, k) weight = weight.view(b * self.out_channels, c, self.kernel_size, self.kernel_size) b, c, h, w = x.shape x = x.view(1, b * c, h, w) out = F.conv2d(x, weight, padding=self.padding, groups=b) out = out.view(b, self.out_channels, *out.shape[2:4]) return out ``` However, only constant weights are supported in Barracuda. How can I change F.conv2d?

claunia closed this issue 2026-01-29 21:45:10 +00:00

claunia commented 2026-01-29 21:45:12 +00:00

Author

Owner

Copy Link

@kelisiya commented on GitHub (Mar 24, 2022):

+1

@kelisiya commented on GitHub (Mar 24, 2022): +1

claunia commented 2026-01-29 21:45:13 +00:00

Author

Owner

Copy Link

@hhuiwang commented on GitHub (May 6, 2022):

+1

@hhuiwang commented on GitHub (May 6, 2022): +1

claunia commented 2026-01-29 21:45:15 +00:00

Author

Owner

Copy Link

@wl2136 commented on GitHub (Aug 5, 2022):

+1

@wl2136 commented on GitHub (Aug 5, 2022): +1

claunia commented 2026-01-29 21:45:16 +00:00

Author

Owner

Copy Link

@Zwei-Rakete commented on GitHub (Aug 9, 2022):

same question

@Zwei-Rakete commented on GitHub (Aug 9, 2022): same question

claunia commented 2026-01-29 21:45:18 +00:00

Author

Owner

Copy Link

@baher3d commented on GitHub (Aug 15, 2022):

+1

@baher3d commented on GitHub (Aug 15, 2022): +1

claunia commented 2026-01-29 21:45:19 +00:00

Author

Owner

Copy Link

@magicse commented on GitHub (Aug 15, 2022):

+1

@magicse commented on GitHub (Aug 15, 2022): +1

claunia commented 2026-01-29 21:45:20 +00:00

Author

Owner

Copy Link

@magicse commented on GitHub (Sep 20, 2022):

Hi @jjeamin , here some research to convert GFPGANv1.3-to-ncnn

@magicse commented on GitHub (Sep 20, 2022): Hi @jjeamin , here some research to convert [GFPGANv1.3-to-ncnn](https://github.com/magicse/GFPGANv1.3-to-ncnn)

claunia commented 2026-01-29 21:45:22 +00:00

Author

Owner

Copy Link

@Zwei-Rakete commented on GitHub (Sep 20, 2022):

This should be work, I covert operations on weight to input(x), then GFPGAN model will be constant.
这种方法实测有效，我将对weight的操作等效修改为对x的操作，这样卷积的权重就是固定参数，模型就会从动态转换为静态。

class ModulatedConv2d(nn.Module):
    def __init__(self,
                 in_channels,
                 out_channels,
                 kernel_size,
                 num_style_feat,
                 demodulate=True,
                 sample_mode=None,
                 eps=1e-8):
        super(ModulatedConv2d, self).__init__()
        self.in_channels = in_channels
        self.out_channels = out_channels
        self.kernel_size = kernel_size`
        self.demodula`te = demodulate
        self.sample_mode = sample_mode
        self.eps = eps

        # modulation inside each modulated conv
        self.modulation = nn.Linear(num_style_feat, in_channels, bias=True)
        # initialization
        default_init_weights(self.modulation, scale=1, bias_fill=1, a=0, mode='fan_in', nonlinearity='linear')

        self.weight = nn.Parameter(
            torch.randn(1, out_channels, in_channels, kernel_size, kernel_size) /
            math.sqrt(in_channels * kernel_size**2))
        self.padding = kernel_size // 2
        self.conv2d = nn.Conv2d(in_channels, out_channels, kernel_size, stride=1, padding=self.padding, bias=False)
        self.conv2d.weight.data = self.weight.view(1 * self.out_channels, self.in_channels,self.kernel_size,self.kernel_size)

    def forward(self, x, style):
        b, c, h, w = x.shape  # c = c_in
        # weight modulation
        style = self.modulation(style).view(b, c, 1, 1)
        x = x * style
        if self.demodulate:
            if self.sample_mode == 'upsample':
                x = F.interpolate(x, scale_factor=2, mode='bilinear', align_corners=False)
            elif self.sample_mode == 'downsample':
                x = F.interpolate(x, scale_factor=0.5, mode='bilinear', align_corners=False)
            x = self.conv2d(x)
            weight = self.weight * style
            demod = torch.rsqrt(weight.pow(2).sum([2, 3, 4]) + self.eps)
            out = x * demod.view(b, self.out_channels, 1, 1)
        else:
            if self.sample_mode == 'upsample':
                x = F.interpolate(x, scale_factor=2, mode='bilinear', align_corners=False)
            elif self.sample_mode == 'downsample':
                x = F.interpolate(x, scale_factor=0.5, mode='bilinear', align_corners=False)
            x = x.view(1, b * c, h, w)
            out = self.conv2d(x)

        out = out.view(b, self.out_channels, *out.shape[2:4])

        return out

@Zwei-Rakete commented on GitHub (Sep 20, 2022): This should be work, I covert operations on weight to input(x), then GFPGAN model will be constant. 这种方法实测有效，我将对weight的操作等效修改为对x的操作，这样卷积的权重就是固定参数，模型就会从动态转换为静态。 ``` class ModulatedConv2d(nn.Module): def __init__(self, in_channels, out_channels, kernel_size, num_style_feat, demodulate=True, sample_mode=None, eps=1e-8): super(ModulatedConv2d, self).__init__() self.in_channels = in_channels self.out_channels = out_channels self.kernel_size = kernel_size` self.demodula`te = demodulate self.sample_mode = sample_mode self.eps = eps # modulation inside each modulated conv self.modulation = nn.Linear(num_style_feat, in_channels, bias=True) # initialization default_init_weights(self.modulation, scale=1, bias_fill=1, a=0, mode='fan_in', nonlinearity='linear') self.weight = nn.Parameter( torch.randn(1, out_channels, in_channels, kernel_size, kernel_size) / math.sqrt(in_channels * kernel_size**2)) self.padding = kernel_size // 2 self.conv2d = nn.Conv2d(in_channels, out_channels, kernel_size, stride=1, padding=self.padding, bias=False) self.conv2d.weight.data = self.weight.view(1 * self.out_channels, self.in_channels,self.kernel_size,self.kernel_size) def forward(self, x, style): b, c, h, w = x.shape # c = c_in # weight modulation style = self.modulation(style).view(b, c, 1, 1) x = x * style if self.demodulate: if self.sample_mode == 'upsample': x = F.interpolate(x, scale_factor=2, mode='bilinear', align_corners=False) elif self.sample_mode == 'downsample': x = F.interpolate(x, scale_factor=0.5, mode='bilinear', align_corners=False) x = self.conv2d(x) weight = self.weight * style demod = torch.rsqrt(weight.pow(2).sum([2, 3, 4]) + self.eps) out = x * demod.view(b, self.out_channels, 1, 1) else: if self.sample_mode == 'upsample': x = F.interpolate(x, scale_factor=2, mode='bilinear', align_corners=False) elif self.sample_mode == 'downsample': x = F.interpolate(x, scale_factor=0.5, mode='bilinear', align_corners=False) x = x.view(1, b * c, h, w) out = self.conv2d(x) out = out.view(b, self.out_channels, *out.shape[2:4]) return out ```

claunia commented 2026-01-29 21:45:23 +00:00

Author

Owner

Copy Link

@magicse commented on GitHub (Sep 20, 2022):

some test converting for stylegan2

@magicse commented on GitHub (Sep 20, 2022): some test converting for stylegan2 

claunia commented 2026-01-29 21:45:24 +00:00

Author

Owner

Copy Link

@magicse commented on GitHub (Sep 21, 2022):

Here project with GFPGANv1.3.onnx gfpgan
Link to model GFPGANv1.3.onnx.prototxt

@magicse commented on GitHub (Sep 21, 2022): Here project with GFPGANv1.3.onnx [gfpgan](https://github.com/axinc-ai/ailia-models/tree/master/generative_adversarial_networks/gfpgan) Link to model [GFPGANv1.3.onnx.prototxt](https://netron.app/?url=https://storage.googleapis.com/ailia-models/gfpgan/GFPGANv1.3.onnx.prototxt)

claunia commented 2026-01-29 21:45:25 +00:00

Author

Owner

Copy Link

@jjxxmiin commented on GitHub (Sep 22, 2022):

WOW
Many Thanks!!!!

@jjxxmiin commented on GitHub (Sep 22, 2022): WOW Many Thanks!!!!

claunia commented 2026-01-29 21:45:26 +00:00

Author

Owner

Copy Link

@zimenglan-sysu-512 commented on GitHub (Jul 26, 2023):

This should be work, I covert operations on weight to input(x), then GFPGAN model will be constant. 这种方法实测有效，我将对weight的操作等效修改为对x的操作，这样卷积的权重就是固定参数，模型就会从动态转换为静态。

class ModulatedConv2d(nn.Module):
    def __init__(self,
                 in_channels,
                 out_channels,
                 kernel_size,
                 num_style_feat,
                 demodulate=True,
                 sample_mode=None,
                 eps=1e-8):
        super(ModulatedConv2d, self).__init__()
        self.in_channels = in_channels
        self.out_channels = out_channels
        self.kernel_size = kernel_size`
        self.demodula`te = demodulate
        self.sample_mode = sample_mode
        self.eps = eps

        # modulation inside each modulated conv
        self.modulation = nn.Linear(num_style_feat, in_channels, bias=True)
        # initialization
        default_init_weights(self.modulation, scale=1, bias_fill=1, a=0, mode='fan_in', nonlinearity='linear')

        self.weight = nn.Parameter(
            torch.randn(1, out_channels, in_channels, kernel_size, kernel_size) /
            math.sqrt(in_channels * kernel_size**2))
        self.padding = kernel_size // 2
        self.conv2d = nn.Conv2d(in_channels, out_channels, kernel_size, stride=1, padding=self.padding, bias=False)
        self.conv2d.weight.data = self.weight.view(1 * self.out_channels, self.in_channels,self.kernel_size,self.kernel_size)

    def forward(self, x, style):
        b, c, h, w = x.shape  # c = c_in
        # weight modulation
        style = self.modulation(style).view(b, c, 1, 1)
        x = x * style
        if self.demodulate:
            if self.sample_mode == 'upsample':
                x = F.interpolate(x, scale_factor=2, mode='bilinear', align_corners=False)
            elif self.sample_mode == 'downsample':
                x = F.interpolate(x, scale_factor=0.5, mode='bilinear', align_corners=False)
            x = self.conv2d(x)
            weight = self.weight * style
            demod = torch.rsqrt(weight.pow(2).sum([2, 3, 4]) + self.eps)
            out = x * demod.view(b, self.out_channels, 1, 1)
        else:
            if self.sample_mode == 'upsample':
                x = F.interpolate(x, scale_factor=2, mode='bilinear', align_corners=False)
            elif self.sample_mode == 'downsample':
                x = F.interpolate(x, scale_factor=0.5, mode='bilinear', align_corners=False)
            x = x.view(1, b * c, h, w)
            out = self.conv2d(x)

        out = out.view(b, self.out_channels, *out.shape[2:4])

        return out

did it work for converting to onnx and the sesults are equal to the torcch model?

@zimenglan-sysu-512 commented on GitHub (Jul 26, 2023): > This should be work, I covert operations on weight to input(x), then GFPGAN model will be constant. 这种方法实测有效，我将对weight的操作等效修改为对x的操作，这样卷积的权重就是固定参数，模型就会从动态转换为静态。 > > ``` > class ModulatedConv2d(nn.Module): > def __init__(self, > in_channels, > out_channels, > kernel_size, > num_style_feat, > demodulate=True, > sample_mode=None, > eps=1e-8): > super(ModulatedConv2d, self).__init__() > self.in_channels = in_channels > self.out_channels = out_channels > self.kernel_size = kernel_size` > self.demodula`te = demodulate > self.sample_mode = sample_mode > self.eps = eps > > # modulation inside each modulated conv > self.modulation = nn.Linear(num_style_feat, in_channels, bias=True) > # initialization > default_init_weights(self.modulation, scale=1, bias_fill=1, a=0, mode='fan_in', nonlinearity='linear') > > self.weight = nn.Parameter( > torch.randn(1, out_channels, in_channels, kernel_size, kernel_size) / > math.sqrt(in_channels * kernel_size**2)) > self.padding = kernel_size // 2 > self.conv2d = nn.Conv2d(in_channels, out_channels, kernel_size, stride=1, padding=self.padding, bias=False) > self.conv2d.weight.data = self.weight.view(1 * self.out_channels, self.in_channels,self.kernel_size,self.kernel_size) > > def forward(self, x, style): > b, c, h, w = x.shape # c = c_in > # weight modulation > style = self.modulation(style).view(b, c, 1, 1) > x = x * style > if self.demodulate: > if self.sample_mode == 'upsample': > x = F.interpolate(x, scale_factor=2, mode='bilinear', align_corners=False) > elif self.sample_mode == 'downsample': > x = F.interpolate(x, scale_factor=0.5, mode='bilinear', align_corners=False) > x = self.conv2d(x) > weight = self.weight * style > demod = torch.rsqrt(weight.pow(2).sum([2, 3, 4]) + self.eps) > out = x * demod.view(b, self.out_channels, 1, 1) > else: > if self.sample_mode == 'upsample': > x = F.interpolate(x, scale_factor=2, mode='bilinear', align_corners=False) > elif self.sample_mode == 'downsample': > x = F.interpolate(x, scale_factor=0.5, mode='bilinear', align_corners=False) > x = x.view(1, b * c, h, w) > out = self.conv2d(x) > > out = out.view(b, self.out_channels, *out.shape[2:4]) > > return out > ``` did it work for converting to onnx and the sesults are equal to the torcch model?

claunia commented 2026-01-29 21:45:26 +00:00

Author

Owner

Copy Link

@shartoo commented on GitHub (Dec 20, 2024):

This should be work, I covert operations on weight to input(x), then GFPGAN model will be constant. 这种方法实测有效，我将对weight的操作等效修改为对x的操作，这样卷积的权重就是固定参数，模型就会从动态转换为静态。

class ModulatedConv2d(nn.Module):
    def __init__(self,
                 in_channels,
                 out_channels,
                 kernel_size,
                 num_style_feat,
                 demodulate=True,
                 sample_mode=None,
                 eps=1e-8):
        super(ModulatedConv2d, self).__init__()
        self.in_channels = in_channels
        self.out_channels = out_channels
        self.kernel_size = kernel_size`
        self.demodula`te = demodulate
        self.sample_mode = sample_mode
        self.eps = eps

        # modulation inside each modulated conv
        self.modulation = nn.Linear(num_style_feat, in_channels, bias=True)
        # initialization
        default_init_weights(self.modulation, scale=1, bias_fill=1, a=0, mode='fan_in', nonlinearity='linear')

        self.weight = nn.Parameter(
            torch.randn(1, out_channels, in_channels, kernel_size, kernel_size) /
            math.sqrt(in_channels * kernel_size**2))
        self.padding = kernel_size // 2
        self.conv2d = nn.Conv2d(in_channels, out_channels, kernel_size, stride=1, padding=self.padding, bias=False)
        self.conv2d.weight.data = self.weight.view(1 * self.out_channels, self.in_channels,self.kernel_size,self.kernel_size)

    def forward(self, x, style):
        b, c, h, w = x.shape  # c = c_in
        # weight modulation
        style = self.modulation(style).view(b, c, 1, 1)
        x = x * style
        if self.demodulate:
            if self.sample_mode == 'upsample':
                x = F.interpolate(x, scale_factor=2, mode='bilinear', align_corners=False)
            elif self.sample_mode == 'downsample':
                x = F.interpolate(x, scale_factor=0.5, mode='bilinear', align_corners=False)
            x = self.conv2d(x)
            weight = self.weight * style
            demod = torch.rsqrt(weight.pow(2).sum([2, 3, 4]) + self.eps)
            out = x * demod.view(b, self.out_channels, 1, 1)
        else:
            if self.sample_mode == 'upsample':
                x = F.interpolate(x, scale_factor=2, mode='bilinear', align_corners=False)
            elif self.sample_mode == 'downsample':
                x = F.interpolate(x, scale_factor=0.5, mode='bilinear', align_corners=False)
            x = x.view(1, b * c, h, w)
            out = self.conv2d(x)

        out = out.view(b, self.out_channels, *out.shape[2:4])

        return out

主要改动和原因：

1. 调制方式改变：

• 原始版本：将style调制应用到卷积权重上
• 修改版本：将style调制应用到输入特征图上

2. 卷积操作改变：

   • 原始版本：使用动态生成的权重进行分组卷积
   • 修改版本：使用标准的静态卷积层(self.conv2d)

3. 去调制(demodulation)处理：

• 原始版本：在权重上进行去调制
• 修改版本：在卷积输出上进行去调制

这些修改可以让Generator变成静态的原因：

1. 数学等价性：

• 这���种操作在数学上是等价的，但后者避免了动态权重

2. 静态权重：

• 修改后的版本使用固定的conv2d层，权重不再随style动态变化
• 所有的动态操作都转移到了特征图上

3 . ONNX兼容性：

• ONNX更容易处理输入特征的动态操作
• 静态卷积权重更容易被ONNX和其他推理框架优化

这种修改保持了模型的功能，同时提高了模型的部署友好性，使其可以被导出为ONNX格式。

ONNX 的计算图特性：

1. ONNX 需要构建静态的计算图
2. 每个操作节点的输入输出形状需要是确定的
3. 权重通常被视为常量参数

F.conv2d 的特殊性：

1. PyTorch 的 F.conv2d 期望权重是固定的模型参数
2. 当权重是动态计算的中间结果时，ONNX 无法正确追踪和导出这种操作
3. 特别是当权重依赖于输入 style 时，这种动态性质更难处理

分组卷积的复杂性：

# 这种操作在ONNX中特别难处理
input = input.view(1, batch * in_channel, height, width)
out = F.conv2d(input, weight, padding=self.padding, groups=batch)

@shartoo commented on GitHub (Dec 20, 2024): > This should be work, I covert operations on weight to input(x), then GFPGAN model will be constant. 这种方法实测有效，我将对weight的操作等效修改为对x的操作，这样卷积的权重就是固定参数，模型就会从动态转换为静态。 > > ``` > class ModulatedConv2d(nn.Module): > def __init__(self, > in_channels, > out_channels, > kernel_size, > num_style_feat, > demodulate=True, > sample_mode=None, > eps=1e-8): > super(ModulatedConv2d, self).__init__() > self.in_channels = in_channels > self.out_channels = out_channels > self.kernel_size = kernel_size` > self.demodula`te = demodulate > self.sample_mode = sample_mode > self.eps = eps > > # modulation inside each modulated conv > self.modulation = nn.Linear(num_style_feat, in_channels, bias=True) > # initialization > default_init_weights(self.modulation, scale=1, bias_fill=1, a=0, mode='fan_in', nonlinearity='linear') > > self.weight = nn.Parameter( > torch.randn(1, out_channels, in_channels, kernel_size, kernel_size) / > math.sqrt(in_channels * kernel_size**2)) > self.padding = kernel_size // 2 > self.conv2d = nn.Conv2d(in_channels, out_channels, kernel_size, stride=1, padding=self.padding, bias=False) > self.conv2d.weight.data = self.weight.view(1 * self.out_channels, self.in_channels,self.kernel_size,self.kernel_size) > > def forward(self, x, style): > b, c, h, w = x.shape # c = c_in > # weight modulation > style = self.modulation(style).view(b, c, 1, 1) > x = x * style > if self.demodulate: > if self.sample_mode == 'upsample': > x = F.interpolate(x, scale_factor=2, mode='bilinear', align_corners=False) > elif self.sample_mode == 'downsample': > x = F.interpolate(x, scale_factor=0.5, mode='bilinear', align_corners=False) > x = self.conv2d(x) > weight = self.weight * style > demod = torch.rsqrt(weight.pow(2).sum([2, 3, 4]) + self.eps) > out = x * demod.view(b, self.out_channels, 1, 1) > else: > if self.sample_mode == 'upsample': > x = F.interpolate(x, scale_factor=2, mode='bilinear', align_corners=False) > elif self.sample_mode == 'downsample': > x = F.interpolate(x, scale_factor=0.5, mode='bilinear', align_corners=False) > x = x.view(1, b * c, h, w) > out = self.conv2d(x) > > out = out.view(b, self.out_channels, *out.shape[2:4]) > > return out > ``` 主要改动和原因： 1. 调制方式改变： + 原始版本：将style调制应用到卷积权重上 + 修改版本：将style调制应用到输入特征图上 2. 卷积操作改变： + 原始版本：使用动态生成的权重进行分组卷积 + 修改版本：使用标准的静态卷积层(self.conv2d) 3. 去调制(demodulation)处理： + 原始版本：在权重上进行去调制 + 修改版本：在卷积输出上进行去调制 这些修改可以让Generator变成静态的原因： 1. 数学等价性： + 这���种操作在数学上是等价的，但后者避免了动态权重 2. 静态权重： + 修改后的版本使用固定的conv2d层，权重不再随style动态变化 + 所有的动态操作都转移到了特征图上 3 . ONNX兼容性： + ONNX更容易处理输入特征的动态操作 + 静态卷积权重更容易被ONNX和其他推理框架优化 这种修改保持了模型的功能，同时提高了模型的部署友好性，使其可以被导出为ONNX格式。 ONNX 的计算图特性： 1. ONNX 需要构建静态的计算图 2. 每个操作节点的输入输出形状需要是确定的 3. 权重通常被视为常量参数 F.conv2d 的特殊性： 1. PyTorch 的 F.conv2d 期望权重是固定的模型参数 2. 当权重是动态计算的中间结果时，ONNX 无法正确追踪和导出这种操作 3. 特别是当权重依赖于输入 style 时，这种动态性质更难处理 分组卷积的复杂性： ``` # 这种操作在ONNX中特别难处理 input = input.view(1, batch * in_channel, height, width) out = F.conv2d(input, weight, padding=self.padding, groups=batch) ```

Sign in to join this conversation.

No Branch/Tag Specified

Branches Tags

master

v1.3.8

v1.3.7

v1.3.6

v1.3.5

v1.3.4

v1.3.3

v1.3.2

v1.3.1

v1.3.0

v0.2.4

v0.2.3

v0.2.2

v.0.2.1

v0.2.0

v0.1.0

Labels

Clear labels

pull-request

Mirrored from GitHub Pull Request

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

claunia

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: TencentARC/GFPGAN#181