似乎没人用4*4卷积了！cifar10训练上82

2026/8/2 7:27:18

平时用惯了两个线性层结尾：

layers.emplace_back(std::make_shared<Linear>(cublas, batch, 128 * 64, 500));
layers.emplace_back(std::make_shared<LeakyRL>(cudnn, batch, 500, 1, 1));
layers.emplace_back(std::make_shared<Linear>(cublas, batch, 500, 10));

想起以前cpu下自己写的全连接方式，可以试一试，代替一次linear层：

layers.emplace_back(std::make_shared<Conv2D>(cudnn, batch, 256,300, 4, 4, 4, 1));

即把256*4*4的数据通过4*4卷积成300*1*1数据

昨天，架构增加了一个残差块，接近82分，但翻不过去！

今天用这个4*4卷积成功突破！闯关成功！

架构如下：

layers.emplace_back(std::make_shared<Conv2D>(cudnn, batch, 5, 64, 32, 32, 3, 1, 1));
layers.emplace_back(std::make_shared<residualExt22>(cudnn, batch, 64, 32, 32));
layers.emplace_back(std::make_shared<MaxPool2D>(cudnn, batch, 64, 32, 32, 2, 2, 0, 2));

layers.emplace_back(std::make_shared<Conv2D>(cudnn, batch, 64, 128, 16, 16, 3, 1, 1));
layers.emplace_back(std::make_shared<residualExt22>(cudnn, batch, 128, 16, 16));
layers.emplace_back(std::make_shared<MaxPool2D>(cudnn, batch, 128, 16, 16, 2, 2, 0, 2));

layers.emplace_back(std::make_shared<Conv2D>(cudnn, batch, 128, 256, 8, 8, 3, 1, 1));
layers.emplace_back(std::make_shared<residualExt22>(cudnn, batch, 256, 8, 8));
layers.emplace_back(std::make_shared<MaxPool2D>(cudnn, batch, 256, 8, 8, 2, 2, 0, 2));

layers.emplace_back(std::make_shared<Conv2D>(cudnn, batch, 256,300, 4, 4, 4, 1));
layers.emplace_back(std::make_shared<BN>(cudnn, batch, 300, 1, 1));
layers.emplace_back(std::make_shared<LeakyRL>(cudnn, batch, 300, 1, 1));
layers.emplace_back(std::make_shared<Linear>(cublas, batch, 300, 10));

得分如下：

rb均值: 0.8797101974,rb方差:5.678951263428
rb均值: -8.3503847122,rb方差:27.435239791870
rb均值: 1.8536375761,rb方差:24.807447433472
rb均值: 4.2101426125,rb方差:16.878156661987
rb均值: -3.8696627617,rb方差:5.029622554779
rb均值: -12.3703460693,rb方差:29.529457092285
rb均值: -2.0291111469,rb方差:8.738946914673
rb均值: -3.2213602066,rb方差:6.971586704254
rb均值: -3.2548468113,rb方差:14.214648246765
时间: 24602.048828 ms
train Classification result: 96.43% ok (used 49984 images)
时间: 1960.859985 ms
Test Classification result:82.07% ok (used 9984 images)
learn rate：0.0001
轮次：16
rb均值: 0.8008456230,rb方差:5.773095607758
rb均值: -8.5729656219,rb方差:27.482904434204
rb均值: 1.7287417650,rb方差:25.392423629761
rb均值: 4.1368126869,rb方差:16.951080322266
rb均值: -3.8994140625,rb方差:5.067879676819
rb均值: -12.6236686707,rb方差:30.174797058105
rb均值: -2.1486938000,rb方差:8.853424072266
rb均值: -3.2606005669,rb方差:7.034667015076
rb均值: -3.3508059978,rb方差:14.302834510803
时间: 24564.140625 ms
train Classification result: 97.57% ok (used 49984 images)
时间: 1951.093018 ms
Test Classification result:81.98% ok (used 9984 images)
learn rate：0.0001
轮次：17
rb均值: 0.7735943794,rb方差:5.724789142609
rb均值: -8.6432456970,rb方差:27.800550460815
rb均值: 1.7252777815,rb方差:25.578634262085
rb均值: 4.1205477715,rb方差:17.033407211304
rb均值: -3.9014611244,rb方差:5.074986934662
rb均值: -12.6995344162,rb方差:30.028085708618
rb均值: -2.1647324562,rb方差:8.935814857483
rb均值: -3.2678849697,rb方差:7.184212207794
rb均值: -3.3836572170,rb方差:14.296825408936
时间: 24613.261719 ms
train Classification result: 98.22% ok (used 49984 images)
时间: 1946.271973 ms
Test Classification result:82.05% ok (used 9984 images)
learn rate：1e-05
轮次：18
rb均值: 0.7709599733,rb方差:5.753355026245
rb均值: -8.6881046295,rb方差:27.816370010376
rb均值: 1.7303854227,rb方差:25.849842071533
rb均值: 4.1074061394,rb方差:17.110939025879
rb均值: -3.9176218510,rb方差:5.036983966827
rb均值: -12.7407503128,rb方差:30.236000061035
rb均值: -2.1752357483,rb方差:8.898382186890
rb均值: -3.2971680164,rb方差:7.075766563416
rb均值: -3.3980994225,rb方差:14.323348999023
时间: 24574.095703 ms
train Classification result: 98.41% ok (used 49984 images)
时间: 1955.838013 ms
Test Classification result:82.37% ok (used 9984 images)
learn rate：1e-05
轮次：19
rb均值: 0.7680509090,rb方差:5.763551712036
rb均值: -8.7100591660,rb方差:28.046590805054
rb均值: 1.7029346228,rb方差:25.735727310181
rb均值: 4.1084098816,rb方差:17.077848434448
rb均值: -3.9339706898,rb方差:5.079279422760
rb均值: -12.7671136856,rb方差:30.269668579102
rb均值: -2.1830737591,rb方差:8.921528816223
rb均值: -3.3042364120,rb方差:7.090327262878
rb均值: -3.4057095051,rb方差:14.365463256836
时间: 24811.109375 ms
train Classification result: 98.65% ok (used 49984 images)
时间: 1964.407959 ms
Test Classification result:82.57% ok (used 9984 images)
learn rate：1e-05
轮次：20
请按任意键继续. . .（看样子，还能上）