readme.ja.json

{
 " Patches Are All You Need?": " \u5fc5\u8981\u306a\u306e\u306f\u30d1\u30c3\u30c1\u3060\u3051\uff1f",
 "<h1><a href=\"https://nn.labml.ai/conv_mixer/index.html\">Patches Are All You Need?</a></h1>\n<p>This is a <a href=\"https://pytorch.org\">PyTorch</a> implementation of the paper <a href=\"https://arxiv.org/abs/2201.09792\">Patches Are All You Need?</a>.</p>\n<p>ConvMixer is Similar to <a href=\"https://nn.labml.ai/transformers/mlp_mixer/index.html\">MLP-Mixer</a>. MLP-Mixer separates mixing of spatial and channel dimensions, by applying an MLP across spatial dimension and then an MLP across the channel dimension (spatial MLP replaces the <a href=\"https://nn.labml.ai/transformers/vit/index.html\">ViT</a> attention and channel MLP is the <a href=\"https://nn.labml.ai/transformers/feed_forward.html\">FFN</a> of ViT).</p>\n<p>ConvMixer uses a 1x1 convolution for channel mixing and a depth-wise convolution for spatial mixing. Since it&#x27;s a convolution instead of a full MLP across the space, it mixes only the nearby batches in contrast to ViT or MLP-Mixer. Also, the MLP-mixer uses MLPs of two layers for each mixing and ConvMixer uses a single layer for each mixing.</p>\n<p>The paper recommends removing the residual connection across the channel mixing (point-wise convolution) and having only a residual connection over the spatial mixing (depth-wise convolution). They also use <a href=\"https://nn.labml.ai/normalization/batch_norm/index.html\">Batch normalization</a> instead of <a href=\"../normalization/layer_norm/index.html\">Layer normalization</a>.</p>\n<p>Here&#x27;s <a href=\"https://nn.labml.ai/conv_mixer/experiment.html\">an experiment</a> that trains ConvMixer on CIFAR-10. </p>\n": "<h1><a href=\"https://nn.labml.ai/conv_mixer/index.html\">\u5fc5\u8981\u306a\u306e\u306f\u30d1\u30c3\u30c1\u3060\u3051\uff1f</a></h1>\n<p><a href=\"https://pytorch.org\">\u3053\u308c\u306f\u7d19\u306e\u30d1\u30c3\u30c1\u3092PyTorch\u3067\u5b9f\u88c5\u3057\u305f\u3082\u306e\u3067\u3059</a><a href=\"https://arxiv.org/abs/2201.09792\">\u3002\u5fc5\u8981\u306a\u306e\u306f\u30d1\u30c3\u30c1\u3060\u3051\u3067\u3059\u304b</a>\uff1f</p>\u3002\n<p><a href=\"https://nn.labml.ai/transformers/mlp_mixer/index.html\">ConvMixer\u306fMLP\u30df\u30ad\u30b5\u30fc\u306b\u4f3c\u3066\u3044\u307e\u3059\u3002</a></p><a href=\"https://nn.labml.ai/transformers/feed_forward.html\">MLP-Mixer\u306f\u3001\u7a7a\u9593\u6b21\u5143\u5168\u4f53\u306bMLP\u3092\u9069\u7528\u3057\u3001\u6b21\u306b\u30c1\u30e3\u30cd\u30eb\u6b21\u5143\u5168\u4f53\u306bMLP\u3092\u9069\u7528\u3059\u308b\u3053\u3068\u3067\u3001\u7a7a\u9593\u6b21\u5143\u3068\u30c1\u30e3\u30cd\u30eb\u6b21\u5143\u306e\u6df7\u5408\u3092\u5206\u96e2\u3057\u307e\u3059\uff08\u7a7a\u9593MLP\u306fvIT\u306e\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u306b\u4ee3\u308f\u308a\u3001<a href=\"https://nn.labml.ai/transformers/vit/index.html\">\u30c1\u30e3\u30cd\u30ebMLP\u306fVIT\u306eFFN\u3067\u3059</a>\uff09\u3002</a>\n<p>ConvMixer\u306f\u3001\u30c1\u30e3\u30f3\u30cd\u30eb\u30df\u30ad\u30b7\u30f3\u30b0\u306b1x1\u306e\u30b3\u30f3\u30dc\u30ea\u30e5\u30fc\u30b7\u30e7\u30f3\u3092\u4f7f\u7528\u3057\u3001\u7a7a\u9593\u30df\u30ad\u30b7\u30f3\u30b0\u306b\u5965\u884c\u304d\u30b3\u30f3\u30dc\u30ea\u30e5\u30fc\u30b7\u30e7\u30f3\u3092\u4f7f\u7528\u3057\u307e\u3059\u3002\u30b9\u30da\u30fc\u30b9\u5168\u4f53\u3067\u30d5\u30ebMLP\u3067\u306f\u306a\u304f\u7573\u307f\u8fbc\u307f\u306a\u306e\u3067\u3001VIT\u3084MLP\u30df\u30ad\u30b5\u30fc\u3068\u306f\u5bfe\u7167\u7684\u306b\u3001\u8fd1\u304f\u306e\u30d0\u30c3\u30c1\u306e\u307f\u3092\u30df\u30ad\u30b7\u30f3\u30b0\u3057\u307e\u3059\u3002\u307e\u305f\u3001MLP\u30df\u30ad\u30b5\u30fc\u306f\u30df\u30ad\u30b7\u30f3\u30b0\u3054\u3068\u306b2\u5c64\u306eMLP\u3092\u4f7f\u7528\u3057\u3001ConvMixer\u306f\u30df\u30ad\u30b7\u30f3\u30b0\u3054\u3068\u306b1\u5c64\u306eMLP\u3092\u4f7f\u7528\u3057\u307e\u3059</p>\u3002\n<p>\u3053\u306e\u8ad6\u6587\u3067\u306f\u3001\u30c1\u30e3\u30cd\u30eb\u30df\u30ad\u30b7\u30f3\u30b0\u5168\u4f53\u306e\u6b8b\u7559\u63a5\u7d9a\u3092\u524a\u9664\u3057\uff08\u70b9\u5358\u4f4d\u306e\u7573\u307f\u8fbc\u307f\uff09\u3001\u7a7a\u9593\u30df\u30ad\u30b7\u30f3\u30b0\u3067\u306f\u6b8b\u7559\u63a5\u7d9a\u306e\u307f\u306b\u3059\u308b\uff08\u6df1\u3055\u65b9\u5411\u306e\u7573\u307f\u8fbc\u307f\uff09\u3053\u3068\u3092\u63a8\u5968\u3057\u3066\u3044\u307e\u3059\u3002\u307e\u305f\u3001</p><a href=\"https://nn.labml.ai/normalization/batch_norm/index.html\"><a href=\"../normalization/layer_norm/index.html\">\u30ec\u30a4\u30e4\u30fc\u6b63\u898f\u5316\u306e\u4ee3\u308f\u308a\u306b\u30d0\u30c3\u30c1\u6b63\u898f\u5316\u3092\u4f7f\u7528\u3057\u307e\u3059</a></a>\u3002\n<p>\u3053\u308c\u306f<a href=\"https://nn.labml.ai/conv_mixer/experiment.html\">\u3001CIFAR-10 \u3067 ConvMixer \u3092\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u3059\u308b\u5b9f\u9a13\u3067\u3059</a>\u3002</p>\n"
}