readme.ja.json

{
 "<h1><a href=\"https://nn.labml.ai/distillation/index.html\">Distilling the Knowledge in a Neural Network</a></h1>\n<p>This is a <a href=\"https://pytorch.org\">PyTorch</a> implementation/tutorial of the paper <a href=\"https://arxiv.org/abs/1503.02531\">Distilling the Knowledge in a Neural Network</a>.</p>\n<p>It&#x27;s a way of training a small network using the knowledge in a trained larger network; i.e. distilling the knowledge from the large network.</p>\n<p>A large model with regularization or an ensemble of models (using dropout) generalizes better than a small model when trained directly on the data and labels. However, a small model can be trained to generalize better with help of a large model. Smaller models are better in production: faster, less compute, less memory.</p>\n<p>The output probabilities of a trained model give more information than the labels because it assigns non-zero probabilities to incorrect classes as well. These probabilities tell us that a sample has a chance of belonging to certain classes. For instance, when classifying digits, when given an image of digit <em>7</em>, a generalized model will give a high probability to 7 and a small but non-zero probability to 2, while assigning almost zero probability to other digits. Distillation uses this information to train a small model better. </p>\n": "<h1><a href=\"https://nn.labml.ai/distillation/index.html\">\u30cb\u30e5\u30fc\u30e9\u30eb\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u3067\u306e\u77e5\u8b58\u306e\u62bd\u51fa</a></h1>\n<p>\u3053\u308c\u306f\u3001<a href=\"https://arxiv.org/abs/1503.02531\">\u8ad6\u6587\u300c\u30cb\u30e5\u30fc\u30e9\u30eb\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u306b\u304a\u3051\u308b\u77e5\u8b58\u306e\u62bd\u51fa</a>\u300d<a href=\"https://pytorch.org\">\u306ePyTorch\u5b9f\u88c5/\u30c1\u30e5\u30fc\u30c8\u30ea\u30a2\u30eb\u3067\u3059</a>\u3002</p>\n<p>\u3053\u308c\u306f\u3001\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u6e08\u307f\u306e\u5927\u898f\u6a21\u306a\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u306e\u77e5\u8b58\u3092\u4f7f\u7528\u3057\u3066\u5c0f\u898f\u6a21\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u3092\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u3059\u308b\u65b9\u6cd5\u3067\u3059\u3002\u3064\u307e\u308a\u3001\u5927\u898f\u6a21\u306a\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u304b\u3089\u77e5\u8b58\u3092\u62bd\u51fa\u3059\u308b\u65b9\u6cd5\u3067\u3059\u3002</p>\n<p>\u30c7\u30fc\u30bf\u3084\u30e9\u30d9\u30eb\u3067\u76f4\u63a5\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u3057\u305f\u5834\u5408\u3001\u6b63\u5247\u5316\u3092\u884c\u3063\u305f\u5927\u898f\u6a21\u306a\u30e2\u30c7\u30eb\u3084 (\u30c9\u30ed\u30c3\u30d7\u30a2\u30a6\u30c8\u3092\u4f7f\u7528\u3057\u305f) \u30e2\u30c7\u30eb\u306e\u30a2\u30f3\u30b5\u30f3\u30d6\u30eb\u306f\u3001\u5c0f\u3055\u306a\u30e2\u30c7\u30eb\u3088\u308a\u3082\u4e00\u822c\u5316\u304c\u5bb9\u6613\u3067\u3059\u3002\u305f\u3060\u3057\u3001\u5c0f\u3055\u3044\u30e2\u30c7\u30eb\u3067\u3082\u3001\u5927\u304d\u306a\u30e2\u30c7\u30eb\u306e\u52a9\u3051\u3092\u501f\u308a\u3066\u3088\u308a\u4e00\u822c\u5316\u3057\u3084\u3059\u3044\u3088\u3046\u306b\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u3067\u304d\u307e\u3059\u3002\u672c\u756a\u74b0\u5883\u3067\u306f\u3001\u30e2\u30c7\u30eb\u304c\u5c0f\u3055\u3044\u307b\u3069\u901f\u304f\u3001\u51e6\u7406\u80fd\u529b\u304c\u5c11\u306a\u304f\u3001\u30e1\u30e2\u30ea\u3082\u5c11\u306a\u304f\u3066\u6e08\u307f\u307e\u3059\u3002</p>\n<p>\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u6e08\u307f\u30e2\u30c7\u30eb\u306e\u51fa\u529b\u78ba\u7387\u306f\u3001\u8aa4\u3063\u305f\u30af\u30e9\u30b9\u306b\u3082\u30bc\u30ed\u4ee5\u5916\u306e\u78ba\u7387\u3092\u5272\u308a\u5f53\u3066\u308b\u305f\u3081\u3001\u30e9\u30d9\u30eb\u3088\u308a\u3082\u591a\u304f\u306e\u60c5\u5831\u3092\u63d0\u4f9b\u3057\u307e\u3059\u3002\u3053\u308c\u3089\u306e\u78ba\u7387\u304b\u3089\u3001\u30b5\u30f3\u30d7\u30eb\u304c\u7279\u5b9a\u306e\u30af\u30e9\u30b9\u306b\u5c5e\u3057\u3066\u3044\u308b\u53ef\u80fd\u6027\u304c\u3042\u308b\u3053\u3068\u304c\u308f\u304b\u308a\u307e\u3059\u3002\u305f\u3068\u3048\u3070\u3001\u6570\u5b57\u3092\u5206\u985e\u3059\u308b\u969b\u3001<em>7 \u6841\u306e\u753b\u50cf\u304c\u4e0e\u3048\u3089\u308c\u305f\u5834\u5408\u3001\u4e00\u822c\u5316\u30e2\u30c7\u30eb\u3067\u306f 7</em> \u306b\u306f\u9ad8\u3044\u78ba\u7387\u30012 \u306b\u306f\u5c0f\u3055\u3044\u306a\u304c\u3089\u3082\u30bc\u30ed\u3067\u306f\u306a\u3044\u78ba\u7387\u304c\u4e0e\u3048\u3089\u308c\u3001\u4ed6\u306e\u6570\u5b57\u306b\u306f\u307b\u307c\u30bc\u30ed\u306e\u78ba\u7387\u3092\u5272\u308a\u5f53\u3066\u307e\u3059\u3002\u84b8\u7559\u3067\u306f\u3001\u3053\u306e\u60c5\u5831\u3092\u5229\u7528\u3057\u3066\u5c0f\u578b\u30e2\u30c7\u30eb\u306e\u5b66\u7fd2\u52b9\u679c\u3092\u9ad8\u3081\u307e\u3059</p>\u3002\n",
 "Distilling the Knowledge in a Neural Network": "\u30cb\u30e5\u30fc\u30e9\u30eb\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u3067\u306e\u77e5\u8b58\u306e\u62bd\u51fa"
}