upload_hf_model.ipynb

Kernel: Python 3

!pip install transformers
!sudo apt-get install git-lfs

Requirement already satisfied: transformers in /usr/local/lib/python3.6/dist-packages (4.1.1)
Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from transformers) (1.19.4)
Requirement already satisfied: requests in /usr/local/lib/python3.6/dist-packages (from transformers) (2.23.0)
Requirement already satisfied: tokenizers==0.9.4 in /usr/local/lib/python3.6/dist-packages (from transformers) (0.9.4)
Requirement already satisfied: filelock in /usr/local/lib/python3.6/dist-packages (from transformers) (3.0.12)
Requirement already satisfied: dataclasses; python_version < "3.7" in /usr/local/lib/python3.6/dist-packages (from transformers) (0.8)
Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.6/dist-packages (from transformers) (2019.12.20)
Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.6/dist-packages (from transformers) (4.41.1)
Requirement already satisfied: sacremoses in /usr/local/lib/python3.6/dist-packages (from transformers) (0.0.43)
Requirement already satisfied: packaging in /usr/local/lib/python3.6/dist-packages (from transformers) (20.8)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests->transformers) (2020.12.5)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests->transformers) (1.24.3)
Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests->transformers) (2.10)
Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests->transformers) (3.0.4)
Requirement already satisfied: six in /usr/local/lib/python3.6/dist-packages (from sacremoses->transformers) (1.15.0)
Requirement already satisfied: joblib in /usr/local/lib/python3.6/dist-packages (from sacremoses->transformers) (1.0.0)
Requirement already satisfied: click in /usr/local/lib/python3.6/dist-packages (from sacremoses->transformers) (7.1.2)
Requirement already satisfied: pyparsing>=2.0.2 in /usr/local/lib/python3.6/dist-packages (from packaging->transformers) (2.4.7)
Reading package lists... Done
Building dependency tree       
Reading state information... Done
git-lfs is already the newest version (2.3.4-1).
0 upgraded, 0 newly installed, 0 to remove and 15 not upgraded.

!git config --global user.email "[email protected]"
!git config --global user.name "Julien Chaumond"

!transformers-cli login

!pwd

/content

!transformers-cli repo create policy-distilbert-7d

2020-12-26 10:02:28.904892: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.10.1
git version 2.17.1
Error: unknown flag: --version

Sorry, no usage text found for "git-lfs"

You are about to create julien-c/policy-distilbert-7d
Proceed? [Y/n] Y

Your repo now lives at:
  https://huggingface.co/julien-c/policy-distilbert-7d

You can clone it locally with the command below, and commit/push as usual.

  git clone https://huggingface.co/julien-c/policy-distilbert-7d

!git clone https://julien-c:[e-mail]/julien-c/policy-distilbert-7d

Cloning into 'policy-distilbert-7d'...
remote: Enumerating objects: 3, done.
remote: Counting objects: 100% (3/3), done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.

!ls -al

total 20
drwxr-xr-x 1 root root 4096 Dec 26 10:03 .
drwxr-xr-x 1 root root 4096 Dec 26 09:42 ..
drwxr-xr-x 1 root root 4096 Dec 21 17:29 .config
drwxr-xr-x 3 root root 4096 Dec 26 10:03 policy-distilbert-7d
drwxr-xr-x 1 root root 4096 Dec 21 17:29 sample_data

%cd policy-distilbert-7d

/content/policy-distilbert-7d

!wget https://huggingface.co/distilbert-base-uncased/resolve/main/config.json
!wget https://huggingface.co/distilbert-base-uncased/resolve/main/pytorch_model.bin

--2020-12-26 10:03:55--  https://huggingface.co/distilbert-base-uncased/resolve/main/config.json
Resolving huggingface.co (huggingface.co)... 34.201.172.85
Connecting to huggingface.co (huggingface.co)|34.201.172.85|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 442 [application/json]
Saving to: ‘config.json’

config.json         100%[===================>]     442  --.-KB/s    in 0s      

2020-12-26 10:03:55 (56.7 MB/s) - ‘config.json’ saved [442/442]

--2020-12-26 10:03:55--  https://huggingface.co/distilbert-base-uncased/resolve/main/pytorch_model.bin
Resolving huggingface.co (huggingface.co)... 34.201.172.85
Connecting to huggingface.co (huggingface.co)|34.201.172.85|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://cdn-lfs.huggingface.co/distilbert-base-uncased/e60d71610916da4787c5513c81bc026d415708528295502fb3e1a6fe1485ea7c [following]
--2020-12-26 10:03:55--  https://cdn-lfs.huggingface.co/distilbert-base-uncased/e60d71610916da4787c5513c81bc026d415708528295502fb3e1a6fe1485ea7c
Resolving cdn-lfs.huggingface.co (cdn-lfs.huggingface.co)... 13.226.49.106, 13.226.49.46, 13.226.49.103, ...
Connecting to cdn-lfs.huggingface.co (cdn-lfs.huggingface.co)|13.226.49.106|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 267967963 (256M) [application/macbinary]
Saving to: ‘pytorch_model.bin’

pytorch_model.bin   100%[===================>] 255.55M  87.1MB/s    in 2.9s    

2020-12-26 10:03:58 (87.1 MB/s) - ‘pytorch_model.bin’ saved [267967963/267967963]

!git lfs install

Updated git hooks.
Git LFS initialized.

!git add .

!git commit -m "from Google Colab"

[main 41a7c98] from Google Colab
 2 files changed, 24 insertions(+)
 create mode 100644 config.json
 create mode 100644 pytorch_model.bin

!git log

commit 41a7c98f1285a7e5ef19095dab11f0ac71ac1406 (HEAD -> main)
Author: Julien Chaumond <[email protected]>
Date:   Sat Dec 26 10:04:20 2020 +0000

    from Google Colab

commit 4630180ab4844ecc97c7f187c3d5bc54a568fac0 (origin/main, origin/HEAD)
Author: system <[email protected]>
Date:   Sat Dec 26 10:02:45 2020 +0000

    initial commit

!git push

Git LFS: (1 of 1 files) 255.55 MB / 255.55 MB
Counting objects: 4, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (4/4), done.
Writing objects: 100% (4/4), 712 bytes | 712.00 KiB/s, done.
Total 4 (delta 0), reused 0 (delta 0)
To https://huggingface.co/julien-c/policy-distilbert-7d
   4630180..41a7c98  main -> main

Check out resulting commit: https://huggingface.co/julien-c/policy-distilbert-7d/commit/41a7c98f1285a7e5ef19095dab11f0ac71ac1406