Book a Demo!
CoCalc Logo Icon
StoreFeaturesDocsShareSupportNewsAboutPoliciesSign UpSign In
suyashi29
GitHub Repository: suyashi29/python-su
 Path: blob/master/Generative NLP Models using Python/6.1 Transformer Models.ipynb
7216 views
Kernel: Python (torch_fix)
pip install torch


import os
os.environ["HF_HUB_DISABLE_SYMLINKS_WARNING"] = "1"

import os
os.environ["CUDA_VISIBLE_DEVICES"] = "-1"
import torch
C:\Users\Suyashi144893\AppData\Local\anaconda3\envs\torch_fix\lib\site-packages\torch\_subclasses\functional_tensor.py:283: UserWarning: Failed to initialize NumPy: No module named 'numpy' (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\pytorch\torch\csrc\utils\tensor_numpy.cpp:84.) cpu = _conversion_method_template(device=torch.device("cpu")) 
import torch
import numpy as np

print(torch.__version__)
print(np.__version__)
print("CUDA:", torch.cuda.is_available())
2.10.0+cpu 2.2.6 CUDA: False

  • #PyTorch is working

  • #But GPU acceleration is not being used (CUDA: false)

Simplified Python code demonstrating how self-attention might be applied in a neural machine translation scenario using the transformers library

This code uses the BERT model from the transformers library to tokenize the input text, compute its hidden states, and extract the self-attention weights. These weights indicate how much each token attends to every other token in each layer of the model. However, note that BERT is not specifically trained for machine translation, so this is just an illustration of self-attention in a language model context.

Steps

  • Tokenization: The input text "Ashi is beautiful." is tokenized into its constituent tokens using the BERT tokenizer. Each token is represented by an integer ID. Let's denote the tokenized input as 𝑋 X.

  • Model Computation: The tokenized input 𝑋 X is fed into the BERT model, which consists of multiple layers of self-attention and feedforward neural networks. The BERT model processes the input tokens and produces hidden states for each token. Let's denote the hidden states as 𝐻

  • Self-Attention: During each layer of the BERT model, self-attention is applied to the input tokens. The self-attention mechanism computes attention scores between each token and every other token in the sequence. These attention scores are calculated using the formula:

  • Self-Attention Weights: The self-attention weights represent the importance of each token attending to every other token in the sequence. These weights are computed for each layer of the model. In the code, the mean of the attention weights across the sequence dimension is calculated for each layer and printed out.

pip install torch

## ose error:

conda create -n torch_fix python=3.10 -y
conda activate torch_fix
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
pip install ipykernel
python -m ipykernel install --user --name=torch_fix --display-name "Python (torch_fix)"
conda install -c conda-forge sentencepiece

### in Jupyter notebook 
pip intstall transformers 
pip install torch
pip install transformers torch sentencepiece

import torch
from transformers import BertModel, BertTokenizer, BertConfig

# Load pre-trained BERT model and tokenizer
machine_T = 'bert-base-multilingual-cased'
tokenizer = BertTokenizer.from_pretrained(machine_T )
model = BertModel.from_pretrained(machine_T )

# Input text
input_text = input("enter your text : ")

# Tokenize input text
input_ids = tokenizer.encode(input_text, add_special_tokens=True, return_tensors="pt")

# Get BERT model's output
outputs = model(input_ids)
C:\Users\Suyashi144893\AppData\Local\anaconda3\envs\torch_fix\lib\site-packages\tqdm\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html from .autonotebook import tqdm as notebook_tqdm Loading weights: 100%|████████████████████████████████████████████████████████████| 199/199 [00:00<00:00, 10694.00it/s] BertModel LOAD REPORT from: bert-base-multilingual-cased Key | Status | | -------------------------------------------+------------+--+- cls.predictions.transform.dense.bias | UNEXPECTED | | cls.predictions.transform.LayerNorm.weight | UNEXPECTED | | cls.predictions.transform.dense.weight | UNEXPECTED | | cls.predictions.bias | UNEXPECTED | | cls.seq_relationship.weight | UNEXPECTED | | cls.predictions.transform.LayerNorm.bias | UNEXPECTED | | cls.seq_relationship.bias | UNEXPECTED | | Notes: - UNEXPECTED :can be ignored when loading from different task/architecture; not ok if you expect identical arch.
enter your text : Hello 
outputs
BaseModelOutputWithPoolingAndCrossAttentions(last_hidden_state=tensor([[[ 0.5917, -0.4587, 0.5820, ..., 0.2063, -0.1083, -0.1181], [ 0.6138, -0.6512, 0.6274, ..., 0.1371, -0.2170, -0.3211], [ 0.8002, -0.5411, 0.8092, ..., 0.0173, -0.1424, -0.0998]]], grad_fn=<NativeLayerNormBackward0>), pooler_output=tensor([[ 5.9969e-02, -2.9528e-02, 4.0020e-01, 8.9153e-02, -7.6358e-02, 1.1700e-01, 2.2558e-01, 2.2368e-01, -1.5202e-01, 1.2092e-01, -3.3085e-02, -2.8979e-03, -3.5918e-01, 1.3999e-01, 2.7579e-01, -2.0411e-02, 1.1153e-01, 1.9183e-01, 2.1063e-01, -2.7483e-01, -9.9749e-01, -3.0571e-01, -5.3179e-02, -2.8360e-01, -2.5086e-01, 1.4877e-01, -8.1033e-02, 7.2817e-02, 1.4959e-01, -3.9660e-01, -4.1933e-02, -9.9817e-01, 5.5237e-01, 3.2668e-01, 4.3992e-01, -9.1618e-02, -1.7267e-02, 5.7098e-03, 2.4953e-01, -1.7852e-01, 1.1814e-02, 1.2091e-01, -1.1904e-01, -7.7950e-02, 4.5230e-01, -1.9725e-01, -5.0464e-02, 4.7424e-02, -4.2836e-01, -1.8238e-02, 1.0977e-02, -1.6073e-02, 2.6587e-01, 1.9053e-01, 1.5416e-01, -6.3297e-02, -2.3615e-02, -8.1964e-02, 1.8489e-01, -2.3479e-01, 6.7886e-02, 1.5042e-01, 2.0807e-01, -1.7040e-01, -2.9881e-01, -2.0684e-01, 4.8278e-02, 5.0205e-02, 2.7569e-02, -1.6108e-02, -2.2583e-01, -1.5076e-01, -3.0563e-01, 5.6014e-03, -4.1851e-02, -2.0374e-01, 3.2329e-01, 2.4174e-01, 1.1792e-01, -4.7416e-02, -2.9508e-01, -4.3588e-01, -2.0144e-01, 1.5361e-01, -6.8126e-02, 3.3846e-01, 2.0481e-01, -3.7374e-01, 1.2818e-01, 2.1754e-02, 1.7560e-01, 5.3715e-02, -8.2860e-02, -8.8868e-02, -1.6861e-01, -3.6583e-01, -2.3073e-01, -7.3909e-02, -9.4930e-02, -1.9170e-01, -7.3342e-03, -1.5028e-01, 1.3858e-02, -8.8288e-02, -6.2695e-02, -2.0720e-01, 1.2873e-01, 9.9554e-02, 1.0772e-01, 4.2834e-02, -7.5886e-02, -1.6059e-01, -2.1158e-02, 1.7143e-01, -1.0153e-01, 5.2400e-01, -2.0954e-01, 2.6050e-01, -6.0447e-02, -2.2621e-01, -1.2111e-01, 9.9737e-01, 7.8637e-02, -1.5479e-04, -7.0324e-02, 1.0001e-01, -2.4532e-01, 4.3483e-01, 2.5222e-01, 5.1069e-02, 1.8711e-01, 2.8339e-02, -3.2788e-01, -6.1781e-02, -2.1478e-01, -1.4524e-01, -1.4704e-01, 1.5843e-01, -3.2705e-01, 8.9687e-02, 7.9512e-02, 2.4066e-01, 1.2148e-01, 5.9500e-02, 1.9551e-02, -5.6383e-02, 4.2235e-01, -3.5410e-02, 9.9683e-01, 3.3365e-01, -1.0914e-01, -7.8143e-02, 4.5720e-01, -3.6139e-01, -9.4052e-02, -2.9103e-01, -1.7178e-01, -3.2769e-01, 2.6754e-01, 3.7255e-01, 8.4494e-02, 1.9836e-03, -1.8689e-01, -5.8495e-02, 2.2463e-01, -2.7782e-01, -6.5879e-02, 1.8703e-01, 9.9822e-02, 1.0137e-01, -1.1256e-01, 4.6567e-01, 2.2670e-01, -2.3757e-01, -5.5120e-02, 1.8358e-01, 6.8885e-02, -4.8863e-02, -1.5539e-01, 8.6935e-02, 3.3127e-01, -3.3166e-01, -1.0855e-01, -5.7433e-02, -2.1928e-02, -3.3799e-01, 4.8820e-02, -9.3553e-02, -9.0431e-02, 1.4344e-01, -1.5496e-01, 1.5766e-01, -2.4360e-01, 2.1582e-01, 3.4157e-01, 1.0599e-01, -2.0447e-01, 2.1597e-01, 4.8452e-01, 2.2034e-01, 3.5009e-02, 7.3436e-02, 1.2471e-01, 4.2982e-01, -2.4757e-01, -5.9791e-01, 3.1461e-01, 2.4756e-01, 2.2558e-01, -1.5604e-01, -2.3694e-01, -3.4975e-01, 3.5500e-01, 3.2177e-01, -2.5057e-01, 1.1095e-01, 3.5307e-01, -3.3087e-01, 5.9897e-02, 5.1021e-02, -1.8246e-02, -1.5076e-01, -3.6455e-01, -3.8838e-01, 4.9575e-02, 9.8375e-02, 2.0098e-01, 2.4716e-01, 1.3382e-01, -4.0611e-01, 3.0370e-02, -2.1231e-01, 4.4431e-02, -1.9845e-02, 3.0191e-02, 4.4657e-01, -8.1858e-02, 1.7452e-01, -3.6958e-01, -4.7298e-02, 1.8224e-01, -1.9083e-01, 3.3819e-01, 5.6905e-01, 1.7529e-01, -2.5879e-01, 2.9544e-01, 1.3149e-01, 1.2007e-01, -9.7963e-02, 1.7445e-01, -1.8260e-01, 4.8496e-01, 2.7463e-01, 7.5850e-02, -9.9746e-01, 1.7813e-01, 1.1883e-01, 3.6701e-01, -3.7606e-02, 3.6496e-01, 4.3491e-01, 2.1549e-01, 4.9902e-01, -2.0131e-01, -1.6312e-01, -4.4063e-01, -2.8601e-01, -2.4484e-01, 1.9112e-03, -3.7161e-01, -1.4300e-01, -2.6625e-01, -3.6160e-02, -1.4741e-01, 1.5084e-01, 1.9427e-01, -7.5823e-01, 2.1791e-01, 2.5877e-01, -1.2773e-01, -1.2449e-01, 2.7816e-01, -9.9719e-01, 2.0384e-01, -1.7850e-01, 4.5598e-02, 1.5705e-01, -1.1915e-01, 1.5505e-02, 3.2123e-01, 1.4321e-01, 1.7238e-01, -8.8783e-02, -7.9058e-02, 1.8648e-01, -3.2061e-02, -2.9808e-02, 1.5732e-01, 1.5358e-02, 2.8621e-01, 6.3961e-02, 2.2671e-01, 1.4830e-01, 3.0473e-02, 3.0433e-01, -1.7904e-01, 1.4617e-01, 3.9398e-01, 3.0933e-02, -1.3927e-01, -4.7411e-02, 3.5211e-01, -2.5358e-01, 3.0047e-01, -1.7789e-02, -2.0274e-01, 5.5363e-02, 3.4920e-02, -7.2669e-02, -7.0308e-02, 1.2434e-01, -2.5466e-01, 9.9768e-01, 2.5095e-01, -7.0067e-02, -5.2189e-01, 2.3797e-01, 4.1827e-01, -1.0887e-01, -5.9949e-01, -1.2234e-01, 1.5537e-01, 3.0183e-01, 2.3639e-01, -1.2411e-02, 6.9772e-02, 8.8934e-02, 7.2452e-02, 5.0240e-02, 5.5824e-02, -3.0316e-01, 1.4688e-01, -1.4542e-01, -1.8090e-01, 1.1232e-01, -1.4023e-01, 6.5263e-02, -4.2248e-01, 2.6356e-02, 2.9222e-01, 8.5037e-02, 8.3515e-02, 3.4675e-02, 4.1259e-02, 2.7126e-01, 1.8883e-01, -9.2570e-02, -1.6072e-01, -2.2847e-01, -7.8879e-02, -6.0544e-02, -1.6559e-01, 8.0641e-02, 2.0416e-01, -5.1342e-02, 2.8973e-01, -9.4009e-02, -7.0425e-02, -1.7127e-01, 3.3810e-01, -9.9806e-01, -1.9202e-02, 1.9828e-01, -3.0111e-01, 4.1362e-01, -7.3153e-02, -3.5004e-02, 1.4504e-01, 1.7425e-01, 3.5599e-03, 1.0566e-01, -1.7804e-01, -3.0556e-02, 4.1378e-02, 1.0979e-01, 3.1022e-01, 1.8557e-01, 2.1294e-01, -1.1286e-01, 7.7546e-02, -3.2865e-01, -5.2912e-02, 3.3346e-01, -8.9447e-02, -1.5653e-01, -4.9547e-02, 2.4388e-01, 1.9072e-01, 2.8262e-02, 4.1071e-01, -2.4039e-01, -1.8263e-01, 3.2457e-01, 9.8918e-02, -2.0052e-01, -3.1968e-01, 9.2117e-02, -8.1688e-02, 2.2843e-01, -1.1804e-03, 2.5090e-01, 1.8174e-01, -2.8723e-02, -2.3781e-01, -2.2093e-01, -6.2892e-02, 9.8965e-02, -3.3219e-01, -2.7744e-01, -1.3207e-01, 9.9742e-01, 7.0327e-02, 3.4208e-01, -4.5168e-01, 4.0007e-01, 3.2408e-01, -1.3243e-02, 2.8286e-01, 5.0396e-02, 1.6667e-01, -2.4661e-01, 5.0037e-02, -8.0211e-02, 2.2650e-01, -8.1374e-03, 1.7261e-01, 7.4401e-02, -2.2746e-01, 6.8310e-02, -2.5272e-01, -4.0856e-01, -7.0748e-01, 2.8515e-01, 2.3769e-01, -2.6715e-01, -5.1703e-01, -8.1558e-02, -1.5937e-01, 5.4645e-02, -2.4245e-01, 5.5708e-02, 2.2595e-01, 6.4999e-02, 3.2584e-01, -4.2017e-02, 9.9736e-01, -1.8473e-01, 1.3916e-01, -4.7836e-02, -3.7824e-02, -1.6771e-01, -1.6767e-01, -3.0335e-01, 1.8001e-01, -9.7848e-02, 1.4782e-01, -4.9802e-01, 1.3550e-01, 7.6381e-02, 6.3978e-02, -1.2072e-02, -3.6520e-02, -3.1372e-01, 2.9353e-01, 1.0607e-02, -1.0208e-01, -2.7822e-01, 1.6078e-01, -5.0989e-01, 1.8656e-01, -5.0234e-02, 6.4833e-02, -3.0711e-01, 1.2582e-01, -1.9269e-02, 2.6781e-01, -2.3377e-01, 8.6382e-02, -1.8650e-01, -1.0073e-01, -4.6782e-01, -1.7406e-02, -2.0020e-01, 9.9749e-01, -2.4608e-02, 1.7318e-01, -1.8265e-01, 1.4780e-01, -2.0103e-01, 4.7512e-01, 1.2639e-01, 7.3818e-02, 1.6959e-01, 3.2306e-01, -3.6860e-01, 3.1404e-01, -7.6970e-02, -5.7854e-01, 5.6602e-02, 4.5709e-01, 3.8136e-02, 1.8349e-01, 4.5581e-01, 7.8982e-02, 1.3219e-01, -1.5332e-01, 7.2412e-02, 1.3165e-01, 1.7046e-01, 2.2885e-01, 2.5871e-01, -1.2678e-02, -7.3951e-02, -4.9309e-01, 9.9819e-01, 9.9628e-01, -2.5750e-02, 1.9030e-01, -2.2099e-01, -1.6245e-01, -1.9312e-01, 5.3803e-03, 8.6229e-02, 2.6053e-01, 9.4428e-02, -2.9584e-01, -1.7609e-01, -9.5494e-02, 6.0100e-02, 1.0895e-01, -1.3635e-01, 5.5727e-02, -2.3048e-01, 9.7328e-02, 3.7507e-01, 2.2147e-01, -2.3173e-03, 2.5493e-01, 2.4562e-01, -1.7073e-01, -1.9163e-01, -1.1118e-02, -3.3140e-01, 9.9232e-03, 7.2381e-02, 5.8398e-02, -9.9716e-01, 7.3113e-02, -1.8559e-01, -2.6608e-01, 4.7819e-01, 7.4917e-02, 1.9444e-01, -5.2799e-02, -3.6409e-01, -2.6557e-01, 2.4623e-01, 4.4844e-02, 1.1082e-01, -1.8033e-01, -2.4645e-01, 3.1054e-01, -1.9978e-01, 6.1873e-03, -3.3965e-01, -2.5551e-01, -1.5982e-02, 2.0613e-02, -2.7569e-01, -3.1128e-02, -3.3368e-01, 2.4278e-02, 2.4145e-01, -1.7751e-02, 1.5775e-02, -2.2276e-01, 2.8615e-01, -7.8051e-02, -1.0230e-01, 2.4267e-01, 1.8090e-01, 1.2858e-01, -1.3220e-02, -2.7613e-01, -7.8747e-02, -2.5742e-01, -1.1929e-01, 2.7459e-01, -1.1241e-01, 3.2656e-01, -8.7801e-02, 3.4712e-03, -2.1399e-01, 6.5081e-02, 3.0578e-01, 1.1775e-01, -1.6849e-01, 4.3242e-01, 5.1911e-01, 2.3756e-02, 4.0207e-01, 7.2790e-02, -3.5139e-01, -3.4423e-01, 9.9770e-01, 2.3580e-01, 1.8302e-01, -1.5551e-01, -3.4704e-02, -1.2189e-01, 8.1130e-02, 4.2769e-01, -1.0948e-01, 6.5104e-01, -2.8204e-01, 1.8948e-01, 2.8437e-01, 3.2862e-01, 1.0675e-02, 1.4780e-01, 2.0146e-01, 1.7602e-01, 3.5416e-01, 3.9548e-01, 3.2059e-01, 4.1251e-01, 1.3758e-01, 1.9462e-01, 1.9243e-02, 3.7949e-01, 1.2713e-02, -1.8807e-01, 4.1209e-01, -1.9652e-01, 1.8252e-02, 2.6968e-02, 8.7959e-02, -9.5017e-02, -5.9194e-02, -2.5948e-01, -2.8553e-02, -2.2109e-01, 3.3540e-01, -2.1638e-01, 5.6245e-02, -9.5250e-02, -2.4076e-01, 3.5068e-01, -3.4758e-01, 3.5926e-01, -3.4302e-02, 3.6915e-02, -2.2418e-01, 6.6422e-02, 1.9171e-02, -4.1770e-01, -7.1583e-02, -3.3430e-01, 2.0526e-01, 2.4508e-01, -7.4681e-02, 2.9844e-01, -1.3024e-01, 4.2759e-01, 1.0793e-01, -2.4166e-01, -9.2636e-02, -9.9746e-01, 1.1442e-01, 2.2176e-01, -2.7835e-02, -6.6700e-02, -2.2908e-01, 1.5853e-01, 3.7068e-01, -1.4861e-01, -2.3097e-01, -1.4553e-01, 1.2633e-01, -1.9639e-01, -1.2390e-01, 3.1364e-01, -2.4779e-01, -2.4920e-01, -2.1438e-01, -3.5619e-02, 2.2667e-01, 3.3857e-01, -3.9294e-01, 2.7900e-01, -2.0619e-01, 2.0817e-01, -2.0552e-01, 1.1617e-01, -1.0186e-01, -1.5929e-01, -2.7417e-03, -2.9296e-01, -2.9401e-01, 1.7255e-02, 3.4470e-01, -1.1942e-01, 2.2924e-02, 1.9109e-01, -6.5098e-02, 3.7304e-01, -2.3259e-01, 3.4568e-01, -1.9573e-01, 2.5953e-01, -4.1360e-01, -1.6410e-01, -5.2876e-01, -2.5299e-01, 1.0047e-01, 3.1621e-01, -1.0462e-01, 8.6058e-02, 5.4909e-02, 1.8881e-01, 1.1468e-01, 1.5064e-01, 1.0535e-01, -6.1509e-02, -7.0513e-02, -3.7463e-01, 2.2138e-01, -1.6765e-01, 6.1603e-02, -8.0533e-01, -6.9980e-02, 2.3973e-01, 3.3635e-01, 4.9510e-01, -3.6996e-01, -7.7451e-02, -4.6428e-01, -5.7513e-02, 3.0907e-02, 3.3044e-01, 1.5560e-01, 1.2385e-01, 2.3155e-01, -2.4292e-01, -1.9017e-01, 5.9568e-01, -1.6784e-01, 2.7504e-02, 2.5809e-01, 1.9123e-01, 7.5970e-01, 2.1786e-01, 3.2746e-01, -9.6603e-02, -2.1835e-01, 3.9978e-01, 2.8238e-01]], grad_fn=<TanhBackward0>), hidden_states=None, past_key_values=None, attentions=None, cross_attentions=None)

# Check if the model supports attention weights
config = BertConfig.from_pretrained(machine_T )
if config.output_attentions:
    # Extract hidden states
    hidden_states = outputs.last_hidden_state

    # Self-attention
    self_attention_weights = outputs.attentions

    # Print self-attention weights
    print("Self-attention weights:")
    for layer, attn_weights in enumerate(self_attention_weights):
        print(f"Layer {layer+1}: {attn_weights.mean(dim=1)}")

# Decoding input text
decoded_output = tokenizer.decode(input_ids[0], skip_special_tokens=True)
print("Decoded output from positional encoder:", decoded_output)

# Get the model's final output
final_output = outputs[0]

# Print the final output
print("Final output:", final_output)

print("Shape of the final Output",final_output.shape)
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
Decoded output from positional encoder: Hello Final output: tensor([[[ 0.5917, -0.4587, 0.5820, ..., 0.2063, -0.1083, -0.1181], [ 0.6138, -0.6512, 0.6274, ..., 0.1371, -0.2170, -0.3211], [ 0.8002, -0.5411, 0.8092, ..., 0.0173, -0.1424, -0.0998]]], grad_fn=<NativeLayerNormBackward0>) Shape of the final Output torch.Size([1, 3, 768])

Langauge Translation

from transformers import MarianTokenizer, MarianMTModel

model_name = 'Helsinki-NLP/opus-mt-en-hi'

tokenizer = MarianTokenizer.from_pretrained(model_name)
model = MarianMTModel.from_pretrained(model_name)

text = "Hello, how are you?"

inputs = tokenizer(text, return_tensors="pt", padding=True)
translated = model.generate(**inputs)

output = tokenizer.decode(translated[0], skip_special_tokens=True)

print(output)
C:\Users\Suyashi144893\AppData\Local\anaconda3\envs\torch_fix\lib\site-packages\tqdm\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html from .autonotebook import tqdm as notebook_tqdm C:\Users\Suyashi144893\AppData\Local\anaconda3\envs\torch_fix\lib\site-packages\transformers\models\marian\tokenization_marian.py:176: UserWarning: Recommended: pip install sacremoses. warnings.warn("Recommended: pip install sacremoses.") Loading weights: 100%|████████████████████████████████████████████████████████████| 258/258 [00:00<00:00, 15891.95it/s] The tied weights mapping and config for this model specifies to tie model.shared.weight to model.decoder.embed_tokens.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning The tied weights mapping and config for this model specifies to tie model.shared.weight to model.encoder.embed_tokens.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
हैलो, तुम कैसे हो? 
from transformers import MarianMTModel, MarianTokenizer

# Load the pre-trained translation model and tokenizer for English to German
model_name = "Helsinki-NLP/opus-mt-en-de"
tokenizer = MarianTokenizer.from_pretrained(model_name)
model = MarianMTModel.from_pretrained(model_name)

# Input text in English
input_text = input("Enter the Text: ")

# Tokenize the input text
inputs = tokenizer(input_text, return_tensors="pt")

# Perform translation
outputs = model.generate(**inputs)

# Decode the translated text
decoded_text = tokenizer.decode(outputs[0], skip_special_tokens=True)

# Print the decoded text
print("Decoded text in German:", decoded_text)
Loading weights: 100%|████████████████████████████████████████████████████████████| 258/258 [00:00<00:00, 15427.27it/s] The tied weights mapping and config for this model specifies to tie model.shared.weight to model.decoder.embed_tokens.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning The tied weights mapping and config for this model specifies to tie model.shared.weight to model.encoder.embed_tokens.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning
Enter the Text: I am happy Decoded text in German: Ich bin glücklich
C:\Users\Suyashi144893\AppData\Local\anaconda3\envs\torch_fix\lib\site-packages\huggingface_hub\file_download.py:129: UserWarning: `huggingface_hub` cache-system uses symlinks by default to efficiently store duplicated files but your machine does not support them in C:\Users\Suyashi144893\.cache\huggingface\hub\models--Helsinki-NLP--opus-mt-en-de. Caching files will still work but in a degraded version that might require more space on your disk. This warning can be disabled by setting the `HF_HUB_DISABLE_SYMLINKS_WARNING` environment variable. For more details, see https://huggingface.co/docs/huggingface_hub/how-to-cache#limitations. To support symlinks on Windows, you either need to activate Developer Mode or to run Python as an administrator. In order to activate developer mode, see this article: https://docs.microsoft.com/en-us/windows/apps/get-started/enable-your-device-for-development warnings.warn(message) 
pip install gradio

Multi Langauge Translation Model 

from transformers import MarianMTModel, MarianTokenizer

# English → Target language models
models = {
    "fr": "Helsinki-NLP/opus-mt-en-fr",
    "de": "Helsinki-NLP/opus-mt-en-de",
    "es": "Helsinki-NLP/opus-mt-en-es",
    "hi": "Helsinki-NLP/opus-mt-en-hi"
}

loaded_models = {}

def load_model(target_lang):
    if target_lang not in models:
        raise ValueError("Invalid language selected")

    if target_lang not in loaded_models:
        model_name = models[target_lang]
        tokenizer = MarianTokenizer.from_pretrained(model_name)
        model = MarianMTModel.from_pretrained(model_name)
        loaded_models[target_lang] = (tokenizer, model)

    return loaded_models[target_lang]


def translate(text, target_lang):
    tokenizer, model = load_model(target_lang)

    tokens = tokenizer(text, return_tensors="pt", padding=True)
    output = model.generate(**tokens)

    return tokenizer.decode(output[0], skip_special_tokens=True)


# ---- USER INPUT ----
if __name__ == "__main__":
    text = input("Enter English text: ")
    lang = input("Choose target language (fr/de/es/hi): ")

    print("Translated:", translate(text, lang))
Enter English text: hi Choose target language (fr/de/es/hi): fr
C:\Users\Suyashi144893\AppData\Local\anaconda3\envs\torch_fix\lib\site-packages\huggingface_hub\file_download.py:129: UserWarning: `huggingface_hub` cache-system uses symlinks by default to efficiently store duplicated files but your machine does not support them in C:\Users\Suyashi144893\.cache\huggingface\hub\models--Helsinki-NLP--opus-mt-en-fr. Caching files will still work but in a degraded version that might require more space on your disk. This warning can be disabled by setting the `HF_HUB_DISABLE_SYMLINKS_WARNING` environment variable. For more details, see https://huggingface.co/docs/huggingface_hub/how-to-cache#limitations. To support symlinks on Windows, you either need to activate Developer Mode or to run Python as an administrator. In order to activate developer mode, see this article: https://docs.microsoft.com/en-us/windows/apps/get-started/enable-your-device-for-development warnings.warn(message) Loading weights: 100%|████████████████████████████████████████████████████████████| 258/258 [00:00<00:00, 12108.43it/s] The tied weights mapping and config for this model specifies to tie model.shared.weight to model.decoder.embed_tokens.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning The tied weights mapping and config for this model specifies to tie model.shared.weight to model.encoder.embed_tokens.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning
Translated: Bonjour.

Improvement with Gradio App

import gradio as gr
from transformers import MarianMTModel, MarianTokenizer

models = {
    "French": "Helsinki-NLP/opus-mt-en-fr",
    "German": "Helsinki-NLP/opus-mt-en-de",
    "Spanish": "Helsinki-NLP/opus-mt-en-es",
    "Hindi": "Helsinki-NLP/opus-mt-en-hi"
}

loaded_models = {}

def load_model(model_name):
    if model_name not in loaded_models:
        tokenizer = MarianTokenizer.from_pretrained(model_name)
        model = MarianMTModel.from_pretrained(model_name)
        loaded_models[model_name] = (tokenizer, model)
    return loaded_models[model_name]


def translate(text, language):
    if not text.strip():
        return "Please enter English text"

    model_name = models[language]
    tokenizer, model = load_model(model_name)

    tokens = tokenizer(text, return_tensors="pt", padding=True)
    output = model.generate(**tokens)

    return tokenizer.decode(output[0], skip_special_tokens=True)


app = gr.Interface(
    fn=translate,
    inputs=[
        gr.Textbox(
            label="Enter English Text",
            placeholder="Type your sentence in English..."
        ),
        gr.Dropdown(
            choices=list(models.keys()),
            label="Select Target Language"
        )
    ],
    outputs="text",
    title="English to Multi-Language Translator"
)

app.launch()
* Running on local URL: http://127.0.0.1:7860 * To create a public link, set `share=True` in `launch()`.
C:\Users\Suyashi144893\AppData\Local\anaconda3\envs\torch_fix\lib\site-packages\transformers\models\marian\tokenization_marian.py:176: UserWarning: Recommended: pip install sacremoses. warnings.warn("Recommended: pip install sacremoses.") Loading weights: 100%|████████████████████████████████████████████████████████████| 258/258 [00:00<00:00, 15735.27it/s] The tied weights mapping and config for this model specifies to tie model.shared.weight to model.decoder.embed_tokens.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning The tied weights mapping and config for this model specifies to tie model.shared.weight to model.encoder.embed_tokens.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning C:\Users\Suyashi144893\AppData\Local\anaconda3\envs\torch_fix\lib\site-packages\transformers\models\marian\tokenization_marian.py:176: UserWarning: Recommended: pip install sacremoses. warnings.warn("Recommended: pip install sacremoses.") Loading weights: 100%|█████████████████████████████████████████████████████████████| 258/258 [00:00<00:00, 8017.56it/s] The tied weights mapping and config for this model specifies to tie model.shared.weight to model.decoder.embed_tokens.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning The tied weights mapping and config for this model specifies to tie model.shared.weight to model.encoder.embed_tokens.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning

Switching to a single multilingual model is the right architecture for scalability and avoids managing multiple models.

Implementation using Facebook M2M100, which supports 100+ languages.

Language Mapping

languages = { "English": "en", "Hindi": "hi", "French": "fr", "German": "de", "Spanish": "es", "Chinese (Simplified)": "zh", "Arabic": "ar", "Russian": "ru", "Portuguese": "pt", "Italian": "it", "Japanese": "ja", "Korean": "ko", "Dutch": "nl", "Turkish": "tr", "Polish": "pl", "Swedish": "sv", "Danish": "da", "Finnish": "fi", "Greek": "el", "Czech": "cs", "Hungarian": "hu", "Romanian": "ro", "Ukrainian": "uk", "Thai": "th", "Vietnamese": "vi", "Indonesian": "id", "Malay": "ms", "Bengali": "bn", "Tamil": "ta", "Telugu": "te", "Marathi": "mr", "Gujarati": "gu", "Punjabi": "pa", "Urdu": "ur" }

from transformers import M2M100ForConditionalGeneration, M2M100Tokenizer

model_name = "facebook/m2m100_418M"

tokenizer = M2M100Tokenizer.from_pretrained(model_name)
model = M2M100ForConditionalGeneration.from_pretrained(model_name)


def translate(text, source_lang, target_lang):
    tokenizer.src_lang = source_lang

    encoded = tokenizer(text, return_tensors="pt")

    generated_tokens = model.generate(
        **encoded,
        forced_bos_token_id=tokenizer.get_lang_id(target_lang)
    )

    return tokenizer.decode(generated_tokens[0], skip_special_tokens=True)


# ---- USER INPUT ----
if __name__ == "__main__":
    text = input("Enter text: ")
    src = input("Enter source language code (e.g., en, hi): ")
    tgt = input("Enter target language code (e.g., fr, de, es, hi): ")

    result = translate(text, src, tgt)
    print("Translated:", result)
C:\Users\Suyashi144893\AppData\Local\anaconda3\envs\torch_fix\lib\site-packages\huggingface_hub\file_download.py:129: UserWarning: `huggingface_hub` cache-system uses symlinks by default to efficiently store duplicated files but your machine does not support them in C:\Users\Suyashi144893\.cache\huggingface\hub\models--facebook--m2m100_418M. Caching files will still work but in a degraded version that might require more space on your disk. This warning can be disabled by setting the `HF_HUB_DISABLE_SYMLINKS_WARNING` environment variable. For more details, see https://huggingface.co/docs/huggingface_hub/how-to-cache#limitations. To support symlinks on Windows, you either need to activate Developer Mode or to run Python as an administrator. In order to activate developer mode, see this article: https://docs.microsoft.com/en-us/windows/apps/get-started/enable-your-device-for-development warnings.warn(message) Loading weights: 100%|█████████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 9320.23it/s] The tied weights mapping and config for this model specifies to tie model.shared.weight to lm_head.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning The tied weights mapping and config for this model specifies to tie model.shared.weight to model.decoder.embed_tokens.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning The tied weights mapping and config for this model specifies to tie model.shared.weight to model.encoder.embed_tokens.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning
Enter text: i am travelling Enter source language code (e.g., en, hi): en Enter target language code (e.g., fr, de, es, hi): de Translated: Ich reiste

using Gradio

import gradio as gr
from transformers import M2M100ForConditionalGeneration, M2M100Tokenizer

# Load model
model_name = "facebook/m2m100_418M"
tokenizer = M2M100Tokenizer.from_pretrained(model_name)
model = M2M100ForConditionalGeneration.from_pretrained(model_name)

# Language mapping
languages = {
    "English": "en",
    "Hindi": "hi",
    "French": "fr",
    "German": "de",
    "Spanish": "es",
    "Chinese (Simplified)": "zh",
    "Arabic": "ar",
    "Russian": "ru",
    "Portuguese": "pt",
    "Italian": "it",
    "Japanese": "ja",
    "Korean": "ko",
    "Dutch": "nl",
    "Turkish": "tr",
    "Polish": "pl",
    "Swedish": "sv",
    "Danish": "da",
    "Finnish": "fi",
    "Greek": "el",
    "Czech": "cs",
    "Hungarian": "hu",
    "Romanian": "ro",
    "Ukrainian": "uk",
    "Thai": "th",
    "Vietnamese": "vi",
    "Indonesian": "id",
    "Malay": "ms",
    "Bengali": "bn",
    "Tamil": "ta",
    "Telugu": "te",
    "Marathi": "mr",
    "Gujarati": "gu",
    "Punjabi": "pa",
    "Urdu": "ur"
}

# Sort for better UX
language_list = sorted(languages.keys())


def translate(text, source_lang, target_lang):
    if not text.strip():
        return "Please enter text"

    tokenizer.src_lang = languages[source_lang]

    encoded = tokenizer(text, return_tensors="pt")

    generated_tokens = model.generate(
        **encoded,
        forced_bos_token_id=tokenizer.get_lang_id(languages[target_lang])
    )

    return tokenizer.decode(generated_tokens[0], skip_special_tokens=True)


# UI
app = gr.Interface(
    fn=translate,
    inputs=[
        gr.Textbox(
            label="Enter Text",
            placeholder="Type your sentence..."
        ),
        
        # Searchable dropdown
        gr.Dropdown(
            choices=language_list,
            label="Source Language",
            value="English",
            filterable=True   # <-- THIS enables search
        ),
        
        gr.Dropdown(
            choices=language_list,
            label="Target Language",
            value="Hindi",
            filterable=True   # <-- searchable here too
        )
    ],
    outputs="text",
    title="Multilingual Translator"
)

app.launch()
Loading weights: 100%|████████████████████████████████████████████████████████████| 512/512 [00:00<00:00, 11268.56it/s] The tied weights mapping and config for this model specifies to tie model.shared.weight to lm_head.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning The tied weights mapping and config for this model specifies to tie model.shared.weight to model.decoder.embed_tokens.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning The tied weights mapping and config for this model specifies to tie model.shared.weight to model.encoder.embed_tokens.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning
* Running on local URL: http://127.0.0.1:7861 * To create a public link, set `share=True` in `launch()`. 
pip install transformers torch sentencepiece

Full Multi-Task NLP App

import gradio as gr
from transformers import pipeline

# -----------------------------
# Load Models (CPU friendly)
# -----------------------------
sentiment = pipeline(
    "sentiment-analysis",
    model="distilbert-base-uncased-finetuned-sst-2-english"
)

ner = pipeline(
    "ner",
    model="dslim/bert-base-NER",
    aggregation_strategy="simple"
)

qa = pipeline(
    "question-answering",
    model="distilbert-base-cased-distilled-squad"
)

summarizer = pipeline(
    "summarization",
    model="t5-small"
)

text2text = pipeline(
    "text2text-generation",
    model="t5-small"
)

# -----------------------------
# Functions
# -----------------------------

def analyze_sentiment(text):
    if not text.strip():
        return "Please enter text"
    return sentiment(text)

def extract_entities(text):
    if not text.strip():
        return "Please enter text"
    return ner(text)

def answer_question(context, question):
    if not context.strip() or not question.strip():
        return "Please enter both context and question"
    return qa({"context": context, "question": question})

def summarize_text(text):
    if not text.strip():
        return "Please enter text"
    return summarizer(text, max_length=80, min_length=20, do_sample=False)

def paragraph_to_points(text):
    if not text.strip():
        return "Please enter text"
    prompt = "Convert paragraph into bullet points: " + text
    return text2text(prompt, max_length=120)[0]["generated_text"]


# -----------------------------
# UI (Tabs)
# -----------------------------

with gr.Blocks(title="Multi-Task NLP App") as app:

    gr.Markdown("# 🧠 Multi-Task NLP using Transformers")
    gr.Markdown("Perform Sentiment Analysis, NER, Q&A, Summarization, and Text Structuring")

    with gr.Tab("Sentiment Analysis"):
        txt1 = gr.Textbox(label="Enter Text")
        out1 = gr.JSON(label="Sentiment Output")
        btn1 = gr.Button("Analyze")
        btn1.click(analyze_sentiment, inputs=txt1, outputs=out1)

    with gr.Tab("Named Entity Recognition"):
        txt2 = gr.Textbox(label="Enter Text")
        out2 = gr.JSON(label="Entities")
        btn2 = gr.Button("Extract")
        btn2.click(extract_entities, inputs=txt2, outputs=out2)

    with gr.Tab("Question Answering"):
        context = gr.Textbox(label="Context")
        question = gr.Textbox(label="Question")
        out3 = gr.JSON(label="Answer")
        btn3 = gr.Button("Get Answer")
        btn3.click(answer_question, inputs=[context, question], outputs=out3)

    with gr.Tab("Summarization"):
        txt4 = gr.Textbox(label="Enter Text")
        out4 = gr.JSON(label="Summary")
        btn4 = gr.Button("Summarize")
        btn4.click(summarize_text, inputs=txt4, outputs=out4)

    with gr.Tab("Paragraph → Bullet Points"):
        txt5 = gr.Textbox(label="Enter Paragraph")
        out5 = gr.Textbox(label="Bullet Points")
        btn5 = gr.Button("Convert")
        btn5.click(paragraph_to_points, inputs=txt5, outputs=out5)


# -----------------------------
# Launch
# -----------------------------
app.launch()
Loading weights: 100%|█████████████████████████████████████████████████████████████| 104/104 [00:00<00:00, 2896.18it/s] Loading weights: 100%|█████████████████████████████████████████████████████████████| 199/199 [00:00<00:00, 2355.36it/s] BertForTokenClassification LOAD REPORT from: dslim/bert-base-NER Key | Status | | -------------------------+------------+--+- bert.pooler.dense.bias | UNEXPECTED | | bert.pooler.dense.weight | UNEXPECTED | | Notes: - UNEXPECTED :can be ignored when loading from different task/architecture; not ok if you expect identical arch.
--------------------------------------------------------------------------- KeyError Traceback (most recent call last) Cell In[10], line 18 7 sentiment = pipeline( 8 "sentiment-analysis", 9 model="distilbert-base-uncased-finetuned-sst-2-english" 10 ) 12 ner = pipeline( 13 "ner", 14 model="dslim/bert-base-NER", 15 aggregation_strategy="simple" 16 ) ---> 18 qa = pipeline( 19 "question-answering", 20 model="distilbert-base-cased-distilled-squad" 21 ) 23 summarizer = pipeline( 24 "summarization", 25 model="t5-small" 26 ) 28 text2text = pipeline( 29 "text2text-generation", 30 model="t5-small" 31 )
File ~\AppData\Local\anaconda3\envs\torch_fix\lib\site-packages\transformers\pipelines\__init__.py:742, in pipeline(task, model, config, tokenizer, feature_extractor, image_processor, processor, revision, use_fast, token, device, device_map, dtype, trust_remote_code, model_kwargs, pipeline_class, **kwargs) 735 pipeline_class = get_class_from_dynamic_module( 736 class_ref, 737 model, 738 code_revision=code_revision, 739 **hub_kwargs, 740 ) 741 else: --> 742 normalized_task, targeted_task, task_options = check_task(task) 743 if pipeline_class is None: 744 pipeline_class = targeted_task["impl"]
File ~\AppData\Local\anaconda3\envs\torch_fix\lib\site-packages\transformers\pipelines\__init__.py:355, in check_task(task) 319 def check_task(task: str) -> tuple[str, dict, Any]: 320 """ 321 Checks an incoming task string, to validate it's correct and return the default Pipeline and Model classes, and 322 default models if they exist. (...) 353 354 """ --> 355 return PIPELINE_REGISTRY.check_task(task)
File ~\AppData\Local\anaconda3\envs\torch_fix\lib\site-packages\transformers\pipelines\base.py:1341, in PipelineRegistry.check_task(self, task) 1338 targeted_task = self.supported_tasks[task] 1339 return task, targeted_task, None -> 1341 raise KeyError(f"Unknown task {task}, available tasks are {self.get_supported_tasks()}")
KeyError: "Unknown task question-answering, available tasks are ['any-to-any', 'audio-classification', 'automatic-speech-recognition', 'depth-estimation', 'document-question-answering', 'feature-extraction', 'fill-mask', 'image-classification', 'image-feature-extraction', 'image-segmentation', 'image-text-to-text', 'keypoint-matching', 'mask-generation', 'ner', 'object-detection', 'sentiment-analysis', 'table-question-answering', 'text-classification', 'text-generation', 'text-to-audio', 'text-to-speech', 'token-classification', 'video-classification', 'zero-shot-audio-classification', 'zero-shot-classification', 'zero-shot-image-classification', 'zero-shot-object-detection']"