parser_model.cpython-310.pyc

o

<WdS&�@s�dZd
dl
Z
d
dlZd
dlZd
dlmZ
d
dlm
mZ	
Gdd�dej
�Zedkrpe
j
dd�
Zejdd	d
dd�
ejd
dd
dd�
e��Zejdejd�Zee�
Zdd�Zdd�Zejrde�
ed�

ejrre�
ed�

dSdSdS)z�
CS224N 2021-2022: Homework 3
parser_model.py: Feed-Forward Neural Network for Dependency Parsing
Sahil Chopra <[email protected]>
Haoshen Hong <[email protected]>
�Ncs6eZ
dZd
Z		d�f
dd�	Zdd	�Zd
d�Z�ZS)
�ParserModela� Feedforward neural network with an embedding layer and two hidden layers.
    The ParserModel will predict which transition should be applied to a
    given partial parse configuration.

    PyTorch Notes:
        - Note that "ParserModel" is a subclass of the "nn.Module" class. In PyTorch all neural networks
            are a subclass of this "nn.Module".
        - The "__init__" method is where you define all the layers and parameters
            (embedding layers, linear layers, dropout layers, etc.).
        - "__init__" gets automatically called when you create a new instance of your class, e.g.
            when you write "m = ParserModel()".
        - Other methods of ParserModel can access variables that have "self." prefix. Thus,
            you should add the "self." prefix layers, values, etc. that you want to utilize
            in other ParserModel methods.
        - For further documentation on "nn.Module" please see https://pytorch.org/docs/stable/nn.html.
    �$�����?cs�tt
|���
||_||_||_|
jd
|_||_t	�
t�|
�
�
|_
t	�
t�|j|j|j��
|_t	�
t�|j�
�
|_t	j�|j�

t	j�|j�

t	j|jd�
|_t	�
t�|j|j��
|_t	�
t�|j�
�
|_t	j�|j�

t	j�|j�

dS)a]
 Initialize the parser model.

        @param embeddings (ndarray): word embeddings (num_words, embedding_size)
        @param n_features (int): number of input features
        @param hidden_size (int): number of hidden units
        @param n_classes (int): number of output classes
        @param dropout_prob (float): dropout probability
        �
)
�
pN)�superr�__init__�
n_features�	n_classes�dropout_prob�shapeZ
embed_size�hidden_size�nn�	Parameter�torch�tensor�
embeddings�empty�embed_to_hidden_weight�embed_to_hidden_bias�init�xavier_uniform_�uniform_�Dropout�dropout�hidden_to_logits_weight�hidden_to_logits_bias)�selfrrrrr
�
�	__class__��>/Users/yimingwang/Desktop/cs224n/assignment/a3/parser_model.pyr
!s 














zParserModel.__init__cCs,|
jd
}t
�|jd
t
�|
�
��|d�}|S)a>
 Utilize `w` to select embeddings from embedding matrix `self.embeddings`
            @param w (Tensor): input tensor of word indices (batch_size, n_features)

            @return x (Tensor): tensor of embeddings for words represented in w
                                (batch_size, n_features * embed_size)
        r
�����)rr�index_selectr�flatten�view)r�
w�
N�
xr"r"r#�embedding_lookupVs

zParserModel.embedding_lookupcCs8|�|
�
}t
�|�|j�
|j�
}|�|j�
|j}|S)
a_ Run the model forward.

            Note that we will not apply the softmax function here because it is included in the loss function nn.CrossEntropyLoss

            PyTorch Notes:
                - Every nn.Module object (PyTorch model) has a `forward` function.
                - When you apply your nn.Module to an input tensor `w` this function is applied to the tensor.
                    For example, if you created an instance of your ParserModel and applied it to some `w` as follows,
                    the `forward` function would called on `w` and the result would be stored in the `output` variable:
                        model = ParserModel()
                        output = model(w) # this calls the forward function
                - For more details checkout: https://pytorch.org/docs/stable/nn.html#torch.nn.Module.forward

        @param w (Tensor): input tensor of tokens (batch_size, n_features)

        @return logits (Tensor): tensor of predictions (output after applying the layers of the network)
                                 without applying softmax (batch_size, n_classes)
        )r+�
F�relu�mmrrrr)rr(r*�
h�logitsr"r"r#�forwardxs


zParserModel.forward)rrrr)�__name__�
__module__�__qualname__�__doc__r
r+r1�
__classcell__r"r"r r#rs

�5"r�__main__z'Simple sanity check for parser_model.py)
�descriptionz-ez--embedding�
store_truez)sanity check for embeding_lookup function)�action�helpz-fz	--forwardz!sanity check for forward function)�d��
�dtypecCsJtj
d
ddtjd�}t�|�
}
t�|
j��d
k�
s#Jdt	|
�
d�
�
dS)Nr
r<��rr>z The result of embedding lookup: z contains non-zero elements.)
r�randint�long�modelr+�np�all�data�numpy�repr)�inds�selectedr"r"r#�check_embedding�s




�

�rLcCsLtj
d
ddtjd�}t|�
}
d}|
j|ks$Jdt|
j�
dt|�
�
�
dS)Nr
r<r@r>)rArz The result shape of forward is: z which doesn't match expected )rrBrCrDrrI)�inputs�outZexpected_out_shaper"r"r#�
check_forward�s




�

�rOz%Embedding_lookup sanity check passes!zForward sanity check passes!)r5�argparserHrEr�torch.nnrZtorch.nn.functional�
functionalr,�Modulerr2�ArgumentParser�parser�add_argument�
parse_args�args�zeros�float32rrDrLrO�	embedding�printr1r"r"r"r#�<module>
s2










�