Book a Demo!
CoCalc Logo Icon
StoreFeaturesDocsShareSupportNewsAboutPoliciesSign UpSign In
Yiming-Wange
GitHub Repository: Yiming-Wange/CS224n-2023-solution
 Path: blob/main/a5/mingpt-demo/mingpt/__pycache__/utils.cpython-310.pyc
3764 views
o

M��c��@sVddlZddlZddlZddlmZddlmZdd�Zdd�Z	e�
�dd	d
��ZdS)�N)�
functionalcCs0t�|�tj�|�t�|�tj�|�dS)N)�random�seed�np�torchZmanual_seed�cudaZmanual_seed_all)r�r�J/Users/yimingwang/Desktop/cs224n/assignment/a5/mingpt-demo/mingpt/utils.py�set_seeds

r
cCs<t�||�\}}|��}td�|||dd�dgfk<|S)NZInf�����)r�topk�clone�float)�logits�k�v�ix�outrrr	�top_k_logits
s r��?Fc
Cs�|��}|��t|�D]V}|�d�|kr|n
|dd�|d�f}||�\}	}
|	dd�ddd�f|}	|dur>t|	|�}	tj|	dd�}|rOtj|dd�}n
tj	|ddd�\}
}tj
||fdd�}q|S)ai
    take a conditioning sequence of indices in x (of shape (b,t)) and predict the next token in
    the sequence, feeding the predictions back into the model each time. Clearly the sampling
    has quadratic complexity unlike an RNN that is only linear, and has a finite context window
    of block_size, unlike an RNN that has an infinite context window.
    �Nr)�dim)Znum_samples)rr)Zget_block_size�eval�range�sizer�FZsoftmaxrZmultinomialr�cat)
�model�xZstepsZtemperature�sampleZtop_k�
block_sizerZx_condr�_Zprobsrrrr	rs(
r)rFN)r�numpyrrZtorch.nn�nnrrr
rZno_gradrrrrr	�<module>s