sgd.cpython-39.pyc

a

ˬd�
�@sddZd
dl
Z
d
dlZd
dlZd
dlZd
dlmZ
dd�Z	dd�Z
dd	d
�
Zdd�Ze
d
kr`e�
dS)i��NcCs�d
}t�d�
D]2}
t
t�t�|
�
�
d
�d�
d�
}||kr|}q|d
kr�d|}d|}t�|�
}t|d��}
t	�|
�
}Wd�
n1s�0


Y
|||fS|ddfSdS)	zb
    A helper function that loads previously saved parameters and resets
    iteration start.
    r
zsaved_params_*.npy�
_��saved_params_%d.npy�saved_state_%d.pickle�rbN)
�glob�int�op�splitext�basename�split�np�load�open�pickle)�st�
f�iter�params_fileZ
state_file�params�state�r�5/Users/yimingwang/Desktop/cs224n/assignment/a2/sgd.py�load_saved_paramss

"







(

rcCsVd
|}t�
||
�
td|d�� }t�t��|�
Wd�
n1sH0


Y
dS)Nrr�wb)r
�saverr�dump�random�getstate)rrrrrrr�save_params"s



rF�
cCs�d
}|r@t�\}}	}
|dkr0|	}
|d||9}|
rDt
�|
�

nd}|
}|sTdd�}d}t|d|d�D]�}
d}||�
\}}|||8}||�
}|
|dkr�|s�|}nd|d	|}td
|
|f�

|
tdkr�|r�t|
|�
|
|dkrj|d9}qj|S)a� Stochastic Gradient Descent

    Implement the stochastic gradient descent method in this function.

    Arguments:
    f -- the function to optimize, it should take a single
         argument and yield two outputs, a loss and the gradient
         with respect to the arguments
    x0 -- the initial point to start SGD from
    step -- the step size for SGD
    iterations -- total iterations to run SGD for
    postprocessing -- postprocessing function for the parameters
                      if necessary. In the case of word2vec we will need to
                      normalize the word vectors to have unit length.
    PRINT_EVERY -- specifies how many iterations to output loss

    Return:
    x -- the parameter value after SGD finishes
    i Nr
��?c


Ss|S)
Nr�
�
xrrr�<lambda>P�zsgd.<locals>.<lambda>N�
gffffff�?g�������?ziter %d: %f)rr�setstate�range�print�SAVE_PARAMS_EVERYr)r�x0�stepZ
iterationsZpostprocessingZuseSaved�PRINT_EVERYZANNEAL_EVERYZ
start_iterZoldxrr#Zexplossr�loss�gradrrr�sgd)s8














r0cCs�d
d�}td�

t
|ddddd�}
td	|
�
t|
�
d
k
s<J�
t
|ddddd�}td|�
t|�
d
k
shJ�
t
|d
dddd�}td|�
t|�
d
k
s�J�
td�

td�

td�

dS)Nc

Sst�
|d
�
|d
fS)Nr)r
�sumr"rrrr$or%zsanity_check.<locals>.<lambda>zRunning sanity checks...r!g{�G�z�?i��d)
r-ztest 1 result:g�����ư>gztest 2 result:g��ztest 3 result:z(----------------------------------------zALL TESTS PASSED)r)r0�abs)�quad�t1�t2�t3rrr�sanity_checkns












r8�__main__)NFr )r*rrr�numpyr
�os.path�pathr	rrr0r8�__name__rrrr�<module>s




�
E