bleu.cpython-36.pyc

3

��?Z
�@s*dZd
dl
Z
d
dlZdd�Zd	dd�
ZdS)
a;
Python implementation of BLEU and smooth-BLEU.
This module provides a Python implementation of BLEU and smooth-BLEU.
Smooth BLEU is computed following the method outlined in the paper:
Chin-Yew Lin, Franz Josef Och. ORANGE: a method for evaluating automatic
evaluation metrics for machine translation. COLING 2004.
�NcCshtj
�}xZtd
|
d
�D]H}xBtdt|�
|d
�D](}t||||��
}||d
7<q4WqW|S)a
Extracts all n-grams upto a given maximum order from an input segment.
    Args:
      segment: text segment from which n-grams will be extracted.
      max_order: maximum length in tokens of the n-grams returned by this
          methods.
    Returns:
      The Counter containing all n-grams upto max_order in segment
      with a count of how many times each n-gram occurred.
    �
r
)�collections�Counter�range�len�tuple)�segment�	max_orderZngram_counts�order�
i�ngram�r
�;/home/jovyan/work/Week 3/Neural Machine Translation/bleu.py�_get_ngramss





r�Fcsd
g
�}d
g
�}d
}d
}x�t||
�D]�\}}	|t
dd�|D�
�
7}|t|	�
7}tj�}
x|D]}|
t|��O}
q`Wt|	��}||
@}
x(|
D] }|t|�
d|
|7<q�WxBtd�d�D]0}t|	�
|d}|d
kr�||d|7<q�Wq(Wd
g
�}xhtd
��D]Z}|�
r8||d||d||<n0||d
k�
r`t||�
||||<nd||<�
qWt
|�
d
k�
r�t�f
dd�|D�
�
}t	j
|�
}nd
}|d
k�
r�t|�
|nd}|dk�
r�d}n |d
k�
r�t	j
dd|�
nd}||}||||||fS)aqComputes BLEU score of translated segments against one or more references.
    Args:
      reference_corpus: list of lists of references for each translation. Each
          reference should be tokenized into a list of tokens.
      translation_corpus: list of translations to score. Each translation
          should be tokenized into a list of tokens.
      max_order: Maximum n-gram order to use when computing BLEU score.
      smooth: Whether or not to apply Lin et al. 2004 smoothing.
    Returns:
      3-Tuple with the BLEU score, n-gram precisions, geometric mean of n-gram
      precisions and brevity penalty.
    r
c
ss|]}
t|
�
V
qdS)
N)
r)�.0�
rr
r
r�	<genexpr>Aszcompute_bleu.<locals>.<genexpr>rg�?gc
3s |]}
d�tj
|
�
V
qd
S)g�?N)�math�log)r�
p)
r	r
rr]s)�zip�minrrrrr�float�sumr�exp)Zreference_corpusZtranslation_corpusr	�smoothZmatches_by_orderZpossible_matches_by_orderZreference_lengthZtranslation_lengthZ
references�translationZmerged_ref_ngram_counts�	referenceZtranslation_ngram_counts�overlaprr
�possible_matchesZ
precisionsrZ	p_log_sumZgeo_mean�ratio�bp�bleur
)
r	r�compute_bleu-sL

















 
















 r$)rF)�__doc__rrrr$r
r
r
r�<module>s