Path: blob/master/trees/__pycache__/tree.cpython-36.pyc
2585 views
3
ţ�[C � @ sR d dl ZG dd� d�Zdd� Zdd� Zdd � Zd
d � Zd d
� Zddd�ZegZ dS )� Nc @ s: e Zd ZdZddd�Zdd � Zd
d � Zd d
� Zdd� ZdS )�Treea
Classification tree using information gain with entropy as impurity
Parameters
----------
max_features : int or None, default None
The number of features to consider when looking for the best split,
None uses all features
min_samples_split : int, default 10
The minimum number of samples required to split an internal node
max_depth : int, default 3
Maximum depth of the tree
minimum_gain : float, default 1e-7
Minimum information gain required for splitting
� N�H�����z>�
c C s || _ || _|| _|| _d S )N)� max_depth� max_features� minimum_gain�min_samples_split)�selfr r r r � r �//Users/mingyuliu/machine-learning/trees/tree.py�__init__ s z
Tree.__init__c C s� t j|�jd | _| jdks,| j|jd kr8|jd | _t j|jd �| _t||| j| j | j| j
| j| j|jd � | _ | jt j | j� _| S )z4pass in the 2d-array dataset and the response columnr N� )
�np�unique�shape�n_classr �zeros�feature_importance�_create_decision_treer r r �tree�sum)r
�X�yr r r �fit s
zTree.fitc C s | j |�}tj|dd�}|S )Nr )�axis)�
predict_probar �argmax)r
r �proba�predr r r �predict3 s
z Tree.predictc C sP t j|jd | jf�}x4t|jd �D ]"}| j||d d �f | j�||< q&W |S )Nr )r �emptyr r �range� _predict_rowr )r
r r �ir r r r 8 s "zTree.predict_probac C sH |d r|d S ||d |d kr4| j ||d �S | j ||d �S dS )zPredict single row�is_leaf�prob� split_col� threshold�left�rightN)r# )r
�rowr r r r r# ? s
zTree._predict_row)r Nr r ) �__name__�
__module__� __qualname__�__doc__r
r r r r# r r r r r s
r c C s� y�|dkst �| jd |ks t �t| ||�\} }
} | |ks>t �|| | jd | | 7 < t| || |
�\} }
}}t| ||d ||||||� }t|
||d ||||||� }W n: t k
r� tj||d�}||jd }d|d�}|S X d||| |
d�}|S )zIrecursively grow the decision tree until it reaches the stopping criteriar r )� minlengthT)r% r& F)r% r) r* r' r( )�AssertionErrorr �_find_best_split�_splitr r �bincount)r r r r r r r r �n_row�column�value�gain�left_X�right_X�left_y�right_y�
left_child� right_child�countsr& �leaf�noder r r r J s4
r c
C s� t jj| jd |dd�}d\}}}t|�}xd|D ]\}t| |�} xL| D ]D}
t| |||
dd�} |t|| � } |dksx| |krD||
| }}}qDW q0W |||fS )z?Greedy algorithm to find the best feature and value for a splitr F)�replaceN)�return_X)NNN)r �random�choicer �_compute_entropy� _find_splitsr3 �_compute_splits_entropy)
r r r �subset�max_col�max_valZmax_gainZparent_entropyr6 � split_valuesr7 �splitsr8 r r r r2 t s
r2 c C s: t j| dd�\}}|| jd }t j|t j|� � }|S )z$entropy score using a fix log base 2T)�
return_countsr )r r r r �log2)�split�_r? �p�entropyr r r rF � s rF c C sl t j| dd�|f �}t j|jd d �}x<td|jd �D ](}||d || d }|||d < q<W |S )z�
find all possible split values (threshold),
by getting unique values in a sorted order
and finding cutoff point (average) between every two values
Nr r � )r r r! r r" )r r6 ZX_uniquerL r$ �averager r r rG � s rG c C s6 d}x,|D ]$}||j d | j d t|� 7 }q
W |S )z8compute the entropy for the splits (the two child nodes)r )r rF )r rM Zsplits_entropyrP r r r rH � s
$rH Tc C sh | dd�|f |k}| dd�|f |k}|| || }}|sF||fS | | | | } }
| |
||fS dS )z4split the response column using the cutoff thresholdNr ) r r r6 r7 rC � left_mask�
right_maskr; r< r9 r: r r r r3 � s r3 )T)
�numpyr r r r2 rF rG rH r3 �__all__r r r r �<module> s F*