Path: blob/master/trees/__pycache__/tree.cpython-35.pyc
2585 views
X�YC � @ sw d d l Z Gd d � d � Z d d � Z d d � Z d d � Z d
d � Z d d
� Z d d d � Z e g Z d S)� Nc @ s^ e Z d Z d Z d d d d d d � Z d d � Z d
d � Z d d
� Z d d � Z d S)�Treea
Classification tree using information gain with entropy as impurity
Parameters
----------
max_features : int or None, default None
The number of features to consider when looking for the best split,
None uses all features
min_samples_split : int, default 10
The minimum number of samples required to split an internal node
max_depth : int, default 3
Maximum depth of the tree
minimum_gain : float, default 1e-7
Minimum information gain required for splitting
� NgH�����z>�
c C s( | | _ | | _ | | _ | | _ d S)N)� max_depth� max_features� minimum_gain�min_samples_split)�selfr r r r � r
�+/Users/ethen/machine-learning/trees/tree.py�__init__ s z
Tree.__init__c C s� t j | � j d | _ | j d k s> | j | j d k rN | j d | _ t j | j d � | _ t | | | j | j | j | j
| j | j | j d � | _ | j t j | j � _ | S)z4pass in the 2d-array dataset and the response columnr N� )
�np�unique�shape�n_classr �zeros�feature_importance�_create_decision_treer r r �tree�sum)r �X�yr
r
r �fit s % zTree.fitc C s( | j | � } t j | d d �} | S)N�axisr
)�
predict_probar �argmax)r r �proba�predr
r
r �predict3 s z Tree.predictc C sj t j | j d | j f � } xD t | j d � D]/ } | j | | d d � f | j � | | <q3 W| S)Nr )r �emptyr r �range� _predict_rowr )r r r �ir
r
r r 8 s -zTree.predict_probac C sV | d r | d S| | d | d k r> | j | | d � S| j | | d � Sd S)zPredict single row�is_leaf�prob� split_col� threshold�left�rightN)r" )r �rowr r
r
r r" ? s
zTree._predict_row) �__name__�
__module__� __qualname__�__doc__r r r r r" r
r
r
r r s r c C sc y� | d k s t � | j d | k s. t � t | | | � \ } }
} | | k s[ t � | | | j d | | 7<t | | | |
� \ } }
} } t | | | d | | | | | | � } t |
| | d | | | | | | � } WnN t k
r:t j | d | �} | | j d } d d d | i } | SYn Xd d d | d | d
| d |
i } | S) zIrecursively grow the decision tree until it reaches the stopping criteriar r
� minlengthr$ Tr% Fr( r) r& r' )�AssertionErrorr �_find_best_split�_splitr r �bincount)r r r r r r r r �n_row�column�value�gain�left_X�right_X�left_y�right_y�
left_child� right_child�countsr% �leaf�noder
r
r r J s4 !
r c
C s� t j j | j d | d d �} d \ } } } t | � } x� | D]� } t | | � } xh | D]` }
t | | | |
d d �} | t | | � } | d k s� | | k r` | |
| } } } q` WqD W| | | f S)z?Greedy algorithm to find the best feature and value for a splitr
�replaceFN�return_X)NNN)r �random�choicer �_compute_entropy� _find_splitsr2 �_compute_splits_entropy)
r r r �subset�max_col�max_valZmax_gainZparent_entropyr5 � split_valuesr6 �splitsr7 r
r
r r1 t s "
r1 c C sM t j | d d �\ } } | | j d } t j | t j | � � } | S)z$entropy score using a fix log base 2�
return_countsTr )r r r r �log2)�split�_r> �p�entropyr
r
r rE � s rE c C s� t j | d d � | f � } t j | j d d � } xF t d | j d � D]. } | | d | | d } | | | d <qP W| S)z�
find all possible split values (threshold),
by getting unique values in a sorted order
and finding cutoff point (average) between every two values
Nr r
� )r r r r r! )r r5 ZX_uniquerK r# �averager
r
r rF � s rF c C sA d } x4 | D], } | | j d | j d t | � 7} q
W| S)z8compute the entropy for the splits (the two child nodes)r )r rE )r rL Zsplits_entropyrO r
r
r rG � s
*rG Tc C s� | d d � | f | k } | d d � | f | k } | | | | } } | s] | | f S| | | | } }
| |
| | f Sd S)z4split the response column using the cutoff thresholdNr
) r r r5 r6 rB � left_mask�
right_maskr: r; r8 r9 r
r
r r2 � s
r2 )
�numpyr r r r1 rE rF rG r2 �__all__r
r
r
r �<module> s F *