dataset.cpython-36.pyc

3

��Y�%�@s@dd
lZ
dd
lZddlmZ
d	dd�
ZGdd�d�Zdd�Zd
S)
�N)
�cachecCs(|
d
krtj
|�
d}
tj|
td�|S)a�

    Generate the One-Hot encoded class-labels from an array of integers.
    For example, if class_number=2 and num_classes=4 then
    the one-hot encoded label is the float array: [0. 0. 1. 0.]
    :param class_numbers:
        Array of integers with class-numbers.
        Assume the integers are from zero to num_classes-1 inclusive.
    :param num_classes:
        Number of classes. If None then use max(class_numbers)+1.
    :return:
        2-dim array of shape: [len(class_numbers), num_classes]
    N�
)
�dtype)�np�max�eye�float)�
class_numbers�num_classes�r�#/home/jovyan/work/Resnet/dataset.py�one_hot_encodeds
r
c@s8eZ
dZddd�
Zdd�Zddd�
Zd	d
�Zdd�Zd
S)�DataSet�.jpgc	Cs
tj
j|
�
}
|
|_td
d�|D�
�
|_g|_g|_g|_g|_	g|_
d|_x�tj|
�
D]�}tj
j
|
|�}tj
j|�
rV|jj|�

|j|�
}|jj|�

|j}|g
t|�
}|j	j|�

|jtj
j
|d��
}|jj|�

|g
t|�
}|j
j|�

|jd7_qVWdS)a�
        Create a data-set consisting of the filenames in the given directory
        and sub-dirs that match the given filename-extensions.
        For example, the knifey-spoony data-set (see knifey.py) has the
        following dir-structure:
        knifey-spoony/forky/
        knifey-spoony/knifey/
        knifey-spoony/spoony/
        knifey-spoony/forky/test/
        knifey-spoony/knifey/test/
        knifey-spoony/spoony/test/
        This means there are 3 classes called: forky, knifey, and spoony.
        If we set in_dir = "knifey-spoony/" and create a new DataSet-object
        then it will scan through these directories and create a training-set
        and test-set for each of these classes.
        The training-set will contain a list of all the *.jpg filenames
        in the following directories:
        knifey-spoony/forky/
        knifey-spoony/knifey/
        knifey-spoony/spoony/
        The test-set will contain a list of all the *.jpg filenames
        in the following directories:
        knifey-spoony/forky/test/
        knifey-spoony/knifey/test/
        knifey-spoony/spoony/test/
        See the TensorFlow Tutorial #09 for a usage example.
        :param in_dir:
            Root-dir for the files in the data-set.
            This would be 'knifey-spoony/' in the example above.
        :param exts:
            String or tuple of strings with valid filename-extensions.
            Not case-sensitive.
        :return:
            Object instance.
        c
ss|]}
|
j�V
qdS)
N)
�lower)�.0�extrrr�	<genexpr>bsz#DataSet.__init__.<locals>.<genexpr>r
�testrN)�os�path�abspath�in_dir�tuple�exts�class_names�	filenames�filenames_testr	�class_numbers_testr
�listdir�join�isdir�append�_get_filenames�extend�len)	�selfrr�nameZcurrent_dirrZclass_numberr	rrrr�__init__6s.&
zDataSet.__init__cCsBg}tj
j|
�
r>x,tj|
�
D]}|j�j|j�
r|j|�

qW|S)
a

        Create and return a list of filenames with matching extensions in the given directory.
        :param dir:
            Directory to scan for files. Sub-dirs are not scanned.
        :return:
            List of filenames. Only filenames. Does not include the directory.
        )rr�existsrr�endswithrr")r&�dirr�filenamerrrr#�s


zDataSet._get_filenamesFccsb|
r|j}|j
}d
}n|j}|j}d}x6t||�D](\}}tjj|j|j	|||�}|V
q2WdS)z�
        Get the full paths for the files in the data-set.
        :param test:
            Boolean. Return the paths for the test-set (True) or training-set (False).
        :return:
            Iterator with strings for the path-names.
        ztest/�N)
rrrr	�ziprrr rr)r&rrr	Ztest_dirr,�clsrrrr�	get_paths�s	

zDataSet.get_pathsc

Cs&t|j
��
tj|j�
t|j|jd
�fS)z�
        Return the list of paths for the files in the training-set,
        and the list of class-numbers as integers,
        and the class-numbers as one-hot encoded arrays.
        )r	r
)�listr0r�asarrayr	r
r
)
r&rrr�get_training_set�s




zDataSet.get_training_setc

Cs*t|j
d
d�
�
tj|j�
t|j|jd�fS)z�
        Return the list of paths for the files in the test-set,
        and the list of class-numbers as integers,
        and the class-numbers as one-hot encoded arrays.
        T)
r)r	r
)r1r0rr2rr
r
)
r&rrr�get_test_set�s



zDataSet.get_test_setN)
r)
F)�__name__�
__module__�__qualname__r(r#r0r3r4rrrrr5s


l
rcCstd
|
�

t
|t|
d�}|S)a�
    Wrapper-function for creating a DataSet-object, which will be
    loaded from a cache-file if it already exists, otherwise a new
    object will be created and saved to the cache-file.
    This is useful if you need to ensure the ordering of the
    filenames is consistent every time you load the data-set,
    for example if you use the DataSet-object in combination
    with Transfer Values saved to another cache-file, see e.g.
    Tutorial #09 for an example of this.
    :param cache_path:
        File-path for the cache-file.
    :param in_dir:
        Root-dir for the files in the data-set.
        This is an argument for the DataSet-init function.
    :return:
        The DataSet-object.
    z$Creating dataset from the files in: )�
cache_path�fnr)�printrr)r8r�datasetrrr�load_cached�s

r<)
N)�numpyrrrr
rr<rrrr�<module>s


=