8.1.1.3.1.8. blueoil.datasets.dataset_iterator

8.1.1.3.1.8.1. Module Contents

8.1.1.3.1.8.1.1. Classes

_MultiProcessDatasetPrefetchThread

A class that represents a thread of control.

_SimpleDatasetReader

_TFDSReader

DatasetIterator

8.1.1.3.1.8.1.2. Functions

_prefetch_setup(dataset)

_apply_augmentations(dataset, image, label)

_process_one_data(i)

_concat_data(data_list)

_xorshift32(r)

_generate_tfds_map_func(dataset)

Return callable object

blueoil.datasets.dataset_iterator._dataset
blueoil.datasets.dataset_iterator._prefetch_setup(dataset)
blueoil.datasets.dataset_iterator._apply_augmentations(dataset, image, label)
blueoil.datasets.dataset_iterator._process_one_data(i)
blueoil.datasets.dataset_iterator._concat_data(data_list)
blueoil.datasets.dataset_iterator._xorshift32(r)
class blueoil.datasets.dataset_iterator._MultiProcessDatasetPrefetchThread(dataset, result_queue, seed)

Bases: threading.Thread

A class that represents a thread of control.

This class can be safely subclassed in a limited fashion. There are two ways to specify the activity: by passing a callable object to the constructor, or by overriding the run() method in a subclass.

gen_ids(self)
gen_task(self, task_batch_size)
chunks(self, l, n)

Yield successive n-sized chunks from l.

refresh_pool(self)
loop_body(self)
run(self)

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

class blueoil.datasets.dataset_iterator._SimpleDatasetReader(dataset, seed, shuffle=True)
_gen_ids(self, size)

Generate ids which length is size.

read(self)

Return batch size data.

blueoil.datasets.dataset_iterator._generate_tfds_map_func(dataset)

Return callable object

class blueoil.datasets.dataset_iterator._TFDSReader(dataset, local_rank)
read(self)

Return batch size data.

class blueoil.datasets.dataset_iterator.DatasetIterator(dataset, enable_prefetch=False, seed=0, local_rank=-1)
available_subsets = ['train', 'validation']

docstring for DatasetIterator.

property num_per_epoch(self)
property classes(self)
property num_classes(self)
property num_max_boxes(self)
property label_colors(self)
extend_dir(self)
__iter__(self)
__next__(self)
feed(self)
__len__(self)
update_dataset(self, indices)

Update own dataset by indices.

get_shuffle_index(self)

Return list of shuffled index.

close(self)
blueoil.datasets.dataset_iterator.cifar10