Sliceable¶

This module support sliceable feature. Please note that this module will be removed after Chainer implements sliceable feature.

ConcatenatedDataset¶

class chainercv.chainer_experimental.datasets.sliceable.ConcatenatedDataset(*datasets)[source]¶

A sliceable version of chainer.datasets.ConcatenatedDataset.

Here is an example.

>>> dataset_a = TupleDataset([0, 1, 2], [0, 1, 4])
>>> dataset_b = TupleDataset([3, 4, 5], [9, 16, 25])
>>>
>>> dataset = ConcatenatedDataset(dataset_a, dataset_b)
>>> dataset.slice[:, 0][:]  # [0, 1, 2, 3, 4, 5]

Parameters: datasets – The underlying datasets. Each dataset should inherit Sliceabledataset and should have the same keys.

get_example_by_keys(index, key_indices)[source]¶

Return data of an example by keys

Parameters

index (int) – An index of an example.
key_indices (tuple of ints) – A tuple of indices of requested keys.

Returns

tuple of data

property keys¶

Return names of all keys

Returns: string or tuple of strings

GetterDataset¶

class chainercv.chainer_experimental.datasets.sliceable.GetterDataset[source]¶

A sliceable dataset class that is defined with getters.

This is a dataset class with getters. Please refer to the tutorial for more detailed explanation.

Here is an example.

>>> class SliceableLabeledImageDataset(GetterDataset):
>>>     def __init__(self, pairs, root='.'):
>>>         super(SliceableLabeledImageDataset, self).__init__()
>>>         with open(pairs) as f:
>>>             self._pairs = [l.split() for l in f]
>>>         self._root = root
>>>
>>>         self.add_getter('img', self.get_image)
>>>         self.add_getter('label', self.get_label)
>>>
>>>     def __len__(self):
>>>         return len(self._pairs)
>>>
>>>     def get_image(self, i):
>>>         path, _ = self._pairs[i]
>>>         return read_image(os.path.join(self._root, path))
>>>
>>>     def get_label(self, i):
>>>         _, label = self._pairs[i]
>>>         return np.int32(label)
>>>
>>> dataset = SliceableLabeledImageDataset('list.txt')
>>>
>>> # get a subset with label = 0, 1, 2
>>> # no images are loaded
>>> indices = [i for i, label in
...            enumerate(dataset.slice[:, 'label']) if label in {0, 1, 2}]
>>> dataset_012 = dataset.slice[indices]

add_getter(keys, getter)[source]¶

Register a getter function

Parameters

keys (string or tuple of strings) – The name(s) of data that the getter function returns.
getter (callable) – A getter function that takes an index and returns data of the corresponding example.

get_example_by_keys(index, key_indices)[source]¶

Return data of an example by keys

Parameters

index (int) – An index of an example.
key_indices (tuple of ints) – A tuple of indices of requested keys.

Returns

tuple of data

property keys¶

Return names of all keys

Returns: string or tuple of strings

TupleDataset¶

class chainercv.chainer_experimental.datasets.sliceable.TupleDataset(*datasets)[source]¶

A sliceable version of chainer.datasets.TupleDataset.

Here is an example.

>>> # omit keys
>>> dataset = TupleDataset([0, 1, 2], [0, 1, 4])
>>> dataset.keys  # (None, None)
>>> dataset.slice[:, 0][:]  # [0, 1, 2]
>>>
>>> dataset_more = TupleDataset(dataset, [0, 1, 8])
>>> dataset_more.keys  # (None, None, None)
>>> dataset_more.slice[:, [1, 2]][:]  # [(0, 0), (1, 1), (4, 8)]
>>>
>>> # specify the name of a key
>>> named_dataset = TupleDataset(('feat0', [0, 1, 2]), [0, 1, 4])
>>> named_dataset.keys  # ('feat0', None)
>>> # slice takes both key and index (or their mixture)
>>> named_dataset.slice[:, ['feat0', 1]][:]  # [(0, 0), (1, 1), (2, 4)]

Parameters

datasets –

The underlying datasets. The following datasets are acceptable.

An inheritance of :class:~chainer.datasets.sliceable.SliceableDataset`.
A tuple of a name and a data array. The data array should be list or numpy.ndarray.
A data array. In this case, the name of key is None.

get_example_by_keys(index, key_indices)[source]¶

Return data of an example by keys

Parameters

index (int) – An index of an example.
key_indices (tuple of ints) – A tuple of indices of requested keys.

Returns

tuple of data

property keys¶

Return names of all keys

Returns: string or tuple of strings

TransformDataset¶

class chainercv.chainer_experimental.datasets.sliceable.TransformDataset(dataset, keys, transform=None)[source]¶

A sliceable version of chainer.datasets.TransformDataset.

Note that it reuqires keys to determine the names of returned values.

Here is an example.

>>> def transfrom(in_data):
>>>     img, bbox, label = in_data
>>>     ...
>>>     return new_img, new_label
>>>
>>> dataset = TramsformDataset(dataset, ('img', 'label'), transform)
>>> dataset.keys  # ('img', 'label')

Parameters

dataset – The underlying dataset. This dataset should have __len__() and __getitem__().
keys (string or tuple of strings) – The name(s) of data that the transform function returns. If this parameter is omitted, __init__() fetches a sample from the underlying dataset to determine the number of data.
transform (callable) – A function that is called to transform values returned by the underlying dataset’s __getitem__().