Datasets¶
DirectoryParsingLabelDataset¶
-
class
chainercv.datasets.
DirectoryParsingLabelDataset
(root, check_img_file=None, color=True, numerical_sort=False)¶ A label dataset whose label names are the names of the subdirectories.
The label names are the names of the directories that locate a layer below the root directory. All images locating under the subdirectoies will be categorized to classes with subdirectory names. An image is parsed only when the function
check_img_file
returnsTrue
by taking the path to the image as an argument. Ifcheck_img_file
isNone
, the path with any image extensions will be parsed.Example
A directory structure should be one like below.
root |-- class_0 | |-- img_0.png | |-- img_1.png | --- class_1 |-- img_0.png
>>> from chainercv.datasets import DirectoryParsingLabelDataset >>> dataset = DirectoryParsingLabelDataset('root') >>> dataset.paths ['root/class_0/img_0.png', 'root/class_0/img_1.png', 'root_class_1/img_0.png'] >>> dataset.labels array([0, 0, 1])
Parameters: - root (str) – The root directory.
- check_img_file (callable) – A function to determine if a file should be included in the dataset.
- color (bool) – If
True
, this dataset read images as color images. - numerical_sort (bool) – Label names are sorted numerically.
This means that label
2
is before label10
, which is not the case when string sort is used. Regardless of this option, string sort is used for the order of files with the same label. The default value isFalse
.
directory_parsing_label_names¶
-
chainercv.datasets.
directory_parsing_label_names
(root, numerical_sort=False)¶ Get label names from the directories that are named by them.
The label names are the names of the directories that locate a layer below the root directory.
The label names can be used together with
chainercv.datasets.DirectoryParsingLabelDataset
. The index of a label name corresponds to the label id that is used by the dataset to refer the label.Parameters: - Retruns:
- list of strings: Sorted names of classes.
TransformDataset¶
TransformDataset¶
-
class
chainercv.datasets.
TransformDataset
(dataset, transform)¶ Dataset that indexes data of a base dataset and transforms it.
This dataset wraps a base dataset by modifying the behavior of the base dataset’s
__getitem__()
. Arrays returned by__getitem__()
of the base dataset with an integer index are transformed by the given functiontransform
.The function
transform
takes, as an argument,in_data
, which is output of the base dataset’s__getitem__()
, and returns transformed arrays as output. Please see the following example.>>> from chainer.datasets import get_mnist >>> from chainercv.datasets import TransformDataset >>> dataset, _ = get_mnist() >>> def transform(in_data): >>> img, label = in_data >>> img -= 0.5 # scale to [-0.5, -0.5] >>> return img, label >>> dataset = TransformDataset(dataset, transform)
Note
The index used to access data is either an integer or a slice. If it is a slice, the base dataset is assumed to return a list of outputs each corresponding to the output of the integer indexing.
Note
This class is deprecated. Please use
chainer.datasets.TransformDataset
instead.Parameters: - dataset – Underlying dataset. The index of this dataset corresponds to the index of the base dataset.
- transform (callable) – A function that is called to transform values
returned by the underlying dataset’s
__getitem__()
.
ADE20K¶
ADE20KSemanticSegmentationDataset¶
-
class
chainercv.datasets.
ADE20KSemanticSegmentationDataset
(data_dir='auto', split='train')¶ Semantic segmentation dataset for ADE20K.
This is ADE20K dataset distributed in MIT Scene Parsing Benchmark website. It has 20,210 training images and 2,000 validation images.
Parameters: - data_dir (string) – Path to the dataset directory. The directory should
contain the
ADEChallengeData2016
directory. And that directory should contain at leastimages
andannotations
directries. Ifauto
is given, the dataset is automatically downloaded into$CHAINER_DATASET_ROOT/pfnet/chainercv/ade20k
. - split ({'train', 'val'}) – Select from dataset splits used in MIT Scene Parsing Benchmark dataset (ADE20K).
- data_dir (string) – Path to the dataset directory. The directory should
contain the
ADE20KTestImageDataset¶
-
class
chainercv.datasets.
ADE20KTestImageDataset
(data_dir='auto')¶ Image dataset for test split of ADE20K.
This is an image dataset of test split in ADE20K dataset distributed at MIT Scene Parsing Benchmark website. It has 3,352 test images.
Parameters: data_dir (string) – Path to the dataset directory. The directory should contain the release_test
dir. Ifauto
is given, the dataset is automatically downloaded into$CHAINER_DATASET_ROOT/pfnet/chainercv/ade20k
.
CamVid¶
CamVidDataset¶
-
class
chainercv.datasets.
CamVidDataset
(data_dir='auto', split='train')¶ Semantic segmentation dataset for CamVid u.
Parameters: - data_dir (string) – Path to the root of the training data. If this is
auto
, this class will automatically download data for you under$CHAINER_DATASET_ROOT/pfnet/chainercv/camvid
. - split ({'train', 'val', 'test'}) – Select from dataset splits used in CamVid Dataset.
- data_dir (string) – Path to the root of the training data. If this is
CityscapesSemanticSegmentationDataset¶
-
class
chainercv.datasets.
CityscapesSemanticSegmentationDataset
(data_dir=None, label_resolution=None, split='train', ignore_labels=True)¶ Semantic segmentation dataset for Cityscapes dataset.
Note
Please manually downalod the data because it is not allowed to re-distribute Cityscapes dataset.
Parameters: - data_dir (string) – Path to the dataset directory. The directory should
contain at least two directories,
leftImg8bit
and eithergtFine
orgtCoarse
. IfNone
is given, it uses$CHAINER_DATSET_ROOT/pfnet/chainercv/cityscapes
by default. - label_resolution ({'fine', 'coarse'}) – The resolution of the labels. It
should be either
fine
orcoarse
. - split ({'train', 'val'}) – Select from dataset splits used in Cityscapes dataset.
- ignore_labels (bool) – If True, the labels marked
ignoreInEval
defined in the original cityscapesScripts<https://github.com/mcordts/cityscapesScripts>_ will be replaced with-1
in theget_example()
method. The default value isTrue
.
- data_dir (string) – Path to the dataset directory. The directory should
contain at least two directories,
CUB¶
CUBLabelDataset¶
-
class
chainercv.datasets.
CUBLabelDataset
(data_dir='auto', return_bb=False, prob_map_dir='auto', return_prob_map=False)¶ Caltech-UCSD Birds-200-2011 dataset with annotated class labels.
When queried by an index, this dataset returns a corresponding
img, label
, a tuple of an image and class id. The image is in RGB and CHW format. The class id are between 0 and 199. Ifreturn_bb = True
, a bounding boxbb
is appended to the tuple. Ifreturn_prob_map = True
, a probability mapprob_map
is appended.A bounding box is a one-dimensional array of shape \((4,)\). The elements of the bounding box corresponds to
(y_min, x_min, y_max, x_max)
, where the four attributes are coordinates of the top left and the bottom right vertices. This information can optionally be retrieved from the dataset by settingreturn_bb = True
.The probability map of a bird shows how likely the bird is located at each pixel. If the value is close to 1, it is likely that the bird locates at that pixel. The shape of this array is \((H, W)\), where \(H\) and \(W\) are height and width of the image respectively. This information can optionally be retrieved from the dataset by setting
return_prob_map = True
.Parameters: - data_dir (string) – Path to the root of the training data. If this is
auto
, this class will automatically download data for you under$CHAINER_DATASET_ROOT/pfnet/chainercv/cub
. - return_bb (bool) – If
True
, this returns a bounding box around a bird. The default value isFalse
. - prob_map_dir (string) – Path to the root of the probability maps.
If this is
auto
, this class will automatically download data for you under$CHAINER_DATASET_ROOT/pfnet/chainercv/cub
. - return_prob_map (bool) – Decide whether to include a probability map of
the bird in a tuple served for a query. The default value is
False
.
- data_dir (string) – Path to the root of the training data. If this is
CUBKeypointDataset¶
-
class
chainercv.datasets.
CUBKeypointDataset
(data_dir='auto', return_bb=False, prob_map_dir='auto', return_prob_map=False)¶ Caltech-UCSD Birds-200-2011 dataset with annotated keypoints.
An index corresponds to each image.
When queried by an index, this dataset returns the corresponding
img, keypoint, kp_mask
, a tuple of an image, keypoints and a keypoint mask that indicates visible keypoints in the image. The data type of the three elements arefloat32, float32, bool
. Ifreturn_bb = True
, a bounding boxbb
is appended to the tuple. Ifreturn_prob_map = True
, a probability mapprob_map
is appended.keypoints are packed into a two dimensional array of shape \((K, 2)\), where \(K\) is the number of keypoints. Note that \(K=15\) in CUB dataset. Also note that not all fifteen keypoints are visible in an image. When a keypoint is not visible, the values stored for that keypoint are undefined. The second axis corresponds to the \(y\) and \(x\) coordinates of the keypoints in the image.
A keypoint mask array indicates whether a keypoint is visible in the image or not. This is a boolean array of shape \((K,)\).
A bounding box is a one-dimensional array of shape \((4,)\). The elements of the bounding box corresponds to
(y_min, x_min, y_max, x_max)
, where the four attributes are coordinates of the top left and the bottom right vertices. This information can optionally be retrieved from the dataset by settingreturn_bb = True
.The probability map of a bird shows how likely the bird is located at each pixel. If the value is close to 1, it is likely that the bird locates at that pixel. The shape of this array is \((H, W)\), where \(H\) and \(W\) are height and width of the image respectively. This information can optionally be retrieved from the dataset by setting
return_prob_map = True
.Parameters: - data_dir (string) – Path to the root of the training data. If this is
auto
, this class will automatically download data for you under$CHAINER_DATASET_ROOT/pfnet/chainercv/cub
. - return_bb (bool) – If
True
, this returns a bounding box around a bird. The default value isFalse
. - prob_map_dir (string) – Path to the root of the probability maps.
If this is
auto
, this class will automatically download data for you under$CHAINER_DATASET_ROOT/pfnet/chainercv/cub
. - return_prob_map (bool) – Decide whether to include a probability map of
the bird in a tuple served for a query. The default value is
False
.
- data_dir (string) – Path to the root of the training data. If this is
OnlineProducts¶
OnlineProductsDataset¶
-
class
chainercv.datasets.
OnlineProductsDataset
(data_dir='auto', split='train')¶ Dataset class for Stanford Online Products Dataset.
When queried by an index, this dataset returns a corresponding
img, class_id, super_class_id
, a tuple of an image, a class id and a coarse level class id. Images are in RGB and CHW format. Class ids start from 0. The name of the \(l\) th coarse level class is \(l\) th element ofchainercv.datasets.online_products_super_label_names
.The
split
selects train and test split of the dataset as done in [1]. The train split contains the first 11318 classes and the test split contains the remaining 11316 classes.[1] Hyun Oh Song, Yu Xiang, Stefanie Jegelka, Silvio Savarese. Deep Metric Learning via Lifted Structured Feature Embedding. arXiv 2015. Parameters: - data_dir (string) – Path to the root of the training data. If this is
auto
, this class will automatically download data for you under$CHAINER_DATASET_ROOT/pfnet/chainercv/online_products
. - split ({'train', 'test'}) – Select a split of the dataset.
- data_dir (string) – Path to the root of the training data. If this is
PASCAL VOC¶
VOCBboxDataset¶
-
class
chainercv.datasets.
VOCBboxDataset
(data_dir='auto', split='train', year='2012', use_difficult=False, return_difficult=False)¶ Bounding box dataset for PASCAL VOC.
The index corresponds to each image.
When queried by an index, if
return_difficult == False
, this dataset returns a correspondingimg, bbox, label
, a tuple of an image, bounding boxes and labels. This is the default behaviour. Ifreturn_difficult == True
, this dataset returns correspondingimg, bbox, label, difficult
.difficult
is a boolean array that indicates whether bounding boxes are labeled as difficult or not.The bounding boxes are packed into a two dimensional tensor of shape \((R, 4)\), where \(R\) is the number of bounding boxes in the image. The second axis represents attributes of the bounding box. They are
(y_min, x_min, y_max, x_max)
, where the four attributes are coordinates of the top left and the bottom right vertices.The labels are packed into a one dimensional tensor of shape \((R,)\). \(R\) is the number of bounding boxes in the image. The class name of the label \(l\) is \(l\) th element of
chainercv.datasets.voc_bbox_label_names
.The array
difficult
is a one dimensional boolean array of shape \((R,)\). \(R\) is the number of bounding boxes in the image. Ifuse_difficult
isFalse
, this array is a boolean array with allFalse
.The type of the image, the bounding boxes and the labels are as follows.
img.dtype == numpy.float32
bbox.dtype == numpy.float32
label.dtype == numpy.int32
difficult.dtype == numpy.bool
Parameters: - data_dir (string) – Path to the root of the training data. If this is
auto
, this class will automatically download data for you under$CHAINER_DATASET_ROOT/pfnet/chainercv/voc
. - split ({'train', 'val', 'trainval', 'test'}) – Select a split of the
dataset.
test
split is only available for 2007 dataset. - year ({'2007', '2012'}) – Use a dataset prepared for a challenge
held in
year
. - use_difficult (bool) – If true, use images that are labeled as difficult in the original annotation.
- return_difficult (bool) – If true, this dataset returns a boolean array
that indicates whether bounding boxes are labeled as difficult
or not. The default value is
False
.
VOCSemanticSegmentationDataset¶
-
class
chainercv.datasets.
VOCSemanticSegmentationDataset
(data_dir='auto', split='train')¶ Semantic segmentation dataset for PASCAL VOC2012.
The class name of the label \(l\) is \(l\) th element of
chainercv.datasets.voc_semantic_segmentation_label_names
.Parameters: - data_dir (string) – Path to the root of the training data. If this is
auto
, this class will automatically download data for you under$CHAINER_DATASET_ROOT/pfnet/chainercv/voc
. - split ({'train', 'val', 'trainval'}) – Select a split of the dataset.
- data_dir (string) – Path to the root of the training data. If this is