Shortcuts

DataSource

class flash.core.data.data_source.DataSource[source]

The DataSource class encapsulates two hooks: load_data and load_sample.

The to_datasets() method can then be used to automatically construct data sets from the hooks.

generate_dataset(data, running_stage)[source]

Generate a single dataset with the given input to load_data() for the given running_stage.

Parameters
  • data (Optional[~DATA_TYPE]) – The input to load_data() to use to create the dataset.

  • running_stage (RunningStage) – The running_stage for this dataset.

Return type

Union[AutoDataset, IterableAutoDataset, None]

Returns

The constructed BaseAutoDataset.

static load_data(data, dataset=None)[source]

Given the data argument, the load_data hook produces a sequence or iterable of samples or sample metadata. The data argument can be anything, but this method should return a sequence or iterable of mappings from string (e.g. “input”, “target”, “bbox”, etc.) to data (e.g. a target value) or metadata (e.g. a filename). Where possible, any heavy data loading should be performed in load_sample(). If the output is an iterable rather than a sequence (that is, it doesn’t have length) then the generated dataset will be an IterableDataset.

Parameters
  • data (~DATA_TYPE) – The data required to load the sequence or iterable of samples or sample metadata.

  • dataset (Optional[Any]) – Overriding methods can optionally include the dataset argument. Any attributes set on the dataset (e.g. num_classes) will also be set on the generated dataset.

Return type

Union[Sequence[Mapping[str, Any]], Iterable[Mapping[str, Any]]]

Returns

A sequence or iterable of samples or sample metadata to be used as inputs to load_sample().

Example:

# data: "."
# output: [{"input": "./cat/1.png", "target": 1}, ..., {"input": "./dog/10.png", "target": 0}]

output: Sequence[Mapping[str, Any]] = load_data(data)
static load_sample(sample, dataset=None)[source]

Given an element from the output of a call to load_data(), this hook should load a single data sample. The keys and values in the sample argument will be same as the keys and values in the outputs of load_data().

Parameters
  • sample (Mapping[str, Any]) – An element (sample or sample metadata) from the output of a call to load_data().

  • dataset (Optional[Any]) – Overriding methods can optionally include the dataset argument. Any attributes set on the dataset (e.g. num_classes) will also be set on the generated dataset.

Return type

Any

Returns

The loaded sample as a mapping with string keys (e.g. “input”, “target”) that can be processed by the pre_tensor_transform().

Example:

# sample: {"input": "./cat/1.png", "target": 1}
# output: {"input": PIL.Image, "target": 1}

output: Mapping[str, Any] = load_sample(sample)
to_datasets(train_data=None, val_data=None, test_data=None, predict_data=None)[source]

Construct data sets (of type BaseAutoDataset) from this data source by calling load_data() with each of the *_data arguments. If an argument is given as None then no dataset will be created for that stage (train, val, test, predict).

Parameters
Return type

Tuple[Optional[BaseAutoDataset], …]

Returns

A tuple of train_dataset, val_dataset, test_dataset, predict_dataset. If any *_data argument is not passed to this method then the corresponding *_dataset will be None.

Read the Docs v: latest
Versions
latest
stable
0.5.0
0.4.0
0.3.2
0.3.1
0.3.0
0.2.3
0.2.2
0.2.1
0.2.0
0.1.0post1
Downloads
pdf
html
On Read the Docs
Project Home
Builds

Free document hosting provided by Read the Docs.