VideoClassifier

class flash.video.classification.model.VideoClassifier(num_classes=None, labels=None, backbone='x3d_xs', backbone_kwargs=None, pretrained=True, loss_fn=torch.nn.functional.cross_entropy, optimizer='Adam', lr_scheduler=None, metrics=torchmetrics.Accuracy, learning_rate=None, head=None)

Task that classifies videos.

Parameters

• num_classes (Optional[int]) – Number of classes to classify.

• backbone (Union[str, Module]) – A string mapped to pytorch_video backbones or nn.Module, defaults to "slowfast_r50".

• backbone_kwargs (Optional[Dict]) – Arguments to customize the backbone from PyTorchVideo.

• pretrained (bool) – Use a pretrained backbone, defaults to True.

• loss_fn (TypeVar(LOSS_FN_TYPE, Callable, Mapping, Sequence, None)) – Loss function for training, defaults to torch.nn.functional.cross_entropy().

• optimizer (TypeVar(OPTIMIZER_TYPE, str, Callable, Tuple[str, Dict[str, Any]], None)) – Optimizer to use for training, defaults to torch.optim.SGD.

• lr_scheduler (Optional[TypeVar(LR_SCHEDULER_TYPE, str, Callable, Tuple[str, Dict[str, Any]], Tuple[str, Dict[str, Any], Dict[str, Any]], None)]) – The scheduler or scheduler class to use.

• metrics (TypeVar(METRICS_TYPE, Metric, Mapping, Sequence, None)) – Metrics to compute for training and evaluation. Can either be an metric from the torchmetrics package, a custom metric inherenting from torchmetrics.Metric, a callable function or a list/dict containing a combination of the aforementioned. In all cases, each metric needs to have the signature metric(preds,target) and return a single scalar tensor. Defaults to torchmetrics.Accuracy.

• learning_rate (Optional[float]) – Learning rate to use for training, defaults to 1e-3.

• head (Union[function, Module, None]) – either a nn.Module or a callable function that converts the features extrated from the backbone into class log probabilities (assuming default loss function). If None, will default to using a single linear layer.

classmethod available_finetuning_strategies(cls)

Returns a list containing the keys of the available Finetuning Strategies.

Return type

List[str]

classmethod available_lr_schedulers(cls)

Returns a list containing the keys of the available LR schedulers.

Return type

List[str]

classmethod available_optimizers(cls)

Returns a list containing the keys of the available Optimizers.

Return type

List[str]

modules_to_freeze()

Return the module attributes of the model to be frozen.

Return type

Union[Module, Iterable[Union[Module, Iterable]]]