This layer provides functionality that enables you to treat CVAT projects and tasks as PyTorch datasets.
The code of this layer is located in the
To use it, you must install the
cvat_sdk distribution with the
import torch import torchvision.models from cvat_sdk import make_client from cvat_sdk.pytorch import ProjectVisionDataset, ExtractSingleLabelIndex # create a PyTorch model model = torchvision.models.resnet34( weights=torchvision.models.ResNet34_Weights.IMAGENET1K_V1) model.eval() # log into the CVAT server with make_client(host="localhost", credentials=('user', 'password')) as client: # get the dataset comprising all tasks for the Validation subset of project 12345 dataset = ProjectVisionDataset(client, project_id=12345, include_subsets=['Validation'], # use transforms that fit our neural network transform=torchvision.models.ResNet34_Weights.IMAGENET1K_V1.transforms(), target_transform=ExtractSingleLabelIndex()) # print the number of images in the dataset (in other words, the number of frames # in the included tasks) print(len(dataset)) # get a sample from the dataset image, target = dataset # evaluate the network on the sample and compare the output to the target output = model(image) if torch.equal(output, target): print("correct prediction") else: print("incorrect prediction")
The key components of this layer are the dataset classes,
representing data & annotations contained in a CVAT project or task, respectively.
Both of them are subclasses of the
torch.utils.data.Dataset abstract class.
The interface of
Dataset is essentially that of a sequence
whose elements are samples from the dataset.
In the case of
TaskVisionDataset, each sample represents a frame from the task
and its associated annotations.
The order of the samples is the same as the order of frames in the task.
Deleted frames are omitted.
In the case of
each sample is a sample from one of the project’s tasks,
as if obtained from a
TaskVisionDataset instance created for that task.
The full sequence of samples is built by concatenating the sequences of samples
from all included tasks in an unspecified order
that is guaranteed to be consistent between executions.
For details on what tasks are included, see Task filtering.
Both dataset classes are instantiated by passing in an instance of
and the ID of the project or task:
dataset = ProjectVisionDataset(client, 123) dataset = TaskVisionDataset(client, 456)
The referenced project or task must contain image data. Video data is currently not supported.
The constructors of these classes also support several keyword-only parameters:
target_transform: see Transform support.
label_name_to_index: see Label index assignment.
ProjectVisionDatasetonly): see Task filtering.
update_policy: see Caching.
During construction, the dataset objects either populate or validate the local data cache (see Caching for details). Any necessary requests to the CVAT server are performed at this time. After construction, the objects make no more network requests.
Indexing a dataset produces a sample. A sample has the form of a tuple with the following components:
PIL.Image.Image): the image.
cvat_sdk.pytorch.Target): the annotations and auxiliary data.
The target object contains the following attributes:
list[cvat_sdk.models.LabeledImage]): tag annotations associated with the current frame.
list[cvat_sdk.models.LabeledShape]): shape annotations associated with the current frame.
Mapping[int, int]): see Label index assignment.
Note that track annotations are currently inaccessible.
The dataset classes support torchvision-like transforms that you can supply to preprocess each sample before it’s returned. You can use this to convert the samples to a more convenient format or to preprocess the data. The transforms are supplied via the following constructor parameters:
transforms: a callable that accepts two arguments (the image and the target) and returns a tuple with two elements.
transform: a callable that accepts an image.
target_transform: a callable that accepts a target.
Let the sample value prior to any transformations be
Here is what indexing the dataset will return for various combinations of
transforms cannot be supplied at the same time
cvat_sdk.pytorch module contains some target transform classes
that are intended for common use cases.
Label index assignment
The annotation model classes (
reference labels by their IDs on the CVAT server.
This is usually not very useful for machine learning code,
since those IDs are unpredictable and will be different between different projects,
even if semantically the set of labels is the same.
Therefore, the dataset classes assign to each label a unique index that
is intended to be a project-independent identifier.
These indices are accessible via the
on each sample’s target.
This attribute maps IDs on the server to the assigned index.
The mapping is the same for every sample.
By default, the dataset classes arrange all label IDs in an unspecified order that remains consistent across executions, and assign them sequential indices, starting with 0.
You can override this behavior and specify your own label indices
label_name_to_index constructor parameter.
This parameter accepts a mapping from label name to index.
The mapping must contain a key for each label in the project/task.
When this parameter is specified, label indices are assigned
by looking up each label’s name in the provided mapping and using the result.
Note: this section applies only to
By default, a
ProjectVisionDataset includes samples
from every task belonging to the project.
You can change this using the following constructor parameters:
Callable[[models.ITaskRead], bool]): if set, the callable will be called for every task, with an instance of
ITaskReadcorresponding to that task passed as the argument. Only tasks for which
Trueis returned will be included.
Container[str]): if set, only tasks whose subset is a member of the container will be included.
Both parameters can be set, in which case tasks must fulfull both criteria to be included.
The images and annotations of a dataset can be substantial in size,
so they are not downloaded from the server every time a dataset object is created.
Instead, they are loaded from a cache on the local file system,
which is maintained during dataset object construction
according to the policy set by the
update_policy constructor parameter.
The available policies are:
UpdatePolicy.IF_MISSING_OR_STALE: If some data is already cached, query the server to determine if it is out of date. If so, discard it. Then, download all necessary data that is missing from the cache and cache it.
This is the default policy.
UpdatePolicy.NEVER: If some necessary data is missing from the cache, raise an exception. Don’t make any network requests.
Note that this policy permits the use of stale data.
By default, the cache is located in a platform-specific per-user directory.
You can change this location with the
cache_dir setting in the
The layer provides some classes whose instances are callables
suitable for usage with the
target_transform dataset constructor parameter
that are intended to simplify working with CVAT datasets in common scenarios.
Intended for object detection tasks.
Iterable[str]). The values must be from the following list:
Effect: Gathers all shape annotations from the input target object
whose types are contained in the value of
Then returns a dictionary with the following string keys
N is the number of gathered shapes):
"boxes"(a floating-point tensor of shape
4). Each row represents the bounding box the corresponding shape in the following format:
[x_min, y_min, x_max, y_max].
"labels"(an integer tensor of shape
N). Each element is the index of the label of the corresponding shape.
Intended for image classification tasks.
Constructor parameters: None.
Effect: If the input target object contains no tag annotations
or more than one tag annotation, raises
Otherwise, returns the index of the label in the solitary tag annotation
as a zero-dimensional tensor.