How to add a new annotation format support

Instructions on adding support for new annotation formats. This section on GitHub.
  1. Add a python script to dataset_manager/formats
  2. Add an import statement to registry.py.
  3. Implement some importers and exporters as the format requires.

Each format is supported by an importer and exporter.

It can be a function or a class decorated with importer or exporter from registry.py. Examples:

@importer(name="MyFormat", version="1.0", ext="ZIP")
def my_importer(file_object, task_data, **options):
  ...

@importer(name="MyFormat", version="2.0", ext="XML")
class my_importer(file_object, task_data, **options):
  def __call__(self, file_object, task_data, **options):
    ...

@exporter(name="MyFormat", version="1.0", ext="ZIP"):
def my_exporter(file_object, task_data, **options):
  ...

Each decorator defines format parameters such as:

  • name

  • version

  • file extension. For the importer it can be a comma-separated list. These parameters are combined to produce a visible name. It can be set explicitly by the display_name argument.

Importer arguments:

  • file_object - a file with annotations or dataset
  • task_data - an instance of TaskData class.

Exporter arguments:

  • file_object - a file for annotations or dataset

  • task_data - an instance of TaskData class.

  • options - format-specific options. save_images is the option to distinguish if dataset or just annotations are requested.

TaskData provides many task properties and interfaces to add and read task annotations.

Public members:

  • TaskData. Attribute - class, namedtuple('Attribute', 'name, value')

  • TaskData. LabeledShape - class, namedtuple('LabeledShape', 'type, frame, label, points, occluded, attributes, group, z_order')

  • TrackedShape - namedtuple('TrackedShape', 'type, points, occluded, frame, attributes, outside, keyframe, z_order')

  • Track - class, namedtuple('Track', 'label, group, shapes')

  • Tag - class, namedtuple('Tag', 'frame, label, attributes, group')

  • Frame - class, namedtuple('Frame', 'frame, name, width, height, labeled_shapes, tags')

  • TaskData. shapes - property, an iterator over LabeledShape objects

  • TaskData. tracks - property, an iterator over Track objects

  • TaskData. tags - property, an iterator over Tag objects

  • TaskData. meta - property, a dictionary with task information

  • TaskData. group_by_frame() - method, returns an iterator over Frame objects, which groups annotation objects by frame. Note that TrackedShape s will be represented as LabeledShape s.

  • TaskData. add_tag(tag) - method, tag should be an instance of the Tag class

  • TaskData. add_shape(shape) - method, shape should be an instance of the Shape class

  • TaskData. add_track(track) - method, track should be an instance of the Track class

Sample exporter code:

...
# dump meta info if necessary
...
# iterate over all frames
for frame_annotation in task_data.group_by_frame():
  # get frame info
  image_name = frame_annotation.name
  image_width = frame_annotation.width
  image_height = frame_annotation.height
  # iterate over all shapes on the frame
  for shape in frame_annotation.labeled_shapes:
    label = shape.label
    xtl = shape.points[0]
    ytl = shape.points[1]
    xbr = shape.points[2]
    ybr = shape.points[3]
    # iterate over shape attributes
    for attr in shape.attributes:
      attr_name = attr.name
      attr_value = attr.value
...
# dump annotation code
file_object.write(...)
...

Sample importer code:

...
#read file_object
...
for parsed_shape in parsed_shapes:
  shape = task_data.LabeledShape(
    type="rectangle",
    points=[0, 0, 100, 100],
    occluded=False,
    attributes=[],
    label="car",
    outside=False,
    frame=99,
  )
task_data.add_shape(shape)

Format specifications