Dataloop’s SDK and CLI documentation¶

Example:

dl.organizations.add_member(email='user@domain.com',
                            organization_id='organization_id',
                            role=dl.MemberOrgRole.MEMBER)

cache_action(organization_id: Optional[str] = None, organization_name: Optional[str] = None, organization: Optional[Organization] = None, mode=CacheAction.APPLY, pod_type=PodType.SMALL)[source]¶

Add or remove Cache for the org

Prerequisites: You must be an organization owner

You must provide at least ONE of the following params: organization, organization_name, or organization_id.

Parameters

organization_id (str) – Organization id
organization_name (str) – Organization name
organization (entities.Organization) – Organization object
mode (str) – dl.CacheAction.APPLY or dl.CacheAction.DESTROY
pod_type (entities.PodType) – dl.PodType.SMALL, dl.PodType.MEDIUM, dl.PodType.HIGH

Returns

True if success

Return type

Example:

dl.organizations.enable_cache(organization_id='organization_id',
                              mode=dl.CacheAction.APPLY)

delete_member(user_id: str, organization_id: Optional[str] = None, organization_name: Optional[str] = None, organization: Optional[Organization] = None, sure: bool = False, really: bool = False) → bool[source]¶

Delete member from the Organization.

Prerequisites: Must be an organization owner to delete members.

You must provide at least ONE of the following params: organization_id, organization_name, organization.

Parameters

user_id (str) – user id
organization_id (str) – Organization id
organization_name (str) – Organization name
organization (entities.Organization) – Organization object
sure (bool) – Are you sure you want to delete?
really (bool) – Really really sure?

Returns

True if success and error if not

Return type

dtlpy.entities.organization.Organization

Example:

dl.organizations.delete_member(user_id='user_id',
                                organization_id='organization_id',
                                sure=True,
                                really=True)

get(organization_id: Optional[str] = None, organization_name: Optional[str] = None, fetch: Optional[bool] = None) → Organization[source]¶

Get Organization object to be able to use it in your code.

Prerequisites: You must be a superuser to use this method.

You must provide at least ONE of the following params: organization_name or organization_id.

Parameters

organization_id (str) – optional - search by id
organization_name (str) – optional - search by name
fetch – optional - fetch entity from platform, default taken from cookie

Returns

Organization object

Return type

Example:

dl.organizations.get(organization_id='organization_id')

list() → List[Organization][source]¶

Lists all the organizations in Dataloop.

Prerequisites: You must be a superuser to use this method.

Returns: List of Organization objects
Return type: list

Example:

dl.organizations.list()

list_groups(organization: Optional[Organization] = None, organization_id: Optional[str] = None, organization_name: Optional[str] = None)[source]¶

List all organization groups (groups that were created within the organization).

Prerequisites: You must be an organization owner to use this method.

You must provide at least ONE of the following params: organization, organization_name, or organization_id.

Parameters

organization (entities.Organization) – Organization object
organization_id (str) – Organization id
organization_name (str) – Organization name

Returns

groups list

Return type

Example:

dl.organizations.list_groups(organization_id='organization_id')

list_integrations(organization: Optional[Organization] = None, organization_id: Optional[str] = None, organization_name: Optional[str] = None, only_available=False)[source]¶

List all organization integrations with external cloud storage.

Prerequisites: You must be an organization owner to use this method.

You must provide at least ONE of the following params: organization_id, organization_name, or organization.

Parameters

organization (entities.Organization) – Organization object
organization_id (str) – Organization id
organization_name (str) – Organization name
only_available (bool) – if True list only the available integrations

Returns

integrations list

Return type

Example:

dl.organizations.list_integrations(organization='organization-entity',
                                    only_available=True)

list_members(organization: Optional[Organization] = None, organization_id: Optional[str] = None, organization_name: Optional[str] = None, role: Optional[MemberOrgRole] = None)[source]¶

List all organization members.

Prerequisites: You must be an organization owner to use this method.

You must provide at least ONE of the following params: organization_id, organization_name, or organization.

Parameters

organization (entities.Organization) – Organization object
organization_id (str) – Organization id
organization_name (str) – Organization name
role (entities.MemberOrgRole) – MemberOrgRole.ADMIN, MemberOrgRole.OWNER, MemberOrgRole.MEMBER

Returns

projects list

Return type

dtlpy.entities.organization.Organization

Example:

dl.organizations.list_members(organization='organization-entity',
                            role=dl.MemberOrgRole.MEMBER)

update(plan: str, organization: Optional[Organization] = None, organization_id: Optional[str] = None, organization_name: Optional[str] = None) → Organization[source]¶

Update an organization.

Prerequisites: You must be a superuser to update an organization.

You must provide at least ONE of the following params: organization, organization_name, or organization_id.

Parameters

plan (str) – OrganizationsPlans.FREEMIUM, OrganizationsPlans.PREMIUM
organization (entities.Organization) – Organization object
organization_id (str) – Organization id
organization_name (str) – Organization name

Returns

organization object

Return type

Example:

dl.organizations.update(organization='organization-entity',
                        plan=dl.OrganizationsPlans.FREEMIUM)

update_member(email: str, role: MemberOrgRole = MemberOrgRole.MEMBER, organization_id: Optional[str] = None, organization_name: Optional[str] = None, organization: Optional[Organization] = None)[source]¶

Update member role.

Prerequisites: You must be an organization owner to update a member’s role.

You must provide at least ONE of the following params: organization, organization_name, or organization_id.

Parameters

email (str) – the member’s email
role (str) – MemberOrgRole.ADMIN, MemberOrgRole.OWNER, MemberOrgRole.MEMBER
organization_id (str) – Organization id
organization_name (str) – Organization name
organization (entities.Organization) – Organization object

Returns

json of the member fields

Return type

Example:

dl.organizations.update_member(email='user@domain.com',
                                organization_id='organization_id',
                                 role=dl.MemberOrgRole.MEMBER)

Integrations¶

Integrations Repository

class Integrations(client_api: ApiClient, org: Optional[Organization] = None, project: Optional[Project] = None)[source]¶

Bases: object

Integrations Repository

The Integrations class allows you to manage data integrtion from your external storage (e.g., S3, GCS, Azure) into your Dataloop’s Dataset storage, as well as sync data in your Dataloop’s Datasets with data in your external storage.

For more information on Organization Storgae Integration see the Dataloop documentation and SDK External Storage.

create(integrations_type: ExternalStorage, name: str, options: dict)[source]¶

Create an integration between an external storage and the organization.

Examples for options include: s3 - {key: “”, secret: “”}; gcs - {key: “”, secret: “”, content: “”}; azureblob - {key: “”, secret: “”, clientId: “”, tenantId: “”}; key_value - {key: “”, value: “”} aws-sts - {key: “”, secret: “”, roleArns: “”}

Prerequisites: You must be an owner in the organization.

Parameters

integrations_type (str) – integrations type dl.ExternalStorage
name (str) – integrations name
options (dict) – dict of storage secrets

Returns

success

Return type

Example:

project.integrations.create(integrations_type=dl.ExternalStorage.S3,
                name='S3ntegration',
                options={key: "Access key ID", secret: "Secret access key"})

delete(integrations_id: str, sure: bool = False, really: bool = False) → bool[source]¶

Delete integrations from the organization.

Prerequisites: You must be an organization owner to delete an integration.

Parameters

integrations_id (str) – integrations id
sure (bool) – Are you sure you want to delete?
really (bool) – Really really sure?

Returns

success

Return type

dtlpy.entities.integration.Integration

Example:

project.integrations.delete(integrations_id='integrations_id', sure=True, really=True)

get(integrations_id: str)[source]¶

Get organization integrations. Use this method to access your integration and be able to use it in your code.

Prerequisites: You must be an owner in the organization.

Parameters: integrations_id (str) – integrations id
Returns: Integration object
Return type: dtlpy.entities.integration.Integration

Example:

project.integrations.get(integrations_id='integrations_id')

list(only_available=False)[source]¶

List all the organization’s integrations with external storage.

Prerequisites: You must be an owner in the organization.

Parameters: only_available (bool) – if True list only the available integrations.
Returns: groups list
Return type: list

Example:

project.integrations.list(only_available=True)

update(new_name: str, integrations_id: str)[source]¶

Update the integration’s name.

Prerequisites: You must be an owner in the organization.

Parameters

new_name (str) – new name
integrations_id (str) – integrations id

Returns

Integration object

Return type

Example:

project.integrations.update(integrations_id='integrations_id', new_name="new_integration_name")

Projects¶

class Projects(client_api: ApiClient, org=None)[source]¶

Bases: object

Projects Repository

The Projects class allows the user to manage projects and their properties.

For more information on Projects see the Dataloop documentation and SDK documentation.

add_member(email: str, project_id: str, role: MemberRole = MemberRole.DEVELOPER)[source]¶

Add a member to the project.

Prerequisites: You must be in the role of an owner to add a member to a project.

Parameters

email (str) – member email
project_id (str) – project id
role – dl.MemberRole.OWNER, dl.MemberRole.DEVELOPER, dl.MemberRole.ANNOTATOR, dl.MemberRole.ANNOTATION_MANAGER

Returns

dict that represent the user

Return type

Example:

dl.projects.add_member(project_id='project_id', email='user@dataloop.ai', role=dl.MemberRole.DEVELOPER)

checkout(identifier: Optional[str] = None, project_name: Optional[str] = None, project_id: Optional[str] = None, project: Optional[Project] = None)[source]¶

Checkout (switch) to a project to work on it.

Prerequisites: All users can open a project in the web.

You must provide at least ONE of the following params: project_id, project_name.

Parameters

identifier (str) – project name or partial id
project_name (str) – project name
project_id (str) – project id
project (dtlpy.entities.project.Project) – project entity

Example:

dl.projects.checkout(project_id='project_id')

create(project_name: str, checkout: bool = False) → Project[source]¶

Create a new project.

Prerequisites: Any user can create a project.

Parameters

project_name (str) – project name
checkout – checkout

Returns

Project object

Return type

Example:

dl.projects.create(project_name='project_name')

delete(project_name: Optional[str] = None, project_id: Optional[str] = None, sure: bool = False, really: bool = False) → bool[source]¶

Delete a project forever!

Prerequisites: You must be in the role of an owner to delete a project.

Parameters

project_name (str) – optional - search by name
project_id (str) – optional - search by id
sure (bool) – Are you sure you want to delete?
really (boll) – Really really sure?

Returns

True if sucess error if not

Return type

Example:

dl.projects.delete(project_id='project_id', sure=True, really=True)

get(project_name: Optional[str] = None, project_id: Optional[str] = None, checkout: bool = False, fetch: Optional[bool] = None, log_error=True) → Project[source]¶

Get a Project object.

Prerequisites: You must be in the role of an owner to get a project object.

You must check out to a project or provide at least one of the following params: project_id, project_name

Parameters

project_name (str) – optional - search by name
project_id (str) – optional - search by id
checkout (bool) – checkout
fetch (bool) – optional - fetch entity from platform, default taken from cookie
log_error (bool) – optional - show the logs errors

Returns

Project object

Return type

Example:

dl.projects.get(project_id='project_id')

list() → List[Project][source]¶

Get users’ project list.

Prerequisites: You must be a superuser to list all users’ projects.

Returns: List of Project objects

Example:

dl.projects.list()

list_members(project: Project, role: Optional[MemberRole] = None)[source]¶

List the project members.

Prerequisites: You must be in the role of an owner to list project members.

Parameters

project (dtlpy.entities.project.Project) – project entity
role – dl.MemberRole.OWNER, dl.MemberRole.DEVELOPER, dl.MemberRole.ANNOTATOR, dl.MemberRole.ANNOTATION_MANAGER

Returns

list of the project members

Return type

Example:

dl.projects.list_members(project_id='project_id', role=dl.MemberRole.DEVELOPER)

open_in_web(project_name: Optional[str] = None, project_id: Optional[str] = None, project: Optional[Project] = None)[source]¶

Open the project in our web platform.

Prerequisites: All users can open a project in the web.

Parameters

project_name (str) – project name
project_id (str) – project id
project (dtlpy.entities.project.Project) – project entity

Example:

dl.projects.open_in_web(project_id='project_id')

remove_member(email: str, project_id: str)[source]¶

Remove a member from the project.

Prerequisites: You must be in the role of an owner to delete a member from a project.

Parameters

email (str) – member email
project_id (str) – project id

Returns

dict that represents the user

Return type

Example:

dl.projects.remove_member(project_id='project_id', email='user@dataloop.ai')

update(project: Project, system_metadata: bool = False) → Project[source]¶

Update a project information (e.g., name, member roles, etc.).

Prerequisites: You must be in the role of an owner to add a member to a project.

Parameters

project (dtlpy.entities.project.Project) – project entity
system_metadata (bool) – True, if you want to change metadata system

Returns

Project object

Return type

Example:

dl.projects.delete(project='project_entity')

update_member(email: str, project_id: str, role: MemberRole = MemberRole.DEVELOPER)[source]¶

Update member’s information/details in the project.

Prerequisites: You must be in the role of an owner to update a member.

Parameters

email (str) – member email
project_id (str) – project id
role – dl.MemberRole.OWNER, dl.MemberRole.DEVELOPER, dl.MemberRole.ANNOTATOR, dl.MemberRole.ANNOTATION_MANAGER

Returns

dict that represent the user

Return type

Example:

dl.projects.update_member(project_id='project_id', email='user@dataloop.ai', role=dl.MemberRole.DEVELOPER)

Datasets¶

Datasets Repository

class Datasets(client_api: ApiClient, project: Optional[Project] = None)[source]¶

Bases: object

Datasets Repository

The Datasets class allows the user to manage datasets. Read more about datasets in our documentation and SDK documentation.

checkout(identifier: Optional[str] = None, dataset_name: Optional[str] = None, dataset_id: Optional[str] = None, dataset: Optional[Dataset] = None)[source]¶

Checkout (switch) to a dataset to work on it.

Prerequisites: You must be an owner or developer to use this method.

You must provide at least ONE of the following params: dataset_id, dataset_name.

Parameters

identifier (str) – project name or partial id
dataset_name (str) – dataset name
dataset_id (str) – dataset id
dataset (dtlpy.entities.dataset.Dataset) – dataset object

Example:

project.datasets.checkout(dataset_id='dataset_id')

clone(dataset_id: str, clone_name: str, filters: Optional[Filters] = None, with_items_annotations: bool = True, with_metadata: bool = True, with_task_annotations_status: bool = True)[source]¶

Clone a dataset. Read more about cloning datatsets and items in our documentation and SDK documentation.

Prerequisites: You must be in the role of an owner or developer.

Parameters

dataset_id (str) – id of the dataset you wish to clone
clone_name (str) – new dataset name
filters (dtlpy.entities.filters.Filters) – Filters entity or a query dict
with_items_annotations (bool) – true to clone with items annotations
with_metadata (bool) – true to clone with metadata
with_task_annotations_status (bool) – true to clone with task annotations’ status

Returns

dataset object

Return type

Example:

project.datasets.clone(dataset_id='dataset_id',
                      clone_name='dataset_clone_name',
                      with_metadata=True,
                      with_items_annotations=False,
                      with_task_annotations_status=False)

create(dataset_name: str, labels=None, attributes=None, ontology_ids=None, driver: Optional[Driver] = None, driver_id: Optional[str] = None, checkout: bool = False, expiration_options: Optional[ExpirationOptions] = None, index_driver: IndexDriver = IndexDriver.V1, recipe_id: Optional[str] = None) → Dataset[source]¶

Create a new dataset

Prerequisites: You must be in the role of an owner or developer.

Parameters

dataset_name (str) – dataset name
labels (list) – dictionary of {tag: color} or list of label entities
attributes (list) – dataset’s ontology’s attributes
ontology_ids (list) – optional - dataset ontology
driver (dtlpy.entities.driver.Driver) – optional - storage driver Driver object or driver name
driver_id (str) – optional - driver id
checkout (bool) – bool. cache the dataset to work locally
expiration_options (ExpirationOptions) – dl.ExpirationOptions object that contain definitions for dataset like MaxItemDays
index_driver (str) – dl.IndexDriver, dataset driver version
recipe_id (str) – optional - recipe id

Returns

Dataset object

Return type

Example:

project.datasets.create(dataset_name='dataset_name', ontology_ids='ontology_ids')

delete(dataset_name: Optional[str] = None, dataset_id: Optional[str] = None, sure: bool = False, really: bool = False)[source]¶

Delete a dataset forever!

Prerequisites: You must be an owner or developer to use this method.

Example:

project.datasets.delete(dataset_id='dataset_id', sure=True, really=True)

Parameters

dataset_name (str) – optional - search by name
dataset_id (str) – optional - search by id
sure (bool) – Are you sure you want to delete?
really (bool) – Really really sure?

Returns

True is success

Return type

directory_tree(dataset: Optional[Dataset] = None, dataset_name: Optional[str] = None, dataset_id: Optional[str] = None)[source]¶

Get dataset’s directory tree.

Prerequisites: You must be an owner or developer to use this method.

You must provide at least ONE of the following params: dataset, dataset_name, dataset_id.

Parameters

dataset (dtlpy.entities.dataset.Dataset) – dataset object
dataset_name (str) – dataset name
dataset_id (str) – dataset id

Returns

DirectoryTree

Example:

project.datasets.directory_tree(dataset='dataset_entity')

static download_annotations(dataset: Dataset, local_path: Optional[str] = None, filters: Optional[Filters] = None, annotation_options: Optional[ViewAnnotationOptions] = None, annotation_filters: Optional[Filters] = None, overwrite: bool = False, thickness: int = 1, with_text: bool = False, remote_path: Optional[str] = None, include_annotations_in_output: bool = True, export_png_files: bool = False, filter_output_annotations: bool = False, alpha: Optional[float] = None, export_version=ExportVersion.V1) → str[source]¶

Download dataset’s annotations by filters.

You may filter the dataset both for items and for annotations and download annotations.

Optional – download annotations as: mask, instance, image mask of the item.

Prerequisites: You must be in the role of an owner or developer.

Parameters

dataset (dtlpy.entities.dataset.Dataset) – dataset object
local_path (str) – local folder or filename to save to.
filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
annotation_options (list) – download annotations options: list(dl.ViewAnnotationOptions)
annotation_filters (dtlpy.entities.filters.Filters) – Filters entity to filter annotations for download
overwrite (bool) – optional - default = False
thickness (int) – optional - line thickness, if -1 annotation will be filled, default =1
with_text (bool) – optional - add text to annotations, default = False
remote_path (str) – DEPRECATED and ignored
include_annotations_in_output (bool) – default - False , if export should contain annotations
export_png_files (bool) – default - if True, semantic annotations should be exported as png files
filter_output_annotations (bool) – default - False, given an export by filter - determine if to filter out annotations
alpha (float) – opacity value [0 1], default 1
export_version (str) – exported items will have original extension in filename, V1 - no original extension in filenames

Returns

local_path of the directory where all the downloaded item

Return type

Example:

project.datasets.download_annotations(dataset='dataset_entity',
                                     local_path='local_path',
                                     annotation_options=dl.ViewAnnotationOptions,
                                     overwrite=False,
                                     thickness=1,
                                     with_text=False,
                                     alpha=1
                                     )

get(dataset_name: Optional[str] = None, dataset_id: Optional[str] = None, checkout: bool = False, fetch: Optional[bool] = None) → Dataset[source]¶

Get dataset by name or id.

Prerequisites: You must be an owner or developer to use this method.

You must provide at least ONE of the following params: dataset_id, dataset_name.

Parameters

dataset_name (str) – optional - search by name
dataset_id (str) – optional - search by id
checkout (bool) – True to checkout
fetch (bool) – optional - fetch entity from platform, default taken from cookie

Returns

Dataset object

Return type

Example:

project.datasets.get(dataset_id='dataset_id')

list(name=None, creator=None) → List[Dataset][source]¶

List all datasets.

Prerequisites: You must be an owner or developer to use this method.

Parameters

name (str) – list by name
creator (str) – list by creator

Returns

List of datasets

Return type

Example:

project.datasets.list(name='name')

merge(merge_name: str, dataset_ids: str, project_ids: str, with_items_annotations: bool = True, with_metadata: bool = True, with_task_annotations_status: bool = True, wait: bool = True)[source]¶

Merge a dataset. See our SDK docs for more information.

Prerequisites: You must be an owner or developer to use this method.

Parameters

merge_name (str) – new dataset name
dataset_ids (str) – id’s of the datatsets you wish to merge
project_ids (str) – project id
with_items_annotations (bool) – with items annotations
with_metadata (bool) – with metadata
with_task_annotations_status (bool) – with task annotations status
wait (bool) – wait for the command to finish

Returns

True if success

Return type

Example:

project.datasets.clone(dataset_ids=['dataset_id1','dataset_id2'],
                      merge_name='dataset_merge_name',
                      with_metadata=True,
                      with_items_annotations=False,
                      with_task_annotations_status=False)

open_in_web(dataset_name: Optional[str] = None, dataset_id: Optional[str] = None, dataset: Optional[Dataset] = None)[source]¶

Open the dataset in web platform.

Prerequisites: You must be an owner or developer to use this method.

Parameters

dataset_name (str) – dataset name
dataset_id (str) – dataset id
dataset (dtlpy.entities.dataset.Dataset) – dataset object

Example:

project.datasets.open_in_web(dataset_id='dataset_id')

set_readonly(state: bool, dataset: Dataset)[source]¶

Set dataset readonly mode.

Prerequisites: You must be in the role of an owner or developer.

Parameters

state (bool) – state to update readonly mode
dataset (dtlpy.entities.dataset.Dataset) – dataset object

Example:

project.datasets.set_readonly(dataset='dataset_entity', state=True)

sync(dataset_id: str, wait: bool = True)[source]¶

Sync dataset with external storage.

Prerequisites: You must be in the role of an owner or developer.

Parameters

dataset_id (str) – to sync dataset
wait (bool) – wait for the command to finish

Returns

True if success

Return type

Example:

project.datasets.sync(dataset_id='dataset_id')

update(dataset: Dataset, system_metadata: bool = False, patch: Optional[dict] = None) → Dataset[source]¶

Update dataset field.

Prerequisites: You must be an owner or developer to use this method.

Parameters

dataset (dtlpy.entities.dataset.Dataset) – dataset object
system_metadata (bool) – True, if you want to change metadata system
patch (dict) – Specific patch request

Returns

Dataset object

Return type

dtlpy.entities.driver.Driver

Example:

project.datasets.update(dataset='dataset_entity')

upload_annotations(dataset, local_path, filters: Optional[Filters] = None, clean=False, remote_root_path='/', export_version=ExportVersion.V1)[source]¶

Upload annotations to dataset.

Example for remote_root_path: If the item filepath is a/b/item and remote_root_path is /a the start folder will be b instead of a

Prerequisites: You must have a dataset with items that are related to the annotations. The relationship between the dataset and annotations is shown in the name. You must be in the role of an owner or developer.

Parameters

dataset (dtlpy.entities.dataset.Dataset) – dataset to upload to
local_path (str) – str - local folder where the annotations files is
filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
clean (bool) – True to remove the old annotations
remote_root_path (str) – the remote root path to match remote and local items
export_version (str) – exported items will have original extension in filename, V1 - no original extension in filenames

Example:

project.datasets.upload_annotations(dataset='dataset_entity',
                                     local_path='local_path',
                                     clean=False,
                                     export_version=dl.ExportVersion.V1
                                     )

Drivers¶

class Drivers(client_api: ApiClient, project: Optional[Project] = None)[source]¶

Bases: object

Drivers Repository

The Drivers class allows users to manage drivers that are used to connect with external storage. Read more about external storage in our documentation and SDK documentation.

create(name: str, driver_type: ExternalStorage, integration_id: str, bucket_name: str, project_id: Optional[str] = None, allow_external_delete: bool = True, region: Optional[str] = None, storage_class: str = '', path: str = '')[source]¶

Create a storage driver.

Prerequisites: You must be in the role of an owner or developer.

Parameters

name (str) – the driver name
driver_type (str) – ExternalStorage.S3, ExternalStorage.GCS, ExternalStorage.AZUREBLOB
integration_id (str) – the integration id
bucket_name (str) – the external bucket name
project_id (str) – project id
allow_external_delete (bool) – true to allow deleting files from external storage when files are deleted in your Dataloop storage
region (str) – relevant only for s3 - the bucket region
storage_class (str) – rilevante only for s3
path (str) – Optional. By default path is the root folder. Path is case sensitive integration

Returns

driver object

Return type

Example:

project.drivers.create(name='driver_name',
           driver_type=dl.ExternalStorage.S3,
           integration_id='integration_id',
           bucket_name='bucket_name',
           project_id='project_id',
           region='ey-west-1')

get(driver_name: Optional[str] = None, driver_id: Optional[str] = None) → Driver[source]¶

Get a Driver object to use in your code.

Prerequisites: You must be in the role of an owner or developer.

You must provide at least ONE of the following params: driver_name, driver_id.

Parameters

driver_name (str) – optional - search by name
driver_id (str) – optional - search by id

Returns

Driver object

Return type

dtlpy.entities.driver.Driver

Example:

project.drivers.get(driver_id='driver_id')

list() → List[Driver][source]¶

Get the project’s drivers list.

Prerequisites: You must be in the role of an owner or developer.

Returns: List of Drivers objects
Return type: list

Example:

project.drivers.list()

Items¶

class Items(client_api: ApiClient, datasets: Optional[Datasets] = None, dataset: Optional[Dataset] = None, dataset_id=None, items_entity=None, project=None)[source]¶

Bases: object

Items Repository

The Items class allows you to manage items in your datasets. For information on actions related to items see Organizing Your Dataset, Item Metadata, and Item Metadata-Based Filtering.

clone(item_id: str, dst_dataset_id: str, remote_filepath: Optional[str] = None, metadata: Optional[dict] = None, with_annotations: bool = True, with_metadata: bool = True, with_task_annotations_status: bool = False, allow_many: bool = False, wait: bool = True)[source]¶

Clone item. Read more about cloning datatsets and items in our documentation and SDK documentation.

Prerequisites: You must be in the role of an owner or developer.

Parameters

item_id (str) – item to clone
dst_dataset_id (str) – destination dataset id
remote_filepath (str) – complete filepath
metadata (dict) – new metadata to add
with_annotations (bool) – clone annotations
with_metadata (bool) – clone metadata
with_task_annotations_status (bool) – clone task annotations status
allow_many (bool) – bool if True, using multiple clones in single dataset is allowed, (default=False)
wait (bool) – wait for the command to finish

Returns

Item object

Return type

Example:

dataset.items.clone(item_id='item_id',
        dst_dataset_id='dist_dataset_id',
        with_metadata=True,
        with_task_annotations_status=False,
        with_annotations=False)

delete(filename: Optional[str] = None, item_id: Optional[str] = None, filters: Optional[Filters] = None)[source]¶

Delete item from platform.

Prerequisites: You must be in the role of an owner or developer.

You must provide at least ONE of the following params: item id, filename, filters.

Parameters

filename (str) – optional - search item by remote path
item_id (str) – optional - search item by id
filters (dtlpy.entities.filters.Filters) – optional - delete items by filter

Returns

True if success

Return type

Example:

dataset.items.delete(item_id='item_id')

download(filters: Optional[Filters] = None, items=None, local_path: Optional[str] = None, file_types: Optional[list] = None, save_locally: bool = True, to_array: bool = False, annotation_options: Optional[ViewAnnotationOptions] = None, annotation_filters: Optional[Filters] = None, overwrite: bool = False, to_items_folder: bool = True, thickness: int = 1, with_text: bool = False, without_relative_path=None, avoid_unnecessary_annotation_download: bool = False, include_annotations_in_output: bool = True, export_png_files: bool = False, filter_output_annotations: bool = False, alpha: float = 1, export_version=ExportVersion.V1)[source]¶

Download dataset items by filters.

Filters the dataset for items and saves them locally.

Optional – download annotation, mask, instance, and image mask of the item.

Prerequisites: You must be in the role of an owner or developer.

Parameters

filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
items (List[dtlpy.entities.item.Item] or dtlpy.entities.item.Item) – download Item entity or item_id (or a list of item)
local_path (str) – local folder or filename to save to.
file_types (list) – a list of file type to download. e.g [‘video/webm’, ‘video/mp4’, ‘image/jpeg’, ‘image/png’]
save_locally (bool) – bool. save to disk or return a buffer
to_array (bool) – returns Ndarray when True and local_path = False
annotation_options (list) – download annotations options: list(dl.ViewAnnotationOptions)
annotation_filters (dtlpy.entities.filters.Filters) – Filters entity to filter annotations for download
overwrite (bool) – optional - default = False
to_items_folder (bool) – Create ‘items’ folder and download items to it
thickness (int) – optional - line thickness, if -1 annotation will be filled, default =1
with_text (bool) – optional - add text to annotations, default = False
without_relative_path (bool) – bool - download items without the relative path from platform
avoid_unnecessary_annotation_download (bool) – default - False
include_annotations_in_output (bool) – default - False , if export should contain annotations
export_png_files (bool) – default - if True, semantic annotations should be exported as png files
filter_output_annotations (bool) – default - False, given an export by filter - determine if to filter out annotations
alpha (float) – opacity value [0 1], default 1
export_version (str) – exported items will have original extension in filename, V1 - no original extension in filenames

Returns

generator of local_path per each downloaded item

Return type

generator or single item

Example:

dataset.items.download(local_path='local_path',
                     annotation_options=dl.ViewAnnotationOptions,
                     overwrite=False,
                     thickness=1,
                     with_text=False,
                     alpha=1,
                     save_locally=True
                     )

get(filepath: Optional[str] = None, item_id: Optional[str] = None, fetch: Optional[bool] = None, is_dir: bool = False) → Item[source]¶

Get Item object

Prerequisites: You must be in the role of an owner or developer.

Parameters

filepath (str) – optional - search by remote path
item_id (str) – optional - search by id
fetch (bool) – optional - fetch entity from platform, default taken from cookie
is_dir (bool) – True if you want to get an item from dir type

Returns

Item object

Return type

Example:

dataset.items.get(item_id='item_id')

get_all_items(filters: Optional[Filters] = None) → [<class 'dtlpy.entities.item.Item'>][source]¶

Get all items in dataset.

Prerequisites: You must be in the role of an owner or developer.

Parameters: filters (dtlpy.entities.filters.Filters) – dl.Filters entity to filters items
Returns: list of all items
Return type: list

Example:

dataset.items.get_all_items()

list(filters: Optional[Filters] = None, page_offset: Optional[int] = None, page_size: Optional[int] = None) → PagedEntities[source]¶

List items in a dataset.

Prerequisites: You must be in the role of an owner or developer.

Parameters

filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
page_offset (int) – start page
page_size (int) – page size

Returns

Pages object

Return type

Example:

dataset.items.list(page_offset=0, page_size=100)

make_dir(directory, dataset: Optional[Dataset] = None) → Item[source]¶

Create a directory in a dataset.

Prerequisites: All users.

Parameters

directory (str) – name of directory
dataset (dtlpy.entities.dataset.Dataset) – dataset object

Returns

Item object

Return type

Example:

dataset.items.make_dir(directory='directory_name')

move_items(destination: str, filters: Optional[Filters] = None, items=None, dataset: Optional[Dataset] = None) → bool[source]¶

Move items to another directory. If directory does not exist we will create it

Prerequisites: You must be in the role of an owner or developer.

Parameters

destination (str) – destination directory
filters (dtlpy.entities.filters.Filters) – optional - either this or items. Query of items to move
items – optional - either this or filters. A list of items to move
dataset (dtlpy.entities.dataset.Dataset) – dataset object

Returns

True if success

Return type

Example:

dataset.items.move_items(destination='directory_name')

open_in_web(filepath=None, item_id=None, item=None)[source]¶

Open the item in web platform

Prerequisites: You must be in the role of an owner or developer or be an annotation manager/annotator with access to that item through task.

Parameters

filepath (str) – item file path
item_id (str) – item id
item (dtlpy.entities.item.Item) – item entity

Example:

dataset.items.open_in_web(item_id='item_id')

set_items_entity(entity)[source]¶

Set the item entity type to Artifact, Item, or Codebase.

Parameters: entity (entities.Item, entities.Artifact, entities.Codebase) – entity type [entities.Item, entities.Artifact, entities.Codebase]

update(item: Optional[Item] = None, filters: Optional[Filters] = None, update_values=None, system_update_values=None, system_metadata: bool = False)[source]¶

Update item metadata.

Prerequisites: You must be in the role of an owner or developer.

You must provide at least ONE of the following params: update_values, system_update_values.

Parameters

item (dtlpy.entities.item.Item) – Item object
filters (dtlpy.entities.filters.Filters) – optional update filtered items by given filter
update_values – optional field to be updated and new values
system_update_values – values in system metadata to be updated
system_metadata (bool) – True, if you want to update the metadata system

Returns

Item object

Return type

Example:

dataset.items.update(item='item_entity')

update_status(status: ItemStatus, items=None, item_ids=None, filters=None, dataset=None, clear=False)[source]¶

Update item status in task

Prerequisites: You must be in the role of an owner or developer or annotation manager who has been assigned a task with the item.

You must provide at least ONE of the following params: items, item_ids, filters.

Parameters

status (str) – ItemStatus.COMPLETED, ItemStatus.APPROVED, ItemStatus.DISCARDED
items (list) – list of items
item_ids (list) – list of items id
filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
dataset (dtlpy.entities.dataset.Dataset) – dataset object
clear (bool) – to delete status

Example:

dataset.items.update_status(item_ids='item_id', status=dl.ItemStatus.COMPLETED)

upload(local_path: str, local_annotations_path: ~typing.Optional[str] = None, remote_path: str = '/', remote_name: ~typing.Optional[str] = None, file_types: ~typing.Optional[~dtlpy.repositories.items.Items.list] = None, overwrite: bool = False, item_metadata: ~typing.Optional[dict] = None, output_entity=<class 'dtlpy.entities.item.Item'>, no_output: bool = False, export_version: str = ExportVersion.V1)[source]¶

Upload local file to dataset. Local filesystem will remain unchanged. If “*” at the end of local_path (e.g. “/images/*”) items will be uploaded without the head directory.

Prerequisites: Any user can upload items.

Parameters

local_path (str) – list of local file, local folder, BufferIO, numpy.ndarray or url to upload
local_annotations_path (str) – path to dataloop format annotations json files.
remote_path (str) – remote path to save.
remote_name (str) – remote base name to save. when upload numpy.ndarray as local path, remote_name with .jpg or .png ext is mandatory
file_types (list) – list of file type to upload. e.g [‘.jpg’, ‘.png’]. default is all
item_metadata (dict) – metadata dict to upload to item or ExportMetadata option to export metadata from annotation file
overwrite (bool) – optional - default = False
output_entity – output type
no_output (bool) – do not return the items after upload
export_version (str) – exported items will have original extension in filename, V1 - no original extension in filenames

Returns

Output (generator/single item)

Return type

generator or single item

Example:

dataset.items.upload(local_path='local_path',
                     local_annotations_path='local_annotations_path',
                     overwrite=True,
                     item_metadata={'Hellow': 'Word'}
                     )

Annotations¶

class Annotations(client_api: ApiClient, item=None, dataset=None, dataset_id=None)[source]¶

Bases: object

Annotations Repository

The Annotation class allows you to manage the annotations of data items. For information on annotations explore our documentation at Classification SDK, Annotation Labels and Attributes, Show Video with Annotations.

builder()[source]¶

Create Annotation collection.

Prerequisites: You must have an item to be annotated. You must have the role of an owner or developer or be assigned a task that includes that item as an annotation manager or annotator.

Returns: Annotation collection object
Return type: dtlpy.entities.annotation_collection.AnnotationCollection

Example:

item.annotations.builder()

delete(annotation: Optional[Annotation] = None, annotation_id: Optional[str] = None, filters: Optional[Filters] = None) → bool[source]¶

Remove an annotation from item.

Prerequisites: You must have an item that has been annotated. You must have the role of an owner or developer or be assigned a task that includes that item as an annotation manager or annotator.

Parameters

annotation (dtlpy.entities.annotation.Annotation) – Annotation object
annotation_id (str) – annotation id
filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters

Returns

True/False

Return type

Example:

item.annotations.delete(annotation_id='annotation_id')

download(filepath: str, annotation_format: ViewAnnotationOptions = ViewAnnotationOptions.MASK, img_filepath: Optional[str] = None, height: Optional[float] = None, width: Optional[float] = None, thickness: int = 1, with_text: bool = False, alpha: float = 1)[source]¶

Save annotation to file.

Prerequisites: You must have an item that has been annotated. You must have the role of an owner or developer or be assigned a task that includes that item as an annotation manager or annotator.

Parameters

filepath (str) – Target download directory
annotation_format (list) – optional - list(dl.ViewAnnotationOptions)
img_filepath (str) – img file path - needed for img_mask
height (float) – optional - image height
width (float) – optional - image width
thickness (int) – optional - annotation format, default =1
with_text (bool) – optional - draw annotation with text, default = False
alpha (float) – opacity value [0 1], default 1

Returns

file path to where save the annotations

Return type

Example:

item.annotations.download(
              filepath='file_path',
              annotation_format=dl.ViewAnnotationOptions.MASK,
              img_filepath='img_filepath',
              height=100,
              width=100,
              thickness=1,
              with_text=False,
              alpha=1)

get(annotation_id: str) → Annotation[source]¶

Get a single annotation.

Prerequisites: You must have an item that has been annotated. You must have the role of an owner or developer or be assigned a task that includes that item as an annotation manager or annotator.

Parameters: annotation_id (str) – annotation id
Returns: Annotation object or None
Return type: dtlpy.entities.annotation.Annotation

Example:

item.annotations.get(annotation_id='annotation_id')

list(filters: Optional[Filters] = None, page_offset: Optional[int] = None, page_size: Optional[int] = None)[source]¶

List Annotations of a specific item. You must get the item first and then list the annotations with the desired filters.

Prerequisites: You must have an item that has been annotated. You must have the role of an owner or developer or be assigned a task that includes that item as an annotation manager or annotator.

Parameters

filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
page_offset (int) – starting page
page_size (int) – size of page

Returns

Pages object

Return type

Example:

item.annotations.list(filters=dl.Filters(
                             resource=dl.FiltersResource.ANNOTATION,
                             field='type',
                             values='box'),
          page_size=100,
          page_offset=0)

show(image=None, thickness: int = 1, with_text: bool = False, height: Optional[float] = None, width: Optional[float] = None, annotation_format: ViewAnnotationOptions = ViewAnnotationOptions.MASK, alpha: float = 1)[source]¶

Show annotations. To use this method, you must get the item first and then show the annotations with the desired filters. The method returns an array showing all the annotations.

Prerequisites: You must have an item that has been annotated. You must have the role of an owner or developer or be assigned a task that includes that item as an annotation manager or annotator.

Parameters

image (ndarray) – empty or image to draw on
thickness (int) – line thickness
with_text (bool) – add label to annotation
height (float) – height
width (float) – width
annotation_format (str) – options: list(dl.ViewAnnotationOptions)
alpha (float) – opacity value [0 1], default 1

Returns

ndarray of the annotations

Return type

ndarray

Example:

item.annotations.show(image='nd array',
          thickness=1,
          with_text=False,
          height=100,
          width=100,
          annotation_format=dl.ViewAnnotationOptions.MASK,
          alpha=1)

update(annotations, system_metadata=False)[source]¶

Update an existing annotation. For example, you may change the annotation’s label and then use the update method.

Prerequisites: You must have an item that has been annotated. You must have the role of an owner or developer or be assigned a task that includes that item as an annotation manager or annotator.

Parameters

annotation (dtlpy.entities.annotation.Annotation) – Annotation object
system_metadata (bool) – bool - True, if you want to change metadata system

Returns

True if successful or error if unsuccessful

Return type

bool

Example:

item.annotations.update(annotation='annotation')

update_status(annotation: Optional[Annotation] = None, annotation_id: Optional[str] = None, status: AnnotationStatus = AnnotationStatus.ISSUE) → Annotation[source]¶

Set status on annotation.

Prerequisites: You must have an item that has been annotated. You must have the role of an owner or developer or be assigned a task that includes that item as an annotation manager.

Parameters

annotation (dtlpy.entities.annotation.Annotation) – Annotation object
annotation_id (str) – optional - annotation id to set status
status (str) – can be AnnotationStatus.ISSUE, AnnotationStatus.APPROVED, AnnotationStatus.REVIEW, AnnotationStatus.CLEAR

Returns

Annotation object

Return type

dtlpy.entities.recipe.Recipe

Example:

item.annotations.update_status(annotation_id='annotation_id', status=dl.AnnotationStatus.ISSUE)

upload(annotations)[source]¶

Upload a new annotation/annotations. You must first create the annotation using the annotation builder method.

Prerequisites: Any user can upload annotations.

Parameters: annotations (List[dtlpy.entities.annotation.Annotation] or dtlpy.entities.annotation.Annotation) – list or single annotation of type Annotation
Returns: list of annotation objects
Return type: list

Example:

item.annotations.upload(annotations='builder')

Recipes¶

class Recipes(client_api: ApiClient, dataset: Optional[Dataset] = None, project: Optional[Project] = None, project_id: Optional[str] = None)[source]¶

Bases: object

Recipes Repository

The Recipes class allows you to manage recipes and their properties. For more information on Recipes, see our documentation and SDK documentation.

clone(recipe: Optional[Recipe] = None, recipe_id: Optional[str] = None, shallow: bool = False)[source]¶

Clone recipe.

Prerequisites: You must be in the role of an owner or developer.

Parameters

recipe (dtlpy.entities.recipe.Recipe) – Recipe object
recipe_id (str) – Recipe id
shallow (bool) – If True, link to existing ontology, clones all ontologies that are linked to the recipe as well

Returns

Cloned ontology object

Return type

Example:

dataset.recipes.clone(recipe_id='recipe_id')

create(project_ids=None, ontology_ids=None, labels=None, recipe_name=None, attributes=None) → Recipe[source]¶

Create a new Recipe. Note: If the param ontology_ids is None, an ontology will be created first.

Prerequisites: You must be in the role of an owner or developer.

Parameters

project_ids – project ids
ontology_ids – ontology ids
labels – labels
recipe_name – recipe name
attributes – attributes

Returns

Recipe entity

Return type

dtlpy.entities.recipe.Recipe

Example:

dataset.recipes.create(recipe_name='My Recipe', labels=labels))

delete(recipe_id: str, force: bool = False)[source]¶

Delete recipe from platform.

Prerequisites: You must be in the role of an owner or developer.

Parameters

recipe_id (str) – recipe id
force (bool) – force delete recipe

Returns

True if success

Return type

dtlpy.entities.recipe.Recipe

Example:

dataset.recipes.delete(recipe_id='recipe_id')

get(recipe_id: str) → Recipe[source]¶

Get a Recipe object to use in your code.

Prerequisites: You must be in the role of an owner or developer.

Parameters: recipe_id (str) – recipe id
Returns: Recipe object
Return type: dtlpy.entities.recipe.Recipe

Example:

dataset.recipes.get(recipe_id='recipe_id')

list(filters: Optional[Filters] = None) → List[Recipe][source]¶

List recipes for a dataset.

Prerequisites: You must be in the role of an owner or developer.

Parameters: filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
Returns: list of all recipes
Retype: list

Example:

dataset.recipes.list()

open_in_web(recipe: Optional[Recipe] = None, recipe_id: Optional[str] = None)[source]¶

Open the recipe in web platform.

Prerequisites: All users.

Parameters

recipe (dtlpy.entities.recipe.Recipe) – recipe entity
recipe_id (str) – recipe id

Example:

dataset.recipes.open_in_web(recipe_id='recipe_id')

update(recipe: Recipe, system_metadata=False) → Recipe[source]¶

Update recipe.

Prerequisites: You must be in the role of an owner or developer.

Parameters

recipe (dtlpy.entities.recipe.Recipe) – Recipe object
system_metadata (bool) – True, if you want to change metadata system

Returns

Recipe object

Return type

Example:

dataset.recipes.update(recipe='recipe_entity')

Ontologies¶

class Ontologies(client_api: ApiClient, recipe: Optional[Recipe] = None, project: Optional[Project] = None, dataset: Optional[Dataset] = None)[source]¶

Bases: object

Ontologies Repository

The Ontologies class allows users to manage ontologies and their properties. Read more about ontology in our SDK docs.

create(labels, title=None, project_ids=None, attributes=None) → Ontology[source]¶

Create a new ontology.

Prerequisites: You must be in the role of an owner or developer.

Parameters

labels – recipe tags
title (str) – ontology title, name
project_ids (list) – recipe project/s
attributes (list) – recipe attributes

Returns

Ontology object

Return type

dtlpy.entities.ontology.Ontology

Example:

recipe.ontologies.create(labels='labels_entity',
                      title='new_ontology',
                      project_ids='project_ids')

delete(ontology_id)[source]¶

Delete Ontology from the platform.

Prerequisites: You must be in the role of an owner or developer.

Parameters: ontology_id – ontology id
Returns: True if success
Return type: bool

Example:

recipe.ontologies.delete(ontology_id='ontology_id')

delete_attributes(ontology_id, keys: list)[source]¶

Delete a bulk of attributes

Parameters

ontology_id (str) – ontology id
keys (list) – Keys of attributes to delete

Returns

True if success

Return type

dtlpy.entities.ontology.Ontology

Example:

ontology.delete_attributes(['1'])

get(ontology_id: str) → Ontology[source]¶

Get Ontology object to use in your code.

Prerequisites: You must be in the role of an owner or developer.

Parameters: ontology_id (str) – ontology id
Returns: Ontology object
Return type: dtlpy.entities.ontology.Ontology

Example:

recipe.ontologies.get(ontology_id='ontology_id')

static labels_to_roots(labels)[source]¶

Converts labels dictionary to a list of platform representation of labels.

Parameters: labels (dict) – labels dict
Returns: platform representation of labels

list(project_ids=None) → List[Ontology][source]¶

List ontologies for recipe

Prerequisites: You must be in the role of an owner or developer.

Parameters: project_ids –
Returns: list of all the ontologies

Example:

recipe.ontologies.list(project_ids='project_ids')

update(ontology: Ontology, system_metadata=False) → Ontology[source]¶

Update the Ontology metadata.

Prerequisites: You must be in the role of an owner or developer.

Parameters

ontology (dtlpy.entities.ontology.Ontology) – Ontology object
system_metadata (bool) – bool - True, if you want to change metadata system

Returns

Ontology object

Return type

Example:

recipe.ontologies.delete(ontology='ontology_entity')

update_attributes(ontology_id: str, title: str, key: str, attribute_type: AttributesTypes, scope: Optional[list] = None, optional: Optional[bool] = None, multi: Optional[bool] = None, values: Optional[list] = None, attribute_range: Optional[AttributesRange] = None)[source]¶

ADD a new attribute or update if exist

Parameters

ontology_id (str) – ontology_id
title (str) – attribute title
key (str) – the key of the attribute must br unique
attribute_type (AttributesTypes) – dl.AttributesTypes your attribute type
scope (list) – list of the labels or * for all labels
optional (bool) – optional attribute
multi (bool) – if can get multiple selection
values (list) – list of the attribute values ( for checkbox and radio button)
attribute_range (dict or AttributesRange) – dl.AttributesRange object

Returns

true in success

Return type

Example:

ontology.update_attributes(key='1',
                           title='checkbox',
                           attribute_type=dl.AttributesTypes.CHECKBOX,
                           values=[1,2,3])

Tasks¶

class Tasks(client_api: ApiClient, project: Optional[Project] = None, dataset: Optional[Dataset] = None, project_id: Optional[str] = None)[source]¶

Bases: object

Tasks Repository

The Tasks class allows the user to manage tasks and their properties. For more information, read in our SDK documentation about Creating Tasks, Redistributing and Reassigning Tasks, and Task Assignment.

add_items(task: Optional[Task] = None, task_id=None, filters: Optional[Filters] = None, items=None, assignee_ids=None, query=None, workload=None, limit=None, wait=True) → Task[source]¶

Add items to a Task.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned to be owner of the annotation task.

Parameters

task (dtlpy.entities.task.Task) – task entity
task_id (str) – task id
filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
items (list) – list of items to add to the task
assignee_ids (list) – list to assignee who works in the task
query (dict) – query to filter the items use it
workload (list) – list of the work load ber assignee and work load
limit (int) – task limit
wait (bool) – wait for the command to finish

Returns

task entity

Return type

Example:

dataset.tasks.add_items(task= 'task_entity',
                    items = [items])

create(task_name, due_date=None, assignee_ids=None, workload=None, dataset=None, task_owner=None, task_type='annotation', task_parent_id=None, project_id=None, recipe_id=None, assignments_ids=None, metadata=None, filters=None, items=None, query=None, available_actions=None, wait=True, check_if_exist: Filters = False, limit=None, batch_size=None, max_batch_workload=None, allowed_assignees=None) → Task[source]¶

Create a new Annotation Task.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned to be owner of the annotation task.

Parameters

task_name (str) – task name
due_date (float) – date by which the task should be finished; for example, due_date = datetime.datetime(day= 1, month= 1, year= 2029).timestamp()
assignee_ids (list) – list of assignee
workload (List[WorkloadUnit]) – list WorkloadUnit for the task assignee
dataset (entities.Dataset) – dataset entity
task_owner (str) – task owner
task_type (str) – “annotation” or “qa”
task_parent_id (str) – optional if type is qa - parent task id
project_id (str) – project id
recipe_id (str) – recipe id
assignments_ids (list) – assignments ids
metadata (dict) – metadata for the task
filters (entities.Filters) – filter to the task
items (List[entities.Item]) – item to insert to the task
query (entities.Filters) – filter to the task
available_actions (list) – list of available actions to the task
wait (bool) – wait for the command to finish
check_if_exist (entities.Filters) – dl.Filters check if task exist according to filter
limit (int) – task limit
batch_size (int) – Pulling batch size (items) . Restrictions - Min 3, max 100
max_batch_workload (int) – Max items in assignment . Restrictions - Min batchSize + 2 , max batchSize * 2
allowed_assignees (list) – It’s like the workload, but without percentage.

Returns

Annotation Task object

Return type

Example:

dataset.tasks.create(task= 'task_entity',
                    due_date = datetime.datetime(day= 1, month= 1, year= 2029).timestamp(),
                    assignee_ids =[ 'annotator1@dataloop.ai', 'annotator2@dataloop.ai'])

create_qa_task(task: Task, assignee_ids, due_date=None, filters=None, items=None, query=None, workload=None, metadata=None, available_actions=None, wait=True, batch_size=None, max_batch_workload=None, allowed_assignees=None) → Task[source]¶

Create a new QA Task.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned to be owner of the annotation task.

Parameters

task (dtlpy.entities.task.Task) – parent task
assignee_ids (list) – list of assignee
due_date (float) – date by which the task should be finished; for example, due_date = datetime.datetime(day= 1, month= 1, year= 2029).timestamp()
filters (entities.Filters) – filter to the task
items (List[entities.Item]) – item to insert to the task
query (entities.Filters) – filter to the task
workload (List[WorkloadUnit]) – list WorkloadUnit for the task assignee
metadata (dict) – metadata for the task
available_actions (list) – list of available actions to the task
wait (bool) – wait for the command to finish
batch_size (int) – Pulling batch size (items) . Restrictions - Min 3, max 100
max_batch_workload (int) – Max items in assignment . Restrictions - Min batchSize + 2 , max batchSize * 2
allowed_assignees (list) – It’s like the workload, but without percentage.

Returns

task object

Return type

Example:

dataset.tasks.create_qa_task(task= 'task_entity',
                            due_date = datetime.datetime(day= 1, month= 1, year= 2029).timestamp(),
                            assignee_ids =[ 'annotator1@dataloop.ai', 'annotator2@dataloop.ai'])

delete(task: Optional[Task] = None, task_name: Optional[str] = None, task_id: Optional[str] = None, wait: bool = True)[source]¶

Delete an Annotation Task.

Prerequisites: You must be in the role of an owner or developer or annotation manager who created that task.

Parameters

task (dtlpy.entities.task.Task) – task entity
task_name (str) – task name
task_id (str) – task id
wait (bool) – wait for the command to finish

Returns

True is success

Return type

Example:

dataset.tasks.delete(task_id='task_id')

get(task_name=None, task_id=None) → Task[source]¶

Get an Annotation Task object to use in your code.

Prerequisites: You must be in the role of an owner or developer or annotation manager who has been assigned the task.

Parameters

task_name (str) – optional - search by name
task_id (str) – optional - search by id

Returns

task object

Return type

list or dtlpy.entities.paged_entities.PagedEntities

Example:

dataset.tasks.get(task_id='task_id')

get_items(task_id: Optional[str] = None, task_name: Optional[str] = None, dataset: Optional[Dataset] = None, filters: Optional[Filters] = None) → PagedEntities[source]¶

Get the task items to use in your code.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned to be owner of the annotation task.

If a filters param is provided, you will receive a PagedEntity output of the task items. If no filter is provided, you will receive a list of the items.

Parameters

task_id (str) – task id
task_name (str) – task name
dataset (dtlpy.entities.dataset.Dataset) – dataset entity
filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters

Returns

list of the items or PagedEntity output of items

Return type

Example:

dataset.tasks.get_items(task_id= 'task_id')

list(project_ids=None, status=None, task_name=None, pages_size=None, page_offset=None, recipe=None, creator=None, assignments=None, min_date=None, max_date=None, filters: Optional[Filters] = None) → Union[List[Task], PagedEntities][source]¶

List all Annotation Tasks.

Prerequisites: You must be in the role of an owner or developer or annotation manager who has been assigned the task.

Parameters

project_ids – list of project ids
status (str) – status
task_name (str) – task name
pages_size (int) – pages size
page_offset (int) – page offset
recipe (dtlpy.entities.recipe.Recipe) – recipe entity
creator (str) – creator
assignments (dtlpy.entities.assignment.Assignment recipe) – assignments entity
min_date (double) – double min date
max_date (double) – double max date
filters (dtlpy.entities.filters.Filters) – dl.Filters entity to filters items

Returns

List of Annotation Task objects

Example:

dataset.tasks.list(project_ids='project_ids',pages_size=100, page_offset=0)

open_in_web(task_name: Optional[str] = None, task_id: Optional[str] = None, task: Optional[Task] = None)[source]¶

Open the task in the web platform.

Prerequisites: You must be in the role of an owner or developer or annotation manager who has been assigned the task.

Parameters

task_name (str) – task name
task_id (str) – task id
task (dtlpy.entities.task.Task) – task entity

Example:

dataset.tasks.open_in_web(task_id='task_id')

query(filters=None, project_ids=None)[source]¶

List all tasks by filter.

Prerequisites: You must be in the role of an owner or developer or annotation manager who has been assigned the task.

Parameters

filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
project_ids (list) – list of project ids

Returns

Paged entity

Return type

Example:

dataset.tasks.query(project_ids='project_ids')

remove_items(task: Optional[Task] = None, task_id=None, filters: Optional[Filters] = None, query=None, items=None, wait=True)[source]¶

remove items from Task.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned to be owner of the annotation task.

Parameters

task (dtlpy.entities.task.Task) – task entity
task_id (str) – task id
filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
query (dict) – query yo filter the items use it
items (list) – list of items to add to the task
wait (bool) – wait for the command to finish

Returns

True if success and an error if failed

Return type

Examples:

dataset.tasks.remove_items(task= 'task_entity',
                            items = [items])

set_status(status: str, operation: str, task_id: str, item_ids: List[str])[source]¶

Update an item status within a task.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned to be owner of the annotation task.

Parameters

status (str) – string the describes the status
operation (str) – ‘create’ or ‘delete’
task_id (str) – task id
item_ids (list) – List[str] id items ids

Returns

True if success

Return type

Example:

dataset.tasks.set_status(task_id= 'task_id', status='complete', operation='create')

update(task: Optional[Task] = None, system_metadata=False) → Task[source]¶

Update an Annotation Task.

Prerequisites: You must be in the role of an owner or developer or annotation manager who created that task.

Parameters

task (dtlpy.entities.task.Task) – task entity
system_metadata (bool) – True, if you want to change metadata system

Returns

Annotation Task object

Return type

dtlpy.entities.assignment.Assignment

Example:

dataset.tasks.update(task='task_entity')

Assignments¶

class Assignments(client_api: ApiClient, project: Optional[Project] = None, task: Optional[Task] = None, dataset: Optional[Dataset] = None, project_id=None)[source]¶

Bases: object

Assignments Repository

The Assignments class allows users to manage assignments and their properties. Read more about Task Assignment in our SDK documentation.

create(assignee_id: str, task: Optional[Task] = None, filters: Optional[Filters] = None, items: Optional[list] = None) → Assignment[source]¶

Create a new assignment.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned as owner of the annotation task.

Parameters

assignee_id (str) – the assignee for the assignment
task (dtlpy.entities.task.Task) – task entity
filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
items (list) – list of items

Returns

Assignment object

Return type

dtlpy.entities.assignment.Assignment assignment

Example:

task.assignments.create(assignee_id='annotator1@dataloop.ai')

get(assignment_name: Optional[str] = None, assignment_id: Optional[str] = None)[source]¶

Get Assignment object to use it in your code.

Parameters

assignment_name (str) – optional - search by name
assignment_id (str) – optional - search by id

Returns

Assignment object

Return type

Example:

task.assignments.get(assignment_id='assignment_id')

get_items(assignment: Optional[Assignment] = None, assignment_id=None, assignment_name=None, dataset=None, filters=None) → PagedEntities[source]¶

Get all the items in the assignment.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned as owner of the annotation task.

Parameters

assignment (dtlpy.entities.assignment.Assignment) – assignment entity
assignment_id (str) – assignment id
assignment_name (str) – assignment name
dataset (dtlpy.entities.dataset.Dataset) – dataset entity
filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters

Returns

pages of the items

Return type

dtlpy.entities.assignment.Assignment

Example:

task.assignments.get_items(assignment_id='assignment_id')

list(project_ids: Optional[list] = None, status: Optional[str] = None, assignment_name: Optional[str] = None, assignee_id: Optional[str] = None, pages_size: Optional[int] = None, page_offset: Optional[int] = None, task_id: Optional[int] = None) → List[Assignment][source]¶

Get Assignment list to be able to use it in your code.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned as owner of the annotation task.

Parameters

project_ids (list) – list of project ids
status (str) – assignment status
assignment_name (str) – assignment name
assignee_id (str) – the user that assignee the assignment to it
pages_size (int) – pages size
page_offset (int) – page offset
task_id (str) – task id

Returns

List of Assignment objects

Return type

miscellaneous.List[dtlpy.entities.assignment.Assignment]

Example:

task.assignments.list(status='complete', assignee_id='user@dataloop.ai', pages_size=100, page_offset=0)

open_in_web(assignment_name: Optional[str] = None, assignment_id: Optional[str] = None, assignment: Optional[str] = None)[source]¶

Open the assignment in the platform.

Prerequisites: All users.

Parameters

assignment_name (str) – assignment name
assignment_id (str) – assignment id
assignment (dtlpy.entities.assignment.Assignment) – assignment object

Example:

task.assignments.open_in_web(assignment_id='assignment_id')

reassign(assignee_id: str, assignment: Optional[Assignment] = None, assignment_id: Optional[str] = None, task: Optional[Task] = None, task_id: Optional[str] = None, wait: bool = True)[source]¶

Reassign an assignment.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned as owner of the annotation task.

Parameters

assignee_id (str) – the id of the user whom you want to assign the assignment to
assignment (dtlpy.entities.assignment.Assignment) – assignment object
assignment_id – assignment id
task (dtlpy.entities.task.Task) – task object
task_id (str) – task id
wait (bool) – wait for the command to finish

Returns

Assignment object

Return type

Example:

task.assignments.reassign(assignee_ids='annotator1@dataloop.ai')

redistribute(workload: Workload, assignment: Optional[Assignment] = None, assignment_id: Optional[str] = None, task: Optional[Task] = None, task_id: Optional[str] = None, wait: bool = True)[source]¶

Redistribute an assignment.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned as owner of the annotation task.

Example:

Parameters

workload (dtlpy.entities.assignment.Workload) – workload object that contain the assignees and the work load
assignment (dtlpy.entities.assignment.Assignment) – assignment object
assignment_id (str) – assignment id
task (dtlpy.entities.task.Task) – task object
task_id (str) – task id
wait (bool) – wait for the command to finish

Returns

Assignment object

Return type

dtlpy.entities.assignment.Assignment assignment

task.assignments.redistribute(workload=dl.Workload([dl.WorkloadUnit(assignee_id="annotator1@dataloop.ai", load=50),
                                                    dl.WorkloadUnit(assignee_id="annotator2@dataloop.ai", load=50)]))

set_status(status: str, operation: str, item_id: str, assignment_id: str) → bool[source]¶

Set item status within assignment.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned as owner of the annotation task.

Parameters

status (str) – status
operation (str) – created/deleted
item_id (str) – item id
assignment_id (str) – assignment id

Returns

True id success

Return type

Example:

task.assignments.set_status(assignment_id='assignment_id',
                            status='complete',
                            operation='created',
                            item_id='item_id')

update(assignment: Optional[Assignment] = None, system_metadata: bool = False) → Assignment[source]¶

Update an assignment.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned as owner of the annotation task.

Parameters

assignment (dtlpy.entities.assignment.Assignment assignment) – assignment entity
system_metadata (bool) – True, if you want to change metadata system

Returns

Assignment object

Return type

dtlpy.entities.assignment.Assignment assignment

Example:

task.assignments.update(assignment='assignment_entity', system_metadata=False)

Packages¶

class LocalServiceRunner(client_api: ApiClient, packages, cwd=None, multithreading=False, concurrency=10, package: Optional[Package] = None, module_name='default_module', function_name='run', class_name='ServiceRunner', entry_point='main.py', mock_file_path=None)[source]¶

Bases: object

Service Runner Class

get_field(field_name, field_type, mock_json, project=None, mock_inputs=None)[source]¶

Get field in mock json.

Parameters

field_name – field name
field_type – field type
mock_json – mock json
project – project
mock_inputs – mock inputs

Returns

get_mainpy_run_service()[source]¶

Get mainpy run service

Returns

run_local_project(project=None)[source]¶

Run local project

Parameters: project – project entity

class Packages(client_api: ApiClient, project: Optional[Project] = None)[source]¶

Bases: object

Packages Repository

The Packages class allows users to manage packages (code used for running in Dataloop’s FaaS) and their properties. Read more about Packages.

build_requirements(filepath) → list[source]¶

Build a requirement list (list of packages your code requires to run) from a file path. The file listing the requirements MUST BE a txt file.

Prerequisites: You must be in the role of an owner or developer.

Parameters: filepath – path of the requirements file
Returns: a list of dl.PackageRequirement
Return type: list

static build_trigger_dict(actions, name='default_module', filters=None, function='run', execution_mode: TriggerExecutionMode = 'Once', type_t: TriggerType = 'Event')[source]¶

Build a trigger dictionary to trigger FaaS. Read more about FaaS Triggers.

Prerequisites: You must be in the role of an owner or developer.

Parameters

actions – list of dl.TriggerAction
name (str) – trigger name
filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
function (str) – function name
execution_mode (str) – execution mode dl.TriggerExecutionMode
type_t (str) – trigger type dl.TriggerType

Returns

trigger dict

Return type

Example:

project.packages.build_trigger_dict(actions=dl.TriggerAction.CREATED,
                                  function='run',
                                  execution_mode=dl.TriggerExecutionMode.ONCE)

static check_cls_arguments(cls, missing, function_name, function_inputs)[source]¶

Check class arguments. This method checks that the package function is correct.

Prerequisites: You must be in the role of an owner or developer.

Parameters

cls – packages class
missing (list) – list of the missing params
function_name (str) – name of function
function_inputs (list) – list of function inputs

checkout(package: Optional[Package] = None, package_id: Optional[str] = None, package_name: Optional[str] = None)[source]¶

Checkout (switch) to a package.

Prerequisites: You must be in the role of an owner or developer.

Parameters

package (dtlpy.entities.package.Package) – package entity
package_id (str) – package id
package_name (str) – package name

Example:

project.packages.checkout(package='package_entity')

delete(package: Optional[Package] = None, package_name=None, package_id=None)[source]¶

Delete a Package object.

Prerequisites: You must be in the role of an owner or developer.

Parameters

package (dtlpy.entities.package.Package) – package entity
package_id (str) – package id
package_name (str) – package name

Returns

True if success

Return type

Example:

project.packages.delete(package_name='package_name')

deploy(package_id: Optional[str] = None, package_name: Optional[str] = None, package: Optional[Package] = None, service_name: Optional[str] = None, project_id: Optional[str] = None, revision: Optional[str] = None, init_input: Optional[Union[List[FunctionIO], FunctionIO, dict]] = None, runtime: Optional[Union[KubernetesRuntime, dict]] = None, sdk_version: Optional[str] = None, agent_versions: Optional[dict] = None, bot: Optional[Union[Bot, str]] = None, pod_type: Optional[InstanceCatalog] = None, verify: bool = True, checkout: bool = False, module_name: Optional[str] = None, run_execution_as_process: Optional[bool] = None, execution_timeout: Optional[int] = None, drain_time: Optional[int] = None, on_reset: Optional[str] = None, max_attempts: Optional[int] = None, force: bool = False, secrets: Optional[list] = None, **kwargs) → Service[source]¶

Deploy a package. A service is required to run the code in your package.

Prerequisites: You must be in the role of an owner or developer.

Parameters

package_id (str) – package id
package_name (str) – package name
package (dtlpy.entities.package.Package) – package entity
service_name (str) – service name
project_id (str) – project id
revision (str) – package revision - default=latest
init_input – config to run at startup
runtime (dict) – runtime resources
sdk_version (str) –
- optional - string - sdk version
agent_versions (dict) –
- dictionary - - optional -versions of sdk, agent runner and agent proxy
bot (str) – bot email
pod_type (str) – pod type dl.InstanceCatalog
verify (bool) – verify the inputs
checkout (bool) – checkout
module_name (str) – module name
run_execution_as_process (bool) – run execution as process
execution_timeout (int) – execution timeout
drain_time (int) – drain time
on_reset (str) – on reset
max_attempts (int) – Maximum execution retries in-case of a service reset
force (bool) – optional - terminate old replicas immediately
secrets (list) – list of the integrations ids

Returns

Service object

Return type

Example:

project.packages.deploy(service_name=package_name,
                        execution_timeout=3 * 60 * 60,
                        module_name=module.name,
                        runtime=dl.KubernetesRuntime(
                            concurrency=10,
                            pod_type=dl.InstanceCatalog.REGULAR_S,
                            autoscaler=dl.KubernetesRabbitmqAutoscaler(
                                min_replicas=1,
                                max_replicas=20,
                                queue_length=20
                            )
                        )
                    )

deploy_from_file(project, json_filepath)[source]¶

Deploy package and service from a JSON file.

Prerequisites: You must be in the role of an owner or developer.

Parameters

project (dtlpy.entities.project.Project) – project entity
json_filepath (str) – path of the file to deploy

Returns

the package and the services

Example:

project.packages.deploy_from_file(project='project_entity', json_filepath='json_filepath')

static generate(name=None, src_path: Optional[str] = None, service_name: Optional[str] = None, package_type='default_package_type')[source]¶

Generate a new package. Provide a file path to a JSON file with all the details of the package and service to generate the package.

Prerequisites: You must be in the role of an owner or developer.

Parameters

name (str) – name
src_path (str) – source file path
service_name (str) – service name
package_type (str) – package type from PackageCatalog

Example:

project.packages.generate(name='package_name',
                          src_path='src_path')

get(package_name: Optional[str] = None, package_id: Optional[str] = None, checkout: bool = False, fetch=None) → Package[source]¶

Get Package object to use in your code.

Prerequisites: You must be in the role of an owner or developer.

Parameters

package_id (str) – package id
package_name (str) – package name
checkout (bool) – checkout
fetch – optional - fetch entity from platform, default taken from cookie

Returns

Package object

Return type

Example:

project.packages.get(package_id='package_id')

list(filters: Optional[Filters] = None, project_id: Optional[str] = None) → PagedEntities[source]¶

List project packages.

Prerequisites: You must be in the role of an owner or developer.

Parameters

filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
project_id (str) – project id

Returns

Paged entity

Return type

Example:

project.packages.list()

open_in_web(package: Optional[Package] = None, package_id: Optional[str] = None, package_name: Optional[str] = None)[source]¶

Open the package in the web platform.

Prerequisites: You must be in the role of an owner or developer.

Parameters

package (dtlpy.entities.package.Package) – package entity
package_id (str) – package id
package_name (str) – package name

Example:

project.packages.open_in_web(package_id='package_id')

pull(package: Package, version=None, local_path=None, project_id=None)[source]¶

Pull (download) the package to a local path.

Prerequisites: You must be in the role of an owner or developer.

Parameters

package (dtlpy.entities.package.Package) – package entity
version –
local_path –
project_id –

Returns

local path where the package pull

Return type

Example:

project.packages.pull(package='package_entity', local_path='local_path')

push(project: Optional[Project] = None, project_id: Optional[str] = None, package_name: Optional[str] = None, src_path: Optional[str] = None, codebase: Optional[Union[GitCodebase, ItemCodebase, FilesystemCodebase]] = None, modules: Optional[List[PackageModule]] = None, is_global: Optional[bool] = None, checkout: bool = False, revision_increment: Optional[str] = None, version: Optional[str] = None, ignore_sanity_check: bool = False, service_update: bool = False, service_config: Optional[dict] = None, slots: Optional[List[PackageSlot]] = None, requirements: Optional[List[PackageRequirement]] = None) → Package[source]¶

Push your local package to the UI.

Prerequisites: You must be in the role of an owner or developer.

Project will be taken in the following hierarchy: project(input) -> project_id(input) -> self.project(context) -> checked out

Parameters

project (dtlpy.entities.project.Project) – optional - project entity to deploy to. default from context or checked-out
project_id (str) – optional - project id to deploy to. default from context or checked-out
package_name (str) – package name
src_path (str) – path to package codebase
codebase (dtlpy.entities.codebase.Codebase) – codebase object
modules (list) – list of modules PackageModules of the package
is_global (bool) – is package is global or local
checkout (bool) – checkout package to local dir
revision_increment (str) – optional - str - version bumping method - major/minor/patch - default = None
version (str) – semver version f the package
ignore_sanity_check (bool) – NOT RECOMMENDED - skip code sanity check before pushing
service_update (bool) – optional - bool - update the service
service_config (dict) – json of service - a service that have config from the main service if wanted
slots (list) – optional - list of slots PackageSlot of the package
requirements (list) – requirements - list of package requirements

Returns

Package object

Return type

Example:

project.packages.push(package_name='package_name',
                        modules=[module],
                        version='1.0.0',
                        src_path=os.getcwd()
                    )

revisions(package: Optional[Package] = None, package_id: Optional[str] = None)[source]¶

Get the package revisions history.

Prerequisites: You must be in the role of an owner or developer.

Parameters

package (dtlpy.entities.package.Package) – package entity
package_id (str) – package id

Example:

project.packages.revisions(package='package_entity')

test_local_package(cwd: Optional[str] = None, concurrency: Optional[int] = None, package: Optional[Package] = None, module_name: str = 'default_module', function_name: str = 'run', class_name: str = 'ServiceRunner', entry_point: str = 'main.py', mock_file_path: Optional[str] = None)[source]¶

Test local package in local environment.

Prerequisites: You must be in the role of an owner or developer.

Parameters

cwd (str) – path to the file
concurrency (int) – the concurrency of the test
package (dtlpy.entities.package.Package) – entities.package
module_name (str) – module name
function_name (str) – function name
class_name (str) – class name
entry_point (str) – the file to run like main.py
mock_file_path (str) – the mock file that have the inputs

Returns

list created by the function that tested the output

Return type

Example:

project.packages.test_local_package(cwd='path_to_package',
                                    package='package_entity',
                                    function_name='run')

update(package: Package, revision_increment: Optional[str] = None) → Package[source]¶

Update Package changes to the platform.

Prerequisites: You must be in the role of an owner or developer.

Parameters

package (dtlpy.entities.package.Package) –
revision_increment – optional - str - version bumping method - major/minor/patch - default = None

Returns

Package object

Return type

Example:

project.packages.delete(package='package_entity')

Codebases¶

class Codebases(client_api: ApiClient, project: Optional[Project] = None, dataset: Optional[Dataset] = None, project_id: Optional[str] = None)[source]¶

Bases: object

Codebase Repository

The Codebases class allows the user to manage codebases and their properties. The codebase is the code the user uploads for the user’s packages to run. Read more about codebase in our FaaS (function as a service).

clone_git(codebase: Codebase, local_path: str)[source]¶

Clone code base

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters

codebase (dtlpy.entities.codebase.Codebase) – codebase object
local_path (str) – local path

Returns

path where the clone will be

Return type

str

Example:

package.codebases.clone_git(codebase='codebase_entity', local_path='local_path')

get(codebase_name: Optional[str] = None, codebase_id: Optional[str] = None, version: Optional[str] = None)[source]¶

Get a Codebase object to use in your code.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Example:

package.codebases.get(codebase_name='codebase_name')

Parameters

codebase_name (str) – optional - search by name
codebase_id (str) – optional - search by id
version (str) – codebase version. default is latest. options: “all”, “latest” or ver number - “10”

Returns

Codebase object

static get_current_version(all_versions_pages, zip_md)[source]¶

This method returns the current version of the codebase and other versions found.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters

all_versions_pages (codebase) – codebase object
zip_md – zipped file of codebase

Returns

current version and all versions found of codebase

Return type

int, int

Example:

package.codebases.get_current_version(all_versions_pages='codebase_entity', zip_md='path')

list() → PagedEntities[source]¶

List all codebases.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Example:

package.codebases.list()

Returns: Paged entity
Return type: dtlpy.entities.paged_entities.PagedEntities

list_versions(codebase_name: str)[source]¶

List all codebase versions.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Example:

package.codebases.list_versions(codebase_name='codebase_name')

Parameters: codebase_name (str) – code base name
Returns: list of versions
Return type: list

pack(directory: str, name: Optional[str] = None, description: str = '')[source]¶

Zip a local code directory and post to codebases.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters

directory (str) – local directory to pack
name (str) – codebase name
description (dtr) – codebase description

Returns

Codebase object

Return type

dtlpy.entities.codebase.Codebase

Example:

package.codebases.pack(directory='path_dir', name='codebase_name')

pull_git(codebase, local_path)[source]¶

Pull (download) a codebase.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters

codebase (dtlpy.entities.codebase.Codebase) – codebase object
local_path (str) – local path

Returns

path where the Pull will be

Return type

Example:

package.codebases.pull_git(codebase='codebase_entity', local_path='local_path')

unpack(codebase: Optional[Codebase] = None, codebase_name: Optional[str] = None, codebase_id: Optional[str] = None, local_path: Optional[str] = None, version: Optional[str] = None)[source]¶

Unpack codebase locally. Download source code and unzip.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters

codebase (dtlpy.entities.codebase.Codebase) – dl.Codebase object
codebase_name (str) – search by name
codebase_id (str) – search by id
local_path (str) – local path to save codebase
version (str) – codebase version to unpack. default - latest

Returns

String (dirpath)

Return type

Example:

package.codebases.unpack(codebase='codebase_entity', local_path='local_path')

Services¶

class ServiceLog(_json: dict, service: Service, services: Services, start=None, follow=None, execution_id=None, function_name=None, replica_id=None, system=False)[source]¶

Bases: object

Service Log

view(until_completed)[source]¶

View logs

Parameters: until_completed –

class Services(client_api: ApiClient, project: Optional[Project] = None, package: Optional[Package] = None, project_id=None)[source]¶

Bases: object

Services Repository

The Services class allows the user to manage services and their properties. Services are created from the packages users create. See our documentation for more information about services.

activate_slots(service: Service, project_id: Optional[str] = None, task_id: Optional[str] = None, dataset_id: Optional[str] = None, org_id: Optional[str] = None, user_email: Optional[str] = None, slots: Optional[List[PackageSlot]] = None, role=None, prevent_override: bool = True, visible: bool = True, icon: str = 'fas fa-magic', **kwargs)[source]¶

Activate service slots (creates buttons in the UI that activate services).

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters

service (dtlpy.entities.service.Service) – service entity
project_id (str) – project id
task_id (str) – task id
dataset_id (str) – dataset id
org_id (str) – org id
user_email (str) – user email
slots (list) – list of entities.PackageSlot
role (str) – user role MemberOrgRole.ADMIN, MemberOrgRole.owner, MemberOrgRole.MEMBER
prevent_override (bool) – True to prevent override
visible (bool) – visible
icon (str) – icon
kwargs – all additional arguments

Returns

list of user setting for activated slots

Return type

Example:

package.services.activate_slots(service='service_entity',
                                project_id='project_id',
                                slots=List[entities.PackageSlot],
                                icon='fas fa-magic')

checkout(service: Optional[Service] = None, service_name: Optional[str] = None, service_id: Optional[str] = None)[source]¶

Checkout (switch) to a service.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters

service (dtlpy.entities.service.Service) – Service entity
service_name (str) – service name
service_id (str) – service id

Example:

package.services.checkout(service_id='service_id')

delete(service_name: Optional[str] = None, service_id: Optional[str] = None)[source]¶

Delete Service object

Prerequisites: You must be in the role of an owner or developer. You must have a package.

You must provide at least ONE of the following params: service_id, service_name.

Parameters

service_name (str) – by name
service_id (str) – by id

Returns

True

Return type

Example:

package.services.delete(service_id='service_id')

deploy(service_name: Optional[str] = None, package: Optional[Package] = None, bot: Optional[Union[Bot, str]] = None, revision: Optional[str] = None, init_input: Optional[Union[List[FunctionIO], FunctionIO, dict]] = None, runtime: Optional[Union[KubernetesRuntime, dict]] = None, pod_type: Optional[InstanceCatalog] = None, sdk_version: Optional[str] = None, agent_versions: Optional[dict] = None, verify: bool = True, checkout: bool = False, module_name: Optional[str] = None, project_id: Optional[str] = None, driver_id: Optional[str] = None, func: Optional[Callable] = None, run_execution_as_process: Optional[bool] = None, execution_timeout: Optional[int] = None, drain_time: Optional[int] = None, max_attempts: Optional[int] = None, on_reset: Optional[str] = None, force: bool = False, secrets: Optional[list] = None, **kwargs) → Service[source]¶

Deploy service.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters

service_name (str) – name
package (dtlpy.entities.package.Package) – package entity
bot (str) – bot email
revision (str) – package revision of version
init_input – config to run at startup
runtime (dict) – runtime resources
pod_type (str) – pod type dl.InstanceCatalog
sdk_version (str) –
- optional - string - sdk version
agent_versions (str) –
- dictionary - - optional -versions of sdk
verify (bool) – if true, verify the inputs
checkout (bool) – if true, checkout (switch) to service
module_name (str) – module name
project_id (str) – project id
driver_id (str) – driver id
func (Callable) – function to deploy
run_execution_as_process (bool) – if true, run execution as process
execution_timeout (int) – execution timeout in seconds
drain_time (int) – drain time in seconds
max_attempts (int) – maximum execution retries in-case of a service reset
on_reset (str) – what happens on reset
force (bool) – optional - if true, terminate old replicas immediately
secrets (list) – list of the integrations ids
kwargs – list of additional arguments

Returns

Service object

Return type

Example:

package.services.deploy(service_name=package_name,
                        execution_timeout=3 * 60 * 60,
                        module_name=module.name,
                        runtime=dl.KubernetesRuntime(
                            concurrency=10,
                            pod_type=dl.InstanceCatalog.REGULAR_S,
                            autoscaler=dl.KubernetesRabbitmqAutoscaler(
                                min_replicas=1,
                                max_replicas=20,
                                queue_length=20
                            )
                        )
                    )

deploy_from_local_folder(cwd=None, service_file=None, bot=None, checkout=False, force=False) → Service[source]¶

Deploy from local folder in local environment.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters

cwd (str) – optional - package working directory. Default=cwd
service_file (str) – optional - service file. Default=None
bot (str) – bot
checkout – checkout
force (bool) – optional - terminate old replicas immediately

Returns

Service object

Return type

Example:

package.services.deploy_from_local_folder(cwd='file_path',
                                          service_file='service_file')

execute(service: Optional[Service] = None, service_id: Optional[str] = None, service_name: Optional[str] = None, sync: bool = False, function_name: Optional[str] = None, stream_logs: bool = False, execution_input=None, resource=None, item_id=None, dataset_id=None, annotation_id=None, project_id=None) → Execution[source]¶

Execute a function on an existing service.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters

service (dtlpy.entities.service.Service) – service entity
service_id (str) – service id
service_name (str) – service name
sync (bool) – wait for function to end
function_name (str) – function name to run
stream_logs (bool) – prints logs of the new execution. only works with sync=True
execution_input – input dictionary or list of FunctionIO entities
resource (str) – dl.PackageInputType - input type.
item_id (str) – str - optional - input to function
dataset_id (str) – str - optional - input to function
annotation_id (str) – str - optional - input to function
project_id (str) – str - resource’s project

Returns

entities.Execution

Return type

Example:

package.services.execute(service='service_entity',
                        function_name='run',
                        item_id='item_id',
                        project_id='project_id')

get(service_name=None, service_id=None, checkout=False, fetch=None) → Service[source]¶

Get service to use in your code.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters

service_name (str) – optional - search by name
service_id (str) – optional - search by id
checkout (bool) – if true, checkout (switch) to service
fetch – optional - fetch entity from platform, default taken from cookie

Returns

Service object

Return type

Example:

package.services.get(service_id='service_id')

list(filters: Optional[Filters] = None) → PagedEntities[source]¶

List all services (services can be listed for a package or for a project).

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters: filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
Returns: Paged entity
Return type: dtlpy.entities.paged_entities.PagedEntities

Example:

package.services.list()

log(service, size=100, checkpoint=None, start=None, end=None, follow=False, text=None, execution_id=None, function_name=None, replica_id=None, system=False, view=True, until_completed=True)[source]¶

Get service logs.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters

service (dtlpy.entities.service.Service) – service object
size (int) – size
checkpoint (dict) – the information from the lst point checked in the service
start (str) – iso format time
end (str) – iso format time
follow (bool) – if true, keep stream future logs
text (str) – text
execution_id (str) – execution id
function_name (str) – function name
replica_id (str) – replica id
system (bool) – system
view (bool) – if true, print out all the logs
until_completed (bool) – wait until completed

Returns

ServiceLog entity

Return type

ServiceLog

Example:

package.services.log(service='service_entity')

name_validation(name: str)[source]¶

Validation service name.

Prerequisites: You must be in the role of an owner or developer.

Parameters: name (str) – service name

Example:

package.services.name_validation(name='name')

open_in_web(service: Optional[Service] = None, service_id: Optional[str] = None, service_name: Optional[str] = None)[source]¶

Open the service in web platform

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters

service_name (str) – service name
service_id (str) – service id
service (dtlpy.entities.service.Service) – service entity

Example:

package.services.open_in_web(service_id='service_id')

pause(service_name: Optional[str] = None, service_id: Optional[str] = None, force: bool = False)[source]¶

Pause service.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

You must provide at least ONE of the following params: service_id, service_name

Parameters

service_name (str) – service name
service_id (str) – service id
force (bool) – optional - terminate old replicas immediately

Returns

True if success

Return type

Example:

package.services.pause(service_id='service_id')

resume(service_name: Optional[str] = None, service_id: Optional[str] = None, force: bool = False)[source]¶

Resume service.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

You must provide at least ONE of the following params: service_id, service_name.

Parameters

service_name (str) – service name
service_id (str) – service id
force (bool) – optional - terminate old replicas immediately

Returns

json of the service

Return type

Example:

package.services.resume(service_id='service_id')

revisions(service: Optional[Service] = None, service_id: Optional[str] = None)[source]¶

Get service revisions history.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

You must provide at leats ONE of the following params: service, service_id

Parameters

service (dtlpy.entities.service.Service) – Service entity
service_id (str) – service id

Example:

package.services.revisions(service_id='service_id')

status(service_name=None, service_id=None)[source]¶

Get service status.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

You must provide at least ONE of the following params: service_id, service_name

Parameters

service_name (str) – service name
service_id (str) – service id

Returns

status json

Return type

Example:

package.services.status(service_id='service_id')

update(service: Service, force: bool = False) → Service[source]¶

Update service changes to platform.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters

service (dtlpy.entities.service.Service) – Service entity
force (bool) – optional - terminate old replicas immediately

Returns

Service entity

Return type

Example:

package.services.update(service='service_entity')

Bots¶

class Bots(client_api: ApiClient, project: Project)[source]¶

Bases: object

Bots Repository

The Bots class allows the user to manage bots and their properties. See our documentation for more information on bots.

create(name: str, return_credentials: bool = False)[source]¶

Create a new Bot.

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters

name (str) – bot name
return_credentials (str) – True will return the password when created

Returns

Bot object

Return type

dtlpy.entities.bot.Bot

Example:

service.bots.delete(name='bot', return_credentials=False)

delete(bot_id: Optional[str] = None, bot_email: Optional[str] = None)[source]¶

Delete a Bot.

Prerequisites: You must be in the role of an owner or developer. You must have a service.

You must provide at least ONE of the following params: bot_id, bot_email

Parameters

bot_id (str) – bot id to delete
bot_email (str) – bot email to delete

Returns

True if successful

Return type

Example:

service.bots.delete(bot_id='bot_id')

get(bot_email: Optional[str] = None, bot_id: Optional[str] = None, bot_name: Optional[str] = None)[source]¶

Get a Bot object.

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters

bot_email (str) – get bot by email
bot_id (str) – get bot by id
bot_name (str) – get bot by name

Returns

Bot object

Return type

dtlpy.entities.bot.Bot

Example:

service.bots.get(bot_id='bot_id')

list() → List[Bot][source]¶

Get a project or package bots list.

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Returns: List of Bots objects
Return type: list

Example:

service.bots.list()

Triggers¶

class Triggers(client_api: ApiClient, project: Optional[Project] = None, service: Optional[Service] = None, project_id: Optional[str] = None, pipeline: Optional[Pipeline] = None)[source]¶

Bases: object

Triggers Repository

The Triggers class allows users to manage triggers and their properties. Triggers activate services. See our documentation for more information on triggers.

create(service_id: Optional[str] = None, trigger_type: TriggerType = TriggerType.EVENT, name: Optional[str] = None, webhook_id=None, function_name='run', project_id=None, active=True, filters=None, resource: TriggerResource = TriggerResource.ITEM, actions: Optional[TriggerAction] = None, execution_mode: TriggerExecutionMode = TriggerExecutionMode.ONCE, start_at=None, end_at=None, inputs=None, cron=None, pipeline_id=None, pipeline=None, pipeline_node_id=None, root_node_namespace=None, **kwargs) → BaseTrigger[source]¶

Create a Trigger. Can create two types: a cron trigger or an event trigger. Inputs are different for each type

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Inputs for all types:

Parameters

service_id (str) – Id of services to be triggered
trigger_type (str) – can be cron or event. use enum dl.TriggerType for the full list
name (str) – name of the trigger
webhook_id (str) – id for webhook to be called
function_name (str) – the function name to be called when triggered (must be defined in the package)
project_id (str) – project id where trigger will work
active (bool) – optional - True/False, default = True, if true trigger is active

Inputs for event trigger: :param dtlpy.entities.filters.Filters filters: optional - Item/Annotation metadata filters, default = none :param str resource: optional - Dataset/Item/Annotation/ItemStatus, default = Item :param str actions: optional - Created/Updated/Deleted, default = create :param str execution_mode: how many times trigger should be activated; default is “Once”. enum dl.TriggerExecutionMode

Inputs for cron trigger: :param start_at: iso format date string to start activating the cron trigger :param end_at: iso format date string to end the cron activation :param inputs: dictionary “name”:”val” of inputs to the function :param str cron: cron spec specifying when it should run. more information: https://en.wikipedia.org/wiki/Cron :param str pipeline_id: Id of pipeline to be triggered :param pipeline: pipeline entity to be triggered :param str pipeline_node_id: Id of pipeline root node to be triggered :param root_node_namespace: namespace of pipeline root node to be triggered

Returns: Trigger entity
Return type: dtlpy.entities.trigger.Trigger

Example:

service.triggers.create(name='triggername',
                      execution_mode=dl.TriggerExecutionMode.ONCE,
                      resource='Item',
                      actions='Created',
                      function_name='run',
                      filters={'$and': [{'hidden': False},
                                        {'type': 'file'}]}
                      )

delete(trigger_id=None, trigger_name=None)[source]¶

Delete Trigger object

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters

trigger_id (str) – trigger id
trigger_name (str) – trigger name

Returns

True is successful error if not

Return type

dtlpy.entities.trigger.Trigger

Example:

service.triggers.delete(trigger_id='trigger_id')

get(trigger_id=None, trigger_name=None) → BaseTrigger[source]¶

Get Trigger object

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters

trigger_id (str) – trigger id
trigger_name (str) – trigger name

Returns

Trigger entity

Return type

Example:

service.triggers.get(trigger_id='trigger_id')

list(filters: Optional[Filters] = None) → PagedEntities[source]¶

List triggers of a project, package, or service.

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters: filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
Returns: Paged entity
Return type: dtlpy.entities.paged_entities.PagedEntities

Example:

service.triggers.list()

name_validation(name: str)[source]¶

This method validates the trigger name. If name is not valid, this method will return an error. Otherwise, it will not return anything.

Parameters: name (str) – trigger name

resource_information(resource, resource_type, action='Created')[source]¶

Returns which function should run on an item (based on global triggers).

Prerequisites: You must be a superuser to run this method.

Parameters

resource – ‘Item’ / ‘Dataset’ / etc
resource_type – dictionary of the resource object
action – ‘Created’ / ‘Updated’ / etc.

Example:

service.triggers.resource_information(resource='Item', resource_type=item_object, action='Created')

update(trigger: BaseTrigger) → BaseTrigger[source]¶

Update trigger

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters: trigger (dtlpy.entities.trigger.Trigger) – Trigger entity
Returns: Trigger entity
Return type: dtlpy.entities.trigger.Trigger

Example:

service.triggers.update(trigger='trigger_entity')

Executions¶

class Executions(client_api: ApiClient, service: Optional[Service] = None, project: Optional[Project] = None)[source]¶

Bases: object

Service Executions Repository

The Executions class allows the users to manage executions (executions of services) and their properties. See our documentation for more information about executions.

create(service_id: Optional[str] = None, execution_input: Optional[list] = None, function_name: Optional[str] = None, resource: Optional[PackageInputType] = None, item_id: Optional[str] = None, dataset_id: Optional[str] = None, annotation_id: Optional[str] = None, project_id: Optional[str] = None, sync: bool = False, stream_logs: bool = False, return_output: bool = False, return_curl_only: bool = False, timeout: Optional[int] = None) → Execution[source]¶

Execute a function on an existing service

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters

service_id (str) – service id to execute on
execution_input (List[FunctionIO] or dict) – input dictionary or list of FunctionIO entities
function_name (str) – function name to run
resource (str) – input type.
item_id (str) – optional - item id as input to function
dataset_id (str) – optional - dataset id as input to function
annotation_id (str) – optional - annotation id as input to function
project_id (str) – resource’s project
sync (bool) – if true, wait for function to end
stream_logs (bool) – prints logs of the new execution. only works with sync=True
return_output (bool) – if True and sync is True - will return the output directly
return_curl_only (bool) – return the cURL of the creation WITHOUT actually do it
timeout (int) – int, seconds to wait until TimeoutError is raised. if <=0 - wait until done - by default wait take the service timeout

Returns

execution object

Return type

Example:

service.executions.create(function_name='function_name', item_id='item_id', project_id='project_id')

get(execution_id: Optional[str] = None, sync: bool = False) → Execution[source]¶

Get Service execution object

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters

execution_id (str) – execution id
sync (bool) – if true, wait for the execution to finish

Returns

Service execution object

Return type

Example:

service.executions.get(execution_id='execution_id')

increment(execution: Execution)[source]¶

Increment the number of attempts that an execution is allowed to attempt to run a service that is not responding.

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters: execution (dtlpy.entities.execution.Execution) –
Returns: int
Return type: int

Example:

service.executions.increment(execution='execution_entity')

list(filters: Optional[Filters] = None) → PagedEntities[source]¶

List service executions

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters: filters (dtlpy.entities.filters.Filters) – dl.Filters entity to filters items
Returns: Paged entity
Return type: dtlpy.entities.paged_entities.PagedEntities

Example:

service.executions.list()

logs(execution_id: str, follow: bool = True, until_completed: bool = True)[source]¶

executions logs

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters

execution_id (str) – execution id
follow (bool) – if true, keep stream future logs
until_completed (bool) – if true, wait until completed

Returns

executions logs

Example:

service.executions.logs(execution_id='execution_id')

progress_update(execution_id: str, status: Optional[ExecutionStatus] = None, percent_complete: Optional[int] = None, message: Optional[str] = None, output: Optional[str] = None, service_version: Optional[str] = None)[source]¶

Update Execution Progress.

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters

execution_id (str) – execution id
status (str) – ExecutionStatus
percent_complete (int) – percent work done
message (str) – message
output (str) – the output of the execution
service_version (str) – service version

Returns

Service execution object

Return type

Example:

service.executions.progress_update(execution_id='execution_id', status='complete', percent_complete=100)

rerun(execution: Execution, sync: bool = False)[source]¶

Rerun execution

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters

execution (dtlpy.entities.execution.Execution) –
sync (bool) – wait for the execution to finish

Returns

Execution object

Return type

Example:

service.executions.rerun(execution='execution_entity')

terminate(execution: Execution)[source]¶

Terminate Execution

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters: execution (dtlpy.entities.execution.Execution) –
Returns: execution object
Return type: dtlpy.entities.execution.Execution

Example:

service.executions.terminate(execution='execution_entity')

update(execution: Execution) → Execution[source]¶

Update execution changes to platform

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters: execution (dtlpy.entities.execution.Execution) – execution entity
Returns: Service execution object
Return type: dtlpy.entities.execution.Execution

Example:

service.executions.update(execution='execution_entity')

wait(execution_id: str, timeout: Optional[int] = None)[source]¶

Get Service execution object.

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters

execution_id (str) – execution id
timeout (int) – seconds to wait until TimeoutError is raised. if <=0 - wait until done - by default wait take the service timeout

Returns

Service execution object

Return type

Example:

service.executions.wait(execution_id='execution_id')

Pipelines¶

class Pipelines(client_api: ApiClient, project: Optional[Project] = None)[source]¶

Bases: object

Pipelines Repository

The Pipelines class allows users to manage pipelines and their properties. See our documentation for more information on pipelines.

create(name: Optional[str] = None, project_id: Optional[str] = None, pipeline_json: Optional[dict] = None) → Pipeline[source]¶

Create a new pipeline.

prerequisites: You must be an owner or developer to use this method.

Parameters

name (str) – pipeline name
project_id (str) – project id
pipeline_json (dict) – json containing the pipeline fields

Returns

Pipeline object

Return type

Example:

project.pipelines.create(name='pipeline_name')

delete(pipeline: Optional[Pipeline] = None, pipeline_name: Optional[str] = None, pipeline_id: Optional[str] = None)[source]¶

Delete Pipeline object.

prerequisites: You must be an owner or developer to use this method.

Parameters

pipeline (dtlpy.entities.pipeline.Pipeline) – pipeline entity
pipeline_id (str) – pipeline id
pipeline_name (str) – pipeline name

Returns

True if success

Return type

dtlpy.entities.pipeline_execution.PipelineExecution

Example:

project.pipelines.delete(pipeline_id='pipeline_id')

execute(pipeline: Optional[Pipeline] = None, pipeline_id: Optional[str] = None, pipeline_name: Optional[str] = None, execution_input=None)[source]¶

Execute a pipeline and return the pipeline execution as an object.

prerequisites: You must be an owner or developer to use this method.

Parameters

pipeline (dtlpy.entities.pipeline.Pipeline) – pipeline entity
pipeline_id (str) – pipeline id
pipeline_name (str) – pipeline name
execution_input – list of the dl.FunctionIO or dict of pipeline input - example {‘item’: ‘item_id’}

Returns

entities.PipelineExecution object

Return type

Example:

project.pipelines.execute(pipeline='pipeline_entity', execution_input= {'item': 'item_id'} )

get(pipeline_name=None, pipeline_id=None, fetch=None) → Pipeline[source]¶

Get Pipeline object to use in your code.

prerequisites: You must be an owner or developer to use this method.

You must provide at least ONE of the following params: pipeline_name, pipeline_id.

Parameters

pipeline_id (str) – pipeline id
pipeline_name (str) – pipeline name
fetch – optional - fetch entity from platform, default taken from cookie

Returns

Pipeline object

Return type

Example:

project.pipelines.get(pipeline_id='pipeline_id')

install(pipeline: Optional[Pipeline] = None)[source]¶

Install (start) a pipeline.

prerequisites: You must be an owner or developer to use this method.

Parameters: pipeline (dtlpy.entities.pipeline.Pipeline) – pipeline entity
Returns: Composition object

Example:

project.pipelines.install(pipeline='pipeline_entity')

list(filters: Optional[Filters] = None, project_id: Optional[str] = None) → PagedEntities[source]¶

List project pipelines.

prerequisites: You must be an owner or developer to use this method.

Parameters

filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
project_id (str) – project id

Returns

Paged entity

Return type

dtlpy.entities.pipeline_execution.PipelineExecution

Example:

project.pipelines.get()

open_in_web(pipeline: Optional[Pipeline] = None, pipeline_id: Optional[str] = None, pipeline_name: Optional[str] = None)[source]¶

Open the pipeline in web platform.

prerequisites: Must be owner or developer to use this method.

Parameters

pipeline (dtlpy.entities.pipeline.Pipeline) – pipeline entity
pipeline_id (str) – pipeline id
pipeline_name (str) – pipeline name

Example:

project.pipelines.open_in_web(pipeline_id='pipeline_id')

pause(pipeline: Optional[Pipeline] = None)[source]¶

Pause a pipeline.

prerequisites: You must be an owner or developer to use this method.

Parameters: pipeline (dtlpy.entities.pipeline.Pipeline) – pipeline entity
Returns: Composition object

Example:

project.pipelines.pause(pipeline='pipeline_entity')

reset(pipeline: Optional[Pipeline] = None, pipeline_id: Optional[str] = None, pipeline_name: Optional[str] = None, stop_if_running: bool = False)[source]¶

Reset pipeline counters.

prerequisites: You must be an owner or developer to use this method.

Parameters

pipeline (dtlpy.entities.pipeline.Pipeline) – pipeline entity - optional
pipeline_id (str) – pipeline_id - optional
pipeline_name (str) – pipeline_name - optional
stop_if_running (bool) – If the pipeline is installed it will stop the pipeline and reset the counters.

Returns

bool

Example:

project.pipelines.reset(pipeline='pipeline_entity')

stats(pipeline: Optional[Pipeline] = None, pipeline_id: Optional[str] = None, pipeline_name: Optional[str] = None)[source]¶

Get pipeline counters.

prerequisites: You must be an owner or developer to use this method.

Parameters

pipeline (dtlpy.entities.pipeline.Pipeline) – pipeline entity - optional
pipeline_id (str) – pipeline_id - optional
pipeline_name (str) – pipeline_name - optional

Returns

PipelineStats

Return type

dtlpy.entities.pipeline.PipelineStats

Example:

project.pipelines.stats(pipeline='pipeline_entity')

update(pipeline: Optional[Pipeline] = None) → Pipeline[source]¶

Update pipeline changes to platform.

prerequisites: You must be an owner or developer to use this method.

Parameters: pipeline (dtlpy.entities.pipeline.Pipeline) – pipeline entity
Returns: Pipeline object
Return type: dtlpy.entities.pipeline.Pipeline

Example:

project.pipelines.update(pipeline='pipeline_entity')

Pipeline Executions¶

class PipelineExecutions(client_api: ApiClient, project: Optional[Project] = None, pipeline: Optional[Pipeline] = None)[source]¶

Bases: object

PipelineExecutions Repository

The PipelineExecutions class allows users to manage pipeline executions. See our documentation for more information on pipelines.

create(pipeline_id: Optional[str] = None, execution_input=None)[source]¶

Execute a pipeline and return the execute.

prerequisites: You must be an owner or developer to use this method.

Parameters

pipeline_id – pipeline id
execution_input – list of the dl.FunctionIO or dict of pipeline input - example {‘item’: ‘item_id’}

Returns

entities.PipelineExecution object

Return type

Example:

pipeline.pipeline_executions.create(pipeline_id='pipeline_id', execution_input={'item': 'item_id'})

get(pipeline_execution_id: str, pipeline_id: Optional[str] = None) → PipelineExecution[source]¶

Get Pipeline Execution object

prerequisites: You must be an owner or developer to use this method.

Parameters

pipeline_execution_id (str) – pipeline execution id
pipeline_id (str) – pipeline id

Returns

PipelineExecution object

Return type

dtlpy.entities.pipeline_execution.PipelineExecution

Example:

pipeline.pipeline_executions.get(pipeline_id='pipeline_id')

list(filters: Optional[Filters] = None) → PagedEntities[source]¶

List project pipeline executions.

prerequisites: You must be an owner or developer to use this method.

Parameters: filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
Returns: Paged entity
Return type: dtlpy.entities.paged_entities.PagedEntities

Example:

pipeline.pipeline_executions.list()

General Commands¶

class Commands(client_api: ApiClient)[source]¶

Bases: object

Service Commands repository

abort(command_id: str)[source]¶

Abort Command

Parameters: command_id (str) – command id
Returns

get(command_id: Optional[str] = None, url: Optional[str] = None) → Command[source]¶

Get Service command object

Parameters

command_id (str) –
url (str) – command url

Returns

Command object

list()[source]¶

List of commands

Returns: list of commands

wait(command_id, timeout=0, step=None, url=None, backoff_factor=0.1)[source]¶

Wait for command to finish

backoff_factor: A backoff factor to apply between attempts after the second try {backoff factor} * (2 ** ({number of total retries} - 1)) seconds. If the backoff_factor is 0.1, then sleep() will sleep for [0.0s, 0.2s, 0.4s, …] between retries. It will never be longer than 8 sec

Parameters

command_id (str) – Command id to wait to
timeout (int) – int, seconds to wait until TimeoutError is raised. if 0 - wait until done
step (int) – int, seconds between polling
url (str) – url to the command
backoff_factor (float) – A backoff factor to apply between attempts after the second try

Returns

Command object

Download Commands¶

Upload Commands¶

Entities¶

Organization¶

class CacheAction(value)[source]¶

Bases: str, Enum

An enumeration.

class MemberOrgRole(value)[source]¶

Bases: str, Enum

An enumeration.

class Organization(members: list, groups: list, account: dict, created_at, updated_at, id, name, logo_url, plan, owner, created_by, client_api: ApiClient, repositories=NOTHING)[source]¶

Bases: BaseEntity

Organization entity

add_member(email, role: ~dtlpy.entities.organization.MemberOrgRole = <enum 'MemberOrgRole'>)[source]¶

Add members to your organization. Read about members and groups [here](https://dataloop.ai/docs/org-members-groups).

Prerequisities: To add members to an organization, you must be in the role of an “owner” in that organization.

Parameters

email (str) – the member’s email
role (str) – MemberOrgRole.ADMIN, MemberOrgRole.OWNER, MemberOrgRole.MEMBER

Returns

True if successful or error if unsuccessful

Return type

cache_action(mode=CacheAction.APPLY, pod_type=PodType.SMALL)[source]¶

Open the organizations in web platform

Parameters

mode (str) – dl.CacheAction.APPLY or dl.CacheAction.DESTROY
pod_type (dl.PodType) – dl.PodType.SMALL, dl.PodType.MEDIUM, dl.PodType.HIGH

Returns

True if success

Return type

delete_member(user_id: str, sure: bool = False, really: bool = False)[source]¶

Delete member from the Organization.

Prerequisites: Must be an organization “owner” to delete members.

Parameters

user_id (str) – user id
sure (bool) – Are you sure you want to delete?
really (bool) – Really really sure?

Returns

True if success and error if not

Return type

dtlpy.entities.organization.Organization

classmethod from_json(_json, client_api, is_fetched=True)[source]¶

Build a Project entity object from a json

Parameters

is_fetched (bool) – is Entity fetched from Platform
_json (dict) – _json response from host
client_api (dl.ApiClient) – ApiClient entity

Returns

Organization object

Return type

list_groups()[source]¶

List all organization groups (groups that were created within the organization).

Prerequisites: You must be an organization “owner” to use this method.

Returns: groups list
Return type: list

list_members(role: Optional[MemberOrgRole] = None)[source]¶

List all organization members.

Prerequisites: You must be an organization “owner” to use this method.

Parameters: role (str) – MemberOrgRole.ADMIN, MemberOrgRole.OWNER, MemberOrgRole.MEMBER
Returns: projects list
Return type: list

open_in_web()[source]¶: Open the organizations in web platform

to_json()[source]¶

Returns platform _json format of object

Returns: platform json format of object
Return type: dict

update(plan: str)[source]¶

Update Organization.

Prerequisities: You must be an Organization superuser to update an organization.

Parameters: plan (str) – OrganizationsPlans.FREEMIUM, OrganizationsPlans.PREMIUM
Returns: organization object

update_member(email: str, role: MemberOrgRole = MemberOrgRole.MEMBER)[source]¶

Update member role.

Prerequisities: You must be an organization “owner” to update a member’s role.

Parameters

email (str) – the member’s email
role (str) – MemberOrgRole.ADMIN, MemberOrgRole.OWNER, MemberOrgRole.MEMBER

Returns

json of the member fields

Return type

class OrganizationsPlans(value)[source]¶

Bases: str, Enum

An enumeration.

class PodType(value)[source]¶

Bases: str, Enum

An enumeration.

Integration¶

class Integration(id, name, type, org, created_at, created_by, update_at, client_api: ApiClient, project=None)[source]¶

Bases: BaseEntity

Integration object

delete(sure: bool = False, really: bool = False) → bool[source]¶

Delete integrations from the Organization

Parameters

sure (bool) – are you sure you want to delete?
really (bool) – really really?

Returns

True

Return type

classmethod from_json(_json: dict, client_api: ApiClient, is_fetched=True)[source]¶

Build a Integration entity object from a json

Parameters

_json – _json response from host
client_api – ApiClient entity
is_fetched – is Entity fetched from Platform

Returns

Integration object

to_json()[source]¶

Returns platform _json format of object

Returns: platform json format of object
Return type: dict

update(new_name: str)[source]¶

Update the integrations name

Parameters: new_name (str) – new name

Project¶

class MemberRole(value)[source]¶

Bases: str, Enum

An enumeration.

class Project(contributors, created_at, creator, id, name, org, updated_at, role, account, is_blocked, feature_constraints, client_api: ApiClient, repositories=NOTHING)[source]¶

Bases: BaseEntity

Project entity

add_member(email, role: MemberRole = MemberRole.DEVELOPER)[source]¶

Add a member to the project.

Parameters: email (str) – member email

::param role: dl.MemberRole.OWNER, dl.MemberRole.DEVELOPER, dl.MemberRole.ANNOTATOR, dl.MemberRole.ANNOTATION_MANAGER :return: dict that represent the user :rtype: dict

checkout()[source]¶: Checkout the project

delete(sure=False, really=False)[source]¶

Delete the project forever!

Parameters

sure (bool) – are you sure you want to delete?
really (bool) – really really?

Returns

True

Return type

classmethod from_json(_json, client_api, is_fetched=True)[source]¶

Build a Project entity object from a json

Parameters

is_fetched (bool) – is Entity fetched from Platform
_json (dict) – _json response from host
client_api (dl.ApiClient) – ApiClient entity

Returns

Project object

Return type

list_members(role: Optional[MemberRole] = None)[source]¶

List the project members.

Parameters: role – dl.MemberRole.OWNER, dl.MemberRole.DEVELOPER, dl.MemberRole.ANNOTATOR, dl.MemberRole.ANNOTATION_MANAGER
Returns: list of the project members
Return type: list

open_in_web()[source]¶: Open the project in web platform

remove_member(email)[source]¶

Remove a member from the project.

Parameters: email (str) – member email
Returns: dict that represent the user
Return type: dict

to_json()[source]¶

Returns platform _json format of object

Returns: platform json format of object
Return type: dict

update(system_metadata=False)[source]¶

Update the project

Parameters: system_metadata (bool) – to update system metadata
Returns: Project object
Return type: dtlpy.entities.project.Project

update_member(email, role: MemberRole = MemberRole.DEVELOPER)[source]¶

Update member’s information/details from the project.

Parameters

email (str) – member email
role – dl.MemberRole.OWNER, dl.MemberRole.DEVELOPER, dl.MemberRole.ANNOTATOR, dl.MemberRole.ANNOTATION_MANAGER

Returns

dict that represent the user

Return type

User¶

class User(created_at, updated_at, name, last_name, username, avatar, email, role, type, org, id, project, client_api=None, users=None)[source]¶

Bases: BaseEntity

User entity

classmethod from_json(_json, project, client_api, users=None)[source]¶

Build a User entity object from a json

Parameters

_json (dict) – _json response from host
project (dtlpy.entities.project.Project) – project entity
client_api – ApiClient entity
users – Users repository

Returns

User object

Return type

dtlpy.entities.user.User

to_json()[source]¶

Returns platform _json format of object

Returns: platform json format of object
Return type: dict

Dataset¶

class Dataset(id, url, name, annotated, creator, projects, items_count, metadata, directoryTree, export, expiration_options, index_driver, created_at, items_url, readable_type, access_level, driver, readonly, client_api: ApiClient, project=None, datasets=None, repositories=NOTHING, ontology_ids=None, labels=None, directory_tree=None, recipe=None, ontology=None)[source]¶

Bases: BaseEntity

Dataset object

add_label(label_name, color=None, children=None, attributes=None, display_label=None, label=None, recipe_id=None, ontology_id=None, icon_path=None)[source]¶

Add single label to dataset

Prerequisites: You must have a dataset with items that are related to the annotations. The relationship between the dataset and annotations is shown in the name. You must be in the role of an owner or developer.

Parameters

label_name (str) – str - label name
color (tuple) – color
children – children (sub labels)
attributes (list) – attributes
display_label (str) – display_label
label (dtlpy.entities.label.Label) – label
recipe_id (str) – optional recipe id
ontology_id (str) – optional ontology id
icon_path (str) – path to image to be display on label

Returns

label entity

Return type

dtlpy.entities.label.Label

Example:

dataset.add_label(label_name='person', color=(34, 6, 231), attributes=['big', 'small'])

add_labels(label_list, ontology_id=None, recipe_id=None)[source]¶

Add labels to dataset

Prerequisites: You must have a dataset with items that are related to the annotations. The relationship between the dataset and annotations is shown in the name. You must be in the role of an owner or developer.

Parameters

label_list (list) – label list
ontology_id (str) – optional ontology id
recipe_id (str) – optional recipe id

Returns

label entities

Example:

dataset.add_labels(label_list=label_list)

checkout()[source]¶: Checkout the dataset

clone(clone_name, filters=None, with_items_annotations=True, with_metadata=True, with_task_annotations_status=True)[source]¶

Clone dataset

Prerequisites: You must be in the role of an owner or developer.

Parameters

clone_name (str) – new dataset name
filters (dtlpy.entities.filters.Filters) – Filters entity or a query dict
with_items_annotations (bool) – clone all item’s annotations
with_metadata (bool) – clone metadata
with_task_annotations_status (bool) – clone task annotations status

Returns

dataset object

Return type

Example:

dataset.clone(dataset_id='dataset_id',
              clone_name='dataset_clone_name',
              with_metadata=True,
              with_items_annotations=False,
              with_task_annotations_status=False)

delete(sure=False, really=False)[source]¶

Delete a dataset forever!

Prerequisites: You must be an owner or developer to use this method.

Parameters

sure (bool) – are you sure you want to delete?
really (bool) – really really?

Returns

True is success

Return type

Example:

dataset.delete(sure=True, really=True)

delete_attributes(keys: list, recipe_id: Optional[str] = None, ontology_id: Optional[str] = None)[source]¶

Delete a bulk of attributes

Parameters

recipe_id (str) – recipe id
ontology_id (str) – ontology id
keys (list) – Keys of attributes to delete

Returns

True if success

Return type

delete_labels(label_names)[source]¶

Delete labels from dataset’s ontologies

Prerequisites: You must be in the role of an owner or developer.

Parameters: label_names – label object/ label name / list of label objects / list of label names

Example:

dataset.delete_labels(label_names=['myLabel1', 'Mylabel2'])

download(filters=None, local_path=None, file_types=None, annotation_options: Optional[ViewAnnotationOptions] = None, annotation_filters=None, overwrite=False, to_items_folder=True, thickness=1, with_text=False, without_relative_path=None, alpha=1, export_version=ExportVersion.V1)[source]¶

Download dataset by filters. Filtering the dataset for items and save them local Optional - also download annotation, mask, instance and image mask of the item

Prerequisites: You must be in the role of an owner or developer.

Parameters

filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
local_path (str) – local folder or filename to save to.
file_types (list) – a list of file type to download. e.g [‘video/webm’, ‘video/mp4’, ‘image/jpeg’, ‘image/png’]
annotation_options (list(dtlpy.entities.annotation.ViewAnnotationOptions)) – download annotations options: list(dl.ViewAnnotationOptions) not relevant for JSON option
annotation_filters (dtlpy.entities.filters.Filters) – Filters entity to filter annotations for download not relevant for JSON option
overwrite (bool) – optional - default = False
to_items_folder (bool) – Create ‘items’ folder and download items to it
thickness (int) – optional - line thickness, if -1 annotation will be filled, default =1
with_text (bool) – optional - add text to annotations, default = False
without_relative_path (bool) – bool - download items without the relative path from platform
alpha (float) – opacity value [0 1], default 1
export_version (str) – V2 - exported items will have original extension in filename, V1 - no original extension in filenames

Returns

List of local_path per each downloaded item

Example:

dataset.download(local_path='local_path',
                 annotation_options=[dl.ViewAnnotationOptions.JSON, dl.ViewAnnotationOptions.MASK],
                 overwrite=False,
                 thickness=1,
                 with_text=False,
                 alpha=1,
                 save_locally=True
                 )

download_annotations(local_path=None, filters=None, annotation_options: Optional[ViewAnnotationOptions] = None, annotation_filters=None, overwrite=False, thickness=1, with_text=False, remote_path=None, include_annotations_in_output=True, export_png_files=False, filter_output_annotations=False, alpha=1, export_version=ExportVersion.V1)[source]¶

Download dataset by filters. Filtering the dataset for items and save them local Optional - also download annotation, mask, instance and image mask of the item

Prerequisites: You must be in the role of an owner or developer.

Parameters

local_path (str) – local folder or filename to save to.
filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
annotation_options (list(dtlpy.entities.annotation.ViewAnnotationOptions)) – download annotations options: list(dl.ViewAnnotationOptions)
annotation_filters (dtlpy.entities.filters.Filters) – Filters entity to filter annotations for download
overwrite (bool) – optional - default = False
thickness (int) – optional - line thickness, if -1 annotation will be filled, default =1
with_text (bool) – optional - add text to annotations, default = False
remote_path (str) – DEPRECATED and ignored
include_annotations_in_output (bool) – default - False , if export should contain annotations
export_png_files (bool) – default - if True, semantic annotations should be exported as png files
filter_output_annotations (bool) – default - False, given an export by filter - determine if to filter out annotations
alpha (float) – opacity value [0 1], default 1
export_version (str) – exported items will have original extension in filename, V1 - no original extension in filenames

Returns

local_path of the directory where all the downloaded item

Return type

Example:

dataset.download_annotations(dataset='dataset_entity',
                             local_path='local_path',
                             annotation_options=[dl.ViewAnnotationOptions.JSON, dl.ViewAnnotationOptions.MASK],
                             overwrite=False,
                             thickness=1,
                             with_text=False,
                             alpha=1
                             )

download_partition(partition, local_path=None, filters=None, annotation_options=None)[source]¶

Download a specific partition of the dataset to local_path This function is commonly used with dl.ModelAdapter which implements thc convert to specific model structure

Parameters

partition (dl.SnapshotPartitionType) – dl.SnapshotPartitionType name of the partition
local_path (str) – local path directory to download the data
filters (dtlpy.entities.filters.Filters) – dl.entities.Filters to add the specific partitions constraint to

:return List str of the new downloaded path of each item

classmethod from_json(project: Project, _json: dict, client_api: ApiClient, datasets=None, is_fetched=True)[source]¶

Build a Dataset entity object from a json

Parameters

project – dataset’s project
_json (dict) – _json response from host
client_api – ApiClient entity
datasets – Datasets repository
is_fetched (bool) – is Entity fetched from Platform

Returns

Dataset object

Return type

get_partitions(partitions, filters=None, batch_size: Optional[int] = None)[source]¶

Returns PagedEntity of items from one or more partitions

Parameters

partitions – dl.entities.SnapshotPartitionType or a list. Name of the partitions
filters (dtlpy.entities.filters.Filters) – dl.Filters to add the specific partitions constraint to
batch_size – int how many items per page

Returns

dl.PagedEntities of dl.Item preforms items.list()

get_recipe_ids()[source]¶

Get dataset recipe Ids

Returns: list of recipe ids
Return type: list

open_in_web()[source]¶: Open the dataset in web platform

static serialize_labels(labels_dict)[source]¶

Convert hex color format to rgb

Parameters: labels_dict (dict) – dict of labels
Returns: dict of converted labels

set_partition(partition, filters=None)[source]¶

Updates all items returned by filters in the dataset to specific partition

Parameters

partition – dl.entities.SnapshotPartitionType to set to
filters (dtlpy.entities.filters.Filters) – dl.entities.Filters to add the specific partitions constraint to

Returns

dl.PagedEntities

set_readonly(state: bool)[source]¶

Set dataset readonly mode

Prerequisites: You must be in the role of an owner or developer.

Parameters: state (bool) – state

Example:

dataset.set_readonly(state=True)

switch_recipe(recipe_id=None, recipe=None)[source]¶

Switch the recipe that linked to the dataset with the given one

Parameters

recipe_id (str) – recipe id
recipe (dtlpy.entities.recipe.Recipe) – recipe entity

Example:

dataset.switch_recipe(recipe_id='recipe_id')

sync(wait=True)[source]¶

Sync dataset with external storage

Prerequisites: You must be in the role of an owner or developer.

Parameters: wait (bool) – wait for the command to finish
Returns: True if success
Return type: bool

Example:

dataset.sync()

to_json()[source]¶

Returns platform _json format of object

Returns: platform json format of object
Return type: dict

update(system_metadata=False)[source]¶

Update dataset field

Prerequisites: You must be an owner or developer to use this method.

Parameters: system_metadata (bool) – bool - True, if you want to change metadata system
Returns: Dataset object
Return type: dtlpy.entities.dataset.Dataset

Example:

dataset.update()

update_attributes(title: str, key: str, attribute_type, recipe_id: Optional[str] = None, ontology_id: Optional[str] = None, scope: Optional[list] = None, optional: Optional[bool] = None, multi: Optional[bool] = None, values: Optional[list] = None, attribute_range=None)[source]¶

ADD a new attribute or update if exist

Parameters

ontology_id (str) – ontology_id
title (str) – attribute title
key (str) – the key of the attribute must br unique
attribute_type (AttributesTypes) – dl.AttributesTypes your attribute type
scope (list) – list of the labels or * for all labels
optional (bool) – optional attribute
multi (bool) – if can get multiple selection
values (list) – list of the attribute values ( for checkbox and radio button)
attribute_range (dict or AttributesRange) – dl.AttributesRange object

Returns

true in success

Return type

Example:

dataset.update_attributes(ontology_id='ontology_id',
                          key='1',
                          title='checkbox',
                          attribute_type=dl.AttributesTypes.CHECKBOX,
                          values=[1,2,3])

update_label(label_name, color=None, children=None, attributes=None, display_label=None, label=None, recipe_id=None, ontology_id=None, upsert=False, icon_path=None)[source]¶

Add single label to dataset

Prerequisites: You must have a dataset with items that are related to the annotations. The relationship between the dataset and annotations is shown in the name. You must be in the role of an owner or developer.

Parameters

label_name (str) – str - label name
color (tuple) – color
children – children (sub labels)
attributes (list) – attributes
display_label (str) – display_label
label (dtlpy.entities.label.Label) – label
recipe_id (str) – optional recipe id
ontology_id (str) – optional ontology id
icon_path (str) – path to image to be display on label

Returns

label entity

Return type

dtlpy.entities.label.Label

Example:

dataset.update_label(label_name='person', color=(34, 6, 231), attributes=['big', 'small'])

update_labels(label_list, ontology_id=None, recipe_id=None, upsert=False)[source]¶

Add labels to dataset

Prerequisites: You must have a dataset with items that are related to the annotations. The relationship between the dataset and annotations is shown in the name. You must be in the role of an owner or developer.

Parameters

label_list (list) – label list
ontology_id (str) – optional ontology id
recipe_id (str) – optional recipe id
upsert (bool) – if True will add in case it does not existing

Returns

label entities

Return type

dtlpy.entities.label.Label

Example:

dataset.update_labels(label_list=label_list)

upload_annotations(local_path, filters=None, clean=False, remote_root_path='/', export_version=ExportVersion.V1)[source]¶

Upload annotations to dataset.

Prerequisites: You must have a dataset with items that are related to the annotations. The relationship between the dataset and annotations is shown in the name. You must be in the role of an owner or developer.

Parameters

local_path (str) – str - local folder where the annotations files is.
filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
clean (bool) – bool - if True it remove the old annotations
remote_root_path (str) – str - the remote root path to match remote and local items
export_version (str) – V2 - exported items will have original extension in filename, V1 - no original extension in filenames

For example, if the item filepath is a/b/item and remote_root_path is /a the start folder will be b instead of a

Example:

dataset.upload_annotations(dataset='dataset_entity',
                         local_path='local_path',
                         clean=False,
                         export_version=dl.ExportVersion.V1
                         )

class ExpirationOptions(item_max_days: Optional[int] = None)[source]¶

Bases: object

ExpirationOptions object

class IndexDriver(value)[source]¶

Bases: str, Enum

An enumeration.

Driver¶

class Driver(bucket_name, creator, allow_external_delete, allow_external_modification, created_at, region, path, type, integration_id, metadata, name, id, client_api: ApiClient)[source]¶

Bases: BaseEntity

Driver entity

classmethod from_json(_json, client_api, is_fetched=True)[source]¶

Build a Driver entity object from a json

Parameters

_json – _json response from host
client_api – ApiClient entity
is_fetched – is Entity fetched from Platform

Returns

Driver object

to_json()[source]¶

Returns platform _json format of object

Returns: platform json format of object
Return type: dict

class ExternalStorage(value)[source]¶

Bases: str, Enum

An enumeration.

Item¶

class ExportMetadata(value)[source]¶

Bases: Enum

An enumeration.

class Item(annotations_link, dataset_url, thumbnail, created_at, dataset_id, annotated, metadata, filename, stream, name, type, url, id, hidden, dir, spec, creator, annotations_count, client_api: ApiClient, platform_dict, dataset, project, project_id, repositories=NOTHING)[source]¶

Bases: BaseEntity

Item object

clone(dst_dataset_id=None, remote_filepath=None, metadata=None, with_annotations=True, with_metadata=True, with_task_annotations_status=False, allow_many=False, wait=True)[source]¶

Clone item

Parameters

dst_dataset_id (str) – destination dataset id
remote_filepath (str) – complete filepath
metadata (dict) – new metadata to add
with_annotations (bool) – clone annotations
with_metadata (bool) – clone metadata
with_task_annotations_status (bool) – clone task annotations status
allow_many (bool) – bool if True, using multiple clones in single dataset is allowed, (default=False)
wait (bool) – wait for the command to finish

Returns

Item object

Return type

Example:

item.clone(item_id='item_id',
        dst_dataset_id='dist_dataset_id',
        with_metadata=True,
        with_task_annotations_status=False,
        with_annotations=False)

delete()[source]¶

Delete item from platform

Returns: True
Return type: bool

download(local_path=None, file_types=None, save_locally=True, to_array=False, annotation_options: Optional[ViewAnnotationOptions] = None, overwrite=False, to_items_folder=True, thickness=1, with_text=False, annotation_filters=None, alpha=1, export_version=ExportVersion.V1)[source]¶

Download dataset by filters. Filtering the dataset for items and save them local Optional - also download annotation, mask, instance and image mask of the item

Parameters

local_path (str) – local folder or filename to save to.
file_types (list) – a list of file type to download. e.g [‘video/webm’, ‘video/mp4’, ‘image/jpeg’, ‘image/png’]
save_locally (bool) – bool. save to disk or return a buffer
to_array (bool) – returns Ndarray when True and local_path = False
annotation_options (list) – download annotations options: list(dl.ViewAnnotationOptions)
annotation_filters (dtlpy.entities.filters.Filters) – Filters entity to filter annotations for download
overwrite (bool) – optional - default = False
to_items_folder (bool) – Create ‘items’ folder and download items to it
thickness (int) – optional - line thickness, if -1 annotation will be filled, default =1
with_text (bool) – optional - add text to annotations, default = False
alpha (float) – opacity value [0 1], default 1
export_version (str) – exported items will have original extension in filename, V1 - no original extension in filenames

Returns

generator of local_path per each downloaded item

Return type

generator or single item

Example:

item.download(local_path='local_path',
             annotation_options=dl.ViewAnnotationOptions.MASK,
             overwrite=False,
             thickness=1,
             with_text=False,
             alpha=1,
             save_locally=True
             )

classmethod from_json(_json, client_api, dataset=None, project=None, is_fetched=True)[source]¶

Build an item entity object from a json

Parameters

project (dtlpy.entities.project.Project) – project entity
_json (dict) – _json response from host
dataset (dtlpy.entities.dataset.Dataset) – dataset in which the annotation’s item is located
.client_api (dlApiClient) – ApiClient entity
is_fetched (bool) – is Entity fetched from Platform

Returns

Item object

Return type

move(new_path)[source]¶

Move item from one folder to another in Platform If the directory doesn’t exist it will be created

Parameters: new_path (str) – new full path to move item to.
Returns: True if update successfully
Return type: bool

open_in_web()[source]¶

Open the items in web platform

Returns

set_description(text: str)[source]¶

Update Item description

Parameters: text (str) – if None or “” description will be deleted

:return

to_json()[source]¶

Returns platform _json format of object

Returns: platform json format of object
Return type: dict

update(system_metadata=False)[source]¶

Update items metadata

Parameters: system_metadata (bool) – bool - True, if you want to change metadata system
Returns: Item object
Return type: dtlpy.entities.item.Item

update_status(status: str, clear: bool = False, assignment_id: Optional[str] = None, task_id: Optional[str] = None)[source]¶

update item status

Parameters

status (str) – “completed” ,”approved” ,”discard”
clear (bool) – if true delete status
assignment_id (str) – assignment id
task_id (str) – task id

:return :True/False :rtype: bool

Example:

item.update_status(status='complete',
                   operation='created',
                   task_id='task_id')

class ItemStatus(value)[source]¶

Bases: str, Enum

An enumeration.

class ModalityRefTypeEnum(value)[source]¶

Bases: str, Enum

State enum

class ModalityTypeEnum(value)[source]¶

Bases: str, Enum

State enum

Item Link¶

class LinkTypeEnum(value)[source]¶

Bases: str, Enum

State enum

Annotation¶

class Annotation(id, url, item_url, item, item_id, creator, created_at, updated_by, updated_at, type, source, dataset_url, platform_dict, metadata, fps, hash=None, dataset_id=None, status=None, object_id=None, automated=None, item_height=None, item_width=None, label_suggestions=None, annotation_definition: Optional[BaseAnnotationDefinition] = None, frames=None, current_frame=0, end_frame=0, end_time=0, start_frame=0, start_time=0, dataset=None, datasets=None, annotations=None, Annotation__client_api=None, items=None, recipe_2_attributes=None)[source]¶

Bases: BaseEntity

Annotations object

add_frame(annotation_definition, frame_num=None, fixed=True, object_visible=True)[source]¶

Add a frame state to annotation

Parameters

annotation_definition – annotation type object - must be same type as annotation
frame_num (int) – frame number
fixed (bool) – is fixed
object_visible (bool) – does the annotated object is visible

Returns

True if success

Return type

Example:

annotation.add_frame(frame_num=10,
                annotation_definition=dl.Box(top=10,left=10,bottom=100, right=100,label='labelName'))
                )

add_frames(annotation_definition, frame_num=None, end_frame_num=None, start_time=None, end_time=None, fixed=True, object_visible=True)[source]¶

Add a frames state to annotation

Prerequisites: Any user can upload annotations.

Parameters

annotation_definition – annotation type object - must be same type as annotation
frame_num (int) – first frame number
end_frame_num (int) – last frame number
start_time – starting time for video
end_time – ending time for video
fixed (bool) – is fixed
object_visible (bool) – does the annotated object is visible

Returns

Example:

annotation.add_frames(frame_num=10,
                annotation_definition=dl.Box(top=10,left=10,bottom=100, right=100,label='labelName'))
                )

delete()[source]¶

Remove an annotation from item

Prerequisites: You must have an item that has been annotated. You must have the role of an owner or developer or be assigned a task that includes that item as an annotation manager or annotator.

Returns: True if success
Return type: bool

Example:

annotation.delete()

download(filepath: str, annotation_format: ViewAnnotationOptions = ViewAnnotationOptions.MASK, height: Optional[float] = None, width: Optional[float] = None, thickness: int = 1, with_text: bool = False, alpha: float = 1)[source]¶

Save annotation to file

Prerequisites: Any user can upload annotations.

Parameters

filepath (str) – local path to where annotation will be downloaded to
annotation_format (list) – options: list(dl.ViewAnnotationOptions)
height (float) – image height
width (float) – image width
thickness (int) – thickness
with_text (bool) – get mask with text
alpha (float) – opacity value [0 1], default 1

Returns

filepath

Return type

Example:

annotation.download(filepath='filepath', annotation_format=dl.ViewAnnotationOptions.MASK)

classmethod from_json(_json, item=None, client_api=None, annotations=None, is_video=None, fps=None, item_metadata=None, dataset=None, is_audio=None)[source]¶

Create an annotation object from platform json

Parameters

_json (dict) – platform json
item (dtlpy.entities.item.Item) – item
client_api – ApiClient entity
annotations –
is_video (bool) – is video
fps – video fps
item_metadata – item metadata
dataset – dataset entity
is_audio (bool) – is audio

Returns

annotation object

Return type

classmethod new(item=None, annotation_definition=None, object_id=None, automated=True, metadata=None, frame_num=None, parent_id=None, start_time=None, item_height=None, item_width=None)[source]¶

Create a new annotation object annotations

Prerequisites: Any user can upload annotations.

Parameters

item (dtlpy.entities.item.Items) – item to annotate
annotation_definition – annotation type object
object_id (str) – object_id
automated (bool) – is automated
metadata (dict) – metadata
frame_num (int) – optional - first frame number if video annotation
parent_id (str) – add parent annotation ID
start_time – optional - start time if video annotation
item_height (float) – annotation item’s height
item_width (float) – annotation item’s width

Returns

annotation object

Return type

Example:

annotation.new(item='item_entity,
                annotation_definition=dl.Box(top=10,left=10,bottom=100, right=100,label='labelName'))
                )

set_frame(frame)[source]¶

Set annotation to frame state

Prerequisites: Any user can upload annotations.

Parameters: frame (int) – frame number
Returns: True if success
Return type: bool

Example:

annotation.set_frame(frame=10)

show(image=None, thickness=None, with_text=False, height=None, width=None, annotation_format: ViewAnnotationOptions = ViewAnnotationOptions.MASK, color=None, label_instance_dict=None, alpha=1, frame_num=None)[source]¶

Show annotations mark the annotation of the image array and return it

Prerequisites: Any user can upload annotations.

Parameters

image – empty or image to draw on
thickness (int) – line thickness
with_text (bool) – add label to annotation
height (float) – height
width (float) – width
annotation_format (dl.ViewAnnotationOptions) – list(dl.ViewAnnotationOptions)
color (tuple) – optional - color tuple
label_instance_dict – the instance labels
alpha (float) – opacity value [0 1], default 1
frame_num (int) – for video annotation, show specific fame

Returns

list or single ndarray of the annotations

Exampls:

annotation.show(image='ndarray',
                thickness=1,
                annotation_format=dl.VIEW_ANNOTATION_OPTIONS_MASK,
                )

to_json()[source]¶

Convert annotation object to a platform json representatio

Returns: platform json
Return type: dict

update(system_metadata=False)[source]¶

Update an existing annotation in host.

Prerequisites: You must have an item that has been annotated. You must have the role of an owner or developer or be assigned a task that includes that item as an annotation manager or annotator.

Parameters: system_metadata – True, if you want to change metadata system
Returns: Annotation object
Return type: dtlpy.entities.annotation.Annotation

Example:

annotation.update()

update_status(status: AnnotationStatus = AnnotationStatus.ISSUE)[source]¶

Set status on annotation

Prerequisites: You must have an item that has been annotated. You must have the role of an owner or developer or be assigned a task that includes that item as an annotation manager.

Parameters: status (str) – can be AnnotationStatus.ISSUE, AnnotationStatus.APPROVED, AnnotationStatus.REVIEW, AnnotationStatus.CLEAR
Returns: Annotation object
Return type: dtlpy.entities.annotation.Annotation

Example:

annotation.update_status(status=dl.AnnotationStatus.ISSUE)

upload()[source]¶

Create a new annotation in host

Prerequisites: Any user can upload annotations.

Returns: Annotation entity
Return type: dtlpy.entities.annotation.Annotation

class AnnotationStatus(value)[source]¶

Bases: str, Enum

An enumeration.

class AnnotationType(value)[source]¶

Bases: str, Enum

An enumeration.

class ExportVersion(value)[source]¶

Bases: str, Enum

An enumeration.

class FrameAnnotation(annotation, annotation_definition, frame_num, fixed, object_visible, recipe_2_attributes=None, interpolation=False)[source]¶

Bases: BaseEntity

FrameAnnotation object

classmethod from_snapshot(annotation, _json, fps)[source]¶

new frame state to annotation

Parameters

annotation – annotation
_json – annotation type object - must be same type as annotation
fps – frame number

Returns

FrameAnnotation object

classmethod new(annotation, annotation_definition, frame_num, fixed, object_visible=True)[source]¶

new frame state to annotation

Parameters

annotation – annotation
annotation_definition – annotation type object - must be same type as annotation
frame_num – frame number
fixed – is fixed
object_visible – does the annotated object is visible

Returns

FrameAnnotation object

show(**kwargs)[source]¶: Show annotation as ndarray :param kwargs: see annotation definition :return: ndarray of the annotation

class ViewAnnotationOptions(value)[source]¶

Bases: str, Enum

The Annotations file types to download (JSON, MASK, INSTANCE, ANNOTATION_ON_IMAGE, VTT, OBJECT_ID).

State	Description
JSON	Dataloop json format
MASK	PNG file that contains drawing annotations on it
INSTANCE	An image file that contains 2D annotations
ANNOTATION_ON_IMAGE	The source image with the annotations drawing in it
VTT	An text file contains supplementary information about a web video
OBJECT_ID	An image file that contains 2D annotations

Collection of Annotation entities¶

class AnnotationCollection(item=None, annotations=NOTHING, dataset=None, colors=None)[source]¶

Bases: BaseEntity

Collection of Annotation entity

add(annotation_definition, object_id=None, frame_num=None, end_frame_num=None, start_time=None, end_time=None, automated=True, fixed=True, object_visible=True, metadata=None, parent_id=None, model_info=None)[source]¶

Add annotations to collection

Parameters

annotation_definition – dl.Polygon, dl.Segmentation, dl.Point, dl.Box etc
object_id – Object id (any id given by user). If video - must input to match annotations between frames
frame_num – video only, number of frame
end_frame_num – video only, the end frame of the annotation
start_time – video only, start time of the annotation
end_time – video only, end time of the annotation
automated –
fixed – video only, mark frame as fixed
object_visible – video only, does the annotated object is visible
metadata – optional- metadata dictionary for annotation
parent_id – set a parent for this annotation (parent annotation ID)
model_info – optional - set model on annotation {‘name’,:’’, ‘confidence’:0}

Returns

delete()[source]¶

Remove an annotation from item

Prerequisites: You must have an item that has been annotated. You must have the role of an owner or developer or be assigned a task that includes that item as an annotation manager or annotator.

Returns: True if success
Return type: bool

Example:

builder.delete()

download(filepath, img_filepath=None, annotation_format: ViewAnnotationOptions = ViewAnnotationOptions.MASK, height=None, width=None, thickness=1, with_text=False, orientation=0, alpha=1)[source]¶

Save annotations to file

Prerequisites: Any user can upload annotations.

Parameters

filepath (str) – path to save annotation
img_filepath (str) – img file path - needed for img_mask
annotation_format (dl.ViewAnnotationOptions) – how to show thw annotations. options: list(dl.ViewAnnotationOptions)
height (int) – height
width (int) – width
thickness (int) – thickness
with_text (bool) – add a text to the image
orientation (int) – the image orientation
alpha (float) – opacity value [0 1], default 1

Returns

file path of the downlaod annotation

Return type

Example:

builder.download(filepath='filepath', annotation_format=dl.ViewAnnotationOptions.MASK)

from_instance_mask(mask, instance_map=None)[source]¶

convert annotation from instance mask format

Parameters

mask – the mask annotation
instance_map – labels

classmethod from_json(_json: list, item=None, is_video=None, fps=25, height=None, width=None, client_api=None, is_audio=None)[source]¶

Create an annotation collection object from platform json

Parameters

_json (dict) – platform json
item (dtlpy.entities.item.Item) – item
client_api – ApiClient entity
is_video (bool) – is video
fps – video fps
height (float) – height
width (float) – width
is_audio (bool) – is audio

Returns

annotation object

Return type

from_vtt_file(filepath)[source]¶

convert annotation from vtt format

Parameters: filepath (str) – path to the file

get_frame(frame_num)[source]¶

Get frame

Parameters: frame_num (int) – frame num
Returns: AnnotationCollection

print(to_return=False, columns=None)[source]¶

Parameters

to_return –
columns –

show(image=None, thickness=None, with_text=False, height=None, width=None, annotation_format: ViewAnnotationOptions = ViewAnnotationOptions.MASK, label_instance_dict=None, color=None, alpha=1, frame_num=None)[source]¶

Show annotations according to annotation_format

Prerequisites: Any user can upload annotations.

Parameters

image (ndarray) – empty or image to draw on
height (int) – height
width (int) – width
thickness (int) – line thickness
with_text (bool) – add label to annotation
annotation_format (dl.ViewAnnotationOptions) – how to show thw annotations. options: list(dl.ViewAnnotationOptions)
label_instance_dict (dict) – instance label map {‘Label’: 1, ‘More’: 2}
color (tuple) – optional - color tuple
alpha (float) – opacity value [0 1], default 1
frame_num (int) – for video annotation, show specific frame

Returns

ndarray of the annotations

Example:

builder.show(image='ndarray',
            thickness=1,
            annotation_format=dl.VIEW_ANNOTATION_OPTIONS_MASK,
            )

to_json()[source]¶

Convert annotation object to a platform json representation

Returns: platform json
Return type: dict

update(system_metadata=True)[source]¶

Update an existing annotation in host.

Prerequisites: You must have an item that has been annotated. You must have the role of an owner or developer or be assigned a task that includes that item as an annotation manager or annotator.

Parameters: system_metadata – True, if you want to change metadata system
Returns: Annotation object
Return type: dtlpy.entities.annotation.Annotation

Example:

builder.update()

upload()[source]¶

Create a new annotation in host

Prerequisites: Any user can upload annotations.

Returns: Annotation entity
Return type: dtlpy.entities.annotation.Annotation

Example:

builder.upload()

Annotation Definition¶

Box Annotation Definition¶

class Box(left=None, top=None, right=None, bottom=None, label=None, attributes=None, description=None, angle=None)[source]¶

Bases: BaseAnnotationDefinition

Box annotation object Can create a box using 2 point using: “top”, “left”, “bottom”, “right” (to form a box [(left, top), (right, bottom)]) For rotated box add the “angel”

classmethod from_segmentation(mask, label, attributes=None)[source]¶

Convert binary mask to Polygon

Parameters

mask – binary mask (0,1)
label – annotation label
attributes – annotations list of attributes

Returns

Box annotations list to each separated segmentation

show(image, thickness, with_text, height, width, annotation_format, color, alpha=1)[source]¶: Show annotation as ndarray :param image: empty or image to draw on :param thickness: :param with_text: not required :param height: item height :param width: item width :param annotation_format: options: list(dl.ViewAnnotationOptions) :param color: color :param alpha: opacity value [0 1], default 1 :return: ndarray

Classification Annotation Definition¶

class Classification(label, attributes=None, description=None)[source]¶

Bases: BaseAnnotationDefinition

Classification annotation object

show(image, thickness, with_text, height, width, annotation_format, color, alpha=1)[source]¶: Show annotation as ndarray :param image: empty or image to draw on :param thickness: :param with_text: not required :param height: item height :param width: item width :param annotation_format: options: list(dl.ViewAnnotationOptions) :param color: color :param alpha: opacity value [0 1], default 1 :return: ndarray

Cuboid Annotation Definition¶

class Cube(label, front_tl, front_tr, front_br, front_bl, back_tl, back_tr, back_br, back_bl, angle=None, attributes=None, description=None)[source]¶

Bases: BaseAnnotationDefinition

Cube annotation object

classmethod from_boxes_and_angle(front_left, front_top, front_right, front_bottom, back_left, back_top, back_right, back_bottom, label, angle=0, attributes=None)[source]¶: Create cuboid by given front and back boxes with angle the angle calculate fom the center of each box

show(image, thickness, with_text, height, width, annotation_format, color, alpha=1)[source]¶: Show annotation as ndarray :param image: empty or image to draw on :param thickness: :param with_text: not required :param height: item height :param width: item width :param annotation_format: options: list(dl.ViewAnnotationOptions) :param color: color :param alpha: opacity value [0 1], default 1 :return: ndarray

Item Description Definition¶

class Description(text, description=None)[source]¶

Bases: BaseAnnotationDefinition

Subtitle annotation object

Ellipse Annotation Definition¶

class Ellipse(x, y, rx, ry, angle, label, attributes=None, description=None)[source]¶

Bases: BaseAnnotationDefinition

Ellipse annotation object

show(image, thickness, with_text, height, width, annotation_format, color, alpha=1)[source]¶: Show annotation as ndarray :param image: empty or image to draw on :param thickness: :param with_text: not required :param height: item height :param width: item width :param annotation_format: options: list(dl.ViewAnnotationOptions) :param color: color :param alpha: opacity value [0 1], default 1 :return: ndarray

Note Annotation Definition¶

class Message(msg_id: Optional[str] = None, creator: Optional[str] = None, msg_time=None, body: Optional[str] = None)[source]¶

Bases: object

Note message object

class Note(left, top, right, bottom, label, attributes=None, messages=None, status='issue', assignee=None, create_time=None, creator=None, description=None)[source]¶

Bases: Box

Note annotation object

Point Annotation Definition¶

class Point(x, y, label, attributes=None, description=None)[source]¶

Bases: BaseAnnotationDefinition

Point annotation object

show(image, thickness, with_text, height, width, annotation_format, color, alpha=1)[source]¶: Show annotation as ndarray :param image: empty or image to draw on :param thickness: :param with_text: not required :param height: item height :param width: item width :param annotation_format: options: list(dl.ViewAnnotationOptions) :param color: color :param alpha: opacity value [0 1], default 1 :return: ndarray

Polygon Annotation Definition¶

class Polygon(geo, label, attributes=None, description=None)[source]¶

Bases: BaseAnnotationDefinition

Polygon annotation object

classmethod from_segmentation(mask, label, attributes=None, epsilon=None, max_instances=1, min_area=0)[source]¶

Convert binary mask to Polygon

Parameters

mask – binary mask (0,1)
label – annotation label
attributes – annotations list of attributes
epsilon – from opencv: specifying the approximation accuracy. This is the maximum distance between the original curve and its approximation. if 0 all points are returns
max_instances – number of max instances to return. if None all wil be returned
min_area – remove polygons with area lower thn this threshold (pixels)

Returns

Polygon annotation

show(image, thickness, with_text, height, width, annotation_format, color, alpha=1)[source]¶: Show annotation as ndarray :param image: empty or image to draw on :param thickness: :param with_text: not required :param height: item height :param width: item width :param annotation_format: options: list(dl.ViewAnnotationOptions) :param color: color :param alpha: opacity value [0 1], default 1 :return: ndarray

Polyline Annotation Definition¶

class Polyline(geo, label, attributes=None, description=None)[source]¶

Bases: BaseAnnotationDefinition

Polyline annotation object

show(image, thickness, with_text, height, width, annotation_format, color, alpha=1)[source]¶: Show annotation as ndarray :param image: empty or image to draw on :param thickness: :param with_text: not required :param height: item height :param width: item width :param annotation_format: options: list(dl.ViewAnnotationOptions) :param color: color :param alpha: opacity value [0 1], default 1 :return: ndarray

Pose Annotation Definition¶

class Pose(label, template_id, instance_id=None, attributes=None, points=None, description=None)[source]¶

Bases: BaseAnnotationDefinition

Classification annotation object

show(image, thickness, with_text, height, width, annotation_format, color, alpha=1)[source]¶: Show annotation as ndarray :param image: empty or image to draw on :param thickness: :param with_text: not required :param height: item height :param width: item width :param annotation_format: options: list(dl.ViewAnnotationOptions) :param color: color :param alpha: opacity value [0 1], default 1 :return: ndarray

Segmentation Annotation Definition¶

class Segmentation(geo, label, attributes=None, description=None)[source]¶

Bases: BaseAnnotationDefinition

Segmentation annotation object

classmethod from_polygon(geo, label, shape, attributes=None)[source]¶

Parameters

geo – list of x,y coordinates of the polygon ([[x,y],[x,y]…]
label – annotation’s label
shape – image shape (h,w)
attributes –

Returns

show(image, thickness, with_text, height, width, annotation_format, color, alpha=1)[source]¶: Show annotation as ndarray :param image: empty or image to draw on :param thickness: :param with_text: not required :param height: item height :param width: item width :param annotation_format: options: list(dl.ViewAnnotationOptions) :param color: color :param alpha: opacity value [0 1], default 1 :return: ndarray

to_box()[source]¶

Returns: Box annotations list to each separated segmentation

Audio Annotation Definition¶

class Subtitle(text, label, attributes=None, description=None)[source]¶

Bases: BaseAnnotationDefinition

Subtitle annotation object

Undefined Annotation Definition¶

class UndefinedAnnotationType(type, label, coordinates, attributes=None, description=None)[source]¶

Bases: BaseAnnotationDefinition

UndefinedAnnotationType annotation object

show(image, thickness, with_text, height, width, annotation_format, color, alpha=1)[source]¶: Show annotation as ndarray :param image: empty or image to draw on :param thickness: :param with_text: not required :param height: item height :param width: item width :param annotation_format: options: list(dl.ViewAnnotationOptions) :param color: color :param alpha: opacity value [0 1], default 1 :return: ndarray

Similarity¶

class Collection(type: CollectionTypes, name, items=None)[source]¶

Bases: object

Base Collection Entity

add(ref, type: SimilarityTypeEnum = SimilarityTypeEnum.ID)[source]¶: Add item to collection :param ref: :param type: url, id

pop(ref)[source]¶

Parameters: ref –

to_json()[source]¶

Returns platform _json format of object

Returns: platform json format of object
Return type: dict

class CollectionItem(type: SimilarityTypeEnum, ref)[source]¶

Bases: object

Base CollectionItem

class CollectionTypes(value)[source]¶

Bases: str, Enum

An enumeration.

class MultiView(name, items=None)[source]¶

Bases: Collection

Multi Entity

property items¶: list of the collection items

to_json()[source]¶

Returns platform _json format of object

Returns: platform json format of object
Return type: dict

class MultiViewItem(type, ref)[source]¶

Bases: CollectionItem

Single multi view item

class Similarity(ref, name=None, items=None)[source]¶

Bases: Collection

Similarity Entity

property items¶: list of the collection items

property target¶: Target item for similarity

to_json()[source]¶

Returns platform _json format of object

Returns: platform json format of object
Return type: dict

class SimilarityItem(type, ref, target=False)[source]¶

Bases: CollectionItem

Single similarity item

class SimilarityTypeEnum(value)[source]¶

Bases: str, Enum

State enum

Filter¶

class Filters(field=None, values=None, operator: Optional[FiltersOperations] = None, method: Optional[FiltersMethod] = None, custom_filter=None, resource: FiltersResource = FiltersResource.ITEM, use_defaults=True, context=None, page_size=None)[source]¶

Bases: object

Filters entity to filter items from pages in platform

add(field, values, operator: Optional[FiltersOperations] = None, method: Optional[FiltersMethod] = None)[source]¶

Add filter

Parameters

field (str) – Metadata field / attribute
values (str or list) – field values
operator (dl.FiltersOperations) – optional - in, gt, lt, eq, ne
method (dl.FiltersMethod) – Optional - or/and

Example:

filter.add(field='metadata.user', values=['1','2'], operator=dl.FiltersOperations.IN)

add_join(field, values, operator: Optional[FiltersOperations] = None, method: FiltersMethod = FiltersMethod.AND)[source]¶

join a query to the filter

Parameters

field (str) – Metadata field / attribute
values (str or list) – field values
operator (dl.FiltersOperations) – optional - in, gt, lt, eq, ne
method – optional - str - FiltersMethod.AND, FiltersMethod.OR

Example:

filter.add_join(field='metadata.user', values=['1','2'], operator=dl.FiltersOperations.IN)

generate_url_query_params(url)[source]¶

generate url query params

Parameters: url (str) –

has_field(field)[source]¶

is filter has field

Parameters: field (str) – field to check
Returns: Ture is have it
Return type: bool

open_in_web(resource)[source]¶

Open the filter in the platform data browser (in a new web browser)

Parameters: resource (str) – dl entity to apply filter on. currently only supports dl.Dataset

platform_url(resource) → str[source]¶

Build a url with filters param to open in web browser

Parameters: resource (str) – dl entity to apply filter on. currently only supports dl.Dataset
Returns: url string
Return type: str

pop(field)[source]¶

Pop filed

Parameters: field (str) – field to pop

pop_join(field)[source]¶

Pop join

Parameters: field (str) – field to pop

prepare(operation=None, update=None, query_only=False, system_update=None, system_metadata=False)[source]¶

To dictionary for platform call

Parameters

operation (str) – operation
update – update
query_only (bool) – query only
system_update – system update
system_metadata – True, if you want to change metadata system

Returns

dict of the filter

Return type

dtlpy.entities.ontology.Ontology

sort_by(field, value: FiltersOrderByDirection = FiltersOrderByDirection.ASCENDING)[source]¶

sort the filter

Parameters

field (str) – field to sort by it
value (dl.FiltersOrderByDirection) – FiltersOrderByDirection.ASCENDING, FiltersOrderByDirection.DESCENDING

Example:

filter.sort_by(field='metadata.user', values=dl.FiltersOrderByDirection.ASCENDING)

class FiltersKnownFields(value)[source]¶

Bases: str, Enum

An enumeration.

class FiltersMethod(value)[source]¶

Bases: str, Enum

An enumeration.

class FiltersOperations(value)[source]¶

Bases: str, Enum

An enumeration.

class FiltersOrderByDirection(value)[source]¶

Bases: str, Enum

An enumeration.

class FiltersResource(value)[source]¶

Bases: str, Enum

An enumeration.

Recipe¶

class Recipe(id, creator, url, title, project_ids, description, ontology_ids, instructions, examples, custom_actions, metadata, ui_settings, client_api: ApiClient, dataset=None, project=None, repositories=NOTHING)[source]¶

Bases: BaseEntity

Recipe object

clone(shallow=False)[source]¶

Clone Recipe

Parameters: shallow (bool) – If True, link ot existing ontology, clones all ontology that are link to the recipe as well
Returns: Cloned ontology object
Return type: dtlpy.entities.recipe.Recipe

delete(force: bool = False)[source]¶

Delete recipe from platform

Parameters: force (bool) – force delete recipe
Returns: True
Return type: bool

classmethod from_json(_json, client_api, dataset=None, project=None, is_fetched=True)[source]¶

Build a Recipe entity object from a json

Parameters

_json (dict) – _json response from host
Dataset (dtlpy.entities.dataset.Dataset) – Dataset entity
project (dtlpy.entities.project.Project) – project entity
client_api (dl.ApiClient) – ApiClient entity
is_fetched (bool) – is Entity fetched from Platform

Returns

Recipe object

get_annotation_template_id(template_name)[source]¶

Get annotation template id by template name

Parameters: template_name (str) –
Returns: template id or None if does not exist

open_in_web()[source]¶

Open the recipes in web platform

Returns

to_json()[source]¶

Returns platform _json format of object

Returns: platform json format of object
Return type: dict

update(system_metadata=False)[source]¶

Update Recipe

Parameters: system_metadata (bool) – bool - True, if you want to change metadata system
Returns: Recipe object
Return type: dtlpy.entities.recipe.Recipe

Ontology¶

class Ontology(client_api: ApiClient, id, creator, url, title, labels, metadata, attributes, recipe=None, dataset=None, project=None, repositories=NOTHING, instance_map=None, color_map=None)[source]¶

Bases: BaseEntity

Ontology object

add_label(label_name, color=None, children=None, attributes=None, display_label=None, label=None, add=True, icon_path=None, update_ontology=False)[source]¶

Add a single label to ontology

Parameters

label_name (str) – str - label name
color (tuple) – color
children – children (sub labels)
attributes (list) – attributes
display_label (str) – display_label
label (dtlpy.entities.label.Label) – label
add (bool) – to add or not
icon_path (str) – path to image to be display on label
update_ontology (bool) – update the ontology, default = False for backward compatible

Returns

Label entity

Return type

dtlpy.entities.label.Label

Example:

ontology.add_label(label_name='person', color=(34, 6, 231), attributes=['big', 'small'])

add_labels(label_list, update_ontology=False)[source]¶

Adds a list of labels to ontology

Parameters

label_list (list) – list of labels [{“value”: {“tag”: “tag”, “displayLabel”: “displayLabel”, “color”: “#color”, “attributes”: [attributes]}, “children”: [children]}]
update_ontology (bool) – update the ontology, default = False for backward compatible

Returns

List of label entities added

Example:

ontology.add_labels(label_list=label_list)

property color_map¶

rgb}

Returns: dict
Return type: dict
Type: Color mapping of labels, {label

delete()[source]¶

Delete recipe from platform

Returns: True

delete_attributes(keys: list)[source]¶

Delete a bulk of attributes

Parameters: keys (list) – Keys of attributes to delete
Returns: True if success
Return type: bool

Example:

ontology.delete_attributes(['1'])

delete_labels(label_names)[source]¶

Delete labels from ontology

Parameters: label_names – label object/ label name / list of label objects / list of label names
Returns

classmethod from_json(_json, client_api, recipe, dataset=None, project=None, is_fetched=True)[source]¶

Build an Ontology entity object from a json

Parameters

is_fetched (bool) – is Entity fetched from Platform
project (dtlpy.entities.project.Project) – project entity
dataset (dtlpy.entities.dataset.Dataset) – dataset
_json (dict) – _json response from host
recipe (dtlpy.entities.recipe.Recipe) – ontology’s recipe
client_api (dl.ApiClient) – ApiClient entity

Returns

Ontology object

Return type

property instance_map¶

instance mapping for creating instance mask

Return dictionary {label: map_id}
Return type: dict

to_json()[source]¶

Returns platform _json format of object

Returns: platform json format of object
Return type: dict

update(system_metadata=False)[source]¶

Update items metadata

Parameters: system_metadata (bool) – bool - True, if you want to change metadata system
Returns: Ontology object

update_attributes(title: str, key: str, attribute_type, scope: Optional[list] = None, optional: Optional[bool] = None, multi: Optional[bool] = None, values: Optional[list] = None, attribute_range=None)[source]¶

ADD a new attribute or update if exist

Parameters

title (str) – attribute title
key (str) – the key of the attribute must br unique
attribute_type (AttributesTypes) – dl.AttributesTypes your attribute type
scope (list) – list of the labels or * for all labels
optional (bool) – optional attribute
multi (bool) – if can get multiple selection
values (list) – list of the attribute values ( for checkbox and radio button)
attribute_range (dict or AttributesRange) – dl.AttributesRange object

Returns

true in success

Return type

update_label(label_name, color=None, children=None, attributes=None, display_label=None, label=None, add=True, icon_path=None, upsert=False, update_ontology=False)[source]¶

Update a single label to ontology

Parameters

label_name (str) – str - label name
color (tuple) – color
children – children (sub labels)
attributes (list) – attributes
display_label (str) – display_label
label (dtlpy.entities.label.Label) – label
add (bool) – to add or not
icon_path (str) – path to image to be display on label
update_ontology (bool) – update the ontology, default = False for backward compatible
upsert (bool) – if True will add in case it does not existing

Returns

Label entity

Return type

dtlpy.entities.label.Label

Example:

ontology.update_label(label_name='person', color=(34, 6, 231), attributes=['big', 'small'])

update_labels(label_list, upsert=False, update_ontology=False)[source]¶

Update a list of labels to ontology

Parameters

label_list (list) – list of labels [{“value”: {“tag”: “tag”, “displayLabel”: “displayLabel”, “color”: “#color”, “attributes”: [attributes]}, “children”: [children]}]
upsert (bool) – if True will add in case it does not existing
update_ontology (bool) – update the ontology, default = False for backward compatible

Returns

List of label entities added

Example:

ontology.update_labels(label_list=label_list)

Label¶

Task¶

class Task(name, status, project_id, metadata, id, url, task_owner, item_status, creator, due_date, dataset_id, spec, recipe_id, query, assignmentIds, annotation_status, progress, for_review, issues, updated_at, created_at, available_actions, total_items, client_api, current_assignments=None, assignments=None, project=None, dataset=None, tasks=None, settings=None)[source]¶

Bases: object

Task object

add_items(filters=None, items=None, assignee_ids=None, workload=None, limit=None, wait=True, query=None)[source]¶

Add items to Task

Parameters

filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
items (list) – list of items to add to the task
assignee_ids (list) – list to assignee who works in the task
workload (list) – list of the work load ber assignee and work load
limit (int) – task limit
wait (bool) – wait for the command to finish
query (dict) – query to filter the items use it

Returns

task entity

Return type

create_assignment(assignment_name, assignee_id, items=None, filters=None)[source]¶

Create a new assignment

Parameters

assignment_name (str) – assignment name
assignee_id (list) – list of assignee for the assignment
items (list) – items list for the assignment
filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters

Returns

Assignment object

Return type

dtlpy.entities.assignment.Assignment assignment

Example:

task.create_assignment(assignee_id='annotator1@dataloop.ai')

create_qa_task(due_date, assignee_ids, filters=None, items=None, query=None, workload=None, metadata=None, available_actions=None, wait=True, batch_size=None, max_batch_workload=None, allowed_assignees=None)[source]¶

Create a new QA Task

Parameters

due_date (float) – date to when finish the task
assignee_ids (list) – list of assignee
filters (entities.Filters) – filter to the task
items (List[entities.Item]) – item to insert to the task
query (entities.Filters) – filter to the task
workload (List[WorkloadUnit]) – list WorkloadUnit for the task assignee
metadata (dict) – metadata for the task
available_actions (list) – list of available actions to the task
wait (bool) – wait for the command to finish
batch_size (int) – Pulling batch size (items) . Restrictions - Min 3, max 100
max_batch_workload (int) – Max items in assignment . Restrictions - Min batchSize + 2 , max batchSize * 2
allowed_assignees (list) – It’s like the workload, but without percentage.

Returns

task object

Return type

Example:

task.create_qa_task(due_date = datetime.datetime(day= 1, month= 1, year= 2029).timestamp(),
                    assignee_ids =[ 'annotator1@dataloop.ai', 'annotator2@dataloop.ai'])

delete(wait=True)[source]¶

Delete task from platform

Parameters: wait (bool) – wait for the command to finish
Returns: True
Return type: bool

get_items(filters=None)[source]¶

Get the task items

Parameters: filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
Returns: list of the items or PagedEntity output of items
Return type: list or dtlpy.entities.paged_entities.PagedEntities

open_in_web()[source]¶

Open the task in web platform

Returns

remove_items(filters: Optional[Filters] = None, query=None, items=None, wait=True)[source]¶

remove items from Task.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned to be owner of the annotation task.

Parameters

filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
query (dict) – query yo filter the items use it
items (list) – list of items to add to the task
wait (bool) – wait for the command to finish

Returns

task entity

Return type

set_status(status: str, operation: str, item_ids: List[str])[source]¶

Update item status within task

Parameters

status (str) – string the describes the status
operation (str) – ‘create’ or ‘delete’
item_ids (list) – List[str] id items ids

Returns

True if success

Return type

to_json()[source]¶

Returns platform _json format of object

Returns: platform json format of object
Return type: dict

update(system_metadata=False)[source]¶

Update an Annotation Task

Parameters: system_metadata (bool) – True, if you want to change metadata system

Assignment¶

class Assignment(name, annotator, status, project_id, metadata, id, url, task_id, dataset_id, annotation_status, item_status, total_items, for_review, issues, client_api, task=None, assignments=None, project=None, dataset=None, datasets=None)[source]¶

Bases: BaseEntity

Assignment object

get_items(dataset=None, filters=None)[source]¶

Get all the items in the assignment

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned as owner of the annotation task.

Parameters

dataset (dtlpy.entities.dataset.Dataset) – dataset entity
filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters

Returns

pages of the items

Return type

dtlpy.entities.assignment.Assignment

Example:

task.assignments.get_items()

open_in_web()[source]¶

Open the assignment in web platform

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned as owner of the annotation task.

Returns

Example:

assignment.open_in_web()

reassign(assignee_id, wait=True)[source]¶

Reassign an assignment

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned as owner of the annotation task.

Parameters

assignee_id (str) – the user that assignee the assignment to it
wait (bool) – wait for the command to finish

Returns

Assignment object

Return type

Example:

assignment.reassign(assignee_ids='annotator1@dataloop.ai')

redistribute(workload, wait=True)[source]¶

Redistribute an assignment

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned as owner of the annotation task.

Parameters

workload (dtlpy.entities.assignment.Workload) – workload object that contain the assignees and the work load
wait (bool) – wait for the command to finish

Returns

Assignment object

Return type

dtlpy.entities.assignment.Assignment assignment

Example:

assignment.redistribute(workload=dl.Workload([dl.WorkloadUnit(assignee_id="annotator1@dataloop.ai", load=50),
                                             dl.WorkloadUnit(assignee_id="annotator2@dataloop.ai", load=50)]))

set_status(status: str, operation: str, item_id: str)[source]¶

Set item status within assignment

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned as owner of the annotation task.

Parameters

status (str) – status
operation (str) – created/deleted
item_id (str) – item id

Returns

True id success

Return type

Example:

assignment.set_status(status='complete',
                        operation='created',
                        item_id='item_id')

to_json()[source]¶

Returns platform _json format of object

Returns: platform json format of object
Return type: dict

update(system_metadata=False)[source]¶

Update an assignment

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned as owner of the annotation task.

Parameters: system_metadata (bool) – True, if you want to change metadata system
Returns: Assignment object
Return type: dtlpy.entities.assignment.Assignment assignment

Example:

assignment.update(system_metadata=False)

class Workload(workload: list = NOTHING)[source]¶

Bases: object

Workload object

add(assignee_id)[source]¶

add a assignee

Parameters: assignee_id –

classmethod generate(assignee_ids, loads=None)[source]¶: generate the loads for the given assignee :param assignee_ids: :param loads:

class WorkloadUnit(assignee_id: str, load: float = 0)[source]¶

Bases: object

WorkloadUnit object

Package¶

class Package(id, url, version, created_at, updated_at, name, codebase, modules, slots: list, ui_hooks, creator, is_global, type, service_config, project_id, project, client_api: ApiClient, revisions=None, repositories=NOTHING, artifacts=None, codebases=None, requirements=None)[source]¶

Bases: BaseEntity

Package object

checkout()[source]¶

Checkout as package

Returns

delete()[source]¶

Delete Package object

Returns: True

deploy(service_name=None, revision=None, init_input=None, runtime=None, sdk_version=None, agent_versions=None, verify=True, bot=None, pod_type=None, module_name=None, run_execution_as_process=None, execution_timeout=None, drain_time=None, on_reset=None, max_attempts=None, force=False, secrets: Optional[list] = None, **kwargs)[source]¶

Deploy package

Parameters

service_name (str) – service name
revision (str) – package revision - default=latest
init_input – config to run at startup
runtime (dict) – runtime resources
sdk_version (str) –
- optional - string - sdk version
agent_versions (dict) –
- dictionary - - optional -versions of sdk, agent runner and agent proxy
bot (str) – bot email
pod_type (str) – pod type dl.InstanceCatalog
verify (bool) – verify the inputs
module_name (str) – module name
run_execution_as_process (bool) – run execution as process
execution_timeout (int) – execution timeout
drain_time (int) – drain time
on_reset (str) – on reset
max_attempts (int) – Maximum execution retries in-case of a service reset
force (bool) – optional - terminate old replicas immediately
secrets (list) – list of the integrations ids

Returns

Service object

Return type

Example:

package.deploy(service_name=package_name,
                execution_timeout=3 * 60 * 60,
                module_name=module.name,
                runtime=dl.KubernetesRuntime(
                    concurrency=10,
                    pod_type=dl.InstanceCatalog.REGULAR_S,
                    autoscaler=dl.KubernetesRabbitmqAutoscaler(
                        min_replicas=1,
                        max_replicas=20,
                        queue_length=20
                    )
                )
            )

classmethod from_json(_json, client_api, project, is_fetched=True)[source]¶

Turn platform representation of package into a package entity

Parameters

_json (dict) – platform representation of package
client_api (dl.ApiClient) – ApiClient entity
project (dtlpy.entities.project.Project) – project entity
is_fetched – is Entity fetched from Platform

Returns

Package entity

Return type

open_in_web()[source]¶: Open the package in web platform

pull(version=None, local_path=None)[source]¶

Pull local package

Parameters

version (str) – version
local_path (str) – local path

Example:

package.pull(local_path='local_path')

push(codebase: Optional[Union[GitCodebase, ItemCodebase]] = None, src_path: Optional[str] = None, package_name: Optional[str] = None, modules: Optional[list] = None, checkout: bool = False, revision_increment: Optional[str] = None, service_update: bool = False, service_config: Optional[dict] = None)[source]¶

Push local package

Parameters

codebase (dtlpy.entities.codebase.Codebase) – PackageCode object - defines how to store the package code
checkout (bool) – save package to local checkout
src_path (str) – location of pacjage codebase folder to zip
package_name (str) – name of package
modules (list) – list of PackageModule
revision_increment (str) – optional - str - version bumping method - major/minor/patch - default = None
service_update (bool) – optional - bool - update the service
service_config (dict) – optional - json of service - a service that have config from the main service if wanted

Returns

package entity

Return type

Example:

packages.push(package_name='package_name',
            modules=[module],
            version='1.0.0',
            src_path=os.getcwd()
        )

test(cwd=None, concurrency=None, module_name='default_module', function_name='run', class_name='ServiceRunner', entry_point='main.py')[source]¶

Test local package in local environment.

Parameters

cwd (str) – path to the file
concurrency (int) – the concurrency of the test
module_name (str) – module name
function_name (str) – function name
class_name (str) – class name
entry_point (str) – the file to run like main.py

Returns

list created by the function that tested the output

Return type

Example:

package.test(cwd='path_to_package',
            function_name='run')

to_json()[source]¶

Turn Package entity into a platform representation of Package

Returns: platform json of package
Return type: dict

update()[source]¶

Update Package changes to platform

Returns: Package entity

class RequirementOperator(value)[source]¶

Bases: str, Enum

An enumeration.

Package Function¶

class PackageFunction(outputs=NOTHING, name=NOTHING, description='', inputs=NOTHING, display_name=None, display_icon=None)[source]¶

Bases: BaseEntity

Webhook object

class PackageInputType(value)[source]¶

Bases: str, Enum

An enumeration.

Package Module¶

class PackageModule(name=NOTHING, init_inputs=NOTHING, entry_point='main.py', class_name='ServiceRunner', functions=NOTHING)[source]¶

Bases: BaseEntity

PackageModule object

add_function(function)[source]¶

Parameters: function –

Slot¶

class PackageSlot(module_name='default_module', function_name='run', display_name=None, display_scopes: Optional[list] = None, display_icon=None, post_action: SlotPostAction = NOTHING, default_inputs: Optional[list] = None, input_options: Optional[list] = None)[source]¶

Bases: BaseEntity

Webhook object

class SlotDisplayScopeResource(value)[source]¶

Bases: str, Enum

An enumeration.

class SlotPostActionType(value)[source]¶

Bases: str, Enum

An enumeration.

class UiBindingPanel(value)[source]¶

Bases: str, Enum

An enumeration.

Codebase¶

Service¶

class InstanceCatalog(value)[source]¶

Bases: str, Enum

The Service Pode size.

State	Description
REGULAR_XS	regular pod with extra small size
REGULAR_S	regular pod with small size
REGULAR_M	regular pod with medium size
REGULAR_L	regular pod with large size
REGULAR_XL	regular pod with extra large size
HIGHMEM_XS	highmem pod with extra small size
HIGHMEM_S	highmem pod with small size
HIGHMEM_M	highmem pod with medium size
HIGHMEM_L	highmem pod with large size
HIGHMEM_XL	highmem pod with extra large size
GPU_K80_S	GPU pod with small size
GPU_K80_M	GPU pod with medium size

class KubernetesAutuscalerType(value)[source]¶

Bases: str, Enum

The Service Autuscaler Type (RABBITMQ, CPU).

State	Description
RABBITMQ	Service Autuscaler will be in RABBITMQ
CPU	Service Autuscaler will be in in local CPU

class OnResetAction(value)[source]¶

Bases: str, Enum

The Execution action when the service reset (RERUN, FAILED).

State	Description
RERUN	When the service resting rerun the execution
FAILED	When the service resting fail the execution

class RuntimeType(value)[source]¶

Bases: str, Enum

Service culture Runtime (KUBERNETES).

State	Description
KUBERNETES	Service run in kubernetes culture

class Service(created_at, updated_at, creator, version, package_id, package_revision, bot, use_user_jwt, init_input, versions, module_name, name, url, id, active, driver_id, secrets, runtime, queue_length_limit, run_execution_as_process: bool, execution_timeout, drain_time, on_reset: OnResetAction, project_id, is_global, max_attempts, package, client_api: ApiClient, revisions=None, project=None, repositories=NOTHING)[source]¶

Bases: BaseEntity

Service object

activate_slots(project_id: Optional[str] = None, task_id: Optional[str] = None, dataset_id: Optional[str] = None, org_id: Optional[str] = None, user_email: Optional[str] = None, slots=None, role=None, prevent_override: bool = True, visible: bool = True, icon: str = 'fas fa-magic', **kwargs) → object[source]¶

Activate service slots

Parameters

project_id (str) – project id
task_id (str) – task id
dataset_id (str) – dataset id
org_id (str) – org id
user_email (str) – user email
slots (list) – list of entities.PackageSlot
role (str) – user role MemberOrgRole.ADMIN, MemberOrgRole.owner, MemberOrgRole.MEMBER
prevent_override (bool) – True to prevent override
visible (bool) – visible
icon (str) – icon
kwargs – all additional arguments

Returns

list of user setting for activated slots

Return type

Example:

service.activate_slots(project_id='project_id',
                        slots=List[entities.PackageSlot],
                        icon='fas fa-magic')

checkout()[source]¶

Checkout

Returns

delete()[source]¶

Delete Service object

Returns: True
Return type: bool

execute(execution_input=None, function_name=None, resource=None, item_id=None, dataset_id=None, annotation_id=None, project_id=None, sync=False, stream_logs=True, return_output=True)[source]¶

Execute a function on an existing service

Parameters

execution_input (List[FunctionIO] or dict) – input dictionary or list of FunctionIO entities
function_name (str) – function name to run
resource (str) – input type.
item_id (str) – optional - item id as input to function
dataset_id (str) – optional - dataset id as input to function
annotation_id (str) – optional - annotation id as input to function
project_id (str) – resource’s project
sync (bool) – if true, wait for function to end
stream_logs (bool) – prints logs of the new execution. only works with sync=True
return_output (bool) – if True and sync is True - will return the output directly

Returns

execution object

Return type

Example:

service.execute(function_name='function_name', item_id='item_id', project_id='project_id')

classmethod from_json(_json: dict, client_api: ApiClient, package=None, project=None, is_fetched=True)[source]¶

Build a service entity object from a json

Parameters

_json (dict) – platform json
client_api (dl.ApiClient) – ApiClient entity
package (dtlpy.entities.package.Package) – package entity
project (dtlpy.entities.project.Project) – project entity
is_fetched (bool) – is Entity fetched from Platform

Returns

service object

Return type

log(size=None, checkpoint=None, start=None, end=None, follow=False, text=None, execution_id=None, function_name=None, replica_id=None, system=False, view=True, until_completed=True)[source]¶

Get service logs

Parameters

size (int) – size
checkpoint (dict) – the information from the lst point checked in the service
start (str) – iso format time
end (str) – iso format time
follow (bool) – if true, keep stream future logs
text (str) – text
execution_id (str) – execution id
function_name (str) – function name
replica_id (str) – replica id
system (bool) – system
view (bool) – if true, print out all the logs
until_completed (bool) – wait until completed

Returns

ServiceLog entity

Return type

ServiceLog

Example:

service.log()

open_in_web()[source]¶

Open the service in web platform

Returns

pause()[source]¶

Returns

resume()[source]¶

Returns

status()[source]¶

Get Service status

Returns: status json
Return type: dict

to_json()[source]¶

Returns platform _json format of object

Returns: platform json format of object
Return type: dict

update(force=False)[source]¶

Update Service changes to platform

Parameters: force (bool) – force update
Returns: Service entity
Return type: dtlpy.entities.service.Service

Bot¶

class Bot(created_at, updated_at, name, last_name, username, avatar, email, role, type, org, id, project, client_api=None, users=None, bots=None, password=None)[source]¶

Bases: User

Bot entity

delete()[source]¶

Delete the bot

Returns: True
Return type: bool

classmethod from_json(_json, project, client_api, bots=None)[source]¶

Build a Bot entity object from a json

Parameters

_json – _json response from host
project – project entity
client_api – ApiClient entity
bots – Bots repository

Returns

User object

to_json()[source]¶

Returns platform _json format of object

Returns: platform json format of object
Return type: dict

Trigger¶

class BaseTrigger(id, url, created_at, updated_at, creator, name, active, type, scope, is_global, input, function_name, service_id, webhook_id, pipeline_id, special, project_id, spec, service, project, client_api: ApiClient, op_type='service', repositories=NOTHING)[source]¶

Bases: BaseEntity

Trigger Entity

delete()[source]¶

Delete Trigger object

Returns: True

classmethod from_json(_json, client_api, project, service=None)[source]¶

Build a trigger entity object from a json

Parameters

_json (dict) – platform json
client_api (dl.ApiClient) – ApiClient entity
project (dtlpy.entities.project.Project) – project entity
service (dtlpy.entities.service.Service) – service entity

Returns

to_json()[source]¶

Returns platform _json format of object

Returns: platform json format of object
Return type: dict

update()[source]¶

Update Trigger object

Returns: Trigger entity

class CronTrigger(id, url, created_at, updated_at, creator, name, active, type, scope, is_global, input, function_name, service_id, webhook_id, pipeline_id, special, project_id, spec, service, project, client_api: ApiClient, op_type='service', repositories=NOTHING, start_at=None, end_at=None, cron=None)[source]¶

Bases: BaseTrigger

classmethod from_json(_json, client_api, project, service=None)[source]¶

Build a trigger entity object from a json

Parameters

_json – platform json
client_api – ApiClient entity
project – project entity
service – service entity

Returns

to_json()[source]¶

Returns platform _json format of object

Returns: platform json format of object
Return type: dict

class Trigger(id, url, created_at, updated_at, creator, name, active, type, scope, is_global, input, function_name, service_id, webhook_id, pipeline_id, special, project_id, spec, service, project, client_api: ApiClient, op_type='service', repositories=NOTHING, filters=None, execution_mode=TriggerExecutionMode.ONCE, actions=TriggerAction.CREATED, resource=TriggerResource.ITEM)[source]¶

Bases: BaseTrigger

Trigger Entity

classmethod from_json(_json, client_api, project, service=None)[source]¶

Build a trigger entity object from a json

Parameters

_json – platform json
client_api – ApiClient entity
project (dtlpy.entities.project.Project) – project entity
service (dtlpy.entities.service.Service) – service entity

Returns

to_json()[source]¶

Returns platform _json format of object

Returns: platform json format of object
Return type: dict

class TriggerAction(value)[source]¶

Bases: str, Enum

An enumeration.

class TriggerExecutionMode(value)[source]¶

Bases: str, Enum

An enumeration.

class TriggerResource(value)[source]¶

Bases: str, Enum

An enumeration.

class TriggerType(value)[source]¶

Bases: str, Enum

An enumeration.

Execution¶

class Execution(id, url, creator, created_at, updated_at, input, output, feedback_queue, status, status_log, sync_reply_to, latest_status, function_name, duration, attempts, max_attempts, to_terminate: bool, trigger_id, service_id, project_id, service_version, package_id, package_name, client_api: ApiClient, service, project=None, repositories=NOTHING, pipeline: Optional[dict] = None)[source]¶

Bases: BaseEntity

Service execution entity

classmethod from_json(_json, client_api, project=None, service=None, is_fetched=True)[source]¶

Parameters

_json (dict) – platform json
client_api (dl.ApiClient) – ApiClient entity
project (dtlpy.entities.project.Project) – project entity
service (dtlpy.entities.service.Service) –
is_fetched – is Entity fetched from Platform

increment()[source]¶

Increment attempts

Returns

logs(follow=False)[source]¶

Print logs for execution

Parameters: follow – keep stream future logs

progress_update(status: Optional[ExecutionStatus] = None, percent_complete: Optional[int] = None, message: Optional[str] = None, output: Optional[str] = None, service_version: Optional[str] = None)[source]¶

Update Execution Progress

Parameters

status (str) – ExecutionStatus
percent_complete (int) – percent complete
message (str) – message to update the progress state
output (str) – output
service_version (str) – service version

Returns

Service execution object

rerun()[source]¶

Re-run

Returns: Execution object

terminate()[source]¶

Terminate execution

Returns: execution object

to_json()[source]¶

Returns platform _json format of object

Returns: platform json format of object
Return type: dict

update()[source]¶

Update execution changes to platform

Returns: execution entity

wait()[source]¶

Wait for execution

Returns: Service execution object

class ExecutionStatus(value)[source]¶

Bases: str, Enum

An enumeration.

Pipeline¶

class Pipeline(id, name, creator, org_id, connections, created_at, updated_at, start_nodes, project_id, composition_id, url, preview, description, revisions, project, client_api: ApiClient, repositories=NOTHING)[source]¶

Bases: BaseEntity

Package object

delete()[source]¶

Delete pipeline object

Returns: True

execute(execution_input=None)[source]¶

execute a pipeline and return the execute

Parameters: execution_input – list of the dl.FunctionIO or dict of pipeline input - example {‘item’: ‘item_id’}
Returns: entities.PipelineExecution object

classmethod from_json(_json, client_api, project, is_fetched=True)[source]¶

Turn platform representation of pipeline into a pipeline entity

Parameters

_json (dict) – platform representation of package
client_api (dl.ApiClient) – ApiClient entity
project (dtlpy.entities.project.Project) – project entity
is_fetched (bool) – is Entity fetched from Platform

Returns

Pipeline entity

Return type

install()[source]¶

install pipeline

Returns: Composition entity

open_in_web()[source]¶

Open the pipeline in web platform

Returns

pause()[source]¶

pause pipeline

Returns: Composition entity

reset(stop_if_running: bool = False)[source]¶

Resets pipeline counters

Parameters: stop_if_running (bool) – If the pipeline is installed it will stop the pipeline and reset the counters.
Returns: bool

set_start_node(node: PipelineNode)[source]¶

Set the start node of the pipeline

Parameters: node (PipelineNode) – node to be the start node

stats()[source]¶

Get pipeline counters

Returns: PipelineStats
Return type: dtlpy.entities.pipeline.PipelineStats

to_json()[source]¶

Turn Package entity into a platform representation of Package

Returns: platform json of package
Return type: dict

update()[source]¶

Update pipeline changes to platform

Returns: pipeline entity

Pipeline Execution¶

class PipelineExecution(id, nodes, executions, status, created_at, updated_at, pipeline_id, max_attempts, pipeline, client_api: ApiClient, repositories=NOTHING)[source]¶

Bases: BaseEntity

Package object

classmethod from_json(_json, client_api, pipeline, is_fetched=True)[source]¶

Turn platform representation of pipeline_execution into a pipeline_execution entity

Parameters

_json (dict) – platform representation of package
client_api (dl.ApiClient) – ApiClient entity
pipeline (dtlpy.entities.pipeline.Pipeline) – Pipeline entity
is_fetched (bool) – is Entity fetched from Platform

Returns

Pipeline entity

Return type

to_json()[source]¶

Turn Package entity into a platform representation of Package

Returns: platform json of package
Return type: dict

Other¶

Pages¶

class PagedEntities(client_api: ApiClient, page_offset, page_size, filters, items_repository, has_next_page=False, total_pages_count=0, items_count=0, service_id=None, project_id=None, order_by_type=None, order_by_direction=None, execution_status=None, execution_resource_type=None, execution_resource_id=None, execution_function_name=None, items=[])[source]¶

Bases: object

Pages object

get_page(page_offset=None, page_size=None)[source]¶

Get page

Parameters

page_offset – page offset
page_size – page size

go_to_page(page=0)[source]¶

Brings specified page of items from host

Parameters: page – page number
Returns

next_page()[source]¶

Brings the next page of items from host

Returns

prev_page()[source]¶

Brings the previous page of items from host

Returns

process_result(result)[source]¶

Parameters: result – json object

return_page(page_offset=None, page_size=None)[source]¶

Return page

Parameters

page_offset – page offset
page_size – page size

Base Entity¶

Command¶

class Command(id, url, status, created_at, updated_at, type, progress, spec, error, client_api: ApiClient, repositories=NOTHING)[source]¶

Bases: BaseEntity

Com entity

abort()[source]¶

abort command

Returns

classmethod from_json(_json, client_api, is_fetched=True)[source]¶

Build a Command entity object from a json

Parameters

_json – _json response from host
client_api – ApiClient entity
is_fetched – is Entity fetched from Platform

Returns

Command object

in_progress()[source]¶

Check if command is still in one of the in progress statuses

Returns: True if command still in progress
Return type: bool

to_json()[source]¶

Returns platform _json format of object

Returns: platform json format of object
Return type: dict

wait(timeout=0, step=None, backoff_factor=0.1)[source]¶

Wait for Command to finish

Parameters

timeout (int) – int, seconds to wait until TimeoutError is raised. if 0 - wait until done
step (int) – int, seconds between polling
backoff_factor (float) – A backoff factor to apply between attempts after the second try

Returns

Command object

class CommandsStatus(value)[source]¶

Bases: str, Enum

An enumeration.

Directory Tree¶

class DirectoryTree(_json)[source]¶

Bases: object

Dataset DirectoryTree

class SingleDirectory(value, directory_tree, children=None)[source]¶

Bases: object

DirectoryTree single directory

Utilities¶

converter¶

class Converter(concurrency=6, return_error_filepath=False)[source]¶

Bases: object

Annotation Converter

attach_agent_progress(progress: Progress, progress_update_frequency: Optional[int] = None)[source]¶

Attach agent progress.

Parameters

progress (Progress) – the progress object that follows the work
progress_update_frequency (int) – progress update frequency in percentages

convert(annotations, from_format: str, to_format: str, conversion_func=None, item=None)[source]¶

Convert annotation list or single annotation.

Prerequisites: You must be an owner or developer to use this method.

Parameters

item (dtlpy.entities.item.Item) – item entity
annotations (list or AnnotationCollection) – annotations list to convert
from_format (str) – AnnotationFormat to convert to – AnnotationFormat.COCO, AnnotationFormat.YOLO, AnnotationFormat.VOC, AnnotationFormat.DATALOOP
to_format (str) – AnnotationFormat to convert to – AnnotationFormat.COCO, AnnotationFormat.YOLO, AnnotationFormat.VOC, AnnotationFormat.DATALOOP
conversion_func (Callable) – Custom conversion service

Returns

the annotations

convert_dataset(dataset, to_format: str, local_path: str, conversion_func=None, filters=None, annotation_filter=None)[source]¶

Convert entire dataset.

Prerequisites: You must be an owner or developer to use this method.

Parameters

dataset (dtlpy.entities.dataet.Dataset) – dataset entity
to_format (str) – AnnotationFormat to convert to – AnnotationFormat.COCO, AnnotationFormat.YOLO, AnnotationFormat.VOC, AnnotationFormat.DATALOOP
local_path (str) – path to save the result to
conversion_func (Callable) – Custom conversion service
filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filter parameters
annotation_filter (dtlpy.entities.filters.Filters) – Filter entity

Returns

the error log file path if there are errors and the coco json if the format is coco

convert_directory(local_path: str, to_format: AnnotationFormat, from_format: AnnotationFormat, dataset, conversion_func=None)[source]¶

Convert annotation files in entire directory.

Prerequisites: You must be an owner or developer to use this method.

Parameters

local_path (str) – path to the directory
to_format (str) – AnnotationFormat to convert to – AnnotationFormat.COCO, AnnotationFormat.YOLO, AnnotationFormat.VOC, AnnotationFormat.DATALOOP
from_format (str) – AnnotationFormat to convert from – AnnotationFormat.COCO, AnnotationFormat.YOLO, AnnotationFormat.VOC, AnnotationFormat.DATALOOP
dataset (dtlpy.entities.dataset.Dataset) – dataset entity
conversion_func (Callable) – Custom conversion service

Returns

the error log file path if there are errors

convert_file(to_format: str, from_format: str, file_path: str, save_locally: bool = False, save_to: Optional[str] = None, conversion_func=None, item=None, pbar=None, upload: bool = False, **_)[source]¶

Convert file containing annotations.

Prerequisites: You must be an owner or developer to use this method.

Parameters

to_format (str) – AnnotationFormat to convert to – AnnotationFormat.COCO, AnnotationFormat.YOLO, AnnotationFormat.VOC, AnnotationFormat.DATALOOP
from_format (str) – AnnotationFormat to convert from – AnnotationFormat.COCO, AnnotationFormat.YOLO, AnnotationFormat.VOC, AnnotationFormat.DATALOOP
file_path (str) – path of the file to convert
pbar (tqdm) – tqdm object that follows the work (progress bar)
upload (bool) – if True upload
save_locally (bool) – If True, save locally
save_to (str) – path to save the result to
conversion_func (Callable) – Custom conversion service
item (dtlpy.entities.item.Item) – item entity

Returns

annotation list, errors

static custom_format(annotation, conversion_func, i_annotation=None, annotations=None, from_format=None, item=None, **_)[source]¶

Custom convert function.

Prerequisites: You must be an owner or developer to use this method.

Parameters

annotation (dtlpy.entities.annotation.Annotation or dict) – annotations to convert
conversion_func (Callable) – Custom conversion service
i_annotation (int) – annotation index
annotations (list) – list of annotations

param str from_format: AnnotationFormat to convert to – AnnotationFormat.COCO, AnnotationFormat.YOLO, AnnotationFormat.VOC, AnnotationFormat.DATALOOP :param dtlpy.entities.item.Item item: item entity :return: converted Annotation

from_coco(annotation, **kwargs)[source]¶

Convert from COCO format to DATALOOP format. Use this as conversion_func param for functions that ask for this param.

Prerequisites: You must be an owner or developer to use this method.

Parameters

annotation – annotations to convert
kwargs – additional params

Returns

converted Annotation entity

Return type

static from_voc(annotation, **_)[source]¶

Convert from VOC format to DATALOOP format. Use this as conversion_func for functions that ask for this param.

Prerequisites: You must be an owner or developer to use this method.

Parameters: annotation – annotations to convert
Returns: converted Annotation entity
Return type: dtlpy.entities.annotation.Annotation

from_yolo(annotation, item=None, **kwargs)[source]¶

Convert from YOLO format to DATALOOP format. Use this as conversion_func param for functions that ask for this param.

Prerequisites: You must be an owner or developer to use this method.

Parameters

annotation – annotations to convert
item (dtlpy.entities.item.Item) – item entity
kwargs – additional params

Returns

converted Annotation entity

Return type

save_to_file(save_to, to_format, annotations, item=None)[source]¶

Save annotations to a file.

Prerequisites: You must be an owner or developer to use this method.

Parameters

save_to (str) – path to save the result to
to_format – AnnotationFormat to convert to – AnnotationFormat.COCO, AnnotationFormat.YOLO, AnnotationFormat.VOC, AnnotationFormat.DATALOOP
annotations (list) – annotation list to convert
item (dtlpy.entities.item.Item) – item entity

static to_coco(annotation, item=None, **_)[source]¶

Convert from DATALOOP format to COCO format. Use this as conversion_func param for functions that ask for this param.

Prerequisites: You must be an owner or developer to use this method.

Parameters

annotation (dtlpy.entities.annotation.Annotation or dict) – annotations to convert
item (dtlpy.entities.item.Item) – item entity
**_ –
additional params

Returns

converted Annotation

Return type

static to_voc(annotation, item=None, **_)[source]¶

Convert from DATALOOP format to VOC format. Use this as conversion_func param for functions that ask for this param.

Prerequisites: You must be an owner or developer to use this method.

Parameters

annotation (dtlpy.entities.annotation.Annotation or dict) – annotations to convert
item (dtlpy.entities.item.Item) – item entity
**_ –
additional params

Returns

converted Annotation

Return type