Repositories

Organizations

class Organizations(client_api: ApiClient)[source]

Bases: object

Organizations Repository

Read our documentation and SDK documentation to learn more about Organizations in the Dataloop platform.

add_member(email: str, role: MemberOrgRole = MemberOrgRole.MEMBER, organization_id: Optional[str] = None, organization_name: Optional[str] = None, organization: Optional[Organization] = None)[source]

Add members to your organization. Read about members and groups here.

Prerequisities: To add members to an organization, you must be an owner in that organization.

You must provide at least ONE of the following params: organization, organization_name, or organization_id.

Parameters
  • email (str) – the member’s email

  • role (str) – MemberOrgRole.ADMIN, MemberOrgRole.OWNER, MemberOrgRole.MEMBER

  • organization_id (str) – Organization id

  • organization_name (str) – Organization name

  • organization (entities.Organization) – Organization object

Returns

True if successful or error if unsuccessful

Return type

bool

Example:

dl.organizations.add_member(email='user@domain.com',
                            organization_id='organization_id',
                            role=dl.MemberOrgRole.MEMBER)
cache_action(organization_id: Optional[str] = None, organization_name: Optional[str] = None, organization: Optional[Organization] = None, mode=CacheAction.APPLY, pod_type=PodType.SMALL)[source]

Add or remove Cache for the org

Prerequisites: You must be an organization owner

You must provide at least ONE of the following params: organization, organization_name, or organization_id.

Parameters
  • organization_id (str) – Organization id

  • organization_name (str) – Organization name

  • organization (entities.Organization) – Organization object

  • mode (str) – dl.CacheAction.APPLY or dl.CacheAction.DESTROY

  • pod_type (entities.PodType) – dl.PodType.SMALL, dl.PodType.MEDIUM, dl.PodType.HIGH

Returns

True if success

Return type

bool

Example:

dl.organizations.enable_cache(organization_id='organization_id',
                              mode=dl.CacheAction.APPLY)
delete_member(user_id: str, organization_id: Optional[str] = None, organization_name: Optional[str] = None, organization: Optional[Organization] = None, sure: bool = False, really: bool = False) bool[source]

Delete member from the Organization.

Prerequisites: Must be an organization owner to delete members.

You must provide at least ONE of the following params: organization_id, organization_name, organization.

Parameters
  • user_id (str) – user id

  • organization_id (str) – Organization id

  • organization_name (str) – Organization name

  • organization (entities.Organization) – Organization object

  • sure (bool) – Are you sure you want to delete?

  • really (bool) – Really really sure?

Returns

True if success and error if not

Return type

bool

Example:

dl.organizations.delete_member(user_id='user_id',
                                organization_id='organization_id',
                                sure=True,
                                really=True)
get(organization_id: Optional[str] = None, organization_name: Optional[str] = None, fetch: Optional[bool] = None) Organization[source]

Get Organization object to be able to use it in your code.

Prerequisites: You must be a superuser to use this method.

You must provide at least ONE of the following params: organization_name or organization_id.

Parameters
  • organization_id (str) – optional - search by id

  • organization_name (str) – optional - search by name

  • fetch – optional - fetch entity from platform, default taken from cookie

Returns

Organization object

Return type

dtlpy.entities.organization.Organization

Example:

dl.organizations.get(organization_id='organization_id')
list() List[Organization][source]

Lists all the organizations in Dataloop.

Prerequisites: You must be a superuser to use this method.

Returns

List of Organization objects

Return type

list

Example:

dl.organizations.list()
list_groups(organization: Optional[Organization] = None, organization_id: Optional[str] = None, organization_name: Optional[str] = None)[source]

List all organization groups (groups that were created within the organization).

Prerequisites: You must be an organization owner to use this method.

You must provide at least ONE of the following params: organization, organization_name, or organization_id.

Parameters
  • organization (entities.Organization) – Organization object

  • organization_id (str) – Organization id

  • organization_name (str) – Organization name

Returns

groups list

Return type

list

Example:

dl.organizations.list_groups(organization_id='organization_id')
list_integrations(organization: Optional[Organization] = None, organization_id: Optional[str] = None, organization_name: Optional[str] = None, only_available=False)[source]

List all organization integrations with external cloud storage.

Prerequisites: You must be an organization owner to use this method.

You must provide at least ONE of the following params: organization_id, organization_name, or organization.

Parameters
  • organization (entities.Organization) – Organization object

  • organization_id (str) – Organization id

  • organization_name (str) – Organization name

  • only_available (bool) – if True list only the available integrations

Returns

integrations list

Return type

list

Example:

dl.organizations.list_integrations(organization='organization-entity',
                                    only_available=True)
list_members(organization: Optional[Organization] = None, organization_id: Optional[str] = None, organization_name: Optional[str] = None, role: Optional[MemberOrgRole] = None)[source]

List all organization members.

Prerequisites: You must be an organization owner to use this method.

You must provide at least ONE of the following params: organization_id, organization_name, or organization.

Parameters
  • organization (entities.Organization) – Organization object

  • organization_id (str) – Organization id

  • organization_name (str) – Organization name

  • role (entities.MemberOrgRole) – MemberOrgRole.ADMIN, MemberOrgRole.OWNER, MemberOrgRole.MEMBER

Returns

projects list

Return type

list

Example:

dl.organizations.list_members(organization='organization-entity',
                            role=dl.MemberOrgRole.MEMBER)
update(plan: str, organization: Optional[Organization] = None, organization_id: Optional[str] = None, organization_name: Optional[str] = None) Organization[source]

Update an organization.

Prerequisites: You must be a superuser to update an organization.

You must provide at least ONE of the following params: organization, organization_name, or organization_id.

Parameters
  • plan (str) – OrganizationsPlans.FREEMIUM, OrganizationsPlans.PREMIUM

  • organization (entities.Organization) – Organization object

  • organization_id (str) – Organization id

  • organization_name (str) – Organization name

Returns

organization object

Return type

dtlpy.entities.organization.Organization

Example:

dl.organizations.update(organization='organization-entity',
                        plan=dl.OrganizationsPlans.FREEMIUM)
update_member(email: str, role: MemberOrgRole = MemberOrgRole.MEMBER, organization_id: Optional[str] = None, organization_name: Optional[str] = None, organization: Optional[Organization] = None)[source]

Update member role.

Prerequisites: You must be an organization owner to update a member’s role.

You must provide at least ONE of the following params: organization, organization_name, or organization_id.

Parameters
  • email (str) – the member’s email

  • role (str) – MemberOrgRole.ADMIN, MemberOrgRole.OWNER, MemberOrgRole.MEMBER

  • organization_id (str) – Organization id

  • organization_name (str) – Organization name

  • organization (entities.Organization) – Organization object

Returns

json of the member fields

Return type

dict

Example:

dl.organizations.update_member(email='user@domain.com',
                                organization_id='organization_id',
                                 role=dl.MemberOrgRole.MEMBER)

Integrations

Integrations Repository

class Integrations(client_api: ApiClient, org: Optional[Organization] = None, project: Optional[Project] = None)[source]

Bases: object

Integrations Repository

The Integrations class allows you to manage data integrtion from your external storage (e.g., S3, GCS, Azure) into your Dataloop’s Dataset storage, as well as sync data in your Dataloop’s Datasets with data in your external storage.

For more information on Organization Storgae Integration see the Dataloop documentation and SDK External Storage.

create(integrations_type: ExternalStorage, name: str, options: dict)[source]

Create an integration between an external storage and the organization.

Examples for options include: s3 - {key: “”, secret: “”}; gcs - {key: “”, secret: “”, content: “”}; azureblob - {key: “”, secret: “”, clientId: “”, tenantId: “”}; key_value - {key: “”, value: “”} aws-sts - {key: “”, secret: “”, roleArns: “”}

Prerequisites: You must be an owner in the organization.

Parameters
  • integrations_type (str) – integrations type dl.ExternalStorage

  • name (str) – integrations name

  • options (dict) – dict of storage secrets

Returns

success

Return type

bool

Example:

project.integrations.create(integrations_type=dl.ExternalStorage.S3,
                name='S3ntegration',
                options={key: "Access key ID", secret: "Secret access key"})
delete(integrations_id: str, sure: bool = False, really: bool = False) bool[source]

Delete integrations from the organization.

Prerequisites: You must be an organization owner to delete an integration.

Parameters
  • integrations_id (str) – integrations id

  • sure (bool) – Are you sure you want to delete?

  • really (bool) – Really really sure?

Returns

success

Return type

bool

Example:

project.integrations.delete(integrations_id='integrations_id', sure=True, really=True)
get(integrations_id: str)[source]

Get organization integrations. Use this method to access your integration and be able to use it in your code.

Prerequisites: You must be an owner in the organization.

Parameters

integrations_id (str) – integrations id

Returns

Integration object

Return type

dtlpy.entities.integration.Integration

Example:

project.integrations.get(integrations_id='integrations_id')
list(only_available=False)[source]

List all the organization’s integrations with external storage.

Prerequisites: You must be an owner in the organization.

Parameters

only_available (bool) – if True list only the available integrations.

Returns

groups list

Return type

list

Example:

project.integrations.list(only_available=True)
update(new_name: str, integrations_id: str)[source]

Update the integration’s name.

Prerequisites: You must be an owner in the organization.

Parameters
  • new_name (str) – new name

  • integrations_id (str) – integrations id

Returns

Integration object

Return type

dtlpy.entities.integration.Integration

Example:

project.integrations.update(integrations_id='integrations_id', new_name="new_integration_name")

Projects

class Projects(client_api: ApiClient, org=None)[source]

Bases: object

Projects Repository

The Projects class allows the user to manage projects and their properties.

For more information on Projects see the Dataloop documentation and SDK documentation.

add_member(email: str, project_id: str, role: MemberRole = MemberRole.DEVELOPER)[source]

Add a member to the project.

Prerequisites: You must be in the role of an owner to add a member to a project.

Parameters
  • email (str) – member email

  • project_id (str) – project id

  • role – dl.MemberRole.OWNER, dl.MemberRole.DEVELOPER, dl.MemberRole.ANNOTATOR, dl.MemberRole.ANNOTATION_MANAGER

Returns

dict that represent the user

Return type

dict

Example:

dl.projects.add_member(project_id='project_id', email='user@dataloop.ai', role=dl.MemberRole.DEVELOPER)
checkout(identifier: Optional[str] = None, project_name: Optional[str] = None, project_id: Optional[str] = None, project: Optional[Project] = None)[source]

Checkout (switch) to a project to work on it.

Prerequisites: All users can open a project in the web.

You must provide at least ONE of the following params: project_id, project_name.

Parameters

Example:

dl.projects.checkout(project_id='project_id')
create(project_name: str, checkout: bool = False) Project[source]

Create a new project.

Prerequisites: Any user can create a project.

Parameters
  • project_name (str) – project name

  • checkout – checkout

Returns

Project object

Return type

dtlpy.entities.project.Project

Example:

dl.projects.create(project_name='project_name')
delete(project_name: Optional[str] = None, project_id: Optional[str] = None, sure: bool = False, really: bool = False) bool[source]

Delete a project forever!

Prerequisites: You must be in the role of an owner to delete a project.

Parameters
  • project_name (str) – optional - search by name

  • project_id (str) – optional - search by id

  • sure (bool) – Are you sure you want to delete?

  • really (boll) – Really really sure?

Returns

True if sucess error if not

Return type

bool

Example:

dl.projects.delete(project_id='project_id', sure=True, really=True)
get(project_name: Optional[str] = None, project_id: Optional[str] = None, checkout: bool = False, fetch: Optional[bool] = None, log_error=True) Project[source]

Get a Project object.

Prerequisites: You must be in the role of an owner to get a project object.

You must check out to a project or provide at least one of the following params: project_id, project_name

Parameters
  • project_name (str) – optional - search by name

  • project_id (str) – optional - search by id

  • checkout (bool) – checkout

  • fetch (bool) – optional - fetch entity from platform, default taken from cookie

  • log_error (bool) – optional - show the logs errors

Returns

Project object

Return type

dtlpy.entities.project.Project

Example:

dl.projects.get(project_id='project_id')
list() List[Project][source]

Get users’ project list.

Prerequisites: You must be a superuser to list all users’ projects.

Returns

List of Project objects

Example:

dl.projects.list()
list_members(project: Project, role: Optional[MemberRole] = None)[source]

List the project members.

Prerequisites: You must be in the role of an owner to list project members.

Parameters
  • project (dtlpy.entities.project.Project) – project entity

  • role – dl.MemberRole.OWNER, dl.MemberRole.DEVELOPER, dl.MemberRole.ANNOTATOR, dl.MemberRole.ANNOTATION_MANAGER

Returns

list of the project members

Return type

list

Example:

dl.projects.list_members(project_id='project_id', role=dl.MemberRole.DEVELOPER)
open_in_web(project_name: Optional[str] = None, project_id: Optional[str] = None, project: Optional[Project] = None)[source]

Open the project in our web platform.

Prerequisites: All users can open a project in the web.

Parameters

Example:

dl.projects.open_in_web(project_id='project_id')
remove_member(email: str, project_id: str)[source]

Remove a member from the project.

Prerequisites: You must be in the role of an owner to delete a member from a project.

Parameters
  • email (str) – member email

  • project_id (str) – project id

Returns

dict that represents the user

Return type

dict

Example:

dl.projects.remove_member(project_id='project_id', email='user@dataloop.ai')
update(project: Project, system_metadata: bool = False) Project[source]

Update a project information (e.g., name, member roles, etc.).

Prerequisites: You must be in the role of an owner to add a member to a project.

Parameters
Returns

Project object

Return type

dtlpy.entities.project.Project

Example:

dl.projects.delete(project='project_entity')
update_member(email: str, project_id: str, role: MemberRole = MemberRole.DEVELOPER)[source]

Update member’s information/details in the project.

Prerequisites: You must be in the role of an owner to update a member.

Parameters
  • email (str) – member email

  • project_id (str) – project id

  • role – dl.MemberRole.OWNER, dl.MemberRole.DEVELOPER, dl.MemberRole.ANNOTATOR, dl.MemberRole.ANNOTATION_MANAGER

Returns

dict that represent the user

Return type

dict

Example:

dl.projects.update_member(project_id='project_id', email='user@dataloop.ai', role=dl.MemberRole.DEVELOPER)

Datasets

Datasets Repository

class Datasets(client_api: ApiClient, project: Optional[Project] = None)[source]

Bases: object

Datasets Repository

The Datasets class allows the user to manage datasets. Read more about datasets in our documentation and SDK documentation.

checkout(identifier: Optional[str] = None, dataset_name: Optional[str] = None, dataset_id: Optional[str] = None, dataset: Optional[Dataset] = None)[source]

Checkout (switch) to a dataset to work on it.

Prerequisites: You must be an owner or developer to use this method.

You must provide at least ONE of the following params: dataset_id, dataset_name.

Parameters

Example:

project.datasets.checkout(dataset_id='dataset_id')
clone(dataset_id: str, clone_name: str, filters: Optional[Filters] = None, with_items_annotations: bool = True, with_metadata: bool = True, with_task_annotations_status: bool = True)[source]

Clone a dataset. Read more about cloning datatsets and items in our documentation and SDK documentation.

Prerequisites: You must be in the role of an owner or developer.

Parameters
  • dataset_id (str) – id of the dataset you wish to clone

  • clone_name (str) – new dataset name

  • filters (dtlpy.entities.filters.Filters) – Filters entity or a query dict

  • with_items_annotations (bool) – true to clone with items annotations

  • with_metadata (bool) – true to clone with metadata

  • with_task_annotations_status (bool) – true to clone with task annotations’ status

Returns

dataset object

Return type

dtlpy.entities.dataset.Dataset

Example:

project.datasets.clone(dataset_id='dataset_id',
                      clone_name='dataset_clone_name',
                      with_metadata=True,
                      with_items_annotations=False,
                      with_task_annotations_status=False)
create(dataset_name: str, labels=None, attributes=None, ontology_ids=None, driver: Optional[Driver] = None, driver_id: Optional[str] = None, checkout: bool = False, expiration_options: Optional[ExpirationOptions] = None, index_driver: IndexDriver = IndexDriver.V1, recipe_id: Optional[str] = None) Dataset[source]

Create a new dataset

Prerequisites: You must be in the role of an owner or developer.

Parameters
  • dataset_name (str) – dataset name

  • labels (list) – dictionary of {tag: color} or list of label entities

  • attributes (list) – dataset’s ontology’s attributes

  • ontology_ids (list) – optional - dataset ontology

  • driver (dtlpy.entities.driver.Driver) – optional - storage driver Driver object or driver name

  • driver_id (str) – optional - driver id

  • checkout (bool) – bool. cache the dataset to work locally

  • expiration_options (ExpirationOptions) – dl.ExpirationOptions object that contain definitions for dataset like MaxItemDays

  • index_driver (str) – dl.IndexDriver, dataset driver version

  • recipe_id (str) – optional - recipe id

Returns

Dataset object

Return type

dtlpy.entities.dataset.Dataset

Example:

project.datasets.create(dataset_name='dataset_name', ontology_ids='ontology_ids')
delete(dataset_name: Optional[str] = None, dataset_id: Optional[str] = None, sure: bool = False, really: bool = False)[source]

Delete a dataset forever!

Prerequisites: You must be an owner or developer to use this method.

Example:

project.datasets.delete(dataset_id='dataset_id', sure=True, really=True)
Parameters
  • dataset_name (str) – optional - search by name

  • dataset_id (str) – optional - search by id

  • sure (bool) – Are you sure you want to delete?

  • really (bool) – Really really sure?

Returns

True is success

Return type

bool

directory_tree(dataset: Optional[Dataset] = None, dataset_name: Optional[str] = None, dataset_id: Optional[str] = None)[source]

Get dataset’s directory tree.

Prerequisites: You must be an owner or developer to use this method.

You must provide at least ONE of the following params: dataset, dataset_name, dataset_id.

Parameters
Returns

DirectoryTree

Example:

project.datasets.directory_tree(dataset='dataset_entity')
static download_annotations(dataset: Dataset, local_path: Optional[str] = None, filters: Optional[Filters] = None, annotation_options: Optional[ViewAnnotationOptions] = None, annotation_filters: Optional[Filters] = None, overwrite: bool = False, thickness: int = 1, with_text: bool = False, remote_path: Optional[str] = None, include_annotations_in_output: bool = True, export_png_files: bool = False, filter_output_annotations: bool = False, alpha: Optional[float] = None, export_version=ExportVersion.V1) str[source]

Download dataset’s annotations by filters.

You may filter the dataset both for items and for annotations and download annotations.

Optional – download annotations as: mask, instance, image mask of the item.

Prerequisites: You must be in the role of an owner or developer.

Parameters
  • dataset (dtlpy.entities.dataset.Dataset) – dataset object

  • local_path (str) – local folder or filename to save to.

  • filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters

  • annotation_options (list) – download annotations options: list(dl.ViewAnnotationOptions)

  • annotation_filters (dtlpy.entities.filters.Filters) – Filters entity to filter annotations for download

  • overwrite (bool) – optional - default = False

  • thickness (int) – optional - line thickness, if -1 annotation will be filled, default =1

  • with_text (bool) – optional - add text to annotations, default = False

  • remote_path (str) – DEPRECATED and ignored

  • include_annotations_in_output (bool) – default - False , if export should contain annotations

  • export_png_files (bool) – default - if True, semantic annotations should be exported as png files

  • filter_output_annotations (bool) – default - False, given an export by filter - determine if to filter out annotations

  • alpha (float) – opacity value [0 1], default 1

  • export_version (str) – exported items will have original extension in filename, V1 - no original extension in filenames

Returns

local_path of the directory where all the downloaded item

Return type

str

Example:

project.datasets.download_annotations(dataset='dataset_entity',
                                     local_path='local_path',
                                     annotation_options=dl.ViewAnnotationOptions,
                                     overwrite=False,
                                     thickness=1,
                                     with_text=False,
                                     alpha=1
                                     )
get(dataset_name: Optional[str] = None, dataset_id: Optional[str] = None, checkout: bool = False, fetch: Optional[bool] = None) Dataset[source]

Get dataset by name or id.

Prerequisites: You must be an owner or developer to use this method.

You must provide at least ONE of the following params: dataset_id, dataset_name.

Parameters
  • dataset_name (str) – optional - search by name

  • dataset_id (str) – optional - search by id

  • checkout (bool) – True to checkout

  • fetch (bool) – optional - fetch entity from platform, default taken from cookie

Returns

Dataset object

Return type

dtlpy.entities.dataset.Dataset

Example:

project.datasets.get(dataset_id='dataset_id')
list(name=None, creator=None) List[Dataset][source]

List all datasets.

Prerequisites: You must be an owner or developer to use this method.

Parameters
  • name (str) – list by name

  • creator (str) – list by creator

Returns

List of datasets

Return type

list

Example:

project.datasets.list(name='name')
merge(merge_name: str, dataset_ids: str, project_ids: str, with_items_annotations: bool = True, with_metadata: bool = True, with_task_annotations_status: bool = True, wait: bool = True)[source]

Merge a dataset. See our SDK docs for more information.

Prerequisites: You must be an owner or developer to use this method.

Parameters
  • merge_name (str) – new dataset name

  • dataset_ids (str) – id’s of the datatsets you wish to merge

  • project_ids (str) – project id

  • with_items_annotations (bool) – with items annotations

  • with_metadata (bool) – with metadata

  • with_task_annotations_status (bool) – with task annotations status

  • wait (bool) – wait for the command to finish

Returns

True if success

Return type

bool

Example:

project.datasets.clone(dataset_ids=['dataset_id1','dataset_id2'],
                      merge_name='dataset_merge_name',
                      with_metadata=True,
                      with_items_annotations=False,
                      with_task_annotations_status=False)
open_in_web(dataset_name: Optional[str] = None, dataset_id: Optional[str] = None, dataset: Optional[Dataset] = None)[source]

Open the dataset in web platform.

Prerequisites: You must be an owner or developer to use this method.

Parameters

Example:

project.datasets.open_in_web(dataset_id='dataset_id')
set_readonly(state: bool, dataset: Dataset)[source]

Set dataset readonly mode.

Prerequisites: You must be in the role of an owner or developer.

Parameters

Example:

project.datasets.set_readonly(dataset='dataset_entity', state=True)
sync(dataset_id: str, wait: bool = True)[source]

Sync dataset with external storage.

Prerequisites: You must be in the role of an owner or developer.

Parameters
  • dataset_id (str) – to sync dataset

  • wait (bool) – wait for the command to finish

Returns

True if success

Return type

bool

Example:

project.datasets.sync(dataset_id='dataset_id')
update(dataset: Dataset, system_metadata: bool = False, patch: Optional[dict] = None) Dataset[source]

Update dataset field.

Prerequisites: You must be an owner or developer to use this method.

Parameters
Returns

Dataset object

Return type

dtlpy.entities.dataset.Dataset

Example:

project.datasets.update(dataset='dataset_entity')
upload_annotations(dataset, local_path, filters: Optional[Filters] = None, clean=False, remote_root_path='/', export_version=ExportVersion.V1)[source]

Upload annotations to dataset.

Example for remote_root_path: If the item filepath is a/b/item and remote_root_path is /a the start folder will be b instead of a

Prerequisites: You must have a dataset with items that are related to the annotations. The relationship between the dataset and annotations is shown in the name. You must be in the role of an owner or developer.

Parameters
  • dataset (dtlpy.entities.dataset.Dataset) – dataset to upload to

  • local_path (str) – str - local folder where the annotations files is

  • filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters

  • clean (bool) – True to remove the old annotations

  • remote_root_path (str) – the remote root path to match remote and local items

  • export_version (str) – exported items will have original extension in filename, V1 - no original extension in filenames

Example:

project.datasets.upload_annotations(dataset='dataset_entity',
                                     local_path='local_path',
                                     clean=False,
                                     export_version=dl.ExportVersion.V1
                                     )

Drivers

class Drivers(client_api: ApiClient, project: Optional[Project] = None)[source]

Bases: object

Drivers Repository

The Drivers class allows users to manage drivers that are used to connect with external storage. Read more about external storage in our documentation and SDK documentation.

create(name: str, driver_type: ExternalStorage, integration_id: str, bucket_name: str, integration_type: ExternalStorage, project_id: Optional[str] = None, allow_external_delete: bool = True, region: Optional[str] = None, storage_class: str = '', path: str = '')[source]

Create a storage driver.

Prerequisites: You must be in the role of an owner or developer.

Parameters
  • name (str) – the driver name

  • driver_type (str) – ExternalStorage.S3, ExternalStorage.GCS, ExternalStorage.AZUREBLOB

  • integration_id (str) – the integration id

  • bucket_name (str) – the external bucket name

  • integration_type (str) – ExternalStorage.S3, ExternalStorage.GCS, ExternalStorage.AZUREBLOB, ExternalStorage.AWS_STS

  • project_id (str) – project id

  • allow_external_delete (bool) – true to allow deleting files from external storage when files are deleted in your Dataloop storage

  • region (str) – relevant only for s3 - the bucket region

  • storage_class (str) – rilevante only for s3

  • path (str) – Optional. By default path is the root folder. Path is case sensitive integration

Returns

driver object

Return type

dtlpy.entities.driver.Driver

Example:

project.drivers.create(name='driver_name',
           driver_type=dl.ExternalStorage.S3,
           integration_id='integration_id',
           bucket_name='bucket_name',
           project_id='project_id',
           region='ey-west-1')
delete(driver_name: Optional[str] = None, driver_id: Optional[str] = None, sure: bool = False, really: bool = False)[source]

Delete a driver forever!

Prerequisites: You must be an owner or developer to use this method.

Example:

project.drivers.delete(dataset_id='dataset_id', sure=True, really=True)
Parameters
  • driver_name (str) – optional - search by name

  • driver_id (str) – optional - search by id

  • sure (bool) – Are you sure you want to delete?

  • really (bool) – Really really sure?

Returns

True if success

Return type

bool

get(driver_name: Optional[str] = None, driver_id: Optional[str] = None) Driver[source]

Get a Driver object to use in your code.

Prerequisites: You must be in the role of an owner or developer.

You must provide at least ONE of the following params: driver_name, driver_id.

Parameters
  • driver_name (str) – optional - search by name

  • driver_id (str) – optional - search by id

Returns

Driver object

Return type

dtlpy.entities.driver.Driver

Example:

project.drivers.get(driver_id='driver_id')
list() List[Driver][source]

Get the project’s drivers list.

Prerequisites: You must be in the role of an owner or developer.

Returns

List of Drivers objects

Return type

list

Example:

project.drivers.list()

Items

class Items(client_api: ApiClient, datasets: Optional[Datasets] = None, dataset: Optional[Dataset] = None, dataset_id=None, items_entity=None, project=None)[source]

Bases: object

Items Repository

The Items class allows you to manage items in your datasets. For information on actions related to items see Organizing Your Dataset, Item Metadata, and Item Metadata-Based Filtering.

clone(item_id: str, dst_dataset_id: str, remote_filepath: Optional[str] = None, metadata: Optional[dict] = None, with_annotations: bool = True, with_metadata: bool = True, with_task_annotations_status: bool = False, allow_many: bool = False, wait: bool = True)[source]

Clone item. Read more about cloning datatsets and items in our documentation and SDK documentation.

Prerequisites: You must be in the role of an owner or developer.

Parameters
  • item_id (str) – item to clone

  • dst_dataset_id (str) – destination dataset id

  • remote_filepath (str) – complete filepath

  • metadata (dict) – new metadata to add

  • with_annotations (bool) – clone annotations

  • with_metadata (bool) – clone metadata

  • with_task_annotations_status (bool) – clone task annotations status

  • allow_many (bool) – bool if True, using multiple clones in single dataset is allowed, (default=False)

  • wait (bool) – wait for the command to finish

Returns

Item object

Return type

dtlpy.entities.item.Item

Example:

dataset.items.clone(item_id='item_id',
        dst_dataset_id='dist_dataset_id',
        with_metadata=True,
        with_task_annotations_status=False,
        with_annotations=False)
delete(filename: Optional[str] = None, item_id: Optional[str] = None, filters: Optional[Filters] = None)[source]

Delete item from platform.

Prerequisites: You must be in the role of an owner or developer.

You must provide at least ONE of the following params: item id, filename, filters.

Parameters
  • filename (str) – optional - search item by remote path

  • item_id (str) – optional - search item by id

  • filters (dtlpy.entities.filters.Filters) – optional - delete items by filter

Returns

True if success

Return type

bool

Example:

dataset.items.delete(item_id='item_id')
download(filters: Optional[Filters] = None, items=None, local_path: Optional[str] = None, file_types: Optional[list] = None, save_locally: bool = True, to_array: bool = False, annotation_options: Optional[ViewAnnotationOptions] = None, annotation_filters: Optional[Filters] = None, overwrite: bool = False, to_items_folder: bool = True, thickness: int = 1, with_text: bool = False, without_relative_path=None, avoid_unnecessary_annotation_download: bool = False, include_annotations_in_output: bool = True, export_png_files: bool = False, filter_output_annotations: bool = False, alpha: float = 1, export_version=ExportVersion.V1)[source]

Download dataset items by filters.

Filters the dataset for items and saves them locally.

Optional – download annotation, mask, instance, and image mask of the item.

Prerequisites: You must be in the role of an owner or developer.

Parameters
  • filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters

  • items (List[dtlpy.entities.item.Item] or dtlpy.entities.item.Item) – download Item entity or item_id (or a list of item)

  • local_path (str) – local folder or filename to save to.

  • file_types (list) – a list of file type to download. e.g [‘video/webm’, ‘video/mp4’, ‘image/jpeg’, ‘image/png’]

  • save_locally (bool) – bool. save to disk or return a buffer

  • to_array (bool) – returns Ndarray when True and local_path = False

  • annotation_options (list) – download annotations options: list(dl.ViewAnnotationOptions)

  • annotation_filters (dtlpy.entities.filters.Filters) – Filters entity to filter annotations for download

  • overwrite (bool) – optional - default = False

  • to_items_folder (bool) – Create ‘items’ folder and download items to it

  • thickness (int) – optional - line thickness, if -1 annotation will be filled, default =1

  • with_text (bool) – optional - add text to annotations, default = False

  • without_relative_path (bool) – bool - download items without the relative path from platform

  • avoid_unnecessary_annotation_download (bool) – default - False

  • include_annotations_in_output (bool) – default - False , if export should contain annotations

  • export_png_files (bool) – default - if True, semantic annotations should be exported as png files

  • filter_output_annotations (bool) – default - False, given an export by filter - determine if to filter out annotations

  • alpha (float) – opacity value [0 1], default 1

  • export_version (str) – exported items will have original extension in filename, V1 - no original extension in filenames

Returns

generator of local_path per each downloaded item

Return type

generator or single item

Example:

dataset.items.download(local_path='local_path',
                     annotation_options=dl.ViewAnnotationOptions,
                     overwrite=False,
                     thickness=1,
                     with_text=False,
                     alpha=1,
                     save_locally=True
                     )
get(filepath: Optional[str] = None, item_id: Optional[str] = None, fetch: Optional[bool] = None, is_dir: bool = False) Item[source]

Get Item object

Prerequisites: You must be in the role of an owner or developer.

Parameters
  • filepath (str) – optional - search by remote path

  • item_id (str) – optional - search by id

  • fetch (bool) – optional - fetch entity from platform, default taken from cookie

  • is_dir (bool) – True if you want to get an item from dir type

Returns

Item object

Return type

dtlpy.entities.item.Item

Example:

dataset.items.get(item_id='item_id')
get_all_items(filters: Optional[Filters] = None) [<class 'dtlpy.entities.item.Item'>][source]

Get all items in dataset.

Prerequisites: You must be in the role of an owner or developer.

Parameters

filters (dtlpy.entities.filters.Filters) – dl.Filters entity to filters items

Returns

list of all items

Return type

list

Example:

dataset.items.get_all_items()
list(filters: Optional[Filters] = None, page_offset: Optional[int] = None, page_size: Optional[int] = None) PagedEntities[source]

List items in a dataset.

Prerequisites: You must be in the role of an owner or developer.

Parameters
Returns

Pages object

Return type

dtlpy.entities.paged_entities.PagedEntities

Example:

dataset.items.list(page_offset=0, page_size=100)
make_dir(directory, dataset: Optional[Dataset] = None) Item[source]

Create a directory in a dataset.

Prerequisites: All users.

Parameters
Returns

Item object

Return type

dtlpy.entities.item.Item

Example:

dataset.items.make_dir(directory='directory_name')
move_items(destination: str, filters: Optional[Filters] = None, items=None, dataset: Optional[Dataset] = None) bool[source]

Move items to another directory. If directory does not exist we will create it

Prerequisites: You must be in the role of an owner or developer.

Parameters
Returns

True if success

Return type

bool

Example:

dataset.items.move_items(destination='directory_name')
open_in_web(filepath=None, item_id=None, item=None)[source]

Open the item in web platform

Prerequisites: You must be in the role of an owner or developer or be an annotation manager/annotator with access to that item through task.

Parameters

Example:

dataset.items.open_in_web(item_id='item_id')
set_items_entity(entity)[source]

Set the item entity type to Artifact, Item, or Codebase.

Parameters

entity (entities.Item, entities.Artifact, entities.Codebase) – entity type [entities.Item, entities.Artifact, entities.Codebase]

update(item: Optional[Item] = None, filters: Optional[Filters] = None, update_values=None, system_update_values=None, system_metadata: bool = False)[source]

Update item metadata.

Prerequisites: You must be in the role of an owner or developer.

You must provide at least ONE of the following params: update_values, system_update_values.

Parameters
  • item (dtlpy.entities.item.Item) – Item object

  • filters (dtlpy.entities.filters.Filters) – optional update filtered items by given filter

  • update_values – optional field to be updated and new values

  • system_update_values – values in system metadata to be updated

  • system_metadata (bool) – True, if you want to update the metadata system

Returns

Item object

Return type

dtlpy.entities.item.Item

Example:

dataset.items.update(item='item_entity')
update_status(status: ItemStatus, items=None, item_ids=None, filters=None, dataset=None, clear=False)[source]

Update item status in task

Prerequisites: You must be in the role of an owner or developer or annotation manager who has been assigned a task with the item.

You must provide at least ONE of the following params: items, item_ids, filters.

Parameters

Example:

dataset.items.update_status(item_ids='item_id', status=dl.ItemStatus.COMPLETED)
upload(local_path: str, local_annotations_path: ~typing.Optional[str] = None, remote_path: str = '/', remote_name: ~typing.Optional[str] = None, file_types: ~typing.Optional[~dtlpy.repositories.items.Items.list] = None, overwrite: bool = False, item_metadata: ~typing.Optional[dict] = None, output_entity=<class 'dtlpy.entities.item.Item'>, no_output: bool = False, export_version: str = ExportVersion.V1, item_description: ~typing.Optional[str] = None)[source]

Upload local file to dataset. Local filesystem will remain unchanged. If “*” at the end of local_path (e.g. “/images/*”) items will be uploaded without the head directory.

Prerequisites: Any user can upload items.

Parameters
  • local_path (str) – list of local file, local folder, BufferIO, numpy.ndarray or url to upload

  • local_annotations_path (str) – path to dataloop format annotations json files.

  • remote_path (str) – remote path to save.

  • remote_name (str) – remote base name to save. when upload numpy.ndarray as local path, remote_name with .jpg or .png ext is mandatory

  • file_types (list) – list of file type to upload. e.g [‘.jpg’, ‘.png’]. default is all

  • item_metadata (dict) – metadata dict to upload to item or ExportMetadata option to export metadata from annotation file

  • overwrite (bool) – optional - default = False

  • output_entity – output type

  • no_output (bool) – do not return the items after upload

  • export_version (str) – exported items will have original extension in filename, V1 - no original extension in filenames

  • item_description (str) – add a string description to the uploaded item

Returns

Output (generator/single item)

Return type

generator or single item

Example:

dataset.items.upload(local_path='local_path',
                     local_annotations_path='local_annotations_path',
                     overwrite=True,
                     item_metadata={'Hellow': 'Word'}
                     )

Annotations

class Annotations(client_api: ApiClient, item=None, dataset=None, dataset_id=None)[source]

Bases: object

Annotations Repository

The Annotation class allows you to manage the annotations of data items. For information on annotations explore our documentation at: Classification SDK, Annotation Labels and Attributes, Show Video with Annotations.

builder()[source]

Create Annotation collection.

Prerequisites: You must have an item to be annotated. You must have the role of an owner or developer

or be assigned a task that includes that item as an annotation manager or annotator.

Returns

Annotation collection object

Return type

dtlpy.entities.annotation_collection.AnnotationCollection

Example:

item.annotations.builder()
delete(annotation: Optional[Annotation] = None, annotation_id: Optional[str] = None, filters: Optional[Filters] = None) bool[source]

Remove an annotation from item.

Prerequisites: You must have an item that has been annotated. You must have the role of an owner or developer or be assigned a task that includes that item as an annotation manager or annotator.

Parameters
Returns

True/False

Return type

bool

Example:

item.annotations.delete(annotation_id='annotation_id')
download(filepath: str, annotation_format: ViewAnnotationOptions = ViewAnnotationOptions.JSON, img_filepath: Optional[str] = None, height: Optional[float] = None, width: Optional[float] = None, thickness: int = 1, with_text: bool = False, alpha: float = 1)[source]

Save annotation to file.

Prerequisites: You must have an item that has been annotated. You must have the role of an owner or developer or be assigned a task that includes that item as an annotation manager or annotator.

Parameters
  • filepath (str) – Target download directory

  • annotation_format (list) – optional - list(dl.ViewAnnotationOptions)

  • img_filepath (str) – img file path - needed for img_mask

  • height (float) – optional - image height

  • width (float) – optional - image width

  • thickness (int) – optional - annotation format, default =1

  • with_text (bool) – optional - draw annotation with text, default = False

  • alpha (float) – opacity value [0 1], default 1

Returns

file path to where save the annotations

Return type

str

Example:

item.annotations.download(
              filepath='file_path',
              annotation_format=dl.ViewAnnotationOptions.MASK,
              img_filepath='img_filepath',
              height=100,
              width=100,
              thickness=1,
              with_text=False,
              alpha=1)
get(annotation_id: str) Annotation[source]

Get a single annotation.

Prerequisites: You must have an item that has been annotated. You must have the role of an owner or developer or be assigned a task that includes that item as an annotation manager or annotator.

Parameters

annotation_id (str) – annotation id

Returns

Annotation object or None

Return type

dtlpy.entities.annotation.Annotation

Example:

item.annotations.get(annotation_id='annotation_id')
list(filters: Optional[Filters] = None, page_offset: Optional[int] = None, page_size: Optional[int] = None)[source]

List Annotations of a specific item. You must get the item first and then list the annotations with the desired filters.

Prerequisites: You must have an item that has been annotated. You must have the role of an owner or developer or be assigned a task that includes that item as an annotation manager or annotator.

Parameters
Returns

Pages object

Return type

dtlpy.entities.paged_entities.PagedEntities

Example:

item.annotations.list(filters=dl.Filters(
                             resource=dl.FiltersResource.ANNOTATION,
                             field='type',
                             values='box'),
          page_size=100,
          page_offset=0)
show(image=None, thickness: int = 1, with_text: bool = False, height: Optional[float] = None, width: Optional[float] = None, annotation_format: ViewAnnotationOptions = ViewAnnotationOptions.MASK, alpha: float = 1)[source]

Show annotations. To use this method, you must get the item first and then show the annotations with the desired filters. The method returns an array showing all the annotations.

Prerequisites: You must have an item that has been annotated. You must have the role of an owner or developer or be assigned a task that includes that item as an annotation manager or annotator.

Parameters
  • image (ndarray) – empty or image to draw on

  • thickness (int) – line thickness

  • with_text (bool) – add label to annotation

  • height (float) – height

  • width (float) – width

  • annotation_format (str) – options: list(dl.ViewAnnotationOptions)

  • alpha (float) – opacity value [0 1], default 1

Returns

ndarray of the annotations

Return type

ndarray

Example:

item.annotations.show(image='nd array',
          thickness=1,
          with_text=False,
          height=100,
          width=100,
          annotation_format=dl.ViewAnnotationOptions.MASK,
          alpha=1)
update(annotations, system_metadata=False)[source]

Update an existing annotation. For example, you may change the annotation’s label and then use the update method.

Prerequisites: You must have an item that has been annotated. You must have the role of an owner or

developer or be assigned a task that includes that item as an annotation manager or annotator.

Parameters
Returns

True if successful or error if unsuccessful

Return type

bool

Example:

item.annotations.update(annotation='annotation')
update_status(annotation: Optional[Annotation] = None, annotation_id: Optional[str] = None, status: AnnotationStatus = AnnotationStatus.ISSUE) Annotation[source]

Set status on annotation.

Prerequisites: You must have an item that has been annotated. You must have the role of an owner or developer or be assigned a task that includes that item as an annotation manager.

Parameters
Returns

Annotation object

Return type

dtlpy.entities.annotation.Annotation

Example:

item.annotations.update_status(annotation_id='annotation_id', status=dl.AnnotationStatus.ISSUE)
upload(annotations) AnnotationCollection[source]

Upload a new annotation/annotations. You must first create the annotation using the annotation builder method.

Prerequisites: Any user can upload annotations.

Parameters

annotations (List[dtlpy.entities.annotation.Annotation] or dtlpy.entities.annotation.Annotation) – list or

single annotation of type Annotation :return: list of annotation objects :rtype: entities.AnnotationCollection

Example:

item.annotations.upload(annotations='builder')

Recipes

class Recipes(client_api: ApiClient, dataset: Optional[Dataset] = None, project: Optional[Project] = None, project_id: Optional[str] = None)[source]

Bases: object

Recipes Repository

The Recipes class allows you to manage recipes and their properties. For more information on Recipes, see our documentation and SDK documentation.

clone(recipe: Optional[Recipe] = None, recipe_id: Optional[str] = None, shallow: bool = False)[source]

Clone recipe.

Prerequisites: You must be in the role of an owner or developer.

Parameters
  • recipe (dtlpy.entities.recipe.Recipe) – Recipe object

  • recipe_id (str) – Recipe id

  • shallow (bool) – If True, link to existing ontology, clones all ontologies that are linked to the recipe as well

Returns

Cloned ontology object

Return type

dtlpy.entities.recipe.Recipe

Example:

dataset.recipes.clone(recipe_id='recipe_id')
create(project_ids=None, ontology_ids=None, labels=None, recipe_name=None, attributes=None, annotation_instruction_file=None) Recipe[source]

Create a new Recipe. Note: If the param ontology_ids is None, an ontology will be created first.

Prerequisites: You must be in the role of an owner or developer.

Parameters
  • project_ids (str) – project ids

  • ontology_ids (str or list) – ontology ids

  • labels – labels

  • recipe_name (str) – recipe name

  • attributes – attributes

  • annotation_instruction_file (str) – file path or url of the recipe instruction

Returns

Recipe entity

Return type

dtlpy.entities.recipe.Recipe

Example:

dataset.recipes.create(recipe_name='My Recipe', labels=labels))
delete(recipe_id: str, force: bool = False)[source]

Delete recipe from platform.

Prerequisites: You must be in the role of an owner or developer.

Parameters
  • recipe_id (str) – recipe id

  • force (bool) – force delete recipe

Returns

True if success

Return type

bool

Example:

dataset.recipes.delete(recipe_id='recipe_id')
get(recipe_id: str) Recipe[source]

Get a Recipe object to use in your code.

Prerequisites: You must be in the role of an owner or developer.

Parameters

recipe_id (str) – recipe id

Returns

Recipe object

Return type

dtlpy.entities.recipe.Recipe

Example:

dataset.recipes.get(recipe_id='recipe_id')
list(filters: Optional[Filters] = None) List[Recipe][source]

List recipes for a dataset.

Prerequisites: You must be in the role of an owner or developer.

Parameters

filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters

Returns

list of all recipes

Retype

list

Example:

dataset.recipes.list()
open_in_web(recipe: Optional[Recipe] = None, recipe_id: Optional[str] = None)[source]

Open the recipe in web platform.

Prerequisites: All users.

Parameters

Example:

dataset.recipes.open_in_web(recipe_id='recipe_id')
update(recipe: Recipe, system_metadata=False) Recipe[source]

Update recipe.

Prerequisites: You must be in the role of an owner or developer.

Parameters
Returns

Recipe object

Return type

dtlpy.entities.recipe.Recipe

Example:

dataset.recipes.update(recipe='recipe_entity')

Ontologies

class Ontologies(client_api: ApiClient, recipe: Optional[Recipe] = None, project: Optional[Project] = None, dataset: Optional[Dataset] = None)[source]

Bases: object

Ontologies Repository

The Ontologies class allows users to manage ontologies and their properties. Read more about ontology in our SDK docs.

create(labels, title=None, project_ids=None, attributes=None) Ontology[source]

Create a new ontology.

Prerequisites: You must be in the role of an owner or developer.

Parameters
  • labels – recipe tags

  • title (str) – ontology title, name

  • project_ids (list) – recipe project/s

  • attributes (list) – recipe attributes

Returns

Ontology object

Return type

dtlpy.entities.ontology.Ontology

Example:

recipe.ontologies.create(labels='labels_entity',
                      title='new_ontology',
                      project_ids='project_ids')
delete(ontology_id)[source]

Delete Ontology from the platform.

Prerequisites: You must be in the role of an owner or developer.

Parameters

ontology_id – ontology id

Returns

True if success

Return type

bool

Example:

recipe.ontologies.delete(ontology_id='ontology_id')
delete_attributes(ontology_id, keys: list)[source]

Delete a bulk of attributes

Parameters
  • ontology_id (str) – ontology id

  • keys (list) – Keys of attributes to delete

Returns

True if success

Return type

bool

Example:

ontology.delete_attributes(['1'])
get(ontology_id: str) Ontology[source]

Get Ontology object to use in your code.

Prerequisites: You must be in the role of an owner or developer.

Parameters

ontology_id (str) – ontology id

Returns

Ontology object

Return type

dtlpy.entities.ontology.Ontology

Example:

recipe.ontologies.get(ontology_id='ontology_id')
static labels_to_roots(labels)[source]

Converts labels dictionary to a list of platform representation of labels.

Parameters

labels (dict) – labels dict

Returns

platform representation of labels

list(project_ids=None) List[Ontology][source]

List ontologies for recipe

Prerequisites: You must be in the role of an owner or developer.

Parameters

project_ids

Returns

list of all the ontologies

Example:

recipe.ontologies.list(project_ids='project_ids')
update(ontology: Ontology, system_metadata=False) Ontology[source]

Update the Ontology metadata.

Prerequisites: You must be in the role of an owner or developer.

Parameters
Returns

Ontology object

Return type

dtlpy.entities.ontology.Ontology

Example:

recipe.ontologies.delete(ontology='ontology_entity')
update_attributes(ontology_id: str, title: str, key: str, attribute_type: AttributesTypes, scope: Optional[list] = None, optional: Optional[bool] = None, values: Optional[list] = None, attribute_range: Optional[AttributesRange] = None)[source]

ADD a new attribute or update if exist

Parameters
  • ontology_id (str) – ontology_id

  • title (str) – attribute title

  • key (str) – the key of the attribute must br unique

  • attribute_type (AttributesTypes) – dl.AttributesTypes your attribute type

  • scope (list) – list of the labels or * for all labels

  • optional (bool) – optional attribute

  • values (list) – list of the attribute values ( for checkbox and radio button)

  • attribute_range (dict or AttributesRange) – dl.AttributesRange object

Returns

true in success

Return type

bool

Example:

ontology.update_attributes(key='1',
                           title='checkbox',
                           attribute_type=dl.AttributesTypes.CHECKBOX,
                           values=[1,2,3])

Tasks

class Tasks(client_api: ApiClient, project: Optional[Project] = None, dataset: Optional[Dataset] = None, project_id: Optional[str] = None)[source]

Bases: object

Tasks Repository

The Tasks class allows the user to manage tasks and their properties. For more information, read in our SDK documentation about Creating Tasks, Redistributing and Reassigning Tasks, and Task Assignment.

add_items(task: Optional[Task] = None, task_id=None, filters: Optional[Filters] = None, items=None, assignee_ids=None, query=None, workload=None, limit=None, wait=True) Task[source]

Add items to a Task.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned to be owner of the annotation task.

Parameters
  • task (dtlpy.entities.task.Task) – task entity

  • task_id (str) – task id

  • filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters

  • items (list) – list of items to add to the task

  • assignee_ids (list) – list to assignee who works in the task

  • query (dict) – query to filter the items use it

  • workload (list) – list of the work load ber assignee and work load

  • limit (int) – task limit

  • wait (bool) – wait for the command to finish

Returns

task entity

Return type

dtlpy.entities.task.Task

Example:

dataset.tasks.add_items(task= 'task_entity',
                    items = [items])
create(task_name, due_date=None, assignee_ids=None, workload=None, dataset=None, task_owner=None, task_type='annotation', task_parent_id=None, project_id=None, recipe_id=None, assignments_ids=None, metadata=None, filters=None, items=None, query=None, available_actions=None, wait=True, check_if_exist: Filters = False, limit=None, batch_size=None, max_batch_workload=None, allowed_assignees=None) Task[source]

Create a new Annotation Task.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned to be owner of the annotation task.

Parameters
  • task_name (str) – task name

  • due_date (float) – date by which the task should be finished; for example, due_date = datetime.datetime(day= 1, month= 1, year= 2029).timestamp()

  • assignee_ids (list) – list of assignee

  • workload (List[WorkloadUnit]) – list WorkloadUnit for the task assignee

  • dataset (entities.Dataset) – dataset entity

  • task_owner (str) – task owner

  • task_type (str) – “annotation” or “qa”

  • task_parent_id (str) – optional if type is qa - parent task id

  • project_id (str) – project id

  • recipe_id (str) – recipe id

  • assignments_ids (list) – assignments ids

  • metadata (dict) – metadata for the task

  • filters (entities.Filters) – filter to the task

  • items (List[entities.Item]) – item to insert to the task

  • query (entities.Filters) – filter to the task

  • available_actions (list) – list of available actions to the task

  • wait (bool) – wait for the command to finish

  • check_if_exist (entities.Filters) – dl.Filters check if task exist according to filter

  • limit (int) – task limit

  • batch_size (int) – Pulling batch size (items) . Restrictions - Min 3, max 100

  • max_batch_workload (int) – Max items in assignment . Restrictions - Min batchSize + 2 , max batchSize * 2

  • allowed_assignees (list) – It’s like the workload, but without percentage.

Returns

Annotation Task object

Return type

dtlpy.entities.task.Task

Example:

dataset.tasks.create(task= 'task_entity',
                    due_date = datetime.datetime(day= 1, month= 1, year= 2029).timestamp(),
                    assignee_ids =[ 'annotator1@dataloop.ai', 'annotator2@dataloop.ai'])
create_qa_task(task: Task, assignee_ids, due_date=None, filters=None, items=None, query=None, workload=None, metadata=None, available_actions=None, wait=True, batch_size=None, max_batch_workload=None, allowed_assignees=None) Task[source]

Create a new QA Task.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned to be owner of the annotation task.

Parameters
  • task (dtlpy.entities.task.Task) – parent task

  • assignee_ids (list) – list of assignee

  • due_date (float) – date by which the task should be finished; for example, due_date = datetime.datetime(day= 1, month= 1, year= 2029).timestamp()

  • filters (entities.Filters) – filter to the task

  • items (List[entities.Item]) – item to insert to the task

  • query (entities.Filters) – filter to the task

  • workload (List[WorkloadUnit]) – list WorkloadUnit for the task assignee

  • metadata (dict) – metadata for the task

  • available_actions (list) – list of available actions to the task

  • wait (bool) – wait for the command to finish

  • batch_size (int) – Pulling batch size (items) . Restrictions - Min 3, max 100

  • max_batch_workload (int) – Max items in assignment . Restrictions - Min batchSize + 2 , max batchSize * 2

  • allowed_assignees (list) – It’s like the workload, but without percentage.

Returns

task object

Return type

dtlpy.entities.task.Task

Example:

dataset.tasks.create_qa_task(task= 'task_entity',
                            due_date = datetime.datetime(day= 1, month= 1, year= 2029).timestamp(),
                            assignee_ids =[ 'annotator1@dataloop.ai', 'annotator2@dataloop.ai'])
delete(task: Optional[Task] = None, task_name: Optional[str] = None, task_id: Optional[str] = None, wait: bool = True)[source]

Delete an Annotation Task.

Prerequisites: You must be in the role of an owner or developer or annotation manager who created that task.

Parameters
Returns

True is success

Return type

bool

Example:

dataset.tasks.delete(task_id='task_id')
get(task_name=None, task_id=None) Task[source]

Get an Annotation Task object to use in your code.

Prerequisites: You must be in the role of an owner or developer or annotation manager who has been assigned the task.

Parameters
  • task_name (str) – optional - search by name

  • task_id (str) – optional - search by id

Returns

task object

Return type

dtlpy.entities.task.Task

Example:

dataset.tasks.get(task_id='task_id')
get_items(task_id: Optional[str] = None, task_name: Optional[str] = None, dataset: Optional[Dataset] = None, filters: Optional[Filters] = None) PagedEntities[source]

Get the task items to use in your code.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned to be owner of the annotation task.

If a filters param is provided, you will receive a PagedEntity output of the task items. If no filter is provided, you will receive a list of the items.

Parameters
Returns

list of the items or PagedEntity output of items

Return type

list or dtlpy.entities.paged_entities.PagedEntities

Example:

dataset.tasks.get_items(task_id= 'task_id')
list(project_ids=None, status=None, task_name=None, pages_size=None, page_offset=None, recipe=None, creator=None, assignments=None, min_date=None, max_date=None, filters: Optional[Filters] = None) Union[List[Task], PagedEntities][source]

List all Annotation Tasks.

Prerequisites: You must be in the role of an owner or developer or annotation manager who has been assigned the task.

Parameters
  • project_ids – list of project ids

  • status (str) – status

  • task_name (str) – task name

  • pages_size (int) – pages size

  • page_offset (int) – page offset

  • recipe (dtlpy.entities.recipe.Recipe) – recipe entity

  • creator (str) – creator

  • assignments (dtlpy.entities.assignment.Assignment recipe) – assignments entity

  • min_date (double) – double min date

  • max_date (double) – double max date

  • filters (dtlpy.entities.filters.Filters) – dl.Filters entity to filters items

Returns

List of Annotation Task objects

Example:

dataset.tasks.list(project_ids='project_ids',pages_size=100, page_offset=0)
open_in_web(task_name: Optional[str] = None, task_id: Optional[str] = None, task: Optional[Task] = None)[source]

Open the task in the web platform.

Prerequisites: You must be in the role of an owner or developer or annotation manager who has been assigned the task.

Parameters

Example:

dataset.tasks.open_in_web(task_id='task_id')
query(filters=None, project_ids=None)[source]

List all tasks by filter.

Prerequisites: You must be in the role of an owner or developer or annotation manager who has been assigned the task.

Parameters
Returns

Paged entity

Return type

dtlpy.entities.paged_entities.PagedEntities

Example:

dataset.tasks.query(project_ids='project_ids')
remove_items(task: Optional[Task] = None, task_id=None, filters: Optional[Filters] = None, query=None, items=None, wait=True)[source]

remove items from Task.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned to be owner of the annotation task.

Parameters
Returns

True if success and an error if failed

Return type

bool

Examples:

dataset.tasks.remove_items(task= 'task_entity',
                            items = [items])
set_status(status: str, operation: str, task_id: str, item_ids: List[str])[source]

Update an item status within a task.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned to be owner of the annotation task.

Parameters
  • status (str) – string the describes the status

  • operation (str) – ‘create’ or ‘delete’

  • task_id (str) – task id

  • item_ids (list) – List[str] id items ids

Returns

True if success

Return type

bool

Example:

dataset.tasks.set_status(task_id= 'task_id', status='complete', operation='create')
update(task: Optional[Task] = None, system_metadata=False) Task[source]

Update an Annotation Task.

Prerequisites: You must be in the role of an owner or developer or annotation manager who created that task.

Parameters
Returns

Annotation Task object

Return type

dtlpy.entities.task.Task

Example:

dataset.tasks.update(task='task_entity')

Assignments

class Assignments(client_api: ApiClient, project: Optional[Project] = None, task: Optional[Task] = None, dataset: Optional[Dataset] = None, project_id=None)[source]

Bases: object

Assignments Repository

The Assignments class allows users to manage assignments and their properties. Read more about Task Assignment in our SDK documentation.

create(assignee_id: str, task: Optional[Task] = None, filters: Optional[Filters] = None, items: Optional[list] = None) Assignment[source]

Create a new assignment.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned as owner of the annotation task.

Parameters
Returns

Assignment object

Return type

dtlpy.entities.assignment.Assignment assignment

Example:

task.assignments.create(assignee_id='annotator1@dataloop.ai')
get(assignment_name: Optional[str] = None, assignment_id: Optional[str] = None)[source]

Get Assignment object to use it in your code.

Parameters
  • assignment_name (str) – optional - search by name

  • assignment_id (str) – optional - search by id

Returns

Assignment object

Return type

dtlpy.entities.assignment.Assignment

Example:

task.assignments.get(assignment_id='assignment_id')
get_items(assignment: Optional[Assignment] = None, assignment_id=None, assignment_name=None, dataset=None, filters=None) PagedEntities[source]

Get all the items in the assignment.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned as owner of the annotation task.

Parameters
Returns

pages of the items

Return type

dtlpy.entities.paged_entities.PagedEntities

Example:

task.assignments.get_items(assignment_id='assignment_id')
list(project_ids: Optional[list] = None, status: Optional[str] = None, assignment_name: Optional[str] = None, assignee_id: Optional[str] = None, pages_size: Optional[int] = None, page_offset: Optional[int] = None, task_id: Optional[int] = None) List[Assignment][source]

Get Assignment list to be able to use it in your code.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned as owner of the annotation task.

Parameters
  • project_ids (list) – list of project ids

  • status (str) – assignment status

  • assignment_name (str) – assignment name

  • assignee_id (str) – the user that assignee the assignment to it

  • pages_size (int) – pages size

  • page_offset (int) – page offset

  • task_id (str) – task id

Returns

List of Assignment objects

Return type

miscellaneous.List[dtlpy.entities.assignment.Assignment]

Example:

task.assignments.list(status='complete', assignee_id='user@dataloop.ai', pages_size=100, page_offset=0)
open_in_web(assignment_name: Optional[str] = None, assignment_id: Optional[str] = None, assignment: Optional[str] = None)[source]

Open the assignment in the platform.

Prerequisites: All users.

Parameters

Example:

task.assignments.open_in_web(assignment_id='assignment_id')
reassign(assignee_id: str, assignment: Optional[Assignment] = None, assignment_id: Optional[str] = None, task: Optional[Task] = None, task_id: Optional[str] = None, wait: bool = True)[source]

Reassign an assignment.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned as owner of the annotation task.

Parameters
Returns

Assignment object

Return type

dtlpy.entities.assignment.Assignment

Example:

task.assignments.reassign(assignee_ids='annotator1@dataloop.ai')
redistribute(workload: Workload, assignment: Optional[Assignment] = None, assignment_id: Optional[str] = None, task: Optional[Task] = None, task_id: Optional[str] = None, wait: bool = True)[source]

Redistribute an assignment.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned as owner of the annotation task.

Example:

Parameters
Returns

Assignment object

Return type

dtlpy.entities.assignment.Assignment assignment

task.assignments.redistribute(workload=dl.Workload([dl.WorkloadUnit(assignee_id="annotator1@dataloop.ai", load=50),
                                                    dl.WorkloadUnit(assignee_id="annotator2@dataloop.ai", load=50)]))
set_status(status: str, operation: str, item_id: str, assignment_id: str) bool[source]

Set item status within assignment.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned as owner of the annotation task.

Parameters
  • status (str) – status

  • operation (str) – created/deleted

  • item_id (str) – item id

  • assignment_id (str) – assignment id

Returns

True id success

Return type

bool

Example:

task.assignments.set_status(assignment_id='assignment_id',
                            status='complete',
                            operation='created',
                            item_id='item_id')
update(assignment: Optional[Assignment] = None, system_metadata: bool = False) Assignment[source]

Update an assignment.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned as owner of the annotation task.

Parameters
  • assignment (dtlpy.entities.assignment.Assignment assignment) – assignment entity

  • system_metadata (bool) – True, if you want to change metadata system

Returns

Assignment object

Return type

dtlpy.entities.assignment.Assignment assignment

Example:

task.assignments.update(assignment='assignment_entity', system_metadata=False)

Packages

class LocalServiceRunner(client_api: ApiClient, packages, cwd=None, multithreading=False, concurrency=10, package: Optional[Package] = None, module_name='default_module', function_name='run', class_name='ServiceRunner', entry_point='main.py', mock_file_path=None)[source]

Bases: object

Service Runner Class

get_field(field_name, field_type, mock_json, project=None, mock_inputs=None)[source]

Get field in mock json.

Parameters
  • field_name – field name

  • field_type – field type

  • mock_json – mock json

  • project – project

  • mock_inputs – mock inputs

Returns

get_mainpy_run_service()[source]

Get mainpy run service

Returns

run_local_project(project=None)[source]

Run local project

Parameters

project – project entity

class Packages(client_api: ApiClient, project: Optional[Project] = None)[source]

Bases: object

Packages Repository

The Packages class allows users to manage packages (code used for running in Dataloop’s FaaS) and their properties. Read more about Packages.

build_requirements(filepath) list[source]

Build a requirement list (list of packages your code requires to run) from a file path. The file listing the requirements MUST BE a txt file.

Prerequisites: You must be in the role of an owner or developer.

Parameters

filepath – path of the requirements file

Returns

a list of dl.PackageRequirement

Return type

list

static build_trigger_dict(actions, name='default_module', filters=None, function='run', execution_mode: TriggerExecutionMode = 'Once', type_t: TriggerType = 'Event')[source]

Build a trigger dictionary to trigger FaaS. Read more about FaaS Triggers.

Prerequisites: You must be in the role of an owner or developer.

Parameters
  • actions – list of dl.TriggerAction

  • name (str) – trigger name

  • filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters

  • function (str) – function name

  • execution_mode (str) – execution mode dl.TriggerExecutionMode

  • type_t (str) – trigger type dl.TriggerType

Returns

trigger dict

Return type

dict

Example:

project.packages.build_trigger_dict(actions=dl.TriggerAction.CREATED,
                                  function='run',
                                  execution_mode=dl.TriggerExecutionMode.ONCE)
static check_cls_arguments(cls, missing, function_name, function_inputs)[source]

Check class arguments. This method checks that the package function is correct.

Prerequisites: You must be in the role of an owner or developer.

Parameters
  • cls – packages class

  • missing (list) – list of the missing params

  • function_name (str) – name of function

  • function_inputs (list) – list of function inputs

checkout(package: Optional[Package] = None, package_id: Optional[str] = None, package_name: Optional[str] = None)[source]

Checkout (switch) to a package.

Prerequisites: You must be in the role of an owner or developer.

Parameters

Example:

project.packages.checkout(package='package_entity')
delete(package: Optional[Package] = None, package_name=None, package_id=None)[source]

Delete a Package object.

Prerequisites: You must be in the role of an owner or developer.

Parameters
Returns

True if success

Return type

bool

Example:

project.packages.delete(package_name='package_name')
deploy(package_id: Optional[str] = None, package_name: Optional[str] = None, package: Optional[Package] = None, service_name: Optional[str] = None, project_id: Optional[str] = None, revision: Optional[str] = None, init_input: Optional[Union[List[FunctionIO], FunctionIO, dict]] = None, runtime: Optional[Union[KubernetesRuntime, dict]] = None, sdk_version: Optional[str] = None, agent_versions: Optional[dict] = None, bot: Optional[Union[Bot, str]] = None, pod_type: Optional[InstanceCatalog] = None, verify: bool = True, checkout: bool = False, module_name: Optional[str] = None, run_execution_as_process: Optional[bool] = None, execution_timeout: Optional[int] = None, drain_time: Optional[int] = None, on_reset: Optional[str] = None, max_attempts: Optional[int] = None, force: bool = False, secrets: Optional[list] = None, **kwargs) Service[source]

Deploy a package. A service is required to run the code in your package.

Prerequisites: You must be in the role of an owner or developer.

Parameters
  • package_id (str) – package id

  • package_name (str) – package name

  • package (dtlpy.entities.package.Package) – package entity

  • service_name (str) – service name

  • project_id (str) – project id

  • revision (str) – package revision - default=latest

  • init_input – config to run at startup

  • runtime (dict) – runtime resources

  • sdk_version (str) –

    • optional - string - sdk version

  • agent_versions (dict) –

    • dictionary - - optional -versions of sdk, agent runner and agent proxy

  • bot (str) – bot email

  • pod_type (str) – pod type dl.InstanceCatalog

  • verify (bool) – verify the inputs

  • checkout (bool) – checkout

  • module_name (str) – module name

  • run_execution_as_process (bool) – run execution as process

  • execution_timeout (int) – execution timeout

  • drain_time (int) – drain time

  • on_reset (str) – on reset

  • max_attempts (int) – Maximum execution retries in-case of a service reset

  • force (bool) – optional - terminate old replicas immediately

  • secrets (list) – list of the integrations ids

Returns

Service object

Return type

dtlpy.entities.service.Service

Example:

project.packages.deploy(service_name=package_name,
                        execution_timeout=3 * 60 * 60,
                        module_name=module.name,
                        runtime=dl.KubernetesRuntime(
                            concurrency=10,
                            pod_type=dl.InstanceCatalog.REGULAR_S,
                            autoscaler=dl.KubernetesRabbitmqAutoscaler(
                                min_replicas=1,
                                max_replicas=20,
                                queue_length=20
                            )
                        )
                    )
deploy_from_file(project, json_filepath)[source]

Deploy package and service from a JSON file.

Prerequisites: You must be in the role of an owner or developer.

Parameters
Returns

the package and the services

Example:

project.packages.deploy_from_file(project='project_entity', json_filepath='json_filepath')
static generate(name=None, src_path: Optional[str] = None, service_name: Optional[str] = None, package_type='default_package_type')[source]

Generate a new package. Provide a file path to a JSON file with all the details of the package and service to generate the package.

Prerequisites: You must be in the role of an owner or developer.

Parameters
  • name (str) – name

  • src_path (str) – source file path

  • service_name (str) – service name

  • package_type (str) – package type from PackageCatalog

Example:

project.packages.generate(name='package_name',
                          src_path='src_path')
get(package_name: Optional[str] = None, package_id: Optional[str] = None, checkout: bool = False, fetch=None) Package[source]

Get Package object to use in your code.

Prerequisites: You must be in the role of an owner or developer.

Parameters
  • package_id (str) – package id

  • package_name (str) – package name

  • checkout (bool) – checkout

  • fetch – optional - fetch entity from platform, default taken from cookie

Returns

Package object

Return type

dtlpy.entities.package.Package

Example:

project.packages.get(package_id='package_id')
list(filters: Optional[Filters] = None, project_id: Optional[str] = None) PagedEntities[source]

List project packages.

Prerequisites: You must be in the role of an owner or developer.

Parameters
Returns

Paged entity

Return type

dtlpy.entities.paged_entities.PagedEntities

Example:

project.packages.list()
open_in_web(package: Optional[Package] = None, package_id: Optional[str] = None, package_name: Optional[str] = None)[source]

Open the package in the web platform.

Prerequisites: You must be in the role of an owner or developer.

Parameters

Example:

project.packages.open_in_web(package_id='package_id')
pull(package: Package, version=None, local_path=None, project_id=None)[source]

Pull (download) the package to a local path.

Prerequisites: You must be in the role of an owner or developer.

Parameters
Returns

local path where the package pull

Return type

str

Example:

project.packages.pull(package='package_entity', local_path='local_path')
push(project: Optional[Project] = None, project_id: Optional[str] = None, package_name: Optional[str] = None, src_path: Optional[str] = None, codebase: Optional[Union[GitCodebase, ItemCodebase, FilesystemCodebase]] = None, modules: Optional[List[PackageModule]] = None, is_global: Optional[bool] = None, checkout: bool = False, revision_increment: Optional[str] = None, version: Optional[str] = None, ignore_sanity_check: bool = False, service_update: bool = False, service_config: Optional[dict] = None, slots: Optional[List[PackageSlot]] = None, requirements: Optional[List[PackageRequirement]] = None) Package[source]

Push your local package to the UI.

Prerequisites: You must be in the role of an owner or developer.

Project will be taken in the following hierarchy: project(input) -> project_id(input) -> self.project(context) -> checked out

Parameters
  • project (dtlpy.entities.project.Project) – optional - project entity to deploy to. default from context or checked-out

  • project_id (str) – optional - project id to deploy to. default from context or checked-out

  • package_name (str) – package name

  • src_path (str) – path to package codebase

  • codebase (dtlpy.entities.codebase.Codebase) – codebase object

  • modules (list) – list of modules PackageModules of the package

  • is_global (bool) – is package is global or local

  • checkout (bool) – checkout package to local dir

  • revision_increment (str) – optional - str - version bumping method - major/minor/patch - default = None

  • version (str) – semver version f the package

  • ignore_sanity_check (bool) – NOT RECOMMENDED - skip code sanity check before pushing

  • service_update (bool) – optional - bool - update the service

  • service_config (dict) – json of service - a service that have config from the main service if wanted

  • slots (list) – optional - list of slots PackageSlot of the package

  • requirements (list) – requirements - list of package requirements

Returns

Package object

Return type

dtlpy.entities.package.Package

Example:

project.packages.push(package_name='package_name',
                        modules=[module],
                        version='1.0.0',
                        src_path=os.getcwd()
                    )
revisions(package: Optional[Package] = None, package_id: Optional[str] = None)[source]

Get the package revisions history.

Prerequisites: You must be in the role of an owner or developer.

Parameters

Example:

project.packages.revisions(package='package_entity')
test_local_package(cwd: Optional[str] = None, concurrency: Optional[int] = None, package: Optional[Package] = None, module_name: str = 'default_module', function_name: str = 'run', class_name: str = 'ServiceRunner', entry_point: str = 'main.py', mock_file_path: Optional[str] = None)[source]

Test local package in local environment.

Prerequisites: You must be in the role of an owner or developer.

Parameters
  • cwd (str) – path to the file

  • concurrency (int) – the concurrency of the test

  • package (dtlpy.entities.package.Package) – entities.package

  • module_name (str) – module name

  • function_name (str) – function name

  • class_name (str) – class name

  • entry_point (str) – the file to run like main.py

  • mock_file_path (str) – the mock file that have the inputs

Returns

list created by the function that tested the output

Return type

list

Example:

project.packages.test_local_package(cwd='path_to_package',
                                    package='package_entity',
                                    function_name='run')
update(package: Package, revision_increment: Optional[str] = None) Package[source]

Update Package changes to the platform.

Prerequisites: You must be in the role of an owner or developer.

Parameters
Returns

Package object

Return type

dtlpy.entities.package.Package

Example:

project.packages.delete(package='package_entity')

Codebases

class Codebases(client_api: ApiClient, project: Optional[Project] = None, dataset: Optional[Dataset] = None, project_id: Optional[str] = None)[source]

Bases: object

Codebase Repository

The Codebases class allows the user to manage codebases and their properties. The codebase is the code the user uploads for the user’s packages to run. Read more about codebase in our FaaS (function as a service).

clone_git(codebase: Codebase, local_path: str)[source]

Clone code base

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters
  • codebase (dtlpy.entities.codebase.Codebase) – codebase object

  • local_path (str) – local path

Returns

path where the clone will be

Return type

str

Example:

package.codebases.clone_git(codebase='codebase_entity', local_path='local_path')
get(codebase_name: Optional[str] = None, codebase_id: Optional[str] = None, version: Optional[str] = None)[source]

Get a Codebase object to use in your code.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Example:

package.codebases.get(codebase_name='codebase_name')
Parameters
  • codebase_name (str) – optional - search by name

  • codebase_id (str) – optional - search by id

  • version (str) – codebase version. default is latest. options: “all”, “latest” or ver number - “10”

Returns

Codebase object

static get_current_version(all_versions_pages, zip_md)[source]

This method returns the current version of the codebase and other versions found.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters
  • all_versions_pages (codebase) – codebase object

  • zip_md – zipped file of codebase

Returns

current version and all versions found of codebase

Return type

int, int

Example:

package.codebases.get_current_version(all_versions_pages='codebase_entity', zip_md='path')
list() PagedEntities[source]

List all codebases.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Example:

package.codebases.list()
Returns

Paged entity

Return type

dtlpy.entities.paged_entities.PagedEntities

list_versions(codebase_name: str)[source]

List all codebase versions.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Example:

package.codebases.list_versions(codebase_name='codebase_name')
Parameters

codebase_name (str) – code base name

Returns

list of versions

Return type

list

pack(directory: str, name: Optional[str] = None, description: str = '')[source]

Zip a local code directory and post to codebases.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters
  • directory (str) – local directory to pack

  • name (str) – codebase name

  • description (dtr) – codebase description

Returns

Codebase object

Return type

dtlpy.entities.codebase.Codebase

Example:

package.codebases.pack(directory='path_dir', name='codebase_name')
pull_git(codebase, local_path)[source]

Pull (download) a codebase.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters
  • codebase (dtlpy.entities.codebase.Codebase) – codebase object

  • local_path (str) – local path

Returns

path where the Pull will be

Return type

str

Example:

package.codebases.pull_git(codebase='codebase_entity', local_path='local_path')
unpack(codebase: Optional[Codebase] = None, codebase_name: Optional[str] = None, codebase_id: Optional[str] = None, local_path: Optional[str] = None, version: Optional[str] = None)[source]

Unpack codebase locally. Download source code and unzip.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters
  • codebase (dtlpy.entities.codebase.Codebase) – dl.Codebase object

  • codebase_name (str) – search by name

  • codebase_id (str) – search by id

  • local_path (str) – local path to save codebase

  • version (str) – codebase version to unpack. default - latest

Returns

String (dirpath)

Return type

str

Example:

package.codebases.unpack(codebase='codebase_entity', local_path='local_path')

Services

class ServiceLog(_json: dict, service: Service, services: Services, start=None, follow=None, execution_id=None, function_name=None, replica_id=None, system=False)[source]

Bases: object

Service Log

view(until_completed)[source]

View logs

Parameters

until_completed

class Services(client_api: ApiClient, project: Optional[Project] = None, package: Optional[Package] = None, project_id=None)[source]

Bases: object

Services Repository

The Services class allows the user to manage services and their properties. Services are created from the packages users create. See our documentation for more information about services.

activate_slots(service: Service, project_id: Optional[str] = None, task_id: Optional[str] = None, dataset_id: Optional[str] = None, org_id: Optional[str] = None, user_email: Optional[str] = None, slots: Optional[List[PackageSlot]] = None, role=None, prevent_override: bool = True, visible: bool = True, icon: str = 'fas fa-magic', **kwargs)[source]

Activate service slots (creates buttons in the UI that activate services).

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters
  • service (dtlpy.entities.service.Service) – service entity

  • project_id (str) – project id

  • task_id (str) – task id

  • dataset_id (str) – dataset id

  • org_id (str) – org id

  • user_email (str) – user email

  • slots (list) – list of entities.PackageSlot

  • role (str) – user role MemberOrgRole.ADMIN, MemberOrgRole.owner, MemberOrgRole.MEMBER

  • prevent_override (bool) – True to prevent override

  • visible (bool) – visible

  • icon (str) – icon

  • kwargs – all additional arguments

Returns

list of user setting for activated slots

Return type

list

Example:

package.services.activate_slots(service='service_entity',
                                project_id='project_id',
                                slots=List[entities.PackageSlot],
                                icon='fas fa-magic')
checkout(service: Optional[Service] = None, service_name: Optional[str] = None, service_id: Optional[str] = None)[source]

Checkout (switch) to a service.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters

Example:

package.services.checkout(service_id='service_id')
delete(service_name: Optional[str] = None, service_id: Optional[str] = None)[source]

Delete Service object

Prerequisites: You must be in the role of an owner or developer. You must have a package.

You must provide at least ONE of the following params: service_id, service_name.

Parameters
  • service_name (str) – by name

  • service_id (str) – by id

Returns

True

Return type

bool

Example:

package.services.delete(service_id='service_id')
deploy(service_name: Optional[str] = None, package: Optional[Package] = None, bot: Optional[Union[Bot, str]] = None, revision: Optional[str] = None, init_input: Optional[Union[List[FunctionIO], FunctionIO, dict]] = None, runtime: Optional[Union[KubernetesRuntime, dict]] = None, pod_type: Optional[InstanceCatalog] = None, sdk_version: Optional[str] = None, agent_versions: Optional[dict] = None, verify: bool = True, checkout: bool = False, module_name: Optional[str] = None, project_id: Optional[str] = None, driver_id: Optional[str] = None, func: Optional[Callable] = None, run_execution_as_process: Optional[bool] = None, execution_timeout: Optional[int] = None, drain_time: Optional[int] = None, max_attempts: Optional[int] = None, on_reset: Optional[str] = None, force: bool = False, secrets: Optional[list] = None, **kwargs) Service[source]

Deploy service.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters
  • service_name (str) – name

  • package (dtlpy.entities.package.Package) – package entity

  • bot (str) – bot email

  • revision (str) – package revision of version

  • init_input – config to run at startup

  • runtime (dict) – runtime resources

  • pod_type (str) – pod type dl.InstanceCatalog

  • sdk_version (str) –

    • optional - string - sdk version

  • agent_versions (str) –

    • dictionary - - optional -versions of sdk

  • verify (bool) – if true, verify the inputs

  • checkout (bool) – if true, checkout (switch) to service

  • module_name (str) – module name

  • project_id (str) – project id

  • driver_id (str) – driver id

  • func (Callable) – function to deploy

  • run_execution_as_process (bool) – if true, run execution as process

  • execution_timeout (int) – execution timeout in seconds

  • drain_time (int) – drain time in seconds

  • max_attempts (int) – maximum execution retries in-case of a service reset

  • on_reset (str) – what happens on reset

  • force (bool) – optional - if true, terminate old replicas immediately

  • secrets (list) – list of the integrations ids

  • kwargs – list of additional arguments

Returns

Service object

Return type

dtlpy.entities.service.Service

Example:

package.services.deploy(service_name=package_name,
                        execution_timeout=3 * 60 * 60,
                        module_name=module.name,
                        runtime=dl.KubernetesRuntime(
                            concurrency=10,
                            pod_type=dl.InstanceCatalog.REGULAR_S,
                            autoscaler=dl.KubernetesRabbitmqAutoscaler(
                                min_replicas=1,
                                max_replicas=20,
                                queue_length=20
                            )
                        )
                    )
deploy_from_local_folder(cwd=None, service_file=None, bot=None, checkout=False, force=False) Service[source]

Deploy from local folder in local environment.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters
  • cwd (str) – optional - package working directory. Default=cwd

  • service_file (str) – optional - service file. Default=None

  • bot (str) – bot

  • checkout – checkout

  • force (bool) – optional - terminate old replicas immediately

Returns

Service object

Return type

dtlpy.entities.service.Service

Example:

package.services.deploy_from_local_folder(cwd='file_path',
                                          service_file='service_file')
execute(service: Optional[Service] = None, service_id: Optional[str] = None, service_name: Optional[str] = None, sync: bool = False, function_name: Optional[str] = None, stream_logs: bool = False, execution_input=None, resource=None, item_id=None, dataset_id=None, annotation_id=None, project_id=None) Execution[source]

Execute a function on an existing service.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters
  • service (dtlpy.entities.service.Service) – service entity

  • service_id (str) – service id

  • service_name (str) – service name

  • sync (bool) – wait for function to end

  • function_name (str) – function name to run

  • stream_logs (bool) – prints logs of the new execution. only works with sync=True

  • execution_input – input dictionary or list of FunctionIO entities

  • resource (str) – dl.PackageInputType - input type.

  • item_id (str) – str - optional - input to function

  • dataset_id (str) – str - optional - input to function

  • annotation_id (str) – str - optional - input to function

  • project_id (str) – str - resource’s project

Returns

entities.Execution

Return type

dtlpy.entities.execution.Execution

Example:

package.services.execute(service='service_entity',
                        function_name='run',
                        item_id='item_id',
                        project_id='project_id')
get(service_name=None, service_id=None, checkout=False, fetch=None) Service[source]

Get service to use in your code.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters
  • service_name (str) – optional - search by name

  • service_id (str) – optional - search by id

  • checkout (bool) – if true, checkout (switch) to service

  • fetch – optional - fetch entity from platform, default taken from cookie

Returns

Service object

Return type

dtlpy.entities.service.Service

Example:

package.services.get(service_id='service_id')
list(filters: Optional[Filters] = None) PagedEntities[source]

List all services (services can be listed for a package or for a project).

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters

filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters

Returns

Paged entity

Return type

dtlpy.entities.paged_entities.PagedEntities

Example:

package.services.list()
log(service, size=100, checkpoint=None, start=None, end=None, follow=False, text=None, execution_id=None, function_name=None, replica_id=None, system=False, view=True, until_completed=True)[source]

Get service logs.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters
  • service (dtlpy.entities.service.Service) – service object

  • size (int) – size

  • checkpoint (dict) – the information from the lst point checked in the service

  • start (str) – iso format time

  • end (str) – iso format time

  • follow (bool) – if true, keep stream future logs

  • text (str) – text

  • execution_id (str) – execution id

  • function_name (str) – function name

  • replica_id (str) – replica id

  • system (bool) – system

  • view (bool) – if true, print out all the logs

  • until_completed (bool) – wait until completed

Returns

ServiceLog entity

Return type

ServiceLog

Example:

package.services.log(service='service_entity')
name_validation(name: str)[source]

Validation service name.

Prerequisites: You must be in the role of an owner or developer.

Parameters

name (str) – service name

Example:

package.services.name_validation(name='name')
open_in_web(service: Optional[Service] = None, service_id: Optional[str] = None, service_name: Optional[str] = None)[source]

Open the service in web platform

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters

Example:

package.services.open_in_web(service_id='service_id')
pause(service_name: Optional[str] = None, service_id: Optional[str] = None, force: bool = False)[source]

Pause service.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

You must provide at least ONE of the following params: service_id, service_name

Parameters
  • service_name (str) – service name

  • service_id (str) – service id

  • force (bool) – optional - terminate old replicas immediately

Returns

True if success

Return type

bool

Example:

package.services.pause(service_id='service_id')
resume(service_name: Optional[str] = None, service_id: Optional[str] = None, force: bool = False)[source]

Resume service.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

You must provide at least ONE of the following params: service_id, service_name.

Parameters
  • service_name (str) – service name

  • service_id (str) – service id

  • force (bool) – optional - terminate old replicas immediately

Returns

json of the service

Return type

dict

Example:

package.services.resume(service_id='service_id')
revisions(service: Optional[Service] = None, service_id: Optional[str] = None)[source]

Get service revisions history.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

You must provide at leats ONE of the following params: service, service_id

Parameters

Example:

package.services.revisions(service_id='service_id')
status(service_name=None, service_id=None)[source]

Get service status.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

You must provide at least ONE of the following params: service_id, service_name

Parameters
  • service_name (str) – service name

  • service_id (str) – service id

Returns

status json

Return type

dict

Example:

package.services.status(service_id='service_id')
update(service: Service, force: bool = False) Service[source]

Update service changes to platform.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters
Returns

Service entity

Return type

dtlpy.entities.service.Service

Example:

package.services.update(service='service_entity')

Bots

class Bots(client_api: ApiClient, project: Project)[source]

Bases: object

Bots Repository

The Bots class allows the user to manage bots and their properties. See our documentation for more information on bots.

create(name: str, return_credentials: bool = False)[source]

Create a new Bot.

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters
  • name (str) – bot name

  • return_credentials (str) – True will return the password when created

Returns

Bot object

Return type

dtlpy.entities.bot.Bot

Example:

service.bots.delete(name='bot', return_credentials=False)
delete(bot_id: Optional[str] = None, bot_email: Optional[str] = None)[source]

Delete a Bot.

Prerequisites: You must be in the role of an owner or developer. You must have a service.

You must provide at least ONE of the following params: bot_id, bot_email

Parameters
  • bot_id (str) – bot id to delete

  • bot_email (str) – bot email to delete

Returns

True if successful

Return type

bool

Example:

service.bots.delete(bot_id='bot_id')
get(bot_email: Optional[str] = None, bot_id: Optional[str] = None, bot_name: Optional[str] = None)[source]

Get a Bot object.

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters
  • bot_email (str) – get bot by email

  • bot_id (str) – get bot by id

  • bot_name (str) – get bot by name

Returns

Bot object

Return type

dtlpy.entities.bot.Bot

Example:

service.bots.get(bot_id='bot_id')
list() List[Bot][source]

Get a project or package bots list.

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Returns

List of Bots objects

Return type

list

Example:

service.bots.list()

Triggers

class Triggers(client_api: ApiClient, project: Optional[Project] = None, service: Optional[Service] = None, project_id: Optional[str] = None, pipeline: Optional[Pipeline] = None)[source]

Bases: object

Triggers Repository

The Triggers class allows users to manage triggers and their properties. Triggers activate services. See our documentation for more information on triggers.

create(service_id: Optional[str] = None, trigger_type: TriggerType = TriggerType.EVENT, name: Optional[str] = None, webhook_id=None, function_name='run', project_id=None, active=True, filters=None, resource: TriggerResource = TriggerResource.ITEM, actions: Optional[TriggerAction] = None, execution_mode: TriggerExecutionMode = TriggerExecutionMode.ONCE, start_at=None, end_at=None, inputs=None, cron=None, pipeline_id=None, pipeline=None, pipeline_node_id=None, root_node_namespace=None, **kwargs) BaseTrigger[source]

Create a Trigger. Can create two types: a cron trigger or an event trigger. Inputs are different for each type

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Inputs for all types:

Parameters
  • service_id (str) – Id of services to be triggered

  • trigger_type (str) – can be cron or event. use enum dl.TriggerType for the full list

  • name (str) – name of the trigger

  • webhook_id (str) – id for webhook to be called

  • function_name (str) – the function name to be called when triggered (must be defined in the package)

  • project_id (str) – project id where trigger will work

  • active (bool) – optional - True/False, default = True, if true trigger is active

Inputs for event trigger: :param dtlpy.entities.filters.Filters filters: optional - Item/Annotation metadata filters, default = none :param str resource: optional - Dataset/Item/Annotation/ItemStatus, default = Item :param str actions: optional - Created/Updated/Deleted, default = create :param str execution_mode: how many times trigger should be activated; default is “Once”. enum dl.TriggerExecutionMode

Inputs for cron trigger: :param start_at: iso format date string to start activating the cron trigger :param end_at: iso format date string to end the cron activation :param inputs: dictionary “name”:”val” of inputs to the function :param str cron: cron spec specifying when it should run. more information: https://en.wikipedia.org/wiki/Cron :param str pipeline_id: Id of pipeline to be triggered :param pipeline: pipeline entity to be triggered :param str pipeline_node_id: Id of pipeline root node to be triggered :param root_node_namespace: namespace of pipeline root node to be triggered

Returns

Trigger entity

Return type

dtlpy.entities.trigger.Trigger

Example:

service.triggers.create(name='triggername',
                      execution_mode=dl.TriggerExecutionMode.ONCE,
                      resource='Item',
                      actions='Created',
                      function_name='run',
                      filters={'$and': [{'hidden': False},
                                        {'type': 'file'}]}
                      )
delete(trigger_id=None, trigger_name=None)[source]

Delete Trigger object

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters
  • trigger_id (str) – trigger id

  • trigger_name (str) – trigger name

Returns

True is successful error if not

Return type

bool

Example:

service.triggers.delete(trigger_id='trigger_id')
get(trigger_id=None, trigger_name=None) BaseTrigger[source]

Get Trigger object

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters
  • trigger_id (str) – trigger id

  • trigger_name (str) – trigger name

Returns

Trigger entity

Return type

dtlpy.entities.trigger.Trigger

Example:

service.triggers.get(trigger_id='trigger_id')
list(filters: Optional[Filters] = None) PagedEntities[source]

List triggers of a project, package, or service.

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters

filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters

Returns

Paged entity

Return type

dtlpy.entities.paged_entities.PagedEntities

Example:

service.triggers.list()
name_validation(name: str)[source]

This method validates the trigger name. If name is not valid, this method will return an error. Otherwise, it will not return anything.

Parameters

name (str) – trigger name

resource_information(resource, resource_type, action='Created')[source]

Returns which function should run on an item (based on global triggers).

Prerequisites: You must be a superuser to run this method.

Parameters
  • resource – ‘Item’ / ‘Dataset’ / etc

  • resource_type – dictionary of the resource object

  • action – ‘Created’ / ‘Updated’ / etc.

Example:

service.triggers.resource_information(resource='Item', resource_type=item_object, action='Created')
update(trigger: BaseTrigger) BaseTrigger[source]

Update trigger

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters

trigger (dtlpy.entities.trigger.Trigger) – Trigger entity

Returns

Trigger entity

Return type

dtlpy.entities.trigger.Trigger

Example:

service.triggers.update(trigger='trigger_entity')

Executions

class Executions(client_api: ApiClient, service: Optional[Service] = None, project: Optional[Project] = None)[source]

Bases: object

Service Executions Repository

The Executions class allows the users to manage executions (executions of services) and their properties. See our documentation for more information about executions.

create(service_id: Optional[str] = None, execution_input: Optional[list] = None, function_name: Optional[str] = None, resource: Optional[PackageInputType] = None, item_id: Optional[str] = None, dataset_id: Optional[str] = None, annotation_id: Optional[str] = None, project_id: Optional[str] = None, sync: bool = False, stream_logs: bool = False, return_output: bool = False, return_curl_only: bool = False, timeout: Optional[int] = None) Execution[source]

Execute a function on an existing service

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters
  • service_id (str) – service id to execute on

  • execution_input (List[FunctionIO] or dict) – input dictionary or list of FunctionIO entities

  • function_name (str) – function name to run

  • resource (str) – input type.

  • item_id (str) – optional - item id as input to function

  • dataset_id (str) – optional - dataset id as input to function

  • annotation_id (str) – optional - annotation id as input to function

  • project_id (str) – resource’s project

  • sync (bool) – if true, wait for function to end

  • stream_logs (bool) – prints logs of the new execution. only works with sync=True

  • return_output (bool) – if True and sync is True - will return the output directly

  • return_curl_only (bool) – return the cURL of the creation WITHOUT actually do it

  • timeout (int) – int, seconds to wait until TimeoutError is raised. if <=0 - wait until done - by default wait take the service timeout

Returns

execution object

Return type

dtlpy.entities.execution.Execution

Example:

service.executions.create(function_name='function_name', item_id='item_id', project_id='project_id')
get(execution_id: Optional[str] = None, sync: bool = False) Execution[source]

Get Service execution object

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters
  • execution_id (str) – execution id

  • sync (bool) – if true, wait for the execution to finish

Returns

Service execution object

Return type

dtlpy.entities.execution.Execution

Example:

service.executions.get(execution_id='execution_id')
increment(execution: Execution)[source]

Increment the number of attempts that an execution is allowed to attempt to run a service that is not responding.

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters

execution (dtlpy.entities.execution.Execution) –

Returns

int

Return type

int

Example:

service.executions.increment(execution='execution_entity')
list(filters: Optional[Filters] = None) PagedEntities[source]

List service executions

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters

filters (dtlpy.entities.filters.Filters) – dl.Filters entity to filters items

Returns

Paged entity

Return type

dtlpy.entities.paged_entities.PagedEntities

Example:

service.executions.list()
logs(execution_id: str, follow: bool = True, until_completed: bool = True)[source]

executions logs

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters
  • execution_id (str) – execution id

  • follow (bool) – if true, keep stream future logs

  • until_completed (bool) – if true, wait until completed

Returns

executions logs

Example:

service.executions.logs(execution_id='execution_id')
progress_update(execution_id: str, status: Optional[ExecutionStatus] = None, percent_complete: Optional[int] = None, message: Optional[str] = None, output: Optional[str] = None, service_version: Optional[str] = None)[source]

Update Execution Progress.

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters
  • execution_id (str) – execution id

  • status (str) – ExecutionStatus

  • percent_complete (int) – percent work done

  • message (str) – message

  • output (str) – the output of the execution

  • service_version (str) – service version

Returns

Service execution object

Return type

dtlpy.entities.execution.Execution

Example:

service.executions.progress_update(execution_id='execution_id', status='complete', percent_complete=100)
rerun(execution: Execution, sync: bool = False)[source]

Rerun execution

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters
Returns

Execution object

Return type

dtlpy.entities.execution.Execution

Example:

service.executions.rerun(execution='execution_entity')
terminate(execution: Execution)[source]

Terminate Execution

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters

execution (dtlpy.entities.execution.Execution) –

Returns

execution object

Return type

dtlpy.entities.execution.Execution

Example:

service.executions.terminate(execution='execution_entity')
update(execution: Execution) Execution[source]

Update execution changes to platform

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters

execution (dtlpy.entities.execution.Execution) – execution entity

Returns

Service execution object

Return type

dtlpy.entities.execution.Execution

Example:

service.executions.update(execution='execution_entity')
wait(execution_id: str, timeout: Optional[int] = None)[source]

Get Service execution object.

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters
  • execution_id (str) – execution id

  • timeout (int) – seconds to wait until TimeoutError is raised. if <=0 - wait until done - by default wait take the service timeout

Returns

Service execution object

Return type

dtlpy.entities.execution.Execution

Example:

service.executions.wait(execution_id='execution_id')

Pipelines

class Pipelines(client_api: ApiClient, project: Optional[Project] = None)[source]

Bases: object

Pipelines Repository

The Pipelines class allows users to manage pipelines and their properties. See our documentation for more information on pipelines.

create(name: Optional[str] = None, project_id: Optional[str] = None, pipeline_json: Optional[dict] = None) Pipeline[source]

Create a new pipeline.

prerequisites: You must be an owner or developer to use this method.

Parameters
  • name (str) – pipeline name

  • project_id (str) – project id

  • pipeline_json (dict) – json containing the pipeline fields

Returns

Pipeline object

Return type

dtlpy.entities.pipeline.Pipeline

Example:

project.pipelines.create(name='pipeline_name')
delete(pipeline: Optional[Pipeline] = None, pipeline_name: Optional[str] = None, pipeline_id: Optional[str] = None)[source]

Delete Pipeline object.

prerequisites: You must be an owner or developer to use this method.

Parameters
Returns

True if success

Return type

bool

Example:

project.pipelines.delete(pipeline_id='pipeline_id')
execute(pipeline: Optional[Pipeline] = None, pipeline_id: Optional[str] = None, pipeline_name: Optional[str] = None, execution_input=None)[source]

Execute a pipeline and return the pipeline execution as an object.

prerequisites: You must be an owner or developer to use this method.

Parameters
  • pipeline (dtlpy.entities.pipeline.Pipeline) – pipeline entity

  • pipeline_id (str) – pipeline id

  • pipeline_name (str) – pipeline name

  • execution_input – list of the dl.FunctionIO or dict of pipeline input - example {‘item’: ‘item_id’}

Returns

entities.PipelineExecution object

Return type

dtlpy.entities.pipeline_execution.PipelineExecution

Example:

project.pipelines.execute(pipeline='pipeline_entity', execution_input= {'item': 'item_id'} )
get(pipeline_name=None, pipeline_id=None, fetch=None) Pipeline[source]

Get Pipeline object to use in your code.

prerequisites: You must be an owner or developer to use this method.

You must provide at least ONE of the following params: pipeline_name, pipeline_id.

Parameters
  • pipeline_id (str) – pipeline id

  • pipeline_name (str) – pipeline name

  • fetch – optional - fetch entity from platform, default taken from cookie

Returns

Pipeline object

Return type

dtlpy.entities.pipeline.Pipeline

Example:

project.pipelines.get(pipeline_id='pipeline_id')
install(pipeline: Optional[Pipeline] = None)[source]

Install (start) a pipeline.

prerequisites: You must be an owner or developer to use this method.

Parameters

pipeline (dtlpy.entities.pipeline.Pipeline) – pipeline entity

Returns

Composition object

Example:

project.pipelines.install(pipeline='pipeline_entity')
list(filters: Optional[Filters] = None, project_id: Optional[str] = None) PagedEntities[source]

List project pipelines.

prerequisites: You must be an owner or developer to use this method.

Parameters
Returns

Paged entity

Return type

dtlpy.entities.paged_entities.PagedEntities

Example:

project.pipelines.get()
open_in_web(pipeline: Optional[Pipeline] = None, pipeline_id: Optional[str] = None, pipeline_name: Optional[str] = None)[source]

Open the pipeline in web platform.

prerequisites: Must be owner or developer to use this method.

Parameters

Example:

project.pipelines.open_in_web(pipeline_id='pipeline_id')
pause(pipeline: Optional[Pipeline] = None)[source]

Pause a pipeline.

prerequisites: You must be an owner or developer to use this method.

Parameters

pipeline (dtlpy.entities.pipeline.Pipeline) – pipeline entity

Returns

Composition object

Example:

project.pipelines.pause(pipeline='pipeline_entity')
reset(pipeline: Optional[Pipeline] = None, pipeline_id: Optional[str] = None, pipeline_name: Optional[str] = None, stop_if_running: bool = False)[source]

Reset pipeline counters.

prerequisites: You must be an owner or developer to use this method.

Parameters
  • pipeline (dtlpy.entities.pipeline.Pipeline) – pipeline entity - optional

  • pipeline_id (str) – pipeline_id - optional

  • pipeline_name (str) – pipeline_name - optional

  • stop_if_running (bool) – If the pipeline is installed it will stop the pipeline and reset the counters.

Returns

bool

Example:

project.pipelines.reset(pipeline='pipeline_entity')
stats(pipeline: Optional[Pipeline] = None, pipeline_id: Optional[str] = None, pipeline_name: Optional[str] = None)[source]

Get pipeline counters.

prerequisites: You must be an owner or developer to use this method.

Parameters
Returns

PipelineStats

Return type

dtlpy.entities.pipeline.PipelineStats

Example:

project.pipelines.stats(pipeline='pipeline_entity')
update(pipeline: Optional[Pipeline] = None) Pipeline[source]

Update pipeline changes to platform.

prerequisites: You must be an owner or developer to use this method.

Parameters

pipeline (dtlpy.entities.pipeline.Pipeline) – pipeline entity

Returns

Pipeline object

Return type

dtlpy.entities.pipeline.Pipeline

Example:

project.pipelines.update(pipeline='pipeline_entity')

Pipeline Executions

class PipelineExecutions(client_api: ApiClient, project: Optional[Project] = None, pipeline: Optional[Pipeline] = None)[source]

Bases: object

PipelineExecutions Repository

The PipelineExecutions class allows users to manage pipeline executions. See our documentation for more information on pipelines.

create(pipeline_id: Optional[str] = None, execution_input=None)[source]

Execute a pipeline and return the execute.

prerequisites: You must be an owner or developer to use this method.

Parameters
  • pipeline_id – pipeline id

  • execution_input – list of the dl.FunctionIO or dict of pipeline input - example {‘item’: ‘item_id’}

Returns

entities.PipelineExecution object

Return type

dtlpy.entities.pipeline_execution.PipelineExecution

Example:

pipeline.pipeline_executions.create(pipeline_id='pipeline_id', execution_input={'item': 'item_id'})
get(pipeline_execution_id: str, pipeline_id: Optional[str] = None) PipelineExecution[source]

Get Pipeline Execution object

prerequisites: You must be an owner or developer to use this method.

Parameters
  • pipeline_execution_id (str) – pipeline execution id

  • pipeline_id (str) – pipeline id

Returns

PipelineExecution object

Return type

dtlpy.entities.pipeline_execution.PipelineExecution

Example:

pipeline.pipeline_executions.get(pipeline_id='pipeline_id')
list(filters: Optional[Filters] = None) PagedEntities[source]

List project pipeline executions.

prerequisites: You must be an owner or developer to use this method.

Parameters

filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters

Returns

Paged entity

Return type

dtlpy.entities.paged_entities.PagedEntities

Example:

pipeline.pipeline_executions.list()

General Commands

class Commands(client_api: ApiClient)[source]

Bases: object

Service Commands repository

abort(command_id: str)[source]

Abort Command

Parameters

command_id (str) – command id

Returns

get(command_id: Optional[str] = None, url: Optional[str] = None) Command[source]

Get Service command object

Parameters
  • command_id (str) –

  • url (str) – command url

Returns

Command object

list()[source]

List of commands

Returns

list of commands

wait(command_id, timeout=0, step=None, url=None, backoff_factor=0.1)[source]

Wait for command to finish

backoff_factor: A backoff factor to apply between attempts after the second try {backoff factor} * (2 ** ({number of total retries} - 1)) seconds. If the backoff_factor is 0.1, then sleep() will sleep for [0.0s, 0.2s, 0.4s, …] between retries. It will never be longer than 8 sec

Parameters
  • command_id (str) – Command id to wait to

  • timeout (int) – int, seconds to wait until TimeoutError is raised. if 0 - wait until done

  • step (int) – int, seconds between polling

  • url (str) – url to the command

  • backoff_factor (float) – A backoff factor to apply between attempts after the second try

Returns

Command object

Download Commands

Upload Commands