Repositories ¶

Example:

dl.organizations.add_member(email='user@domain.com',
                            organization_id='organization_id',
                            role=dl.MemberOrgRole.MEMBER)

cache_action(organization_id: Optional[str] = None, organization_name: Optional[str] = None, organization: Optional[Organization] = None, mode=CacheAction.APPLY, pod_type=PodType.SMALL)[source]¶

Add or remove Cache for the org

Prerequisites: You must be an organization owner

You must provide at least ONE of the following params: organization, organization_name, or organization_id.

Parameters

organization_id (str) – Organization id
organization_name (str) – Organization name
organization (entities.Organization) – Organization object
mode (str) – dl.CacheAction.APPLY or dl.CacheAction.DESTROY
pod_type (entities.PodType) – dl.PodType.SMALL, dl.PodType.MEDIUM, dl.PodType.HIGH

Returns

True if success

Return type

Example:

dl.organizations.enable_cache(organization_id='organization_id',
                              mode=dl.CacheAction.APPLY)

delete_member(user_id: str, organization_id: Optional[str] = None, organization_name: Optional[str] = None, organization: Optional[Organization] = None, sure: bool = False, really: bool = False) → bool[source]¶

Delete member from the Organization.

Prerequisites: Must be an organization owner to delete members.

You must provide at least ONE of the following params: organization_id, organization_name, organization.

Parameters

user_id (str) – user id
organization_id (str) – Organization id
organization_name (str) – Organization name
organization (entities.Organization) – Organization object
sure (bool) – Are you sure you want to delete?
really (bool) – Really really sure?

Returns

True if success and error if not

Return type

dtlpy.entities.organization.Organization

Example:

dl.organizations.delete_member(user_id='user_id',
                                organization_id='organization_id',
                                sure=True,
                                really=True)

get(organization_id: Optional[str] = None, organization_name: Optional[str] = None, fetch: Optional[bool] = None) → Organization[source]¶

Get Organization object to be able to use it in your code.

Prerequisites: You must be a superuser to use this method.

You must provide at least ONE of the following params: organization_name or organization_id.

Parameters

organization_id (str) – optional - search by id
organization_name (str) – optional - search by name
fetch – optional - fetch entity from platform, default taken from cookie

Returns

Organization object

Return type

Example:

dl.organizations.get(organization_id='organization_id')

list() → List[Organization][source]¶

Lists all the organizations in Dataloop.

Prerequisites: You must be a superuser to use this method.

Returns: List of Organization objects
Return type: list

Example:

dl.organizations.list()

list_groups(organization: Optional[Organization] = None, organization_id: Optional[str] = None, organization_name: Optional[str] = None)[source]¶

List all organization groups (groups that were created within the organization).

Prerequisites: You must be an organization owner to use this method.

You must provide at least ONE of the following params: organization, organization_name, or organization_id.

Parameters

organization (entities.Organization) – Organization object
organization_id (str) – Organization id
organization_name (str) – Organization name

Returns

groups list

Return type

Example:

dl.organizations.list_groups(organization_id='organization_id')

list_integrations(organization: Optional[Organization] = None, organization_id: Optional[str] = None, organization_name: Optional[str] = None, only_available=False)[source]¶

List all organization integrations with external cloud storage.

Prerequisites: You must be an organization owner to use this method.

You must provide at least ONE of the following params: organization_id, organization_name, or organization.

Parameters

organization (entities.Organization) – Organization object
organization_id (str) – Organization id
organization_name (str) – Organization name
only_available (bool) – if True list only the available integrations

Returns

integrations list

Return type

Example:

dl.organizations.list_integrations(organization='organization-entity',
                                    only_available=True)

list_members(organization: Optional[Organization] = None, organization_id: Optional[str] = None, organization_name: Optional[str] = None, role: Optional[MemberOrgRole] = None)[source]¶

List all organization members.

Prerequisites: You must be an organization owner to use this method.

You must provide at least ONE of the following params: organization_id, organization_name, or organization.

Parameters

organization (entities.Organization) – Organization object
organization_id (str) – Organization id
organization_name (str) – Organization name
role (entities.MemberOrgRole) – MemberOrgRole.ADMIN, MemberOrgRole.OWNER, MemberOrgRole.MEMBER

Returns

projects list

Return type

dtlpy.entities.organization.Organization

Example:

dl.organizations.list_members(organization='organization-entity',
                            role=dl.MemberOrgRole.MEMBER)

update(plan: str, organization: Optional[Organization] = None, organization_id: Optional[str] = None, organization_name: Optional[str] = None) → Organization[source]¶

Update an organization.

Prerequisites: You must be a superuser to update an organization.

You must provide at least ONE of the following params: organization, organization_name, or organization_id.

Parameters

plan (str) – OrganizationsPlans.FREEMIUM, OrganizationsPlans.PREMIUM
organization (entities.Organization) – Organization object
organization_id (str) – Organization id
organization_name (str) – Organization name

Returns

organization object

Return type

Example:

dl.organizations.update(organization='organization-entity',
                        plan=dl.OrganizationsPlans.FREEMIUM)

update_member(email: str, role: MemberOrgRole = MemberOrgRole.MEMBER, organization_id: Optional[str] = None, organization_name: Optional[str] = None, organization: Optional[Organization] = None)[source]¶

Update member role.

Prerequisites: You must be an organization owner to update a member’s role.

You must provide at least ONE of the following params: organization, organization_name, or organization_id.

Parameters

email (str) – the member’s email
role (str) – MemberOrgRole.ADMIN, MemberOrgRole.OWNER, MemberOrgRole.MEMBER
organization_id (str) – Organization id
organization_name (str) – Organization name
organization (entities.Organization) – Organization object

Returns

json of the member fields

Return type

Example:

dl.organizations.update_member(email='user@domain.com',
                                organization_id='organization_id',
                                 role=dl.MemberOrgRole.MEMBER)

Integrations¶

Integrations Repository

class Integrations(client_api: ApiClient, org: Optional[Organization] = None, project: Optional[Project] = None)[source]¶

Bases: object

Integrations Repository

The Integrations class allows you to manage data integrtion from your external storage (e.g., S3, GCS, Azure) into your Dataloop’s Dataset storage, as well as sync data in your Dataloop’s Datasets with data in your external storage.

For more information on Organization Storgae Integration see the Dataloop documentation and SDK External Storage.

create(integrations_type: ExternalStorage, name: str, options: dict)[source]¶

Create an integration between an external storage and the organization.

Examples for options include: s3 - {key: “”, secret: “”}; gcs - {key: “”, secret: “”, content: “”}; azureblob - {key: “”, secret: “”, clientId: “”, tenantId: “”}; key_value - {key: “”, value: “”} aws-sts - {key: “”, secret: “”, roleArns: “”}

Prerequisites: You must be an owner in the organization.

Parameters

integrations_type (str) – integrations type dl.ExternalStorage
name (str) – integrations name
options (dict) – dict of storage secrets

Returns

success

Return type

Example:

project.integrations.create(integrations_type=dl.ExternalStorage.S3,
                name='S3ntegration',
                options={key: "Access key ID", secret: "Secret access key"})

delete(integrations_id: str, sure: bool = False, really: bool = False) → bool[source]¶

Delete integrations from the organization.

Prerequisites: You must be an organization owner to delete an integration.

Parameters

integrations_id (str) – integrations id
sure (bool) – Are you sure you want to delete?
really (bool) – Really really sure?

Returns

success

Return type

dtlpy.entities.integration.Integration

Example:

project.integrations.delete(integrations_id='integrations_id', sure=True, really=True)

get(integrations_id: str)[source]¶

Get organization integrations. Use this method to access your integration and be able to use it in your code.

Prerequisites: You must be an owner in the organization.

Parameters: integrations_id (str) – integrations id
Returns: Integration object
Return type: dtlpy.entities.integration.Integration

Example:

project.integrations.get(integrations_id='integrations_id')

list(only_available=False)[source]¶

List all the organization’s integrations with external storage.

Prerequisites: You must be an owner in the organization.

Parameters: only_available (bool) – if True list only the available integrations.
Returns: groups list
Return type: list

Example:

project.integrations.list(only_available=True)

update(new_name: str, integrations_id: str)[source]¶

Update the integration’s name.

Prerequisites: You must be an owner in the organization.

Parameters

new_name (str) – new name
integrations_id (str) – integrations id

Returns

Integration object

Return type

Example:

project.integrations.update(integrations_id='integrations_id', new_name="new_integration_name")

Projects¶

class Projects(client_api: ApiClient, org=None)[source]¶

Bases: object

Projects Repository

The Projects class allows the user to manage projects and their properties.

For more information on Projects see the Dataloop documentation and SDK documentation.

add_member(email: str, project_id: str, role: MemberRole = MemberRole.DEVELOPER)[source]¶

Add a member to the project.

Prerequisites: You must be in the role of an owner to add a member to a project.

Parameters

email (str) – member email
project_id (str) – The Id of the project
role – The required role for the user. Use the enum dl.MemberRole

Returns

dict that represent the user

Return type

dtlpy.entities.project.Project

Example:

dl.projects.add_member(project_id='project_id', email='user@dataloop.ai', role=dl.MemberRole.DEVELOPER)

checkout(identifier: Optional[str] = None, project_name: Optional[str] = None, project_id: Optional[str] = None, project: Optional[Project] = None)[source]¶

Checkout (switch) to a project to work on.

Prerequisites: All users can open a project in the web.

You must provide at least ONE of the following params: project_id, project_name.

Parameters

identifier (str) – project name or partial id that you wish to switch
project_name (str) – The Name of the project
project_id (str) – The Id of the project
project (dtlpy.entities.project.Project) – project object

Example:

dl.projects.checkout(project_id='project_id')

create(project_name: str, checkout: bool = False) → Project[source]¶

Create a new project.

Prerequisites: Any user can create a project.

Parameters

project_name (str) – The Name of the project
checkout (bool) – set the project as a default project object (cookies)

Returns

Project object

Return type

Example:

dl.projects.create(project_name='project_name')

delete(project_name: Optional[str] = None, project_id: Optional[str] = None, sure: bool = False, really: bool = False) → bool[source]¶

Delete a project forever!

Prerequisites: You must be in the role of an owner to delete a project.

Parameters

project_name (str) – optional - search by name
project_id (str) – optional - search by id
sure (bool) – Are you sure you want to delete?
really (bool) – Really really sure?

Returns

True if success, error if not

Return type

dtlpy.entities.project.Project

Example:

dl.projects.delete(project_id='project_id', sure=True, really=True)

get(project_name: Optional[str] = None, project_id: Optional[str] = None, checkout: bool = False, fetch: Optional[bool] = None, log_error=True) → Project[source]¶

Get a Project object.

Prerequisites: You must be in the role of an owner to get a project object.

You must check out to a project or provide at least one of the following params: project_id, project_name

Parameters

project_name (str) – optional - search by name
project_id (str) – optional - search by id
checkout (bool) – set the project as a default project object (cookies)
fetch (bool) – optional - fetch entity from platform (True), default taken from cookie
log_error (bool) – optional - show the logs errors

Returns

Project object

Return type

Example:

dl.projects.get(project_id='project_id')

list() → List[Project][source]¶

Get the user’s project list

Prerequisites: You must be a superuser to list all users’ projects.

Returns: List of Project objects

Example:

dl.projects.list()

list_members(project: Project, role: Optional[MemberRole] = None)[source]¶

Get a list of the project members.

Prerequisites: You must be in the role of an owner to list project members.

Parameters

project (dtlpy.entities.project.Project) – Project object
role – The required role for the user. Use the enum dl.MemberRole

Returns

list of the project members

Return type

Example:

dl.projects.list_members(project_id='project_id', role=dl.MemberRole.DEVELOPER)

open_in_web(project_name: Optional[str] = None, project_id: Optional[str] = None, project: Optional[Project] = None)[source]¶

Open the project in our web platform.

Prerequisites: All users can open a project in the web.

Parameters

project_name (str) – The Name of the project
project_id (str) – The Id of the project
project (dtlpy.entities.project.Project) – project object

Example:

dl.projects.open_in_web(project_id='project_id')

remove_member(email: str, project_id: str)[source]¶

Remove a member from the project.

Prerequisites: You must be in the role of an owner to delete a member from a project.

Parameters

email (str) – member email
project_id (str) – The Id of the project

Returns

dict that represents the user

Return type

dtlpy.entities.project.Project

Example:

dl.projects.remove_member(project_id='project_id', email='user@dataloop.ai')

update(project: Project, system_metadata: bool = False) → Project[source]¶

Update a project information (e.g., name, member roles, etc.).

Prerequisites: You must be in the role of an owner to add a member to a project.

Parameters

project (dtlpy.entities.project.Project) – project object
system_metadata (bool) – optional - True, if you want to change metadata system

Returns

Project object

Return type

Example:

dl.projects.delete(project='project_entity')

update_member(email: str, project_id: str, role: MemberRole = MemberRole.DEVELOPER)[source]¶

Update member’s information/details in the project.

Prerequisites: You must be in the role of an owner to update a member.

Parameters

email (str) – member email
project_id (str) – The Id of the project
role – The required role for the user. Use the enum dl.MemberRole

Returns

dict that represent the user

Return type

Example:

dl.projects.update_member(project_id='project_id', email='user@dataloop.ai', role=dl.MemberRole.DEVELOPER)

Datasets¶

Datasets Repository

class Datasets(client_api: ApiClient, project: Optional[Project] = None)[source]¶

Bases: object

Datasets Repository

The Datasets class allows the user to manage datasets. Read more about datasets in our documentation and SDK documentation.

checkout(identifier: Optional[str] = None, dataset_name: Optional[str] = None, dataset_id: Optional[str] = None, dataset: Optional[Dataset] = None)[source]¶

Checkout (switch) to a dataset to work on it.

Prerequisites: You must be an owner or developer to use this method.

You must provide at least ONE of the following params: dataset_id, dataset_name.

Parameters

identifier (str) – project name or partial id that you wish to switch
dataset_name (str) – The Name of the dataset
dataset_id (str) – The Id of the dataset
dataset (dtlpy.entities.dataset.Dataset) – dataset object

Example:

project.datasets.checkout(dataset_id='dataset_id')

clone(dataset_id: str, clone_name: str, filters: Optional[Filters] = None, with_items_annotations: bool = True, with_metadata: bool = True, with_task_annotations_status: bool = True)[source]¶

Clone a dataset. Read more about cloning datatsets and items in our documentation and SDK documentation.

Prerequisites: You must be in the role of an owner or developer.

Parameters

dataset_id (str) – id of the dataset you wish to clone
clone_name (str) – new dataset name
filters (dtlpy.entities.filters.Filters) – Filters entity or a query dict
with_items_annotations (bool) – true to clone with items annotations
with_metadata (bool) – true to clone with metadata
with_task_annotations_status (bool) – true to clone with task annotations’ status

Returns

dataset object

Return type

Example:

project.datasets.clone(dataset_id='dataset_id',
                      clone_name='dataset_clone_name',
                      with_metadata=True,
                      with_items_annotations=False,
                      with_task_annotations_status=False)

create(dataset_name: str, labels=None, attributes=None, ontology_ids=None, driver: Optional[Driver] = None, driver_id: Optional[str] = None, checkout: bool = False, expiration_options: Optional[ExpirationOptions] = None, index_driver: Optional[IndexDriver] = None, recipe_id: Optional[str] = None) → Dataset[source]¶

Create a new dataset

Prerequisites: You must be in the role of an owner or developer.

Parameters

dataset_name (str) – The Name of the dataset
labels (list) – dictionary of {tag: color} or list of label entities
attributes (list) – dataset’s ontology’s attributes
ontology_ids (list) – optional - dataset ontology
driver (dtlpy.entities.driver.Driver) – optional - storage driver Driver object or driver name
driver_id (str) – optional - driver id
checkout (bool) – set the dataset as a default dataset object (cookies)
expiration_options (ExpirationOptions) – dl.ExpirationOptions object that contain definitions for dataset like MaxItemDays
index_driver (str) – dl.IndexDriver, dataset driver version
recipe_id (str) – optional - recipe id

Returns

Dataset object

Return type

Example:

project.datasets.create(dataset_name='dataset_name', ontology_ids='ontology_ids')

delete(dataset_name: Optional[str] = None, dataset_id: Optional[str] = None, sure: bool = False, really: bool = False)[source]¶

Delete a dataset forever!

Prerequisites: You must be an owner or developer to use this method.

Example:

project.datasets.delete(dataset_id='dataset_id', sure=True, really=True)

Parameters

dataset_name (str) – optional - search by name
dataset_id (str) – optional - search by id
sure (bool) – Are you sure you want to delete?
really (bool) – Really really sure?

Returns

True is success

Return type

directory_tree(dataset: Optional[Dataset] = None, dataset_name: Optional[str] = None, dataset_id: Optional[str] = None)[source]¶

Get dataset’s directory tree.

Prerequisites: You must be an owner or developer to use this method.

You must provide at least ONE of the following params: dataset, dataset_name, dataset_id.

Parameters

dataset (dtlpy.entities.dataset.Dataset) – dataset object
dataset_name (str) – The Name of the dataset
dataset_id (str) – The Id of the dataset

Returns

DirectoryTree

Example:

project.datasets.directory_tree(dataset='dataset_entity')

static download_annotations(dataset: Dataset, local_path: Optional[str] = None, filters: Optional[Filters] = None, annotation_options: Optional[ViewAnnotationOptions] = None, annotation_filters: Optional[Filters] = None, overwrite: bool = False, thickness: int = 1, with_text: bool = False, remote_path: Optional[str] = None, include_annotations_in_output: bool = True, export_png_files: bool = False, filter_output_annotations: bool = False, alpha: Optional[float] = None, export_version=ExportVersion.V1) → str[source]¶

Download dataset’s annotations by filters.

You may filter the dataset both for items and for annotations and download annotations.

Optional – download annotations as: mask, instance, image mask of the item.

Prerequisites: You must be in the role of an owner or developer.

Parameters

dataset (dtlpy.entities.dataset.Dataset) – dataset object
local_path (str) – local folder or filename to save to.
filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
annotation_options (list) – type of download annotations: list(dl.ViewAnnotationOptions)
annotation_filters (dtlpy.entities.filters.Filters) – Filters entity to filter annotations for download
overwrite (bool) – optional - default = False to overwrite the existing files
thickness (int) – optional - line thickness, if -1 annotation will be filled, default =1
with_text (bool) – optional - add text to annotations, default = False
remote_path (str) – DEPRECATED and ignored
include_annotations_in_output (bool) – default - False , if export should contain annotations
export_png_files (bool) – default - if True, semantic annotations should be exported as png files
filter_output_annotations (bool) – default - False, given an export by filter - determine if to filter out annotations
alpha (float) – opacity value [0 1], default 1
export_version (str) – exported items will have original extension in filename, V1 - no original extension in filenames

Returns

local_path of the directory where all the downloaded item

Return type

Example:

project.datasets.download_annotations(dataset='dataset_entity',
                                     local_path='local_path',
                                     annotation_options=dl.ViewAnnotationOptions,
                                     overwrite=False,
                                     thickness=1,
                                     with_text=False,
                                     alpha=1
                                     )

get(dataset_name: Optional[str] = None, dataset_id: Optional[str] = None, checkout: bool = False, fetch: Optional[bool] = None) → Dataset[source]¶

Get dataset by name or id.

Prerequisites: You must be an owner or developer to use this method.

You must provide at least ONE of the following params: dataset_id, dataset_name.

Parameters

dataset_name (str) – optional - search by name
dataset_id (str) – optional - search by id
checkout (bool) – set the dataset as a default dataset object (cookies)
fetch (bool) – optional - fetch entity from platform (True), default taken from cookie

Returns

Dataset object

Return type

Example:

project.datasets.get(dataset_id='dataset_id')

list(name=None, creator=None) → List[Dataset][source]¶

List all datasets.

Prerequisites: You must be an owner or developer to use this method.

Parameters

name (str) – list by name
creator (str) – list by creator

Returns

List of datasets

Return type

Example:

project.datasets.list(name='name')

merge(merge_name: str, dataset_ids: list, project_ids: str, with_items_annotations: bool = True, with_metadata: bool = True, with_task_annotations_status: bool = True, wait: bool = True)[source]¶

Merge a dataset. See our SDK docs for more information.

Prerequisites: You must be an owner or developer to use this method.

Parameters

merge_name (str) – new dataset name
dataset_ids (list) – list id’s of the datatsets you wish to merge
project_ids (str) – the project id that include the datasets
with_items_annotations (bool) – true to merge with items annotations
with_metadata (bool) – true to merge with metadata
with_task_annotations_status (bool) – true to merge with task annotations’ status
wait (bool) – wait for the command to finish

Returns

True if success

Return type

Example:

project.datasets.merge(dataset_ids=['dataset_id1','dataset_id2'],
                      merge_name='dataset_merge_name',
                      with_metadata=True,
                      with_items_annotations=False,
                      with_task_annotations_status=False)

open_in_web(dataset_name: Optional[str] = None, dataset_id: Optional[str] = None, dataset: Optional[Dataset] = None)[source]¶

Open the dataset in web platform.

Prerequisites: You must be an owner or developer to use this method.

Parameters

dataset_name (str) – The Name of the dataset
dataset_id (str) – The Id of the dataset
dataset (dtlpy.entities.dataset.Dataset) – dataset object

Example:

project.datasets.open_in_web(dataset_id='dataset_id')

set_readonly(state: bool, dataset: Dataset)[source]¶

Set dataset readonly mode.

Prerequisites: You must be in the role of an owner or developer.

Parameters

state (bool) – state to update readonly mode
dataset (dtlpy.entities.dataset.Dataset) – dataset object

Example:

project.datasets.set_readonly(dataset='dataset_entity', state=True)

sync(dataset_id: str, wait: bool = True)[source]¶

Sync dataset with external storage.

Prerequisites: You must be in the role of an owner or developer.

Parameters

dataset_id (str) – The Id of the dataset to sync
wait (bool) – wait for the command to finish

Returns

True if success

Return type

Example:

project.datasets.sync(dataset_id='dataset_id')

update(dataset: Dataset, system_metadata: bool = False, patch: Optional[dict] = None) → Dataset[source]¶

Update dataset field.

Prerequisites: You must be an owner or developer to use this method.

Parameters

dataset (dtlpy.entities.dataset.Dataset) – dataset object
system_metadata (bool) – True, if you want to change metadata system
patch (dict) – Specific patch request

Returns

Dataset object

Return type

dtlpy.entities.driver.Driver

Example:

project.datasets.update(dataset='dataset_entity')

upload_annotations(dataset, local_path, filters: Optional[Filters] = None, clean=False, remote_root_path='/', export_version=ExportVersion.V1)[source]¶

Upload annotations to dataset.

Example for remote_root_path: If the item filepath is a/b/item and remote_root_path is /a the start folder will be b instead of a

Prerequisites: You must have a dataset with items that are related to the annotations. The relationship between the dataset and annotations is shown in the name. You must be in the role of an owner or developer.

Parameters

dataset (dtlpy.entities.dataset.Dataset) – dataset to upload to
local_path (str) – str - local folder where the annotations files is
filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
clean (bool) – True to remove the old annotations
remote_root_path (str) – the remote root path to match remote and local items
export_version (str) – exported items will have original extension in filename, V1 - no original extension in filenames

Example:

project.datasets.upload_annotations(dataset='dataset_entity',
                                     local_path='local_path',
                                     clean=False,
                                     export_version=dl.ExportVersion.V1
                                     )

Drivers¶

class Drivers(client_api: ApiClient, project: Optional[Project] = None)[source]¶

Bases: object

Drivers Repository

The Drivers class allows users to manage drivers that are used to connect with external storage. Read more about external storage in our documentation and SDK documentation.

create(name: str, driver_type: ExternalStorage, integration_id: str, bucket_name: str, integration_type: ExternalStorage, project_id: Optional[str] = None, allow_external_delete: bool = True, region: Optional[str] = None, storage_class: str = '', path: str = '')[source]¶

Create a storage driver.

Prerequisites: You must be in the role of an owner or developer.

Parameters

name (str) – the driver name
driver_type (str) – ExternalStorage.S3, ExternalStorage.GCS, ExternalStorage.AZUREBLOB
integration_id (str) – the integration id
bucket_name (str) – the external bucket name
integration_type (str) – ExternalStorage.S3, ExternalStorage.GCS, ExternalStorage.AZUREBLOB, ExternalStorage.AWS_STS
project_id (str) – project id
allow_external_delete (bool) – true to allow deleting files from external storage when files are deleted in your Dataloop storage
region (str) – relevant only for s3 - the bucket region
storage_class (str) – rilevante only for s3
path (str) – Optional. By default path is the root folder. Path is case sensitive integration

Returns

driver object

Return type

Example:

project.drivers.create(name='driver_name',
           driver_type=dl.ExternalStorage.S3,
           integration_id='integration_id',
           bucket_name='bucket_name',
           project_id='project_id',
           region='ey-west-1')

delete(driver_name: Optional[str] = None, driver_id: Optional[str] = None, sure: bool = False, really: bool = False)[source]¶

Delete a driver forever!

Prerequisites: You must be an owner or developer to use this method.

Example:

project.drivers.delete(dataset_id='dataset_id', sure=True, really=True)

Parameters

driver_name (str) – optional - search by name
driver_id (str) – optional - search by id
sure (bool) – Are you sure you want to delete?
really (bool) – Really really sure?

Returns

True if success

Return type

dtlpy.entities.driver.Driver

get(driver_name: Optional[str] = None, driver_id: Optional[str] = None) → Driver[source]¶

Get a Driver object to use in your code.

Prerequisites: You must be in the role of an owner or developer.

You must provide at least ONE of the following params: driver_name, driver_id.

Parameters

driver_name (str) – optional - search by name
driver_id (str) – optional - search by id

Returns

Driver object

Return type

Example:

project.drivers.get(driver_id='driver_id')

list() → List[Driver][source]¶

Get the project’s drivers list.

Prerequisites: You must be in the role of an owner or developer.

Returns: List of Drivers objects
Return type: list

Example:

project.drivers.list()

Items¶

class Items(client_api: ApiClient, datasets: Optional[Datasets] = None, dataset: Optional[Dataset] = None, dataset_id=None, items_entity=None, project=None)[source]¶

Bases: object

Items Repository

The Items class allows you to manage items in your datasets. For information on actions related to items see Organizing Your Dataset, Item Metadata, and Item Metadata-Based Filtering.

clone(item_id: str, dst_dataset_id: str, remote_filepath: Optional[str] = None, metadata: Optional[dict] = None, with_annotations: bool = True, with_metadata: bool = True, with_task_annotations_status: bool = False, allow_many: bool = False, wait: bool = True)[source]¶

Clone item. Read more about cloning datatsets and items in our documentation and SDK documentation.

Prerequisites: You must be in the role of an owner or developer.

Parameters

item_id (str) – item to clone
dst_dataset_id (str) – destination dataset id
remote_filepath (str) – complete filepath
metadata (dict) – new metadata to add
with_annotations (bool) – clone annotations
with_metadata (bool) – clone metadata
with_task_annotations_status (bool) – clone task annotations status
allow_many (bool) – bool if True, using multiple clones in single dataset is allowed, (default=False)
wait (bool) – wait for the command to finish

Returns

Item object

Return type

Example:

dataset.items.clone(item_id='item_id',
        dst_dataset_id='dist_dataset_id',
        with_metadata=True,
        with_task_annotations_status=False,
        with_annotations=False)

delete(filename: Optional[str] = None, item_id: Optional[str] = None, filters: Optional[Filters] = None)[source]¶

Delete item from platform.

Prerequisites: You must be in the role of an owner or developer.

You must provide at least ONE of the following params: item id, filename, filters.

Parameters

filename (str) – optional - search item by remote path
item_id (str) – optional - search item by id
filters (dtlpy.entities.filters.Filters) – optional - delete items by filter

Returns

True if success

Return type

Example:

dataset.items.delete(item_id='item_id')

download(filters: Optional[Filters] = None, items=None, local_path: Optional[str] = None, file_types: Optional[list] = None, save_locally: bool = True, to_array: bool = False, annotation_options: Optional[ViewAnnotationOptions] = None, annotation_filters: Optional[Filters] = None, overwrite: bool = False, to_items_folder: bool = True, thickness: int = 1, with_text: bool = False, without_relative_path=None, avoid_unnecessary_annotation_download: bool = False, include_annotations_in_output: bool = True, export_png_files: bool = False, filter_output_annotations: bool = False, alpha: float = 1, export_version=ExportVersion.V1)[source]¶

Download dataset items by filters.

Filters the dataset for items and saves them locally.

Optional – download annotation, mask, instance, and image mask of the item.

Prerequisites: You must be in the role of an owner or developer.

Parameters

filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
items (List[dtlpy.entities.item.Item] or dtlpy.entities.item.Item) – download Item entity or item_id (or a list of item)
local_path (str) – local folder or filename to save to.
file_types (list) – a list of file type to download. e.g [‘video/webm’, ‘video/mp4’, ‘image/jpeg’, ‘image/png’]
save_locally (bool) – bool. save to disk or return a buffer
to_array (bool) – returns Ndarray when True and local_path = False
annotation_options (list) – download annotations options: list(dl.ViewAnnotationOptions)
annotation_filters (dtlpy.entities.filters.Filters) – Filters entity to filter annotations for download
overwrite (bool) – optional - default = False
to_items_folder (bool) – Create ‘items’ folder and download items to it
thickness (int) – optional - line thickness, if -1 annotation will be filled, default =1
with_text (bool) – optional - add text to annotations, default = False
without_relative_path (bool) – bool - download items without the relative path from platform
avoid_unnecessary_annotation_download (bool) – default - False
include_annotations_in_output (bool) – default - False , if export should contain annotations
export_png_files (bool) – default - if True, semantic annotations should be exported as png files
filter_output_annotations (bool) – default - False, given an export by filter - determine if to filter out annotations
alpha (float) – opacity value [0 1], default 1
export_version (str) – exported items will have original extension in filename, V1 - no original extension in filenames

Returns

generator of local_path per each downloaded item

Return type

generator or single item

Example:

dataset.items.download(local_path='local_path',
                     annotation_options=dl.ViewAnnotationOptions,
                     overwrite=False,
                     thickness=1,
                     with_text=False,
                     alpha=1,
                     save_locally=True
                     )

get(filepath: Optional[str] = None, item_id: Optional[str] = None, fetch: Optional[bool] = None, is_dir: bool = False) → Item[source]¶

Get Item object

Prerequisites: You must be in the role of an owner or developer.

Parameters

filepath (str) – optional - search by remote path
item_id (str) – optional - search by id
fetch (bool) – optional - fetch entity from platform, default taken from cookie
is_dir (bool) – True if you want to get an item from dir type

Returns

Item object

Return type

Example:

dataset.items.get(item_id='item_id')

get_all_items(filters: Optional[Filters] = None) → [<class 'dtlpy.entities.item.Item'>][source]¶

Get all items in dataset.

Prerequisites: You must be in the role of an owner or developer.

Parameters: filters (dtlpy.entities.filters.Filters) – dl.Filters entity to filters items
Returns: list of all items
Return type: list

Example:

dataset.items.get_all_items()

list(filters: Optional[Filters] = None, page_offset: Optional[int] = None, page_size: Optional[int] = None) → PagedEntities[source]¶

List items in a dataset.

Prerequisites: You must be in the role of an owner or developer.

Parameters

filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
page_offset (int) – start page
page_size (int) – page size

Returns

Pages object

Return type

Example:

dataset.items.list(page_offset=0, page_size=100)

make_dir(directory, dataset: Optional[Dataset] = None) → Item[source]¶

Create a directory in a dataset.

Prerequisites: All users.

Parameters

directory (str) – name of directory
dataset (dtlpy.entities.dataset.Dataset) – dataset object

Returns

Item object

Return type

Example:

dataset.items.make_dir(directory='directory_name')

move_items(destination: str, filters: Optional[Filters] = None, items=None, dataset: Optional[Dataset] = None) → bool[source]¶

Move items to another directory. If directory does not exist we will create it

Prerequisites: You must be in the role of an owner or developer.

Parameters

destination (str) – destination directory
filters (dtlpy.entities.filters.Filters) – optional - either this or items. Query of items to move
items – optional - either this or filters. A list of items to move
dataset (dtlpy.entities.dataset.Dataset) – dataset object

Returns

True if success

Return type

Example:

dataset.items.move_items(destination='directory_name')

open_in_web(filepath=None, item_id=None, item=None)[source]¶

Open the item in web platform

Prerequisites: You must be in the role of an owner or developer or be an annotation manager/annotator with access to that item through task.

Parameters

filepath (str) – item file path
item_id (str) – item id
item (dtlpy.entities.item.Item) – item entity

Example:

dataset.items.open_in_web(item_id='item_id')

set_items_entity(entity)[source]¶

Set the item entity type to Artifact, Item, or Codebase.

Parameters: entity (entities.Item, entities.Artifact, entities.Codebase) – entity type [entities.Item, entities.Artifact, entities.Codebase]

update(item: Optional[Item] = None, filters: Optional[Filters] = None, update_values=None, system_update_values=None, system_metadata: bool = False)[source]¶

Update item metadata.

Prerequisites: You must be in the role of an owner or developer.

You must provide at least ONE of the following params: update_values, system_update_values.

Parameters

item (dtlpy.entities.item.Item) – Item object
filters (dtlpy.entities.filters.Filters) – optional update filtered items by given filter
update_values – optional field to be updated and new values
system_update_values – values in system metadata to be updated
system_metadata (bool) – True, if you want to update the metadata system

Returns

Item object

Return type

Example:

dataset.items.update(item='item_entity')

update_status(status: ItemStatus, items=None, item_ids=None, filters=None, dataset=None, clear=False)[source]¶

Update item status in task

Prerequisites: You must be in the role of an owner or developer or annotation manager who has been assigned a task with the item.

You must provide at least ONE of the following params: items, item_ids, filters.

Parameters

status (str) – ItemStatus.COMPLETED, ItemStatus.APPROVED, ItemStatus.DISCARDED
items (list) – list of items
item_ids (list) – list of items id
filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
dataset (dtlpy.entities.dataset.Dataset) – dataset object
clear (bool) – to delete status

Example:

dataset.items.update_status(item_ids='item_id', status=dl.ItemStatus.COMPLETED)

upload(local_path: str, local_annotations_path: ~typing.Optional[str] = None, remote_path: str = '/', remote_name: ~typing.Optional[str] = None, file_types: ~typing.Optional[~dtlpy.repositories.items.Items.list] = None, overwrite: bool = False, item_metadata: ~typing.Optional[dict] = None, output_entity=<class 'dtlpy.entities.item.Item'>, no_output: bool = False, export_version: str = ExportVersion.V1, item_description: ~typing.Optional[str] = None)[source]¶

Upload local file to dataset. Local filesystem will remain unchanged. If “*” at the end of local_path (e.g. “/images/*”) items will be uploaded without the head directory.

Prerequisites: Any user can upload items.

Parameters

local_path (str) – list of local file, local folder, BufferIO, numpy.ndarray or url to upload
local_annotations_path (str) – path to dataloop format annotations json files.
remote_path (str) – remote path to save.
remote_name (str) – remote base name to save. when upload numpy.ndarray as local path, remote_name with .jpg or .png ext is mandatory
file_types (list) – list of file type to upload. e.g [‘.jpg’, ‘.png’]. default is all
item_metadata (dict) – metadata dict to upload to item or ExportMetadata option to export metadata from annotation file
overwrite (bool) – optional - default = False
output_entity – output type
no_output (bool) – do not return the items after upload
export_version (str) – exported items will have original extension in filename, V1 - no original extension in filenames
item_description (str) – add a string description to the uploaded item

Returns

Output (generator/single item)

Return type

generator or single item

Example:

dataset.items.upload(local_path='local_path',
                     local_annotations_path='local_annotations_path',
                     overwrite=True,
                     item_metadata={'Hellow': 'Word'}
                     )

Annotations¶

class Annotations(client_api: ApiClient, item=None, dataset=None, dataset_id=None)[source]¶

Bases: object

Annotations Repository

The Annotation class allows you to manage the annotations of data items. For information on annotations explore our documentation at: Classification SDK, Annotation Labels and Attributes, Show Video with Annotations.

builder()[source]¶

Create Annotation collection.

Prerequisites: You must have an item to be annotated. You must have the role of an owner or developer: or be assigned a task that includes that item as an annotation manager or annotator.

Returns: Annotation collection object
Return type: dtlpy.entities.annotation_collection.AnnotationCollection

Example:

item.annotations.builder()

delete(annotation: Optional[Annotation] = None, annotation_id: Optional[str] = None, filters: Optional[Filters] = None) → bool[source]¶

Remove an annotation from item.

Prerequisites: You must have an item that has been annotated. You must have the role of an owner or developer or be assigned a task that includes that item as an annotation manager or annotator.

Parameters

annotation (dtlpy.entities.annotation.Annotation) – Annotation object
annotation_id (str) – The id of the annotation
filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters

Returns

True/False

Return type

Example:

item.annotations.delete(annotation_id='annotation_id')

download(filepath: str, annotation_format: ViewAnnotationOptions = ViewAnnotationOptions.JSON, img_filepath: Optional[str] = None, height: Optional[float] = None, width: Optional[float] = None, thickness: int = 1, with_text: bool = False, alpha: float = 1)[source]¶

Save annotation to file.

Prerequisites: You must have an item that has been annotated. You must have the role of an owner or developer or be assigned a task that includes that item as an annotation manager or annotator.

Parameters

filepath (str) – Target download directory
annotation_format (str) – the format that want to download ,options: list(dl.ViewAnnotationOptions)
img_filepath (str) – img file path - needed for img_mask
height (float) – optional - image height
width (float) – optional - image width
thickness (int) – optional - line thickness, default=1
with_text (bool) – optional - draw annotation with text, default = False
alpha (float) – opacity value [0 1], default 1

Returns

file path to where save the annotations

Return type

Example:

item.annotations.download(
              filepath='file_path',
              annotation_format=dl.ViewAnnotationOptions.MASK,
              img_filepath='img_filepath',
              height=100,
              width=100,
              thickness=1,
              with_text=False,
              alpha=1)

get(annotation_id: str) → Annotation[source]¶

Get a single annotation.

Prerequisites: You must have an item that has been annotated. You must have the role of an owner or developer or be assigned a task that includes that item as an annotation manager or annotator.

Parameters: annotation_id (str) – The id of the annotation
Returns: Annotation object or None
Return type: dtlpy.entities.annotation.Annotation

Example:

item.annotations.get(annotation_id='annotation_id')

list(filters: Optional[Filters] = None, page_offset: Optional[int] = None, page_size: Optional[int] = None)[source]¶

List Annotations of a specific item. You must get the item first and then list the annotations with the desired filters.

Prerequisites: You must have an item that has been annotated. You must have the role of an owner or developer or be assigned a task that includes that item as an annotation manager or annotator.

Parameters

filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
page_offset (int) – starting page
page_size (int) – size of page

Returns

Pages object

Return type

dtlpy.entities.annotation.Annotation

Example:

item.annotations.list(filters=dl.Filters(
                             resource=dl.FiltersResource.ANNOTATION,
                             field='type',
                             values='box'),
          page_size=100,
          page_offset=0)

show(image=None, thickness: int = 1, with_text: bool = False, height: Optional[float] = None, width: Optional[float] = None, annotation_format: ViewAnnotationOptions = ViewAnnotationOptions.MASK, alpha: float = 1)[source]¶

Show annotations. To use this method, you must get the item first and then show the annotations with the desired filters. The method returns an array showing all the annotations.

Prerequisites: You must have an item that has been annotated. You must have the role of an owner or developer or be assigned a task that includes that item as an annotation manager or annotator.

Parameters

image (ndarray) – empty or image to draw on
thickness (int) – optional - line thickness, default=1
with_text (bool) – add label to annotation
height (float) – item height
width (float) – item width
annotation_format (str) – the format that want to show ,options: list(dl.ViewAnnotationOptions)
alpha (float) – opacity value [0 1], default 1

Returns

ndarray of the annotations

Return type

ndarray

Example:

item.annotations.show(image='nd array',
          thickness=1,
          with_text=False,
          height=100,
          width=100,
          annotation_format=dl.ViewAnnotationOptions.MASK,
          alpha=1)

update(annotations, system_metadata=False)[source]¶

Update an existing annotation. For example, you may change the annotation’s label and then use the update method.

Prerequisites: You must have an item that has been annotated. You must have the role of an owner or: developer or be assigned a task that includes that item as an annotation manager or annotator.

Parameters

annotations (dtlpy.entities.annotation.Annotation) – Annotation object
system_metadata (bool) – bool - True, if you want to change metadata system

Returns

True if successful or error if unsuccessful

Return type

bool

Example:

item.annotations.update(annotation='annotation')

update_status(annotation: Optional[Annotation] = None, annotation_id: Optional[str] = None, status: AnnotationStatus = AnnotationStatus.ISSUE) → Annotation[source]¶

Set status on annotation.

Prerequisites: You must have an item that has been annotated. You must have the role of an owner or developer or be assigned a task that includes that item as an annotation manager.

Parameters

annotation (dtlpy.entities.annotation.Annotation) – Annotation object
annotation_id (str) – optional - annotation id to set status
status (str) – can be AnnotationStatus.ISSUE, APPROVED, REVIEW, CLEAR

Returns

Annotation object

Return type

Example:

item.annotations.update_status(annotation_id='annotation_id', status=dl.AnnotationStatus.ISSUE)

upload(annotations) → AnnotationCollection[source]¶

Upload a new annotation/annotations. You must first create the annotation using the annotation builder method.

Prerequisites: Any user can upload annotations.

Parameters: annotations (List[dtlpy.entities.annotation.Annotation] or dtlpy.entities.annotation.Annotation) – list or

single annotation of type Annotation :return: list of annotation objects :rtype: entities.AnnotationCollection

Example:

item.annotations.upload(annotations='builder')

Recipes¶

class Recipes(client_api: ApiClient, dataset: Optional[Dataset] = None, project: Optional[Project] = None, project_id: Optional[str] = None)[source]¶

Bases: object

Recipes Repository

The Recipes class allows you to manage recipes and their properties. For more information on Recipes, see our documentation and SDK documentation.

clone(recipe: Optional[Recipe] = None, recipe_id: Optional[str] = None, shallow: bool = False)[source]¶

Clone recipe.

Prerequisites: You must be in the role of an owner or developer.

Parameters

recipe (dtlpy.entities.recipe.Recipe) – Recipe object
recipe_id (str) – Recipe id
shallow (bool) – If True, link to existing ontology, clones all ontologies that are linked to the recipe as well

Returns

Cloned ontology object

Return type

dtlpy.entities.recipe.Recipe

Example:

dataset.recipes.clone(recipe_id='recipe_id')

create(project_ids=None, ontology_ids=None, labels=None, recipe_name=None, attributes=None, annotation_instruction_file=None) → Recipe[source]¶

Create a new Recipe. Note: If the param ontology_ids is None, an ontology will be created first.

Prerequisites: You must be in the role of an owner or developer.

Parameters

project_ids (str) – project ids
ontology_ids (str or list) – ontology ids
labels – labels
recipe_name (str) – recipe name
attributes – attributes
annotation_instruction_file (str) – file path or url of the recipe instruction

Returns

Recipe entity

Return type

dtlpy.entities.recipe.Recipe

Example:

dataset.recipes.create(recipe_name='My Recipe', labels=labels))

delete(recipe_id: str, force: bool = False)[source]¶

Delete recipe from platform.

Prerequisites: You must be in the role of an owner or developer.

Parameters

recipe_id (str) – recipe id
force (bool) – force delete recipe

Returns

True if success

Return type

dtlpy.entities.recipe.Recipe

Example:

dataset.recipes.delete(recipe_id='recipe_id')

get(recipe_id: str) → Recipe[source]¶

Get a Recipe object to use in your code.

Prerequisites: You must be in the role of an owner or developer.

Parameters: recipe_id (str) – recipe id
Returns: Recipe object
Return type: dtlpy.entities.recipe.Recipe

Example:

dataset.recipes.get(recipe_id='recipe_id')

list(filters: Optional[Filters] = None) → List[Recipe][source]¶

List recipes for a dataset.

Prerequisites: You must be in the role of an owner or developer.

Parameters: filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
Returns: list of all recipes
Retype: list

Example:

dataset.recipes.list()

open_in_web(recipe: Optional[Recipe] = None, recipe_id: Optional[str] = None)[source]¶

Open the recipe in web platform.

Prerequisites: All users.

Parameters

recipe (dtlpy.entities.recipe.Recipe) – recipe entity
recipe_id (str) – recipe id

Example:

dataset.recipes.open_in_web(recipe_id='recipe_id')

update(recipe: Recipe, system_metadata=False) → Recipe[source]¶

Update recipe.

Prerequisites: You must be in the role of an owner or developer.

Parameters

recipe (dtlpy.entities.recipe.Recipe) – Recipe object
system_metadata (bool) – True, if you want to change metadata system

Returns

Recipe object

Return type

Example:

dataset.recipes.update(recipe='recipe_entity')

Ontologies¶

class Ontologies(client_api: ApiClient, recipe: Optional[Recipe] = None, project: Optional[Project] = None, dataset: Optional[Dataset] = None)[source]¶

Bases: object

Ontologies Repository

The Ontologies class allows users to manage ontologies and their properties. Read more about ontology in our SDK docs.

create(labels, title=None, project_ids=None, attributes=None) → Ontology[source]¶

Create a new ontology.

Prerequisites: You must be in the role of an owner or developer.

Parameters

labels – recipe tags
title (str) – ontology title, name
project_ids (list) – recipe project/s
attributes (list) – recipe attributes

Returns

Ontology object

Return type

dtlpy.entities.ontology.Ontology

Example:

recipe.ontologies.create(labels='labels_entity',
                      title='new_ontology',
                      project_ids='project_ids')

delete(ontology_id)[source]¶

Delete Ontology from the platform.

Prerequisites: You must be in the role of an owner or developer.

Parameters: ontology_id – ontology id
Returns: True if success
Return type: bool

Example:

recipe.ontologies.delete(ontology_id='ontology_id')

delete_attributes(ontology_id, keys: list)[source]¶

Delete a bulk of attributes

Parameters

ontology_id (str) – ontology id
keys (list) – Keys of attributes to delete

Returns

True if success

Return type

dtlpy.entities.ontology.Ontology

Example:

ontology.delete_attributes(['1'])

get(ontology_id: str) → Ontology[source]¶

Get Ontology object to use in your code.

Prerequisites: You must be in the role of an owner or developer.

Parameters: ontology_id (str) – ontology id
Returns: Ontology object
Return type: dtlpy.entities.ontology.Ontology

Example:

recipe.ontologies.get(ontology_id='ontology_id')

static labels_to_roots(labels)[source]¶

Converts labels dictionary to a list of platform representation of labels.

Parameters: labels (dict) – labels dict
Returns: platform representation of labels

list(project_ids=None) → List[Ontology][source]¶

List ontologies for recipe

Prerequisites: You must be in the role of an owner or developer.

Parameters: project_ids –
Returns: list of all the ontologies

Example:

recipe.ontologies.list(project_ids='project_ids')

update(ontology: Ontology, system_metadata=False) → Ontology[source]¶

Update the Ontology metadata.

Prerequisites: You must be in the role of an owner or developer.

Parameters

ontology (dtlpy.entities.ontology.Ontology) – Ontology object
system_metadata (bool) – bool - True, if you want to change metadata system

Returns

Ontology object

Return type

Example:

recipe.ontologies.delete(ontology='ontology_entity')

update_attributes(ontology_id: str, title: str, key: str, attribute_type: AttributesTypes, scope: Optional[list] = None, optional: Optional[bool] = None, values: Optional[list] = None, attribute_range: Optional[AttributesRange] = None)[source]¶

ADD a new attribute or update if exist

Parameters

ontology_id (str) – ontology_id
title (str) – attribute title
key (str) – the key of the attribute must br unique
attribute_type (AttributesTypes) – dl.AttributesTypes your attribute type
scope (list) – list of the labels or * for all labels
optional (bool) – optional attribute
values (list) – list of the attribute values ( for checkbox and radio button)
attribute_range (dict or AttributesRange) – dl.AttributesRange object

Returns

true in success

Return type

Example:

ontology.update_attributes(key='1',
                           title='checkbox',
                           attribute_type=dl.AttributesTypes.CHECKBOX,
                           values=[1,2,3])

Tasks¶

class Tasks(client_api: ApiClient, project: Optional[Project] = None, dataset: Optional[Dataset] = None, project_id: Optional[str] = None)[source]¶

Bases: object

Tasks Repository

The Tasks class allows the user to manage tasks and their properties. For more information, read in our SDK documentation about Creating Tasks, Redistributing and Reassigning Tasks, and Task Assignment.

add_items(task: Optional[Task] = None, task_id=None, filters: Optional[Filters] = None, items=None, assignee_ids=None, query=None, workload=None, limit=None, wait=True) → Task[source]¶

Add items to a Task.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned to be owner of the annotation task.

Parameters

task (dtlpy.entities.task.Task) – task object
task_id (str) – the Id of the task
filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
items (list) – list of items (item Ids or objects) to add to the task
assignee_ids (list) – list to assignee who works in the task
query (dict) – query to filter the items for the task
workload (list) – list of WorkloadUnit objects. Customize distribution (percentage) between the task assignees. For example: [dl.WorkloadUnit(annotator@hi.com, 80), dl.WorkloadUnit(annotator2@hi.com, 20)]
limit (int) – the limit items that task can include
wait (bool) – wait until add items will to finish

Returns

task entity

Return type

Example:

dataset.tasks.add_items(task= 'task_entity',
                    items = [items])

create(task_name, due_date=None, assignee_ids=None, workload=None, dataset=None, task_owner=None, task_type='annotation', task_parent_id=None, project_id=None, recipe_id=None, assignments_ids=None, metadata=None, filters=None, items=None, query=None, available_actions=None, wait=True, check_if_exist: Filters = False, limit=None, batch_size=None, max_batch_workload=None, allowed_assignees=None, priority=TaskPriority.MEDIUM) → Task[source]¶

Create a new Task (Annotation or QA).

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned to be owner of the annotation task.

Parameters

task_name (str) – the name of the task
due_date (float) – date by which the task should be finished; for example, due_date=datetime.datetime(day=1, month=1, year=2029).timestamp()
assignee_ids (list) – list the task assignees (contributors) that should be working on the task. Provide a list of users’ emails
workload (List[WorkloadUnit] List[WorkloadUnit]) – list of WorkloadUnit objects. Customize distribution (percentage) between the task assignees. For example: [dl.WorkloadUnit(annotator@hi.com, 80), dl.WorkloadUnit(annotator2@hi.com, 20)]
dataset (entities.Dataset) – dataset object, the dataset that refer to the task
task_owner (str) – task owner. Provide user email
task_type (str) – task type “annotation” or “qa”
task_parent_id (str) – optional if type is qa - parent annotation task id
project_id (str) – the Id of the project where task will be created
recipe_id (str) – recipe id for the task
assignments_ids (list) – assignments ids to the task
metadata (dict) – metadata for the task
filters (entities.Filters) – dl.Filters entity to filter items for the task
items (List[entities.Item]) – list of items (item Id or objects) to insert to the task
query (dict DQL) – filter items for the task
available_actions (list) – list of available actions (statuses) that will be available for the task items; The default statuses are: “Completed” and “Discarded”
wait (bool) – wait until create task finish
check_if_exist (entities.Filters) – dl.Filters check if task exist according to filter
limit (int) – the limit items that the task can include
batch_size (int) – Pulling batch size (items), use with pulling allocation method. Restrictions - Min 3, max 100
max_batch_workload (int) – max_batch_workload: Max items in assignment, use with pulling allocation method. Restrictions - Min batchSize + 2, max batchSize * 2
allowed_assignees (list) – list the task assignees (contributors) that should be working on the task. Provide a list of users’ emails
priority (entities.TaskPriority) – priority of the task options in entities.TaskPriority

Returns

Task object

Return type

Example:

dataset.tasks.create(task= 'task_entity',
                    due_date = datetime.datetime(day= 1, month= 1, year= 2029).timestamp(),
                    assignee_ids =[ 'annotator1@dataloop.ai', 'annotator2@dataloop.ai'])

create_qa_task(task: Task, assignee_ids, due_date=None, filters=None, items=None, query=None, workload=None, metadata=None, available_actions=None, wait=True, batch_size=None, max_batch_workload=None, allowed_assignees=None, priority=TaskPriority.MEDIUM) → Task[source]¶

Create a new QA Task.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned to be owner of the annotation task.

Parameters

task (dtlpy.entities.task.Task) – the parent annotation task object
assignee_ids (list) – list the QA task assignees (contributors) that should be working on the task. Provide a list of users’ emails
due_date (float) – date by which the QA task should be finished; for example, due_date=datetime.datetime(day=1, month=1, year=2029).timestamp()
filters (entities.Filters) – dl.Filters entity to filter items for the task
items (List[entities.Item]) – list of items (item Id or objects) to insert to the task
query (dict DQL) – filter items for the task
workload (List[WorkloadUnit]) – list of WorkloadUnit objects. Customize distribution (percentage) between the task assignees. For example: [dl.WorkloadUnit(annotator@hi.com, 80), dl.WorkloadUnit(annotator2@hi.com, 20)]
metadata (dict) – metadata for the task
available_actions (list) – list of available actions (statuses) that will be available for the task items; The default statuses are: “Approved” and “Discarded”
wait (bool) – wait until create task finish
batch_size (int) – Pulling batch size (items), use with pulling allocation method. Restrictions - Min 3, max 100
max_batch_workload (int) – Max items in assignment, use with pulling allocation method. Restrictions - Min batchSize + 2, max batchSize * 2
allowed_assignees (list) – list the task assignees (contributors) that should be working on the task. Provide a list of users’ emails
priority (entities.TaskPriority) – priority of the task options in entities.TaskPriority

Returns

task object

Return type

Example:

dataset.tasks.create_qa_task(task= 'task_entity',
                            due_date = datetime.datetime(day= 1, month= 1, year= 2029).timestamp(),
                            assignee_ids =[ 'annotator1@dataloop.ai', 'annotator2@dataloop.ai'])

delete(task: Optional[Task] = None, task_name: Optional[str] = None, task_id: Optional[str] = None, wait: bool = True)[source]¶

Delete the Task.

Prerequisites: You must be in the role of an owner or developer or annotation manager who created that task.

Parameters

task (dtlpy.entities.task.Task) – the task object
task_name (str) – the name of the task
task_id (str) – the Id of the task
wait (bool) – wait until delete task finish

Returns

True is success

Return type

Example:

dataset.tasks.delete(task_id='task_id')

get(task_name=None, task_id=None) → Task[source]¶

Get a Task object to use in your code.

Prerequisites: You must be in the role of an owner or developer or annotation manager who has been assigned the task.

Parameters

task_name (str) – optional - search by name
task_id (str) – optional - search by id

Returns

task object

Return type

list or dtlpy.entities.paged_entities.PagedEntities

Example:

dataset.tasks.get(task_id='task_id')

get_items(task_id: Optional[str] = None, task_name: Optional[str] = None, dataset: Optional[Dataset] = None, filters: Optional[Filters] = None) → PagedEntities[source]¶

Get the task items to use in your code.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned to be owner of the annotation task.

If a filters param is provided, you will receive a PagedEntity output of the task items. If no filter is provided, you will receive a list of the items.

Parameters

task_id (str) – the Id of the task
task_name (str) – the name of the task
dataset (dtlpy.entities.dataset.Dataset) – dataset object that refer to the task
filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters

Returns

list of the items or PagedEntity output of items

Return type

Example:

dataset.tasks.get_items(task_id= 'task_id')

list(project_ids=None, status=None, task_name=None, pages_size=None, page_offset=None, recipe=None, creator=None, assignments=None, min_date=None, max_date=None, filters: Optional[Filters] = None) → Union[List[Task], PagedEntities][source]¶

List all tasks.

Prerequisites: You must be in the role of an owner or developer or annotation manager who has been assigned the task.

Parameters

project_ids – search tasks by given list of project ids
status (str) – search tasks by a given task status
task_name (str) – search tasks by a given task name
pages_size (int) – pages size of the output generator
page_offset (int) – page offset of the output generator
recipe (dtlpy.entities.recipe.Recipe) – Search tasks that use a given recipe. Provide the required recipe object
creator (str) – search tasks created by a given creator (user email)
assignments (dtlpy.entities.assignment.Assignment recipe) – assignments object
min_date (double) – search all tasks created AFTER a given date, use a milliseconds format. For example: 1661780622008
max_date (double) – search all tasks created BEFORE a given date, use a milliseconds format. For example: 1661780622008
filters (dtlpy.entities.filters.Filters) – dl.Filters entity to filters tasks using DQL

Returns

List of Task objects

Example:

dataset.tasks.list(project_ids='project_ids',pages_size=100, page_offset=0)

open_in_web(task_name: Optional[str] = None, task_id: Optional[str] = None, task: Optional[Task] = None)[source]¶

Open the task in the web platform.

Prerequisites: You must be in the role of an owner or developer or annotation manager who has been assigned the task.

Parameters

task_name (str) – the name of the task
task_id (str) – the Id of the task
task (dtlpy.entities.task.Task) – the task object

Example:

dataset.tasks.open_in_web(task_id='task_id')

query(filters=None, project_ids=None)[source]¶

List all tasks by filter.

Prerequisites: You must be in the role of an owner or developer or annotation manager who has been assigned the task.

Parameters

filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
project_ids (list) – list of project ids of the required tasks

Returns

Paged entity - task pages generator

Return type

Example:

dataset.tasks.query(project_ids='project_ids')

remove_items(task: Optional[Task] = None, task_id=None, filters: Optional[Filters] = None, query=None, items=None, wait=True)[source]¶

remove items from Task.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned to be owner of the annotation task.

Parameters

task (dtlpy.entities.task.Task) – task object
task_id (str) – the Id of the task
filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
query (dict) – query to filter the items use it
items (list) – list of items to add to the task
wait (bool) – wait until remove items finish

Returns

True if success and an error if failed

Return type

Examples:

dataset.tasks.remove_items(task= 'task_entity',
                            items = [items])

set_status(status: str, operation: str, task_id: str, item_ids: List[str])[source]¶

Update an item status within a task.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned to be owner of the annotation task.

Parameters

status (str) – string the describes the status
operation (str) – the status action need ‘create’ or ‘delete’
task_id (str) – the Id of the task
item_ids (list) – List[str] id items ids

Returns

True if success

Return type

Example:

dataset.tasks.set_status(task_id= 'task_id', status='complete', operation='create')

update(task: Optional[Task] = None, system_metadata=False) → Task[source]¶

Update a Task.

Prerequisites: You must be in the role of an owner or developer or annotation manager who created that task.

Parameters

task (dtlpy.entities.task.Task) – the task object
system_metadata (bool) – True, if you want to change metadata system

Returns

Task object

Return type

dtlpy.entities.assignment.Assignment

Example:

dataset.tasks.update(task='task_entity')

Assignments¶

class Assignments(client_api: ApiClient, project: Optional[Project] = None, task: Optional[Task] = None, dataset: Optional[Dataset] = None, project_id=None)[source]¶

Bases: object

Assignments Repository

The Assignments class allows users to manage assignments and their properties. Read more about Task Assignment in our SDK documentation.

create(assignee_id: str, task: Optional[Task] = None, filters: Optional[Filters] = None, items: Optional[list] = None) → Assignment[source]¶

Create a new assignment.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned as owner of the annotation task.

Parameters

assignee_id (str) – the email of the user that want to assign the assignment
task (dtlpy.entities.task.Task) – the task object that include the assignment
filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
items (list) – list of items (item Id or objects) to insert to the assignment

Returns

Assignment object

Return type

dtlpy.entities.assignment.Assignment assignment

Example:

task.assignments.create(assignee_id='annotator1@dataloop.ai')

get(assignment_name: Optional[str] = None, assignment_id: Optional[str] = None)[source]¶

Get Assignment object to use it in your code.

Parameters

assignment_name (str) – optional - search by name
assignment_id (str) – optional - search by id

Returns

Assignment object

Return type

Example:

task.assignments.get(assignment_id='assignment_id')

get_items(assignment: Optional[Assignment] = None, assignment_id=None, assignment_name=None, dataset=None, filters=None) → PagedEntities[source]¶

Get all the items in the assignment.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned as owner of the annotation task.

Parameters

assignment (dtlpy.entities.assignment.Assignment) – assignment object
assignment_id – the Id of the assignment
assignment_name (str) – the name of the assignment
dataset (dtlpy.entities.dataset.Dataset) – dataset object, the dataset that refer to the assignment
filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters

Returns

pages of the items

Return type

dtlpy.entities.assignment.Assignment

Example:

task.assignments.get_items(assignment_id='assignment_id')

list(project_ids: Optional[list] = None, status: Optional[str] = None, assignment_name: Optional[str] = None, assignee_id: Optional[str] = None, pages_size: Optional[int] = None, page_offset: Optional[int] = None, task_id: Optional[int] = None) → List[Assignment][source]¶

Get Assignment list to be able to use it in your code.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned as owner of the annotation task.

Parameters

project_ids (list) – search assignment by given list of project ids
status (str) – search assignment by a given task status
assignment_name (str) – search assignment by a given assignment name
assignee_id (str) – the user email that assignee the assignment to it
pages_size (int) – pages size of the output generator
page_offset (int) – page offset of the output generator
task_id (str) – search assignment by given task id

Returns

List of Assignment objects

Return type

miscellaneous.List[dtlpy.entities.assignment.Assignment]

Example:

task.assignments.list(status='complete', assignee_id='user@dataloop.ai', pages_size=100, page_offset=0)

open_in_web(assignment_name: Optional[str] = None, assignment_id: Optional[str] = None, assignment: Optional[str] = None)[source]¶

Open the assignment in the platform.

Prerequisites: All users.

Parameters

assignment_name (str) – the name of the assignment
assignment_id (str) – the Id of the assignment
assignment (dtlpy.entities.assignment.Assignment) – assignment object

Example:

task.assignments.open_in_web(assignment_id='assignment_id')

reassign(assignee_id: str, assignment: Optional[Assignment] = None, assignment_id: Optional[str] = None, task: Optional[Task] = None, task_id: Optional[str] = None, wait: bool = True)[source]¶

Reassign an assignment.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned as owner of the annotation task.

Parameters

assignee_id (str) – the email of the user that want to assign the assignment
assignment (dtlpy.entities.assignment.Assignment) – assignment object
assignment_id – the Id of the assignment
task (dtlpy.entities.task.Task) – task object
task_id (str) – the Id of the task that include the assignment
wait (bool) – wait until reassign assignment finish

Returns

Assignment object

Return type

Example:

task.assignments.reassign(assignee_ids='annotator1@dataloop.ai')

redistribute(workload: Workload, assignment: Optional[Assignment] = None, assignment_id: Optional[str] = None, task: Optional[Task] = None, task_id: Optional[str] = None, wait: bool = True)[source]¶

Redistribute an assignment.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned as owner of the annotation task.

Example:

Parameters

workload (dtlpy.entities.assignment.Workload) – list of WorkloadUnit objects. Customize distribution (percentage) between the task assignees. For example: [dl.WorkloadUnit(annotator@hi.com, 80), dl.WorkloadUnit(annotator2@hi.com, 20)]
assignment (dtlpy.entities.assignment.Assignment) – assignment object
assignment_id (str) – the Id of the assignment
task (dtlpy.entities.task.Task) – the task object that include the assignment
task_id (str) – the Id of the task that include the assignment
wait (bool) – wait until redistribute assignment finish

Returns

Assignment object

Return type

dtlpy.entities.assignment.Assignment assignment

task.assignments.redistribute(workload=dl.Workload([dl.WorkloadUnit(assignee_id="annotator1@dataloop.ai", load=50),
                                                    dl.WorkloadUnit(assignee_id="annotator2@dataloop.ai", load=50)]))

set_status(status: str, operation: str, item_id: str, assignment_id: str) → bool[source]¶

Set item status within assignment.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned as owner of the annotation task.

Parameters

status (str) – string the describes the status
operation (str) – the status action need ‘create’ or ‘delete’
item_id (str) – item id that want to set his status
assignment_id – the Id of the assignment

Returns

True id success

Return type

Example:

task.assignments.set_status(assignment_id='assignment_id',
                            status='complete',
                            operation='created',
                            item_id='item_id')

update(assignment: Optional[Assignment] = None, system_metadata: bool = False) → Assignment[source]¶

Update an assignment.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned as owner of the annotation task.

Parameters

assignment (dtlpy.entities.assignment.Assignment assignment) – assignment entity
system_metadata (bool) – True, if you want to change metadata system

Returns

Assignment object

Return type

dtlpy.entities.assignment.Assignment assignment

Example:

task.assignments.update(assignment='assignment_entity', system_metadata=False)

Packages¶

class LocalServiceRunner(client_api: ApiClient, packages, cwd=None, multithreading=False, concurrency=10, package: Optional[Package] = None, module_name='default_module', function_name='run', class_name='ServiceRunner', entry_point='main.py', mock_file_path=None)[source]¶

Bases: object

Service Runner Class

get_field(field_name, field_type, mock_json, project=None, mock_inputs=None)[source]¶

Get field in mock json.

Parameters

field_name – field name
field_type – field type
mock_json – mock json
project – project
mock_inputs – mock inputs

Returns

get_mainpy_run_service()[source]¶

Get mainpy run service

Returns

run_local_project(project=None)[source]¶

Run local project

Parameters: project – project entity

class Packages(client_api: ApiClient, project: Optional[Project] = None)[source]¶

Bases: object

Packages Repository

The Packages class allows users to manage packages (code used for running in Dataloop’s FaaS) and their properties. Read more about Packages.

build(package: Package, module_name=None, init_inputs=None, local_path=None, from_local=None)[source]¶

Instantiate a module from the package code. Returns a loaded instance of the runner class

Parameters

package – Package entity
module_name – Name of the module to build the runner class
init_inputs (str) – dictionary of the class init variables (if exists). will be used to init the module class
local_path (str) – local path of the package (if from_local=False - codebase will be downloaded)
from_local (bool) – bool. if true - codebase will not be downloaded (only use local files)

Returns

dl.BaseServiceRunner

build_requirements(filepath) → list[source]¶

Build a requirement list (list of packages your code requires to run) from a file path. The file listing the requirements MUST BE a txt file.

Prerequisites: You must be in the role of an owner or developer.

Parameters: filepath – path of the requirements file
Returns: a list of dl.PackageRequirement
Return type: list

static build_trigger_dict(actions, name='default_module', filters=None, function='run', execution_mode: TriggerExecutionMode = 'Once', type_t: TriggerType = 'Event')[source]¶

Build a trigger dictionary to trigger FaaS. Read more about FaaS Triggers.

Prerequisites: You must be in the role of an owner or developer.

Parameters

actions – list of dl.TriggerAction
name (str) – trigger name
filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
function (str) – function name
execution_mode (str) – execution mode dl.TriggerExecutionMode
type_t (str) – trigger type dl.TriggerType

Returns

trigger dict

Return type

Example:

project.packages.build_trigger_dict(actions=dl.TriggerAction.CREATED,
                                  function='run',
                                  execution_mode=dl.TriggerExecutionMode.ONCE)

static check_cls_arguments(cls, missing, function_name, function_inputs)[source]¶

Check class arguments. This method checks that the package function is correct.

Prerequisites: You must be in the role of an owner or developer.

Parameters

cls – packages class
missing (list) – list of the missing params
function_name (str) – name of function
function_inputs (list) – list of function inputs

checkout(package: Optional[Package] = None, package_id: Optional[str] = None, package_name: Optional[str] = None)[source]¶

Checkout (switch) to a package.

Prerequisites: You must be in the role of an owner or developer.

Parameters

package (dtlpy.entities.package.Package) – package entity
package_id (str) – package id
package_name (str) – package name

Example:

project.packages.checkout(package='package_entity')

delete(package: Optional[Package] = None, package_name=None, package_id=None)[source]¶

Delete a Package object.

Prerequisites: You must be in the role of an owner or developer.

Parameters

package (dtlpy.entities.package.Package) – package entity
package_id (str) – package id
package_name (str) – package name

Returns

True if success

Return type

Example:

project.packages.delete(package_name='package_name')

deploy(package_id: Optional[str] = None, package_name: Optional[str] = None, package: Optional[Package] = None, service_name: Optional[str] = None, project_id: Optional[str] = None, revision: Optional[str] = None, init_input: Optional[Union[List[FunctionIO], FunctionIO, dict]] = None, runtime: Optional[Union[KubernetesRuntime, dict]] = None, sdk_version: Optional[str] = None, agent_versions: Optional[dict] = None, bot: Optional[Union[Bot, str]] = None, pod_type: Optional[InstanceCatalog] = None, verify: bool = True, checkout: bool = False, module_name: Optional[str] = None, run_execution_as_process: Optional[bool] = None, execution_timeout: Optional[int] = None, drain_time: Optional[int] = None, on_reset: Optional[str] = None, max_attempts: Optional[int] = None, force: bool = False, secrets: Optional[list] = None, **kwargs) → Service[source]¶

Deploy a package. A service is required to run the code in your package.

Prerequisites: You must be in the role of an owner or developer.

Parameters

package_id (str) – package id
package_name (str) – package name
package (dtlpy.entities.package.Package) – package entity
service_name (str) – service name
project_id (str) – project id
revision (str) – package revision - default=latest
init_input – config to run at startup
runtime (dict) – runtime resources
sdk_version (str) –
- optional - string - sdk version
agent_versions (dict) –
- dictionary - - optional -versions of sdk, agent runner and agent proxy
bot (str) – bot email
pod_type (str) – pod type dl.InstanceCatalog
verify (bool) – verify the inputs
checkout (bool) – checkout
module_name (str) – module name
run_execution_as_process (bool) – run execution as process
execution_timeout (int) – execution timeout
drain_time (int) – drain time
on_reset (str) – on reset
max_attempts (int) – Maximum execution retries in-case of a service reset
force (bool) – optional - terminate old replicas immediately
secrets (list) – list of the integrations ids

Returns

Service object

Return type

dtlpy.entities.package.Package

Example:

project.packages.deploy(service_name=package_name,
                        execution_timeout=3 * 60 * 60,
                        module_name=module.name,
                        runtime=dl.KubernetesRuntime(
                            concurrency=10,
                            pod_type=dl.InstanceCatalog.REGULAR_S,
                            autoscaler=dl.KubernetesRabbitmqAutoscaler(
                                min_replicas=1,
                                max_replicas=20,
                                queue_length=20
                            )
                        )
                    )

deploy_from_file(project, json_filepath)[source]¶

Deploy package and service from a JSON file.

Prerequisites: You must be in the role of an owner or developer.

Parameters

project (dtlpy.entities.project.Project) – project entity
json_filepath (str) – path of the file to deploy

Returns

the package and the services

Example:

project.packages.deploy_from_file(project='project_entity', json_filepath='json_filepath')

static generate(name=None, src_path: Optional[str] = None, service_name: Optional[str] = None, package_type='default_package_type')[source]¶

Generate a new package. Provide a file path to a JSON file with all the details of the package and service to generate the package.

Prerequisites: You must be in the role of an owner or developer.

Parameters

name (str) – name
src_path (str) – source file path
service_name (str) – service name
package_type (str) – package type from PackageCatalog

Example:

project.packages.generate(name='package_name',
                          src_path='src_path')

get(package_name: Optional[str] = None, package_id: Optional[str] = None, checkout: bool = False, fetch=None) → Package[source]¶

Get Package object to use in your code.

Prerequisites: You must be in the role of an owner or developer.

Parameters

package_id (str) – package id
package_name (str) – package name
checkout (bool) – set the package as a default package object (cookies)
fetch – optional - fetch entity from platform, default taken from cookie

Returns

Package object

Return type

Example:

project.packages.get(package_id='package_id')

list(filters: Optional[Filters] = None, project_id: Optional[str] = None) → PagedEntities[source]¶

List project packages.

Prerequisites: You must be in the role of an owner or developer.

Parameters

filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
project_id (str) – project id

Returns

Paged entity

Return type

Example:

project.packages.list()

open_in_web(package: Optional[Package] = None, package_id: Optional[str] = None, package_name: Optional[str] = None)[source]¶

Open the package in the web platform.

Prerequisites: You must be in the role of an owner or developer.

Parameters

package (dtlpy.entities.package.Package) – package entity
package_id (str) – package id
package_name (str) – package name

Example:

project.packages.open_in_web(package_id='package_id')

pull(package: Package, version=None, local_path=None, project_id=None)[source]¶

Pull (download) the package to a local path.

Prerequisites: You must be in the role of an owner or developer.

Parameters

package (dtlpy.entities.package.Package) – package entity
version (str) – the package version to pull
local_path – the path of where to save the package
project_id – the project id that include the package

Returns

local path where the package pull

Return type

Example:

project.packages.pull(package='package_entity', local_path='local_path')

push(project: Optional[Project] = None, project_id: Optional[str] = None, package_name: Optional[str] = None, src_path: Optional[str] = None, codebase: Optional[Union[GitCodebase, ItemCodebase, FilesystemCodebase]] = None, modules: Optional[List[PackageModule]] = None, is_global: Optional[bool] = None, checkout: bool = False, revision_increment: Optional[str] = None, version: Optional[str] = None, ignore_sanity_check: bool = False, service_update: bool = False, service_config: Optional[dict] = None, slots: Optional[List[PackageSlot]] = None, requirements: Optional[List[PackageRequirement]] = None, package_type=None, metadata=None) → Package[source]¶

Push your local package to the UI.

Prerequisites: You must be in the role of an owner or developer.

Project will be taken in the following hierarchy: project(input) -> project_id(input) -> self.project(context) -> checked out

Parameters

project (dtlpy.entities.project.Project) – optional - project entity to deploy to. default from context or checked-out
project_id (str) – optional - project id to deploy to. default from context or checked-out
package_name (str) – package name
src_path (str) – path to package codebase
codebase (dtlpy.entities.codebase.Codebase) – codebase object
modules (list) – list of modules PackageModules of the package
is_global (bool) – is package is global or local
checkout (bool) – checkout package to local dir
revision_increment (str) – optional - str - version bumping method - major/minor/patch - default = None
version (str) – semver version f the package
ignore_sanity_check (bool) – NOT RECOMMENDED - skip code sanity check before pushing
service_update (bool) – optional - bool - update the service

:param dict service_config : Service object as dict. Contains the spec of the default service to create. :param list slots: optional - list of slots PackageSlot of the package :param list requirements: requirements - list of package requirements :param str package_type: default ‘faas’, options: ‘app’, ‘ml :param dict metadata: dictionary of system and user metadata

Returns: Package object
Return type: dtlpy.entities.package.Package

Example:

project.packages.push(package_name='package_name',
                        modules=[module],
                        version='1.0.0',
                        src_path=os.getcwd()
                    )

revisions(package: Optional[Package] = None, package_id: Optional[str] = None)[source]¶

Get the package revisions history.

Prerequisites: You must be in the role of an owner or developer.

Parameters

package (dtlpy.entities.package.Package) – package entity
package_id (str) – package id

Example:

project.packages.revisions(package='package_entity')

test_local_package(cwd: Optional[str] = None, concurrency: Optional[int] = None, package: Optional[Package] = None, module_name: str = 'default_module', function_name: str = 'run', class_name: str = 'ServiceRunner', entry_point: str = 'main.py', mock_file_path: Optional[str] = None)[source]¶

Test local package in local environment.

Prerequisites: You must be in the role of an owner or developer.

Parameters

cwd (str) – path to the file
concurrency (int) – the concurrency of the test
package (dtlpy.entities.package.Package) – entities.package
module_name (str) – module name
function_name (str) – function name
class_name (str) – class name
entry_point (str) – the file to run like main.py
mock_file_path (str) – the mock file that have the inputs

Returns

list created by the function that tested the output

Return type

dtlpy.entities.package.Package

Example:

project.packages.test_local_package(cwd='path_to_package',
                                    package='package_entity',
                                    function_name='run')

update(package: Package, revision_increment: Optional[str] = None) → Package[source]¶

Update Package changes to the platform.

Prerequisites: You must be in the role of an owner or developer.

Parameters

package (dtlpy.entities.package.Package) –
revision_increment – optional - str - version bumping method - major/minor/patch - default = None

Returns

Package object

Return type

Example:

project.packages.delete(package='package_entity')

Codebases¶

class Codebases(client_api: ApiClient, project: Optional[Project] = None, dataset: Optional[Dataset] = None, project_id: Optional[str] = None)[source]¶

Bases: object

Codebase Repository

The Codebases class allows the user to manage codebases and their properties. The codebase is the code the user uploads for the user’s packages to run. Read more about codebase in our FaaS (function as a service).

clone_git(codebase: Codebase, local_path: str)[source]¶

Clone code base

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters

codebase (dtlpy.entities.codebase.Codebase) – codebase object
local_path (str) – local path

Returns

path where the clone will be

Return type

str

Example:

package.codebases.clone_git(codebase='codebase_entity', local_path='local_path')

get(codebase_name: Optional[str] = None, codebase_id: Optional[str] = None, version: Optional[str] = None)[source]¶

Get a Codebase object to use in your code.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Example:

package.codebases.get(codebase_name='codebase_name')

Parameters

codebase_name (str) – optional - search by name
codebase_id (str) – optional - search by id
version (str) – codebase version. default is latest. options: “all”, “latest” or ver number - “10”

Returns

Codebase object

static get_current_version(all_versions_pages, zip_md)[source]¶

This method returns the current version of the codebase and other versions found.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters

all_versions_pages (codebase) – codebase object
zip_md – zipped file of codebase

Returns

current version and all versions found of codebase

Return type

int, int

Example:

package.codebases.get_current_version(all_versions_pages='codebase_entity', zip_md='path')

list() → PagedEntities[source]¶

List all codebases.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Example:

package.codebases.list()

Returns: Paged entity
Return type: dtlpy.entities.paged_entities.PagedEntities

list_versions(codebase_name: str)[source]¶

List all codebase versions.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Example:

package.codebases.list_versions(codebase_name='codebase_name')

Parameters: codebase_name (str) – code base name
Returns: list of versions
Return type: list

pack(directory: str, name: Optional[str] = None, extension: str = 'zip', description: str = '', ignore_directories: Optional[List[str]] = None)[source]¶

Zip a local code directory and post to codebases.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters

directory (str) – local directory to pack
name (str) – codebase name
extension (str) – the extension of the file
description (str) – codebase description
ignore_directories (list[str]) – directories to ignore.

Returns

Codebase object

Return type

dtlpy.entities.codebase.Codebase

Example:

package.codebases.pack(directory='path_dir', name='codebase_name')

pull_git(codebase, local_path)[source]¶

Pull (download) a codebase.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters

codebase (dtlpy.entities.codebase.Codebase) – codebase object
local_path (str) – local path

Returns

path where the Pull will be

Return type

Example:

package.codebases.pull_git(codebase='codebase_entity', local_path='local_path')

unpack(codebase: Optional[Codebase] = None, codebase_name: Optional[str] = None, codebase_id: Optional[str] = None, local_path: Optional[str] = None, version: Optional[str] = None)[source]¶

Unpack codebase locally. Download source code and unzip.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters

codebase (dtlpy.entities.codebase.Codebase) – dl.Codebase object
codebase_name (str) – search by name
codebase_id (str) – search by id
local_path (str) – local path to save codebase
version (str) – codebase version to unpack. default - latest

Returns

String (dirpath)

Return type

Example:

package.codebases.unpack(codebase='codebase_entity', local_path='local_path')

Services¶

class ServiceLog(_json: dict, service: Service, services: Services, start=None, follow=None, execution_id=None, function_name=None, replica_id=None, system=False)[source]¶

Bases: object

Service Log

view(until_completed)[source]¶

View logs

Parameters: until_completed –

class Services(client_api: ApiClient, project: Optional[Project] = None, package: Optional[Package] = None, project_id=None)[source]¶

Bases: object

Services Repository

The Services class allows the user to manage services and their properties. Services are created from the packages users create. See our documentation for more information about services.

activate_slots(service: Service, project_id: Optional[str] = None, task_id: Optional[str] = None, dataset_id: Optional[str] = None, org_id: Optional[str] = None, user_email: Optional[str] = None, slots: Optional[List[PackageSlot]] = None, role=None, prevent_override: bool = True, visible: bool = True, icon: str = 'fas fa-magic', **kwargs)[source]¶

Activate service slots (creates buttons in the UI that activate services).

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters

service (dtlpy.entities.service.Service) – service entity
project_id (str) – project id
task_id (str) – task id
dataset_id (str) – dataset id
org_id (str) – org id
user_email (str) – user email
slots (list) – list of entities.PackageSlot
role (str) – user role MemberOrgRole.ADMIN, MemberOrgRole.owner, MemberOrgRole.MEMBER
prevent_override (bool) – True to prevent override
visible (bool) – visible
icon (str) – icon
kwargs – all additional arguments

Returns

list of user setting for activated slots

Return type

Example:

package.services.activate_slots(service='service_entity',
                                project_id='project_id',
                                slots=List[entities.PackageSlot],
                                icon='fas fa-magic')

checkout(service: Optional[Service] = None, service_name: Optional[str] = None, service_id: Optional[str] = None)[source]¶

Checkout (switch) to a service.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters

service (dtlpy.entities.service.Service) – Service entity
service_name (str) – service name
service_id (str) – service id

Example:

package.services.checkout(service_id='service_id')

delete(service_name: Optional[str] = None, service_id: Optional[str] = None)[source]¶

Delete Service object

Prerequisites: You must be in the role of an owner or developer. You must have a package.

You must provide at least ONE of the following params: service_id, service_name.

Parameters

service_name (str) – by name
service_id (str) – by id

Returns

True

Return type

Example:

package.services.delete(service_id='service_id')

deploy(service_name: Optional[str] = None, package: Optional[Package] = None, bot: Optional[Union[Bot, str]] = None, revision: Optional[str] = None, init_input: Optional[Union[List[FunctionIO], FunctionIO, dict]] = None, runtime: Optional[Union[KubernetesRuntime, dict]] = None, pod_type: Optional[InstanceCatalog] = None, sdk_version: Optional[str] = None, agent_versions: Optional[dict] = None, verify: bool = True, checkout: bool = False, module_name: Optional[str] = None, project_id: Optional[str] = None, driver_id: Optional[str] = None, func: Optional[Callable] = None, run_execution_as_process: Optional[bool] = None, execution_timeout: Optional[int] = None, drain_time: Optional[int] = None, max_attempts: Optional[int] = None, on_reset: Optional[str] = None, force: bool = False, secrets: Optional[list] = None, **kwargs) → Service[source]¶

Deploy service.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters

service_name (str) – name
package (dtlpy.entities.package.Package) – package entity
bot (str) – bot email
revision (str) – package revision of version
init_input – config to run at startup
runtime (dict) – runtime resources
pod_type (str) – pod type dl.InstanceCatalog
sdk_version (str) –
- optional - string - sdk version
agent_versions (str) –
- dictionary - - optional -versions of sdk
verify (bool) – if true, verify the inputs
checkout (bool) – if true, checkout (switch) to service
module_name (str) – module name
project_id (str) – project id
driver_id (str) – driver id
func (Callable) – function to deploy
run_execution_as_process (bool) – if true, run execution as process
execution_timeout (int) – execution timeout in seconds
drain_time (int) – drain time in seconds
max_attempts (int) – maximum execution retries in-case of a service reset
on_reset (str) – what happens on reset
force (bool) – optional - if true, terminate old replicas immediately
secrets (list) – list of the integrations ids
kwargs – list of additional arguments

Returns

Service object

Return type

Example:

package.services.deploy(service_name=package_name,
                        execution_timeout=3 * 60 * 60,
                        module_name=module.name,
                        runtime=dl.KubernetesRuntime(
                            concurrency=10,
                            pod_type=dl.InstanceCatalog.REGULAR_S,
                            autoscaler=dl.KubernetesRabbitmqAutoscaler(
                                min_replicas=1,
                                max_replicas=20,
                                queue_length=20
                            )
                        )
                    )

deploy_from_local_folder(cwd=None, service_file=None, bot=None, checkout=False, force=False) → Service[source]¶

Deploy from local folder in local environment.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters

cwd (str) – optional - package working directory. Default=cwd
service_file (str) – optional - service file. Default=None
bot (str) – bot
checkout – checkout
force (bool) – optional - terminate old replicas immediately

Returns

Service object

Return type

Example:

package.services.deploy_from_local_folder(cwd='file_path',
                                          service_file='service_file')

execute(service: Optional[Service] = None, service_id: Optional[str] = None, service_name: Optional[str] = None, sync: bool = False, function_name: Optional[str] = None, stream_logs: bool = False, execution_input=None, resource=None, item_id=None, dataset_id=None, annotation_id=None, project_id=None) → Execution[source]¶

Execute a function on an existing service.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters

service (dtlpy.entities.service.Service) – service entity
service_id (str) – service id
service_name (str) – service name
sync (bool) – wait for function to end
function_name (str) – function name to run
stream_logs (bool) – prints logs of the new execution. only works with sync=True
execution_input – input dictionary or list of FunctionIO entities
resource (str) – dl.PackageInputType - input type.
item_id (str) – str - optional - input to function
dataset_id (str) – str - optional - input to function
annotation_id (str) – str - optional - input to function
project_id (str) – str - resource’s project

Returns

entities.Execution

Return type

Example:

package.services.execute(service='service_entity',
                        function_name='run',
                        item_id='item_id',
                        project_id='project_id')

get(service_name=None, service_id=None, checkout=False, fetch=None) → Service[source]¶

Get service to use in your code.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters

service_name (str) – optional - search by name
service_id (str) – optional - search by id
checkout (bool) – if true, checkout (switch) to service
fetch – optional - fetch entity from platform, default taken from cookie

Returns

Service object

Return type

Example:

package.services.get(service_id='service_id')

list(filters: Optional[Filters] = None) → PagedEntities[source]¶

List all services (services can be listed for a package or for a project).

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters: filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
Returns: Paged entity
Return type: dtlpy.entities.paged_entities.PagedEntities

Example:

package.services.list()

log(service, size=100, checkpoint=None, start=None, end=None, follow=False, text=None, execution_id=None, function_name=None, replica_id=None, system=False, view=True, until_completed=True)[source]¶

Get service logs.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters

service (dtlpy.entities.service.Service) – service object
size (int) – size
checkpoint (dict) – the information from the lst point checked in the service
start (str) – iso format time
end (str) – iso format time
follow (bool) – if true, keep stream future logs
text (str) – text
execution_id (str) – execution id
function_name (str) – function name
replica_id (str) – replica id
system (bool) – system
view (bool) – if true, print out all the logs
until_completed (bool) – wait until completed

Returns

ServiceLog entity

Return type

ServiceLog

Example:

package.services.log(service='service_entity')

name_validation(name: str)[source]¶

Validation service name.

Prerequisites: You must be in the role of an owner or developer.

Parameters: name (str) – service name

Example:

package.services.name_validation(name='name')

open_in_web(service: Optional[Service] = None, service_id: Optional[str] = None, service_name: Optional[str] = None)[source]¶

Open the service in web platform

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters

service_name (str) – service name
service_id (str) – service id
service (dtlpy.entities.service.Service) – service entity

Example:

package.services.open_in_web(service_id='service_id')

pause(service_name: Optional[str] = None, service_id: Optional[str] = None, force: bool = False)[source]¶

Pause service.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

You must provide at least ONE of the following params: service_id, service_name

Parameters

service_name (str) – service name
service_id (str) – service id
force (bool) – optional - terminate old replicas immediately

Returns

True if success

Return type

Example:

package.services.pause(service_id='service_id')

resume(service_name: Optional[str] = None, service_id: Optional[str] = None, force: bool = False)[source]¶

Resume service.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

You must provide at least ONE of the following params: service_id, service_name.

Parameters

service_name (str) – service name
service_id (str) – service id
force (bool) – optional - terminate old replicas immediately

Returns

json of the service

Return type

Example:

package.services.resume(service_id='service_id')

revisions(service: Optional[Service] = None, service_id: Optional[str] = None)[source]¶

Get service revisions history.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

You must provide at leats ONE of the following params: service, service_id

Parameters

service (dtlpy.entities.service.Service) – Service entity
service_id (str) – service id

Example:

package.services.revisions(service_id='service_id')

status(service_name=None, service_id=None)[source]¶

Get service status.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

You must provide at least ONE of the following params: service_id, service_name

Parameters

service_name (str) – service name
service_id (str) – service id

Returns

status json

Return type

Example:

package.services.status(service_id='service_id')

update(service: Service, force: bool = False) → Service[source]¶

Update service changes to platform.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters

service (dtlpy.entities.service.Service) – Service entity
force (bool) – optional - terminate old replicas immediately

Returns

Service entity

Return type

Example:

package.services.update(service='service_entity')

Bots¶

class Bots(client_api: ApiClient, project: Project)[source]¶

Bases: object

Bots Repository

The Bots class allows the user to manage bots and their properties. See our documentation for more information on bots.

create(name: str, return_credentials: bool = False)[source]¶

Create a new Bot.

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters

name (str) – bot name
return_credentials (str) – True will return the password when created

Returns

Bot object

Return type

dtlpy.entities.bot.Bot

Example:

service.bots.delete(name='bot', return_credentials=False)

delete(bot_id: Optional[str] = None, bot_email: Optional[str] = None)[source]¶

Delete a Bot.

Prerequisites: You must be in the role of an owner or developer. You must have a service.

You must provide at least ONE of the following params: bot_id, bot_email

Parameters

bot_id (str) – bot id to delete
bot_email (str) – bot email to delete

Returns

True if successful

Return type

Example:

service.bots.delete(bot_id='bot_id')

get(bot_email: Optional[str] = None, bot_id: Optional[str] = None, bot_name: Optional[str] = None)[source]¶

Get a Bot object.

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters

bot_email (str) – get bot by email
bot_id (str) – get bot by id
bot_name (str) – get bot by name

Returns

Bot object

Return type

dtlpy.entities.bot.Bot

Example:

service.bots.get(bot_id='bot_id')

list() → List[Bot][source]¶

Get a project or package bots list.

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Returns: List of Bots objects
Return type: list

Example:

service.bots.list()

Triggers¶

class Triggers(client_api: ApiClient, project: Optional[Project] = None, service: Optional[Service] = None, project_id: Optional[str] = None, pipeline: Optional[Pipeline] = None)[source]¶

Bases: object

Triggers Repository

The Triggers class allows users to manage triggers and their properties. Triggers activate services. See our documentation for more information on triggers.

create(service_id: Optional[str] = None, trigger_type: TriggerType = TriggerType.EVENT, name: Optional[str] = None, webhook_id=None, function_name='run', project_id=None, active=True, filters=None, resource: TriggerResource = TriggerResource.ITEM, actions: Optional[TriggerAction] = None, execution_mode: TriggerExecutionMode = TriggerExecutionMode.ONCE, start_at=None, end_at=None, inputs=None, cron=None, pipeline_id=None, pipeline=None, pipeline_node_id=None, root_node_namespace=None, **kwargs) → BaseTrigger[source]¶

Create a Trigger. Can create two types: a cron trigger or an event trigger. Inputs are different for each type

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Inputs for all types:

Parameters

service_id (str) – Id of services to be triggered
trigger_type (str) – can be cron or event. use enum dl.TriggerType for the full list
name (str) – name of the trigger
webhook_id (str) – id for webhook to be called
function_name (str) – the function name to be called when triggered (must be defined in the package)
project_id (str) – project id where trigger will work
active (bool) – optional - True/False, default = True, if true trigger is active

Inputs for event trigger: :param dtlpy.entities.filters.Filters filters: optional - Item/Annotation metadata filters, default = none :param str resource: optional - Dataset/Item/Annotation/ItemStatus, default = Item :param str actions: optional - Created/Updated/Deleted, default = create :param str execution_mode: how many times trigger should be activated; default is “Once”. enum dl.TriggerExecutionMode

Inputs for cron trigger: :param start_at: iso format date string to start activating the cron trigger :param end_at: iso format date string to end the cron activation :param inputs: dictionary “name”:”val” of inputs to the function :param str cron: cron spec specifying when it should run. more information: https://en.wikipedia.org/wiki/Cron :param str pipeline_id: Id of pipeline to be triggered :param pipeline: pipeline entity to be triggered :param str pipeline_node_id: Id of pipeline root node to be triggered :param root_node_namespace: namespace of pipeline root node to be triggered

Returns: Trigger entity
Return type: dtlpy.entities.trigger.Trigger

Example:

service.triggers.create(name='triggername',
                      execution_mode=dl.TriggerExecutionMode.ONCE,
                      resource='Item',
                      actions='Created',
                      function_name='run',
                      filters={'$and': [{'hidden': False},
                                        {'type': 'file'}]}
                      )

delete(trigger_id=None, trigger_name=None)[source]¶

Delete Trigger object

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters

trigger_id (str) – trigger id
trigger_name (str) – trigger name

Returns

True is successful error if not

Return type

dtlpy.entities.trigger.Trigger

Example:

service.triggers.delete(trigger_id='trigger_id')

get(trigger_id=None, trigger_name=None) → BaseTrigger[source]¶

Get Trigger object

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters

trigger_id (str) – trigger id
trigger_name (str) – trigger name

Returns

Trigger entity

Return type

Example:

service.triggers.get(trigger_id='trigger_id')

list(filters: Optional[Filters] = None) → PagedEntities[source]¶

List triggers of a project, package, or service.

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters: filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
Returns: Paged entity
Return type: dtlpy.entities.paged_entities.PagedEntities

Example:

service.triggers.list()

name_validation(name: str)[source]¶

This method validates the trigger name. If name is not valid, this method will return an error. Otherwise, it will not return anything.

Parameters: name (str) – trigger name

resource_information(resource, resource_type, action='Created')[source]¶

Returns which function should run on an item (based on global triggers).

Prerequisites: You must be a superuser to run this method.

Parameters

resource – ‘Item’ / ‘Dataset’ / etc
resource_type – dictionary of the resource object
action – ‘Created’ / ‘Updated’ / etc.

Example:

service.triggers.resource_information(resource='Item', resource_type=item_object, action='Created')

update(trigger: BaseTrigger) → BaseTrigger[source]¶

Update trigger

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters: trigger (dtlpy.entities.trigger.Trigger) – Trigger entity
Returns: Trigger entity
Return type: dtlpy.entities.trigger.Trigger

Example:

service.triggers.update(trigger='trigger_entity')

Executions¶

class Executions(client_api: ApiClient, service: Optional[Service] = None, project: Optional[Project] = None)[source]¶

Bases: object

Service Executions Repository

The Executions class allows the users to manage executions (executions of services) and their properties. See our documentation for more information about executions.

create(service_id: Optional[str] = None, execution_input: Optional[list] = None, function_name: Optional[str] = None, resource: Optional[PackageInputType] = None, item_id: Optional[str] = None, dataset_id: Optional[str] = None, annotation_id: Optional[str] = None, project_id: Optional[str] = None, sync: bool = False, stream_logs: bool = False, return_output: bool = False, return_curl_only: bool = False, timeout: Optional[int] = None) → Execution[source]¶

Execute a function on an existing service

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters

service_id (str) – service id to execute on
execution_input (List[FunctionIO] or dict) – input dictionary or list of FunctionIO entities
function_name (str) – function name to run
resource (str) – input type.
item_id (str) – optional - item id as input to function
dataset_id (str) – optional - dataset id as input to function
annotation_id (str) – optional - annotation id as input to function
project_id (str) – resource’s project
sync (bool) – if true, wait for function to end
stream_logs (bool) – prints logs of the new execution. only works with sync=True
return_output (bool) – if True and sync is True - will return the output directly
return_curl_only (bool) – return the cURL of the creation WITHOUT actually do it
timeout (int) – int, seconds to wait until TimeoutError is raised. if <=0 - wait until done - by default wait take the service timeout

Returns

execution object

Return type

Example:

service.executions.create(function_name='function_name', item_id='item_id', project_id='project_id')

get(execution_id: Optional[str] = None, sync: bool = False) → Execution[source]¶

Get Service execution object

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters

execution_id (str) – execution id
sync (bool) – if true, wait for the execution to finish

Returns

Service execution object

Return type

Example:

service.executions.get(execution_id='execution_id')

increment(execution: Execution)[source]¶

Increment the number of attempts that an execution is allowed to attempt to run a service that is not responding.

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters: execution (dtlpy.entities.execution.Execution) –
Returns: int
Return type: int

Example:

service.executions.increment(execution='execution_entity')

list(filters: Optional[Filters] = None) → PagedEntities[source]¶

List service executions

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters: filters (dtlpy.entities.filters.Filters) – dl.Filters entity to filters items
Returns: Paged entity
Return type: dtlpy.entities.paged_entities.PagedEntities

Example:

service.executions.list()

logs(execution_id: str, follow: bool = True, until_completed: bool = True)[source]¶

executions logs

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters

execution_id (str) – execution id
follow (bool) – if true, keep stream future logs
until_completed (bool) – if true, wait until completed

Returns

executions logs

Example:

service.executions.logs(execution_id='execution_id')

progress_update(execution_id: str, status: Optional[ExecutionStatus] = None, percent_complete: Optional[int] = None, message: Optional[str] = None, output: Optional[str] = None, service_version: Optional[str] = None)[source]¶

Update Execution Progress.

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters

execution_id (str) – execution id
status (str) – ExecutionStatus
percent_complete (int) – percent work done
message (str) – message
output (str) – the output of the execution
service_version (str) – service version

Returns

Service execution object

Return type

Example:

service.executions.progress_update(execution_id='execution_id', status='complete', percent_complete=100)

rerun(execution: Execution, sync: bool = False)[source]¶

Rerun execution

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters

execution (dtlpy.entities.execution.Execution) –
sync (bool) – wait for the execution to finish

Returns

Execution object

Return type

Example:

service.executions.rerun(execution='execution_entity')

terminate(execution: Execution)[source]¶

Terminate Execution

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters: execution (dtlpy.entities.execution.Execution) –
Returns: execution object
Return type: dtlpy.entities.execution.Execution

Example:

service.executions.terminate(execution='execution_entity')

update(execution: Execution) → Execution[source]¶

Update execution changes to platform

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters: execution (dtlpy.entities.execution.Execution) – execution entity
Returns: Service execution object
Return type: dtlpy.entities.execution.Execution

Example:

service.executions.update(execution='execution_entity')

wait(execution_id: str, timeout: Optional[int] = None)[source]¶

Get Service execution object.

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters

execution_id (str) – execution id
timeout (int) – seconds to wait until TimeoutError is raised. if <=0 - wait until done - by default wait take the service timeout

Returns

Service execution object

Return type

dtlpy.entities.pipeline.Pipeline

Example:

service.executions.wait(execution_id='execution_id')

Pipelines¶

class Pipelines(client_api: ApiClient, project: Optional[Project] = None)[source]¶

Bases: object

Pipelines Repository

The Pipelines class allows users to manage pipelines and their properties. See our documentation for more information on pipelines.

create(name: Optional[str] = None, project_id: Optional[str] = None, pipeline_json: Optional[dict] = None) → Pipeline[source]¶

Create a new pipeline.

prerequisites: You must be an owner or developer to use this method.

Parameters

name (str) – pipeline name
project_id (str) – project id
pipeline_json (dict) – json containing the pipeline fields

Returns

Pipeline object

Return type

Example:

project.pipelines.create(name='pipeline_name')

delete(pipeline: Optional[Pipeline] = None, pipeline_name: Optional[str] = None, pipeline_id: Optional[str] = None)[source]¶

Delete Pipeline object.

prerequisites: You must be an owner or developer to use this method.

Parameters

pipeline (dtlpy.entities.pipeline.Pipeline) – pipeline entity
pipeline_id (str) – pipeline id
pipeline_name (str) – pipeline name

Returns

True if success

Return type

dtlpy.entities.pipeline_execution.PipelineExecution

Example:

project.pipelines.delete(pipeline_id='pipeline_id')

execute(pipeline: Optional[Pipeline] = None, pipeline_id: Optional[str] = None, pipeline_name: Optional[str] = None, execution_input=None)[source]¶

Execute a pipeline and return the pipeline execution as an object.

prerequisites: You must be an owner or developer to use this method.

Parameters

pipeline (dtlpy.entities.pipeline.Pipeline) – pipeline entity
pipeline_id (str) – pipeline id
pipeline_name (str) – pipeline name
execution_input – list of the dl.FunctionIO or dict of pipeline input - example {‘item’: ‘item_id’}

Returns

entities.PipelineExecution object

Return type

Example:

project.pipelines.execute(pipeline='pipeline_entity', execution_input= {'item': 'item_id'} )

get(pipeline_name=None, pipeline_id=None, fetch=None) → Pipeline[source]¶

Get Pipeline object to use in your code.

prerequisites: You must be an owner or developer to use this method.

You must provide at least ONE of the following params: pipeline_name, pipeline_id.

Parameters

pipeline_id (str) – pipeline id
pipeline_name (str) – pipeline name
fetch – optional - fetch entity from platform, default taken from cookie

Returns

Pipeline object

Return type

dtlpy.entities.pipeline.Pipeline

Example:

project.pipelines.get(pipeline_id='pipeline_id')

install(pipeline: Optional[Pipeline] = None)[source]¶

Install (start) a pipeline.

prerequisites: You must be an owner or developer to use this method.

Parameters: pipeline (dtlpy.entities.pipeline.Pipeline) – pipeline entity
Returns: Composition object

Example:

project.pipelines.install(pipeline='pipeline_entity')

list(filters: Optional[Filters] = None, project_id: Optional[str] = None) → PagedEntities[source]¶

List project pipelines.

prerequisites: You must be an owner or developer to use this method.

Parameters

filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters
project_id (str) – project id

Returns

Paged entity

Return type