Repositories

Organizations

class Organizations(client_api: ApiClient)[source]

Bases: object

Organizations Repository

Read our documentation to learn more about Organizations in the Dataloop platform.

add_member(email: str, role: MemberOrgRole = MemberOrgRole.MEMBER, organization_id: Optional[str] = None, organization_name: Optional[str] = None, organization: Optional[Organization] = None)[source]

Add members to your organization. Read about members and groups here.

Prerequisities: To add members to an organization, you must be an owner in that organization.

You must provide at least ONE of the following params: organization, organization_name, or organization_id.

Parameters:

  • email (str) – the member’s email

  • role (str) – MemberOrgRole.ADMIN, MemberOrgRole.OWNER, MemberOrgRole.MEMBER, MemberOrgRole.WORKER

  • organization_id (str) – Organization id

  • organization_name (str) – Organization name

  • organization (entities.Organization) – Organization object

Returns:

True if successful or error if unsuccessful

Return type:

bool

Example:

success = dl.organizations.add_member(email='user@domain.com',
                            organization_id='organization_id',
                            role=dl.MemberOrgRole.MEMBER)
cache_action(organization_id: Optional[str] = None, organization_name: Optional[str] = None, organization: Optional[Organization] = None, mode=CacheAction.APPLY, pod_type=PodType.SMALL)[source]

Add or remove Cache for the org

Prerequisites: You must be an organization owner

You must provide at least ONE of the following params: organization, organization_name, or organization_id.

Parameters:

  • organization_id (str) – Organization id

  • organization_name (str) – Organization name

  • organization (entities.Organization) – Organization object

  • mode (str) – dl.CacheAction.APPLY or dl.CacheAction.DESTROY

  • pod_type (entities.PodType) – dl.PodType.SMALL, dl.PodType.MEDIUM, dl.PodType.HIGH

Returns:

True if success

Return type:

bool

Example:

success = dl.organizations.enable_cache(organization_id='organization_id',
                              mode=dl.CacheAction.APPLY)
delete_member(user_id: str, organization_id: Optional[str] = None, organization_name: Optional[str] = None, organization: Optional[Organization] = None, sure: bool = False, really: bool = False) → bool[source]

Delete member from the Organization.

Prerequisites: Must be an organization owner to delete members.

You must provide at least ONE of the following params: organization_id, organization_name, organization.

Parameters:

  • user_id (str) – user id

  • organization_id (str) – Organization id

  • organization_name (str) – Organization name

  • organization (entities.Organization) – Organization object

  • sure (bool) – Are you sure you want to delete?

  • really (bool) – Really really sure?

Returns:

True if success and error if not

Return type:

bool

Example:

success = dl.organizations.delete_member(user_id='user_id',
                                organization_id='organization_id',
                                sure=True,
                                really=True)
get(organization_id: Optional[str] = None, organization_name: Optional[str] = None, fetch: Optional[bool] = None) → Organization[source]

Get Organization object to be able to use it in your code.

Prerequisites: You must be a superuser to use this method.

You must provide at least ONE of the following params: organization_name or organization_id.

Parameters:

  • organization_id (str) – optional - search by id

  • organization_name (str) – optional - search by name

  • fetch – optional - fetch entity from platform, default taken from cookie

Returns:

Organization object

Return type:

dtlpy.entities.organization.Organization

Example:

org = dl.organizations.get(organization_id='organization_id')
list() → List[Organization][source]

Lists all the organizations in Dataloop.

Prerequisites: You must be a superuser to use this method.

Returns:

List of Organization objects

Return type:

list

Example:

organizations = dl.organizations.list()
list_groups(organization: Optional[Organization] = None, organization_id: Optional[str] = None, organization_name: Optional[str] = None)[source]

List all organization groups (groups that were created within the organization).

Prerequisites: You must be an organization owner to use this method.

You must provide at least ONE of the following params: organization, organization_name, or organization_id.

Parameters:

  • organization (entities.Organization) – Organization object

  • organization_id (str) – Organization id

  • organization_name (str) – Organization name

Returns:

groups list

Return type:

list

Example:

groups_list = dl.organizations.list_groups(organization_id='organization_id')
list_integrations(organization: Optional[Organization] = None, organization_id: Optional[str] = None, organization_name: Optional[str] = None, only_available=False)[source]

List all organization integrations with external cloud storage.

Prerequisites: You must be an organization owner to use this method.

You must provide at least ONE of the following params: organization_id, organization_name, or organization.

Parameters:

  • organization (entities.Organization) – Organization object

  • organization_id (str) – Organization id

  • organization_name (str) – Organization name

  • only_available (bool) – if True list only the available integrations

Returns:

integrations list

Return type:

list

Example:

list_integrations = dl.organizations.list_integrations(organization='organization-entity',
                                    only_available=True)
list_members(organization: Optional[Organization] = None, organization_id: Optional[str] = None, organization_name: Optional[str] = None, role: Optional[MemberOrgRole] = None)[source]

List all organization members.

Prerequisites: You must be an organization owner to use this method.

You must provide at least ONE of the following params: organization_id, organization_name, or organization.

Parameters:

  • organization (entities.Organization) – Organization object

  • organization_id (str) – Organization id

  • organization_name (str) – Organization name

  • role (entities.MemberOrgRole) – MemberOrgRole.ADMIN, MemberOrgRole.OWNER, MemberOrgRole.MEMBER, MemberOrgRole.WORKER

Returns:

projects list

Return type:

list

Example:

list_members = dl.organizations.list_members(organization='organization-entity',
                            role=dl.MemberOrgRole.MEMBER)
update(plan: str, organization: Optional[Organization] = None, organization_id: Optional[str] = None, organization_name: Optional[str] = None) → Organization[source]

Update an organization.

Prerequisites: You must be a superuser to update an organization.

You must provide at least ONE of the following params: organization, organization_name, or organization_id.

Parameters:

  • plan (str) – OrganizationsPlans.FREEMIUM, OrganizationsPlans.PREMIUM

  • organization (entities.Organization) – Organization object

  • organization_id (str) – Organization id

  • organization_name (str) – Organization name

Returns:

organization object

Return type:

dtlpy.entities.organization.Organization

Example:

org = dl.organizations.update(organization='organization-entity',
                        plan=dl.OrganizationsPlans.FREEMIUM)
update_member(email: str, role: MemberOrgRole = MemberOrgRole.MEMBER, organization_id: Optional[str] = None, organization_name: Optional[str] = None, organization: Optional[Organization] = None)[source]

Update member role.

Prerequisites: You must be an organization owner to update a member’s role.

You must provide at least ONE of the following params: organization, organization_name, or organization_id.

Parameters:

  • email (str) – the member’s email

  • role (str) – MemberOrgRole.ADMIN, MemberOrgRole.OWNER, MemberOrgRole.MEMBER, MemberOrgRole.WORKER

  • organization_id (str) – Organization id

  • organization_name (str) – Organization name

  • organization (entities.Organization) – Organization object

Returns:

json of the member fields

Return type:

dict

Example:

member_json = dl.organizations.update_member(email='user@domain.com',
                                organization_id='organization_id',
                                 role=dl.MemberOrgRole.MEMBER)

Integrations

Integrations Repository

class Integrations(client_api: ApiClient, org: Optional[Organization] = None, project: Optional[Project] = None)[source]

Bases: object

Integrations Repository

The Integrations class allows you to manage data integrations from your external storage (e.g., S3, GCS, Azure) into your Dataloop’s Dataset storage, as well as sync data in your Dataloop’s Datasets with data in your external storage.

For more information on Organization Storage Integration see the Dataloop documentation and developers’ docs.

create(integrations_type: IntegrationType, name: str, options: dict)[source]

Create an integration between an external storage and the organization.

Examples for options include: s3 - {key: “”, secret: “”}; gcs - {key: “”, secret: “”, content: “”}; azureblob - {key: “”, secret: “”, clientId: “”, tenantId: “”}; key_value - {key: “”, value: “”} aws-sts - {key: “”, secret: “”, roleArns: “”} aws-cross - {} gcp-cross - {}

Prerequisites: You must be an owner in the organization.

Parameters:

  • integrations_type (IntegrationType) – integrations type dl.IntegrationType

  • name (str) – integrations name

  • options (dict) – dict of storage secrets

Returns:

success

Return type:

bool

Example:

project.integrations.create(integrations_type=dl.IntegrationType.S3,
                name='S3ntegration',
                options={key: "Access key ID", secret: "Secret access key"})
delete(integrations_id: str, sure: bool = False, really: bool = False) → bool[source]

Delete integrations from the organization.

Prerequisites: You must be an organization owner to delete an integration.

Parameters:

  • integrations_id (str) – integrations id

  • sure (bool) – Are you sure you want to delete?

  • really (bool) – Really really sure?

Returns:

success

Return type:

bool

Example:

project.integrations.delete(integrations_id='integrations_id', sure=True, really=True)
get(integrations_id: str)[source]

Get organization integrations. Use this method to access your integration and be able to use it in your code.

Prerequisites: You must be an owner in the organization.

Parameters:

integrations_id (str) – integrations id

Returns:

Integration object

Return type:

dtlpy.entities.integration.Integration

Example:

project.integrations.get(integrations_id='integrations_id')
list(only_available=False)[source]

List all the organization’s integrations with external storage.

Prerequisites: You must be an owner in the organization.

Parameters:

only_available (bool) – if True list only the available integrations.

Returns:

groups list

Return type:

list

Example:

project.integrations.list(only_available=True)
update(new_name: Optional[str] = None, integrations_id: Optional[str] = None, integration: Optional[Integration] = None, new_options: Optional[dict] = None)[source]

Update the integration’s name.

Prerequisites: You must be an owner in the organization.

Parameters:

  • new_name (str) – new name

  • integrations_id (str) – integrations id

  • integration (Integration) – integration object

  • new_options (dict) – new value

Returns:

Integration object

Return type:

dtlpy.entities.integration.Integration

Examples for options include: s3 - {key: “”, secret: “”}; gcs - {key: “”, secret: “”, content: “”}; azureblob - {key: “”, secret: “”, clientId: “”, tenantId: “”}; key_value - {key: “”, value: “”} aws-sts - {key: “”, secret: “”, roleArns: “”} aws-cross - {roleArn: “”} gcp-cross - {“email: “”, “resourceName”: “”}

Example:

project.integrations.update(integrations_id='integrations_id', new_options={roleArn: ""})

Projects

class Projects(client_api: ApiClient, org=None)[source]

Bases: object

Projects Repository

The Projects class allows the user to manage projects and their properties.

For more information on Projects see the Dataloop documentation.

add_member(email: str, project_id: str, role: MemberRole = MemberRole.DEVELOPER)[source]

Add a member to the project.

Prerequisites: You must be in the role of an owner to add a member to a project.

Parameters:

  • email (str) – member email

  • project_id (str) – The Id of the project

  • role – The required role for the user. Use the enum dl.MemberRole

Returns:

dict that represent the user

Return type:

dict

Example:

user_json = dl.projects.add_member(project_id='project_id', email='user@dataloop.ai', role=dl.MemberRole.DEVELOPER)
checkout(identifier: Optional[str] = None, project_name: Optional[str] = None, project_id: Optional[str] = None, project: Optional[Project] = None)[source]

Checkout (switch) to a project to work on.

Prerequisites: All users can open a project in the web.

You must provide at least ONE of the following params: project_id, project_name.

Parameters:

  • identifier (str) – project name or partial id that you wish to switch

  • project_name (str) – The Name of the project

  • project_id (str) – The Id of the project

  • project (dtlpy.entities.project.Project) – project object

Example:

dl.projects.checkout(project_id='project_id')
create(project_name: str, checkout: bool = False) → Project[source]

Create a new project.

Prerequisites: Any user can create a project.

Parameters:

  • project_name (str) – The Name of the project

  • checkout (bool) – set the project as a default project object (cookies)

Returns:

Project object

Return type:

dtlpy.entities.project.Project

Example:

project = dl.projects.create(project_name='project_name')
delete(project_name: Optional[str] = None, project_id: Optional[str] = None, sure: bool = False, really: bool = False) → bool[source]

Delete a project forever!

Prerequisites: You must be in the role of an owner to delete a project.

Parameters:

  • project_name (str) – optional - search by name

  • project_id (str) – optional - search by id

  • sure (bool) – Are you sure you want to delete?

  • really (bool) – Really really sure?

Returns:

True if success, error if not

Return type:

bool

Example:

is_deleted = dl.projects.delete(project_id='project_id', sure=True, really=True)
get(project_name: Optional[str] = None, project_id: Optional[str] = None, checkout: bool = False, fetch: Optional[bool] = None, log_error=True) → Project[source]

Get a Project object.

Prerequisites: You must be in the role of an owner to get a project object.

You must check out to a project or provide at least one of the following params: project_id, project_name

Parameters:

  • project_name (str) – optional - search by name

  • project_id (str) – optional - search by id

  • checkout (bool) – set the project as a default project object (cookies)

  • fetch (bool) – optional - fetch entity from platform (True), default taken from cookie

  • log_error (bool) – optional - show the logs errors

Returns:

Project object

Return type:

dtlpy.entities.project.Project

Example:

project = dl.projects.get(project_id='project_id')
list() → List[Project][source]

Get the user’s project list

Prerequisites: You must be a superuser to list all users’ projects.

Returns:

List of Project objects

Example:

projects = dl.projects.list()
list_members(project: Project, role: Optional[MemberRole] = None)[source]

Get a list of the project members.

Prerequisites: You must be in the role of an owner to list project members.

Parameters:
Returns:

list of the project members

Return type:

list

Example:

users_jsons_list = dl.projects.list_members(project_id='project_id', role=dl.MemberRole.DEVELOPER)
open_in_web(project_name: Optional[str] = None, project_id: Optional[str] = None, project: Optional[Project] = None)[source]

Open the project in our web platform.

Prerequisites: All users can open a project in the web.

Parameters:

Example:

dl.projects.open_in_web(project_id='project_id')
remove_member(email: str, project_id: str)[source]

Remove a member from the project.

Prerequisites: You must be in the role of an owner to delete a member from a project.

Parameters:

  • email (str) – member email

  • project_id (str) – The Id of the project

Returns:

dict that represents the user

Return type:

dict

Example:

user_json = dl.projects.remove_member(project_id='project_id', email='user@dataloop.ai')
update(project: Project, system_metadata: bool = False) → Project[source]

Update a project information (e.g., name, member roles, etc.).

Prerequisites: You must be in the role of an owner to add a member to a project.

Parameters:
Returns:

Project object

Return type:

dtlpy.entities.project.Project

Example:

project = dl.projects.delete(project='project_entity')
update_member(email: str, project_id: str, role: MemberRole = MemberRole.DEVELOPER)[source]

Update member’s information/details in the project.

Prerequisites: You must be in the role of an owner to update a member.

Parameters:

  • email (str) – member email

  • project_id (str) – The Id of the project

  • role – The required role for the user. Use the enum dl.MemberRole

Returns:

dict that represent the user

Return type:

dict

Example:

user_json = = dl.projects.update_member(project_id='project_id', email='user@dataloop.ai', role=dl.MemberRole.DEVELOPER)

Datasets

Datasets Repository

class Datasets(client_api: ApiClient, project: Optional[Project] = None)[source]

Bases: object

Datasets Repository

The Datasets class allows the user to manage datasets. Read more about datasets in our documentation and SDK documentation.

checkout(identifier: Optional[str] = None, dataset_name: Optional[str] = None, dataset_id: Optional[str] = None, dataset: Optional[Dataset] = None)[source]

Checkout (switch) to a dataset to work on it.

Prerequisites: You must be an owner or developer to use this method.

You must provide at least ONE of the following params: dataset_id, dataset_name.

Parameters:

  • identifier (str) – project name or partial id that you wish to switch

  • dataset_name (str) – The Name of the dataset

  • dataset_id (str) – The Id of the dataset

  • dataset (dtlpy.entities.dataset.Dataset) – dataset object

Example:

project.datasets.checkout(dataset_id='dataset_id')
clone(dataset_id: str, clone_name: str, filters: Optional[Filters] = None, with_items_annotations: bool = True, with_metadata: bool = True, with_task_annotations_status: bool = True)[source]

Clone a dataset. Read more about cloning datatsets and items in our documentation and SDK documentation.

Prerequisites: You must be in the role of an owner or developer.

Parameters:

  • dataset_id (str) – id of the dataset you wish to clone

  • clone_name (str) – new dataset name

  • filters (dtlpy.entities.filters.Filters) – Filters entity or a query dict

  • with_items_annotations (bool) – true to clone with items annotations

  • with_metadata (bool) – true to clone with metadata

  • with_task_annotations_status (bool) – true to clone with task annotations’ status

Returns:

dataset object

Return type:

dtlpy.entities.dataset.Dataset

Example:

dataset = project.datasets.clone(dataset_id='dataset_id',
                      clone_name='dataset_clone_name',
                      with_metadata=True,
                      with_items_annotations=False,
                      with_task_annotations_status=False)
create(dataset_name: str, labels=None, attributes=None, ontology_ids=None, driver: Optional[Driver] = None, driver_id: Optional[str] = None, checkout: bool = False, expiration_options: Optional[ExpirationOptions] = None, index_driver: Optional[IndexDriver] = None, recipe_id: Optional[str] = None) → Dataset[source]

Create a new dataset

Prerequisites: You must be in the role of an owner or developer.

Parameters:

  • dataset_name (str) – The Name of the dataset

  • labels (list) – dictionary of {tag: color} or list of label entities

  • attributes (list) – dataset’s ontology’s attributes

  • ontology_ids (list) – optional - dataset ontology

  • driver (dtlpy.entities.driver.Driver) – optional - storage driver Driver object or driver name

  • driver_id (str) – optional - driver id

  • checkout (bool) – set the dataset as a default dataset object (cookies)

  • expiration_options (ExpirationOptions) – dl.ExpirationOptions object that contain definitions for dataset like MaxItemDays

  • index_driver (str) – dl.IndexDriver, dataset driver version

  • recipe_id (str) – optional - recipe id

Returns:

Dataset object

Return type:

dtlpy.entities.dataset.Dataset

Example:

dataset = project.datasets.create(dataset_name='dataset_name', ontology_ids='ontology_ids')
delete(dataset_name: Optional[str] = None, dataset_id: Optional[str] = None, sure: bool = False, really: bool = False)[source]

Delete a dataset forever!

Prerequisites: You must be an owner or developer to use this method.

Example:

is_deleted = project.datasets.delete(dataset_id='dataset_id', sure=True, really=True)
Parameters:

  • dataset_name (str) – optional - search by name

  • dataset_id (str) – optional - search by id

  • sure (bool) – Are you sure you want to delete?

  • really (bool) – Really really sure?

Returns:

True is success

Return type:

bool

directory_tree(dataset: Optional[Dataset] = None, dataset_name: Optional[str] = None, dataset_id: Optional[str] = None)[source]

Get dataset’s directory tree.

Prerequisites: You must be an owner or developer to use this method.

You must provide at least ONE of the following params: dataset, dataset_name, dataset_id.

Parameters:
Returns:

DirectoryTree

Example:

static download_annotations(dataset: Dataset, local_path: Optional[str] = None, filters: Optional[Filters] = None, annotation_options: Optional[ViewAnnotationOptions] = None, annotation_filters: Optional[Filters] = None, overwrite: bool = False, thickness: int = 1, with_text: bool = False, remote_path: Optional[str] = None, include_annotations_in_output: bool = True, export_png_files: bool = False, filter_output_annotations: bool = False, alpha: Optional[float] = None, export_version=ExportVersion.V1) → str[source]

Download dataset’s annotations by filters.

You may filter the dataset both for items and for annotations and download annotations.

Optional – download annotations as: mask, instance, image mask of the item.

Prerequisites: You must be in the role of an owner or developer.

Parameters:

  • dataset (dtlpy.entities.dataset.Dataset) – dataset object

  • local_path (str) – local folder or filename to save to.

  • filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters

  • annotation_options (list) – type of download annotations: list(dl.ViewAnnotationOptions)

  • annotation_filters (dtlpy.entities.filters.Filters) – Filters entity to filter annotations for download

  • overwrite (bool) – optional - default = False to overwrite the existing files

  • thickness (int) – optional - line thickness, if -1 annotation will be filled, default =1

  • with_text (bool) – optional - add text to annotations, default = False

  • remote_path (str) – DEPRECATED and ignored

  • include_annotations_in_output (bool) – default - False , if export should contain annotations

  • export_png_files (bool) – default - if True, semantic annotations should be exported as png files

  • filter_output_annotations (bool) – default - False, given an export by filter - determine if to filter out annotations

  • alpha (float) – opacity value [0 1], default 1

  • export_version (str) – exported items will have original extension in filename, V1 - no original extension in filenames

Returns:

local_path of the directory where all the downloaded item

Return type:

str

Example:

file_path = project.datasets.download_annotations(dataset='dataset_entity',
                                     local_path='local_path',
                                     annotation_options=dl.ViewAnnotationOptions,
                                     overwrite=False,
                                     thickness=1,
                                     with_text=False,
                                     alpha=1
                                     )
get(dataset_name: Optional[str] = None, dataset_id: Optional[str] = None, checkout: bool = False, fetch: Optional[bool] = None) → Dataset[source]

Get dataset by name or id.

Prerequisites: You must be an owner or developer to use this method.

You must provide at least ONE of the following params: dataset_id, dataset_name.

Parameters:

  • dataset_name (str) – optional - search by name

  • dataset_id (str) – optional - search by id

  • checkout (bool) – set the dataset as a default dataset object (cookies)

  • fetch (bool) – optional - fetch entity from platform (True), default taken from cookie

Returns:

Dataset object

Return type:

dtlpy.entities.dataset.Dataset

Example:

dataset = project.datasets.get(dataset_id='dataset_id')
list(name=None, creator=None) → List[Dataset][source]

List all datasets.

Prerequisites: You must be an owner or developer to use this method.

Parameters:

  • name (str) – list by name

  • creator (str) – list by creator

Returns:

List of datasets

Return type:

list

Example:

datasets = project.datasets.list(name='name')
merge(merge_name: str, dataset_ids: list, project_ids: str, with_items_annotations: bool = True, with_metadata: bool = True, with_task_annotations_status: bool = True, wait: bool = True)[source]

Merge a dataset. See our SDK docs for more information.

Prerequisites: You must be an owner or developer to use this method.

Parameters:

  • merge_name (str) – new dataset name

  • dataset_ids (list) – list id’s of the datatsets you wish to merge

  • project_ids (str) – the project id that include the datasets

  • with_items_annotations (bool) – true to merge with items annotations

  • with_metadata (bool) – true to merge with metadata

  • with_task_annotations_status (bool) – true to merge with task annotations’ status

  • wait (bool) – wait for the command to finish

Returns:

True if success

Return type:

bool

Example:

success = project.datasets.merge(dataset_ids=['dataset_id1','dataset_id2'],
                      merge_name='dataset_merge_name',
                      with_metadata=True,
                      with_items_annotations=False,
                      with_task_annotations_status=False)
open_in_web(dataset_name: Optional[str] = None, dataset_id: Optional[str] = None, dataset: Optional[Dataset] = None)[source]

Open the dataset in web platform.

Prerequisites: You must be an owner or developer to use this method.

Parameters:

Example:

project.datasets.open_in_web(dataset_id='dataset_id')
set_readonly(state: bool, dataset: Dataset)[source]

Set dataset readonly mode.

Prerequisites: You must be in the role of an owner or developer.

Parameters:

Example:

project.datasets.set_readonly(dataset='dataset_entity', state=True)
sync(dataset_id: str, wait: bool = True)[source]

Sync dataset with external storage.

Prerequisites: You must be in the role of an owner or developer.

Parameters:

  • dataset_id (str) – The Id of the dataset to sync

  • wait (bool) – wait for the command to finish

Returns:

True if success

Return type:

bool

Example:

success = project.datasets.sync(dataset_id='dataset_id')
update(dataset: Dataset, system_metadata: bool = False, patch: Optional[dict] = None) → Dataset[source]

Update dataset field.

Prerequisites: You must be an owner or developer to use this method.

Parameters:
Returns:

Dataset object

Return type:

dtlpy.entities.dataset.Dataset

Example:

dataset = project.datasets.update(dataset='dataset_entity')
upload_annotations(dataset, local_path, filters: Optional[Filters] = None, clean=False, remote_root_path='/', export_version=ExportVersion.V1)[source]

Upload annotations to dataset.

Example for remote_root_path: If the item filepath is a/b/item and remote_root_path is /a the start folder will be b instead of a

Prerequisites: You must have a dataset with items that are related to the annotations. The relationship between the dataset and annotations is shown in the name. You must be in the role of an owner or developer.

Parameters:

  • dataset (dtlpy.entities.dataset.Dataset) – dataset to upload to

  • local_path (str) – str - local folder where the annotations files are

  • filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters

  • clean (bool) – True to remove the old annotations

  • remote_root_path (str) – the remote root path to match remote and local items

  • export_version (str) – exported items will have original extension in filename, V1 - no original extension in filenames

Example:

project.datasets.upload_annotations(dataset='dataset_entity',
                                     local_path='local_path',
                                     clean=False,
                                     export_version=dl.ExportVersion.V1
                                     )

Drivers

class Drivers(client_api: ApiClient, project: Optional[Project] = None)[source]

Bases: object

Drivers Repository

The Drivers class allows users to manage drivers that are used to connect with external storage. Read more about external storage in our documentation and developers’ docs.

create(name: str, driver_type: ExternalStorage, integration_id: str, bucket_name: str, integration_type: IntegrationType, project_id: Optional[str] = None, allow_external_delete: bool = True, region: Optional[str] = None, storage_class: str = '', path: str = '')[source]

Create a storage driver.

Prerequisites: You must be in the role of an owner or developer.

Parameters:

  • name (str) – the driver name

  • driver_type (ExternalStorage) – dl.ExternalStorage (Enum). For all options run: list(dl.ExsternalStorage)

  • integration_id (str) – the integration id

  • bucket_name (str) – the external bucket name

  • integration_type (IntegrationType) – dl.IntegrationType (Enum). For all options run: list(dl.IntegrationType)

  • project_id (str) – project id

  • allow_external_delete (bool) – true to allow deleting files from external storage when files are deleted in your Dataloop storage

  • region (str) – relevant only for s3 - the bucket region

  • storage_class (str) – relevant only for s3

  • path (str) – Optional. By default path is the root folder. Path is case sensitive integration

Returns:

driver object

Return type:

dtlpy.entities.driver.Driver

Example:

project.drivers.create(name='driver_name',
           driver_type=dl.ExternalStorage.S3,
           integration_id='integration_id',
           bucket_name='bucket_name',
           project_id='project_id',
           region='ey-west-1')
delete(driver_name: Optional[str] = None, driver_id: Optional[str] = None, sure: bool = False, really: bool = False)[source]

Delete a driver forever!

Prerequisites: You must be an owner or developer to use this method.

Example:

project.drivers.delete(dataset_id='dataset_id', sure=True, really=True)
Parameters:

  • driver_name (str) – optional - search by name

  • driver_id (str) – optional - search by id

  • sure (bool) – Are you sure you want to delete?

  • really (bool) – Really really sure?

Returns:

True if success

Return type:

bool

get(driver_name: Optional[str] = None, driver_id: Optional[str] = None) → Driver[source]

Get a Driver object to use in your code.

Prerequisites: You must be in the role of an owner or developer.

You must provide at least ONE of the following params: driver_name, driver_id.

Parameters:

  • driver_name (str) – optional - search by name

  • driver_id (str) – optional - search by id

Returns:

Driver object

Return type:

dtlpy.entities.driver.Driver

Example:

project.drivers.get(driver_id='driver_id')
list() → List[Driver][source]

Get the project’s drivers list.

Prerequisites: You must be in the role of an owner or developer.

Returns:

List of Drivers objects

Return type:

list

Example:

project.drivers.list()

Items

class Items(client_api: ApiClient, datasets: Optional[Datasets] = None, dataset: Optional[Dataset] = None, dataset_id=None, items_entity=None, project=None)[source]

Bases: object

Items Repository

The Items class allows you to manage items in your datasets. For information on actions related to items see https://developers.dataloop.ai/tutorials/data_management/upload_and_manage_items/chapter/

clone(item_id: str, dst_dataset_id: str, remote_filepath: Optional[str] = None, metadata: Optional[dict] = None, with_annotations: bool = True, with_metadata: bool = True, with_task_annotations_status: bool = False, allow_many: bool = False, wait: bool = True)[source]

Clone item. Read more about cloning datatsets and items in our documentation and SDK documentation.

Prerequisites: You must be in the role of an owner or developer.

Parameters:

  • item_id (str) – item to clone

  • dst_dataset_id (str) – destination dataset id

  • remote_filepath (str) – complete filepath

  • metadata (dict) – new metadata to add

  • with_annotations (bool) – clone annotations

  • with_metadata (bool) – clone metadata

  • with_task_annotations_status (bool) – clone task annotations status

  • allow_many (bool) – bool if True, using multiple clones in single dataset is allowed, (default=False)

  • wait (bool) – wait for the command to finish

Returns:

Item object

Return type:

dtlpy.entities.item.Item

Example:

dataset.items.clone(item_id='item_id',
        dst_dataset_id='dist_dataset_id',
        with_metadata=True,
        with_task_annotations_status=False,
        with_annotations=False)
delete(filename: Optional[str] = None, item_id: Optional[str] = None, filters: Optional[Filters] = None)[source]

Delete item from platform.

Prerequisites: You must be in the role of an owner or developer.

You must provide at least ONE of the following params: item id, filename, filters.

Parameters:

  • filename (str) – optional - search item by remote path

  • item_id (str) – optional - search item by id

  • filters (dtlpy.entities.filters.Filters) – optional - delete items by filter

Returns:

True if success

Return type:

bool

Example:

dataset.items.delete(item_id='item_id')
download(filters: Optional[Filters] = None, items=None, local_path: Optional[str] = None, file_types: Optional[list] = None, save_locally: bool = True, to_array: bool = False, annotation_options: Optional[ViewAnnotationOptions] = None, annotation_filters: Optional[Filters] = None, overwrite: bool = False, to_items_folder: bool = True, thickness: int = 1, with_text: bool = False, without_relative_path=None, avoid_unnecessary_annotation_download: bool = False, include_annotations_in_output: bool = True, export_png_files: bool = False, filter_output_annotations: bool = False, alpha: float = 1, export_version=ExportVersion.V1)[source]

Download dataset items by filters.

Filters the dataset for items and saves them locally.

Optional – download annotation, mask, instance, and image mask of the item.

Prerequisites: You must be in the role of an owner or developer.

Parameters:

  • filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters

  • items (List[dtlpy.entities.item.Item] or dtlpy.entities.item.Item) – download Item entity or item_id (or a list of item)

  • local_path (str) – local folder or filename to save to.

  • file_types (list) – a list of file type to download. e.g [‘video/webm’, ‘video/mp4’, ‘image/jpeg’, ‘image/png’]

  • save_locally (bool) – bool. save to disk or return a buffer

  • to_array (bool) – returns Ndarray when True and local_path = False

  • annotation_options (list) – download annotations options: list(dl.ViewAnnotationOptions)

  • annotation_filters (dtlpy.entities.filters.Filters) – Filters entity to filter annotations for download

  • overwrite (bool) – optional - default = False

  • to_items_folder (bool) – Create ‘items’ folder and download items to it

  • thickness (int) – optional - line thickness, if -1 annotation will be filled, default =1

  • with_text (bool) – optional - add text to annotations, default = False

  • without_relative_path (bool) – bool - download items without the relative path from platform

  • avoid_unnecessary_annotation_download (bool) – default - False

  • include_annotations_in_output (bool) – default - False , if export should contain annotations

  • export_png_files (bool) – default - if True, semantic annotations should be exported as png files

  • filter_output_annotations (bool) – default - False, given an export by filter - determine if to filter out annotations

  • alpha (float) – opacity value [0 1], default 1

  • export_version (str) – exported items will have original extension in filename, V1 - no original extension in filenames

Returns:

generator of local_path per each downloaded item

Return type:

generator or single item

Example:

dataset.items.download(local_path='local_path',
                     annotation_options=dl.ViewAnnotationOptions,
                     overwrite=False,
                     thickness=1,
                     with_text=False,
                     alpha=1,
                     save_locally=True
                     )
get(filepath: Optional[str] = None, item_id: Optional[str] = None, fetch: Optional[bool] = None, is_dir: bool = False) → Item[source]

Get Item object

Prerequisites: You must be in the role of an owner or developer.

Parameters:

  • filepath (str) – optional - search by remote path

  • item_id (str) – optional - search by id

  • fetch (bool) – optional - fetch entity from platform, default taken from cookie

  • is_dir (bool) – True if you want to get an item from dir type

Returns:

Item object

Return type:

dtlpy.entities.item.Item

Example:

dataset.items.get(item_id='item_id')
get_all_items(filters: Optional[Filters] = None) → [<class 'dtlpy.entities.item.Item'>][source]

Get all items in dataset.

Prerequisites: You must be in the role of an owner or developer.

Parameters:

filters (dtlpy.entities.filters.Filters) – dl.Filters entity to filters items

Returns:

list of all items

Return type:

list

Example:

dataset.items.get_all_items()
list(filters: Optional[Filters] = None, page_offset: Optional[int] = None, page_size: Optional[int] = None) → PagedEntities[source]

List items in a dataset.

Prerequisites: You must be in the role of an owner or developer.

Parameters:
Returns:

Pages object

Return type:

dtlpy.entities.paged_entities.PagedEntities

Example:

dataset.items.list(page_offset=0, page_size=100)
make_dir(directory, dataset: Optional[Dataset] = None) → Item[source]

Create a directory in a dataset.

Prerequisites: All users.

Parameters:
Returns:

Item object

Return type:

dtlpy.entities.item.Item

Example:

dataset.items.make_dir(directory='directory_name')
move_items(destination: str, filters: Optional[Filters] = None, items=None, dataset: Optional[Dataset] = None) → bool[source]

Move items to another directory. If directory does not exist we will create it

Prerequisites: You must be in the role of an owner or developer.

Parameters:
Returns:

True if success

Return type:

bool

Example:

dataset.items.move_items(destination='directory_name')
open_in_web(filepath=None, item_id=None, item=None)[source]

Open the item in web platform

Prerequisites: You must be in the role of an owner or developer or be an annotation manager/annotator with access to that item through task.

Parameters:

Example:

dataset.items.open_in_web(item_id='item_id')
set_items_entity(entity)[source]

Set the item entity type to Artifact, Item, or Codebase.

Parameters:

entity (entities.Item, entities.Artifact, entities.Codebase) – entity type [entities.Item, entities.Artifact, entities.Codebase]

update(item: Optional[Item] = None, filters: Optional[Filters] = None, update_values=None, system_update_values=None, system_metadata: bool = False)[source]

Update item metadata.

Prerequisites: You must be in the role of an owner or developer.

You must provide at least ONE of the following params: update_values, system_update_values.

Parameters:

  • item (dtlpy.entities.item.Item) – Item object

  • filters (dtlpy.entities.filters.Filters) – optional update filtered items by given filter

  • update_values – optional field to be updated and new values

  • system_update_values – values in system metadata to be updated

  • system_metadata (bool) – True, if you want to update the metadata system

Returns:

Item object

Return type:

dtlpy.entities.item.Item

Example:

dataset.items.update(item='item_entity')
update_status(status: ItemStatus, items=None, item_ids=None, filters=None, dataset=None, clear=False)[source]

Update item status in task

Prerequisites: You must be in the role of an owner or developer or annotation manager who has been assigned a task with the item.

You must provide at least ONE of the following params: items, item_ids, filters.

Parameters:

Example:

dataset.items.update_status(item_ids='item_id', status=dl.ItemStatus.COMPLETED)
upload(local_path: str, local_annotations_path: ~typing.Optional[str] = None, remote_path: str = '/', remote_name: ~typing.Optional[str] = None, file_types: ~typing.Optional[~dtlpy.repositories.items.Items.list] = None, overwrite: bool = False, item_metadata: ~typing.Optional[dict] = None, output_entity=<class 'dtlpy.entities.item.Item'>, no_output: bool = False, export_version: str = ExportVersion.V1, item_description: ~typing.Optional[str] = None)[source]

Upload local file to dataset. Local filesystem will remain unchanged. If “*” at the end of local_path (e.g. “/images/*”) items will be uploaded without the head directory.

Prerequisites: Any user can upload items.

Parameters:

  • local_path (str) – list of local file, local folder, BufferIO, numpy.ndarray or url to upload

  • local_annotations_path (str) – path to dataloop format annotations json files.

  • remote_path (str) – remote path to save.

  • remote_name (str) – remote base name to save. when upload numpy.ndarray as local path, remote_name with .jpg or .png ext is mandatory

  • file_types (list) – list of file type to upload. e.g [‘.jpg’, ‘.png’]. default is all

  • item_metadata (dict) – metadata dict to upload to item or ExportMetadata option to export metadata from annotation file

  • overwrite (bool) – optional - default = False

  • output_entity – output type

  • no_output (bool) – do not return the items after upload

  • export_version (str) – exported items will have original extension in filename, V1 - no original extension in filenames

  • item_description (str) – add a string description to the uploaded item

Returns:

Output (generator/single item)

Return type:

generator or single item

Example:

dataset.items.upload(local_path='local_path',
                     local_annotations_path='local_annotations_path',
                     overwrite=True,
                     item_metadata={'Hellow': 'Word'}
                     )

Annotations

class Annotations(client_api: ApiClient, item=None, dataset=None, dataset_id=None)[source]

Bases: object

Annotations Repository

The Annotation class allows you to manage the annotations of data items. For information on annotations explore our documentation at: Classification SDK, Annotation Labels and Attributes, Show Video with Annotations.

builder()[source]

Create Annotation collection.

Prerequisites: You must have an item to be annotated. You must have the role of an owner or developer

or be assigned a task that includes that item as an annotation manager or annotator.

Returns:

Annotation collection object

Return type:

dtlpy.entities.annotation_collection.AnnotationCollection

Example:

annotation_collection = item.annotations.builder()
delete(annotation: Optional[Annotation] = None, annotation_id: Optional[str] = None, filters: Optional[Filters] = None) → bool[source]

Remove an annotation from item.

Prerequisites: You must have an item that has been annotated. You must have the role of an owner or developer or be assigned a task that includes that item as an annotation manager or annotator.

Parameters:
Returns:

True/False

Return type:

bool

Example:

is_deleted = item.annotations.delete(annotation_id='annotation_id')
download(filepath: str, annotation_format: ViewAnnotationOptions = ViewAnnotationOptions.JSON, img_filepath: Optional[str] = None, height: Optional[float] = None, width: Optional[float] = None, thickness: int = 1, with_text: bool = False, alpha: float = 1)[source]

Save annotation to file.

Prerequisites: You must have an item that has been annotated. You must have the role of an owner or developer or be assigned a task that includes that item as an annotation manager or annotator.

Parameters:

  • filepath (str) – Target download directory

  • annotation_format (str) – the format that want to download ,options: list(dl.ViewAnnotationOptions)

  • img_filepath (str) – img file path - needed for img_mask

  • height (float) – optional - image height

  • width (float) – optional - image width

  • thickness (int) – optional - line thickness, default=1

  • with_text (bool) – optional - draw annotation with text, default = False

  • alpha (float) – opacity value [0 1], default 1

Returns:

file path to where save the annotations

Return type:

str

Example:

file_path = item.annotations.download(
              filepath='file_path',
              annotation_format=dl.ViewAnnotationOptions.MASK,
              img_filepath='img_filepath',
              height=100,
              width=100,
              thickness=1,
              with_text=False,
              alpha=1)
get(annotation_id: str) → Annotation[source]

Get a single annotation.

Prerequisites: You must have an item that has been annotated. You must have the role of an owner or developer or be assigned a task that includes that item as an annotation manager or annotator.

Parameters:

annotation_id (str) – The id of the annotation

Returns:

Annotation object or None

Returns:

Annotation object or None

Return type:

dtlpy.entities.annotation.Annotation

Example:

annotation = item.annotations.get(annotation_id='annotation_id')
list(filters: Optional[Filters] = None, page_offset: Optional[int] = None, page_size: Optional[int] = None)[source]

List Annotations of a specific item. You must get the item first and then list the annotations with the desired filters.

Prerequisites: You must have an item that has been annotated. You must have the role of an owner or developer or be assigned a task that includes that item as an annotation manager or annotator.

Parameters:
Returns:

Pages object

Return type:

dtlpy.entities.paged_entities.PagedEntities

Example:

annotations = item.annotations.list(filters=dl.Filters(
                             resource=dl.FiltersResource.ANNOTATION,
                             field='type',
                             values='box'),
          page_size=100,
          page_offset=0)
show(image=None, thickness: int = 1, with_text: bool = False, height: Optional[float] = None, width: Optional[float] = None, annotation_format: ViewAnnotationOptions = ViewAnnotationOptions.MASK, alpha: float = 1)[source]

Show annotations. To use this method, you must get the item first and then show the annotations with the desired filters. The method returns an array showing all the annotations.

Prerequisites: You must have an item that has been annotated. You must have the role of an owner or developer or be assigned a task that includes that item as an annotation manager or annotator.

Parameters:

  • image (ndarray) – empty or image to draw on

  • thickness (int) – optional - line thickness, default=1

  • with_text (bool) – add label to annotation

  • height (float) – item height

  • width (float) – item width

  • annotation_format (str) – the format that want to show ,options: list(dl.ViewAnnotationOptions)

  • alpha (float) – opacity value [0 1], default 1

Returns:

ndarray of the annotations

Return type:

ndarray

Example:

image = item.annotations.show(image='nd array',
          thickness=1,
          with_text=False,
          height=100,
          width=100,
          annotation_format=dl.ViewAnnotationOptions.MASK,
          alpha=1)
update(annotations, system_metadata=False)[source]

Update an existing annotation. For example, you may change the annotation’s label and then use the update method.

Prerequisites: You must have an item that has been annotated. You must have the role of an owner or

developer or be assigned a task that includes that item as an annotation manager or annotator.

Parameters:
Returns:

True if successful or error if unsuccessful

Return type:

bool

Example:

annotations = item.annotations.update(annotation='annotation')
update_status(annotation: Optional[Annotation] = None, annotation_id: Optional[str] = None, status: AnnotationStatus = AnnotationStatus.ISSUE) → Annotation[source]

Set status on annotation.

Prerequisites: You must have an item that has been annotated. You must have the role of an owner or developer or be assigned a task that includes that item as an annotation manager.

Parameters:
Returns:

Annotation object

Return type:

dtlpy.entities.annotation.Annotation

Example:

annotation = item.annotations.update_status(annotation_id='annotation_id', status=dl.AnnotationStatus.ISSUE)
upload(annotations) → AnnotationCollection[source]

Upload a new annotation/annotations. You must first create the annotation using the annotation builder method.

Prerequisites: Any user can upload annotations.

Parameters:

annotations (List[dtlpy.entities.annotation.Annotation] or dtlpy.entities.annotation.Annotation) – list or

single annotation of type Annotation :return: list of annotation objects :rtype: entities.AnnotationCollection

Example:

annotations = item.annotations.upload(annotations='builder')

Recipes

class Recipes(client_api: ApiClient, dataset: Optional[Dataset] = None, project: Optional[Project] = None, project_id: Optional[str] = None)[source]

Bases: object

Recipes Repository

The Recipes class allows you to manage recipes and their properties. For more information on Recipes, see our documentation and developers’ documentation.

clone(recipe: Optional[Recipe] = None, recipe_id: Optional[str] = None, shallow: bool = False)[source]

Clone recipe.

Prerequisites: You must be in the role of an owner or developer.

Parameters:

  • recipe (dtlpy.entities.recipe.Recipe) – Recipe object

  • recipe_id (str) – Recipe id

  • shallow (bool) – If True, link to existing ontology, clones all ontologies that are linked to the recipe as well

Returns:

Cloned ontology object

Return type:

dtlpy.entities.recipe.Recipe

Example:

dataset.recipes.clone(recipe_id='recipe_id')
create(project_ids=None, ontology_ids=None, labels=None, recipe_name=None, attributes=None, annotation_instruction_file=None) → Recipe[source]

Create a new Recipe. Note: If the param ontology_ids is None, an ontology will be created first.

Prerequisites: You must be in the role of an owner or developer.

Parameters:

  • project_ids (str) – project ids

  • ontology_ids (str or list) – ontology ids

  • labels – labels

  • recipe_name (str) – recipe name

  • attributes – attributes

  • annotation_instruction_file (str) – file path or url of the recipe instruction

Returns:

Recipe entity

Return type:

dtlpy.entities.recipe.Recipe

Example:

dataset.recipes.create(recipe_name='My Recipe', labels=labels))
delete(recipe_id: str, force: bool = False)[source]

Delete recipe from platform.

Prerequisites: You must be in the role of an owner or developer.

Parameters:

  • recipe_id (str) – recipe id

  • force (bool) – force delete recipe

Returns:

True if success

Return type:

bool

Example:

dataset.recipes.delete(recipe_id='recipe_id')
get(recipe_id: str) → Recipe[source]

Get a Recipe object to use in your code.

Prerequisites: You must be in the role of an owner or developer.

Parameters:

recipe_id (str) – recipe id

Returns:

Recipe object

Return type:

dtlpy.entities.recipe.Recipe

Example:

dataset.recipes.get(recipe_id='recipe_id')
list(filters: Optional[Filters] = None) → List[Recipe][source]

List recipes for a dataset.

Prerequisites: You must be in the role of an owner or developer.

Parameters:

filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters

Returns:

list of all recipes

Retype:

list

Example:

dataset.recipes.list()
open_in_web(recipe: Optional[Recipe] = None, recipe_id: Optional[str] = None)[source]

Open the recipe in web platform.

Prerequisites: All users.

Parameters:

Example:

dataset.recipes.open_in_web(recipe_id='recipe_id')
update(recipe: Recipe, system_metadata=False) → Recipe[source]

Update recipe.

Prerequisites: You must be in the role of an owner or developer.

Parameters:
Returns:

Recipe object

Return type:

dtlpy.entities.recipe.Recipe

Example:

dataset.recipes.update(recipe='recipe_entity')

Ontologies

class Ontologies(client_api: ApiClient, recipe: Optional[Recipe] = None, project: Optional[Project] = None, dataset: Optional[Dataset] = None)[source]

Bases: object

Ontologies Repository

The Ontologies class allows users to manage ontologies and their properties. Read more about ontology in our developers docs.

create(labels, title=None, project_ids=None, attributes=None) → Ontology[source]

Create a new ontology.

Prerequisites: You must be in the role of an owner or developer.

Parameters:

  • labels – recipe tags

  • title (str) – ontology title, name

  • project_ids (list) – recipe project/s

  • attributes (list) – recipe attributes

Returns:

Ontology object

Return type:

dtlpy.entities.ontology.Ontology

Example:

recipe.ontologies.create(labels='labels_entity',
                      title='new_ontology',
                      project_ids='project_ids')
delete(ontology_id)[source]

Delete Ontology from the platform.

Prerequisites: You must be in the role of an owner or developer.

Parameters:

ontology_id – ontology id

Returns:

True if success

Return type:

bool

Example:

recipe.ontologies.delete(ontology_id='ontology_id')
delete_attributes(ontology_id, keys: list)[source]

Delete a bulk of attributes

Parameters:

  • ontology_id (str) – ontology id

  • keys (list) – Keys of attributes to delete

Returns:

True if success

Return type:

bool

Example:

ontology.delete_attributes(['1'])
get(ontology_id: str) → Ontology[source]

Get Ontology object to use in your code.

Prerequisites: You must be in the role of an owner or developer.

Parameters:

ontology_id (str) – ontology id

Returns:

Ontology object

Return type:

dtlpy.entities.ontology.Ontology

Example:

recipe.ontologies.get(ontology_id='ontology_id')
static labels_to_roots(labels)[source]

Converts labels dictionary to a list of platform representation of labels.

Parameters:

labels (dict) – labels dict

Returns:

platform representation of labels

list(project_ids=None) → List[Ontology][source]

List ontologies for recipe

Prerequisites: You must be in the role of an owner or developer.

Parameters:

project_ids –

Returns:

list of all the ontologies

Example:

recipe.ontologies.list(project_ids='project_ids')
update(ontology: Ontology, system_metadata=False) → Ontology[source]

Update the Ontology metadata.

Prerequisites: You must be in the role of an owner or developer.

Parameters:
Returns:

Ontology object

Return type:

dtlpy.entities.ontology.Ontology

Example:

recipe.ontologies.delete(ontology='ontology_entity')
update_attributes(ontology_id: str, title: str, key: str, attribute_type: AttributesTypes, scope: Optional[list] = None, optional: Optional[bool] = None, values: Optional[list] = None, attribute_range: Optional[AttributesRange] = None)[source]

ADD a new attribute or update if exist

Parameters:

  • ontology_id (str) – ontology_id

  • title (str) – attribute title

  • key (str) – the key of the attribute must br unique

  • attribute_type (AttributesTypes) – dl.AttributesTypes your attribute type

  • scope (list) – list of the labels or * for all labels

  • optional (bool) – optional attribute

  • values (list) – list of the attribute values ( for checkbox and radio button)

  • attribute_range (dict or AttributesRange) – dl.AttributesRange object

Returns:

true in success

Return type:

bool

Example:

ontology.update_attributes(key='1',
                           title='checkbox',
                           attribute_type=dl.AttributesTypes.CHECKBOX,
                           values=[1,2,3])

Tasks

class Tasks(client_api: ApiClient, project: Optional[Project] = None, dataset: Optional[Dataset] = None, project_id: Optional[str] = None)[source]

Bases: object

Tasks Repository

The Tasks class allows the user to manage tasks and their properties. For more information, read in our developers’ documentation about Creating Tasks, and Redistributing and Reassigning Tasks.

add_items(task: Optional[Task] = None, task_id=None, filters: Optional[Filters] = None, items=None, assignee_ids=None, query=None, workload=None, limit=None, wait=True) → Task[source]

Add items to a Task.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned to be owner of the annotation task.

Parameters:

  • task (dtlpy.entities.task.Task) – task object

  • task_id (str) – the Id of the task

  • filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters

  • items (list) – list of items (item Ids or objects) to add to the task

  • assignee_ids (list) – list to assignee who works in the task

  • query (dict) – query to filter the items for the task

  • workload (list) – list of WorkloadUnit objects. Customize distribution (percentage) between the task assignees. For example: [dl.WorkloadUnit(annotator@hi.com, 80), dl.WorkloadUnit(annotator2@hi.com, 20)]

  • limit (int) – the limit items that task can include

  • wait (bool) – wait until add items will to finish

Returns:

task entity

Return type:

dtlpy.entities.task.Task

Example:

dataset.tasks.add_items(task= 'task_entity',
                    items = [items])
create(task_name, due_date=None, assignee_ids=None, workload=None, dataset=None, task_owner=None, task_type='annotation', task_parent_id=None, project_id=None, recipe_id=None, assignments_ids=None, metadata=None, filters=None, items=None, query=None, available_actions=None, wait=True, check_if_exist: Filters = False, limit=None, batch_size=None, max_batch_workload=None, allowed_assignees=None, priority=TaskPriority.MEDIUM, consensus_task_type=None, consensus_percentage=None, consensus_assignees=None) → Task[source]

Create a new Task (Annotation or QA).

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned to be owner of the annotation task.

Parameters:

  • task_name (str) – the name of the task

  • due_date (float) – date by which the task should be finished; for example, due_date=datetime.datetime(day=1, month=1, year=2029).timestamp()

  • assignee_ids (list) – list the task assignees (contributors) that should be working on the task. Provide a list of users’ emails

  • workload (List[WorkloadUnit] List[WorkloadUnit]) – list of WorkloadUnit objects. Customize distribution (percentage) between the task assignees. For example: [dl.WorkloadUnit(annotator@hi.com, 80), dl.WorkloadUnit(annotator2@hi.com, 20)]

  • dataset (entities.Dataset) – dataset object, the dataset that refer to the task

  • task_owner (str) – task owner. Provide user email

  • task_type (str) – task type “annotation” or “qa”

  • task_parent_id (str) – optional if type is qa - parent annotation task id

  • project_id (str) – the Id of the project where task will be created

  • recipe_id (str) – recipe id for the task

  • assignments_ids (list) – assignments ids to the task

  • metadata (dict) – metadata for the task

  • filters (entities.Filters) – dl.Filters entity to filter items for the task

  • items (List[entities.Item]) – list of items (item Id or objects) to insert to the task

  • query (dict DQL) – filter items for the task

  • available_actions (list) – list of available actions (statuses) that will be available for the task items; The default statuses are: “completed” and “discard”

  • wait (bool) – wait until create task finish

  • check_if_exist (entities.Filters) – dl.Filters check if task exist according to filter

  • limit (int) – the limit items that the task can include

  • batch_size (int) – Pulling batch size (items), use with pulling allocation method. Restrictions - Min 3, max 100

  • max_batch_workload (int) – max_batch_workload: Max items in assignment, use with pulling allocation method. Restrictions - Min batchSize + 2, max batchSize * 2

  • allowed_assignees (list) – list the task assignees (contributors) that should be working on the task. Provide a list of users’ emails

  • priority (entities.TaskPriority) – priority of the task options in entities.TaskPriority

  • consensus_task_type (entities.ConsensusTaskType) – consensus_task_type of the task options in entities.ConsensusTaskType

  • consensus_percentage (int) – percentage of items to be copied to multiple annotators (consensus items)

  • consensus_assignees (int) – the number of different annotators per item (number of copies per item)

Returns:

Task object

Return type:

dtlpy.entities.task.Task

Example:

dataset.tasks.create(task= 'task_entity',
                    due_date = datetime.datetime(day= 1, month= 1, year= 2029).timestamp(),
                    assignee_ids =[ 'annotator1@dataloop.ai', 'annotator2@dataloop.ai'])
create_qa_task(task: Task, assignee_ids, due_date=None, filters=None, items=None, query=None, workload=None, metadata=None, available_actions=None, wait=True, batch_size=None, max_batch_workload=None, allowed_assignees=None, priority=TaskPriority.MEDIUM) → Task[source]

Create a new QA Task.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned to be owner of the annotation task.

Parameters:

  • task (dtlpy.entities.task.Task) – the parent annotation task object

  • assignee_ids (list) – list the QA task assignees (contributors) that should be working on the task. Provide a list of users’ emails

  • due_date (float) – date by which the QA task should be finished; for example, due_date=datetime.datetime(day=1, month=1, year=2029).timestamp()

  • filters (entities.Filters) – dl.Filters entity to filter items for the task

  • items (List[entities.Item]) – list of items (item Id or objects) to insert to the task

  • query (dict DQL) – filter items for the task

  • workload (List[WorkloadUnit]) – list of WorkloadUnit objects. Customize distribution (percentage) between the task assignees. For example: [dl.WorkloadUnit(annotator@hi.com, 80), dl.WorkloadUnit(annotator2@hi.com, 20)]

  • metadata (dict) – metadata for the task

  • available_actions (list) – list of available actions (statuses) that will be available for the task items; The default statuses are: “approved” and “discard”

  • wait (bool) – wait until create task finish

  • batch_size (int) – Pulling batch size (items), use with pulling allocation method. Restrictions - Min 3, max 100

  • max_batch_workload (int) – Max items in assignment, use with pulling allocation method. Restrictions - Min batchSize + 2, max batchSize * 2

  • allowed_assignees (list) – list the task assignees (contributors) that should be working on the task. Provide a list of users’ emails

  • priority (entities.TaskPriority) – priority of the task options in entities.TaskPriority

Returns:

task object

Return type:

dtlpy.entities.task.Task

Example:

dataset.tasks.create_qa_task(task= 'task_entity',
                            due_date = datetime.datetime(day= 1, month= 1, year= 2029).timestamp(),
                            assignee_ids =[ 'annotator1@dataloop.ai', 'annotator2@dataloop.ai'])
delete(task: Optional[Task] = None, task_name: Optional[str] = None, task_id: Optional[str] = None, wait: bool = True)[source]

Delete the Task.

Prerequisites: You must be in the role of an owner or developer or annotation manager who created that task.

Parameters:

  • task (dtlpy.entities.task.Task) – the task object

  • task_name (str) – the name of the task

  • task_id (str) – the Id of the task

  • wait (bool) – wait until delete task finish

Returns:

True is success

Return type:

bool

Example:

dataset.tasks.delete(task_id='task_id')
get(task_name=None, task_id=None) → Task[source]

Get a Task object to use in your code.

Prerequisites: You must be in the role of an owner or developer or annotation manager who has been assigned the task.

Parameters:

  • task_name (str) – optional - search by name

  • task_id (str) – optional - search by id

Returns:

task object

Return type:

dtlpy.entities.task.Task

Example:

dataset.tasks.get(task_id='task_id')
get_items(task_id: Optional[str] = None, task_name: Optional[str] = None, dataset: Optional[Dataset] = None, filters: Optional[Filters] = None, get_consensus_items: bool = False, task: Optional[Task] = None) → PagedEntities[source]

Get the task items to use in your code.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned to be owner of the annotation task.

If a filters param is provided, you will receive a PagedEntity output of the task items. If no filter is provided, you will receive a list of the items.

Parameters:

  • task_id (str) – the id of the task

  • task_name (str) – the name of the task

  • get_consensus_items (bool) – get the items from the consensus assignment

  • task (dtlpy.entities.Task) – task object

  • dataset (dtlpy.entities.dataset.Dataset) – dataset object that refer to the task

  • filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters

Returns:

list of the items or PagedEntity output of items

Return type:

list or dtlpy.entities.paged_entities.PagedEntities

Example:

dataset.tasks.get_items(task_id= 'task_id')
list(project_ids=None, status=None, task_name=None, pages_size=None, page_offset=None, recipe=None, creator=None, assignments=None, min_date=None, max_date=None, filters: Optional[Filters] = None) → Union[List[Task], PagedEntities][source]

List all tasks.

Prerequisites: You must be in the role of an owner or developer or annotation manager who has been assigned the task.

Parameters:

  • project_ids – search tasks by given list of project ids

  • status (str) – search tasks by a given task status

  • task_name (str) – search tasks by a given task name

  • pages_size (int) – pages size of the output generator

  • page_offset (int) – page offset of the output generator

  • recipe (dtlpy.entities.recipe.Recipe) – Search tasks that use a given recipe. Provide the required recipe object

  • creator (str) – search tasks created by a given creator (user email)

  • assignments (dtlpy.entities.assignment.Assignment recipe) – assignments object

  • min_date (double) – search all tasks created AFTER a given date, use a milliseconds format. For example: 1661780622008

  • max_date (double) – search all tasks created BEFORE a given date, use a milliseconds format. For example: 1661780622008

  • filters (dtlpy.entities.filters.Filters) – dl.Filters entity to filters tasks using DQL

Returns:

List of Task objects

Example:

dataset.tasks.list(project_ids='project_ids',pages_size=100, page_offset=0)
open_in_web(task_name: Optional[str] = None, task_id: Optional[str] = None, task: Optional[Task] = None)[source]

Open the task in the web platform.

Prerequisites: You must be in the role of an owner or developer or annotation manager who has been assigned the task.

Parameters:

Example:

dataset.tasks.open_in_web(task_id='task_id')
query(filters=None, project_ids=None)[source]

List all tasks by filter.

Prerequisites: You must be in the role of an owner or developer or annotation manager who has been assigned the task.

Parameters:

  • filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters

  • project_ids (list) – list of project ids of the required tasks

Returns:

Paged entity - task pages generator

Return type:

dtlpy.entities.paged_entities.PagedEntities

Example:

dataset.tasks.query(project_ids='project_ids')
remove_items(task: Optional[Task] = None, task_id=None, filters: Optional[Filters] = None, query=None, items=None, wait=True)[source]

remove items from Task.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned to be owner of the annotation task.

Parameters:
Returns:

True if success and an error if failed

Return type:

bool

Examples:

dataset.tasks.remove_items(task= 'task_entity',
                            items = [items])
set_status(status: str, operation: str, task_id: str, item_ids: List[str])[source]

Update an item status within a task.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned to be owner of the annotation task.

Parameters:

  • status (str) – string the describes the status

  • operation (str) – the status action need ‘create’ or ‘delete’

  • task_id (str) – the Id of the task

  • item_ids (list) – List[str] id items ids

Returns:

True if success

Return type:

bool

Example:

dataset.tasks.set_status(task_id= 'task_id', status='complete', operation='create')
update(task: Optional[Task] = None, system_metadata=False) → Task[source]

Update a Task.

Prerequisites: You must be in the role of an owner or developer or annotation manager who created that task.

Parameters:
Returns:

Task object

Return type:

dtlpy.entities.task.Task

Example:

dataset.tasks.update(task='task_entity')

Assignments

class Assignments(client_api: ApiClient, project: Optional[Project] = None, task: Optional[Task] = None, dataset: Optional[Dataset] = None, project_id=None)[source]

Bases: object

Assignments Repository

The Assignments class allows users to manage assignments and their properties. Read more about Task Assignment in our Developers documentation.

create(assignee_id: str, task: Optional[Task] = None, filters: Optional[Filters] = None, items: Optional[list] = None) → Assignment[source]

Create a new assignment.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned as owner of the annotation task.

Parameters:

  • assignee_id (str) – the email of the user that want to assign the assignment

  • task (dtlpy.entities.task.Task) – the task object that include the assignment

  • filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters

  • items (list) – list of items (item Id or objects) to insert to the assignment

Returns:

Assignment object

Return type:

dtlpy.entities.assignment.Assignment assignment

Example:

assignment = task.assignments.create(assignee_id='annotator1@dataloop.ai')
get(assignment_name: Optional[str] = None, assignment_id: Optional[str] = None)[source]

Get Assignment object to use it in your code.

Parameters:

  • assignment_name (str) – optional - search by name

  • assignment_id (str) – optional - search by id

Returns:

Assignment object

Return type:

dtlpy.entities.assignment.Assignment

Example:

assignment = task.assignments.get(assignment_id='assignment_id')
get_items(assignment: Optional[Assignment] = None, assignment_id=None, assignment_name=None, dataset=None, filters=None) → PagedEntities[source]

Get all the items in the assignment.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned as owner of the annotation task.

Parameters:
Returns:

pages of the items

Return type:

dtlpy.entities.paged_entities.PagedEntities

Example:

items = task.assignments.get_items(assignment_id='assignment_id')
list(project_ids: Optional[list] = None, status: Optional[str] = None, assignment_name: Optional[str] = None, assignee_id: Optional[str] = None, pages_size: Optional[int] = None, page_offset: Optional[int] = None, task_id: Optional[int] = None) → List[Assignment][source]

Get Assignment list to be able to use it in your code.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned as owner of the annotation task.

Parameters:

  • project_ids (list) – search assignment by given list of project ids

  • status (str) – search assignment by a given task status

  • assignment_name (str) – search assignment by a given assignment name

  • assignee_id (str) – the user email that assignee the assignment to it

  • pages_size (int) – pages size of the output generator

  • page_offset (int) – page offset of the output generator

  • task_id (str) – search assignment by given task id

Returns:

List of Assignment objects

Return type:

miscellaneous.List[dtlpy.entities.assignment.Assignment]

Example:

assignments = task.assignments.list(status='complete', assignee_id='user@dataloop.ai', pages_size=100, page_offset=0)
open_in_web(assignment_name: Optional[str] = None, assignment_id: Optional[str] = None, assignment: Optional[str] = None)[source]

Open the assignment in the platform.

Prerequisites: All users.

Parameters:

Example:

task.assignments.open_in_web(assignment_id='assignment_id')
reassign(assignee_id: str, assignment: Optional[Assignment] = None, assignment_id: Optional[str] = None, task: Optional[Task] = None, task_id: Optional[str] = None, wait: bool = True)[source]

Reassign an assignment.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned as owner of the annotation task.

Parameters:

  • assignee_id (str) – the email of the user that want to assign the assignment

  • assignment (dtlpy.entities.assignment.Assignment) – assignment object

  • assignment_id – the Id of the assignment

  • task (dtlpy.entities.task.Task) – task object

  • task_id (str) – the Id of the task that include the assignment

  • wait (bool) – wait until reassign assignment finish

Returns:

Assignment object

Return type:

dtlpy.entities.assignment.Assignment

Example:

assignment = task.assignments.reassign(assignee_ids='annotator1@dataloop.ai')
redistribute(workload: Workload, assignment: Optional[Assignment] = None, assignment_id: Optional[str] = None, task: Optional[Task] = None, task_id: Optional[str] = None, wait: bool = True)[source]

Redistribute an assignment.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned as owner of the annotation task.

Example:

Parameters:
Returns:

Assignment object

Return type:

dtlpy.entities.assignment.Assignment assignment

assignment = task.assignments.redistribute(workload=dl.Workload([dl.WorkloadUnit(assignee_id="annotator1@dataloop.ai", load=50),
                                                    dl.WorkloadUnit(assignee_id="annotator2@dataloop.ai", load=50)]))
set_status(status: str, operation: str, item_id: str, assignment_id: str) → bool[source]

Set item status within assignment.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned as owner of the annotation task.

Parameters:

  • status (str) – string the describes the status

  • operation (str) – the status action need ‘create’ or ‘delete’

  • item_id (str) – item id that want to set his status

  • assignment_id – the Id of the assignment

Returns:

True id success

Return type:

bool

Example:

success = task.assignments.set_status(assignment_id='assignment_id',
                            status='complete',
                            operation='created',
                            item_id='item_id')
update(assignment: Optional[Assignment] = None, system_metadata: bool = False) → Assignment[source]

Update an assignment.

Prerequisites: You must be in the role of an owner, developer, or annotation manager who has been assigned as owner of the annotation task.

Parameters:

  • assignment (dtlpy.entities.assignment.Assignment assignment) – assignment entity

  • system_metadata (bool) – True, if you want to change metadata system

Returns:

Assignment object

Return type:

dtlpy.entities.assignment.Assignment assignment

Example:

assignment = task.assignments.update(assignment='assignment_entity', system_metadata=False)

Packages

class LocalServiceRunner(client_api: ApiClient, packages, cwd=None, multithreading=False, concurrency=10, package: Optional[Package] = None, module_name='default_module', function_name='run', class_name='ServiceRunner', entry_point='main.py', mock_file_path=None)[source]

Bases: object

Service Runner Class

get_field(field_name, field_type, mock_json, project=None, mock_inputs=None)[source]

Get field in mock json.

Parameters:

  • field_name – field name

  • field_type – field type

  • mock_json – mock json

  • project – project

  • mock_inputs – mock inputs

Returns:

get_mainpy_run_service()[source]

Get mainpy run service

Returns:

run_local_project(project=None)[source]

Run local project

Parameters:

project – project entity

class Packages(client_api: ApiClient, project: Optional[Project] = None)[source]

Bases: object

Packages Repository

The Packages class allows users to manage packages (code used for running in Dataloop’s FaaS) and their properties. Read more about Packages.

build(package: Package, module_name=None, init_inputs=None, local_path=None, from_local=None)[source]

Instantiate a module from the package code. Returns a loaded instance of the runner class

Parameters:

  • package – Package entity

  • module_name – Name of the module to build the runner class

  • init_inputs (str) – dictionary of the class init variables (if exists). will be used to init the module class

  • local_path (str) – local path of the package (if from_local=False - codebase will be downloaded)

  • from_local (bool) – bool. if true - codebase will not be downloaded (only use local files)

Returns:

dl.BaseServiceRunner

build_requirements(filepath) → list[source]

Build a requirement list (list of packages your code requires to run) from a file path. The file listing the requirements MUST BE a txt file.

Prerequisites: You must be in the role of an owner or developer.

Parameters:

filepath – path of the requirements file

Returns:

a list of dl.PackageRequirement

Return type:

list

static build_trigger_dict(actions, name='default_module', filters=None, function='run', execution_mode: TriggerExecutionMode = 'Once', type_t: TriggerType = 'Event')[source]

Build a trigger dictionary to trigger FaaS. Read more about FaaS Triggers.

Prerequisites: You must be in the role of an owner or developer.

Parameters:

  • actions – list of dl.TriggerAction

  • name (str) – trigger name

  • filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters

  • function (str) – function name

  • execution_mode (str) – execution mode dl.TriggerExecutionMode

  • type_t (str) – trigger type dl.TriggerType

Returns:

trigger dict

Return type:

dict

Example:

project.packages.build_trigger_dict(actions=dl.TriggerAction.CREATED,
                                  function='run',
                                  execution_mode=dl.TriggerExecutionMode.ONCE)
static check_cls_arguments(cls, missing, function_name, function_inputs)[source]

Check class arguments. This method checks that the package function is correct.

Prerequisites: You must be in the role of an owner or developer.

Parameters:

  • cls – packages class

  • missing (list) – list of the missing params

  • function_name (str) – name of function

  • function_inputs (list) – list of function inputs

checkout(package: Optional[Package] = None, package_id: Optional[str] = None, package_name: Optional[str] = None)[source]

Checkout (switch) to a package.

Prerequisites: You must be in the role of an owner or developer.

Parameters:

Example:

project.packages.checkout(package='package_entity')
delete(package: Optional[Package] = None, package_name=None, package_id=None)[source]

Delete a Package object.

Prerequisites: You must be in the role of an owner or developer.

Parameters:
Returns:

True if success

Return type:

bool

Example:

project.packages.delete(package_name='package_name')
deploy(package_id: Optional[str] = None, package_name: Optional[str] = None, package: Optional[Package] = None, service_name: Optional[str] = None, project_id: Optional[str] = None, revision: Optional[str] = None, init_input: Optional[Union[List[FunctionIO], FunctionIO, dict]] = None, runtime: Optional[Union[KubernetesRuntime, dict]] = None, sdk_version: Optional[str] = None, agent_versions: Optional[dict] = None, bot: Optional[Union[Bot, str]] = None, pod_type: Optional[InstanceCatalog] = None, verify: bool = True, checkout: bool = False, module_name: Optional[str] = None, run_execution_as_process: Optional[bool] = None, execution_timeout: Optional[int] = None, drain_time: Optional[int] = None, on_reset: Optional[str] = None, max_attempts: Optional[int] = None, force: bool = False, secrets: Optional[list] = None, **kwargs) → Service[source]

Deploy a package. A service is required to run the code in your package.

Prerequisites: You must be in the role of an owner or developer.

Parameters:

  • package_id (str) – package id

  • package_name (str) – package name

  • package (dtlpy.entities.package.Package) – package entity

  • service_name (str) – service name

  • project_id (str) – project id

  • revision (str) – package revision - default=latest

  • init_input – config to run at startup

  • runtime (dict) – runtime resources

  • sdk_version (str) –

    • optional - string - sdk version

  • agent_versions (dict) –

    • dictionary - - optional -versions of sdk, agent runner and agent proxy

  • bot (str) – bot email

  • pod_type (str) – pod type dl.InstanceCatalog

  • verify (bool) – verify the inputs

  • checkout (bool) – checkout

  • module_name (str) – module name

  • run_execution_as_process (bool) – run execution as process

  • execution_timeout (int) – execution timeout

  • drain_time (int) – drain time

  • on_reset (str) – on reset

  • max_attempts (int) – Maximum execution retries in-case of a service reset

  • force (bool) – optional - terminate old replicas immediately

  • secrets (list) – list of the integrations ids

Returns:

Service object

Return type:

dtlpy.entities.service.Service

Example:

project.packages.deploy(service_name=package_name,
                        execution_timeout=3 * 60 * 60,
                        module_name=module.name,
                        runtime=dl.KubernetesRuntime(
                            concurrency=10,
                            pod_type=dl.InstanceCatalog.REGULAR_S,
                            autoscaler=dl.KubernetesRabbitmqAutoscaler(
                                min_replicas=1,
                                max_replicas=20,
                                queue_length=20
                            )
                        )
                    )
deploy_from_file(project, json_filepath)[source]

Deploy package and service from a JSON file.

Prerequisites: You must be in the role of an owner or developer.

Parameters:
Returns:

the package and the services

Example:

project.packages.deploy_from_file(project='project_entity', json_filepath='json_filepath')
static generate(name=None, src_path: Optional[str] = None, service_name: Optional[str] = None, package_type='default_package_type')[source]

Generate a new package. Provide a file path to a JSON file with all the details of the package and service to generate the package.

Prerequisites: You must be in the role of an owner or developer.

Parameters:

  • name (str) – name

  • src_path (str) – source file path

  • service_name (str) – service name

  • package_type (str) – package type from PackageCatalog

Example:

project.packages.generate(name='package_name',
                          src_path='src_path')
get(package_name: Optional[str] = None, package_id: Optional[str] = None, checkout: bool = False, fetch=None) → Package[source]

Get Package object to use in your code.

Prerequisites: You must be in the role of an owner or developer.

Parameters:

  • package_id (str) – package id

  • package_name (str) – package name

  • checkout (bool) – set the package as a default package object (cookies)

  • fetch – optional - fetch entity from platform, default taken from cookie

Returns:

Package object

Return type:

dtlpy.entities.package.Package

Example:

project.packages.get(package_id='package_id')
list(filters: Optional[Filters] = None, project_id: Optional[str] = None) → PagedEntities[source]

List project packages.

Prerequisites: You must be in the role of an owner or developer.

Parameters:
Returns:

Paged entity

Return type:

dtlpy.entities.paged_entities.PagedEntities

Example:

project.packages.list()
open_in_web(package: Optional[Package] = None, package_id: Optional[str] = None, package_name: Optional[str] = None)[source]

Open the package in the web platform.

Prerequisites: You must be in the role of an owner or developer.

Parameters:

Example:

project.packages.open_in_web(package_id='package_id')
pull(package: Package, version=None, local_path=None, project_id=None)[source]

Pull (download) the package to a local path.

Prerequisites: You must be in the role of an owner or developer.

Parameters:

  • package (dtlpy.entities.package.Package) – package entity

  • version (str) – the package version to pull

  • local_path – the path of where to save the package

  • project_id – the project id that include the package

Returns:

local path where the package pull

Return type:

str

Example:

project.packages.pull(package='package_entity', local_path='local_path')
push(project: Optional[Project] = None, project_id: Optional[str] = None, package_name: Optional[str] = None, src_path: Optional[str] = None, codebase: Optional[Union[GitCodebase, ItemCodebase, FilesystemCodebase]] = None, modules: Optional[List[PackageModule]] = None, is_global: Optional[bool] = None, checkout: bool = False, revision_increment: Optional[str] = None, version: Optional[str] = None, ignore_sanity_check: bool = False, service_update: bool = False, service_config: Optional[dict] = None, slots: Optional[List[PackageSlot]] = None, requirements: Optional[List[PackageRequirement]] = None, package_type=None, metadata=None) → Package[source]

Push your local package to the UI.

Prerequisites: You must be in the role of an owner or developer.

Project will be taken in the following hierarchy: project(input) -> project_id(input) -> self.project(context) -> checked out

Parameters:

  • project (dtlpy.entities.project.Project) – optional - project entity to deploy to. default from context or checked-out

  • project_id (str) – optional - project id to deploy to. default from context or checked-out

  • package_name (str) – package name

  • src_path (str) – path to package codebase

  • codebase (dtlpy.entities.codebase.Codebase) – codebase object

  • modules (list) – list of modules PackageModules of the package

  • is_global (bool) – is package is global or local

  • checkout (bool) – checkout package to local dir

  • revision_increment (str) – optional - str - version bumping method - major/minor/patch - default = None

  • version (str) – semver version f the package

  • ignore_sanity_check (bool) – NOT RECOMMENDED - skip code sanity check before pushing

  • service_update (bool) – optional - bool - update the service

:param dict service_config : Service object as dict. Contains the spec of the default service to create. :param list slots: optional - list of slots PackageSlot of the package :param list requirements: requirements - list of package requirements :param str package_type: default ‘faas’, options: ‘app’, ‘ml :param dict metadata: dictionary of system and user metadata

Returns:

Package object

Return type:

dtlpy.entities.package.Package

Example:

project.packages.push(package_name='package_name',
                        modules=[module],
                        version='1.0.0',
                        src_path=os.getcwd()
                    )
revisions(package: Optional[Package] = None, package_id: Optional[str] = None)[source]

Get the package revision history.

Prerequisites: You must be in the role of an owner or developer.

Parameters:

Example:

project.packages.revisions(package='package_entity')
test_local_package(cwd: Optional[str] = None, concurrency: Optional[int] = None, package: Optional[Package] = None, module_name: str = 'default_module', function_name: str = 'run', class_name: str = 'ServiceRunner', entry_point: str = 'main.py', mock_file_path: Optional[str] = None)[source]

Test local package in local environment.

Prerequisites: You must be in the role of an owner or developer.

Parameters:

  • cwd (str) – path to the file

  • concurrency (int) – the concurrency of the test

  • package (dtlpy.entities.package.Package) – entities.package

  • module_name (str) – module name

  • function_name (str) – function name

  • class_name (str) – class name

  • entry_point (str) – the file to run like main.py

  • mock_file_path (str) – the mock file that have the inputs

Returns:

list created by the function that tested the output

Return type:

list

Example:

project.packages.test_local_package(cwd='path_to_package',
                                    package='package_entity',
                                    function_name='run')
update(package: Package, revision_increment: Optional[str] = None) → Package[source]

Update Package changes to the platform.

Prerequisites: You must be in the role of an owner or developer.

Parameters:
Returns:

Package object

Return type:

dtlpy.entities.package.Package

Example:

project.packages.delete(package='package_entity')

Codebases

class Codebases(client_api: ApiClient, project: Optional[Project] = None, dataset: Optional[Dataset] = None, project_id: Optional[str] = None)[source]

Bases: object

Codebase Repository

The Codebases class allows the user to manage codebases and their properties. The codebase is the code the user uploads for the user’s packages to run. Read more about codebase in our FaaS (function as a service).

clone_git(codebase: Codebase, local_path: str)[source]

Clone code base

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters:

  • codebase (dtlpy.entities.codebase.Codebase) – codebase object

  • local_path (str) – local path

Returns:

path where the clone will be

Return type:

str

Example:

package.codebases.clone_git(codebase='codebase_entity', local_path='local_path')
get(codebase_name: Optional[str] = None, codebase_id: Optional[str] = None, version: Optional[str] = None)[source]

Get a Codebase object to use in your code.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Example:

package.codebases.get(codebase_name='codebase_name')
Parameters:

  • codebase_name (str) – optional - search by name

  • codebase_id (str) – optional - search by id

  • version (str) – codebase version. default is latest. options: “all”, “latest” or ver number - “10”

Returns:

Codebase object

static get_current_version(all_versions_pages, zip_md)[source]

This method returns the current version of the codebase and other versions found.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters:

  • all_versions_pages (codebase) – codebase object

  • zip_md – zipped file of codebase

Returns:

current version and all versions found of codebase

Return type:

int, int

Example:

package.codebases.get_current_version(all_versions_pages='codebase_entity', zip_md='path')
list() → PagedEntities[source]

List all codebases.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Example:

package.codebases.list()
Returns:

Paged entity

Return type:

dtlpy.entities.paged_entities.PagedEntities

list_versions(codebase_name: str) → PagedEntities[source]

List all codebase versions.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Example:

package.codebases.list_versions(codebase_name='codebase_name')
Parameters:

codebase_name (str) – code base name

Returns:

list of versions

Return type:

list

pack(directory: str, name: Optional[str] = None, extension: str = 'zip', description: str = '', ignore_directories: Optional[List[str]] = None)[source]

Zip a local code directory and post to codebases.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters:

  • directory (str) – local directory to pack

  • name (str) – codebase name

  • extension (str) – the extension of the file

  • description (str) – codebase description

  • ignore_directories (list[str]) – directories to ignore.

Returns:

Codebase object

Return type:

dtlpy.entities.codebase.Codebase

Example:

package.codebases.pack(directory='path_dir', name='codebase_name')
pull_git(codebase, local_path)[source]

Pull (download) a codebase.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters:

  • codebase (dtlpy.entities.codebase.Codebase) – codebase object

  • local_path (str) – local path

Returns:

path where the Pull will be

Return type:

str

Example:

package.codebases.pull_git(codebase='codebase_entity', local_path='local_path')
unpack(codebase: Optional[Codebase] = None, codebase_name: Optional[str] = None, codebase_id: Optional[str] = None, local_path: Optional[str] = None, version: Optional[str] = None)[source]

Unpack codebase locally. Download source code and unzip.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters:

  • codebase (dtlpy.entities.codebase.Codebase) – dl.Codebase object

  • codebase_name (str) – search by name

  • codebase_id (str) – search by id

  • local_path (str) – local path to save codebase

  • version (str) – codebase version to unpack. default - latest

Returns:

String (dirpath)

Return type:

str

Example:

package.codebases.unpack(codebase='codebase_entity', local_path='local_path')

Services

class ServiceLog(_json: dict, service: Service, services: Services, start=None, follow=None, execution_id=None, function_name=None, replica_id=None, system=False, model_id=None, model_operation=None, project_id=None)[source]

Bases: object

Service Log

view(until_completed)[source]

View logs

Parameters:

until_completed –

class Services(client_api: ApiClient, project: Optional[Project] = None, package: Optional[Package] = None, project_id=None)[source]

Bases: object

Services Repository

The Services class allows the user to manage services and their properties. Services are created from the packages users create. See our documentation for more information about services.

activate_slots(service: Service, project_id: Optional[str] = None, task_id: Optional[str] = None, dataset_id: Optional[str] = None, org_id: Optional[str] = None, user_email: Optional[str] = None, slots: Optional[List[PackageSlot]] = None, role=None, prevent_override: bool = True, visible: bool = True, icon: str = 'fas fa-magic', **kwargs)[source]

Activate service slots (creates buttons in the UI that activate services).

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters:

  • service (dtlpy.entities.service.Service) – service entity

  • project_id (str) – project id

  • task_id (str) – task id

  • dataset_id (str) – dataset id

  • org_id (str) – org id

  • user_email (str) – user email

  • slots (list) – list of entities.PackageSlot

  • role (str) – user role MemberOrgRole.ADMIN, MemberOrgRole.owner, MemberOrgRole.MEMBER, MemberOrgRole.WORKER

  • prevent_override (bool) – True to prevent override

  • visible (bool) – visible

  • icon (str) – icon

  • kwargs – all additional arguments

Returns:

list of user setting for activated slots

Return type:

list

Example:

setting = package.services.activate_slots(service='service_entity',
                                project_id='project_id',
                                slots=List[entities.PackageSlot],
                                icon='fas fa-magic')
checkout(service: Optional[Service] = None, service_name: Optional[str] = None, service_id: Optional[str] = None)[source]

Checkout (switch) to a service.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters:

Example:

package.services.checkout(service_id='service_id')
delete(service_name: Optional[str] = None, service_id: Optional[str] = None)[source]

Delete Service object

Prerequisites: You must be in the role of an owner or developer. You must have a package.

You must provide at least ONE of the following params: service_id, service_name.

Parameters:

  • service_name (str) – by name

  • service_id (str) – by id

Returns:

True

Return type:

bool

Example:

is_deleted = package.services.delete(service_id='service_id')
deploy(service_name: Optional[str] = None, package: Optional[Package] = None, bot: Optional[Union[Bot, str]] = None, revision: Optional[str] = None, init_input: Optional[Union[List[FunctionIO], FunctionIO, dict]] = None, runtime: Optional[Union[KubernetesRuntime, dict]] = None, pod_type: Optional[InstanceCatalog] = None, sdk_version: Optional[str] = None, agent_versions: Optional[dict] = None, verify: bool = True, checkout: bool = False, module_name: Optional[str] = None, project_id: Optional[str] = None, driver_id: Optional[str] = None, func: Optional[Callable] = None, run_execution_as_process: Optional[bool] = None, execution_timeout: Optional[int] = None, drain_time: Optional[int] = None, max_attempts: Optional[int] = None, on_reset: Optional[str] = None, force: bool = False, secrets: Optional[list] = None, **kwargs) → Service[source]

Deploy service.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters:

  • service_name (str) – name

  • package (dtlpy.entities.package.Package) – package entity

  • bot (str) – bot email

  • revision (str) – package revision of version

  • init_input – config to run at startup

  • runtime (dict) – runtime resources

  • pod_type (str) – pod type dl.InstanceCatalog

  • sdk_version (str) –

    • optional - string - sdk version

  • agent_versions (str) –

    • dictionary - - optional -versions of sdk

  • verify (bool) – if true, verify the inputs

  • checkout (bool) – if true, checkout (switch) to service

  • module_name (str) – module name

  • project_id (str) – project id

  • driver_id (str) – driver id

  • func (Callable) – function to deploy

  • run_execution_as_process (bool) – if true, run execution as process

  • execution_timeout (int) – execution timeout in seconds

  • drain_time (int) – drain time in seconds

  • max_attempts (int) – maximum execution retries in-case of a service reset

  • on_reset (str) – what happens on reset

  • force (bool) – optional - if true, terminate old replicas immediately

  • secrets (list) – list of the integrations ids

  • kwargs – list of additional arguments

Returns:

Service object

Return type:

dtlpy.entities.service.Service

Example:

service = package.services.deploy(service_name=package_name,
                        execution_timeout=3 * 60 * 60,
                        module_name=module.name,
                        runtime=dl.KubernetesRuntime(
                            concurrency=10,
                            pod_type=dl.InstanceCatalog.REGULAR_S,
                            autoscaler=dl.KubernetesRabbitmqAutoscaler(
                                min_replicas=1,
                                max_replicas=20,
                                queue_length=20
                            )
                        )
                    )
deploy_from_local_folder(cwd=None, service_file=None, bot=None, checkout=False, force=False) → Service[source]

Deploy from local folder in local environment.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters:

  • cwd (str) – optional - package working directory. Default=cwd

  • service_file (str) – optional - service file. Default=None

  • bot (str) – bot

  • checkout – checkout

  • force (bool) – optional - terminate old replicas immediately

Returns:

Service object

Return type:

dtlpy.entities.service.Service

Example:

service = package.services.deploy_from_local_folder(cwd='file_path',
                                          service_file='service_file')
execute(service: Optional[Service] = None, service_id: Optional[str] = None, service_name: Optional[str] = None, sync: bool = False, function_name: Optional[str] = None, stream_logs: bool = False, execution_input=None, resource=None, item_id=None, dataset_id=None, annotation_id=None, project_id=None) → Execution[source]

Execute a function on an existing service.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters:

  • service (dtlpy.entities.service.Service) – service entity

  • service_id (str) – service id

  • service_name (str) – service name

  • sync (bool) – wait for function to end

  • function_name (str) – function name to run

  • stream_logs (bool) – prints logs of the new execution. only works with sync=True

  • execution_input – input dictionary or list of FunctionIO entities

  • resource (str) – dl.PackageInputType - input type.

  • item_id (str) – str - optional - input to function

  • dataset_id (str) – str - optional - input to function

  • annotation_id (str) – str - optional - input to function

  • project_id (str) – str - resource’s project

Returns:

entities.Execution

Return type:

dtlpy.entities.execution.Execution

Example:

execution = package.services.execute(service='service_entity',
                        function_name='run',
                        item_id='item_id',
                        project_id='project_id')
get(service_name=None, service_id=None, checkout=False, fetch=None) → Service[source]

Get service to use in your code.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters:

  • service_name (str) – optional - search by name

  • service_id (str) – optional - search by id

  • checkout (bool) – if true, checkout (switch) to service

  • fetch – optional - fetch entity from platform, default taken from cookie

Returns:

Service object

Return type:

dtlpy.entities.service.Service

Example:

service = package.services.get(service_id='service_id')
list(filters: Optional[Filters] = None) → PagedEntities[source]

List all services (services can be listed for a package or for a project).

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters:

filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters

Returns:

Paged entity

Return type:

dtlpy.entities.paged_entities.PagedEntities

Example:

services = package.services.list()
log(service, size=100, checkpoint=None, start=None, end=None, follow=False, text=None, execution_id=None, function_name=None, replica_id=None, system=False, view=True, until_completed=True, log_level='DEBUG', model_id: Optional[str] = None, model_operation: Optional[str] = None, project_id: Optional[str] = None)[source]

Get service logs.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters:

  • service (dtlpy.entities.service.Service) – service object

  • size (int) – size

  • checkpoint (dict) – the information from the lst point checked in the service

  • start (str) – iso format time

  • end (str) – iso format time

  • follow (bool) – if true, keep stream future logs

  • text (str) – text

  • execution_id (str) – execution id

  • function_name (str) – function name

  • replica_id (str) – replica id

  • system (bool) – system

  • view (bool) – if true, print out all the logs

  • until_completed (bool) – wait until completed

  • log_level (str) – the log level to display dl.LoggingLevel

  • model_id (str) – model id

  • model_operation (str) – model operation action

  • project_id (str) – project id

Returns:

ServiceLog entity

Return type:

ServiceLog

Example:

service_logs = package.services.log(service='service_entity')
name_validation(name: str)[source]

Validation service name.

Prerequisites: You must be in the role of an owner or developer.

Parameters:

name (str) – service name

Example:

package.services.name_validation(name='name')
open_in_web(service: Optional[Service] = None, service_id: Optional[str] = None, service_name: Optional[str] = None)[source]

Open the service in web platform

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters:

Example:

package.services.open_in_web(service_id='service_id')
pause(service_name: Optional[str] = None, service_id: Optional[str] = None, force: bool = False)[source]

Pause service.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

You must provide at least ONE of the following params: service_id, service_name

Parameters:

  • service_name (str) – service name

  • service_id (str) – service id

  • force (bool) – optional - terminate old replicas immediately

Returns:

True if success

Return type:

bool

Example:

success = package.services.pause(service_id='service_id')
resume(service_name: Optional[str] = None, service_id: Optional[str] = None, force: bool = False)[source]

Resume service.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

You must provide at least ONE of the following params: service_id, service_name.

Parameters:

  • service_name (str) – service name

  • service_id (str) – service id

  • force (bool) – optional - terminate old replicas immediately

Returns:

json of the service

Return type:

dict

Example:

service_json = package.services.resume(service_id='service_id')
revisions(service: Optional[Service] = None, service_id: Optional[str] = None)[source]

Get service revisions history.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

You must provide at leats ONE of the following params: service, service_id

Parameters:

Example:

service_revision = package.services.revisions(service_id='service_id')
status(service_name=None, service_id=None)[source]

Get service status.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

You must provide at least ONE of the following params: service_id, service_name

Parameters:

  • service_name (str) – service name

  • service_id (str) – service id

Returns:

status json

Return type:

dict

Example:

status_json = package.services.status(service_id='service_id')
update(service: Service, force: bool = False) → Service[source]

Update service changes to platform.

Prerequisites: You must be in the role of an owner or developer. You must have a package.

Parameters:
Returns:

Service entity

Return type:

dtlpy.entities.service.Service

Example:

service = package.services.update(service='service_entity')

Bots

class Bots(client_api: ApiClient, project: Project)[source]

Bases: object

Bots Repository

The Bots class allows the user to manage bots and their properties.

create(name: str, return_credentials: bool = False)[source]

Create a new Bot.

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters:

  • name (str) – bot name

  • return_credentials (str) – True will return the password when created

Returns:

Bot object

Return type:

dtlpy.entities.bot.Bot

Example:

service.bots.delete(name='bot', return_credentials=False)
delete(bot_id: Optional[str] = None, bot_email: Optional[str] = None)[source]

Delete a Bot.

Prerequisites: You must be in the role of an owner or developer. You must have a service.

You must provide at least ONE of the following params: bot_id, bot_email

Parameters:

  • bot_id (str) – bot id to delete

  • bot_email (str) – bot email to delete

Returns:

True if successful

Return type:

bool

Example:

service.bots.delete(bot_id='bot_id')
get(bot_email: Optional[str] = None, bot_id: Optional[str] = None, bot_name: Optional[str] = None)[source]

Get a Bot object.

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters:

  • bot_email (str) – get bot by email

  • bot_id (str) – get bot by id

  • bot_name (str) – get bot by name

Returns:

Bot object

Return type:

dtlpy.entities.bot.Bot

Example:

service.bots.get(bot_id='bot_id')
list() → List[Bot][source]

Get a project or package bots list.

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Returns:

List of Bots objects

Return type:

list

Example:

bots_list = service.bots.list()

Triggers

class Triggers(client_api: ApiClient, project: Optional[Project] = None, service: Optional[Service] = None, project_id: Optional[str] = None, pipeline: Optional[Pipeline] = None)[source]

Bases: object

Triggers Repository

The Triggers class allows users to manage triggers and their properties. Triggers activate services. See our documentation for more information on triggers.

create(service_id: Optional[str] = None, trigger_type: TriggerType = TriggerType.EVENT, name: Optional[str] = None, webhook_id=None, function_name='run', project_id=None, active=True, filters=None, resource: TriggerResource = TriggerResource.ITEM, actions: Optional[TriggerAction] = None, execution_mode: TriggerExecutionMode = TriggerExecutionMode.ONCE, start_at=None, end_at=None, inputs=None, cron=None, pipeline_id=None, pipeline=None, pipeline_node_id=None, root_node_namespace=None, **kwargs) → BaseTrigger[source]

Create a Trigger. Can create two types: a cron trigger or an event trigger. Inputs are different for each type

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Inputs for all types:

Parameters:

  • service_id (str) – Id of services to be triggered

  • trigger_type (str) – can be cron or event. use enum dl.TriggerType for the full list

  • name (str) – name of the trigger

  • webhook_id (str) – id for webhook to be called

  • function_name (str) – the function name to be called when triggered (must be defined in the package)

  • project_id (str) – project id where trigger will work

  • active (bool) – optional - True/False, default = True, if true trigger is active

Inputs for event trigger: :param dtlpy.entities.filters.Filters filters: optional - Item/Annotation metadata filters, default = none :param str resource: optional - Dataset/Item/Annotation/ItemStatus, default = Item :param str actions: optional - Created/Updated/Deleted, default = create :param str execution_mode: how many times trigger should be activated; default is “Once”. enum dl.TriggerExecutionMode

Inputs for cron trigger: :param start_at: iso format date string to start activating the cron trigger :param end_at: iso format date string to end the cron activation :param inputs: dictionary “name”:”val” of inputs to the function :param str cron: cron spec specifying when it should run. more information: https://en.wikipedia.org/wiki/Cron :param str pipeline_id: Id of pipeline to be triggered :param pipeline: pipeline entity to be triggered :param str pipeline_node_id: Id of pipeline root node to be triggered :param root_node_namespace: namespace of pipeline root node to be triggered

Returns:

Trigger entity

Return type:

dtlpy.entities.trigger.Trigger

Example:

service.triggers.create(name='triggername',
                      execution_mode=dl.TriggerExecutionMode.ONCE,
                      resource='Item',
                      actions='Created',
                      function_name='run',
                      filters={'$and': [{'hidden': False},
                                        {'type': 'file'}]}
                      )
delete(trigger_id=None, trigger_name=None)[source]

Delete Trigger object

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters:

  • trigger_id (str) – trigger id

  • trigger_name (str) – trigger name

Returns:

True is successful error if not

Return type:

bool

Example:

service.triggers.delete(trigger_id='trigger_id')
get(trigger_id=None, trigger_name=None) → BaseTrigger[source]

Get Trigger object

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters:

  • trigger_id (str) – trigger id

  • trigger_name (str) – trigger name

Returns:

Trigger entity

Return type:

dtlpy.entities.trigger.Trigger

Example:

service.triggers.get(trigger_id='trigger_id')
list(filters: Optional[Filters] = None) → PagedEntities[source]

List triggers of a project, package, or service.

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters:

filters (dtlpy.entities.filters.Filters) – Filters entity or a dictionary containing filters parameters

Returns:

Paged entity

Return type:

dtlpy.entities.paged_entities.PagedEntities

Example:

service.triggers.list()
name_validation(name: str)[source]

This method validates the trigger name. If name is not valid, this method will return an error. Otherwise, it will not return anything.

Parameters:

name (str) – trigger name

resource_information(resource, resource_type, action='Created')[source]

Returns which function should run on an item (based on global triggers).

Prerequisites: You must be a superuser to run this method.

Parameters:

  • resource – ‘Item’ / ‘Dataset’ / etc

  • resource_type – dictionary of the resource object

  • action – ‘Created’ / ‘Updated’ / etc.

Example:

service.triggers.resource_information(resource='Item', resource_type=item_object, action='Created')
update(trigger: BaseTrigger) → BaseTrigger[source]

Update trigger

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters:

trigger (dtlpy.entities.trigger.Trigger) – Trigger entity

Returns:

Trigger entity

Return type:

dtlpy.entities.trigger.Trigger

Example:

service.triggers.update(trigger='trigger_entity')

Executions

class Executions(client_api: ApiClient, service: Optional[Service] = None, project: Optional[Project] = None)[source]

Bases: object

Service Executions Repository

The Executions class allows the users to manage executions (executions of services) and their properties. See our documentation for more information about executions.

create(service_id: Optional[str] = None, execution_input: Optional[list] = None, function_name: Optional[str] = None, resource: Optional[PackageInputType] = None, item_id: Optional[str] = None, dataset_id: Optional[str] = None, annotation_id: Optional[str] = None, project_id: Optional[str] = None, sync: bool = False, stream_logs: bool = False, return_output: bool = False, return_curl_only: bool = False, timeout: Optional[int] = None) → Execution[source]

Execute a function on an existing service

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters:

  • service_id (str) – service id to execute on

  • execution_input (List[FunctionIO] or dict) – input dictionary or list of FunctionIO entities

  • function_name (str) – function name to run

  • resource (str) – input type.

  • item_id (str) – optional - item id as input to function

  • dataset_id (str) – optional - dataset id as input to function

  • annotation_id (str) – optional - annotation id as input to function

  • project_id (str) – resource’s project

  • sync (bool) – if true, wait for function to end

  • stream_logs (bool) – prints logs of the new execution. only works with sync=True

  • return_output (bool) – if True and sync is True - will return the output directly

  • return_curl_only (bool) – return the cURL of the creation WITHOUT actually do it

  • timeout (int) – int, seconds to wait until TimeoutError is raised. if <=0 - wait until done - by default wait take the service timeout

Returns:

execution object

Return type:

dtlpy.entities.execution.Execution

Example:

service.executions.create(function_name='function_name', item_id='item_id', project_id='project_id')
create_batch(service_id: str, filters, function_name: Optional[str] = None, execution_inputs: Optional[list] = None, wait=True)[source]

Execute a function on an existing service

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters:

  • service_id (str) – service id to execute on

  • filters – Filters entity for a filtering before execute

  • function_name (str) – function name to run

  • execution_inputs (List[FunctionIO] or dict) – input dictionary or list of FunctionIO entities

  • wait (bool) – wait until create task finish

Returns:

execution object

Return type:

dtlpy.entities.execution.Execution

Example:

command = service.executions.create_batch(
            execution_inputs=dl.FunctionIO(type=dl.PackageInputType.STRING, value='test', name='string'),
            filters=dl.Filters(field='dir', values='/test'),
            function_name='run')
get(execution_id: Optional[str] = None, sync: bool = False) → Execution[source]

Get Service execution object

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters:

  • execution_id (str) – execution id

  • sync (bool) – if true, wait for the execution to finish

Returns:

Service execution object

Return type:

dtlpy.entities.execution.Execution

Example:

service.executions.get(execution_id='execution_id')
increment(execution: Execution)[source]

Increment the number of attempts that an execution is allowed to attempt to run a service that is not responding.

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters:

execution (dtlpy.entities.execution.Execution) –

Returns:

int

Return type:

int

Example:

service.executions.increment(execution='execution_entity')
list(filters: Optional[Filters] = None) → PagedEntities[source]

List service executions

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters:

filters (dtlpy.entities.filters.Filters) – dl.Filters entity to filters items

Returns:

Paged entity

Return type:

dtlpy.entities.paged_entities.PagedEntities

Example:

service.executions.list()
logs(execution_id: str, follow: bool = True, until_completed: bool = True)[source]

executions logs

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters:

  • execution_id (str) – execution id

  • follow (bool) – if true, keep stream future logs

  • until_completed (bool) – if true, wait until completed

Returns:

executions logs

Example:

service.executions.logs(execution_id='execution_id')
progress_update(execution_id: str, status: Optional[ExecutionStatus] = None, percent_complete: Optional[int] = None, message: Optional[str] = None, output: Optional[str] = None, service_version: Optional[str] = None)[source]

Update Execution Progress.

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters:

  • execution_id (str) – execution id

  • status (str) – ExecutionStatus

  • percent_complete (int) – percent work done

  • message (str) – message

  • output (str) – the output of the execution

  • service_version (str) – service version

Returns:

Service execution object

Return type:

dtlpy.entities.execution.Execution

Example:

service.executions.progress_update(execution_id='execution_id', status='complete', percent_complete=100)
rerun(execution: Execution, sync: bool = False)[source]

Rerun execution

Prerequisites: You must be in the role of an owner or developer. You must have a service.

Parameters:
Returns:

Execution object

Return type:

dtlpy.entities.execution.Execution

Example:

service.executions.rerun(execution='execution_entity')
terminate(execution: