cloud-function (cf)¶
For information about concepts and other user interfaces, see the Overview from the NVCF Documentation.
Cloud Function Commands
usage: ngc cloud-function [--debug] [--format_type <fmt>] [-h]
{available-gpus,cluster,function,gpu,task,telemetry-endpoint}
...
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
cloud-function¶
- {available-gpus,cluster,function,gpu,task,telemetry-endpoint}
Possible choices: available-gpus, cluster, fn, function, gpu, instance-type, task, telemetry-endpoint
Sub-commands¶
function (fn)¶
Cloud function function commands.
ngc cloud-function function [--debug] [--format_type <fmt>] [-h]
{authorization,create,deploy,info,instance,invoke,list,remove,remove-rate-limit,update-rate-limit,update-secret}
...
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
function¶
- {authorization,create,deploy,info,instance,invoke,list,remove,remove-rate-limit,update-rate-limit,update-secret}
Possible choices: auth, authorization, create, delete, deploy, info, instance, invoke, list, ls, remove, remove-rate-limit, rm, update-rate-limit, update-secret
Sub-commands¶
deploy¶
Description of the deployment command
ngc cloud-function function deploy [--debug] [--format_type <fmt>] [-h]
{create,info,list,log,remove,restart,update}
...
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
deploy¶
- {create,info,list,log,remove,restart,update}
Possible choices: create, delete, info, list, log, ls, redeploy, remove, restart, rm, update
Sub-commands¶
list (ls)¶
List active deployments
ngc cloud-function function deploy list [--debug] [--format_type <fmt>] [-h]
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
info¶
Info about a function deployment
ngc cloud-function function deploy info [--debug] [--format_type <fmt>] [-h]
<function-id>:<function-version-id>
Positional Arguments¶
- <function-id>:<function-version-id>
Function. Format: function-id:function-version
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
remove (rm, delete)¶
Undeploy a function
ngc cloud-function function deploy remove [--debug] [--format_type <fmt>]
[--graceful] [-h]
<function-id>:<function-version-id>
Positional Arguments¶
- <function-id>:<function-version-id>
Function. Format: function-id:function-version
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
- --graceful
Allow current tasks to complete before deleting deployment.
restart (redeploy)¶
restart
ngc cloud-function function deploy restart [--debug] [--format_type <fmt>]
[-h]
<function-id>:<function-version-id>
Positional Arguments¶
- <function-id>:<function-version-id>
Function. Format: function-id:function-version
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
update¶
Update an existing deployment
ngc cloud-function function deploy update [--configuration CONFIGURATION]
[--configuration-file <file.yaml>]
[--debug]
[--deployment-specification <dep-spec>]
[--format_type <fmt>]
[--set <key1.key2=value>]
[--targeted-deployment-specification <targeted-dep-spec>]
[-h]
<function-id>:<function-version-id>
Positional Arguments¶
- <function-id>:<function-version-id>
Function. Format: function-id:function-version
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
- --configuration
Helm chart overrides json will apply cross the functions, only for Helm chart deployment
- --configuration-file, -f
Helm chart overrides yaml. Can apply multiple times.The priority will be given to the last (right-most) file specified.
- --set
Set configuration values (e.g., --set image.tag=latest). Can apply multiple times.
- --deployment-specification, --dep-spec
Deployment specs with GPU and Backend details, can specify multiple times. (Format: backend:gpu:instance_type:min_instances:max_instances[:max_request_concurrency])
- --targeted-deployment-specification, --targeted-dep-spec
Deployment specs with GPU and instance type, can specify multiple times.Clusters are mandatory for non GFN backends.(Format: <gpu:instance_type:min_instances:max_instances[:max_request_concurrency][:clusters(cluster_1,cluster_2)][:regions(region_1,region_2)][:attributes(attribute_1,attribute_2)][:preferredOrder])
create¶
Create a deployment
ngc cloud-function function deploy create [--configuration CONFIGURATION]
[--configuration-file <file.yaml>]
[--debug]
[--deployment-specification <backend:gpu:instance_type:min_instances:max_instances[:maxRequestConcurrency]>]
[--format_type <fmt>]
[--set key1.key2=value]
[--targeted-deployment-specification <gpu:instance_type:min_instances:max_instances[:max_request_concurrency][:clusters(cluster_1,cluster_2)][:regions(region_1,region_2)][:attributes(attribute_1,attribute_2)][:preferredOrder]>]
[-h]
<function-id>:<function-version-id>
Positional Arguments¶
- <function-id>:<function-version-id>
Function. Format: function-id:function-version
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
- --configuration
Helm chart overrides json will apply cross the functions, only for Helm chart deployment
- --configuration-file, -f
Helm chart overrides yaml. Can apply multiple times. The priority will be given to the last (right-most) file specified.
- --set
Set configuration values (e.g., --set image.tag=latest). Can apply multiple times.
- --deployment-specification, --dep-spec
Deployment specs with GPU and Backend details, can specify multiple times
- --targeted-deployment-specification, --targeted-dep-spec
Deployment specs with GPU and instance type, can specify multiple times.Clusters are mandatory for non GFN backends.
log¶
Query Logs for Deployment
ngc cloud-function function deploy log [--debug] [--duration <t>]
[--end-time <t>] [--format_type <fmt>]
[--start-time <t>] [-h]
function-id:[<function-id>]
Positional Arguments¶
- function-id:[<function-id>]
Function. Format: function-id:function-version
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
- --start-time
Specifies the start time for querying logs. Format: [yyyy-MM-dd::HH:mm:ss].
- --end-time
Specifies the end time for querying logs. Format: [yyyy-MM-dd::HH:mm:ss]. Default: now
- --duration
Specifies the duration of time, either after begin-time or before end-timelogs. Format: [nD][nH][nM][nS]. Default 1 day, doesn't respect decimal measurements
instance¶
description of the instance command
ngc cloud-function function instance [--debug] [--format_type <fmt>] [-h]
{execute,list,logs} ...
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
instance¶
- {execute,list,logs}
Possible choices: exec, execute, list, logs, ls
Sub-commands¶
list (ls)¶
List a function version instances
ngc cloud-function function instance list [--debug] [--format_type <fmt>] [-h]
<function-id>:[<function-version-id>]
Positional Arguments¶
- <function-id>:[<function-version-id>]
Function. Format: function-id:[version]
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
execute (exec)¶
Info about a version
ngc cloud-function function instance execute [--command <command>]
[--container-name <container-name>]
[--debug] [--format_type <fmt>]
[--instance-id <instance-id>]
[--pod-name <pod-name>]
[--timeout <timeout>] [-h]
<function-id>:[<function-version-id>]
Positional Arguments¶
- <function-id>:[<function-version-id>]
Function. Format: function-id:[version]
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
- --container-name
Target container to execute container
- --pod-name
Target pod which including the target container
- --instance-id
Target instance id.
- --command
Command to execute
- --timeout
Timeout for the command execution
logs¶
Read the target instance logs
ngc cloud-function function instance logs [--container-name <container-name>]
[--debug] [--format_type <fmt>]
[--instance-id <instance-id>]
[--pod-name <pod-name>] [-h]
<function-id>:[<function-version-id>]
Positional Arguments¶
- <function-id>:[<function-version-id>]
Function. Format: function-id:[version]
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
- --container-name
Target container to query logs. Must be used with --pod-name
- --pod-name
Target pod which including the target container. Must be used with --container-name
- --instance-id
Target instance id.
list (ls)¶
List a functions
ngc cloud-function function list [--access-filter <filter>]
[--column <column>] [--debug]
[--format_type <fmt>] [--name-pattern <name>]
[-h]
[[<function-id>]]
Positional Arguments¶
- [<function-id>]
Function ID
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
- --access-filter
Possible choices: authorized, private, public
Filter functions by access, choices are: [private,public,authorized]
- --name-pattern
Filter functions by names, supports globs.
- --column
Specify output column as column[=header], header is optional, default is id[=Id]. Valid columns are activeInstances[="Active Instances"], containerImage[="Container Image URL"], helmChart[="Helm Chart URL"], helmChartServiceName[="Helm Chart Service Name"], models[="Model URLs"], id[="Function ID"], description[=Description], status[=Status], version[="Version ID"]. Use quotes with spaces. Multiple column arguments are allowed.
info¶
Info about a version
ngc cloud-function function info [--debug] [--format_type <fmt>] [-h]
<function-id>:<function-version-id>
Positional Arguments¶
- <function-id>:<function-version-id>
Function. Format: function-id:[version]
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
remove (rm, delete)¶
Delete a version
ngc cloud-function function remove [--debug] [--format_type <fmt>] [-h]
<function-id>:[<function-version-id>]
Positional Arguments¶
- <function-id>:[<function-version-id>]
Function. Format: function-id:[version]
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
update-secret¶
Update a function's secret values.
ngc cloud-function function update-secret [--debug] [--format_type <fmt>]
[--json-secret-file <filename>]
[--secret <name:value>] [-h]
<function-id>:[<function-version-id>]
Positional Arguments¶
- <function-id>:[<function-version-id>]
Function. Format: function-id:[version]
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
- --secret
Secret name/value pair
- --json-secret-file
Takes in a file, assumes the key as the filename, value is the file contents. Only works with json files
update-rate-limit¶
Update a function's rate limit values.
ngc cloud-function function update-rate-limit [--debug] [--format_type <fmt>]
[--rate-limit-exempt-nca-id <rate-limit-exempt-id>]
--rate-limit-pattern
<rate-limit-pattern>
[--rate-limit-sync-check] [-h]
<function-id>:[<function-version-id>]
Positional Arguments¶
- <function-id>:[<function-version-id>]
Function. Format: function-id:[version]
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
- --rate-limit-exempt-nca-id
Exempt NCA id.
- --rate-limit-sync-check
Rate limit sync check.
Required named arguments¶
- --rate-limit-pattern
Specify rate Limit, format NUMBER-S|M|H|D, ex: 3-S.
remove-rate-limit¶
Delete a function's rate limit.
ngc cloud-function function remove-rate-limit [--debug] [--format_type <fmt>]
[-h]
<function-id>:[<function-version-id>]
Positional Arguments¶
- <function-id>:[<function-version-id>]
Function. Format: function-id:[version]
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
create¶
Create a new function
ngc cloud-function function create [--api-body-format <api-body-format>]
[--container-args <container-args>]
[--container-environment-variable <key:value>]
[--container-image <org>/[<team>/]<image>:<tag>]
[--debug] [--description <description>]
[--format_type <fmt>]
[--function-type <function-type>]
[--health-expected-status-code <health-expected-status-code>]
[--health-port <health-port>]
[--health-protocol <health-protocol>]
[--health-timeout <health-timeout>]
[--health-uri <health-uri>]
[--helm-chart <org>/[<team>/]<helm-chart>:<tag>]
[--helm-chart-service <helm-chart-service-name>]
[--inference-port <inference-port>]
--inference-url <inference-url>
[--json-secret-file <filename>]
[--logs-telemetry-id <logs-telemetry-id>]
[--metrics-telemetry-id <metrics-telemetry-id>]
[--model [<override-name>:]<org>/[<team>/]<image>:version]
--name <name>
[--rate-limit-exempt-nca-id <rate-limit-exempt-id>]
[--rate-limit-pattern <rate-limit-pattern>]
[--rate-limit-sync-check | --no-rate-limit-sync-check]
[--resource <org>/[<team>/]<resource>:version]
[--secret <name:value>] [--tag <tag>]
[--traces-telemetry-id <traces-telemetry-id>]
[-h]
[[<function-id>]]
Positional Arguments¶
- [<function-id>]
Function ID
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
- --helm-chart
Helm Chart in NGC used for deployment.
- --helm-chart-service
Must be provided if a helm chart function.
- --health-uri
Health endpoint for inferencing
- --health-port
Health port for inferencing
- --health-timeout
Health timeout for inferencing
- --health-protocol
Possible choices: HTTP, gRPC
Health protocol for inferencing
- --health-expected-status-code
Health status code.
- --inference-port
Optional port number where the inference listener is running - defaults to 8000 for HTTPS, 8001 for GRPC
- --container-args
Args to be passed in for inferencing
- --container-environment-variable
Environment settings for inferencing
- --secret
Secret name/value pair
- --json-secret-file
Takes in a file, assumes the key as the filename, value is the file contents. Only works with json files
- --model
List of models - could be empty with custom container, can accept multiple
- --resource
Optional List of resources, can accept multiple
- --tag
Tag to identify the function by, can accept multiple
- --description
Optional description for function/version
- --container-image
Custom container Image
- --api-body-format
Possible choices: CUSTOM, PREDICT_V2
Information about the request body format
- --function-type
Possible choices: DEFAULT, STREAMING
Health timeout for inferencing
- --metrics-telemetry-id
UUID representing the metrics telemetry.
- --logs-telemetry-id
UUID representing the logs telemetry.
- --traces-telemetry-id
UUID representing the traces telemetry.
- --rate-limit-pattern
Rate Limit.
- --rate-limit-exempt-nca-id
Exempt NCA id.
- --rate-limit-sync-check, --no-rate-limit-sync-check
Rate limit sync check.
Required named arguments¶
- --name
Function name must start with lowercase/uppercase/digit and can only contain lowercase, uppercase, digit, hyphen, and underscore characters
- --inference-url
Serves as entrypoint for Triton to Custom container
invoke¶
Inference a given function,
ngc cloud-function function invoke [--debug] [--format_type <fmt>] -f <file>
[-h] [-s]
<function-id>:[<function-version-id>]
Positional Arguments¶
- <function-id>:[<function-version-id>]
Function. Format: function-id:[version]
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
- -s, --stream
Invoke function with text/event-stream in the header
Required named arguments¶
- -f, --file
JSON file in format expected by given function. When stream is true, you may need to modify this payload.
cluster¶
Cloud function cluster commands.
ngc cloud-function cluster [--debug] [--format_type <fmt>] [-h]
{create,delete,info,list} ...
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
cluster¶
- {create,delete,info,list}
Possible choices: create, delete, info, list, ls, register, remove, rm
Sub-commands¶
list (ls)¶
List clusters.
ngc cloud-function cluster list [--debug] [--format_type <fmt>] [-h]
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
info¶
Info about a cluster
ngc cloud-function cluster info [--debug] [--format_type <fmt>] [-h]
<cluster-id>
Positional Arguments¶
- <cluster-id>
Cluster Id
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
delete (rm, remove)¶
Delete a cluster
ngc cloud-function cluster delete [--debug] [--format_type <fmt>] [-h]
<cluster-id>
Positional Arguments¶
- <cluster-id>
Cluster Id
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
create (register)¶
Register a new cluster with the specified details
ngc cloud-function cluster create [--attribute ATTRIBUTE]
[--authorized-nca-id AUTHORIZED_NCA_ID]
[--capability {CachingSupport,DynamicGPUDiscovery,LogPosting}]
--cloud-provider
{AWS,AZURE,DGX-CLOUD,GCP,OCI,ON-PREM}
[--cluster-description CLUSTER_DESCRIPTION]
--cluster-group-name CLUSTER_GROUP_NAME
--cluster-name CLUSTER_NAME
[--custom-attribute CUSTOM_ATTRIBUTE]
[--debug] [--format_type <fmt>]
[--nvca-version NVCA_VERSION] --region
{ap-east-1,eu-central-1,eu-north-1,eu-south-1,eu-west-1,us-east-1,us-west-1,us-west-2}
--ssa-client-id SSA_CLIENT_ID [-h]
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
- --cluster-description
Optional description providing additional context about the cluster.
- --capability
Possible choices: CachingSupport, DynamicGPUDiscovery, LogPosting
Capabilities
- --attribute
Attributes
- --custom-attribute
Custom attributes
- --authorized-nca-id
Authorized NCA IDs
- --nvca-version
NVCA version
Required named arguments¶
- --cluster-name
The name for the cluster. This field is not changeable once configured.
- --cluster-group-name
The name of the cluster group, typically identical to the cluster name,allowing function deployment across grouped clusters.
- --cloud-provider
Possible choices: AWS, AZURE, DGX-CLOUD, GCP, OCI, ON-PREM
The cloud platform on which the cluster is deployed.
- --region
Possible choices: ap-east-1, eu-central-1, eu-north-1, eu-south-1, eu-west-1, us-east-1, us-west-1, us-west-2
The region where the cluster is deployed.
- --ssa-client-id
SSA client ID
gpu (instance-type)¶
Cloud function gpu commands.
ngc cloud-function gpu [--debug] [--format_type <fmt>] [-h] {info,list} ...
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
gpu¶
- {info,list}
Possible choices: get, info, list, ls
Sub-commands¶
list (ls)¶
List allocated gpus.
ngc cloud-function gpu list [--debug] [--format_type <fmt>] [-h]
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
info (get)¶
Info for a given gpu.
ngc cloud-function gpu info [--debug] [--format_type <fmt>] [-h]
Positional Arguments¶
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
task¶
Cloud function task commands.
ngc cloud-function task [--debug] [--format_type <fmt>] [-h]
{cancel,create,delete,events,info,list,logs,results,update-secret}
...
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
task¶
- {cancel,create,delete,events,info,list,logs,results,update-secret}
Possible choices: cancel, create, delete, events, info, list, logs, ls, remove, results, rm, update-secret
Sub-commands¶
create¶
Create a new task
ngc cloud-function task create [--configuration CONFIGURATION]
[--configuration-file <file.yaml>]
[--container-args <container-args>]
[--container-environment-variable <key:value>]
[--container-image <org>/[<team>/]<image>:<tag>]
[--debug] [--description <description>]
[--format_type <fmt>] --gpu-specification
<gpu:instance_type[:backend]>
[--helm-chart <org>/[<team>/]<helm-chart>:<tag>]
[--logs-telemetry-id <logs-telemetry-id>]
[--max-queued-duration <max-queued-duration>]
[--max-runtime-duration <max-runtime-duration>]
[--metrics-telemetry-id <metrics-telemetry-id>]
[--model <model>] --name <name>
[--resource <resource>]
[--result-handling-strategy <result-handling-strategy>]
[--result-location <result-location>]
[--secret <name:value>]
[--set <key1.key2=value>] [--tag <tag>]
[--termination-grace-period-duration <termination-grace-period-duration>]
[--traces-telemetry-id <traces-telemetry-id>]
[-h]
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
- --configuration
Helm chart overrides json will apply cross the functions, only for Helm chart deployment
- --configuration-file, -f
Helm chart overrides yaml. Can apply multiple times.The priority will be given to the last (right-most) file specified.
- --set
Set configuration values (e.g., --set image.tag=latest). Can apply multiple times.
- --container-image
Custom container image
- --container-args
Args to be passed in for inferencing
- --container-environment-variable
Environment settings for inferencing
- --model
Name of model(s) to use in the task.
- --resource
Name of resource(s) to use in the task.
- --tag
Tags to identify the function by, can accept multiple
- --description
Optional description for function/version
- --max-runtime-duration
Specifies the maximum runtime duration. Format: [nD][nH][nM][nS].
- --max-queued-duration
Specifies the maximum queued duration. Format: [nD][nH][nM][nS].
- --termination-grace-period-duration
Specifies the duration for grace period after termination. Format: [nD][nH][nM][nS].
- --result-handling-strategy
Possible choices: NONE, UPLOAD
Specifies how results will be handled. Choices: UPLOAD, NONE
- --result-location
Specifies where result will be stored.}
- --secret
Secret name/value pair
- --helm-chart
Helm Chart in NGC used for deployment.
- --metrics-telemetry-id
UUID representing the metrics telemetry.
- --logs-telemetry-id
UUID representing the logs telemetry.
- --traces-telemetry-id
UUID representing the traces telemetry.
Required named arguments¶
- --name
Task name
- --gpu-specification
GPU specification. Format: gpu:instance_type[:backend]
update-secret¶
Update a task's secret value. Can specify multiple times.
ngc cloud-function task update-secret [--debug] [--format_type <fmt>] --secret
<name:value> [-h]
<task-id>
Positional Arguments¶
- <task-id>
Task ID
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
Required named arguments¶
- --secret
Secret name/value pair
list (ls)¶
List of tasks
ngc cloud-function task list [--debug] [--format_type <fmt>] [-h]
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
info¶
Info about a task
ngc cloud-function task info [--debug] [--format_type <fmt>] [-h] <task-id>
Positional Arguments¶
- <task-id>
Task ID
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
delete (remove, rm)¶
Delete a task
ngc cloud-function task delete [--debug] [--format_type <fmt>] [-h] <task-id>
Positional Arguments¶
- <task-id>
Task ID
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
cancel¶
Cancel a task
ngc cloud-function task cancel [--debug] [--format_type <fmt>] [-h] <task-id>
Positional Arguments¶
- <task-id>
Task ID
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
events¶
List of events for a task
ngc cloud-function task events [--debug] [--format_type <fmt>] [-h] <task-id>
Positional Arguments¶
- <task-id>
Task ID
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
results¶
List of results for a task
ngc cloud-function task results [--debug] [--format_type <fmt>] [-h] <task-id>
Positional Arguments¶
- <task-id>
Task ID
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
logs¶
List of logs for a task
ngc cloud-function task logs [--debug] [--format_type <fmt>] [-h] <task-id>
Positional Arguments¶
- <task-id>
Task ID
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
telemetry-endpoint¶
Cloud function telemetry-endpoint commands.
ngc cloud-function telemetry-endpoint [--debug] [--format_type <fmt>] [-h]
{create,list,remove} ...
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
telemetry-endpoint¶
- {create,list,remove}
Possible choices: add, create, delete, list, ls, remove, rm
Sub-commands¶
list (ls)¶
List telemetries.
ngc cloud-function telemetry-endpoint list [--debug] [--format_type <fmt>]
[-h]
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
create (add)¶
Create a new telemetry endpoint.
ngc cloud-function telemetry-endpoint create [--client-cert <client-cert>]
[--debug] --endpoint <endpoint>
[--format_type <fmt>]
[--instance <instance>] --key
<key> --name <name> --protocol
<protocol> --provider <provider>
[--self-cert <self-cert>] --type
<telemetry-type> [-h]
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
- --instance
Instance id for GRAFANA endpoints.
- --client-cert
Client cert for PROMETHEUS endpoints.
- --self-cert
Optional self certificate for PROMETHEUS endpoints.
Required named arguments¶
- --name
Telemetry Name
- --key
Telemetry key
- --endpoint
Telemetry endpoint URL
- --protocol
Possible choices: GRPC, HTTP
Protocol used for communication
- --provider
Possible choices: AZURE_MONITOR, DATADOG, GRAFANA_CLOUD, PROMETHEUS, SERVICENOW
Protocol used for communication
- --type
Possible choices: LOGS, METRICS, TRACES
Set of telemetry data types
remove (delete, rm)¶
Delete a telemetry endpoint.
ngc cloud-function telemetry-endpoint remove [--debug] [--format_type <fmt>]
[-h]
<telemetry-id>
Positional Arguments¶
- <telemetry-id>
Id of telemtry endpoint to delete
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
available-gpus¶
List available GPUs in your Org, Admin Only
ngc cloud-function available-gpus [--debug] [--format_type <fmt>] [-h]
Named Arguments¶
- --debug
Enable debug mode.
- --format_type
Possible choices: ascii, csv, json
Specify the output format type. Supported formats are: ['ascii', 'csv', 'json']. Only commands that produce tabular data support csv format. Default: ascii
Examples¶
Functions¶
How to list functions?¶
$ ngc cf fn list
+-----------------------+------------------------+------------------------+------------------------+------------------------+----------+
| Name | Id | Version | Container | Helm Chart | Status |
+-----------------------+------------------------+------------------------+------------------------+------------------------+----------+
| example-helm | 33352d58-10ae-4a38-ade | 28980bb7-844e-40e6-a05 | | https://helm.ngc.nvidi | ACTIVE |
| | 9-ca2468798315 | b-411236a6cc9a | | a.com/lpeb1gw2rozl/cha | |
| | | | | rts/peek-helm-chart- | |
| | | | | test-0.1.0.tgz | |
| example-container | 74510cba-6d29-43f6-88b | 6094e300-375f-42b2-9e7 | canary.nvcr.io/org | | ACTIVE |
| | 4-e1c3d055f3ab | b-f3d87ce59353 | /example_container | | |
+-----------------------+------------------------+------------------------+------------------------+------------------------+----------+
How to create a function?¶
$ ngc cf function create --name peek-test-e2e --inference-url /health --container-image lpeb1gw2rozl/pytriton_echo_sample:latest --secret key:value --secret key2:value2 --description "test-function"
--------------------------------------------------------------------
Function Information
Name: peek-test-e2e
Version: 6698aa4d-1ea3-4f95-b6cb-4bf879fe538a
ID: a7346e6f-4755-47cf-9163-6212e8db0900
Status: INACTIVE
Inference URL: /health
Description: test-function
Container Image: nvcr.io/lpeb1gw2rozl/pytriton_echo_sample:latest
Secret Keys: key2, key
Health Endpoint Information
URI: /v2/health/ready
Port: 8000
Timeout: PT10S
Protocol: HTTP
Expected Status Code: 200
--------------------------------------------------------------------
How to create a function with a json secret?¶
$ ngc cf function create --name peek-test-e2e --inference-url /health --container-image lpeb1gw2rozl/pytriton_echo_sample:latest --json-secret-file key.json
--------------------------------------------------------------------
Function Information
Name: peek-test-e2e
Version: 6698aa4d-1ea3-4f95-b6cb-4bf879fe538a
ID: a7346e6f-4755-47cf-9163-6212e8db0900
Status: INACTIVE
Inference URL: /health
Description: test-function
Container Image: nvcr.io/lpeb1gw2rozl/pytriton_echo_sample:latest
Secret Keys: key
Health Endpoint Information
URI: /v2/health/ready
Port: 8000
Timeout: PT10S
Protocol: HTTP
Expected Status Code: 200
--------------------------------------------------------------------
How to allow others to invoke your function across all versions?¶
$ ngc cf fn auth add 7835a350-921e-4fd3-8004-554b84c1367b --authorized-party "2lxz_k9-GnzBXPFt0OCagYnVXWwoUxNMJ3-IeT-qSYA"
-----------------------------------------------------------------------
Function Authorization Information
Function ID: 7835a350-921e-4fd3-8004-554b84c1367b
Author NCA ID: CMYBKSNNjtg1TQmSke-gHNGgMlFvA-dCRAI8gcHOBcw
Authorized Party NCAIds: 2lxz_k9-GnzBXPFt0OCagYnVXWwoUxNMJ3-IeT-qSYA
-----------------------------------------------------------------------
Tasks¶
How to create a task?¶
$ ngc cf task create --name task-name --container-image stg.nvcr.io/ax3ysqem02xw/testing-jet/server:main_11178569 --gpu-specification T10:g6.full:GFN --description description --max-runtime-duration 1H --max-queued-duration 1H --termination-grace-period-duration 1H --result-handling-strategy NONE
-------------------------------------------------------------------------------------------------------------------
Task Information
Name: task-name
ID: f2a41879-1ee4-46bf-9b51-5f3da7dd139b
Status: QUEUED
Description: description
Created: 2024-12-04T20:15:13.910726527Z
Last Updated:
Configuration Details
Container: stg.nvcr.io/ax3ysqem02xw/testing-jet/server:main_11178569
GPU Type: T10
Instance Type: g6.full
Max Runtime Duration: 1h
Max Queued Duration: 1h
Termination Grace Period: 1h
Result Handling: NONE
Result Path:
-------------------------------------------------------------------------------------------------------------------
How to list tasks?¶
$ ngc cf task list
+-----------------------+-------------------------------+--------------------------------------+--------------------------+--------------------------+
| Name | Status | Id | Created At | Last Updated |
+-----------------------+-------------------------------+--------------------------------------+--------------------------+--------------------------+
| yura-task-3 | EXCEEDED_MAX_RUNTIME_DURATION | 15bc4cb5-e09b-4b2c-af7d-19750f342b59 | 2024-11-27T20:03:50.017Z | 2024-11-27T20:16:34.447Z |
| yura-task-2 | CANCELED | 93847acd-1604-4c57-a293-f43265eb5cc0 | 2024-11-21T19:19:03.801Z | 2024-11-22T18:30:20.695Z |
+-----------------------+-------------------------------+--------------------------------------+--------------------------+--------------------------+
How to view a task's information?¶
$ ngc cf task info 1bd74059-c494-4b1f-bc6a-f028677546f4
--------------------------------------------------------------------
Task Information
Name: randomcontainer
ID: 1bd74059-c494-4b1f-bc6a-f028677546f4
Status: QUEUED
Description:
Created: 2024-10-16T19:09:08.006Z
Last Updated:
Configuration Details
Container: stg.nvcr.io/ax3ysqemo02xw/testing-jet/server:main_11257917
GPU Type: L40
Instance Type: fl40_1.br20_2xlarge
Max Runtime Duration: 10 minutes
Max Queued Duration: 3 days
Termination Grace Period: 10 minutes
Result Handling: NONE
Result Path:
--------------------------------------------------------------------
How to delete a task?¶
$ ngc cf task delete 1bd74059-c494-4b1f-bc6a-f028677546f4
Successfully deleted task 1bd74059-c494-4b1f-bc6a-f028677546f4.
How to cancel a task?¶
$ ngc cf task delete 1bd74059-c494-4b1f-bc6a-f028677546f4
Successfully canceled task: 1bd74059-c494-4b1f-bc6a-f028677546f4.
How to view a task's events?¶
$ ngc cf task events 1bd74059-c494-4b1f-bc6a-f028677546f4
+-----------------------+-------------------------------+-------------------------------------------------------------------------------------------------------------------------------------+
| Event Id | Created At | Message |
+--------------------------------------+--------------------------+---------------------------------------------------------------------------------------------------------------------------+
| 21027187-9426-4b26-b907-c5432943ff7f | 2024-10-19T10:13:46.213Z | Changing status from 'QUEUED' to 'ERRORED' with error 'SIS Request Id cbcc58d5-d580-45d8-ad62-8dbae76f1a4d: Request State |
| | | CANCELED; Instance Id UNKNOWN: Instance State UNKNOWN; Error Message - Backend 'null', GPU 'L40', Instance Type |
| | | 'gl40_1.br20_2xlarge': Capacity not available at this time' |
+--------------------------------------+--------------------------+---------------------------------------------------------------------------------------------------------------------------+
How to view a task's results?¶
$ ngc cf task events 1bd74059-c494-4b1f-bc6a-f028677546f4
How to view a task's logs?¶
$ ngc cf task events 1bd74059-c494-4b1f-bc6a-f028677546f4
+-----------------------------+----------------------------------------------------------------------------------------------------------------------------------+-----------+
| Timestamp | Message | Source |
+-----------------------------+----------------------------------------------------------------------------------------------------------------------------------+-----------+
| 2024-12-17 18:39:22.769 UTC | 2024-12-17T18:39:22.768Z INFO 1 --- [gdn-nvct-service] [io-8080-exec-10] c.n.k.n.s.task.TaskManagementService : Task id | inference |
| | '7e06ca3a-b907-4a71-8ae9-ba69f44d0210': Creating and launching new task with name 'yura-task-12' in account 'CMYBKSNNjtg1TQmSke- | |
| | gHNGgMlFvA-dCRAI8gcHOBcw' | |
| 2024-12-17 18:39:23.259 UTC | 2024-12-17T18:39:23.258Z DEBUG 1 --- [gdn-nvct-service] [io-8080-exec-10] c.n.k.nvct.service.task.TaskService : Task id | inference |
| | '7e06ca3a-b907-4a71-8ae9-ba69f44d0210': Saving task | |
| 2024-12-17 18:39:23.321 UTC | 2024-12-17T18:39:23.321Z DEBUG 1 --- [gdn-nvct-service] [io-8080-exec-10] c.n.k.nvct.service.task.TaskService : Task id | inference |
| | '7e06ca3a-b907-4a71-8ae9-ba69f44d0210': Saved task | |
| 2024-12-17 18:39:23.726 UTC | 2024-12-17T18:39:23.726Z INFO 1 --- [gdn-nvct-service] [io-8080-exec-10] c.n.kaizen.nvct.service.sis.SisService : Task id | inference |
| | '7e06ca3a-b907-4a71-8ae9-ba69f44d0210', SIS requestId 'b84f3b26-2ffe-489a-9489-1cf78518d138', GPU 'A100': Requested '1' instance | |
| 2024-12-17 18:39:24.471 UTC | 2024-12-17T18:39:24.471Z DEBUG 1 --- [gdn-nvct-service] [nio-8080-exec-7] c.n.k.nvct.service.task.TaskService : Task id | inference |
| | '7e06ca3a-b907-4a71-8ae9-ba69f44d0210': Fetching task | |
| 2024-12-17 18:39:24.476 UTC | 2024-12-17T18:39:24.476Z DEBUG 1 --- [gdn-nvct-service] [nio-8080-exec-7] c.n.k.nvct.service.task.TaskService : Task id | inference |
| | '7e06ca3a-b907-4a71-8ae9-ba69f44d0210': Fetched task | |
| 2024-12-17 18:39:24.476 UTC | 2024-12-17T18:39:24.476Z DEBUG 1 --- [gdn-nvct-service] [nio-8080-exec-7] c.n.k.nvct.service.task.TaskService : Task id | inference |
| | '7e06ca3a-b907-4a71-8ae9-ba69f44d0210': Fetching task | |
| 2024-12-17 18:39:24.476 UTC | 2024-12-17T18:39:24.476Z DEBUG 1 --- [gdn-nvct-service] [nio-8080-exec-7] c.n.k.n.s.task.TaskManagementService : Task id | inference |
| | '7e06ca3a-b907-4a71-8ae9-ba69f44d0210': Retrieving | |
| 2024-12-17 18:39:24.478 UTC | 2024-12-17T18:39:24.478Z DEBUG 1 --- [gdn-nvct-service] [nio-8080-exec-7] c.n.k.nvct.service.task.TaskService : Task id | inference |
| | '7e06ca3a-b907-4a71-8ae9-ba69f44d0210': Fetching task metadata | |
| 2024-12-17 18:39:24.478 UTC | 2024-12-17T18:39:24.478Z DEBUG 1 --- [gdn-nvct-service] [nio-8080-exec-7] c.n.k.nvct.service.task.TaskService : Task id | inference |
| | '7e06ca3a-b907-4a71-8ae9-ba69f44d0210': Fetched task | |
+-----------------------------+----------------------------------------------------------------------------------------------------------------------------------+-----------+
How to update a task's secrets?¶
$ ngc cf task update-secret 1bd74059-c494-4b1f-bc6a-f028677546f4 --secret secret1:new-value
Successfully updated secret.