ballerinax/azure.datalake Ballerina library

1.5.1

Overview

This is a generated connector for Azure DataLake Storage (Gen2) API version 2019-10-31 OpenAPI specification.

Azure Data Lake Storage provides storage for Hadoop and other big data workloads.

Prerequisites

Create an Azure account
Create an Azure Data Lake Storage(Gen2) account
Obtain tokens
- Use this guide to obtain the credentials which are needed to create the <CLIENT_ID> and <CLIENT_SECRET>

Quickstart

To use the Azure Data Lake Storage connector in your Ballerina application, update the .bal file as follows:

Step 1 - Import connector

First, import the ballerinax/azure.datalake module into the Ballerina project.


import ballerinax/azure.datalake;

Step 2 - Create a new connector instance

You can now make the connection configuration using the access token.


datalake:ClientConfig clientConfig = {
    auth : {
        token: token
    }
};

datalake:Client baseClient = check new Client(clientConfig, serviceUrl = "https://{accountName}.{dnsSuffix}");

Step 3 - Invoke connector operation

List file systems


public function main() {
    datalake:FilesystemList|error fileSystemList = baseClient->filesystemList("account");
    if (fileSystemList is datalake:FilesystemList) {
        log:printInfo("Filesyatem List " + fileSystemList.toString());
    } else {
        log:printInfo("Error Occured");
    }
}

Use bal run command to compile and run the Ballerina program

Clients

azure.datalake: Client

Isolated

This is a generated connector for Azure DataLake Storage (Gen2) API version 2019-10-31 OpenAPI specification. Azure Data Lake Storage provides storage for Hadoop and other big data workloads.

Constructor

Gets invoked to initialize the connector. The connector initialization requires setting the API credentials. Create a Azure Data Lake account and obtain OAuth tokens following this guide.

init (ConnectionConfig config, string serviceUrl)

config ConnectionConfig - The configurations to be used when initializing the connector

serviceUrl string - URL of the target service

filesystemList

Isolated FunctionRemote Function

function filesystemList(string 'resource, string? prefix, string? continuation, int? maxResults, string? xMsClientRequestId, int? timeout, string? xMsDate, string? xMsVersion) returns FilesystemList|error

List Filesystems

Parameters

'resource string - The value must be "account" for all account operations.

prefix string? (default ()) - Filters results to filesystems within the specified prefix.

continuation string? (default ()) - The number of filesystems returned with each invocation is limited. If the number of filesystems to be returned exceeds this limit, a continuation token is returned in the response header x-ms-continuation. When a continuation token is returned in the response, it must be specified in a subsequent invocation of the list operation to continue listing the filesystems.

maxResults int? (default ()) - An optional value that specifies the maximum number of items to return. If omitted or greater than 5,000, the response will include up to 5,000 items.

xMsClientRequestId string? (default ()) - A UUID recorded in the analytics logs for troubleshooting and correlation.

timeout int? (default ()) - An optional operation timeout value in seconds. The period begins when the request is received by the service. If the timeout value elapses before the operation completes, the operation fails.

xMsDate string? (default ()) - Specifies the Coordinated Universal Time (UTC) for the request. This is required when using shared key authorization.

xMsVersion string? (default ()) - Specifies the version of the REST protocol used for processing the request. This is required when using shared key authorization.

Return Type

FilesystemList|error - OK

pathList

Isolated FunctionRemote Function

function pathList(string filesystem, string 'resource, boolean recursive, string? xMsClientRequestId, int? timeout, string? xMsDate, string? xMsVersion, string? directory, string? continuation, int? maxResults, boolean? upn) returns PathList|error

List Paths

Parameters

filesystem string - The filesystem identifier. The value must start and end with a letter or number and must contain only letters, numbers, and the dash (-) character. Consecutive dashes are not permitted. All letters must be lowercase. The value must have between 3 and 63 characters.

'resource string - The value must be "filesystem" for all filesystem operations.

recursive boolean - If "true", all paths are listed; otherwise, only paths at the root of the filesystem are listed. If "directory" is specified, the list will only include paths that share the same root.

xMsClientRequestId string? (default ()) - A UUID recorded in the analytics logs for troubleshooting and correlation.

timeout int? (default ()) - An optional operation timeout value in seconds. The period begins when the request is received by the service. If the timeout value elapses before the operation completes, the operation fails.

xMsDate string? (default ()) - Specifies the Coordinated Universal Time (UTC) for the request. This is required when using shared key authorization.

xMsVersion string? (default ()) - Specifies the version of the REST protocol used for processing the request. This is required when using shared key authorization.

directory string? (default ()) - Filters results to paths within the specified directory. An error occurs if the directory does not exist.

continuation string? (default ()) - The number of paths returned with each invocation is limited. If the number of paths to be returned exceeds this limit, a continuation token is returned in the response header x-ms-continuation. When a continuation token is returned in the response, it must be specified in a subsequent invocation of the list operation to continue listing the paths.

maxResults int? (default ()) - An optional value that specifies the maximum number of items to return. If omitted or greater than 5,000, the response will include up to 5,000 items.

upn boolean? (default ()) - Optional. Valid only when Hierarchical Namespace is enabled for the account. If "true", the user identity values returned in the owner and group fields of each list entry will be transformed from Azure Active Directory Object IDs to User Principal Names. If "false", the values will be returned as Azure Active Directory Object IDs. The default value is false. Note that group and application Object IDs are not translated because they do not have unique friendly names.

Return Type

PathList|error - Ok

filesystemCreate

Isolated FunctionRemote Function

function filesystemCreate(string filesystem, string 'resource, string? xMsClientRequestId, int? timeout, string? xMsDate, string? xMsVersion, string? xMsProperties) returns Response|error

Create Filesystem

Parameters

filesystem string - The filesystem identifier. The value must start and end with a letter or number and must contain only letters, numbers, and the dash (-) character. Consecutive dashes are not permitted. All letters must be lowercase. The value must have between 3 and 63 characters.

'resource string - The value must be "filesystem" for all filesystem operations.

xMsClientRequestId string? (default ()) - A UUID recorded in the analytics logs for troubleshooting and correlation.

timeout int? (default ()) - An optional operation timeout value in seconds. The period begins when the request is received by the service. If the timeout value elapses before the operation completes, the operation fails.

xMsDate string? (default ()) - Specifies the Coordinated Universal Time (UTC) for the request. This is required when using shared key authorization.

xMsVersion string? (default ()) - Specifies the version of the REST protocol used for processing the request. This is required when using shared key authorization.

xMsProperties string? (default ()) - User-defined properties to be stored with the filesystem, in the format of a comma-separated list of name and value pairs "n1=v1, n2=v2, ...", where each value is a base64 encoded string. Note that the string may only contain ASCII characters in the ISO-8859-1 character set.

Return Type

Response|error - Created

filesystemDelete

Isolated FunctionRemote Function

function filesystemDelete(string filesystem, string 'resource, string? xMsClientRequestId, int? timeout, string? xMsDate, string? xMsVersion, string? ifModifiedSince, string? ifUnmodifiedSince) returns Response|error

Delete Filesystem

Parameters

filesystem string - The filesystem identifier. The value must start and end with a letter or number and must contain only letters, numbers, and the dash (-) character. Consecutive dashes are not permitted. All letters must be lowercase. The value must have between 3 and 63 characters.

'resource string - The value must be "filesystem" for all filesystem operations.

xMsClientRequestId string? (default ()) - A UUID recorded in the analytics logs for troubleshooting and correlation.

timeout int? (default ()) - An optional operation timeout value in seconds. The period begins when the request is received by the service. If the timeout value elapses before the operation completes, the operation fails.

xMsDate string? (default ()) - Specifies the Coordinated Universal Time (UTC) for the request. This is required when using shared key authorization.

xMsVersion string? (default ()) - Specifies the version of the REST protocol used for processing the request. This is required when using shared key authorization.

ifModifiedSince string? (default ()) - Optional. A date and time value. Specify this header to perform the operation only if the resource has been modified since the specified date and time.

ifUnmodifiedSince string? (default ()) - Optional. A date and time value. Specify this header to perform the operation only if the resource has not been modified since the specified date and time.

Return Type

Response|error - Accepted

filesystemSetproperties

Isolated FunctionRemote Function

function filesystemSetproperties(string filesystem, string 'resource, string? xMsClientRequestId, int? timeout, string? xMsDate, string? xMsVersion, string? xMsProperties, string? ifModifiedSince, string? ifUnmodifiedSince) returns Response|error

Set Filesystem Properties

Parameters

filesystem string - The filesystem identifier. The value must start and end with a letter or number and must contain only letters, numbers, and the dash (-) character. Consecutive dashes are not permitted. All letters must be lowercase. The value must have between 3 and 63 characters.

'resource string - The value must be "filesystem" for all filesystem operations.

xMsClientRequestId string? (default ()) - A UUID recorded in the analytics logs for troubleshooting and correlation.

timeout int? (default ()) - An optional operation timeout value in seconds. The period begins when the request is received by the service. If the timeout value elapses before the operation completes, the operation fails.

xMsDate string? (default ()) - Specifies the Coordinated Universal Time (UTC) for the request. This is required when using shared key authorization.

xMsVersion string? (default ()) - Specifies the version of the REST protocol used for processing the request. This is required when using shared key authorization.

xMsProperties string? (default ()) - Optional. User-defined properties to be stored with the filesystem, in the format of a comma-separated list of name and value pairs "n1=v1, n2=v2, ...", where each value is a base64 encoded string. Note that the string may only contain ASCII characters in the ISO-8859-1 character set. If the filesystem exists, any properties not included in the list will be removed. All properties are removed if the header is omitted. To merge new and existing properties, first get all existing properties and the current E-Tag, then make a conditional request with the E-Tag and include values for all properties.

ifModifiedSince string? (default ()) - Optional. A date and time value. Specify this header to perform the operation only if the resource has been modified since the specified date and time.

ifUnmodifiedSince string? (default ()) - Optional. A date and time value. Specify this header to perform the operation only if the resource has not been modified since the specified date and time.

Return Type

Response|error - Ok

filesystemGetproperties

Isolated FunctionRemote Function

function filesystemGetproperties(string filesystem, string 'resource, string? xMsClientRequestId, int? timeout, string? xMsDate, string? xMsVersion) returns Response|error

Get Filesystem Properties.

Parameters

filesystem string - The filesystem identifier. The value must start and end with a letter or number and must contain only letters, numbers, and the dash (-) character. Consecutive dashes are not permitted. All letters must be lowercase. The value must have between 3 and 63 characters.

'resource string - The value must be "filesystem" for all filesystem operations.

xMsClientRequestId string? (default ()) - A UUID recorded in the analytics logs for troubleshooting and correlation.

timeout int? (default ()) - An optional operation timeout value in seconds. The period begins when the request is received by the service. If the timeout value elapses before the operation completes, the operation fails.

xMsDate string? (default ()) - Specifies the Coordinated Universal Time (UTC) for the request. This is required when using shared key authorization.

xMsVersion string? (default ()) - Specifies the version of the REST protocol used for processing the request. This is required when using shared key authorization.

Return Type

Response|error - Ok

pathRead

Isolated FunctionRemote Function

function pathRead(string filesystem, string pathParam, string? xMsClientRequestId, int? timeout, string? xMsDate, string? xMsVersion, string? range, string? xMsLeaseId, boolean? xMsRangeGetContentMd5, string? ifMatch, string? ifNoneMatch, string? ifModifiedSince, string? ifUnmodifiedSince) returns string|error

Read File

Parameters

filesystem string - The filesystem identifier.

pathParam string - The file or directory path.

xMsClientRequestId string? (default ()) - A UUID recorded in the analytics logs for troubleshooting and correlation.

timeout int? (default ()) - An optional operation timeout value in seconds. The period begins when the request is received by the service. If the timeout value elapses before the operation completes, the operation fails.

xMsDate string? (default ()) - Specifies the Coordinated Universal Time (UTC) for the request. This is required when using shared key authorization.

xMsVersion string? (default ()) - Specifies the version of the REST protocol used for processing the request. This is required when using shared key authorization.

range string? (default ()) - The HTTP Range request header specifies one or more byte ranges of the resource to be retrieved.

xMsLeaseId string? (default ()) - Optional. If this header is specified, the operation will be performed only if both of the following conditions are met: i) the path's lease is currently active and ii) the lease ID specified in the request matches that of the path.

xMsRangeGetContentMd5 boolean? (default ()) - Optional. When this header is set to "true" and specified together with the Range header, the service returns the MD5 hash for the range, as long as the range is less than or equal to 4MB in size. If this header is specified without the Range header, the service returns status code 400 (Bad Request). If this header is set to true when the range exceeds 4 MB in size, the service returns status code 400 (Bad Request).

ifMatch string? (default ()) - Optional. An ETag value. Specify this header to perform the operation only if the resource's ETag matches the value specified. The ETag must be specified in quotes.

ifNoneMatch string? (default ()) - Optional. An ETag value or the special wildcard ("*") value. Specify this header to perform the operation only if the resource's ETag does not match the value specified. The ETag must be specified in quotes.

ifModifiedSince string? (default ()) - Optional. A date and time value. Specify this header to perform the operation only if the resource has been modified since the specified date and time.

ifUnmodifiedSince string? (default ()) - Optional. A date and time value. Specify this header to perform the operation only if the resource has not been modified since the specified date and time.

Return Type

string|error - Ok

pathCreate

Isolated FunctionRemote Function

function pathCreate(string filesystem, string pathParam, string? xMsClientRequestId, int? timeout, string? xMsDate, string? xMsVersion, string? 'resource, string? continuation, string? mode, string? cacheControl, string? contentEncoding, string? contentLanguage, string? contentDisposition, string? xMsCacheControl, string? xMsContentType, string? xMsContentEncoding, string? xMsContentLanguage, string? xMsContentDisposition, string? xMsRenameSource, string? xMsLeaseId, string? xMsSourceLeaseId, string? xMsProperties, string? xMsPermissions, string? xMsUmask, string? ifMatch, string? ifNoneMatch, string? ifModifiedSince, string? ifUnmodifiedSince, string? xMsSourceIfMatch, string? xMsSourceIfNoneMatch, string? xMsSourceIfModifiedSince, string? xMsSourceIfUnmodifiedSince) returns Response|error

Create File | Create Directory | Rename File | Rename Directory

Parameters

filesystem string - The filesystem identifier.

pathParam string - The file or directory path.

xMsClientRequestId string? (default ()) - A UUID recorded in the analytics logs for troubleshooting and correlation.

timeout int? (default ()) - An optional operation timeout value in seconds. The period begins when the request is received by the service. If the timeout value elapses before the operation completes, the operation fails.

xMsDate string? (default ()) - Specifies the Coordinated Universal Time (UTC) for the request. This is required when using shared key authorization.

xMsVersion string? (default ()) - Specifies the version of the REST protocol used for processing the request. This is required when using shared key authorization.

'resource string? (default ()) - Required only for Create File and Create Directory. The value must be "file" or "directory".

continuation string? (default ()) - Optional. When renaming a directory, the number of paths that are renamed with each invocation is limited. If the number of paths to be renamed exceeds this limit, a continuation token is returned in this response header. When a continuation token is returned in the response, it must be specified in a subsequent invocation of the rename operation to continue renaming the directory.

mode string? (default ()) - Optional. Valid only when namespace is enabled. This parameter determines the behavior of the rename operation. The value must be "legacy" or "posix", and the default value will be "posix".

cacheControl string? (default ()) - Optional. The service stores this value and includes it in the "Cache-Control" response header for "Read File" operations for "Read File" operations.

contentEncoding string? (default ()) - Optional. Specifies which content encodings have been applied to the file. This value is returned to the client when the "Read File" operation is performed.

contentLanguage string? (default ()) - Optional. Specifies the natural language used by the intended audience for the file.

contentDisposition string? (default ()) - Optional. The service stores this value and includes it in the "Content-Disposition" response header for "Read File" operations.

xMsCacheControl string? (default ()) - Optional. The service stores this value and includes it in the "Cache-Control" response header for "Read File" operations.

xMsContentType string? (default ()) - Optional. The service stores this value and includes it in the "Content-Type" response header for "Read File" operations.

xMsContentEncoding string? (default ()) - Optional. The service stores this value and includes it in the "Content-Encoding" response header for "Read File" operations.

xMsContentLanguage string? (default ()) - Optional. The service stores this value and includes it in the "Content-Language" response header for "Read File" operations.

xMsContentDisposition string? (default ()) - Optional. The service stores this value and includes it in the "Content-Disposition" response header for "Read File" operations.

xMsRenameSource string? (default ()) - An optional file or directory to be renamed. The value must have the following format: "/{filesystem}/{path}". If "x-ms-properties" is specified, the properties will overwrite the existing properties; otherwise, the existing properties will be preserved. This value must be a URL percent-encoded string. Note that the string may only contain ASCII characters in the ISO-8859-1 character set.

xMsLeaseId string? (default ()) - Optional. A lease ID for the path specified in the URI. The path to be overwritten must have an active lease and the lease ID must match.

xMsSourceLeaseId string? (default ()) - Optional for rename operations. A lease ID for the source path. The source path must have an active lease and the lease ID must match.

xMsProperties string? (default ()) - Optional. User-defined properties to be stored with the file or directory, in the format of a comma-separated list of name and value pairs "n1=v1, n2=v2, ...", where each value is a base64 encoded string. Note that the string may only contain ASCII characters in the ISO-8859-1 character set.

xMsPermissions string? (default ()) - Optional and only valid if Hierarchical Namespace is enabled for the account. Sets POSIX access permissions for the file owner, the file owning group, and others. Each class may be granted read, write, or execute permission. The sticky bit is also supported. Both symbolic (rwxrw-rw-) and 4-digit octal notation (e.g. 0766) are supported.

xMsUmask string? (default ()) - Optional and only valid if Hierarchical Namespace is enabled for the account. When creating a file or directory and the parent folder does not have a default ACL, the umask restricts the permissions of the file or directory to be created. The resulting permission is given by p & ^u, where p is the permission and u is the umask. For example, if p is 0777 and u is 0057, then the resulting permission is 0720. The default permission is 0777 for a directory and 0666 for a file. The default umask is 0027. The umask must be specified in 4-digit octal notation (e.g. 0766).

ifMatch string? (default ()) - Optional. An ETag value. Specify this header to perform the operation only if the resource's ETag matches the value specified. The ETag must be specified in quotes.

ifNoneMatch string? (default ()) - Optional. An ETag value or the special wildcard ("*") value. Specify this header to perform the operation only if the resource's ETag does not match the value specified. The ETag must be specified in quotes.

ifModifiedSince string? (default ()) - Optional. A date and time value. Specify this header to perform the operation only if the resource has been modified since the specified date and time.

ifUnmodifiedSince string? (default ()) - Optional. A date and time value. Specify this header to perform the operation only if the resource has not been modified since the specified date and time.

xMsSourceIfMatch string? (default ()) - Optional. An ETag value. Specify this header to perform the rename operation only if the source's ETag matches the value specified. The ETag must be specified in quotes.

xMsSourceIfNoneMatch string? (default ()) - Optional. An ETag value or the special wildcard ("*") value. Specify this header to perform the rename operation only if the source's ETag does not match the value specified. The ETag must be specified in quotes.

xMsSourceIfModifiedSince string? (default ()) - Optional. A date and time value. Specify this header to perform the rename operation only if the source has been modified since the specified date and time.

xMsSourceIfUnmodifiedSince string? (default ()) - Optional. A date and time value. Specify this header to perform the rename operation only if the source has not been modified since the specified date and time.

Return Type

Response|error - The file or directory was created.

pathLease

Isolated FunctionRemote Function

function pathLease(string filesystem, string pathParam, string xMsLeaseAction, string? xMsClientRequestId, int? timeout, string? xMsDate, string? xMsVersion, int? xMsLeaseDuration, int? xMsLeaseBreakPeriod, string? xMsLeaseId, string? xMsProposedLeaseId, string? ifMatch, string? ifNoneMatch, string? ifModifiedSince, string? ifUnmodifiedSince) returns Response|error

Lease Path

Parameters

filesystem string - The filesystem identifier.

pathParam string - The file or directory path.

xMsLeaseAction string - There are five lease actions: "acquire", "break", "change", "renew", and "release". Use "acquire" and specify the "x-ms-proposed-lease-id" and "x-ms-lease-duration" to acquire a new lease. Use "break" to break an existing lease. When a lease is broken, the lease break period is allowed to elapse, during which time no lease operation except break and release can be performed on the file. When a lease is successfully broken, the response indicates the interval in seconds until a new lease can be acquired. Use "change" and specify the current lease ID in "x-ms-lease-id" and the new lease ID in "x-ms-proposed-lease-id" to change the lease ID of an active lease. Use "renew" and specify the "x-ms-lease-id" to renew an existing lease. Use "release" and specify the "x-ms-lease-id" to release a lease.

xMsClientRequestId string? (default ()) - A UUID recorded in the analytics logs for troubleshooting and correlation.

timeout int? (default ()) - An optional operation timeout value in seconds. The period begins when the request is received by the service. If the timeout value elapses before the operation completes, the operation fails.

xMsDate string? (default ()) - Specifies the Coordinated Universal Time (UTC) for the request. This is required when using shared key authorization.

xMsVersion string? (default ()) - Specifies the version of the REST protocol used for processing the request. This is required when using shared key authorization.

xMsLeaseDuration int? (default ()) - The lease duration is required to acquire a lease, and specifies the duration of the lease in seconds. The lease duration must be between 15 and 60 seconds or -1 for infinite lease.

xMsLeaseBreakPeriod int? (default ()) - The lease break period duration is optional to break a lease, and specifies the break period of the lease in seconds. The lease break duration must be between 0 and 60 seconds.

xMsLeaseId string? (default ()) - Required when "x-ms-lease-action" is "renew", "change" or "release". For the renew and release actions, this must match the current lease ID.

xMsProposedLeaseId string? (default ()) - Required when "x-ms-lease-action" is "acquire" or "change". A lease will be acquired with this lease ID if the operation is successful.

ifMatch string? (default ()) - Optional. An ETag value. Specify this header to perform the operation only if the resource's ETag matches the value specified. The ETag must be specified in quotes.

ifNoneMatch string? (default ()) - Optional. An ETag value or the special wildcard ("*") value. Specify this header to perform the operation only if the resource's ETag does not match the value specified. The ETag must be specified in quotes.

ifModifiedSince string? (default ()) - Optional. A date and time value. Specify this header to perform the operation only if the resource has been modified since the specified date and time.

ifUnmodifiedSince string? (default ()) - Optional. A date and time value. Specify this header to perform the operation only if the resource has not been modified since the specified date and time.

Return Type

Response|error - The "renew", "change" or "release" action was successful.

pathDelete

Isolated FunctionRemote Function

function pathDelete(string filesystem, string pathParam, string? xMsClientRequestId, int? timeout, string? xMsDate, string? xMsVersion, boolean? recursive, string? continuation, string? xMsLeaseId, string? ifMatch, string? ifNoneMatch, string? ifModifiedSince, string? ifUnmodifiedSince) returns Response|error

Delete File | Delete Directory

Parameters

filesystem string - The filesystem identifier.

pathParam string - The file or directory path.

xMsClientRequestId string? (default ()) - A UUID recorded in the analytics logs for troubleshooting and correlation.

timeout int? (default ()) - An optional operation timeout value in seconds. The period begins when the request is received by the service. If the timeout value elapses before the operation completes, the operation fails.

xMsDate string? (default ()) - Specifies the Coordinated Universal Time (UTC) for the request. This is required when using shared key authorization.

xMsVersion string? (default ()) - Specifies the version of the REST protocol used for processing the request. This is required when using shared key authorization.

recursive boolean? (default ()) - Required and valid only when the resource is a directory. If "true", all paths beneath the directory will be deleted. If "false" and the directory is non-empty, an error occurs.

continuation string? (default ()) - Optional. When deleting a directory, the number of paths that are deleted with each invocation is limited. If the number of paths to be deleted exceeds this limit, a continuation token is returned in this response header. When a continuation token is returned in the response, it must be specified in a subsequent invocation of the delete operation to continue deleting the directory.

xMsLeaseId string? (default ()) - The lease ID must be specified if there is an active lease.

ifMatch string? (default ()) - Optional. An ETag value. Specify this header to perform the operation only if the resource's ETag matches the value specified. The ETag must be specified in quotes.

ifNoneMatch string? (default ()) - Optional. An ETag value or the special wildcard ("*") value. Specify this header to perform the operation only if the resource's ETag does not match the value specified. The ETag must be specified in quotes.

ifModifiedSince string? (default ()) - Optional. A date and time value. Specify this header to perform the operation only if the resource has been modified since the specified date and time.

ifUnmodifiedSince string? (default ()) - Optional. A date and time value. Specify this header to perform the operation only if the resource has not been modified since the specified date and time.

Return Type

Response|error - The file was deleted.

pathUpdate

Isolated FunctionRemote Function

function pathUpdate(string filesystem, string pathParam, string action, byte[] payload, string? xMsClientRequestId, int? timeout, string? xMsDate, string? xMsVersion, int? position, boolean? retainUncommittedData, boolean? close, int? contentLength, string? contentMd5, string? xMsLeaseId, string? xMsCacheControl, string? xMsContentType, string? xMsContentDisposition, string? xMsContentEncoding, string? xMsContentLanguage, string? xMsContentMd5, string? xMsProperties, string? xMsOwner, string? xMsGroup, string? xMsPermissions, string? xMsAcl, string? ifMatch, string? ifNoneMatch, string? ifModifiedSince, string? ifUnmodifiedSince) returns Response|error

Append Data | Flush Data | Set Properties | Set Access Control

Parameters

filesystem string - The filesystem identifier.

pathParam string - The file or directory path.

action string - The action must be "append" to upload data to be appended to a file, "flush" to flush previously uploaded data to a file, "setProperties" to set the properties of a file or directory, or "setAccessControl" to set the owner, group, permissions, or access control list for a file or directory. Note that Hierarchical Namespace must be enabled for the account in order to use access control. Also note that the Access Control List (ACL) includes permissions for the owner, owning group, and others, so the x-ms-permissions and x-ms-acl request headers are mutually exclusive.

payload byte[] - Valid only for append operations. The data to be uploaded and appended to the file.

xMsClientRequestId string? (default ()) - A UUID recorded in the analytics logs for troubleshooting and correlation.

timeout int? (default ()) - An optional operation timeout value in seconds. The period begins when the request is received by the service. If the timeout value elapses before the operation completes, the operation fails.

xMsDate string? (default ()) - Specifies the Coordinated Universal Time (UTC) for the request. This is required when using shared key authorization.

xMsVersion string? (default ()) - Specifies the version of the REST protocol used for processing the request. This is required when using shared key authorization.

position int? (default ()) - This parameter allows the caller to upload data in parallel and control the order in which it is appended to the file. It is required when uploading data to be appended to the file and when flushing previously uploaded data to the file. The value must be the position where the data is to be appended. Uploaded data is not immediately flushed, or written, to the file. To flush, the previously uploaded data must be contiguous, the position parameter must be specified and equal to the length of the file after all data has been written, and there must not be a request entity body included with the request.

retainUncommittedData boolean? (default ()) - Valid only for flush operations. If "true", uncommitted data is retained after the flush operation completes; otherwise, the uncommitted data is deleted after the flush operation. The default is false. Data at offsets less than the specified position are written to the file when flush succeeds, but this optional parameter allows data after the flush position to be retained for a future flush operation.

close boolean? (default ()) - Azure Storage Events allow applications to receive notifications when files change. When Azure Storage Events are enabled, a file changed event is raised. This event has a property indicating whether this is the final change to distinguish the difference between an intermediate flush to a file stream and the final close of a file stream. The close query parameter is valid only when the action is "flush" and change notifications are enabled. If the value of close is "true" and the flush operation completes successfully, the service raises a file change notification with a property indicating that this is the final update (the file stream has been closed). If "false" a change notification is raised indicating the file has changed. The default is false. This query parameter is set to true by the Hadoop ABFS driver to indicate that the file stream has been closed."

contentLength int? (default ()) - Required for "Append Data" and "Flush Data". Must be 0 for "Flush Data". Must be the length of the request content in bytes for "Append Data".

contentMd5 string? (default ()) - Optional. An MD5 hash of the request content. This header is valid on "Append" and "Flush" operations. This hash is used to verify the integrity of the request content during transport. When this header is specified, the storage service compares the hash of the content that has arrived with this header value. If the two hashes do not match, the operation will fail with error code 400 (Bad Request). Note that this MD5 hash is not stored with the file. This header is associated with the request content, and not with the stored content of the file itself.

xMsLeaseId string? (default ()) - The lease ID must be specified if there is an active lease.

xMsCacheControl string? (default ()) - Optional and only valid for flush and set properties operations. The service stores this value and includes it in the "Cache-Control" response header for "Read File" operations.

xMsContentType string? (default ()) - Optional and only valid for flush and set properties operations. The service stores this value and includes it in the "Content-Type" response header for "Read File" operations.

xMsContentDisposition string? (default ()) - Optional and only valid for flush and set properties operations. The service stores this value and includes it in the "Content-Disposition" response header for "Read File" operations.

xMsContentEncoding string? (default ()) - Optional and only valid for flush and set properties operations. The service stores this value and includes it in the "Content-Encoding" response header for "Read File" operations.

xMsContentLanguage string? (default ()) - Optional and only valid for flush and set properties operations. The service stores this value and includes it in the "Content-Language" response header for "Read File" operations.

xMsContentMd5 string? (default ()) - Optional and only valid for "Flush & Set Properties" operations. The service stores this value and includes it in the "Content-Md5" response header for "Read & Get Properties" operations. If this property is not specified on the request, then the property will be cleared for the file. Subsequent calls to "Read & Get Properties" will not return this property unless it is explicitly set on that file again.

xMsProperties string? (default ()) - Optional. User-defined properties to be stored with the file or directory, in the format of a comma-separated list of name and value pairs "n1=v1, n2=v2, ...", where each value is a base64 encoded string. Note that the string may only contain ASCII characters in the ISO-8859-1 character set. Valid only for the setProperties operation. If the file or directory exists, any properties not included in the list will be removed. All properties are removed if the header is omitted. To merge new and existing properties, first get all existing properties and the current E-Tag, then make a conditional request with the E-Tag and include values for all properties.

xMsOwner string? (default ()) - Optional and valid only for the setAccessControl operation. Sets the owner of the file or directory.

xMsGroup string? (default ()) - Optional and valid only for the setAccessControl operation. Sets the owning group of the file or directory.

xMsPermissions string? (default ()) - Optional and only valid if Hierarchical Namespace is enabled for the account. Sets POSIX access permissions for the file owner, the file owning group, and others. Each class may be granted read, write, or execute permission. The sticky bit is also supported. Both symbolic (rwxrw-rw-) and 4-digit octal notation (e.g. 0766) are supported. Invalid in conjunction with x-ms-acl.

xMsAcl string? (default ()) - Optional and valid only for the setAccessControl operation. Sets POSIX access control rights on files and directories. The value is a comma-separated list of access control entries that fully replaces the existing access control list (ACL). Each access control entry (ACE) consists of a scope, a type, a user or group identifier, and permissions in the format "[scope:][type]:[id]:[permissions]". The scope must be "default" to indicate the ACE belongs to the default ACL for a directory; otherwise scope is implicit and the ACE belongs to the access ACL. There are four ACE types: "user" grants rights to the owner or a named user, "group" grants rights to the owning group or a named group, "mask" restricts rights granted to named users and the members of groups, and "other" grants rights to all users not found in any of the other entries. The user or group identifier is omitted for entries of type "mask" and "other". The user or group identifier is also omitted for the owner and owning group. The permission field is a 3-character sequence where the first character is 'r' to grant read access, the second character is 'w' to grant write access, and the third character is 'x' to grant execute permission. If access is not granted, the '-' character is used to denote that the permission is denied. For example, the following ACL grants read, write, and execute rights to the file owner and john.doe@contoso, the read right to the owning group, and nothing to everyone else: "user::rwx,user:john.doe@contoso:rwx,group::r--,other::---,mask=rwx". Invalid in conjunction with x-ms-permissions.

ifMatch string? (default ()) - Optional for Flush Data and Set Properties, but invalid for Append Data. An ETag value. Specify this header to perform the operation only if the resource's ETag matches the value specified. The ETag must be specified in quotes.

ifNoneMatch string? (default ()) - Optional for Flush Data and Set Properties, but invalid for Append Data. An ETag value or the special wildcard ("*") value. Specify this header to perform the operation only if the resource's ETag does not match the value specified. The ETag must be specified in quotes.

ifModifiedSince string? (default ()) - Optional for Flush Data and Set Properties, but invalid for Append Data. A date and time value. Specify this header to perform the operation only if the resource has been modified since the specified date and time.

ifUnmodifiedSince string? (default ()) - Optional for Flush Data and Set Properties, but invalid for Append Data. A date and time value. Specify this header to perform the operation only if the resource has not been modified since the specified date and time.

Return Type

Response|error - The data was flushed (written) to the file or the properties were set successfully.

pathGetproperties

Isolated FunctionRemote Function

function pathGetproperties(string filesystem, string pathParam, string? xMsClientRequestId, int? timeout, string? xMsDate, string? xMsVersion, string? action, boolean? upn, string? fsAction, string? xMsLeaseId, string? ifMatch, string? ifNoneMatch, string? ifModifiedSince, string? ifUnmodifiedSince) returns Response|error

Get Properties | Get Status | Get Access Control List | Check Access

Parameters

filesystem string - The filesystem identifier.

pathParam string - The file or directory path.

xMsClientRequestId string? (default ()) - A UUID recorded in the analytics logs for troubleshooting and correlation.

timeout int? (default ()) - An optional operation timeout value in seconds. The period begins when the request is received by the service. If the timeout value elapses before the operation completes, the operation fails.

xMsDate string? (default ()) - Specifies the Coordinated Universal Time (UTC) for the request. This is required when using shared key authorization.

xMsVersion string? (default ()) - Specifies the version of the REST protocol used for processing the request. This is required when using shared key authorization.

action string? (default ()) - Optional. If the value is "getStatus" only the system defined properties for the path are returned. If the value is "getAccessControl" the access control list is returned in the response headers (Hierarchical Namespace must be enabled for the account), otherwise the properties are returned.

upn boolean? (default ()) - Optional. Valid only when Hierarchical Namespace is enabled for the account. If "true", the user identity values returned in the x-ms-owner, x-ms-group, and x-ms-acl response headers will be transformed from Azure Active Directory Object IDs to User Principal Names. If "false", the values will be returned as Azure Active Directory Object IDs. The default value is false. Note that group and application Object IDs are not translated because they do not have unique friendly names.

fsAction string? (default ()) - Required only for check access action. Valid only when Hierarchical Namespace is enabled for the account. File system operation read/write/execute in string form, matching regex pattern '[rwx-]{3}'

xMsLeaseId string? (default ()) - Optional. If this header is specified, the operation will be performed only if both of the following conditions are met: i) the path's lease is currently active and ii) the lease ID specified in the request matches that of the path.

ifMatch string? (default ()) - Optional. An ETag value. Specify this header to perform the operation only if the resource's ETag matches the value specified. The ETag must be specified in quotes.

ifNoneMatch string? (default ()) - Optional. An ETag value or the special wildcard ("*") value. Specify this header to perform the operation only if the resource's ETag does not match the value specified. The ETag must be specified in quotes.

ifModifiedSince string? (default ()) - Optional. A date and time value. Specify this header to perform the operation only if the resource has been modified since the specified date and time.

ifUnmodifiedSince string? (default ()) - Optional. A date and time value. Specify this header to perform the operation only if the resource has not been modified since the specified date and time.

Return Type

Response|error - Returns all properties for the file or directory.

Records

azure.datalake: ClientHttp1Settings

Closed record

Provides settings related to HTTP/1.x protocol.

Fields

keepAlive KeepAlive(default http:KEEPALIVE_AUTO) - Specifies whether to reuse a connection for multiple requests

chunking Chunking(default http:CHUNKING_AUTO) - The chunking behaviour of the request

proxy ProxyConfig? - Proxy server related options

azure.datalake: ConnectionConfig

Closed record

Provides a set of configurations for controlling the behaviours when communicating with a remote HTTP endpoint.

Fields

auth BearerTokenConfig - Configurations related to client authentication

httpVersion HttpVersion(default http:HTTP_2_0) - The HTTP version understood by the client

http1Settings ClientHttp1Settings? - Configurations related to HTTP/1.x protocol

http2Settings ClientHttp2Settings? - Configurations related to HTTP/2 protocol

timeout decimal(default 60) - The maximum time to wait (in seconds) for a response before closing the connection

forwarded string(default "disable") - The choice of setting forwarded/x-forwarded header

poolConfig PoolConfiguration? - Configurations associated with request pooling

cache CacheConfig? - HTTP caching related configurations

compression Compression(default http:COMPRESSION_AUTO) - Specifies the way of handling compression (accept-encoding) header

circuitBreaker CircuitBreakerConfig? - Configurations associated with the behaviour of the Circuit Breaker

retryConfig RetryConfig? - Configurations associated with retrying

responseLimits ResponseLimitConfigs? - Configurations associated with inbound response size limits

secureSocket ClientSecureSocket? - SSL/TLS-related options

proxy ProxyConfig? - Proxy server related options

validation boolean(default true) - Enables the inbound payload validation functionality which provided by the constraint package. Enabled by default

azure.datalake: DataLakeStorageError

The service error response object.

Fields

'error DatalakestorageerrorError? - The service error response object.

azure.datalake: DatalakestorageerrorError

The service error response object.

Fields

code string? - The service error code

message string? - The service error message

azure.datalake: Filesystem

Information about file system

Fields

name string? - The path of the file system

lastModified string? - The timestamp when the file system was modified for the last time

eTag string? - Unique identifier of the file system

azure.datalake: FilesystemList

Fields

filesystems Filesystem[]? -

azure.datalake: Path

The information about files and folders.

Fields

name string? - The path of the file or folder

isDirectory boolean? - Determines if the resource is a folder or a file

lastModified string? - The timestamp when the file was modified for the last time

eTag string? - Unique identifier of the file or folder

contentLength int? - Determines the size of the file in bytes

owner string? - The name of the owner

group string? - The name of a group

permissions string? - Permissions set to the file/folder

azure.datalake: PathList

Fields

paths Path[]? -

azure.datalake: ProxyConfig

Closed record

Proxy server configurations to be used with the HTTP client endpoint.

Fields

host string(default "") - Host name of the proxy server

port int(default 0) - Proxy server port

userName string(default "") - Proxy server username

password string(default "") - Proxy server password

Import

import ballerinax/azure.datalake;

Metadata

Released date: over 1 year ago

Version: 1.5.1

License: Apache-2.0

Compatibility

Platform: any

Ballerina version: 2201.4.1

GraalVM compatible: Yes

Pull count

Total: 8

Current verison: 3

Weekly downloads

Source repository

Keywords

Business Intelligence/Analytics

Cost/Paid

Vendor/Microsoft

Contributors

Other versions

1.5.1

1.5.0 1.4.0 1.2.0 1.1.0

Dependencies

ballerina/url/2.2.3 ballerina/http/2.6.0

Cookie policy

Delete policy

clients

records

ballerinax/azure.datalake Ballerina library

Overview

Prerequisites

Quickstart

Step 1 - Import connector

Step 2 - Create a new connector instance

Step 3 - Invoke connector operation

Clients

azure.datalake: Client

Constructor

filesystemList

Parameters

Return Type

pathList

Parameters

Return Type

filesystemCreate

Parameters

Return Type

filesystemDelete

Parameters

Return Type

filesystemSetproperties

Parameters

Return Type

filesystemGetproperties

Parameters

Return Type

pathRead

Parameters

Return Type

pathCreate

Parameters

Return Type

pathLease

Parameters

Return Type

pathDelete

Parameters

Return Type

pathUpdate

Parameters

Return Type

pathGetproperties

Parameters

Return Type

Records

azure.datalake: ClientHttp1Settings

Fields

azure.datalake: ConnectionConfig

Fields

azure.datalake: DataLakeStorageError

Fields

azure.datalake: DatalakestorageerrorError

Fields

azure.datalake: Filesystem

Fields

azure.datalake: FilesystemList

Fields

azure.datalake: Path

Fields

azure.datalake: PathList

Fields

azure.datalake: ProxyConfig

Fields