Filesystem Resource¶

A Filesystem connection is a set of connection parameters needed to perform download/upload operations using a remote file location.

This connection can be stored in two ways:

related to the object which is going to use it (Custom source option)
as a Rulex Platform Resource of Filesystem type (Saved source option), which can be used at any time and in any operation.

If this connection is stored as a Filesystem resource, the user can define a set of permissions linked to this resource (as explained in the permissions page) to limit/control the access other users can have on the inserted connection.

To use a Filesystem resource in any panel, a filesystem connection is required, and the user must have view permissions set to allow on the resource itself, while modify permissions are required to see its internal parameters.

Tip

By creating a Filesystem resource with view permissions for all users and modify permission only for the administrator, you are exposing a connection without sharing username and password used in the authentication procedure.

Filesystem connection configuration¶

Rulex Platform supports the following remote locations:

The configuration interface and its parameters vary for every filesystem type. Further details are listed in the paragraphs below.

Note

The Local Filesystem file location is considered as a filesystem, and it corresponds to the client machine disk. As this filesystem doesn’t require any connection parameter, it is available only as a Custom connection. To use the Local Filesystem in Rulex Platform Cloud/server version, you can find more information in this note.

Warning

Rulex’s technical documentation does not and cannot provide comprehensive guidelines on the use, limitations and constraints of third-party software programs, beyond how Rulex integrates with this software.

Please read the technical documentation of the third-party software itself for up-to-date information.

Sharedrive File System

This connection allows Rulex Platform to connect to Samba or NFS share drive, reachable from the hosting machine or from the cloud/server.

The connection parameters needed for this connection are listed in the following table (parameters in bold are mandatory). To define a Custom connection to a Sharedrive Filesystem in Rulex Platform API calls, the user must specify the value sharedrive in the uri field.

Name	Description	API param key	Default
Host	The server host of the chosen share drive location.	host
Port	The server port of the chosen share drive location.	port	445
Username	If an authentication is required, enter the username exchanged with the share drive server.	username
Password	If an authentication is required, enter the password exchanged with the share drive server.	password
Domain	The domain the share drive location owns, if available.	domain
Share	In Samba Share drive any server can contain more than one exposed share drive. The user must specify which server Rulex Platform must connect to.	share

Http/s Server

This connection allows Rulex Platform to perform Web/SOAP/REST API call towards any HTTP/S Server, reachable from the hosting machine or from the cloud/server.

The connection parameters needed for this connection are reported in the following table (parameters in bold are mandatory). To define a Custom connection towards an HTTP/S Server in Rulex Platform API calls the user must specify the value http in uri field.

Name	Description	API param key	Default
Authentication	The authentication type to use in contacting the HTTP/S Server. The possible values are: Basic Authentication (basic): the user performs a Basic Authentication by inserting the required username and password. NTLM Authentication (ntlm): the user performs a NTLM Negotiation to the server by inserting the required username and password. Digest Authentication (digest): the user performs a Digest Negotiation to the server by inserting the required username and password. Bearer Authentication (bearer): the user adds the Bearer Token provided to the Authentication header in the current call. OAuth2 Authentication (oauth2): the user performs a client-credential or password OAuth2 authentication flow by gathering a bearer token from the OAuth2 authentication endpoint. Insecure (insecure): no authentication is performed.	auth	basic
Username	If an authentication is required, enter the username linked to the share drive server.	username
Password	If an authentication is required, enter the password linked to the share drive server.	password
Bearer Token	The Bearer Token to insert in the Authentication header (Mandatory if the Authentication is set to Bearer Authentication).	password
Authentication Url	The OAuth2 authentication endpoint used for OAuth2 authentication client-credential or password flows. (Mandatory if the Authentication is set to OAuth2 Authentication).	authurl
OAuth2 type	The OAuth2 grant type which selects the type of OAuth2 authentication flow. (Mandatory if the Authentication is set to OAuth2 Authentication). Possible values are: Client Credential with Basic Auth (ccba): client-credential authentication flow with client authentication performed using Basic Authentication. Client Credential with Body Auth (ccbd): client-credential authentication flow with client authentication performed by sharing in the body client id and client secret. Client Credential with JWT Token (ccjwt): client-credential authentication flow with client authentication performed by using a signed JWT procedure. Password Auth (password): password authentication flow.	granttype	ccba
Client ID	The application client identifier used in OAuth2 client credential flow to identify the client. (Mandatory if the Authentication set to OAuth2 Authentication and the OAuth2 type is set to Client Credential with Basic Auth, Client Credential with Body Auth or Client Credential with signed JWT)	username
Client Secret	The application client secret passphrase used in OAuth2 client credential flow to authenticate the client. (Mandatory if the Authentication is set to OAuth2 Authentication and the OAuth2 type is set to Client Credential with Basic Auth or Client Credential with Body Auth)	password
JWT Token	The JWT Token used in OAuth2 client credential flow to authenticate the client. (Mandatory if the Authentication is set to OAuth2 Authentication and OAuth2 type is set to Client Credential with signed JWT)	password
Scope	The application authentication scope requested in OAuth2 authentication. (Mandatory if the Authentication is set to OAuth2 Authentication). The RFC8707 resource claim is supported in the HTTP/S connector, for more information check the corresponding page.	share
Proxy Settings	A dictionary containing Proxy settings information if a Proxy needs to be used. For more information see Proxy Panel.	proxy
Headers	A list of strings written using the syntax Label:Value containing additional Headers to be added to the HTTP/S call.	headers

FTP/S Server

This connection allows Rulex Platform to connect to an FTP/FTPS server, reachable from the hosting machine or from the cloud/server.

The connection parameters required for this connection are reported in the following table (parameters in bold are mandatory.). To define a Custom connection to an FTP/FTPS server in Rulex Platform API calls the user must specify the value ftp in uri field.

Name	Description	API param key	Default
Host	The server host of the chosen ftp location.	host
Port	The server port of the chosen ftp location.	port	21
Username	If an authentication is required, enter the username linked to the ftp server.	username
Password	If an authentication is required, enter the password linked to the ftp server.	password
Proxy Settings	A dictionary containing Proxy settings information, if a Proxy needs to be used. For more information see Proxy Panel.	proxy

Amazon AWS S3

This connection allows Rulex Platform to connect to an Amazon AWS S3 bucket reachable from the hosting machine or from the cloud/server.

The connection parameters required for this connection are reported in the following table (parameters in bold are mandatory.). To define a Custom connection to an Amazon AWS S3 filesystem in Rulex Platform API calls, the user must specify the value s3 in uri field.

Name	Description	API param key
Bucket	The Amazon AWS S3 Bucket used in the connection.	bucket
Username/Access Key ID	The username used to authenticate to the Amazon AWS S3 service.	username
Password/Secret Access Key	The password used to authenticate to the Amazon AWS S3 service.	password
Authentication	The type of authentication method used: one of Standard or Short-term credential. As Standard, the IAM authentication is intended.	auth
Encryption	The server-side encryption eventually enforced: one of AES256 (default), SSE-KMS (aws:kms), DSSE-KMS (aws:dsse:kms).	granttype
Bucket Region	The regional endpoint used to make requests.	path
Session token	If Authentication is Short-term credential it contains the session token used to authenticate.	connstring
Key ID	When SSE-KMS or DSSE-KMS are used, it stores the Storage key used in the encryption.	domain
Proxy Settings	A dictionary containing Proxy settings information, if a Proxy needs to be used. For more information see Proxy Panel.	proxy

Microsoft SharePoint

This connection allows Rulex Platform to connect to Microsoft SharePoint sites reachable from the hosting machine or from the cloud/server.

The connection parameters required for this connection are reported in the following table (parameters in bold are mandatory.). To define a Custom connection to a Microsoft SharePoint filesystem in Rulex Platform API calls, the user must specify the value sharepoint in uri field.

Name	Description	API param key	Default
Base Url	The Microsoft SharePoint site Url location.	host
Authentication	The type of authentication used to connect to Microsoft SharePoint service; the possible values are: Legacy Sharepoint Authentication (direct): to connect directly to Microsoft Sharepoint service using the Username and Password combination. Azure Authentication (azure): to authenticate using an Only-App Azure AD Authentication towards Microsoft Graph API and redirect it to Microsoft Sharepoint service. For more information about this type of authentication see this section. Refresh token (refreshtoken): to connect to Microsoft Sharepoint service using a refresh token. For more information on this mechanism, go to the corresponding section.	auth	direct
Refresh Token	The Refresh Token obtained by a client credential OAuth2 flow which can be used to obtain a new access token at any run. (Mandatory if the Authentication type is Refresh Token)	connstring
Tenant ID	The tenant identifier. (Mandatory if Refresh Token authentication is selected)	domain
Username	The username used to authenticate. (Mandatory if the Legacy Sharepoint Authentication is selected)	username
Password	The password used to authenticate. (Mandatory if the Legacy Sharepoint Authentication is selected)	password
App ID	The application identifier of the SharePoint service principal app registered on the Azure tenant. (Mandatory if the Azure Authentication is selected)	username
Tenant ID	The Azure tenant identifier. (Mandatory if the Azure Authentication is selected)	domain
JWT Auth Token	The signed JWT token associated to the Sharepoint service principal, generated as explained here. (Mandatory if the Legacy Sharepoint Authentication is selected)	password
Proxy Settings	A dictionary containing Proxy settings information if a Proxy needs to be used. For more information see Proxy Panel.	proxy

Hadoop HDFS File System

This connection allows Rulex Platform to connect to Hadoop File System (HDFS) reachable from the hosting machine or from the cloud/server.

The connection parameters needed for this connection are reported in the following table (parameters in bold are mandatory.). To define a Custom connection to an HDFS filesystem in Rulex Platform API calls, the user must specify the value hdfs in the uri field.

Name	Description	API param key
Filesystem url	The url location of your Hadoop filesystem.	url
Port	The port used by your Hadoop filesystem, if any.	port
Username	The username linked to the Hadoop filesystem during the authentication procedure.	username
Password	The password linked to the Hadoop filesystem during the authentication procedure.	password

Azure Blob Storage

This connection allows Rulex Platform to connect to an Azure Storage Account (Blob service) reachable from the hosting machine or from the cloud/server.

Warning

This connector works only for Azure Storage Account Blob service (1st or 2nd generation).

For Azure Storage Account Files service use the Sharedrive File System connection while for Azure Storage Account Table service define a Database connection by configuring it as a SQL Server database.

Warning

If the SAS key type is container, so it has limited permissions on the filesystem, read the dedicated section in the import page.

The connection parameters needed for this connection are reported in the table below (parameters in bold are mandatory.). To define a Custom connection to an Azure Blob Storage filesystem in Rulex Platform API call use the value astorage in uri field.

Name	Description	API param key	Default
Account Storage Name	The Azure Storage Account name.	username
Type Key	The key type used in the authentication. The possible values are: Account Key (sharedkey) or SAS key (signkey).	auth	sharedkey
Key/SAS Token	The Key or the SAS Token used to authenticate to the Azure Storage Account: it is the Access Key if the Account Key type value is Key, while it is the full SAS token if SAS key is selected.	password
Proxy Settings	A dictionary containing the Proxy settings information if a Proxy needs to be used. For more information see Proxy Panel.	proxy

SFTP/S Server

This connection allows Rulex Platform to connect to a SFTP/SFTPS server reachable from the hosting machine or from the cloud/server/server/server.

The connection parameters needed for this connection are reported in the following table (parameters in bold are mandatory.). To define a Custom connection to a SFTP server in Rulex Platform API calls the user must specify the value sftp in the uri field.

Name	Description	API param key	Default
Host	The server host of the chosen sftp location.	host
Port	The server port of the chosen sftp location.	port	22
Username	The username exchanged with the sftp server during the authentication process.	username
Password	The password exchanged with the sftp server during the authentication process.	password

Google Drive

This connection allows Rulex Platform to connect to Google drive, reachable from the hosting machine or from the cloud/server.

The connection parameters needed for this connection are reported in the following table (parameters in bold are mandatory.). To define a Custom connection to a Google Drive filesystem in Rulex Platform API calls the user must specify the value google in the uri field.

Name	Description	API param key
Api key	API key endpoint pointing to the chosen Google Drive filesystem.	host
Client ID	Client identifier of the registered application on Google domain.	username
Client Secret	Client secret passphrase of the registered application on Google domain.	password
Refresh Token	Refresh Token obtained by a client credential OAuth2 flow which can be used to obtain a new access token at any run.	connstring

Microsoft Outlook

This connection allows Rulex Platform to connect to a Microsoft Outlook account through Microsoft Graph API, reachable from the hosting machine or from the cloud/server.

The connection parameters needed for this connection are reported in the following table (parameters in bold are mandatory.). To define a Custom connection to a Microsoft Outlook filesystem in Rulex Platform API calls the user must specify the value outlook in the uri field.

Name	Description	API param key
Mail account	The mail address account to connect.	url
Client ID	The client identifier of the delegated application registered on Azure tenant for Microsoft Graph API access (see this section).	username
Authentication	The method to use for the authentication. The possible value is Refresh Token (refreshtoken). For more information on this mechanism, go to the corresponding section.	auth
Refresh Token	The Refresh Token obtained by a client credential OAuth2 flow which can be used to obtain a new access token at any run. (Mandatory if the Authentication type is Refresh Token)	connstring
Tenant ID	The identifier of the Azure tenant.	domain
Secret Name	The name of the secret created in the vault to store the refresh token. (only if the Authentication type is Interactive)	authurl

Creating a Filesystem Resource¶

To create a Filesystem resource, you need to open the Explorer panel (for more information refer to the corresponding page) and follow the procedure below:

Procedure

Click the Explore Resources icon to open the Explorer panel.
Deactivate the primary resource filter by toggling off on the Primary filter on the upper right side of the Explorer panel to add general resources.
Hover the mouse over the Plus button.
Select Add new Filesystem and a dedicated window will appear on the screen.
Select the Filesystem type you want to connect to (see supported types).
Configure the connection parameters following the provided guide. A test connection will be performed; if unsuccessful, an error message is displayed, offering also the opportunity to continue working anyway.
Type a unique name for the new resource.
Click Create: the new Filesystem resource is now added to the list.

Once the Filesystem resource is defined, it can be referred as a Saved source in any location where a remote connection can be used.

As an example, you’ll find below the most important Filesystem saved connection applications:

In any of the applications listed above, it is still possible to define the connection while performing the operation itself, by inserting a Custom connection in the parameters of the section above.

Proxy panel¶

Sometimes, for standalone installation, the used network needs to be protected with proxy firewall or by using a security VPN. Usually, the Rulex Platform software inherits proxy configuration directly from the system but when this does not occur (for example with some VPNs which enforce the proxy directly on the connection and not on the entire operating system), the proxy configuration needs to be imposed on the external connection itself. This can be done by using the Proxy panel which is contained in any filesystem remote connection configuration pane.

To open the Proxy panel you can click on the Change settings button located at the bottom of any remote connection configuration pane.

The Proxy panel is composed by four different entries:

Host: the IP host of the proxy.
Port: the IP port of the proxy.
Username: for secured proxy the username to be used in the proxy bypass.
Password: for secured proxy the password to be used in the proxy bypass.

Refresh Token Mechanism¶

Every time a Refresh Token is specified as the Authentication type, a button appears next to the Authentication drop-down list.

This button, called Generate Token, allows users to open an additional window, where they can complete their MFA authentication to the provider or, if they are already logged in, they can generate the refresh token and visualize the corresponding window.

The Refresh Token window is made of a main area, where the refresh token is visualized, and of a series of button located in the bottom part of the window. The following buttons can be found:

Save your refresh token in vault: select the vault where the refresh token will be stored.
Secret name: type the secret name which will be used to store the refresh token.
Save to vault: click this button to save the refresh token in the previously specified vault.
Copy to clipboard: click this button to copy the refresh token.

Note

To generate a refresh token, it is required an application with delegated access scope. As explained here, the permission to be set on the delegated application should be the most general possible (we suggest for Sharepoint Sites.ReadWrite.All for example), since they then will be limited by user permission. To know more about these procedures, check out the Sharepoint and the Outlook app creation dedicated pages.

Authentication application mediated¶

Rulex Platform connectors need to be executed without the physical presence of the user for the majority of the computation time. Moreover, if an interactive authentication is required, this should be asked once even if used for several connections.

The general approach of the majority of the identity providers available now on the market is to use a client credential authentication flow which exploits a constructed application to manage a machine to machine connection.

This definition of these client applications has to be executed on the identity provider portal (Azure, Google for example) towards you want to connect to.

Note

The Rulex Platform filesystem remote connections which now require a client application definition are:

Sharepoint connector with Azure authentication
Outlook connector
Google connector

Please refer to the official documentation of these providers for the complete step-by-step guide of this client creation. Here and in the correlated pages the attention is concentrated about most critical configuration steps or about the use of the already created application within the Rulex Platform filesystem configuration.

Sharepoint Azure authentication connection requires an Azure authentication app with on behalf as the user rights. Step-by-step guide to create this type of application with the correct rights to access the desired Sharepoint sites and the way to obtain the necessary signed JWT to be used in Rulex Platform is described in this side page.

Outlook connection requires an Azure authentication app with delegated permission on the desired user. The application needs the following permissions on Microsoft Graph service:

User.Read - Sign in and read user profile
Mail.Read - Read user mail
Mail.ReadWrite - Read and write access to user mail (only used in export operations)
Mail.Send - Send mail as a user (only used in export operations)

A step-by-step guide to create this type of application with the correct rights and the way to obtain the necessary information to configure the corresponding panel in Rulex Platform is available in this dedicated page.

Google connection requires a Google authentication app acting with delegated permission on the desired drive user storage. Please refer to this link, for a general guide about Oauth2 app authentication in the Google framework.

Filesystem GOLD Reference¶

Files in GOLD are mediated through the File class:

Filesystem connections in GOLD are mediated through the Connectors class: