arc.job.ssh

A module for SSHing into servers. Used for giving commands, uploading, and downloading files.

class arc.job.ssh.SSHClient(server: str = '')[source]

This is a class for communicating with remote servers via SSH.

Parameters

server (str) – The server name as specified in ARCs’s settings file under servers as a key.

server

The server name as specified in ARCs’s settings file under servers as a key.

Type

str

address

The server’s address.

Type

str

un

The username to use on the server.

Type

str

key

A path to a file containing the RSA SSH private key to the server.

Type

str

_ssh

A high-level representation of a session with an SSH server.

Type

paramiko.SSHClient

_sftp

SFTP client used to perform remote file operations.

Type

paramiko.sftp_client.SFTPClient

change_mode(mode: str, path: str, recursive: bool = False, remote_path: str = '') → None[source]

Change the mode to a file or a directory.

Parameters
  • mode (str) – The mode change to be applied, can be either octal or symbolic.

  • path (str) – The path to the file or the directory to be changed.

  • recursive (bool, optional) – Whether to recursively change the mode to all files under a directory.``True`` for recursively change.

  • remote_path (str, optional) – The directory path at which the command will be executed.

check_job_status(job_id: int) → str[source]

Check job’s status.

Parameters

job_id (int) – The job’s ID.

Returns: str

Possible statuses: before_submission, running, errored on node xx, done, and errored: …

check_running_jobs_ids() → list[source]

Check all jobs submitted by the user on a server.

Returns: list

A list of job IDs.

close() → None[source]

Close the connection to paramiko SSHClient and SFTPClient

connect() → None[source]

A modulator function for _connect(). Connect to the server.

Raises

ServerError – Cannot connect to the server with maximum times to try

delete_job(job_id: Union[int, str]) → None[source]

Deletes a running job.

Parameters

job_id (Union[int, str]) – The job’s ID.

delete_jobs(jobs: Optional[List[Union[str, int]]] = None) → None[source]

Delete all of the jobs on a specific server.

Parameters

jobs (List[Union[str, int]], optional) – Specific ARC job IDs to delete.

download_file(remote_file_path: str, local_file_path: str) → None[source]

Download a file from the server.

Parameters
  • remote_file_path (str) – The remote path to be downloaded from.

  • local_file_path (str) – The local path to be downloaded to.

Raises

ServerError – If the file cannot be downloaded with maximum times to try

find_package(package_name: str) → list[source]

Find the path to the package.

Parameters

package_name (str) – The name of the package to search for.

list_available_nodes() → list[source]

List available nodes on the server.

Returns

lines of the node hostnames.

Return type

list

list_dir(remote_path: str = '') → list[source]

List directory contents.

Parameters

remote_path (str, optional) – The directory path at which the command will be executed.

submit_job(remote_path: str) → Tuple[str, int][source]

Submit a job to the server.

Parameters

remote_path (str) – The remote path contains the input file and the submission script.

Returns: Tuple[str, int]
  • A string indicate the status of job submission. Either errored or submitted.

  • The job ID of the submitted job.

upload_file(remote_file_path: str, local_file_path: str = '', file_string: str = '') → None[source]

Upload a local file or contents from a string to the remote server.

Parameters
  • remote_file_path (str) – The path to write into on the remote server.

  • local_file_path (Optional[str]) – The local file path to be copied to the remote location.

  • file_string (Optional[str]) – The file content to be copied and saved as the remote file.

Raises
  • InputError – If both local_file_path or file_string are invalid, or local_file_path does not exists.

  • ServerError – If the file cannot be uploaded with maximum times to try

arc.job.ssh.check_connections(function: Callable[[], Any]) → Callable[[], Any][source]

A decorator designned for SSHClient``to check SSH connections before calling a method. It first checks if ``self._ssh is available in a SSHClient instance and then checks if you can send ls and get response to make sure your connection still alive. If connection is bad, this decorator will reconnect the SSH channel, to avoid connection related error when executing the method.

arc.job.ssh.check_job_status_in_stdout(job_id: int, stdout: Union[list, str], server: str) → str[source]

A helper function for checking job status.

Parameters
  • job_id (int) – the job ID recognized by the server.

  • stdout (Union[list, str]) – The output of a queue status check.

  • server (str) – The server name.

Returns

The job status on the server (‘running’, ‘done’, or ‘errored’).

Return type

str

arc.job.ssh.delete_all_arc_jobs(server_list: list, jobs: Optional[List[str]] = None) → None[source]

Delete all ARC-spawned jobs (with job name starting with a and a digit) from :list:servers (servers could also be a string of one server name) Make sure you know what you’re doing, so unrelated jobs won’t be deleted… Useful when terminating ARC while some (ghost) jobs are still running.

Parameters
  • server_list (list) – List of servers to delete ARC jobs from.

  • jobs (Optional[List[str]]) – Specific ARC job IDs to delete.