UBC Wiki - User contributions [en]

Sandbox:SSH Proxy Service Interim Technical Documentation

2026-06-30T16:41:00Z

JolPittet: Add silent mode example

== Scope of Service ==
The '''SSH Proxy Service''' ('''SSHP''') provides an SSH tunnel between UBC '''SSH Proxy Pools''' and Internal '''SSH Destination Hosts'''. Once the connection via the SSH Proxy Pool to the SSH Destination Host is established, then other protocols may be emitted from the SSH Destination Host at the discretion of the SSH Destination Host system administrator. This service is intended for '''end-user use''' of SSH services.

It does not:

* allow other protocols to be emitted from the '''SSH Proxy Pools'''
* provide a full replacement for the [https://it.ubc.ca/services/email-voice-internet/myvpn UBC myVPN service].
* provide any user control or management of accounts or authentication on SSH Destination Hosts
* provide an approved method for administration of systems over SSH using privileged accounts. Remote SSH systems administration with privileged accounts should be facilitated through the use of the high-security [https://it.ubc.ca/services/accounts-passwords/privileged-access-management-pam UBC PAM (Privileged Access Management)] service. PAM offers session recording, privileged account management, and a host of other security features which make it the right choice for systems administration work.
[[File:Sshproxyservicescope.png|frameless|931x931px]]

== Connectivity Information ==
{| class="wikitable"
|+
!Purpose
!Address
!Port
!Credentials
|-
|SSH Proxy Pool - UBC Vancouver
|p01.sshp.cybersecurity.ubc.ca
142.103.90.160/28
|22
|CWL Production Account
|-
|SSH Key Registration Web Portal
|https://sshp.cybersecurity.ubc.ca/
|443
|CWL Production Account
|}
{| style="border: 1px #98B2C3 solid; padding: 0; border-radius: 0.25rem; padding-right: .3em;"
|-
| ℹ || If you are a server administrator who would like to provide access to a server through the SSH Proxy Service, please see the '''Information for System Administrators and Destination Host Administrators''' section at the bottom of this page.
|}
{| class="wikitable" width=100% align="center"valign="top"
!style="background: #faecc8;"| Warning
|-
|style="background: #faf6ed; align=left; padding:3px 7px 3px 7px;"|Avoid the use of SSH Agent forwarding! SSH Agent forwarding exposes your authentication to the server you’re connecting to. By default, an attacker with control of the server (i.e. root access) can communicate with your agent and use your key to authenticate to other servers without any notification (i.e. impersonate you). For this reason, one must be careful when using SSH agent forwarding. Defaulting to always forwarding the agent is '''strongly''' discouraged. Note also that while the attacker can use your key as long as the agent is running and forwarded, they cannot steal/download the key for offline/later use. <ref name="Mozilla OpenSSH Guidelines">{{cite web |url=https://infosec.mozilla.org/guidelines/openssh#ssh-agent-forwarding |title=Mozilla OpenSSH Guidelines |publisher=Mozilla}}</ref>
|}

=== Authentication Options ===
'''Interactive'''

* CWL accounts with MFA.
* MFA required on every authentication.
* Convenient and easy to use - best suited for ad-hoc work.

'''Non-Interactive (Key-based)'''

* CWL usernames with SSH keys.
* MFA not required on authentication.
* CWL with MFA is required to register SSH keys at https://sshp.cybersecurity.ubc.ca/
* Best-suited for heavy users or non-interactive connections.

{| style="border: 1px #98B2C3 solid; padding: 0; border-radius: 0.25rem; padding-right: .3em;"
|-
| ℹ || At this time, there is no key expiration on SSH keys, but this may change in the future.
|}

== SSH Key Handling ==
Do not re-use an existing key for the SSH Proxy Service unless required.

Keys should be generated following the [https://infosec.mozilla.org/guidelines/openssh#key-generation|Mozilla OpenSSH guidelines], as per UBC Information Security Standard M3, section 4.

Example:

$ ssh-keygen -t ed25519 -f ~/.ssh/id_ubcsshp_$(date +%Y%m%d) -C "UBC key for SSHP"

This creates one keypair consisting of a public and a private key. Register the '''public''' key in the SSH Proxy Key Registration Portal - '''NOT the private key'''.

== Use Case Patterns ==
See the [https://en.wikibooks.org/wiki/OpenSSH/Cookbook/Proxies_and_Jump_Hosts#Jump_Hosts_--_Passing_Through_a_Gateway_or_Two Proxies and Jump Hosts section of the OpenSSH Cookbook] for many other scenarios.

=== Interactive Auth to SSHP ===
Connecting through SSHP Proxy Pool p01.sshp.cybersecurity.ubc.ca interactively with SSH to the Destination Host can be accomplished transparently via use of the '''-J''' parameter.

$ ssh -J <cwlusername>@p01.sshp.cybersecurity.ubc.ca <username>@<destination_host>
==== Example ====
Connect to '''foo.ubc.ca''' as user '''jbdoe-dest''' using CWL username '''jbdoe-cwl''' via SSHP:

$ ssh -J jbdoe-cwl@p01.sshp.cybersecurity.ubc.ca jbdoe-dest@foo.ubc.ca

=== Non-Interactive (SSH Key-based) Auth to SSHP ===
Connecting through SSHP using SSH key-based auth requires you first create an SSHP Proxy Pool-specific '''Host''' directive in ~/.ssh/config.

''~/.ssh/config

Host p01.sshp.cybersecurity.ubc.ca
IdentityFile ~/.ssh/id_ed25519_ubcsshp_20221206
User jbdoe-cwl

The ''IdentityFile'' is the private key corresponding to the private which you registered with the [https://sshp.cybersecurity.ubc.ca/ SSH Proxy Key Registration Portal].

The ''User'' is your CWL username. If you do not specify a username in the ''config'' file, you will need to specify it in the connection string (i.e. jbdoe-cwl@p01.sshp.cybersecurity.ubc.ca).

After that, you can connect using the same '''-J''' parameter used in the Interactive section above.

Because the public key has been registered with the SSH Proxy Key Registration Portal, the private key will be used during the authentication with the SSH Proxy Pool, and you will not be required to perform an interactive login with the SSH Proxy Pool -- however, you will still need to authenticate with the SSH Destination Host.

$ ssh -J p01.sshp.cybersecurity.ubc.ca jdoe-dest@ssh-target.misc.ubc.ca
#####################################################################
UBC SSH Proxy Service <nowiki>https://it.ubc.ca</nowiki>
------------------------------ NOTICE -------------------------------
You are connecting to the UBC SSH Proxy Service.

After authentication succeeds here, your SSH client will connect
through this proxy to your destination host.

Any subsequent username, password, MFA, or host key prompts may be
originating from your destination host, not from this proxy service.

If authentication or access fails after the proxy connection has
been established, contact the administrator of your destination host.

Repeated failed authentication attempts within a short period of time
may result in a temporary network block affecting access to this
proxy service from your source IP address.

The use of this proxy service may be monitored and recorded for
administrative and security purposes. Monitoring is limited to
activity on the proxy service itself and does not include activity
on your destination host, which may be monitored separately. Anyone
accessing this service expressly consents to such monitoring and is
advised that if such monitoring reveals possible evidence of criminal
activity, UBC may provide that evidence to law enforcement officials.
---------------------------------------------------------------------

Enter passphrase for key 'id_ed25519_ubcsshp_20221206':
SSH Proxy Session ID: ITSO-A4E59.2176796

SSH Proxy Pool login successful.

=====================================================================
DESTINATION HOST SESSION BEGINS BELOW THIS LINE
For further assistance, contact the destination host administrator.
=====================================================================

(jdoe-dest@ssh-target.misc.ubc.ca) Password:
Updates Information Summary: available
6 Security notice(s)
3 Important Security notice(s)
2 Medium Security notice(s)
1 Low Security notice(s)

, #_
~\_ ####_ Amazon Linux 2023
~~ \_#####\
~~ \###|
~~ \#/ ___ <nowiki>https://aws.amazon.com/linux/amazon-linux-2023</nowiki>
~~ V~' '->
<nowiki> ~~~ /</nowiki>
~~._. _/
_/ _/
_/m/'
Last failed login: Fri May 15 23:26:30 UTC 2026 from 10.93.46.42 on ssh:notty
There was 1 failed login attempt since the last successful login.
Last login: Fri May 15 22:29:02 2026 from 10.93.46.42
[jdoe-dest@ssh-target ~]$

=== SSH Port Forwarding through SSHP ===
SSH port forwarding [also known as SSH tunneling] may be used through the SSH Proxy Service provided that the tunnel terminates on an authorized SSH Destination Host.

The SSH Proxy Service does not permit SSH Proxy Hosts to be used as port forwarding endpoints. However, once a connection to an SSH Destination Host has been established through the SSH Proxy Service, standard SSH port forwarding features, if supported by the SSH Destination Host, remain available.

For an introduction to SSH tunneling concepts and usage patterns, see:

* [https://help.ubuntu.com/community/SSH/OpenSSH/PortForwarding Ubuntu Community Documentation: SSH/OpenSSH/PortForwarding]
* [https://en.wikibooks.org/wiki/OpenSSH/Cookbook/Tunnels Wikibooks OpenSSH/Cookbook/Tunnels]

==== Example ====
Create a local tunnel to www.ubc.ca:443 through an SSH Destination Host:
ssh -N -J <cwlusername>@p01.sshp.cybersecurity.ubc.ca -L 8443:www.ubc.ca:443 \
<destination_username>@<destination_host>
The resulting traffic path is:

User Workstation:8443 ==> SSH Proxy Host ==> SSH Destination Host ==> www.ubc.ca:443

The SSH tunnel is established on the SSH Destination Host. Any outbound connections generated by the tunnel originate from the SSH Destination Host and are subject to the policies, network controls, and authorization decisions of the SSH Destination Host administrator.

==== Limitation ====
SSH Proxy Hosts cannot be used as general-purpose TCP forwarding endpoints.

For example, the following connection attempt is not permitted and will fail because it does not use the destination as the tunnel endpoint - it uses the proxy host itself:
ssh -N -L 8443:www.ubc.ca:443 <cwlusername>@p01.sshp.cybersecurity.ubc.ca
When traffic attempts to use the tunnel, the SSH client will receive an error similar to:
channel 2: open failed: administratively prohibited: open failed
This behavior is intentional and prevents SSH Proxy Hosts from being used as generic network relays.

=== Using SSHP via ssh config ===
Connecting with SSH to a Destination Host can be accomplished transparently via use of the '''ProxyJump''' configuration setting for any given host defined in the ~/.ssh/config file.

''~/.ssh/config

Host <destination_host>
ProxyJump p01.sshp.cybersecurity.ubc.ca

However, doing this will result in the p01.sshp.cybersecurity.ubc.ca host being passed the local username of the source system, which may not be desired. As such, you may want to also define a '''Host''' section for the SSHP Proxy Pool that forces a specific username to be used.

==== Example 1 ====
Connect to foo.ubc.ca through SSHP Proxy Pool p01.sshp.cybersecurity.ubc.ca with CWL username of jbdoe-cwl:

''~/.ssh/config
Host p01.sshp.cybersecurity.ubc.ca
User jbdoe-cwl

Host foo.ubc.ca
ProxyJump p01.sshp.cybersecurity.ubc.ca

With that in place, you can connect to foo.ubc.ca as user jbdoe-dest:

$ ssh jbdoe-dest@foo.ubc.ca

'''Note: ''' This will route ALL ssh connections to the Destination Host through SSHP from the source host, including those that originate from on-campus.

==== Example 2 ====
Use SSHP with an SSHP-specific SSH key for all hosts under UBC subdomain foo.ubc.ca, using a CWL username of jbdoe-cwl to login to SSHP.

''~/.ssh/config''

Host p01.sshp.cybersecurity.ubc.ca
User jbdoe-cwl
IdentityFile ~/.ssh/id_ed25519_ubcsshp_20221206

Host foo.ubc.ca
ProxyJump p01.sshp.cybersecurity.ubc.ca

With that in place, you can connect to foo.ubc.ca as user jbdoe-dest, without being prompted for your CWL password.

$ ssh jbdoe-dest@foo.ubc.ca

==== Example 3 ====
Use SSHP for all hosts under UBC subdomain bar.ubc.ca:

''~/.ssh/config

Match Host *.bar.ubc.ca
ProxyJump p01.sshp.cybersecurity.ubc.ca

With that in place, you can connect to foo.bar.ubc.ca as user jbdoe-dest (note that this will use whatever your local account username is to auth to SSHP):

$ ssh jbdoe-dest@foo.bar.ubc.ca

==== Example 4 ====
Use SSHP for all hosts under UBC subdomain bar.ubc.ca except for foo.bar.ubc.ca and only off-campus:

''~/.ssh/config''
Match Host *.bar.ubc.ca,!foo.bar.ubc.ca exec "[\[ `curl --no-progress-meter ifconfig.me` != 142\.103\.* ]]"
ProxyJump p01.sshp.cybersecurity.ubc.ca
(Remove the \ - it's necessary here because [ is special.)
Tune the IP address expression to whatever is "on-campus" for you.
{| class="wikitable"

|+ Identity Rubric
|-
! SSH Command Line !! ssh_config !! Local Username !! SSHP Username !! Destination System Username !! Login UX
|-
| <code>$ ssh -J p01.sshp.cybersecurity.ubc.ca foo.ubc.ca</code> || none || <code>jbdoe-local</code> || <code>jbdoe-local</code> || <code>jbdoe-local</code> || SSHP: Interactive
foo.ubc.ca: Interactive
|-
| <code>$ ssh -J jbdoe-cwl@p01.sshp.cybersecurity.ubc.ca jbdoe-dest@foo.ubc.ca</code> || none || <code>jbdoe-local</code> || <code>jbdoe-cwl</code> || <code>jbdoe-dest</code> || SSHP: Interactive
foo.ubc.ca: Interactive
|-
| <code>$ ssh jbdoe-dest@foo.ubc.ca</code> ||
Host foo.ubc.ca
ProxyJump p01.sshp.cybersecurity.ubc.ca
|| <code>jbdoe-local</code> || <code>jbdoe-local</code> || <code>jbdoe-dest</code> || SSHP: Interactive
foo.ubc.ca: Interactive
|-
| <code>$ ssh foo.bar.ubc.ca</code> ||
Host p01.sshp.cybersecurity.ubc.ca
User jbdoe-cwl

Match *.bar.ubc.ca
ProxyJump p01.sshp.cybersecurity.ubc.ca
|| <code>jbdoe-local</code> || <code>jbdoe-cwl</code> || <code>jbdoe-dest</code> || SSHP: Interactive
foo.bar.ubc.ca: Interactive
|-
| <code>$ ssh foo.ubc.ca</code> ||
Host p01.sshp.cybersecurity.ubc.ca
User jbdoe-cwl
IdentityFile ~/.ssh/id_ed25519_ubcsshp_20221206

Host foo.ubc.ca
User jbdoe-dest
ProxyJump p01.sshp.cybersecurity.ubc.ca
|| <code>jbdoe-local</code> || <code>jbdoe-cwl</code> || <code>jbdoe-dest</code> || SSHP: Non-Interactive
foo.ubc.ca: Interactive
|-
| <code>$ ssh foo.ubc.ca</code> ||
Host p01.sshp.cybersecurity.ubc.ca
User jbdoe-cwl
IdentityFile ~/.ssh/id_ed25519_ubcsshp_20221206

Match *.ubc.ca
User jbdoe-dest
IdentityFile ~/.ssh/id_rsa_ubc_20201101
ProxyJump p01.sshp.cybersecurity.ubc.ca
|| <code>jbdoe-local</code> || <code>jbdoe-cwl</code> || <code>jbdoe-dest</code> || SSHP: Non-Interactive
foo.ubc.ca: Non-Interactive
|}

=== Silent mode: Suppressing proxy-side output for non-interactive workflows ===

Some non-interactive workflows require clean STDOUT/STDERR from the SSH Destination Host, such as database dumps, scripts, rsync-like tools, or SSH used as a transport.

Avoid suppressing STDERR on the outer SSH command, for example:

ssh destination-host 'mysqldump ...' 2>/dev/null

This suppresses all STDERR from the SSH command, including useful errors from the destination host or remote command.

Instead, use ProxyCommand to suppress STDERR '''only''' from the SSH Proxy Pool connection:

Host p01.sshp.cybersecurity.ubc.ca
User <cwlusername>
IdentityFile ~/.ssh/id_ed25519_ubcsshp_20221206

Host destination-host
User <destination_username>
ProxyCommand ssh p01.sshp.cybersecurity.ubc.ca -W %h:%p 2>/dev/null

With this pattern, proxy-side banner/session output is suppressed, while STDERR from the SSH Destination Host and remote command is still displayed.

== Common Issues and Troubleshooting ==
{| class="wikitable"
!'''Error Message'''
!'''Was SSH Proxy Session ID† Displayed'''
!'''Was Destination Host Notice‡ Displayed'''
!'''Probable Causes'''
!'''Correction'''
|- valign="top"
|
Error 500: SSH Proxy Service error. Submit a ticket at <nowiki>https://it.ubc.ca/selfservice</nowiki>.

Connection closed by <ip address> port 22

Connection closed by UNKNOWN port 65535
|No
|No
|SSH Proxy Service server-side error.
|This is a service error that has to be corrected by the SSH Proxy Service administrators. Submit a ticket to notify us of the error.
|- valign="top"
|
Error 503: SSH Proxy Service is currently offline. Please try again later.

Connection closed by <ip address> port 22

Connection closed by UNKNOWN port 65535
|No
|No
|SSH Proxy Service has been intentionally taken offline.
|Try again later, watch status.it.ubc.ca for updates.
|- valign="top"
|
<after three password attempts>

jdoe@p01.sshp.devl.infosec.it.ubc.ca: Permission denied (publickey,keyboard-interactive).

Connection closed by UNKNOWN port 65535
|No
|No
|Invalid CWL username or password.
|Check to ensure that your username and password are valid.
|- valign="top"
|
(jdoe@p01.sshp.devl.infosec.it.ubc.ca) Password:

(jdoe@p01.sshp.devl.infosec.it.ubc.ca) Duo two-factor login for jdoe

Enter a passcode:
|No
|No
|Account is enrolled in Duo MFA, but does not have a license for Duo.
|Submit a ticket to request a Duo MFA license.
|- valign="top"
|
<SSH key provided as authentication parameter>

(jdoe@p01.sshp.devl.infosec.it.ubc.ca) Password:
|No
|No
|'''Cause 1'''

Public SSH key is not registered in portal.

'''Cause 2'''

Public SSH key is registered in portal, but hasn't been pushed to the proxy pool yet.

'''Cause 3'''

Private SSH key cannot be read.
|'''Correction 1'''

Register public SSH key in portal.

'''Correction 2'''

Wait 2-5 minutes, try again.

'''Correction 3'''

Ensure that permissions on your private SSH key are correct. See this video for more information.
|- valign="top"
|
no such identity: <filename>: No such file or directory

(jdoe@p01.sshp.devl.infosec.it.ubc.ca) Password:
|No
|No
|Private SSH key does not exist.
|Check name for typo, check to ensure file exists as displayed.
|- valign="top"
|
no such identity: <filename>: Permission denied

(jdoe@p01.sshp.devl.infosec.it.ubc.ca) Password:
|No
|No
|Permissions on directory holding private key is too restrictive.
|Set permissions on key directory to be readable by current user.
|- valign="top"
|
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@ WARNING: UNPROTECTED PRIVATE KEY FILE! @

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

Permissions <permissions> for '<filename>' are too open.

It is required that your private key files are NOT accessible by others.

This private key will be ignored.

Load key "<filename>": bad permissions

(jdoe@p01.sshp.devl.infosec.it.ubc.ca) Password:
|No
|No
|Private key file has permissions that are too open.
|Set permissions on file to 0400 (user: read, group: none, other:none). See this video for more information.
|- valign="top"
|
Load key "<filename>": Permission denied

(jdoe@p01.sshp.devl.infosec.it.ubc.ca) Password:
|No
|No
|Private key file has permissions that are too restrictive.
|Set permissions on file to 0400 (user: read, group: none, other:none). See this video for more information.
|- valign="top"
|
SSH Proxy Session ID: <session id>

Error 403: Authorization denied by SSH Proxy Service policy.

Connection closed by <ip address> port 22

Connection closed by UNKNOWN port 65535
|Yes
|No
|'''Cause 1'''

User account is disabled.

'''Cause 2'''

User does not have the required permissions to use the SSH Proxy Service.
|'''Correction 1'''

Check to ensure your CWL account is working by logging in at <nowiki>https://myaccount.ubc.ca</nowiki>.

'''Correction 2'''

Submit a ticket for the SSH Proxy Service.
|- valign="top"
|
channel 0: open failed: administratively prohibited: open failed

stdio forwarding failed

Connection closed by UNKNOWN port 65535
|Yes
|Yes
|'''Cause 1'''

Typo in the Destination Host name or port (a default port of 22 is assumed but some Destination Hosts may be using a non-standard port).

'''Cause 2'''

Attempting to reach a Destination Host that is not authorized.
|'''Correction 1'''

Check to ensure that the Destination Host name (and port, if provided) are correct.

'''Correction 2'''

Request that the Destination Host be registered with the SSH Proxy Service.
|- valign="top"
|
channel 0: open failed: connect failed: open failed

Connection to p01.sshp.cybersecurity.ubc.ca closed.
|Yes
|Yes
|Attempting to login to the proxy host directly via SSH.
|Use the proxy pool via the -J parameter or ProxyJump option.
|- valign="top"
|
<very long delay before message>

channel 0: open failed: connect failed: Connection timed out

stdio forwarding failed

Connection closed by UNKNOWN port 65535
|Yes
|Yes
|Destination host is authorized but is not responding.
|Contact the Destination Host administrator for assistance. The host may be offline, or a firewall on the destination side is blocking the connection attempt.
|- valign="top"
|
<displayed by itself AFTER the Destination Host Notice>

Connection closed by UNKNOWN port 65535
|Yes
|Yes
|Destination Host is authorized and destination port is open, but it is not communicating using SSH.
|Contact the Destination Host administrator for assistance.
|}
{| class="wikitable"
|† The '''SSH Proxy Session ID''' will only be shown when authentication to the proxy pool is successful. If you see the SSH Proxy Session ID, the username, password, MFA challenges have passed, or a valid SSH key was provided.

An example of a successful authentication displaying an SSH Proxy Session ID is:
SSH Proxy Session ID: ITSO-A4E59.0353334

SSH Proxy Pool login successful.
‡ The '''Destination Host Notice''' refers to the following text which appears after the SSH Proxy Service has authenticated and authorized the user, and determined that the destination is authorized.
=====================================================================

DESTINATION HOST SESSION BEGINS BELOW THIS LINE

For further assistance, contact the destination host administrator.

=====================================================================
|}

== Information for System Administrators and Destination Host Administrators ==

=== Modifying Firewall Rules ===
To allow inbound communication from a specific SSH Proxy Pool, open your inbound firewalls to the address listed in the '''Connectivity Information''' section of this page for your specific Destination Host and Port.

=== Adding Destination Hosts ===
To request a host be added as a Destination Host, please submit a [https://ubc.service-now.com/selfservice?id=sc_cat_item_v2&sys_id=9440ab761b136c500dbaec21b24bcbbb Cybersecurity Services Support Request] through the UBC Self-Service Portal. Provide the following information:

# Your department and role
# Destination Hostname (FQDN)
## Only Destination Hosts with FQDNs are eligible for this service
# Destination IP Address
# Destination Port
# Technical Owner Name
# Technical Owner Email
# Business Owner Name
# Business Owner Email
# Reason

A request has been submitted to have a custom form for this type of request.

== Support / Feedback ==
For any questions, comments, suggestions or feedback regarding the SSH Proxy Service, please submit a [https://ubc.service-now.com/selfservice?id=sc_cat_item_v2&sys_id=9440ab761b136c500dbaec21b24bcbbb Cybersecurity Services Support Request] through the UBC Self-Service Portal.

== References ==
{{Reflist}}

Sandbox:SSH Proxy Service Interim Technical Documentation

2026-06-30T00:32:06Z

JolPittet: Typo fix missing ProxyJump directive

== Scope of Service ==
The '''SSH Proxy Service''' ('''SSHP''') provides an SSH tunnel between UBC '''SSH Proxy Pools''' and Internal '''SSH Destination Hosts'''. Once the connection via the SSH Proxy Pool to the SSH Destination Host is established, then other protocols may be emitted from the SSH Destination Host at the discretion of the SSH Destination Host system administrator. This service is intended for '''end-user use''' of SSH services.

It does not:

* allow other protocols to be emitted from the '''SSH Proxy Pools'''
* provide a full replacement for the [https://it.ubc.ca/services/email-voice-internet/myvpn UBC myVPN service].
* provide any user control or management of accounts or authentication on SSH Destination Hosts
* provide an approved method for administration of systems over SSH using privileged accounts. Remote SSH systems administration with privileged accounts should be facilitated through the use of the high-security [https://it.ubc.ca/services/accounts-passwords/privileged-access-management-pam UBC PAM (Privileged Access Management)] service. PAM offers session recording, privileged account management, and a host of other security features which make it the right choice for systems administration work.
[[File:Sshproxyservicescope.png|frameless|931x931px]]

== Connectivity Information ==
{| class="wikitable"
|+
!Purpose
!Address
!Port
!Credentials
|-
|SSH Proxy Pool - UBC Vancouver
|p01.sshp.cybersecurity.ubc.ca
142.103.90.160/28
|22
|CWL Production Account
|-
|SSH Key Registration Web Portal
|https://sshp.cybersecurity.ubc.ca/
|443
|CWL Production Account
|}
{| style="border: 1px #98B2C3 solid; padding: 0; border-radius: 0.25rem; padding-right: .3em;"
|-
| ℹ || If you are a server administrator who would like to provide access to a server through the SSH Proxy Service, please see the '''Information for System Administrators and Destination Host Administrators''' section at the bottom of this page.
|}
{| class="wikitable" width=100% align="center"valign="top"
!style="background: #faecc8;"| Warning
|-
|style="background: #faf6ed; align=left; padding:3px 7px 3px 7px;"|Avoid the use of SSH Agent forwarding! SSH Agent forwarding exposes your authentication to the server you’re connecting to. By default, an attacker with control of the server (i.e. root access) can communicate with your agent and use your key to authenticate to other servers without any notification (i.e. impersonate you). For this reason, one must be careful when using SSH agent forwarding. Defaulting to always forwarding the agent is '''strongly''' discouraged. Note also that while the attacker can use your key as long as the agent is running and forwarded, they cannot steal/download the key for offline/later use. <ref name="Mozilla OpenSSH Guidelines">{{cite web |url=https://infosec.mozilla.org/guidelines/openssh#ssh-agent-forwarding |title=Mozilla OpenSSH Guidelines |publisher=Mozilla}}</ref>
|}

=== Authentication Options ===
'''Interactive'''

* CWL accounts with MFA.
* MFA required on every authentication.
* Convenient and easy to use - best suited for ad-hoc work.

'''Non-Interactive (Key-based)'''

* CWL usernames with SSH keys.
* MFA not required on authentication.
* CWL with MFA is required to register SSH keys at https://sshp.cybersecurity.ubc.ca/
* Best-suited for heavy users or non-interactive connections.

{| style="border: 1px #98B2C3 solid; padding: 0; border-radius: 0.25rem; padding-right: .3em;"
|-
| ℹ || At this time, there is no key expiration on SSH keys, but this may change in the future.
|}

== SSH Key Handling ==
Do not re-use an existing key for the SSH Proxy Service unless required.

Keys should be generated following the [https://infosec.mozilla.org/guidelines/openssh#key-generation|Mozilla OpenSSH guidelines], as per UBC Information Security Standard M3, section 4.

Example:

$ ssh-keygen -t ed25519 -f ~/.ssh/id_ubcsshp_$(date +%Y%m%d) -C "UBC key for SSHP"

This creates one keypair consisting of a public and a private key. Register the '''public''' key in the SSH Proxy Key Registration Portal - '''NOT the private key'''.

== Use Case Patterns ==
See the [https://en.wikibooks.org/wiki/OpenSSH/Cookbook/Proxies_and_Jump_Hosts#Jump_Hosts_--_Passing_Through_a_Gateway_or_Two Proxies and Jump Hosts section of the OpenSSH Cookbook] for many other scenarios.

=== Interactive Auth to SSHP ===
Connecting through SSHP Proxy Pool p01.sshp.cybersecurity.ubc.ca interactively with SSH to the Destination Host can be accomplished transparently via use of the '''-J''' parameter.

$ ssh -J <cwlusername>@p01.sshp.cybersecurity.ubc.ca <username>@<destination_host>
==== Example ====
Connect to '''foo.ubc.ca''' as user '''jbdoe-dest''' using CWL username '''jbdoe-cwl''' via SSHP:

$ ssh -J jbdoe-cwl@p01.sshp.cybersecurity.ubc.ca jbdoe-dest@foo.ubc.ca

=== Non-Interactive (SSH Key-based) Auth to SSHP ===
Connecting through SSHP using SSH key-based auth requires you first create an SSHP Proxy Pool-specific '''Host''' directive in ~/.ssh/config.

''~/.ssh/config

Host p01.sshp.cybersecurity.ubc.ca
IdentityFile ~/.ssh/id_ed25519_ubcsshp_20221206
User jbdoe-cwl

The ''IdentityFile'' is the private key corresponding to the private which you registered with the [https://sshp.cybersecurity.ubc.ca/ SSH Proxy Key Registration Portal].

The ''User'' is your CWL username. If you do not specify a username in the ''config'' file, you will need to specify it in the connection string (i.e. jbdoe-cwl@p01.sshp.cybersecurity.ubc.ca).

After that, you can connect using the same '''-J''' parameter used in the Interactive section above.

Because the public key has been registered with the SSH Proxy Key Registration Portal, the private key will be used during the authentication with the SSH Proxy Pool, and you will not be required to perform an interactive login with the SSH Proxy Pool -- however, you will still need to authenticate with the SSH Destination Host.

$ ssh -J p01.sshp.cybersecurity.ubc.ca jdoe-dest@ssh-target.misc.ubc.ca
#####################################################################
UBC SSH Proxy Service <nowiki>https://it.ubc.ca</nowiki>
------------------------------ NOTICE -------------------------------
You are connecting to the UBC SSH Proxy Service.

After authentication succeeds here, your SSH client will connect
through this proxy to your destination host.

Any subsequent username, password, MFA, or host key prompts may be
originating from your destination host, not from this proxy service.

If authentication or access fails after the proxy connection has
been established, contact the administrator of your destination host.

Repeated failed authentication attempts within a short period of time
may result in a temporary network block affecting access to this
proxy service from your source IP address.

The use of this proxy service may be monitored and recorded for
administrative and security purposes. Monitoring is limited to
activity on the proxy service itself and does not include activity
on your destination host, which may be monitored separately. Anyone
accessing this service expressly consents to such monitoring and is
advised that if such monitoring reveals possible evidence of criminal
activity, UBC may provide that evidence to law enforcement officials.
---------------------------------------------------------------------

Enter passphrase for key 'id_ed25519_ubcsshp_20221206':
SSH Proxy Session ID: ITSO-A4E59.2176796

SSH Proxy Pool login successful.

=====================================================================
DESTINATION HOST SESSION BEGINS BELOW THIS LINE
For further assistance, contact the destination host administrator.
=====================================================================

(jdoe-dest@ssh-target.misc.ubc.ca) Password:
Updates Information Summary: available
6 Security notice(s)
3 Important Security notice(s)
2 Medium Security notice(s)
1 Low Security notice(s)

, #_
~\_ ####_ Amazon Linux 2023
~~ \_#####\
~~ \###|
~~ \#/ ___ <nowiki>https://aws.amazon.com/linux/amazon-linux-2023</nowiki>
~~ V~' '->
<nowiki> ~~~ /</nowiki>
~~._. _/
_/ _/
_/m/'
Last failed login: Fri May 15 23:26:30 UTC 2026 from 10.93.46.42 on ssh:notty
There was 1 failed login attempt since the last successful login.
Last login: Fri May 15 22:29:02 2026 from 10.93.46.42
[jdoe-dest@ssh-target ~]$

=== SSH Port Forwarding through SSHP ===
SSH port forwarding [also known as SSH tunneling] may be used through the SSH Proxy Service provided that the tunnel terminates on an authorized SSH Destination Host.

The SSH Proxy Service does not permit SSH Proxy Hosts to be used as port forwarding endpoints. However, once a connection to an SSH Destination Host has been established through the SSH Proxy Service, standard SSH port forwarding features, if supported by the SSH Destination Host, remain available.

For an introduction to SSH tunneling concepts and usage patterns, see:

* [https://help.ubuntu.com/community/SSH/OpenSSH/PortForwarding Ubuntu Community Documentation: SSH/OpenSSH/PortForwarding]
* [https://en.wikibooks.org/wiki/OpenSSH/Cookbook/Tunnels Wikibooks OpenSSH/Cookbook/Tunnels]

==== Example ====
Create a local tunnel to www.ubc.ca:443 through an SSH Destination Host:
ssh -N -J <cwlusername>@p01.sshp.cybersecurity.ubc.ca -L 8443:www.ubc.ca:443 \
<destination_username>@<destination_host>
The resulting traffic path is:

User Workstation:8443 ==> SSH Proxy Host ==> SSH Destination Host ==> www.ubc.ca:443

The SSH tunnel is established on the SSH Destination Host. Any outbound connections generated by the tunnel originate from the SSH Destination Host and are subject to the policies, network controls, and authorization decisions of the SSH Destination Host administrator.

==== Limitation ====
SSH Proxy Hosts cannot be used as general-purpose TCP forwarding endpoints.

For example, the following connection attempt is not permitted and will fail because it does not use the destination as the tunnel endpoint - it uses the proxy host itself:
ssh -N -L 8443:www.ubc.ca:443 <cwlusername>@p01.sshp.cybersecurity.ubc.ca
When traffic attempts to use the tunnel, the SSH client will receive an error similar to:
channel 2: open failed: administratively prohibited: open failed
This behavior is intentional and prevents SSH Proxy Hosts from being used as generic network relays.

=== Using SSHP via ssh config ===
Connecting with SSH to a Destination Host can be accomplished transparently via use of the '''ProxyJump''' configuration setting for any given host defined in the ~/.ssh/config file.

''~/.ssh/config

Host <destination_host>
ProxyJump p01.sshp.cybersecurity.ubc.ca

However, doing this will result in the p01.sshp.cybersecurity.ubc.ca host being passed the local username of the source system, which may not be desired. As such, you may want to also define a '''Host''' section for the SSHP Proxy Pool that forces a specific username to be used.

==== Example 1 ====
Connect to foo.ubc.ca through SSHP Proxy Pool p01.sshp.cybersecurity.ubc.ca with CWL username of jbdoe-cwl:

''~/.ssh/config
Host p01.sshp.cybersecurity.ubc.ca
User jbdoe-cwl

Host foo.ubc.ca
ProxyJump p01.sshp.cybersecurity.ubc.ca

With that in place, you can connect to foo.ubc.ca as user jbdoe-dest:

$ ssh jbdoe-dest@foo.ubc.ca

'''Note: ''' This will route ALL ssh connections to the Destination Host through SSHP from the source host, including those that originate from on-campus.

==== Example 2 ====
Use SSHP with an SSHP-specific SSH key for all hosts under UBC subdomain foo.ubc.ca, using a CWL username of jbdoe-cwl to login to SSHP.

''~/.ssh/config''

Host p01.sshp.cybersecurity.ubc.ca
User jbdoe-cwl
IdentityFile ~/.ssh/id_ed25519_ubcsshp_20221206

Host foo.ubc.ca
ProxyJump p01.sshp.cybersecurity.ubc.ca

With that in place, you can connect to foo.ubc.ca as user jbdoe-dest, without being prompted for your CWL password.

$ ssh jbdoe-dest@foo.ubc.ca

==== Example 3 ====
Use SSHP for all hosts under UBC subdomain bar.ubc.ca:

''~/.ssh/config

Match Host *.bar.ubc.ca
ProxyJump p01.sshp.cybersecurity.ubc.ca

With that in place, you can connect to foo.bar.ubc.ca as user jbdoe-dest (note that this will use whatever your local account username is to auth to SSHP):

$ ssh jbdoe-dest@foo.bar.ubc.ca

==== Example 4 ====
Use SSHP for all hosts under UBC subdomain bar.ubc.ca except for foo.bar.ubc.ca and only off-campus:

''~/.ssh/config''
Match Host *.bar.ubc.ca,!foo.bar.ubc.ca exec "[\[ `curl --no-progress-meter ifconfig.me` != 142\.103\.* ]]"
ProxyJump p01.sshp.cybersecurity.ubc.ca
(Remove the \ - it's necessary here because [ is special.)
Tune the IP address expression to whatever is "on-campus" for you.
{| class="wikitable"

|+ Identity Rubric
|-
! SSH Command Line !! ssh_config !! Local Username !! SSHP Username !! Destination System Username !! Login UX
|-
| <code>$ ssh -J p01.sshp.cybersecurity.ubc.ca foo.ubc.ca</code> || none || <code>jbdoe-local</code> || <code>jbdoe-local</code> || <code>jbdoe-local</code> || SSHP: Interactive
foo.ubc.ca: Interactive
|-
| <code>$ ssh -J jbdoe-cwl@p01.sshp.cybersecurity.ubc.ca jbdoe-dest@foo.ubc.ca</code> || none || <code>jbdoe-local</code> || <code>jbdoe-cwl</code> || <code>jbdoe-dest</code> || SSHP: Interactive
foo.ubc.ca: Interactive
|-
| <code>$ ssh jbdoe-dest@foo.ubc.ca</code> ||
Host foo.ubc.ca
ProxyJump p01.sshp.cybersecurity.ubc.ca
|| <code>jbdoe-local</code> || <code>jbdoe-local</code> || <code>jbdoe-dest</code> || SSHP: Interactive
foo.ubc.ca: Interactive
|-
| <code>$ ssh foo.bar.ubc.ca</code> ||
Host p01.sshp.cybersecurity.ubc.ca
User jbdoe-cwl

Match *.bar.ubc.ca
ProxyJump p01.sshp.cybersecurity.ubc.ca
|| <code>jbdoe-local</code> || <code>jbdoe-cwl</code> || <code>jbdoe-dest</code> || SSHP: Interactive
foo.bar.ubc.ca: Interactive
|-
| <code>$ ssh foo.ubc.ca</code> ||
Host p01.sshp.cybersecurity.ubc.ca
User jbdoe-cwl
IdentityFile ~/.ssh/id_ed25519_ubcsshp_20221206

Host foo.ubc.ca
User jbdoe-dest
ProxyJump p01.sshp.cybersecurity.ubc.ca
|| <code>jbdoe-local</code> || <code>jbdoe-cwl</code> || <code>jbdoe-dest</code> || SSHP: Non-Interactive
foo.ubc.ca: Interactive
|-
| <code>$ ssh foo.ubc.ca</code> ||
Host p01.sshp.cybersecurity.ubc.ca
User jbdoe-cwl
IdentityFile ~/.ssh/id_ed25519_ubcsshp_20221206

Match *.ubc.ca
User jbdoe-dest
IdentityFile ~/.ssh/id_rsa_ubc_20201101
ProxyJump p01.sshp.cybersecurity.ubc.ca
|| <code>jbdoe-local</code> || <code>jbdoe-cwl</code> || <code>jbdoe-dest</code> || SSHP: Non-Interactive
foo.ubc.ca: Non-Interactive
|}

== Common Issues and Troubleshooting ==
{| class="wikitable"
!'''Error Message'''
!'''Was SSH Proxy Session ID† Displayed'''
!'''Was Destination Host Notice‡ Displayed'''
!'''Probable Causes'''
!'''Correction'''
|- valign="top"
|
Error 500: SSH Proxy Service error. Submit a ticket at <nowiki>https://it.ubc.ca/selfservice</nowiki>.

Connection closed by <ip address> port 22

Connection closed by UNKNOWN port 65535
|No
|No
|SSH Proxy Service server-side error.
|This is a service error that has to be corrected by the SSH Proxy Service administrators. Submit a ticket to notify us of the error.
|- valign="top"
|
Error 503: SSH Proxy Service is currently offline. Please try again later.

Connection closed by <ip address> port 22

Connection closed by UNKNOWN port 65535
|No
|No
|SSH Proxy Service has been intentionally taken offline.
|Try again later, watch status.it.ubc.ca for updates.
|- valign="top"
|
<after three password attempts>

jdoe@p01.sshp.devl.infosec.it.ubc.ca: Permission denied (publickey,keyboard-interactive).

Connection closed by UNKNOWN port 65535
|No
|No
|Invalid CWL username or password.
|Check to ensure that your username and password are valid.
|- valign="top"
|
(jdoe@p01.sshp.devl.infosec.it.ubc.ca) Password:

(jdoe@p01.sshp.devl.infosec.it.ubc.ca) Duo two-factor login for jdoe

Enter a passcode:
|No
|No
|Account is enrolled in Duo MFA, but does not have a license for Duo.
|Submit a ticket to request a Duo MFA license.
|- valign="top"
|
<SSH key provided as authentication parameter>

(jdoe@p01.sshp.devl.infosec.it.ubc.ca) Password:
|No
|No
|'''Cause 1'''

Public SSH key is not registered in portal.

'''Cause 2'''

Public SSH key is registered in portal, but hasn't been pushed to the proxy pool yet.

'''Cause 3'''

Private SSH key cannot be read.
|'''Correction 1'''

Register public SSH key in portal.

'''Correction 2'''

Wait 2-5 minutes, try again.

'''Correction 3'''

Ensure that permissions on your private SSH key are correct. See this video for more information.
|- valign="top"
|
no such identity: <filename>: No such file or directory

(jdoe@p01.sshp.devl.infosec.it.ubc.ca) Password:
|No
|No
|Private SSH key does not exist.
|Check name for typo, check to ensure file exists as displayed.
|- valign="top"
|
no such identity: <filename>: Permission denied

(jdoe@p01.sshp.devl.infosec.it.ubc.ca) Password:
|No
|No
|Permissions on directory holding private key is too restrictive.
|Set permissions on key directory to be readable by current user.
|- valign="top"
|
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@ WARNING: UNPROTECTED PRIVATE KEY FILE! @

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

Permissions <permissions> for '<filename>' are too open.

It is required that your private key files are NOT accessible by others.

This private key will be ignored.

Load key "<filename>": bad permissions

(jdoe@p01.sshp.devl.infosec.it.ubc.ca) Password:
|No
|No
|Private key file has permissions that are too open.
|Set permissions on file to 0400 (user: read, group: none, other:none). See this video for more information.
|- valign="top"
|
Load key "<filename>": Permission denied

(jdoe@p01.sshp.devl.infosec.it.ubc.ca) Password:
|No
|No
|Private key file has permissions that are too restrictive.
|Set permissions on file to 0400 (user: read, group: none, other:none). See this video for more information.
|- valign="top"
|
SSH Proxy Session ID: <session id>

Error 403: Authorization denied by SSH Proxy Service policy.

Connection closed by <ip address> port 22

Connection closed by UNKNOWN port 65535
|Yes
|No
|'''Cause 1'''

User account is disabled.

'''Cause 2'''

User does not have the required permissions to use the SSH Proxy Service.
|'''Correction 1'''

Check to ensure your CWL account is working by logging in at <nowiki>https://myaccount.ubc.ca</nowiki>.

'''Correction 2'''

Submit a ticket for the SSH Proxy Service.
|- valign="top"
|
channel 0: open failed: administratively prohibited: open failed

stdio forwarding failed

Connection closed by UNKNOWN port 65535
|Yes
|Yes
|'''Cause 1'''

Typo in the Destination Host name or port (a default port of 22 is assumed but some Destination Hosts may be using a non-standard port).

'''Cause 2'''

Attempting to reach a Destination Host that is not authorized.
|'''Correction 1'''

Check to ensure that the Destination Host name (and port, if provided) are correct.

'''Correction 2'''

Request that the Destination Host be registered with the SSH Proxy Service.
|- valign="top"
|
channel 0: open failed: connect failed: open failed

Connection to p01.sshp.cybersecurity.ubc.ca closed.
|Yes
|Yes
|Attempting to login to the proxy host directly via SSH.
|Use the proxy pool via the -J parameter or ProxyJump option.
|- valign="top"
|
<very long delay before message>

channel 0: open failed: connect failed: Connection timed out

stdio forwarding failed

Connection closed by UNKNOWN port 65535
|Yes
|Yes
|Destination host is authorized but is not responding.
|Contact the Destination Host administrator for assistance. The host may be offline, or a firewall on the destination side is blocking the connection attempt.
|- valign="top"
|
<displayed by itself AFTER the Destination Host Notice>

Connection closed by UNKNOWN port 65535
|Yes
|Yes
|Destination Host is authorized and destination port is open, but it is not communicating using SSH.
|Contact the Destination Host administrator for assistance.
|}
{| class="wikitable"
|† The '''SSH Proxy Session ID''' will only be shown when authentication to the proxy pool is successful. If you see the SSH Proxy Session ID, the username, password, MFA challenges have passed, or a valid SSH key was provided.

An example of a successful authentication displaying an SSH Proxy Session ID is:
SSH Proxy Session ID: ITSO-A4E59.0353334

SSH Proxy Pool login successful.
‡ The '''Destination Host Notice''' refers to the following text which appears after the SSH Proxy Service has authenticated and authorized the user, and determined that the destination is authorized.
=====================================================================

DESTINATION HOST SESSION BEGINS BELOW THIS LINE

For further assistance, contact the destination host administrator.

=====================================================================
|}

== Information for System Administrators and Destination Host Administrators ==

=== Modifying Firewall Rules ===
To allow inbound communication from a specific SSH Proxy Pool, open your inbound firewalls to the address listed in the '''Connectivity Information''' section of this page for your specific Destination Host and Port.

=== Adding Destination Hosts ===
To request a host be added as a Destination Host, please submit a [https://ubc.service-now.com/selfservice?id=sc_cat_item_v2&sys_id=9440ab761b136c500dbaec21b24bcbbb Cybersecurity Services Support Request] through the UBC Self-Service Portal. Provide the following information:

# Your department and role
# Destination Hostname (FQDN)
## Only Destination Hosts with FQDNs are eligible for this service
# Destination IP Address
# Destination Port
# Technical Owner Name
# Technical Owner Email
# Business Owner Name
# Business Owner Email
# Reason

A request has been submitted to have a custom form for this type of request.

== Support / Feedback ==
For any questions, comments, suggestions or feedback regarding the SSH Proxy Service, please submit a [https://ubc.service-now.com/selfservice?id=sc_cat_item_v2&sys_id=9440ab761b136c500dbaec21b24bcbbb Cybersecurity Services Support Request] through the UBC Self-Service Portal.

== References ==
{{Reflist}}