Published: Dec 24, 2024 by Isaac Johnson
Today let’s look into few decent open source externalized access options that we can use to enter into our network.
One of the ones we’ll look at is Nexterm that not only allow us to connect to hosts with a variety of protocols but also install some Opensource apps such as Uptime Kuma.
I’ll demonstrate a simple way to expose an SSH server just using a container in your cluster and a firewall rule.
Lastly, I’ll dig into Waveterm which is a new SSH Client (and more) for Linux, Windows and Mac.
Let’s start with Nexterm.
Nexterm
I was inspired to check out Nexterm from this MariusHosting blog where he uses Portainer.
We can see the Nexterm setup with its Docker compose file (found here as well as here).
Docker Compose
services:
nexterm:
ports:
- "6989:6989"
restart: always
volumes:
- /docker/nexterm:/app/data
image: germannewsmaker/nexterm:1.0.2-OPEN-PREVIEW
volumes:
nexterm:
I’ll quickly convert that to a Kubernetes YAML manifest with a PVC, Deployment and Service
Deployment YAML
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: nexterm-pvc
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi # Adjust storage size as needed
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nexterm
spec:
replicas: 1
selector:
matchLabels:
app: nexterm
template:
metadata:
labels:
app: nexterm
spec:
containers:
- name: nexterm
image: germannewsmaker/nexterm:1.0.2-OPEN-PREVIEW
ports:
- containerPort: 6989
volumeMounts:
- mountPath: /app/data
name: nexterm-storage
volumes:
- name: nexterm-storage
persistentVolumeClaim:
claimName: nexterm-pvc
---
apiVersion: v1
kind: Service
metadata:
name: nexterm-service
spec:
selector:
app: nexterm
ports:
- protocol: TCP
port: 6989
targetPort: 6989
Now let’s apply
$ kubectl apply -f ./test.yaml
persistentvolumeclaim/nexterm-pvc created
deployment.apps/nexterm created
service/nexterm-service created
I can now connect
$ kubectl port-forward svc/nexterm-service 6989:6989
Forwarding from 127.0.0.1:6989 -> 6989
Forwarding from [::1]:6989 -> 6989
The first step is to create an account
I can now see a landing page
I’ll go to Networking and add Nexterm there
I get an error about no SSH servers
I’ll go to servers and add a folder
From there I can add a server
I’ll add a Macbook Air
I gave it a password in the auth section the tried connecting with it. Worked great
Ingress
Since we have a password, assuming this is a decently secure app, let’s expose it externally
First, I’ll need an A record we can use for an ingress
$ gcloud dns --project=myanthosproject2 record-sets create Nexterm.steeped.space --zone="steepedspace" --type="A" --ttl="300" --rrdatas="75.73.224.240"
NAME TYPE TTL DATA
Nexterm.steeped.space. A 300 75.73.224.240
Let me then create an ingress
$ cat ingress.Nexterm.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
cert-manager.io/cluster-issuer: gcpleprod2
ingress.kubernetes.io/proxy-body-size: "0"
ingress.kubernetes.io/ssl-redirect: "true"
kubernetes.io/ingress.class: nginx
kubernetes.io/tls-acme: "true"
nginx.ingress.kubernetes.io/proxy-body-size: "0"
nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.org/client-max-body-size: "0"
nginx.org/proxy-connect-timeout: "3600"
nginx.org/proxy-read-timeout: "3600"
nginx.org/websocket-services: nexterm-service
labels:
app.kubernetes.io/instance: nexterm-service
app.kubernetes.io/name: nexterm-service
name: nexterm-service
spec:
rules:
- host: Nexterm.steeped.space
http:
paths:
- backend:
service:
name: nexterm-service
port:
number: 6989
path: /
pathType: ImplementationSpecific
tls:
- hosts:
- Nexterm.steeped.space
secretName: nexterm-servicegcp-tls
Now apply
$ kubectl apply -f ./ingress.Nexterm.yaml
ingress.networking.k8s.io/nexterm-service created
Once I see it satisified the cert
$ kubectl get cert nexterm-servicegcp-tls
NAME READY SECRET AGE
nexterm-servicegcp-tls True nexterm-servicegcp-tls 84s
And I can use it to get through to a host
I can then install an app to that host
A window pops up and shows it plans to actually install a lot of things
I’m slightly worried as it installed Docker on that host which already runs as a Kubernetes master node and I’m pretty sure had docker as it’s the primary utility box used in my AWX job runner
When I clicked open it tries to fire up a browser to the internal IP. As my traffic is coming from Sweden at the moment, that does not work so hot
However, If I route into a window in my network, I can reach it
I tested other things on my VNC session and they weren’t as laggy but as you can see, the speed test works, but the interface is definitely sluggish.
Using it
While testing, I found one of my services was down
Let’s figure this out.
First, I need to determine if it’s a containerized service in Kubernetes or just one running in docker and exposed via Kubernetes.
To do that, I’ll get the ingress and look at the service
$ kubectl get ingress -A | grep enclosed
default enclosedingress <none> enclosed.tpk.pw 80, 443 45d
$ kubectl get ingress enclosedingress -o yaml | grep -C 3 service:
http:
paths:
- backend:
service:
name: enclosed-external-ip
port:
number: 80
Usually I need to look into the service to check on selectors, but I did myself a favour by naming it external IP
$ kubectl get endpoints | grep enclosed
enclosed-external-ip 192.168.1.100:8713 45d
Let’s now use Nexterm to get there.
I’ll create a folder for Docker boxes
And add a server
I can make an entry for that 192.168.1.100 (builder-T100) and connect
I fear I did some unneccessary pruning. I know it was there but it sure isn’t now
I was able to look up the blog from 10-29 and run the docker command again
builder@builder-T100:~$ docker run -d --name enclosed -e PORT=8713 -v /home/builder/enclosed:/app/.data -p 8713:8713 corentinth/enclosed:latest
Unable to find image 'corentinth/enclosed:latest' locally
latest: Pulling from corentinth/enclosed
da9db072f522: Already exists
03d2f4babaac: Pull complete
a1c7bd30f9ab: Pull complete
c3f44fc696cb: Pull complete
4a4bb4ad0c79: Pull complete
08c6502d02c0: Pull complete
fbcf34564597: Pull complete
d5c1ac7fe4ce: Pull complete
Digest: sha256:ffadec06634cd201a3dfb02dbd19097a1cf2677fe8d85ef3cfa50282e799cb29
Status: Downloaded newer image for corentinth/enclosed:latest
7b52245e735bb567720b3bcca79ece326bdd028bcf2c535b0008469e78d5191a
And I can see it’s up and running
Nexterm with Btop++
I can also use this a nice wy to get at BTOP++ to view real time stats of the host
SSH in K8s
Let’s start by creating a username and password and base64’ing the values
$ echo -n 'builder' | base64
YnVpbGRlcg==
$ echo -n 'TestTest1234' | base64
VGVzdFRlc3QxMjM0
Next, I’m going to create a secret with those values
$ cat creds.yaml
apiVersion: v1
kind: Secret
metadata:
name: ssh-secret
type: Opaque
data:
username: YnVpbGRlcg==
password: VGVzdFRlc3QxMjM0
$ kubectl apply -f ./creds.yaml
secret/ssh-secret created
And use it in a deployment that has an ubuntu image with SSHD preloaded:
$ cat manifest.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: ssh-server
spec:
replicas: 1
selector:
matchLabels:
app: ssh-server
template:
metadata:
labels:
app: ssh-server
spec:
containers:
- name: ssh-server
image: rastasheep/ubuntu-sshd:18.04
ports:
- containerPort: 22
env:
- name: SSH_USERNAME
valueFrom:
secretKeyRef:
name: ssh-secret
key: username
- name: SSH_PASSWORD
valueFrom:
secretKeyRef:
name: ssh-secret
key: password
volumeMounts:
- name: ssh-keys
mountPath: /root/.ssh
volumes:
- name: ssh-keys
emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
name: ssh-service
spec:
type: LoadBalancer
selector:
app: ssh-server
ports:
- protocol: TCP
port: 22
targetPort: 22
port: 5522
nodePort: 30022
$ kubectl apply -f ./manifest.yaml
deployment.apps/ssh-server created
service/ssh-service created
This uses the default root and root to login, and did not actually create the builder user
builder@builder-T100:~$ ssh -p 5522 root@192.168.1.57
root@192.168.1.57's password:
root@ssh-server-7678596449-6wsxs:~# passwd builder
passwd: user 'builder' does not exist
root@ssh-server-7678596449-6wsxs:~# ^C
root@ssh-server-7678596449-6wsxs:~#
I wonder if I could just create a Dockerfile with a user/pass
$ cat Dockerfile
FROM ubuntu:latest
# Install SSH server
RUN apt-get update && \
apt-get install -y openssh-server && \
apt-get clean
# Set the root password for the SSH server (CHANGE THIS PASSWORD!)
RUN echo 'root:TestTest1234' | chpasswd
# Permit root login via SSH
RUN sed -i 's/#PermitRootLogin prohibit-password/PermitRootLogin yes/' /etc/ssh/sshd_config
# Enable password authentication
RUN sed -i 's/#PasswordAuthentication yes/PasswordAuthentication yes/' /etc/ssh/sshd_config
# Create privilege separation directory with proper permissions
RUN mkdir /run/sshd && chown root:root /run/sshd && chmod 0755 /run/sshd
# SSH port (optional, change if needed)
EXPOSE 22
# Start SSH service
CMD ["/usr/sbin/sshd", "-D"]
I can then build
$ docker build -t harbor.freshbrewed.science/freshbrewedprivate/testsshserver:0.3 .
[+] Building 1.1s (9/9) FINISHED docker:default
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 709B 0.0s
=> [internal] load metadata for docker.io/library/ubuntu:latest 1.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [1/5] FROM docker.io/library/ubuntu:latest@sha256:80dd3c3b9c6cecb9f1667e9290b3bc61b78c2678c02cbdae5f0fea92cc6734ab 0.0s
=> CACHED [2/5] RUN apt-get update && apt-get install -y openssh-server && apt-get clean 0.0s
=> CACHED [3/5] RUN echo 'root:TestTest1234' | chpasswd 0.0s
=> CACHED [4/5] RUN sed -i 's/#PermitRootLogin prohibit-password/PermitRootLogin yes/' /etc/ssh/sshd_config 0.0s
=> CACHED [5/5] RUN sed -i 's/#PasswordAuthentication yes/PasswordAuthentication yes/' /etc/ssh/sshd_config 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:a7c36966cf91acef38ad6e62d14122bbac8fb5223f36dad3ef8e52508e8378cd 0.0s
=> => naming to harbor.freshbrewed.science/freshbrewedprivate/testsshserver:0.1 0.0s
What's Next?
View a summary of image vulnerabilities and recommendations → docker scout quickview
and push
$ docker push harbor.freshbrewed.science/freshbrewedprivate/testsshserver:0.3
The push refers to repository [harbor.freshbrewed.science/freshbrewedprivate/testsshserver]
7b67a675a8b3: Pushed
cbe99aa61f16: Pushed
fbdd987b8a4e: Pushed
e1e114296ab1: Pushed
687d50f2f6a6: Pushed
0.1: digest: sha256:96ed3aa9cf826d7f6fdaf52b2dd520b6cfdf35002c81a7bacadc5dfa87892cf3 size: 1364
I can now try using that in the deployment
$ cat manifest.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: ssh-server
spec:
replicas: 1
selector:
matchLabels:
app: ssh-server
template:
metadata:
labels:
app: ssh-server
spec:
containers:
- name: ssh-server
image: harbor.freshbrewed.science/freshbrewedprivate/testsshserver:0.3
ports:
- containerPort: 22
imagePullSecrets:
- name: myharborreg
---
apiVersion: v1
kind: Service
metadata:
name: ssh-service
spec:
type: LoadBalancer
selector:
app: ssh-server
ports:
- protocol: TCP
port: 22
targetPort: 22
port: 5522
nodePort: 30022
$ kubectl apply -f ./manifest.yaml
deployment.apps/ssh-server configured
service/ssh-service unchanged
This seemed to work
builder@builder-T100:~$ ssh -p 5522 root@192.168.1.33
The authenticity of host '[192.168.1.33]:5522 ([192.168.1.33]:5522)' can't be established.
ED25519 key fingerprint is SHA256:gcLkbLpLj4QGZVt3njKf+3GbrW3I4RqtodWI1v+LrQE.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '[192.168.1.33]:5522' (ED25519) to the list of known hosts.
root@192.168.1.33's password:
Welcome to Ubuntu 24.04.1 LTS (GNU/Linux 5.15.0-106-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/pro
This system has been minimized by removing packages and content that are
not required on a system that users do not log into.
To restore this content, you can run the 'unminimize' command.
The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.
root@ssh-server-84c8dbf79c-lg6d8:~#
Now, if I want to reach this outside of my network, I need to expose a hole through the firewall
And I can test by sending all my traffic out to Sweden and coming back
I, of course, removed the hole when done and then the deployment
$ kubectl delete -f ./manifest.yaml
deployment.apps "ssh-server" deleted
service "ssh-service" deleted
Waveterm
Let’s check out on more SSH related tool, but this one is a client and not a server.
I found Waveterm from the TLDR newsletter.
I’ll start by downloading for windows here (they have binaries for Windows, Mac and Linux).
That launches a quick installer
which then launches WaveTerm
I’m shown a few keybindings before being let loose
The first thing I did was switch my left shell from a Powershell prompt to a bash shell on WSL. I then created a new shell tab and clicked connect to (gear icon)
I’ll try adding a connection to my dockerh ost
I typed in the password
and was prompted again if I want to use WSL extensions. This time i did check “apply to all”
Here we can see how we can rearrange blocks and add tabs
The thing about WaveTerm is it works WAY better on a large monitor. Moving it to fullscreen on my primary display makes a rather usable experience (I’m not sure how this will translate in a blog image, but use your biggest monitor to view fullscreen)
In the above, I am able to watch my build, the Ansible jobs and have some common terminals I might need/
Summary
Today we checked out NexTerm and Waveterm. The former is an interesting web-based SSH client that can also install apps (and docker) to linux hosts. I’m not certain I would use the app install again because I’m not keen on it updating docker each go, but the SSH client feature is handy.
WaveTerm is an interesting option. I’ll have to compare it to the more more tried and true TMux to see which windowed option I like best. I do like the integrated browsers as I’m often looking at code as well as content in Github like PRs or Ansible or Azure DevOps.
Lastly, while i did demonstrate firing up a pod to expose SSH, I would more likely just expose SSH from a Pi in my stack or a Linux machine that is more utility. However, it was a good example of how one could use Kubernetes to accomplish a similar goal of NexTerm.
