Published: Jul 16, 2024 by Isaac Johnson

I’ve been meaning to get back to some Open-Source apps that would allow me to remotely upload and download files. With an eye for simplicity, I decided I would look at SFTPGo and Filegator as both seemed reasonably up to date and active.

SFTPGo

Like others, I found this originally on a MariusHosting Post.

The app is hosted and developed in github.com/drakkan/sftpgo.

That said, I did find an active helm repo that can deliver this to Kubernetes for us.

Let’s add the helm repo

$ helm repo add skm https://charts.sagikazarmark.dev
"skm" has been added to your repositories

$ helm repo update
Hang tight while we grab the latest from your chart repositories...
...Unable to get an update from the "freshbrewed" chart repository (https://harbor.freshbrewed.science/chartrepo/library):
        failed to fetch https://harbor.freshbrewed.science/chartrepo/library/index.yaml : 404 Not Found
...Unable to get an update from the "myharbor" chart repository (https://harbor.freshbrewed.science/chartrepo/library):
        failed to fetch https://harbor.freshbrewed.science/chartrepo/library/index.yaml : 404 Not Found
...Successfully got an update from the "nfs" chart repository
...Successfully got an update from the "bitwarden" chart repository
...Successfully got an update from the "kube-state-metrics" chart repository
...Successfully got an update from the "ngrok" chart repository
...Successfully got an update from the "jfelten" chart repository
...Successfully got an update from the "actions-runner-controller" chart repository
...Successfully got an update from the "akomljen-charts" chart repository
...Successfully got an update from the "backstage" chart repository
...Successfully got an update from the "openfunction" chart repository
...Successfully got an update from the "ingress-nginx" chart repository
...Successfully got an update from the "portainer" chart repository
...Successfully got an update from the "zabbix-community" chart repository
...Successfully got an update from the "btungut" chart repository
...Successfully got an update from the "jetstack" chart repository
...Successfully got an update from the "sonarqube" chart repository
...Successfully got an update from the "kubecost" chart repository
...Successfully got an update from the "gitea-charts" chart repository
...Successfully got an update from the "castai-helm" chart repository
...Successfully got an update from the "harbor" chart repository
...Successfully got an update from the "openproject" chart repository
...Successfully got an update from the "nginx-stable" chart repository
...Successfully got an update from the "uptime-kuma" chart repository
...Successfully got an update from the "incubator" chart repository
...Successfully got an update from the "ananace-charts" chart repository
...Successfully got an update from the "gitlab" chart repository
...Successfully got an update from the "makeplane" chart repository
...Successfully got an update from the "prometheus-community" chart repository
...Successfully got an update from the "rhcharts" chart repository
...Successfully got an update from the "opencost" chart repository
...Successfully got an update from the "opencost-charts" chart repository
...Successfully got an update from the "adwerx" chart repository
...Successfully got an update from the "novum-rgi-helm" chart repository
...Successfully got an update from the "skm" chart repository
...Successfully got an update from the "azure-samples" chart repository
...Successfully got an update from the "dapr" chart repository
...Successfully got an update from the "confluentinc" chart repository
...Successfully got an update from the "kuma" chart repository
...Successfully got an update from the "lifen-charts" chart repository
...Successfully got an update from the "kiwigrid" chart repository
...Successfully got an update from the "hashicorp" chart repository
...Successfully got an update from the "elastic" chart repository
...Successfully got an update from the "longhorn" chart repository
...Successfully got an update from the "open-telemetry" chart repository
...Successfully got an update from the "sumologic" chart repository
...Successfully got an update from the "spacelift" chart repository
...Successfully got an update from the "datadog" chart repository
...Successfully got an update from the "rook-release" chart repository
...Successfully got an update from the "signoz" chart repository
...Successfully got an update from the "argo-cd" chart repository
...Successfully got an update from the "openzipkin" chart repository
...Successfully got an update from the "rancher-latest" chart repository
...Unable to get an update from the "epsagon" chart repository (https://helm.epsagon.com):
        Get "https://helm.epsagon.com/index.yaml": dial tcp: lookup helm.epsagon.com on 10.255.255.254:53: server misbehaving
...Successfully got an update from the "crossplane-stable" chart repository
...Successfully got an update from the "grafana" chart repository
...Successfully got an update from the "newrelic" chart repository
...Successfully got an update from the "bitnami" chart repository
Update Complete. ⎈Happy Helming!⎈

Let’s try a chart with no extra values

$ helm install sftpgo -n sftpgo --create-namespace skm/sftpgo
NAME: sftpgo
LAST DEPLOYED: Wed Jul 10 08:50:04 2024
NAMESPACE: sftpgo
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
1. Get the application URL by running these commands:
  export POD_NAME=$(kubectl get pods --namespace sftpgo -l "app.kubernetes.io/name=sftpgo,app.kubernetes.io/instance=sftpgo" -o jsonpath="{.items[0].metadata.name}")
  export CONTAINER_PORT=$(kubectl get pod --namespace sftpgo $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")
  echo "Visit http://127.0.0.1:8080 to use your application"
  kubectl --namespace sftpgo port-forward $POD_NAME 8080:$CONTAINER_PORT

Let’s try a port-forward to the service

$ kubectl port-forward svc/sftpgo -n sftpgo 8888:80
Forwarding from 127.0.0.1:8888 -> 8080
Forwarding from [::1]:8888 -> 8080
Handling connection for 8888
Handling connection for 8888
Handling connection for 8888

We can see an empty system at this stage, but we do have an admin user

If I log out and come back, we can see that admin account is persisted (so first one in is admin)

In users, I can add a new user

I thought I might be able to port-forward to 2022 on the pod

But my admin and “isaac” user does not accept the pass

That’s when I realized the “Users” for FTP need to be defined in “Users” not “Admins”.

Let’s add an ftp user

That worked just fine. I could push and list a file

Since my load balancing is done with NGinx, I likely will have some challenges with SFTP.

I figured that using a NodePort service might solve this.

I created a new NodePort service with the same selectors to send TCP over to 2022.

$ kubectl get svc -n sftpgo my-sftpgo-service -o yaml
apiVersion: v1
kind: Service
metadata:
  creationTimestamp: "2024-07-10T14:20:49Z"
  labels:
    app: my-sftpgo-service
  name: my-sftpgo-service
  namespace: sftpgo
  resourceVersion: "5201202"
  uid: 73a95eda-5fd7-49b5-8516-26c758a1c769
spec:
  clusterIP: 10.43.82.171
  clusterIPs:
  - 10.43.82.171
  externalTrafficPolicy: Cluster
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - name: 2022-2022
    nodePort: 30844
    port: 2022
    protocol: TCP
    targetPort: 2022
  selector:
    app.kubernetes.io/instance: sftpgo
    app.kubernetes.io/name: sftpgo
  sessionAffinity: None
  type: NodePort
status:
  loadBalancer: {}

Which when launch shows up

$ kubectl get svc -n sftpgo
NAME                TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                   AGE
sftpgo              ClusterIP   10.43.102.30   <none>        22/TCP,80/TCP,10000/TCP   31m
my-sftpgo-service   NodePort    10.43.82.171   <none>        2022:30844/TCP            59s

I can now test

$ sftp -P 30844 myftpuser@192.168.1.81
The authenticity of host '[192.168.1.81]:30844 ([192.168.1.81]:30844)' can't be established.
ECDSA key fingerprint is SHA256:gIon3V2MTrSjuKOFIaml3i01hfYW3jZAm+yMTVptQwo.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '[192.168.1.81]:30844' (ECDSA) to the list of known hosts.
Password:
Connected to 192.168.1.81.
sftp> ls
test.txt

I would not even need to move this to my production cluster.

Knowing I have a NodePort, I could expose a port to direct traffic to a Node on that Test Cluster

I’ll now use a GCP Cloud Shell (just to get an external Linux instance) to test SFTP back to my pod in the test cluster

Then I’ll remove that profile from my firewall.

One thing you’ll note is that there are no PVCs.

$ kubectl get pvcs -n sftpgo
error: the server doesn't have a resource type "pvcs"

One quick and easy solution is to mount a dir on the host

We can use the values:

$ cat sftp-values.yaml
volumes:
- name: tmpdir
  hostPath:
    path: /tmp/

volumeMounts:
- name: tmpdir
  mountPath: /tmp/tmpdir

Using those in a helm deploy

$ helm install sftpgo -n sftpgo -f ./sftp-values.yaml skm/sftpgo

I added a user with “/tmp” as the root dir

This will let me see the hostpath mount when I sftp

$ sftp -P 30844 mysftpuser@192.168.1.81
Password:
Connected to 192.168.1.81.
sftp> ls
tmpdir
sftp> cd tmpdir
sftp> put sftp-values.yaml
Uploading sftp-values.yaml to /tmpdir/sftp-values.yaml
sftp-values.yaml                                                                                                                                                        100%  111    60.5KB/s   00:00
sftp> exit

Hopping onto the Kubernetes node, I can see that delivered file

builder@anna-MacBookAir:/tmp$ ls -ltra | tail -n3
drwxrwxr-x 18 builder builder  4096 Jul 10 03:00 jekyll
-rw-r--r--  1 builder builder   111 Jul 10 09:42 sftp-values.yaml
drwxrwxrwt 29 root    root    12288 Jul 10 09:42 .

Connections

We can view active connections

Here you can see how a disconnect works

FileGator

Another tool I was turned on to by a MariusHosting Post is Filegator. The Install Guide has a few docker options.

So let’s start with Docker

builder@builder-T100:~/filegator$ docker run -p 8098:8080 -d filegator/filegator
Unable to find image 'filegator/filegator:latest' locally
latest: Pulling from filegator/filegator
6177a7f9989f: Pull complete 
bb4ea0ade393: Pull complete 
064c8ff4a89f: Pull complete 
835cacc3785c: Pull complete 
e34a344384e1: Pull complete 
be9df37f9cbb: Pull complete 
c0a5847d4744: Pull complete 
d60c0388eeb2: Pull complete 
c7c72fbfe2e5: Pull complete 
671bb7cbb446: Pull complete 
82a30a053cc6: Pull complete 
20f5530e4bb7: Pull complete 
4b774ba9951f: Pull complete 
d31e195367d5: Pull complete 
d635b61874c0: Pull complete 
c2beedeaa9c6: Pull complete 
15d7ea74f7fc: Pull complete 
4f4fb700ef54: Pull complete 
a310975d10d2: Pull complete 
6e1b9db332e4: Pull complete 
7c5a8edf8f5a: Pull complete 
ca5584a87aae: Pull complete 
65285fb793ce: Pull complete 
1f86e896400f: Pull complete 
17726cdda671: Pull complete 
d7fde4f92d1a: Pull complete 
4f2d7d3472ea: Pull complete 
Digest: sha256:9d40cebbeec0c7a8ee605deb6d8b8d44b08621e2158b8b54388e4dffafabd0cb
Status: Downloaded newer image for filegator/filegator:latest
516ac31e2554809419131375d806e8f08e8410433c74a8c66b5b28fae14dbc5c

I can then login with the local IP

Logging in with admin and password admin123 shows us the main page

I can upload a file

I can add users or change their settings. For instance, I changed the “guest” user to have Read and Download (default is no permissions)

Which will confirm I’m okay opening this up to the world

I can now verify that anonymous users can view and download files

I did try stopping and starting the container. I verified the files persist on a stop/start

builder@builder-T100:~/filegator$ docker ps | grep gator
516ac31e2554   filegator/filegator                                              "docker-php-entrypoi…"   3 hours ago     Up 3 hours              80/tcp, 0.0.0.0:8098->8080/tcp, :::8098->8080/tcp                                                           thirsty_hawking
builder@builder-T100:~/filegator$ docker stop 516ac31e2554
516ac31e2554
builder@builder-T100:~/filegator$ docker start 516ac31e2554
516ac31e2554

Jumping on the container, it’s clear it uses two folders, /private and /repository to store changes and files

-rw-r--r-- 1 www-data www-data  12214 Apr 24 12:25 .phpunit.result.cache
drwxr-xr-x 1 root     root       4096 Apr 24 12:26 ..
drwxr-xr-x 1 www-data www-data   4096 Apr 24 12:26 .
drwxrwxr-x 5 www-data www-data   4096 Jul 13 15:17 private
drwxrwxr-x 2 www-data www-data   4096 Jul 13 15:20 repository
www-data@516ac31e2554:~/filegator$ cd repository/
www-data@516ac31e2554:~/filegator/repository$ ls
cert.pem
www-data@516ac31e2554:~/filegator/repository$ pwd
/var/www/filegator/repository
www-data@516ac31e2554:~/filegator/repository$ ls -lra ../private/
total 36
-rwxrwxr-x 1 www-data www-data  314 Apr 24 12:23 users.json.blank
-rw-r--r-- 1 www-data www-data  326 Jul 13 17:47 users.json
drwxr-xr-x 2 www-data www-data 4096 Jul 13 15:20 tmp
drwxrwxr-x 2 www-data www-data 4096 Jul 13 15:17 sessions
drwxrwxr-x 2 www-data www-data 4096 Jul 13 15:19 logs
-rwxrwxr-x 1 www-data www-data   14 Apr 24 12:23 .htaccess
-rwxrwxr-x 1 www-data www-data   17 Apr 24 12:23 .gitignore
drwxr-xr-x 1 www-data www-data 4096 Apr 24 12:26 ..
drwxrwxr-x 5 www-data www-data 4096 Jul 13 15:17 .

I stopped and removed that container then launched with some volume mounts:

$ mkdir ./repository2 && chmod 777 ./repository2
$ mkdir ./private2 && chmod 777 ./private2
$ docker run -p 8098:8080 -d \
  -v /home/builder/filegator/repository2:/var/www/filegator/repository \
  -v /home/builder/filegator/private2:/var/www/filegator/private \
  filegator/filegator

I did need to copy some blank files over. Seems it needs logs and a blank users.json

builder@builder-T100:~/filegator$ mkdir private2/logs
builder@builder-T100:~/filegator$ chmod 777 private2/logs
builder@builder-T100:~/filegator$ cp private/users.json.blank private2/
logs/ tmp/  
builder@builder-T100:~/filegator$ cp private/users.json.blank private2/users.json
builder@builder-T100:~/filegator$ cp private/users.json.blank private2/users.json.blank

I’m now going to log in as admin and add a new user

I noticed the users did not persist. I’m guessing it’s permissions related

Once I set the perms

builder@builder-T100:~/filegator$ ls -ltra ./private2
total 24
drwxrwxr-x 13 builder  builder  4096 Jul 13 13:11 ..
drwxr-xr-x  2 www-data www-data 4096 Jul 13 13:15 tmp
drwxrwxrwx  2 builder  builder  4096 Jul 13 13:15 logs
-rwxrwxr-x  1 builder  builder   314 Jul 13 13:16 users.json
-rwxrwxr-x  1 builder  builder   314 Jul 13 13:16 users.json.blank
drwxrwxrwx  4 builder  builder  4096 Jul 13 13:16 .
builder@builder-T100:~/filegator$ ls -ltra ./private2/logs
total 16
drwxrwxrwx 2 builder  builder  4096 Jul 13 13:15 .
drwxrwxrwx 4 builder  builder  4096 Jul 13 13:16 ..
-rw-r--r-- 1 www-data www-data 4838 Jul 13 13:20 app.log
builder@builder-T100:~/filegator$ chmod 777 ./private2/users.json
builder@builder-T100:~/filegator$ chmod 777 ./private2/users.json.blank 

It worked

I tested a stop and remove, then recreating the container and found the users persisted. So now i have a solution that might be a bit more durable.

External ingress

Let’s expose this through the production cluster using an external endpoint.

First, I’ll need an A Record

$ az account set --subscription "Pay-As-You-Go" && az network dns record-set a add-record -g idjdnsrg -z tpk.pw -a 75.73.224.240 -n filegator
{
  "ARecords": [
    {
      "ipv4Address": "75.73.224.240"
    }
  ],
  "TTL": 3600,
  "etag": "7967af16-b75d-4114-924e-158157ed572f",
  "fqdn": "filegator.tpk.pw.",
  "id": "/subscriptions/d955c0ba-13dc-44cf-a29a-8fed74cbb22d/resourceGroups/idjdnsrg/providers/Microsoft.Network/dnszones/tpk.pw/A/filegator",
  "name": "filegator",
  "provisioningState": "Succeeded",
  "resourceGroup": "idjdnsrg",
  "targetResource": {},
  "trafficManagementProfile": {},
  "type": "Microsoft.Network/dnszones/A"
}

We can then apply an ingress

$ cat filegator.yaml

---
apiVersion: v1
kind: Endpoints
metadata:
  name: filegator-external-ip
subsets:
- addresses:
  - ip: 192.168.1.100
  ports:
  - name: filegatorint
    port: 8098
    protocol: TCP
---
apiVersion: v1
kind: Service
metadata:
  name: filegator-external-ip
spec:
  clusterIP: None
  clusterIPs:
  - None
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  - IPv6
  ipFamilyPolicy: RequireDualStack
  ports:
  - name: filegator
    port: 80
    protocol: TCP
    targetPort: 8098
  sessionAffinity: None
  type: ClusterIP
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    cert-manager.io/cluster-issuer: azuredns-tpkpw
    kubernetes.io/ingress.class: nginx
    kubernetes.io/tls-acme: "true"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
    nginx.org/websocket-services: filegator-external-ip
  generation: 1
  labels:
    app.kubernetes.io/instance: filegatoringress
  name: filegatoringress
spec:
  rules:
  - host: filegator.tpk.pw
    http:
      paths:
      - backend:
          service:
            name: filegator-external-ip
            port:
              number: 80
        path: /
        pathType: ImplementationSpecific
  tls:
  - hosts:
    - filegator.tpk.pw
    secretName: filegator-tls

$ kubectl apply -f ./filegator.yaml
endpoints/filegator-external-ip created
service/filegator-external-ip created
ingress.networking.k8s.io/filegatoringress created

Once I see the cert satisified

$ kubectl get cert filegator-tls
NAME            READY   SECRET          AGE
filegator-tls   True    filegator-tls   2m32s

I can now reach it externally with proper TLS

I logged in and changed the admin user password first

I tested uploading a file

I also went on my phone and uploaded from there

Which back on the PC we can see that for small images, we get an inline preview

The files are ending up on the volume mount on my dockerhost

builder@builder-T100:~/filegator$ ls -ltra repository2/
total 484
drwxrwxr-x 13 builder  builder    4096 Jul 13 13:11  ..
-rw-r--r--  1 www-data www-data 203659 Jul 13 13:39 'BWCA camping packing list.docx'
-rw-r--r--  1 www-data www-data 282301 Jul 13 13:47  Screenshot_20240713_134734_Chrome.jpg
drwxrwxrwx  2 builder  builder    4096 Jul 13 13:47  .

I could stop here and just swap mounts, but I think I will instead follow the pattern I used with Vaultwarden backups and use a crontab to backup periodically to a mounted fileshare to one of the NASes

In our /etc/fstab, we see the mount to a NAS

builder@builder-T100:~/filegator$ cat /etc/fstab | grep filestation
192.168.1.116:/volume1/docker-filestation  /mnt/filestation nfs    auto,nofail,noatime,nolock,intr,tcp,actimeo=1800        0       0

In my crontab, I’ll add a new backup file

builder@builder-T100:~/filegator/repository2$ crontab -l
@reboot /sbin/swapoff -a
45 3 * * * tar -zcvf /mnt/filestation/vaultbackups.$(date '+\%Y-\%m-\%d_\%Hh\%Mm').tgz /home/builder/vaultwarden/data
55 3 * * * tar -zcvf /mnt/filestation/filegator.$(date '+\%Y-\%m-\%d_\%Hh\%Mm').tgz /home/builder/filegator/repository2

I’ll do a quick test to see if it works

builder@builder-T100:~/filegator/repository2$ date
Sun Jul 14 06:14:59 AM CDT 2024
builder@builder-T100:~/filegator/repository2$ crontab -e
crontab: installing new crontab

builder@builder-T100:~/filegator/repository2$ crontab -l | grep filegator
16 6 * * * tar -zcvf /mnt/filestation/filegator.$(date '+\%Y-\%m-\%d_\%Hh\%Mm').tgz /home/builder/filegator/repository2

And I can confirm I now have a daily backup to the NAS

Summary

We tested out two different but quite nice Open-Source file transfer tools; SFTPGo and Filegator. SFTPGo was easy to setup and because it’s primarily for exposing SFTP, we were able to serve it from a test cluster through the firewall.

With Filegator, we quickly setup a docker instance on a dockerhost and tested locally. We then changed to exposing through a production Kubernetes cluster to allow us TLS ingress with a proper domain. Lastly, we tweaked the volume mounts and added a daily backup to durable storage to provide proper DR in case of a pod crash or docker failure.

Of the two, SFTPGo is far more configurable and could be used with groups and teams. There were a lot of user features I didn’t explore. Filegator, while simple, is also very easy to understand and use. Of the two, I’ll likely keep Filegator in the mix as a simple fast upload tool for on-the-go screenshots and sharing.