Published: Dec 3, 2024 by Isaac Johnson
We need to talk about BlueSky. It was founded in 2019 (as an experimental initiative within Twitter) and gained it’s independence in 2021. It’s only been open to the general public since February 2024 (about 9 months at time of writing).
In a post-twitter world, I had assumed either Meta Threads or Mastodon would take over. I personally use both - the former for social and the latter for technical. However, BlueSky stayed on my radar and I would hear my DTNS hosts bring it up from time to time. I was on the private beta, but did not really use it.
Then it just took off. As Musk, the new owner of Twitter got nuttier and nuttier and went fully “Dark Maga” (his words) in the end, BlueSky Social developers saw these “Elon Musk Events”, as they termed them, drive millions of users to BlueSky in spikes. I’m not going to paraphrase Wikipedia for this, but you can read there, with sources, all the various large-scale spiking events. Most recently, I heard of Taylor Swift sending all her followers over basically telling Musk, “ohh, look what you made me do” (I have daughters, judge away).
It’s in this post-election haze we now see reports that as of November 19th, Bluesky officially crossed 20 million users (that is 3x growth in 3 months).
What now
So now that we have some real viable contenders, it might be worth tackling how we use the API to post. I’m not looking to encourage spam, but those of us that write lots of content really don’t want to be on deck to fire up mobile apps and retype our blurbs over and over manually. I consider API driven posts more toil-reduction than spam-enablement.
Note: I avoid politics on this blog, but as I am anti-Nazi and believe all people are equal and fully support LBGTQA+ and women’s reproductive freedoms, my leanings are evident and explain why I deleted my Twitter some time ago. Though I still keep things like Instagram if only to watch the TSA feed (which is hilarious if you’ve never looked)
The API
To get started, I’ll assume you have gotten yourself a BlueSky account from https://bsky.app/ or https://blueskyproject.io/. I’m at https://bsky.app/profile/isaacj.bsky.social myself.
Next we need to get an access Java Web Token (JWT). Since my “domain” is bsky.social ($userid.$domain – e.g. isaacj.bsky.social), I’ll be useing “bsky.social” as my base on the REST calls (after HTTPS)
To get the Bearer Token we need for the rest of the calls (the JWT), we can use the “com.atproto.server.createSession” endpoint
$ curl -X POST https://bsky.social/xrpc/com.atproto.server.createSession -H "Content-Type: application/json" -d '{"identifier": "isaacj.bsky.social", "password": "PutYourPasswordHere"}'
{"did":"did:plc:asdfasdfasdfasdf","didDoc":{"@context":["https://www.w3.org/ns/did/v1","https://w3id.org/security/multikey/v1","https://w3id.org/security/suites/secp256k1-2019/v1"],"id":"did:plc:aaasdfasfasdfasdfasdf","alsoKnownAs":["at://isaacj.bsky.social"],"verificationMethod":[{"id":"did:plc:r7e4l3eicmgbdnturl2wotmz#atproto","type":"Multikey","controller":"did:plc:asdfasdfsafasdf","publicKeyMultibase":"zQ3shp8kuLJyvwH4fZZ2prAQsTfYLRBM3F7GMtexU7crPwSjs"}],"service":[{"id":"#atproto_pds","type":"AtprotoPersonalDataServer","serviceEndpoint":"https://morel.us-east.host.bsky.network"}]},"handle":"isaacj.bsky.social","email":"isaac@freshbrewed.science","emailConfirmed":true,"emailAuthFactor":false,"accessJwt":"LoremipsumdolorsitametconsecteturadipiscingelitQuisquesempertinciduntiaculis","refreshJwt":"Inhachabitasseplateadictumst.Etiamporttitorerosenimnecpretiumeratmalesuadanon","active":true}
I piped that to a file and showed I could just pull the accessJWT with jq:
$ cat o.json | jq -r .accessJwt
LoremipsumdolorsitametconsecteturadipiscingelitQuisquesempertinciduntiaculis
In this example we see after I authed, I got back a JSON block with an accessJwt we can now use to post:
$ curl -X POST https://bsky.social/xrpc/com.atproto.repo.createRecord -H "Authorization: Bearer LoremipsumdolorsitametconsecteturadipiscingelitQuisquesempertinciduntiaculis" -H "Content-Type: application/json" -d "{\"repo\": \"isaacj.bsky.social\", \"collection\": \"app.bsky.feed.post\", \"record\": {\"text\": \"Have the ice fishing itch, but it will be a bit before we have a proper layer. Cool out, but I'm tempted to pull the canoe out today for chilly paddle.\", \"createdAt\": \"$(date -u +%Y-%m-%dT%H:%M:%SZ)\"}}"
{"uri":"at://did:plc:r7e4l3eicmgbdnturl2wotmz/app.bsky.feed.post/3lboyvkanaz2n","cid":"bafyreifgd37yjn4yxockhds5tfwdnkoees4ivkc6evuwqbvzkc3lxmnhxe","commit":{"cid":"bafyreie5eougvtzkexxcrs6xutkq4jt2nivdlndslsnf44uaogqgolgvfe","rev":"3lboyvkav2z2n"},"validationStatus":"valid"}
I confirmed it posted on the web interface
Fantastic! At the very least, we have a curl based approach that will work.
Tying it into CICD
With Mastodon, I pull from my Jekyll feeds to build a payload to post with a curl line. Since November, when I moved YOURLS to production, it also gets a valid short URL - this is more important now as there are character limits in BlueSky.
We can start by looking at that existing Mastodon posting block in my Github Actions YAML:
- name: Post to Mastodon
run: |
# clean
rm ./title.txt || true
rm ./payload.json || true
rm ./target.url || true
rm ./yourls.output.json || true
export LATESTFILE=`ls -l _posts/ | grep ".* 20[0-9][0-9]-[0-9][0-9].*markdown" | tail -n1 | sed 's/.* \(20[0-9][0-9]-[0-9][0-9].*markdown\)/\1/'`
export LATESTURL=`ls -l _posts/ | grep ".* 20[0-9][0-9]-[0-9][0-9].*markdown" | tail -n1 | sed 's/.* \(20[0-9][0-9]-[0-9][0-9]-[0-9][0-9]-\)\(.*\)\.markdown/\2.html/'`
echo '{"status":"' | tr -d '\n' > payload.json
cat _posts/$LATESTFILE | grep "^title: " | head -n 1 | sed 's/^title: "\(.*\)"/\1/' | tr -d '\n'> title.txt
cat _posts/$LATESTFILE | grep "^social: " | head -n 1 | sed 's/^social: "\(.*\)"/\1/' | tr -d '\n'>> payload.json
cat _posts/$LATESTFILE | grep "^date: " | head -n 1 | sed "s/^date: .\([0-9]*\)-\([0-9]*\)-\([0-9]*\) .*/https:\/\/freshbrewed.science\/\1\/\2\/\3\/$LATESTURL/g" | sed 's/.markdown/.html/' | tr -d '\n' > target.url
curl --data-urlencode "url=`cat target.url`" --data-urlencode "title=`cat title.txt`" --data-urlencode 'format=json' --data-urlencode 'action=shorturl' --data-urlencode 'signature=adfasdfasdf' "https://go.tpk.pw/yourls-api.php" | tee yourls.output.json
echo " " | tr -d '\n' >> payload.json
cat yourls.output.json | jq -r '.shorturl' | tr -d '\n' >> payload.json
echo '"}' >> payload.json
cat payload.json | base64
# allow a way to not repost on image updates and hotfixes
log=$(git log -n 1)
if [[ $log != *"SKIPSOCIAL"* ]]; then
echo "CHECK-OK: posting to social"
curl -X POST -H "Authorization: Bearer $MASTODONAPI" -H 'Content-Type: application/json' -d @payload.json https://noc.social/api/v1/statuses
else
echo "CHECK-SKIP: Skip Posting To Social.. would have posted:"
cat payload.json
fi
env:
MASTODONAPI: $
GHTOKEN: $
As you can see here, I find the latest post to get the translated URL then parse the top for the ‘title, social, and date’.
I also have a special check for “SKIPSOCIAL” in my GIT logs - this way if I need to do a quick correction (like a hotfix to correct a misspelling, or sneak in a backsplash update) it won’t double post out to socials.
To get started, I set out to create a local bash script I could use to test:
#!/bin/bash
set -x
#FROM SECRETS IN GH
export BSKYUSER=isaacj.bsky.social
export BSKYPASS=MyPasswordHereEscapeSpecialCharactersIfNeeded
## Idempotency for fun and profit
rm ./title.txt || true
rm ./bsky.payload.json || true
rm ./target.url || true
rm ./yourls.output.json || true
rm ./bskyauth.sjon || true
export PDSHOST="https://bsky.social"
curl -X POST $PDSHOST/xrpc/com.atproto.server.createSession \
-H "Content-Type: application/json" \
-d '{"identifier": "'"$BSKYUSER"'", "password": "'"$BSKYPASS"'"}' | tee bskyauth.json
export ACCESSJWT="`cat ./bskyauth.json | jq -r .accessJwt | tr -d '\n'`"
export DTIME="`date -u +%Y-%m-%dT%H:%M:%SZ`"
export LATESTFILE=`ls -l _posts/ | grep ".* 20[0-9][0-9]-[0-9][0-9].*markdown" | tail -n1 | sed 's/.* \(20[0-9][0-9]-[0-9][0-9].*markdown\)/\1/'`
export LATESTURL=`ls -l _posts/ | grep ".* 20[0-9][0-9]-[0-9][0-9].*markdown" | tail -n1 | sed 's/.* \(20[0-9][0-9]-[0-9][0-9]-[0-9][0-9]-\)\(.*\)\.markdown/\2.html/'`
# Get title and social text
cat _posts/$LATESTFILE | grep "^title: " | head -n 1 | sed 's/^title: "\(.*\)"/\1/' | tr -d '\n'> title.txt
cat _posts/$LATESTFILE | grep "^social: " | head -n 1 | sed 's/^social: "\(.*\)"/\1/' | tr -d '\n'> social.txt
# Prepare full URL
cat _posts/$LATESTFILE | grep "^date: " | head -n 1 | sed "s/^date: .\([0-9]*\)-\([0-9]*\)-\([0-9]*\) .*/https:\/\/freshbrewed.science\/\1\/\2\/\3\/$LATESTURL/g" | sed 's/.markdown/.html/' | tr -d '\n' > target.url
# Get Short URL
curl --data-urlencode "url=`cat target.url`" --data-urlencode "title=`cat title.txt`" --data-urlencode 'format=json' --data-urlencode 'action=shorturl' --data-urlencode 'signature=asdfasdfasdf' "https://go.tpk.pw/yourls-api.php" | tee yourls.output.json
# Just append to the social text block
cat yourls.output.json | jq -r '.shorturl' | tr -d '\n' >> social.txt
# Create the BlueSky payload
echo "{\"repo\": \"$BSKYUSER\", \"collection\":\"app.bsky.feed.post\", \"record\": {\"text\": \"`cat ./social.txt`\", \"createdAt\": \"$DTIME\"}}" > bsky.payload.json
# DEBUG - show the curl statement
echo "--------------------"
echo curl -X POST "$PDSHOST/xrpc/com.atproto.repo.createRecord" -H "Authorization: Bearer $ACCESSJWT" -H "Content-Type: application/json" -2 @bsky.payload.json
When run, as this is just a test, I see the payload is
'{"repo": "isaacj.bsky.social", "collection":"app.bsky.feed.post", "record": {"text": "Following up on the prior two writeups on OneDev, In this third and final, I'\''ll cover some more advanced setup including email templates, invites, and labels. I will show impersonation, child projects and forks and branch restrictions. I will touch on time management, backups and license management as well", "createdAt": "2024-11-24T16:52:32Z"}}'
And the posting line looks like
curl -X POST https://bsky.social/xrpc/com.atproto.repo.createRecord -H 'Authorization: Bearer Loremipsumdolorsit ametconsecteturadipiscingelit.Quisquesempertinciduntiaculis.Suspendisseatnequepulvinarvulputateturpisfeugiatcongue nibh.Sedauctormagnaapulvinarrutrum' -H 'Content-Type: application/json' -2 @bsky.payload.json
That payload looks about right
$ cat bsky.payload.json | jq
{
"repo": "isaacj.bsky.social",
"collection": "app.bsky.feed.post",
"record": {
"text": "Following up on the prior two writeups on OneDev, In this third and final, I'll cover some more advanced setup including email templates, invites, and labels. I will show impersonation, child projects and forks and branch restrictions. I will touch on time management, backups adn license management as well",
"createdAt": "2024-11-24T16:52:32Z"
}
}
Since I’m happy with that, at least enough to test, I’ll update my Github Actions Secrets to pass in the Username and Password. I don’t really need to make the username a secret, but it does make it easier to update later if they are co-located
I next updated the Github Workflow block to use the variables as well as put out a little debug code
- name: Post to BlueSky
run: |
# clean
rm ./title.txt || true
rm ./bsky.payload.json || true
rm ./target.url || true
rm ./yourls.output.json || true
rm ./bskyauth.sjon || true
export PDSHOST="https://bsky.social"
curl -X POST $PDSHOST/xrpc/com.atproto.server.createSession \
-H "Content-Type: application/json" \
-d '{"identifier": "'"$BSKYUSER"'", "password": "'"$BSKYPASS"'"}' | tee bskyauth.json
export ACCESSJWT="`cat ./bskyauth.json | jq -r .accessJwt | tr -d '\n'`"
export DTIME="`date -u +%Y-%m-%dT%H:%M:%SZ`"
export LATESTFILE=`ls -l _posts/ | grep ".* 20[0-9][0-9]-[0-9][0-9].*markdown" | tail -n1 | sed 's/.* \(20[0-9][0-9]-[0-9][0-9].*markdown\)/\1/'`
export LATESTURL=`ls -l _posts/ | grep ".* 20[0-9][0-9]-[0-9][0-9].*markdown" | tail -n1 | sed 's/.* \(20[0-9][0-9]-[0-9][0-9]-[0-9][0-9]-\)\(.*\)\.markdown/\2.html/'`
cat _posts/$LATESTFILE | grep "^title: " | head -n 1 | sed 's/^title: "\(.*\)"/\1/' | tr -d '\n'> title.txt
cat _posts/$LATESTFILE | grep "^social: " | head -n 1 | sed 's/^social: "\(.*\)"/\1/' | tr -d '\n'> social.txt
# I suspect I need an extra space
echo " " | tr -d '\n' >> social.txt
cat _posts/$LATESTFILE | grep "^date: " | head -n 1 | sed "s/^date: .\([0-9]*\)-\([0-9]*\)-\([0-9]*\) .*/https:\/\/freshbrewed.science\/\1\/\2\/\3\/$LATESTURL/g" | sed 's/.markdown/.html/' | tr -d '\n' > target.url
curl --data-urlencode "url=`cat target.url`" --data-urlencode "title=`cat title.txt`" --data-urlencode 'format=json' --data-urlencode 'action=shorturl' --data-urlencode 'signature=asdfasfasdf' "https://go.tpk.pw/yourls-api.php" | tee yourls.output.json
cat yourls.output.json | jq -r '.shorturl' | tr -d '\n' >> social.txt
echo "{\"repo\": \"$BSKYUSER\", \"collection\":\"app.bsky.feed.post\", \"record\": {\"text\": \"`cat ./social.txt`\", \"createdAt\": \"$DTIME\"}}" > bsky.payload.json
set -x
cat bsky.payload.json
echo "================ now posting ================"
curl -X POST "$PDSHOST/xrpc/com.atproto.repo.createRecord" -H "Authorization: Bearer $ACCESSJWT" -H "Content-Type: application/json" -2 @bsky.payload.json
env:
BSKYUSER: $
BSKYPASS: $
GHTOKEN: $
I then added this to the candidate post for OneDev part 2 which is set to post in the next day. Rather than post meaningless dribble to verify my flow, I’ll just work this post over time to tweak any payload issues (one of the benefits to writing things with enough runway).
I did not see it above when I wrote it, but my first live test failed:
The typo was in
curl -X POST "$PDSHOST/xrpc/com.atproto.repo.createRecord" -H "Authorization: Bearer $ACCESSJWT" -H "Content-Type: application/json" -2 @bsky.payload.json
Where -2 @bsky.payload.json should have been -d @bsky.payload.json
Instead of hotfixing, I’ll try and manually use the line since I did kick the payload out to the Github output
$ cat bsky.payload.json | jq
{
"repo": "***",
"collection": "app.bsky.feed.post",
"record": {
"text": "Following up on the first OneDev post, I'll cover using OneDev for artifacts, build agent setup, pipelines, and container pushes. It is a powerful Open-Source CICD suite so there is lots to cover. https://go.tpk.pw/888yq",
"createdAt": "2024-11-26T09:14:05Z"
}
}
I need to replace “repo” with my username - a consequence of using a Github secret is it is properly masked in output
$ cat bsky.payload.json | jq
{
"repo": "isaacj.bsky.social",
"collection": "app.bsky.feed.post",
"record": {
"text": "Following up on the first OneDev post, I'll cover using OneDev for artifacts, build agent setup, pipelines, and container pushes. It is a powerful Open-Source CICD suite so there is lots to cover. https://go.tpk.pw/888yq",
"createdAt": "2024-11-26T09:14:05Z"
}
}
Now let’s try locally (after I set all the variables, of course)
$ curl -X POST "$PDSHOST/xrpc/com.atproto.repo.createRecord" -H "Authorization: Bearer $ACCESSJWT" -H "Content-Type: application/json" -d @bsky.payload.json
{"error":"SyntaxError","message":"Unexpected non-whitespace character after JSON at position 204"}
JQ agrees, I did something goof
$ curl -X POST "$PDSHOST/xrpc/com.atproto.repo.createRecord" -H "Authorization: Bearer $ACCESSJWT" -H "Content-Type: application/json" -d @bsky.payload.json
{"uri":"at://did:plc:r7e4l3eicmgbdnturl2wotmz/app.bsky.feed.post/3lbtzeys42j2d","cid":"bafyreigsaqkzga5jlncwim47pqiuixa4zb2lsctv6nvawykbpu66c5rgny","commit":{"cid":"bafyreiabgbqmfhvgil7gvrzcpbgffgtihulw6ougpdzy7zuz52mwtabff4","rev":"3lbtzeysjpz2d"},"validationStatus":"valid"}
While this did post, it did not handle the link
As I dug into links in their documentation, I realized this might be a bit trickier than I first thought.
In fact, their guidance is to use the AT Protocol Libraries to write native python or typescript
Building a Microservice
Okay, why fart around when we can just get to the end and build a container?
note: I committed the final code to https://github.com/idjohnson/pybsposter for easy reference
In short, there are three files.
requirements.txt
flask
atproto
A Dockerfile to serve
# Use an official Python runtime as a parent image
FROM python:3.9-slim
# Set the working directory to /app
WORKDIR /app
# Copy the current directory contents into the container at /app
COPY . /app
# Install any needed packages specified in requirements.txt
RUN pip install -r requirements.txt
# Make port 5000 available to the world outside this container
EXPOSE 5000
# Define environment variable
ENV NAME World
# Run app.py when the container launches
CMD ["python", "app.py"]
And an app.py:
from flask import Flask, request, jsonify
from atproto import Client, client_utils
app = Flask(__name__)
@app.route('/post', methods=['POST'])
def handle_post():
data = request.json
username = data.get('USERNAME')
password = data.get('PASSWORD')
text = data.get('TEXT')
link = data.get('LINK')
client = Client()
profile = client.login(username, password)
text = client_utils.TextBuilder().text(text).link(link,link)
post = client.send_post(text)
client.like(post.uri, post.cid)
response = {
"YOU ARE: ": profile.display_name,
"TEXT": text,
"LINK": link
}
return jsonify(response)
if __name__ == '__main__':
app.run(host='0.0.0.0', port=5000)
I’ll try building it local
$ docker build -t pybsposter:0.1 .
[+] Building 20.6s (10/10) FINISHED docker:default
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 525B 0.0s
=> [internal] load metadata for docker.io/library/python:3.9-slim 1.3s
=> [auth] library/python:pull token for registry-1.docker.io 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [1/4] FROM docker.io/library/python:3.9-slim@sha256:6250eb7983c08b3cf5a7db9309f8630d3ca03dd152158fa37a3f8daaf397085d 7.0s
=> => resolve docker.io/library/python:3.9-slim@sha256:6250eb7983c08b3cf5a7db9309f8630d3ca03dd152158fa37a3f8daaf397085d 0.0s
=> => sha256:4920a3bd5f7ed3269b647eb643846a0652dc21daa31763b4afc848701981d141 3.51MB / 3.51MB 0.7s
=> => sha256:77edb37367fad6d17c53a3cabdf41a57c0221a49f77250d97e3f5bb1fc1ed6e0 14.93MB / 14.93MB 1.1s
=> => sha256:6250eb7983c08b3cf5a7db9309f8630d3ca03dd152158fa37a3f8daaf397085d 10.41kB / 10.41kB 0.0s
=> => sha256:43e98aa4594b2a62ace026fb04338453f799bd6012b7933ecd442d6876787cb5 1.75kB / 1.75kB 0.0s
=> => sha256:6a22698eab0ea915af39918e5d2e4f27e49afc6944fd1d97466e13820324bc62 5.41kB / 5.41kB 0.0s
=> => sha256:2d429b9e73a6cf90a5bb85105c8118b30a1b2deedeae3ea9587055ffcb80eb45 29.13MB / 29.13MB 1.6s
=> => sha256:02c34c079cc82f150c24eae4d136fd997632cd64c1f922aa8c5974be33792ae5 255B / 255B 0.8s
=> => extracting sha256:2d429b9e73a6cf90a5bb85105c8118b30a1b2deedeae3ea9587055ffcb80eb45 3.1s
=> => extracting sha256:4920a3bd5f7ed3269b647eb643846a0652dc21daa31763b4afc848701981d141 0.3s
=> => extracting sha256:77edb37367fad6d17c53a3cabdf41a57c0221a49f77250d97e3f5bb1fc1ed6e0 1.6s
=> => extracting sha256:02c34c079cc82f150c24eae4d136fd997632cd64c1f922aa8c5974be33792ae5 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 1.38kB 0.0s
=> [2/4] WORKDIR /app 0.3s
=> [3/4] COPY . /app 0.0s
=> [4/4] RUN pip install -r requirements.txt 11.4s
=> exporting to image 0.5s
=> => exporting layers 0.5s
=> => writing image sha256:f009e380a6a0db6fdc7d15fb30a7bc896db2187758208405bcaf35713c7a6825 0.0s
=> => naming to docker.io/library/pybsposter:0.1 0.0s
1 warning found (use docker --debug to expand):
- LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 17)
Then run it
$ docker run -d -p 5550:5000 pybsposter:0.1
I created a payload
$ cat payload.json
{ "USERNAME": "isaacj.bsky.social", "PASSWORD": "asdfasdfasdf", "TEXT": "Following up on the first OneDev post, I'll cover using OneDev for artifacts, build agent setup, pipelines, and container pushes. It is a powerful Open-Source CICD suite so there is lots to cover.", "LINK": "https://go.tpk.pw/888yq" }
Then tested
$ curl -X POST http://localhost:5550 -H "Content-Type: application/json" -d @payload.json
[2024-11-26 12:41:04,355] ERROR in app: Exception on /post [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1511, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 919, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 917, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 902, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
File "/app/app.py", line 29, in handle_post
return jsonify(response)
File "/usr/local/lib/python3.9/site-packages/flask/json/__init__.py", line 170, in jsonify
return current_app.json.response(*args, **kwargs) # type: ignore[return-value]
File "/usr/local/lib/python3.9/site-packages/flask/json/provider.py", line 214, in response
f"{self.dumps(obj, **dump_args)}\n", mimetype=self.mimetype
File "/usr/local/lib/python3.9/site-packages/flask/json/provider.py", line 179, in dumps
return json.dumps(obj, **kwargs)
File "/usr/local/lib/python3.9/json/__init__.py", line 234, in dumps
return cls(
File "/usr/local/lib/python3.9/json/encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/local/lib/python3.9/json/encoder.py", line 257, in iterencode
return _iterencode(o, 0)
File "/usr/local/lib/python3.9/site-packages/flask/json/provider.py", line 121, in _default
raise TypeError(f"Object of type {type(o).__name__} is not JSON serializable")
TypeError: Object of type TextBuilder is not JSON serializable
172.17.0.1 - - [26/Nov/2024 12:41:04] "POST /post HTTP/1.1" 500 -
While I see the error, which we will address, it did indeed post:
This is because we cannot natively use a text builder object as the output in a jsonify which expects strings.
I fixed the code (and disabled the self-like since that seems a bit much)
from flask import Flask, request, jsonify
from atproto import Client, client_utils
app = Flask(__name__)
@app.route('/post', methods=['POST'])
def handle_post():
data = request.json
username = data.get('USERNAME')
password = data.get('PASSWORD')
text = data.get('TEXT')
link = data.get('LINK')
client = Client()
profile = client.login(username, password)
builder = client_utils.TextBuilder().text(text).link(link,link)
text_string = str(builder) # Convert TextBuilder to a string
post = client.send_post(builder)
# Don't really need to like my own posts
# client.like(post.uri, post.cid)
response = {
"YOU ARE:": profile.display_name,
"TEXT": text_string,
"LINK": link
}
return jsonify(response)
if __name__ == '__main__':
app.run(host='0.0.0.0', port=5000)
Let’s give that a try. I’ll build with Dockerhub names so others can easily use
builder@DESKTOP-QADGF36:~/Workspaces/pyBSPoster$ docker build -t idjohnson/pybsposter:0.2 .
[+] Building 0.4s (9/9) FINISHED docker:default
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 525B 0.0s
=> [internal] load metadata for docker.io/library/python:3.9-slim 0.3s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [1/4] FROM docker.io/library/python:3.9-slim@sha256:6250eb7983c08b3cf5a7db9309f8630d3ca03dd152158fa37a3f8daaf397085d 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 1.18kB 0.0s
=> CACHED [2/4] WORKDIR /app 0.0s
=> CACHED [3/4] COPY . /app 0.0s
=> CACHED [4/4] RUN pip install -r requirements.txt 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:90d962489959960f8932c64c5fb5e86720e5ad840f028036f37b80e6e4fd90bd 0.0s
=> => naming to docker.io/idjohnson/pybsposter:0.2 0.0s
builder@DESKTOP-QADGF36:~/Workspaces/pyBSPoster$ docker build -t idjohnson/pybsposter:latest .
[+] Building 0.4s (9/9) FINISHED docker:default
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 525B 0.0s
=> [internal] load metadata for docker.io/library/python:3.9-slim 0.3s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [1/4] FROM docker.io/library/python:3.9-slim@sha256:6250eb7983c08b3cf5a7db9309f8630d3ca03dd152158fa37a3f8daaf397085d 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 1.18kB 0.0s
=> CACHED [2/4] WORKDIR /app 0.0s
=> CACHED [3/4] COPY . /app 0.0s
=> CACHED [4/4] RUN pip install -r requirements.txt 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:90d962489959960f8932c64c5fb5e86720e5ad840f028036f37b80e6e4fd90bd 0.0s
=> => naming to docker.io/idjohnson/pybsposter:latest
Then push to Dockerhub
builder@DESKTOP-QADGF36:~/Workspaces/pyBSPoster$ docker push idjohnson/pybsposter:latest
The push refers to repository [docker.io/idjohnson/pybsposter]
16ded61f8e99: Pushed
5b5603393e2d: Pushed
d4265f990c9f: Pushed
aacba17e24d9: Mounted from library/python
f751ad7c65c4: Mounted from library/python
7822e749b484: Mounted from library/python
c3548211b826: Mounted from library/python
latest: digest: sha256:2f15d73313442ea7153491dca3146f917745bf63d73d386db09545770299ad59 size: 1785
builder@DESKTOP-QADGF36:~/Workspaces/pyBSPoster$ docker push idjohnson/pybsposter:0.2
The push refers to repository [docker.io/idjohnson/pybsposter]
16ded61f8e99: Layer already exists
5b5603393e2d: Layer already exists
d4265f990c9f: Layer already exists
aacba17e24d9: Layer already exists
f751ad7c65c4: Layer already exists
7822e749b484: Layer already exists
c3548211b826: Layer already exists
0.2: digest: sha256:2f15d73313442ea7153491dca3146f917745bf63d73d386db09545770299ad59 size: 1785
I’d like to deploy this, and risk the misuse, but make it easy to just call with REST
I’ll make and launch a Kubernetes YAML manifest with a deployment and service
$ cat deploy.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: pybsposter
spec:
replicas: 1
selector:
matchLabels:
app: pybsposter
template:
metadata:
labels:
app: pybsposter
spec:
containers:
- name: pybsposter
image: idjohnson/pybsposter:latest
ports:
- containerPort: 5000
---
apiVersion: v1
kind: Service
metadata:
name: pybsposter
spec:
selector:
app: pybsposter
ports:
- protocol: TCP
port: 80
targetPort: 5000
I’ll apply and then check for a pod to come up
$ kubectl apply -f ./deploy.yaml
deployment.apps/pybsposter created
service/pybsposter created
$ kubectl get pods -l app=pybsposter
NAME READY STATUS RESTARTS AGE
pybsposter-d9cc5878d-hht85 1/1 Running 0 14s
Next, for ingress I’ll need an A Record
$ gcloud dns --project=myanthosproject2 record-sets create bskyposter.steeped.space --zone="steepedspace" --type="A" --ttl="300" --rrdatas="75.73.224.240"
NAME TYPE TTL DATA
bskyposter.steeped.space. A 300 75.73.224.240
I’ll then create an ingress
$ cat ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
cert-manager.io/cluster-issuer: gcpleprod2
ingress.kubernetes.io/proxy-body-size: "0"
ingress.kubernetes.io/ssl-redirect: "true"
kubernetes.io/ingress.class: nginx
kubernetes.io/tls-acme: "true"
nginx.ingress.kubernetes.io/proxy-body-size: "0"
nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.org/client-max-body-size: "0"
nginx.org/proxy-connect-timeout: "3600"
nginx.org/proxy-read-timeout: "3600"
nginx.org/websocket-services: pybsposter
name: pybspostergcpingress
spec:
rules:
- host: bskyposter.steeped.space
http:
paths:
- backend:
service:
name: pybsposter
port:
number: 80
path: /
pathType: ImplementationSpecific
tls:
- hosts:
- bskyposter.steeped.space
secretName: pybspostergcp-tls
$ kubectl apply -f ./ingress.yaml
ingress.networking.k8s.io/pybspostergcpingress created
When the cert is ready
$ kubectl get certs pybspostergcp-tls
NAME READY SECRET AGE
pybspostergcp-tls False pybspostergcp-tls 46s
$ kubectl get certs pybspostergcp-tls
NAME READY SECRET AGE
pybspostergcp-tls True pybspostergcp-tls 72s
We can test
$ curl -X POST https://bskyposter.steeped.space/post -H "Content-Type: application/json" -d @payload.json
<!doctype html>
<html lang=en>
<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.</p>
Checking the logs, it’s clear that I need to pay attention to limits on message sizes
[2024-11-26 13:40:59,734] ERROR in app: Exception on /post [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1511, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 919, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 917, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 902, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
File "/app/app.py", line 20, in handle_post
post = client.send_post(builder)
File "/usr/local/lib/python3.9/site-packages/atproto_client/client/client.py", line 173, in send_post
return self.app.bsky.feed.post.create(repo, record)
File "/usr/local/lib/python3.9/site-packages/atproto_client/namespaces/sync_ns.py", line 788, in create
response = self._client.invoke_procedure(
File "/usr/local/lib/python3.9/site-packages/atproto_client/client/base.py", line 115, in invoke_procedure
return self._invoke(InvokeType.PROCEDURE, url=self._build_url(nsid), params=params, data=data, **kwargs)
File "/usr/local/lib/python3.9/site-packages/atproto_client/client/client.py", line 41, in _invoke
return super()._invoke(invoke_type, **kwargs)
File "/usr/local/lib/python3.9/site-packages/atproto_client/client/base.py", line 122, in _invoke
return self.request.post(**kwargs)
File "/usr/local/lib/python3.9/site-packages/atproto_client/request.py", line 165, in post
return _parse_response(self._send_request('POST', *args, **kwargs))
File "/usr/local/lib/python3.9/site-packages/atproto_client/request.py", line 155, in _send_request
_handle_request_errors(e)
File "/usr/local/lib/python3.9/site-packages/atproto_client/request.py", line 54, in _handle_request_errors
raise exception
File "/usr/local/lib/python3.9/site-packages/atproto_client/request.py", line 153, in _send_request
return _handle_response(response)
File "/usr/local/lib/python3.9/site-packages/atproto_client/request.py", line 79, in _handle_response
raise exceptions.BadRequestError(error_response)
atproto_client.exceptions.BadRequestError: Response(success=False, status_code=400, content=XrpcError(error='InvalidRequest', message='Invalid app.bsky.feed.post record: Record/text must not be longer than 300 graphemes'), headers={'x-powered-by': 'Express', 'access-control-allow-origin': '*', 'cache-control': 'private', 'vary': 'Authorization, Accept-Encoding', 'ratelimit-limit': '5000', 'ratelimit-remaining': '4986', 'ratelimit-reset': '1732628466', 'ratelimit-policy': '5000;w=3600', 'content-type': 'application/json; charset=utf-8', 'content-length': '123', 'etag': 'W/"7b-5OYjOucrkx456vG6nIBiH/VZWIA"', 'date': 'Tue, 26 Nov 2024 13:40:59 GMT', 'keep-alive': 'timeout=90', 'strict-transport-security': 'max-age=63072000'})
10.42.0.20 - - [26/Nov/2024 13:40:59] "POST /post HTTP/1.1" 500 -
10.42.0.20 - - [26/Nov/2024 13:41:11] "GET / HTTP/1.1" 404 -
10.42.0.20 - - [26/Nov/2024 13:42:06] "GET / HTTP/1.1" 404 -
10.42.0.20 - - [26/Nov/2024 13:42:07] "GET /favicon.ico HTTP/1.1" 404 -
I reduced the post size
$ cat payload.json
{ "USERNAME": "isaacj.bsky.social", "PASSWORD": "asdfasdfasdfasdf", "TEXT": "I realized I just posted about Part 2 of OneDev but missing part 1. OneDev is a pretty nice Open-Source CICD Suite and in the first post I cover GIT repos, Pipelines, Issue Tracking and more. ", "LINK": "https://go.tpk.pw/6f07k" }
and tried again
$ curl -X POST https://bskyposter.steeped.space/post -H "Content-Type: application/json" -d @payload.json
{"LINK":"https://go.tpk.pw/6f07k","TEXT":"<atproto_client.utils.text_builder.TextBuilder object at 0x7f77b73d21f0>","YOU ARE:":"Isaac Johnson"}
I’ll update my Github workflow to use that new URL and payload
- name: Post to BlueSky
run: |
# clean
rm ./title.txt || true
rm ./bsky.payload.json || true
rm ./target.url || true
rm ./yourls.output.json || true
export LATESTFILE=`ls -l _posts/ | grep ".* 20[0-9][0-9]-[0-9][0-9].*markdown" | tail -n1 | sed 's/.* \(20[0-9][0-9]-[0-9][0-9].*markdown\)/\1/'`
export LATESTURL=`ls -l _posts/ | grep ".* 20[0-9][0-9]-[0-9][0-9].*markdown" | tail -n1 | sed 's/.* \(20[0-9][0-9]-[0-9][0-9]-[0-9][0-9]-\)\(.*\)\.markdown/\2.html/'`
cat _posts/$LATESTFILE | grep "^title: " | head -n 1 | sed 's/^title: "\(.*\)"/\1/' | tr -d '\n'> title.txt
cat _posts/$LATESTFILE | grep "^social: " | head -n 1 | sed 's/^social: "\(.*\)"/\1/' | tr -d '\n'> social.txt
# I suspect I need an extra space
echo " " | tr -d '\n' >> social.txt
cat _posts/$LATESTFILE | grep "^date: " | head -n 1 | sed "s/^date: .\([0-9]*\)-\([0-9]*\)-\([0-9]*\) .*/https:\/\/freshbrewed.science\/\1\/\2\/\3\/$LATESTURL/g" | sed 's/.markdown/.html/' | tr -d '\n' > target.url
curl --data-urlencode "url=`cat target.url`" --data-urlencode "title=`cat title.txt`" --data-urlencode 'format=json' --data-urlencode 'action=shorturl' --data-urlencode 'signature=asdfsadfsadf' "https://go.tpk.pw/yourls-api.php" | tee yourls.output.json
cat yourls.output.json | jq -r '.shorturl' | tr -d '\n' > link.txt
echo "================ bluesky ================"
echo "{\"USERNAME\": \"$BSKYUSER\", \"PASSWORD\": \"$BSKYPASS\", \"TEXT\": \"`cat ./social.txt`\", \"LINK\": \"`cat ./social.txt`\"}" > bsky.payload.json
set -x
cat bsky.payload.json
echo "================ now posting ================"
curl -X POST https://bskyposter.steeped.space/post -H "Content-Type: application/json" -d @bsky.payload.json
env:
BSKYUSER: $
BSKYPASS: $
GHTOKEN: $
As I mentioned at the start of this section, you can find the code including Dockerfile, python and Manifest at https://github.com/idjohnson/pybsposter.
I realized later there (at present) is a 300-character limit on social posts. Most of the time I’m under the limit but sometimes I go over. I added a step to check at 275 and if longer, cut the social at 275 and create an elipse:
char_count=$(wc -c ./social.txt)
cat _posts/$LATESTFILE | grep "^date: " | head -n 1 | sed "s/^date: .\([0-9]*\)-\([0-9]*\)-\([0-9]*\) .*/https:\/\/freshbrewed.science\/\1\/\2\/\3\/$LATESTURL/g" | sed 's/.markdown/.html/' | tr -d '\n' > target.url
curl --data-urlencode "url=`cat target.url`" --data-urlencode "title=`cat title.txt`" --data-urlencode 'format=json' --data-urlencode 'action=shorturl' --data-urlencode 'signature=6b15f178b9' "https://go.tpk.pw/yourls-api.php" | tee yourls.output.json
cat yourls.output.json | jq -r '.shorturl' | tr -d '\n' > link.txt
echo "================ bluesky ================"
if [ "$char_count" -gt 275 ]; then
echo "{\"USERNAME\": \"$BSKYUSER\", \"PASSWORD\": \"$BSKYPASS\", \"TEXT\": \"`head -c 275 ./social.txt`...\", \"LINK\": \"`cat ./social.txt`\"}" > bsky.payload.json
else
echo "{\"USERNAME\": \"$BSKYUSER\", \"PASSWORD\": \"$BSKYPASS\", \"TEXT\": \"`cat ./social.txt`\", \"LINK\": \"`cat ./social.txt`\"}" > bsky.payload.json
fi
One more quick fix and that was to fix the links (used social by mistake) and avoid posting if we are skipping social (like this quick update where I realized i biffed the link):
- name: Post to BlueSky
run: |
# clean
rm ./title.txt || true
rm ./bsky.payload.json || true
rm ./target.url || true
rm ./yourls.output.json || true
export LATESTFILE=`ls -l _posts/ | grep ".* 20[0-9][0-9]-[0-9][0-9].*markdown" | tail -n1 | sed 's/.* \(20[0-9][0-9]-[0-9][0-9].*markdown\)/\1/'`
export LATESTURL=`ls -l _posts/ | grep ".* 20[0-9][0-9]-[0-9][0-9].*markdown" | tail -n1 | sed 's/.* \(20[0-9][0-9]-[0-9][0-9]-[0-9][0-9]-\)\(.*\)\.markdown/\2.html/'`
cat _posts/$LATESTFILE | grep "^title: " | head -n 1 | sed 's/^title: "\(.*\)"/\1/' | tr -d '\n'> title.txt
cat _posts/$LATESTFILE | grep "^social: " | head -n 1 | sed 's/^social: "\(.*\)"/\1/' | tr -d '\n'> social.txt
# I suspect I need an extra space
echo " " | tr -d '\n' >> social.txt
char_count=$(wc -c ./social.txt)
cat _posts/$LATESTFILE | grep "^date: " | head -n 1 | sed "s/^date: .\([0-9]*\)-\([0-9]*\)-\([0-9]*\) .*/https:\/\/freshbrewed.science\/\1\/\2\/\3\/$LATESTURL/g" | sed 's/.markdown/.html/' | tr -d '\n' > target.url
curl --data-urlencode "url=`cat target.url`" --data-urlencode "title=`cat title.txt`" --data-urlencode 'format=json' --data-urlencode 'action=shorturl' --data-urlencode 'signature=6b15f178b9' "https://go.tpk.pw/yourls-api.php" | tee yourls.output.json
cat yourls.output.json | jq -r '.shorturl' | tr -d '\n' > link.txt
echo "================ bluesky ================"
if [ "$char_count" -gt 275 ]; then
echo "{\"USERNAME\": \"$BSKYUSER\", \"PASSWORD\": \"$BSKYPASS\", \"TEXT\": \"`head -c 275 ./social.txt`...\", \"LINK\": \"`cat ./link.txt`\"}" > bsky.payload.json
else
echo "{\"USERNAME\": \"$BSKYUSER\", \"PASSWORD\": \"$BSKYPASS\", \"TEXT\": \"`cat ./social.txt`\", \"LINK\": \"`cat ./link.txt`\"}" > bsky.payload.json
fi
# allow a way to not repost on image updates and hotfixes
log=$(git log -n 1)
set -x
cat bsky.payload.json
if [[ $log != *"SKIPSOCIAL"* ]]; then
echo "================ now posting ================"
curl -X POST https://bskyposter.steeped.space/post -H "Content-Type: application/json" -d @bsky.payload.json
else
echo "CHECK-SKIP: Skip Posting To Social (BSky).. would have posted:"
cat bsky.payload.json
fi
Summary
We can see it works just fine:
I plan to improve it more over time, but at present, this simple containerized service should do the trick. I do plan to move the limit check into the container instead of my Github runner.
As for the container, you can find the code including Dockerfile, python and Manifest at https://github.com/idjohnson/pybsposter and the image on Dockerhub at idjohnson/pybsposter.
