Spent two days automating some of the processes around findstarlink.com, and updating some of the code that had started bit-rotting.
Most of FindStarlink's operations run as individual AWS Lambda functions, that are triggered periodically by CloudWatch Events (and Schedules). But a few processes are still done manually, mainly due to a mix of laziness and also being a bit tricky to automate. I also needed to migrate the existing automations to a newer NodeJS runtime in AWS Lambda, since the current runtime was nearing end-of-life support.
What was automated?
One such process is for making a database entry whenever a new batch of Starlink satellites launch. I used to make the database entry manually (well, JSON entry), and this was becoming a real problem because I've been increasingly erratic and late with updating the DB. Delaying this entry results in users not knowing about new launches for several hours after the launch, and Starlink trains are best seen within 24 hours after launch. I used to perform some manual validations after a launch, but I don't think that's necessary anymore. So removing the manual validation step made it straightforward to automate. I used the Launch Library API to get the latest launch info, and automated the rest of the steps.
Another problematic manual process is replacing the dummy NORAD ID (assigned to a satellite upon launch by Celestrak) with the provisional NORAD ID (assigned a day or two later). Delaying this replacement results in outdated orbit predictions, often leading to increasingly inaccurate predictions for users. Finding the like-for-like replacement isn't very straightforward, because we need to search for a satellite with similar coordinates and orbital path as the one being replaced. But a simple Manhattan-distance check seemed good-enough for the job, and it's been applied to the automation. It also determines the list of satellites that are part of the train, and stores that in the database.
What's left?
I'm still left with two manual processes.
One is replacing the provisional NORAD ID with the official NORAD ID (assigned a week after launch). This is different from what I automated yesterday - that one replaced the dummy ID with the provisional ID, this one replaces the provisional ID with the official one (assigned a week after launch).
This process is a bit trickier to automate, because after a week, the particular satellite being tracked may no longer even be in the "train", i.e. it may have drifted away to a different orbit. This isn't a problem when done manually, because I can see the entire train in my dashboard, and visually pick the correct "leader" of the train. But doing this algorithmically would involve calculating the orbital paths of all the satellites in that "train", and automatically picking a new leader. So this automation isn't very hard to do, but isn't trivial either.
The other manual process is removing the DB entries for older trains that have spread out completely. This is usually necessary 3-4 weeks after a launch. Again, this is similar in complexity to the previous process, because I need to calculate the orbital paths of all the satellites in the train, and check if they're still within a threshold distance and threshold path angle to be considered a train. The user impact of this process being late is low, because the system already warns users automatically about low visibility chances once a train is older than 7 days.
Future ideas
I'd also like to somehow store the main changes to the database as versions. The database is just a JSON file, and I'd like to store the change history for things like insertions, deletions, and NORAD ID changes. Would be ideal if the Lambda function could just commit to the GitHub repo.
Maybe this entire setup could be replaced with a GitHub action that runs periodically, and commits to the repo.