• We had an issue with the proxy layer on Fly.io, that should be now resolved.
• Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected.

A long incident!

The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident.

What have we done?
In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions.

What will we do next

We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service.