Incidents | Turso Incidents reported on status page for Turso https://status.turso.tech/ https://d1lppblt9t2x15.cloudfront.net/logos/ce36349b1a2ff16fe9785f064af061c0.png Incidents | Turso https://status.turso.tech/ en Database and group token generation unavailable https://status.turso.tech/incident/707994 Mon, 18 Aug 2025 15:59:00 -0000 https://status.turso.tech/incident/707994#67490f6d5a0eec57d21114eff014ba80387a140756be66638cef1ac33f387b3b Tokens generation is back to normal Database and group token generation unavailable https://status.turso.tech/incident/707994 Mon, 18 Aug 2025 15:34:00 -0000 https://status.turso.tech/incident/707994#6ed0335f2c2393da087bcd742a25ef63163bb12158d2b673311db064284707c8 New tokens for databases and groups cannot be generated at the moment. Our team is working to resolve the issue. Singapore, Singapore recovered https://status.turso.tech/ Mon, 04 Aug 2025 15:27:05 +0000 https://status.turso.tech/#f834ba785d6bce63841d7981d844c2fc1af8115d0e8522909d908abe8a26e7b8 Singapore, Singapore recovered Global recovered https://status.turso.tech/ Mon, 04 Aug 2025 15:27:05 +0000 https://status.turso.tech/#f834ba785d6bce63841d7981d844c2fc1af8115d0e8522909d908abe8a26e7b8 Global recovered Singapore, Singapore went down https://status.turso.tech/ Mon, 04 Aug 2025 15:22:17 +0000 https://status.turso.tech/#f834ba785d6bce63841d7981d844c2fc1af8115d0e8522909d908abe8a26e7b8 Singapore, Singapore went down Global went down https://status.turso.tech/ Mon, 04 Aug 2025 15:22:17 +0000 https://status.turso.tech/#f834ba785d6bce63841d7981d844c2fc1af8115d0e8522909d908abe8a26e7b8 Global went down Johannesburg, South Africa recovered https://status.turso.tech/ Mon, 04 Aug 2025 15:13:47 +0000 https://status.turso.tech/#2fcf3f6f9dc4579980fbcc72788369aef193d94336536586f365d8e40662c868 Johannesburg, South Africa recovered Global recovered https://status.turso.tech/ Mon, 04 Aug 2025 15:13:47 +0000 https://status.turso.tech/#2fcf3f6f9dc4579980fbcc72788369aef193d94336536586f365d8e40662c868 Global recovered Johannesburg, South Africa went down https://status.turso.tech/ Sat, 02 Aug 2025 08:35:45 +0000 https://status.turso.tech/#2fcf3f6f9dc4579980fbcc72788369aef193d94336536586f365d8e40662c868 Johannesburg, South Africa went down Global went down https://status.turso.tech/ Sat, 02 Aug 2025 08:35:45 +0000 https://status.turso.tech/#2fcf3f6f9dc4579980fbcc72788369aef193d94336536586f365d8e40662c868 Global went down Amsterdam, Netherlands recovered https://status.turso.tech/ Thu, 24 Jul 2025 14:50:47 +0000 https://status.turso.tech/#c6b7ded75e1a1854c398735ab910d55b5a768dd212494f579b9cb33a9116155b Amsterdam, Netherlands recovered Global recovered https://status.turso.tech/ Thu, 24 Jul 2025 14:50:47 +0000 https://status.turso.tech/#c6b7ded75e1a1854c398735ab910d55b5a768dd212494f579b9cb33a9116155b Global recovered Amsterdam, Netherlands went down https://status.turso.tech/ Thu, 24 Jul 2025 14:40:16 +0000 https://status.turso.tech/#c6b7ded75e1a1854c398735ab910d55b5a768dd212494f579b9cb33a9116155b Amsterdam, Netherlands went down Global went down https://status.turso.tech/ Thu, 24 Jul 2025 14:40:16 +0000 https://status.turso.tech/#c6b7ded75e1a1854c398735ab910d55b5a768dd212494f579b9cb33a9116155b Global went down AWS ap-south-1 (Mumbai) recovered https://status.turso.tech/ Thu, 17 Jul 2025 00:43:28 +0000 https://status.turso.tech/#3364ac86f77e00e7a7d7a349737e6f888e6801c2804004bc1e72a38711601991 AWS ap-south-1 (Mumbai) recovered AWS ap-south-1 (Mumbai) went down https://status.turso.tech/ Thu, 17 Jul 2025 00:41:05 +0000 https://status.turso.tech/#3364ac86f77e00e7a7d7a349737e6f888e6801c2804004bc1e72a38711601991 AWS ap-south-1 (Mumbai) went down Sao Paulo, Brazil recovered https://status.turso.tech/ Mon, 14 Jul 2025 20:11:19 +0000 https://status.turso.tech/#7f6d79e76a9b580be335db2fa36a9e3f9b6c480d9a204f486dd25008ef27551c Sao Paulo, Brazil recovered Global recovered https://status.turso.tech/ Mon, 14 Jul 2025 20:11:19 +0000 https://status.turso.tech/#7f6d79e76a9b580be335db2fa36a9e3f9b6c480d9a204f486dd25008ef27551c Global recovered Sao Paulo, Brazil went down https://status.turso.tech/ Mon, 14 Jul 2025 19:56:49 +0000 https://status.turso.tech/#7f6d79e76a9b580be335db2fa36a9e3f9b6c480d9a204f486dd25008ef27551c Sao Paulo, Brazil went down Global went down https://status.turso.tech/ Mon, 14 Jul 2025 19:56:49 +0000 https://status.turso.tech/#7f6d79e76a9b580be335db2fa36a9e3f9b6c480d9a204f486dd25008ef27551c Global went down Sao Paulo, Brazil recovered https://status.turso.tech/ Mon, 14 Jul 2025 19:29:16 +0000 https://status.turso.tech/#5c51ca41cd144918aa1dbc38c9c262dd2233f8e17db397fdf58c70e3efdbf516 Sao Paulo, Brazil recovered Global recovered https://status.turso.tech/ Mon, 14 Jul 2025 19:29:16 +0000 https://status.turso.tech/#5c51ca41cd144918aa1dbc38c9c262dd2233f8e17db397fdf58c70e3efdbf516 Global recovered Sao Paulo, Brazil went down https://status.turso.tech/ Mon, 14 Jul 2025 19:23:35 +0000 https://status.turso.tech/#5c51ca41cd144918aa1dbc38c9c262dd2233f8e17db397fdf58c70e3efdbf516 Sao Paulo, Brazil went down Global went down https://status.turso.tech/ Mon, 14 Jul 2025 19:23:35 +0000 https://status.turso.tech/#5c51ca41cd144918aa1dbc38c9c262dd2233f8e17db397fdf58c70e3efdbf516 Global went down Sao Paulo, Brazil recovered https://status.turso.tech/ Mon, 14 Jul 2025 18:08:07 +0000 https://status.turso.tech/#875dd1dc97f833653a7f8c37b34ac01593e4ebf6670e4a62e4df394f9a823c3c Sao Paulo, Brazil recovered Global recovered https://status.turso.tech/ Mon, 14 Jul 2025 18:08:07 +0000 https://status.turso.tech/#875dd1dc97f833653a7f8c37b34ac01593e4ebf6670e4a62e4df394f9a823c3c Global recovered Sao Paulo, Brazil went down https://status.turso.tech/ Mon, 14 Jul 2025 18:04:45 +0000 https://status.turso.tech/#875dd1dc97f833653a7f8c37b34ac01593e4ebf6670e4a62e4df394f9a823c3c Sao Paulo, Brazil went down Global went down https://status.turso.tech/ Mon, 14 Jul 2025 18:04:45 +0000 https://status.turso.tech/#875dd1dc97f833653a7f8c37b34ac01593e4ebf6670e4a62e4df394f9a823c3c Global went down Availability of databases in aws-us-east-1 region degraded https://status.turso.tech/incident/619383 Mon, 14 Jul 2025 15:42:00 -0000 https://status.turso.tech/incident/619383#c41d95a02653f04019967228ccbb1c48060f4fcc924a5f9af5a614eb2398d3a3 S3 availability in the region returned back to normal and so all DBs are back online. There were partial S3 availability issues in the region due to which some DBs was unavailable. Sao Paulo, Brazil recovered https://status.turso.tech/ Mon, 14 Jul 2025 14:50:33 +0000 https://status.turso.tech/#89ced257bf3b943b316736fdba1ec1fe5d0ada535faf7e024d421459048a6b5d Sao Paulo, Brazil recovered Global recovered https://status.turso.tech/ Mon, 14 Jul 2025 14:50:33 +0000 https://status.turso.tech/#89ced257bf3b943b316736fdba1ec1fe5d0ada535faf7e024d421459048a6b5d Global recovered Sao Paulo, Brazil went down https://status.turso.tech/ Mon, 14 Jul 2025 14:48:20 +0000 https://status.turso.tech/#89ced257bf3b943b316736fdba1ec1fe5d0ada535faf7e024d421459048a6b5d Sao Paulo, Brazil went down Global went down https://status.turso.tech/ Mon, 14 Jul 2025 14:48:20 +0000 https://status.turso.tech/#89ced257bf3b943b316736fdba1ec1fe5d0ada535faf7e024d421459048a6b5d Global went down Network issues in IAD Fly region https://status.turso.tech/incident/618134 Fri, 11 Jul 2025 19:03:00 -0000 https://status.turso.tech/incident/618134#99755258c7335ae34fe7eaa08e6ade4d694b3e642073f94cacdb2650cf8759d3 Fly provider has networking issues in IAD region at the moment. Connectivity to the Turso databases with primaries/replicas in that region at Fly can be unstable. Network issues in IAD Fly region https://status.turso.tech/incident/618134 Fri, 11 Jul 2025 19:03:00 -0000 https://status.turso.tech/incident/618134#99755258c7335ae34fe7eaa08e6ade4d694b3e642073f94cacdb2650cf8759d3 Fly provider has networking issues in IAD region at the moment. Connectivity to the Turso databases with primaries/replicas in that region at Fly can be unstable. Seattle, Washington (US) recovered https://status.turso.tech/ Thu, 10 Jul 2025 11:57:56 +0000 https://status.turso.tech/#86d011fbdbfec4d2c68d9acb68c62c5108664c99ea802f0b92b1b5192cedc844 Seattle, Washington (US) recovered Global recovered https://status.turso.tech/ Thu, 10 Jul 2025 11:57:56 +0000 https://status.turso.tech/#86d011fbdbfec4d2c68d9acb68c62c5108664c99ea802f0b92b1b5192cedc844 Global recovered Seattle, Washington (US) went down https://status.turso.tech/ Thu, 10 Jul 2025 11:39:53 +0000 https://status.turso.tech/#86d011fbdbfec4d2c68d9acb68c62c5108664c99ea802f0b92b1b5192cedc844 Seattle, Washington (US) went down Global went down https://status.turso.tech/ Thu, 10 Jul 2025 11:39:53 +0000 https://status.turso.tech/#86d011fbdbfec4d2c68d9acb68c62c5108664c99ea802f0b92b1b5192cedc844 Global went down Miami, Florida (US) recovered https://status.turso.tech/ Wed, 09 Jul 2025 13:03:02 +0000 https://status.turso.tech/#82ed74fad20cc5667162aedd4ad0ac648080e4175a18291bdc4c0b89647a7abd Miami, Florida (US) recovered Global recovered https://status.turso.tech/ Wed, 09 Jul 2025 13:03:02 +0000 https://status.turso.tech/#82ed74fad20cc5667162aedd4ad0ac648080e4175a18291bdc4c0b89647a7abd Global recovered Miami, Florida (US) went down https://status.turso.tech/ Wed, 09 Jul 2025 13:00:39 +0000 https://status.turso.tech/#82ed74fad20cc5667162aedd4ad0ac648080e4175a18291bdc4c0b89647a7abd Miami, Florida (US) went down Global went down https://status.turso.tech/ Wed, 09 Jul 2025 13:00:39 +0000 https://status.turso.tech/#82ed74fad20cc5667162aedd4ad0ac648080e4175a18291bdc4c0b89647a7abd Global went down Miami, Florida (US) recovered https://status.turso.tech/ Wed, 09 Jul 2025 12:53:26 +0000 https://status.turso.tech/#e64b18a0e5de4c55b5f4b3558d9680e72308e8c0f03fa95f70bcb72f19401737 Miami, Florida (US) recovered Global recovered https://status.turso.tech/ Wed, 09 Jul 2025 12:53:26 +0000 https://status.turso.tech/#e64b18a0e5de4c55b5f4b3558d9680e72308e8c0f03fa95f70bcb72f19401737 Global recovered Miami, Florida (US) went down https://status.turso.tech/ Wed, 09 Jul 2025 12:49:39 +0000 https://status.turso.tech/#e64b18a0e5de4c55b5f4b3558d9680e72308e8c0f03fa95f70bcb72f19401737 Miami, Florida (US) went down Global went down https://status.turso.tech/ Wed, 09 Jul 2025 12:49:39 +0000 https://status.turso.tech/#e64b18a0e5de4c55b5f4b3558d9680e72308e8c0f03fa95f70bcb72f19401737 Global went down Bucharest, Romania recovered https://status.turso.tech/ Fri, 04 Jul 2025 10:53:01 +0000 https://status.turso.tech/#092ef2389b4830e67be6fad3b7686b4439f9cd82b30a7bd9e2982cd0bbbd5a1f Bucharest, Romania recovered Global recovered https://status.turso.tech/ Fri, 04 Jul 2025 10:53:01 +0000 https://status.turso.tech/#092ef2389b4830e67be6fad3b7686b4439f9cd82b30a7bd9e2982cd0bbbd5a1f Global recovered Bucharest, Romania went down https://status.turso.tech/ Fri, 04 Jul 2025 10:49:57 +0000 https://status.turso.tech/#092ef2389b4830e67be6fad3b7686b4439f9cd82b30a7bd9e2982cd0bbbd5a1f Bucharest, Romania went down Global went down https://status.turso.tech/ Fri, 04 Jul 2025 10:49:57 +0000 https://status.turso.tech/#092ef2389b4830e67be6fad3b7686b4439f9cd82b30a7bd9e2982cd0bbbd5a1f Global went down Bucharest, Romania recovered https://status.turso.tech/ Fri, 04 Jul 2025 09:27:36 +0000 https://status.turso.tech/#9856a93e50e9e7f20a5ec985e8df57457e11de7f98b2e20c12b44baf1005274d Bucharest, Romania recovered Global recovered https://status.turso.tech/ Fri, 04 Jul 2025 09:27:36 +0000 https://status.turso.tech/#9856a93e50e9e7f20a5ec985e8df57457e11de7f98b2e20c12b44baf1005274d Global recovered Bucharest, Romania went down https://status.turso.tech/ Fri, 04 Jul 2025 09:23:54 +0000 https://status.turso.tech/#9856a93e50e9e7f20a5ec985e8df57457e11de7f98b2e20c12b44baf1005274d Bucharest, Romania went down Global went down https://status.turso.tech/ Fri, 04 Jul 2025 09:23:54 +0000 https://status.turso.tech/#9856a93e50e9e7f20a5ec985e8df57457e11de7f98b2e20c12b44baf1005274d Global went down AWS ap-south-1 (Mumbai) recovered https://status.turso.tech/ Mon, 26 May 2025 08:39:14 +0000 https://status.turso.tech/#31d5859e0bbb2d348fd0caf2436b5cf152a9a60d6ab6a6ba74e47fee7fd7a01c AWS ap-south-1 (Mumbai) recovered AWS ap-south-1 (Mumbai) went down https://status.turso.tech/ Mon, 26 May 2025 08:31:32 +0000 https://status.turso.tech/#31d5859e0bbb2d348fd0caf2436b5cf152a9a60d6ab6a6ba74e47fee7fd7a01c AWS ap-south-1 (Mumbai) went down Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Outage on Fly.io regions https://status.turso.tech/incident/555843 Sat, 03 May 2025 07:38:00 -0000 https://status.turso.tech/incident/555843#f923cc0e493efac54470152cf78185f7ddd21e4533e242563ce66dca118cc901 * We had an issue with the proxy layer on Fly.io, that should be now resolved. * Users on AWS using legacy URLs (.turso.io) were also affected because those URLs still go through Fly. But if you had switched to the AWS-specific URLs (.region.aws.turso.io) you were not affected. **A long incident!** The incident took long to resolve because with all proxies coming down everywhere, they all tried to come back at the same time, overloading other servers. Fly.io also started rate limiting us because of the excessive errors, contributing to the duration of the incident. **What have we done?** In the short term, we disable all automation to bring proxies online automatically, and started bringing them up manually. We are doing it one by one. Not all regions are back. You can connect to your database in every region, but if you are coming to a region where the proxy is not yet deployed, latencies will be higher. Eventually we will have the proxy back in all regions. **What will we do next** We will work on ways to prevent this in the future by changing the way proxy startup works, so that if there is a global issue again, all proxies coming back online at the same time won't overload any other service. Downtime for databases hosted in Fly regions https://status.turso.tech/incident/555842 Sat, 03 May 2025 06:10:00 -0000 https://status.turso.tech/incident/555842#60b90d2cba8f30676a2ecb07cf0bcd9abb2ce5e816e365c63b93f8a52eda8d61 We have identified the issue and rolled out the fixes. All databases should be functional. Downtime for databases hosted in Fly regions https://status.turso.tech/incident/555842 Sat, 03 May 2025 02:38:00 -0000 https://status.turso.tech/incident/555842#ae9a09545df9f0910491016394ad6e088fe5d03ece00de3e9172e991ded1dffa Databases hosted on Fly are unreachable Writes for DBs at AWS region ap-northeast-1 are partially unavilable https://status.turso.tech/incident/546344 Wed, 16 Apr 2025 17:34:00 -0000 https://status.turso.tech/incident/546344#323da0235d8021623efe6afb5e869c04f84b40767719ccafacd86d22872e133d We are seeing increasing error rates for writes to S3 Express One Zone in ap-northeast-1 region. Issue partially affects writes to the DBs in that region. We are actively investigating the issue with AWS support team. Writes for DBs at AWS region ap-northeast-1 are unavilable https://status.turso.tech/incident/545528 Tue, 15 Apr 2025 08:43:00 -0000 https://status.turso.tech/incident/545528#702cea54096d5dd9ff917938570dd4739120eac5d7a58d97df354c7132fea3fe AWS has connectivity issues at ap-northeast-1 AZ at the moment and this incident affects S3 Express One Zone: https://health.aws.amazon.com/health/status database creation may fail for certain users https://status.turso.tech/incident/539629 Fri, 04 Apr 2025 01:12:00 -0000 https://status.turso.tech/incident/539629#823cc62fdd802320a555c764dc3e5e057544c07d4e69046ae6f80b20d3eb0035 We have resolved the issue and database creation should work normally now Degraded state in the API https://status.turso.tech/incident/528278 Fri, 14 Mar 2025 15:32:00 -0000 https://status.turso.tech/incident/528278#68400fdd2d7b883d3c810e0348238abdfb8a468a99cf0d57794b6aa3d0962f96 We are investigating an issue in the API that may be preventing access to databases created in the last 4 hours. Update: a patch has been applied and the situation is resolved. Network issues in AMS region https://status.turso.tech/incident/519844 Thu, 27 Feb 2025 10:30:00 -0000 https://status.turso.tech/incident/519844#75bc2eeedd6a06872d9a6a5b223c548b1e918826569a555785e307683cd01761 The issue has been fixed by downstream infra provider Network issues in AMS region https://status.turso.tech/incident/519844 Thu, 27 Feb 2025 10:30:00 -0000 https://status.turso.tech/incident/519844#75bc2eeedd6a06872d9a6a5b223c548b1e918826569a555785e307683cd01761 The issue has been fixed by downstream infra provider Network issues in AMS region https://status.turso.tech/incident/519844 Thu, 27 Feb 2025 09:11:00 -0000 https://status.turso.tech/incident/519844#666a21679918b3844add4211197289f11a1f8be9e1d8808ef9f1eda42fa86973 Fly cloud provider experiences networking issues with multiple hosts in AMS. Databases (primaries and replicas) in that region can experience unavailability. Network issues in AMS region https://status.turso.tech/incident/519844 Thu, 27 Feb 2025 09:11:00 -0000 https://status.turso.tech/incident/519844#666a21679918b3844add4211197289f11a1f8be9e1d8808ef9f1eda42fa86973 Fly cloud provider experiences networking issues with multiple hosts in AMS. Databases (primaries and replicas) in that region can experience unavailability. Potential database unavailability https://status.turso.tech/incident/513785 Sun, 16 Feb 2025 10:12:00 -0000 https://status.turso.tech/incident/513785#56a67c172d1ee829bf84a300d28e1e5a91a7cca73e2bfc4e73e46502068ab0d0 Connectivity in IAD were recovered on Fly provider side. DB instances in that region and Turso control plane are fully operational. Potential database unavailability https://status.turso.tech/incident/513785 Sun, 16 Feb 2025 10:12:00 -0000 https://status.turso.tech/incident/513785#56a67c172d1ee829bf84a300d28e1e5a91a7cca73e2bfc4e73e46502068ab0d0 Connectivity in IAD were recovered on Fly provider side. DB instances in that region and Turso control plane are fully operational. Potential database unavailability https://status.turso.tech/incident/513785 Sun, 16 Feb 2025 10:12:00 -0000 https://status.turso.tech/incident/513785#56a67c172d1ee829bf84a300d28e1e5a91a7cca73e2bfc4e73e46502068ab0d0 Connectivity in IAD were recovered on Fly provider side. DB instances in that region and Turso control plane are fully operational. Potential database unavailability https://status.turso.tech/incident/513785 Sun, 16 Feb 2025 10:12:00 -0000 https://status.turso.tech/incident/513785#56a67c172d1ee829bf84a300d28e1e5a91a7cca73e2bfc4e73e46502068ab0d0 Connectivity in IAD were recovered on Fly provider side. DB instances in that region and Turso control plane are fully operational. Database instances in BOG are not available https://status.turso.tech/incident/512318 Thu, 13 Feb 2025 12:08:00 -0000 https://status.turso.tech/incident/512318#58811fb86a927e9e9a12eed4167ac94273319fac23e6cf10b2217fa56d6ea649 This incident has been resolved Database instances in BOG are not available https://status.turso.tech/incident/512318 Thu, 13 Feb 2025 12:08:00 -0000 https://status.turso.tech/incident/512318#58811fb86a927e9e9a12eed4167ac94273319fac23e6cf10b2217fa56d6ea649 This incident has been resolved Increased latency and API errors https://status.turso.tech/incident/467662 Tue, 26 Nov 2024 08:15:00 -0000 https://status.turso.tech/incident/467662#7995e6727c3da2bd37d52505d0273e4d2474ad89518567b1e0d39b350173aefd This incident has been resolved Increased latency and API errors https://status.turso.tech/incident/467662 Tue, 26 Nov 2024 08:15:00 -0000 https://status.turso.tech/incident/467662#7995e6727c3da2bd37d52505d0273e4d2474ad89518567b1e0d39b350173aefd This incident has been resolved Increased latency and API errors https://status.turso.tech/incident/467662 Tue, 26 Nov 2024 04:34:00 -0000 https://status.turso.tech/incident/467662#6dcd62f4bc8e5bcf641c6aeb5acde79a30cd265434cd7993e7a9ad53f1c80c0a Some infrastructure operations might still time out or fail Increased latency and API errors https://status.turso.tech/incident/467662 Tue, 26 Nov 2024 04:34:00 -0000 https://status.turso.tech/incident/467662#6dcd62f4bc8e5bcf641c6aeb5acde79a30cd265434cd7993e7a9ad53f1c80c0a Some infrastructure operations might still time out or fail Increased latency and API errors https://status.turso.tech/incident/467662 Tue, 26 Nov 2024 03:47:00 -0000 https://status.turso.tech/incident/467662#fde92540ff291fd0ee3158492f2d61f424a478ed4c6c6eabf9c4388134f7802f The systems are back to operating normally. We're still monitoring for any instabilities. Increased latency and API errors https://status.turso.tech/incident/467662 Tue, 26 Nov 2024 03:47:00 -0000 https://status.turso.tech/incident/467662#fde92540ff291fd0ee3158492f2d61f424a478ed4c6c6eabf9c4388134f7802f The systems are back to operating normally. We're still monitoring for any instabilities. Increased latency and API errors https://status.turso.tech/incident/467662 Mon, 25 Nov 2024 22:14:00 -0000 https://status.turso.tech/incident/467662#a99d7eaeeb0894b7cec45b31e5eedbc5f1f1059c209a7bcb6639690ef1f66112 Due to an ongoing incident with one of our providers, users may see increased latency and errors when querying databases. Turso API may also be unstable when modifying infrastructure. Increased latency and API errors https://status.turso.tech/incident/467662 Mon, 25 Nov 2024 22:14:00 -0000 https://status.turso.tech/incident/467662#a99d7eaeeb0894b7cec45b31e5eedbc5f1f1059c209a7bcb6639690ef1f66112 Due to an ongoing incident with one of our providers, users may see increased latency and errors when querying databases. Turso API may also be unstable when modifying infrastructure. Emergency network maintenance in ARN https://status.turso.tech/incident/464453 Wed, 20 Nov 2024 05:00:00 +0000 https://status.turso.tech/incident/464453#aa7b96639b336a9b415a79ef092cb575ffe6c26790099fa8de801251e5a89eaf Maintenance completed Emergency network maintenance in ARN https://status.turso.tech/incident/464453 Wed, 20 Nov 2024 05:00:00 +0000 https://status.turso.tech/incident/464453#aa7b96639b336a9b415a79ef092cb575ffe6c26790099fa8de801251e5a89eaf Maintenance completed Emergency network maintenance in ARN https://status.turso.tech/incident/464453 Wed, 20 Nov 2024 02:00:00 -0000 https://status.turso.tech/incident/464453#2cccfcd9981651b8b470b9cbc36ba0fad9b945f51b7ccc3f2ac89d6b0c6adb88 Our cloud provider is performing an emergency switch replacement during this window. An up to one hour network outage is expected during this maintenance window. We've migrated most of the existing databases out of ARN. New databases should be automatically created in nearby regions. Due to the emergency nature of the maintenance we could not migrate all databases in time, so some customers could still be affected. Emergency network maintenance in ARN https://status.turso.tech/incident/464453 Wed, 20 Nov 2024 02:00:00 -0000 https://status.turso.tech/incident/464453#2cccfcd9981651b8b470b9cbc36ba0fad9b945f51b7ccc3f2ac89d6b0c6adb88 Our cloud provider is performing an emergency switch replacement during this window. An up to one hour network outage is expected during this maintenance window. We've migrated most of the existing databases out of ARN. New databases should be automatically created in nearby regions. Due to the emergency nature of the maintenance we could not migrate all databases in time, so some customers could still be affected. Group and DB creation on Fly locations is unavailable, also some dbs are experiencing elevated increases in 502s https://status.turso.tech/incident/448872 Wed, 23 Oct 2024 02:10:00 -0000 https://status.turso.tech/incident/448872#76d0ddb6fc32c2531e4bcc6a3aed1f2ae1663d63fdc2ec256434204090c3256f Issue was resolved by the infra provider Group and DB creation on Fly locations is unavailable, also some dbs are experiencing elevated increases in 502s https://status.turso.tech/incident/448872 Tue, 22 Oct 2024 18:45:00 -0000 https://status.turso.tech/incident/448872#45bb91526c06ff18053f2b0139e66823146d2e16d306ed00b1a6a08a2c95d2a9 Fly.io updated their incident with the information that applications were also impacted. users might see increasing 502 errors when connecting with their databases Group and DB creation on Fly locations is unavailable, also some dbs are experiencing elevated increases in 502s https://status.turso.tech/incident/448872 Tue, 22 Oct 2024 18:10:00 -0000 https://status.turso.tech/incident/448872#ef97b6ee93cafff924edc65b1dbca6c1eba5a23befa2b9f3aa9bdd763facec21 due to an incident in fly.io's machines API, db and group creation is degraded https://status.flyio.net/incidents/j3610m921p24 DEN region unavailable https://status.turso.tech/incident/447392 Sat, 19 Oct 2024 22:19:00 -0000 https://status.turso.tech/incident/447392#5632254fe12c8bfe74b6af2af5f99fe56121e9cd9b378a5878ddb0a4349a9330 The faulty hardware was replaced and the region is back online DEN region unavailable https://status.turso.tech/incident/447392 Sat, 19 Oct 2024 22:19:00 -0000 https://status.turso.tech/incident/447392#5632254fe12c8bfe74b6af2af5f99fe56121e9cd9b378a5878ddb0a4349a9330 The faulty hardware was replaced and the region is back online DEN region unavailable https://status.turso.tech/incident/447392 Sat, 19 Oct 2024 04:28:00 -0000 https://status.turso.tech/incident/447392#cced47d835fa59906a3f946866abdccb979781240c333b6ca9067d1a7f57edac DEN region was unavailable due to a networking failure on a downstream cloud provider DEN region unavailable https://status.turso.tech/incident/447392 Sat, 19 Oct 2024 04:28:00 -0000 https://status.turso.tech/incident/447392#cced47d835fa59906a3f946866abdccb979781240c333b6ca9067d1a7f57edac DEN region was unavailable due to a networking failure on a downstream cloud provider Disruption in arn and ams regions https://status.turso.tech/incident/442972 Fri, 11 Oct 2024 22:41:00 -0000 https://status.turso.tech/incident/442972#e9286595da2f53c3fe3f018d3aa76a160456fbee85c99124116c55476e7d571d The issues have been fixed by our infra provider Disruption in arn and ams regions https://status.turso.tech/incident/442972 Fri, 11 Oct 2024 22:41:00 -0000 https://status.turso.tech/incident/442972#e9286595da2f53c3fe3f018d3aa76a160456fbee85c99124116c55476e7d571d The issues have been fixed by our infra provider Disruption in arn and ams regions https://status.turso.tech/incident/442972 Fri, 11 Oct 2024 22:41:00 -0000 https://status.turso.tech/incident/442972#e9286595da2f53c3fe3f018d3aa76a160456fbee85c99124116c55476e7d571d The issues have been fixed by our infra provider Disruption in arn and ams regions https://status.turso.tech/incident/442972 Fri, 11 Oct 2024 22:41:00 -0000 https://status.turso.tech/incident/442972#e9286595da2f53c3fe3f018d3aa76a160456fbee85c99124116c55476e7d571d The issues have been fixed by our infra provider Disruption in arn and ams regions https://status.turso.tech/incident/442972 Fri, 11 Oct 2024 10:30:00 -0000 https://status.turso.tech/incident/442972#401bedf12c3bc2cb2947ee1e283872da6b9abd0d23cc8aa6b792903732a963ca Our infra provider is having capacity issues in arn and ams regions. Some of the existing databases also might face some connectivity issues. Disruption in arn and ams regions https://status.turso.tech/incident/442972 Fri, 11 Oct 2024 10:30:00 -0000 https://status.turso.tech/incident/442972#401bedf12c3bc2cb2947ee1e283872da6b9abd0d23cc8aa6b792903732a963ca Our infra provider is having capacity issues in arn and ams regions. Some of the existing databases also might face some connectivity issues. Disruption in arn and ams regions https://status.turso.tech/incident/442972 Fri, 11 Oct 2024 10:30:00 -0000 https://status.turso.tech/incident/442972#401bedf12c3bc2cb2947ee1e283872da6b9abd0d23cc8aa6b792903732a963ca Our infra provider is having capacity issues in arn and ams regions. Some of the existing databases also might face some connectivity issues. Disruption in arn and ams regions https://status.turso.tech/incident/442972 Fri, 11 Oct 2024 10:30:00 -0000 https://status.turso.tech/incident/442972#401bedf12c3bc2cb2947ee1e283872da6b9abd0d23cc8aa6b792903732a963ca Our infra provider is having capacity issues in arn and ams regions. Some of the existing databases also might face some connectivity issues. LHR region unavailable https://status.turso.tech/incident/438244 Wed, 02 Oct 2024 17:33:00 -0000 https://status.turso.tech/incident/438244#de50f0c216249cc50917fe219cda08d49fcbc345e275211b9259e6daf4824c40 Maintenance was performed and workloads from LHR came back online LHR region unavailable https://status.turso.tech/incident/438244 Wed, 02 Oct 2024 17:33:00 -0000 https://status.turso.tech/incident/438244#de50f0c216249cc50917fe219cda08d49fcbc345e275211b9259e6daf4824c40 Maintenance was performed and workloads from LHR came back online LHR region unavailable https://status.turso.tech/incident/438244 Wed, 02 Oct 2024 13:10:00 -0000 https://status.turso.tech/incident/438244#601eb6bd9d030f1440764be55c51f43c620769e227df47541dd85e5cc444dd6a The LHR region went through emergency maintenance which resulted in unavailability. In situations like this, we're usually able to migrate workloads to different data centers before the maintenance window. However, due to the short notice, we weren't able to migrate all databases safely before the maintenance. LHR region unavailable https://status.turso.tech/incident/438244 Wed, 02 Oct 2024 13:10:00 -0000 https://status.turso.tech/incident/438244#601eb6bd9d030f1440764be55c51f43c620769e227df47541dd85e5cc444dd6a The LHR region went through emergency maintenance which resulted in unavailability. In situations like this, we're usually able to migrate workloads to different data centers before the maintenance window. However, due to the short notice, we weren't able to migrate all databases safely before the maintenance. SCL region unavailable https://status.turso.tech/incident/434898 Thu, 26 Sep 2024 15:10:00 -0000 https://status.turso.tech/incident/434898#060246fee0c5feef88b7537e79d9cd71903cd9aa547b75fc922b172cf97e3400 The issue has been resolved SCL region unavailable https://status.turso.tech/incident/434898 Thu, 26 Sep 2024 15:10:00 -0000 https://status.turso.tech/incident/434898#060246fee0c5feef88b7537e79d9cd71903cd9aa547b75fc922b172cf97e3400 The issue has been resolved Maintenance on LATAM data centers https://status.turso.tech/incident/434836 Thu, 26 Sep 2024 03:30:00 +0000 https://status.turso.tech/incident/434836#4fee0b6ef108b6455407df03fa8de94c94f4990f81d4a0650ff63220d9cd297a Maintenance completed Maintenance on LATAM data centers https://status.turso.tech/incident/434836 Thu, 26 Sep 2024 03:30:00 +0000 https://status.turso.tech/incident/434836#4fee0b6ef108b6455407df03fa8de94c94f4990f81d4a0650ff63220d9cd297a Maintenance completed Maintenance on LATAM data centers https://status.turso.tech/incident/434836 Thu, 26 Sep 2024 03:30:00 +0000 https://status.turso.tech/incident/434836#4fee0b6ef108b6455407df03fa8de94c94f4990f81d4a0650ff63220d9cd297a Maintenance completed Maintenance on LATAM data centers https://status.turso.tech/incident/434836 Thu, 26 Sep 2024 03:30:00 +0000 https://status.turso.tech/incident/434836#4fee0b6ef108b6455407df03fa8de94c94f4990f81d4a0650ff63220d9cd297a Maintenance completed Maintenance on LATAM data centers https://status.turso.tech/incident/434836 Thu, 26 Sep 2024 03:30:00 +0000 https://status.turso.tech/incident/434836#4fee0b6ef108b6455407df03fa8de94c94f4990f81d4a0650ff63220d9cd297a Maintenance completed Maintenance on LATAM data centers https://status.turso.tech/incident/434836 Thu, 26 Sep 2024 03:30:00 +0000 https://status.turso.tech/incident/434836#4fee0b6ef108b6455407df03fa8de94c94f4990f81d4a0650ff63220d9cd297a Maintenance completed Maintenance on LATAM data centers https://status.turso.tech/incident/434836 Thu, 26 Sep 2024 03:30:00 +0000 https://status.turso.tech/incident/434836#4fee0b6ef108b6455407df03fa8de94c94f4990f81d4a0650ff63220d9cd297a Maintenance completed SCL region unavailable https://status.turso.tech/incident/434898 Thu, 26 Sep 2024 02:39:00 -0000 https://status.turso.tech/incident/434898#de2ff31ab47b5514fb247c236a4283a46ea68b1f0277a40675372a2255a38aa3 During the emergency LATAM maintenance, SCL region lost connectivity. Our cloud provider is working to reestablish it. SCL region unavailable https://status.turso.tech/incident/434898 Thu, 26 Sep 2024 02:39:00 -0000 https://status.turso.tech/incident/434898#de2ff31ab47b5514fb247c236a4283a46ea68b1f0277a40675372a2255a38aa3 During the emergency LATAM maintenance, SCL region lost connectivity. Our cloud provider is working to reestablish it. Maintenance on LATAM data centers https://status.turso.tech/incident/434836 Thu, 26 Sep 2024 01:00:00 -0000 https://status.turso.tech/incident/434836#e90bb99600717e6f80ee107eb0ff8a519729ce744380074a034c124d36386a88 After networking instability earlier today, our infrastructure provider needed to perform emergency maintenance on their Latin America data centers. Network outages below 15 minutes may be expected. Maintenance on LATAM data centers https://status.turso.tech/incident/434836 Thu, 26 Sep 2024 01:00:00 -0000 https://status.turso.tech/incident/434836#e90bb99600717e6f80ee107eb0ff8a519729ce744380074a034c124d36386a88 After networking instability earlier today, our infrastructure provider needed to perform emergency maintenance on their Latin America data centers. Network outages below 15 minutes may be expected. Maintenance on LATAM data centers https://status.turso.tech/incident/434836 Thu, 26 Sep 2024 01:00:00 -0000 https://status.turso.tech/incident/434836#e90bb99600717e6f80ee107eb0ff8a519729ce744380074a034c124d36386a88 After networking instability earlier today, our infrastructure provider needed to perform emergency maintenance on their Latin America data centers. Network outages below 15 minutes may be expected. Maintenance on LATAM data centers https://status.turso.tech/incident/434836 Thu, 26 Sep 2024 01:00:00 -0000 https://status.turso.tech/incident/434836#e90bb99600717e6f80ee107eb0ff8a519729ce744380074a034c124d36386a88 After networking instability earlier today, our infrastructure provider needed to perform emergency maintenance on their Latin America data centers. Network outages below 15 minutes may be expected. Maintenance on LATAM data centers https://status.turso.tech/incident/434836 Thu, 26 Sep 2024 01:00:00 -0000 https://status.turso.tech/incident/434836#e90bb99600717e6f80ee107eb0ff8a519729ce744380074a034c124d36386a88 After networking instability earlier today, our infrastructure provider needed to perform emergency maintenance on their Latin America data centers. Network outages below 15 minutes may be expected. Maintenance on LATAM data centers https://status.turso.tech/incident/434836 Thu, 26 Sep 2024 01:00:00 -0000 https://status.turso.tech/incident/434836#e90bb99600717e6f80ee107eb0ff8a519729ce744380074a034c124d36386a88 After networking instability earlier today, our infrastructure provider needed to perform emergency maintenance on their Latin America data centers. Network outages below 15 minutes may be expected. Maintenance on LATAM data centers https://status.turso.tech/incident/434836 Thu, 26 Sep 2024 01:00:00 -0000 https://status.turso.tech/incident/434836#e90bb99600717e6f80ee107eb0ff8a519729ce744380074a034c124d36386a88 After networking instability earlier today, our infrastructure provider needed to perform emergency maintenance on their Latin America data centers. Network outages below 15 minutes may be expected. Degraded performance in IAD, EWR and BOS regions https://status.turso.tech/incident/431815 Thu, 19 Sep 2024 18:04:00 -0000 https://status.turso.tech/incident/431815#58dc99de589627b4d1c3fa2f80b36485a8f4a336219e597f4dee11195b6361e6 The cloud provider has applied mitigations to the issue in the IAD, EWR, and BOS regions. Customers with databases in these regions should not experience any issues at this time. Degraded performance in IAD, EWR and BOS regions https://status.turso.tech/incident/431815 Thu, 19 Sep 2024 18:04:00 -0000 https://status.turso.tech/incident/431815#58dc99de589627b4d1c3fa2f80b36485a8f4a336219e597f4dee11195b6361e6 The cloud provider has applied mitigations to the issue in the IAD, EWR, and BOS regions. Customers with databases in these regions should not experience any issues at this time. Degraded performance in IAD, EWR and BOS regions https://status.turso.tech/incident/431815 Thu, 19 Sep 2024 18:04:00 -0000 https://status.turso.tech/incident/431815#58dc99de589627b4d1c3fa2f80b36485a8f4a336219e597f4dee11195b6361e6 The cloud provider has applied mitigations to the issue in the IAD, EWR, and BOS regions. Customers with databases in these regions should not experience any issues at this time. Degraded performance in IAD, EWR and BOS regions https://status.turso.tech/incident/431815 Thu, 19 Sep 2024 18:04:00 -0000 https://status.turso.tech/incident/431815#58dc99de589627b4d1c3fa2f80b36485a8f4a336219e597f4dee11195b6361e6 The cloud provider has applied mitigations to the issue in the IAD, EWR, and BOS regions. Customers with databases in these regions should not experience any issues at this time. Degraded performance in IAD, EWR and BOS regions https://status.turso.tech/incident/431815 Thu, 19 Sep 2024 18:04:00 -0000 https://status.turso.tech/incident/431815#58dc99de589627b4d1c3fa2f80b36485a8f4a336219e597f4dee11195b6361e6 The cloud provider has applied mitigations to the issue in the IAD, EWR, and BOS regions. Customers with databases in these regions should not experience any issues at this time. Degraded performance in IAD, EWR and BOS regions https://status.turso.tech/incident/431815 Thu, 19 Sep 2024 18:04:00 -0000 https://status.turso.tech/incident/431815#58dc99de589627b4d1c3fa2f80b36485a8f4a336219e597f4dee11195b6361e6 The cloud provider has applied mitigations to the issue in the IAD, EWR, and BOS regions. Customers with databases in these regions should not experience any issues at this time. Degraded performance in IAD, EWR and BOS regions https://status.turso.tech/incident/431815 Thu, 19 Sep 2024 16:47:00 -0000 https://status.turso.tech/incident/431815#3c8f9453c3cfbc3af18bf1f20b5d313e0b2531e3784887db3c547b36f8749fca Our cloud provider is experiencing an issue in the IAD, EWR, and BOS regions (Ashburn, Virginia; Secaucus, New Jersey; and Boston, Massachusetts) that is affecting databases and degrading performance. This issue primarily impacts customers with databases under high load. Degraded performance in IAD, EWR and BOS regions https://status.turso.tech/incident/431815 Thu, 19 Sep 2024 16:47:00 -0000 https://status.turso.tech/incident/431815#3c8f9453c3cfbc3af18bf1f20b5d313e0b2531e3784887db3c547b36f8749fca Our cloud provider is experiencing an issue in the IAD, EWR, and BOS regions (Ashburn, Virginia; Secaucus, New Jersey; and Boston, Massachusetts) that is affecting databases and degrading performance. This issue primarily impacts customers with databases under high load. Degraded performance in IAD, EWR and BOS regions https://status.turso.tech/incident/431815 Thu, 19 Sep 2024 16:47:00 -0000 https://status.turso.tech/incident/431815#3c8f9453c3cfbc3af18bf1f20b5d313e0b2531e3784887db3c547b36f8749fca Our cloud provider is experiencing an issue in the IAD, EWR, and BOS regions (Ashburn, Virginia; Secaucus, New Jersey; and Boston, Massachusetts) that is affecting databases and degrading performance. This issue primarily impacts customers with databases under high load. Degraded performance in IAD, EWR and BOS regions https://status.turso.tech/incident/431815 Thu, 19 Sep 2024 16:47:00 -0000 https://status.turso.tech/incident/431815#3c8f9453c3cfbc3af18bf1f20b5d313e0b2531e3784887db3c547b36f8749fca Our cloud provider is experiencing an issue in the IAD, EWR, and BOS regions (Ashburn, Virginia; Secaucus, New Jersey; and Boston, Massachusetts) that is affecting databases and degrading performance. This issue primarily impacts customers with databases under high load. Degraded performance in IAD, EWR and BOS regions https://status.turso.tech/incident/431815 Thu, 19 Sep 2024 16:47:00 -0000 https://status.turso.tech/incident/431815#3c8f9453c3cfbc3af18bf1f20b5d313e0b2531e3784887db3c547b36f8749fca Our cloud provider is experiencing an issue in the IAD, EWR, and BOS regions (Ashburn, Virginia; Secaucus, New Jersey; and Boston, Massachusetts) that is affecting databases and degrading performance. This issue primarily impacts customers with databases under high load. Degraded performance in IAD, EWR and BOS regions https://status.turso.tech/incident/431815 Thu, 19 Sep 2024 16:47:00 -0000 https://status.turso.tech/incident/431815#3c8f9453c3cfbc3af18bf1f20b5d313e0b2531e3784887db3c547b36f8749fca Our cloud provider is experiencing an issue in the IAD, EWR, and BOS regions (Ashburn, Virginia; Secaucus, New Jersey; and Boston, Massachusetts) that is affecting databases and degrading performance. This issue primarily impacts customers with databases under high load. Maintenance operations for primaries in Chicago (ORD region) https://status.turso.tech/incident/430485 Tue, 17 Sep 2024 11:51:00 -0000 https://status.turso.tech/incident/430485#3970651e5e21e97ab58ebe24d55289fb0a8d874a1d3bd74634aaec0070160310 Maintenance operations were finished successfully. Maintenance operations for primaries in Chicago (ORD region) https://status.turso.tech/incident/430485 Tue, 17 Sep 2024 11:51:00 -0000 https://status.turso.tech/incident/430485#3970651e5e21e97ab58ebe24d55289fb0a8d874a1d3bd74634aaec0070160310 Maintenance operations were finished successfully. Maintenance operations for primaries in Chicago (ORD region) https://status.turso.tech/incident/430485 Tue, 17 Sep 2024 10:40:00 -0000 https://status.turso.tech/incident/430485#49aad25525e676d3d72f54dac36010e83d10ccc12ad18ef8d601419786400965 We are proactively performing maintenance operations with primaries in the ORD region to mitigate the effects of scheduled network maintenance by our cloud provider in the same region. Primaries in the ORD region may experience a few seconds of unavailability during the maintenance operations. Maintenance operations for primaries in Chicago (ORD region) https://status.turso.tech/incident/430485 Tue, 17 Sep 2024 10:40:00 -0000 https://status.turso.tech/incident/430485#49aad25525e676d3d72f54dac36010e83d10ccc12ad18ef8d601419786400965 We are proactively performing maintenance operations with primaries in the ORD region to mitigate the effects of scheduled network maintenance by our cloud provider in the same region. Primaries in the ORD region may experience a few seconds of unavailability during the maintenance operations. Infrastructure operations dirupted https://status.turso.tech/incident/428811 Fri, 13 Sep 2024 18:12:00 -0000 https://status.turso.tech/incident/428811#d4c4bcb18402f8c87ba3d677f5d1352e591b1997234c71694a7031b324d04075 The issue was resolved Infrastructure operations dirupted https://status.turso.tech/incident/428811 Fri, 13 Sep 2024 17:39:00 -0000 https://status.turso.tech/incident/428811#f6aac560ec3a565864d4af3244497c605fde49166a9e65b8a82b87d72ec4e5d7 Operations that require changes in infrastructure were disrupted due to an incident on a cloud provider. This includes group/locations creation and deletion, unarchiving, and scale to one after a group went idle. Database creation and deletion disrupted https://status.turso.tech/incident/426108 Sun, 08 Sep 2024 20:29:00 -0000 https://status.turso.tech/incident/426108#190b743b83a1408367c4e36402c7ac1b658ceb06d5ac7b669a56fed90cff20bd The issue was resolved API, CLI and usage metrics are Degraded https://status.turso.tech/incident/422676 Sun, 01 Sep 2024 20:11:00 -0000 https://status.turso.tech/incident/422676#adf0f3aa35c090d71ecb3a96be3ed272b4d5fc1e057e665863c8658c549ee471 network issue was solved API, CLI and usage metrics are Degraded https://status.turso.tech/incident/422676 Sun, 01 Sep 2024 19:40:00 -0000 https://status.turso.tech/incident/422676#372b9241dbc5322b33a3cb28331e20ac3ea3ba6531de707fdcfcc9930e47040f usage metrics, CLI and API are degraded due to a network failure in a cloud provider. We are working with them to restablish the system to its normalcy. Server API outage https://status.turso.tech/incident/421210 Thu, 29 Aug 2024 10:32:00 -0000 https://status.turso.tech/incident/421210#7c53041dfd4ee8c8e1df07867d679f1181824d250448be6f22109c80c0ffceda Server API now working normally Server API outage https://status.turso.tech/incident/421210 Thu, 29 Aug 2024 10:29:00 -0000 https://status.turso.tech/incident/421210#9a67649386e7f9082fa775d6603629a3d94a1d893c8ab82a4c2251f6510471a4 We are currently experiencing an outage of API server. Idle groups partially unavailable https://status.turso.tech/incident/418452 Sat, 24 Aug 2024 03:11:00 -0000 https://status.turso.tech/incident/418452#bea0c537f4de380641b5a8944fa58845d974f2ebe7c106f8307850ce62dfde0c Groups of starter plan users that were marked as idle may be unavailable due to an issue with a downstream provider. We're working to mitigate the issue. Infrastructure operations partially disrupted https://status.turso.tech/incident/416755 Wed, 21 Aug 2024 00:08:00 -0000 https://status.turso.tech/incident/416755#c5095cef46db6609a3134b94121894c3fcc0c2910428e634791ca73542a97bb7 Due to an incident in a provider, operations that involve infrastructure changes may be disrupted. Analytics is down https://status.turso.tech/incident/414326 Thu, 15 Aug 2024 08:09:00 -0000 https://status.turso.tech/incident/414326#481fc6a67a153c94eee3a11607110d243943a1de6ae50d144aa73a5484627c3f Patch has been deployed and analytics service is restored. Networking Failure in Some Locations https://status.turso.tech/incident/413422 Tue, 13 Aug 2024 21:10:00 -0000 https://status.turso.tech/incident/413422#cbedfff20e88a7492c6e4ef63712fcde7df2b39c861b72123e3290c2663fea0d We are still working with our cloud provider to pinpoint the cause of the incident, but traffic is back to normal Networking Failure in Some Locations https://status.turso.tech/incident/413422 Tue, 13 Aug 2024 21:10:00 -0000 https://status.turso.tech/incident/413422#cbedfff20e88a7492c6e4ef63712fcde7df2b39c861b72123e3290c2663fea0d We are still working with our cloud provider to pinpoint the cause of the incident, but traffic is back to normal Networking Failure in Some Locations https://status.turso.tech/incident/413422 Tue, 13 Aug 2024 21:10:00 -0000 https://status.turso.tech/incident/413422#cbedfff20e88a7492c6e4ef63712fcde7df2b39c861b72123e3290c2663fea0d We are still working with our cloud provider to pinpoint the cause of the incident, but traffic is back to normal Networking Failure in Some Locations https://status.turso.tech/incident/413422 Tue, 13 Aug 2024 21:10:00 -0000 https://status.turso.tech/incident/413422#cbedfff20e88a7492c6e4ef63712fcde7df2b39c861b72123e3290c2663fea0d We are still working with our cloud provider to pinpoint the cause of the incident, but traffic is back to normal Networking Failure in Some Locations https://status.turso.tech/incident/413422 Tue, 13 Aug 2024 21:10:00 -0000 https://status.turso.tech/incident/413422#cbedfff20e88a7492c6e4ef63712fcde7df2b39c861b72123e3290c2663fea0d We are still working with our cloud provider to pinpoint the cause of the incident, but traffic is back to normal Networking Failure in Some Locations https://status.turso.tech/incident/413422 Tue, 13 Aug 2024 21:10:00 -0000 https://status.turso.tech/incident/413422#cbedfff20e88a7492c6e4ef63712fcde7df2b39c861b72123e3290c2663fea0d We are still working with our cloud provider to pinpoint the cause of the incident, but traffic is back to normal Networking Failure in Some Locations https://status.turso.tech/incident/413422 Tue, 13 Aug 2024 21:00:00 -0000 https://status.turso.tech/incident/413422#cbf48245d3a6804cd2cbc5a3dd3a798d1d4c96ed7b8499eda02a0506002dc33a Several of our location suffered a drastic reduction of traffic during the time of the incident, the cause is attributed to an ongoing incident of one of our cloud providers. Networking Failure in Some Locations https://status.turso.tech/incident/413422 Tue, 13 Aug 2024 21:00:00 -0000 https://status.turso.tech/incident/413422#cbf48245d3a6804cd2cbc5a3dd3a798d1d4c96ed7b8499eda02a0506002dc33a Several of our location suffered a drastic reduction of traffic during the time of the incident, the cause is attributed to an ongoing incident of one of our cloud providers. Networking Failure in Some Locations https://status.turso.tech/incident/413422 Tue, 13 Aug 2024 21:00:00 -0000 https://status.turso.tech/incident/413422#cbf48245d3a6804cd2cbc5a3dd3a798d1d4c96ed7b8499eda02a0506002dc33a Several of our location suffered a drastic reduction of traffic during the time of the incident, the cause is attributed to an ongoing incident of one of our cloud providers. Networking Failure in Some Locations https://status.turso.tech/incident/413422 Tue, 13 Aug 2024 21:00:00 -0000 https://status.turso.tech/incident/413422#cbf48245d3a6804cd2cbc5a3dd3a798d1d4c96ed7b8499eda02a0506002dc33a Several of our location suffered a drastic reduction of traffic during the time of the incident, the cause is attributed to an ongoing incident of one of our cloud providers. Networking Failure in Some Locations https://status.turso.tech/incident/413422 Tue, 13 Aug 2024 21:00:00 -0000 https://status.turso.tech/incident/413422#cbf48245d3a6804cd2cbc5a3dd3a798d1d4c96ed7b8499eda02a0506002dc33a Several of our location suffered a drastic reduction of traffic during the time of the incident, the cause is attributed to an ongoing incident of one of our cloud providers. Networking Failure in Some Locations https://status.turso.tech/incident/413422 Tue, 13 Aug 2024 21:00:00 -0000 https://status.turso.tech/incident/413422#cbf48245d3a6804cd2cbc5a3dd3a798d1d4c96ed7b8499eda02a0506002dc33a Several of our location suffered a drastic reduction of traffic during the time of the incident, the cause is attributed to an ongoing incident of one of our cloud providers. Service Degration https://status.turso.tech/incident/410440 Wed, 07 Aug 2024 19:26:00 -0000 https://status.turso.tech/incident/410440#aacab030afe3df2f7a86d5f8b387312b8a10b17c514f86569f4b25bf3fd9621b Our website, and some databases are experiencing failures due to incidents in 2 cloud providers we use. Service Degration https://status.turso.tech/incident/410440 Wed, 07 Aug 2024 19:26:00 -0000 https://status.turso.tech/incident/410440#aacab030afe3df2f7a86d5f8b387312b8a10b17c514f86569f4b25bf3fd9621b Our website, and some databases are experiencing failures due to incidents in 2 cloud providers we use. Service Degration https://status.turso.tech/incident/410440 Wed, 07 Aug 2024 19:26:00 -0000 https://status.turso.tech/incident/410440#aacab030afe3df2f7a86d5f8b387312b8a10b17c514f86569f4b25bf3fd9621b Our website, and some databases are experiencing failures due to incidents in 2 cloud providers we use. Partial outage for databases with primaries in the AMS region https://status.turso.tech/incident/406921 Wed, 31 Jul 2024 09:47:00 -0000 https://status.turso.tech/incident/406921#82aeec402acbfe57449ceb69f6bec28981e3094426131feb499657d4bdf36021 The partial outage for databases with primaries in the AMS region has been completely resolved. Partial outage for databases with primaries in the AMS region https://status.turso.tech/incident/406921 Tue, 30 Jul 2024 15:00:00 -0000 https://status.turso.tech/incident/406921#ebd702dd7a4857f33576e50513dc4a987b39a9e680fbbe9d26d4b306702129d6 Migration of primaries located in the AMS region was performed to avoid downtime due to a maintenance window of our cloud provider. Due to the misconfiguration of clusters that had at least two locations during maintenance, replicas were not updated, which led to unavailability when requests arrived at replicas. If the request arrived at the primary, no issues were experienced. As soon as we became aware of the issue, a fix was applied, and the clusters returned to normal operation. Disruption on group and replica creation https://status.turso.tech/incident/405441 Mon, 29 Jul 2024 11:15:00 -0000 https://status.turso.tech/incident/405441#c366fa33af713b4e900e4c911443ade0975d4581b5cc618de315602f46d81342 Resolved by upstream infra provider Disruption on group and replica creation https://status.turso.tech/incident/405441 Mon, 29 Jul 2024 09:17:00 -0000 https://status.turso.tech/incident/405441#10b377d2980f627d6605eb9d6ec1f68ce55917271247ccd2722107e9e2853470 All commands that require infrastructure change are disrupted due to an incident on our cloud provider. Existing databases are not affected. Database and Group APIs not working https://status.turso.tech/incident/403175 Wed, 24 Jul 2024 19:49:00 -0000 https://status.turso.tech/incident/403175#4295d24884de17b5ead7a2a1bfe40677ca4ac1ed1dfce677bf924c478412f126 We have solved the incident Database and Group APIs not working https://status.turso.tech/incident/403175 Wed, 24 Jul 2024 19:41:00 -0000 https://status.turso.tech/incident/403175#7a5f48b33f7fd8055c6101fa9b9d71cf9009cc5708569884cbecf2f6cf95fd9b One of our cloud provider is currently having a failure in their APIs, we are working with them to solve the issue Services degraded in Singapore region https://status.turso.tech/incident/385480 Sun, 16 Jun 2024 02:04:00 -0000 https://status.turso.tech/incident/385480#cce26475335ba8e11cc89a5d8084ad462d05e68e3e25c91fd623088ee1288297 Incident was resolved Services degraded in Singapore region https://status.turso.tech/incident/385480 Sun, 16 Jun 2024 02:04:00 -0000 https://status.turso.tech/incident/385480#cce26475335ba8e11cc89a5d8084ad462d05e68e3e25c91fd623088ee1288297 Incident was resolved Infrastructure allocation disrupted at South America https://status.turso.tech/incident/381597 Sun, 09 Jun 2024 00:49:00 -0000 https://status.turso.tech/incident/381597#d07e018649896405fbbb3915f005d48771413a746869942ac843993c50a5a7fa The underlying issue was resolved. We're still monitoring the platform for any further problems. Infrastructure allocation disrupted at South America https://status.turso.tech/incident/381597 Sat, 08 Jun 2024 23:07:00 -0000 https://status.turso.tech/incident/381597#d38052b8268dc53cc4ff5e65c00590f999b469ffc23c26b11dd98949d1fb3758 Operations that require infrastructure allocation at South America may be disrupted or degraded. We're working with our cloud provider to resolve the issue. Dashboard, API and Databases Outage https://status.turso.tech/incident/377586 Fri, 31 May 2024 17:27:00 -0000 https://status.turso.tech/incident/377586#071a9ef6e9f825b0c542164b3fab7fbb0cf4d63dc942a2e1a82f337635e0282b They have rolledback the networking change and our services were back to normal Dashboard, API and Databases Outage https://status.turso.tech/incident/377586 Fri, 31 May 2024 17:15:00 -0000 https://status.turso.tech/incident/377586#9ea1b139bdb4d82c1ce14c1cff146df5496fe88d948a51cb3a5be09aa567b99d Our cloud provider had a network failure that impacted our main API, dashboard and some of our customers databases. Usage metrics are unavailable https://status.turso.tech/incident/376003 Tue, 28 May 2024 19:11:00 -0000 https://status.turso.tech/incident/376003#e3ef87722a3fb92dac965a7e040b797f90584cc89d19401bd343f97702c5cefd A fix has been implemented and the problem was solved Usage metrics are unavailable https://status.turso.tech/incident/376003 Tue, 28 May 2024 18:55:00 -0000 https://status.turso.tech/incident/376003#be48f52b5890f5ca33c3ac147029fbf1efe02543d742285cc2362235fb9803ad We have identified the cause and we are working on a solution Usage metrics are unavailable https://status.turso.tech/incident/376003 Tue, 28 May 2024 18:54:00 -0000 https://status.turso.tech/incident/376003#d12eeb600b905d2bec155204cdbfa48e1a7ad3a6e7a164b09fbd9fff69a6db1c We are currently experiencing a failure in the usage metrics server, and all usage metrics are unavailable Increased errors and latency https://status.turso.tech/incident/361325 Sat, 27 Apr 2024 02:53:00 -0000 https://status.turso.tech/incident/361325#76183fa87f47190af1da2518956eef7a9411a2ed7f91ec908434bd8a6c4bb9b5 The issue has been resolved by the downstream provider Increased errors and latency https://status.turso.tech/incident/361325 Fri, 26 Apr 2024 23:44:00 -0000 https://status.turso.tech/incident/361325#07526af8221d31b33ec55866ff2c3a68a433dddc350303741c469775065676d0 We're seeing increased errors and latency on some parts of the platform. We're investigating the cause. Increased error rate and latency https://status.turso.tech/incident/360285 Wed, 24 Apr 2024 18:40:00 -0000 https://status.turso.tech/incident/360285#ef9a09cc8531e84de5e0b1d3cb4d6f8ade9e4ca2c308d495cbe6a14aed63e961 The issue was resolved by our cloud provider. API partially disrupted https://status.turso.tech/incident/354297 Fri, 12 Apr 2024 01:20:00 -0000 https://status.turso.tech/incident/354297#fcbb1c026bb90dec88dacc7a62c32d99e5ec8167d7d124e39a99b87b96bacb7d The issue has been fixed. API partially disrupted https://status.turso.tech/incident/354297 Fri, 12 Apr 2024 00:23:00 -0000 https://status.turso.tech/incident/354297#e6babf119f0874988fe197a57c15bb5e948b2e19fe43f3ef2700bccbd642a2ed API operations that require the creation and validation of database\group credentials are disrupted. This includes creating new groups since they must be configured with public keys to validate tokens. Infrastructure allocation disrupted https://status.turso.tech/incident/343086 Tue, 19 Mar 2024 16:06:00 -0000 https://status.turso.tech/incident/343086#1ddc3bdbc695b4ff4cdbc4c699da8e508c0e3ad12fc1e20b315395fc506f5441 The cloud provider has fixed the issue. Infrastructure allocation disrupted https://status.turso.tech/incident/343086 Tue, 19 Mar 2024 15:33:00 -0000 https://status.turso.tech/incident/343086#f03925cc2cd49ae321bc582c80992fd4262a6ba4ca5a104ce81ab9f6aa8c5e2f Infrastructure allocation is disrupted due to a failure on a downstream cloud provider. Group creation and the addition of new locations may be affected. Group Creation is failing https://status.turso.tech/incident/341289 Fri, 15 Mar 2024 14:47:00 -0000 https://status.turso.tech/incident/341289#53e96c9db1a5d8233d8200355271b5bc3e15e850b43378700ae96f86d3584c28 Problem was fixed Group Creation is failing https://status.turso.tech/incident/341289 Fri, 15 Mar 2024 14:25:00 -0000 https://status.turso.tech/incident/341289#0dc9decbe0516ef750ee8c3193a25d43b7c75027afe51ea76ea93a544b8fb0f5 Group creation API is being impacted by an issue with our cloud provider. We are in contact with them. New group creation and replication are failing https://status.turso.tech/incident/323953 Thu, 08 Feb 2024 07:20:00 -0000 https://status.turso.tech/incident/323953#3d0cfbd2d13d23f7fd0b21ebce4a8df8115bb1659cf65afc2b45255b46e611ff All systems are operating normally now New group creation and replication are failing https://status.turso.tech/incident/323953 Thu, 08 Feb 2024 05:11:00 -0000 https://status.turso.tech/incident/323953#b1c272e76eb61c141ed18898c65ef82b88af62794be7e6794e86033016ebeeb8 Due to the issues at cloud provider, group creation and replication are failing. Existing database operations are working as usual Database Creation Service is Degraded https://status.turso.tech/incident/323718 Wed, 07 Feb 2024 21:36:00 -0000 https://status.turso.tech/incident/323718#2ec8fce8d94b1469041e0abca9f21496bc7dbf1649b8c0783c421d84ed82c2c9 All systems are operating normally now Database Creation Service is Degraded https://status.turso.tech/incident/323718 Wed, 07 Feb 2024 20:46:00 -0000 https://status.turso.tech/incident/323718#b46919c9dc0a290ed4d6e77e51d5718d9f2cadde419c702be7cf44ed118cb510 We are not being able to create databases that require new instances. This will affect all users that are creating groups or don't have a group yet. New group creation and replication are failing https://status.turso.tech/incident/307873 Wed, 03 Jan 2024 11:30:00 -0000 https://status.turso.tech/incident/307873#ba0086e0c186895829d0f418ed65e2b1979e9566dbabfbc6294e4ceff52d928f fixed by the cloud provider New group creation and replication are failing https://status.turso.tech/incident/307873 Wed, 03 Jan 2024 01:30:00 -0000 https://status.turso.tech/incident/307873#eda94533f22c2655fb4f3f41179cd50b1b7264f28dd88480ef549a98ae2a4a8b Due to the issues at cloud provider, group creation and replication are failing. Existing database operations are working as usual Instance creation disrupted https://status.turso.tech/incident/307646 Tue, 02 Jan 2024 21:00:00 -0000 https://status.turso.tech/incident/307646#81a7369012154cbfd7dd3d76621f4d922c640bd0e1800223c49639a0ee3c1357 fixed by the cloud provider Instance creation disrupted https://status.turso.tech/incident/307646 Tue, 02 Jan 2024 20:21:00 -0000 https://status.turso.tech/incident/307646#a0b98a6ac73c5eb912ffa3a4809d799d322801c5e77ca330d9bdbdc5f7876819 Creating new groups and replicating a group in a new location is disrupted due to an incident on our cloud provider. We're in touch with them. Server API is down https://status.turso.tech/incident/289845 Mon, 20 Nov 2023 15:18:00 -0000 https://status.turso.tech/incident/289845#124b5ec31279b98688e30ebea79d30e02e891f0c84efe26d0926b1a4c06ad278 A fix was deployed and we are working normally Server API is down https://status.turso.tech/incident/289845 Mon, 20 Nov 2023 15:09:00 -0000 https://status.turso.tech/incident/289845#870c88e8b7c8bdce0a884719c7c956a547b0a52f4540f25737ec8fe764f7afdb We are currently experiencing an outage caused by a bug deployed to production. We've already identified the root cause and we are working on a solution. Service disruption caused by a network failure in an upstream provider https://status.turso.tech/incident/287023 Tue, 14 Nov 2023 02:12:00 -0000 https://status.turso.tech/incident/287023#6e393a2e1fd0b7a89d56d02332dc0597b470ec979f6aeedd831f6e0ca411d731 The fix was deployed by our upstream provider and everything is back to normal. Service disruption caused by a network failure in an upstream provider https://status.turso.tech/incident/287023 Tue, 14 Nov 2023 01:47:00 -0000 https://status.turso.tech/incident/287023#7ca4fa0efb9d87e79abb7b442c5b42013e2090e88432753c87ca9b03f17f55d4 A disruption in one of our upstream services caused by networking issues made most of our services experience disruption. Creation and destruction of databases is disrupted https://status.turso.tech/incident/285396 Fri, 10 Nov 2023 18:09:00 -0000 https://status.turso.tech/incident/285396#2a1d56904661300cd8cb62f5b1d17acadcb57f2403cc9b9d8183916c1b6870b1 We've deployed a fix and everything is working normally now. Creation and destruction of databases is disrupted https://status.turso.tech/incident/285396 Fri, 10 Nov 2023 17:40:00 -0000 https://status.turso.tech/incident/285396#20a2549ba01f53bfbce2f162e27b456e640f4a42c25c4c105b8cb8a9f891f504 You are currently unable to create or destroy databases because of suspected underlying infrastructure issue. Creation of new instances disrupted https://status.turso.tech/incident/279682 Mon, 30 Oct 2023 13:40:00 -0000 https://status.turso.tech/incident/279682#f8b5baf49649e45a60fde31f5309ad3680249bd42bf643005a5886eab85d9d94 The issue is resolved. Creation of new instances disrupted https://status.turso.tech/incident/279682 Mon, 30 Oct 2023 08:19:00 -0000 https://status.turso.tech/incident/279682#bf6befdf313904fc657db6e1011094fbd745d83948de682df6850b1c01407031 Creation of new instances is disrupted. Group creation disrupted https://status.turso.tech/incident/277137 Tue, 24 Oct 2023 18:47:00 -0000 https://status.turso.tech/incident/277137#8f82e75f27d65939d7c5aae8c7b77af8a75b515e412b053020b1698c355ab54d The incident was resolved. Group creation disrupted https://status.turso.tech/incident/277137 Tue, 24 Oct 2023 18:26:00 -0000 https://status.turso.tech/incident/277137#751f365015de5b4c413cac4a44c498bf05697b96a8517045a3062c0c9766a7c7 Group creation is failing due to a problem with a downstream infrastructure provider. We're working with them to fix it. Database Creation and Replication are not working https://status.turso.tech/incident/273530 Mon, 16 Oct 2023 17:20:00 -0000 https://status.turso.tech/incident/273530#d9c7df50d2978cc2fca361969b3f79d34c6c6dee45eb2e69e0810eb51377b322 The issue was resolved. Database Creation and Replication are not working https://status.turso.tech/incident/273530 Mon, 16 Oct 2023 17:12:00 -0000 https://status.turso.tech/incident/273530#7586b98695c42789313b2d652c1c607765c3ea253520f975902967f8520af781 We are monitoring the results of the fix that was deployted Database Creation and Replication are not working https://status.turso.tech/incident/273530 Mon, 16 Oct 2023 16:49:00 -0000 https://status.turso.tech/incident/273530#030fb5108813ca6f1d9c3f01a0ac396ac7edecdb283889b98aff45d0f6fb1144 Our cloud provider is working on a fix Database Creation and Replication are not working https://status.turso.tech/incident/273530 Mon, 16 Oct 2023 16:23:00 -0000 https://status.turso.tech/incident/273530#a10ed8b304172511f04d49cc8e74a9bd94b2324c6bfa6a28ebf0ef63ebb34667 Our cloud provider is having issues with provisioning volumes, we are working with them to restore our service back to normal. Databases already created are working normally. Disruption on database management APIs https://status.turso.tech/incident/258902 Wed, 13 Sep 2023 21:00:00 -0000 https://status.turso.tech/incident/258902#9e25ceb924b737cd6e09186ca2dbbf83004798597fefdd5f85977ac980fb24d8 The issue has been resolved. Disruption on database management APIs https://status.turso.tech/incident/258902 Wed, 13 Sep 2023 20:30:00 -0000 https://status.turso.tech/incident/258902#8a909bc087996365479bcd5959e2b525b8f1b2245cdd876b5eb3d6495f8124c8 Any operations that require us to change infrastructure are disrupted due to an issue with our cloud provider. Disruption on database creation and replication https://status.turso.tech/incident/258467 Tue, 12 Sep 2023 20:25:00 -0000 https://status.turso.tech/incident/258467#edaee0d4a9f6c0f42ac723428d5a1834f4434bad6359355492b65afebedaa17b The issue has been resolved. Disruption on database creation and replication https://status.turso.tech/incident/258467 Tue, 12 Sep 2023 18:56:00 -0000 https://status.turso.tech/incident/258467#79e1188fef8acefe9b55544ef247d75d86570577818e954eda62756adf202ebd All commands that require infrastructure change are disrupted due to an incident on our cloud provider. Database creation is slow https://status.turso.tech/incident/254740 Mon, 04 Sep 2023 09:16:00 -0000 https://status.turso.tech/incident/254740#d6f4808970d19f118e5847f3ca15fa1c54faf5ea311267e89033d5a3b7b8dd6f Fly is currently having a major outage on the API layer, which is causing Turso database creation to be slow. Database creation is slow https://status.turso.tech/incident/190551 Wed, 29 Mar 2023 12:08:00 -0000 https://status.turso.tech/incident/190551#41c11804a825fb1d422684e5033251fc405b232240fc911eb545eb2686587dcf Due to infrastructure provider issues, database creation is slower than usual.