Migrating my code from Gatsby to Astro was a pretty simple affair. Sure, there are a few differences in how both engines handle the build process, or deal with additional functionality like sitemap generation, but overall I was able to muddle through with the help of their respective documentation and a little trail-and-error.
However, when it came to handling URLs and redirects during deployment, it wasn't quite so simple. This is neither the fault of Gatsby nor Astro, but rather a complication of the hosting service that I use for the front end of theAdhocracy: Netlify.
For the most part, Netlify has been a fantastic service with lots of clever functionality that makes hosting a static, HTML-first website a breeze. But every now and then, you run into the spectre of back-end development 👻 ‒ some small piece of server-side functionality that trips you up or isn't entirely obvious. On this project, I hit two of these little niggles:
- Page redirects;
- Trailing slashes on URLs.
After a bit of back-and-forth I managed to get both configured to match how this site has worked for years with the Gatsby codebase, but there were a couple of foot-guns I felt were worth documenting.
Redirecting Pages
Let's start with the (arguably) more important issue. Due to this being the fourth version of theAdhocracy, there's now over a decade of accreted changes to the URL structure and information architecture that the site uses. In particular, the migration from WordPress to a page structure that I had full control over (the v2-to-v3 migration) saw a significant shift away from a date-dominant URL structure, towards a more personalised and meaningful system[1].
As a result, I have several redirect rules baked into the old codebase, ensuring that I'm minimising the amount of link rot I'm personally responsible for. For the most part, these are broad rules that bulk redirect whole patterns (e.g. /[year]/[month]/[day]/[slug] becomes /wrote/[slug]) but there are also a dozen or so more specific redirects, where individual articles have been broken up, deleted, or modified so heavily a new URL made more sense.
Overall, I'm actually pretty proud of how few redirect rules I have (largely thanks to those catch-all redirects, but still), but the unfortunate reality is that they are effectively permanent technical debt. I can't really ever know if I can remove them, because I don't know who or what has linked to those pages elsewhere. So even with a full codebase migration, the redirects have to remain.
Redirecting with Astro
Astro has pretty decent baked-in functionality for handling redirects. Given that part of the point of this migration is to try and do things a bit more "by the book", I wanted to used this native functionality. So that's where I started.
On my old Gatsby codebase, redirects were being handled by a _redirects file placed in the static folder. Gatsby automatically clones any files in that directory into the root of your build output, which I assumed was a Gatsby-specific method (spoiler: this was wrong 😉). Each rule was pretty simple: a singe line that contained the origin URL, destination URL, and an HTTP code (all of which were set to 301). Both origin and destination URLs can have dynamic sections (e.g. /blog/[slug] would match any value for [slug]).
Back in the Astro codebase, I added the redirects argument to my astro.config.mjs file; converted the old rules from that defunct _redirects file into a JSON structure; rewrote the format of dynamic paths to one that Astro expects; and that was that. My pages were being redirected 🎉
Or at least, I thought they were. Locally, everything worked perfectly well. But when I set the changes live on Netlify, static paths worked, but anything with a dynamic element was broken.
Addendum: Since writing this section, I've realised that my Astro config was actually set to use server as the rendering method, likely so that I could use server-side form handling for search pages. I have since changed to the hybrid method, but haven't retested how this change affects redirect handling, so just be aware that this may have been a factor.
Adapting for Netlify
Once I began digging into how Netlify handles redirects, I realised that the old _redirects file in the Gatsby codebase was, in fact, a Netlify specific file. There are a few ways that you can define server-level redirects for Netlify, but this is the lowest tech and simplest way to do so (and, I imagine, the only supported way back when I was setting up the Gatsby site).
Now I'll save you some time and say that the trick to getting Netlify to perform redirects is to simply use that _redirects file. Don't try and use Astro redirects at all. Put it in your public folder within the Astro codebase (this works the same way as the static folder does with Gatsby) and just leave it at that. Don't try to do what I did, which was use both at the same time 😅
Why? Well, there are a couple of reasons, but what it boils down to is how Netlify (or, more specifically, the Netlify Astro adaptor) works. Y'see, Netlify attempts to standardise both sets of redirect rules, hoisting anything out of the Astro config file and combining it with rules found in the _redirects file.
You can dig into this yourself by using the Deploy File Browser (it's at the very bottom of any deploy page on Netlify) to actually take a peek at the output file. If you do that, you won't see a _redirects file at all. Instead, there will be a netlify.toml file in the root of the codebase, which contains each redirect rule, rewritten to work the way Netlify needs them to.
The problem with that is you can no longer control the order of the rules. Netlify will clone the _redirects file first, then hoist any rules out of the Astro config. But redirect rules are applied in a very basic manner. Rules are read top-to-bottom, and the first that matches is used. So if you put your dynamic rules in the _redirects folder, any more specific static rules will never be reached.
For instance, I have a dynamic redirect that takes anything at the path /article/[slug] and shifts it over to /wrote/[slug], but I also have a few old articles that are now reviews or notes, such as /article/gretel-and-the-dark-spoilers which was moved to /review/book/gretel-and-the-dark. Now the dynamic redirect was forcing the page to /wrote/gretel-and-the-dark-spoilers, which doesn't exist and therefore resulted in a 404 😬
So just use the Netlify-specific system, and ignore Astro 😉
Aside: But wait! If Netlify hoists the rules out of the Astro config and rewrites them, why didn't the dynamic paths work? Well, that seems to have something to do with server rendering. Netlify points dynamic rules at a server-side function, rather than at the provided destination URL. That should work fine if you're purely using server-side rendering (I think; I haven't tested this), but I'm using Astro as an SSG, so it was trying to access some kind of edge function which isn't enabled 🤦♀️
What About Local Development?
Yeah... that's a tricky one. Netlify rules obviously only apply on a Netlify server. You can test them locally if you use the Netlify CLI to run your developer environment, which seemed to work for me, with some limitations. But it doesn't seem possible to support a normal localhost set up, unfortunately.
Personally, I don't need local redirects to work, as that doesn't really serve any specific purpose. They exist to handle incoming links from third-party sources, not first-party ones. That said, if anyone knows of a way to get both working, please let me know 🙏
Trailing Slashes
The smaller and less significant issue I had was removing trailing slashes from all URLs. Why? I think they're a bit ugly, and they make pattern matching slightly harder, but the main reason was that I've now not used them for long enough that changing over to having them would likely cause some SEO issues[2]. Once way or another, they had to go. But where were they even coming from?
Astro: trailingSlash
As the old Gatsby site didn't use trailing slashes ‒ and I couldn't find anything specifically overriding that behaviour ‒ I assumed it was an Astro default. I took a look in the Astro docs and pretty quickly found the config attribute trailingSlash, which can be set to never. Perfect!
Locally that seemed to do the trick, but (once again!) pushing it up to Netlify had no impact. After a bit more digging around, I discovered that the trailingSlash attribute only targets the development server. It's useful to have this aligned with your production environment, but it does not control whether trailing slashes are used. Dang 😅
Netlify: Pretty URLs
Netlify has a specific piece of functionality called "Pretty URLs" which seems to control how trailing slashes are handled, but again the name and description are a bit misleading. Whilst disabling Pretty URLs can fix this issue, it's also not advisable. Under the hood, the setting doesn't really care whether you use trailing slashes or not, but it does force them to be consistent. And, crucially, it prevents SEO duplication.
Without it enabled, the following URLs are all considered to be unique pages:
/blog/article-name/blog/article-name//blog/article-name.html/blog/article-name/index.html
With it enabled, they will all redirect to either /blog/article-name or /blog/article-name/, ensuring that you aren't penalised for having lots of "duplicate content" on your site.
So how does it pick which to use? That took a bit more forensic spelunking through various support threads and blog posts, but the answer appears to be that it's determined by your file/folder structure. If you have a named page file, then there's no trailing slash; if you have an unnamed index page file, you get a slash. So:
- If your blog post is located at
/blog/article-name/index.html, then Netlify normalises it to/blog/article-name/, effectively preserving the fact that the page name is derived from a folder; - If your blog post is located at
/blog/article-name.html, then Netlify normalises it to/blog/article-name, hiding the file extension.
And that's the key: the file extension! It's not really controlling trailing slashes or not, but rather how it "prettifies" file extensions and folder names.
Astro: build.format
As a result, the way to control trailing slashes is to control how Astro outputs files and folders through the build process. Enter the build config attribute 👀
Back in your astro.config.mjs file you can define the build.format attribute with either file or directory. If you pick the former, pages will be output as article-name.html; if you pick the latter, they'll be output as /article-name/index.html. In other words, the name of your page is given to either a file or a directory (i.e. folder), hence the values.
Astro will automagically infer which output you want if the build.format attribute is not set, and it does so based on how you've created your page templates. Use a lot of index.astro templates, and it will default to a directory value, and vice versa. Adaptors (like the Netlify adaptor) can also provide strong hints that influence which attribute is used, and I suspect Netlify likely leans in the direction of directory, purely because everything else they do leans that way 😉
Thankfully, the build.format config attribute can override that inferred behaviour, or any behaviour set by an adaptor on your behalf. In my case, I wanted the following:
export default defineConfig({
build: {
format: "file"
},
trailingSlash: "never"
});And yes, you do still want to use trailingSlash. The docs make this clearer, but basically forcing your build.format will also change the output of Astro.url, so you want to enforce the corresponding strict behaviour, which in this case is never.
With all that said, and after a couple hours of debugging, I had redirects and trailing slashes behaving as I wanted! It ended up being a little config needed on both sides for each "fix". But hey, at least everything was ultimately documented somewhere! And now, hopefully, this post will help surface the information with a little less digging 😉
