A Local Data Store in Astro

I've been playing around with Astro for a new side project and, so far, I've been having a complete blast. I absolutely love the templating/build engine that Astro provides; it hits a sweet spot between complex functionality, developer experience, and prioritising native web technologies that feels extremely easy to grok. Want to write some HTML? Just write some HTML. Want to conditionally render different HTML based on a given value?ย Use JavaScriptย directly in your HTML and Astro will sort it out on the server. It takes all of the bits that Iย like about JSX and lets me keep using them, whilst fixing/removing all of the annoying quirks (*cough* className *cough*). So if you already know how to write a webpage in HTML and CSS, Astro is very easy to pick up. If you know how to useย JSX, Astro is also easy to pick up. Fantastic!

For fully "static" sites[1] without any user interactivity, this makes Astro an absolute joy; barebones templating with very little additional knowledge needed. For heavily dynamic sites, you can import React, Vue, Svelte, insert-front-end-framework components and use them as you would normally in other, much larger and more heavyweight frameworks like Next or Nuxt. Double fantastic! ๐Ÿป

For my own needs, the site I'm working on is functionally static, but does need to have a small amount of interactivity when it comes to forms and layout. Specifically, I want a user to be able to input their email address and then see what settings they've previously, well, set. I could store those locally in a cookie or via local storage, but I want this to be available across devices.ย I could store it in a database, but I don't want to maintain an entire back end for a few UI elements. Instead, I want to use a JSON file hosted alongside the rest of the code, so that I can query it based on a provided email address and retrieve any relevant information.

But Should You, Though?

Let's get this out of the way right at the top: I am not suggesting that anyone should use this pattern at scale, or (god forbid!) for sensitive data. There are a wealth of reasons why what I'm about to propose should be treated as completely unsafe, impractical, and generally bad practice.

For starters, there's no validation going on here. Anyone who types in a valid email address will get access to that user's settings, including the ability to edit them. Obviously, that is a bad system for just about any site or service. For me, though, I personally know every single user that will visit this website. The site itself is not indexable, so it won't pop up in Google or Bing or DuckDuckGo. And the data being stored isn't sensitive (the most sensitive information is probably the email address, so if that's already compromised then an attacker can't learn much else).

Okay, but what about the editing capabilities? Couldn't one friend decide to mess with another? Yes, that's a perfectly possible outcome here (if any of my friends realise the site's fatal flaw; luckily, I don't hang out with many other developers ๐Ÿ˜‰).ย But this actually pairs nicely with the second major limitation of this system: manual updates.

Yes, you could write a submission script that modifies the JSON file at the heart of this not-quite-a-database whenever a user updates their settings, but I'm not going to do that (yet). Every change comes through me, and I manually update the JSON. That way,ย I can vet the changes and if anything seems odd, query them with the person directly[2].

Is this system foolproof? No. Hence why I don't suggest using it. But as a quick'n'dirty set up that gets you quite a lot of functionality, I think it's a decent foundation that you could build a more robust system on top of.

A Rough Outline

Caveats and warnings aside, what am I actually proposing? In it's simplest form, the pattern looks like this:

  1. A homepage with an email input element and a button to trigger submission;
  2. A submit function that captures the email address and redirects the user to a new page, appending the email address as a URL search parameter;
  3. A page template that uses Astro's server-side JavaScript execution to grab the email address from the URL, perform a lookup on a JSON object, and return any associated data;
  4. And some conditional HTML that either renders a settings page โ€“ prepopulated based on the user's fetched details โ€“ or an email input/button combo, in case someone lands there directly.

In a nutshell, that's everything. You can extend this with error messages or various other nice-to-haves (and I may yet do so), but right now this is my least engineered solution that does what I need; the MVP, if you will.

Why Server Side?

Let's say we don't have a homepage input and instead send our user directly to our /settings page. Here they are greeted with an email input and button, just like before. We could write some client-side JavaScript that captures the data from the input, fetches our JSON blob, performs the lookup, and then dynamically injects the settings form onto the page, no redirects necessary.

This is (to be a little overly simplistic) what frameworks like React do all of the time: render one UI state and then update it based on user input. And Astro can absolutely do this, because vanilla JavaScript can do it. Add a <script> element to the bottom of your Astro file, stick all of the necessary functionality in there to do the JSON lookup, add an event listener to capture the submit event from your input, and you should be golden. Toss on some innerHTML magic and you've got our exact user journey without any pesky page loads.

However, this system breaks entirely if client-side JavaScript is disabled, or fails to run. Doing this on the server neatly sidesteps that entire issue and comes with some additional benefits on top. By leaning into web-native tools like the HTML <form> element, we can leave several of the steps to the browser (such as the redirect; creation of the search parameter; and URL encoding), giving us less code to maintain and a more robust solution.

Better still, it prevents data leakage. Don't forget, anything done on the client can be snooped. So that data fetch for the JSON blob? Sure, it's easy to do, but you just exposed your entire database. Now I've mentioned above that this isn't the most secure system, but if a user can enter any email address (even invalid ones) and get access to every email address the system knows about, as well as all of the user details and settings, with a single (and again, invalid) action, well, that is a security hole too large even for me ๐Ÿ˜…[3]

Setting Up Astro

First, let's create a new Astro project. Open a terminal and enter:

npm create astro@latest

Walk through the steps as they appear in the CLI to install the project (I selected "a few best practices", "yes"ย to install dependencies, and "Strict" for TypeScript), then navigate to your new project folder. Personally, I'd now launch my IDE; if you use VS Code, just type code into the terminal and it will load up the Astro project directly.

We can broadly ignore the various config files, but we do need to make one edit. Open the astro.config.mjs in the root directory and add the following line to the defineConfig function, like so:

export default defineConfig({
    output: "server",
});

This is all that is needed to enable server side functionality ๐Ÿคฏ

Now we just need to focus on the /src folder. You can delete everything in there (or edit it) and instead just create the files/folders listed below. Feel free to keep the other boilerplate files in the project, or delete any you don't want.ย Your /src folder should end up looking like this (plus various settings and config files, node modules, and VS code stuff):

/src
    /components
        EmailInput.astro
    /layouts
        Layout.astro
    /pages
        index.astro
        settings.astro

(I'm going to assume that you have at least a basic understanding of how Astro works; if not, do spend some time looking over the docs and going through the resources linked below ๐Ÿ˜Š)

Our <Layout/> component will contain an extremely basic boilerplate, based on Astro's own default setup. Copy and paste the following into that file:

---
export interface Props {
    title: string;
}

const { title } = Astro.props;
---

<!DOCTYPE html>
<html lang="en">
    <head>
        <meta charset="UTF-8" />
        <meta name="viewport" content="width=device-width" />
        <link
            rel="icon"
            type="image/svg+xml"
            href="data:image/svg+xml,<svg xmlns=%22http://www.w3.org/2000/svg%22 viewBox=%220 0 100 100%22><text y=%22.9em%22 font-size=%2290%22>๐Ÿ‘‹</text></svg>"
        />
        <meta name="generator" content={Astro.generator} />
        <title>{title}</title>
    </head>
    <body>
        <slot />
    </body>
</html>
<style is:global>
    :root {
        --accent-gradient: linear-gradient(45deg, #923fe0, #f739a1, white 60%);
    }
    html {
        font-family: system-ui, sans-serif;
        background-color: #f6f6f6;
    }
    code {
        font-family: Menlo, Monaco, Lucida Console, Liberation Mono,
            DejaVu Sans Mono, Bitstream Vera Sans Mono, Courier New, monospace;
    }
</style>

This gives us some global styles; an emoji favicon; and a configurable page title.

Next, because we want our email capture form to appear on both pages, we can abstract this into an <EmailInput> component. Here's the code for that file:

<form action="/settings" method="get">
    <input type="text" name="email" />
    <button>Edit Settings</button>
</form>

Now that we have the common structures defined, we can create our index.astro file:

---
import Layout from "../layouts/Layout.astro";
import EmailInput from "../components/EmailInput.astro";
---

<Layout title="Homepage">
    <main>
        <h1>Welcome Back!</h1>
        <h2>Change Your Settings?</h2>
        <EmailInput/>
    </main>
</Layout>

And finally, in the settings.astro file, add the following:

---
import Layout from "../layouts/Layout.astro";
import EmailInput from "../components/EmailInput.astro";

const userOK = false;
---

<Layout title="Settings">
    <h1>User Settings</h1>
    {
        userOK ? (
            <>
                <p>Welcome Back {}!</p>
            </>
        ) : (
            <EmailInput />
        )
    }
</Layout>

The above is deliberately boilerplate. Right now, it doesn't really work, and it certainly doesn't do what we want.ย However, we've set up the basic foundations for what we need:

  • A homepage with a form that can capture an email address and submit that data to your settings page;
  • A settings page that can conditionally render the same email capture form, or some user information;
  • And some boilerplate code to contain it all.

You can give it a test now by running npm run dev in your terminal and opening a new browser tab to http://localhost:3000 ๐Ÿ‘

A screenshot of the homepage showing a large heading that reads
Hopefully this is what you see when you launch localhost.

Adding Some Data

The next step is to provide a data file for our user settings to live within, and get it ready for manipulation.

In the /src folder, create a new subfolder called /data. Within that, create a file called settings.json, and paste the following:

{
    "jesse": {
        "name": "Jessica Rabbit",
        "email": "jesse@test.local",
        "powerMode": true
    },
    "james": {
        "name": "James Bond",
        "email": "james@test.local",
        "powerMode": false
    }
}

We now have an extremely basic pseudo-database with two entries: Jesse and James. These each have a set of attributes, comprised of a known email address, their name, and a fictional setting that they can update.

If we return to the settings.astro file, we can now import this dataset by simply adding an import to the top of our file, e.g:

---
import Layout from "../layouts/Layout.astro";
import EmailInput from "../components/EmailInput.astro";
import settings from "../data/settings.json";

const userOK = false;
---

And that's it. Seriously. We've fetched our data and made it ready to manipulate, easy as pie ๐Ÿ˜Š

Oh okay, we're not quite done yet. Next we need to convert the data into a format that will make finding the relevant user a little easier. There are many ways to do this, but the method I prefer is to convert the JSON blob into an array of objects. That's actually why I've structured the JSON the way that I have: by using a named key linked to a child object, we can create an array of just those children very easily.

First, let's create another new folder within /src โ€“ I tend to call it something like /utils or /utility. Inside that new folder, we can create a functions.ts file.


โ„นINFO: This step isn't strictly necessary; you could create everything that we're about to add to this file within the settings.astro page template itself. But I prefer to split stuff like this out, both for reusability and to help keep my code a little easier to read and maintain in the future.


We'll create a new function within the file called jsonToArray, like so:

// Function: Convert JSON object to array
export const jsonToArray = (json: any) => {
    var result: any[] = [];
    var keys = Object.keys(json);
    keys.forEach(function (key) {
        result.push(json[key]);
    });
    return result;
};

What this function does is fairly simple:

  1. You input a JSON object;
  2. Create an empty array;
  3. Generate a separate array of top-level keys (i.e. our names, such as james) from the JSON;
  4. And finally loop through those keys, querying the JSON blob for the value of each one, and saving that value to the empty array.

The output is an array of the child object of each entry in our original JSON dataset. This means you lose the named keys (james/jesse etc.) but as we will be basing all further data manipulation on the email address, that doesn't matter.

If you want to, you can import the jsonToArray function into the settings.astro file and pass it the settings data blob, then log out the response to the console to test that everything is working. We won't actually use it that way, but it's a useful test to run ๐Ÿ˜‰


๐Ÿ’ก TIP: In the above snippet you will notice there are a few type annotations, specifically json: any and result: any[]. Because we've defined the filed as .ts, it will expect valid TypeScript in the functions. If you don't want this, feel free to use a .js file instead. If you do want type safety, the above should work, but it is better to create genuine types for your data.

Personally, I'd create a types.ts file alongside functions.ts in your /utils folder. Within that file, I'd create a User type and a Users type, that look something like this:

export type User = {
    name: string;
    email: string;
    powerMode: boolean;
};

export type Users = {
    [key: string]: User;
};

You can then import these types into your functions file and swap out the dubious any values for json: Users and result: User[] respectively ๐Ÿ˜Š


Matching the Email Address

If you test out the site at this stage, you can see that most of the email form logic is already working. You can enter an email address on either page, press the button, and it will magically just appear in the URL as you are redirected. What we need to do now is retrieve that email address from the URL, so that we can search for it in our data.

Luckily, Astro has a built-in function for doing just that: โœจ Astro.url โœจ

This handy little utility returns a standard JavaScript URI object, which means we can extract the URL parameters with a dash of deconstruction and the native get() function. If you add this below your import statements in the settings.astro file, and then input an email into the form, you should see the address get logged to the terminal:

{...}
import settings from "../data/settings.json";

// Retrieve email from the URL
const { searchParams } = Astro.url;
const email = searchParams.get("email");
console.log(email)
{...}

Ah, but wait, it probably doesn't look quite right[4]. The value logged out will have been URI encoded. This is a good thing, as it makes the value safe to use in our URL parameter and won't cause any odd browser behaviours, but it does mean we'll need to decode it before we can compare it to the email addresses stored in our dataset.

To do that, we'll create another helper function in our functions.ts file, called lookupUser(). Paste the following below the jsonToArray function (if you've set up types, swap out the any values with Users/User ๐Ÿ˜‰):

// Function: Fetch user details from database by email
export const lookupUser = (email: string, json: any) => {
    const array = jsonToArray(json);
    email = decodeURI(email);
    const user = array.filter((user: any) => {
        return user.email === email;
    });
    return user[0];
};

Again, we can walk this through step-by-step:

  1. Convert the provided JSON blob to an array;
  2. Decode the email address from the URL parameter;
  3. Filter the array and only return entries where the email address matches the one from the URL;
  4. As we know that an email address is effectively a unique value, we can return the only entry in the output array (index "0").

With our two helper functions now working together, we can head back to the settings.astro file and integrate them. Because we're doing everything on the server, we can pop this in our Astro header rather than a <script> element, like so:

---
import Layout from "../layouts/Layout.astro";
import EmailInput from "../components/EmailInput.astro";
import settings from "../data/settings.json";
import { lookupUser } from "../utils/functions";

// Retrieve email from the URL
const { searchParams } = Astro.url;
const email = searchParams.get("email");

// Fetch user settings
let user;
if (email) {
    user = lookupUser(email, settings);
}

// Check that user settings exist
const userOK = user?.name;
---

You'll notice that we've slightly amended the userOK value as well. This is now serving its intended purpose: confirming that the lookup function has returned valid data. I'm using user?.name to check that we have a value that I know all users will have (the ? allows the logic to fail gracefully without having to write a bunch of if statements). You could also check the typeof value, or chain several value comparisons, or even check that the user object isn't undefined โ€“ it's up to you.

The previous if statement is doing something similar. I'm using it to ensure that we only perform a lookup if the URL contains an email parameter in the first place, because otherwise that function is guaranteed to fail.


๐Ÿ’ก TIP: You could add some additional logic here to trigger an error message if an invalid email is provided, which would mean the lookup fails. Create another validation check (e.g. const error = email !== '' && !userOK ? true : false;) and then add a conditional render to your HTML template:

{ error && <p>Sorry, the email you've entered is unknown. Please check for typos and try again.</p> }

Our final step is to do something with that retrieved data on the page. Right now, I've left a handy welcome message in place that we can hook up, just to show what's going on. In your HTML, edit this line like so:

<p>Welcome Back {user?.name}!</p>

And that's it ๐ŸŽ‰

We now have a fully working template that fetches data based on the user's input, manipulates and verifies it, and updates the page with the relevant information. We've got failsafes and fallbacks in place to catch user errors, and a process that can be adapted to fit a wide range of scenarios.

Again, I wouldn't recommend using this for anything too risky โ€“ such as a login form โ€“ but if you just need a quick and easy data store that people can access, you're good to go. I may yet decide to add real-time editing capabilities to this setup, so if I do I'll write a follow up post and link it here ๐Ÿ˜‰ In the meantime, I hope this was vaguely useful. Astro continues to impress me with its flexibility and ease of use, and it's incredibly refreshing to look at a solution like this and realise that not only is it fast and pretty easy to set up, but it works without JavaScript, in any browser, extremely robustly. I wouldn't have been able to do that with React at all!

Explore Other Articles

Further Reading & Sources

Conversation

Want to take part?

Comments are powered by Webmentions; if you know what that means, do your thing ๐Ÿ‘

Footnotes

  • <p>A quick (and dirty) way to fetch local data from a JSON file and modify the look of a page in Astro, completely natively.</p>
  • Murray Champernowne.
Article permalink