How do you design a FaunaDB database? Feedback about designing/debugging/observability of FaunaDB and FQL

Help
1

From Slack discussion Workspace Deleted | Slack

I’m fine with the traditional tables/columns and relationships, but I’m getting stuck hard on anything that’s related to faunadb concepts, such as indexes, roles, membership, FQL. Not having a way to “draw” anything visual is a real limitation and it’s slowing me down a lot.

Hey people, does this help? Modernizing from PostgreSQL to Serverless with Fauna Part 1
Although it was an article focused on postgres, I specifically wrote this to help in such decisions. - Brecht De Rooms

Thanks for the blog article, I’ll read it :slightly_smiling_face:
I’ve already read (mutliple times) the whole FaunaDB documentation.

I feel like what’s missing to FaunaDB is some kind of framework about how to do things the right way. By “the right way”, I mean in a testable, debugable, observable way.

The current documentation explains how to do things, but don’t provide strong foundations about how to handle an actual project. That’s up to the developers to figure that part out, and it’s not straightforward. Real-world examples like Fwitter help, I’ve been reading tons of OSS code about faunadb projects and got inspiration from many.

What I need the most regarding FQL is:

  • Write queries/indexes/roles in a testable/reproductible way
  • Test queries and associated permissions (really bad DX about permissions right now, very hard to pinpoint the actual issue because error message don’t provide any guidance)
  • Understand what permissions a query/index needs to work properly (kinda related to previous point)
  • Quickly test a query using different roles to ensure it works as I expect (or, to pinpoint misconfiguration issues)
  • Having something that’s visual to design things would be very useful, I don’t know what exactly, I just know I get lost with my mental model when designing indexes, roles

A few ideas:

  • Executing FQL from the Web Shell with different roles will help.
  • Having some kind of UI “Role checker” when using FQL in roles would be quite useful too, I spent hours yesterday before figuring out my roles read/update didn’t behave the way I meant it. And it still doesn’t work as I want and I don’t know how to fix it. Testing is too slow. Having some built-in “scenario/presets” for each roles to test them through the Web UI would be quite efficient. Something like a list of persona with different roles/permissions (based on the existing roles in the db) and displaying whether read/write/update/delete works for each role would be a huge gain.
  • GitHub - fauna-labs/fauna-schema-migrate: The Fauna Schema Migrate tool helps you set up Fauna resources as code and perform schema migrations. is meant to help with automation/migration, haven’t used it yet (I have 2 collections and it felt like “too much” at this point, at first glance, but I might reconsider that)
2

This is good feedback, and thanks for adding it to the forum upon my request. I’m super interested in how such a drawing tool could look for Fauna and brainstorm around that, but it also seems a lot of work at this point.

how to test

I am currently writing ‘fauna packages’ which are all heavily tested. How I typically do it is:

  • set up fauna resources in a child database per test suite
  • split up queries in functions in the host languages so I can easily test parts of the query. In case of roles that means you could test predicate lambda or privilege lambdas separately.

We do not have good content explaining this and my aim is to explain how I do it in blogposts based on the content I’m writing currently. Functions like ‘Debug’ in Eigil’s lib could help as well: GitHub - shiftx/faunadb-fql-lib

Sneak preview of how my tests look:

image
image1062×1231 273 KB

And sneak preview of how my code looks, fragmented in small snippets which I can theoretically test separately.

image
image977×1002 112 KB

tools for testing

That said, we really need easier testing within the dashboard, providing a way to run queries with a specific role is planned, although I can’t provide you with ETAs.

Other things that could be useful in the long run is permission logs that can only be viewed by the fauna admin. This is also an idea that came up a few times.

how to model

When it comes to how to model there are already quite some questions from many users, there is currently not a right way since Fauna is quite powerful in the sense that it gives you all advantages of native documents as well as the advantages of a traditional relational database. This means you can model both normalized and denormalized and whether you should pick one over the other depends on convenience and tradeoffs, typically it boils down to whether you optimize reads over writes. I’ve wrote a two part blogs for people coming from another database on this topic: Modernizing from PostgreSQL to Serverless with Fauna Part 1

“the right way”

We are currently working on a set of best practices and/or anti-patterns. It’ll take some time though but we are working on it.

The documentation is, of course, reference documentation. It’s part of my job to help developers set up real-life applications. Therefore I’m super interested in your feedback this is very helpful!

Any feedback welcome on how it could be of more help to you, as you probaly saw, the tool has only been released recently.

1 Like
3

This is quite interesting. I will give this some thought, maybe it could be feasible to develop such a standalone role tester as an experiment and see how helpful it is :thinking:. If it is, it can then become a part of the product.

1 Like
4

Agreed, I’m thinking out loud but maybe it could be an option similar to the current observer parameter (JS driver) which would have access to such advanced debug logs and could print them right in the browser when provided with an admin token (typically, during development).

export function getClient(secret: string) {
  const options: ClientConfig = {
    secret,

    // Custom observer used to log responses for easier debug
    observer: (res, client) => {
      console.debug('FaunaDB response:', res);
    },
  };

  return new faunadb.Client(options);
}
5

I was rather thinking in the direction of what I’ve seen in other applications. A debug log in the dashboard itself which you can filter. Something that returns information on permissions directly in the driver with specific permissions could indeed be useful, especially for testing.

6

Yeah, I also thought first about a debug logs accessible from the Web dashboard, but then I immediately thought it would be even more efficient if those logs could be included directly within the tools used by the developers, which is the browser is many cases. (especially when working with real-time)

7

I’ve been following this thread in Slack, and it’s a breath of fresh air to see admin staff actually respond to user feedback this quickly and helpfully. After playing with Fauna off and on for a month or so, I’m 99% sure we’re going to switch to it for the next version of my business’s website — I love Fauna that much, even as a traditionally frontend developer who’s been thrust into a backend role and who feels consistently out of his depth. Heck, the GraphQL possibilities are what enticed me in the first place, but I’ve settled on an all-FQL approach because of its power.

That said, I wish I didn’t have to tab back and forth between the FQL cheat sheet, API references, the spaceship tutorial and the Fwitter repository just to figure out how to implement a function that feels like it should be obvious. I wish the API references contained examples, whenever possible, of how to apply any given function to a real querying scenario. Like I said, I’ve been using this for a month and a lot of my development process is trial-and-error in the FQL shell until I figure out what works and adapt it back to Node.

Anyway, thanks again for being so responsive and for making such an innovative database solution. Coming from Firestore, Fauna is a godsend!