How to quickly clear child database between tests?

Help
1

I’m using jest to test my database’s roles and permissions. Creating a new child database between each test takes way too long and is expensive.

Is there a way to quickly clear a database between tests without deleting and recreating?

2

If the word quickly is crucial then deleting the test database is the fastest way since else you bump into the cache of collections/functions/index names. I’m surprised that deleting and creating a database is taking a lot of time though since for me that process takes about 1 second or less.

Could you give us a glimpse on how you are doing it? Maybe a script or something that you could give us insight to.

3

Here is an example of our accounts service. The delete for this database takes about 2 seconds which is excellent. However, we have to wait another 14 seconds between each test for creation.

Looking at this now, I guess I don’t need to delete roles and functions right? I can organize my collections, indexes, and tokens together and just recreate those. But what happens when we add more? I could easily see us having hundreds of collections.

// Creating a DB:

$ ts-node-dev -r dotenv/config scripts/create.ts
[INFO] 13:49:21 ts-node-dev ver. 1.1.1 (using ts-node ver. 9.1.1, typescript ver. 4.1.3)
   [ Executed ] 'database - create child database if it doesnt exist'
   [ Executed ] 'database admin key - create child database admin key'
   [ Executed ] 'collections/indexes - oauth_applications'
   [ Executed ] 'collections/indexes - oauth_codes'
   [ Executed ] 'collections/indexes - oauth_sessions'
   [ Executed ] 'collections/indexes - oauth_transactions'
   [ Executed ] 'collections/indexes - accounts'
   [ Executed ] 'functionrole - get_account_function_role'
   [ Executed ] 'functionrole - create_account_function_role'
   [ Executed ] 'functionrole - create_oauth_transaction_function_role'
   [ Executed ] 'functionrole - create_oauth_code_function_role'
   [ Executed ] 'functionrole - verify_oauth_code_function_role'
   [ Executed ] 'functionrole - verify_client_credentials_function_role'
   [ Executed ] 'functionrole - verify_refresh_token_function_role'
   [ Executed ] 'functionrole - create_access_and_refresh_token_function_role'
   [ Executed ] 'function - get_account_function'
   [ Executed ] 'function - create_account_function'
   [ Executed ] 'function - create_oauth_transaction_function'
   [ Executed ] 'function - create_oauth_code_function'
   [ Executed ] 'function - verify_oauth_code_function'
   [ Executed ] 'function - verify_client_credentials_function'
   [ Executed ] 'function - verify_refresh_token_function'
   [ Executed ] 'function - create_access_and_refresh_token_function'
   [ Executed ] 'keyrole - server_key_role'
✨  Done in 14.64s.

// Example test:

import {
  Client,
  CREATE_ACCOUNT_FUNCTION,
  createTestDatabase,
  query,
} from "@booksmart/faunadb";

const { Call, UDF } = query;

describe("accounts-api", () => {
  let client: Client;

  beforeEach(async () => {
    client = await createTestDatabase("accounts-api-test");
  });

  test("can create a new business", async () => {
    const options = {
      givenName: "John",
      familyName: "Smith",
      email: "johnsmith@example.com",
    };

    const response = await client.query(
      Call(UDF(CREATE_ACCOUNT_FUNCTION), options)
    );

    expect(response.data).toMatchObject(options);
  });
});
4

There are multiple things that you can do here.

Group in one transaction
I’m assuming that every ‘Executed’ I see here is a separate transaction. You can combine many creations in one transaction with Do. That’ll make you run into problems once you start creating collections/indexes that depend on each other. In that case you would turn it into a Let(). For example, here is the query generated by a tool I just wrote and am going to release on npm/github shortly. Maybe I should provide a way to use it as a pure library instead of as a commandline tool as well for testing purposes.

image
image911×1162 167 KB

To test the library itself I used its internal functions to set up resources, each test sets up its own database and most tests run in 1-2s.

Create/Update
The alternative is not to delete the database and write a CreateOrUpdate to update everything. You would still want to do that in as few transactions as possible though. For indexes, it won’t work since those have to be dropped. For a test it’s much cleaner to use child databases so I would go with the first approach.

1 Like
5

Congratulations on your new migration tool! I’m excited to check it out.

I’ll rework my setup to use Let. Never thought of that!