How long does it take for index to be available?

[hrishikesh]

Hello.

I’m trying to use FaunaDB as a backend for comments on my website. I’m storing each page as a collection. I need to dynamically create a collection and its index whenever the first time a page tries to load or post comments. Since FaunaDB will throw an error when a collection or index with a specified name already exists, I have to check for that.

So, I created a function which will first check if a collection with a specific name exists, if it does, I directly call my addComment or getComment function. However, if it doesn’t exist, I create a collection, then create an index and then call the functions I mentioned.

This is how I was using it till now:

From my client-side code, I was calling setupComments function (severless function) which would check the existence and once that’s done, I was calling addComment and getComments (again, serverless functions), according to the context. This was creating 2 problems:

1. Calling one serverless function, waiting for its response, then calling another function was taking time. It was taking just a second, but still.

2. In serveless functions, each invocation is counted. So, I didn’t want to increase costs by calling setupComments function just to check the existence of the required elements (collection and index).

Thus, I created created a module setupComments which handles the setting up part for me and required this module in both, addComment as well as getComment function. So now, I am able to save the invocation of an additional function by bundling it. But this has put me in a different trouble. Seems like there’s a delay between an index being created and it being available to use.

From my console.log() statements, I can see that the collection and index are ready.

However, when my getComment function actually looks for that index immediately after it’s done creating, it fails.

The same thing will now work if I refresh the page.

Here’s my repo if it helps: OpinionJS/src at main · Hrishikesh-K/OpinionJS · GitHub. There you can see the setupComments and the other two functions. From my understanding, I’ve written the code correctly, especially since it works after a page refresh, but still, I won’t mind correcting it if I’ve made a mistake.

[summer]

Hello @hrishikesh, and welcome to the Fauna community!

Index build times depend on the amount of data being indexed, but you can check the index to see if it is active before you try using it. The following query:

Select("active",Get(Index( "dept_by_name")))

Should return either true or false.

You can read more about this here: CreateIndex | Fauna Documentation

[hrishikesh]

Thank you for the reply @summer.

But does it take time to create an empty index? Like, I’m creating an index right after I create a collection, and the collection is empty at that point.

Also, even if I check for the index to exist like you suggested, I don’t think it’s possible for me to use it. Because, I need to perform a query on the index anyhow, I always need the answer to be true. Is it possible for me to wait till the index is done creating? I am using promises, so I assumed that my code will execute only after the process is done, but looks like that’s not happening.

[hrishikesh]

I am re-reading the docs and just found something interesting:

The CreateIndex function adds a new index to the database with the specified parameters. After the transaction containing the CreateIndex is completed, the index is immediately available for reads. (The index may not be used in the transaction it was created, and it may not be created in the same transaction as its source collection(s).)

So the question is, what exactly is a transaction? When I do something like:

createACollection.then(() => {
  createAnIndex
})

is that one transaction or something?

[ewan]

Fauna is a transactional database. Every time you run a query, it is running in a transaction.

When an index is created, you cannot attempt to use it in the same transaction. You have to create documents that the index would cover in one or more subsequent transactions.

Your strategy of creating a collection and then creating an index, before you create any documents within the collection, it the one we normally advise.

With no documents to index, the index should become active immediately. When a collection contains more than 128 documents, indexing is handled as a background task to avoid a blocking operation.

How long it takes to complete an index build depends on how many documents, how big the indexed documents might be, the operations involved in any defined bindings, and overall system load at the time.

If you’re noticing that “empty” indexes are not active for some amount of time, any timing information would be helpful, and an indication of prevalence (does this happen for you for all indexes created, or only some, or only sporadically) would be very helpful.

[hrishikesh]

I just checked, I’m really sorry. While migrating as mentioned in the opening post, I missed making the index creation an async operation. So, it was getting created, but the code after that (one requiring the index) was getting executed and thus the error. All in all, it was an error caused by missing one word.

Apologies for the confusion and thanks a lot for your time. Your response actually made me recheck my code.