How does Map/Get impact pricing? Does it cost (n+1) reads? How does it compare to putting all values in an index and what is the correct approach?

It’s technically n+1 query. As I understand, it should be good enough for small listings, but for the bigger, we should project the whole model to index itself. Am I correct?

Hi Daniel,

I took the liberty to modify the title of the question slightly to make it easier for other people to find this question. The first thing that we need to know to answer this question (which I assume you already know) is how your query affects pricing.

How query structure impacts pricing

1 Get = 1 read

Literally, using the Get function)

client.query(
  Get(Ref(Collection('spells'), '181388642046968320'))
)

This would return the full document with all the attributes and cost one read operation. Mapping over a set of references and getting each of them would cost N read operations

 Map(
     ['181388642046968320', '181388642046968320 ', '181388642046968320'], 
     Lambda('id',  Get(Ref(Collection('spells'), Var('id')))
 )

In this case, that should be 3 read operations iirc.

1 Index Page = 1 read

Using Match on an Index and Paginating the result will return a page, using the after cursor to get the next result set is another page)

client.query(
  Paginate(Match(Index('spells_by_element'), 'fire'))
)

One page contains multiple elements and what elements it contains depends on the values you specified in the index. An index always returns an array of arrays. If I would have a user and would make an index on it with name and email the index will forever (you can’t modify indexes) return an array of the structure:

[
   [ John, john@some-email-provider.com]
   [ Mary, mary@some-email-provider.com]
]

Index Page + Gets ?

When we start to mix indexes with a Map/Get, we get indeed what you mentioned.

client.query(
  Map(
    Paginate(Match(Index('all_user_references'))),
    Lambda(['userRef'], Get(Var('userRef'))
  )
)

This index is just an index that returns the ref (either specified ref in values or left it empty) of the user and has no terms. So we get an array of refs that we can map over. Mapping over this is essentially a Map/Get over one index Page.

One index page = 1 read (returns N references)
One get per reference = N reads

Back to your original question, is this the right approach?

Indexes VS Map/Get

When reading this, you already get a feeling that you can optimize your reads a lot by using indexes and that is certainly true. Whether you should do that optimization depends on what you value most: price vs flexibility.

Indexes are fixed, if you add a value to your documents, it will not appear in your queries, if you want it to appear you have to update your index and probably update your queries since which values it returns depends on the configuration of the index (each attribute that was added to values).

Map/Get is however more flexible since if you added an attribute to your documents and want to read it as well, you do not need to rebuild the index. Map/Get will just continue working and return the new value automatically.

At a certain point when you want to join data together in one call, you will need Map/Get. The Join function is a bit a confusing name since it’s more like a traverse, it takes values from one index and replaces them with another index.

My personal advice would be to start with Map/Get in development. Once you are certain about your access patterns and the structure of your documents you can look for places to optimise your calls by creating indexes with all the values you need and replace the Map/Get with just index matches.

Much depends on your use-case though, you will notice in the Fwitter app that I chose to use Map/Get almost everywhere due to the flexibility and easier joins except for searches since they are so frequent (every letter typed might cause a search).

5 Likes

I’d also just add a brief addendum: indexes also carry restrictions that don’t apply to documents in terms of their size. You can quite happily store mutimegabyte documents in fauna and retrieve them with no difficulties. If you try to flatten all of the data out into an index then you will run slap bang into size limits on terms and values. Of course in future those restrictions might get lifted, but we aren’t there yet.

Another consideration (although I am sure the cross over point is so far out there this probably isn’t worth considering) is that the more data you cram into your index the more redundant data you are storing.

1 Like

We publish a page in our documentation that describes which kinds of operations result in billable operations: https://docs.fauna.com/fauna/current/concepts/billing

2 Likes