"Value not found at path" when trying to use Filter

Help
,
1

I’ve been trying to build a product page using data from my Fauna DB. As such I’m wanting to allow for filtering based upon the product’s properties. I’ve been following this Fauna article here: Combination field filtering - Fauna Documentation

I’ve tried to build a simplified version where there are two properties: background-color and texture. I input the below into the shell:

Paginate(
  Filter(
    Documents(Collection("my_collection")),
    Lambda(
      "x",
      Let(
        {
          p: Get(Var("x"))
        },
        And(
          Equals(Select(["data", "texture"], Var("x")), "Solid", null),
          Equals(Select(["data", "background-color"], Var("p")), "Terracotta", null),
        )
      )
    )
  )
)

However, I get the following error:

Error: [
  {
    "position": [
      "paginate",
      "filter",
      "expr",
      "in",
      "and",
      0,
      "equals",
      0,
      "from"
    ],
    "code": "value not found",
    "description": "Value not found at path [data,texture]."
  }
]

p

[]

I’m unsure how to resolve this so that I can return a list of documents based upon multiple filters. I’ve attempted various other approaches found on these forums however they don’t seem to bring my closer to a solution.

2

I think I see two issues here. First, you are placing the default select value null outside of the Select call. And second, you are incorrectly trying to pull data out of a reference (Var("x")) instead of its dereferenced document (Var("p"))

Try this:

Paginate(
  Filter(
    Documents(Collection("my_collection")),
    Lambda(
      "x",
      Let(
        {
          p: Get(Var("x"))
        },
        And(
          Equals(Select(["data", "texture"], Var("p"), null), "Solid"),
          Equals(Select(["data", "background-color"], Var("p"), null), "Terracotta"),
        )
      )
    )
  )
)

This query is very inefficient since it is doing a table scan. Instead, I would create an index where the terms are texture and background. Then you can get the set of all the documents in your collection that have for instance “Solid” and “Terracotta” in a single efficient call.

3 Likes
3

@wallslide’s answer provides the solution you should be looking for. I just want to expand on what he said here:

Agreed, it should be pretty rare to ONLY use Filter. Like Wallslide said, a full table scan for this query is going to be very inefficient.

The docs page you link to tries to help you find the balance between creating an index for everything and using Filter to fill in the gaps.

So you can create a single Index to look up the result in O(1) time.

CreateIndex({
  name: "my_collection_by_texture_background-color",
  terms: [
    { field: ["data", "texture"] },
    { field: ["data", "background-color"] }
  ]
}

// and use it like
Paginate(Match(
  Index("my_collection_by_texture_background"),
  "Solid",
  "Terracotta"
))

Okay, so now you have a query for matching by texture and background. What if you also need to query just by texture? Now what if you also need to query just by background? What if you need to also sort the values?

Indexes provide the most efficient way to read data, so it is tempting to naively create another Index that perfectly matches every possible query pattern. While that’s fine if you only have a few query patterns, your Write cost and latency can suffer if you start adding dozens of indexes. At some point, you will want to find a balance between adding more indexes and falling back on Filter, and the subject docs page tries to help demonstrate one way to do that.

Fauna doesn’t use a query planner, because we have a consumption-based pricing model and the black box of a query-planner would make predicting costs impossible. YOU are the query planner*. So the goal is then to take a set of query patterns, choose which indexes can be used to narrow down the data set most efficiently, and use Filter to complete the query (if you are not lucky enough that your chosen indexes are sufficient by themselves).

Example

In your case, suppose that texture and background-color are very common filter requirements, but not necessarily at the same time. You might want to have indexes for each

CreateIndex({
  name: "my_collection_by_texture",
  terms: [
    { field: ["data", "texture"] },
  ]
}
CreateIndex({
  name: "my_collection_by_background-color",
  terms: [
    { field: ["data", "background-color"] }
  ]
}

Let’s assume texture is typically more specific than background-color: in that case, you might choose to only use the texture index here:

// Map(
    Paginate(
      Filter(
        Match(Index("my_collection_by_texture_background"), "Solid"),
        Lambda(
          "ref",
          Equals("Terracotta", Select(["data", "background-color"], Get(Var("ref"))))
        )
      )
    ),
//  Lambda("ref", Get(Var("ref")))
// )

Other Notes on Filter

Filter inside of Paginate is “lazy”. That is, this query won’t necessarily scan the entire set (index match on texture). The query will iterate through the set, until it can satisfy the page size requested (default = 64). That means two things:

  1. If there is a lot of overlap between the index set and what you are filtering for (i.e. not many documents rejected by the filter), then the read cost approaches that of reading a single index.
  2. If there is little overlap (i.e. most documents are rejected by the filter), then the query will have to read a lot of documents, up to the entire set, in order to fulfill the page size requested or at least as much as it can.
2 Likes
4

Thanks for your detailed response.

In my case, there are 11 different categories to filter by (all up the collection is 3000 documents), which is why I leaned towards the filter option rather than using indices. Would you say the most efficient approach would to be create an index for each filter that could then be leveraged if a user is only filtering by 1 category, and if the user is filtering using one category but wishes to apply more than 1 filter term (eg “blue” and "green) an Union() would be used.

Then use a collection filter if the user is filtering by more than 1 category?