Index call not returning expected documents

Help
1

Collection definition:

collection MonthlyPropertyExpenseSubitems {
  history_days 30

  index byMonthlyPropertyExpense {
    terms [.expense]
  }

  index tempIncorrectEntry {
    terms [.hasIncorrectEntry]
  }

  compute hasIncorrectEntry = (.monthlyPropertyEntryId != null)
}

At least one document where the compute field is true:

MonthlyPropertyExpenseSubitems.byId("398548655440659022")

{
  id: "398548655440659022",
  coll: MonthlyPropertyExpenseSubitems,
  ts: Time("2024-05-22T02:21:06.600Z"),
  hasIncorrectEntry: true,
  expense: MonthlyPropertyExpense("395429960643248718"),
  monthlyPropertyEntryId: MonthlyPropertyEntry("395429960627520078"),
  amount: 100000
}

Actual result of calling the index:

MonthlyPropertyExpenseSubitems.tempIncorrectEntry(true)

{
  data: []
}

Partial definition:

MonthlyPropertyExpenseSubitems.definition

{
  name: "MonthlyPropertyExpenseSubitems",
  coll: Collection,
  ts: Time("2024-05-25T19:55:35.130Z"),
  constraints: [],
  indexes: {
    byMonthlyPropertyExpense: {
      terms: [
        {
          field: ".expense",
          mva: false
        }
      ],
      queryable: true,
      status: "complete"
    },
    tempIncorrectEntry: {
      terms: [
        {
          field: ".hasIncorrectEntry",
          mva: false
        }
      ],
      queryable: true,
      status: "complete"
    }
  },
  computed_fields: {
    hasIncorrectEntry: {
      body: "(.monthlyPropertyEntryId != null)"
    }
  },
...

I’m not sure why the result of the index call is empty. This is calling as admin from the dashboard shell. I know at least one document should match. Calling MonthlyPropertyExpenseSubitems.tempIncorrectEntry(false) seems to match a lot of documents so it seems like all documents are being evaluated as false.

2

Hi @wallslide This looks like it should work. We’ll take a closer look at it.

1 Like
3

So, the fact that there are no results for the index is expected.

The issue that you have is that the computed field that relies on a Document Reference. To check if a reference is null (i.e. != null), Fauna will attempt to read the Reference. Reads are not permitted when indexing, so no index entry is created. The reason why Fauna won’t index on a calculation with side-effect like reads is that creating/deleting/updating the documents pointed to could put the index in a bad state.

We now have added some tasks to update the docs around this.

Working around this depends on what your intent actually is. Are you trying to find documents that have bad references? Or do you just need to check if the field is set?

Checking if a field is set

Instead of != null, you should use isa Null. Doing so will not attempt to read the document reference to determine if it’s valid, but will return true if the field is missing.

  compute hasIncorrectEntry = (.monthlyPropertyEntryId isa Null)

Checking for invalid references

If you need to inspect the reference, then you cannot rely on an index. You might consider paginating through the collection to add a hasIncorrectEntry flag to the document, then you can index on that.

Note that this is similar to how building an index works, including read and write costs. By going through and adding the field yourself, however, you are asserting that your application will not introduce any inconsistencies. Like an unsafe operation in Rust – Fauna won’t do that for you, because there could be unknown side-effects, but you can do it manually if you know what you are doing.

1 Like
4

Out of curiosity, why then did MonthlyPropertyExpenseSubitems.tempIncorrectEntry(false) return documents?

I accidentally changed the code which creates the document to save that field as monthlyPropertyEntryId instead of monthlyPropertyEntry. So now I need to find all the documents with the incorrect field name and change it back to the correct one. So I’m just trying to check the existence of monthlyPropertyEntryId, and not whether the reference is valid or not.

Thanks!

5

When .monthlyPropertyEntryId is not set, running .monthlyPropertyEntryId != null returns false, so those would be correctly indexed.

When .monthlyPropertyEntryId is a document reference, Fauna should fail to index the field as soon as a read is detected, and no index entry should be created.

If you find anything contrary to that, let us know, as that would be a problem.

1 Like
6

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.