Index with binding not returning for some documents

Help
1

Hi there

I’ve got an index which looks like this:

CreateIndex({
  "name": "narratives-read-count-by-id",
  "unique": false,
  "serialized": true,
  "source": {
    "collection": Collection("narratives"),
    "fields": {
      "nonNullReadCount": Query(Lambda("doc", Select(["data", "readCount"], Var("doc"), 0)))
    }
  },
  "terms": [{
    "field": ["data", "narrativeId"]
  }],
  "values": [{
    "binding": "nonNullReadCount"
  }, {
    "field": ["ref"]
  }]
})

Essentially, for a document in the narratives collection, it should return either the readCount or 0. The point of this being that I should be able to remove null values or non-existent results. My understanding was that with this query, every document should be returned because readCount will always be set as it defaults to 0.

However, I have two documents, both without a readCount value. One of them is being returned correctly by the index and the other is not. I had initially thought that this was because one document was created before the index was, but when viewing each document’s history, both were created before the index was.

Any idea what might be going wrong here, or if I’ve misunderstood something?

2

Can you confirm that data.narrativeId is not null for the missing document? An entry will not be written to the Index if all of the terms resolve to null, or if all of the values resolve to null.

Thank you for checking on the document history, which should rule out the delay due to caching. (note: stay tuned to the link to see progress on that).

You can also test if something is wrong by creating a new index with the same definition and see if it behaves differently from the existing one.

3

Yea, both have data.narrativeId set. We also have another index like this:

CreateIndex({
  name: 'narratives-by-id',
  serialized: true,
  unique: false,
  source: Collection("narratives"),
  terms: [ { field: [ 'data', 'narrativeId' ] } ]
})

And this does work for both of them, so that side of the data is working fine.

Interesting point about creating the index with the same definition, I’ll give that a go and see if that ends up with the same issue.

As a side note, I also tried updating the document that isn’t showing up because I thought maybe the update would trigger it to reindex or something, but that also didn’t work.

4

Ok, @ptpaterson so I recreated the index with the same definition and changed the name. The new one returns successfully, the old one does not:

Exists(Match(Index("narratives-read-count-by-id"), "narrative-1")) === true
Exists(Match(Index("narratives-read-count-by-id"), "narrative-2")) === false

// new one
Exists(Match(Index("narratives-read-count-by-id-test"), "narrative-1")) === true
Exists(Match(Index("narratives-read-count-by-id-test"), "narrative-2")) === true

So, I’m guessing the immediate fix would be to delete the index and re-create it? Or create a new one and update our application logic to use the new one. However, this does open up a couple of questions:

  • How would we be sure that the new index has successfully indexed all documents ( I guess we could loop through all the docs and ensure it exists )
  • This makes me a little nervous when making releases moving forward that new indexes won’t be created correctly
  • Is there any way to re-index these? Or force an index to update? Maybe delete the document and re-create it?
  • Is there any info we can provide to help track down the source of the issue? Fully understand software has bugs and Fauna has been great, so more than happy to help diagnose the issue.

Cheers!

5

It is possible that your account may have been affected by the recent event (See status.fauna.com) that caused some documents to be omitted from indexes. Your questions are reasonable ones, and we are sensitive to how events like these impact our users.

We are constantly working to add improvements and controls to our service on a weekly basis. We have resolved the underlying cause of the event, so we expect that you shouldn’t see this particular issue again; if you do, please reach out to us at support.fauna.com so that we can help right away.

Addressing your current index issue

To start using the new index you can swap the names, rather than delete and create a new one with the same name. This may allow your application to stay the same and minimize downtime during the transition. You will need to delete the first index, then wait up to about a minute to be able to reuse the name.

// Step 1
Delete(Index("narratives-read-count-by-id"))

// wait about 1 minute

// Step 2
Update(
  Index("narratives-read-count-by-id-test"),
  { name: "narratives-read-count-by-id" }
)

As a reminder for any readers, if you are a Team or Business tier customer, your plan comes with professional support through our helpdesk at support.fauna.com. We do not always monitor the forums or Slack, so filling out a priority ticket is the best and quickest way to alert us to any issues. We will also get paged 24/7 for any Priority 1 production outage that our Team/Business customers report in the helpdesk.

6

Hmm ok, it’s unlikely that it was that event as we created the index around a month ago, but I guess it could have been effected via an update or something. Who knows. Anyway that’s fine, I’ll continue to monitor it and will let you know if anything else comes up.

Thanks for your help!

7

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.