Bulk Upsert - query help needed :)

[ptpaterson]

Replace and Update

@madsem

Which brings me to the question, what is the difference between Replace & Update ?
Only that Update makes a partial replacement & merge?

You got it! Replace completely overwrites the data field. Update will merge the existing data field with what you provide.

Scaling your queries

Regarding performance, this pattern may run into 2 problems at scale

1. simultaneous Read + Write operations on the same Index/Document should be expected to encounter contention if performed about 5 or more times per second. I’ve walked through an example contention scenario before here in the forums.
2. Even though you set history_days to 0, cleanup of history is performed as a background task. This means you’ll always have some buildup of history that will reach a kind of equilibrium with garbage collection. Assuming an extreme case that avoids contention but still has a lot of updates, you might see increased query time and cost for reading Index history.

@madsem

as we need to update this in realtime while a redirect is happening. Therefore we cannot use techniques as described here: Scaling write throughput | Fauna Documentation

Even if you use the Event Sourcing pattern and create periodic snapshots, you can still read all the latest data in real-time. Basically, read the snapshot and all events since the snapshot and calculate the latest value on the fly.

You of course need serialized guarantees for the new_value = current_value + 1 pattern to work. If you could work out an algorithm that can handle eventual consistency, then you can avoid contention by making your indexes nonserialized. Note that this also, by definition, cannot enforce uniqueness.

At this point, this topic is getting pretty off-topic from the original post. If you have additional questions, I encourage you to create a new topic to drill into scaling your query.