Aggregating analytics on documents

Help
4

I might have oversimplified how contention errors can come up.

A read-write query on a serialized index, regardless of which documents it is reading and which ones it is writing, will be Strictly Serialized. In fact, in this case you need that serialization guarantee to always be sure that every request always gets the latest stat record. Since the requests also write to the index, they change which document is the latest. Every request looking for that latest document has to retry until it is finally its turn, but if it retries too many times, then a contention error will happen.

Here is an example pseudo query and what that might look like in practice

Let(
  {
    surveyRef: Ref(Collection('surveys'), '12345678'),
    
    // The query is writing, so the query will enter the transaction pipeline.
    // The new document created is "locked" by this transaction.
    surveyResult: Create(Collection('survey_results'), {
      data: {
        survey: Var('surveyRef'),
        /* ... survey result data */
      },
    }),

    // Index('latest_stats') is added to the transaction log.
    // Any writes to the index from earlier transactions will require this transaction to restart.
    lastSurveyStatSet: Match(Index('latest_stats_by_survey'), Var('surveyRef')),

    // Creating 'survey_stats' Documents means that we are both 
    // reading AND writing to Index('latest_stats_by_survey').
    // Changes to the index here mean changes to the resulting Set of the 
    // previous Match function.
    result: If(
      Exists(Var('lastSurveyStatSet')),
      Create(Collection('survey_stats'), { /* ... aggregate stats */ })
      Create(Collection('survey_stats'), { /* ... create first stats doc */ })
    ),
  },
  Var('result')
)

Example going step by step:

  1. 10 requests to create survey_results come in at roughly the same time.
  2. They all eagerly start running their transaction
  3. Each proceeds with creating a survey_results Doc
  4. Each proceeds to read the latest_stats_by_survey Index
  5. The earliest transaction creates a survey_stats Doc, which also writes to the latest_stats_by_survey Index. The 9 later transactions no longer have the latest survey_stats document and need to start over.
  6. The first transaction completes. 9 requests to create survey_results are pending.
  7. They all eagerly start running their transaction
  8. … and the process repeats until the transactions succeed or retry 5 times.