Sorting Union with Join

I’m trying to refactor some queries that I’ve written to be more efficient.

So I’ve been following this documentation.

One of my new queries looks like this:

Map(
    Paginate(
        Join(
            Union(matches),
            Index(
                "sort_recipes_by_author_asc_updatedAt_desc"
            )
        ),
        options // <= size and the after cursor
    ),
    Lambda((author, updatedAt, ref) => Get(ref))
)
);

matches is being built like this:

const matches = ids.map((id: string) =>
    Match(Index("find_recipes_by_chefId"), id)
);

The sort index looks. like this:

{
  name: "sort_recipes_by_author_asc_updatedAt_desc",
  unique: false,
  serialized: true,
  source: "Recipe",
  terms: [
    {
      field: ["ref"]
    }
  ],
  values: [
    {
      field: ["data", "author"]
    },
    {
      field: ["data", "updatedAt"],
      reverse: true
    },
    {
      field: ["ref"]
    }
  ]
}

What I’m expecting is for the data to be sorted in “groups,” meaning that the resulting data will be sorted by chefName ascending then by updatedAt descending.

What I’m getting is: { before: [ null ], data: [] }

When I use the FaunaDB UI, can get a result from the sort query using a single ref:

image

While troubleshooting this, I created a few other indexes, and in doing so, I serendipitously created an index with a value that does not exist on a Recipe: chefName.

{
  name: "sort_recipes_by_chefName_desc_updatedAt_desc",
  unique: false,
  serialized: true,
  source: "Recipe",
  terms: [
    {
      field: ["ref"]
    }
  ],
  values: [
    {
      field: ["data", "chefName"], <= does not exist in Recipe doc data
    },
    {
      field: ["data", "updatedAt"],
      reverse: true
    },
    {
      field: ["ref"]
    }
  ]
}

Using this as the sorting index, I do get data, but it’s unsorted, as–I believe–chefName is null:

image

Again author is definitely a data value in Recipe, while chefName is not.

So, I’m confused. There’s a 99% chance that I’m doing something wrong here, but I can’t figure it out after hours of battling this.

Any help would be most appreciated.

So, I’ve narrowed this down to the index being used to sort.

If I use this index, it works:

{
  name: "sort_recipes_by_updatedAt_desc",
  unique: false,
  serialized: true,
  source: "Recipe",
  terms: [
    {
      field: ["ref"]
    }
  ],
  values: [
    {
      field: ["data", "updatedAt"],
      reverse: true
    },
    {
      field: ["ref"]
    }
  ]
}

If I use this index, I get no data:

{
  name: "sort_recipes_by_author_asc_updatedAt_desc",
  unique: false,
  serialized: true,
  source: "Recipe",
  terms: [
    {
      field: ["ref"]
    }
  ],
  values: [
    {
      field: ["data", "author"]
    },
    {
      field: ["data", "updatedAt"],
      reverse: true
    },
    {
      field: ["ref"]
    }
  ]
}
Map(
    Paginate(
        Join(
            Union(matches),
            Index("sort_recipes_by_updatedAt_desc") // <= this works
            // Index("sort_recipes_by_author_asc_updatedAt_desc") <= this does not work
        ),
        options
    ),
    // Lambda((author, updatedAt, ref) => Get(ref))
    Lambda((updatedAt, ref) => Get(ref))
)

The only real difference being that sort indexes have different values.

What is your options object? Note that null is a valid value for the cursor. null values are sorted last, so providing after: null will effectively skip all actual values. Make sure if you don’t want to provide a cursor it is undefined and not null.

Reverse sorting with Join

Unfortunately, there is an issue with pagination when using Join on a reverse index. This includes setting reverse: true in the values as well as using the Reverse function.

It looks like your immediate problem is related to the cursor, and not the pagination issue, but this is something you will run into with sort_recipes_by_updatedAt_desc at some point.

You can check out a discussion of the issue and a workaround here:

Hi @ptpaterson, thanks for taking the time to reply.

The options object looks like this:

{
    size: size,
    after: faunaQueries.toExpr(
        cursor as string | undefined
    ),
}

The size variable is set to 10, but I’ve tried bumpin that way over the number of possible recipes (e.g. 1000) and still no data, so I don’t think that’s an issue here.

Thanks for telling me about this issue with reverse: true and Join.. As such, I’ve moved away from Join, as I want to sort using descending order in many cases. Maybe that should be flagged in the docs, until that issue is sorted . If it is, I didn’t see it.

Anyhow, with Join no longer an option, I created this index:

{
  name:
    "find_recipes_by_chefId_isPublished_author_asc_updatedAt_desc_isPublished_desc",
  unique: false,
  serialized: true,
  source: "Recipe",
  terms: [
    {
      field: ["data", "chefId"]
    },
    {
      field: ["data", "isPublished"]
    }
  ],
  values: [
    {
      field: ["data", "author"]
    },
    {
      field: ["data", "updatedAt"],
      reverse: true
    },
    {
      field: ["data", "isPublished"],
      reverse: true
    },
    {
      field: ["ref"]
    }
  ]
}

The query has become (removing Join):

Map(
    Paginate(Union(matches), options),
    Lambda((author, updatedAt, isPublished, ref) =>
        Get(ref)
    )
)

With matches using the new index like:

const matches = ids.map((id: string) =>
    Match(
        Index(
            "find_recipes_by_chefId_isPublished_author_asc_updatedAt_desc_isPublished_desc"
        ),
        [id, id === ADMIN_ID ? true : false]
    )
);

FYI: the [id, id === ADMIN_ID ? true : false] bit is there to only show published recipes when the id is that of the ADMIN’s and all other user’s recipes are set to isPublished: false at this time. It works, so that’s not an issue, I believe.

Matches look like this when logged to the console (stringified), so I know they are there:

[{"match":{"index":"find_recipes_by_chefId_isPublished_author_asc_updatedAt_desc_isPublished_desc"},"terms":["4d9fd59c-6ce6-xxxx-adb2-49e394ead573",false]},
{"match":{"index":"find_recipes_by_chefId_isPublished_author_asc_updatedAt_desc_isPublished_desc"},"terms":["f6b508e3-a58a-xxxx-9a3c-7397df67e3a5",false]},
{"match":{"index":"find_recipes_by_chefId_isPublished_author_asc_updatedAt_desc_isPublished_desc"},"terms":["552d4c33-7e5a-xxxx-938f-ef550dd6c271",false]}]

The ids are correct and false is correct in all cases.

The query works in the FaunaDB UI for single queries:

Troubleshooting further, I’ve used different queries in matches, adjusting the Lambda in the query as necessary.

Sorting on author only returns no data. Not an error, just no data:

{
  name: "find_recipes_by_chefId_isPublished_author_asc",
  unique: false,
  serialized: true,
  source: "Recipe",
  terms: [
    {
      field: ["data", "chefId"]
    },
    {
      field: ["data", "isPublished"]
    }
  ],
  values: [
    {
      field: ["data", "author"]
    },
    {
      field: ["ref"]
    }
  ]
}

Sorting on updatedAt only returns data as expected, but I’m only sorting on one of the fields that I want to:

{
  name: "find_recipes_by_chefId_isPublished_updatedAt_desc",
  unique: false,
  serialized: true,
  source: "Recipe",
  terms: [
    {
      field: ["data", "chefId"]
    },
    {
      field: ["data", "isPublished"]
    }
  ],
  values: [
    {
      field: ["data", "updatedAt"],
      reverse: true
    },
    {
      field: ["ref"]
    }
  ]
}

It’s weird, but to me, using author as a value in the index seems to be the culprit here.

And, sadly, that’s one of the things I really want to sort on.

Any ideas toward solving this mystery would be greatly appreciated.

Hi @ptpaterson,

I’ve discovered something.

If after is null, this causes the query to return no data.

Prior I created my options object like this:

{
    size: size,
    after: faunaQueries.toExpr(
        cursor as string | undefined
    )
}

It appears that .toExpr returns null if cursor is undefined.

I’ve changed it to be this, and now I’m getting data :tada:

{
    size: size,
    after: faunaQueries.toExpr(
        cursor as string | undefined
    ) || undefined,
}

If you hadn’t asked me what the options object looked like, I would have never figured this out. So thank you :pray:

PS: TypeScript didn’t catch this because I had options set to Record<string,any> :blush:

Now it’s: options: {size: number, after: Expr | undefined}.

1 Like

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.