[Feature request / Help] pagination tokens for FQL

Currently when you run a query that looks similar to:

Map(
  Paginate(
    Match(
      Index("usersByOrganization"),
      Ref(Collection("organizations"), "123")
    ),
    { size: 10 }
  ),
  Lambda(
    ["ref", "user", "role"],
    Let(
      {
        user: Get(Var("user")),
      },
      Merge(Var("user"), {
        data: Merge(Select(["data"], Var("user")), {
          role: Var("role"),
        }),
      })
    )
  )
);

It will return an after and before which will look somewhat like:

[
  Ref(Collection("organizationUsers"), "123"),
  Ref(Collection("users"), "123"),
  "Admin",
  Ref(Collection("organizationUsers"), "123"),
]

This comes with a problem, it’s that we need to pass this to the frontend so that it can pass it back to the backend when it needs to fetch the next results, however, we really can’t pass it along looking like that, right? So we want to somehow find a way to convert this into a more workable string like, one solution is to encode it into base64 which will work BUT will produce a very long string, like so:

async function main() {
  // converted the refs into multi-dimensional array where `item[0]` is the `collection` and `item[1]` is the `id` provided that `item` is an array, otherwise it will just be a string.
  const data = [
    ['organizationUsers', '341496517539922002'],
    ['users', '341496448941031506'],
    'Admin',
    ['organizationUsers', '341496517539922002']
  ];

  const base64Data = Buffer.from(
    JSON.stringify(data),
    'utf-8'
  ).toString('base64');

  console.log(base64Data);
}

main();

That will produce W1sib3JnYW5pemF0aW9uVXNlcnMiLCIzNDE0OTY1MTc1Mzk5MjIwMDIiXSxbInVzZXJzIiwiMzQxNDk2NDQ4OTQxMDMxNTA2Il0sIkFkbWluIixbIm9yZ2FuaXphdGlvblVzZXJzIiwiMzQxNDk2NTE3NTM5OTIyMDAyIl1d which is 168 characters long! It just doesn’t feel right to use this.

Do you guys know any way to produce a much smaller token which I can send to the frontend and it can send back as a query string param e.g. /users-list?nextToken=asdasdasda.

I’ve thought of another way to circumvent this, which is to save the after into redis or something similar – heck, save it to faunadb :joy: but I think that’s a bit too much, right? considering that this is not permanent data but something that the frontend might not even need if the user didn’t go as far as to fetch the subsequent results.

I’ve been searching on google too but I can’t find relevant results to what I want, maybe I just can’t find the right keywords.

Feature Request

Maybe make this a built-in in FQL instead of returning an array for after and before, it should just return a string which can be used to fetch the next results.

1 Like

I think we are asking for more or less the same thing? Simplify Paginate .after/.before Cursors

1 Like

Pagination cursors need all of the relevant information to dead-reckon to the correct position in the Index, or any Set which can involve multiple Indexes. Even if we encode the cursor as a string, all of the information needs to be serialized.

For passing back and forth between your client and server, you can transform the JSON back into FQL using the function parseJSON, which is exported from the javascript driver.

That said, the JSON representation of cursors, particularly if they contain References, can be unwieldy – I agree with you. Encoding the cursor as a string is something we did with the GraphQL API. The cursors are not meant to be transparent; that is, if you need to specify a range of values yourself, use the Range function, don’t manipulate cursors directly. In that case, encoding a cursor in an easier-to-handle way makes sense. I’ve highlighted your request with our Product team and will bump @hunp’s feature request!

2 Likes

Thanks for the explanation. Hoping for a resolution soon.