Dynamically setting Paginate options like page_after fails with error

[Polecat7]

Hi there,

Anyone managed to write a function that supports dynamically setting or not setting page_after, page_before options on a paginate depending on options passed into a UDF?

As a test, I store an object like this page_size : {size:3} which you can see in the below code. When I pass that into the paginate as a Var(‘page_size’) it errors out with:

“code”: “invalid expression”,
“description”: “No form/function found, or invalid argument keys: { paginate, raw }.”

Map(
  [
    [
      "TestArray",
      "options",
      Ref(Collection("Options"), "351011260955886162"),
      {
        page_size: 3,			// This is optional
        page_after : null,		// These can be removed, or null if not avail.
        page_before : null		// These can be removed, or null if not avail.
      }
    ]
  ],
  Lambda(
    ["collection_id", "field_name", "field_value", "config"],

    Let(
      {
        index_idx: Select(["index_idx"], Var("config"), 0),
        index_name: Concat([
          Var("collection_id"),
          "_by_",
          Var("field_name"),
          "_ref"
        ]),
        
        page_size : { 
          size : 3

          // Default to 100 items (to be set as const via JS)
          //size : Select(["page_size"], Var("config"), 100)
        },
        page_before_raw : Select(["page_before"], Var("config"), null),
        page_after_raw : Select(["page_after"], Var("config"), null),
        page_before : If(IsNull(Var('page_before_raw')), {}, {'page_before' : Var('page_before_raw')}),
        page_after : If(IsNull(Var('page_after_raw')), {}, {'page_after' : Var('page_after_raw')}),
        
        // You MUST put a [] around the second pair, otherwise merge does NOT work
        page_options : Merge(Var('page_size'), [Var('page_before'), Var('page_after')]),
      },

      Map(
			
        // Problem here - cannnot set Paginate paging with variables...
        Paginate(
          Match(Index(Var("index_name")), Var("field_value"), 30),

          // Fauna does not like this...
          Var('page_size')
        ),  //, Var('page_options')

        Lambda(
          "index_result",

          Let(
            {},
            {
              //sel : Select([Var('index_idx')], Var('index_result'), ''),
              sel: Select([Var("index_idx")], Var("index_result"), ""),
              idx: Var("index_idx"),
              index_result: Var("index_result"),
              index_name: Var("index_name"),
              x : Var('page_options')
            }
          )
        )
      )
    )
  )
)

If I remove

Var(‘page_size’)

then the code works.
If I use the form below as an argument to paginate it also works:

{size:Select([“page_size”], Var(“config”), 100)}

In the test output, page_options is correctly building to the object - we can see that x below has the value {size:30}

{
  sel: "",
  idx: 0,
  index_result: Ref(Collection("TestArray"), "351186618918371928"),
  index_name: "TestArray_by_options_ref",
  x: {
    size: 30
  }
}

Ultimately the aim is to pass

Var(‘page_options’)

which dynamically contains the config based on what is passed in. But paginate does not like it.

Does anyone have any ideas on how I could solve this?
I was reading other posts, and it occurred to me that using range, and doing some other calculations to manage paging might be possible, but for now, I’d prefer to know how others are dynamically setting page_after/page_before when they are needed, and not when they are not.

All documentation I have found has statically typed text, I have not found an example of passing in variables. If anyone knows of documentation that explains this, or a way to solve this, I would be very grateful.

Thanks for taking time to read this!

[ewan]

What you are attempting to do does not work. Effectively, the Paginate options must be statically defined. This particular issue is covered in the documentation: Paginate - Fauna Documentation

However, it would certainly be useful if they could be dynamically defined. Feel free to create a “feature request” post asking for this capability.

[ewan]

You can do something like this:

Let(
  {
    size: 3,
  },
  Paginate(
    Documents(Collection("Letters")),
    {
      size: Var("size"),
    }
  )
)

The contents of the options object can be dynamic, but the options object itself must be static.

[ptpaterson]

The GraphQL docs provide an example of a UDF that handles a Page. User-defined functions (for custom resolvers) - Fauna Documentation

It comes down to branching logic to assign a cursor or not.

If(
  Equals(Var("page_before"), null),
  If(
    Equals(Var("page_after"), null),
      Paginate(Var("match"), { size: Var("page_size") }),
      Paginate(Var("match"), { size: Var("page_size"), after: Var("page_after") })
  ),
  Paginate(Var("match"), { size: Var("page_size"), before: Var("page_before") }),
)

NOTE: null is a valid value for a cursor. You might consider ContainsPath("page_before", Var("config")) if null is a possible cursor in your use case.

[Polecat7]

Many thanks to both @ewan and @ptpaterson for your help with this.

@ewan thank you for the link to the documentation. I had looked at that before, but not seen the warning in the pink box, which directly states, as you say that a dynamic object is not supported:

Other FQL functions accept a param_object , which could be composed with functions that return Objects, such as Merge. Unlike those functions, Paginate requires that these parameters be expressed as a JavaScript object within the driver code (they are not sent to the server within an object).

I could not think why I had not seen this. I searched for the link on my dev computer this morning and noted that the pink box did not display. Mmmm… and I figured it out: on example code, you can change the format to JS, Shell, Go, etc. When set to Shell, the Parameters and Returns sections roll up to just a header - so I cannot see the pink warning box. Instead, it looks like this:

So at least I now know why - and that I should watch out for this in the future.

@ptpaterson as you note, branching logic is the way to go. I was hoping to avoid that, but if that’s the way, OK.
Thank you also for pointing out Null is a valid value and that contains path is a better way to do things! You have probably just saved me hours of potential bug hunting months down the road.

For anyone needing a reminder, the document states

It is possible to fetch the last page of a set without knowing how many pages are available by specifying a synthetic before cursor: * { before: null }: for indexes that have no values fields defined, or indexes where the first values field uses the default order.

I’ll go the branching route, and use your other suggestions.
A good day to you both, happy holidays! (Though we probably all have a piles of work, do take some time off.)

[Polecat7]

This is a follow up post showing how I got this working, following suggestions from @ewan and @ptpaterson, for anyone interested.
It’s relatively straightforward.

This particular test happens to dynamically build the name of the index it needs to query; but this is nothing to do with the Paginate option build we are considering here.

The options passed in are of the form

{
  size : n,
  [page_before: ref | null,]
  [page_after : ref]
}

Keep in mind that depending on the inputs expected by your indexes, your page before and after values may need to contain other entries.

I have tested all the different combinations of:

• size
• size, page_before:ref
• size, page_before:null
• size, page_after:ref

and they all work correctly.
Thanks again for the support!

Map(
  [
    [
      "TestArray",
      "options",
      Ref(Collection("Options"), "351011260955886162"),
      {
        // Optional
        page_size: 3,

        // If not needed do not pass in, to jump to last page, pass as null
        //page_before : null,

        // If not needed, do not pass in
        //page_after: [
          //Ref(Collection("TestArray"), "351724795175371351")
          //Ref(Collection("TestArray"), "351724872490025559")
        //]
      }
    ]
  ],
  Lambda(
    ["collection_id", "field_name", "field_value", "config"],

    Let(
      {
        index_idx: Select(["index_idx"], Var("config"), 0),
        index_name: Concat([
          Var("collection_id"),
          "_by_",
          Var("field_name"),
          "_ref"
        ]),

        // Default to 100 items (TODO; set 100 from DEFAULT_PAGE_SIZE as const via JS)
        page_size: Select(["page_size"], Var("config"), 100),
        page_before: Select(["page_before"], Var("config"), null),
        page_after: Select(["page_after"], Var("config"), null),
        has_page_before: ContainsPath("page_before", Var("config")),
        has_page_after: ContainsPath("page_after", Var("config")),

        // (TODO; set 30 from STATUS_ACTIVE as const via JS)
        match_field_value: Match(
          Index(Var("index_name")),
          Var("field_value"),
          30
        )
      },

      Map(
        If(
          Not(Or(Var("has_page_before"), Var("has_page_after"))),
          Paginate(Var("match_field_value"), { size: Var("page_size") }),
          If(
            Var("has_page_after"),
            Paginate(Var("match_field_value"), {
              size: Var("page_size"),
              after: Var("page_after")
            }),
            Paginate(Var("match_field_value"), {
              size: Var("page_size"),
              before: Var("page_before")
            })
          )
        ),

        Lambda(
          "index_result",
          Let(
            {},
            // TEST OUTPUT
            {
              result: If(
                IsArray(Var("index_result")),
                Select(
                  [Var("index_idx")],
                  Var("index_result"),
                  "INVALID_INDEX"
                ),
                Var("index_result")
              ),
              raw: Var("index_result"),
              index_name: Var("index_name")
            }
          )
        )
      )
    )
  )
)

[ewan]

I’m glad you got it working!

Thanks for the heads-up on the odd presentation when the Shell language is selected. Switching languages mostly just hides/shows content blocks on the page, and it looks like we missed providing Shell content there. We’ll get that fixed up, probably in the new year.