Index Sorting that only returns Ref of document

Is it possible to sort an index and return the refs only?

CreateIndex(
name: “latest_objects_by_date”,
source: Collection(“Objects”),
values: [
{ field: [“ts”] }, // Somehow hide this in return but also sort them
{ field: [“ref”] }
]
)

I want it to return as a set so I can use filter on it

Hi @zuiluj,
You can use such index, get the result this way and use Filter() on the result:

Filter(
  Map(Paginate(Match("latest_objects_by_date"),{size:5}),Lambda('x',Select([1],Var('x')))),
  Lambda('i',Equals(<your Ref here>, Var('i')))
)

Let me know if this help.

Luigi

1 Like

You can also get sorting in a round about way using join. Assuming you have a index like this

CreateIndex({
   name: "ref_by_ref",
   source: "_",
   terms: ["ref"]
})

Then you can always join any index that contains a ref in the values into it to get a sorted set of refs. In your case

Join($match, Lambda(["ts", "ref"], Match(Index("ref_by_ref"), Var("ref"))))

should work. See Join for details.

edit – above approach is not recommended

When transforming a Set in this way, it is recommended to use Singleton and not an additional index, because it will reduce Read Ops and perform better. Writes and storage may also be cheaper without the additional index.

Join(
  $match,
  Lambda(["ts", "ref"], Singleton(Var("ref")))
))
1 Like

While this is extremely helpful. The reason I need it as set is because there is a paginate after the filter.
Why?
It’s in my graphQL schema (so I can use graphQL _size pagination). Paginate requires a set so I need to pass a set to filter so it will return a set too

This might be what I need.
So I tried creating an index that takes a ref
The sorted index works fine but when I use it with Join it’s not sorted anymore.
Any idea why? (Mind that I did what you said exactly and I still used the same index the one I posted)

It is sorted, but by reference. I guess I need to understand what the complete query is before I give you any more bad advice :slight_smile:

1 Like

We’ll need to see that code to properly help you :wink:
In the meantime the short answer to your question is that you can not get an index sorted on other values than a ref and only return the ref. Happy to help further once you provide us with a bit more details on your use-case (preferably the part of the query you are talking about that will wrap around this index, the part that is the reason you need a Set). We might be able to rewrite that part for you to accomplish your goals.

Why? FaunaDB gives you access to the raw power of the index and somewhat exposes how they are structured (ordered). There are databases that do differently but you have to realise they will be trying to do things behind your back that might not be efficient, we prefer to give you efficient constructs and if you need something differently, you’ll have to use Paginate/Map/Filter but you’ll realise exactly what the impact of those is without magic that happens behind your back :slight_smile:, hope that makes sense.

1 Like

A comprehensive answer!
For the code, this is how it essentially goes:

fauna_client.query(
        q.update(q.function("filter_collection"), {
            "body": q.query(
                q.lambda_(["input", "size", "after", "before"],
                    q.let({ 
                        "result": 
                            q.filter_(
                                # Lambda to apply filter
                                q.lambda_("doc",
                                    q.and_(
                                        # 'Make' field filters
                                        q.equals(
                                            q.select(["data", "filter_field"], q.get(q.var("doc"))), 
                                            q.var("filter_field")
                                            )
                                        )
                                    ),
                                q.if_(
                                    q.equals(q.var("index_collection2"), None),
                                        # Index to look into
                                        q.match(q.index("index_collection1")), 
                                        q.match(q.index("index_collection2"))
                                    )
                                ),
                        "page": q.if_(
                            q.equals(q.var("before"), None),
                            q.if_(
                                q.equals(q.var("after"), None),
                                    q.paginate(set=q.var("result"), size=q.var("size")),
                                    q.paginate(set=q.var("result"), size=q.var("size"), after=q.var("after"))
                            ),
                            q.paginate(set=q.var("result"), size=q.var("size"), before=q.var("before"))
                        )
                    },
                    # Show results by acquiring every document in the paginated result
                    q.map_(
                        q.lambda_("docref",
                            q.get(q.var("docref"))
                        ),
                        q.var("page")
                        )
                    )
                )
            )
        })
    )

That’s how I my UDF constructed right now. Mainly, I want the sorted index for the filter_ index
I mainly want to use a UDF since GraphQL doesn’t have out-of-the-box filter yet. I’m looking forward for it also

Right now, I managed to make it work but by using sorted Ref. It seems to work well but if there great practices let me know!

Disclaimer: I’m not that familiar with the python syntax.

So the essence of your query is:

q.let({
  "result": q.filter_(
      q.and_(
        field filters q.equals(
          q.select(["data", "filter_field"], q.get(q.var("doc"))),
          q.var("filter_field")
        )
      )
    ),
    q.if_(
      q.equals(q.var("index_collection2"), None), 
      q.match(q.index("index_collection2"))
    )
  ),
  "page": q.if_(
    q.equals(q.var("before"), None),
    q.if_(
      q.equals(q.var("after"), None),
      q.paginate(set = q.var("result"), size = q.var("size")),
      q.paginate(set = q.var("result"), size = q.var("size"), after = q.var("after"))
    ),
    q.paginate(set = q.var("result"), size = q.var("size"), before = q.var("before"))
  )
}, #Show results by acquiring every document in the paginated result q.map_(
  q.lambda_("docref",
    q.get(q.var("docref"))
  ),
  q.var("page")
))

I’m still confused by the question.
Doesn’t it solve your problem to just use the index as you defined it:

CreateIndex(
   name: “latest_objects_by_date”,
   source: Collection(“Objects”),
   values: [
       { field: [“ts”] }, // Somehow hide this in return but also sort them
       { field: [“ref”] }
    ]
)

Along with a slight modification to your query (namely adding two parameters in the lambda part at the end and only using the ref parameter)

...
}, #Show results by acquiring every document in the paginated result q.map_(
  lambda ts, ref:  q.get(q.var(ref)),
  q.var("page")
))

(not 100% sure if I got that right, as mentioned, never coded with the python driver)