Help with Collection references & data modeling

Help
1

New to FaunaDB and have been able to get the basic stuff working (creating and reading collections and indexes.) I am now trying to create some data references in my collections and I am a little lost on how to go about it. Most of the examples I find have the References hardcoded, and in my case, I don’t have this data. I have some keys that I can use to query an index, but not sure how to use that to get a Ref to pass into my other query.

Say I have a collection of Shop objects with a unique “key” data attribute. I also have an index called shops_by_key.

I also have a collection called “Carts”, which will be used to hold references to Shops. So a Cart has-many Shops. In my head, the FQL document/entry looks something like this:

{
  "ref": Ref(Collection("carts"), "287656564382237192"),
  "ts": 1610730727970000,
  "data": {
      shopRefs: [
           Ref(Collection("shops"), "287656564382237194"),
           Ref(Collection("shops"), "287656564382237195"),
           Ref(Collection("shops"), "287656564382237197"),
           ....
      ]
  }
}

Within my app, I am making a request that sends over a collection of shop objects, but I have no reference to the Ref ID in fauna anywhere. I just have the unique shop “key” that I can use to look up the shop from the “shop_by_key” index.

So question is, how do I use this information to construct a Create/Update command using FQL, which takes an array of shop objects (with unique key), looks up each shop from the “shop_by_key” index, and passes those References into the Create Cart query.

Hope I was able to explain that so that it makes sense. Thank you

2

So I got a bit further on this and figured out how to get references from an index. I am now running into another issue which I cannot figure out. I have the following function:

async function saveCart(shops: Shop[], userId: string) {
    const shopKeys = shops.map((shop) => shop.attributes.key)

    const matchExpr = q.Match(q.Index('carts_by_userid'), userId)
    const shortId = q.Let(
      { gId: generateId() },
      q.If(q.Exists(matchExpr), q.Select(['data', 'shortId'], q.Get(matchExpr)), q.Var('gId'))
    )

    const data = {
      shortId,
      shopRefs: q.Map(
        shopKeys,
        q.Lambda(['shopKey'], q.Select('ref', q.Get(q.Match(q.Index('shops_by_key'), q.Var('shopKey')))))
      ),
    }

    return await serverClient.query(q.Update(q.Select('ref', q.Get(matchExpr)), data))
  }

What I am trying to do is update a cart that belongs to userId and pass it in shopRefs, which are internal faunadb references to the shops collection. I am getting the shop references using my shops_by_key index and mapping over shopKeys. When the individual parts of this query in the CLI, things seem ok. But when I run the whole thing, I get the following error:

Unhandled rejection: [NotFound: instance not found] {
  description: 'Set not found.',
  requestResult: RequestResult {
    method: 'POST',
    path: '',
    query: null,
    requestRaw: '{
      "update": {
        "select": "ref",
        "from": {
          "get": {
            "match": { "index": "carts_by_userid" },
            "terms": "3ac1b839d511a997985af76e4b3338494f06f5c2fcdc891c6f4d38a581cc1c"
          }
        }
      },
      "params": {
        "object": {
          "shortId": {
            "let": [{ "gId": "B3KngmQU" }],
            "in": {
              "if": {
                "exists": {
                  "match": { "index": "carts_by_userid" },
                  "terms": "3ac1b839d511a997985af76e4b3338494f06f5c2fcdc891c6f4d38a581cc1c"
                }
              },
              "then": {
                "select": ["data", "shortId"],
                "from": {
                  "get": {
                    "match": { "index": "carts_by_userid" },
                    "terms": "3ac1b839d511a997985af76e4b3338494f06f5c2fcdc891c6f4d38a581cc1c"
                  }
                }
              },
              "else": { "var": "gId" }
            }
          },
          "shopRefs": {
            "map": {
              "lambda": ["shopKey"],
              "expr": {
                "select": "ref",
                "from": { "get": { "match": { "index": "shops_by_key" }, "terms": { "var": "shopKey" } } }
              }
            },
            "collection": [
              "https://www.etsy.com",
              "https://www.amazon.com",
              "https://www.target.com",
              "https://shop.coolmaterial.com",
              "https://www.thirdlove.com",
              "https://www.bestbuy.com",
              "https://www.bhphotovideo.com",
              "https://kutfromthekloth.com",
              "https://www.homedepot.com",
              "https://www.barnesandnoble.com",
              "https://www.costco.com",
              "https://www.lowes.com"
            ]
          }
        }
      }
    }',
    requestContent: Expr { raw: [Object] },
    responseRaw: '{"errors":[{"position":["params","object","shopRefs","map","expr","from"],"code":"instance not found","description":"Set not found."}]}',
    responseContent: { errors: [Array] },
    statusCode: 404
  }
}
3

For those following along, found the issue. Seems that some of the shops in my array of shop keys did not exist in my shops collection/index, which is why it was failing. When I removed those invalid keys, the query ran just fine!

Now I need to figure out what to do about missing shop keys. Ideally, I would like to create those entries if they don’t exist (I have the data when making this request). So if anyone has any ideas/clues, would appreciate it!

4

Hey, first of all, sorry that I didn’t react yesterday. It was 11pm here and apparently I was too tired to understand your question :slight_smile: .

For this last question, FQL excels at that, and you already have the partial answer in the first part of your question (the one assigned to the var shortId). Creating those entries if they don’t exist is yet another If(Exists(…), , Create())

I’ll try to write it by heart so there might be syntax errors. You would replace the lambda with something like this:

q.Lambda(
    ['shopKey'],
    Let({
        match: q.Match(q.Index('shops_by_key'), q.Var('shopKey')),
    },
    q.If(
          q.Exists(q.Var(' match')),  
          q.Select('ref', q.Get(q.Var('match'))),
          q.Create(Collection('shops'), { data: { ... }})
    )
)

The bigger question probably is why some shops keys exist without a corresponding shop. Of course, I don’t know how they are created but if possible I would try to make sure a key and a shop are created in one transaction.

Extra resources to become an FQL ninja in case you didn’t find these:

1 Like
5

Thank you for the reply and resources @databrecht. I’m slowly getting the hang of FQL (I think I had my “ah ha!” moment.) I was able to create my collections along with the proper references. I am now trying to figure out the best way to read/retrieve the data, along with references.

When I query my collection via an index, like so:

q.Get(q.Match(q.Index('carts_by_shortid'), shortId)) // `shortId` is passed in

I get a result that looks something like this:

{
  ref: Ref(Collection("carts"), "287726086250299917"),
  ts: 1611678815460000,
  data: {
    shops: [
      {
        items: [
          Ref(Collection("items"), "288722607432278541"),
          Ref(Collection("items"), "288722607432271373"),
          Ref(Collection("items"), "288722607432275469")
        ],
        shopRef: Ref(Collection("shops"), "288722607868487181")
      },
      {
        items: [
          Ref(Collection("items"), "288722607436464653"),
          Ref(Collection("items"), "288722607432282637"),
          Ref(Collection("items"), "288722607432280589"),
          Ref(Collection("items"), "288722607432272397"),
          Ref(Collection("items"), "288722607439610381")
        ],
        shopRef: Ref(Collection("shops"), "288722607868480013")
      },
      ....etc
    ]
  }
}

I then wrote this query to get the above to pull the referenced items, but not sure if its the best way to go about this.

q.Map(
  q.Select(['data', 'shops'], q.Get(q.Match(q.Index('carts_by_shortid'), shortId))),
  q.Lambda(['shop'],
    {
      items: q.Map(q.Select('items', q.Var('shop')), q.Lambda(['itemRef'],
        q.Let({
          item: q.Get(q.Var('itemRef')),
          shop: q.Get(q.Select(['data', 'shopRef'], q.Var('item'))),
          shopType: q.Get(q.Select(['data', 'typeRef'], q.Var('shop'))),
        },
        {
          data: q.Select(['data'], q.Var('item')),
          shop: {
            data: q.Select(['data'], q.Var('shop')),
            type: q.Var('shopType'),
          }
        }
      ))),
    }
  )
)

The results do come back correctly, but I have to basically reconstruct the data structure and whatnot. Is there a way to tell Faunadb, “pull this data, a long with all of its referenced data”?

The other question I had was, is it possible to use the JS spread (...) operator on some of the FQL results. For example I would like to express this:

shop: {
  data: q.Select(['data'], q.Var('shop')),
  type: q.Var('shopType'),
}

as

shop: {
  ...(q.Select(['data'], q.Var('shop'))),
  type: q.Var('shopType'),
}

But that doesn’t seem to work