I’m getting an inconsistent 403 on document updates with a mildly complex ABAC predicate defined. I need some help figuring out what to try next. I have run into a wall several times on this. Each time I thought it was something I was doing and completely rewrote my ABAC predicates. But, now, I’m wondering if it might be somewhat out of my control. I have a simple update:
When I call this update outside of a JS .reduce it always works. When I call it inside of a JS .reduce it may or may not work. If I do it three times, it might work once. I’ve been checking my predicate CheckPrivilege against q.Ref(Items(), item._id) before running the update. It always returns true when I would expect it to.
// NOTE: itemKey is the same as its _id
await Object.keys(store).reduce(async (
agg,
itemKey
)=>{
console.log(await
client.query(CheckPrivilege(
q.Ref(Items(), itemKey),
"write"
))) // always true when I would expect
console.log(await client.query(
q.Update(q.Ref(Items(), itemKey), {
data : {
name : store[itemKey].name,
itemType : store[itemKey].itemType
}
})
)) // sometimes works
const result = await client.query<ItemDocI>(
Commit(store[itemKey])
); // this actually seems to work even more rarely
return {
...await agg,
...result ? {
[result.ref.id] :
translateFromFauna(result)
} : {}
}
}, {})
Is there any chance that too large of a predicate could be timing out and returning false? (edited)
For a little info on how my ABAC system works:
Auth0 identity.
User looked up in collection (indexed by Auth0 identity).
User role information looked up from User.
User role information assessed against a Resource document which each Item has.
I was able to decrease the chance of this issue by reducing over the store within my FQL expression as opposed to reducing in JS. However, I’m still concerned about this as potential behavior that occurs with lots of concurrent users.
We likely need more information to discover where the problem might exist. Feel free to DM me if you’d rather not share the details publicly.
What does Items() return?
What does CheckPrivilege do?
How many documents might be involved?
What do your ABAC predicates look like?
It looks like you are running several queries in succession. Have you considered combining them into a single query?
In general, there are a couple of details to keep in mind while working with ABAC roles:
ABAC roles operate on any matching combination of membership and privileges. Depending on which identities and privileges are involved, your ABAC predicates might be firing on most/all queries, and they have to be evaluated before the body of the current query can be processed.
While ABAC roles can be used to filter out documents that an identity should not access, they can be expensive on collections with many documents. Consider a simple query such as Paginate(Documents(Collection('big_collection'))). In order to provide a single results page, thousands or millions of documents might need to be evaluated.
In those situations, it is often better to use an index that returns documents based on an “owner” identity field because an index can provide a group of matching documents without evaluating all of the other documents in the same collection.
I’m not concerned about sharing most of the details. Here are some notes from me:
ABAC System
I’m currently rewriting my ABAC system. Hopefully, I will reduce complexity on my end. But, I think it needs to be done anyway.
Check Privilege used to call a UDF that ran an index match, up to three Gets on refs, two Intersections, and control flow depth of up to 5 (per operation).
The new version of Check Privilege runs a map over a set bound by the number of Items which a user can access, control flow of depth two, n of subset of set index Matches and Gets, and a max privilege method which I describe below.
On a higher level, I’m switching from… (a) defining all privilege information on a Resource Document which describes privileges for all Actors and their “manner of interaction” with an Item,
to (b) defining individual AppRole documents describing the privilege for an Actor against an Item.
Previously, Check Privilege traversed to said Resource Document. Now, I’ve abstracted the “manner of interaction” bit away into Actors.
Check Privilege now matches a user to all of its actors then computes the maximum privilege set amongst the AppRoles associated with these Actors and a given Item. Max privilege computation is O(actors for a user for an item).
I believe this is conceptually the better way to do what I was doing. But, was trying to avoid the rewrite. Oh, well…
I’m still trying to debug this, so I’m not sure if it’s actually any better for the 403 issue at this point.
Single query
Rob had me go for reducing within the query, which works just fine if I keep the document sizes small (which isn’t a problem). However…
Scale
I still encounter the behavior if I make requests in relatively close succession, which had me concerned that this issue would occur with lots of concurrent users. I believe Rob thought there was a particular pattern that might help which is why he prompted me to post here.
Yeah, we figured it out. It looks like Fauna was making too many requests to Auth0 for whatever reason. The solution ended up being to wrap the update calls in a UDF and not rely on Fauna’s ABAC.