Run query against two databases in Fauna natively (Not Fauna driver)

Feature Requests
1

I want to run a FQL query/UDF on Fauna, that fetches documents from a Collection in Database 1 and creates documents in another Collection in Database 2.

Something like

let database1 = Database(<key>).Section.all()
let database2 = Database(<key>).Section
 
database1.map(section => database2.create(section))

This use case is currently only possible with the fauna driver, but supporting this in Fauna directly would lead to a significant performance increase because of the missing network traffic.

2

Hi @Mike!

Reading child databases is currently possibly in FQL v4, but not writes.

Neither reads nor writes to child databases are possible yet in v10, but reading child databases is on the roadmap. We are also considering the ability to write to child databases.

It looks like you are trying to copy data from one child DB to another. Can you share more about that use case?

Side note:

Writes are not supported when using map. This is because map is a lazy set operation, so writes could potentially change the Set out from under the computation.

If you want to perform writes while iterating over a Set, you should use .forEach.

You should also be careful about the scalability of reading from one Set to use directly in writes. Our docs for forEach note that “Because this method scans the full set, the query can time out for large sets.”

The page limit for v10 is 16000. With a process that must read a large set and do something with it, you may find a more stable solution to:

  • read/paginate the necessary data locally, then
  • perform any necessary write operations with the data

Splitting these requests into smaller chunks could help to gracefully handle throughput limits as well.

3

Regarding your question about the use case:

We have created a database migration tool

  • Sync/Migrate CMS data to a product DB’s
  • Stage data from Dev to Preview to Production DB’s
  • Migrate a full database to another database
  • Partial migration of schema or data

To further improve it, I would like to run the whole migration in Fauna to increase performance and to profit from ACID compliance.

For more insights, you can also ask Bryan; I shared with him all the insights :slight_smile: