Problem with document streaming

[olazabal]

I use Fauna document streaming in my browser app. It works great for 99.9% but I have one customer where the stream does not work. Seems like the stream is started but the client never gets the ‘snapshot’ event.

Is there some browser setting or firewall issue that could block streams from working? How does the document streaming work under the hood in Fauna JS driver? Is it using something else than HTTP/2 Server Push? I read that support for that has been disabled in Chromium (Removing HTTP/2 Server Push from Chrome - Chrome Developers).

[ptpaterson]

Fauna does not use server push. Fauna’s streaming protocol is inspired by (but not implemented with) Server Sent Events (SSE). Learn - Fauna Documentation

Similar to the SSE protocol, events are communicated over a text-based channel. Each event is formatted as a single-line JSON object. Unlike SSE, Fauna adds an additional line ending, \r\n, to delimit payloads, which helps with JSON parsing when network middleware splits the event payload into multiple packets.

From the client’s perspective, it’s just a regular request that takes a long time to download all of the data.

In practice for the javascript driver:

• For “fetch” environments, like the browser or Cloudflare workers, we use the fetch API.
• In Node, we default to using the built-in http2 module, but still make a standard POST request.

In both cases, the client and server keep the connection open so the server can continue to send additional packets of data all as a part of a single request.

So why does it not work for your customer?

That is an excellent question!

Missing documents

If the resource you are requesting a stream for doesn’t exist, then the stream will fail to start. Make sure that your customer is not trying to access any missing documents or Set that doesn’t exist.

Did the stream get started?

Seems like the stream is started but the client never gets the ‘snapshot’ event.

Does your customer receive the “start” event? The “start” event is what indicates that the request to open the stream was successful.

The “snapshot” event actually comes from a standard Get(your_doc) FQL query, that is executed once the “start” event is received. The order of operations is

1. Send a POST request to the streaming endpoint
2. When the connection for the stream request is successfully made, the driver issues a “start” event. The driver also now sends a regular query to Get your document.
3. When the query to Get your document returns, the driver issues a “snapshot” event with the result
4. When the request receives data, and that data signals the end of a streamed event with the \r\n delimiter, the driver issues a “version” event.

Browser support

Do you know precisely what browser they are using? We only test against and support stable Chrome, Firefox, and Safari. Also IE 11, if you need that

If they are using an unsupported browser, that could be part of the issue.

If they are using a supported browser, I recommend opening a support ticket at support.fauna.com or by emailing support@fauna.com. Please have your customer capture a HAR file that includes the requests to open fauna streams and attach with the ticket. Generating a HAR file for troubleshooting – Zendesk help

[olazabal]

Thanks for the reply and all that info, great to know how it works under the hood and that the stream for the client is just a regular long request.

Ok, I added some logging and the customers client actually does NOT receive “start” event after calling start() for the stream. But also no errors. The weird thing is that this reproduces in many browsers and computers in the same customers network, as if they had something preventing the stream from starting. Everywhere else the stream to the same document works. Well I’ll try to get into their environment to do some debugging. If necessary I’ll generate HAR file and file a ticket.

[wallslide]

Could you please paste the code that you are using to setup the stream. I wonder if the stream is getting started before you are defining event handlers somehow?

[olazabal]

Below is the code for setupping the stream. So at the customer, we get “starting project stream…” but never “project stream started” in the console. I’m waiting to to get to the customers environment to do some debugging, solving this will be much easier then.

function createProjectStream(id) {
	console.log("creating project stream");
	
	if (zeit.projectStream) {
		console.log("closing project stream");
		zeit.projectStream.close();
	}

	zeit.projectStream = client.stream.document(makeProjectRef(id))
	.on('start', () => {
		console.log("project stream started");
	}).on('snapshot', snapshot => {
		console.log("project stream snapshot received");
		setProjectData(snapshot.data);
	}).on('version', version => {
		setProjectData(version.document.data);
	}).on('error', () => {
		if (zeit.maintenance || zeit.error)
			return;

		console.log("error in project stream. Retrying in 5s.");
		setTimeout(() => { createProjectStream(id) }, 5000);
	});

	console.log("starting project stream...");
	zeit.projectStream.start();
}

[olazabal]

Ok, right after I posted the above, I got msg from the customer saying they got it working by tweaking some SSL settings in their web proxy. So “problem” solved. I just wonder if any web page in the customers network can work, if they have such tight settings in their systems blocking requests.

But this was good lesson to learn how the streaming actually works.