Unix socket API?

[lrowe]

It struck me that wait_for_requests_paused() is essentially a blocking read so what if tenant programs could open a unix socket and run a standard web server?

• Most web server frameworks already support serving requests over unix sockets.
• Unix sockets avoid overhead of networking stack.
• Would incur additional overhead from serializing/deserializing request but that is only of the order of 10us per request which for many applications is negligible.
• Easier testing since you can simply run the tenant program under linux and make requests against the unix socket.
• With the experimental executable_heap support presumably dlopen should work so you could run existing linux binaries which would be particularly helpful for applications running in interpreted languages. (This might also require changing the main arguments protocol so that the exact arguments could be specified.)

The biggest downside would probably be the need to support multiple mechanisms like epoll/select/blocking reads.

While I'm mostly interested in Deno right now I don't expect it would help there immediately since Deno runs its web server on a separate thread. It would probably need a --single-threaded option adding to disable v8 background threads and run the web server in thread.

[fwsGonzo]

I have thought about this and spent alot of effort in this area already. The primary research tool and leading driver for this is my libriscv emulator. In it, I have attempted to add proper emulation of event fd's, signals and epoll/select. It can fetch websites using HTTPS in Zig, but not any other frameworks/run-times that I know of. I suspect that programs just don't support cooperative threading, as they rely on preemption for many features. Some kind of preemption is apparently necessary to drive forward progress. Adding preemption to TinyKVM changes the architecture and adds new moving parts, which is scary. Second, Varnish doesn't have the original request so it has to be reconstructed for the VMOD program. Then it has to be written to the guest through readv emulation, which adds overhead. The way the VMOD works now is that everything is presented to you before we even enter the guest, and you can query Varnish for additional request information. This adds little overhead.

Support for file descriptors is already present through the file API and since they work the same for sockets, there's not too much difference. However, supporting a modern Web framework is a gigantic effort because of the complexity of todays frameworks. Nobody calls bind into connect into write/witev anymore. There's many moving parts to make it asynchronous, such as inter-thread message queues, eventfd's etc. There's additional complexity in errno return values, data alignments and keeping overhead down. I experimented with taking over (stealing) unencrypted fd's from Varnish so that this VMOD could directly deliver payloads. Ultimately, it's defeating the point of being embedded in Varnish and just remains a curiosity. Not to mention nobody uses plain unencrypted requests anymore. I can't take over an encrypted fd in Varnish without a larger change internally, unfortunately. Varnish acts as a shield to backends/origins, and so trying to "work around" that is probably counter-productive.

Support for dlopen() is definitely happening. I'll have to port all the syscall emulation from this VMOD into TinyKVM proper for that first. After that, adding support for mmap fd's is fairly straightforward, and that also means that we can now load normal dynamic executables. Removing the requirement for static or static-pie executables. OpenCV is hard to build statically, and I look forward to being able to use that without any special build. Good point about the main_arguments. It does seem like a breaking change is necessary.

I agree with your points about easier integration and testing. However, this VMOD is currently designed to go through a preparation phase (main VM), and then spend as little effort as possible generating a response (request VMs) using that preparation. I think this always calls for a modicum of specialized handling, in order to make the most of it. To be honest, I'm just happy that everything seems to be working at this point. It's all very complex already!

I hope this doesn't scream "never", because over time I will definitely port over most of the syscall emulation already present in libriscv. There's no reason not to, as it's all very similar. But, it's not high on my list right now. I want to spend some time to improve on what's already there, if that makes sense. As mentioned, I'm just very happy that things are seemingly working, and I want to see if the system can remain stable over time, as that is the ultimate test.