Embedding into UE4

[kamrann]

Hi there. I've just come across the libary and am wondering if it's suitable to be used in such a context. Essentially, where all the plot data is generated programmatically, and then plots are rendered using the internal API of UE4 (or any other similar framework).

Would this mean implementing a custom backend? From a brief look through the code, I'm unsure if that interface is the right place since it seems to have functionality related to high level user input and window management.

And if a backend is the right way to go, is the core library currently using that interface fully? I'm confused by the parameters of a number of the drawing functions, and the OpenGL example doesn't implement them so I'm wondering if that's because the library itself is not yet ready for custom backends?
Thanks!

[alandefreitas]

Hi @kamrann,

You're right. I would delegate matplot++ for benchmarks, experiments, or any interface where you only need the plots.

At the moment, if you need integration with OpenGL, Qt, etc... the backend interface and the backends themselves are not quite there yet.

There are people working on it, but it's currently more like a proof of concept, demonstrating how things are implemented separately. There's nothing very useful out of the box, unless you're interested in working on the interface itself.

There are a number of issues and discussions about the backend interface, in case you're more interested.

[kamrann]

Thanks Alan. Yep, I've started to have a look through, but I'll dig in some more.

Could you just clarify one thing - is realtime plotting (with data being updated live at high frequency) an intended use case for the library? I'm not concerned about whether it's ready to use out of the box, just as to whether that's within scope. I don't have any background in Matlab or similar environments, so just want to be sure I'm not investigating something that isn't really aimed at my use cases.

[alandefreitas]

Real-time plotting is perfectly fine. It’s very useful for benchmarks and it’s one of the most common use cases. It won’t look good if you have many FPS though.

[kamrann]

Great, thanks for the clarification.
I'll look into what would be needed for a minimal UE4 backend, then can maybe give you some feedback on the interface once I've got to grips with the code. Cheers.

[alandefreitas]

The backend interface is supposed to roughly look like the matplotlib interface. With the difference that matplotlib uses Agg, which is 2d only, while this library has 3D plot types. We started trying to reuse class names from matlab and matplotlib not to confuse people, but a better name for that would be a canvas_interface.

Besides being 3d, our second problem is that our main backend is gnuplot pipes, which are very slow. To make things a little faster, we had to include this consumes_gnuplot_commands to the interface, which bypasses many of the common drawing functions. The figure_type will send_gnuplot_draw_commands or send_draw_commands depending on the backend type.

Many axes_objects can only send gnuplot commands while some can do both (like the OpenGL backend example). This is not such a big problem, because (i) calling the backend functions directly is much easier than reverse-engineering efficient Gnuplot commands, and (ii) most axes objects can be expressed as a composition of line objects. We can even include these functions as we need them. What's really important is that the objects are modeled 100% separately from the backend, which they are.

The biggest problem is coming up with the backend in a way that works well cross-platform. I've been working on that as a separate project. I started with OpenGL, then moved to Vulkan. Then made something that would work also with WebGL / WASM and some non-interactive backends (useful for Cling, for instance). One disadvantage here is that this is much more efficient but it blocks the main thread, although there are a few ways to deal with that. I'm trying to integrate everything before merging it with this repo. From that point on, then it should be trivial to create backends like Qt or WxWidgets, which support OpenGL canvases if I'm not wrong. This takes time though, not because it's particularly difficult to implement, but because I have a lot of work on other repos going on in parallel.

[kamrann]

Thanks for the detailed explanation. I'll close this and post new threads as I have more specific questions.