Initial raytraced lighting progress (bevy_solari) #19058

JMS55 · 2025-05-04T16:56:14Z

Bevy Solari

Preface

See release notes.
Please talk to me in #rendering-dev on discord or open a github discussion if you have questions about the long term plan, and keep discussion in this PR limited to the contents of the PR :)

Connections

Works towards Ray tracing support? #639, UE5 Megalights-like #16408.
Spawned StandardMaterial::emissive units not documented #18993.
Need to fix RT stuff in naga_oil first Hardcode fix for RT naga_oil#116.

This PR

After nearly two years, I've revived the raytraced lighting effort I first started in #10000.

Unlike that PR, which has realtime techniques, I've limited this PR to:

RaytracingScenePlugin - BLAS and TLAS building, geometry and texture binding, sampling functions.
PathtracingPlugin - A non-realtime path tracer intended to serve as a testbed and reference.

What's implemented?

BLAS building on mesh load
Emissive lights
Directional lights with soft shadows
Diffuse (lambert, not Bevy's diffuse BRDF) and emissive materials
A reference path tracer with:
- Antialiasing
- Direct light sampling (next event estimation) with 0/1 MIS weights
- Importance-sampled BRDF bounces
- Russian roulette

What's not implemented?

Anything realtime, including a real-time denoiser
Integration with Bevy's rasterized gbuffer
Specular materials
Non-opaque geometry
Any sort of CPU or GPU optimizations
- BLAS compaction, proper bindless, and further RT APIs are things that we need wgpu to add
PointLights, SpotLights, or skyboxes / environment lighting
Support for materials other than StandardMaterial (and only a subset of properties are supported)
Skinned/morphed or otherwise animating/deformed meshes
Mipmaps
Adaptive self-intersection ray bias
A good way for developers to detect whether the user's GPU supports RT or not, and fallback to baked lighting.
Documentation and actual finalized APIs (literally everything is subject to change)

End-user Usage

Have a GPU that supports RT with inline ray queries
Add SolariPlugin to your app
Ensure any Mesh asset you want to use for raytracing has enable_raytracing: true (defaults to true), and that it uses the standard uncompressed position/normal/uv_0/tangent vertex attribute set, triangle list topology, and 32-bit indices.
- If you don't want to build a BLAS and use the mesh for RT, set enable_raytracing to false.
Add the RaytracingMesh3d component to your entity (separate from Mesh3d or MeshletMesh3d).

Testing

Did you test these changes? If so, how?
- Ran the solari example.
Are there any parts that need more testing?
- Other test scenes probably. Normal mapping would be good to test.
How can other people (reviewers) test your changes? Is there anything specific they need to know?
- See the solari.rs example for how to setup raytracing.
If relevant, what platforms did you test these changes on, and are there any important ones you can't test?
- Windows 11, NVIDIA RTX 3080.

… sample of the integral

SparkyPotato · 2025-05-04T18:11:28Z

crates/bevy_solari/src/pathtracer/pathtracer.wgsl

+            let cos_theta = dot(-ray_direction, ray_hit.world_normal);
+            let cosine_hemisphere_pdf = cos_theta / PI; // Weight for the next bounce because we importance sampled the diffuse BRDF for the next ray direction
+            throughput *= (diffuse_brdf * cos_theta) / cosine_hemisphere_pdf;


This should all just cancel out to throughput *= base_color.

Yes but I don't want to do that, because then it'll make things more confusing in the future.

This isn't meant to be fast, just accurate and easy to understand.

Probably can put the math (and cancellation) in a comment?

I'd really rather leave it explicit 😅

SparkyPotato · 2025-05-04T18:28:03Z

crates/bevy_solari/src/pathtracer/pathtracer.wgsl

+            let p = tonemapping_luminance(throughput);
+            if rand_f(&rng) > p { break; }
+            throughput /= p;


For low luminances (that don't russian roulette), this would lead to dividing by a very small number (which can lead to infs). I would probably clamp that with something like:

let p = max(0.05, 1.0 - tonemapping_luminance(throughput)); if rand_f(&rng) < p { break; } throughput /= 1.0 - p;

I'll have to test this.

I had something somewhat like this before, but it lead to bias and failing the white furnace test.

Yeah this is basically what I had before, but it breaks the white furnace test. I think it's fine in practice.

That's weird, it should only lead to an increase in variance, not bias.

crates/bevy_solari/src/pathtracer/pathtracer.wgsl

atlv24 · 2025-05-06T06:11:19Z

crates/bevy_solari/src/lib.rs

+
+impl SolariPlugin {
+    /// [`WgpuFeatures`] required for this plugin to function.
+    pub fn required_wgpu_features() -> WgpuFeatures {


(unrelated) we're really overdue for structured plugins. this is the kind of thing that should be centralized by a trait and collected at initialization to request the minimal feature set.

atlv24

Reviewed only the rust side for now, will do wgsl later. Looks good so far!

atlv24 · 2025-05-06T07:32:23Z

release-content/release-notes/bevy_solari.md

+pull_requests: [19058]
+---
+
+(TODO: Embed solari example screenshot here)


this TODO and the one below needs resolving

Not in this PR. Instructions say not to embed images now, and that we'll do that once the release notes are transferred to the bevy website repo for the release. Otherwise we bloat this repo with multimedia.

atlv24 · 2025-05-07T23:44:04Z

Cargo.toml

@@ -4328,3 +4349,6 @@ name = "Extended Bindless Material"
 description = "Demonstrates bindless `ExtendedMaterial`"
 category = "Shaders"
 wasm = false
+
+[patch.crates-io]
+naga_oil = { git = "https://github.com/JMS55/naga_oil", rev = "82476f83023da017be4963eaafc0f7b30e811bc3" }


we're gonna need to wait until we can use a published version here

atlv24 · 2025-05-07T23:45:50Z

crates/bevy_mesh/src/mesh.rs

@@ -203,6 +205,7 @@ impl Mesh {
            morph_targets: None,
            morph_target_names: None,
            asset_usage,
+            enable_raytracing: true,


Does this being true mean its building a BLAS for all meshes even when bevy_solari is not in use?

How about instead, defaulting to false, and having a method on Mesh which checks the set of vertex attributes is {{POSITION, NORMAL, UV_0, TANGENT}} before setting the bool to true, and returns an error otherwise?

Does this being true mean its building a BLAS for all meshes even when bevy_solari is not in use?

Depends by what you mean by "not in use". If the solari scene plugin is added to the app, your GPU supports RT, and the mesh is compatible and has enable_raytracing: true, then yes the mesh will get a BLAS built.

If the plugin is not loaded (it's not part of default apps) / your GPU does not support RT, or the mesh is not compatible or has RT off, then no a BLAS won't be built.

How about instead, defaulting to false, and having a method on Mesh which checks the set of vertex attributes is {{POSITION, NORMAL, UV_0, TANGENT}} before setting the bool to true, and returns an error otherwise?

The raytracing_enabled field is not meant to be "whether or not the mesh is compatible for RT". The field is meant for cases where you don't want to build a BLAS for a mesh even if it's compatible. E.g. for higher level LODs if you're using exclusively lower level LODs for RT to reduce BLAS and traversal cost. I'll update the docs to make this explicit.

crates/bevy_solari/src/pathtracer/node.rs

atlv24 · 2025-05-08T00:28:41Z

crates/bevy_solari/src/scene/binder.rs

+                    min_binding_size: None,
+                },
+                count: MAX_MESH_COUNT,
+            },


I thought there was a shorthand api for this

Not for entries with count, idt. If someone knows otherwise, let me know.

Co-authored-by: atlv <email@atlasdostal.com>

atlv24

Very cool, good job

atlv24 · 2025-05-17T02:46:27Z

crates/bevy_solari/src/scene/sampling.wgsl

+fn sample_cosine_hemisphere(normal: vec3<f32>, rng: ptr<function, u32>) -> vec3<f32> {
+    let cos_theta = 1.0 - 2.0 * rand_f(rng);
+    let phi = 2.0 * PI * rand_f(rng);
+    let sin_theta = sqrt(max(1.0 - cos_theta * cos_theta, 0.0));


i assume the max accounts for the precision problems the reference describes

Must be. I wrote this code a few years ago, tbh it's just copied from an older version of solari so I no longer remember.

atlv24 · 2025-05-17T03:01:54Z

crates/bevy_solari/src/scene/raytracing_scene_bindings.wgsl

+    world_normal: vec3<f32>,
+    geometric_world_normal: vec3<f32>,


docs here one what the distinction is would be good: im guessing world_normal is the normal as dictated by the mesh, with vertex interpolation, and geometric_world_normal is the normal of the triangle.

Yeah basically.

I'm not going to spend time documenting it because this is basically all subject to change. I haven't decided exactly what the APIs are going to look like yet because some passes probably don't even care about normals, or materials, and for shadow rays we don't have BVH intersections and have to use vertex and index buffers where for other rays we can get the positions directly from the BVH, etc.

I plan to figure things out over time and write the APIs as I need them.

atlv24 · 2025-05-17T03:05:22Z

crates/bevy_solari/src/scene/raytracing_scene_bindings.wgsl

+
+    let local_normal = mat3x3(vertices[0].normal, vertices[1].normal, vertices[2].normal) * barycentrics; // TODO: Use barycentric lerp, ray_hit.object_to_world, cross product geo normal
+    var world_normal = normalize(mat3x3(transform[0].xyz, transform[1].xyz, transform[2].xyz) * local_normal);
+    let geometric_world_normal = world_normal;


is this assuming "flat" normals? should it be taking a cross product of the edges instead?

Not sure, I wrote this code a while ago. But I think I had tested on bistro at some point and it looked fine.

atlv24 · 2025-05-17T03:07:13Z

crates/bevy_solari/src/pathtracer/pathtracer.wgsl

+
+    // Setup RNG
+    let pixel_index = global_id.x + global_id.y * u32(view.viewport.z);
+    let frame_index = u32(old_color.a) * 5782582u;


this doesn't look like its a frame index, ~~plus isn't old_color.a in [0..1]? so casting it to u32 is basically always zero? so this is not really contributing anything~~ i see old_color.a is the frame count, perhaps move the multiplication down into the rng initialization instead so that the name makes sense

JMS55 added 30 commits March 26, 2025 22:57

WIP

e28c8c8

Add logo

f42c25b

Guard extra buffer usages behind a cargo feature

58507ba

Better pattern to contain RT buffer usages entirely within bevy_solari

fdf123f

Add extra mesh allocator buffer usages instead of overwriting

f9e5aa2

Merge commit '301f61845a26107ab36f83d2702f471642be25b1' into solari6

cfb1ba2

Add BLAS creation code

92997a0

WIP scene bindings

7224fcd

Add binding array helpers

e7b9aa5

WIP scene binder

c9dd3ab

WIP scene binding

c635e31

WIP example setup

182e88b

Misc

27abb14

Bugfix

135bcb4

WIP pathtracer

e9ea256

Fix?

324f1be

Switch to uploading geometry slices instead of descriptors

491effd

Fixes

60d86b3

Use patched naga_oil

e60addf

Scene binder WIP

e387883

Simplify code

231ca08

Add material bindings

0b0db2c

Misc reorder

5ff7c4d

Fix mesh binding array shader types

7ebea4a

Merge commit '73f03327c234f935ba7328220f6a25e3b33e1899' into solari6

45bd908

Note down some bugs

f0e13b2

Note that RaytracingMesh3d cannot be used standalone

f2873c0

Add missing import

c8eddb3

Reset pathtracer accumulation on view

6e78a94

Hide modules

d4e4dba

JMS55 added 11 commits April 29, 2025 20:59

Rename irradiance to radiance in the path tracer, since it's a single…

7bc608f

… sample of the integral

Adjust RAY_T_MIN

b9134ee

Fix directional light units

5f79f7a

Misc

4c6abfe

Rename directional light color to illuminance

49166b1

Merge commit 'cd67bac544a874f668c706b837c8b3093dfb9545' into solari6

a4ef1f0

Make sun size configurable

044474a

Fix incorrect throughput calculation, add comments

f26e0c1

Remove russian roulette bias

7dc5458

Change example clear color for raster

d25d52e

Misc

9adcdd0

github-project-automation bot added this to Rendering May 4, 2025

alice-i-cecile added this to the 0.17 milestone May 4, 2025

Add release notes

2dc2589

JMS55 marked this pull request as ready for review May 4, 2025 18:06

alice-i-cecile added the D-Complex Quite challenging from either a design or technical perspective. Ask for help! label May 4, 2025

Fix release notes

eb48343

SparkyPotato reviewed May 4, 2025

View reviewed changes

JMS55 added 2 commits May 4, 2025 12:50

Fix overflows with high sample counts

d80f376

Specify commit of forked naga_oil

7753735

atlv24 reviewed May 6, 2025

View reviewed changes

atlv24 reviewed May 8, 2025

View reviewed changes

JMS55 and others added 2 commits May 8, 2025 18:38

Update crates/bevy_solari/src/pathtracer/node.rs

51c5f56

Co-authored-by: atlv <email@atlasdostal.com>

Add more docs to Mesh::enable_raytracing

86f2aa2

atlv24 approved these changes May 17, 2025

View reviewed changes

janhohenheim added the S-Needs-Review Needs reviewer attention (from anyone!) to move forward label May 17, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Initial raytraced lighting progress (bevy_solari) #19058

Initial raytraced lighting progress (bevy_solari) #19058

JMS55 commented May 4, 2025 •

edited

Loading

SparkyPotato May 4, 2025

JMS55 May 4, 2025

SparkyPotato May 4, 2025

JMS55 May 4, 2025

SparkyPotato May 4, 2025

JMS55 May 4, 2025

JMS55 May 4, 2025

SparkyPotato May 5, 2025

atlv24 May 6, 2025

atlv24 left a comment

atlv24 May 6, 2025

JMS55 May 9, 2025 •

edited

Loading

atlv24 May 7, 2025

atlv24 May 7, 2025

atlv24 May 7, 2025

JMS55 May 9, 2025

atlv24 May 8, 2025

JMS55 May 9, 2025

atlv24 left a comment

atlv24 May 17, 2025

JMS55 May 17, 2025

atlv24 May 17, 2025

JMS55 May 17, 2025

atlv24 May 17, 2025

JMS55 May 17, 2025

atlv24 May 17, 2025

Initial raytraced lighting progress (bevy_solari) #19058

Are you sure you want to change the base?

Initial raytraced lighting progress (bevy_solari) #19058

Conversation

JMS55 commented May 4, 2025 • edited Loading

Bevy Solari

Preface

Connections

This PR

What's implemented?

What's not implemented?

End-user Usage

Testing

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

atlv24 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JMS55 May 9, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

atlv24 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JMS55 commented May 4, 2025 •

edited

Loading

JMS55 May 9, 2025 •

edited

Loading