I can see that relighting is still a work in progress, as the virtual spot lights tends to look flat and fake.
I understand that you are just making brighter splats that fall inside the spotlight cone and darker the ones behind lots of splats.
Do you know if there are plans for gaussian splats to capture unlit albedo, roughness and metalness? So we can relight in a more realistic manner?
Also, environment radiosity doesnt seem to translate to the splats, am I right?
There are many ways to relight Gaussian splats. However, the highest quality results are currently coming from raytracing/path tracing render engines (such as Octane and VRay), with 2D diffusion models in second place. Relighting with GSOPs nodes does not yield as high quality, but can be baked into the model and exported elsewhere. This is the only approach that stores the relit information in the original splat scene.
That said, you are correct that in order to relight more accurately, we need material properties encoded in the splats as well. I believe this will come sooner than later with inverse rendering and material decomposition, or technology like Beeble Switchlight (https://beeble.ai). This data can ultimately be predicted from multiple views and trained into the splats.
"Also, environment radiosity doesnt seem to translate to the splats, am I right?"
Splats do not have their own radiosity in that sense, but if you have a virtual environment, its radiosity can be translated to the splats.
Back in 2001 I was the math consultant for "A Beautiful Mind". One spends a lot of time waiting on a film set. Eventually one wonders why.
The majority of wait time was the cinematographer lighting each scene. I imagined a workflow where secondary digital cameras captured 3D information, and all lighting took place in post production. Film productions hemorrhage money by the second; this would be a massive cost saving.
I described this idea to a venture capitalist friend, who concluded one already needed to be a player to pull this off. I mentioned this to an acquaintance at Pixar (a logical player) and they went silent.
Still, we don't shoot movies this way. Not there yet...
the problem with SDF engines is that you have to reinvent everything, as current pipelines rely on triangles.
That means:
- Software to model using SDF (like Womp)
- Technique to animate skeletons using SDFs
- Tool to procedural texture surfaces using SDFs
At least he solved the physics part, which is also complex.
And also, his way of carving is by instantiating new elements, which works for small carves, but if you plan to have lots of tunels, then the number of instances is going to skyrocket.
Although a decent chunk of modern tooling is there to handle the limitations of triangles. And modelling is often using higher-level abstractions that are only turned into triangles at the end of the process.
It´s about normalizing something we think it could lead to problems.
I dont want to romanticize the game in his mind, so when he grows up people ask him to play poker and he sees it as "that nice game we played at home!".
It is a game with very strong connections with gambling. There are thousands of other games without that association which are as rewarding as poker.
park chess players gamble.
Going to a chess tournament with a prize pool and paying an entry fee is gambling.
In germany magic the gathering tournaments are banned since they are deemed as gambling.
Looks like a repurposed VitePress docs template, which is a perfectly fine solution for text-heavy content. The site appears to be open-source, there are links to the repo at the bottom of each page: https://github.com/xiaoiver/infinite-canvas-tutorial
As a Software Engineer I found it hard to grasp the concepts explained here.
First it says we lose electrons by deleting information. But AFAIK we are losing electrons everywhere, most gates will operate on negation of a current, which I understand is what they refeer to losing electrons. So, are all gates bad now?
Also, why keeping a history of all memory changes will prevent losing heat? You will have to keep all that memory running so...
And finally, why would this be useful? Who needs to go back in time in their computations??
Theoretically, a computer that never forgets anything can run without consuming any power (and thus never heating). That kind of computer would be called reversible (or adiabatic) as it would require its gates to be reversible (i.e. any computation can be undone). You would still need to expend energy to set the initial state (input) and copy the result (output).
Obviously, in real life, most power consumed by computers is lost by wire resistance, not through "forgetting" memory in logic gates. You would need superconducting wires and gates to build an actually reversible CPU.
Also, you would need to "uncompute" the result of a computation to bring back your reversible computer from its result back to its initial state, which may be problematic. Or you can expend energy to erase the state.
Quantum computers are reversible computers, if you seek a real life example. Quantum logic gates are reversible and can all be inverted.
It's a thermodynamics thing. Reversible processes are the most efficient (something to do with entropy). Deleting information means it's no longer reversible. This is an entirely theoretical thing. There are theoretical limits to energy usage of computation based on this, but actual computers are nowhere near these theoretical limits, at all.
Edit: and yes, most of the logical operations in a regular chip like AND, OR, NAND etc are irreversible (in isolation, anyway)
> but actual computers are nowhere near these theoretical limits, at all.
The Landauer limit at ambient temperature gives something of the order of 10⁻²¹ J to irreversibly flip a bit. While, if I read this paper[1] correctly, current transistors are around 10⁻¹⁵ J. So, definitely not coming to AI "soon".
> Also, why keeping a history of all memory changes will prevent losing heat?
How much power does a persistent storage (hard drive, SSD) require to preserve its stored data? Zero, which is why it emits zero heat.
> Who needs to go back in time in their computations??
At its most basic level, erasing/overwriting data requires energy. This generates a lot of heat. Heat dissipation is a major obstacle to scaling chips down even further. If you can design a computer that doesn't need to erase nearly as much data, you generate orders of magnitude less heat, and this potentially opens up more scaling potential and considerable power savings.
On one side Im amazed by the amount of good work you have done, but on the other, I feel it lacks true useful scenarios.
For instance, I would love to install it in my server to handle my own server files, but it doesnt support mounting a folder to access from the OS.
Or I would love to have an SSH client, or a terminal that is executed in the server, to run my own nodejs apps.
Also some form of login/pass would be helpfull in case somebody got access to the URL.
But none of them are available.
I understand than the goal was to see whats possible in a browser, but to make it more appealing to people I would love to see some real usecases covered.
It does indeed have a long way to go if I want to add usefulness beyond my personal website use case.
It is possible to "map" a local drive in several ways, such as via the desktop right click menu's "Map directory", and also if you built it locally you could put a folder in "public" which would then be mapped.
But it isn't the same as what you are describing and that would likely require a server component so I could bypass the limits of the browser.
I hope to eventually have an answer for all the use cases you mention, but because this is just a side project and my main goal is a client-side personal website, I don't expect those features in the 2020's, maybe the 2030's. :-)
I can see that relighting is still a work in progress, as the virtual spot lights tends to look flat and fake. I understand that you are just making brighter splats that fall inside the spotlight cone and darker the ones behind lots of splats.
Do you know if there are plans for gaussian splats to capture unlit albedo, roughness and metalness? So we can relight in a more realistic manner?
Also, environment radiosity doesnt seem to translate to the splats, am I right?
Thanks