Forum

#1 2019-07-15 12:50:35

Narann
Member

Instances: Connections VS RenderGraph ArchRef

Hello devs!

We want to provide a way to support a massive amount of instances and I'm facing some potential problems I want to discuss with you.

Notice I didn't test anything massive enough to assume anything. Those are mostly guessing on how Guerilla would behave.

1) The "classic" way to do instancing is to connect the Instances plug. For each transform nodes. This would generate a huge amount of connections.

I suspect this would increase gproject save and loading times a lot. We already use this on our current pipeline and the gproject text file is 90-95% of depend("$40.Instances","$38|foo_000:Root.Instances") lines.

Are my assumptions correct?

2) The "new" way to handle instancing is to use a RenderGraph. With that, I can have a SceneGraphNode named destination with empty nodes in it foo_001, foo_002, foo_003, etc…

And, in the RenderGrapth a Path node with a simple "destination|foo_.+" in it to generate instances under every foo_ node.

But this could make Guerilla crazy if, for a reason or anothe RenderGraph need to be evaluated.

So I don't know what to choose.

3) Which method did you use to handle the Mona Island data set?

Any help is welcome.

A big thanks in advance.

Offline

 

#2 2019-07-16 10:39:28

doubleailes
Guerilla dev, the guy to hug

Re: Instances: Connections VS RenderGraph ArchRef

Hi Narann,

In the Moana Island data set, i use an houndini point cloud. From the json database i extracted all the 4x4 matrix, convert it to a particle with a quaternion and add a id to indicate the object linked to it.
In guerilla, i used a dynamic attributes "ObjectId" to connect the right asset to the right particle.

Here is a video about it:
https://youtu.be/XeX1YfXIA_o

Moana instances were more than 60M ( 20M grain of sand ), which is massive IMHO.

For a massive amount i would not recomanded the 1), tried it with moana and i failed. And of course the 2) seems too expensive for a big set.

The fourth method ( 4) ) can be the spawn node, but i never used it for a big set, but i think it will be more efficient than the 2)

There is a 5) in the 2.2 wink

cheers.
P.


Philippe Llerena

Offline

 

#3 2019-07-16 11:22:39

Ben
Guerilla dev, the guy to hug

Re: Instances: Connections VS RenderGraph ArchRef

Hi Narann,

I second doubleailes on this. 1 and 2 will basically generate the same scene tree, with the difference of 1 resulting in a huge gproject file.
2 requires you to have all instances roots in the scene tree. This is clearly not adequate when dealing with 10+ million instances.

The point cloud solution is definitely the way to go if you're looking for 10M, 100M instances. Moreover, it's going to be much faster because there's no more scene tree traversal at this point, and instances are roughly duplicated instead of traversed.

Ben


Benjamin 'Ben' Legros
Guerilla developer
http://www.guerillarender.com

Offline

 

#4 2019-07-16 11:36:07

Narann
Member

Re: Instances: Connections VS RenderGraph ArchRef

doubleailes :

There is a 5) in the 2.2 wink

What a teasing !

Thanks ! I didn't think about spawn but it makes more sense ! I will try.

What do you use for non-Mona Island sets (aka big sets but not too much) ? Still particles ? Brutal Alembic ? (looks like a rock metal group name)

Edit to Ben:

The problem with Alembic point cloud is that it make our pipeline heavier. For now, our "massive instances" are JSON files. No Alembic involved (this mean, no Maya either). This allow us to let artists modifies per instance property in Guerilla and republish a JSON file.

The text base approach is more easier to manipulates from a pipeline perspective.

If point cloud is the way to go. Do you think Partio could be an option ? It could be easier for us to convert our JSON structure to Partio than Alembic.

Thank you guys !

Last edited by Narann (2019-07-16 12:12:19)

Offline

 

#5 2019-07-16 13:05:02

Ben
Guerilla dev, the guy to hug

Re: Instances: Connections VS RenderGraph ArchRef

Well, if you've got the JSON (which I assume contains placement matrices) you "could" implement your own archive reader (in C++) which would translate your JSON to a point cloud.

This means you need a C++ plugin for that, but if your format is not too verbose, this should not be overwhelmingly difficult.


Benjamin 'Ben' Legros
Guerilla developer
http://www.guerillarender.com

Offline

 

#6 2019-07-16 13:25:52

doubleailes
Guerilla dev, the guy to hug

Re: Instances: Connections VS RenderGraph ArchRef

I send you an exemple of python solution to read json.


Philippe Llerena

Offline

 

#7 2019-07-16 19:44:49

Narann
Member

Re: Instances: Connections VS RenderGraph ArchRef

Thanks Ben !

We support both Windows and Linux so C++ is, unfortunately, an extensive option.

I will look into your gproject asap, thanks !

Offline

 

#8 2019-07-31 18:55:00

Fulcrum35
New member

Re: Instances: Connections VS RenderGraph ArchRef

Thanks for the advice regarding the sand particle clouds, I was kinda stuck with a project of a short animation involving something resembling these house in Larnaca  in a post apocalyptic setting, overrun by sand, and was at a loss trying to animate said sand.

Last edited by Fulcrum35 (2019-08-06 15:12:08)

Offline

 

Board footer

Powered by PunBB
© Copyright 2002–2008 PunBB