WIP: My approach to a (poor man's?) particle system for Fabric

malbrechtmalbrecht Fabric for Houdini Posts: 752 ✭✭✭

Moin,

as I have stated elsewhere, I am (too slowly) in the process of creating some basic "Particle System" for my own Fabric experiments. Ideas on this as well as ways to display particles in DCCs have been discussed here: http://forums.fabricengine.com/discussion/comment/673/

Unfortunately my time is very limited, so I have no idea how long it will take me to make this thing usable, but maybe others are interested in chiming in?
I will update this thread whenever I have something new to suggest/discuss.

One question I'd like to tackle right away: What is the best way to hook in callback functions? What I would like to be able to do is define callback functions for "collision", "rest", "influence radius entered" or whatever on the particle "control" instance (which I currently call a particleSimulation).
I have seen some code fragments that in fact simply stated a user defined function's name as value in an object's property - is that allowed?

Marc


Marc Albrecht - marc-albrecht.de - does things.

«13

Comments

  • malbrechtmalbrecht Fabric for Houdini Posts: 752 ✭✭✭
    edited February 2016

    Based on the discussion mentioned above, this is the current definition of a particle and a simulation. Note that this is untested, just a "note down" from the discussion, I am working on the needed core functions (like initializing the clusters based on the simulation radius etc.) - I do have some constructors laid out, but I'd like to have some feedback on whether this makes sense so far before posting too much.


    /*
      WIP class for a flexible particle system in Fabric
      Note that most, if not all, of this may, will or has to change.
      If you make enhancements to this (like deleting and rewriting it from scratch),
      please inform malbrecht@marc-albrecht.de
    */
    
    /// \cond
    require Math;
    /// \endcond
    
    /**
     global particle simulation variables and, eventually, functions
    */
    object maParticleSimulation
    {
      Boolean active; // global activation state to allow "freeze" states
      Vec3 position; // initial position of newly created particles (do we need this?)
      Vec3 gravity; // direction and strength (in m/s^2) of gravity
      Float32 mass; // initial mass for new particles in kg (should this be weight in Newton?)
      Size clusters; // number of segments per axis to limit neighborhood searches ("octree mokup"), the actual amount of clusters is this value^3 (three axises)
      UInt32 cluster[][]; // Array of arrays of particle IDs. Note that a particle (ID) can get listed in more than one cluster, if it is less than "influence" away from the respective border
      Float32 framespersecond; // helper for simple access to frames resolution
      Vec3 farest; // position of the farest particle in this simulation, used to update the clusters
      } 
    
    /**
      particle object that holds all information relevant to a single point in a simulation
    */
    object maParticle   
    {
      Boolean active; // is this particle even alive?
      Vec3 position; // position in space 
      Quat rotation; // rotation as Quat to allow for direction dependent rotation
      Vec3 scaling; // scaling as Vec3 to allow for deformation data in all axises
      Vec3 velocity; // velocity in m/s
      Float32 mass; // in kg (should this be changed to weight in Newton?) 
      Float32 attraction; // used for stickyness, glueing, negative values for repulsion, this value means "strength"
      Float32 influence; // used in conjunction with attraction as a radius around the point measured in m. "attraction" is weakened in 1/m^2 (should this only be stored as a global to save space?)
      UInt32 age; // ticked in frames
      Float32 temperature; // could be used for various other things. Like ... heat. Or energy stored in motion.  
      Float32 cooling; // temperature decrease factor. Per frame. Could be used for cooling. Or loss of heat or so.
    
      UInt32 cluster; // "home" cluster this particle is in, used to minimize calculations on position versus cluster - note that a particle can be listed in up to four clusters if it is closer than the "influence" distance to a border!
    };
    

    Marc Albrecht - marc-albrecht.de - does things.

  • yamahigashiyamahigashi Posts: 26

    Moin moin,

    I've written such verlet particle solver before( FE 1.15), i wrote this for rigging SpringStrand ( ICE vs Splice ) and I'll share this. that is still wip and also not integrated with canvas, but i hope this will help.

    https://github.com/yamahigashi/FabricCPS

  • malbrechtmalbrecht Fabric for Houdini Posts: 752 ✭✭✭
    edited February 2016

    おはよう

    thanks a lot, this will definitely be very helpful!

    I quickly browsed over the files and would like to ask: I thought about storing previous positions as well, but decided against it, as Fabric's bullet implementation right now does not allow to "go back in time" either, so users probably are OK with baking simulated animations anyway. For calculation purpose the current velocity in each particle should suffice, I think, as it should get updated when forces influence the particles direction.
    My main reason against storing additional positions was that it takes up a lot of memory. If you are dealing with 10 Million particles, depending on math precision and storage needed for that, you may need around 230MB just for a single additional position.
    So my actual question is: What were the reasons for you for storing previous and initial positions? If there are good arguments, I definitely have to copy that ^_^

    Initial positions - I think - should be set/created outside the simulation, as the simulation should get reset on its start and only after the reset initials can be changed. So, likewise, my question here is: What is the advantage of storing the initial positions?

    Right now I feel like not using geometry inside the simulation at all, but better dealing with "abstract" data only. My plan is to provide export as "XFO", so that a "CloneMeshInstance" can be used based on the particle-simulation created field of positions, giving users an instant "replicator" use instead of ready-baked geometry. Elseway, a pointcloud could be created/exported or a default particle shape (geometry) could be pushed into a big geometry cluster by the simulation as a "service". I think this approach is more flexible than having particles "cemented" in a geometry from the start.
    Does this make sense or am I thinking too abstract?

    Marc


    Marc Albrecht - marc-albrecht.de - does things.

  • yamahigashiyamahigashi Posts: 26

    when i wrote that, I estimated particles count about 10 - 100 order (and intended for "real time use" purpose). So achiving 10m order, there are more better approach may be. (over 2k particles on GPU make my pc crash, on CPU a bit more ok)

    1. storing "previous position"

    that is for verlet is that will be necessary both "prev pos" and "velocity" in the process ( also needed in appling force and constraint phase). If there is one var thus another one is determineable of course, It seems a bit waste but allocating these initially and storing both make better performance.

    2. storing "initial state"

    This is for "satisfy constraint" phase. Spring behaviour needs initial (=rest) position. And I want to manipulate these positions manually at runtime. thus General particle doesn't needs this actually, i think.

    3. particle's data type

    Using general type for particle has some pros points i think. reuseable, like "brush" feature that is includede in samples already (but its not unfinished in my code haha)
    imho, for representing what "look like something", that is better using the "Something Type" when available. In this case, particle looks like Points for me (pointcloud is more better). Geometry types can have additional attributes and that is very convenient.

    But it may be "too big" is that using these types, so never know which is better until done it. in my case, abstract data only way make not much differnt so I chose deriving the Points type.

    Export result

    Elseway, a pointcloud could be created/exported or a default particle shape (geometry) could be pushed into a big geometry cluster by the simulation as a "service". I think this approach is more flexible than having particles "cemented" in a geometry from the start.

    yeah, i agree

  • Roy NieterauRoy Nieterau Posts: 258 ✭✭✭
    edited February 2016

    I think a Particle system is best implemented using the Points datatype that's already there and store the required values like velocity as its Attributes. This also means that the amount of Attributes can grow/shrink depending on your needs.

    At the same time you keep access to the drawing methods of its GeometryAttributes and its SpatialQuery support.

    Actually just now noticed @yamahigashi also mentioned this with "particle looks like Points for me".

  • malbrechtmalbrecht Fabric for Houdini Posts: 752 ✭✭✭

    Hi,

    thanks - I'll think about that. What I do not fully grasp, yet, is how I would go and create additional properties on the Points object. Sure, I could have an array of Points in the simulation-container (array) and additional arrays beside that - but that's convoluted and not what I am looking for as a "flexible" solution.
    How would I add properties to the existing Point datatype - basically inheriting the Point class and adding my own stuff in KL?

    Marc


    Marc Albrecht - marc-albrecht.de - does things.

  • HelgeHelge Moderator, Fabric Employee Posts: 314 Fabric Employee

    Hey Marc,

    with the new attribute presets in the next release that will be trivial to do. So just hang in there for a small while. With these presets you'll be able to define any kind of property on the fly, pull the data out, and put it back in. You'll also be able to perform simulation easily, so it sounds like Roy is spot on here: You shall build your particle system based on the Points type.

    I hope this helps!

    Research Engineer @ Fabric Software

  • HelgeHelge Moderator, Fabric Employee Posts: 314 Fabric Employee

    As another side note: For small data structures like a particle in this case you shouldn't use an object but a struct. Structs are tightly packed and not reference counted, so much faster to pass around and easier to process. Furthermore, if you decide to use the GPU later for compute, structs are fully supported on the GPU, while objects are not.

    Research Engineer @ Fabric Software

  • malbrechtmalbrecht Fabric for Houdini Posts: 752 ✭✭✭

    Hi, Helge,

    So just hang in there for a small while.

    ... that I do quite convincingly! :)

    There's enough stuff to do around the basic datatype (like getting all the initializer and calculation routines figured out), so I'll happily fiddle around with my "quick'n dirty" scenario and shift everything over to the real deal, as soon as that's being dealt.
    So .... hanging in, but not doing nothing while doing that ...

    Marc


    Marc Albrecht - marc-albrecht.de - does things.

  • malbrechtmalbrecht Fabric for Houdini Posts: 752 ✭✭✭

    ... and on the second comment: Structs versus objects - the reason for going with an object was my wish to use callback functions. I don't know if that is even possible. If NOT, a struct is just fine for me.

    Marc


    Marc Albrecht - marc-albrecht.de - does things.

  • HelgeHelge Moderator, Fabric Employee Posts: 314 Fabric Employee

    It certainly is possible, just that the data type won't store them. I suggest to go with

    a) a fast and tiny data structure (struct) to store the data
    b) KL interfaces for evaluating things (evaluators), which can be implemented as objects

    I hope this makes sense. Please share further thoughts and experiments! I am also willing to share some stuff in case you guys are interested. But let's see some of your own results first :smile:

    Research Engineer @ Fabric Software

  • malbrechtmalbrecht Fabric for Houdini Posts: 752 ✭✭✭

    see some of your own results first

    Good idea.

    My priorities on this project are:

    • live, earn the money to do that
    • do things I love to do, earn the money to do that
    • continue beta-ing (experimenting, trying to break) Fabric in modo
    • do experiments outside modo, but inside Fabric
    • continue work on the particle system as part of the above, but currently not connected to "earn the money to do that"

    So ... I'll continue posting here and thank you all for the input - it may take some time before I can show some impressive dust clouds settling on a sandy beach being rolled over by a perfect fluid simulation mimicing the Northsea.

    Marc


    Marc Albrecht - marc-albrecht.de - does things.

  • HelgeHelge Moderator, Fabric Employee Posts: 314 Fabric Employee

    :smile: like I stated before - I am willing to help. With the new work we've put into attribute presets it won't be long before the dust clouds settle. :wink:

    Research Engineer @ Fabric Software

  • malbrechtmalbrecht Fabric for Houdini Posts: 752 ✭✭✭

    Waiting for a call from a client I tried another run through geometry attributes - am I right to assume that I can simply do this:

    • create a point (...cloud)
    • GetOrCreateVec3Attribute("velocity") on that one
    • continue with whatever properties I deem delicious
    • plug that point cloud into my simulation container

    if so - I'll try to rewobble what I have into this framework. And: What about callbacks? Is that something to hang in on or am I simply too blond again?

    Marc


    Marc Albrecht - marc-albrecht.de - does things.

  • HelgeHelge Moderator, Fabric Employee Posts: 314 Fabric Employee

    Can you describe what you are trying to achieve with what you refer to as callbacks please?

    Research Engineer @ Fabric Software

  • malbrechtmalbrecht Fabric for Houdini Posts: 752 ✭✭✭

    Can you describe what you are trying to achieve with what you refer to as callbacks please?

    Sorry, yes, I was afraid I was already too elaborate :)

    I want the sim-system to provide basic simulation functions like "please now move all the points according to their velocity vectors, take the known forces/constrains into consideration and create the new positions". That's a method of the simulation class, obviously. But then I'd like to have a trigger telling that method to do basic collision checks (I have those "influence radii" that aren't exactly collisions, but particles may enter the influence area of other particles).
    When the simulation method finds (because the trigger told it to look out for that) "collisions", it should be able to call user-defined functions to deal with those.

    The idea is to save as much back-and-forth and re-iterating over the point cloud for additional checks as possible. So a "callback" or "hook function" could be defined by whoever is, e.g., creating a new solver.

    Example: Creating a sand solver you will probably have "guide particles" and "secondary particles". When a secondary particle enters the influence area of a guide particle, the simulation (going over the particle cloud anyway) could immediately call the hook/callback to do whatever is wanted (like tagging the particle in a specific way).
    Or, even simpler, the particle falls under a defined y-value (i.e. "hits the ground"). The callback might give it a mirrored motion vector, decreased by some dampening factor.

    Having this kind of "plugged in" callbacks writing solvers would be easier and those might run faster than if you had to iterate over all points several times (simulation once or twice, solver again and maybe again).

    Does this make sense?

    Marc


    Marc Albrecht - marc-albrecht.de - does things.

  • Roy NieterauRoy Nieterau Posts: 258 ✭✭✭
    edited February 2016

    Actually what you're going for sounds similar to stacking functionality triggering one after the other. Like a deformation stack (I think Max does that).

    Anyway, a node graph can act like such a stack offering great control but also complexity and as such isn't as fool proof.

    A stack could be mimicked using an array of DFGbindings where one is called after the other with the points as arguments plus whatever the function requires to do its thing like damping or for a target for some flocking or steering behavior.

    I haven't jumped into DFGBindings but I'm assuming this is how they could be used. With the current DFGBindings it means you'll process each step one after the other like:

    • damping
    • attract to point
    • flocking steering behaviour

    Once a form of "for loop" blocks are in FE this could become even a "per point operator stack".

    Edit: Actually I think per point processing could even be possible with the DFGBinding if you make bindings that operate on a single point and put those in a stack that applies these bindings per point. (Again, not completely sure it works like this)

    Also you were spot on @malbrecht with how you would add/retrieve data like that. @Helge seems to say this is even easier with the new attribute presets (are these public already?). Anyway, the point is that you'd add the point attribute only when you need it in your stack. Does that make sense?

    typed on phone, sorry for any possible typos

  • malbrechtmalbrecht Fabric for Houdini Posts: 752 ✭✭✭

    Moin, @Roy Nieterau ,

    yeah - I was able to hack together a quick graph while talking to a client on the phone last night. It works the way outlined above, adding Attributes to the Point cloud. I am now creating the simulation "container" only as a struct (defined as an extension) and that also seems to work (yip yip!). So, in theory, I should be able to create a dummy solver next.
    Got some refugee-helping work over the next couple of days, but I hope to get the basic sim functions put in over the weekend, so maybe I can get a first "cloud of dust settling on a ground plane" over the weekend.

    Marc

    P.S. I really hope this community can keep up its current sense of cooperation. I am really grateful for the fast and serious help I am getting here with all my crazy experiments and can only help that some of it might help others, if not at least inspire them to do better! ;-)


    Marc Albrecht - marc-albrecht.de - does things.

  • craouettecraouette Posts: 113

    Hi,

    may I join? I was thinking too to make a particle system with Fabric... I already went for the Points and attributes, but I think something is missing: "Global" attributes. For a particle system, we will need to have global attributes, for example all particles have the same mass, so store it once, use it everywhere. To be opposed to particle dependent attributes (the one already supported in the daily builds). But from the point of view of the algorithm using attributes, global attribute or a per particle attribute won't be important, just call the function massAtt= getFloat32Attribute("mass") to retrieve the attribute (global or per particle) and use it with m = massAtt[index].

    The second point is how to manage live and death of particles... A big array with an "active" flag per particle can be used or an array with the size changing according to the number of active particles can be used.... the 1st one have the drawback that if 10 particles are alive on an array of 1 000 000, the 1 000 000 particle should be processed to know only 10 are alive.... the 2nd has the drawback of moving data around each time a particle died or is created... may be there is a better option?

    Pierre

  • HelgeHelge Moderator, Fabric Employee Posts: 314 Fabric Employee

    Hey Pierre,

    thanks for your input! Two suggestions here:

    • Go with a data structure based on the Points, which has a secondary GeometryAttributes member for "global" or "singleton" attributes.
    • Have also an "activeIndices" UInt32[] index list which is always used to perform the work, so when a particle dies you switch its ID with the last one and shrink the array by one. For example. Newly emitted indices would then just have to be appended to that list.

    Research Engineer @ Fabric Software

  • craouettecraouette Posts: 113

    Hey Helge,

    For the second, yep... should have think of it myself!!!
    for the first one, will this allow to access an attribute independently of its type (singleton/per particle)? Ok, it is simple to overwrite the getAttribute function, to look for the geometricAttributes of Points first and then the singleton ones if nothing is found in the 1st step... but is it possible to overwrite the array access?
    otherwise, it is possible to always access attibute arrays with m = massAtt.values[max(index, massAtt.size())]....

  • HelgeHelge Moderator, Fabric Employee Posts: 314 Fabric Employee

    Hey Craouette,

    well you'd add new methods. I suggest to have something like getGlobalAttribute / setGlobalAttribute or so.

    Research Engineer @ Fabric Software

  • malbrechtmalbrecht Fabric for Houdini Posts: 752 ✭✭✭

    Hi, Pierre,

    thanks for joining in. The global structure I am using is currently named "maParticleSimulation", in this I store global defaults (like initial positions, masses etc) as well as an on-off-switch. Each particle has its own "active" state as well, so it can "die". But I think Helge's suggestion of having an additional "activeParticles" array is a good helper for those cases where, like you said, only a very few out of many are still active.

    I'll try to post some WIP tonight ...

    Marc


    Marc Albrecht - marc-albrecht.de - does things.

  • craouettecraouette Posts: 113

    I think Helge proposed something like this:

    require Math, FabricStatistics, Geometry;
    
    object Particles : Points {
      GeometryAttributes globalAttributes;
    
    };
    
    function Particles () {
      this.globalAttributes = GeometryAttributes();
      report("particles created");
    }
    
    /// Clears all data, including points and attributes. Version will be incremented.
    function Particles.clear!() {
      this.parent.clear();
      this.globalAttributes.resize(0);
    }
    
    
    
    /// Returns true if an attribute of a given name exists.
    inline Boolean Particles.hasAttribute?(String name) {
      if(this.attributes.has(name))
        return true;
      return this.globalAttributes.has(name);
    }
    
    /// Returns true if it an attribute of a given name and type exists.
    inline Boolean Particles.hasAttribute?(String name, Type attributeType) {
      if(this.attributes.has(name))
        return this.attributes.has(name, attributeType);
      return this.globalAttributes.has(name, attributeType);
    }
    
    
    
    function Particles.attachNewGlobalAttribute!(Ref<GeometryAttribute> attribute) {
      this.globalAttributes.attachNewAttribute(attribute);
      this.globalAttributes.resize(1);
    }
    
    
    
    
    /// Removes the attribute that has this name.
    function Particles.removeAttribute!(String name) {
      if(this.attributes.has(name)) {
        this.attributes.removeAttribute(name);
        return;
      }
      this.globalAttributes.removeAttribute(name);
    }
    
    
    
    /// Returns a contained attribute of a specific type from its name, or 'null' if not found or another type.
    inline Ref<GeometryAttribute> Particles.getAttribute(String name, Type expectedAttributeType) {
      if(this.attributes.has(name))
        return this.getAttribute(name, expectedAttributeType);
      return this.globalAttributes.getAttribute(name, expectedAttributeType);
    }
    

    then, you can add attributes on the fly, depending on what you need... it can be the previous position for a verlet integration or the velocity for an euler one. it can be the mass unique for all particles or a mass per particle. it can the size used for collision detection, and using bullet behind. For some particles systems, the front facing ones used for smokes for example, you just need the position (the orientation is computed at display time, depending on the point position and the camera position), but for objects dropped in a box, you need the full Xfo. Using the GeometryAttributes opens the door to experiment whatever you can imagine... as long as the user is able to provide a way to update the attribute from one time step to the other!

  • HelgeHelge Moderator, Fabric Employee Posts: 314 Fabric Employee

    that looks great. thanks for that.

    Research Engineer @ Fabric Software

  • craouettecraouette Posts: 113

    by the way, it would be useful if we have a way to reload an extension... once you ask for it, even if there is error, it is not possible to "reload" it once the reported error are connected. The only way is to leave canvas and relaunch it again.

  • HelgeHelge Moderator, Fabric Employee Posts: 314 Fabric Employee

    You can in fact. You need to specify the extension in the "Required Extensions" field, and then you can right click a node and do "Reload Extensions"

    Research Engineer @ Fabric Software

  • craouettecraouette Posts: 113

    yes... didn't notice it. sorry for the trouble!!!

  • craouettecraouette Posts: 113

    one more question...

    all compounds on GeometryAttributes use a PolygonMesh as input... for some, getAtPolygonPoint, I understand, but for other, setAtPoint, it can be for PolygonMesh, Lines or Points.... so, it should take BaseGeometry as input, no?

  • malbrechtmalbrecht Fabric for Houdini Posts: 752 ✭✭✭

    Not sure if this helps - but I use GeometryAttributes on Points without any problem. No need to use a PolygonMesh.


    Marc Albrecht - marc-albrecht.de - does things.

«13
Sign In or Register to comment.