Implementation: Initial Thoughts

Max certainly presents some unique issues related to deeply integrating FE. I am assuming the various FabricMax node types FE provides are meant to contend with Max's lack of something like a dependency graph.

Here are some thoughts and questions that are coming up for me as I work with FE in Max. I realize there may not be answers to them currently, but I think it's useful to note sort of things that we are running into:

  1. Is there a way to prevent Max from triggering unnecessary updates? Max doesn't seem to know when an update is actually needed, so unnecessary evaluations are kicked off: clicking and not editing an input port on a transform controller, for instance.3
  2. Array attributes don't appear to be available. Very limiting. Max limitation? Other options?
  3. Moving data between Max and Fabric: How best to set up a multi node solver? For example, if I wanted to write a custom limb solver and drive part of an existing character, what would that look like in Max? Should I use the FabricMax transform controllers, or should I use a standard Max controller and write to its inputs using parameter wiring, etc?
  4. Ideally I would prefer to use FE to handle as much as possible. Introducing Max specific idioms, like parameter wiring and scene hierarchy, etc., makes the implementation more complex and less portable.
  5. In a test I did the transform controllers didn't perform very well, but it could have been due to the nature of my implementation. In reference to #3 & #4 above, I wanted to keep as much functionality in FE as possible: I wanted to use FE constructs to move data around, not Max's parameter wiring. To this end I created a singleton to hold some data that could be accessed by each of the 451 involved controllers. This was slow and took up a lot of memory.
  6. We have to use DirectX, so unfortunately, we can't use any of FE's drawing capabilities within Max. This is very disappointing because a big attraction of FE has been the possibility to provide rich visual feedback without sacrificing performance.

At this point I have no strong opinions about FE's integration within Max - just questions. I am still looking at ways of solving the above outlined problems. It's very important to us that our implementation be portable because we plan on migrating away from Max.

-Judah

Comments

  • borjaborja Administrator, Fabric Employee Posts: 480 admin

    Hi Judah

    Thank you very much for your feedback!. Answers quoted

    @Judah said:
    Max certainly presents some unique issues related to deeply integrating FE. I am assuming the various FabricMax node types FE provides are meant to contend with Max's lack of something like a dependency graph.

    Here are some thoughts and questions that are coming up for me as I work with FE in Max. I realize there may not be answers to them currently, but I think it's useful to note sort of things that we are running into:

    1. Is there a way to prevent Max from triggering unnecessary updates? Max doesn't seem to know when an update is actually needed, so unnecessary evaluations are kicked off: clicking and not editing an input port on a transform controller, for instance.3

    We have filed FABMAX-40 to take a look at this

    1. Array attributes don't appear to be available. Very limiting. Max limitation? Other options?

    Array attributes are working but only available within Maxscript. We have FABMAX-22 to improve on it.

    1. Moving data between Max and Fabric: How best to set up a multi node solver? For example, if I wanted to write a custom limb solver and drive part of an existing character, what would that look like in Max? Should I use the FabricMax transform controllers, or should I use a standard Max controller and write to its inputs using parameter wiring, etc?

    We have not done extensive thinking on workflows so I am afraid I can't tell which one is going to be faster or more robust without doing real testing (that is one of the reasons why we are in beta :) )

    1. Ideally I would prefer to use FE to handle as much as possible. Introducing Max specific idioms, like parameter wiring and scene hierarchy, etc., makes the implementation more complex and less portable.

    Indeed. All your feedback will be welcome. Do you have any suggestion on how to handle the lack of a proper DG?

    1. In a test I did the transform controllers didn't perform very well, but it could have been due to the nature of my implementation. In reference to #3 & #4 above, I wanted to keep as much functionality in FE as possible: I wanted to use FE constructs to move data around, not Max's parameter wiring. To this end I created a singleton to hold some data that could be accessed by each of the 451 involved controllers. This was slow and took up a lot of memory.

    Yes, we have discussed about it and have FABMAX-39 to handle the slowdown.

    1. We have to use DirectX, so unfortunately, we can't use any of FE's drawing capabilities within Max. This is very disappointing because a big attraction of FE has been the possibility to provide rich visual feedback without sacrificing performance.

    I will raise this issue, but DirectX support is not in our roadmap right now.

    Borja Morales
    Technical Product Manager
    Fabric Software Inc.

  • instinctvfxinstinctvfx Posts: 3

    I'd like to add a +1 for the last issue. OpenGL is not really being worked on since ages and Nitrous is all DX only. I sadly do not expect that to change for the better.

  • JudahJudah Posts: 64

    Borja,
    Thanks for the detailed response.

    The initial FabricMax beta post contains some interesting information about MaxScript access. Is there documentation somewhere that goes into more detail?

    Related: In reference to the array port types, can you expand on that a little? I was able to create a Vec3[] port on a FabricMax node, but I'm not sure how to interface with it, or if this is currently possible. Actually, looks like I can just create any old arbitrary port type without a complaint - Probably in support of the polymorphic nodes?

    Ok, I finally figured out how to create a MaxNode port. Turns out it can't be created directly, but once you have created a regular port type it can be converted. Flow is a little strange. I can't seem to create an output port, though. From the notes on the initial beta posting it sounds like that should be possible?

    At any rate, as I noted in my original post, even if these other types are supported (array, and MaxNodes, etc.), I am really looking for a way to use the DCC idioms as little as possible. We can't migrate away from Max immediately, but that is the eventual goal. To this end I'd like to keep as much as possible in Fabric proper.

    I'm not sure what is causing the slowdown for me, but I will do some more testing. I am performing two map lookups, but I have a hard time believing this would be the source of such a significant slowdown, especially when each map contains only a single object.

    Ideally, the controllers could be treated more like simple write targets. There is currently a lot of other logic within that handles data loading, retrieval and the associated failure cases. If there was a coordinating agent that could handle the data management aspects it would be cleaner and it seems like more informed optimizations could be performed.

    Can the creation/destruction of a FabricMax node be tracked? I'm sort of wondering if registering the controllers upon instantiation might allow a different type of relationship to be set up. This would allow the controller to be referenced by a solver, rather than the other way around. Then the solver could run it's calculation and write directly to the controllers involved. Would just need to be able to detect if the controllers are still valid on each iteration.

  • borjaborja Administrator, Fabric Employee Posts: 480 admin

    @Judah said:
    Borja,
    Thanks for the detailed response.

    The initial FabricMax beta post contains some interesting information about MaxScript access. Is there documentation somewhere that goes into more detail?

    Related: In reference to the array port types, can you expand on that a little? I was able to create a Vec3[] port on a FabricMax node, but I'm not sure how to interface with it, or if this is currently possible. Actually, looks like I can just create any old arbitrary port type without a complaint - Probably in support of the polymorphic nodes?

    We don't have yet proper documentation on maxscript access. Let me try to prepare some samples on this, including the use of arrays. You need to support any arbitrary type in case you have extensions registering custom datatypes.

    Ok, I finally figured out how to create a MaxNode port. Turns out it can't be created directly, but once you have created a regular port type it can be converted. Flow is a little strange. I can't seem to create an output port, though. From the notes on the initial beta posting it sounds like that should be possible?

    We have the ticket FABMAX-32 to allow creating a MaxNode port on creation, rather than having to create and then edit.

    Can the creation/destruction of a FabricMax node be tracked? I'm sort of wondering if registering the controllers upon instantiation might allow a different type of relationship to be set up. This would allow the controller to be referenced by a solver, rather than the other way around. Then the solver could run it's calculation and write directly to the controllers involved. Would just need to be able to detect if the controllers are still valid on each iteration.

    I am not sure about this one. Will talk with the main developer of the plugin!

    Borja Morales
    Technical Product Manager
    Fabric Software Inc.

  • StephenTStephenT Fabric for MotionBuilder Posts: 77

    Hi Judah


    Moving data between Max and Fabric: How best to set up a multi node solver? For example, if I wanted to write a custom limb solver and drive part of an existing character, what would that look like in Max? Should I use the FabricMax transform controllers, or should I use a standard Max controller and write to its inputs using parameter wiring, etc?

    For this, I would suggest creating a FabricReferenceTarget with an output array of Mat44, then create N Fabric transform controllers that take the Mat44 array as input, and simply return an indexed value from that array. The advantage of working with the FabricControllers vs writing to inputs is that you integrate correctly within Max's dependency graph.

    See the Controller maxscript unit test for an example of connecting an output on one graph with the inport on another graph.

    I do plan to implement out-port exposure to max (similar to the way in-ports are exposed), which will allow for parameter wiring etc, but in general that will probably be slower because with the values will be converted between Max & Fabric more frequently than necessary. I doubt that will be perceptible in your case, but with thousands of elements it could definitely be concern.


    I was able to create a Vec3[] port on a FabricMax node, but I'm not sure how to interface with it, or if this is currently possible.

    This is currently a bug in the max implementation, we recognize Vec3Array port type, but not the Vec3[] type. To add and set/read values, see below

    (maxscript)
    $.DFGAddPort "v" 0 "Vec3Array" portToConnect:"" extDep:"" metaData:"" execPath:""
    $.v = #( (Point3 0 1 2), (Point3 .5 1.2 9))
    $.v

    We try to handle all conversions in the background, but for unknown types its a bit difficult. We are looking into ways of supporting this, but its non-trivial.


    Can the creation/destruction of a FabricMax node be tracked?

    You could register a callback for node creation and test each node for its type. However, I don't think this is the path you want to be taking. It depends on your implementation of course, but the controllers should always reference the solver. If A depends on B for its value, then B cannot depend on A, as it creates a circular dependency.

    I'm assuming we are still talking about a multi-limb solver here. Solvers can be a little bit more difficult, as you can easily depend on X for an initial position, while then also writing X's final position. In this scenario, one possible solution is to have different nodes for the input as for the output - ie, your artist driven rig is the input to the solver, and it writes to a different set of bones for output. I understand this makes a much messier scene, but it makes much cleaner code!

    However, I'm just guessing here. Could describe your requirements in a bit more detail?

  • borjaborja Administrator, Fabric Employee Posts: 480 admin
    edited February 2016

    @StephenT said:
    For this, I would suggest creating a FabricReferenceTarget with an output array of Mat44, then create N Fabric transform controllers that take the Mat44 array as input, and simply return an indexed value from that array. The advantage of working with the FabricControllers vs writing to inputs is that you integrate correctly within Max's dependency graph.

    See the Controller maxscript unit test for an example of connecting an output on one graph with the inport on another graph.

    In beta 1 we are not deploying the unit tests yet. For reference, I am attaching it here

    Borja Morales
    Technical Product Manager
    Fabric Software Inc.

  • JudahJudah Posts: 64

    @borja: Thanks I got the script. I ran it as is and it worked fine. I increased the numOutputs to 451 to match my bone count and there is indeed a performance improvement over what I was getting, however, not as much as I had hoped. I get about 10 fps.

    @StephenT said:
    For this, I would suggest creating a FabricReferenceTarget with an output array of Mat44, then create N Fabric transform controllers that take the Mat44 array as input, and simply return an indexed value from that array. The advantage of working with the FabricControllers vs writing to inputs is that you integrate correctly within Max's dependency graph.

    This sounds great and I'll try it as soon as I can. My initial attempt was similar, in that the controllers effectively use an index to get a matrix. However, without knowledge of the FabricReferenceTarget, I stored this data (skeletons) in a SkeletonManager singleton that the controllers retrieve. I was first thinking that the performance issue I ran into was related to doing map lookups into this external data store. But I cache the result, so this seems unlikely. Instead, I think it may be that the controllers are directly and individually accessing the skeleton to retrieve the matrix data. This means a getBone() followed by a getReferencePose() followed by a toMat44() for each controller. I should probably cache the entire skeleton output in a single Mat44[] instead.

    See the Controller maxscript unit test for an example of connecting an output on one graph with the inport on another graph.

    This is what I was looking for!

    I do plan to implement out-port exposure to max (similar to the way in-ports are exposed), which will allow for parameter wiring etc, but in general that will probably be slower because with the values will be converted between Max & Fabric more frequently than necessary. I doubt that will be perceptible in your case, but with thousands of elements it could definitely be concern.

    Well, I guess you can go ahead and do that, but I'll not use it :) I am hoping to stay as far out of the Max environ as possible. The scenarios you describe regarding solvers are of great concern to me, due to how Max evaluates, so I am taking whatever steps I can to limit Max's involvement in the computation pathway.

    You could register a callback for node creation and test each node for its type. However, I don't think this is the path you want to be taking. It depends on your implementation of course, but the controllers should always reference the solver. If A depends on B for its value, then B cannot depend on A, as it creates a circular dependency.

    This is definitely not my preferred option. I was really just spit-balling, although the intention was not to create a circular dependency, but to figure out how to write directly and efficiently to the controllers. (Continuing below your next quote...)

    I'm assuming we are still talking about a multi-limb solver here. Solvers can be a little bit more difficult, as you can easily depend on X for an initial position, while then also writing X's final position. In this scenario, one possible solution is to have different nodes for the input as for the output - ie, your artist driven rig is the input to the solver, and it writes to a different set of bones for output. I understand this makes a much messier scene, but it makes much cleaner code!

    However, I'm just guessing here. Could describe your requirements in a bit more detail?

    I just want a fast way to write to a lot of controllers. They are not doing any computation. Having different input nodes from the output nodes is no problem - I consider that a given. Any type of solve would not use the the controllers as input as all. The solve would be as near completely within Fabric as possible. Currently all I'm doing is storing a static skeleton in memory who's data comes from a JSON file on disk. I'm not yet doing any real computation, so I was surprised at the performance.

    Oh yeah. I get a **lot **of these:

    [Fabric] [FABRIC:MT] Failed to persist port 'dataVal' (Mat44[]), objects must inherit from RTValToJSONEncoder and RTValFromJSONDecoder to be persistable
    

    I'm also seeing a number of UI bugs that don't look directly FE related, but I figure I'll mention here just in case. The entire Max UI seems to get confused and draws incoherently all over. Their equivalent of the outliner dies, throwing up a window I can't easilly read because of the corruption, and then turns into a white box with a big red X in it. Beam me up, Scotty...

    @StephenT & @borja,
    Thanks both of you for the help. I can't wait to try these changes.

    -Judah

  • JudahJudah Posts: 64

    @StephenT & @borja,
    I have reworked the controller graph to do a bit less work. Summary of changes:

    1. The matrix data required by the controllers is now returned in a cached Mat44[]. This has reduced the function overhead as outlined in the previous post.
    2. Removed debugging options and associated input ports.
    3. Moved logic out of graph and into KL. It's much easier to follow and more straight forward to tell what code is executing when.

    These changes resulted in some performance improvements, though not quite to the point of acceptability. Framerate went up from around 3.5 fps to 7.5 fps and memory consumption went down by approximately 50% from over 12 gig to about 6. Those graphs certainly are heavy memory consumers. I'd like to do a test that collapses the entire graph into a single KL function to see what the difference is, but I'm not sure I have time at this point.

    I have not yet incorporated the FabricReferenceTarget. I plan to do that next and will let you know the results.

    -Judah

  • borjaborja Administrator, Fabric Employee Posts: 480 admin

    Thanks for keeping us informed Judah.

    We still have the memory and performance overhead ticket in mind, which hopefully will improve on the framerate and memory consumption.

    Borja Morales
    Technical Product Manager
    Fabric Software Inc.

  • StephenTStephenT Fabric for MotionBuilder Posts: 77

    @Judah

    The UI bugs.... yeah, I just can't figure out how that got let through. I'm slightly embarrassed, because it was one of the last things I worked on before leaving Autodesk, and now it seems to crash with a fair regularity. On the other hand, the whole thing is pretty unrecognizable now from when I left, so...

    If you want (and its allowed/possible) I would be happy to take a look at your scene and do a spot of profiling to see if I can figure out where the problems lie. Max has some very silly implementation faults that can cause all sorts of issues, but quite a few of them can be dealt with without too much problems. (Also, I haven't run a profiler over Fabric4Max at all yet, so its possible that some slowdowns exist in that department too).

    The "failed to persist" warnings are a bit annoying. Its Max's auto-save, combined with (I guess) something being a bit problematic in the Fabric function we use to save our data. I don't think we have a issue logged to get rid of it, but one should probably be added (@Borja?)

  • borjaborja Administrator, Fabric Employee Posts: 480 admin

    We are persisting properly Mat44[] in the core these days, so it should be fixed by a new beta version built against the 2.0.2 version that will come soon.

    Borja Morales
    Technical Product Manager
    Fabric Software Inc.

Sign In or Register to comment.