Kludge: New wrapping mechanism: Opaque

pzionpzion Moderator, Fabric Employee Posts: 118 Fabric Employee

In response to feedback from several of you guys, particularly with regards to trying to wrap the Arnold API, I've added a new type wrapping mechanism: add_opaque_type.

It's covered in the docs at https://github.com/fabric-engine/kludge, but the brief summary is that this wrapping mechanism should be used for types that are opaque pointers whose contents are hidden by the API. It's not (currently) possible to add members and methods to these types, but they can be used in Kludge-generated functions.

Note that the kludge discover step may not correctly generate a type declaration for this type, and that's OK; you'll need to insert it manually anyway. Once you added the ext.add_opque_type('TypeName') declaration, the discovered functions that use that type should just "automatically work". At some point I hope to make it automatically use opaque as the default for types that are associated with structures and classes without bodies, but that's not done for the moment.

If you are using the Kludge VM you will need to git pull the latest in the ~/kludge directory to get this feature, and you may also need to edit ~/.ssh/config to point to the right key to do so.

Peter Zion
Fabric Engine

Tagged:

Comments

  • scaronscaron Fabric for Houdini Posts: 171

    Awesome, thanks Peter!

    So how many of us are trying to wrap the Arnold API? :)

    I hate doing the same work over again.

  • jfriedmanjfriedman Fabric for Houdini Posts: 81 ✭✭

    Hello Steven, funny finding you here. :D

  • scaronscaron Fabric for Houdini Posts: 171

    I figured you were in on this. I should be more free to contribute in a open way to a new binding/wrapper for Arnold. Maybe we can talk?

  • jfriedmanjfriedman Fabric for Houdini Posts: 81 ✭✭
  • scaronscaron Fabric for Houdini Posts: 171

    @pzion said:
    If you are using the Kludge VM you will need to git pull the latest in the ~/kludge directory to get this feature, and you may also need to edit ~/.ssh/config to point to the right key to do so.

    I am sorry, I may have missed some important step. I have made a new key and added it to my github account but my access would still need to be added to that repo, correct? Otherwise, I just get 'repository not found'

  • Paul DoylePaul Doyle Administrator, Fabric Employee Posts: 229 admin

    we are going to open the repo up later today, I'll post here once it's done

    CEO at Fabric Software
    Twitter

  • pzionpzion Moderator, Fabric Employee Posts: 118 Fabric Employee

    Repo is now open, you should be able to pull in ~/kludge regardless of how things are set up. @scaron if you still have trouble let me know!

    Peter Zion
    Fabric Engine

  • scaronscaron Fabric for Houdini Posts: 171
    edited November 2016

    @pzion, I was able to pull on the repository. Unfortunately, the files generated from the discover step only had the boilerplate part about being auto-generated by kludge but no python code inside the file. I was very quick to test it last night but I was wondering if anyone has tried to run kludge on an existing project with this new commit? Otherwise, I will run it again with any debug modes on tonight when I get home.

  • pzionpzion Moderator, Fabric Employee Posts: 118 Fabric Employee

    That's weird, do all the unit tests pass if you run py.test at the root of the Kludge checkout?

    @scaron said:
    @pzion, I was able to pull on the repository. Unfortunately, the files generated from the discover step only had the boilerplate part about being auto-generated by kludge but no python code inside the file. I was very quick to test it last night but I was wondering if anyone has tried to run kludge on an existing project with this new commit? Otherwise, I will run it again with any debug modes on tonight when I get home.

    Peter Zion
    Fabric Engine

  • scaronscaron Fabric for Houdini Posts: 171

    I will run the test suite tonight.

  • scaronscaron Fabric for Houdini Posts: 171

    @pzion, Ok, there is no py.test file... there is a test_kludge.py file but it doesn't do anything. I included below my command line and the arguments. Both defns and decls files are empty minus the boilerplate and Arnold.kludge.py just has the cpp_quoted_include listing my input header file and two lines to include the other kludge python files.

    [user@fabric-kludge kludge]$ python kludge discover Arnold ../Arnold/include/ai_universe.h --output-dir ../kludge-arnold -v 6 Using Clang options: -x c++ -isystem /opt/fabric-llvm-3.9-linux-x86_64-gcc_48/include/c++/v1 -isystem /opt/fabric-llvm-3.9-linux-x86_64-gcc_48/lib/clang/3.9.0/include Generated temporary inclusion header: +------------------------------------------------------------------------------- | #include "../Arnold/include/ai_universe.h" +------------------------------------------------------------------------------- Parsing temporary inclusion header with Clang Writing declarations to '../kludge-arnold/Arnold.decls.kludge.py' Writing definitions to '../kludge-arnold/Arnold.defns.kludge.py' Debug: Skipping AST nodes for '/opt/fabric-llvm-3.9-linux-x86_64-gcc_48/include/c++/v1/__config' Debug: Skipping AST nodes for '/opt/fabric-llvm-3.9-linux-x86_64-gcc_48/include/c++/v1/__config' Debug: Skipping AST nodes for '/opt/fabric-llvm-3.9-linux-x86_64-gcc_48/include/c++/v1/__config' Debug: Skipping AST nodes for '/opt/fabric-llvm-3.9-linux-x86_64-gcc_48/include/c++/v1/__config' Debug: Skipping AST nodes for '/usr/include/math.h' Debug: Skipping AST nodes for '/opt/fabric-llvm-3.9-linux-x86_64-gcc_48/include/c++/v1/math.h' Debug: Skipping AST nodes for '/opt/fabric-llvm-3.9-linux-x86_64-gcc_48/include/c++/v1/cmath' Debug: Skipping AST nodes for '/usr/include/stdlib.h' Debug: Skipping AST nodes for '/opt/fabric-llvm-3.9-linux-x86_64-gcc_48/include/c++/v1/stdlib.h' Debug: Skipping AST nodes for '/opt/fabric-llvm-3.9-linux-x86_64-gcc_48/include/c++/v1/cstdlib' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_types.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_types.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_types.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_types.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_types.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_types.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_types.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_types.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_types.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_types.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_types.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_types.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_types.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_types.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_types.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_types.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_types.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_types.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_types.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_types.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_types.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_types.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_types.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_types.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_types.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_types.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_types.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_types.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_types.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_types.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_types.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_types.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_types.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_types.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_types.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_types.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_types.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_vector.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_bbox.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_bbox.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_bbox.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_bbox.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_bbox.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_bbox.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_bbox.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_bbox.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_bbox.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_bbox.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_bbox.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_bbox.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_bbox.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_bbox.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_bbox.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_bbox.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_bbox.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_bbox.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_universe.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_universe.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_universe.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_universe.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_universe.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_universe.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_universe.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_universe.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_universe.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_universe.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_universe.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_universe.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_universe.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_universe.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_universe.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_universe.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_universe.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_universe.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_universe.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_universe.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_universe.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_universe.h' Debug: Skipping AST nodes for '/home/user/kludge/../Arnold/include/ai_universe.h' Writing master to '../kludge-arnold/Arnold.kludge.py'

  • scaronscaron Fabric for Houdini Posts: 171

    @pzion, ok this was maybe just my fault, I am getting proper output with this...

    [user@fabric-kludge kludge]$ python kludge discover Arnold /home/user/Arnold/include --output-dir ../kludge-arnold

    this works with an absolute path to the Arnold include directory... the ../Arnold/include doesn't? I don't have a ton of experience with linux so I might have just operated under the wrong impression.

  • pzionpzion Moderator, Fabric Employee Posts: 118 Fabric Employee

    No, it's a bug in the script! I will push a fix right now, it will be there by the time you see this :-)

    Peter Zion
    Fabric Engine

  • scaronscaron Fabric for Houdini Posts: 171

    Ah, I see you added support for converting that argument into an absolute path for the user :)

    Thanks!

  • scaronscaron Fabric for Houdini Posts: 171

    Hey Peter,

    I am having trouble getting kludge to work with types which have been declared as opaque and being returned from the API to Fabric/kludge as const.

    # node_entry.decls.kludge.py
    # E:\\library\\development\\solidangle\\Arnold-4.2.16.3-windows\\include\\ai_node_entry.h:69:8
    ext_AtNodeEntry = ext.add_opaque_type('AtNodeEntry')
    
    # node_entry.defns.kludge.py
    # E:\\library\\development\\solidangle\\Arnold-4.2.16.3-windows\\include\\ai_node_entry.h:94:50
    ext.add_func('AiNodeEntryLookUp', 'const AtNodeEntry *', [Param('name', 'const AtString')])
    

    So AtNodeEntry is opaque and AiNodeEntryLookUp returns that type as const pointer but the reinterpret_cast keeps complaining about losing qualifiers. It looks like the opaque types registered with kludge don't get the same pointer/ref types wrapped like when you declare an owned type. ie. CxxMyTypeConstPtr, CxxMyTypeConstRef, etc.

  • pzionpzion Moderator, Fabric Employee Posts: 118 Fabric Employee

    Hi Steven,

    I will be looking at this today and keep you posted.

    Peter Zion
    Fabric Engine

  • pzionpzion Moderator, Fabric Employee Posts: 118 Fabric Employee

    OK turned out to be a simple fix. The daily builds that appear a few hours from now should have this fix.

    Peter Zion
    Fabric Engine

  • scaronscaron Fabric for Houdini Posts: 171

    Oh, I am glad! I am making my way through wrapping Arnold the same way I had it pre-kludge. I am almost back to where I was, this is going to help the most since there are lots of uses of the opaque wrappers in Arnold, a couple which are returned as const.

  • jfriedmanjfriedman Fabric for Houdini Posts: 81 ✭✭

    Oh great, I had this issue too. I was editing the C++ to use const_cast instead of reinterpret_cast to get around it. Thanks Peter!

  • scaronscaron Fabric for Houdini Posts: 171

    @pzion just ran on the new build really quickly and it seems to work.

Sign In or Register to comment.