LLDB on Windows

StephenTStephenT Fabric for MotionBuilder Posts: 77

Hi All,

Coming back to fabric after bit of a break - and it looks like LLDB on windows has come along significantly: several projects boast LLDB debugging in Visual Studio. This is great news, and I can't wait to try them out!

Has anyone used one of these projects?

If not - are there any curve-ball requirements on Fabric's side that might prevent me from using these projects to debug KL? I'm happy to compile LLDB or similar if necessary to get this going (and will post back my results)

«1

Comments

  • StephenTStephenT Fabric for MotionBuilder Posts: 77

    On the positive side: installing LLVM 5.0.0 is a breeze! Got it up and running without any issues at all.

    Also great news: it (says it) supports JIT debugging
    (lldb) log list
    jit - log JIT events in the target

    On the downside, even using the LLDB command prompt, I can't attach to a running KL script.
    set FABRIC_DEBUG=2
    start a KL script with an infinite loop to give a process we can attach to

    (lldb) process attach --name kl.exe
    error: attach failed: unknown error

    A bit frustrating, but we seem to be a lot closer!

    How can I find the missing link? Any suggestions from Fabric Co?

  • AhmidouAhmidou Posts: 179 ✭✭

    I would like to know too, this could be very helpful!

  • StephenTStephenT Fabric for MotionBuilder Posts: 77

    Yeah, using a language without debugging tools can be tough.

    To verify that a standard build can even be used for debugging, I've setup zBug/LLDB 4.0 on Ubuntu 16.04, and nothing seems to be connecting right.

    `> process launch --stop-at-entry
    Process 19287 launched: '/usr/bin/python' (x86_64)

    br set -n operator.entry

    Breakpoint 1: no locations (pending).
    WARNING: Unable to resolve breakpoint to any actual locations.

    process continue

    Process 19287 resuming
    Connected to Fabric Engine process
    STOP None
    STOP None
    Process 19287 exited with status 0
    `
    I have used this process in the past to get zBug working with a custom build of LLDB3.5, so I'm mildly confident that it should work, but with a standard install I'm getting no luck.

    I don't think I can get any further without a bit of assistance from FE. Ie - does/should a stock install of LLDB function with Fabric? What is required to get this working?

  • Daniele NieroDaniele Niero Posts: 233 ✭✭

    Add me to the list of who is interested (very) on this.
    If zBug could be use on Windows it would be a massive help for those, like me, that unfortunately need to work on windows.

  • StephenTStephenT Fabric for MotionBuilder Posts: 77

    I've just noticed that Fabric has a build of LLVM 3.9 in their downloads folder: http://dist.fabric-engine.com/llvm/, posted only 5 months ago.

    This kinda implies support for LLDB3.9, which is great, but also implies it's a custom build of 3.9, which is not great, as it suggests we are dependent on Fabric for builds. If thats the case, it seems unlikely we'll be seeing a Windows build any time soon, as 5.0 is the active development branch of LLVM.

  • StephenTStephenT Fabric for MotionBuilder Posts: 77

    Unfortunately, it turns out the build of LLVM on Fabric site does not include LLDB - so is unrelated to zBug & debugging in general.

  • pzionpzion Moderator, Fabric Employee Posts: 118 Fabric Employee

    Hi guys,

    The build of LLVM 3.9 is used by Kludge to access libclang. We have an open item to upgrade the LLVM that Fabric itself uses, but it's not as high priority as other things we are working on right now, and so it's unclear right now when it will happen.

    I will try to find some time to investigate why zBug doesn't work with more recent LLDB versions. zBug is just a wrapper around LLDB; can you use LLDB on its own? Ie. will this work:

    FABRIC_DEBUG=2 lldb kl hello.kl
    (lldb) b operator.entry
    (lldb) r
    

    I would try first on CentOS 6, and if it seems to be working try with Windows.

    Peter Zion
    Fabric Engine

  • StephenTStephenT Fabric for MotionBuilder Posts: 77
    edited February 8

    At this point I seem to be unable to even get the original zBug working (Ubuntu 16.04). The callstack if I trigger a break in the middle of evaluation, it can find the source file, and I can see the Python source frames, but not Fabric.

    Is zBug supported on the latest version of Fabric? Are there any debugging tools being used successfully with KL?

  • pzionpzion Moderator, Fabric Employee Posts: 118 Fabric Employee

    I know that just using gdb to see the stack frames works fine on Linux, I was just using it yesterday. Note however that the "stock" CentOS 6 gdb is quite broken and you need to compile a recent gdb from source.

    We'll file an issue to have zBug looked at.

    Peter Zion
    Fabric Engine

  • pzionpzion Moderator, Fabric Employee Posts: 118 Fabric Employee

    I just downloaded zBug from dist.fabricengine.com and ran it here and it seemed to work fine. It's a bit messy to get the environment set up right, but it definitely works:

    export LD_LIBRARY_PATH=~/zBug-20150206-Linux-x86_64/lib/python2.6/site-packages/PySide
    export PYTHONPATH=~/zBug-20150206-Linux-x86_64/lib/python2.6/site-packages
    FABRIC_DEBUG=2 python2.6 ~/zBug-20150206-Linux-x86_64/zBug kl hello.kl
    

    Then if I b operator.entry and r I get a break on operator.entry and a stack trace containing KL functions.

    Obviously this should be improved, especially since Fabric now ships with PySide; we should just be using the PySide from Fabric. However, we would have to build LLDB against Python 2.7 to do this. Or, as you've tried, it may work to use a more recent LLDB Python module built against python 2.7.

    Peter Zion
    Fabric Engine

  • StephenTStephenT Fabric for MotionBuilder Posts: 77

    Hi Peter!

    Thanks for the help (and apologies - I somehow missed your earlier posts when writing my last post).

    the LLDB command line seems to be working - with version 4.0 no less! I'm not sure whats different with what you suggested and what I tried earlier, but your steps gets a cmd line where I can step & examine variables. zBug doesn't seem to play well well, but that could also be the hacking I did to get it running on Ubuntu (CentOS 6 is just a little old for my blood).

    I'll continue investigating and will post where I get to

  • pzionpzion Moderator, Fabric Employee Posts: 118 Fabric Employee

    OK thanks Steve, I am also trying to get a build of zBug put up that (a) doesn't include PySide (instead using whatever is already on PYTHONPATH) and (b) comes with and uses LLDB 4.0.0.

    Note that even when I do this I'm doubtful that you will be able to just put the right DLL in place and expect it to work on Windows, although I'm not sure it won't work. Fabric tells LLDB that new symbols exist using a standard gdb-type hook; it's possible they've made this work transparently on Windows but I have my doubts.

    Peter Zion
    Fabric Engine

  • pzionpzion Moderator, Fabric Employee Posts: 118 Fabric Employee

    OK everyone, I've uploaded a new build of zBug. zBug itself has not actually changed; it simply ships with LLDB 4.0.0, and has a few modifications to ensure that it will load the "right" LLDB. What this means is:

    • If the lldb module is already on the PYTHONPATH, it will load it.
    • Otherwise, it will load the module that is included in the build.

    In theory, it should be "easy" to substitute a different build of LLDB, or download this archive on Windows and see if an LLDB 4.0.0 build works there. As I mentioned before, I have my doubts, but I would be happy to be proven wrong. There is nothing anywhere that mandates the version of LLDB; in fact, I didn't do much testing other than the obvious "set a breakpoint, see it is hit and that the stack trace contains KL functions".

    Also, this zBug no longer comes with PySide; instead, it uses the PySide that lives on PYTHONPATH, which for us will usually be the version that comes with Fabric.

    The new build is available at http://dist.fabric-engine.com/zBug/zBug-20170208-Linux-x86_64.tar.bz2

    Peter Zion
    Fabric Engine

  • StephenTStephenT Fabric for MotionBuilder Posts: 77

    Hi Peter,

    Thanks for the prompt attention, but I'm actually ok without zBug. I prefer to use something familiar - Visual Studio Code is perfect for the job (actually, its a pretty nice editor all round). Now it's running, I'm happy. Although I'll keep looking at the windows port to see how far along it is.

    @Daniele Niero Although we still don't have proper windows debugging, one method of working is to install a virtual machine of linux, install lldb-4.0 & visual studio code+CodeLLDB, and you'll have a fairly full-featured debugger on your hands. Share your KL code folder so it can be accessed from both linux and windows, and (for now) that is a good-enough solution.

  • pzionpzion Moderator, Fabric Employee Posts: 118 Fabric Employee

    Thanks for the info Steve. Are you actually able to debug using the CodeLLDB plugin on VSCode? If so that's great news, since that would suggest that the debug symbols that Fabric emits are being picked up...

    Peter Zion
    Fabric Engine

  • StephenTStephenT Fabric for MotionBuilder Posts: 77

    Yeah, it's -mostly- working. VSCode seems to struggle with finding file paths when searching for non-local files (eg, the builtin exts), so it's a ways from perfect. However, VS Code seems very customizable (even more so than Sublime) so I'm sure I'll be able to resolve those issues.

    I've converted the sublime KL colorization to VSCode as well, (https://github.com/FrozenKiwi/VSCode-KL, for all who are interested) so it's well on it's way to be a full workable IDE. I've been taking a quick look over it's plugin architecture too, and it looks like it'll be able to support some really advanced features - intellisense, clickable errors when compiling, code validation while typing etc. It seems extremely promising, with a bit of effort it could rival full-fledged native IDE's.

    I don't feel like I'd even miss windows when working in it :)

  • pzionpzion Moderator, Fabric Employee Posts: 118 Fabric Employee

    Ah gotcha, this is running on Linux. Note that the interface for obtaining the KL code from the Fabric core as KL code is compiled and run is quite simple; I believe it is just over a named socket and it just passes some JSON data. At any rate, it's 100% implemented in the zBug Python code, so it's easy enough to see what's going on. In principle it would be straightforward to extend any open-source debugger like CodeLLDB to support it.

    Peter Zion
    Fabric Engine

  • StephenTStephenT Fabric for MotionBuilder Posts: 77

    Yeah, I've had some strange issues with getting LLDB5.0 running on linux (can't find files, despite them being linked in /usr/bin/). The 5.0 branch is the active dev branch, so I'll have another go next week some time.

    Once I've got things running on linux, I'll have another go at trying on windows.

    @pzion Do you have any idea if there would be difficulties using an x32 process to debug an x64 process? I would have thought it was possible, as I believe thats what VS uses on windows, but that might explain LLDB's problems on Windows.

  • pzionpzion Moderator, Fabric Employee Posts: 118 Fabric Employee

    Yes it should be and is probably possible -- I don't know the debugger syscall interface on Windows but I know the ptrace() interface on Linux would let you do this. Certainly the fact that devenv.exe seems to be a 32-bit executable that debugs 64-bit binaries suggests this works on Windows too.

    Peter Zion
    Fabric Engine

  • StephenTStephenT Fabric for MotionBuilder Posts: 77

    As I work through this process, is there a place where I could leave thoughts/suggestions? The setup isn't difficult, but I'm running into a few gotcha's (and I'll most likely end up forking the LLDBCode project). Where do you think would be a good place to post debugging tips etc?

  • borjaborja Administrator, Fabric Employee Posts: 480 admin

    Borja Morales
    Technical Product Manager
    Fabric Software Inc.

  • AhmidouAhmidou Posts: 179 ✭✭

    Hey @StephenT
    Did you also got Intellisense working withVS Code ?

  • StephenTStephenT Fabric for MotionBuilder Posts: 77

    yeah, it works kinda well. It's good for suggesting words, but doesn't do go-to definition or anything like that yet.

    The VM is a little slow though, so VSCode lags very very slightly (so does sublime running on the VM). For lots of typing I tend to use sublime in windows. VSCode was heavily inspired by sublime though, so I find switching between the two (win+sublime/linux+VSCode) fairly painless.

  • AhmidouAhmidou Posts: 179 ✭✭

    Nice, what about executing kl.exe? is it as simple as in Sublime?

  • StephenTStephenT Fabric for MotionBuilder Posts: 77

    much simpler :)

    I haven't put real time into this yet, but VSCode is capable of a lot more than sublime. We might be able to take advantage of a lot of it's more advanced capabilities (for example, connecting it to KL's lexer to get proper intellisense etc).

    Once I'm not way too rushed, I'm looking forward to investing a bit of time into this. I built a syntax highlighter (basically copying the existing KL repo), and I've got a fork of LLDBCode to fix a few of the niggling issues. I'll have a bit more time in a few weeks to look into things then

  • StephenTStephenT Fabric for MotionBuilder Posts: 77

    Slight update:

    With the latest build of LLVM (5.0.0r298093), the debugger does not seem to pick up and JIT'ed code in windows. On windows, using the LLDB command line, if I

    (lldb) image list
    

    it will list all the native compiled libraries.

    [  1]                                                         C:\Windows\System32\ntdll.dll 
    [  2]                                                         C:\Windows\System32\kernel32.dll 
    [  3]                                                         C:\Windows\System32\KernelBase.dll 
    [  4]                                                         D:\External\FabricEngine-2.4.0-Windows-x86_64\bin\FabricCore-2.4.dll 
    ...
    

    If I execute the same command in linux, I get a bunch of:

    [ 37] A2CFD394-0000-0000-0000-000000000000 0x00007ffff15eb030 JIT(0x7ffff15eb030) (0x00007ffff15eb030)
    [ 38] 68ACBB8F-0000-0000-0000-000000000000                    /home/staylor/src/FabricEngine-2.4.0-Linux-x86_64/Exts/Builtin/FabricSynchronization/libFabricSynchronization-Linux-x86_64.so 
    [ 39] 27F038AF-0000-0000-0000-000000000000 0x000000000295f930 JIT(0x295f930) (0x000000000295f930)
    [ 40] 6D2334DE-0000-0000-0000-000000000000 0x0000000002ddea20 JIT(0x2ddea20) (0x0000000002ddea20)
    

    So, clearly the LLDB environment isn't picking up on the JIT notification from the KL compiler. I haven't looked into whether this is due to LLDB not processing the notification, or if it's due to KL not emitting the notification on module compilation.

    Note - this doesn't appear to have anything to do with the FABRIC_DEBUG/FABRIC_OPT_TYPE, even without setting these variables I can break in KL in lldb/linux

  • AhmidouAhmidou Posts: 179 ✭✭

    I just found this out:
    http://blog.llvm.org/2017/08/llvm-on-windows-now-supports-pdb-debug.html

    Does it mean a KL debug in visual studio will be possible to implement?

  • Daniele NieroDaniele Niero Posts: 233 ✭✭

    I need debugging tools for KL so badly...

  • StephenTStephenT Fabric for MotionBuilder Posts: 77

    I don't think this will help with debugging KL, as KL doesn't use the pdb format. That being said, I haven't looked into windows debugging in a while (as using a VM is pretty much just as good).

    @Daniele Niero I'll write up a tutorial for setting up VS Code for KL soon. I use it all the time, and although it has a few flaws it's utterly indispensable - I don't understand how anyone can do serious work without a debugger.

    It's also way faster to write code in VS Code than anything else. So far I've only created minimal tooling for it, there is still many improvements I have planned. When I have "time".

  • AhmidouAhmidou Posts: 179 ✭✭

    Hey @StephenT, so you're using VS Code with LLDB on linux, right?
    Can you add break points and visual debug?

«1
Sign In or Register to comment.