Animating Data from CSV - Needs some more work

Hi Fellow Fabrickers, Had some time to come back and look some more at what I had started a few weeks ago http://forums.fabricengine.com/discussion/406/working-with-pickle-dictionary-outputting-values#latest

Overview > CSV file of Midi Events > inputting data into animation events
Please find canvas attached, download .csv and play along /help/learn some more. You will need to install Malbrechts extension (https://github.com/Marc-Albrecht/maTools) (cheers again for that) install it under Exts folder, add to FABRIC_DFG_PATH in windows environment paths, for reading in the csv data (and place the .csv file https://dl.dropboxusercontent.com/u/18183777/midiCSVfabric.csv at C:/
.You can modify that in fileBrowser if needed.
Making some breakthroughs but would appreciate anybody nudging me in the right direction on a few things.

What I need to do >
Pipe in conditional from Event Checker for noteOn or noteOFF (Purple block at top) into its matching animation node - you can see there is a boolean and if conditional at the top node so I need to get that straight I think. At the moment you can see the cube position matches its position to it matching vertex on the polyplane - e.g so if input is Note 58 it will match the position of vertex index 58 etc. At the moment its taking every note on and note off event that comes in - so thats where the conditional comes. I would like for only the note on events to pipe into the note on animation node.

2 - I would like an instance or copy of cube to be created on each note on event as well and then a simple animation in the y axis and of course timings would be based on the 'time' node block. I've been looking at the instancing examples but I'm not seeing what I need to do.

Cheers

Comments

  • malbrechtmalbrecht Fabric for Houdini Posts: 752 ✭✭✭

    Moin,

    I hope I got your needs right ... attached is an extended graph, two YELLOW blocks added.

    The upper one checks for note on events by separating the original event string into an array split by underscore ("_"). Then it pulls the second entry and checks if it equals "on". You can do with that result whatever you want ...

    The lower one stacks the positions on a variable holding an array of Xfos. You need a variable to carry the stack from one graph execution to the next. Obviously you need to add a function that resets the stack (array.resize with a value of 0) on frame 0, I did not add that.

    Let me know if I failed! :) Or if I should explain anything better. Or if I got you wrong completely.

    Marc


    Marc Albrecht - marc-albrecht.de - does things.

  • renren Posts: 39

    Hey Marc,
    Thanks for clarifying the conditionals setup and resetting the stack. Haven't had time to dig back into Fabric but that should move things along.

  • seltzdesignseltzdesign Posts: 80
    edited July 2016

    Really interesting. I would like to open your file, but maTools on Github is not available anymore. Is it integrated into canvas now?

    I have been working with csv midi data for a long time now in Grasshopper and vvvv. The key to get anywhere with it is to not rely on Note On and Note Off. You have to get the data into a format where you have one line per event and a calculated note length. Keeping track of the right note ons and offs is really tough.

    Where is the .csv data coming from? Usually there is a tool to track note ons and offs and convert to note length. Max (the one by cycling74) for example has a node just for that. I usually convert actual midi files to .csv or you can use live midi input, but also with a timer for the notelength and not note on/offs. There is a cool little software called GNMidi which can convert midi files to .csv - either in note on off format or combines with a note length. Otherwise vvvv is really cool for converting live midi to a normal format, like in the .csv. I know midi is already integrated into Canvas, but it should really support OSC, which is much more powerful and useful for any kind of data to be piped in.

    My next step is to convert our tool that is done in Grasshopper to Fabric, at least partly, which will be quite a big job, since the tool is quite sophisticated after 2 years of work on it. But hey, nothing like a challange ;)

  • malbrechtmalbrecht Fabric for Houdini Posts: 752 ✭✭✭

    Hi, @seltzdesign - I sent you a PM with the maTools archive.

    I have been starting to work on a Fabric based tool for midi, goal being to provide a platform for VSTi creation in Canvas. Unfortunately recent events have demotivated me from doing anything with Fabric for some time.

    Marc


    Marc Albrecht - marc-albrecht.de - does things.

  • renren Posts: 39

    Hi seltz and malbrecht,
    Been caught up with life in the last while, but dipped back into fabric.

    Where is the .csv data coming from?

    I use a free utility https://www.fourmilab.ch/webtools/midicsv/
    to convert the midi file to a csv, I had a look at the GNMidi program, but I can get what I need with the midicsv utility ( I did notice that the GNMidi program didn't seem to include the velocity data in the csv - might be wrong, as I only had a quick look at it)

    The key to get anywhere with it is to not rely on Note On and Note Off. You have to get the data into a format where you have one line per event and a calculated note length. Keeping track of the right note ons and offs is really tough.

    I've done some tweaks to my original python script, so I'm now able to get this as the main .csv.

    'note,start,end,velocity,track,instrument'
    43, 0.0, 2.304, 60, 6, 'Violoncello'
    55, 0.0, 0.768, 45, 4, 'Violino II'
    59, 0.0, 0.384, 45, 3, 'Violino I'
    71, 0.0, 1.3439999999999999, 75, 5, 'Viola'
    55, 0.384, 2.6879999999999997, 60, 6, 'Violoncello'

    I left the start and end points in for now. If its found more useful to have a single calculated note length - then I'll change that. Just need to try it out and see whats best.

    In addition a .csv file for the controller elements.
    'note,time,velocity,type,track,instrument'
    '30,1.096,90,64,3,concert_piano'
    '30,1.111,76,64,3,concert_piano'
    '30,1.125,71,64,3,concert_piano'
    ( in this instance the controller is '64' sustain pedal-). For those interested what these can include. http://www.nortonmusic.com/midi_cc.html
    I think these are a biggie for getting out more expression in an animation.
    and also a third for the pitch bend wheel
    'note,time,pitch,track,instrument'
    '54,0.0,8192,3,NN19 1'
    '54,8.79365,8192,3,NN19 1'

    I know midi is already integrated into Canvas, but it should really support OSC, which is much more powerful and useful for any kind of data to be piped in.

    Seltz and Malbrecht have you been able to get midi to work in Canvas?. I tried to get it working, but there is a lack of concrete examples and set up information. I've been able to get midi working with Processing 3 so assumed that fabric wouldn't be any harder. Friends have told me about OSC being flexible(so its on my more to learn about list). Music is a 'in the moment' experience which you can't do with a .csv. Still useful though.

    When I'm able I need to get back into canvas and get everything setup.
    If anybody wants to have a play around in the meantime I can upload some converted midi files.
    Good Luck all!

  • malbrechtmalbrecht Fabric for Houdini Posts: 752 ✭✭✭

    Hi,

    sorry, I did not know about "midi being integrated into Canvas". Haven't looked at it. Since one of my experimental projects involves midi, I might have to hunt for that integration - although I think that my "external" approach might not benefit too much from it.

    Marc


    Marc Albrecht - marc-albrecht.de - does things.

  • seltzdesignseltzdesign Posts: 80

    Hey guys,

    I have been away for a little while and now finally going back into Fabric. I am just looking at malbrechts canvas and trying to understand what's happening. Its looking pretty logical so far. I think what you are trying to do is very similar to what I have done in vvvv and grasshopper in the past - create geometry from midi data. In case you are interested, you can see what we have done from a few years ago when we (my boss and me) did a TEDx Talk in Zurich (https://www.youtube.com/watch?v=uxYHlZQSADQ - my part starts some minutes in doing the demo of the quite early vvvv prototype).

    So right now, the task of recreating this in Fabric is very daunting - that vvvv patch is huge and Fabric is still in its infancy compared to vvvv and Grasshopper when it comes to ease of use (not being able to see the data at every input and output of every node).

    What I would be interested in though, is to use Fabric for animations, where I have the transformation matrix data already (from our tool in Grasshopper) for all the stages of the animation (before note on, note on, note off, after note off) and then use Fabric to interpolate between those and instancing the geometry with that. It seems like its not so hard with Fabric. Still a lot to learn though!

    Regarding Midi Support in Fabric - I cant seem to find it right now, but I saw a video on the blog, where they show an example where they are controlling some values (brightness of a light) with a midi controller. That means there is at least some basic support for midi input into fabric that can be fed into nodes. Like I said, the hard part is not the midi input into the program, the hard part is recording each note coming in and funneling everything into the masterlist with all recorded and currently running notes. Especially the currently running notes are difficult, because you need to start a timer for each note and keep track of it till it is finished. The nature of music means that usually there are multiple notes played at different times and lengths.

    @ren: If you want something realtime, which has great midi support and can do pretty much anything including audio, 3D, etc. then I would recommend vvvv. Its node based, so much more powerful (because less debugging) than Processing. Or even look at Max/Msp, which is similar.

    I am not sure how well Fabric is suited to realtime applications. Its seems like its always tied to a timeline, ie. a definitive length of animation.

    Anyways, interesting stuff and i'll keep posted about any progress.

  • malbrechtmalbrecht Fabric for Houdini Posts: 752 ✭✭✭

    Hi,

    I am not sure how well Fabric is suited to realtime applications. Its seems like its always tied to a timeline, ie. a definitive length of animation.

    ... not necessarily. Using the timeline is just a very convenient way of getting a "tick". But you can make Fabric evaluate the graph on every input change - so there is nothing that would keep you from having a time-unlimited evaluation.

    I am very interested in everything animation-related. Since I didn't get much feedback on my first ideas for a Fabric-Animation-Helper system, I haven't done much about it recently, but with Fabric's user base growing, I will keep an eye on it :)

    Marc


    Marc Albrecht - marc-albrecht.de - does things.

  • malbrechtmalbrecht Fabric for Houdini Posts: 752 ✭✭✭

    ... just to add on that: If you don't have any condition set that checks for a specific timeline value (like frame 0 or 1), running the timeline in loop mode will have your graph being executed endlessly as well ...


    Marc Albrecht - marc-albrecht.de - does things.

  • seltzdesignseltzdesign Posts: 80
    edited August 2016

    So, no I finally have some time to play with this. How do I get the MaReadCSV to work. I'm getting an error "no extension or type named 'maTools'. :/

    Also can I attach a sample-file for how my .csv is structured. Its just a bunch of values on each line, seperated by ";". So no field names.
    While we are at it, I also need to flip the array, so that all 1st values are in one array, all second values, etc. Do you know how to do this?

    Thanks!

  • malbrechtmalbrecht Fabric for Houdini Posts: 752 ✭✭✭

    Moin,

    So, no I finally have some time to play with this. How do I get the MaReadCSV to work. I'm getting an error "no extension or type named 'maTools'.

    you should be able to load the csv-reader thing as a graph, remove the requirement (maTools) and just save it as your own preset. If that doesn't work, give me a ping, I'll send you the "full package" (well, I have to strip it of a few features that aren't ready for giving out of hand yet).

    Also can I attach a sample-file for how my .csv is structured. Its just a bunch of values on each line, seperated by ";". So no field names.

    I'll have a look at the CSV reader later tonight, cannot really remember if it supports indices instead of named columns. But that'd be simple to add.

    While we are at it, I also need to flip the array, so that all 1st values are in one array, all second values, etc. Do you know how to do this?

    Sounds do-able. Later tonight ;) It's Friday here and that's a typo. It used to be spelled "Freakday".

    Marc


    Marc Albrecht - marc-albrecht.de - does things.

  • seltzdesignseltzdesign Posts: 80

    Haha, ok, thanks.
    Either the .csv reader should support indices or I can get it to work using the Text Reader. Still trying to understand data operations in Fabric. In Grasshopper data structures are a breeze compared to this and all nodes just work with single values or arrays the same, which is very convenient!!

    Here is the other discussion about the way using Text Reader: http://forums.fabricengine.com/discussion/692/split-text-file-into-lines#latest

  • malbrechtmalbrecht Fabric for Houdini Posts: 752 ✭✭✭

    Not to derail this into a philosophic discourse, but:

    In Grasshopper data structures are a breeze compared to this and all nodes just work with single values or arrays the same, which is very convenient!!

    ... I don't know Grashopper (I do know grashoppers, got plenty of them in the backyard, though), but from your description it sounds like a PHP-ish "auto-casting". That is not a good thing(tm) and I would not find it "convenient", since it is very much capable of screwing things up into a bloody, messy pile of ... I'll stop here and just say: Arrays in Fabric can do with a bit more love, for sure. I do think, however, that they work pretty much the way one expects arrays to work ...

    Marc


    Marc Albrecht - marc-albrecht.de - does things.

  • seltzdesignseltzdesign Posts: 80

    Yeah, I understand what you mean. The reason it works great in Grasshopper (another node based programming language inside Rhino) is that you can view the data and data structure at every node input and output by just hovering over it. Also the connection wire actually shows you if it contains a single value, multiple values (simple array) or multidimensional array. Its very sophisticated and enables some very powerful things with very few nodes. Sure, if you come from more traditional text-based coding, then it will be confusing at first.

    Anyways, each tools has its pro's and con's. Right now I am excited about Fabric's performance and interoperation with Maya.

  • seltzdesignseltzdesign Posts: 80

    Grasshopper actually uses Data Trees. There is a good explaination of them here in case you are interested:

    I got very used to it actually and its amazing how you can work with very complex data structures so easily. But really it works so well, because you dont have to log everything to see whats going on, but can at every junction see whats happening in real-time.

  • malbrechtmalbrecht Fabric for Houdini Posts: 752 ✭✭✭

    here's a quick hack - the "Seltz-CSV-Reader" ...

    • in the reader node, select "onlyIndices" - that will auto-create indices for the columns starting with 0.
    • in the pick-value node, remember that line and column both start with 0.

    I hope this helps!

    Marc


    Marc Albrecht - marc-albrecht.de - does things.

  • seltzdesignseltzdesign Posts: 80

    Thanks. Still getting that maTools error message though, even after commenting out all references to "require maTools".

  • malbrechtmalbrecht Fabric for Houdini Posts: 752 ✭✭✭

    sorry, my fault - note to myself: "Don't do quick hacks. You're old now. Old people don't do quick hacks, they're supposed to enjoy other peoples' hacks".

    Attached is v2 of the not-so-quick-hack.


    Marc Albrecht - marc-albrecht.de - does things.

  • seltzdesignseltzdesign Posts: 80

    Not a problem! So this works nicely. So the output of MaReadCSV is a dictionary? I cant seem to be able to use the normal Dict components on the output.

    I need to get all {0} values, all {1} values, etc. from all lines, then convert to scalar, then to Mat44.

    Lets continue this on the other thread, because this was started by someone else. And also weekend first.. ;)

  • renren Posts: 39

    Hello again,

    That got busy quick :)
    Seltz - interesting talk on the TEDx clip
    marc re:

    I am very interested in everything animation-related. Since I didn't get much feedback on my first ideas for a Fabric-Animation-Helper system, I haven't done much about it recently, but with Fabric's user base growing, I will keep an eye on it

    Have you had a look at mash in maya?
    That should give you some ideas on what goes into an artist friendly setup.

    Here's where I am at the moment with developing the canvas
    https://dl.dropboxusercontent.com/u/18183777/AAA_read_midi_csv.canvas

    things on the the to do list

    • have the resetStackFunction do what it should do and set the stack_of_Xfos back to 0,0,0.
      I saw in some fabric sample scenes that they were calling init execution to reset some xfos arrays.

    • have the keyframe[] data feed into the instances, something simple like make instance appear at the startframe and animate in a transform direction, pull in the velocity element from the keyframe[] and do something like set the size of the geo instance at the start and end times. you can see the details output from the labelled reports to the log messages window. velocity is a nice attribute for animation - if for example piano keys are hit or soft then that will be reflected.

    • on the left you can see the 'select Instrument' node. at the moment in the 'Eq==' node you can see 'piano Ch1'. So only the events related to that instrument are allowed to the rest of the canvas. You can swap over to the other instruments if you follow the notes in the canvas. Any suggestions on anything to hand where the all the instruments can be stored in a interface - where the user can select?

    The idea is for me to bring these all over to Maya, so that's where it will make more sense with the audio playing back (not sure that audio playback is supported in fabric engine)
    If you download these you'll find the .csv file + the sound file.
    my testing canvas is using this first one. A nice mix of short note events on the piano and longer ones on the cello and violin for different animation looks.
    https://dl.dropboxusercontent.com/u/18183777/violin,cello,piano.zip
    this is just a piano, but includes controller data for a sustain pedal
    https://dl.dropboxusercontent.com/u/18183777/piano+sustainController.zip

    Thanks for any help and if anybody just wants to noodle about with it and has some questions / suggestions let me know.

  • malbrechtmalbrecht Fabric for Houdini Posts: 752 ✭✭✭

    Hi, @ren - let me know if you need (specific) help.
    I am willing to try my best, but have to admit that I am having difficulties in following along if weeks pass between bits of conversation :wink:

    Marc


    Marc Albrecht - marc-albrecht.de - does things.

  • renren Posts: 39

    Hi Marc,
    Yeah a bit of data dump there.
    I've sorted out 1st issue i had with resetting the array stack - just needed a break and came back to it.

    Main thing is connecting up the keyframe[] data to animate instances. There are a couple of other posts floating on the forum so I'll try and tease out how the hell its done.

  • seltzdesignseltzdesign Posts: 80

    Hey @ren,

    I will take a look at what you sent and see how I can help. In the end I am looking for something probably very similar in Fabric. I have built the same thing already in vvvv and then again in Grasshopper, so I know WHAT needs to happen, but not necessarily HOW to do it in Fabric - but the concepts are the same.. interpolate Keyframe data between keyframes for each note, map different parameters to different transforms, etc.

    I dont think the Fabric tool will have as many bells and whistles as the Grasshopper tool, because Fabric doesnt really provide a very nice UI for input and making connections. I guess you can build your own UI, but it seems kind of advanced. Maybe its even better doing it with Canvas inside Maya, as you can have nicer Inputs.

    Anyways, will get back on this on Monday - luckily doing stuff like this is my full-time job, so I can answer a little quicker @malbrecht ;)

    @ren - we could have a shared dropbox folder for these experiments if you want to - might be easier than sending files back and forth in the forum. Or we could use something like Github, which I have never really used, but would like to learn anyways.

  • gnmidiexpertgnmidiexpert Posts: 1

    GNMIDI (http://www.gnmidi.com ) csv conversion surely contains all data
    including columns for note on and note off velocities.
    The csv file can be modified and converted back to midi file.

  • seltzdesignseltzdesign Posts: 80

    @gnmidiexpert said:
    GNMIDI (http://www.gnmidi.com ) csv conversion surely contains all data
    including columns for note on and note off velocities.
    The csv file can be modified and converted back to midi file.

    @gnmidiexpert That is actually precisely what we use. I even mention it in the 4th post in this conversation ;)

    @seltzdesign said:
    There is a cool little software called GNMidi which can convert midi files to .csv - either in note on off format or combines with a note >length.

    I now have a second tool, which can do the same that I coded in Max to use in Ableton Live, where you can write a readable .csv file straight from multiple Ableton tracks without the detour to GNMidi. Its still a great little tool and I have actually used the backwards conversion to test the files generated in Ableton. Upon reimport into Ableton they are exactly identical, meaning it works.

Sign In or Register to comment.