Split text file into lines

I am reading a text file with a line of float values separated by comma and new lines.
I would now like to use String.Split to first split the contents in an array of lines and then use Split again to get an array of arrays.

I was trying to use "\n" as the separator in Split, but it doesn't seem to work looking at the reported output.

How can I split by lines?

Tagged:
«1

Comments

  • seltzdesignseltzdesign Posts: 80

    btw: The data is transformation (Tx,Ty,Tz,Sx,Sy,Sz,Rx,Ry,Rz) data and needs to end up being an array of XFO's. I also can't seem to find a String.ToScalarArray node!?

  • malbrechtmalbrecht Fabric for Houdini Posts: 752 ✭✭✭

    Moin,

    works fine here - the trick is to not use "\n" (escaped text) but to actually input the line feed. Open the split node in your value editor (double click on it) and instead of typing "\n" into the separator field, click on the magnifying glas and enter a single line feed as the separator.

    Marc


    Marc Albrecht - marc-albrecht.de - does things.

  • seltzdesignseltzdesign Posts: 80

    Right, ok, I got it. But how can I split it again at the commas? No String.SplitArray! Or do I have to use a block and for loop for that?

    Can you even have arrays inside arrays in Fabric?

  • malbrechtmalbrecht Fabric for Houdini Posts: 752 ✭✭✭

    Yup, for what you want to do, you will need to use a loop or create your own block.

    Can you even have arrays inside arrays in Fabric?

    Sure. See docs.fabric-engine.com/FabricEngine/2.3.0/HTML/KLProgrammingGuide/types.html#index-16


    Marc Albrecht - marc-albrecht.de - does things.

  • seltzdesignseltzdesign Posts: 80

    Okay, interesting and good to know. Still trying to work out how Blocks work and how to create one. Step by step...

  • seltzdesignseltzdesign Posts: 80

    Something like this? It throws an error, so I am still doing something wrong.

  • malbrechtmalbrecht Fabric for Houdini Posts: 752 ✭✭✭

    for arrays you want to use "size", not "length" :)


    Marc Albrecht - marc-albrecht.de - does things.

  • seltzdesignseltzdesign Posts: 80

    Right, ach, always everything is called something else in every language/environment.

    So its getting there, but now I just have one array with all the values, not 18 arrays with 12 values.

    Actually just noticed that they are transformation matrixes (mat44) not XFO's. Any way to convert them straight away? I could have the script that outputs the file write it out in the proper format. Then I would just have to split the lines and use some sort of String.ToMat44 node!?

  • malbrechtmalbrecht Fabric for Houdini Posts: 752 ✭✭✭

    got to leave in a minute ...

    For Mat44 to Xfo you can use "setFromMat44". You may have to convert the string entries to Scalar first, though (using "toScalar"). When trying to develop something visually (using nodes), I create my own set of converters first, put them into subgraphs and thus get a cleaned up, more or less "understandable" graph. But I am an absolute no-visual-developer-developer, so I may be way off with this idea :)

    Marc


    Marc Albrecht - marc-albrecht.de - does things.

  • seltzdesignseltzdesign Posts: 80

    Thats how I would/will do it as well, even though I am a mainly-visual-developer. In the end all this will be one subgraph just for getting the text from file and spitting out an array of XFOs or Mat44s in the end. Then the fun starts.. :smiley:

    Thanks for your help. Have a nice weekend!

  • seltzdesignseltzdesign Posts: 80
    edited August 2016

    So continuing the quest to get transforms from a text file, I played with the seltz-csv-reader2.canvas file and wanted to get not just a single value, but all values with the same key for each line. I wish there was a better explaination how blocks work, because it is very confusing right now.
    When I thought I almost had it, I still get an error message: No such method Float32.push. What am I doing wrong?

    Thanks, Armin.

  • EricTEricT Administrator, Moderator, Fabric Employee Posts: 305 admin

    In that modify preset in the modify block, you are working with a single value from the array. You are essentially trying to use the Array.Push() method on a single Float32 value.

    Eric Thivierge
    Kraken Developer
    Kraken Rigging Framework

  • malbrechtmalbrecht Fabric for Houdini Posts: 752 ✭✭✭
    edited August 2016

    "Push" is a method on an array. So you need to create an array, onto which you can push the float.
    The block (or modifier in there) deals with the single value ON THE array, so you need to pass the array itself into the block code.

    I hope this helps!

    Marc

    P.S. Eric beat me to it ... if you are stuck for another four hours, I'll quick-hack something ... but I am almost absolutely certain you'll get it figured out by then ;)


    Marc Albrecht - marc-albrecht.de - does things.

  • EricTEricT Administrator, Moderator, Fabric Employee Posts: 305 admin

    Just an FYI, we're discussing internally how we can better make users aware of how to use blocks. Stay tuned.

    Eric Thivierge
    Kraken Developer
    Kraken Rigging Framework

  • malbrechtmalbrecht Fabric for Houdini Posts: 752 ✭✭✭

    quick thought (still busy): I think your block is actually fine, you just need to kick out the "push" node, since you are modifying the "current" value on the array stack anyway ...


    Marc Albrecht - marc-albrecht.de - does things.

  • seltzdesignseltzdesign Posts: 80

    Thanks all for the input. Getting closer to figuring this out! I think the hard part is exactly the figuring out what part of the array is already "in" the block and how the values are set.

    Anyways I am now trying with Set[] and passing an empty array in there. It seems to work fine except that I get the same value from the data for each array member instead of a different value for each.

    @malbrecht : this is a long-term thing and I am in the lucky position to be learning new things at work. I see the great potential in Fabric, so I wont be giving up on this anytime soon ;)

  • seltzdesignseltzdesign Posts: 80

    @EricT Yes, please. The possibilities of blocks seems great, but its just really hard to get your head around it, because each block has 2 parts to it - the block itsself and whats in the block. That's why I got confused using the modify block, because it already contains a for loop in the block itsself.

    I think just starting really simple and going through some examples step by step will help. Just watching someone build something and explaining what they are doing really helps to eventually get the hang of something.

    Have you (or someone you know) thought about Fabric Workshops IRL or online? The tutorial videos are great, but are really quick and dont go into much detail. Also I think the online documentation still has miles to go - for example I wasnt sure which node to use for filling an array - if I dont know what something is called already there is simply no way to know how I would do that. I was searching for "array", but got nothing that remotely helped me in my endeavor.

    I think it would be great in Canvas in the Explorer if we could get a tooltip for each entry to see what it does (or even link to documentation), without having to place it first. That way I could have narrowed it down pretty quickly. Lastly I think it would be great to get a help demo file for each node, similar to how Max/Msp or vvvv do it - select a node, press F1 and get a file with explanaition and demo content.

  • EricTEricT Administrator, Moderator, Fabric Employee Posts: 305 admin

    Hi @seltzdesign

    We are aware of the difficulties of finding nodes that you need. We have multiple issues logged for this already.

    For the blocks training we are thinking of the best way to get people the information they need in regards to the way to use them and how to present step by step explanations. Not sure about live workshops but I'll try to discuss with the others.

    For filling and array, there is Array.Fill and Array.FillInterpolated presets. Hope you found them.

    That's a great suggestion about tool tips before you drop nodes. I've logged the issue FE-7211

    Eric Thivierge
    Kraken Developer
    Kraken Rigging Framework

  • seltzdesignseltzdesign Posts: 80

    @EricT That sounds good. I think the tutorial videos are really good, we just need a few more and a bit more indepth.

    I did see the Array.Fill and Array.FillInterpolated presets, but didnt quite know how to use them in conjunction with blocks. It wasnt quite clear wether I pass an empty array in and then let the loop edit each item or what. Anyways, getting closer ;)

  • EricTEricT Administrator, Moderator, Fabric Employee Posts: 305 admin

    The Fill preset takes a single value so String or Vec3 and then creates an array of the size you set with the string or vec3 or other value as the default value for each item in that array.

    FillInterpolated will fill the array and distribute the entry values between the start / end inputs.

    Eric Thivierge
    Kraken Developer
    Kraken Rigging Framework

  • EricTEricT Administrator, Moderator, Fabric Employee Posts: 305 admin

    @seltzdesign

    What is the format of the CSV file? You stated at the top it was 9 values but there are actually 12. How do they map?

    Eric Thivierge
    Kraken Developer
    Kraken Rigging Framework

  • EricTEricT Administrator, Moderator, Fabric Employee Posts: 305 admin

    @seltzdesign well I'm not sure why there are 12 values per line and not 9 but I created a graph that splits the file into an array of lines, then split that into an array split by ",". Then used the first 9 items to build SRT values that I construct an Xfo from.

    I do this on the arrays and created a custom node from the ForLoop to do this. One thing to note is that you can't convert from String > Scalar without using the Util extension so I created a custom function node that is in the graph that does this.

    I create an empty Xfo array and push items into it within the for loop.

    Let me know if you have any more questions.

    Eric Thivierge
    Kraken Developer
    Kraken Rigging Framework

  • borjaborja Administrator, Fabric Employee Posts: 480 admin

    Hi @seltzdesign

    I also filed FE-7212 for the suggestion of adding a document page for each preset with a sample scene.

    Borja Morales
    Technical Product Manager
    Fabric Software Inc.

  • seltzdesignseltzdesign Posts: 80
    edited August 2016

    @EricT Wow, thank you so much, thats really great. Just looking through it to understand what you did.
    Sorry, I seemed to remember it was 9 values (XFO), but they are actually Transformation Matrixes (4x4), thats why there are 12 values (the last 4 values are always 0,0,0,1 for now).

    I edited the file to work with Mat44, but there is still one more slight problem. For some reason the very first entry in the array that is reported is empty and index 1 is actually the first line in the file. Do you have any idea why that is happening?

  • EricTEricT Administrator, Moderator, Fabric Employee Posts: 305 admin

    Hi @seltzdesign,

    Yes I just took a look at your Mat44[] you created that is connected to the XfoArrayFromLines has a count set to 1. That means the array starts with 1 item then the XfoArrayFromLines node adds items in after that one. Just change the count to 0 and it should be correct.

    Eric Thivierge
    Kraken Developer
    Kraken Rigging Framework

  • seltzdesignseltzdesign Posts: 80

    Hello @EricT
    Ah nice one, didn't see that and didnt know you could create an array with 0 elements. Works perfectly now!
    Now for the next step: Loading a file for each frame and linking with timeline. I would imagine, that loading the file for each frame separately while the timeline is playing is probably a performance killer, so I guess I put everything in another for loop, read all the files in a folder, make one large array of arrays, cache it, then get the right array for each frame of the timeline. Does that sound correct to you?

    Thanks, Armin.

  • malbrechtmalbrecht Fabric for Houdini Posts: 752 ✭✭✭

    sounds good to me at least :)


    Marc Albrecht - marc-albrecht.de - does things.

  • EricTEricT Administrator, Moderator, Fabric Employee Posts: 305 admin

    Yes that would be my approach. You're going to essentially cache the arrays. You'll want to look into cache nodes and variable for this work.

    You'll probably also need to look into the FileIO extension so you can setup a small graph to look into a directory, find all the valid files in the sequence, and read them in and cache the arrays.

    Eric Thivierge
    Kraken Developer
    Kraken Rigging Framework

  • seltzdesignseltzdesign Posts: 80

    Update: Ok, just tried it and its reading one file per frame at 40 fps. Still I guess reading all files and caching is the better method. Lets see if I can do this! cracks fingers

  • seltzdesignseltzdesign Posts: 80

    Grr, ok, I give up. I know too little of KL to get this to work. Can someone point me in the right direction of how to push an array into a 2D array please.

    I tried several approaches, but ultimately it always ended in an empty array or wouldnt let me execute the topmost for loop.

    Could someone please take a look how to do this. I am learning a lot, but coming from other programming languages its just not feasable, because I dont know the right syntax. Having to deal with for loops is pretty new to me in visual programming languages as all others I know well dont use/need those.

    I am attaching the test text files. Thanks for your help everyone!

«1
Sign In or Register to comment.