chunking Archives

[Solved] can i convert 3-gram txt to iob for crf suite

October 5, 2022 by Kirat

By the looks of the question, I assume that the input file is in csv format and the IOB2 format looks as though it is space or tab separated tokens. So the simplest way to achieve that format would be to read each line and replace the comma delimiter with a space. # fill in … Read more

[Solved] java code to split text file into chunks based on chunk size

August 24, 2022 by Kirat

That’s because BufferedReader.readLine() reads only a line not the whole file. I assume that the line break characters \r and \n are not part of the normal content you interested in. Maybe that helps. // … StringBuilder sb = new StringBuilder(); String line; while ((line = inputStream.readLine()) != null) { sb.append(line); // if enough content … Read more