Hi,

I have a software which outputs lines of numbers one by one. In another java program, I use BufferedReader as follows:

BufferedReader input = new BufferedReader(new InputStreamReader(pr.getInputStream(),"US-ASCII"));

where pr is the process which runs the software. For each line in the output stream, I try to check whether the first number in the line is greater than a certain value and if it is, I write to another file.

The problem is, when I run the software repeatedly (in a linux cluster) like 10 simulations simultaneously, one of them have got @^@^.... characters followed by some huge numbers. So, I tried eliminating that line alone by using String.matches command but now, it seems even after that I get the error - so, I think it might mean that it is actually during the writing process that the problem occurs. I use BufferedWriter like the following to write it: (I do flush it after writing.)

BufferedWriter out=new BufferedWriter(new FileWriter(tmp_file));

I would greatly appreciate if someone can help. This thing really seems to be a huge bottleneck!

Thanks,
Senthil

 

 

BufferedWriter out=new BufferedWriter(new FileWriter(tmp_file),"US-ASCII");

Also make sure you stop reading at EOF. Also make sure your input really is character data, not binary.

 

 

Oops! Thank you very much. But, how do I verify whether its binary?

Thanks,
Senthil

 

 

Unless you are 100% sure that it is text data, don't use Readers and Writers, use InputStreams and OutputStreams.

 

 

I suggest to use UTF-8 instead of US-ASCII to make your program more robust. What if you ever reads in / writes out a file which contains "special" characters?

Further on agree with ejp: just use InputStream/OutputStream all the way, especially if you don't need to modify anything at character-level in Java.

 

 

Hello! Thanks. It seems I still have the problem.

The simulator just outputs lines of numbers to the linux command line. To redirect the output, I use the >> operator as usual. But as I had to check for the first number in every line, I use the streamreader, read in line by line, split it, check and then write. So, there are no special characters actually. Its 100% numbers only!

Is there any other way I could do the same thing?

Thanks a lot for all your help, once again!

 

 

I tried Scanner class to read in too ... I am still having the problem. I would appreciate any help!

Thanks, in advance!

 

 

Which means at least the numbers are text.


You need to first start by accurately identifying exactly what it is that the programs output. The fact that you see numbers doesn't mean that that is all that is being produced.

But presuming that all that is output is numbers, spaces and end of line characters then you problem is originating somewhere else in your code besides the readers themselves.

 

 

Hi jschell! Thanks!

On your advice, I looked at the source code of the software; it was doing this: System.out.print(output), where output is a StringBuffer with an array of integers appended. Do you think this could have caused the problem?

I have now changed it into System.out.print(output.toString()).

Any suggestions?

 

 

Post your code.

 

 

Hi ejp! Thanks - here is the code!

import java.io.BufferedReader;
import java.io.FileOutputStream;
import java.io.InputStreamReader;
import java.io.PrintStream;
import java.io.;
import java.util.;

//call the class file with the input xml file to simulate
public class RunSim {
public static void main(String[] args) {
String[] xmlFiles = new String[] { "/home/senthil/Simulator/Dessa1.4/" args[0]};
Scanner input=null;
FileWriter ofw=null;
PrintWriter out=null;
for(String file:xmlFiles) {
try {
String[] command = {"java","Test",file, "10000"};
Process pr = Runtime.getRuntime().exec(command); // run the command
input = new Scanner(new InputStreamReader(pr.getInputStream()));
String line=null;
ofw=new FileWriter(file"_out.txt");
out=new PrintWriter(ofw);

while(input.hasNextLine()) {
line=input.nextLine();
System.out.println(line);
if(line.matches("[0-9.\\s]*")) { // I included this as I didn't want the '@^' character - but still, it is being produced.
out.println(line);
out.flush();
String[] str = line.split(" ");
if(!line.equals("Badly formatted XML file")){
if((Float.parseFloat(str[0]) >= 196)) //this is the limit upto which the process has to be run
pr.destroy();
}
}
else
continue;
}
out.close();
int exitVal = pr.waitFor();
System.out.println("Exited with error code "+exitVal);
} catch(Exception e) {
System.out.println(e.toString());
e.printStackTrace();
}
}
}
}

 

 

I doubt that changed the behaviour.

Note however that 'print' is not 'println' yet you are reading lines.

 

 

break;

 

 

That was awesome! Thanks a lot for the support, guys!

I guess, it resolved the problem - since yesterday morning, I haven't got those errors. What was the weirdest thing I was doing here? I thought once the process is destroyed, the while loop automatically exits and there was no need for a break.

But there was one more thing, I noticed. When I place the break command right after the pr.destroy(), for some unknown reasons, only the first line of the simulation was output to the file; the processes were running, though. [I recompiled everything and tried many times. ] Then, I introduced a flag value there and then before closing the while, I just check for the flag's reset value and exit accordingly - now its working perfectly. What could have been the problem?

Thanks once again!

 

 

Buffering inside the process.