Java File IO Operation Efficiency Comparison
Note: This article only compares at the time consumption level, the memory footprint level requires special attention!
1. Description of parameters
Total file size: 2,111,993,850 bytes (2.11 GB)
static String defaultFilePath = "/tmp/";
Buffer size: 8192
static int defaultByteLength = 1024 * 8;
2. Presentation of examples
The data file was read in several ways and 10 consecutive tests were performed:
- FileInputStream + byte[] file byte input stream + byte array reading method
- FileInputStream + Scanner File Byte Input Stream + Scanner reading method
- FileReader + char[] file character input stream + character array method
- BufferedReader buffered character stream method
- FileChannel File Input/Output Pipeline Stream "NIO
3. Comparison results
「5. FileChannel」 > 「1. FileInputStream + byte[]」> 「3. FileReader + char[]」>= 「4. BufferedReader」 > 「2. FileInputStream + Scanner」
In the operation of the file, the file will be distinguished into large files, small files, text files, binary files, etc., according to the different files need to choose the appropriate way to read. Usually large files are recommended to use "FileChannel" for higher efficiency, and small files can be read by IO or NIO.
4. Example code
4.1. FileInputStream + byte[] file byte input stream + byte array reading method
/**
* FileInputStream + byte[] method
* Equivalent to BufferedInputStream byte input buffer stream
* FileInputStream + byte array reading method
* Suitable for: binary files or non-text files.
*/
public void testFileInputStreamWithBytes() {
long startTime = new Date().getTime();
// Use the try wrapper
try (FileInputStream fileInputStream = new FileInputStream(defaultFilePath)){
byte[] reads = new byte[defaultByteLength]; int readCount; reads = new byte[defaultByteLength]; // use try packing
int readCount.
while ((readCount = (reads)) ! = -1) {
// TODO process the data
}
} catch (IOException e) {
("Read file exception [ %s ]%n", ()); }
}
("FileInputStream + byte[] way to read file used a total of %d milliseconds %n", new Date().getTime() - startTime); }
}
The results of the 10 tests are as follows
FileInputStream + byte[] uses 884 milliseconds to read a file.
FileInputStream + byte[] uses 331 milliseconds to read the file.
FileInputStream + byte[] reads the file in 319 milliseconds.
FileInputStream + byte[] uses 420 milliseconds to read the file.
FileInputStream + byte[] reads the file in 333 milliseconds.
FileInputStream + byte[] reads the file for 321 milliseconds.
FileInputStream + byte[] reads the file in 327 milliseconds.
FileInputStream + byte[] reads the file in 339 milliseconds.
FileInputStream + byte[] reads the file in 328 milliseconds.
FileInputStream + byte[] reads the file for 398 milliseconds.
4.2. FileInputStream + Scanner File Byte Input Stream + Scanner Reading Methods
/**
* FileInputStream + Scanner method
* FileInputStream + Scanner reads text.
* For: text files
*/
public void testFileInputStreamWithScanner() {
long startTime = new Date().getTime(); // Use try to wrap the file with a byte input stream.
// Use the try wrapper
try (FileInputStream fileInputStream = new FileInputStream(defaultFilePath)){
Scanner scanner = new Scanner(fileInputStream); while (()) { { new FileInputStream(defaultFilePath)){
while (()) {
().
// TODO processing the data
}
} catch (IOException e) {
("Read file exception [ %s ]%n", ()); }
}
("FileInputStream + Scanner way to read file using a total of %d milliseconds %n", new Date().getTime() - startTime); }
}
The results of the 10 tests are as follows
No buffer, performance drops dramatically!
FileInputStream + Scanner uses 16755 milliseconds to read a file.
FileInputStream + Scanner uses 18744 milliseconds to read the file.
FileInputStream + Scanner uses 17929 milliseconds to read the file.
FileInputStream + Scanner reads the file for 18640 milliseconds.
FileInputStream + Scanner reads the file for 18316 milliseconds.
FileInputStream + Scanner reads the file for 18015 milliseconds.
FileInputStream + Scanner reads the file for 18479 milliseconds.
FileInputStream + Scanner reads the file for 18755 milliseconds.
FileInputStream + Scanner uses 18907 milliseconds to read the file.
FileInputStream + Scanner reads the file for 18783 milliseconds.
4.3. FileReader + char[] file character input stream + character array approach
/**
* FileReader + char[] method
* Equivalent to BufferedReader character input buffer stream
* FileReader + char array method
* For: character files
*/
public void testFileReaderWithChars() {
long startTime = new Date().getTime();
// Use the try wrapper
try (FileReader fileReader = new FileReader(defaultFilePath)){
int readCount.
while ((readCount = (reads)) ! = -1) {
// TODO process the data
}
} catch (IOException e) {
("Read file exception [ %s ]%n", ()); }
}
("FileReader + char[] way to read file used a total of %d milliseconds %n", new Date().getTime() - startTime); }
}
The results of the 10 tests are as follows
FileReader + char[] uses 922 milliseconds to read the file.
FileReader + char[] uses 971 milliseconds to read the file.
FileReader + char[] reads the file for 842 milliseconds.
FileReader + char[] reads the file in 985 milliseconds.
FileReader + char[] reads the file in 868 milliseconds.
FileReader + char[] reads the file in 1207 milliseconds.
FileReader + char[] reads the file in 1031 milliseconds.
FileReader + char[] reads the file in 981 milliseconds.
FileReader + char[] reads the file in 1259 milliseconds.
FileReader + char[] reads the file in 1034 milliseconds.
4.4. BufferedReader buffered character stream method
/**
* BufferedReader way (of life)
* 缓冲字符流way (of life)
* pertain：character file
*/
public void testBufferedReader() {
long startTime = new Date().getTime();
// utilization try wrap
try (BufferedReader fileReader = new BufferedReader(new FileReader(defaultFilePath))){
String line;
while ((line = ()) != null) {
// TODO Processing data
}
} catch (IOException e) {
("Read File Exception[ %s ]%n", ());
}
("BufferedReader way (of life) 读取文件共utilization %d millisecond%n", new Date().getTime() - startTime);
}
The results of the 10 tests are as follows
The BufferedReader method uses 1870 milliseconds to read the file.
The BufferedReader method uses 1895 milliseconds to read the file.
The BufferedReader method uses 1890 milliseconds to read the file.
The BufferedReader method uses 1875 milliseconds to read the file.
The BufferedReader method uses 1829 milliseconds to read the file.
The BufferedReader method uses 2060 milliseconds to read the file.
The BufferedReader method uses 1821 milliseconds to read the file.
BufferedReader reads the file for 1944 milliseconds.
BufferedReader reads a file for 1902 milliseconds.
BufferedReader reads the file for 1860 milliseconds.
4.5. FileChannel File Input/Output Pipeline Stream "NIO
/**
* FileChannel method
* File input and output pipeline streams
*/
public void testFileChannel() {
long startTime = new Date().getTime(); // Use try to wrap the file channel.
// Use the try wrapper
try (FileChannel channel = ((defaultFilePath), )){
// Constructing a B yteBuffer has two methods, and , both of which have the same input parameters but different meanings.
// (capacity) allocates a non-direct buffer; operations on non-direct buffers are performed in Java heap memory, and data is passed through Java heap memory for reading and writing.
// (capacity) is allocated as a direct buffer, direct buffer operations can be passed through local I/O, avoiding data transfers between the Java heap and the local heap.
ByteBuffer buf = (defaultByteLength);
while ((buf) ! = -1) {
();
// TODO process the data
// (new String(())).
(); }
}
} catch (IOException e) {
("Read File Exception [ %s ]%n", ()); }
}
("FileChannel way to read a file using a total of %d milliseconds %n", new Date().getTime() - startTime); }
}
The results of the 10 tests are as follows
The FileChannel method uses 314 milliseconds to read a file.
The FileChannel method uses 293 milliseconds to read the file.
FileChannel reads the file in 332 milliseconds.
FileChannel reads the file for 296 milliseconds.
FileChannel reads the file for 285 milliseconds.
FileChannel reads the file for 290 milliseconds.
FileChannel uses 283 milliseconds to read the file.
FileChannel reads the file for 282 milliseconds.
FileChannel reads the file for 298 milliseconds.
FileChannel reads the file for 280 milliseconds.
5. Concluding remarks
In Java 8, "FileChannel" is an efficient way to handle file I/O, which is more flexible, convenient and efficient than traditional I/O streams such as "FileInputStream" and "FileOutputStream".
6. Code appendix
package ;
import ;
import ;
import ;
import ;
import ;import ;import ;import ;import
import ;import ;import ;import ;import
import ;import ;import ;import ;import
import ;import ;import ;import ;import
import ;import ;import ;import ;import
import ;import ;import ;import ;import
import ;
/**
* File reading
* @date 2024-08-14 16:42:21
*/
public class FileReadTest {
// File path
static String defaultFilePath = "/Users/changbeibei/Desktop/work/-24081412";
// 8k
static int defaultByteLength = 1024 * 8; // 8k
/**
* FileInputStream + byte[] method
* Equivalent to BufferedInputStream.
* FileInputStream + byte[] method * Equivalent to BufferedInputStream byte input buffer * FileInputStream + byte array read method
* Suitable for: binary files or non-text files.
*/
@Test
public void testFileInputStreamWithBytes() {
long startTime = new Date().getTime();
// Use the try wrapper
try (FileInputStream fileInputStream = new FileInputStream(defaultFilePath)){
byte[] reads = new byte[defaultByteLength]; int readCount; reads = new byte[defaultByteLength]; // use try packing
int readCount.
while ((readCount = (reads)) ! = -1) {
// TODO process the data
}
} catch (IOException e) {
("Read file exception [ %s ]%n", ()); }
}
("FileInputStream + byte[] way to read file used a total of %d milliseconds %n", new Date().getTime() - startTime); }
}
/**
* FileInputStream + Scanner method
* FileInputStream + Scanner method.
* For: text files
*/
@Test
public void testFileInputStreamWithScanner() {
long startTime = new Date().getTime(); // Use try to wrap the file input stream.
// Use the try wrapper
try (FileInputStream fileInputStream = new FileInputStream(defaultFilePath)){
Scanner scanner = new Scanner(fileInputStream); while (()) { { new FileInputStream(defaultFilePath)){
while (()) {
().
// TODO processing the data
}
} catch (IOException e) {
("Read file exception [ %s ]%n", ()); }
}
("FileInputStream + Scanner way to read file using a total of %d milliseconds %n", new Date().getTime() - startTime); }
}
/**
* FileReader + char[] method
* Equivalent to BufferedReader character input buffer stream
* FileReader + char array method
* For: character files
*/
@Test
public void testFileReaderWithChars() {
long startTime = new Date().getTime();
// Use the try wrapper
try (FileReader fileReader = new FileReader(defaultFilePath)){
int readCount.
while ((readCount = (reads)) ! = -1) {
// TODO process the data
}
} catch (IOException e) {
("Read file exception [ %s ]%n", ()); }
}
("FileReader + char[] way to read file used a total of %d milliseconds %n", new Date().getTime() - startTime); }
}
/**
* BufferedReader method
* BufferedReader method
* For: character files
*/
@Test
public void testBufferedReader() {
long startTime = new Date().getTime();
// Use the try wrapper
try (BufferedReader fileReader = new BufferedReader(new FileReader(defaultFilePath))){
String line;
while ((line = ()) ! = null) {
// TODO process the data
}
} catch (IOException e) {
("Read file exception [ %s ]%n", ()); }
}
("BufferedReader way to read file used a total of %d milliseconds %n", new Date().getTime() - startTime); }
}
/**
* FileChannel method
* File input and output pipeline streams
*/
@Test
public void testFileChannel() {
long startTime = new Date().getTime(); // Use try to wrap the file channel.
// Use the try wrapper
try (FileChannel channel = ((defaultFilePath), )){
// Constructing a B yteBuffer has two methods, and , both of which have the same input parameters but different meanings.
// (capacity) allocates an undirected buffer; operations on undirected buffers take place in Java heap memory, and reads and writes of data are passed through Java heap memory.
// (capacity) is allocated as a direct buffer, direct buffer operations can be passed through local I/O, avoiding data transfers between the Java heap and the local heap.
ByteBuffer buf = (defaultByteLength);
while ((buf) ! = -1) {
();
// TODO process the data
// (new String(())).
(); }
}
} catch (IOException e) {
("Read File Exception [ %s ]%n", ()); }
}
("FileChannel way to read file using %d milliseconds %n", new Date().getTime() - startTime); }
}
@Test
public void testMain() {
for (int i = 0; i < 10; i++) {
("The %dth test %n", i + 1);
testFileInputStreamWithBytes();
testFileInputStreamWithScanner(); testFileReaderWithScanner(); testFileReaderWithScanner()
testFileReaderWithChars(); testBufferedReader(); testBufferedReader()
testBufferedReader(); testFileChannel(); testFileChannel(); testFileChannel()
testFileChannel();
}
}
}