Java NIO 2.0 : Memory-Mapped Files | MappedByteBuffer Tutorial
2015-10-15 11:20
489 查看
If you know how java IO works at lower level, then you will be aware of buffer handling, memory paging and other such concepts. For conventional file I/O, in which user processes issue
To do both writing and reading in memory mapped files, we start with a
The file created with the above program is 128 MB long, which is probably larger than the space your OS will allow. The file appears to be accessible all at once because only portions of it are brought into memory, and other parts are swapped out. This way a very large file (up to 2 GB) can easily be modified.
Like conventional file handles, file mappings can be writable or read-only. The first two mapping modes,
You’ll notice that there is no
A
The user process sees the file data as memory, so there is no need to issue
As the user process touches the mapped memory space, page faults will be generated automatically to bring in the file data from disk. If the user modifies the mapped memory space, the affected page is automatically marked as dirty and will be subsequently flushed to disk to update the file.
The virtual memory subsystem of the operating system will perform intelligent caching of the pages, automatically managing memory according to system load.
The data is always page-aligned, and no buffer copying is ever needed.
Very large files can be mapped without consuming large amounts of memory to copy the data.
Reading a Memory-Mapped File
Writing into a Memory-Mapped File
read()and
write()system calls to transfer data, there is almost always one or more copy operations to move the data between these filesystem pages in kernel space and a memory area in user space. This is because there is not usually a one-to-one alignment between filesystem pages and user buffers. There is, however, a special type of I/O operation supported by most operating systems that allows user processes to take maximum advantage of the page-oriented nature of system I/O and completely avoid buffer copies. This is called memory-mapped I/O and we are going to learn few things here around memory-mapped files.
Memory-Mapped Files
Memory-mapped I/O uses the filesystem to establish a virtual memory mapping from user space directly to the applicable filesystem pages. With a memory-mapped file, you can pretend that the entire file is in memory and that you can access it by simply treating it as a very large array. This approach greatly simplifies the code you write in order to modify the file.To do both writing and reading in memory mapped files, we start with a
RandomAccessFile, get a channel for that file. Memory mapped byte buffers are created via the
FileChannel.map()method. This class extends the
ByteBufferclass with operations that are specific to memory-mapped file regions. A mapped byte buffer and the file mapping that it represents remain valid until the buffer itself is garbage-collected. Note that you must specify the starting point and the length of the region that you want to map in the file; this means that you have the option to map smaller regions of a large file.
import java.io.RandomAccessFile; import java.nio.MappedByteBuffer; import java.nio.channels.FileChannel; public class MemoryMappedFileExample { static int length = 0x8FFFFFF; // 128 Mb public static void main(String[] args) throws Exception { MappedByteBuffer out = new RandomAccessFile("howtodoinjava.dat", "rw") .getChannel().map(FileChannel.MapMode.READ_WRITE, 0, length); for (int i = 0; i < length; i++) { out.put((byte) 'x'); } System.out.println("Finished writing"); } }
The file created with the above program is 128 MB long, which is probably larger than the space your OS will allow. The file appears to be accessible all at once because only portions of it are brought into memory, and other parts are swapped out. This way a very large file (up to 2 GB) can easily be modified.
Like conventional file handles, file mappings can be writable or read-only. The first two mapping modes,
MapMode.READ_ONLYand
MapMode.READ_WRITE, are fairly obvious. They indicate whether you want the mapping to be read-only or to allow modification of the mapped file. The third mode,
MapMode.PRIVATE, indicates that you want a copy-on-write mapping. This means that any modifications you make via
put( )will result in a private copy of the data that only the
MappedByteBufferinstance can see. No changes will be made to the underlying file, and any changes made will be lost when the buffer is garbage collected. Even though a copy-on-write mapping prevents any changes to the underlying file, you must have opened the file for read/write to set up a
MapMode.PRIVATEmapping. This is necessary for the returned
MappedByteBufferobject to allow
put()s.
You’ll notice that there is no
unmap()method. Once established, a mapping remains in effect until the
MappedByteBufferobject is garbage collected. Also, mapped buffers are not tied to the channel that created them. Closing the associated
FileChanneldoes not destroy the mapping; only disposal of the buffer object itself breaks the mapping.
A
MemoryMappedBufferhas a fixed size, but the file it’s mapped to is elastic. Specifically, if a file’s size changes while the mapping is in effect, some or all of the buffer may become inaccessible, undefined data could be returned, or unchecked exceptions could be thrown. Be careful about how files are manipulated by other threads or external processes when they are memory-mapped.
Advantages of Memory-Mapped Files
Memory-Mapped IO have several advantages over normal I/O:The user process sees the file data as memory, so there is no need to issue
read()or
write()system calls.
As the user process touches the mapped memory space, page faults will be generated automatically to bring in the file data from disk. If the user modifies the mapped memory space, the affected page is automatically marked as dirty and will be subsequently flushed to disk to update the file.
The virtual memory subsystem of the operating system will perform intelligent caching of the pages, automatically managing memory according to system load.
The data is always page-aligned, and no buffer copying is ever needed.
Very large files can be mapped without consuming large amounts of memory to copy the data.
Reading a Memory-Mapped File
To read a file using memory mapped IO, use below code template: import java.io.File; import java.io.RandomAccessFile; import java.nio.MappedByteBuffer; import java.nio.channels.FileChannel; public class MemoryMappedFileReadExample { private static String bigExcelFile = "bigFile.xls"; public static void main(String[] args) throws Exception { //Create file object File file = new File(bigExcelFile); //Get file channel in readonly mode FileChannel fileChannel = new RandomAccessFile(file, "r").getChannel(); //Get direct byte buffer access using channel.map() operation MappedByteBuffer buffer = fileChannel.map(FileChannel.MapMode.READ_ONLY, 0, fileChannel.size()); // the buffer now reads the file as if it were loaded in memory. System.out.println(buffer.isLoaded()); //prints false System.out.println(buffer.capacity()); //Get the size based on content size of file //You can read the file from this buffer the way you like. for (int i = 0; i < buffer.limit(); i++) { System.out.print((char) buffer.get()); //Print the content of file } } }
Writing into a Memory-Mapped File
To write data into a file using memory mapped IO, use below code template: import java.io.File; import java.io.RandomAccessFile; import java.nio.MappedByteBuffer; import java.nio.channels.FileChannel; public class MemoryMappedFileWriteExample { private static String bigExcelFile = "test.txt"; public static void main(String[] args) throws Exception { // Create file object File file = new File(bigExcelFile); //Delete the file; we will create a new file file.delete(); // Get file channel in readonly mode FileChannel fileChannel = new RandomAccessFile(file, "rw").getChannel(); // Get direct byte buffer access using channel.map() operation MappedByteBuffer buffer = fileChannel.map(FileChannel.MapMode.READ_WRITE, 0, 4096 * 8 * 8); //Write the content using put methods buffer.put("howtodoinjava.com".getBytes()); } }
相关文章推荐
- iOS 心得四 GCD倒计时的写法
- iOS 闭包中的[weak self]在什么情况下需要使用,什么情况下可以不加?
- 从新建项目到打包成APK(Cocos2d-x 2.2.1)
- 专题 - Web应用->Web应用中适配不同屏幕
- 【IOS 开发学习总结-OC-49】★★ios开发之UI控件——UIStepper与UIWebView
- iOS容易造成循环引用的三种场景
- Nagios介绍
- android获取SDCard中照片并按照时间顺序显示
- ios 基于NSConnection简单封装的工具类
- 苹果Swift编程语言新手教程【中国版】
- 有关ScrollView嵌套WebView使用时需要注意的一点小问题
- Android Bitmap实战技巧
- android:强大的图片下载和缓存库Picasso
- 关于iOS9中的App Transport Security相关说明及适配
- Building Apps Faster with Android Data Binding
- iOS Font
- android studio -- 图片应该放在drawable还是mipmap?
- 剥开ios 系统sandbox神秘面纱
- iOS:iOS中的几种动画
- eclipse+ADT使用第三方静态库及COCOS2d问题汇总