1 // Copyright (c) 2008-2009 Nokia Corporation and/or its subsidiary(-ies).
2 // All rights reserved.
3 // This component and the accompanying materials are made available
4 // under the terms of "Eclipse Public License v1.0"
5 // which accompanies this distribution, and is available
6 // at the URL "http://www.eclipse.org/legal/epl-v10.html".
8 // Initial Contributors:
9 // Nokia Corporation - initial contribution.
19 #include <f32file64.h>
22 The RFileBuf64 class provides buffered file read/write operations on a single RFile64 object.
23 RFileBuf64::Read() and RFileBuf64::Write() methods may boost the performance of the read/write file operations up to 30%
24 if compared to their RFile64 equivalents. Especially this is true in the case of a set of sequential read or write
25 operations when the file offset of operation N+1 is the end file offset of operation N plus one byte.
27 The RFileBuf64 public methods declarations match the declarations of the most often used RFile64 public methods.
28 That makes the source code migration from RFile64 to RFileBuf64 easier (or from RFileBuf64 to RFile64).
30 The RFileBuf64 capabilities are similar to those of the RFileBuf class, except the fact that RFileBuf
31 uses 32-bit file offsets, RFileBuf64 uses 64-bit file offsets and RFileBuf64 provides optimised read-ahead operations,
32 crafted especially for the case when RFileBuf64 is used with a page based database management system (like SQLite).
36 - an object of RFileBuf64 type must be defined first, specifying the max size (capacity) of the buffer as a parameter
39 RFileBuf64 fbuf(<N>);//<N> is the minimal buffer capacity in bytes
41 - the second step is to initialize the just defined RFileBuf64 object by calling one of the "resource acquisition"
42 methods: RFileBuf64::Create(), RFileBuf64::Open(), RFileBuf64::Temp() or RFileBuf64::AdoptFromClient().
44 In details, to create a file and access it through a RFileBuf64 object:
47 TInt err = fs.Connect();
50 RFileBuf64 fbuf(<N>); //<N> is the buffer capacity in bytes
51 err = fbuf.Create(fs, <file name>, <file mode>);
54 To open an existing file and access it through a RFileBuf64 object:
57 TInt err = fs.Connect();
60 RFileBuf64 fbuf(<N>); //<N> is the buffer capacity in bytes
61 err = fbuf.Open(fs, <file name>, <file mode>);
64 To create a temporary file and access it through a RFileBuf64 object:
67 TInt err = fs.Connect();
70 RFileBuf64 fbuf(<N>); //<N> is the buffer capacity in bytes
71 err = fbuf.Temp(fs, <path>, <file name>, <file mode>);
74 To open a file from handle and access it through a RFileBuf64 object:
77 TInt err = fs.Connect();
80 RFileBuf64 fbuf(<N>); //<N> is the buffer capacity in bytes
81 err = fbuf.AdoptFromClient(<msg>, <fs handle index>, <file handle index>);
84 - if the RFileBuf64 object is initialised successfully, now the public RFileBuf64 methods can be called to perform
85 requested operations on the file:
87 err = fbuf.Write(<file pos>, <data>);
90 err = fbuf.Read(<file pos>, <buf>);
93 Note: do not forget to call RFileBuf64::Flush() at the moment when you want to ensure that the file data
94 (possibly buffered) is really written to the file.
96 - The final step is to close the RFileBuf64 object and the corresponding file thus realising the used resouces:
102 Implementation notes: the current RFileBuf64 implementation is optimised for use by the SQLite OS porting layer.
103 After investigation of SQLite file read/write operations it was found that buffering of
104 two or more logical file writes into a single physical file write has a positive impact (as expected) on the performance
105 of the database write operations. But the picture is quite different for the file read operations. The database data is
106 organised in pages with fixed size. After a database is created and set of insert/update/delete operations is performed
107 on it, after a while the database pages (identified by their numbers) are not sequential in the database file and using
108 a read-ahead buffer with fixed size makes no sense because for each "page read" request of N bytes, the RFileBuf64 object
109 will read up to K bytes, K >= N, where K is the read-ahead value in bytes. Since the "read page" requests in general are
110 not sequential (relatively to the page numbers), obviously the read-ahead data is wasted and the "read page" performance
111 is negatively impacted. This observation is true in general except two cases:
112 - sequential scan of a database table;
113 - reading of a BLOB column;
114 In these two cases it is likely that the table/blob data occupies pages with sequential numbers located in a continuous
115 file area. Then if a read-ahead buffer is used that will have a positive impact on the "read page" performance, because
116 the number of the read IPC calls to the file server will be reduced.
117 In order to satisfy those two orthogonal requirements, the RFileBuf64 implementation uses a "read file offset prediction"
119 - The file buffer object is created with 0 read-ahead value;
120 - After each "file read" operation the buffer implementation "guesses" what might be the file offset of the
121 next file read operation (it is possible because the database data is organised in pages with fixed size and
122 generally all "file read" requests are for reading a database page) and stores the value in one of its data members;
123 - If the file offset of the next file read operation matches the "guessed" offset, the read-ahead value is changed from
124 0 to 1024 bytes (1024 bytes is the default database page size);
125 - Every next match of the "guessed" file offset value doubles the read-ahead value. But the max read-ahead value is
126 capped by the capacity of the buffer;
127 - If the file offset of the next file read operation does not match the "guessed" file offset, then the read-ahead
128 value is set back to 0;
129 Shortly, depending of the nature of the file read requests, the RFileBuf64 object will dynamically change the read-ahead
130 value thus minimising the amount of the wasted read data and improving the "file read" performance in general.
139 friend void SetReadAheadSizeTest();
141 enum {KDefaultReadAheadSize = 1024};//Default size in bytes of the read-ahead buffer
144 RFileBuf64(TInt aMinCapacity);
146 TInt Create(RFs& aFs, const TDesC& aFileName, TUint aFileMode);
147 TInt Open(RFs& aFs, const TDesC& aFileName, TUint aFileMode);
148 TInt Temp(RFs& aFs, const TDesC& aPath, TFileName& aFileName, TUint aFileMode);
149 TInt AdoptFromClient(const RMessage2& aMsg, TInt aFsIndex, TInt aFileIndex);
151 TInt SetReadAheadSize(TInt aBlockSize, TInt aReadRecBufSize);
153 TInt Read(TInt64 aFilePos, TDes8& aDes);
154 TInt Write(TInt64 aFilePos, const TDesC8& aData);
156 TInt Size(TInt64& aFileSize);
157 TInt SetSize(TInt64 aFileSize);
160 TInt Drive(TInt& aDriveNumber, TDriveInfo& aDriveInfo) const;
163 void Invariant() const;
165 TInt DoPostInit(TInt aInitErr);
168 TInt DoSetFileSize(TInt64 aFileSize);
171 TInt DoFileWrite1(TInt64 aNewFilePos);
172 TInt DoFileWrite2(TInt64 aNewFilePos = 0LL);
173 void DoDiscardBufferedReadData();
174 TInt DoSetCapacity(TInt aRwDataLength);
178 TInt iCapacity; //The buffer size. Indicates how much data can be put in.
179 TUint8* iBase; //Pointer to the beginning of the buffer.
180 TInt iLength; //The length of the data currently held in the buffer.
182 TInt64 iFilePos; //The file position associated with the beginning of the buffer.
183 TInt64 iFileSize; //The file size which is the nominal file length including the buffer contents.
184 TInt64 iRealFileSize; //The real file size in disk.
185 RFile64 iFile; //The file object.
187 TBool iDirty; //The buffer contains pending data to be written to the file
188 TInt64 iNextReadFilePos; //The guessed file position of the next "file read" operation
189 TInt iNextReadFilePosHits; //How many times the guessed file position of the "file read" operation was correct
192 TBool iOptimized; //True if the file buffer capacity has been optimized already
197 void ProfilerReset();
199 TInt iFileReadCount; //The number of the non-buffered file reads (RFile64::Read() calls).
200 TInt64 iFileReadAmount; //The amount of the data read from the file.
201 TInt iFileWriteCount; //The number of the non-buffered file writes (RFile64::Write() calls).
202 TInt64 iFileWriteAmount; //The amount of the data written to the file.
203 TInt iFileSizeCount; //The number of non-buffered RFile64::Size() calls.
204 TInt iFileSetSizeCount; //The number of non-buffered RFile64::SetSize() calls.
205 TInt iFileFlushCount; //The number of RFile64::Flush() calls.