| .. _devguide-coding-fileio: |
| |
| ######## |
| File I/O |
| ######## |
| |
| .. contents:: |
| :local: |
| :backlinks: none |
| :depth: 2 |
| |
| Introduction |
| ============ |
| |
| This chapter describes how to use the `FileIO API |
| </native-client/pepper_stable/cpp/classpp_1_1_file_i_o>`_ to read and write |
| files using a local secure data store. |
| |
| You might use the File IO API with the URL Loading APIs to create an overall |
| data download and caching solution for your NaCl applications. For example: |
| |
| #. Use the File IO APIs to check the local disk to see if a file exists that |
| your program needs. |
| #. If the file exists locally, load it into memory using the File IO API. If |
| the file doesn't exist locally, use the URL Loading API to retrieve the |
| file from the server. |
| #. Use the File IO API to write the file to disk. |
| #. Load the file into memory using the File IO API when needed by your |
| application. |
| |
| The example discussed in this chapter is included in the SDK in the directory |
| ``examples/api/file_io``. |
| |
| Reference information |
| ===================== |
| |
| For reference information related to FileIO, see the following documentation: |
| |
| * `file_io.h </native-client/pepper_stable/cpp/file__io_8h>`_ - API to create a |
| FileIO object |
| * `file_ref.h </native-client/pepper_stable/cpp/file__ref_8h>`_ - API to create |
| a file reference or "weak pointer" to a file in a file system |
| * `file_system.h </native-client/pepper_stable/cpp/file__system_8h>`_ - API to |
| create a file system associated with a file |
| |
| Local file I/O |
| ============== |
| |
| Chrome provides an obfuscated, restricted area on disk to which a web app can |
| safely `read and write files |
| <https://developers.google.com/chrome/whitepapers/storage#persistent>`_. The |
| Pepper FileIO, FileRef, and FileSystem APIs (collectively called the File IO |
| APIs) allow you to access this sandboxed local disk so you can read and write |
| files and manage caching yourself. The data is persistent between launches of |
| Chrome, and is not removed unless your application deletes it or the user |
| manually deletes it. There is no limit to the amount of local data you can |
| use, other than the actual space available on the local drive. |
| |
| .. _quota_management: |
| .. _enabling_file_access: |
| |
| Enabling local file I/O |
| ----------------------- |
| |
| The easiest way to enable the writing of persistent local data is to include |
| the `unlimitedStorage permission |
| </extensions/declare_permissions#unlimitedStorage>`_ in your Chrome Web Store |
| manifest file. With this permission you can use the Pepper FileIO API without |
| the need to request disk space at run time. When the user installs the app |
| Chrome displays a message announcing that the app writes to the local disk. |
| |
| If you do not use the ``unlimitedStorage`` permission you must include |
| JavaScript code that calls the `HTML5 Quota Management API |
| <http://updates.html5rocks.com/2011/11/Quota-Management-API-Fast-Facts>`_ to |
| explicitly request local disk space before using the FileIO API. In this case |
| Chrome will prompt the user to accept a requestQuota call every time one is |
| made. |
| |
| Testing local file I/O |
| ---------------------- |
| |
| You should be aware that using the ``unlimitedStorage`` manifest permission |
| constrains the way you can test your app. Three of the four techniques |
| described in :doc:`Running Native Client Applications <../devcycle/running>` |
| read the Chrome Web Store manifest file and enable the ``unlimitedStorage`` |
| permission when it appears, but the first technique (local server) does not. |
| If you want to test the file IO portion of your app with a simple local server, |
| you need to include JavaScript code that calls the HTML5 Quota Management API. |
| When you deliver your application you can replace this code with the |
| ``unlimitedStorage`` manifest permission. |
| |
| The ``file_io`` example |
| ======================= |
| |
| The Native Client SDK includes an example, ``file_io``, that demonstrates how |
| to read and write a local disk file. Since you will probably run the example |
| from a local server without a Chrome Web Store manifest file, the example's |
| index file uses JavaScript to perform the Quota Management setup as described |
| above. The example has these primary files: |
| |
| * ``index.html`` - The HTML code that launches the Native Client module and |
| displays the user interface. |
| * ``example.js`` - JavaScript code that requests quota (as described above). It |
| also listens for user interaction with the user interface, and forwards the |
| requests to the Native Client module. |
| * ``file_io.cc`` - The code that sets up and provides an entry point to the |
| Native Client module. |
| |
| The remainder of this section covers the code in the ``file_io.cc`` file for |
| reading and writing files. |
| |
| File I/O overview |
| ----------------- |
| |
| Like many Pepper APIs, the File IO API includes a set of methods that execute |
| asynchronously and that invoke callback functions in your Native Client module. |
| Unlike most other examples, the ``file_io`` example also demonstrates how to |
| make Pepper calls synchronously on a worker thread. |
| |
| It is illegal to make blocking calls to Pepper on the module's main thread. |
| This restriction is lifted when running on a worker thread---this is called |
| "calling Pepper off the main thread". This often simplifies the logic of your |
| code; multiple asynchronous Pepper functions can be called from one function on |
| your worker thread, so you can use the stack and standard control flow |
| structures normally. |
| |
| The high-level flow for the ``file_io`` example is described below. Note that |
| methods in the namespace ``pp`` are part of the Pepper C++ API. |
| |
| Creating and writing a file |
| --------------------------- |
| |
| Following are the high-level steps involved in creating and writing to a |
| file: |
| |
| #. ``pp::FileIO::Open`` is called with the ``PP_FILEOPEN_FLAG_CREATE`` flag to |
| create a file. Because the callback function is ``pp::BlockUntilComplete``, |
| this thread is blocked until ``Open`` succeeds or fails. |
| #. ``pp::FileIO::Write`` is called to write the contents. Again, the thread is |
| blocked until the call to ``Write`` completes. If there is more data to |
| write, ``Write`` is called again. |
| #. When there is no more data to write, call ``pp::FileIO::Flush``. |
| |
| Opening and reading a file |
| -------------------------- |
| |
| Following are the high-level steps involved in opening and reading a file: |
| |
| #. ``pp::FileIO::Open`` is called to open the file. Because the callback |
| function is ``pp::BlockUntilComplete``, this thread is blocked until Open |
| succeeds or fails. |
| #. ``pp::FileIO::Query`` is called to query information about the file, such as |
| its file size. The thread is blocked until ``Query`` completes. |
| #. ``pp::FileIO::Read`` is called to read the contents. The thread is blocked |
| until ``Read`` completes. If there is more data to read, ``Read`` is called |
| again. |
| |
| Deleting a file |
| --------------- |
| |
| Deleting a file is straightforward: call ``pp::FileRef::Delete``. The thread is |
| blocked until ``Delete`` completes. |
| |
| Making a directory |
| ------------------ |
| |
| Making a directory is also straightforward: call ``pp::File::MakeDirectory``. |
| The thread is blocked until ``MakeDirectory`` completes. |
| |
| Listing the contents of a directory |
| ----------------------------------- |
| |
| Following are the high-level steps involved in listing a directory: |
| |
| #. ``pp::FileRef::ReadDirectoryEntries`` is called, and given a directory entry |
| to list. A callback is given as well; many of the other functions use |
| ``pp::BlockUntilComplete``, but ``ReadDirectoryEntries`` returns results in |
| its callback, so it must be specified. |
| #. When the call to ``ReadDirectoryEntries`` completes, it calls |
| ``ListCallback`` which packages up the results into a string message, and |
| sends it to JavaScript. |
| |
| ``file_io`` deep dive |
| ===================== |
| |
| The ``file_io`` example displays a user interface with a couple of fields and |
| several buttons. Following is a screenshot of the ``file_io`` example: |
| |
| .. image:: /images/fileioexample.png |
| |
| Each radio button is a file operation you can perform, with some reasonable |
| default values for filenames. Try typing a message in the large input box and |
| clicking ``Save``, then switching to the ``Load File`` operation, and |
| clicking ``Load``. |
| |
| Let's take a look at what is going on under the hood. |
| |
| Opening a file system and preparing for file I/O |
| ------------------------------------------------ |
| |
| ``pp::Instance::Init`` is called when an instance of a module is created. In |
| this example, ``Init`` starts a new thread (via the ``pp::SimpleThread`` |
| class), and tells it to open the filesystem: |
| |
| .. naclcode:: |
| |
| virtual bool Init(uint32_t /*argc*/, |
| const char * /*argn*/ [], |
| const char * /*argv*/ []) { |
| file_thread_.Start(); |
| // Open the file system on the file_thread_. Since this is the first |
| // operation we perform there, and because we do everything on the |
| // file_thread_ synchronously, this ensures that the FileSystem is open |
| // before any FileIO operations execute. |
| file_thread_.message_loop().PostWork( |
| callback_factory_.NewCallback(&FileIoInstance::OpenFileSystem)); |
| return true; |
| } |
| |
| When the file thread starts running, it will call ``OpenFileSystem``. This |
| calls ``pp::FileSystem::Open`` and blocks the file thread until the function |
| returns. |
| |
| .. Note:: |
| :class: note |
| |
| Note that the call to ``pp::FileSystem::Open`` uses |
| ``pp::BlockUntilComplete`` as its callback. This is only possible because we |
| are running off the main thread; if you try to make a blocking call from the |
| main thread, the function will return the error |
| ``PP_ERROR_BLOCKS_MAIN_THREAD``. |
| |
| .. naclcode:: |
| |
| void OpenFileSystem(int32_t /*result*/) { |
| int32_t rv = file_system_.Open(1024 * 1024, pp::BlockUntilComplete()); |
| if (rv == PP_OK) { |
| file_system_ready_ = true; |
| // Notify the user interface that we're ready |
| PostMessage("READY|"); |
| } else { |
| ShowErrorMessage("Failed to open file system", rv); |
| } |
| } |
| |
| Handling messages from JavaScript |
| --------------------------------- |
| |
| When you click the ``Save`` button, JavaScript posts a message to the NaCl |
| module with the file operation to perform sent as a string (See :doc:`Messaging |
| System <message-system>` for more details on message passing). The string is |
| parsed by ``HandleMessage``, and new work is added to the file thread: |
| |
| .. naclcode:: |
| |
| virtual void HandleMessage(const pp::Var& var_message) { |
| if (!var_message.is_string()) |
| return; |
| |
| // Parse message into: instruction file_name_length file_name [file_text] |
| std::string message = var_message.AsString(); |
| std::string instruction; |
| std::string file_name; |
| std::stringstream reader(message); |
| int file_name_length; |
| |
| reader >> instruction >> file_name_length; |
| file_name.resize(file_name_length); |
| reader.ignore(1); // Eat the delimiter |
| reader.read(&file_name[0], file_name_length); |
| |
| ... |
| |
| // Dispatch the instruction |
| if (instruction == kLoadPrefix) { |
| file_thread_.message_loop().PostWork( |
| callback_factory_.NewCallback(&FileIoInstance::Load, file_name)); |
| } else if (instruction == kSavePrefix) { |
| ... |
| } |
| } |
| |
| Saving a file |
| ------------- |
| |
| ``FileIoInstance::Save`` is called when the ``Save`` button is pressed. First, |
| it checks to see that the FileSystem has been successfully opened: |
| |
| .. naclcode:: |
| |
| if (!file_system_ready_) { |
| ShowErrorMessage("File system is not open", PP_ERROR_FAILED); |
| return; |
| } |
| |
| It then creates a ``pp::FileRef`` resource with the name of the file. A |
| ``FileRef`` resource is a weak reference to a file in the FileSystem; that is, |
| a file can still be deleted even if there are outstanding ``FileRef`` |
| resources. |
| |
| .. naclcode:: |
| |
| pp::FileRef ref(file_system_, file_name.c_str()); |
| |
| Next, a ``pp::FileIO`` resource is created and opened. The call to |
| ``pp::FileIO::Open`` passes ``PP_FILEOPEFLAG_WRITE`` to open the file for |
| writing, ``PP_FILEOPENFLAG_CREATE`` to create a new file if it doesn't already |
| exist and ``PP_FILEOPENFLAG_TRUNCATE`` to clear the file of any previous |
| content: |
| |
| .. naclcode:: |
| |
| pp::FileIO file(this); |
| |
| int32_t open_result = |
| file.Open(ref, |
| PP_FILEOPENFLAG_WRITE | PP_FILEOPENFLAG_CREATE | |
| PP_FILEOPENFLAG_TRUNCATE, |
| pp::BlockUntilComplete()); |
| if (open_result != PP_OK) { |
| ShowErrorMessage("File open for write failed", open_result); |
| return; |
| } |
| |
| Now that the file is opened, it is written to in chunks. In an asynchronous |
| model, this would require writing a separate function, storing the current |
| state on the free store and a chain of callbacks. Because this function is |
| called off the main thread, ``pp::FileIO::Write`` can be called synchronously |
| and a conventional do/while loop can be used: |
| |
| .. naclcode:: |
| |
| int64_t offset = 0; |
| int32_t bytes_written = 0; |
| do { |
| bytes_written = file.Write(offset, |
| file_contents.data() + offset, |
| file_contents.length(), |
| pp::BlockUntilComplete()); |
| if (bytes_written > 0) { |
| offset += bytes_written; |
| } else { |
| ShowErrorMessage("File write failed", bytes_written); |
| return; |
| } |
| } while (bytes_written < static_cast<int64_t>(file_contents.length())); |
| |
| Finally, the file is flushed to push all changes to disk: |
| |
| .. naclcode:: |
| |
| int32_t flush_result = file.Flush(pp::BlockUntilComplete()); |
| if (flush_result != PP_OK) { |
| ShowErrorMessage("File fail to flush", flush_result); |
| return; |
| } |
| |
| Loading a file |
| -------------- |
| |
| ``FileIoInstance::Load`` is called when the ``Load`` button is pressed. Like |
| the ``Save`` function, ``Load`` first checks to see if the FileSystem has been |
| successfully opened, and creates a new ``FileRef``: |
| |
| .. naclcode:: |
| |
| if (!file_system_ready_) { |
| ShowErrorMessage("File system is not open", PP_ERROR_FAILED); |
| return; |
| } |
| pp::FileRef ref(file_system_, file_name.c_str()); |
| |
| Next, ``Load`` creates and opens a new ``FileIO`` resource, passing |
| ``PP_FILEOPENFLAG_READ`` to open the file for reading. The result is compared |
| to ``PP_ERROR_FILENOTFOUND`` to give a better error message when the file |
| doesn't exist: |
| |
| .. naclcode:: |
| |
| int32_t open_result = |
| file.Open(ref, PP_FILEOPENFLAG_READ, pp::BlockUntilComplete()); |
| if (open_result == PP_ERROR_FILENOTFOUND) { |
| ShowErrorMessage("File not found", open_result); |
| return; |
| } else if (open_result != PP_OK) { |
| ShowErrorMessage("File open for read failed", open_result); |
| return; |
| } |
| |
| Then ``Load`` calls ``pp::FileIO::Query`` to get metadata about the file, such |
| as its size. This is used to allocate a ``std::vector`` buffer that holds the |
| data from the file in memory: |
| |
| .. naclcode:: |
| |
| int32_t query_result = file.Query(&info, pp::BlockUntilComplete()); |
| if (query_result != PP_OK) { |
| ShowErrorMessage("File query failed", query_result); |
| return; |
| } |
| |
| ... |
| |
| std::vector<char> data(info.size); |
| |
| Similar to ``Save``, a conventional while loop is used to read the file into |
| the newly allocated buffer: |
| |
| .. naclcode:: |
| |
| int64_t offset = 0; |
| int32_t bytes_read = 0; |
| int32_t bytes_to_read = info.size; |
| while (bytes_to_read > 0) { |
| bytes_read = file.Read(offset, |
| &data[offset], |
| data.size() - offset, |
| pp::BlockUntilComplete()); |
| if (bytes_read > 0) { |
| offset += bytes_read; |
| bytes_to_read -= bytes_read; |
| } else if (bytes_read < 0) { |
| // If bytes_read < PP_OK then it indicates the error code. |
| ShowErrorMessage("File read failed", bytes_read); |
| return; |
| } |
| } |
| |
| Finally, the contents of the file are sent back to JavaScript, to be displayed |
| on the page. This example uses "``DISP|``" as a prefix command for display |
| information: |
| |
| .. naclcode:: |
| |
| std::string string_data(data.begin(), data.end()); |
| PostMessage("DISP|" + string_data); |
| ShowStatusMessage("Load success"); |
| |
| Deleting a file |
| --------------- |
| |
| ``FileIoInstance::Delete`` is called when the ``Delete`` button is pressed. |
| First, it checks whether the FileSystem has been opened, and creates a new |
| ``FileRef``: |
| |
| .. naclcode:: |
| |
| if (!file_system_ready_) { |
| ShowErrorMessage("File system is not open", PP_ERROR_FAILED); |
| return; |
| } |
| pp::FileRef ref(file_system_, file_name.c_str()); |
| |
| Unlike ``Save`` and ``Load``, ``Delete`` is called on the ``FileRef`` resource, |
| not a ``FileIO`` resource. Note that the result is checked for |
| ``PP_ERROR_FILENOTFOUND`` to give a better error message when trying to delete |
| a non-existent file: |
| |
| .. naclcode:: |
| |
| int32_t result = ref.Delete(pp::BlockUntilComplete()); |
| if (result == PP_ERROR_FILENOTFOUND) { |
| ShowStatusMessage("File/Directory not found"); |
| return; |
| } else if (result != PP_OK) { |
| ShowErrorMessage("Deletion failed", result); |
| return; |
| } |
| |
| Listing files in a directory |
| ---------------------------- |
| |
| ``FileIoInstance::List`` is called when the ``List Directory`` button is |
| pressed. Like all other operations, it checks whether the FileSystem has been |
| opened and creates a new ``FileRef``: |
| |
| .. naclcode:: |
| |
| if (!file_system_ready_) { |
| ShowErrorMessage("File system is not open", PP_ERROR_FAILED); |
| return; |
| } |
| |
| pp::FileRef ref(file_system_, dir_name.c_str()); |
| |
| Unlike the other operations, it does not make a blocking call to |
| ``pp::FileRef::ReadDirectoryEntries``. Since ``ReadDirectoryEntries`` returns |
| the resulting directory entries in its callback, a new callback object is |
| created pointing to ``FileIoInstance::ListCallback``. |
| |
| The ``pp::CompletionCallbackFactory`` template class is used to instantiate a |
| new callback. Notice that the ``FileRef`` resource is passed as a parameter; |
| this will add a reference count to the callback object, to keep the ``FileRef`` |
| resource from being destroyed when the function finishes. |
| |
| .. naclcode:: |
| |
| // Pass ref along to keep it alive. |
| ref.ReadDirectoryEntries(callback_factory_.NewCallbackWithOutput( |
| &FileIoInstance::ListCallback, ref)); |
| |
| ``FileIoInstance::ListCallback`` then gets the results passed as a |
| ``std::vector`` of ``pp::DirectoryEntry`` objects, and sends them to |
| JavaScript: |
| |
| .. naclcode:: |
| |
| void ListCallback(int32_t result, |
| const std::vector<pp::DirectoryEntry>& entries, |
| pp::FileRef /*unused_ref*/) { |
| if (result != PP_OK) { |
| ShowErrorMessage("List failed", result); |
| return; |
| } |
| |
| std::stringstream ss; |
| ss << "LIST"; |
| for (size_t i = 0; i < entries.size(); ++i) { |
| pp::Var name = entries[i].file_ref().GetName(); |
| if (name.is_string()) { |
| ss << "|" << name.AsString(); |
| } |
| } |
| PostMessage(ss.str()); |
| ShowStatusMessage("List success"); |
| } |
| |
| Making a new directory |
| ---------------------- |
| |
| ``FileIoInstance::MakeDir`` is called when the ``Make Directory`` button is |
| pressed. Like all other operations, it checks whether the FileSystem has been |
| opened and creates a new ``FileRef``: |
| |
| .. naclcode:: |
| |
| if (!file_system_ready_) { |
| ShowErrorMessage("File system is not open", PP_ERROR_FAILED); |
| return; |
| } |
| pp::FileRef ref(file_system_, dir_name.c_str()); |
| |
| Then the ``pp::FileRef::MakeDirectory`` function is called. |
| |
| .. naclcode:: |
| |
| int32_t result = ref.MakeDirectory( |
| PP_MAKEDIRECTORYFLAG_NONE, pp::BlockUntilComplete()); |
| if (result != PP_OK) { |
| ShowErrorMessage("Make directory failed", result); |
| return; |
| } |
| ShowStatusMessage("Make directory success"); |