|
| 1 | +# PyScript and Filesystems |
| 2 | + |
| 3 | +As you know, the filesystem is where you store files. For Python to work there |
| 4 | +needs to be a filesystem in which Python packages, modules and data for your |
| 5 | +apps can be found. When you `import` a library, or when you `open` a file, it |
| 6 | +is on the in-browser virtual filesystem that Python looks. |
| 7 | + |
| 8 | +However, things are not as they may seem. |
| 9 | + |
| 10 | +This section clarifies what PyScript means by a filesystem, and the way in |
| 11 | +which PyScript interacts with such a concept. |
| 12 | + |
| 13 | +## Two filesystems |
| 14 | + |
| 15 | +PyScript interacts with two filesystems. |
| 16 | + |
| 17 | +1. The browser, thanks to |
| 18 | + [Emscripten](https://emscripten.org/docs/api_reference/Filesystem-API.html), |
| 19 | + provides a virtual in-memory filesystem. **This has nothing to do with your |
| 20 | + device's local filesystem**, but is contained within the browser based |
| 21 | + sandbox used by PyScript. The [files](../configuration/#files) |
| 22 | + configuration API defines what is found on this filesystem. |
| 23 | +2. PyScript provides an easy to use API for accessing your device's local |
| 24 | + filesystem. It requires permission from the user to mount a folder from the |
| 25 | + local filesystem onto a directory in the browser's virtual filesystem. Think |
| 26 | + of it as gate-keeping a bridge to the outside world of the device's local |
| 27 | + filesystem. |
| 28 | + |
| 29 | +!!! danger |
| 30 | + |
| 31 | + Access to the device's local filesystem **is only available in Chromium |
| 32 | + based browsers**. |
| 33 | + |
| 34 | + Firefox and Safari do not support this capability (yet), and so it is not |
| 35 | + available to PyScript running in these browsers. |
| 36 | + |
| 37 | +## The in-browser filesystem |
| 38 | + |
| 39 | +The filesystem that both Pyodide and MicroPython use by default is the |
| 40 | +[in-browser virtual filesystem](https://emscripten.org/docs/api_reference/Filesystem-API.html). |
| 41 | +Opening files and importing modules takes place in relation to this sandboxed |
| 42 | +environment, configured via the [files](../configuration/#files) entry in your |
| 43 | +settings. |
| 44 | + |
| 45 | +```toml title="Filesystem configuration via TOML." |
| 46 | +[files] |
| 47 | +"https://example.com/myfile.txt": "" |
| 48 | +``` |
| 49 | + |
| 50 | +```python title="Just use the resulting file 'as usual'." |
| 51 | +# Interacting with the virtual filesystem, "as usual". |
| 52 | +with open("myfile.txt", "r") as myfile: |
| 53 | + print(myfile.read()) |
| 54 | +``` |
| 55 | + |
| 56 | +Currently, each time you re-load the page, the filesystem is recreated afresh, |
| 57 | +so any data stored by PyScript to this filesystem will be lost. |
| 58 | + |
| 59 | +!!! info |
| 60 | + |
| 61 | + In the future, we may make it possible to configure the in-browser virtual |
| 62 | + filesystem as persistent across re-loads. |
| 63 | + |
| 64 | +[This article](https://emscripten.org/docs/porting/files/file_systems_overview.html) |
| 65 | +gives an excellent overview of the browser based virtual filesystem's |
| 66 | +implementation and architecture. |
| 67 | + |
| 68 | +The most important key concepts to remember are: |
| 69 | + |
| 70 | +* The PyScript filesystem is contained *within* the browser's sandbox. |
| 71 | +* Each instance of a Python interpreter used by PyScript runs in a separate |
| 72 | + sandbox, and so does NOT share virtual filesystems. |
| 73 | +* All Python related filesytem operations work as expected with this |
| 74 | + filesystem. |
| 75 | +* The virtual filesystem is configured via the |
| 76 | + [files](../configuration/#files) entry in your settings. |
| 77 | +* The virtual filesystem is (currently) NOT persistent between page re-loads. |
| 78 | +* Currently, the filesystem has a maximum capacity of 4GB of data (something |
| 79 | + over which we have no control). |
| 80 | + |
| 81 | +## The device's local filesystem |
| 82 | + |
| 83 | +**Access to the device's local filesystem currently only works on Chromium |
| 84 | +based browsers**. |
| 85 | + |
| 86 | +Your device (the laptop, mobile or tablet) that runs your browser has a |
| 87 | +filesystem provided by a hard drive. Thanks to the |
| 88 | +[`pyscript.fs` namespace in our API](../../api/#pyscriptfs), both MicroPython |
| 89 | +and Pyodide (CPython) gain access to this filesystem should the user of |
| 90 | +your code allow this to happen. |
| 91 | + |
| 92 | +This is a [transient activation](https://developer.mozilla.org/en-US/docs/Glossary/Transient_activation) |
| 93 | +for the purposes of |
| 94 | +[user activation of gated features](https://developer.mozilla.org/en-US/docs/Web/Security/User_activation). |
| 95 | +Put simply, before your code gains access to their local filesystem, an |
| 96 | +explicit agreement needs to be gathered from the user. Part of this process |
| 97 | +involves asking the user to select a target directory on their local |
| 98 | +filesystem, to which PyScript will be given access. |
| 99 | + |
| 100 | +The directory on their local filesystem, selected by the user, is then mounted |
| 101 | +to a given directory inside the browser's virtual filesystem. In this way a |
| 102 | +mapping is made between the sandboxed world of the browser, and the outside |
| 103 | +world of the user's filesystem. |
| 104 | + |
| 105 | +Your code will then be able to perform all the usual filesystem related |
| 106 | +operations provided by Python, within the mounted directory. However, **such |
| 107 | +changes will NOT take effect on the local filesystem UNTIL your code |
| 108 | +explicitly calls the `sync` function**. At this point, the state of the |
| 109 | +in-browser virtual filesystem and the user's local filesystem are synchronised. |
| 110 | + |
| 111 | +The following code demonstrates the simplest use case: |
| 112 | + |
| 113 | +```python title="The core operations of the pyscript.fs API" |
| 114 | +from pyscript import fs |
| 115 | + |
| 116 | +# Ask once for permission to mount any local folder |
| 117 | +# into the virtual filesystem handled by Pyodide/MicroPython. |
| 118 | +# The folder "/local" refers to the directory on the virtual |
| 119 | +# filesystem to which the user-selected directory will be |
| 120 | +# mounted. |
| 121 | +await fs.mount("/local") |
| 122 | + |
| 123 | +# ... DO FILE RELATED OPERATIONS HERE ... |
| 124 | + |
| 125 | +# If changes were made, ensure these are persisted to the local filesystem's |
| 126 | +# folder. |
| 127 | +await fs.sync("/local") |
| 128 | + |
| 129 | +# If needed to free RAM or that specific path, sync and unmount |
| 130 | +await fs.unmount("/local") |
| 131 | +``` |
| 132 | + |
| 133 | +It is possible to use multiple different local directories with the same mount |
| 134 | +point. This is important if your application provides some generic |
| 135 | +functionality on data that might be in different local directories because |
| 136 | +while the nature of the data might be similar, the subject is not. For |
| 137 | +instance, you may have different models for a PyScript based LLM in different |
| 138 | +directories, and may wish to switch between them at runtime using different |
| 139 | +handlers (requiring their own transient action). In which case use |
| 140 | +the following technique: |
| 141 | + |
| 142 | +```python title="Multiple local directories on the same mount point" |
| 143 | +# Mount a local folder specifying a different handler. |
| 144 | +# This requires a user explicit transient action (once). |
| 145 | +await fs.mount("/local", id="v1") |
| 146 | +# ... operate on that folder ... |
| 147 | +await fs.unmount("/local") |
| 148 | + |
| 149 | +# Mount a local folder specifying a different handler. |
| 150 | +# This also requires a user explicit transient action (once). |
| 151 | +await fs.mount("/local", id="v2") |
| 152 | +# ... operate on that folder ... |
| 153 | +await fs.unmount("/local") |
| 154 | + |
| 155 | +# Go back to the original handler or a previous one. |
| 156 | +# No transient action required now. |
| 157 | +await fs.mount("/local", id="v1") |
| 158 | +# ... operate again on that folder ... |
| 159 | +``` |
| 160 | + |
| 161 | +In addition to the mount `path` and handler `id`, the `fs.mount` function can |
| 162 | +take two further arguments: |
| 163 | + |
| 164 | +* `mode` (by default `"readwrite"`) indicates the sort of activity available to |
| 165 | + the user. It can also be set to `read` for read-only access to the local |
| 166 | + filesystem. This is a part of the |
| 167 | + [web-standards](https://developer.mozilla.org/en-US/docs/Web/API/Window/showDirectoryPicker#mode) |
| 168 | + for directory selection. |
| 169 | +* `root` - (by default, `""`) is a hint to the browser for where to start |
| 170 | + picking the path that should be mounted in Python. Valid values are: |
| 171 | + `desktop`, `documents`, `downloads`, `music`, `pictures` or `videos` |
| 172 | + [as per web standards](https://developer.mozilla.org/en-US/docs/Web/API/Window/showDirectoryPicker#startin). |
| 173 | + |
| 174 | +The `sync` and `unmount` functions only accept the mount `path` used in the |
| 175 | +browser's local filesystem. |
0 commit comments