WARNING: WASM support is work-in-progress! Lots of features are not working yet.
This directory contains configuration and helpers to facilitate cross compilation of CPython to WebAssembly (WASM). Python supports Emscripten (wasm32-emscripten) and WASI (wasm32-wasi) targets. Emscripten builds run in modern browsers and JavaScript runtimes like Node.js. WASI builds use WASM runtimes such as wasmtime.
Users and developers are encouraged to use the script Tools/wasm/wasm_build.py. The tool automates the build process and provides assistance with installation of SDKs, running tests, etc.
NOTE: If you are looking for information that is not directly related to building CPython for WebAssembly (or the resulting build), please see https://github.com/psf/webassembly for more information.
For now the build system has two target flavors. The Emscripten/browser target (--with-emscripten-target=browser) is optimized for browsers. It comes with a reduced and preloaded stdlib without tests and threading support. The Emscripten/node target has threading enabled and can access the file system directly.
Cross compiling to the wasm32-emscripten platform needs the Emscripten SDK and a build Python interpreter. Emscripten 3.1.19 or newer are recommended. All commands below are relative to a repository checkout.
Christian Heimes maintains a container image with Emscripten SDK, Python build dependencies, WASI-SDK, wasmtime, and several additional tools.
From within your local CPython repo clone, run one of the following commands:
# Fedora, RHEL, CentOS podman run --rm -ti -v $(pwd):/python-wasm/cpython:Z -w /python-wasm/cpython quay.io/tiran/cpythonbuild:emsdk3 # other docker run --rm -ti -v $(pwd):/python-wasm/cpython -w /python-wasm/cpython quay.io/tiran/cpythonbuild:emsdk3
NOTE: Follow the on-screen instructions how to add the SDK to PATH.
git clone https://github.com/emscripten-core/emsdk.git /opt/emsdk /opt/emsdk/emsdk install latest /opt/emsdk/emsdk activate latest
The EM_COMPILER_WRAPPER must be set after the EMSDK environment is sourced. Otherwise the source script removes the environment variable.
. /opt/emsdk/emsdk_env.sh EM_COMPILER_WRAPPER=ccache
Emscripten SDK provides static builds of core libraries without PIC (position-independent code). Python builds with dlopen support require PIC. To populate the build cache, run:
. /opt/emsdk/emsdk_env.sh embuilder build zlib bzip2 MINIMAL_PIC embuilder --pic build zlib bzip2 MINIMAL_PIC
From within the container, run the following command:
./Tools/wasm/wasm_build.py build
The command is roughly equivalent to:
mkdir -p builddir/build pushd builddir/build ../../configure -C make -j$(nproc) popd
./Tools/wasm/wasm_build.py emscripten-browser
The command is roughly equivalent to:
mkdir -p builddir/emscripten-browser pushd builddir/emscripten-browser CONFIG_SITE=../../Tools/wasm/config.site-wasm32-emscripten \ emconfigure ../../configure -C \ --host=wasm32-unknown-emscripten \ --build=$(../../config.guess) \ --with-emscripten-target=browser \ --with-build-python=$(pwd)/../build/python emmake make -j$(nproc) popd
Serve python.html with a local webserver and open the file in a browser. Python comes with a minimal web server script that sets necessary HTTP headers like COOP, COEP, and mimetypes. Run the script outside the container and from the root of the CPython checkout.
./Tools/wasm/wasm_webserver.py
and open http://localhost:8000/builddir/emscripten-browser/python.html . This directory structure enables the C/C++ DevTools Support (DWARF) to load C and header files with debug builds.
./Tools/wasm/wasm_build.py emscripten-node-dl
The command is roughly equivalent to:
mkdir -p builddir/emscripten-node-dl pushd builddir/emscripten-node-dl CONFIG_SITE=../../Tools/wasm/config.site-wasm32-emscripten \ emconfigure ../../configure -C \ --host=wasm32-unknown-emscripten \ --build=$(../../config.guess) \ --with-emscripten-target=node \ --enable-wasm-dynamic-linking \ --with-build-python=$(pwd)/../build/python emmake make -j$(nproc) popd
node --experimental-wasm-threads --experimental-wasm-bulk-memory --experimental-wasm-bigint builddir/emscripten-node-dl/python.js
(--experimental-wasm-bigint is not needed with recent NodeJS versions)
Emscripten before 3.1.8 has known bugs that can cause memory corruption and resource leaks. 3.1.8 contains several fixes for bugs in date and time functions.
asyncio, urllib, selectors, etc. are not available.AF_INET and AF_INET6 with SOCK_STREAM (TCP) or SOCK_DGRAM (UDP) are available. AF_UNIX is not supported.socketpair does not work.socket.accept crashes the runtime. gethostbyname does not resolve to a real IP address. IPv6 is not available.select module is limited. select.select() crashes the runtime due to lack of exectfd support.ENOSYS or ENOSUP.signal.alarm, itimer, sigaction are not available or do not work correctly. SIGTERM exits the runtime.os.nice and most functions of the resource module are not available.configure option --enable-wasm-pthreads adds compiler flag -pthread and linker flags -sUSE_PTHREADS -sPROXY_TO_PTHREAD.pwd module, grp module, os.setgroups, os.chown, and so on. lchown and lchmod are not available.umask is a no-op.os.link) are not supported.os.pread, os.preadv) are not available.os.mknod and os.mkfifo don't work and are disabled.mmap module is unstable. flush (msync) can crash the runtime.ctypes, readline, ssl, and more.--enable-wasm-dynamic-linking enables dynamic extensions supports. It's currently known to crash in combination with threading.locales module is affected by musl libc issues, gh-90548.obmalloc is disabled by default.ensurepip is not available.ctypes features like c_longlong and c_longdouble may need NodeJS option --experimental-wasm-bigint.pyc files.--enable-test-modules build test modules like _testcapi.Node builds use NODERAWFS.
FS.mount() call.--experimental-wasm-memory64.EM_JS functions must return BigInt().Py_BuildValue() format strings must match size of types. Confusing 32 and 64 bits types leads to memory corruption, see gh-95876 and gh-95878.The simple REPL terminal uses SharedArrayBuffer. For security reasons browsers only provide the feature in secure environents with cross-origin isolation. The webserver must send cross-origin headers and correct MIME types for the JavaScript and WASM files. Otherwise the terminal will fail to load with an error message like Browsers disable shared array buffer.
Place a .htaccess file in the same directory as python.wasm.
# .htaccess
Header set Cross-Origin-Opener-Policy same-origin
Header set Cross-Origin-Embedder-Policy require-corp
AddType application/javascript js
AddType application/wasm wasm
<IfModule mod_deflate.c>
AddOutputFilterByType DEFLATE text/html application/javascript application/wasm
</IfModule>
NOTE: The instructions below assume a Unix-based OS due to cross-compilation for CPython being set up for ./configure.
Developing for WASI requires two additional tools to be installed beyond the typical tools required to build CPython:
All of this is provided in the devcontainer if you don't want to install these tools locally.
Building for WASI requires doing a cross-build where you have a “build” Python to help produce a WASI build of CPython (technically it‘s a “host x host” cross-build because the build Python is also the target Python while the host build is the WASI build; yes, it’s confusing terminology). In the end you should have a build Python in cross-build/build and a WASI build in cross-build/wasm32-wasi.
The easiest way to do a build is to use the wasi.py script. You can either have it perform the entire build process from start to finish in one step, or you can do it in discrete steps that mirror running configure and make for each of the two builds of Python you end up producing (which are beneficial when you only need to do a specific step after getting a complete build, e.g. editing some code and you just need to run make for the WASI build). The script is designed to self-document what actions it is performing on your behalf, both as a way to check its work but also for educaitonal purposes.
The discrete steps for building via wasi.py are:
python Tools/wasm/wasi.py configure-build-python python Tools/wasm/wasi.py make-build-python python Tools/wasm/wasi.py configure-host python Tools/wasm/wasi.py make-host
To do it all in a single command, run:
python Tools/wasm/wasi.py build
That will:
configure for the build Python (same as wasi.py configure-build-python)make for the build Python (wasi.py make-build-python)configure for the WASI build (wasi.py configure-host)make for the WASI build (wasi.py make-host)See the --help for the various options available for each of the subcommands which controls things like the location of the WASI SDK, the command to use with the WASI host/runtime, etc. Also note that you can use -- as a separator for any of the configure-related commands -- including build itself -- to pass arguments to the underlying configure call. For example, if you want a pydebug build that also caches the results from configure, you can do:
python Tools/wasm/wasi.py build -- -C --with-pydebug
The wasi.py script is able to infer details from the build Python, and so you only technically need to specify --with-pydebug once via configure-build-python as this will lead to configure-host detecting its use if you use the discrete steps:
python Tools/wasm/wasi.py configure-build-python -- -C --with-pydebug python Tools/wasm/wasi.py make-build-python python Tools/wasm/wasi.py configure-host -- -C python Tools/wasm/wasi.py make-host
If you used wasi.py to do your build then there will be a cross-build/wasm32-wasi/python.sh file which you can use to run the python.wasm file (see the output from the configure-host subcommand):
cross-build/wasm32-wasi/python.sh --version
While you can run python.wasm directly, Python will fail to start up without certain things being set (e.g. PYTHONPATH for sysconfig data). As such, the python.sh file records these details for you.
import os, sys if sys.platform == "emscripten": # Python on Emscripten ... if sys.platform == "wasi": # Python on WASI ... if os.name == "posix": # WASM platforms identify as POSIX-like. # Windows does not provide os.uname(). machine = os.uname().machine if machine.startswith("wasm"): # WebAssembly (wasm32, wasm64 potentially in the future)
>>> import os, sys >>> os.uname() posix.uname_result( sysname='Emscripten', nodename='emscripten', release='3.1.19', version='#1', machine='wasm32' ) >>> os.name 'posix' >>> sys.platform 'emscripten' >>> sys._emscripten_info sys._emscripten_info( emscripten_version=(3, 1, 10), runtime='Mozilla/5.0 (X11; Linux x86_64; rv:104.0) Gecko/20100101 Firefox/104.0', pthreads=False, shared_memory=False )
>>> sys._emscripten_info sys._emscripten_info( emscripten_version=(3, 1, 19), runtime='Node.js v14.18.2', pthreads=True, shared_memory=True )
>>> import os, sys >>> os.uname() posix.uname_result( sysname='wasi', nodename='(none)', release='0.0.0', version='0.0.0', machine='wasm32' ) >>> os.name 'posix' >>> sys.platform 'wasi'
Emscripten SDK and WASI SDK define several built-in macros. You can dump a full list of built-ins with emcc -dM -E - < /dev/null and /path/to/wasi-sdk/bin/clang -dM -E - < /dev/null.
__wasm__ (also __wasm)__wasm32__ (also __wasm32)__wasm64____EMSCRIPTEN__ (also EMSCRIPTEN)__EMSCRIPTEN_major__, __EMSCRIPTEN_minor__, __EMSCRIPTEN_tiny____wasi__