I just tested this in a brand new virtual environment using the macOS Python 3: bash pipenv shell --python /Applications/Xcode.app/Contents/Developer/usr/bin/python3 Then in that virtual environment I ran: ```bash pip install sqlite-utils

Confirm the right one is on the path:

which sqlite-utils

curl "https://data.nasa.gov/resource/y77d-th95.json" | \ sqlite-utils insert meteorites.db meteorites - --pk=id sqlite-utils extract meteorites.db meteorites recclass This threw the same error reported above. Then I did this:bash rm meteorites.db pip install sqlean.py

curl "https://data.nasa.gov/resource/y77d-th95.json" | \ sqlite-utils insert meteorites.db meteorites - --pk=id sqlite-utils extract meteorites.db meteorites recclass ``` And that second time it worked correctly.

https://sqlite-utils.datasette.io/en/stable/changelog.html#v3-33 - upgrading to sqlite-utils>=3.33 and then installing both sqlean.py and sqlite-dump in the same virtual environment as sqlite-utils should fix this issue.

Ideally a workaround for this right now would be to install pysqlite3 in the same virtual environment:

sqlite-utils install pysqlite3-binary

But pysqlite3-binary doesn't yet ship a wheel for macOS so this probably won't work for most people.

The "easiest" fix at the moment is to use Python from Homebrew - so brew install sqlite-utils for example won't suffer from this problem. Not a great solution for people who aren't using Homebrew though!

Also checked the official Datasette Docker image - I had to run that in Codespaces because it doesn't currently work on my M2 Mac:

``` codespace@codespaces-112c61:/workspaces/sqlite-utils$ docker pull datasetteproject/datasette Using default tag: latest ... codespace@codespaces-112c61:/workspaces/sqlite-utils$ docker run -it datasetteproject/datasette / bin/bash root@75ba34f501ec:/# python Python 3.11.0 (main, Dec 6 2022, 13:31:55) [GCC 10.2.1 20210110] on linux Type "help", "copyright", "credits" or "license" for more information.

import sqlite3 .executescript(""" PRAGMA writable_schema = 1; UPDATE sqlite_master SET sql = 'CREATE TABLE [foos] (id integer primary key)'; PRAGMA writable_schema = 0; """)>>> db = sqlite3.connect(":memory:") db.executescript(""" ... PRAGMA writable_schema = 1; ... UPDATE sqlite_master SET sql = 'CREATE TABLE [foos] (id integer primary key)'; ... PRAGMA writable_schema = 0; ... """) <sqlite3.Cursor object at 0x7fd9b0561140>

``` So that confirms that the official image also has a Python with a SQLite that's not in defensive mode.

I also tested this against the current ubuntu:latest Docker image (on an M2 Mac), in Python 3.10 and 3.11: docker run -it ubuntu:latest /bin/bash Then in the container: ``` apt-get update apt-get install python3 python3

pasted in the above recipe

apt install software-properties-common add-apt-repository ppa:deadsnakes/ppa apt install python3.11 python3.11

pasted it in again

``` In both cases the Python code did not raise an exception, which suggests that on Ubuntu those two Python versions do not have the defensive mode set.

This seems to work: python import sqlite3 db = sqlite3.connect(":memory:") db.executescript(""" PRAGMA writable_schema = 1; UPDATE sqlite_master SET sql = 'CREATE TABLE [foos] (id integer primary key)'; PRAGMA writable_schema = 0; """) It succeeds on my Python 3.11 and raises the following exception on my broken Python 3.9: sqlite3.OperationalError: table sqlite_master may not be modified Removing the PRAGMA writable_schema = 1; causes the same exception to be raised on both Pythons.

I'll ask on the SQLite forum if it's possible to toggle that mode on and off using regular SQL. My hunch is that it isn't.

In which case sqlite-utils should at least know how to catch this error and display a much more readable error message, maybe with a link to further documentation.

A utility function that can detect this mode would be really useful too. I'd probably have to do a test that tries to modify sqlite_master on a new in-memory database to catch if it's possible or not.

Investigating this one now.

The sqlite-utils test suite passes without errors on my Python 3.11.2 installation... but it fails with this error on a Python 3.9.6 installation.

In the broken version's virtual environment directory I ran this: cat pyvenv.cfg home = /Applications/Xcode.app/Contents/Developer/usr/bin implementation = CPython version_info = 3.9.6.final.0 virtualenv = 20.17.1 include-system-site-packages = false base-prefix = /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9 base-exec-prefix = /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9 base-executable = /Applications/Xcode.app/Contents/Developer/usr/bin/python3 So it looks like the Xcode python3 has "defensive" mode turned on for SQLite.

As far as I can tell there's no way to turn it OFF again in Python.

My virtual environment that DOES work has this:

home = /opt/homebrew/opt/python@3.11/bin implementation = CPython version_info = 3.11.2.final.0 virtualenv = 20.17.1 include-system-site-packages = false base-prefix = /opt/homebrew/opt/python@3.11/Frameworks/Python.framework/Versions/3.11 base-exec-prefix = /opt/homebrew/opt/python@3.11/Frameworks/Python.framework/Versions/3.11 base-executable = /opt/homebrew/opt/python@3.11/bin/python3.11 So the Python 3.11 I installed through Homebrew doesn't exhibit this bug.