html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,issue,performed_via_github_app
https://github.com/simonw/datasette/issues/2065#issuecomment-1524709988,https://api.github.com/repos/simonw/datasette/issues/2065,1524709988,IC_kwDOBm6k_c5a4Tpk,9599,2023-04-27T05:09:36Z,2023-04-27T05:09:36Z,OWNER,"That fixed it - after installing `main.zip` again I ran this and it worked:
~/.rye/tools/main-zip/bin/datasette install datasette-graphql
```
% ~/.rye/tools/main-zip/bin/datasette plugins
[
{
""name"": ""datasette-graphql"",
""static"": true,
""templates"": true,
""version"": ""2.2"",
""hooks"": [
""database_actions"",
""extra_template_vars"",
""menu_links"",
""register_routes"",
""startup"",
""table_actions""
]
}
]
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686033652,
https://github.com/simonw/datasette/issues/2065#issuecomment-1524707628,https://api.github.com/repos/simonw/datasette/issues/2065,1524707628,IC_kwDOBm6k_c5a4TEs,9599,2023-04-27T05:06:44Z,2023-04-27T05:06:44Z,OWNER,"I need `pip` as a dependency too:
```
% ~/.rye/tools/main-zip/bin/datasette install datasette-graphql
Traceback (most recent call last):
File ""/Users/simon/.rye/tools/main-zip/bin/datasette"", line 8, in <module>
sys.exit(cli())
^^^^^
File ""/Users/simon/.rye/tools/main-zip/lib/python3.11/site-packages/click/core.py"", line 1130, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/Users/simon/.rye/tools/main-zip/lib/python3.11/site-packages/click/core.py"", line 1055, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File ""/Users/simon/.rye/tools/main-zip/lib/python3.11/site-packages/click/core.py"", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/Users/simon/.rye/tools/main-zip/lib/python3.11/site-packages/click/core.py"", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/Users/simon/.rye/tools/main-zip/lib/python3.11/site-packages/click/core.py"", line 760, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/Users/simon/.rye/tools/main-zip/lib/python3.11/site-packages/datasette/cli.py"", line 365, in install
run_module(""pip"", run_name=""__main__"")
File ""<frozen runpy>"", line 222, in run_module
File ""<frozen runpy>"", line 142, in _get_module_details
ImportError: No module named pip
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686033652,
https://github.com/simonw/datasette/issues/2065#issuecomment-1524699863,https://api.github.com/repos/simonw/datasette/issues/2065,1524699863,IC_kwDOBm6k_c5a4RLX,9599,2023-04-27T04:56:22Z,2023-04-27T04:56:22Z,OWNER,Turned this into a TIL: https://til.simonwillison.net/python/rye,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686033652,
https://github.com/simonw/datasette/issues/2066#issuecomment-1524680160,https://api.github.com/repos/simonw/datasette/issues/2066,1524680160,IC_kwDOBm6k_c5a4MXg,9599,2023-04-27T04:27:50Z,2023-04-27T04:27:50Z,OWNER,That fixed it.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686042269,
https://github.com/simonw/datasette/issues/2066#issuecomment-1524675817,https://api.github.com/repos/simonw/datasette/issues/2066,1524675817,IC_kwDOBm6k_c5a4LTp,9599,2023-04-27T04:21:03Z,2023-04-27T04:21:03Z,OWNER,I went with this to generate the long string: https://github.com/simonw/datasette/blob/0b0c5cd7a94fe3f151a3e10261b5c84ee64f2f18/tests/test_csv.py#L157-L176,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686042269,
https://github.com/simonw/datasette/issues/2066#issuecomment-1524669124,https://api.github.com/repos/simonw/datasette/issues/2066,1524669124,IC_kwDOBm6k_c5a4JrE,9599,2023-04-27T04:10:44Z,2023-04-27T04:10:52Z,OWNER,"I need an alternative way of generating a long string with a shorter URL.
```sql
select group_concat('abcabcabc', '') from json_each(json_array(1, 2, 3, 4, 5, 6, 7, 8, 9, 10))
```
https://latest.datasette.io/_memory?sql=select+group_concat%28%27abcabcabc%27%2C+%27%27%29+from+json_each%28json_array%281%2C+2%2C+3%2C+4%2C+5%2C+6%2C+7%2C+8%2C+9%2C+10%29%29","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686042269,
https://github.com/simonw/datasette/issues/2066#issuecomment-1524666049,https://api.github.com/repos/simonw/datasette/issues/2066,1524666049,IC_kwDOBm6k_c5a4I7B,9599,2023-04-27T04:06:18Z,2023-04-27T04:06:18Z,OWNER,"Most recent `httpx` release is 0.24 a couple of weeks ago. Here's what changed in that version: https://github.com/encode/httpx/compare/0.23.3...0.24.0
It looks like that URL limit is new: https://github.com/encode/httpx/commit/57daabf673705954afa94686c0002801c93d31f3#diff-78d8d93b5dd4c77d99c3e2b46b7286ba71a8fd60e92d8bd4eee5fb200b4f87bfR149-R155
```python
def urlparse(url: str = """", **kwargs: typing.Optional[str]) -> ParseResult:
# Initial basic checks on allowable URLs.
# ---------------------------------------
# Hard limit the maximum allowable URL length.
if len(url) > MAX_URL_LENGTH:
raise InvalidURL(""URL too long"")
```
https://github.com/encode/httpx/blob/32e25497a36e6222cc64a758c98275b450dac28d/httpx/_urlparse.py#L153-L155","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686042269,
https://github.com/simonw/datasette/issues/2066#issuecomment-1524660603,https://api.github.com/repos/simonw/datasette/issues/2066,1524660603,IC_kwDOBm6k_c5a4Hl7,9599,2023-04-27T04:02:55Z,2023-04-27T04:02:55Z,OWNER,"In the debugger:
```
(Pdb) MAX_URL_LENGTH
65536
```
Weird this only seems to be a problem with `httpx` on Python 3.7 though.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686042269,
https://github.com/simonw/datasette/issues/2066#issuecomment-1524659084,https://api.github.com/repos/simonw/datasette/issues/2066,1524659084,IC_kwDOBm6k_c5a4HOM,9599,2023-04-27T04:02:07Z,2023-04-27T04:02:07Z,OWNER,"This is the failing test:
https://github.com/simonw/datasette/blob/249fcf8e3e2a90e763f41b080c1b9ec8017f5005/tests/test_csv.py#L156-L167","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686042269,
https://github.com/simonw/datasette/issues/2066#issuecomment-1524655203,https://api.github.com/repos/simonw/datasette/issues/2066,1524655203,IC_kwDOBm6k_c5a4GRj,9599,2023-04-27T03:59:56Z,2023-04-27T03:59:56Z,OWNER,"In a fresh Datasette checkout I ran:
pipenv shell --python /Users/simon/.pyenv/versions/3.7.16/bin/python
That gave me a virtual environment with 3.7.16 Python.
Then I ran:
pip install -e '.[test]'
Weirdly that gave me a `which pytest` of `/opt/homebrew/bin/pytest` which ran the tests on 3.11.
I figured out the location of the virtual environment with `which python` and then ran this:
```
% /Users/simon/.local/share/virtualenvs/datasette-cZYvnUqY/bin/pytest tests/test_csv.py
============================================================================================== test session starts ===============================================================================================
platform darwin -- Python 3.7.16, pytest-7.3.1, pluggy-1.0.0
SQLite: 3.39.5
rootdir: /private/tmp/datasette
configfile: pytest.ini
plugins: asyncio-0.21.0, timeout-2.1.0, xdist-3.2.1, anyio-3.6.2
asyncio: mode=strict
collected 15 items
tests/test_csv.py ..........F.... [100%]
==================================================================================================== FAILURES ====================================================================================================
________________________________________________________________________________________________ test_max_csv_mb _________________________________________________________________________________________________
app_client_csv_max_mb_one = <datasette.utils.testing.TestClient object at 0x107ab5c08>
def test_max_csv_mb(app_client_csv_max_mb_one):
response = app_client_csv_max_mb_one.get(
(
""/fixtures.csv?sql=select+'{}'+""
""from+compound_three_primary_keys&_stream=1&_size=max""
> ).format(""abcdefg"" * 10000)
)
/private/tmp/datasette/tests/test_csv.py:161:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/Users/simon/.local/share/virtualenvs/datasette-cZYvnUqY/lib/python3.7/site-packages/asgiref/sync.py:240: in __call__
return call_result.result()
/Users/simon/.pyenv/versions/3.7.16/lib/python3.7/concurrent/futures/_base.py:428: in result
return self.__get_result()
/Users/simon/.pyenv/versions/3.7.16/lib/python3.7/concurrent/futures/_base.py:384: in __get_result
raise self._exception
/Users/simon/.local/share/virtualenvs/datasette-cZYvnUqY/lib/python3.7/site-packages/asgiref/sync.py:306: in main_wrap
result = await self.awaitable(*args, **kwargs)
/private/tmp/datasette/datasette/utils/testing.py:76: in get
headers=headers,
/private/tmp/datasette/datasette/utils/testing.py:167: in _request
content=post_body,
/private/tmp/datasette/datasette/app.py:1787: in request
method, self._fix(path, avoid_path_rewrites), **kwargs
/Users/simon/.local/share/virtualenvs/datasette-cZYvnUqY/lib/python3.7/site-packages/httpx/_client.py:1528: in request
extensions=extensions,
/Users/simon/.local/share/virtualenvs/datasette-cZYvnUqY/lib/python3.7/site-packages/httpx/_client.py:346: in build_request
url = self._merge_url(url)
/Users/simon/.local/share/virtualenvs/datasette-cZYvnUqY/lib/python3.7/site-packages/httpx/_client.py:376: in _merge_url
merge_url = URL(url)
/Users/simon/.local/share/virtualenvs/datasette-cZYvnUqY/lib/python3.7/site-packages/httpx/_urls.py:113: in __init__
self._uri_reference = urlparse(url, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
url = ""http://localhost/fixtures.csv?sql=select+'abcdefgabcdefgabcdefgabcdefgabcdefgabcdefgabcdefgabcdefgabcdefgabcdefgabcde...gabcdefgabcdefgabcdefgabcdefgabcdefgabcdefgabcdefgabcdefgabcdefg'+from+compound_three_primary_keys&_stream=1&_size=max""
kwargs = {}
def urlparse(url: str = """", **kwargs: typing.Optional[str]) -> ParseResult:
# Initial basic checks on allowable URLs.
# ---------------------------------------
# Hard limit the maximum allowable URL length.
if len(url) > MAX_URL_LENGTH:
> raise InvalidURL(""URL too long"")
E httpx.InvalidURL: URL too long
/Users/simon/.local/share/virtualenvs/datasette-cZYvnUqY/lib/python3.7/site-packages/httpx/_urlparse.py:155: InvalidURL
============================================================================================ short test summary info =============================================================================================
FAILED tests/test_csv.py::test_max_csv_mb - httpx.InvalidURL: URL too long
========================================================================================== 1 failed, 14 passed in 1.83s ==========================================================================================
(datasette) simon@Simons-MacBook-Pro datasette %
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686042269,
https://github.com/simonw/datasette/issues/2066#issuecomment-1524648995,https://api.github.com/repos/simonw/datasette/issues/2066,1524648995,IC_kwDOBm6k_c5a4Ewj,9599,2023-04-27T03:56:42Z,2023-04-27T03:57:11Z,OWNER,"I don't have 3.7 locally. Trying to install it with `pyenv`.
brew install pyenv
Then:
pyenv install --list | grep 3.7
Installing:
pyenv install 3.7.16
Output:
Installed Python-3.7.16 to /Users/simon/.pyenv/versions/3.7.16
So the executable is `/Users/simon/.pyenv/versions/3.7.16/bin/python`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686042269,
https://github.com/simonw/datasette/issues/2066#issuecomment-1524638233,https://api.github.com/repos/simonw/datasette/issues/2066,1524638233,IC_kwDOBm6k_c5a4CIZ,9599,2023-04-27T03:50:51Z,2023-04-27T03:50:51Z,OWNER,Failure was on 3.7. I'll try that.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686042269,
https://github.com/simonw/datasette/issues/2066#issuecomment-1524637376,https://api.github.com/repos/simonw/datasette/issues/2066,1524637376,IC_kwDOBm6k_c5a4B7A,9599,2023-04-27T03:50:19Z,2023-04-27T03:50:19Z,OWNER,"Having trouble replicating this on my laptop. I tried a fresh virtual environment with fresh packages (in case this is a `httpx` change) but this passed:
pytest tests/test_csv.py","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686042269,
https://github.com/simonw/datasette/issues/2065#issuecomment-1524616740,https://api.github.com/repos/simonw/datasette/issues/2065,1524616740,IC_kwDOBm6k_c5a384k,9599,2023-04-27T03:38:21Z,2023-04-27T03:38:21Z,OWNER,"Tried this:
rye install https://github.com/simonw/datasette/archive/refs/heads/main.zip
But got this error:
Error: Expected one of `@`, `(`, `<`, `=`, `>`, `~`, `!`, `;`, found `:`
I instead downloaded that file and ran:
rye install main.zip
This worked! And now:
```
~/.rye/tools/main-zip/bin/datasette --version
```
```
datasette, version 1.0a2
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1686033652,