github
html_url | issue_url | id | node_id | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
https://github.com/simonw/sqlite-utils/issues/592#issuecomment-1712895580 | https://api.github.com/repos/simonw/sqlite-utils/issues/592 | 1712895580 | IC_kwDOCGYnMM5mGLZc | 9599 | 2023-09-10T17:46:41Z | 2023-09-10T17:46:41Z | OWNER | In working on this I learned that `rowid` values in SQLite are way less stable than I had thought - in particular, they are often entirely rewritten on a `VACUUM`: https://www.sqlite.org/lang_vacuum.html#how_vacuum_works > The VACUUM command may change the [ROWIDs](https://www.sqlite.org/lang_createtable.html#rowid) of entries in any tables that do not have an explicit [INTEGER PRIMARY KEY](https://www.sqlite.org/lang_createtable.html#rowid). So this fix wasn't as valuable as I thought. I need to move away from ever assuming that a `rowid` is a useful foreign key for anything. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1886771493 | |
https://github.com/simonw/sqlite-utils/issues/592#issuecomment-1710935270 | https://api.github.com/repos/simonw/sqlite-utils/issues/592 | 1710935270 | IC_kwDOCGYnMM5l-szm | 9599 | 2023-09-08T00:55:30Z | 2023-09-08T00:55:30Z | OWNER | Yes! That recreated the bug: ``` > assert previous_rows == next_rows E AssertionError: assert equals failed E [ [ E (1, '1', 'Paris'), (1, '1', 'Paris'), E (3, '3', 'New York'), (2, '3', 'New York'), E (4, '4', 'London'), (3, '4', 'London'), E ] ... E ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1886771493 | |
https://github.com/simonw/sqlite-utils/issues/592#issuecomment-1710934448 | https://api.github.com/repos/simonw/sqlite-utils/issues/592 | 1710934448 | IC_kwDOCGYnMM5l-smw | 9599 | 2023-09-08T00:54:03Z | 2023-09-08T00:54:03Z | OWNER | Oh! Maybe the row ID preservation here is a coincidence because the tables are created from scratch and count 1, 2, 3. If I delete a row from the table and then insert some more - breaking the `rowid` sequence - it might show the bug. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1886771493 | |
https://github.com/simonw/sqlite-utils/issues/592#issuecomment-1710933716 | https://api.github.com/repos/simonw/sqlite-utils/issues/592 | 1710933716 | IC_kwDOCGYnMM5l-sbU | 9599 | 2023-09-08T00:52:42Z | 2023-09-08T00:52:42Z | OWNER | I just noticed that the table where I encountered this bug wasn't actually a `rowid` table after all - it had an `id` column that was a text primary key. The reason the `rowid` was important is that's how the FTS mechanism in Datasette relates FTS entries to their rows. But I tried this test and it passed, too: ```python def test_transform_preserves_rowids(fresh_db): fresh_db["places"].insert_all( [ {"id": "1", "name": "Paris", "country": "France"}, {"id": "2", "name": "London", "country": "UK"}, {"id": "3", "name": "New York", "country": "USA"}, ], pk="id", ) previous_rows = list( tuple(row) for row in fresh_db.execute("select rowid, id, name from places") ) # Transform it fresh_db["places"].transform(column_order=("country", "name")) # Should be the same next_rows = list( tuple(row) for row in fresh_db.execute("select rowid, id, name from places") ) assert previous_rows == next_rows ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1886771493 | |
https://github.com/simonw/sqlite-utils/issues/592#issuecomment-1710931605 | https://api.github.com/repos/simonw/sqlite-utils/issues/592 | 1710931605 | IC_kwDOCGYnMM5l-r6V | 9599 | 2023-09-08T00:49:02Z | 2023-09-08T00:49:02Z | OWNER | I tried bumping that up to 10,000 rows instead of just 3 but the test still passed. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1886771493 | |
https://github.com/simonw/sqlite-utils/issues/592#issuecomment-1710930934 | https://api.github.com/repos/simonw/sqlite-utils/issues/592 | 1710930934 | IC_kwDOCGYnMM5l-rv2 | 9599 | 2023-09-08T00:47:57Z | 2023-09-08T00:47:57Z | OWNER | That's odd, I wrote a test for this just now and it passes already: ```python def test_transform_preserves_rowids(fresh_db): # Create a rowid table fresh_db["places"].insert_all( ( {"name": "Paris", "country": "France"}, {"name": "London", "country": "UK"}, {"name": "New York", "country": "USA"}, ), ) assert fresh_db["places"].use_rowid previous_rows = list( tuple(row) for row in fresh_db.execute("select rowid, name from places") ) # Transform it fresh_db["places"].transform(column_order=("country", "name")) # Should be the same next_rows = list( tuple(row) for row in fresh_db.execute("select rowid, name from places") ) assert previous_rows == next_rows ``` So maybe I'm wrong about the cause of that bug? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1886771493 |