home / github

Menu
  • Search all tables
  • GraphQL API

issue_comments

Table actions
  • GraphQL API for issue_comments

6 rows where author_association = "OWNER", issue = 1886771493 and user = 9599 sorted by updated_at descending

✖
✖
✖
✖

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date)

user 1

  • simonw · 6 ✖

issue 1

  • `table.transform()` should preserve `rowid` values · 6 ✖

author_association 1

  • OWNER · 6 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions issue performed_via_github_app
1712895580 https://github.com/simonw/sqlite-utils/issues/592#issuecomment-1712895580 https://api.github.com/repos/simonw/sqlite-utils/issues/592 IC_kwDOCGYnMM5mGLZc simonw 9599 2023-09-10T17:46:41Z 2023-09-10T17:46:41Z OWNER

In working on this I learned that rowid values in SQLite are way less stable than I had thought - in particular, they are often entirely rewritten on a VACUUM:

https://www.sqlite.org/lang_vacuum.html#how_vacuum_works

The VACUUM command may change the ROWIDs of entries in any tables that do not have an explicit INTEGER PRIMARY KEY.

So this fix wasn't as valuable as I thought. I need to move away from ever assuming that a rowid is a useful foreign key for anything.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
`table.transform()` should preserve `rowid` values 1886771493  
1710935270 https://github.com/simonw/sqlite-utils/issues/592#issuecomment-1710935270 https://api.github.com/repos/simonw/sqlite-utils/issues/592 IC_kwDOCGYnMM5l-szm simonw 9599 2023-09-08T00:55:30Z 2023-09-08T00:55:30Z OWNER

Yes! That recreated the bug: ```

  assert previous_rows == next_rows

E AssertionError: assert equals failed E [ [
E (1, '1', 'Paris'), (1, '1', 'Paris'),
E (3, '3', 'New York'), (2, '3', 'New York'),
E (4, '4', 'London'), (3, '4', 'London'),
E ] ... E
```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
`table.transform()` should preserve `rowid` values 1886771493  
1710934448 https://github.com/simonw/sqlite-utils/issues/592#issuecomment-1710934448 https://api.github.com/repos/simonw/sqlite-utils/issues/592 IC_kwDOCGYnMM5l-smw simonw 9599 2023-09-08T00:54:03Z 2023-09-08T00:54:03Z OWNER

Oh! Maybe the row ID preservation here is a coincidence because the tables are created from scratch and count 1, 2, 3.

If I delete a row from the table and then insert some more - breaking the rowid sequence - it might show the bug.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
`table.transform()` should preserve `rowid` values 1886771493  
1710933716 https://github.com/simonw/sqlite-utils/issues/592#issuecomment-1710933716 https://api.github.com/repos/simonw/sqlite-utils/issues/592 IC_kwDOCGYnMM5l-sbU simonw 9599 2023-09-08T00:52:42Z 2023-09-08T00:52:42Z OWNER

I just noticed that the table where I encountered this bug wasn't actually a rowid table after all - it had an id column that was a text primary key.

The reason the rowid was important is that's how the FTS mechanism in Datasette relates FTS entries to their rows.

But I tried this test and it passed, too: python def test_transform_preserves_rowids(fresh_db): fresh_db["places"].insert_all( [ {"id": "1", "name": "Paris", "country": "France"}, {"id": "2", "name": "London", "country": "UK"}, {"id": "3", "name": "New York", "country": "USA"}, ], pk="id", ) previous_rows = list( tuple(row) for row in fresh_db.execute("select rowid, id, name from places") ) # Transform it fresh_db["places"].transform(column_order=("country", "name")) # Should be the same next_rows = list( tuple(row) for row in fresh_db.execute("select rowid, id, name from places") ) assert previous_rows == next_rows

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
`table.transform()` should preserve `rowid` values 1886771493  
1710931605 https://github.com/simonw/sqlite-utils/issues/592#issuecomment-1710931605 https://api.github.com/repos/simonw/sqlite-utils/issues/592 IC_kwDOCGYnMM5l-r6V simonw 9599 2023-09-08T00:49:02Z 2023-09-08T00:49:02Z OWNER

I tried bumping that up to 10,000 rows instead of just 3 but the test still passed.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
`table.transform()` should preserve `rowid` values 1886771493  
1710930934 https://github.com/simonw/sqlite-utils/issues/592#issuecomment-1710930934 https://api.github.com/repos/simonw/sqlite-utils/issues/592 IC_kwDOCGYnMM5l-rv2 simonw 9599 2023-09-08T00:47:57Z 2023-09-08T00:47:57Z OWNER

That's odd, I wrote a test for this just now and it passes already: python def test_transform_preserves_rowids(fresh_db): # Create a rowid table fresh_db["places"].insert_all( ( {"name": "Paris", "country": "France"}, {"name": "London", "country": "UK"}, {"name": "New York", "country": "USA"}, ), ) assert fresh_db["places"].use_rowid previous_rows = list( tuple(row) for row in fresh_db.execute("select rowid, name from places") ) # Transform it fresh_db["places"].transform(column_order=("country", "name")) # Should be the same next_rows = list( tuple(row) for row in fresh_db.execute("select rowid, name from places") ) assert previous_rows == next_rows So maybe I'm wrong about the cause of that bug?

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
`table.transform()` should preserve `rowid` values 1886771493  

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
, [performed_via_github_app] TEXT);
CREATE INDEX [idx_issue_comments_issue]
                ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
                ON [issue_comments] ([user]);
Powered by Datasette · Queries took 1316.992ms · About: github-to-sqlite
  • Sort ascending
  • Sort descending
  • Facet by this
  • Hide this column
  • Show all columns
  • Show not-blank rows