{"html_url": "https://github.com/simonw/sqlite-utils/issues/145#issuecomment-689186423", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/145", "id": 689186423, "node_id": "MDEyOklzc3VlQ29tbWVudDY4OTE4NjQyMw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-08T23:21:23Z", "updated_at": "2020-09-08T23:21:23Z", "author_association": "OWNER", "body": "Fixed in PR #146.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 688659182, "label": "Bug when first record contains fewer columns than subsequent records"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/pull/146#issuecomment-689185393", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/146", "id": 689185393, "node_id": "MDEyOklzc3VlQ29tbWVudDY4OTE4NTM5Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-08T23:17:42Z", "updated_at": "2020-09-08T23:17:42Z", "author_association": "OWNER", "body": "That seems like a reasonable approach to me, especially since this is going to be a pretty rare edge-case.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 688668680, "label": "Handle case where subsequent records (after first batch) include extra columns"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/155#issuecomment-689166404", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/155", "id": 689166404, "node_id": "MDEyOklzc3VlQ29tbWVudDY4OTE2NjQwNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-08T22:20:03Z", "updated_at": "2020-09-08T22:20:03Z", "author_association": "OWNER", "body": "I'm going to update `sqlite-utils optimize` to also take an optional list of tables, for consistency.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 696045581, "label": "rebuild-fts command and table.rebuild_fts() method"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/153#issuecomment-689165985", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/153", "id": 689165985, "node_id": "MDEyOklzc3VlQ29tbWVudDY4OTE2NTk4NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-08T22:18:52Z", "updated_at": "2020-09-08T22:18:52Z", "author_association": "OWNER", "body": "I've reverted this change again, because it turns out using the `rebuild` FTS mechanism is a better way of repairing this issue - see #155.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 695377804, "label": "table.optimize() should delete junk rows from *_fts_docsize"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/155#issuecomment-689163158", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/155", "id": 689163158, "node_id": "MDEyOklzc3VlQ29tbWVudDY4OTE2MzE1OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-08T22:10:27Z", "updated_at": "2020-09-08T22:10:27Z", "author_association": "OWNER", "body": "For the command version:\r\n\r\n    sqlite-utils rebuild-fts mydb.db\r\n\r\nThis will rebuild all detected FTS tables. You can also specify one or more explicit tables:\r\n\r\n    sqlite-utils rebuild-fts mydb.db dogs\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 696045581, "label": "rebuild-fts command and table.rebuild_fts() method"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/dogsheep-beta/issues/19#issuecomment-688626037", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/19", "id": 688626037, "node_id": "MDEyOklzc3VlQ29tbWVudDY4ODYyNjAzNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-08T05:27:07Z", "updated_at": "2020-09-08T05:27:07Z", "author_association": "MEMBER", "body": "A really clever way to do this would be with triggers. The indexer script would add triggers to each of the database tables that it is indexing - each in their own database.\r\n\r\nThose triggers would then maintain a `_index_queue_` table. This table would record the primary key of rows that are added, modified or deleted. The indexer could then work by reading through the `_index_queue_` table, re-indexing (or deleting) just the primary keys listed there, and then emptying the queue once it has finished.\r\n\r\nThis would add a small amount of overhead to insert/update/delete queries run against the table. My hunch is that the overhead would be miniscule, but I could still allow people to opt-out for tables that are so high traffic that this would matter.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 695556681, "label": "Figure out incremental re-indexing"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/dogsheep-beta/issues/19#issuecomment-688625430", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/19", "id": 688625430, "node_id": "MDEyOklzc3VlQ29tbWVudDY4ODYyNTQzMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-08T05:24:50Z", "updated_at": "2020-09-08T05:24:50Z", "author_association": "MEMBER", "body": "I thought about allowing tables to define a incremental indexing SQL query - maybe something that can return just records touched in the past hour, or records since a recorded \"last indexed record\" value.\r\n\r\nThe problem with this is deletes - if you delete a record, how does the indexer know to remove it? See #18  - that's already caused problems.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 695556681, "label": "Figure out incremental re-indexing"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/dogsheep-beta/issues/18#issuecomment-688623097", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/18", "id": 688623097, "node_id": "MDEyOklzc3VlQ29tbWVudDY4ODYyMzA5Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-08T05:15:51Z", "updated_at": "2020-09-08T05:15:51Z", "author_association": "MEMBER", "body": "I'm inclined to go with the first, simpler option. I have longer term plans for efficient incremental index updates based on clever trickery with triggers.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 695553522, "label": "Deleted records stay in the search index"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/dogsheep-beta/issues/18#issuecomment-688622995", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/18", "id": 688622995, "node_id": "MDEyOklzc3VlQ29tbWVudDY4ODYyMjk5NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-08T05:15:21Z", "updated_at": "2020-09-08T05:15:21Z", "author_association": "MEMBER", "body": "Alternatively it could run as it does now but add a `DELETE FROM index1.search_index WHERE key not in (select key from ...)`.\r\n\r\nI'm not sure which would be more efficient.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 695553522, "label": "Deleted records stay in the search index"}, "performed_via_github_app": null}