github
id | node_id | number | title | user | state | locked | assignee | milestone | comments | created_at | updated_at | closed_at | author_association | pull_request | body | repo | type | active_lock_reason | performed_via_github_app | reactions | draft | state_reason |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
648637666 | MDU6SXNzdWU2NDg2Mzc2NjY= | 880 | POST to /db/canned-query that returns JSON should be supported (for API clients) | 9599 | closed | 0 | 5818042 | 11 | 2020-07-01T03:14:43Z | 2020-09-14T21:28:21Z | 2020-09-14T21:25:01Z | OWNER | Now that CSRF is solved for API requests (#835) it would be good to support API requests to the `.json` extension. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/880/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
701294727 | MDU6SXNzdWU3MDEyOTQ3Mjc= | 965 | Documentation for 404.html, 500.html templates | 9599 | closed | 0 | 5818042 | 3 | 2020-09-14T17:36:59Z | 2020-09-14T18:49:49Z | 2020-09-14T18:47:22Z | OWNER | This mechanism is not documented: https://github.com/simonw/datasette/blob/30b98e4d2955073ca2bca92ca7b3d97fcd0191bf/datasette/app.py#L1119-L1129 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/965/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
700728217 | MDU6SXNzdWU3MDA3MjgyMTc= | 964 | raise_404 mechanism for custom templates | 9599 | closed | 0 | 5818042 | 1 | 2020-09-14T03:22:15Z | 2020-09-14T17:49:44Z | 2020-09-14T17:39:34Z | OWNER | > Having tried this out I think it does need a `raise_404()` mechanism - which needs to be smart enough to trigger the default 404 handler without accidentally going into an infinite loop. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/944#issuecomment-691788478_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/964/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
681516976 | MDU6SXNzdWU2ODE1MTY5NzY= | 944 | Path parameters for custom pages | 9599 | closed | 0 | 5818042 | 5 | 2020-08-19T03:25:17Z | 2020-09-14T03:21:45Z | 2020-09-14T02:34:58Z | OWNER | [Custom pages](https://docs.datasette.io/en/stable/custom_templates.html#custom-pages) let you e.g. create a `templates/pages/about.html` page and have it automatically served at `/about`. It would be useful if these pages could capture path patterns. I like the Python format string syntax for this (also used by Starlette): `/foo/bar/{slug}`. So... how about embedding those patterns in the filenames themselves? templates/pages/museums/{slug}.html Would capture any hits to `/museums/something` and use that page to serve them. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/944/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
698791218 | MDU6SXNzdWU2OTg3OTEyMTg= | 50 | favorites --stop_after=N stops after min(N, 200) | 370930 | open | 0 | 2 | 2020-09-11T03:38:14Z | 2020-09-13T05:11:14Z | CONTRIBUTOR | For any number greater than 200, `favorites --stop_after` stops after getting 200 tweets, e.g. ``` $ twitter-to-sqlite favorites tweets.db --stop_after=300 Importing favorites [####################################] 199 $ ``` I don't _think_ this is a limitation of the API (if you omit `--stop_after` you get some very large number, possibly all of them), so I _think_ this is a bug. | 206156866 | issue | { "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/50/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
699947574 | MDU6SXNzdWU2OTk5NDc1NzQ= | 963 | Currently selected array facets are not correctly persisted through hidden form fields | 649467 | closed | 0 | 5818042 | 1 | 2020-09-12T01:49:17Z | 2020-09-12T21:54:29Z | 2020-09-12T21:54:09Z | NONE | Faceted search uses JSON array elements as facets rather than the arrays. However, if a search is "Apply"ed (using the Apply button), the array itself rather than its elements used. To reproduce: https://latest.datasette.io/fixtures/facetable?_sort=pk&_facet=created&_facet=tags&_facet_array=tags Press "Apply", which might be done when removing a filter. Notice that the "tags" facet values are now arrays, not array elements. It appears the "&_facet_array=tags" element of the query string is dropped. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/963/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
569275763 | MDU6SXNzdWU1NjkyNzU3NjM= | 680 | Release automation: automate the bit that posts the GitHub release | 9599 | closed | 0 | 5 | 2020-02-22T03:50:40Z | 2020-09-12T18:18:50Z | 2020-09-12T18:18:50Z | OWNER | The most manual part of [the release process](https://datasette.readthedocs.io/en/stable/contributing.html#release-process) right now is having to post a GitHub release that matches the updated changelog. This is particularly annoying because the changelog is in `.rst` while the GitHub release needs markdown - so I currently manually translate between the two. Having the release script automatically post a GitHub release at the end would be much more convenient. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/680/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
649429772 | MDU6SXNzdWU2NDk0Mjk3NzI= | 886 | Reconsider how _actor_X magic parameter deals with missing values | 9599 | open | 0 | 2 | 2020-07-02T00:00:38Z | 2020-09-11T21:35:26Z | OWNER | I had to build a custom `_actorornull` prefix for [datasette-saved-queries](https://github.com/simonw/datasette-saved-queries/blob/37c00e56ac398e1f9aa342d30357de013a9b37b4/datasette_saved_queries/__init__.py): ```python def actorornull(key, request): if request.actor is None: return None return request.actor.get(key) @hookimpl def register_magic_parameters(): return [ ("actorornull", actorornull), ] ``` Maybe the `actor` magic in Datasette core should do that out of the box? https://github.com/simonw/datasette/blob/f1f581b7ffcd5d8f3ae6c1c654d813a6641410eb/datasette/default_magic_parameters.py#L14-L17 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/886/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
691475400 | MDU6SXNzdWU2OTE0NzU0MDA= | 958 | Upgrade to latest Black (20.8b1) | 9599 | closed | 0 | 5818042 | 0 | 2020-09-02T22:24:19Z | 2020-09-11T21:34:24Z | 2020-09-02T22:25:10Z | OWNER | Black has some changes: https://black.readthedocs.io/en/stable/change_log.html#b0 - in particular: > - re-implemented support for explicit trailing commas: now it works consistently within any bracket pair, including nested structures (#1288 and duplicates) > - Black now reindents docstrings when reindenting code around it (#1053) | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/958/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
699622046 | MDU6SXNzdWU2OTk2MjIwNDY= | 962 | datasette --pdb option for debugging errors | 9599 | closed | 0 | 5818042 | 1 | 2020-09-11T18:33:10Z | 2020-09-11T21:34:24Z | 2020-09-11T18:38:01Z | OWNER | I needed to debug an exception from deep inside a Jinja template the other day. I hacked this together and it helped. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/962/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
684111953 | MDU6SXNzdWU2ODQxMTE5NTM= | 947 | datasette --get exit code should reflect HTTP errors | 9599 | closed | 0 | 5818042 | 1 | 2020-08-23T04:17:08Z | 2020-09-11T21:33:15Z | 2020-09-11T21:33:15Z | OWNER | If you run `datasette . --get /` and the result is a 500 or 404 error (anything that's not a 200 or a 30x) the exit code from the command should not be 0. It should still output the returned content to stdout. This will help with writing soundness checks, as seen in https://til.simonwillison.net/til/til/github-actions_grep-tests.md | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/947/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
697162939 | MDU6SXNzdWU2OTcxNjI5Mzk= | 20 | Add more tags so people can find your project. | 7902810 | open | 0 | 0 | 2020-09-09T21:14:09Z | 2020-09-09T21:14:09Z | NONE | quantified-self habit-tracking google-fit time-tracking wearables quantifiedself for example | 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/20/reactions", "total_count": 1, "+1": 0, "-1": 1, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
697030843 | MDExOlB1bGxSZXF1ZXN0NDgzMDI3NTg3 | 156 | Typos in tests | 96218 | closed | 0 | 1 | 2020-09-09T18:00:58Z | 2020-09-09T18:24:50Z | 2020-09-09T18:21:23Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/156 | One of these is my fault, and the other is one I just happened to come across. They're harmless, but might as well be fixed. | 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/156/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
696908389 | MDU6SXNzdWU2OTY5MDgzODk= | 961 | Verification checks for metadata.json on startup | 9599 | open | 0 | 2 | 2020-09-09T15:21:53Z | 2020-09-09T15:24:31Z | OWNER | I lost a bunch of time yesterday trying to figure out why a Datasette instance wasn't starting up - it turned out it was because I had a `facets:` reference that mentioned a column that did not exist. Catching these on startup would be good. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/961/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
694500679 | MDU6SXNzdWU2OTQ1MDA2Nzk= | 17 | Rename "table" to "type" | 9599 | closed | 0 | 2 | 2020-09-06T19:34:41Z | 2020-09-09T03:03:22Z | 2020-09-09T03:03:22Z | MEMBER | I think "table" is the wrong name for the concept I'm using it for here. Two reasons: firstly, `table` is a reserved word in SQLite. More importantly, it turns out there's not a direct mapping from tables to types of search result. In particular, for GitHub I ended up having two different "tables" of repositories - one for repos created by me, another for repos that I have starred. | 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/17/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
688659182 | MDU6SXNzdWU2ODg2NTkxODI= | 145 | Bug when first record contains fewer columns than subsequent records | 96218 | closed | 0 | 2 | 2020-08-30T05:44:44Z | 2020-09-08T23:21:23Z | 2020-09-08T23:21:23Z | CONTRIBUTOR | `insert_all()` selects the maximum batch size based on the number of fields in the first record. If the first record has fewer fields than subsequent records (and `alter=True` is passed), this can result in SQL statements with more than the maximum permitted number of host parameters. This situation is perhaps unlikely to occur, but could happen if the first record had, say, 10 columns, such that `batch_size` (based on `SQLITE_MAX_VARIABLE_NUMBER = 999`) would be 99. If the next 98 rows had 11 columns, the resulting SQL statement for the first batch would have `10 * 1 + 11 * 98 = 1088` host parameters (and subsequent batches, if the data were consistent from thereon out, would have `99 * 11 = 1089`). I suspect that this bug is masked somewhat by the fact that while: > [`SQLITE_MAX_VARIABLE_NUMBER`](https://www.sqlite.org/limits.html#max_variable_number) ... defaults to 999 for SQLite versions prior to 3.32.0 (2020-05-22) or 32766 for SQLite versions after 3.32.0. it is common that it is increased at compile time. Debian-based systems, for example, seem to ship with a version of sqlite compiled with `SQLITE_MAX_VARIABLE_NUMBER` set to 250,000, and I believe this is the case for homebrew installations too. A test for this issue might look like this: ```python def test_columns_not_in_first_record_should_not_cause_batch_to_be_too_large(fresh_db): # sqlite on homebrew and Debian/Ubuntu etc. is typically compiled with # SQLITE_MAX_VARIABLE_NUMBER set to 250,000, so we need to exceed this value to # trigger the error on these systems. THRESHOLD = 250000 extra_columns = 1 + (THRESHOLD - 1) // 99 records = [ {"c0": "first record"}, # one column in first record -> batch_size = 100 # fill out the batch with 99 records with enough columns to exceed THRESHOLD *[ dict([("c{}".format(i), j) for i in range(extra_columns)]) for j in range(99) ] ] try: fresh_db["too_many_columns"].insert_all(records, a… | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/145/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
688668680 | MDExOlB1bGxSZXF1ZXN0NDc1ODc0NDkz | 146 | Handle case where subsequent records (after first batch) include extra columns | 96218 | closed | 0 | 5 | 2020-08-30T07:13:58Z | 2020-09-08T23:20:37Z | 2020-09-08T23:20:37Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/146 | Addresses #145. I think this should do the job. If it meets with your approval I'll update this PR to include an update to the documentation -- I came across this bug while preparing a PR to update the documentation around `batch_size` in any event. | 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/146/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
695556681 | MDU6SXNzdWU2OTU1NTY2ODE= | 19 | Figure out incremental re-indexing | 9599 | open | 0 | 2 | 2020-09-08T05:23:31Z | 2020-09-08T05:27:07Z | MEMBER | As tables get bigger reindexing everything on a schedule (essentially recreating the entire index from scratch) will start to become a performance bottleneck. | 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/19/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
695553522 | MDU6SXNzdWU2OTU1NTM1MjI= | 18 | Deleted records stay in the search index | 9599 | open | 0 | 2 | 2020-09-08T05:14:23Z | 2020-09-08T05:15:51Z | MEMBER | Here's why: https://github.com/dogsheep/dogsheep-beta/blob/24f7898d41a39218058f174c75ba62f7c0fcfff6/dogsheep_beta/utils.py#L44-L53 That should probably do `DELETE FROM index1.search_index WHERE [table] = ?` first. | 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/18/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
695441530 | MDU6SXNzdWU2OTU0NDE1MzA= | 154 | OperationalError: cannot change into wal mode from within a transaction | 9599 | open | 0 | 2 | 2020-09-07T23:42:44Z | 2020-09-07T23:47:10Z | OWNER | I'm getting this error when running: sqlite-utils enable-wal beta.db `OperationalError: cannot change into wal mode from within a transaction` I'm worried that maybe that's because of this new code from #152: https://github.com/simonw/sqlite-utils/blob/deb2eb013ff85bbc828ebc244a9654f0d9c3139e/sqlite_utils/db.py#L128-L129 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/154/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
695359607 | MDU6SXNzdWU2OTUzNTk2MDc= | 150 | Feature for tracing SQL queries | 9599 | closed | 0 | 0 | 2020-09-07T19:43:08Z | 2020-09-07T21:57:01Z | 2020-09-07T21:57:01Z | OWNER | Debugging `sqlite-utils` when something weird happens (e.g. #149) can be a bit tricky since it runs a bunch of different SQL statements behind the scenes. An optional "tracing" mechanism for seeing what SQL is being executed would be useful. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/150/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
695360889 | MDExOlB1bGxSZXF1ZXN0NDgxNjE2NzA0 | 151 | Tracer mechanism for seeing underlying SQL | 9599 | closed | 0 | 0 | 2020-09-07T19:46:43Z | 2020-09-07T21:57:00Z | 2020-09-07T21:57:00Z | OWNER | simonw/sqlite-utils/pulls/151 | Refs #150. Needs tests and documentation, including for the new `db.execute()` and `db.executescript()` methods. | 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/151/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
695376054 | MDU6SXNzdWU2OTUzNzYwNTQ= | 152 | Turn on recursive_triggers by default | 9599 | closed | 0 | 2 | 2020-09-07T20:26:36Z | 2020-09-07T21:17:48Z | 2020-09-07T20:45:14Z | OWNER | https://www.sqlite.org/pragma.html#pragma_recursive_triggers says: > Prior to SQLite [version 3.6.18](https://www.sqlite.org/releaselog/3_6_18.html) (2009-09-11), recursive triggers were not supported. The behavior of SQLite was always as if this pragma was set to OFF. Support for recursive triggers was added in version 3.6.18 but was initially turned OFF by default, for compatibility. Recursive triggers may be turned on by default in future versions of SQLite. So I think the fix for the complex issue in #149 is to turn on `recursive_triggers` globally by default for `sqlite-utils`. _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/149#issuecomment-688499924_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/152/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
695319258 | MDU6SXNzdWU2OTUzMTkyNTg= | 149 | FTS table with 7 rows has _fts_docsize table with 9,141 rows | 9599 | closed | 0 | 10 | 2020-09-07T18:06:16Z | 2020-09-07T21:16:34Z | 2020-09-07T21:16:34Z | OWNER | I'm seeing a weird issue with some of the SQLite databases that I am using with the FTS5 module. I have a database with a `licenses` table that contains 7 rows: <https://github-to-sqlite.dogsheep.net/github/licenses> The FTS table also has 7 rows: <https://github-to-sqlite.dogsheep.net/github/licenses_fts> Somehow the accompanying `licenses_fts_docsize` shadow table now has 9,141 rows in it! <https://github-to-sqlite.dogsheep.net/github/licenses_fts_docsize> And `licenses_fts_data` has 41 rows - should I expect that to have 7 rows? <https://github-to-sqlite.dogsheep.net/github/licenses_fts_data> I have a hunch that it might be a problem with the triggers. These are the triggers that are updating that FTS table: <https://github-to-sqlite.dogsheep.net/github?sql=select+*+from+sqlite_master+where+type+%3D+%27trigger%27+and+tbl_name+%3D+%27licenses%27> | type | name | tbl_name | rootpage | sql | | --- | --- | --- | --- | --- | | trigger | licenses_ai | licenses | 0 | `CREATE TRIGGER [licenses_ai] AFTER INSERT ON [licenses] BEGIN INSERT INTO [licenses_fts] (rowid, [name]) VALUES (new.rowid, new.[name]); END` | | trigger | licenses_ad | licenses | 0 | `CREATE TRIGGER [licenses_ad] AFTER DELETE ON [licenses] BEGIN INSERT INTO [licenses_fts] ([licenses_fts], rowid, [name]) VALUES('delete', old.rowid, old.[name]); END` | | trigger | licenses_au | licenses | 0 | `CREATE TRIGGER [licenses_au] AFTER UPDATE ON [licenses] BEGIN INSERT INTO [licenses_fts] ([licenses_fts], rowid, [name]) VALUES('delete', old.rowid, old.[name]); INSERT INTO [licenses_fts] (rowid, [name]) VALUES (new.rowid, new.[name]); END` | | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/149/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
695276328 | MDU6SXNzdWU2OTUyNzYzMjg= | 148 | More attractive indentation of created FTS table schema | 9599 | closed | 0 | 1 | 2020-09-07T16:49:30Z | 2020-09-07T18:12:50Z | 2020-09-07T18:12:50Z | OWNER | On https://github-to-sqlite.dogsheep.net/github/licenses_fts the create table SQL is displayed as: ```sql CREATE VIRTUAL TABLE [licenses_fts] USING FTS5 ( [name], content=[licenses] ); ``` It would be more aesthetically pleasing if it looked like this: ```sql CREATE VIRTUAL TABLE [licenses_fts] USING FTS5 ( [name], content=[licenses] ); ``` | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/148/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
693318095 | MDU6SXNzdWU2OTMzMTgwOTU= | 14 | On FTS exception rerun the query with quoting | 9599 | closed | 0 | 0 | 2020-09-04T15:44:18Z | 2020-09-05T16:23:01Z | 2020-09-05T16:23:01Z | MEMBER | Searching for eg `#dogfest` currently throws an FTS exception - but I want to support advanced FTS query tricks as seen in #13. https://dogsheep.simonwillison.net/-/beta?q=%23dogfest > fts5: syntax error near "#" Idea: catch that error and re-run the query with FTS escaping applied! | 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/14/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
692202408 | MDU6SXNzdWU2OTIyMDI0MDg= | 12 | Idea: maps and GeoJSON support | 9599 | open | 0 | 0 | 2020-09-03T18:47:10Z | 2020-09-04T01:45:03Z | MEMBER | It would be cool if the `display_sql` could return a column populated with GeoJSON which would the automatically be displayed on a map in the results (or maybe default JS would look for a `class="geojson"` element output by the `display` template) - ala https://github.com/simonw/datasette-leaflet-geojson Then I could render workout routes on a map, or Swarm checkin points. | 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/12/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
692386625 | MDU6SXNzdWU2OTIzODY2MjU= | 13 | Support advanced FTS queries | 9599 | closed | 0 | 1 | 2020-09-03T21:29:56Z | 2020-09-03T21:40:51Z | 2020-09-03T21:40:51Z | MEMBER | `simon willison NOT screenshot` for example. | 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/13/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
691521965 | MDU6SXNzdWU2OTE1MjE5NjU= | 9 | Mechanism for defining custom display of results | 9599 | closed | 0 | 8 | 2020-09-03T00:14:07Z | 2020-09-03T21:12:14Z | 2020-09-03T21:09:55Z | MEMBER | Part of #3 - in particular I want to make sure my photos are displayed with a thumbnail. | 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/9/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
689810340 | MDU6SXNzdWU2ODk4MTAzNDA= | 3 | Datasette plugin to provide custom page for running faceted, ranked searches | 9599 | closed | 0 | 3 | 2020-09-01T05:00:22Z | 2020-09-03T21:01:41Z | 2020-09-03T21:01:41Z | MEMBER | This will be a page at `/-/beta` which renders using a custom template. It will offer a default timeline view plus search and facet by type/date. | 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/3/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
689847361 | MDU6SXNzdWU2ODk4NDczNjE= | 5 | Add a context column that's not searchable | 9599 | closed | 0 | 1 | 2020-09-01T06:13:42Z | 2020-09-03T18:43:50Z | 2020-09-03T18:43:50Z | MEMBER | I sometimes like to configure titles that are things like "Comment on issue X" or "Photo in Golden Gate Park" - these shouldn't be included in the search index but should be stored so they can be displayed to provide context. Add a column for this - probably called `context` - and make it so it can be populated. | 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/5/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
691557547 | MDU6SXNzdWU2OTE1NTc1NDc= | 10 | Category 3: received | 9599 | closed | 0 | 1 | 2020-09-03T01:40:36Z | 2020-09-03T17:38:51Z | 2020-09-03T17:38:51Z | MEMBER | A category for things that were sent to me: DMs, emails etc. Follows #7. | 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/10/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
692125110 | MDU6SXNzdWU2OTIxMjUxMTA= | 11 | Public / Private mechanism | 9599 | closed | 0 | 1 | 2020-09-03T16:47:03Z | 2020-09-03T17:33:52Z | 2020-09-03T17:33:52Z | MEMBER | Some of the data in Dogsheep is stuff that was written publicly - tweets, blog posts, GitHub commits to public repos. Some of it is private data - emails, photos, direct messages, Swarm checkins, commits to private repos. Being able to filter for just one or the other (or both) would be useful. Especially when giving demos! | 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/11/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
691537426 | MDU6SXNzdWU2OTE1Mzc0MjY= | 959 | Internals API idea: results.dicts in addition to results.rows | 9599 | open | 0 | 0 | 2020-09-03T00:50:17Z | 2020-09-03T00:50:17Z | OWNER | I just wrote this code: ```python results = await database.execute(SEARCH_SQL, {"query": query}) return [dict(r) for r in results.rows] ``` How about having `results.dicts` as a utility property that does that? | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/959/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
687245650 | MDExOlB1bGxSZXF1ZXN0NDc0NzAzMDA3 | 952 | Update black requirement from ~=19.10b0 to >=19.10,<21.0 | 27856297 | closed | 0 | 1 | 2020-08-27T13:31:36Z | 2020-09-02T22:26:17Z | 2020-09-02T22:26:16Z | CONTRIBUTOR | simonw/datasette/pulls/952 | Updates the requirements on [black](https://github.com/psf/black) to permit the latest version. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/psf/black/blob/master/CHANGES.md">black's changelog</a>.</em></p> <blockquote> <h3>20.8b1</h3> <h4><em>Packaging</em></h4> <ul> <li>explicitly depend on Click 7.1.2 or newer as <code>Black</code> no longer works with versions older than 7.0</li> </ul> <h3>20.8b0</h3> <h4><em>Black</em></h4> <ul> <li> <p>re-implemented support for explicit trailing commas: now it works consistently within any bracket pair, including nested structures (<a href="https://github-redirect.dependabot.com/psf/black/issues/1288">#1288</a> and duplicates)</p> </li> <li> <p><code>Black</code> now reindents docstrings when reindenting code around it (<a href="https://github-redirect.dependabot.com/psf/black/issues/1053">#1053</a>)</p> </li> <li> <p><code>Black</code> now shows colored diffs (<a href="https://github-redirect.dependabot.com/psf/black/issues/1266">#1266</a>)</p> </li> <li> <p><code>Black</code> is now packaged using 'py3' tagged wheels (<a href="https://github-redirect.dependabot.com/psf/black/issues/1388">#1388</a>)</p> </li> <li> <p><code>Black</code> now supports Python 3.8 code, e.g. star expressions in return statements (<a href="https://github-redirect.dependabot.com/psf/black/issues/1121">#1121</a>)</p> </li> <li> <p><code>Black</code> no longer normalizes capital R-string prefixes as those have a community-accepted meaning (<a href="https://github-redirect.dependabot.com/psf/black/issues/1244">#1244</a>)</p> </li> <li> <p><code>Black</code> now uses exit code 2 when specified configuration file doesn't exit (<a href="https://github-redirect.dependabot.com/psf/black/issues/1361">#1361</a>)</p> </li> <li> <p><code>Black</code> now works on AWS Lambda (<a href="https://github-redirect.dependabot.com/psf/black/issues/1141">#1141</a>)</p> </li> <li> <p>added <code>--force-exclude</code> argument (<a href="https://github-redirect.dependabot.com/p… | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/952/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
691265198 | MDU6SXNzdWU2OTEyNjUxOTg= | 7 | Mechanism for differentiating between "by me" and "liked by me" | 9599 | closed | 0 | 6 | 2020-09-02T17:44:37Z | 2020-09-02T21:06:28Z | 2020-09-02T21:06:28Z | MEMBER | Some of the content I'm indexing is by me - photos I've taken, tweets I wrote, commits, comments I posted. Some of it is stuff that I've "liked" or "bookmarked" in some way - favourited tweets, Pocket articles, starred GitHub repos. It woud be useful to be able to differentiate between the two. | 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/7/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
691369691 | MDU6SXNzdWU2OTEzNjk2OTE= | 8 | Create a view for running faceted searches | 9599 | closed | 0 | 1 | 2020-09-02T19:44:07Z | 2020-09-02T19:50:47Z | 2020-09-02T19:50:47Z | MEMBER | ```sql select search_index_fts.rank, search_index.rowid, search_index.[table], search_index.key, search_index.title, search_index.timestamp, search_index.search_1 from search_index join search_index_fts on search_index.rowid = search_index_fts.rowid order by search_index_fts.rank, search_index.timestamp desc ``` | 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/8/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
689809225 | MDU6SXNzdWU2ODk4MDkyMjU= | 2 | Apply porter stemming | 9599 | closed | 0 | 2 | 2020-09-01T04:57:55Z | 2020-09-01T20:42:00Z | 2020-09-01T20:40:24Z | MEMBER | This can be on by default. You can turn it off for a table in the config file using `stemming: none` - or maybe `tokenize: none` to match the terminology used by SQLite and `sqlite-utils`: https://sqlite-utils.readthedocs.io/en/stable/python-api.html#enabling-full-text-search | 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/2/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
689850810 | MDU6SXNzdWU2ODk4NTA4MTA= | 6 | Set up a demo instance | 9599 | open | 0 | 0 | 2020-09-01T06:20:24Z | 2020-09-01T06:20:24Z | MEMBER | Once I've got the Datasette plugin to a state where it's worth building a demo: #3 I can use data from my public https://github-to-sqlite.dogsheep.net/ demo plus the Pocket data subset I use for the demo in https://github.com/dogsheep/pocket-to-sqlite/issues/5 - I could pull in the https://dogsheep-photos.dogsheep.net/ photos data too. | 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/6/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
629473827 | MDU6SXNzdWU2Mjk0NzM4Mjc= | 5 | Set up a demo | 26745575 | open | 0 | 1 | 2020-06-02T19:56:49Z | 2020-09-01T06:18:43Z | NONE | First off, thanks for open sourcing this application! This is a suggestion to increase the amount of people that would make use of it: an example in the readme file would help. Currently, users have to clone the app, install it, authorize through pocket, run a command, an then find out if this application does what they hope it does. Another possibility is to add a file `example-output.db`, containing one (mock) Pocket article. Keep up the good work! | 213286752 | issue | { "url": "https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/5/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
503243784 | MDU6SXNzdWU1MDMyNDM3ODQ= | 3 | Extract images into separate tables | 9599 | open | 0 | 1 | 2019-10-07T05:43:01Z | 2020-09-01T06:17:45Z | MEMBER | As already done with authors. Slightly harder because images do not have a universally unique ID. Also need to figure out what to do about there being columns for both `image` and `images`. <img width="1522" alt="memory__items" src="https://user-images.githubusercontent.com/9599/66287418-9ab20680-e88a-11e9-96bf-6c80d881eff0.png"> | 213286752 | issue | { "url": "https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/3/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
689848827 | MDU6SXNzdWU2ODk4NDg4Mjc= | 6 | ISO timestamps | 9599 | open | 0 | 0 | 2020-09-01T06:16:42Z | 2020-09-01T06:16:42Z | MEMBER | The `time_added`, `time_updated` and `time_read` columns currently store data like this: September 19, 2019 - 00:30:30 UTC Should use ISO instead, e.g. `2020-07-26T01:05:24+00:00` | 213286752 | issue | { "url": "https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/6/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
689839399 | MDU6SXNzdWU2ODk4MzkzOTk= | 4 | Optimize the FTS table | 9599 | closed | 0 | 1 | 2020-09-01T05:58:17Z | 2020-09-01T06:10:08Z | 2020-09-01T06:10:08Z | MEMBER | 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/4/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||||
689800307 | MDU6SXNzdWU2ODk4MDAzMDc= | 1 | Add an index on the timestamp column | 9599 | closed | 0 | 0 | 2020-09-01T04:33:37Z | 2020-09-01T04:49:23Z | 2020-09-01T04:49:23Z | MEMBER | Since default view will likely be ordered by timestamp descending. | 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/1/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
688386219 | MDExOlB1bGxSZXF1ZXN0NDc1NjY1OTg0 | 142 | insert_all(..., alter=True) should work for new columns introduced after the first 100 records | 96218 | closed | 0 | 3 | 2020-08-28T22:22:57Z | 2020-08-30T07:28:23Z | 2020-08-28T22:30:14Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/142 | Closes #139. | 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/142/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
688395275 | MDU6SXNzdWU2ODgzOTUyNzU= | 144 | Run some tests against numpy | 9599 | closed | 0 | 2 | 2020-08-28T22:53:00Z | 2020-08-28T22:57:05Z | 2020-08-28T22:57:04Z | OWNER | Accidentally removed in #143: https://github.com/simonw/sqlite-utils/blob/d7d3f962861ef32c5ead8f514c8756f5b6f7c4a0/.travis.yml#L18-L19 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/144/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
686978131 | MDU6SXNzdWU2ODY5NzgxMzE= | 139 | insert_all(..., alter=True) should work for new columns introduced after the first 100 records | 96218 | closed | 0 | 7 | 2020-08-27T06:25:25Z | 2020-08-28T22:48:51Z | 2020-08-28T22:30:14Z | CONTRIBUTOR | Is there a way to make `.insert_all()` work properly when new columns are introduced outside the first 100 records (with or without the `alter=True` argument)? I'm using `.insert_all()` to bulk insert ~3-4k records at a time and it is common for records to need to introduce new columns. However, if new columns are introduced after the first 100 records, `sqlite_utils` doesn't even raise the `OperationalError: table ... has no column named ...` exception; it just silently drops the extra data and moves on. It took me a while to find this little snippet in the [documentation for `.insert_all()`](https://sqlite-utils.readthedocs.io/en/stable/python-api.html#bulk-inserts) (it's not mentioned under [Adding columns automatically on insert/update](https://sqlite-utils.readthedocs.io/en/stable/python-api.html#bulk-inserts)): > The column types used in the CREATE TABLE statement are automatically derived from the types of data in that first batch of rows. **_Any additional or missing columns in subsequent batches will be ignored._** I tried changing the `batch_size` argument to the total number of records, but it seems only to effect the number of rows that are committed at a time, and has no influence on this problem. Is there a way around this that you would suggest? It seems like it should raise an exception at least. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/139/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
688389933 | MDU6SXNzdWU2ODgzODk5MzM= | 143 | Move to GitHub Actions CI | 9599 | closed | 0 | 1 | 2020-08-28T22:34:11Z | 2020-08-28T22:41:35Z | 2020-08-28T22:41:35Z | OWNER | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/143/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||||
687694947 | MDU6SXNzdWU2ODc2OTQ5NDc= | 954 | Remove old register_output_renderer dict mechanism in Datasette 1.0 | 9599 | open | 0 | 3268330 | 1 | 2020-08-28T04:04:23Z | 2020-08-28T04:56:31Z | OWNER | > Documentation says that the old dictionary mechanism will be deprecated by 1.0: > > https://github.com/simonw/datasette/blob/799ecae94824640bdff21f86997f69844048d5c3/docs/plugin_hooks.rst#L460 _Originally posted by @simonw in https://github.com/simonw/datasette/issues/953#issuecomment-682312494_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/954/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
682005535 | MDU6SXNzdWU2ODIwMDU1MzU= | 945 | datasette install -U for upgrading packages | 9599 | closed | 0 | 5818042 | 1 | 2020-08-19T17:12:04Z | 2020-08-28T04:53:14Z | 2020-08-19T17:20:50Z | OWNER | This will also give Homebrew a way to upgrade Datasette itself without having to wait for the latest packaged version to land in Homebrew core. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/945/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
687681018 | MDU6SXNzdWU2ODc2ODEwMTg= | 953 | register_output_renderer render function should be able to return a Response | 9599 | closed | 0 | 5818042 | 1 | 2020-08-28T03:21:21Z | 2020-08-28T04:53:03Z | 2020-08-28T04:03:01Z | OWNER | That plugin hook was designed before Datasette had a documented Response class. It should optionally be allowed to return a Response in addition to the current custom dictionary. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/953/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
683804172 | MDU6SXNzdWU2ODM4MDQxNzI= | 134 | --load-extension option for sqlite-utils query | 9599 | closed | 0 | 4 | 2020-08-21T20:12:42Z | 2020-08-21T21:06:26Z | 2020-08-21T20:54:19Z | OWNER | I got this error: ``` % sqlite-utils calands.db 'create table superunits_with_maps_view_concrete as select * from superunits_with_maps_view' Traceback (most recent call last): ... cursor = db.conn.execute(sql, dict(param)) sqlite3.OperationalError: no such function: AsGeoJSON ``` A `--load-extension=/usr/local/lib/mod_spatialite.dylib` option (imitating the same option for Datasette) would help. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/134/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
671763164 | MDU6SXNzdWU2NzE3NjMxNjQ= | 915 | Refactor TableView class so things like datasette-graphql can reuse the logic | 9599 | closed | 0 | 3 | 2020-08-03T03:13:33Z | 2020-08-18T22:28:37Z | 2020-08-18T22:28:37Z | OWNER | _Originally posted by @simonw in https://github.com/simonw/datasette-graphql/issues/2#issuecomment-667780040_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/915/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
647095487 | MDU6SXNzdWU2NDcwOTU0ODc= | 873 | "datasette -p 0 --root" gives the wrong URL | 9599 | open | 0 | 14 | 2020-06-29T04:03:06Z | 2020-08-18T17:26:10Z | OWNER | ``` $ datasette -p 0 --root http://127.0.0.1:0/-/auth-token?token=2d498c... ``` The port is incorrect. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/873/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
681086659 | MDU6SXNzdWU2ODEwODY2NTk= | 47 | emojis command | 9599 | closed | 0 | 1 | 2020-08-18T14:26:26Z | 2020-08-18T14:52:13Z | 2020-08-18T14:52:13Z | MEMBER | For fun - it can import https://api.github.com/emojis - maybe with an option to fetch the binary representations in addition to the URLs. | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/47/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
613777056 | MDU6SXNzdWU2MTM3NzcwNTY= | 39 | issues foreign key to repo isn't working | 9599 | closed | 0 | 1 | 2020-05-07T05:11:48Z | 2020-08-18T14:24:46Z | 2020-08-18T14:23:56Z | MEMBER | https://github-to-sqlite.dogsheep.net/github/issues?_facet=repo <img width="725" alt="github__issues__2_303_rows_where_sorted_by_updated_at_descending" src="https://user-images.githubusercontent.com/9599/81256810-723be780-8fe6-11ea-9e1f-f37d40f4d3ae.png"> If the foreign key was working those would be repository names. From the schema at the bottom of the page: ``` [repo] TEXT, ``` That's the wrong type and not a foreign key. | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/39/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
679809281 | MDExOlB1bGxSZXF1ZXN0NDY4NDg0MDMx | 941 | Run CI on GitHub Actions, not Travis | 9599 | closed | 0 | 1 | 2020-08-16T19:13:39Z | 2020-08-18T05:09:36Z | 2020-08-18T05:09:35Z | OWNER | simonw/datasette/pulls/941 | Refs #940 | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/941/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
671056788 | MDU6SXNzdWU2NzEwNTY3ODg= | 914 | "Object of type bytes is not JSON serializable" for _nl=on | 9599 | closed | 0 | 1 | 2020-08-01T17:43:10Z | 2020-08-16T21:10:27Z | 2020-08-16T18:26:59Z | OWNER | https://latest.datasette.io/fixtures/binary_data.json?_sort_desc=data&_shape=array returns this: ```json [ { "rowid": 1, "data": "this is binary data" } ] ``` But adding `&_nl=on` returns this: https://latest.datasette.io/fixtures/binary_data.json?_sort_desc=data&_shape=array&_nl=on ```json { "ok": false, "error": "Object of type bytes is not JSON serializable", "status": 500, "title": null } ``` I found this error by running `wget -r 127.0.0.1:8001` against my local `fixtures.db`. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/914/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
679779797 | MDU6SXNzdWU2Nzk3Nzk3OTc= | 939 | extra_ plugin hooks should take the same arguments | 9599 | closed | 0 | 6 | 2020-08-16T16:04:54Z | 2020-08-16T18:25:05Z | 2020-08-16T16:50:29Z | OWNER | - [x] `extra_css_urls(template, database, table, datasette)` - [x] `extra_js_urls(template, database, table, datasette)` - [x] `extra_body_script(template, database, table, view_name, datasette)` - [x] `extra_template_vars(template, database, table, view_name, request, datasette)` _Originally posted by @simonw in https://github.com/simonw/datasette/issues/938#issuecomment-674544691_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/939/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
679660778 | MDExOlB1bGxSZXF1ZXN0NDY4Mzc3MjEy | 937 | Docs now live at docs.datasette.io | 9599 | closed | 0 | 0 | 2020-08-15T23:53:52Z | 2020-08-15T23:57:06Z | 2020-08-15T23:57:05Z | OWNER | simonw/datasette/pulls/937 | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/937/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
679646710 | MDU6SXNzdWU2Nzk2NDY3MTA= | 935 | db.execute_write_fn(create_tables, block=True) hangs a thread if connection fails | 9599 | closed | 0 | 3 | 2020-08-15T21:49:17Z | 2020-08-15T22:35:33Z | 2020-08-15T22:35:33Z | OWNER | Discovered in https://github.com/simonw/latest-datasette-with-all-plugins/issues/3#issuecomment-674449757 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/935/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
679650632 | MDExOlB1bGxSZXF1ZXN0NDY4MzcwNjU4 | 936 | Don't hang in db.execute_write_fn() if connection fails | 9599 | closed | 0 | 2 | 2020-08-15T22:20:12Z | 2020-08-15T22:35:33Z | 2020-08-15T22:35:32Z | OWNER | simonw/datasette/pulls/936 | Refs #935 | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/936/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
679637501 | MDU6SXNzdWU2Nzk2Mzc1MDE= | 934 | --get doesn't fully invoke the startup routine | 9599 | closed | 0 | 0 | 2020-08-15T20:30:25Z | 2020-08-15T20:53:49Z | 2020-08-15T20:53:49Z | OWNER | https://github.com/simonw/datasette/blob/7702ea602188899ee9b0446a874a6a9b546b564d/datasette/cli.py#L417-L433 Spotted this working on https://github.com/simonw/latest-datasette-with-all-plugins/issues/3 - I'd like to be able to use `datasette --get /` as a sanity checking test, but that doesn't work if the init hooks aren't fully executed. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/934/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
677926613 | MDU6SXNzdWU2Nzc5MjY2MTM= | 931 | Docker container is no longer being pushed (it's stuck on 0.45) | 9599 | closed | 0 | 7 | 2020-08-12T19:33:03Z | 2020-08-12T21:36:20Z | 2020-08-12T21:36:20Z | OWNER | e.g. https://travis-ci.org/github/simonw/datasette/jobs/717123725 Here's how it broke: ``` --2020-08-12 03:08:17-- https://www.gaia-gis.it/gaia-sins/freexl-1.0.5.tar.gz Resolving www.gaia-gis.it (www.gaia-gis.it)... 212.83.162.51 Connecting to www.gaia-gis.it (www.gaia-gis.it)|212.83.162.51|:443... connected. HTTP request sent, awaiting response... 404 Not Found 2020-08-12 03:08:18 ERROR 404: Not Found. The command '/bin/sh -c wget "https://www.gaia-gis.it/gaia-sins/freexl-1.0.5.tar.gz" && tar zxf freexl-1.0.5.tar.gz && cd freexl-1.0.5 && ./configure && make && make install' returned a non-zero code: 8 The command "docker build -f Dockerfile -t $REPO:$TRAVIS_TAG ." exited with 8. 0.07s$ docker tag $REPO:$TRAVIS_TAG $REPO:latest Error response from daemon: No such image: [secure]/datasette:0.47.1 The command "docker tag $REPO:$TRAVIS_TAG $REPO:latest" exited with 1. 0.08s$ docker push $REPO The push refers to repository [docker.io/[secure]/datasette] An image does not exist locally with the tag: [secure]/datasette The command "docker push $REPO" exited with 1. cache.2 store build cache ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/931/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
677839979 | MDU6SXNzdWU2Nzc4Mzk5Nzk= | 133 | Release a sdist to PyPI | 9599 | closed | 0 | 1 | 2020-08-12T16:55:09Z | 2020-08-12T17:05:06Z | 2020-08-12T17:05:06Z | OWNER | https://pypi.org/project/sqlite-utils/#files currently just has a wheel. I need this to package for homebrew: https://github.com/simonw/homebrew-datasette/issues/10 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/133/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
677326155 | MDU6SXNzdWU2NzczMjYxNTU= | 930 | Datasette sdist is missing templates (hence broken when installing from Homebrew) | 9599 | closed | 0 | 6 | 2020-08-12T02:20:16Z | 2020-08-12T03:30:59Z | 2020-08-12T03:30:59Z | OWNER | Pretty nasty bug this: I'm getting 500 errors for all pages that try to render a template after installing the newly released Datasette 0.47 - both from `pip install` and via Homebrew. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/930/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
677250834 | MDU6SXNzdWU2NzcyNTA4MzQ= | 926 | datasette fixtures.db --get "/fixtures.json" | 9599 | closed | 0 | 2 | 2020-08-11T22:55:36Z | 2020-08-12T00:26:17Z | 2020-08-12T00:24:42Z | OWNER | I can expose ALL of Datasette's functionality on the command-line (without even running a web server) by adding `--get` and `--post` options to `datasette serve`. datasette fixtures.db --get "/fixtures.json" This would instantiate the Datasette ASGI app, run a fake request for `/fixtures.json` through it, dump the results out to standard output and quit. A `--post` option could do the same for a POST request. Treating that as a stretch goal for the moment. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/926/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
677265716 | MDExOlB1bGxSZXF1ZXN0NDY2NDEwNzU1 | 927 | 'datasette --get' option, refs #926 | 9599 | closed | 0 | 5 | 2020-08-11T23:31:52Z | 2020-08-12T00:24:42Z | 2020-08-12T00:24:41Z | OWNER | simonw/datasette/pulls/927 | Refs #926, #898 | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/927/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
677037043 | MDU6SXNzdWU2NzcwMzcwNDM= | 923 | Add homebrew installation to documentation | 9599 | closed | 0 | 5 | 2020-08-11T16:54:31Z | 2020-08-11T22:53:07Z | 2020-08-11T22:52:46Z | OWNER | > ``` > $ brew tap simonw/datasette > $ brew install simonw/datasette/datasette > $ datasette --version > datasette, version 0.46 > ``` _Originally posted by @simonw in https://github.com/simonw/datasette/issues/335#issuecomment-672088880_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/923/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
677227912 | MDU6SXNzdWU2NzcyMjc5MTI= | 925 | "datasette install" and "datasette uninstall" commands | 9599 | closed | 0 | 3 | 2020-08-11T22:04:32Z | 2020-08-11T22:34:37Z | 2020-08-11T22:32:12Z | OWNER | When installing Datasette plugins it's crucial that they end up in the same virtual environment as Datasette itself. It's not necessarily obvious how to do this, especially if you install Datasette via pipx or homebrew. Solution: `datasette install datasette-vega` and `datasette uninstall datasette-vega` commands that know how to install to the correct place - a very thin wrapper around `pip install`. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/925/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
339505204 | MDU6SXNzdWUzMzk1MDUyMDQ= | 335 | Package datasette for installation using homebrew | 9599 | closed | 0 | 12 | 2018-07-09T15:45:03Z | 2020-08-11T16:54:06Z | 2020-08-11T16:54:06Z | OWNER | https://docs.brew.sh/Python-for-Formula-Authors describes how. > Applications should be installed into a Python virtualenv environment rooted in libexec. This prevents the app’s Python modules from contaminating the system site-packages and vice versa. It recommends using https://github.com/tdsmith/homebrew-pypi-poet | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/335/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
675839512 | MDU6SXNzdWU2NzU4Mzk1MTI= | 132 | Features for enabling and disabling WAL mode | 9599 | closed | 0 | 5 | 2020-08-10T03:25:44Z | 2020-08-10T18:59:35Z | 2020-08-10T18:59:35Z | OWNER | I finally figured out how to enable WAL - turns out it's a property of the database file itself: https://github.com/simonw/til/blob/master/sqlite/enabling-wal-mode.md | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/132/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
675724951 | MDU6SXNzdWU2NzU3MjQ5NTE= | 918 | Security issue: read-only canned queries leak CSRF token in URL | 9599 | closed | 0 | 4 | 2020-08-09T16:03:01Z | 2020-08-09T16:56:48Z | 2020-08-09T16:11:59Z | OWNER | The HTML form for a read-only canned query includes the hidden CSRF token field added in #798 for writable canned queries (#698). This means that submitting those read-only forms exposes the CSRF token in the URL - for example on https://latest.datasette.io/fixtures/neighborhood_search submitting the form took me to: https://latest.datasette.io/fixtures/neighborhood_search?text=down&csrftoken=IlFubnoxVVpLU1NGT3NMVUoi.HbOPd2YH_epQmp8f_aAt0s-MxtU This token could potentially leak to an attacker if the resulting page has a link to an external site on it and the user clicks the link, since the token would be exposed in the referral logs. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/918/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
675727366 | MDU6SXNzdWU2NzU3MjczNjY= | 919 | Travis should not build the master branch, only the main branch | 9599 | closed | 0 | 3 | 2020-08-09T16:18:25Z | 2020-08-09T16:26:18Z | 2020-08-09T16:19:37Z | OWNER | Caused by #849 - since we are mirroring the two branches (to ensure old links to `master` keep working) Travis is building both. The following in `.travis.yml` should fix that: ``` branches: except: - master ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/919/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
675594325 | MDU6SXNzdWU2NzU1OTQzMjU= | 917 | Idea: "datasette publish" option for "only if the data has changed | 9599 | open | 0 | 0 | 2020-08-08T21:58:27Z | 2020-08-08T21:58:27Z | OWNER | This is a pattern I often find myself needing. I usually implement this in GitHub Actions like this: https://github.com/simonw/covid-19-datasette/blob/efa01c39abc832b8641fc2a92840cc3acae2fb08/.github/workflows/scheduled.yml#L52-L63 ```yaml - name: Set variables to decide if we should deploy id: decide_variables run: |- echo "##[set-output name=latest;]$(datasette inspect covid.db | jq '.covid.hash' -r)" echo "##[set-output name=deployed;]$(curl -s https://covid-19.datasettes.com/-/databases.json | jq '.[0].hash' -r)" - name: Set up Cloud Run if: github.event_name == 'workflow_dispatch' || steps.decide_variables.outputs.latest != steps.decide_variables.outputs.deployed uses: GoogleCloudPlatform/github-actions/setup-gcloud@master ``` This is pretty fiddly. It might be good for `datasette publish` to grow a helper option that does effectively this - hashes the databases (and the `metadata.json`) and compares them to the deployed version. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/917/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
673602857 | MDU6SXNzdWU2NzM2MDI4NTc= | 9 | Define a view that displays photos correctly | 9599 | open | 0 | 0 | 2020-08-05T14:53:39Z | 2020-08-05T14:53:39Z | MEMBER | The `photos` table stores data like this: id | createdAt | source | prefix | suffix | width | height | visibility | created ▲ | user -- | -- | -- | -- | -- | -- | -- | -- | -- | -- 5e12c9708506bc000840262a | January 06, 2020 - 05:45:20 UTC | Swarm for iOS 1 | https://fastly.4sqi.net/img/general/ | /15889193_AXxGk4I1nbzUZuyYqObgbXdJNyEHiwj6AUDq0tPZWtw.jpg | 1920 | 1440 | public | 2020-01-06T05:45:20 | 15889193 The photo URL can be derived from those pieces - define a SQL view which does that (using `datasette-json-html` to display the pictures) | 205429375 | issue | { "url": "https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/9/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
671130371 | MDU6SXNzdWU2NzExMzAzNzE= | 130 | Support tokenize option for FTS | 9599 | closed | 0 | 3 | 2020-08-01T19:27:22Z | 2020-08-01T20:51:28Z | 2020-08-01T20:51:14Z | OWNER | FTS5 supports things like porter stemming using a `tokenize=` option: https://www.sqlite.org/fts5.html#tokenizers Something like this in code: ``` CREATE VIRTUAL TABLE [{table}_fts] USING {fts_version} ( {columns}, tokenize='porter', content=[{table}] ); ``` I tried this out just now and it worked exactly as expected. So... `db[table].enable_fts(...) should accept a 'tokenize=` argument, and `sqlite-utils enable-fts ...` should support a `--tokenize` option. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/130/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
660827546 | MDU6SXNzdWU2NjA4Mjc1NDY= | 899 | How to setup a request limit per user | 133845 | closed | 0 | 1 | 2020-07-19T13:08:25Z | 2020-07-31T23:54:42Z | 2020-07-31T23:54:42Z | NONE | Hello, Until now I'm using datasette without any authentication system but I would like to setup a configuration or limiting the number of requests per user (eventually by IP or with a cookie mechanism) and eventually allowing me to ban specific users/IPs. Is there a plugin available for this use case ? If not what are your insights regarding this UC ? Should I write a plugin ? Should I deploy datasette behind a reverse proxy to manage this ? | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/899/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
662322234 | MDExOlB1bGxSZXF1ZXN0NDUzODkwMjky | 901 | Use None as a default arg | 56323389 | closed | 0 | 1 | 2020-07-20T22:18:38Z | 2020-07-31T18:42:39Z | 2020-07-31T18:42:39Z | CONTRIBUTOR | simonw/datasette/pulls/901 | When passing a mutable value as a default argument in a function, the default argument is mutated anytime that value is mutated. This poses a bug risk. Instead, use None as a default and assign the mutable value inside the function. | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/901/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
668064778 | MDU6SXNzdWU2NjgwNjQ3Nzg= | 912 | Add "publishing to Vercel" to the publish docs | 9599 | closed | 0 | 0 | 2020-07-29T18:50:58Z | 2020-07-31T17:06:35Z | 2020-07-31T17:06:35Z | OWNER | https://datasette.readthedocs.io/en/0.45/publish.html#datasette-publish currently only lists Cloud Run, Heroku and Fly. It should list Vercel too. (I should probably rename `datasette-publish-now` to `datasette-publish-vercel`) | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/912/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
665802405 | MDU6SXNzdWU2NjU4MDI0MDU= | 124 | sqlite-utils query should support named parameters | 9599 | closed | 0 | 1 | 2020-07-26T15:25:10Z | 2020-07-30T22:57:51Z | 2020-07-27T03:53:58Z | OWNER | To help out with escaping - so you can run this: sqlite-utils query "insert into foo (blah) values (:blah)" --param blah `something here` | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/124/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
668308777 | MDU6SXNzdWU2NjgzMDg3Nzc= | 129 | "insert-files --sqlar" for creating SQLite archives | 9599 | closed | 0 | 2 | 2020-07-30T02:28:29Z | 2020-07-30T22:41:01Z | 2020-07-30T22:40:55Z | OWNER | A `--sqlar` option could cause `insert-files` to behave in the same way as SQLite's own sqlar mechanism. https://www.sqlite.org/sqlar.html and https://sqlite.org/sqlar/doc/trunk/README.md | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/129/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
666040390 | MDU6SXNzdWU2NjYwNDAzOTA= | 127 | Ability to insert files piped to insert-files stdin | 9599 | closed | 0 | 3 | 2020-07-27T07:09:33Z | 2020-07-30T03:08:52Z | 2020-07-30T03:08:18Z | OWNER | > Inserting files by piping them in should work - but since a filename cannot be derived this will need a `--name blah.gif` option. > > cat blah.gif | sqlite-utils insert-files files.db files - --name=blah.gif > _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/122#issuecomment-664128071_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/127/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
666639051 | MDU6SXNzdWU2NjY2MzkwNTE= | 128 | Support UUID and memoryview types | 9599 | closed | 0 | 1 | 2020-07-27T23:08:34Z | 2020-07-30T01:10:43Z | 2020-07-30T01:10:43Z | OWNER | `psycopg2` can return data from PostgreSQL as `uuid.UUID` or `memoryview` objects. These should to be supported by `sqlite-utils` - mainly for https://github.com/simonw/db-to-sqlite | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/128/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
667467128 | MDU6SXNzdWU2Njc0NjcxMjg= | 909 | AsgiFileDownload: filename not correctly passed | 9599 | closed | 0 | 2 | 2020-07-29T00:41:43Z | 2020-07-30T00:56:17Z | 2020-07-29T21:34:48Z | OWNER | https://github.com/simonw/datasette/blob/3c33b421320c0be81a625ca7307b2e4416a9ed5b/datasette/utils/asgi.py#L396-L405 `self.filename` should be passed to `asgi_send_file()` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/909/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
667840539 | MDExOlB1bGxSZXF1ZXN0NDU4NDM1NTky | 910 | Update pytest requirement from <5.5.0,>=5.2.2 to >=5.2.2,<6.1.0 | 27856297 | closed | 0 | 1 | 2020-07-29T13:21:17Z | 2020-07-29T21:26:05Z | 2020-07-29T21:26:04Z | CONTRIBUTOR | simonw/datasette/pulls/910 | Updates the requirements on [pytest](https://github.com/pytest-dev/pytest) to permit the latest version. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/pytest-dev/pytest/releases">pytest's releases</a>.</em></p> <blockquote> <h2>6.0.0</h2> <h1>pytest 6.0.0 (2020-07-28)</h1> <p>(<strong>Please see the full set of changes for this release also in the 6.0.0rc1 notes below</strong>)</p> <h2>Breaking Changes</h2> <ul> <li> <p><a href="https://github-redirect.dependabot.com/pytest-dev/pytest/issues/5584">#5584</a>: <strong>PytestDeprecationWarning are now errors by default.</strong></p> <p>Following our plan to remove deprecated features with as little disruption as possible, all warnings of type <code>PytestDeprecationWarning</code> now generate errors instead of warning messages.</p> <p><strong>The affected features will be effectively removed in pytest 6.1</strong>, so please consult the <a href="https://docs.pytest.org/en/latest/deprecations.html">Deprecations and Removals</a> section in the docs for directions on how to update existing code.</p> <p>In the pytest <code>6.0.X</code> series, it is possible to change the errors back into warnings as a stopgap measure by adding this to your <code>pytest.ini</code> file:</p> <pre lang="{.sourceCode" data-meta=".ini}"><code>[pytest] filterwarnings = ignore::pytest.PytestDeprecationWarning </code></pre> <p>But this will stop working when pytest <code>6.1</code> is released.</p> <p><strong>If you have concerns</strong> about the removal of a specific feature, please add a comment to <a href="https://github-redirect.dependabot.com/pytest-dev/pytest/issues/5584">#5584</a>.</p> </li> <li> <p><a href="https://github-redirect.dependabot.com/pytest-dev/pytest/issues/7472">#7472</a>: The <code>exec_()</code> and <code>is_true()</code> methods of <code>_pytest._code.Frame</code> have been removed.</p> </li> </ul> <h2>Features</h2> <ul> <li><a href="https://github-redirect.dependabot.com/pytest-dev/pytest/issues/7464">#7464</a>: Added… | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/910/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
668064026 | MDU6SXNzdWU2NjgwNjQwMjY= | 911 | Rethink the --name option to "datasette publish" | 9599 | open | 0 | 3268330 | 0 | 2020-07-29T18:49:49Z | 2020-07-29T18:49:49Z | OWNER | `--name` works inconsistently across the different publish providers - on Cloud Run you should use `--service` instead for example. Need to review it across all of them and either remove it or clarify what it does. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/911/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
665700495 | MDU6SXNzdWU2NjU3MDA0OTU= | 122 | CLI utility for inserting binary files into SQLite | 9599 | closed | 0 | 10 | 2020-07-26T03:27:39Z | 2020-07-27T07:10:41Z | 2020-07-27T07:09:03Z | OWNER | SQLite BLOB columns can store entire binary files. The challenge is inserting them, since they don't neatly fit into JSON objects. It would be great if the `sqlite-utils` CLI had a trick for helping with this. Inspired by https://github.com/simonw/datasette-media/issues/14 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/122/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
665819048 | MDU6SXNzdWU2NjU4MTkwNDg= | 126 | Ability to insert binary data on the CLI using JSON | 9599 | closed | 0 | 2 | 2020-07-26T16:54:14Z | 2020-07-27T04:00:33Z | 2020-07-27T03:59:45Z | OWNER | > I could solve round tripping (at least a bit) by allowing insert to be run with a flag that says "these columns are base64 encoded, store the decoded data in a BLOB". > > That would solve inserting binary data using JSON too. _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/125#issuecomment-664012247_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/126/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
665817570 | MDU6SXNzdWU2NjU4MTc1NzA= | 125 | Output binary columns in "sqlite-utils query" JSON | 9599 | closed | 0 | 4 | 2020-07-26T16:47:02Z | 2020-07-27T00:49:41Z | 2020-07-27T00:48:45Z | OWNER | You get an error if you try to run a query that returns data from a BLOB. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/125/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
665701216 | MDU6SXNzdWU2NjU3MDEyMTY= | 123 | --raw option for outputting binary content | 9599 | closed | 0 | 0 | 2020-07-26T03:35:39Z | 2020-07-26T16:44:11Z | 2020-07-26T16:44:11Z | OWNER | Related to the `insert-files` work in #122 - it should be easy to get binary data back out of the database again. One way to do that could be: sqlite-utils files.db "select content from files where key = 'foo.jpg'" --raw The `--raw` option would cause just the contents of the first column to be output directly to stdout. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/123/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
665400224 | MDU6SXNzdWU2NjU0MDAyMjQ= | 906 | "allow": true for anyone, "allow": false for nobody | 9599 | closed | 0 | 5607421 | 3 | 2020-07-24T20:28:10Z | 2020-07-25T00:07:10Z | 2020-07-25T00:05:04Z | OWNER | The "allow" syntax described at https://datasette.readthedocs.io/en/0.45/authentication.html#defining-permissions-with-allow-blocks currently says this: > An allow block can specify "no-one is allowed to do this" using an empty `{}`: > > ``` > { > "allow": {} > } > ``` `"allow": null` allows all access, though this isn't documented (it should be though). These are not very intuitive. How about also supporting `"allow": true` for "allow anyone" and `"allow": false` for "allow nobody"? | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/906/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
665407663 | MDU6SXNzdWU2NjU0MDc2NjM= | 908 | Interactive debugging tool for "allow" blocks | 9599 | closed | 0 | 5607421 | 3 | 2020-07-24T20:43:44Z | 2020-07-25T00:06:15Z | 2020-07-24T22:56:52Z | OWNER | > It might be good to have a little interactive tool which helps debug these things, since there are quite a few edge-cases and the damage caused if people use them incorrectly is substantial. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/907#issuecomment-663726146_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/908/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
665403403 | MDU6SXNzdWU2NjU0MDM0MDM= | 907 | Allow documentation doesn't explain what happens with multiple allow keys | 9599 | closed | 0 | 5607421 | 2 | 2020-07-24T20:34:40Z | 2020-07-24T22:53:07Z | 2020-07-24T22:53:07Z | OWNER | Documentation here: https://datasette.readthedocs.io/en/0.45/authentication.html#defining-permissions-with-allow-blocks Doesn't explain that with the following "allow" block: ```json { "allow": { "id": "simonw", "role": "staff" } } ``` The rule will match if EITHER the id is simonw OR the role includes staff. The tests are missing this case too: https://github.com/simonw/datasette/blob/028f193dd6233fa116262ab4b07b13df7dcec9be/tests/test_utils.py#L504 Related to #906 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/907/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
442327592 | MDU6SXNzdWU0NDIzMjc1OTI= | 456 | Installing installs the tests package | 7725188 | closed | 0 | 3 | 2019-05-09T16:35:16Z | 2020-07-24T20:39:54Z | 2020-07-24T20:39:54Z | CONTRIBUTOR | Because `setup.py` uses `find_packages` and `tests` is on the top-level, `pip install datasette` will install a top-level package called `tests`, which is probably not desired behavior. The offending line is here: https://github.com/simonw/datasette/blob/bfa2ae0d16d39bb82dbe4da4f3fdc3c7f6257418/setup.py#L40 And only `pip uninstall datasette` with a conflicting package would warn you by default; apparently another package had the same problem, which is why I get this message when uninstalling: ``` $ pip uninstall datasette Uninstalling datasette-0.27: Would remove: /usr/local/bin/datasette /usr/local/lib/python3.7/site-packages/datasette-0.27.dist-info/* /usr/local/lib/python3.7/site-packages/datasette/* /usr/local/lib/python3.7/site-packages/tests/* Would not remove (might be manually added): [ .. snip .. ] Proceed (y/n)? ``` This should be a relatively simple fix, and I could drop a PR if desired! Cheers | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/456/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
662439034 | MDExOlB1bGxSZXF1ZXN0NDUzOTk1MTc5 | 902 | Don't install tests package | 32467826 | closed | 0 | 2 | 2020-07-21T01:08:50Z | 2020-07-24T20:39:54Z | 2020-07-24T20:39:54Z | CONTRIBUTOR | simonw/datasette/pulls/902 | The `exclude` argument to `find_packages` needs an iterable of package names. Fixes: #456 | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/902/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
664793260 | MDU6SXNzdWU2NjQ3OTMyNjA= | 2 | Yak shave | 145425 | open | 0 | 0 | 2020-07-23T22:04:18Z | 2020-07-23T22:04:18Z | NONE | Just a quick note... The 23andme data is not exactly your genome, but a SNP chip of your genome. It's "some of your genotypes." Or about 0.1% of your genome. Nice work in any case! It deserves to be liberated!!!!! | 209590345 | issue | { "url": "https://api.github.com/repos/dogsheep/genome-to-sqlite/issues/2/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
663976976 | MDU6SXNzdWU2NjM5NzY5NzY= | 48 | Add a table of contents to the README | 9599 | closed | 0 | 3 | 2020-07-22T18:54:33Z | 2020-07-23T17:46:07Z | 2020-07-22T19:03:02Z | MEMBER | Using https://github.com/jonschlinkert/markdown-toc | 206156866 | issue | { "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/48/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
663317875 | MDU6SXNzdWU2NjMzMTc4NzU= | 905 | /database.db download should include content-length header | 9599 | closed | 0 | 2 | 2020-07-21T21:23:48Z | 2020-07-22T04:59:46Z | 2020-07-22T04:52:45Z | OWNER | I can do this by modifying this function: https://github.com/simonw/datasette/blob/02dc6298bdbfb1d63e0d2a39ff597b5fcc60e06b/datasette/utils/asgi.py#L248-L270 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/905/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
442402832 | MDExOlB1bGxSZXF1ZXN0Mjc3NTI0MDcy | 458 | setup: add tests to package exclusion | 7725188 | closed | 0 | 1 | 2019-05-09T19:47:21Z | 2020-07-21T01:14:42Z | 2019-05-10T01:54:51Z | CONTRIBUTOR | simonw/datasette/pulls/458 | This PR fixes #456 by adding `tests` to the package exclusion list. Cheers | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/458/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
660355904 | MDU6SXNzdWU2NjAzNTU5MDQ= | 43 | github-to-sqlite tags command for fetching tags | 9599 | closed | 0 | 4 | 2020-07-18T20:14:12Z | 2020-07-18T23:05:56Z | 2020-07-18T21:52:15Z | MEMBER | Fetches paginated data from https://api.github.com/repos/simonw/datasette/tags | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/43/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed |