github
id | node_id | number | title | user | state | locked | assignee | milestone | comments | created_at | updated_at | closed_at | author_association | pull_request | body | repo | type | active_lock_reason | performed_via_github_app | reactions | draft | state_reason |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
871304967 | MDU6SXNzdWU4NzEzMDQ5Njc= | 1315 | settings.json should be picked up by "datasette publish cloudrun" | 9599 | open | 0 | 0 | 2021-04-29T18:16:41Z | 2021-04-29T18:16:41Z | OWNER | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1315/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||||
923602693 | MDU6SXNzdWU5MjM2MDI2OTM= | 276 | support small help flag -h | 601708 | closed | 0 | 0 | 2021-06-17T07:59:31Z | 2021-06-18T14:56:59Z | 2021-06-18T14:56:59Z | CONTRIBUTOR | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/276/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||||
923697888 | MDU6SXNzdWU5MjM2OTc4ODg= | 278 | Support db as first parameter before subcommand, or as environment variable | 601708 | closed | 0 | 3 | 2021-06-17T09:26:29Z | 2021-06-20T22:39:57Z | 2021-06-18T15:43:19Z | CONTRIBUTOR | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/278/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||||
927385540 | MDU6SXNzdWU5MjczODU1NDA= | 8 | any guidance / experience on imessage-to-sqlite ? | 2675621 | open | 0 | 0 | 2021-06-22T15:46:16Z | 2021-06-22T15:46:16Z | NONE | 214746582 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep.github.io/issues/8/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||||
927766296 | MDU6SXNzdWU5Mjc3NjYyOTY= | 291 | Adopt flake8 | 9599 | closed | 0 | 2 | 2021-06-23T01:19:37Z | 2021-06-24T17:50:27Z | 2021-06-24T17:50:27Z | OWNER | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/291/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||||
931752773 | MDU6SXNzdWU5MzE3NTI3NzM= | 294 | Add a `sqlite-utils memory` example to the README | 9599 | closed | 0 | 0 | 2021-06-28T16:35:59Z | 2021-08-18T21:40:03Z | 2021-08-18T21:40:03Z | OWNER | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/294/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||||
836063389 | MDU6SXNzdWU4MzYwNjMzODk= | 17 | Datetime columns are not properly formatted to be recognizes as datetime | 1234956 | open | 0 | 0 | 2021-03-19T14:33:04Z | 2021-03-19T14:33:04Z | NONE | Currently, the datetimes are formatted in a way that is not recognized by datasette-vega for plotting with a `Date/time` type for the axis. For example, if you have datasette running locally with `datasette-vega` installed and have a database that includes resting heart rate: ``` http://localhost:8001/healthkit/rRestingHeartRate#g.mark=line&g.x_column=startDate&g.x_type=temporal&g.y_column=value&g.y_type=quantitative ``` The plot is blank unless you choose `Label` as the type for the date data. The `startDate` (and `creationDate` and `endDate`) columns appear like: `2019-11-14 18:22:18 -0700` If instead the format for this column is changed slightly: `2019-11-14T18:22:18-07:00` they are recognized as proper dates and the charting works as expected. I have a PR that addresses this issue, will submit shortly. | 197882382 | issue | { "url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/17/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
796736607 | MDU6SXNzdWU3OTY3MzY2MDc= | 56 | Not all quoted statuses get fetched? | 42315895 | closed | 0 | 3 | 2021-01-29T09:48:44Z | 2021-02-03T10:36:36Z | 2021-02-03T10:36:36Z | NONE | ![image](https://user-images.githubusercontent.com/42315895/106259325-5f75dc80-621f-11eb-8311-db8f2fe2a257.png) In my database I have 13300 quote tweets, but eta 3600 have `quoted_status` empty. I fetched some of them using `https://api.twitter.com/1.1/statuses/show.json?id=xx` and they did have ids of quoted tweets. | 206156866 | issue | { "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/56/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1761613778 | I_kwDOBm6k_c5pABfS | 2084 | Support facets for columns that contain timestamps | 19492893 | open | 0 | 0 | 2023-06-17T03:33:54Z | 2023-06-17T03:33:54Z | NONE | Django has this very nice filter for datetime fields - <img width="176" alt="image" src="https://github.com/simonw/datasette/assets/19492893/3c66d7c4-1579-4d30-8f08-89d111f4566e"> It would be nice to have something similar to facet by a field that contains a timestamp in datasette too - Which doesn't seem to do anything with timestamps right now... <img width="283" alt="image" src="https://github.com/simonw/datasette/assets/19492893/069083e4-13f5-4b28-9473-a7b9d48839ea"> | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/2084/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
831751367 | MDU6SXNzdWU4MzE3NTEzNjc= | 246 | Escaping FTS search strings | 16001974 | closed | 0 | 4 | 2021-03-15T12:15:09Z | 2021-08-18T18:57:13Z | 2021-08-18T18:43:12Z | CONTRIBUTOR | Thanks for the excellent library, it's very nice to use! I've been building some in memory search functionality for a data annotation tool i'm making, and I got tripped up a little bit with escaping the full text search queries. First I tried using `db.quote(q)`, which doesn't work, because sqlite FTS has it's own (separate)[ query syntax](https://www2.sqlite.org/fts5.html#full_text_query_syntax). You can see this happening here also: http://search-24ways.herokuapp.com/24ways-f8f455f/articles?_search=acces%2A I got around this by aggressively escaping quotes inside the query string like this: ```python quoted = q.replace('"', '""') quoted = f'"{quoted}"' print(quoted) results = db["data"].search(quoted, columns=["id"]) return [x["id"] for x in results] ``` This works in the sense it doesn't crash, but it also removes access to the search query syntax. Given the well specified definition, it might be possible for sqlite-utils to provide a `db.quote_query(q)` which would intelligently escape a query whilst leaving the syntax intact. This would be very nice! | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/246/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1943259395 | I_kwDOEhK-wc5z08kD | 16 | time data '2014-11-21T11:44:12.000Z' does not match format '%Y%m%dT%H%M%SZ' | 3746270 | open | 0 | 0 | 2023-10-14T13:24:39Z | 2023-10-14T13:24:39Z | NONE | ``` evernote-to-sqlite enex evernote.db ./我的笔记.enex Importing from ENEX [#####-------------------------------] 14% Traceback (most recent call last): File "/usr/local/bin/evernote-to-sqlite", line 8, in <module> sys.exit(cli()) ^^^^^ File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1157, in __call__ return self.main(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1688, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, **ctx.params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/click/core.py", line 783, in invoke return __callback(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/evernote_to_sqlite/cli.py", line 31, in enex save_note(db, note) File "/usr/local/lib/python3.11/site-packages/evernote_to_sqlite/utils.py", line 46, in save_note "created": convert_datetime(created), ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/evernote_to_sqlite/utils.py", line 111, in convert_datetime return datetime.datetime.strptime(s, "%Y%m%dT%H%M%SZ").isoformat() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/_strptime.py", line 568, in _strptime_datetime tt, fraction, gmtoff_fraction = _strptime(data_string, format) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.1… | 303218369 | issue | { "url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/16/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
273660425 | MDU6SXNzdWUyNzM2NjA0MjU= | 84 | datasette package --metadata does not work with a relative path | 9599 | closed | 0 | 0 | 2017-11-14T04:00:50Z | 2017-11-15T05:18:35Z | 2017-11-15T05:18:35Z | OWNER | $ datasette package ~/parlgov-db/parlgov.db --metadata=~/parlgov-db/parlgov.json Usage: datasette package [OPTIONS] FILES... Error: Invalid value for "-m" / "--metadata": Could not open file: ~/parlgov-db/parlgov.json: No such file or directory simonw-07542:~ simonw$ cd ~/parlgov-db/ simonw-07542:parlgov-db simonw$ datasette package ~/parlgov-db/parlgov.db --metadata=parlgov.json Sending build context to Docker daemon 4.46MB Step 1/7 : FROM python:3 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/84/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
488833975 | MDU6SXNzdWU0ODg4MzM5NzU= | 3 | Command for running a search and saving tweets for that search | 9599 | closed | 0 | 6 | 2019-09-03T21:29:56Z | 2019-11-04T05:31:56Z | 2019-11-04T05:31:16Z | MEMBER | $ twitter-to-sqlite search dogsheep | 206156866 | issue | { "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/3/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
267513424 | MDU6SXNzdWUyNjc1MTM0MjQ= | 1 | Addressable pages for every row in a table | 9599 | closed | 0 | 2857392 | 6 | 2017-10-23T00:44:16Z | 2017-10-24T14:11:04Z | 2017-10-24T14:11:03Z | OWNER | /database-name-7sha256/table-name/compound-pk /database-name-7sha256/table-name/compound-pk.json Tricky part will be figuring out what the private key is - especially since it could be a compound primary key and it might involve different data types. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
718540751 | MDU6SXNzdWU3MTg1NDA3NTE= | 1012 | For 1.0 update trove classifier in setup.py | 9599 | open | 0 | 3268330 | 5 | 2020-10-10T05:52:08Z | 2021-11-16T13:18:36Z | OWNER | Development Status :: 5 - Production/Stable | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1012/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
275939188 | MDU6SXNzdWUyNzU5MzkxODg= | 143 | Mechanism for "suggested visualizations" | 9599 | closed | 0 | 1 | 2017-11-22T04:10:25Z | 2018-07-10T17:48:34Z | 2018-07-10T17:48:34Z | OWNER | Each visualization should have a way of deciding if it might be appropriate for the current view of data. We can then offer a "suggested visualizations" prompt which shows previews. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/143/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
280745746 | MDU6SXNzdWUyODA3NDU3NDY= | 171 | HTML comments specifying custom templates for page | 9599 | closed | 0 | 2949431 | 1 | 2017-12-09T19:11:13Z | 2017-12-09T21:50:50Z | 2017-12-09T21:48:03Z | OWNER | This would make the custom templating system self-documenting, and save people from having to figure out the right template names for customizing specific pages. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/171/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
737394470 | MDU6SXNzdWU3MzczOTQ0NzA= | 1084 | Table/database action menu cut off if too short | 9599 | closed | 0 | 6055094 | 4 | 2020-11-06T01:55:23Z | 2020-11-21T23:45:59Z | 2020-11-21T23:45:59Z | OWNER | ![3CC0C181-959E-4B20-BE39-806ED93E833E](https://user-images.githubusercontent.com/9599/98316836-03891800-1f90-11eb-9e52-5266baf33296.jpeg) | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1084/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
710506708 | MDU6SXNzdWU3MTA1MDY3MDg= | 978 | Rendering glitch with column headings on mobile | 9599 | closed | 0 | 5971510 | 6 | 2020-09-28T19:04:45Z | 2020-10-08T23:54:40Z | 2020-09-28T22:43:01Z | OWNER | ![6348FED5-1310-47DA-BF54-810E63C96C6F](https://user-images.githubusercontent.com/9599/94474607-bc368d00-0182-11eb-9bb1-9a0e455353f1.jpeg) https://latest-with-plugins.datasette.io/fixtures?sql=select%0D%0A++dateutil_parse%28%2210+october+2020+3pm%22%29%2C%0D%0A++dateutil_easter%28%222020%22%29%2C%0D%0A++dateutil_parse_fuzzy%28%22This+is+due+10+september%22%29%2C%0D%0A++dateutil_parse%28%221%2F2%2F2020%22%29%2C%0D%0A++dateutil_parse%28%222020-03-04%22%29%2C%0D%0A++dateutil_parse_dayfirst%28%222020-03-04%22%29%2C%0D%0A++dateutil_easter%282020%29 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/978/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
913900374 | MDU6SXNzdWU5MTM5MDAzNzQ= | 1369 | Don't show foreign key IDs twice if no label | 9599 | open | 0 | 1 | 2021-06-07T19:47:02Z | 2021-06-07T19:47:24Z | OWNER | ![B5B54D94-A768-4544-A88D-CDCAB417CD3C](https://user-images.githubusercontent.com/9599/121078979-6e9d0600-c78e-11eb-8b70-20e6d29b48b1.jpeg) | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1369/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1469015001 | I_kwDOBm6k_c5Xj2PZ | 1916 | GET requests against POST endpoints should not 500 error | 9599 | closed | 0 | 7867486 | 1 | 2022-11-30T04:04:43Z | 2022-11-30T05:15:19Z | 2022-11-30T05:15:19Z | OWNER | ![CF37BA4D-0677-4DDD-A339-EAF163BB63B7](https://user-images.githubusercontent.com/9599/204705025-6f88e9f7-757d-45e8-a89c-ab97e84781e8.png) | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1916/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
1564769997 | I_kwDOBm6k_c5dRH7N | 2011 | Applied facet did not result in an "x" icon to dismiss it | 9599 | open | 0 | 1 | 2023-01-31T17:57:44Z | 2023-01-31T17:58:54Z | OWNER | ![CleanShot 2023-01-31 at 09 55 56@2x](https://user-images.githubusercontent.com/9599/215843684-1761a230-d490-4f87-be6d-186319366794.png) That's against this data https://data.sfgov.org/City-Management-and-Ethics/Supplier-Contracts/cqi5-hm2d imported using https://datasette.io/plugins/datasette-socrata It's for `Contract Type` of `Non-Purchasing Contract (Rents, etc.)` - so possible that some of the spaces or punctuation in either the name of the value tripped up the code that decides if the X icon should be displayed. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/2011/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1424378012 | I_kwDOBm6k_c5U5kic | 1860 | SQL query field can't begin by a comment | 562352 | closed | 0 | 12 | 2022-10-26T16:55:31Z | 2022-10-27T18:57:37Z | 2022-10-27T04:21:40Z | NONE | ![image](https://user-images.githubusercontent.com/562352/198085197-f26fcd61-4dac-4ca4-a346-e70f88a30ecc.png) SQL comments are **very** useful to explain the meaning of the query. It's currently impossible to put it at the beginning of the field as seen on the screen capture: it leads to an error: `Statement must be a SELECT`. It would be great to make it possible because: * as the request is the title of the page: * it eases the search with search engines * it eases the search in the browsers' url field * it acts as a kind of title: the global meaning of the query is immediately understandable * some tools, such as Slack, are shortening long URLs and displaying the beginning of the URLs (eg. `https://example.org/products?sql=select+%28length%28data_quality_errors_ta[...]+%21%3D+%22%22+group+by+NB_of_issues+order+by+NB_of_issues+desc+limit+200`) Beginning a query with a comment is possible with SQLite. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1860/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1578790070 | I_kwDOCGYnMM5eGmy2 | 527 | `Table.convert()` skips falsey values | 167893 | closed | 0 | 5 | 2023-02-10T00:00:52Z | 2023-05-09T21:15:05Z | 2023-05-08T21:03:24Z | CONTRIBUTOR | # Summary By design, `Table.convert()` does [not attempt](https://github.com/simonw/sqlite-utils/blob/fc221f9b62ed8624b1d2098e564f525c84497969/sqlite_utils/db.py#L2663) conversion of falsey values (`None`, `""`, `0`, ...). This is surprising (directly contradicts the docstring) and `convert()` may quietly skip cells where the user assumed a conversion would take place. # Example Increment a column of integers by one ``` python from sqlite_utils import Database db = Database(memory=True) table = db['table'] col = 'x' table.insert_all([{col: 0}, {col:1}]) print(table.get(1)) # 0 print(table.get(2)) # 1 print() table.convert(col, lambda x: x+1) print(table.get(1)) # got 0, expected 1 ⚠⚠⚠ print(table.get(2)) # got 2, expected 2 ``` Another example might be, say, transforming cells containing empty string to `NULL`. # Discussion This was, I think, a pragmatic choice so that consumers can skip writing guard clauses for these falsey values (particularly from the CLI). But this surprising undocumented behavior can lead to incorrect data. I don't think this is a good trade-off between convenience and correctness. In the absence of this convenience users will either have to write guard clauses into their conversion expressions (or adapt the called function to do the same), so: ``` python fn(value) if value else value ``` instead of: ``` python fn(value) ``` This is more typing and sometimes I will forget, and there will be errors. (But they will be noisy errors, which is a good thing). Such a change will certainly inconvenience some existing consumers; there will be some breakage. But I think this is worth it to avoid quietly not converting some values by default, which can lead to quietly bad data. I have a PR that I will attach, please take a look and see what you think. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/527/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
611284481 | MDU6SXNzdWU2MTEyODQ0ODE= | 38 | [Feature Request] Support Repo Name in Search 🥺 | 5779832 | closed | 0 | 4 | 2020-05-02T22:08:51Z | 2020-05-03T02:34:32Z | 2020-05-02T23:15:11Z | NONE | ## Description Per your [v2.2 release tweet](https://twitter.com/simonw/status/1256700238099693568) I played with the demo, but the output did not match my expectations. ## Expected Behavior Expected a search query for "twitter" contained within the `repo` column to return non-zero results. ## Actual Behavior 😭 [0 rows where repo contains "twitter" sorted by starred_at descending](https://github-to-sqlite.dogsheep.net/github/stars?repo__contains=twitter&_sort_desc=starred_at) ## Best Explanation Per the table schema (see appendix) `repo` is of type `INTEGER` which built from `repo_id` and does not expose the repo name in search. ## Desired Behavior Given that searching for "206156866" is less intuitive than "twitter", it would be great to support this via extending the search capabilities or by adding an additional column. ✅ 104 rows where repo contains "twitter" ❌ [104 rows where repo contains "206156866" sorted by starred_at descending](https://github-to-sqlite.dogsheep.net/github/stars?repo__contains=206156866&_sort_desc=starred_at) ## Appendix ``` CREATE TABLE [stars] ( [user] INTEGER REFERENCES [users]([id]), [repo] INTEGER REFERENCES [repos]([id]), [starred_at] TEXT, PRIMARY KEY ([user], [repo]) ); CREATE INDEX [idx_stars_repo] ON [stars] ([repo]); CREATE INDEX [idx_stars_user] ON [stars] ([user]); ``` | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/38/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1198822563 | I_kwDOBm6k_c5HdJSj | 1706 | [feature] immutable mode for a directory, not just individual sqlite file | 9020979 | open | 0 | 4 | 2022-04-10T00:50:57Z | 2022-12-09T19:11:40Z | CONTRIBUTOR | ## Motivation - I have a directory of sqlite databases - I'd like to use immutable mode when opening them for better performance [docs](https://docs.datasette.io/en/0.54/performance.html#immutable-mode) - Currently using this flag throws the following error IsADirectoryError: [Errno 21] Is a directory: '/name-of-directory' ## Proposal Immutable flag works for both single files and directories datasette -i /folder-of-sqlite-files | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1706/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1182227211 | I_kwDOBm6k_c5Gd1sL | 1692 | [plugins][feature request]: Support additional script tag attributes when loading custom JS | 9020979 | open | 0 | 2 | 2022-03-27T01:16:03Z | 2022-03-30T06:14:51Z | CONTRIBUTOR | ## Motivation - The build system for my new [plugin](https://github.com/hydrosquall/datasette-nteract-data-explorer) has two output JS files, one for browsers that support ES modules, one for browsers that don't. At present, I'm only passing one of them into Datasette. - I'd like to specify the non-es-module script as a fallback for older browsers. I don't want to load it by default, because browsers will only need one, and it's heavy, so for now I'm only supporting modern browsers. To be able to support legacy browsers without slowing down users with modern browsers, I would like to be able to set additional HTML attributes on the tag fallback script, `nomodule` and `defer`. My injected scripts should look something like this: ```html <script type="module" src="/index.my-es-module-bundle.js"></script> <script src="/index.my-legacy-fallback-bundle.js" nomodule="" defer></script> ``` ## Proposal To achieve this, I propose additional optional properties to the API accepted by the `extra_js_urls` hook and custom JS field the `metadata.json` [described here](https://docs.datasette.io/en/stable/custom_templates.html#custom-css-and-javascript). Under this API, I'd write something like this to get the above HTML rendered in Datasette. ```json { "extra_js_urls": [ { "url": "/index.my-es-module-bundle.js", "module": true, }, { "url": "/index.my-legacy-fallback-bundle.js", "nomodule": "", "defer": true } ] } ``` ## Resources - [MDN on the script tag](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/script) - There may be other properties that could be added that are potentially valuable, like `async` or `referrerpolicy`, but I don't have an immediate need for those. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1692/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1211283427 | I_kwDODFdgUs5IMrfj | 72 | feature: display progress bar when downloading multi-page responses | 9020979 | open | 0 | 1 | 2022-04-21T16:37:12Z | 2022-04-21T17:29:31Z | NONE | ## Motivation For a long running command (longer than 1 minute) for a big table (like pull requests or commits), it can be tricky to know if the script is still running, or if a rate limit/error was encountered We know how many pages there are, so it may be possible to indicate how many remain. https://github.com/dogsheep/github-to-sqlite/blob/a6e237f75a4b86963d91dcb5c9582e3a1b3349d6/github_to_sqlite/utils.py#L367 ## Resources - Using the existing Click API: - https://click.palletsprojects.com/en/5.x/utils/#showing-progress-bars - Loading spinner: https://github.com/pavdmyt/yaspin - Progress bar: https://github.com/tqdm/tqdm | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/72/reactions", "total_count": 3, "+1": 3, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1109884720 | I_kwDOBm6k_c5CJ38w | 1609 | Ensure "pip install datasette" still works with Python 3.6 | 9599 | closed | 0 | 12 | 2022-01-21T00:08:10Z | 2022-01-24T19:20:09Z | 2022-01-21T02:24:13Z | OWNER | ## Original title: Can I keep "pip install datasette" working on Python 3.6? I dropped support for 3.6 in: - #1577 I'm getting reports that `pip3 install datasette` throws an error on that Python, even though I haven't made that new release yet - presumably due to lack of pinning of Uvicorn: https://twitter.com/ldodds/status/1484289475195080706 Is it possible to get `pip` on that version of Python to install the highest possible version of the packages that are still known to support Python 3.6? If so, how? | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1609/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1339663518 | I_kwDOBm6k_c5P2aSe | 1784 | Include "entrypoint" option on `--load-extension`? | 15178711 | closed | 0 | 2 | 2022-08-16T00:22:57Z | 2022-08-23T18:34:31Z | 2022-08-23T18:34:31Z | CONTRIBUTOR | ## Problem SQLite extensions have the option to define multiple "entrypoints" in each loadable extension. For example, the upcoming version of `sqlite-lines` will have 2 entrypoints: the default `sqlite3_lines_init` (which SQLite will automatically guess for) and `sqlite3_lines_noread_init`. The `sqlite3_lines_noread_init` version omits functions that read from the filesystem, which is necessary for security purposes when running untrusted SQL (which Datasette does). (Similar multiple entrypoints will also be added for sqlite-http). The `--load-extension` flag, however, doesn't give the option to specify a different entrypoint, so the default one is always used. ## Proposal I want there to be a new command line option of the `--load-extension` flag to specify a custom entrypoint like so: ``` datasette my.db \ --load-extension ./lines0 sqlite3_lines0_noread_init ``` Then, under the hood, this line of code: https://github.com/simonw/datasette/blob/7af67b54b7d9bca43e948510fc62f6db2b748fa8/datasette/app.py#L562 Would look something like this: ```python conn.execute("SELECT load_extension(?, ?)", [extension, entrypoint]) ``` One potential problem: For backward compatibility, I'm not sure if Click allows cli flags to have variable number of options ("arity"). So I guess it could also use a `:` delimiter like `--static`: ``` datasette my.db \ --load-extension ./lines0:sqlite3_lines0_noread_init ``` Or maybe even a new flag name? ``` datasette my.db \ --load-extension-entrypoint ./lines0 sqlite3_lines0_noread_init ``` Personally I prefer the `:` option... and maybe even `--load-extension` -> `--load`? Definitely out of scope for this issue tho ``` datasette my.db \ --load./lines0:sqlite3_lines0_noread_init ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1784/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1698865182 | I_kwDOBm6k_c5lQqAe | 2069 | [BUG] Cannot insert new data to deployed instance | 31861128 | open | 0 | 1 | 2023-05-07T02:59:42Z | 2023-05-07T03:17:35Z | NONE | ## Summary Recently, I deployed an instance of datasette to Vercel with the following plugins: - datasette-auth-tokens - datasette-insert With the above plugins, I was able to insert new data to local sqlite db. However, when it comes to the deployment on Vercel, things behave differently. I observed some errors from the logs console on Vercel: ```console File "/var/task/datasette/database.py", line 179, in _execute_writes conn = self.connect(write=True) File "/var/task/datasette/database.py", line 93, in connect assert not (write and not self.is_mutable) AssertionError ``` <img width="1380" alt="image" src="https://user-images.githubusercontent.com/31861128/236655274-3fd5e16f-4e06-4ba8-87ff-41130b8ebe90.png"> I think it is a potential bug. ## Reproduce <details><summary>metadata.json</summary> </br> ```json { "plugins": { "datasette-insert": { "allow": { "id": "*" } }, "datasette-auth-tokens": { "tokens": [ { "token": { "$env": "INSERT_TOKEN" }, "actor": { "id": "repeater" } } ], "param": "_auth_token" } } } ``` </details> <details><summary>commands</summary> </br> ```bash # deploy datasette publish vercel remote.db \ --project=repeater-bot-sqlite \ --metadata metadata.json \ --install datasette-auth-tokens \ --install datasette-insert \ --vercel-json=vercel.json # test insert cat fixtures/dogs.json | curl --request POST -d @- -H "Authorization: Bearer <token>" \ 'https://repeater-bot-sqlite.vercel.app/-/insert/remote/dogs?pk=id' ``` </details> <details><summary>logs</summary> </br> ```console Traceback (most recent call last): File "/var/task/datasette/app.py", line 1354, in route_path response = await view(request, send) File "/var/task/datasette/app.py", line 1500, in async_view_fn response = await async_call_with_supported_arguments( File "/var/task/datasette/utils… | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/2069/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1575131737 | I_kwDOCGYnMM5d4ppZ | 525 | Repeated calls to `Table.convert()` fail | 167893 | closed | 0 | 4 | 2023-02-07T22:40:47Z | 2023-05-08T21:59:41Z | 2023-05-08T21:54:02Z | CONTRIBUTOR | ## Summary When using the API, repeated calls to `Table.convert()` do not work correctly since all conversions quietly use the callable (function, lambda) from the first call to `convert()` only. Subsequent invocations with different callables use the callable from the first invocation only. ## Example ```python from sqlite_utils import Database db = Database(memory=True) table = db['table'] col = 'x' table.insert_all([{col: 1}]) print(table.get(1)) table.convert(col, lambda x: x*2) print(table.get(1)) def zeroize(x): return 0 #zeroize = lambda x: 0 #zeroize.__name__ = 'zeroize' table.convert(col, zeroize) print(table.get(1)) ``` Output: ``` {'x': 1} {'x': 2} {'x': 4} ``` Expected: ``` {'x': 1} {'x': 2} {'x': 0} ``` ## Explanation This is some relevant [documentation](https://github.com/simonw/sqlite-utils/blob/1491b66dd7439dd87cd5cd4c4684f46eb3c5751b/docs/python-api.rst#registering-custom-sql-functions:~:text=By%20default%20registering%20a%20function%20with%20the%20same%20name%20and%20number%20of%20arguments%20will%20have%20no%20effect). * `Table.convert()` takes a `Callable` to perform data conversion on a column * The `Callable` is passed to `Database.register_function()` * `Database.register_function()` uses the callable's `__name__` attribute for registration * (Aside: all lambdas have a `__name__` of `<lambda>`: I thought this was the problem, and it was close, but not quite) * However `convert()` first wraps the callable by local function [`convert_value()`](https://github.com/simonw/sqlite-utils/blob/fc221f9b62ed8624b1d2098e564f525c84497969/sqlite_utils/db.py#L2661) * Consequently `register_function()` sees name `convert_value` for all invocations from `convert()` * `register_function()` silently ignores registrations using the same name, retaining only the first such registration There's a mismatch between the comments and the code: https://github.com/simonw/sqlite-utils/blob/fc221f9b62ed8624b1d2098e564f525c84497969/sqlite_utils/db.py#L4… | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/525/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1154399841 | I_kwDOBm6k_c5Ezr5h | 1645 | Sensible `cache-control` headers for static assets, including those served by plugins | 697092 | open | 0 | 3268330 | 4 | 2022-02-28T18:12:03Z | 2022-03-08T02:59:29Z | NONE | ## What I'm seeing With `default_cache_ttl = 86400`, I see the following: A table view returns `Cache-control: max-age=86400`: ![Screenshot_20220228_190000](https://user-images.githubusercontent.com/697092/156034352-4d64683e-39c8-49af-81df-0217a5957bbd.png) A static asset returns no `Cache-control` header: ![Screenshot_20220228_185933](https://user-images.githubusercontent.com/697092/156034363-d0b03cc2-5889-4ed2-b601-8c1846b8469a.png) ## What I expected to see I expected the static asset to return a `Cache-control` header indicating that this response can be cached. ## Why this matters I'm productionising a Datasette deployment right now and was looking into putting it behind a Varnish instance. I was surprised to see requests for static assets being served from Datasette rather than Varnish, this is what led me to look more closely at the response headers. While Datasette serves those static assets pretty quickly, I don't see why Datasette should serve them. By their nature, static assets like images and JS files are very cacheable, so it should be easy to serve them from a cache like Varnish. (Note that Varnish can easily be configured to override this header, enabling caching for static assets. But it would be better if this override was not necessary.) ## Discussion It seems clear to me that serving static assets without a `Cache-control` header is not ideal. I see two options here: A. Static assets use the same logic as table / SQL views to set the `Cache-control` header based on `default_cache_ttl`. B. An additional setting for static assets is introduced (`default_static_cache_ttl`, say). | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1645/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
1978603203 | I_kwDOCGYnMM517xbD | 602 | `sqlite-utils transform` removes the `AUTOINCREMENT` keyword | 4472046 | open | 0 | 0 | 2023-11-06T08:48:43Z | 2023-11-06T08:48:43Z | NONE | ### Context We ran into this bug randomly, noticing that deleted `ROWID` would get reused after migrating the DB. Using `transform` to change any column in the table will also unexpectedly strip away the `AUTOINCREMENT` keyword from the primary key definition, even if it was not the transformation target. ### Reproducible example **Original database** ```sql $ sqlite3 test.db << EOF CREATE TABLE mytable ( col1 INTEGER PRIMARY KEY AUTOINCREMENT, col2 TEXT NOT NULL ) EOF $ sqlite3 test.db ".schema mytable" CREATE TABLE mytable ( col1 INTEGER PRIMARY KEY AUTOINCREMENT, col2 TEXT NOT NULL ); ``` **Modified database after sqlite-utils** ```sql $ sqlite-utils transform test.db mytable --rename col2 renamedcol2 $ sqlite3 test.db "SELECT sql FROM sqlite_master WHERE name = 'mytable';" CREATE TABLE IF NOT EXISTS "mytable" ( [col1] INTEGER PRIMARY KEY, [renamedcol2] TEXT NOT NULL ); ``` | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/602/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1161937073 | I_kwDOBm6k_c5FQcCx | 1653 | Mechanism to default a table to sorting by multiple columns | 9599 | open | 0 | 2 | 2022-03-07T21:20:11Z | 2022-03-07T21:23:39Z | OWNER | ### Discussed in https://github.com/simonw/datasette/discussions/1652 <div type='discussions-op-text'> <sup>Originally posted by **zaneselvans** March 7, 2022</sup> It's easy to tell datasette to sort tables using a single column, as [described in the docs](https://docs.datasette.io/en/stable/metadata.html#setting-a-default-sort-order): ```yaml databases: ferc1: tables: f1_edcfu_epda: sort: created_time ``` But is there some way to tell it to sort using a composite key, like you would in an `ORDER BY` clause instead? For example, the way it's being done **[in this query](https://data.catalyst.coop/ferc1?sql=select%0D%0A++rowid%2C%0D%0A++respondent_id%2C%0D%0A++report_year%2C%0D%0A++spplmnt_num%2C%0D%0A++row_number%2C%0D%0A++row_seq%2C%0D%0A++row_prvlg%2C%0D%0A++acct_num%2C%0D%0A++depr_plnt_base%2C%0D%0A++est_avg_srvce_lf%2C%0D%0A++net_salvage%2C%0D%0A++apply_depr_rate%2C%0D%0A++mrtlty_crv_typ%2C%0D%0A++avg_remaining_lf%2C%0D%0A++report_prd%0D%0Afrom%0D%0A++f1_edcfu_epda%0D%0Awhere%0D%0A++respondent_id+%3D+210%0D%0A++AND+report_year+%3D+2020%0D%0Aorder+by%0D%0A++report_year%2C+report_prd%2C+respondent_id%2C+spplmnt_num%2C+row_number%0D%0Alimit%0D%0A++1000)** on our Datasette? ```sql SELECT respondent_id, report_year, spplmnt_num, row_number, row_seq, row_prvlg, acct_num, depr_plnt_base, est_avg_srvce_lf, net_salvage, apply_depr_rate, mrtlty_crv_typ, avg_remaining_lf, report_prd FROM f1_edcfu_epda WHERE respondent_id = 210 AND report_year = 2020 ORDER BY report_year, report_prd, respondent_id, spplmnt_num, row_number LIMIT 1000 ``` The problem here is that by default it's using `rowid` (the SQLite assigned autoincrementing integer key) to order the records, but the table **should** have a natural composite primary key, but the original database that this data is being migrated from doesn't enforce unique primary keys, so there are dupes, and we don't want to drop those rows, and the records are somehow getti… | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1653/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
559964149 | MDU6SXNzdWU1NTk5NjQxNDk= | 665 | Introduce a SQL statement parser in Python | 9599 | open | 0 | 1 | 2020-02-04T20:36:05Z | 2020-02-04T20:36:48Z | OWNER | #254 and #653 are both examples of problems that could be solved using a real SQL parser in Python. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/665/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
323830051 | MDU6SXNzdWUzMjM4MzAwNTE= | 270 | --limit= CLI option for setting limits | 9599 | closed | 0 | 1 | 2018-05-17T00:14:24Z | 2018-05-18T06:19:31Z | 2018-05-18T06:16:39Z | OWNER | #264 calls for four new datasette limit options, on top of the two existing ones: * `--max_returned_rows` * `--sql_time_limit_ms` These are already clogging up `datasette serve --help` a bit. How about this syntax instead? datasette --limit max_returned_rows:100 \ --limit facet_timeout_ms:500 demo.db Then we can add as many new user over-rideable limits as we like without clogging up `--help` too much - though it would be good to have a way of optionally listings their documentation as well. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/270/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
324652142 | MDU6SXNzdWUzMjQ2NTIxNDI= | 274 | Rename --limit to --config, add --help-config | 9599 | closed | 0 | 2 | 2018-05-19T18:57:42Z | 2018-05-20T17:04:55Z | 2018-05-20T17:04:11Z | OWNER | #270 introduced `--limit` but on further thought it should be called `--config` instead. `--page_size` should becomes `--config default_page_size:1000` Add `--help-config` to show full help showing all config settings. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/274/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
976399638 | MDU6SXNzdWU5NzYzOTk2Mzg= | 319 | [Enhancement] Please allow 'insert-files' to insert content as text. | 66709385 | closed | 0 | 10 | 2021-08-22T15:10:46Z | 2021-08-24T23:33:45Z | 2021-08-24T23:33:44Z | NONE | 'insert-files' creates BLOB columns for file contents. Transforming the column to TEXT still keep the content as binary. Even though I'm sure there is a transform that can be applied decoding the text it would be great to have a argument to make 'insert-files' to do it as text (with optional text encoding). The use case is a bunch of htmls (single file) on a directory structure that inserted with this command could be served in Datasette allowing full text search. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/319/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1251710928 | I_kwDOBm6k_c5Km5fQ | 1751 | Add scrollbars to table presentation in default layout | 408765 | closed | 0 | 1 | 2022-05-28T19:44:57Z | 2022-05-28T19:52:17Z | 2022-05-28T19:52:17Z | NONE | (As you will be able to tell from the terminology I use, I am not a frontend guy, but I hope you will understand.) When a table is wide and needs horizontal scrolling to see the columns towards the end, the user needs to scroll horizontally. However, since the container for the HTML table (`div` with class `table-wrapper`) isn't limited by the window size, I first need to vertically scroll near to the bottom of the page in order to scroll horizontally. Then I can scroll back up again. This isn't very user friendly. Instead, I think it would make sense to constrain the table's size (when necessary), so that the vertical and horizontal scrollbars either always are visible or at least not far out of reach. I understand that I could provide my own template and / or CSS, but I think it would probably make sense to adjust the default in this regard. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1751/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
853672224 | MDU6SXNzdWU4NTM2NzIyMjQ= | 1294 | "You can check out any time you like. But you can never leave!" | 192568 | open | 0 | 0 | 2021-04-08T17:02:15Z | 2021-04-08T18:35:50Z | CONTRIBUTOR | (Feel free to rename this one.) - The column gear lets you "Show not-blank rows." Then it places a parameter in the URL, which a web developer would notice, but a lot of users won't notice, or know to delete it. Would be good to toggle "Show not-blank rows" with "Show all rows." (Also would be quite helpful to have a "Show blank rows | Show all rows" option) - The column gear lets you "Sort ascending" and "Sort descending" but then you're stuck with some sort of sorted version thereafter, unless you know to sort the ID column, or to remove the full _sort parameter and its value in the URL. Would be good to offer a "Remove sort" option in the gear. - These requests are in the same camp as: https://github.com/simonw/datasette-vega/issues/36 - I suspect there are other url parameter instances where similar analysis would be helpful, but the three above are the use cases I've run across. UPDATE: - It would be helpful to have a "Previous page" available for all but the first table page. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1294/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1465194249 | I_kwDOCGYnMM5XVRcJ | 514 | upsert of new row with check constraints fails | 193185 | closed | 0 | 5 | 2022-11-26T16:12:23Z | 2023-05-08T21:50:52Z | 2023-05-08T21:50:51Z | NONE | (I originally opened this in https://github.com/simonw/datasette-insert/issues/20, but I see that that library depends on sqlite-utils) In the case of a new row, upsert first adds the row, specifying only its pkeys: https://github.com/simonw/sqlite-utils/blob/965ca0d5f5bffe06cc02cd7741344d1ddddf9d56/sqlite_utils/db.py#L2783-L2787 This means that a table with NON NULL (or other constraint) columns that aren't part of the pkey can't have new rows upserted. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/514/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
988556488 | MDU6SXNzdWU5ODg1NTY0ODg= | 1459 | suggestion: allow `datasette --open` to take a relative URL | 51016 | open | 0 | 1 | 2021-09-05T17:17:07Z | 2021-09-05T19:59:15Z | CONTRIBUTOR | (soft suggestion because I'm not sure I'm using datasette right yet) Over at https://github.com/ctb/2021-sourmash-datasette, I'm playing around with datasette, and I'm creating some static pages to send people to the right facets. There may well be better ways of achieving this end goal, and I will find out if so, I'm sure! But regardless I think it might be neat to support an option to allow `-o/--open` to take a relative URL, that then gets appended to the hostname and port. This would let me improve my documentation. I don't see any downsides, either, but 🤷 there may well be some :) Happy to dig in and provide a PR if it's of interest. I'm not sure off the top of my head how to support an optional value to a parameter in argparse - the current `-o` behavior is kinda nice so it'd be suboptimal to require a url for `-o`. Maybe `--open-url=` or something would work? | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1459/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1538197093 | I_kwDOBm6k_c5brwZl | 1995 | foreign_keys error 500 | 137183 | open | 0 | 0 | 2023-01-18T15:27:36Z | 2023-01-18T16:44:01Z | NONE | **Error 500 expected string or bytes-like object** [espial-new.sqlite3.zip](https://github.com/simonw/datasette/files/10447965/espial-new.sqlite3.zip) run `datasette espial-new.sqlite3` & click on any table other than `User` ``` /home/jon/.local/lib/python3.10/site-packages/datasette/app.py:814 in │ │ expand_foreign_keys │ │ │ │ 811 │ │ │ from {other_table} │ │ 812 │ │ │ where {other_column} in ({placeholders}) │ │ 813 │ │ """.format( │ │ ❱ 814 │ │ │ other_column=escape_sqlite(fk["other_column"]), │ │ 815 │ │ │ label_column=escape_sqlite(label_column), │ │ 816 │ │ │ other_table=escape_sqlite(fk["other_table"]), │ │ 817 │ │ │ placeholders=", ".join(["?"] * len(set(values))), │ │ │ │ ╭───────────────────────────── locals ──────────────────────────────╮ │ │ │ column = 'user_id' │ │ │ │ database = 'espial-new' │ │ │ │ db = <Database: espial-new (mutable, size=53248)> │ │ │ │ fk = { │ │ │ │ │ 'column': 'user_id', │ │ │ │ │ 'other_table': 'user', │ │ │ │ │ 'other_column': None │ │ │ │ } │ │ │ │ foreign_keys = [ │ │ │ │ │ { … | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1995/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
807174161 | MDU6SXNzdWU4MDcxNzQxNjE= | 227 | Error reading csv files with large column data | 295329 | closed | 0 | 4 | 2021-02-12T11:51:47Z | 2021-02-16T11:48:03Z | 2021-02-14T21:17:19Z | NONE | *Feel free to close this issue - I mostly added it for reference for future folks that run into this :)* I have a CSV file with one column that has very long strings. When i try to import this file via the `insert` command I get the following error: ``` sqlite-utils insert database.db table_name file_with_large_column.csv Traceback (most recent call last): File "/usr/local/bin/sqlite-utils", line 10, in <module> sys.exit(cli()) File "/usr/local/lib/python3.7/site-packages/click/core.py", line 829, in __call__ return self.main(*args, **kwargs) File "/usr/local/lib/python3.7/site-packages/click/core.py", line 782, in main rv = self.invoke(ctx) File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params) File "/usr/local/lib/python3.7/site-packages/click/core.py", line 610, in invoke return callback(*args, **kwargs) File "/usr/local/lib/python3.7/site-packages/sqlite_utils/cli.py", line 774, in insert default=default, File "/usr/local/lib/python3.7/site-packages/sqlite_utils/cli.py", line 705, in insert_upsert_implementation docs, pk=pk, batch_size=batch_size, alter=alter, **extra_kwargs File "/usr/local/lib/python3.7/site-packages/sqlite_utils/db.py", line 1852, in insert_all first_record = next(records) File "/usr/local/lib/python3.7/site-packages/sqlite_utils/cli.py", line 703, in <genexpr> docs = (decode_base64_values(doc) for doc in docs) File "/usr/local/lib/python3.7/site-packages/sqlite_utils/cli.py", line 681, in <genexpr> docs = (dict(zip(headers, row)) for row in reader) _csv.Error: field larger than field limit (131072) ``` Built with the docker image `datasetteproject/datasette:0.54` with the following versions: ``` # sqlite-utils --version sqlite-utils, version 3.4.1 # datasette --version datas… | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/227/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1114147905 | I_kwDOBm6k_c5CaIxB | 1612 | Move canned queries closer to the SQL input area | 639012 | closed | 0 | 3268330 | 5 | 2022-01-25T17:06:39Z | 2022-03-19T04:04:49Z | 2022-01-25T18:34:21Z | CONTRIBUTOR | *Original title: Consider placing example queries above the sql input?* Hi! Have been enjoying deploying ad hoc datasettes for collaborators to pick over! I keep finding myself manually "fixing" the database.html template so that the "example queries" (canned queries) appear directly *over* the sql box? So they are sorta more a suggestion for collaborators who aren't inclined to write their own queries? My sense is any time I go to the trouble of writing canned queries my users should see 'em? (( I have also considered a client-side reactive-ish option where selecting a query just places the raw SQL in the box and doesn't execute it, but this seems to end up being an inconvenience, rather than a teaching tool. )) | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1612/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
860625833 | MDU6SXNzdWU4NjA2MjU4MzM= | 1300 | Make row available to `render_cell` plugin hook | 3243482 | closed | 0 | 5 | 2021-04-18T10:14:37Z | 2022-07-07T16:34:05Z | 2022-07-07T16:31:22Z | CONTRIBUTOR | *Original title: **Generating URL for a row inside `render_cell` hook*** Hey, I am using Datasette to view a database that contains video metadata. It has BLOB columns that contain video thumbnails in JPG format (around 100-500KB per row). I've registered an output formatter that extends `datasette.blob_renderer.render_blob` function and serves the column with `image/jpeg` content type. ```python from datasette.blob_renderer import render_blob async def render_jpg(datasette, database, rows, columns, request, table, view_name): response = await render_blob(datasette, database, rows, columns, request, table, view_name) response.content_type = "image/jpeg" response.headers["Content-Disposition"] = f'inline; filename="image.jpg"' return response @hookimpl def register_output_renderer(): return { "extension": "jpg", "render": render_jpg, "can_render": lambda: True, } ``` This works well. I can visit `http://localhost:8001/mydb/videos/1.jpg?_blob_column=thumbnail` and view the image. I want to display the image directly with an `<img>` tag (lazy-loaded of course). So, I need a URL, because embedding base64 would increase the page size too much (each image > 100KB). Datasette generates a link with `.blob` extension for blob columns. It does this by calling `datasette.urls.row_blob` https://github.com/simonw/datasette/blob/7a2ed9f8a119e220b66d67c7b9e07cbab47b1196/datasette/views/table.py#L169-L179 But I have no way of getting the row inside the `render_cell` hook. ```python @hookimpl def render_cell(value, column, table, database, datasette): if isinstance(value, bytes) and imghdr.what(None, value): # generate url return '$renderedLink' ``` Any pointers? | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1300/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1126692066 | I_kwDOCGYnMM5DJ_Ti | 403 | Document how to add a primary key to a rowid table using `sqlite-utils transform --pk` | 536941 | closed | 0 | 4 | 2022-02-08T01:39:40Z | 2022-02-09T04:22:43Z | 2022-02-08T19:33:59Z | CONTRIBUTOR | *Original title: Add option for adding a new, serial, primary key* sometimes we have tables that don't have primary keys, but ought to have them. we *can* use rowid for that, but it would often be nicer to have an explicit primary key. using the current value of rowid would be fine. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/403/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1087913724 | I_kwDOBm6k_c5A2D78 | 1577 | Drop support for Python 3.6 | 9599 | closed | 0 | 3268330 | 6 | 2021-12-23T18:17:03Z | 2022-01-25T23:30:03Z | 2022-01-20T04:31:41Z | OWNER | *Original title: Decide when to drop support for Python 3.6* > `context_vars` can solve this but they were introduced in Python 3.7: https://www.python.org/dev/peps/pep-0567/ > > Python 3.6 support ends in a few days time, and it looks like Glitch has updated to 3.7 now - so maybe I can get away with Datasette needing 3.7 these days? > > Tweeted about that here: https://twitter.com/simonw/status/1473761478155010048 _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1576#issuecomment-999878907_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1577/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
816560819 | MDU6SXNzdWU4MTY1NjA4MTk= | 240 | table.pks_and_rows_where() method returning primary keys along with the rows | 9599 | closed | 0 | 7 | 2021-02-25T15:49:28Z | 2021-02-25T16:39:23Z | 2021-02-25T16:28:23Z | OWNER | *Original title: Easier way to update a row returned from .rows* Here's a surprisingly hard problem I ran into while trying to implement #239 - given a row returned by `db[table].rows` how can you update that row? The problem is that the `db[table].update(...)` method requires a primary key. But if you have a row from the `db[table].rows` iterator it might not even contain the primary key - provided the table is a `rowid` table. Instead, currently, you need to introspect the table and, if `rowid` is a primary key, explicitly include that in the `select=` argument to `table.rows_where(...)` - otherwise it will not be returned. A utility mechanism to make this easier would be very welcome. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/240/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1077620955 | I_kwDOBm6k_c5AOzDb | 1549 | Redesign CSV export to improve usability | 536941 | open | 0 | 3268330 | 5 | 2021-12-11T19:02:12Z | 2022-04-04T11:17:13Z | CONTRIBUTOR | *Original title: Set content type for CSV so that browsers will attempt to download instead opening in the browser* Right now, if the user clicks on the CSV related to a <s>table or a</s> query, the response header for the content type is "content-type: text/plain; charset=utf-8" Most browsers will try to open a file with this content-type in the browser. This is not what most people want to do, and lots of folks don't know that if they want to download the CSV and open it in the a spreadsheet program they next need to save the page through their browser. It would be great if the response header could be something like ``` 'Content-type: text/csv'); 'Content-disposition: attachment;filename=MyVerySpecial.csv'); ``` which would lead browsers to open a download dialog. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1549/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
1250629388 | I_kwDOCGYnMM5KixcM | 440 | CSV files with too many values in a row cause errors | 4068 | closed | 0 | 20 | 2022-05-27T10:54:44Z | 2022-06-14T22:23:01Z | 2022-06-14T20:12:46Z | NONE | *Original title: csv.DictReader can have None as key* In some cases, `csv.DictReader` can have `None` as key for unnamed columns, and a list of values as value. `sqlite_utils.utils.rows_from_file` cannot handle that: ```python url="https://artsdatabanken.no/Fab2018/api/export/csv" db = sqlite_utils.Database(":memory") with urlopen(url) as fab: reader, _ = sqlite_utils.utils.rows_from_file(fab, encoding="utf-16le") db["fab2018"].insert_all(reader, pk="Id") ``` Result: ``` Traceback (most recent call last): File "<stdin>", line 3, in <module> File "/home/user/.local/pipx/venvs/sqlite-utils/lib/python3.8/site-packages/sqlite_utils/db.py", line 2924, in insert_all chunk = list(chunk) File "/home/user/.local/pipx/venvs/sqlite-utils/lib/python3.8/site-packages/sqlite_utils/db.py", line 3454, in fix_square_braces if any("[" in key or "]" in key for key in record.keys()): File "/home/user/.local/pipx/venvs/sqlite-utils/lib/python3.8/site-packages/sqlite_utils/db.py", line 3454, in <genexpr> if any("[" in key or "]" in key for key in record.keys()): TypeError: argument of type 'NoneType' is not iterable ``` Code: https://github.com/simonw/sqlite-utils/blob/59be60c471fd7a2c4be7f75e8911163e618ff5ca/sqlite_utils/db.py#L3454 `sqlite-utils insert` from command line is not affected by this issue. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/440/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
656959584 | MDU6SXNzdWU2NTY5NTk1ODQ= | 893 | pip3 install datasette not serving static on linuxbrew. | 44167 | closed | 0 | 1 | 2020-07-14T23:33:38Z | 2021-06-02T04:29:56Z | 2021-06-02T04:29:56Z | NONE | *This error wasn't thrown* ``` Traceback (most recent call last): File "/home/linuxbrew/.linuxbrew/opt/python@3.8/lib/python3.8/site-packages/datasette/utils/asgi.py", line 289, in inner_static full_path.relative_to(root_path) File "/home/linuxbrew/.linuxbrew/opt/python@3.8/lib/python3.8/pathlib.py", line 904, in relative_to raise ValueError("{!r} does not start with {!r}" ValueError: '/home/linuxbrew/.linuxbrew/lib/python3.8/site-packages/datasette/static/app.css' does not start with '/home/linuxbrew/.linuxbrew/opt/python@3.8/lib/python3.8/site-packages/datasette/static' ``` Linuxbrew install python@3.8 with symbolic links when You call the full_path.relative_to(root_path) throw ValueError. This happened when you install from pip3 when you install with python3 setup.py develop , works good. Well at the end the static wasn't serving. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/893/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1822939274 | I_kwDOBm6k_c5sp9iK | 2113 | Implement and document extras for the new query view page | 9599 | open | 0 | 8755003 | 3 | 2023-07-26T18:24:01Z | 2023-08-09T17:35:22Z | OWNER | - #2109 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/2113/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
1822940263 | I_kwDOBm6k_c5sp9xn | 2114 | Implement canned queries against new query JSON work | 9599 | closed | 0 | 9700784 | 3 | 2023-07-26T18:24:50Z | 2023-08-09T15:26:58Z | 2023-08-09T15:26:57Z | OWNER | - #2109 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/2114/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
1822940964 | I_kwDOBm6k_c5sp98k | 2115 | Ensure all tests pass against new query view JSON | 9599 | closed | 0 | 9700784 | 0 | 2023-07-26T18:25:20Z | 2023-08-08T02:01:39Z | 2023-08-08T02:01:38Z | OWNER | - #2109 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/2115/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
924992318 | MDU6SXNzdWU5MjQ5OTIzMTg= | 281 | Mechanism for explicitly stating CSV or JSON or TSV for sqlite-utils memory | 9599 | closed | 0 | 1 | 2021-06-18T15:04:53Z | 2021-06-19T03:11:59Z | 2021-06-19T03:11:59Z | OWNER | - Implement `filename.json:json` and `-:nl` and suchlike options for specifying the format rather than guessing it - see https://github.com/simonw/sqlite-utils/issues/272#issuecomment-861985944 Follows #272 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/281/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
924990677 | MDU6SXNzdWU5MjQ5OTA2Nzc= | 279 | sqlite-utils memory should handle TSV and JSON in addition to CSV | 9599 | closed | 0 | 7 | 2021-06-18T15:02:54Z | 2021-06-19T03:11:59Z | 2021-06-19T03:11:59Z | OWNER | - Use sniff to detect CSV or TSV (if `:tsv` or `:csv` was not specified) and delimiters Follow-on from #272 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/279/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
276704327 | MDU6SXNzdWUyNzY3MDQzMjc= | 150 | _group_count= feature improvements | 9599 | closed | 0 | 3 | 2017-11-24T22:06:18Z | 2018-05-28T16:41:28Z | 2018-05-28T16:41:28Z | OWNER | - [ ] The "apply filters" form should keep you on the _group_count= page - [ ] Foreign key references should be expand - [ ] Page title should reflect the view you are on | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/150/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
268262480 | MDU6SXNzdWUyNjgyNjI0ODA= | 36 | date, year, month and day querystring lookups | 9599 | closed | 0 | 3 | 2017-10-25T04:23:45Z | 2018-05-28T17:30:53Z | 2018-05-28T17:30:53Z | OWNER | - [ ] `?timestamp___date=2017-07-17` - return every item where the timestamp falls on that date - [ ] `?timestamp___year=2017` - return every item where the timestamp falls within 2017 - [ ] `?timestamp___month=1` - return every item where the month component is January - [ ] `?timestamp___day=10` - return every item where the day-of-the-month component is 10 Follow on from #23 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/36/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
602533539 | MDU6SXNzdWU2MDI1MzM1Mzk= | 4 | Upload all my photos to a secure S3 bucket | 9599 | closed | 0 | 5324096 | 14 | 2020-04-18T19:24:50Z | 2020-04-18T21:58:11Z | 2020-04-18T21:57:13Z | MEMBER | - [x] Create a bucket with bucket credentials - [x] Programmatically upload some recent photos to it (from a notebook) - [x] Turn this into a script | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
601265023 | MDU6SXNzdWU2MDEyNjUwMjM= | 25 | Improvements to demo instance | 9599 | closed | 0 | 1 | 2020-04-16T17:26:55Z | 2020-04-16T18:07:12Z | 2020-04-16T18:07:12Z | MEMBER | - [x] Demo should pull issue-comments as well | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/25/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
732685643 | MDU6SXNzdWU3MzI2ODU2NDM= | 1063 | .csv should link to .blob downloads | 9599 | closed | 0 | 6026070 | 3 | 2020-10-29T21:45:58Z | 2021-06-17T18:12:30Z | 2020-10-29T22:47:45Z | OWNER | - [x] Update `.csv` output to link to these things (and get that `xfail` test to pass) - ~~Add a `.csv?_blob_base64=1` argument that causes them to be output in base64 in the CSV~~ > Moving the CSV work to a separate ticket. _Originally posted by @simonw in https://github.com/simonw/datasette/pull/1061#issuecomment-719042601_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1063/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
679779797 | MDU6SXNzdWU2Nzk3Nzk3OTc= | 939 | extra_ plugin hooks should take the same arguments | 9599 | closed | 0 | 6 | 2020-08-16T16:04:54Z | 2020-08-16T18:25:05Z | 2020-08-16T16:50:29Z | OWNER | - [x] `extra_css_urls(template, database, table, datasette)` - [x] `extra_js_urls(template, database, table, datasette)` - [x] `extra_body_script(template, database, table, view_name, datasette)` - [x] `extra_template_vars(template, database, table, view_name, request, datasette)` _Originally posted by @simonw in https://github.com/simonw/datasette/issues/938#issuecomment-674544691_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/939/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
565041624 | MDU6SXNzdWU1NjUwNDE2MjQ= | 671 | datasette.add_database(name, db) and datasette.remove_database(name) methods | 9599 | closed | 0 | 1 | 2020-02-14T01:05:48Z | 2020-02-14T01:30:35Z | 2020-02-14T01:30:30Z | OWNER | - `datasette.add_database(name, db)` - adds a new named database to the list of connected databases. `db` will be a `Database()` object, which may prove useful in the future for things like #670 and could also allow some plugins to provide in-memory SQLite databases. - `datasette.remove_database(name)` _Originally posted by @simonw in https://github.com/simonw/datasette/issues/417#issuecomment-586047995_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/671/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1432012302 | I_kwDOBm6k_c5VWsYO | 1877 | Refactor and tidy up final write API code | 9599 | closed | 0 | 1 | 2022-11-01T20:00:11Z | 2022-11-29T19:44:16Z | 2022-11-29T19:44:07Z | OWNER | - `views/table.py` has got a bit too big - I think the write classes should be pulled out into a separate module. - [x] There's duplicate logic for deciding if the table and database exist and checking permissions | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1877/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
608613033 | MDU6SXNzdWU2MDg2MTMwMzM= | 745 | Extract the hash-URL mechanism out into a plugin | 9599 | closed | 0 | 2 | 2020-04-28T21:00:38Z | 2020-10-23T19:47:18Z | 2020-10-23T19:47:10Z | OWNER | 0.28 in May 2019 made this feature not-the-default: https://datasette.readthedocs.io/en/stable/changelog.html#v0-28 - see #418 I've not felt the need to use it myself since. I think I should move it into a plugin. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/745/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
860734722 | MDU6SXNzdWU4NjA3MzQ3MjI= | 1302 | Fix disappearing facets | 192568 | open | 0 | 0 | 2021-04-18T18:42:33Z | 2021-04-20T07:40:15Z | CONTRIBUTOR | 1. Clone https://github.com/mroswell/list-N 2. Run `datasette disinfectants.db -o` 3. Select the `Safer_or_Toxic` facet. 4. Select `Toxic`. 5. Close out the `Safer_or_Toxic` facet. 6. Examine `Suggested facets` list. `Safer_or_Toxic` is GONE. 7. Try some other facets. When you select an element, and then close the list, in some cases, the facet properly returns to the `Suggested facet` list... Arrays and dates properly return to the list, but fields with strings don't return to the list. Since my site is devoted to whether disinfectants are Safer or Toxic, having the suggested facet disappear from the suggested facet list is very confusing* to end-users. This, along with a few other issues, unfortunately proved beyond my own programming ability to address. So I hired a Senior-level developer to address a number of issues, including this disappearing act. 8. Open a new terminal. Run `datasette disinfectants.db -m metadata.json --static static:static/ --template-dir templates/ --plugins-dir plugins/ -p 8001 -o` 9. Repeat steps 3-6, but this time, the Safer_or_Toxic facet returns to the list (and the related URL parameters are removed). I'm not sure how to do a pull request for this, because the plugin contains other functionality that goes beyond this bug. I wanted the facets sorted in a certain order (both in the suggested facet list, and the detail lists) (... the detail lists were hopping around all over the place before...) I wanted the duplicate facets removed (leaving only the one where you can facet by individual item in an array.) I wanted the arrays to be presented in a prettier fashion (I did that in the template... That could be moved over to the plugin at some point) I'm thinking it'll be very helpful if applicable parts of my project's plugin (sort_suggested_facets_plugin.py) will be able to be incorporated back into datasette, but I leave that to you to consider. (* The disappearing facet bug was especially confusing because I'm removing the filters and sql from the table page,… | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1302/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
735852274 | MDU6SXNzdWU3MzU4NTIyNzQ= | 1082 | DigitalOcean buildpack memory errors for large sqlite db? | 39538958 | open | 0 | 3 | 2020-11-04T06:35:32Z | 2020-11-04T19:35:44Z | NONE | 1. Have a sqlite db stored in Dropbox 2. Previously tried the Digital Ocean build pack minimal approach (e.g. Procfile, requirements.txt, bin/post_compile) 3. bin/post_compile with wget from Dropbox 4. download of large sqlite db is successful 5. log reveals that when building Docker container, Digital Ocean runs out of memory for 5gb+ sqlite db but works fine for 2gb+ sqlite db | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1082/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1428560020 | I_kwDOBm6k_c5VJhiU | 1872 | SITE-BUSTING ERROR: "render_template() called before await ds.invoke_startup()" | 192568 | closed | 0 | 3 | 2022-10-30T02:28:39Z | 2022-10-30T06:26:01Z | 2022-10-30T06:26:01Z | CONTRIBUTOR | 1. My https://list.saferdisinfectants.org/disinfectants/listN page (linked from https://SaferDisinfectants.org ) has been running beautifully for a year and a half, including a GitHub Actions workflow that's been routinely updating the database. 2. I received a recent report that the list page is down. I don't know when it went down, but the content is replaced with: "render_template() called before await ds.invoke_startup()" 3. The local datasette repo runs without incident. 4. The site is hosted on vercel, linked to my github repo. Perhaps some vercel changes were made, but not by anyone on our side. Here is a screenshot of the current project settings: <img width="877" alt="Screen Shot 2022-10-29 at 9 23 30 PM" src="https://user-images.githubusercontent.com/192568/198859474-341a5504-88b8-452d-bd06-e988cbe75e6c.png"> Here a screenshot of the latest deployment status: <img width="1351" alt="Screen Shot 2022-10-29 at 10 19 21 PM" src="https://user-images.githubusercontent.com/192568/198859641-ad3d99ba-a41d-4b98-88f6-e3a01fe9f238.png"> This is my repository: https://github.com/mroswell/list-N (I notice: datasette==0.59 in my requirements.txt file) Because it's been long while since I actively worked on this or any other datasette project, I forget a lot of what I knew at one point. Perhaps some configuration file could be missing? Or perhaps I just need to know the right incantation to add to that vercel settings page. Help is welcome as the nonprofit org is soon hosting its annual conference, and we'd love to have the page working again. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1872/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
843884745 | MDU6SXNzdWU4NDM4ODQ3NDU= | 1283 | advanced #export causes unexpected scrolling | 192568 | open | 0 | 0 | 2021-03-29T22:46:57Z | 2021-03-29T22:46:57Z | CONTRIBUTOR | 1. Visit a datasette table page 2. Click on the "(advanced)" link. This adds a fragment identifier "#export" to the URL, and scrolls down to the "Advanced export" div with the "export" id. 3. Manually scroll back up, and click on a suggested facet. The fragment identifier is still present, and the app scrolls back down to the "Advanced export" div. I think this is unwanted behavior. The user remedy seems to be to manually remove the "#export" from the URL. This behavior happens in my project, and in: https://covid-19.datasettes.com/covid/economist_excess_deaths (for instance) but not in this table: https://global-power-plants.datasettes.com/global-power-plants/global-power-plants | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1283/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1843600087 | I_kwDOBm6k_c5t4xrX | 2135 | Release notes for 1.0a3 | 9599 | closed | 0 | 9700784 | 3 | 2023-08-09T16:09:26Z | 2023-08-09T19:17:07Z | 2023-08-09T19:17:06Z | OWNER | 118 commits! https://github.com/simonw/datasette/compare/1.0a2...26be9f0445b753fb84c802c356b0791a72269f25 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/2135/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
315548495 | MDU6SXNzdWUzMTU1NDg0OTU= | 225 | /-/(inspect|metadata|plugins)(.json)? introspection | 9599 | closed | 0 | 0 | 2018-04-18T16:14:58Z | 2018-04-19T05:25:33Z | 2018-04-19T05:25:33Z | OWNER | 3 pages (and accompanying .json endpoints) for viewing: * the metadata.json that datasette was loaded with * the output of ds.inspect() * a list of installed plugins, detected by pluggy | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/225/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1178521513 | I_kwDOBm6k_c5GPs-p | 1682 | SQL queries against databases with different routes are broken | 9599 | closed | 0 | 1 | 2022-03-23T18:42:57Z | 2022-03-23T18:48:16Z | 2022-03-23T18:48:16Z | OWNER | 500 error on https://datasette-hashed-urls-preview.vercel.app/fixtures-09f8f95?sql=select+*+from+facetable Here's the trace: ``` File "/Users/simon/.local/share/virtualenvs/datasette-hashed-urls-ssI2fO50/lib/python3.10/site-packages/datasette/views/database.py", line 54, in data return await QueryView(self.ds).data( File "/Users/simon/.local/share/virtualenvs/datasette-hashed-urls-ssI2fO50/lib/python3.10/site-packages/datasette/views/database.py", line 232, in data self.ds.get_database(database), sql File "/Users/simon/.local/share/virtualenvs/datasette-hashed-urls-ssI2fO50/lib/python3.10/site-packages/datasette/app.py", line 401, in get_database return self.databases[name] KeyError: 'fixtures-aa7318b' ``` It looks like this is a Datasette bug, which is frustrating because I just shipped Datasette 0.61 five minutes ago! _Originally posted by @simonw in https://github.com/simonw/datasette-hashed-urls/issues/13#issuecomment-1076693667_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1682/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1453813400 | I_kwDOBm6k_c5Wp26Y | 1901 | Some plugins show "home" breadcrumbs twice in the top left | 95570 | closed | 0 | 8 | 2022-11-17T18:44:58Z | 2022-11-18T07:22:37Z | 2022-11-18T07:02:56Z | CONTRIBUTOR | <img width="1023" alt="Screenshot 2022-11-17 at 10 44 12 AM" src="https://user-images.githubusercontent.com/95570/202531505-b8632f70-d95a-425f-b2f2-6380b153478b.png"> | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1901/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
506432572 | MDU6SXNzdWU1MDY0MzI1NzI= | 21 | Fix & escapes in tweet text | 9599 | closed | 0 | 1 | 2019-10-14T03:37:28Z | 2019-10-15T18:48:16Z | 2019-10-15T18:48:16Z | MEMBER | <img width="1136" alt="twitter__tweets__21_773_rows_where_sorted_by_id_descending" src="https://user-images.githubusercontent.com/9599/66728360-38f91b80-edf9-11e9-95b5-ce6d097fe18e.png"> Shouldn't be storing `&` here. | 206156866 | issue | { "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/21/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1493404423 | I_kwDOBm6k_c5ZA4sH | 1948 | 500 error on permission debug page when testing actors with _r | 9599 | open | 0 | 1 | 2022-12-13T05:22:03Z | 2022-12-13T05:22:19Z | OWNER | <img width="1272" alt="image" src="https://user-images.githubusercontent.com/9599/207233390-ecb1b81b-c9f4-40a3-8131-5537ab23e9fc.png"> The 500 error is silent unless you are looking at the DevTools network pane. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1948/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
583970196 | MDU6SXNzdWU1ODM5NzAxOTY= | 701 | Search box CSS doesn't look great on OS X Safari | 9599 | closed | 0 | 5234079 | 3 | 2020-03-18T20:00:52Z | 2020-03-24T22:57:18Z | 2020-03-24T22:57:18Z | OWNER | <img width="1447" alt="twitter__tweets__68_773_rows_where_sorted_by_id_descending_and_twitter__tweets__955_rows" src="https://user-images.githubusercontent.com/9599/77002262-73bb2e80-6918-11ea-87a3-e3f537365c9f.png"> | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/701/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
493668862 | MDU6SXNzdWU0OTM2Njg4NjI= | 2 | Extract licenses from repos into a separate table | 9599 | closed | 0 | 0 | 2019-09-14T21:33:41Z | 2019-09-14T21:46:58Z | 2019-09-14T21:46:58Z | MEMBER | <img width="588" alt="starred3__repos__800_rows" src="https://user-images.githubusercontent.com/9599/64913845-a2074980-d6fc-11e9-815a-f103ad0c6f56.png"> | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/2/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
715072935 | MDU6SXNzdWU3MTUwNzI5MzU= | 993 | Column action menu should show column type | 9599 | closed | 0 | 5971510 | 2 | 2020-10-05T18:40:49Z | 2020-10-08T23:55:19Z | 2020-10-06T00:33:15Z | OWNER | <img width="616" alt="data__rent_burden_households_and_median_gross_rent_percent__97_rows" src="https://user-images.githubusercontent.com/9599/95118702-97509580-06ff-11eb-8523-39eb1b389acd.png"> | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/993/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
1068791148 | I_kwDOBm6k_c4_tHVs | 1540 | Idea: hover to reveal details of linked row | 9599 | open | 0 | 6 | 2021-12-01T19:28:07Z | 2021-12-09T23:38:39Z | OWNER | <img width="698" alt="fara__item_version__7_rows_where_where__item___5236" src="https://user-images.githubusercontent.com/9599/144300364-ee621811-8f5a-4c7e-9950-199119092cf3.png"> Hovering over that could work a little bit like GitHub issue links: ![hover](https://user-images.githubusercontent.com/9599/144300537-9cd9e9af-ac16-42db-842f-37661bc94063.gif) | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1540/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
464905894 | MDU6SXNzdWU0NjQ5MDU4OTQ= | 545 | Fix header on 404 page | 9599 | closed | 0 | 4471010 | 1 | 2019-07-07T01:47:40Z | 2019-07-07T20:26:55Z | 2019-07-07T20:26:55Z | OWNER | <img width="707" alt="Error_404" src="https://user-images.githubusercontent.com/9599/60762932-83127a00-a01e-11e9-853a-6ea6aa5e8f6d.png"> | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/545/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
607211058 | MDU6SXNzdWU2MDcyMTEwNTg= | 740 | Don't throw 500 error on attempted directory browse | 9599 | closed | 0 | 1 | 2020-04-27T03:50:11Z | 2020-04-27T18:29:15Z | 2020-04-27T18:29:15Z | OWNER | <img width="725" alt="Error_500" src="https://user-images.githubusercontent.com/9599/80332338-76eee780-87ff-11ea-85b3-32704d715cd9.png"> This should be a 403 error instead, because the `--static` mechanism doesn't allow directory browsing. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/740/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
489419782 | MDU6SXNzdWU0ODk0MTk3ODI= | 6 | Extract extended_entities into a media table | 9599 | closed | 0 | 0 | 2019-09-04T21:59:10Z | 2019-09-04T22:08:01Z | 2019-09-04T22:08:01Z | MEMBER | <img width="747" alt="cleo__select___from_tweets_order_by_id_limit_101" src="https://user-images.githubusercontent.com/9599/64294799-880f7f00-cf24-11e9-83b8-75c1ee54182c.png"> | 206156866 | issue | { "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/6/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1362402998 | I_kwDOBm6k_c5RNJ62 | 1802 | Tests reliably failing on Python 3.7 | 9599 | closed | 0 | 15 | 2022-09-05T19:21:16Z | 2022-09-06T00:40:20Z | 2022-09-06T00:40:20Z | OWNER | <img width="819" alt="image" src="https://user-images.githubusercontent.com/9599/188504608-5be0f941-dca2-4f42-83bc-3d04bbad221d.png"> https://github.com/simonw/datasette/runs/8194907739?check_suite_focus=true I thought this might be an intermittent failure but attempts to re-run the tests have not made it pass. End of that trace is: ``` /home/runner/work/datasette/datasette/datasette/app.py:234: in __init__ self._refresh_schemas_lock = asyncio.Lock() /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/locks.py:161: in __init__ self._loop = events.get_event_loop() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <asyncio.unix_events._UnixDefaultEventLoopPolicy object at 0x7fb1fc799fd0> def get_event_loop(self): """Get the event loop for the current context. Returns an instance of EventLoop or raises an exception. """ if (self._local._loop is None and not self._local._set_called and isinstance(threading.current_thread(), threading._MainThread)): self.set_event_loop(self.new_event_loop()) if self._local._loop is None: raise RuntimeError('There is no current event loop in thread %r.' > % threading.current_thread().name) E RuntimeError: There is no current event loop in thread 'MainThread'. ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1802/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
741021342 | MDU6SXNzdWU3NDEwMjEzNDI= | 1086 | Foreign keys with blank titles result in non-clickable links | 9599 | closed | 0 | 6055094 | 3 | 2020-11-11T19:41:09Z | 2020-11-28T23:28:29Z | 2020-11-11T23:46:20Z | OWNER | <img width="851" alt="index__core_snapshot_tags__3_rows_and_swarm__select_venues_name__checkins_id__createdAt__type__timeZoneOffset__editableUntil___like___isMayor__source__venue__createdBy__event__sticker__created__comments_count__entities__shout__checkins_priv" src="https://user-images.githubusercontent.com/9599/98856656-bafbaf80-2412-11eb-8070-4f3a47511973.png"> The HTML looks like this: ```html <td class="col-tag_id type-int"><a href="/index/core_tag/1"></a> <em>1</em></td> ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1086/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
1795051447 | I_kwDOBm6k_c5q_k-3 | 2097 | Drop Python 3.7 | 9599 | closed | 0 | 0 | 2023-07-08T18:39:44Z | 2023-08-23T18:18:00Z | 2023-08-23T18:18:00Z | OWNER | > I'm going to drop Python 3.7. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1153#issuecomment-1627455892_ It's not supported any more: https://devguide.python.org/versions/ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/2097/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1452364777 | I_kwDOBm6k_c5WkVPp | 1896 | Extract logic for resolving a URL to a database / table / row | 9599 | closed | 0 | 8658075 | 4 | 2022-11-16T22:25:20Z | 2022-11-18T22:57:47Z | 2022-11-18T22:56:55Z | OWNER | > In trying to write this I realize that there's a lot of duplicated code with delete row, specifically around resolving the incoming URL into a row (or a database or a table). > > Since this is so common, I think it's worth extracting the logic out first. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1863#issuecomment-1317755263_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1896/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
1435294468 | I_kwDOBm6k_c5VjNsE | 1882 | `/db/-/create` API for creating tables | 9599 | closed | 0 | 8658075 | 12 | 2022-11-03T21:44:32Z | 2022-11-15T19:59:43Z | 2022-11-15T06:00:41Z | OWNER | > It really feels like this should be accompanied by a `/db/-/create` API for creating tables. I had to add that to `sqlite-utils` eventually (initially it only supported creating by passing in an example document): > > https://sqlite-utils.datasette.io/en/stable/cli.html#cli-create-table _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1862#issuecomment-1299073433_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1882/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
1386734383 | I_kwDOBm6k_c5Sp-Mv | 1821 | Release Datasette 0.63a0 | 9599 | closed | 0 | 1 | 2022-09-26T21:15:27Z | 2022-09-26T22:06:39Z | 2022-09-26T22:06:39Z | OWNER | > - The [prepare_jinja2_environment(env, datasette)](https://docs.datasette.io/en/latest/plugin_hooks.html#plugin-hook-prepare-jinja2-environment) plugin hook now accepts an optional `datasette` argument. Hook implementations can also now return an `async` function which will be awaited automatically. ([#1809](https://github.com/simonw/datasette/issues/1809)) > - `--load-extension` option now supports entrypoints. Thanks, Alex Garcia. ([#1789](https://github.com/simonw/datasette/pull/1789)) > - New tutorial: [Cleaning data with sqlite-utils and Datasette](https://datasette.io/tutorials/clean-data). > - Facet size can now be set per-table with the new `facet_size` table metadata option. ([#1804](https://github.com/simonw/datasette/issues/1804)) > - `truncate_cells_html` setting now also affects long URLs in columns. ([#1805](https://github.com/simonw/datasette/issues/1805)) > - `Database(is_mutable=)` now defaults to `True`. ([#1808](https://github.com/simonw/datasette/issues/1808)) > - Non-JavaScript textarea now increases height to fit the SQL query. ([#1786](https://github.com/simonw/datasette/issues/1786)) > - More detailed command descriptions on the [CLI reference](https://docs.datasette.io/en/latest/cli-reference.html#cli-reference) page. ([#1787](https://github.com/simonw/datasette/issues/1787)) > - Datasette no longer enforces upper bounds on its depenedencies. ([#1800](https://github.com/simonw/datasette/issues/1800)) > - Facets are now displayed with better line-breaks in long values. Thanks, Daniel Rech. ([#1794](https://github.com/simonw/datasette/pull/1794)) > - The `settings.json` file used in [Configuration directory mode](https://docs.datasette.io/en/latest/settings.html#config-dir) is now validated on startup. ([#1816](https://github.com/simonw/datasette/issues/1816)) | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1821/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1786243905 | I_kwDOCGYnMM5qd-tB | 564 | Document that running `db.transform()` tidies up the schema indentation | 9599 | closed | 0 | 0 | 2023-07-03T13:59:28Z | 2023-07-22T22:15:34Z | 2023-07-22T22:15:34Z | OWNER | > ... and it turns out running `.transform()` with no arguments still fixes the format of the schema! ```pycon >>> db["log"].add_column("foo", str) <Table log (id, name2, age, weight, foo)> >>> db["log"].add_column("bar", str) <Table log (id, name2, age, weight, foo, bar)> >>> db["log"].add_column("baz", str) <Table log (id, name2, age, weight, foo, bar, baz)> >>> print(db["log"].schema) CREATE TABLE "log" ( [id] INTEGER PRIMARY KEY, [name2] TEXT, [age] INTEGER, [weight] FLOAT , [foo] TEXT, [bar] TEXT, [baz] TEXT) >>> db["log"].transform() <Table log (id, name2, age, weight, foo, bar, baz)> >>> print(db["log"].schema) CREATE TABLE "log" ( [id] INTEGER PRIMARY KEY, [name2] TEXT, [age] INTEGER, [weight] FLOAT, [foo] TEXT, [bar] TEXT, [baz] TEXT ) ``` _Originally posted by @simonw in https://github.com/simonw/llm/issues/65#issuecomment-1618347727_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/564/reactions", "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 1 } |
completed | ||||||
1174302994 | I_kwDOBm6k_c5F_nES | 1667 | Make route matched pattern groups more consistent | 9599 | closed | 0 | 3268330 | 3 | 2022-03-19T16:32:35Z | 2022-03-19T20:37:42Z | 2022-03-19T20:37:41Z | OWNER | > ... highlights how inconsistent the way the capturing works is. Especially `as_format` which can be `None` or `""` or `.json` or `json` or not used at all in the case of `TableView`. https://github.com/simonw/datasette/blob/764738dfcb16cd98b0987d443f59d5baa9d3c332/tests/test_routes.py#L12-L36 _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1666#issuecomment-1073039670_ Part of: - #1660 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1667/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
705840673 | MDU6SXNzdWU3MDU4NDA2NzM= | 972 | Support faceting against arbitrary SQL queries | 9599 | open | 0 | 1 | 2020-09-21T19:00:43Z | 2021-12-15T18:02:20Z | OWNER | > ... support for running facets against arbitrary custom SQL queries is half-done in that facets now execute against wrapped subqueries as-of ea66c45df96479ef66a89caa71fff1a97a862646 > > https://github.com/simonw/datasette/blob/ea66c45df96479ef66a89caa71fff1a97a862646/datasette/facets.py#L192-L200 _Originally posted by @simonw in https://github.com/simonw/datasette/issues/971#issuecomment-696307922_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/972/reactions", "total_count": 3, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 3, "rocket": 0, "eyes": 0 } |
||||||||
1907765514 | I_kwDOBm6k_c5xtjEK | 2195 | `datasette publish` needs support for the new config/metadata split | 9599 | open | 0 | 9 | 2023-09-21T21:08:12Z | 2023-09-21T22:57:48Z | OWNER | > ... which raises the challenge that `datasette publish` doesn't yet know what to do with a config file! _Originally posted by @simonw in https://github.com/simonw/datasette/issues/2194#issuecomment-1730259871_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/2195/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1097087280 | I_kwDOCGYnMM5BZDkw | 368 | Offer `python -m sqlite_utils` as an alternative to `sqlite-utils` | 9599 | closed | 0 | 7558727 | 3 | 2022-01-09T02:29:30Z | 2022-01-10T19:27:20Z | 2022-01-09T02:40:50Z | OWNER | > Add this to `sqlite_utils/cli.py`: > > ```python > if __name__ == "__main__": > cli() > ``` > Now the tool can be run using `python -m sqlite_utils.cli --help` _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/364#issuecomment-1008214998_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/368/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
965210966 | MDU6SXNzdWU5NjUyMTA5NjY= | 314 | Type signatures for `.create_table()` and `.create_table_sql()` and `.create()` and `Table.__init__` | 9599 | closed | 0 | 2 | 2021-08-10T18:03:59Z | 2021-08-18T22:25:21Z | 2021-08-18T22:25:21Z | OWNER | > Adding type signatures to `create_table()` and `.create_table_sql()` is a bit too involved, I'll do that in a separate issue. _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/312#issuecomment-896200682_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/314/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1784794489 | I_kwDOCGYnMM5qYc15 | 562 | Explore the intersection between sqlite-utils and dataclasses | 9599 | open | 0 | 1 | 2023-07-02T19:23:08Z | 2023-07-02T19:26:39Z | OWNER | > Aside: this makes me think it might be cool if `sqlite-utils` had a way of working with dataclasses rather than just dicts, and knew how to create a SQLite table to match a dataclass and maybe how to code-generate dataclasses for a specific table schema (dynamically or even using code-generation that can be written to disk, for better editor integrations). _Originally posted by @simonw in https://github.com/simonw/llm/issues/65#issuecomment-1616742529_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/562/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
972918533 | MDU6SXNzdWU5NzI5MTg1MzM= | 1438 | Query page .csv and .json links are not correctly URL-encoded on Vercel under unknown specific conditions | 9599 | open | 0 | 7 | 2021-08-17T17:35:36Z | 2021-08-18T00:22:23Z | OWNER | > Confirmed: https://thesession.vercel.app/thesession?sql=select+*+from+tunes+where+name+like+%22%25wise+maid%25%22%0D%0A is a page where the URL correctly encoded `%` as `%25` - but then in the HTML on that page that links to the CSV and JSON versions we get this: > > ```html > <p class="export-links">This data as > <a href="/thesession.json?sql=select * from tunes where name like "%wise maid%"">json</a>, > <a href="/thesession.csv?sql=select * from tunes where name like "%wise maid%"&_size=max">CSV</a> > </p> > ``` Those CSV and JSON links are incorrect. _Originally posted by @simonw in https://github.com/simonw/datasette-publish-vercel/issues/48#issuecomment-900497579_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1438/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1114640101 | I_kwDOCGYnMM5CcA7l | 392 | `sqlite-utils bulk --batch-size` option | 9599 | closed | 0 | 4 | 2022-01-26T05:17:11Z | 2022-01-26T18:17:59Z | 2022-01-26T18:17:59Z | OWNER | > Could add support for `--batch-size` as seen in `insert`/`upsert` too - causing it to break the list up into batches and commit for each one. _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/391#issuecomment-1021876055_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/392/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1433576351 | I_kwDOBm6k_c5VcqOf | 1880 | Datasette with many and large databases > Memory use | 525934 | open | 0 | 4 | 2022-11-02T18:10:27Z | 2022-11-16T17:50:29Z | NONE | > Datasette maintains an in-memory SQLite database with details of the the databases, tables and columns for all of the attached databases. The above is from the docs ^. There's two problems here - the number of datasette "instances" in a single server/VM and the size of the database itself. We want the **opposite** of in-memory, including what happens on SQLlite - documented in https://www.sqlite.org/inmemorydb.html From the context in https://github.com/simonw/datasette/issues/1150 - does it mean datasette is memory-bound to the size of the dataset - which might be a deal-breaker for many large-scale use cases? In an extreme case - let's say a single server had 100 SQLlite databases, which would enable 100 "instances" of datasette to run, one per client (e.g. in a SaaS multi-tenant environment). How could we achieve all these goals: 1. Allow any _one_ of these 100 databases to grow to say 2Tb in size 2. Have one datasette instance, which connects to 1 of the 100 instances, based on incoming credentials/tenant ID 3. Minimize memory use entirely - both by datasette and SQLlite, such that almost all operations are executed in real-time on-disk with little to no memory consumption per-tenant, or per-database. Any ideas appreciated - we're looking to use this in a SaaS type of setting - many instances, single server. @simonw great work on datasette, in general! Possibly related to https://github.com/simonw/datasette/issues/1480 but we don't want use any kind of serverless infra - this is a long-running VM/server. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1880/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
628121234 | MDU6SXNzdWU2MjgxMjEyMzQ= | 788 | /-/permissions debugging tool | 9599 | closed | 0 | 5512395 | 2 | 2020-06-01T03:13:47Z | 2020-06-06T00:43:40Z | 2020-06-01T05:01:01Z | OWNER | > Debugging tool idea: `/-/permissions` page which shows you the actor and lets you type in the strings for `action`, `resource_type` and `resource_identifier` - then shows you EVERY plugin hook that would have executed and what it would have said, plus when the chain would have terminated. > > Bonus: if you're logged in as the `root` user (or a user that matches some kind of permission check, maybe a check for `permissions_debug`) you get to see a rolling log of the last 30 permission checks and what the results were across the whole of Datasette. This should make figuring out permissions policies a whole lot easier. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/699#issuecomment-636576603_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/788/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed |