github
id | node_id | number | title | user | state | locked | assignee | milestone | comments | created_at | updated_at | closed_at | author_association | pull_request | body | repo | type | active_lock_reason | performed_via_github_app | reactions | draft | state_reason |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
988493790 | MDExOlB1bGxSZXF1ZXN0NzI3MzkwODM1 | 36 | Correct naming of tool in readme | 2129 | open | 0 | 1 | 2021-09-05T12:05:40Z | 2022-01-06T16:04:46Z | FIRST_TIME_CONTRIBUTOR | dogsheep/dogsheep-photos/pulls/36 | 256834907 | pull | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/36/reactions", "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||||
1094974713 | I_kwDOCGYnMM5BQ_z5 | 362 | upsert --detect-types is broken | 9599 | closed | 0 | 0 | 2022-01-06T05:12:10Z | 2022-01-06T06:54:45Z | 2022-01-06T06:28:34Z | OWNER | Noticed this thanks to syntax highlighting in VS Code showing an unused variable - need to fix it and add a test. <img src="https://user-images.githubusercontent.com/9599/148331804-6295bcf5-01fd-4159-b8ad-60c82acb97b4.png" width="200"> | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/362/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1094890366 | PR_kwDOCGYnMM4wlm3B | 361 | --lines and --text and --convert and --import | 9599 | closed | 0 | 15 | 2022-01-06T01:49:44Z | 2022-01-06T06:37:03Z | 2022-01-06T06:24:54Z | OWNER | simonw/sqlite-utils/pulls/361 | Refs #356 Still TODO: - [x] Get `--lines` working, with tests - [x] Get `--text` working, with tests - [x] Get regular JSON import working with `--convert` with tests - [x] Get `--lines` working with `--convert` with tests - [x] Get `--text` working with `--convert` with tests - [x] Get `--csv` and `--tsv` import working with `--convert` with tests - [x] Get `--nl` working with `--convert` with tests - [x] Documentation for all of the above | 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/361/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1077431957 | I_kwDOCGYnMM5AOE6V | 356 | `sqlite-utils insert --convert` option | 9599 | closed | 0 | 11 | 2021-12-11T07:24:48Z | 2022-01-06T06:30:13Z | 2022-01-06T06:28:53Z | OWNER | Idea come to me while re-reading this: https://simonwillison.net/2021/Aug/6/sqlite-utils-convert/ This is a bit of a hack: ``` cat /tmp/log.txt | \ jq --raw-input '{line: .}' --compact-output | \ sqlite-utils insert /tmp/logs.db log - --nl ``` Would be great if you could pipe lines to `insert` and transform them on the way in. A `--convert python-code` option, modeled after `sqlite-utils convert`, could do this. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/356/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1091850530 | I_kwDODEm0Qs5BFFEi | 63 | Import archive error 'withheld_in_countries' | 521097 | open | 0 | 0 | 2022-01-01T16:58:59Z | 2022-01-01T16:58:59Z | NONE | Importing the twitter archive I received this error: ```bash $ twitter-to-sqlite import archive.db twitter-2021-12-31-<hash>.zip birdwatch-note-rating: not yet implemented birdwatch-note: not yet implemented branch-links: not yet implemented community-tweet: not yet implemented contact: not yet implemented device-token: not yet implemented direct-message-mute: not yet implemented mute: not yet implemented periscope-account-information: not yet implemented periscope-ban-information: not yet implemented periscope-broadcast-metadata: not yet implemented periscope-comments-made-by-user: not yet implemented periscope-expired-broadcasts: not yet implemented periscope-followers: not yet implemented periscope-profile-description: not yet implemented professional-data: not yet implemented protected-history: not yet implemented reply-prompt: not yet implemented screen-name-change: not yet implemented smartblock: not yet implemented spaces-metadata: not yet implemented sso: not yet implemented Traceback (most recent call last): File "/home/paulox/.virtualenvs/dogsheep/bin/twitter-to-sqlite", line 8, in <module> sys.exit(cli()) File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/click/core.py", line 1128, in __call__ return self.main(*args, **kwargs) File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/click/core.py", line 1053, in main rv = self.invoke(ctx) File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/click/core.py", line 1659, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/click/core.py", line 1395, in invoke return ctx.invoke(self.callback, **ctx.params) File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/click/core.py", line 754, in invoke return __callback(*args, **kwargs) File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/twitter_to_sqlite/cli.py", line 759, in impo… | 206156866 | issue | { "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/63/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1091838742 | I_kwDOBm6k_c5BFCMW | 1585 | Fire base caching for `publish cloudrun` | 9599 | open | 0 | 1 | 2022-01-01T15:38:15Z | 2022-01-01T15:40:38Z | OWNER | https://gist.github.com/steren/03d3e58c58c9a53fd49bb78f58541872 has a recipe for this, via https://twitter.com/steren/status/1477038411114446848 Could this enable easier vanity URLs of the format `https://$project_id.web.app/`? How about CDN caching? | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1585/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1091257796 | I_kwDOBm6k_c5BC0XE | 1584 | give error with recursive sql | 58088336 | open | 0 | 0 | 2021-12-30T18:53:16Z | 2021-12-30T18:53:16Z | NONE | I got an error "near "WITH": syntax error" after I upgraded to version 0.59 from 0.52.4. This error is related to recursive sql. It works great on the previous version but it failed after upgraded. Below is an example of sql: WITH RECURSIVE manager_of(position, super_position) AS (SELECT position, case ifnull(INDIRECT_SUPER_POSITION,'') when '' then super_position else INDIRECT_SUPER_POSITION end as SUPER_POSITION FROM position where super_position<>'SGV000000001' and super_position!='' and position <> super_position),chain_manager_of_position(position, level) AS (SELECT super_position, 1 as level FROM manager_of WHERE super_position!='' and (position=:pos or position in (Select position from employee where employee=:ein)) UNION ALL SELECT super_position, level+1 as level FROM manager_of JOIN chain_manager_of_position USING(position)) SELECT * FROM chain_manager_of_position left join employee using(position) where employee is not NULL order by level limit 1 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1584/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1090810196 | I_kwDOBm6k_c5BBHFU | 1583 | consider adding deletion step of cloudbuild artifacts to gcloud publish | 536941 | open | 0 | 1 | 2021-12-30T00:33:23Z | 2021-12-30T00:34:16Z | CONTRIBUTOR | right now, as part of the the publish process images and other artifacts are stored to gcloud's cloud storage before being deployed to cloudrun. after successfully deploying, it would be nice if the the script deleted these artifacts. otherwise, if you have regularly scheduled build process, you can end up paying to store lots of out of date artifacts. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1583/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1090798237 | I_kwDOCGYnMM5BBEKd | 359 | Use RETURNING if available to populate last_pk | 9599 | open | 0 | 0 | 2021-12-29T23:43:23Z | 2021-12-29T23:43:23Z | OWNER | Inspired by this: https://news.ycombinator.com/item?id=29729283 > Because SQLite is effectively serializing all the writes for us, we have zero locking in our code. We used to have to lock when inserting new items (to get the LastInsertRowId), but the newer version of SQLite supports the RETURNING keyword, so we don't even have to lock on inserts now. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/359/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
770598024 | MDU6SXNzdWU3NzA1OTgwMjQ= | 1152 | Efficiently calculate list of databases/tables a user can view | 9599 | open | 0 | 12 | 2020-12-18T06:13:01Z | 2021-12-27T23:04:31Z | OWNER | > The homepage currently performs a massive flurry of permission checks - one for each, database, table and view: https://github.com/simonw/datasette/blob/0.53/datasette/views/index.py#L21-L75 > > A paginated version of this is a little daunting as the permission checks would have to be carried out in every single table just to calculate the count that will be paginated. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1150#issuecomment-747864831_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1152/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
807437089 | MDU6SXNzdWU4MDc0MzcwODk= | 228 | --no-headers option for CSV and TSV | 9599 | closed | 0 | 10 | 2021-02-12T17:56:51Z | 2021-12-26T07:01:31Z | 2021-02-14T22:25:17Z | OWNER | https://bl.iro.bl.uk/work/ns/3037474a-761c-456d-a00c-9ef3c6773f4c has a fascinating CSV file that doesn't have a header row - it starts like this: ```csv Computation and measurement of turbulent flow through idealized turbine blade passages,,"Loizou, Panos A.",https://isni.org/isni/0000000136122593,,University of Manchester,https://isni.org/isni/0000000121662407,1989,Thesis (Ph.D.),,Physical Sciences,,,https://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.232781, "Prolactin and growth hormone secretion in normal, hyperprolactinaemic and acromegalic man",,"Prescott, R. W. G.",https://isni.org/isni/0000000134992122,,University of Newcastle upon Tyne,https://isni.org/isni/0000000104627212,1983,Thesis (Ph.D.),,Biological Sciences,,,https://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.232784, ``` It would be useful if `sqlite-utils insert ... --csv` had a mechanism for importing files like this one. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/228/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1087919372 | I_kwDOBm6k_c5A2FUM | 1578 | Confirm if documented nginx proxy config works for row pages with escaped characters in their primary key | 9599 | open | 0 | 4 | 2021-12-23T18:27:59Z | 2021-12-24T21:33:19Z | OWNER | Found this while working on https://github.com/simonw/datasette-tiddlywiki <img width="1254" alt="image" src="https://user-images.githubusercontent.com/9599/147279097-e02f80f3-cc88-4bdd-a26f-03f924c13b5e.png"> Then clicking on `/tiddlywiki/tiddlers/%24%3A%2FDefaultTiddlers` returns a 404. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1578/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
781262510 | MDU6SXNzdWU3ODEyNjI1MTA= | 1181 | Certain database names results in 404: "Database not found: None" | 1470389 | closed | 0 | 6346396 | 4 | 2021-01-07T12:01:16Z | 2021-12-21T18:25:15Z | 2021-01-25T05:13:19Z | NONE | I have a file named `test-database (1).sqlite`. When requesting the home route `/`, I see datasette is able to read it correctly: <img width="409" alt="Screenshot 2021-01-07 at 12 54 46" src="https://user-images.githubusercontent.com/1470389/103889964-9442b800-50e7-11eb-9548-fbf10ed3b1ea.png"> However, if I click any of the links, datasette replies with: `Error 404 Database not found: None` It seems the hash is crucial, as renaming the file to `database (1).sqlite` makes the error go away. This lines checks for a single dash: https://github.com/simonw/datasette/blob/97fb10c17dd007a275ab743742e93e932335ad67/datasette/views/base.py#L184 ``` $ datasette test-database\ \(1\).sqlite INFO: Started server process [68314] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://127.0.0.1:8001 (Press CTRL+C to quit) INFO: 127.0.0.1:54043 - "GET /favicon.ico HTTP/1.1" 200 OK INFO: 127.0.0.1:54043 - "GET / HTTP/1.1" 200 OK ... INFO: 127.0.0.1:54044 - "GET /favicon.ico HTTP/1.1" 200 OK INFO: 127.0.0.1:54044 - "GET /test-database (1) HTTP/1.1" 404 Not Found ``` Version: ``` $ datasette --version datasette, version 0.53 ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1181/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
1079129258 | PR_kwDOBm6k_c4vynly | 1554 | TableView refactor | 9599 | closed | 0 | 3 | 2021-12-13T23:16:04Z | 2021-12-20T23:52:11Z | 2021-12-20T23:52:04Z | OWNER | simonw/datasette/pulls/1554 | I'm starting a PR with almost nothing in it so I can use the GitHub code commenting feature to add a bunch of comments to the code I intend to refactor. Related issues: - #617 - #715 - #870 - #1518 | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1554/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1084257842 | I_kwDOBm6k_c5AoHYy | 1575 | __call__() got an unexpected keyword argument 'specname' | 9599 | closed | 0 | 1 | 2021-12-20T01:24:04Z | 2021-12-20T01:48:03Z | 2021-12-20T01:47:57Z | OWNER | > I've installed the alpha version but get an error when starting up Datasette: ``` Traceback (most recent call last): File "/Users/tim/.pyenv/versions/stock-exchange/bin/datasette", line 5, in <module> from datasette.cli import cli File "/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/cli.py", line 15, in <module> from .app import Datasette, DEFAULT_SETTINGS, SETTINGS, SQLITE_LIMIT_ATTACHED, pm File "/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/app.py", line 31, in <module> from .views.database import DatabaseDownload, DatabaseView File "/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/views/database.py", line 25, in <module> from datasette.plugins import pm File "/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/plugins.py", line 29, in <module> mod = importlib.import_module(plugin) File "/Users/tim/.pyenv/versions/3.8.5/lib/python3.8/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/filters.py", line 9, in <module> @hookimpl(specname="filters_from_request") TypeError: __call__() got an unexpected keyword argument 'specname' ``` _Originally posted by @wragge in https://github.com/simonw/datasette/issues/1547#issuecomment-997511968_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1575/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
706001517 | MDU6SXNzdWU3MDYwMDE1MTc= | 163 | Idea: conversions= could take Python functions | 9599 | open | 0 | 4 | 2020-09-22T00:37:12Z | 2021-12-20T00:56:52Z | OWNER | Right now you use `conversions=` like this: ```python db["example"].insert({ "name": "The Bigfoot Discovery Museum" }, conversions={"name": "upper(?)"}) ``` How about if you could optionally provide a Python function (or a lambda) like this? ```python db["example"].insert({ "name": "The Bigfoot Discovery Museum" }, conversions={"name": lambda s: s.upper()}) ``` This would work by creating a random name for that function, registering it (similar to #162), executing the SQL and then un-registering the custom function at the end. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/163/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1083657868 | I_kwDOBm6k_c5Al06M | 1565 | Documented JavaScript variables on different templates made available for plugins | 9599 | open | 0 | 8 | 2021-12-17T22:30:51Z | 2021-12-19T22:37:29Z | OWNER | While working on https://github.com/simonw/datasette-leaflet-freedraw/issues/10 I found myself writing this atrocity to figure out the SQL query used for a specific table page: ```javascript let innerSql = Array.from(document.getElementsByTagName("span")).filter( el => el.innerText == "View and edit SQL" )[0].parentElement.getAttribute("title") ``` This is obviously bad - it's very brittle, and will break if I ever change the text on that link (like localizing it for example). Instead, I think pages like that one should have a block of script at the bottom something like this: ```javascript window.datasette = window.datasette || {}; datasette.view_name = 'table'; datasette.table_sql = 'select * from ...'; ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1565/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1084185188 | I_kwDOBm6k_c5An1pk | 1573 | Make trace() a documented internal API | 9599 | open | 0 | 1 | 2021-12-19T20:32:56Z | 2021-12-19T21:13:13Z | OWNER | This should be documented so plugin authors can use it to add their own custom traces: https://github.com/simonw/datasette/blob/8f311d6c1d9f73f4ec643009767749c17b5ca5dd/datasette/tracer.py#L28-L52 Including the new `kwargs` pattern I added in #1571: https://github.com/simonw/datasette/blob/f65817000fdf87ce8a0c23edc40784ebe33b5842/datasette/database.py#L128-L132 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1573/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1076057610 | I_kwDOBm6k_c5AI1YK | 1546 | validating the sql | 50336793 | closed | 0 | 1 | 2021-12-09T21:35:57Z | 2021-12-18T02:05:17Z | 2021-12-18T02:05:16Z | NONE | Could someone tell me that part of the code is responsible for validating the sql that guarantees that only a table can be read | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1546/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
520681725 | MDU6SXNzdWU1MjA2ODE3MjU= | 621 | Syntax for ?_through= that works as a form field | 9599 | open | 0 | 7 | 2019-11-11T00:19:03Z | 2021-12-18T01:42:33Z | OWNER | The current syntax for `?_through=` uses JSON to avoid any risk of confusion with table or column names that contain special characters. This means you can't target a form field at it. We should be able to support both - `?x.y.z=value` for tables and columns with "regular" names, falling back to the current JSON syntax for columns or tables that won't work with the key/value syntax. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/621/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
519613116 | MDU6SXNzdWU1MTk2MTMxMTY= | 617 | Refactor TableView.data() method | 9599 | closed | 0 | 9 | 2019-11-08T01:55:41Z | 2021-12-18T01:41:47Z | 2021-12-11T19:17:11Z | OWNER | This is by far the most complex piece of Datasette - the `TableView.data()` method is over 500 lines long and is increasingly getting in the way of cleanly implementing new features (e.g. #615 and #613). Need to break it up into smaller, cleaner pieces. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/617/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
445850934 | MDU6SXNzdWU0NDU4NTA5MzQ= | 473 | Plugin hook: filters_from_request | 9599 | closed | 0 | 13 | 2019-05-19T18:44:33Z | 2021-12-17T23:11:30Z | 2021-12-17T19:02:17Z | OWNER | I meant to add this as part of the facets plugin mechanism but didn't quite get to it. Original idea was to allow plugins to register extra filters, as seen in `datasette/filters.py`: https://github.com/simonw/datasette/blob/260085838887ee343f4d3b177c422e7aef5ade9d/datasette/filters.py#L83-L98 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/473/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1082743068 | PR_kwDOBm6k_c4v-izc | 1559 | filters_from_request plugin hook, now used in TableView | 9599 | closed | 0 | 6 | 2021-12-16T23:59:33Z | 2021-12-17T23:09:41Z | 2021-12-17T19:02:15Z | OWNER | simonw/datasette/pulls/1559 | New plugin hook, refs #473 Used it to extract the logic from TableView that handles _search and _through and _where - refs #1518 | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1559/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1083246400 | PR_kwDOBm6k_c4wAMK8 | 1562 | Update janus requirement from <0.8,>=0.6.2 to >=0.6.2,<1.1 | 49699333 | closed | 0 | 4 | 2021-12-17T13:11:10Z | 2021-12-17T23:08:29Z | 2021-12-17T23:08:28Z | CONTRIBUTOR | simonw/datasette/pulls/1562 | Updates the requirements on [janus](https://github.com/aio-libs/janus) to permit the latest version. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/aio-libs/janus/releases">janus's releases</a>.</em></p> <blockquote> <h2>janus 1.0.0 release</h2> <ul> <li>Dropped Python 3.6 support</li> <li>Janus is marked as stable, no API changes was made for years</li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/aio-libs/janus/blob/master/CHANGES.rst">janus's changelog</a>.</em></p> <blockquote> <h2>1.0.0 (2021-12-17)</h2> <ul> <li>Drop Python 3.6 support</li> </ul> <h2>0.7.0 (2021-11-24)</h2> <ul> <li>Add SyncQueue and AsyncQueue Protocols to provide type hints for sync and async queues <a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/374">#374</a></li> </ul> <h2>0.6.2 (2021-10-24)</h2> <ul> <li>Fix Python 3.10 compatibility <a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/358">#358</a></li> </ul> <h2>0.6.1 (2020-10-26)</h2> <ul> <li> <p>Raise RuntimeError on queue.join() after queue closing. <a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/295">#295</a></p> </li> <li> <p>Replace <code>timeout</code> type from <code>Optional[int]</code> to <code>Optional[float]</code> <a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/267">#267</a></p> </li> </ul> <h2>0.6.0 (2020-10-10)</h2> <ul> <li> <p>Drop Python 3.5, the minimal supported version is Python 3.6</p> </li> <li> <p>Support Python 3.9</p> </li> <li> <p>Refomat with <code>black</code></p> </li> </ul> <h2>0.5.0 (2020-04-23)</h2> <ul> <li>Remove explicit loop arguments and forbid creating queues outside event loops <a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/246">#246</a></li> </ul> <h2>0.4.0 (2018-07-28)</h2> <ul> <li> <p>Add <code>py.typed</code> macro <a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/89">#89</a></p> </li> <li> <p>Drop p… | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1562/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
793002853 | MDExOlB1bGxSZXF1ZXN0NTYwNzYwMTQ1 | 1204 | WIP: Plugin includes | 9599 | open | 0 | 3 | 2021-01-25T03:59:06Z | 2021-12-17T07:10:49Z | OWNER | simonw/datasette/pulls/1204 | Refs #1191 Next steps: - [ ] Get comfortable that this pattern is the right way to go - [ ] Implement it for all of the other pages, not just the table page - [ ] Add a new set of plugin tests that exercise ALL of these new hook locations - [ ] Document, then ship | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1204/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1 | ||||||
1079422215 | I_kwDOCGYnMM5AVq0H | 357 | pytest-runner is not required | 4067843 | closed | 0 | 1 | 2021-12-14T07:51:24Z | 2021-12-16T20:43:19Z | 2021-12-16T20:43:13Z | NONE | Deprecated pytest-runner is not necessary for running the testsuite. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/357/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
636511683 | MDU6SXNzdWU2MzY1MTE2ODM= | 830 | Redesign register_facet_classes plugin hook | 9599 | open | 0 | 3268330 | 3 | 2020-06-10T20:03:27Z | 2021-12-16T19:58:22Z | OWNER | Nothing uses this plugin hook yet, so the design is not yet proven. I'm going to build a real plugin against it and use that process to inform any design changes that may need to be made. I'll add a warning about this to the documentation. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/830/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
1079111498 | I_kwDOBm6k_c5AUe9K | 1553 | if csv export is truncated in non streaming mode set informative response header | 536941 | open | 0 | 3 | 2021-12-13T22:50:44Z | 2021-12-16T19:17:28Z | CONTRIBUTOR | streaming mode is currently not enabled for custom queries, so the queries will be truncated to max row limit. it would be great if a response is truncated that an header signalling that was set in the header. i need to write some pagination code for getting full results back for a custom query and it would make the code much better if i could reliably known when there is nothing more to limit/offset | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1553/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
741231849 | MDU6SXNzdWU3NDEyMzE4NDk= | 1087 | Idea: ?_extra=urls for getting back URLs to useful things | 9599 | open | 0 | 0 | 2020-11-12T02:55:41Z | 2021-12-15T18:06:16Z | OWNER | Working on https://github.com/simonw/datasette-search-all/issues/10 made me realize that sometimes it can be difficult to calculate the URL for a database, table or row within Datasette. It would be useful to have an optional extra JSON extension (using `?_extra=` from #262) that can help with this. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1087/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
705840673 | MDU6SXNzdWU3MDU4NDA2NzM= | 972 | Support faceting against arbitrary SQL queries | 9599 | open | 0 | 1 | 2020-09-21T19:00:43Z | 2021-12-15T18:02:20Z | OWNER | > ... support for running facets against arbitrary custom SQL queries is half-done in that facets now execute against wrapped subqueries as-of ea66c45df96479ef66a89caa71fff1a97a862646 > > https://github.com/simonw/datasette/blob/ea66c45df96479ef66a89caa71fff1a97a862646/datasette/facets.py#L192-L200 _Originally posted by @simonw in https://github.com/simonw/datasette/issues/971#issuecomment-696307922_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/972/reactions", "total_count": 3, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 3, "rocket": 0, "eyes": 0 } |
||||||||
962391325 | MDU6SXNzdWU5NjIzOTEzMjU= | 1423 | Show count of facet values if ?_facet_size=max | 9599 | closed | 0 | 9 | 2021-08-06T04:42:20Z | 2021-12-15T17:48:40Z | 2021-08-16T18:56:43Z | OWNER | I sometimes want to get a count of the values in a facet - if it's a facet of US states for example I want to know if all 50 are represented. Idea: if `?_facet_size=max` is present, add a count to the facet heading. So on: https://latest.datasette.io/fixtures/compound_three_primary_keys?_facet=content&_facet_size=max&_facet=pk1&_facet=pk2#facet-pk2 It could have something like this: <img width="658" alt="fixtures__compound_three_primary_keys__1_001_rows_and_NY_629_755_•_Untitled-1_—_datasette" src="https://user-images.githubusercontent.com/9599/128456981-1f3d5348-90f1-45cc-9287-0f7bb0015257.png"> Note that the first column shows >1000 - because in that case we've truncated the facet calculation since the maximum allowed returned rows is 1000. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1423/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1072106103 | I_kwDOBm6k_c4_5wp3 | 1542 | feature request: order and dependency of plugins (that use js) | 33631 | open | 0 | 1 | 2021-12-06T12:40:45Z | 2021-12-15T17:47:08Z | NONE | I have been playing with datasette for the last couple of weeks and it is great! I am a big fan of `datasette-cluster-map` and wanted to enhance it a bit with a what I would call a sub-plugin. I basically want to add more controls to the map that cluster map provides. I have been looking into its code and how the plugin management works, but it seems what I am trying to do is not doable without hacks in js. Basically what would like to have is a way to say load my plugin after the plugins I depend on have been loaded and rendered. There seems to be no prior art where plugins have these dependencies on the js level so I was wondering if that could be added or if it exists how to do it. Basically what I want to do is: my-awesome-plugin has a dependency on datastte-cluster-map. Whenever datasette cluster map has finished rendering on page load, call my plugin, but no earlier. To make that work datasette probably needs some total order in which way plugins are loaded intialized. Since I am new to datastte, I may be missing something obvious, so please let me know if the above makes no sense. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1542/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1072135269 | PR_kwDOBm6k_c4vb__Y | 1543 | Bump black from 21.11b1 to 21.12b0 | 49699333 | closed | 0 | 1 | 2021-12-06T13:11:16Z | 2021-12-13T23:22:29Z | 2021-12-13T23:22:29Z | CONTRIBUTOR | simonw/datasette/pulls/1543 | Bumps [black](https://github.com/psf/black) from 21.11b1 to 21.12b0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/psf/black/releases">black's releases</a>.</em></p> <blockquote> <h2>21.12b0</h2> <h3><em>Black</em></h3> <ul> <li>Fix determination of f-string expression spans (<a href="https://github-redirect.dependabot.com/psf/black/issues/2654">#2654</a>)</li> <li>Fix bad formatting of error messages about EOF in multi-line statements (<a href="https://github-redirect.dependabot.com/psf/black/issues/2343">#2343</a>)</li> <li>Functions and classes in blocks now have more consistent surrounding spacing (<a href="https://github-redirect.dependabot.com/psf/black/issues/2472">#2472</a>)</li> </ul> <h4>Jupyter Notebook support</h4> <ul> <li>Cell magics are now only processed if they are known Python cell magics. Earlier, all cell magics were tokenized, leading to possible indentation errors e.g. with <code>%%writefile</code>. (<a href="https://github-redirect.dependabot.com/psf/black/issues/2630">#2630</a>)</li> <li>Fix assignment to environment variables in Jupyter Notebooks (<a href="https://github-redirect.dependabot.com/psf/black/issues/2642">#2642</a>)</li> </ul> <h4>Python 3.10 support</h4> <ul> <li>Point users to using <code>--target-version py310</code> if we detect 3.10-only syntax (<a href="https://github-redirect.dependabot.com/psf/black/issues/2668">#2668</a>)</li> <li>Fix <code>match</code> statements with open sequence subjects, like <code>match a, b:</code> or <code>match a, *b:</code> (<a href="https://github-redirect.dependabot.com/psf/black/issues/2639">#2639</a>) (<a href="https://github-redirect.dependabot.com/psf/black/issues/2659">#2659</a>)</li> <li>Fix <code>match</code>/<code>case</code> statements that contain <code>match</code>/<code>case</code> soft keywords multiple times, like <code>match re.match()</code> (<a href="https://github-redirect.dependabot.com/psf/black/issues/2661">#2661</a>)</li> <li>Fix <code>case</code> statements with an inline … | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1543/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1076834768 | PR_kwDOBm6k_c4vrZxV | 1548 | Update pytest-xdist requirement from <2.5,>=2.2.1 to >=2.2.1,<2.6 | 49699333 | closed | 0 | 1 | 2021-12-10T13:12:06Z | 2021-12-13T23:22:22Z | 2021-12-13T23:22:21Z | CONTRIBUTOR | simonw/datasette/pulls/1548 | Updates the requirements on [pytest-xdist](https://github.com/pytest-dev/pytest-xdist) to permit the latest version. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/pytest-dev/pytest-xdist/blob/master/CHANGELOG.rst">pytest-xdist's changelog</a>.</em></p> <blockquote> <h1>pytest-xdist 2.5.0 (2021-12-10)</h1> <h2>Features</h2> <ul> <li> <p><code>[#722](https://github.com/pytest-dev/pytest-xdist/issues/722) <https://github.com/pytest-dev/pytest-xdist/issues/722></code>_: Full compatibility with pytest 7 - no deprecation warnings or use of legacy features.</p> </li> <li> <p><code>[#733](https://github.com/pytest-dev/pytest-xdist/issues/733) <https://github.com/pytest-dev/pytest-xdist/issues/733></code>_: New <code>--dist=loadgroup</code> option, which ensures all tests marked with <code>@pytest.mark.xdist_group</code> run in the same session/worker. Other tests run distributed as in <code>--dist=load</code>.</p> </li> </ul> <h2>Trivial Changes</h2> <ul> <li> <p><code>[#708](https://github.com/pytest-dev/pytest-xdist/issues/708) <https://github.com/pytest-dev/pytest-xdist/issues/708></code>_: Use <code>@pytest.hookspec</code> decorator to declare hook options in <code>newhooks.py</code> to avoid warnings in <code>pytest 7.0</code>.</p> </li> <li> <p><code>[#719](https://github.com/pytest-dev/pytest-xdist/issues/719) <https://github.com/pytest-dev/pytest-xdist/issues/719></code>_: Use up-to-date <code>setup.cfg</code>/<code>pyproject.toml</code> packaging setup.</p> </li> <li> <p><code>[#720](https://github.com/pytest-dev/pytest-xdist/issues/720) <https://github.com/pytest-dev/pytest-xdist/issues/720></code>_: Require pytest>=6.2.0.</p> </li> <li> <p><code>[#721](https://github.com/pytest-dev/pytest-xdist/issues/721) <https://github.com/pytest-dev/pytest-xdist/issues/721></code>_: Started using type annotations and mypy checking internally. The types are incomplete and not published.</p> </li> </ul> <h1>pytest-xdist 2.4.0 (2021-09-20)</h1> <h… | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1548/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1077628073 | I_kwDOBm6k_c5AO0yp | 1550 | Research option for returning all rows from arbitrary query | 9599 | open | 0 | 2 | 2021-12-11T19:31:11Z | 2021-12-11T23:43:24Z | OWNER | Inspired by thinking about #1549 - returning ALL rows from an arbitrary query is a lot easier if you just run that query and keep iterating over the cursor. I've avoided doing that in the past because it could tie up a connection for a long time - but in private instances this wouldn't be such a problem. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1550/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1077102934 | I_kwDOCGYnMM5AM0lW | 353 | Allow passing a file of code to "sqlite-utils convert" | 536941 | closed | 0 | 8 | 2021-12-10T18:06:14Z | 2021-12-11T01:38:29Z | 2021-12-11T01:09:39Z | CONTRIBUTOR | sqlite-utils is so nice, but the ergonomics of the multiline code in kind of tough. It's really hard (maybe impossible) to make the newlines play well with Makefiles. it would be great to write your code fragment in a separate file and direct it into the sqlite-utils either like ```sqlite-utils convert my.db my_table my_column < custom_code.py``` or ```sqlite-utils convert my.db my_table my_column --custom-code=custom_code.py``` Thanks, as ever, for these great tools! | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/353/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1077243232 | I_kwDOCGYnMM5ANW1g | 354 | Test failure in test_rebuild_fts | 9599 | closed | 0 | 7 | 2021-12-10T21:27:55Z | 2021-12-11T01:08:46Z | 2021-12-11T01:08:46Z | OWNER | Not sure why this has only just started failing, but I'm getting this: https://github.com/simonw/sqlite-utils/runs/4488687639 ``` E sqlite3.DatabaseError: database disk image is malformed sqlite_utils/db.py:425: DatabaseError _______________________ test_rebuild_fts[searchable_fts] _______________________ fresh_db = <Database <sqlite3.Connection object at 0x1084ea9d0>> table_to_fix = 'searchable_fts' @pytest.mark.parametrize("table_to_fix", ["searchable", "searchable_fts"]) def test_rebuild_fts(fresh_db, table_to_fix): table = fresh_db["searchable"] table.insert(search_records[0]) table.enable_fts(["text", "country"]) # Run a search rows = list(table.search("tanuki")) assert len(rows) == 1 assert { "rowid": 1, "text": "tanuki are running tricksters", "country": "Japan", "not_searchable": "foo", }.items() <= rows[0].items() # Delete from searchable_fts_data fresh_db["searchable_fts_data"].delete_where() # This should have broken the index with pytest.raises(sqlite3.DatabaseError): list(table.search("tanuki")) # Running rebuild_fts() should fix it > fresh_db[table_to_fix].rebuild_fts() ``` | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/354/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1066603133 | PR_kwDOCGYnMM4vKAzW | 347 | Test against pysqlite3 running SQLite 3.37 | 9599 | open | 0 | 9 | 2021-11-29T23:17:57Z | 2021-12-11T01:02:19Z | OWNER | simonw/sqlite-utils/pulls/347 | Refs #346 and #344. | 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/347/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
1077322009 | I_kwDOCGYnMM5ANqEZ | 355 | Allow users to pass a full convert() function definition | 9599 | closed | 0 | 4 | 2021-12-10T23:59:58Z | 2021-12-11T00:51:15Z | 2021-12-11T00:49:31Z | OWNER | > I think the fix for this is to change the rules about what code is accepted in both the `-` mode and the literal code string mode: you can pass in a Python expression, OR a fragment that gets turned into a function, OR code that implements its own `def convert(value)` function. So this would work too: > ```sh > sqlite-utils convert my.db mytable col1 ' > def convert(value): > return value.upper() > ' > ``` _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/353#issuecomment-991381679_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/355/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1072780607 | I_kwDOCGYnMM4_8VU_ | 351 | Support `--import xml.etree.ElementTree` in `sqlite-utils convert` | 9599 | closed | 0 | 1 | 2021-12-07T00:40:29Z | 2021-12-11T00:11:25Z | 2021-12-11T00:11:25Z | OWNER | It's not possible to use a module that requires a nested import, such as `xml.etree.ElementTree`, at the moment. I found and fixed this bug in `git-history`, I should replicate that fix (and accompanying documentation) here: https://github.com/simonw/git-history/issues/39 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/351/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1068791148 | I_kwDOBm6k_c4_tHVs | 1540 | Idea: hover to reveal details of linked row | 9599 | open | 0 | 6 | 2021-12-01T19:28:07Z | 2021-12-09T23:38:39Z | OWNER | <img width="698" alt="fara__item_version__7_rows_where_where__item___5236" src="https://user-images.githubusercontent.com/9599/144300364-ee621811-8f5a-4c7e-9950-199119092cf3.png"> Hovering over that could work a little bit like GitHub issue links: ![hover](https://user-images.githubusercontent.com/9599/144300537-9cd9e9af-ac16-42db-842f-37661bc94063.gif) | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1540/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
863884805 | MDU6SXNzdWU4NjM4ODQ4MDU= | 1304 | Document how to send multiple values for "Named parameters" | 9308268 | open | 0 | 4 | 2021-04-21T13:19:06Z | 2021-12-08T03:23:14Z | NONE | https://docs.datasette.io/en/stable/sql_queries.html#named-parameters I thought that I had seen an example of how to do this example below, but I can't seem to find it ```sql select * from bib where bib.bib_record_num in (1008088,1008092) ``` ```sql select * from bib where bib.bib_record_num in (:bib_record_numbers) ``` ![image](https://user-images.githubusercontent.com/9308268/115558839-2333a480-a281-11eb-85e6-ce3bada79140.png) https://ilsweb.cincinnatilibrary.org/collection-analysis/current_collection-204d100?sql=select%0D%0A++*%0D%0Afrom%0D%0A++bib%0D%0Awhere%0D%0A++bib.bib_record_num+in+%28%3Abib_record_numbers%29&bib_record_numbers=1008088%2C1008092 Or, maybe this isn't a fully supported feature. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1304/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1073712378 | I_kwDOBm6k_c4__4z6 | 1544 | Code that detects the label column for a table is case-sensitive | 9599 | closed | 0 | 2 | 2021-12-07T20:01:25Z | 2021-12-07T20:03:43Z | 2021-12-07T20:03:43Z | OWNER | I just noticed that a column called `Name` is not being picked up as the label column for a table. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1544/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1071531082 | I_kwDOCGYnMM4_3kRK | 349 | A way of creating indexes on newly created tables | 9599 | open | 0 | 3 | 2021-12-05T18:56:12Z | 2021-12-07T01:04:37Z | OWNER | I'm writing code for https://github.com/simonw/git-history/issues/33 that creates a table inside a loop: ```python item_pk = db[item_table].lookup( {"_item_id": item_id}, item_to_insert, column_order=("_id", "_item_id"), pk="_id", ) ``` I need to look things up by `_item_id` on this table, which means I need an index on that column (the table can get very big). But there's no mechanism in SQLite utils to detect if the table was created for the first time and add an index to it. And I don't want to run `CREATE INDEX IF NOT EXISTS` every time through the loop. This should work like the `foreign_keys=` mechanism. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/349/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1072435124 | I_kwDOCGYnMM4_7A-0 | 350 | Optional caching mechanism for table.lookup() | 9599 | open | 0 | 3 | 2021-12-06T17:54:25Z | 2021-12-06T17:56:57Z | OWNER | Inspired by work on `git-history` where I used this pattern: ```python column_name_to_id = {} def column_id(column): if column not in column_name_to_id: id = db["columns"].lookup( {"namespace": namespace_id, "name": column}, foreign_keys=(("namespace", "namespaces", "id"),), ) column_name_to_id[column] = id return column_name_to_id[column] ``` If you're going to be doing a large number of `table.lookup(...)` calls and you know that no other script will be modifying the database at the same time you can presumably get a big speedup using a Python in-memory cache - maybe even a LRU one to avoid memory bloat. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/350/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
964322136 | MDU6SXNzdWU5NjQzMjIxMzY= | 1426 | Manage /robots.txt in Datasette core, block robots by default | 9599 | open | 0 | 9 | 2021-08-09T19:56:56Z | 2021-12-04T07:11:29Z | OWNER | See accompanying Twitter thread: https://twitter.com/simonw/status/1424820203603431439 > Datasette currently has a plugin for configuring robots.txt, but I'm beginning to think it should be part of core and crawlers should be blocked by default - having people explicitly opt-in to having their sites crawled and indexed feels a lot safer https://datasette.io/plugins/datasette-block-robots I have a lot of Datasettes deployed now, and tailing logs shows that they are being *hammered* by search engine crawlers even though many of them are not interesting enough to warrant indexing. I'm starting to think blocking crawlers would actually be a better default for most people, provided it was well documented and easy to understand how to allow them. Default-deny is usually a better policy than default-allow! | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1426/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1071071397 | I_kwDODFdgUs4_10Cl | 69 | View that combines issues and issue comments | 9599 | open | 0 | 1 | 2021-12-04T00:34:33Z | 2021-12-04T00:34:52Z | MEMBER | I want to see a reverse chronologically ordered interface onto both issues and comments - essentially a unified log of comments and issues opened across one or multiple projects. | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/69/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1069881276 | I_kwDOBm6k_c4_xRe8 | 1541 | Different default layout for row page | 9599 | open | 0 | 1 | 2021-12-02T18:56:36Z | 2021-12-02T18:56:54Z | OWNER | The row page displays as a table even though it only has one table row. maybe default to the same display as the narrow page version, even for wide pages? | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1541/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1058790545 | I_kwDOBm6k_c4_G9yR | 1519 | base_url is omitted in JSON and CSV views | 157158 | closed | 0 | 22 | 2021-11-19T18:10:45Z | 2021-12-01T17:50:09Z | 2021-11-20T19:11:21Z | NONE | I have a datasette deployment, using Apache2 to reverse proxy: ProxyPass /ged http://thor.phfactor.net:8001 ProxyPreserveHost On In settings.json I have ```json { "base_url": "/ged/", "trace_debug": 1, "template_debug": 1 } ``` and datasette works correctly. However, if you view a query and then click on the 'This data as json, CSV' both links omit the base_url prefix and are therefore 404. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1519/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1067775061 | I_kwDOBm6k_c4_pPRV | 1539 | Research PRAGMA query_only | 9599 | open | 0 | 0 | 2021-11-30T23:30:24Z | 2021-11-30T23:30:24Z | OWNER | https://www.sqlite.org/pragma.html#pragma_query_only > The query_only pragma prevents data changes on database files when enabled. When this pragma is enabled, any attempt to CREATE, DELETE, DROP, INSERT, or UPDATE will result in an [SQLITE_READONLY](https://www.sqlite.org/rescode.html#readonly) error. However, the database is not truly read-only. You can still run a [checkpoint](https://www.sqlite.org/wal.html#ckpt) or a [COMMIT](https://www.sqlite.org/lang_transaction.html) and the return value of the [sqlite3_db_readonly()](https://www.sqlite.org/c3ref/db_readonly.html) routine is not affected. Would it be worth adding this as an extra protection against accidental writes to a DB file over a read-only connection? | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1539/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1059509927 | I_kwDOBm6k_c4_Jtan | 1525 | "Links from other tables" broken for columns starting with underscore | 9599 | closed | 0 | 3 | 2021-11-21T22:55:08Z | 2021-11-30T06:39:01Z | 2021-11-30T06:34:35Z | OWNER | Same bug as #1506, this time it's this link or the row page: <img width="341" alt="image" src="https://user-images.githubusercontent.com/9599/142782167-fe9b6ffa-cd81-4a30-9f3b-401118c498e9.png"> | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1525/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1062414013 | PR_kwDOBm6k_c4u9wQq | 1529 | Update janus requirement from <0.7,>=0.6.2 to >=0.6.2,<0.8 | 49699333 | closed | 0 | 1 | 2021-11-24T13:12:42Z | 2021-11-30T02:37:13Z | 2021-11-30T02:37:13Z | CONTRIBUTOR | simonw/datasette/pulls/1529 | Updates the requirements on [janus](https://github.com/aio-libs/janus) to permit the latest version. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/aio-libs/janus/blob/master/CHANGES.rst">janus's changelog</a>.</em></p> <blockquote> <h2>0.7.0 (2021-11-24)</h2> <ul> <li>Add SyncQueue and AsyncQueue Protocols to provide type hints for sync and async queues <a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/374">#374</a></li> </ul> <h2>0.6.2 (2021-10-24)</h2> <ul> <li>Fix Python 3.10 compatibility <a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/358">#358</a></li> </ul> <h2>0.6.1 (2020-10-26)</h2> <ul> <li> <p>Raise RuntimeError on queue.join() after queue closing. <a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/295">#295</a></p> </li> <li> <p>Replace <code>timeout</code> type from <code>Optional[int]</code> to <code>Optional[float]</code> <a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/267">#267</a></p> </li> </ul> <h2>0.6.0 (2020-10-10)</h2> <ul> <li> <p>Drop Python 3.5, the minimal supported version is Python 3.6</p> </li> <li> <p>Support Python 3.9</p> </li> <li> <p>Refomat with <code>black</code></p> </li> </ul> <h2>0.5.0 (2020-04-23)</h2> <ul> <li>Remove explicit loop arguments and forbid creating queues outside event loops <a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/246">#246</a></li> </ul> <h2>0.4.0 (2018-07-28)</h2> <ul> <li> <p>Add <code>py.typed</code> macro <a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/89">#89</a></p> </li> <li> <p>Drop python 3.4 support and fix minimal version python3.5.3 <a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/88">#88</a></p> </li> <li> <p>Add property with that indicates if queue is closed <a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/86">#86</a></p> </li> </ul> <h2>0.3.2 (2018-07-06)</h2> <ul> <li>Fixed python 3.7 support <a href="https://github-redirect.depe… | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1529/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1057340779 | PR_kwDOBm6k_c4utsKs | 1516 | Bump black from 21.9b0 to 21.11b1 | 49699333 | closed | 0 | 1 | 2021-11-18T13:11:12Z | 2021-11-30T02:35:29Z | 2021-11-30T02:35:29Z | CONTRIBUTOR | simonw/datasette/pulls/1516 | Bumps [black](https://github.com/psf/black) from 21.9b0 to 21.11b1. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/psf/black/releases">black's releases</a>.</em></p> <blockquote> <h2>21.11b1</h2> <h3><em>Black</em></h3> <ul> <li>Bumped regex version minimum to 2021.4.4 to fix Pattern class usage (<a href="https://github-redirect.dependabot.com/psf/black/issues/2621">#2621</a>)</li> </ul> <h2>21.11b0</h2> <h3><em>Black</em></h3> <ul> <li>Warn about Python 2 deprecation in more cases by improving Python 2 only syntax detection (<a href="https://github-redirect.dependabot.com/psf/black/issues/2592">#2592</a>)</li> <li>Add experimental PyPy support (<a href="https://github-redirect.dependabot.com/psf/black/issues/2559">#2559</a>)</li> <li>Add partial support for the match statement. As it's experimental, it's only enabled when <code>--target-version py310</code> is explicitly specified (<a href="https://github-redirect.dependabot.com/psf/black/issues/2586">#2586</a>)</li> <li>Add support for parenthesized with (<a href="https://github-redirect.dependabot.com/psf/black/issues/2586">#2586</a>)</li> <li>Declare support for Python 3.10 for running Black (<a href="https://github-redirect.dependabot.com/psf/black/issues/2562">#2562</a>)</li> </ul> <h3>Integrations</h3> <ul> <li>Fixed vim plugin with Python 3.10 by removing deprecated distutils import (<a href="https://github-redirect.dependabot.com/psf/black/issues/2610">#2610</a>)</li> <li>The vim plugin now parses <code>skip_magic_trailing_comma</code> from pyproject.toml (<a href="https://github-redirect.dependabot.com/psf/black/issues/2613">#2613</a>)</li> </ul> <h2>21.10b0</h2> <h3><em>Black</em></h3> <ul> <li>Document stability policy, that will apply for non-beta releases (<a href="https://github-redirect.dependabot.com/psf/black/issues/2529">#2529</a>)</li> <li>Add new <code>--workers</code> parameter (<a href="https://github-redirect.dependabot.com/psf/black/issues/2514">#2514</a>)</li> <li>Fixed feature detection for po… | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1516/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1053655062 | PR_kwDOBm6k_c4uiE0n | 1508 | Update docutils requirement from <0.18 to <0.19 | 49699333 | closed | 0 | 1 | 2021-11-15T13:15:47Z | 2021-11-30T02:35:19Z | 2021-11-30T02:35:19Z | CONTRIBUTOR | simonw/datasette/pulls/1508 | Updates the requirements on [docutils](http://docutils.sourceforge.net/) to permit the latest version. Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1508/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1066023866 | PR_kwDOBm6k_c4vIJqi | 1537 | Update aiofiles requirement from <0.8,>=0.4 to >=0.4,<0.9 | 49699333 | closed | 0 | 1 | 2021-11-29T13:13:52Z | 2021-11-30T02:29:55Z | 2021-11-30T02:29:54Z | CONTRIBUTOR | simonw/datasette/pulls/1537 | Updates the requirements on [aiofiles](https://github.com/Tinche/aiofiles) to permit the latest version. <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/Tinche/aiofiles/commit/d010ff4d789598213334a32ec3d3f55caaab766c"><code>d010ff4</code></a> 0.8.0</li> <li><a href="https://github.com/Tinche/aiofiles/commit/3e3830daad8f41b1d3548a59c9913e4daf7be327"><code>3e3830d</code></a> Tweak README</li> <li><a href="https://github.com/Tinche/aiofiles/commit/713984eb46a62f8d504f31ee9b1305a55581675d"><code>713984e</code></a> Update README</li> <li><a href="https://github.com/Tinche/aiofiles/commit/95709758e635e9155ee5d96c973676cd0815221a"><code>9570975</code></a> Use os.sep</li> <li><a href="https://github.com/Tinche/aiofiles/commit/2b2500d555769460bc1f226f078d86eeaae06daf"><code>2b2500d</code></a> Add prefix, suffix, dir args to TemporaryDirectory</li> <li><a href="https://github.com/Tinche/aiofiles/commit/8895eb48b6004c8677da5841e2504a73eb8fad5e"><code>8895eb4</code></a> Fix ospath.exists test</li> <li><a href="https://github.com/Tinche/aiofiles/commit/83083555512320cf5c4cb10dd4dbda57c4cfbea1"><code>8308355</code></a> Fix typo: aiofiles.ospath.exists</li> <li><a href="https://github.com/Tinche/aiofiles/commit/b8cf8dd19369c2529181b3a4b8f1c485a2c05dbd"><code>b8cf8dd</code></a> Update Readme with os.makedirs and os.removedirs</li> <li><a href="https://github.com/Tinche/aiofiles/commit/aea353ebdd2771f2e4ae934077eceb12daaa3481"><code>aea353e</code></a> Merge branch 'patch-2'</li> <li><a href="https://github.com/Tinche/aiofiles/commit/761f06ede5925d0cdd22926b0d83564e9c1e1ab5"><code>761f06e</code></a> Fix README</li> <li>Additional commits viewable in <a href="https://github.com/Tinche/aiofiles/compare/v0.4.0...v0.8.0">compare view</a></li> </ul> </details> <br /> Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- … | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1537/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1066563554 | I_kwDOCGYnMM4_knfi | 346 | Way to test SQLite 3.37 (and potentially other versions) in CI | 9599 | open | 0 | 5 | 2021-11-29T22:21:06Z | 2021-11-29T23:12:49Z | OWNER | > Need to figure out a good pattern for testing this in CI too - it will currently skip the new tests if it doesn't have SQLite 3.37 or higher. _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/344#issuecomment-982076924_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/346/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1066501534 | I_kwDOCGYnMM4_kYWe | 345 | `table.strict` introspection boolean for identifying STRICT mode tables | 9599 | closed | 0 | 2 | 2021-11-29T21:05:10Z | 2021-11-29T22:45:26Z | 2021-11-29T22:44:36Z | OWNER | > From the STRICT docs: >> The SQLite parser accepts a comma-separated list of table options after the final close parenthesis in a CREATE TABLE statement. As of this writing (2021-08-23) only two options are recognized: >> >> - STRICT >> - [WITHOUT ROWID](https://www.sqlite.org/withoutrowid.html) > > So I think I need to read the `CREATE TABLE` statement from the `sqlite_master` table, split on the last `)`, split those tokens on `,` and see if `create` is in there (case insensitive). _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/344#issuecomment-982020757_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/345/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1066288689 | I_kwDOBm6k_c4_jkYx | 1538 | Research pattern for re-registering existing Click tools with register_commands | 9599 | closed | 0 | 3 | 2021-11-29T17:09:47Z | 2021-11-29T17:32:44Z | 2021-11-29T17:27:16Z | OWNER | Building a Datasette plugin that imports an existing Click CLI tool and re-registers it is proving hard - Click doesn't really want you to do that. I tried this: ```python from datasette import hookimpl from git_history.cli import file as git_history_file @hookimpl def register_commands(cli): cli.command(name="git-history")(git_history_file.callback) ``` But when I run this: ``` % datasette git-history --help Usage: datasette git-history [OPTIONS] Analyze the history of a specific file and write it to SQLite Options: --help Show this message and exit. ``` The options are all missing - which means that the command doesn't actually work. Will need to research this pattern separately. _Originally posted by @simonw in https://github.com/simonw/git-history/issues/21#issuecomment-981835305_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1538/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1039037439 | PR_kwDOCGYnMM4t0uaI | 333 | Add functionality to read Parquet files. | 2118708 | closed | 0 | 3 | 2021-10-28T23:43:19Z | 2021-11-25T19:47:35Z | 2021-11-25T19:47:35Z | NONE | simonw/sqlite-utils/pulls/333 | I needed this for a project of mine, and I thought it'd be useful to have it in sqlite-utils (It's also mentioned in #248 ). The current implementation works (data is read & data types are inferred correctly. I've added a single straightforward test case, but @simonw please let me know if there are any non-obvious flags/combinations I should test too. | 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/333/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1058896236 | I_kwDOBm6k_c4_HXls | 1522 | Deploy a live instance of demos/apache-proxy | 9599 | closed | 0 | 34 | 2021-11-19T20:32:55Z | 2021-11-23T03:00:34Z | 2021-11-20T18:51:56Z | OWNER | > I'll get this working on my laptop first, but then I want to get it up and running on Cloud Run - maybe with a GitHub Actions workflow in this repo that re-deploys it on manual execution. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1521#issuecomment-974322178_ I started by following https://ahmet.im/blog/cloud-run-multiple-processes-easy-way/ - see example in https://github.com/ahmetb/multi-process-container-lazy-solution | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1522/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1059549523 | I_kwDOBm6k_c4_J3FT | 1526 | Add to vercel.json, rather than overwriting it. | 192568 | closed | 0 | 2 | 2021-11-22T00:47:12Z | 2021-11-22T04:49:45Z | 2021-11-22T04:13:47Z | CONTRIBUTOR | I'd like to be able to add to vercel.json. But Datasette overwrites whatever I put in that file. I originally reported this here: https://github.com/simonw/datasette-publish-vercel/issues/51 In that case, I wanted to do a rewrite... and now I need to do 301 redirects (because we had to rename our site). Can this be addressed? | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1526/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
273944952 | MDU6SXNzdWUyNzM5NDQ5NTI= | 93 | Package as standalone binary | 67420 | closed | 0 | 18 | 2017-11-14T21:14:07Z | 2021-11-21T07:00:23Z | 2021-11-21T07:00:23Z | NONE | hint: more than the docker image a standalone and multiplatform binary (containing the app and the database) could be simpler to distribute. i would like to investigate the possibility to package everything with [pyinstaller](http://www.pyinstaller.org/) adding the database as a [data file](https://pythonhosted.org/PyInstaller/spec-files.html#adding-data-files) | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/93/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1059219106 | I_kwDOBm6k_c4_Imai | 1524 | Improve Apache proxy documentation, link to demo | 9599 | closed | 0 | 4 | 2021-11-20T20:03:14Z | 2021-11-20T23:34:03Z | 2021-11-20T23:34:03Z | OWNER | > The latest demo is now live at https://datasette-apache-proxy-demo.fly.dev/prefix/fixtures/sortable?_facet=pk2 _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1519#issuecomment-974697824_ I'm going to put out 0.59.3 bugfix release with this, but I'd like to first improve the documentation on https://docs.datasette.io/en/stable/deploying.html#apache-proxy-configuration to highlight the new demo. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1524/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
637395097 | MDU6SXNzdWU2MzczOTUwOTc= | 838 | Incorrect URLs when served behind a proxy with base_url set | 79913 | closed | 0 | 6026070 | 14 | 2020-06-11T23:58:55Z | 2021-11-20T19:35:48Z | 2021-11-20T19:35:48Z | NONE | I'm running `datasette serve --config base_url:/foo/ …`, proxying to it with this Apache config: ProxyPass /foo/ http://localhost:8001/ ProxyPassReverse /foo/ http://localhost:8001/ and then accessing it via `https://example.com/foo/`. Although many of the URLs in the pages are correct (presumably because they either use absolute paths which include `base_url` or relative paths), the faceting and pagination links still use fully-qualified URLs pointing at `http://localhost:8001`. I looked into this a little in the source code, and it seems to be an issue anywhere `request.url` or `request.path` is used, as these contain the values for the request between the frontend (Apache) and backend (Datasette) server. Those properties are primarily used via the `path_with_…` family of utility functions and the `Datasette.absolute_url` method. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/838/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
1059209412 | I_kwDOBm6k_c4_IkDE | 1523 | Come up with a more elegant solution for base_url than ds.urls.path() | 9599 | open | 0 | 0 | 2021-11-20T19:05:22Z | 2021-11-20T19:05:22Z | OWNER | While fixing #1519 I added a lot of ugly code that looks like this: https://github.com/simonw/datasette/blob/08947fa76433d18988aa1ee1d929bd8320c75fe2/datasette/facets.py#L228-L230 See these two commits in particular: fe687fd0207c4c56c4778d3e92e3505fc4b18172 and 08947fa76433d18988aa1ee1d929bd8320c75fe2 It would be great to come up with a less verbose and error-prone way of handling this problem. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1523/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1058815557 | I_kwDOBm6k_c4_HD5F | 1521 | Docker configuration for exercising Datasette behind Apache mod_proxy | 9599 | closed | 0 | 10 | 2021-11-19T18:46:18Z | 2021-11-19T20:32:29Z | 2021-11-19T20:32:29Z | OWNER | > Having a live demo running on Cloud Run that proxies through Apache and uses `base_url` would be incredibly useful for replicating and debugging this kind of thing. I wonder how hard it is to run Apache and `mod_proxy` in the same Docker container as Datasette? _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1519#issuecomment-974310208_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1521/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1058803238 | I_kwDOBm6k_c4_HA4m | 1520 | Pattern for avoiding accidental URL over-rides | 9599 | open | 0 | 1 | 2021-11-19T18:28:05Z | 2021-11-19T18:29:26Z | OWNER | Following #1517 I'm experimenting with a plugin that does this: ```python @hookimpl def register_routes(): return [ (r"/(?P<db_name>[^/]+)/(?P<table_and_format>[^/]+?)$", Table().view), ] ``` This is supposed to replace the default table page with new code... but there's a problem: `/-/versions` on that instance now returns 404 `Database '-' does not exist`! Need to figure out a pattern to avoid that happening. Plugins get to add their routes before Datasette's default routes, which is why this is happening here. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1520/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1033678984 | PR_kwDOBm6k_c4tjgJ8 | 1495 | Allow routes to have extra options | 536941 | open | 0 | 5 | 2021-10-22T15:00:45Z | 2021-11-19T15:36:27Z | CONTRIBUTOR | simonw/datasette/pulls/1495 | Right now, datasette routes can only be a 2-tuple of `(regex, view_fn)`. If it was possible for datasette to handle extra options, like [standard Django does](https://docs.djangoproject.com/en/3.2/topics/http/urls/#passing-extra-options-to-view-functions), it would add flexibility for plugin authors. For example, if extra options were enabled, then it would be easy to make a single table the home page (#1284). This plugin would accomplish it. ```python from datasette import hookimpl from datasette.views.table import TableView @hookimpl def register_routes(datasette): return [ (r"^/$", TableView.as_view(datasette), {'db_name': 'DB_NAME', 'table': 'TABLE_NAME'}) ] ``` | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1495/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
1058196641 | I_kwDOCGYnMM4_Esyh | 342 | Extra options to `lookup()` which get passed to `insert()` | 9599 | closed | 0 | 7 | 2021-11-19T06:53:03Z | 2021-11-19T07:26:54Z | 2021-11-19T07:26:54Z | OWNER | For https://github.com/simonw/git-history/issues/12 I found myself wanting to pass extra options to `lookup()` to set the column order, primary key etc. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/342/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1057996111 | I_kwDOBm6k_c4_D71P | 1517 | Let `register_routes()` over-ride default routes within Datasette | 9599 | closed | 0 | 3268330 | 2 | 2021-11-19T00:22:15Z | 2021-11-19T03:20:00Z | 2021-11-19T03:07:27Z | OWNER | See https://github.com/simonw/datasette/issues/878#issuecomment-973554024_ - right now `register_routes()` can't replace default Datasette routes. It would be neat if plugins could do this - especially if there was a neat documented way for them to then re-dispatch to the original route code after making some kind of modification. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1517/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
1056117435 | PR_kwDOBm6k_c4up0R0 | 1514 | Bump black from 21.9b0 to 21.11b0 | 49699333 | closed | 0 | 2 | 2021-11-17T13:13:55Z | 2021-11-18T13:11:17Z | 2021-11-18T13:11:15Z | CONTRIBUTOR | simonw/datasette/pulls/1514 | Bumps [black](https://github.com/psf/black) from 21.9b0 to 21.11b0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/psf/black/releases">black's releases</a>.</em></p> <blockquote> <h2>21.11b0</h2> <h3><em>Black</em></h3> <ul> <li>Warn about Python 2 deprecation in more cases by improving Python 2 only syntax detection (<a href="https://github-redirect.dependabot.com/psf/black/issues/2592">#2592</a>)</li> <li>Add experimental PyPy support (<a href="https://github-redirect.dependabot.com/psf/black/issues/2559">#2559</a>)</li> <li>Add partial support for the match statement. As it's experimental, it's only enabled when <code>--target-version py310</code> is explicitly specified (<a href="https://github-redirect.dependabot.com/psf/black/issues/2586">#2586</a>)</li> <li>Add support for parenthesized with (<a href="https://github-redirect.dependabot.com/psf/black/issues/2586">#2586</a>)</li> <li>Declare support for Python 3.10 for running Black (<a href="https://github-redirect.dependabot.com/psf/black/issues/2562">#2562</a>)</li> </ul> <h3>Integrations</h3> <ul> <li>Fixed vim plugin with Python 3.10 by removing deprecated distutils import (<a href="https://github-redirect.dependabot.com/psf/black/issues/2610">#2610</a>)</li> <li>The vim plugin now parses <code>skip_magic_trailing_comma</code> from pyproject.toml (<a href="https://github-redirect.dependabot.com/psf/black/issues/2613">#2613</a>)</li> </ul> <h2>21.10b0</h2> <h3><em>Black</em></h3> <ul> <li>Document stability policy, that will apply for non-beta releases (<a href="https://github-redirect.dependabot.com/psf/black/issues/2529">#2529</a>)</li> <li>Add new <code>--workers</code> parameter (<a href="https://github-redirect.dependabot.com/psf/black/issues/2514">#2514</a>)</li> <li>Fixed feature detection for positional-only arguments in lambdas (<a href="https://github-redirect.dependabot.com/psf/black/issues/2532">#2532</a>)</li> <li>Bumped typed-ast version minimum to 1.4.3 for 3.10 compatiblity (<a href="https://git… | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1514/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1056746091 | I_kwDOBm6k_c4-_Kpr | 1515 | Handle foreign keys that point to a non-existent table | 9599 | open | 0 | 0 | 2021-11-17T23:40:13Z | 2021-11-18T01:31:56Z | OWNER | Spotted in https://github.com/simonw/datasette-graphql/issues/79 Demo: https://datasette-graphql-demo.datasette.io/fixtures/bad_foreign_key The foreign key links to a 404 page. ![B87009C7-CFCA-4DF9-8FBA-FA3E6CA28EC2](https://user-images.githubusercontent.com/9599/142334788-4d1a4acd-bc87-4426-b333-d46b221afcec.jpeg) | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1515/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1041158024 | PR_kwDOBm6k_c4t7RKr | 1500 | Bump black from 21.9b0 to 21.10b0 | 49699333 | closed | 0 | 2 | 2021-11-01T13:11:23Z | 2021-11-17T13:14:00Z | 2021-11-17T13:13:58Z | CONTRIBUTOR | simonw/datasette/pulls/1500 | Bumps [black](https://github.com/psf/black) from 21.9b0 to 21.10b0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/psf/black/releases">black's releases</a>.</em></p> <blockquote> <h2>21.10b0</h2> <h3><em>Black</em></h3> <ul> <li>Document stability policy, that will apply for non-beta releases (<a href="https://github-redirect.dependabot.com/psf/black/issues/2529">#2529</a>)</li> <li>Add new <code>--workers</code> parameter (<a href="https://github-redirect.dependabot.com/psf/black/issues/2514">#2514</a>)</li> <li>Fixed feature detection for positional-only arguments in lambdas (<a href="https://github-redirect.dependabot.com/psf/black/issues/2532">#2532</a>)</li> <li>Bumped typed-ast version minimum to 1.4.3 for 3.10 compatiblity (<a href="https://github-redirect.dependabot.com/psf/black/issues/2519">#2519</a>)</li> <li>Fixed a Python 3.10 compatibility issue where the loop argument was still being passed even though it has been removed (<a href="https://github-redirect.dependabot.com/psf/black/issues/2580">#2580</a>)</li> <li>Deprecate Python 2 formatting support (<a href="https://github-redirect.dependabot.com/psf/black/issues/2523">#2523</a>)</li> </ul> <h3><em>Blackd</em></h3> <ul> <li>Remove dependency on aiohttp-cors (<a href="https://github-redirect.dependabot.com/psf/black/issues/2500">#2500</a>)</li> <li>Bump required aiohttp version to 3.7.4 (<a href="https://github-redirect.dependabot.com/psf/black/issues/2509">#2509</a>)</li> </ul> <h3><em>Black-Primer</em></h3> <ul> <li>Add primer support for --projects (<a href="https://github-redirect.dependabot.com/psf/black/issues/2555">#2555</a>)</li> <li>Print primer summary after individual failures (<a href="https://github-redirect.dependabot.com/psf/black/issues/2570">#2570</a>)</li> </ul> <h3>Integrations</h3> <ul> <li>Allow to pass <code>target_version</code> in the vim plugin (<a href="https://github-redirect.dependabot.com/psf/black/issues/1319">#1319</a>)</li> <li>Install build tools in docker file and use mul… | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1500/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1055402144 | PR_kwDOBm6k_c4unfnq | 1512 | New pattern for async view classes | 9599 | closed | 0 | 7 | 2021-11-16T21:55:44Z | 2021-11-17T01:39:54Z | 2021-11-17T01:39:44Z | OWNER | simonw/datasette/pulls/1512 | Refs #878 - starting out with the new `AsyncBase` class implementing a pytest-inspired `asyncio` parallel execution mechanism. | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1512/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1 | |||||
1055469073 | I_kwDOBm6k_c4-6S4R | 1513 | Research: CTEs and union all to calculate facets AND query at the same time | 9599 | closed | 0 | 12 | 2021-11-16T22:26:45Z | 2021-11-16T23:41:46Z | 2021-11-16T23:41:46Z | OWNER | Consider this page: https://global-power-plants.datasettes.com/global-power-plants/global-power-plants?_search=plant&_facet=owner&_facet=country_long&_facet=primary_fuel Datasette needs to run the main query for the rows on that page, a count query for the total query, then a separate query for each of those three specified facets. This is a `_search=` query, so it needs to execute the FTS code once for the rows, again for the count, and then three more times for each of the facets. Could running that query as a CTE and doing the other queries as part of the same large query produce significant speed improvements? | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1513/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
812704869 | MDU6SXNzdWU4MTI3MDQ4Njk= | 1237 | ?_pretty=1 option for pretty-printing JSON output | 9599 | open | 0 | 3268330 | 1 | 2021-02-20T20:54:40Z | 2021-11-16T18:28:33Z | OWNER | Suggested by @frankieroberto in https://github.com/simonw/datasette/issues/782#issuecomment-782746755 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1237/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
718540751 | MDU6SXNzdWU3MTg1NDA3NTE= | 1012 | For 1.0 update trove classifier in setup.py | 9599 | open | 0 | 3268330 | 5 | 2020-10-10T05:52:08Z | 2021-11-16T13:18:36Z | OWNER | Development Status :: 5 - Production/Stable | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1012/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
1052247023 | I_kwDOBm6k_c4-uAPv | 1505 | Datasette should have an option to output CSV with semicolons | 9599 | open | 0 | 1 | 2021-11-12T18:02:21Z | 2021-11-16T11:40:52Z | OWNER | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1505/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||||
440222719 | MDU6SXNzdWU0NDAyMjI3MTk= | 448 | _facet_array should work against views | 9599 | closed | 0 | 3268330 | 12 | 2019-05-03T21:08:04Z | 2021-11-16T01:32:05Z | 2021-11-16T01:19:40Z | OWNER | I created this view: https://json-view-facet-bug-demo-j7hipcg4aq-uc.a.run.app/russian-ads-8dbda00/ads_with_targets ``` CREATE VIEW ads_with_targets as select ads.*, json_group_array(targets.name) as target_names from ads join ad_targets on ad_targets.ad_id = ads.id join targets on ad_targets.target_id = targets.id group by ad_targets.ad_id ``` When I try to apply faceting by array it appears to work at first: https://json-view-facet-bug-demo-j7hipcg4aq-uc.a.run.app/russian-ads/ads_with_targets?_facet_array=target_names But actually it's doing the wrong thing - the SQL for the facets uses rowid, but rowid is not present on views at all! These results are incorrect, and clicking to select a facet will fail to produce any rows: https://json-view-facet-bug-demo-j7hipcg4aq-uc.a.run.app/russian-ads/ads_with_targets?_facet_array=target_names&target_names__arraycontains=people_who_match%3Ainterests%3AAfrican-American+Civil+Rights+Movement+%281954%E2%80%9468%29 Here's the SQL it should be using when you select a facet (note that it does not use a rowid): https://json-view-facet-bug-demo-j7hipcg4aq-uc.a.run.app/russian-ads?sql=select+*+from+ads_with_targets+where+id+in+%28%0D%0A++++++++++++select+ads_with_targets.id+from+ads_with_targets%2C+json_each%28ads_with_targets.target_names%29+j%0D%0A++++++++++++where+j.value+%3D+%3Ap0%0D%0A++++++++%29+limit+101&p0=people_who_match%3Ainterests%3ABlack+%28Color%29 So we need to do something a lot smarter here. I'm not sure what the fix will look like, or even if it's feasible given that views don't have a rowid to hook into so the JSON faceting SQL may have to be completely rewritten. ``` datasette publish cloudrun \ russian-ads.db \ --name json-view-facet-bug-demo \ --branch master \ --extra-options "--config sql_time_limit_ms:5000 --config facet_time_limit_ms:5000" ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/448/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
1054246919 | I_kwDOBm6k_c4-1ogH | 1511 | Review plugin hooks for Datasette 1.0 | 9599 | open | 0 | 3268330 | 1 | 2021-11-15T23:26:05Z | 2021-11-16T01:20:14Z | OWNER | I need to perform a detailed review of the plugin interface - especially the plugin hooks like [register_facet_classes()](https://docs.datasette.io/en/stable/plugin_hooks.html#register-facet-classes) which I don't yet have complete confidence in. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1511/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
459590021 | MDU6SXNzdWU0NTk1OTAwMjE= | 519 | Decide what goes into Datasette 1.0 | 9599 | closed | 0 | 3268330 | 4 | 2019-06-23T15:47:41Z | 2021-11-15T23:26:11Z | 2021-11-15T23:26:11Z | OWNER | Datasette ASGI #272 is a big part of it... but 1.0 will generally be an indicator that Datasette is a stable platform for developers to write plugins and custom templates against. So lots to think about. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/519/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
1025726600 | PR_kwDOCGYnMM4tKxHD | 330 | Test against Python 3.10 | 9599 | closed | 0 | 1 | 2021-10-13T21:50:22Z | 2021-11-15T02:59:29Z | 2021-10-13T22:25:05Z | OWNER | simonw/sqlite-utils/pulls/330 | 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/330/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
1005891028 | I_kwDOCGYnMM479K3U | 329 | Rethink approach to [ and ] in column names (currently throws error) | 9599 | closed | 0 | 12 | 2021-09-23T22:14:24Z | 2021-11-15T02:57:51Z | 2021-11-15T02:57:51Z | OWNER | > I think it's best to still keep `[` and `]` out of column names though. Transforming them into `(` and `)` seems reasonable - but should that happen here or in `sqlite-utils`? I think in `sqlite-utils`. _Originally posted by @simonw in https://github.com/simonw/datasette-app/issues/121#issuecomment-926200398_ This is a rethinking of the solution to: - https://github.com/simonw/sqlite-utils/issues/86 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/329/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1053136495 | I_kwDOCGYnMM4-xZZv | 341 | `hash_id: Optional[Any]` should be `hash_id: Optional[str]` | 9599 | closed | 0 | 0 | 2021-11-15T02:12:39Z | 2021-11-15T02:19:31Z | 2021-11-15T02:19:31Z | OWNER | In a few places: https://github.com/simonw/sqlite-utils/blob/54a2269e91ce72b059618662ed133a85f3d42e4a/sqlite_utils/db.py#L642 https://github.com/simonw/sqlite-utils/blob/54a2269e91ce72b059618662ed133a85f3d42e4a/sqlite_utils/db.py#L751 https://github.com/simonw/sqlite-utils/blob/54a2269e91ce72b059618662ed133a85f3d42e4a/sqlite_utils/db.py#L1049 https://github.com/simonw/sqlite-utils/blob/54a2269e91ce72b059618662ed133a85f3d42e4a/sqlite_utils/db.py#L1230 But it's correct here: https://github.com/simonw/sqlite-utils/blob/54a2269e91ce72b059618662ed133a85f3d42e4a/sqlite_utils/db.py#L2470 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/341/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1053122092 | I_kwDOCGYnMM4-xV4s | 339 | `table.lookup()` option to populate additional columns when creating a record | 9599 | closed | 0 | 4 | 2021-11-15T01:41:17Z | 2021-11-15T02:02:34Z | 2021-11-15T02:02:00Z | OWNER | > For the commits table I feel like I want a version of `table.lookup()` that can be passed additional columns to populate only if the record does not exist yet. _Originally posted by @simonw in https://github.com/simonw/git-history/issues/12#issuecomment-967455017_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/339/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1053087862 | I_kwDOCGYnMM4-xNh2 | 338 | dict, list, tuple should all map to TEXT | 9599 | closed | 0 | 0 | 2021-11-15T00:28:01Z | 2021-11-15T00:36:03Z | 2021-11-15T00:36:03Z | OWNER | > This relates to the fact that dictionaries, lists and tuples get special treatment and are converted to JSON strings, using this code: https://github.com/simonw/sqlite-utils/blob/e8d958109ee290cfa1b44ef7a39629bb50ab673e/sqlite_utils/db.py#L2937-L2947 > > So the `COLUMN_TYPE_MAPPING` should include those too - right now it looks like this: https://github.com/simonw/sqlite-utils/blob/e8d958109ee290cfa1b44ef7a39629bb50ab673e/sqlite_utils/db.py#L165-L188 _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/322#issuecomment-968401459_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/338/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
979612115 | MDExOlB1bGxSZXF1ZXN0NzE5OTk4MjI1 | 322 | Add dict type to be mapped as TEXT in sqllite | 2496189 | closed | 0 | 1 | 2021-08-25T20:54:26Z | 2021-11-15T00:27:40Z | 2021-11-15T00:27:40Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/322 | the library deal with Postgres type jsonb as dictionary, add dict type as a TEXT for mapping to sqlite | 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/322/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
988013247 | MDExOlB1bGxSZXF1ZXN0NzI3MDEyOTk2 | 324 | Use python-dateutil package instead of dateutils | 191622 | closed | 0 | 1 | 2021-09-03T18:31:19Z | 2021-11-14T23:25:40Z | 2021-11-14T23:25:40Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/324 | While working on updating `sqlite-utils` for NixOS/Nixpkgs, I came a cross the following: In 5ec6686153e29ae10d4921a1ad4c841f192f20e2, a new dependency was added on `dateutils` (https://pypi.org/project/dateutils/). I believe this is unintentional, and instead `python-dateutil` (https://pypi.org/project/python-dateutil/) was intended. My reasoning is: - `python-dateutil` is imported here in [recipes.py](https://github.com/simonw/sqlite-utils/blob/5ec6686153e29ae10d4921a1ad4c841f192f20e2/sqlite_utils/recipes.py#L1) - The `mypy` `type-python-dateutil` dependency in [setup.py](https://github.com/simonw/sqlite-utils/blob/5ec6686153e29ae10d4921a1ad4c841f192f20e2/setup.py#L36) - `python-dateutil` is a dependency of `dateutils` as seen in the output in [docs/tutorial.ipynb](https://github.com/simonw/sqlite-utils/blob/77c240df56068341561e95e4a412cbfa24dc5bc7/docs/tutorial.ipynb#L43) Seems like the trailing "s" seems to be the source of confusion 😅 I've swapped the dependencies out, hope this helps. | 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/324/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1026794056 | I_kwDOCGYnMM49M6JI | 331 | Mypy error: found module but no type hints or library stubs | 53032010 | closed | 0 | 2 | 2021-10-14T20:29:50Z | 2021-11-14T23:21:08Z | 2021-11-14T23:21:08Z | NONE | ``` Python 3.9.5 mypy 0.910 sqlite-utils 3.17.1 ``` While using sqlite-utils as a library, when I use mypy for static type checking, it throws an error: ``` mypy . src/etl.py:5: error: Skipping analyzing "sqlite_utils": found module but no type hints or library stubs import sqlite_utils ^ src/etl.py:5: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports test/test_etl.py:4: error: Skipping analyzing "sqlite_utils": found module but no type hints or library stubs import sqlite_utils ^ Found 2 errors in 2 files (checked 7 source files) ``` When I add a `py.typed` file to the sqlite-utils package to mark it as PEP 561 compatible, the error goes away. ``` al@nbal ..b/python3.9/site-packages/sqlite_utils (git)-[main] % la total 200 drwx------ 3 al al 4096 Oct 14 22:00 . drwx------ 117 al al 4096 Oct 12 21:12 .. -rw------- 1 al al 64409 Oct 12 21:11 cli.py -rw------- 1 al al 109092 Oct 12 21:11 db.py -rw------- 1 al al 0 Oct 14 22:00 py.typed -rw------- 1 al al 684 Oct 12 21:11 recipes.py -rw------- 1 al al 7988 Oct 12 21:11 utils.py -rw------- 1 al al 113 Oct 12 21:11 __init__.py ``` I would like to suggest adding a `py.typed` file to the repository. See also the mypy docs on creating PEP 561 compatible packages: https://mypy.readthedocs.io/en/stable/installed_packages.html#creating-pep-561-compatible-packages | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/331/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1028056713 | I_kwDOCGYnMM49RuaJ | 332 | `sqlite-utils memory --flatten` option to flatten nested JSON | 22523840 | closed | 0 | 1 | 2021-10-16T14:04:42Z | 2021-11-14T23:05:05Z | 2021-11-14T23:05:05Z | NONE | currently --flatten option works only for `insert` command, it would be cool if it worked for `memory` as well to query nested json | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/332/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1042569687 | I_kwDOCGYnMM4-JFnX | 335 | sqlite-utils index-foreign-keys fails due to pre-existing index | 596279 | closed | 0 | 11 | 2021-11-02T16:22:11Z | 2021-11-14T22:55:56Z | 2021-11-14T22:55:56Z | NONE | While running the command: ```sh sqlite-utils index-foreign-keys $SQLITE_DIR/pudl.sqlite ``` I got the following error: ``` Traceback (most recent call last): File "/home/zane/miniconda3/envs/pudl-dev/bin/sqlite-utils", line 8, in <module> sys.exit(cli()) File "/home/zane/miniconda3/envs/pudl-dev/lib/python3.9/site-packages/click/core.py", line 829, in __call__ return self.main(*args, **kwargs) File "/home/zane/miniconda3/envs/pudl-dev/lib/python3.9/site-packages/click/core.py", line 782, in main rv = self.invoke(ctx) File "/home/zane/miniconda3/envs/pudl-dev/lib/python3.9/site-packages/click/core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/zane/miniconda3/envs/pudl-dev/lib/python3.9/site-packages/click/core.py", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params) File "/home/zane/miniconda3/envs/pudl-dev/lib/python3.9/site-packages/click/core.py", line 610, in invoke return callback(*args, **kwargs) File "/home/zane/miniconda3/envs/pudl-dev/lib/python3.9/site-packages/sqlite_utils/cli.py", line 454, in index_foreign_keys db.index_foreign_keys() File "/home/zane/miniconda3/envs/pudl-dev/lib/python3.9/site-packages/sqlite_utils/db.py", line 902, in index_foreign_keys table.create_index([fk.column]) File "/home/zane/miniconda3/envs/pudl-dev/lib/python3.9/site-packages/sqlite_utils/db.py", line 1563, in create_index self.db.execute(sql) File "/home/zane/miniconda3/envs/pudl-dev/lib/python3.9/site-packages/sqlite_utils/db.py", line 421, in execute return self.conn.execute(sql) sqlite3.OperationalError: index idx_generators_eia860_report_date already exists ``` This DB was created with the foreign key constraint `PRAGMA` enabled and a bunch of column-level `CHECK` constraints. Is this an expected behavior? Should one not try to index foreign keys if FK constraints are already being enforced within the DB? I'm also noticing that the size of the DB after FK … | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/335/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1052851176 | I_kwDOBm6k_c4-wTvo | 1507 | ReadTheDocs build failed for 0.59.2 release | 9599 | closed | 0 | 6 | 2021-11-14T05:24:34Z | 2021-11-14T05:41:55Z | 2021-11-14T05:41:55Z | OWNER | I had to cancel the 0.59.2 release because ReadTheDocs was failing to build the documentation. https://readthedocs.org/projects/datasette/builds/15268454/ ``` /home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/bin/python -m sphinx -T -b html -d _build/doctrees -D language=en . _build/html Running Sphinx v1.8.5 loading translations [en]... done making output directory... building [mo]: targets for 0 po files that are out of date building [html]: targets for 27 source files that are out of date updating environment: 27 added, 0 changed, 0 removed reading sources... [ 3%] authentication Traceback (most recent call last): File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/cmd/build.py", line 304, in build_main app.build(args.force_all, filenames) File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/application.py", line 341, in build self.builder.build_update() File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/builders/__init__.py", line 347, in build_update len(to_build)) File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/builders/__init__.py", line 360, in build updated_docnames = set(self.read()) File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/builders/__init__.py", line 468, in read self._read_serial(docnames) File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/builders/__init__.py", line 490, in _read_serial self.read_doc(docname) File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/builders/__init__.py", line 534, in read_doc doctree = read_doc(self.app, self.env, self.env.doc2path(docname)) File … | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1507/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1050163432 | I_kwDOBm6k_c4-mDjo | 1503 | `?_nocol=` removes that column from the filter interface | 9599 | closed | 0 | 1 | 2021-11-10T18:22:50Z | 2021-11-14T05:08:27Z | 2021-11-14T04:53:07Z | OWNER | e.g. on https://latest.datasette.io/fixtures/sortable?_nocol=sortable <img width="723" alt="fixtures__sortable__201_rows" src="https://user-images.githubusercontent.com/9599/141170777-733b1ab2-c153-4276-8ba2-d975ed84156e.png"> This causes weird behaviour when you e.g. facet by a hidden column, since selecting facets and then re-submitting the form will clear the selected filter. ![nocol-bug](https://user-images.githubusercontent.com/9599/141171135-aded71d1-a4cb-4b7f-a4ea-26828fa98906.gif) | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1503/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1052826038 | I_kwDOBm6k_c4-wNm2 | 1506 | Columns beginning with an underscore do not facet correctly | 9599 | closed | 0 | 1 | 2021-11-14T02:20:32Z | 2021-11-14T04:45:21Z | 2021-11-14T04:45:21Z | OWNER | Datasette treats columns that start with an underscore as querystring parameters it should ignore! <img width="914" alt="bchydro__item_versions__99_918_rows" src="https://user-images.githubusercontent.com/9599/141664864-119990d5-42b2-47f7-91bd-09a74a288af9.png"> Discovered in https://github.com/simonw/git-history/issues/14#issuecomment-968192464 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1506/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1041778507 | I_kwDOCGYnMM4-GEdL | 334 | Filter by datetime objects using rows_where() | 11642379 | closed | 0 | 0 | 2021-11-02T00:44:08Z | 2021-11-13T19:23:21Z | 2021-11-13T19:23:21Z | NONE | Firstly, thanks for this nice utility. It would be nice to have an example in the docs on how to filter by date range using `rows_where()`. This doesn't seem to work: ``` table.rows_where('datetime(created) between datetime("2021-10-31T17:29:59.277428-04:00") AND datetime("2021-11-01T03:44:04.544651+00:00")') ``` I could probably just use `db.query()`, which works for the above, but it would be nice if I could pass in `datetime` objects in `rows_where()`. Thanks. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/334/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
924748955 | MDU6SXNzdWU5MjQ3NDg5NTU= | 1380 | Serve all db files in a folder | 193463 | open | 0 | 5 | 2021-06-18T10:03:32Z | 2021-11-13T08:09:11Z | NONE | I tried to get the `serve` command to serve all the .db files in the `/mnt` folder but is seems that the server does not refresh the list of files. In more detail: * Starting datasette as a docker container with: ``` docker run -p 8001:8001 -v `pwd`:/mnt \ datasetteproject/datasette \ datasette -p 8001 -h 0.0.0.0 /mnt ``` * Datasette correctly serves all the *.db files found in the /mnt folder * When the server is running, if I copy a new file in the $PWD folder, Datasette does not seem to see the new files, forcing me to restart Docker. Is there an option/setting that I overlooked, or is this something missing? BTW, the `--reload` setting, although at first glance is what you think you need, does not seem to do anything in regards of seeing all *.db files. Thanks! | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1380/reactions", "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1051277222 | I_kwDOBm6k_c4-qTem | 1504 | Link to ?_size=max at bottom of table page | 9599 | open | 0 | 0 | 2021-11-11T19:06:33Z | 2021-11-11T19:06:33Z | OWNER | This can have text such as "Show 1,000 rows per page", based on the max size limit setting. Would make it easier for people to see more data at once without having to know how to hack the URL, similar to the `...` for facet sizes I added in #1337. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1504/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1049946823 | I_kwDOBm6k_c4-lOrH | 1502 | Full-text search: No support to unary "-" operator | 516827 | open | 0 | 0 | 2021-11-10T15:11:19Z | 2021-11-10T15:11:19Z | NONE | Reference: https://www.sqlite.org/fts3.html#set_operations_using_the_standard_query_syntax Test: https://fara.datasettes.com/fara/FARA_All_ShortForms?_search=manafort+-freedman&_sort=rowid | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1502/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
1046887492 | PR_kwDODFE5qs4uMsMJ | 9 | Removed space from filename My Activity.json | 91880982 | open | 0 | 0 | 2021-11-08T00:04:31Z | 2021-11-08T00:04:31Z | FIRST_TIME_CONTRIBUTOR | dogsheep/google-takeout-to-sqlite/pulls/9 | File name from google takeout has no space. The code only runs without error if filename is "MyActivity.json" and not "My Activity.json". Is it a new change by Google? | 206649770 | pull | { "url": "https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/9/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
1046271107 | PR_kwDOCGYnMM4uK5z2 | 337 | Default values for `--attach` and `--param` options | 771193 | closed | 0 | 1 | 2021-11-05T21:57:53Z | 2021-11-05T22:33:03Z | 2021-11-05T22:33:02Z | NONE | simonw/sqlite-utils/pulls/337 | It seems that `click` 8.x uses `None` as the default value for `multiple=True` options. This change makes the code forward-compatible with `click` 8.x. See this build failure for more info: https://hydra.nixos.org/build/156926608 | 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/337/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
845794436 | MDU6SXNzdWU4NDU3OTQ0MzY= | 1284 | Feature or Documentation Request: Individual table as home page template | 192568 | open | 0 | 4 | 2021-03-31T03:56:17Z | 2021-11-04T03:15:01Z | CONTRIBUTOR | It would be great to have a sample showing how to move a single database that has a single table, to the index page. I'm trying it now, and find there is a real depth of Datasette and Python understanding that's required to be successful. I've got all the basic jinja concepts down... variables, template control structures, template inheritance, template overrides, css, html, the --template-dir and --static arguments, etc. But copying the table.html file to index.html doesn't work. There are undocumented functions and filters... I can figure some of them out (yay, url_builder.py and utils/__init__.py!) but it's a slog better handled by a much stronger Python developer. One sample would make a world of difference. The ideal form of this documentation would be a diff between the default table.html and how that would look if essentially moved to index.html. The use case is for everyone who wants to create a public-facing website to explore a single table at the root directory. (Maybe a second bit of documentation for people who have a single database with multiple tables.) (Hmm... might be cool to have a setting for that, where it happens automagically! If only one table, then home page is at the table level. if only one database, then home page is at the database level.... as an option.) I suppose I could ignore this, and somehow do this in the DNS settings once I hook up Vercel to a domain name, maybe.. and remove the breadcrumbs in table.html... but for now, a documentation request in the form of a diff... for viewing a single table (or a single database) at the root. (Actually, there's probably room for a whole expanded section on templates. Noticed some nice table metadata in one of the datasette examples, for instance... Hmm... maybe a whole library of solutions in one place... maybe a documentation hackathon! If that's of interest, of course it's a separate issue. ) | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1284/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |