issue_comments
10,495 rows sorted by user
This data as json, CSV (advanced)
issue >30
- Upgrade to CodeMirror 6, add SQL autocomplete 22
- base_url configuration setting 17
- feat: Javascript Plugin API (Custom panels, column menu items with JS actions) 15
- Transformation type `--type DATETIME` 14
- Database page loads too slowly with many large tables (due to table counts) 13
- .json and .csv exports fail to apply base_url 11
- Plugin hook for dynamic metadata 11
- Stream all results for arbitrary SQL and canned queries 10
- Extract columns cannot create foreign key relation: sqlite3.OperationalError: table sqlite_master may not be modified 10
- docker image is duplicating db files somehow 10
- Call for birthday presents: if you're using Datasette, let us know how you're using it here 10
- Documentation with recommendations on running Datasette in production without using Docker 9
- JavaScript plugin hooks mechanism similar to pluggy 9
- De-tangling Metadata before Datasette 1.0 9
- Incorrect URLs when served behind a proxy with base_url set 8
- Improve the display of facets information 8
- Full text search of all tables at once? 7
- Populate "endpoint" key in ASGI scope 7
- Figure out some interesting example SQL queries 7
- Exceeding Cloud Run memory limits when deploying a 4.8G database 7
- create-index should run analyze after creating index 7
- query result page is using 400mb of browser memory 40x size of html page and 400x size of csv data 7
- Add GraphQL endpoint 6
- publish heroku does not work on Windows 10 6
- WIP: Add Gmail takeout mbox import 6
- clean checkout & clean environment has test failures 6
- CLI eats my cursor 6
- Metadata should be a nested arbitrary KV store 5
- Windows installation error 5
- datasette publish lambda plugin 5
- …
id | html_url | issue_url | node_id | user ▼ | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
489353316 | https://github.com/simonw/datasette/issues/187#issuecomment-489353316 | https://api.github.com/repos/simonw/datasette/issues/187 | MDEyOklzc3VlQ29tbWVudDQ4OTM1MzMxNg== | carsonyl 46059 | 2019-05-04T18:36:36Z | 2019-05-04T18:36:36Z | NONE | Hi @simonw - I just hit this issue when trying out Datasette after your PyCon talk today. Datasette is pinned to Sanic 0.7.0, but it looks like 0.8.0 added the option to remove the uvloop dependency for Windows by having an environment variable |
{ "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 1, "eyes": 0 } |
Windows installation error 309033998 | |
620841496 | https://github.com/simonw/datasette/issues/633#issuecomment-620841496 | https://api.github.com/repos/simonw/datasette/issues/633 | MDEyOklzc3VlQ29tbWVudDYyMDg0MTQ5Ng== | nryberg 46165 | 2020-04-28T20:37:50Z | 2020-04-28T20:37:50Z | NONE | Using the Heroku web interface, you can set the WEB_CONCURRENCY = 1 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Publish to Heroku is broken: "WARNING: You must pass the application as an import string to enable 'reload' or 'workers" 522334771 | |
911772943 | https://github.com/dogsheep/evernote-to-sqlite/issues/14#issuecomment-911772943 | https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/14 | IC_kwDOEhK-wc42WI0P | step21 46968 | 2021-09-02T14:53:11Z | 2021-09-02T14:53:11Z | NONE | Additionally, assuming the line numbers match up with the provided enenx file, the mentioned line plus one before and after is as follows: ``` ]]>``` |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
xml.etree.ElementTree.Parse Error - mismatched tag 986829194 | |
374872202 | https://github.com/simonw/datasette/issues/186#issuecomment-374872202 | https://api.github.com/repos/simonw/datasette/issues/186 | MDEyOklzc3VlQ29tbWVudDM3NDg3MjIwMg== | stefanocudini 47107 | 2018-03-21T09:07:22Z | 2018-03-21T09:07:22Z | NONE | --debug is perfect tnk |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
proposal new option to disable user agents cache 306811513 | |
527209840 | https://github.com/simonw/sqlite-utils/pull/56#issuecomment-527209840 | https://api.github.com/repos/simonw/sqlite-utils/issues/56 | MDEyOklzc3VlQ29tbWVudDUyNzIwOTg0MA== | amjith 49260 | 2019-09-02T17:23:21Z | 2019-09-02T17:23:21Z | CONTRIBUTOR | I have updated the other PR with the changes from this one and added tests. I have also changed the escaping from double quotes to brackets. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Escape the table name in populate_fts and search. 487847945 | |
527211047 | https://github.com/simonw/sqlite-utils/pull/57#issuecomment-527211047 | https://api.github.com/repos/simonw/sqlite-utils/issues/57 | MDEyOklzc3VlQ29tbWVudDUyNzIxMTA0Nw== | amjith 49260 | 2019-09-02T17:30:43Z | 2019-09-02T17:30:43Z | CONTRIBUTOR | I have merged the other PR (#56) into this one. I have incorporated your suggestions. Cheers! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add triggers while enabling FTS 487987958 | |
533818697 | https://github.com/simonw/sqlite-utils/issues/61#issuecomment-533818697 | https://api.github.com/repos/simonw/sqlite-utils/issues/61 | MDEyOklzc3VlQ29tbWVudDUzMzgxODY5Nw== | amjith 49260 | 2019-09-21T18:09:01Z | 2019-09-21T18:09:28Z | CONTRIBUTOR | @witeshadow The library version doesn't have helpers around CSV (at least not from what I can see in the code). But here's a snippet that makes it easy to insert from CSV using the library. ``` import csv from sqlite_utils import Database CSV Readercsv_file = open("filename.csv") # open the csv file. reader = csv.reader(csv_file) # Create a CSV reader headers = next(reader) # First line is the header docs = (dict(zip(headers, row)) for row in reader) Now you can use the
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
importing CSV to SQLite as library 491219910 | |
643709037 | https://github.com/simonw/datasette/issues/691#issuecomment-643709037 | https://api.github.com/repos/simonw/datasette/issues/691 | MDEyOklzc3VlQ29tbWVudDY0MzcwOTAzNw== | amjith 49260 | 2020-06-14T02:35:16Z | 2020-06-14T02:35:16Z | CONTRIBUTOR | The server should reload in the Ref: #848 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
--reload sould reload server if code in --plugins-dir changes 574021194 | |
652990131 | https://github.com/simonw/datasette/issues/889#issuecomment-652990131 | https://api.github.com/repos/simonw/datasette/issues/889 | MDEyOklzc3VlQ29tbWVudDY1Mjk5MDEzMQ== | amjith 49260 | 2020-07-02T12:58:11Z | 2020-07-02T13:00:18Z | CONTRIBUTOR | FWIW, this error does NOT happen in datasette 0.45a4. It only started on 0.45a5 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
asgi_wrapper plugin hook is crashing at startup 649907676 | |
653002499 | https://github.com/simonw/datasette/issues/889#issuecomment-653002499 | https://api.github.com/repos/simonw/datasette/issues/889 | MDEyOklzc3VlQ29tbWVudDY1MzAwMjQ5OQ== | amjith 49260 | 2020-07-02T13:22:13Z | 2020-07-02T13:22:13Z | CONTRIBUTOR | I was able to narrow this down to the fact that lifespan protocol is turned on. I see the workaround you've used here: https://github.com/simonw/datasette-debug-asgi/commit/72d568d32a3159c763ce908c0b269736935c6987 If so, maybe it's time to update some of the asg_wrapper plugins. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
asgi_wrapper plugin hook is crashing at startup 649907676 | |
346974336 | https://github.com/simonw/datasette/issues/141#issuecomment-346974336 | https://api.github.com/repos/simonw/datasette/issues/141 | MDEyOklzc3VlQ29tbWVudDM0Njk3NDMzNg== | janimo 50138 | 2017-11-26T00:00:35Z | 2017-11-26T00:00:35Z | NONE | FWIW I worked around this by setting TMPDIR to ~/tmp before running the command. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
datasette publish can fail if /tmp is on a different device 275814941 | |
346987395 | https://github.com/simonw/datasette/issues/124#issuecomment-346987395 | https://api.github.com/repos/simonw/datasette/issues/124 | MDEyOklzc3VlQ29tbWVudDM0Njk4NzM5NQ== | janimo 50138 | 2017-11-26T06:24:08Z | 2017-11-26T06:24:08Z | NONE | Are there performance gains when using immutable as opposed to read-only? From what I see other processes can still modify the DB when immutable, but there are no change notifications. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Option to open readonly but not immutable 275125805 | |
347123991 | https://github.com/simonw/datasette/issues/124#issuecomment-347123991 | https://api.github.com/repos/simonw/datasette/issues/124 | MDEyOklzc3VlQ29tbWVudDM0NzEyMzk5MQ== | janimo 50138 | 2017-11-27T09:25:15Z | 2017-11-27T09:25:15Z | NONE | That's the only reference to immutable I saw as well, making me think that there may be no perceivable advantages over simply using mode=ro. Since the database is never or seldom updated the change notifications should not impact performance. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Option to open readonly but not immutable 275125805 | |
344125441 | https://github.com/simonw/datasette/pull/81#issuecomment-344125441 | https://api.github.com/repos/simonw/datasette/issues/81 | MDEyOklzc3VlQ29tbWVudDM0NDEyNTQ0MQ== | jefftriplett 50527 | 2017-11-14T02:24:54Z | 2017-11-14T02:24:54Z | CONTRIBUTOR | Oops, if I jumped the gun. I saw the project in my github activity feed and saw some low hanging fruit :) |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
:fire: Removes DS_Store 273595473 | |
735279355 | https://github.com/simonw/datasette/pull/1112#issuecomment-735279355 | https://api.github.com/repos/simonw/datasette/issues/1112 | MDEyOklzc3VlQ29tbWVudDczNTI3OTM1NQ== | jefftriplett 50527 | 2020-11-28T19:21:09Z | 2020-11-28T19:21:09Z | CONTRIBUTOR | (Even more annoying is that I see my editor leaked an extra delete space at the end of the line. I'm happy to rebuild this to be less annoying, but you probably don't want the changelog update either way) |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Fix --metadata doc usage 752749485 | |
735281577 | https://github.com/simonw/datasette/issues/493#issuecomment-735281577 | https://api.github.com/repos/simonw/datasette/issues/493 | MDEyOklzc3VlQ29tbWVudDczNTI4MTU3Nw== | jefftriplett 50527 | 2020-11-28T19:39:53Z | 2020-11-28T19:39:53Z | CONTRIBUTOR | I was confused by A short term solution might be pointing out in plugins that their snippet json can go in |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Rename metadata.json to config.json 449886319 | |
748305976 | https://github.com/simonw/datasette/issues/493#issuecomment-748305976 | https://api.github.com/repos/simonw/datasette/issues/493 | MDEyOklzc3VlQ29tbWVudDc0ODMwNTk3Ng== | jefftriplett 50527 | 2020-12-18T20:34:39Z | 2020-12-18T20:34:39Z | CONTRIBUTOR | I can't keep up with the renaming contexts, but I like having the ability to run datasette+ datasette-ripgrep against different configs:
I have one for all of my code and one per client who has lots of code. So as long as I can point to datasette to something, it's easy to work with. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Rename metadata.json to config.json 449886319 | |
1224382336 | https://github.com/simonw/sqlite-utils/issues/467#issuecomment-1224382336 | https://api.github.com/repos/simonw/sqlite-utils/issues/467 | IC_kwDOCGYnMM5I-peA | jefftriplett 50527 | 2022-08-23T17:16:13Z | 2022-08-23T17:16:13Z | CONTRIBUTOR |
I would lean towards not dropping them (or making a I do like that sqlite-utils mostly just works with existing tables but it's also nice to add to existing fields in a few cases. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Mechanism for ensuring a table has all the columns 1348169997 | |
1256781274 | https://github.com/simonw/datasette/issues/1817#issuecomment-1256781274 | https://api.github.com/repos/simonw/datasette/issues/1817 | IC_kwDOBm6k_c5K6PXa | jefftriplett 50527 | 2022-09-23T22:59:46Z | 2022-09-23T22:59:46Z | CONTRIBUTOR | While you are adding features, would you be future-proofing your APIs if you switched over some arguments over to keyword-only arguments or would that be too disruptive? Thinking out loud:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Expose `sql` and `params` arguments to various plugin hooks 1384273985 | |
913001282 | https://github.com/simonw/datasette/pull/1455#issuecomment-913001282 | https://api.github.com/repos/simonw/datasette/issues/1455 | IC_kwDOBm6k_c42a0tC | ctb 51016 | 2021-09-04T16:31:24Z | 2021-09-04T16:31:24Z | CONTRIBUTOR | I love it! maybe 'researchers' instead? Or 'scientists and researchers'? |
{ "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add scientists to target groups 988325628 | |
915279711 | https://github.com/simonw/datasette/issues/1464#issuecomment-915279711 | https://api.github.com/repos/simonw/datasette/issues/1464 | IC_kwDOBm6k_c42jg9f | ctb 51016 | 2021-09-08T14:16:49Z | 2021-09-08T14:16:49Z | CONTRIBUTOR | on commit d57ab156b35ec642 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
clean checkout & clean environment has test failures 991191951 | |
915302885 | https://github.com/simonw/datasette/issues/1464#issuecomment-915302885 | https://api.github.com/repos/simonw/datasette/issues/1464 | IC_kwDOBm6k_c42jmnl | ctb 51016 | 2021-09-08T14:44:50Z | 2021-09-08T14:44:50Z | CONTRIBUTOR | thanks for the response! full errors attached; excerpt: ``` ...
/Users/t/dev/datasette/tests/test_api.py:1115: AssertionError ``` A quick scan of #1223 suggests you're right. Unfortunately, pysqlite3-binary isn't available for Mac OS X, so I can't quickly check that that fixes it; will do so later. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
clean checkout & clean environment has test failures 991191951 | |
917642487 | https://github.com/simonw/datasette/issues/1464#issuecomment-917642487 | https://api.github.com/repos/simonw/datasette/issues/1464 | IC_kwDOBm6k_c42shz3 | ctb 51016 | 2021-09-12T14:03:09Z | 2021-09-12T14:03:09Z | CONTRIBUTOR | haven't had time to get back to this, but idle thought that I'm recording for later investigation: how does the continuous integration handle this installation issue? Is it documented there? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
clean checkout & clean environment has test failures 991191951 | |
974711959 | https://github.com/simonw/datasette/issues/1426#issuecomment-974711959 | https://api.github.com/repos/simonw/datasette/issues/1426 | IC_kwDOBm6k_c46GOyX | tannewt 52649 | 2021-11-20T21:11:51Z | 2021-11-20T21:11:51Z | NONE | I think another thing would be to make |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Manage /robots.txt in Datasette core, block robots by default 964322136 | |
1115542067 | https://github.com/simonw/datasette/issues/1732#issuecomment-1115542067 | https://api.github.com/repos/simonw/datasette/issues/1732 | IC_kwDOBm6k_c5CfdIz | tannewt 52649 | 2022-05-03T01:50:44Z | 2022-05-03T01:50:44Z | NONE | I haven’t set one up unfortunately. My time is very limited because we just had a baby. On Mon, May 2, 2022, at 6:42 PM, Simon Willison wrote:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Custom page variables aren't decoded 1221849746 | |
1671649530 | https://github.com/simonw/datasette/issues/2133#issuecomment-1671649530 | https://api.github.com/repos/simonw/datasette/issues/2133 | IC_kwDOBm6k_c5jo1j6 | HaveF 54462 | 2023-08-09T15:41:14Z | 2023-08-09T15:41:14Z | NONE | Yes, using this approach( I'm curious about the results of the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
[feature request]`datasette install plugins.json` options 1841501975 | |
1672360472 | https://github.com/simonw/datasette/issues/2133#issuecomment-1672360472 | https://api.github.com/repos/simonw/datasette/issues/2133 | IC_kwDOBm6k_c5jrjIY | HaveF 54462 | 2023-08-10T00:31:24Z | 2023-08-10T00:31:24Z | NONE | It looks very nice now. Finally, no more manual installation of plugins one by one. Thank you, Simon! ❤️ |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
[feature request]`datasette install plugins.json` options 1841501975 | |
344810525 | https://github.com/simonw/datasette/issues/46#issuecomment-344810525 | https://api.github.com/repos/simonw/datasette/issues/46 | MDEyOklzc3VlQ29tbWVudDM0NDgxMDUyNQ== | ingenieroariel 54999 | 2017-11-16T04:11:25Z | 2017-11-16T04:11:25Z | CONTRIBUTOR | @simonw On the spatialite support, here is some info to make it work and a screenshot: I used the following Dockerfile: ``` FROM prolocutor/python3-sqlite-ext:3.5.1-spatialite as build RUN mkdir /code ADD . /code/ RUN pip install /code/ EXPOSE 8001 CMD ["datasette", "serve", "/code/ne.sqlite", "--host", "0.0.0.0"] ``` and added this to |
{ "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 1, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Dockerfile should build more recent SQLite with FTS5 and spatialite support 271301468 | |
345002908 | https://github.com/simonw/datasette/issues/46#issuecomment-345002908 | https://api.github.com/repos/simonw/datasette/issues/46 | MDEyOklzc3VlQ29tbWVudDM0NTAwMjkwOA== | ingenieroariel 54999 | 2017-11-16T17:47:49Z | 2017-11-16T17:47:49Z | CONTRIBUTOR | I'll try to find alternatives to the Dockerfile option - I also think we should not use that old one without sources or license. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Dockerfile should build more recent SQLite with FTS5 and spatialite support 271301468 | |
754911290 | https://github.com/simonw/datasette/issues/1171#issuecomment-754911290 | https://api.github.com/repos/simonw/datasette/issues/1171 | MDEyOklzc3VlQ29tbWVudDc1NDkxMTI5MA== | rcoup 59874 | 2021-01-05T21:31:15Z | 2021-01-05T21:31:15Z | NONE | We did this for Sno under macOS — it's a PyInstaller binary/setup which uses Packages for packaging.
FYI (if you ever get to it) for Windows you need to get a code signing certificate. And if you want automated CI, you'll want to get an "EV CodeSigning for HSM" certificate from GlobalSign, which then lets you put the certificate into Azure Key Vault. Which you can use with azuresigntool to sign your code & installer. (Non-EV certificates are a waste of time, the user still gets big warnings at install time). |
{ "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 1, "rocket": 0, "eyes": 0 } |
GitHub Actions workflow to build and sign macOS binary executables 778450486 | |
1747231893 | https://github.com/simonw/sqlite-utils/issues/433#issuecomment-1747231893 | https://api.github.com/repos/simonw/sqlite-utils/issues/433 | IC_kwDOCGYnMM5oJKSV | spookylukey 62745 | 2023-10-04T16:15:09Z | 2023-10-04T16:28:21Z | CONTRIBUTOR | I confirm the bug, as above, and that @jonafato 's patch fixes it for me. However, it's not the right fix. The problem is that ProgressBar is being used in the wrong way. This also results in two lines being printed instead of one, like this:
```json { "version": "0.2.0", "configurations": [ { "name": "Python: Module", "type": "python", "request": "launch", "module": "sqlite_utils", "justMyCode": false, "args": ["insert", "test.db", "test", "--csv", "tests/sniff/example1.csv"] } ] } ``` [edit - deleted my analysis of why the current code is wrong, which was confused and confusing] |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
CLI eats my cursor 1239034903 | |
344424382 | https://github.com/simonw/datasette/issues/93#issuecomment-344424382 | https://api.github.com/repos/simonw/datasette/issues/93 | MDEyOklzc3VlQ29tbWVudDM0NDQyNDM4Mg== | atomotic 67420 | 2017-11-14T22:42:16Z | 2017-11-14T22:42:16Z | NONE | tried quickly, this seems working: ``` ~ pip3 install pyinstaller ~ pyinstaller -F --add-data /usr/local/lib/python3.6/site-packages/datasette/templates:datasette/templates --add-data /usr/local/lib/python3.6/site-packages/datasette/static:datasette/static /usr/local/bin/datasette ~ du -h dist/datasette 6.8M dist/datasette ~ file dist/datasette dist/datasette: Mach-O 64-bit executable x86_64 ``` |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Package as standalone binary 273944952 | |
344430299 | https://github.com/simonw/datasette/issues/93#issuecomment-344430299 | https://api.github.com/repos/simonw/datasette/issues/93 | MDEyOklzc3VlQ29tbWVudDM0NDQzMDI5OQ== | atomotic 67420 | 2017-11-14T23:06:33Z | 2017-11-14T23:06:33Z | NONE | i will look better tomorrow, it's late i surely made some mistake https://asciinema.org/a/ZyAWbetrlriDadwWyVPUWB94H |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Package as standalone binary 273944952 | |
344516406 | https://github.com/simonw/datasette/issues/93#issuecomment-344516406 | https://api.github.com/repos/simonw/datasette/issues/93 | MDEyOklzc3VlQ29tbWVudDM0NDUxNjQwNg== | atomotic 67420 | 2017-11-15T08:09:41Z | 2017-11-15T08:09:41Z | NONE | actually you can use travis to build for linux/macos and appveyor to build for windows. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Package as standalone binary 273944952 | |
1006708046 | https://github.com/dogsheep/dogsheep-photos/pull/36#issuecomment-1006708046 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/36 | IC_kwDOD079W848ASVO | scoates 71983 | 2022-01-06T16:04:46Z | 2022-01-06T16:04:46Z | NONE | This one got me, today, too. 👍 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Correct naming of tool in readme 988493790 | |
645515103 | https://github.com/dogsheep/twitter-to-sqlite/issues/47#issuecomment-645515103 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/47 | MDEyOklzc3VlQ29tbWVudDY0NTUxNTEwMw== | hpk42 73579 | 2020-06-17T17:30:01Z | 2020-06-17T17:30:01Z | NONE | It's the one with python3.7::
On Wed, Jun 17, 2020 at 10:24 -0700, Simon Willison wrote:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Fall back to FTS4 if FTS5 is not available 639542974 | |
1685471752 | https://github.com/simonw/datasette/issues/2145#issuecomment-1685471752 | https://api.github.com/repos/simonw/datasette/issues/2145 | IC_kwDOBm6k_c5kdkII | pkulchenko 77071 | 2023-08-21T01:07:23Z | 2023-08-21T01:07:23Z | NONE | @simonw, since you're referencing "rowid" column by name, I just want to note that there may be an existing rowid column with completely different semantics (https://www.sqlite.org/lang_createtable.html#rowid), which is likely to break this logic. I don't see a good way to detect a proper "rowid" name short of checking if there is a field with that name and using the alternative ( In terms of the original issue, maybe a way to deal with it is to use rowid by default and then use primary key for WITHOUT ROWID tables (as they are guaranteed to be not null), but I suspect it may require significant changes to the API (and doesn't fully address the issue of what value to pass to indicate NULL when editing records). Would it make sense to generate a random string to indicate NULL values when editing? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
If a row has a primary key of `null` various things break 1857234285 | |
1690799608 | https://github.com/simonw/datasette/issues/2143#issuecomment-1690799608 | https://api.github.com/repos/simonw/datasette/issues/2143 | IC_kwDOBm6k_c5kx434 | pkulchenko 77071 | 2023-08-24T00:09:47Z | 2023-08-24T00:10:41Z | NONE | @simonw, FWIW, I do exactly the same thing for one of my projects (both to allow multiple configuration files to be passed on the command line and setting individual values) and it works quite well for me and my users. I even use the same parameter name for both (https://studio.zerobrane.com/doc-configuration#configuration-via-command-line), but I understand why you may want to use different ones for files and individual values. There is one small difference that I accept code snippets, but I don't think it matters much in this case. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
De-tangling Metadata before Datasette 1.0 1855885427 | |
414860009 | https://github.com/simonw/datasette/issues/267#issuecomment-414860009 | https://api.github.com/repos/simonw/datasette/issues/267 | MDEyOklzc3VlQ29tbWVudDQxNDg2MDAwOQ== | annapowellsmith 78156 | 2018-08-21T23:57:51Z | 2018-08-21T23:57:51Z | NONE | Looks to me like hashing, redirects and caching were documented as part of https://github.com/simonw/datasette/commit/788a542d3c739da5207db7d1fb91789603cdd336#diff-3021b0e065dce289c34c3b49b3952a07 - so perhaps this can be closed? :tada: |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Documentation for URL hashing, redirects and cache policy 323716411 | |
1737906995 | https://github.com/simonw/datasette/pull/2155#issuecomment-1737906995 | https://api.github.com/repos/simonw/datasette/issues/2155 | IC_kwDOBm6k_c5nllsz | cadeef 79087 | 2023-09-27T18:44:02Z | 2023-09-27T18:44:02Z | NONE | @simonw Any chance we can get this tiny patch merged for an upcoming release? |
{ "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Fix hupper.start_reloader entry point 1865572575 | |
643083451 | https://github.com/simonw/datasette/issues/838#issuecomment-643083451 | https://api.github.com/repos/simonw/datasette/issues/838 | MDEyOklzc3VlQ29tbWVudDY0MzA4MzQ1MQ== | tsibley 79913 | 2020-06-12T06:04:14Z | 2020-06-12T06:04:14Z | NONE | Hmm, I haven't tried removing Looking a little more at the code, I think the issue here is that the behaviour of In a mount situation, it is perfectly fine to construct URLs reusing the domain and path from the request. In a proxy situation, it never is, as the domain and path in the request are not the domain and path that the non-proxy client actually needs to use. That is, links which include the Apache → Datasette request origin, The tests you pointed to also reflect this in two ways:
Apps that support being proxied automatically support being mounted, but apps that only support being mounted don't automatically support being proxied. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Incorrect URLs when served behind a proxy with base_url set 637395097 | |
655018966 | https://github.com/simonw/sqlite-utils/pull/118#issuecomment-655018966 | https://api.github.com/repos/simonw/sqlite-utils/issues/118 | MDEyOklzc3VlQ29tbWVudDY1NTAxODk2Ng== | tsibley 79913 | 2020-07-07T17:41:06Z | 2020-07-07T17:41:06Z | CONTRIBUTOR | Hmm, while tests pass, this may not work as intended on larger datasets. Looking into it. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add insert --truncate option 651844316 | |
655052451 | https://github.com/simonw/sqlite-utils/pull/118#issuecomment-655052451 | https://api.github.com/repos/simonw/sqlite-utils/issues/118 | MDEyOklzc3VlQ29tbWVudDY1NTA1MjQ1MQ== | tsibley 79913 | 2020-07-07T18:45:23Z | 2020-07-07T18:45:23Z | CONTRIBUTOR | Ah, I see the problem. The truncate is inside a loop I didn't realize was there. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add insert --truncate option 651844316 | |
655239728 | https://github.com/simonw/sqlite-utils/pull/118#issuecomment-655239728 | https://api.github.com/repos/simonw/sqlite-utils/issues/118 | MDEyOklzc3VlQ29tbWVudDY1NTIzOTcyOA== | tsibley 79913 | 2020-07-08T02:16:42Z | 2020-07-08T02:16:42Z | CONTRIBUTOR | I fixed my original oops by moving the I wanted to make the DELETE + INSERT happen all in the same transaction so it was robust, but that was more complicated than I expected. The transaction handling in the Database/Table classes isn't systematic, and this poses big hurdles to making For example, I wanted to do this (whitespace ignored in diff, so indentation change not highlighted): ```diff diff --git a/sqlite_utils/db.py b/sqlite_utils/db.py index d6b9ecf..4107ceb 100644 --- a/sqlite_utils/db.py +++ b/sqlite_utils/db.py @@ -1028,6 +1028,11 @@ class Table(Queryable): batch_size = max(1, min(batch_size, SQLITE_MAX_VARS // num_columns)) self.last_rowid = None self.last_pk = None + with self.db.conn: + # Explicit BEGIN is necessary because Python's sqlite3 doesn't + # issue implicit BEGINs for DDL, only DML. We mix DDL and DML + # below and might execute DDL first, e.g. for table creation. + self.db.conn.execute("BEGIN") if truncate and self.exists(): self.db.conn.execute("DELETE FROM [{}];".format(self.name)) for chunk in chunks(itertools.chain([first_record], records), batch_size): @@ -1038,7 +1043,11 @@ class Table(Queryable): # Use the first batch to derive the table names column_types = suggest_column_types(chunk) column_types.update(columns or {}) - self.create( + # Not self.create() because that is wrapped in its own + # transaction and Python's sqlite3 doesn't support + # nested transactions. + self.db.create_table( + self.name, column_types, pk, foreign_keys, @@ -1139,7 +1148,6 @@ class Table(Queryable): flat_values = list(itertools.chain(*values)) queries_and_params = [(sql, flat_values)]
but that fails in tests because other methods call Stepping back, it would be nice to make the transaction handling systematic and predictable. One way to do this is to make the There is also the caveat that for each transaction, an explicit |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add insert --truncate option 651844316 | |
655643078 | https://github.com/simonw/sqlite-utils/pull/118#issuecomment-655643078 | https://api.github.com/repos/simonw/sqlite-utils/issues/118 | MDEyOklzc3VlQ29tbWVudDY1NTY0MzA3OA== | tsibley 79913 | 2020-07-08T17:05:59Z | 2020-07-08T17:05:59Z | CONTRIBUTOR |
Ah, yes, thanks for this reminder! I've repushed with doc bits added. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add insert --truncate option 651844316 | |
655652679 | https://github.com/simonw/sqlite-utils/issues/121#issuecomment-655652679 | https://api.github.com/repos/simonw/sqlite-utils/issues/121 | MDEyOklzc3VlQ29tbWVudDY1NTY1MjY3OQ== | tsibley 79913 | 2020-07-08T17:24:46Z | 2020-07-08T17:24:46Z | CONTRIBUTOR | Better transaction handling would be really great. Some of my thoughts on implementing better transaction discipline are in https://github.com/simonw/sqlite-utils/pull/118#issuecomment-655239728. My preferences:
```python db = sqlite_utils.Database(path) with db: # ← BEGIN issued here by Database.enter db.insert(…) db.create_view(…) ← COMMIT/ROLLBACK issue here by sqlite3.connection.exit``` |
{ "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Improved (and better documented) support for transactions 652961907 | |
655898722 | https://github.com/simonw/sqlite-utils/issues/121#issuecomment-655898722 | https://api.github.com/repos/simonw/sqlite-utils/issues/121 | MDEyOklzc3VlQ29tbWVudDY1NTg5ODcyMg== | tsibley 79913 | 2020-07-09T04:53:08Z | 2020-07-09T04:53:08Z | CONTRIBUTOR | Yep, I agree that makes more sense for backwards compat and more casual use cases. I think it should be possible for the Database/Queryable methods to DTRT based on seeing if it's within a context-manager-managed transaction. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Improved (and better documented) support for transactions 652961907 | |
790857004 | https://github.com/simonw/datasette/issues/1238#issuecomment-790857004 | https://api.github.com/repos/simonw/datasette/issues/1238 | MDEyOklzc3VlQ29tbWVudDc5MDg1NzAwNA== | tsibley 79913 | 2021-03-04T19:06:55Z | 2021-03-04T19:06:55Z | NONE | @rgieseke Ah, that's super helpful. Thank you for the workaround for now! |
{ "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Custom pages don't work with base_url setting 813899472 | |
795893813 | https://github.com/simonw/datasette/issues/838#issuecomment-795893813 | https://api.github.com/repos/simonw/datasette/issues/838 | MDEyOklzc3VlQ29tbWVudDc5NTg5MzgxMw== | tsibley 79913 | 2021-03-10T18:43:39Z | 2021-03-10T18:43:39Z | NONE | @simonw Unfortunately this issue as I reported it is not actually solved in version 0.55. Every link which is returned by the What I wrote originally still stands:
Would you prefer to re-open this issue or have me create a new one? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Incorrect URLs when served behind a proxy with base_url set 637395097 | |
795939998 | https://github.com/simonw/datasette/issues/838#issuecomment-795939998 | https://api.github.com/repos/simonw/datasette/issues/838 | MDEyOklzc3VlQ29tbWVudDc5NTkzOTk5OA== | tsibley 79913 | 2021-03-10T19:16:55Z | 2021-03-10T19:16:55Z | NONE | Nod. The problem with the tests is that they're ignoring the origin (hostname, port) of links. In a reverse proxy situation, the frontend request origin is different than the backend request origin. The problem is Datasette generates links with the backend request origin. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Incorrect URLs when served behind a proxy with base_url set 637395097 | |
795950636 | https://github.com/simonw/datasette/issues/838#issuecomment-795950636 | https://api.github.com/repos/simonw/datasette/issues/838 | MDEyOklzc3VlQ29tbWVudDc5NTk1MDYzNg== | tsibley 79913 | 2021-03-10T19:24:13Z | 2021-03-10T19:24:13Z | NONE | I think this could be solved by one of:
Option 1 seems like the easiest to me, if you can get away with never having to generate an absolute URL. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Incorrect URLs when served behind a proxy with base_url set 637395097 | |
876213177 | https://github.com/simonw/datasette/issues/1388#issuecomment-876213177 | https://api.github.com/repos/simonw/datasette/issues/1388 | MDEyOklzc3VlQ29tbWVudDg3NjIxMzE3Nw== | aslakr 80737 | 2021-07-08T07:47:17Z | 2021-07-08T07:47:17Z | CONTRIBUTOR |
Yes, in some cases it is easer to use e.g. Apache's ProxyPass Directive with Unix Domain Socket like |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Serve using UNIX domain socket 939051549 | |
360535979 | https://github.com/simonw/datasette/issues/179#issuecomment-360535979 | https://api.github.com/repos/simonw/datasette/issues/179 | MDEyOklzc3VlQ29tbWVudDM2MDUzNTk3OQ== | psychemedia 82988 | 2018-01-25T17:18:24Z | 2018-01-25T17:18:24Z | CONTRIBUTOR | To summarise that thread:
It could also be useful to allow users to import a python file containing custom functions that can that be loaded into scope and made available to custom templates. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
More metadata options for template authors 288438570 | |
401310732 | https://github.com/simonw/datasette/issues/276#issuecomment-401310732 | https://api.github.com/repos/simonw/datasette/issues/276 | MDEyOklzc3VlQ29tbWVudDQwMTMxMDczMg== | psychemedia 82988 | 2018-06-29T10:05:04Z | 2018-06-29T10:07:25Z | CONTRIBUTOR | @russs Different map projections can presumably be handled on the client side using a leaflet plugin to transform the geometry (eg kartena/Proj4Leaflet) although the leaflet side would need to detect or be informed of the original projection? Another possibility would be to provide an easy way/guidance for users to create an FK'd table containing the WGS84 projection of a non-WGS84 geometry in the original/principle table? This could then as a proxy for serving GeoJSON to the leaflet map? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Handle spatialite geometry columns better 324835838 | |
435862009 | https://github.com/simonw/datasette/issues/371#issuecomment-435862009 | https://api.github.com/repos/simonw/datasette/issues/371 | MDEyOklzc3VlQ29tbWVudDQzNTg2MjAwOQ== | psychemedia 82988 | 2018-11-05T12:48:35Z | 2018-11-05T12:48:35Z | CONTRIBUTOR | I think you need to register a domain name you own separately in order to get a non-IP address address? https://www.digitalocean.com/docs/networking/dns/ |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
datasette publish digitalocean plugin 377156339 | |
436037692 | https://github.com/simonw/datasette/issues/370#issuecomment-436037692 | https://api.github.com/repos/simonw/datasette/issues/370 | MDEyOklzc3VlQ29tbWVudDQzNjAzNzY5Mg== | psychemedia 82988 | 2018-11-05T21:15:47Z | 2018-11-05T21:18:37Z | CONTRIBUTOR | In terms of integration with
The |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Integration with JupyterLab 377155320 | |
436042445 | https://github.com/simonw/datasette/issues/370#issuecomment-436042445 | https://api.github.com/repos/simonw/datasette/issues/370 | MDEyOklzc3VlQ29tbWVudDQzNjA0MjQ0NQ== | psychemedia 82988 | 2018-11-05T21:30:42Z | 2018-11-05T21:31:48Z | CONTRIBUTOR | Another route would be something like creating a |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Integration with JupyterLab 377155320 | |
459915995 | https://github.com/simonw/datasette/issues/160#issuecomment-459915995 | https://api.github.com/repos/simonw/datasette/issues/160 | MDEyOklzc3VlQ29tbWVudDQ1OTkxNTk5NQ== | psychemedia 82988 | 2019-02-02T00:43:16Z | 2019-02-02T00:58:20Z | CONTRIBUTOR | Do you have any simple working examples of how to use If Use case is here: https://github.com/psychemedia/jupyterserverproxy-datasette-demo Trying to do a really simple |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Ability to bundle and serve additional static files 278208011 | |
464341721 | https://github.com/simonw/sqlite-utils/issues/8#issuecomment-464341721 | https://api.github.com/repos/simonw/sqlite-utils/issues/8 | MDEyOklzc3VlQ29tbWVudDQ2NDM0MTcyMQ== | psychemedia 82988 | 2019-02-16T12:08:41Z | 2019-02-16T12:08:41Z | NONE | We also get an error if a column name contains a |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Problems handling column names containing spaces or - 403922644 | |
474280581 | https://github.com/simonw/datasette/issues/417#issuecomment-474280581 | https://api.github.com/repos/simonw/datasette/issues/417 | MDEyOklzc3VlQ29tbWVudDQ3NDI4MDU4MQ== | psychemedia 82988 | 2019-03-19T10:06:42Z | 2019-03-19T10:06:42Z | CONTRIBUTOR | This would be really interesting but several possibilities in use arise, I think? For example:
CSV files may also have messy names compared to the table you want. Or for an update CSV, may have the form |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Datasette Library 421546944 | |
474282321 | https://github.com/simonw/datasette/issues/412#issuecomment-474282321 | https://api.github.com/repos/simonw/datasette/issues/412 | MDEyOklzc3VlQ29tbWVudDQ3NDI4MjMyMQ== | psychemedia 82988 | 2019-03-19T10:09:46Z | 2019-03-19T10:09:46Z | CONTRIBUTOR | Does this also relate to https://github.com/simonw/datasette/issues/283 and the ability to |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Linked Data(sette) 411257981 | |
480621924 | https://github.com/simonw/sqlite-utils/issues/18#issuecomment-480621924 | https://api.github.com/repos/simonw/sqlite-utils/issues/18 | MDEyOklzc3VlQ29tbWVudDQ4MDYyMTkyNA== | psychemedia 82988 | 2019-04-07T19:31:42Z | 2019-04-07T19:31:42Z | NONE | I've just noticed that SQLite lets you IGNORE inserts that collide with a pre-existing key. This can be quite handy if you have a dataset that keeps changing in part, and you don't want to upsert and replace pre-existing PK rows but you do want to ignore collisions to existing PK rows. Do |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
.insert/.upsert/.insert_all/.upsert_all should add missing columns 413871266 | |
482994231 | https://github.com/simonw/sqlite-utils/issues/8#issuecomment-482994231 | https://api.github.com/repos/simonw/sqlite-utils/issues/8 | MDEyOklzc3VlQ29tbWVudDQ4Mjk5NDIzMQ== | psychemedia 82988 | 2019-04-14T15:04:07Z | 2019-04-14T15:29:33Z | NONE | PLEASE IGNORE THE BELOW... I did a package update and rebuilt the kernel I was working in... may just have been an old version of sqlite_utils, seems to be working now. (Too many containers / too many environments!) Has an issue been reintroduced here with FTS? eg I'm getting an error thrown by spaces in column names here: ``` /usr/local/lib/python3.7/site-packages/sqlite_utils/db.py in insert_all(self, records, pk, foreign_keys, upsert, batch_size, column_order) def enable_fts(self, columns, fts_version="FTS5"): --> 329 "Enables FTS on the specified columns" 330 sql = """ 331 CREATE VIRTUAL TABLE "{table}_fts" USING {fts_version} ( ``` when trying an Also, if a col has a ``` /usr/local/lib/python3.7/site-packages/sqlite_utils/db.py in insert_all(self, records, pk, foreign_keys, upsert, batch_size, column_order) 327 jsonify_if_needed(record.get(key, None)) for key in all_columns 328 ) --> 329 result = self.db.conn.execute(sql, values) 330 self.db.conn.commit() 331 self.last_id = result.lastrowid OperationalError: near ".": syntax error ``` (Can't post a worked minimal example right now; racing trying to build something against a live timing screen that will stop until next weekend in an hour or two...) PS Hmmm I did a test and they seem to work; I must be messing up s/where else... ``` import sqlite3 from sqlite_utils import Database dbname='testingDB_sqlite_utils.db' !rm $dbnameconn = sqlite3.connect(dbname, timeout=10) Setup database tablesc = conn.cursor() setup=''' CREATE TABLE IF NOT EXISTS "test1" ( "NO" INTEGER, "NAME" TEXT ); CREATE TABLE IF NOT EXISTS "test2" (
"NO" INTEGER,
CREATE TABLE IF NOT EXISTS "test3" (
"NO" INTEGER,
c.executescript(setup) DB = Database(conn) import pandas as pd df1 = pd.DataFrame({'NO':[1,2],'NAME':['a','b']}) DB['test1'].insert_all(df1.to_dict(orient='records')) df2 = pd.DataFrame({'NO':[1,2],'TIME OF DAY':['early on','late']}) DB['test2'].insert_all(df2.to_dict(orient='records')) df3 = pd.DataFrame({'NO':[1,2],'AVG. SPEED (MPH)':['123.3','123.4']}) DB['test3'].insert_all(df3.to_dict(orient='records')) ``` all seem to work ok. I'm still getting errors in my set up though, which is not too different to the text cases? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Problems handling column names containing spaces or - 403922644 | |
483017176 | https://github.com/simonw/datasette/issues/431#issuecomment-483017176 | https://api.github.com/repos/simonw/datasette/issues/431 | MDEyOklzc3VlQ29tbWVudDQ4MzAxNzE3Ng== | psychemedia 82988 | 2019-04-14T16:58:37Z | 2019-04-14T16:58:37Z | CONTRIBUTOR | Hmm... nope... I see an updated timestamp from |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Datasette doesn't reload when database file changes 432870248 | |
483202658 | https://github.com/simonw/datasette/issues/429#issuecomment-483202658 | https://api.github.com/repos/simonw/datasette/issues/429 | MDEyOklzc3VlQ29tbWVudDQ4MzIwMjY1OA== | psychemedia 82988 | 2019-04-15T10:48:01Z | 2019-04-15T10:48:01Z | CONTRIBUTOR | Minor UI observation:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
?_where=sql-fragment parameter for table views 432636432 | |
509013413 | https://github.com/simonw/datasette/issues/507#issuecomment-509013413 | https://api.github.com/repos/simonw/datasette/issues/507 | MDEyOklzc3VlQ29tbWVudDUwOTAxMzQxMw== | psychemedia 82988 | 2019-07-07T16:31:57Z | 2019-07-07T16:31:57Z | CONTRIBUTOR | Chrome and Firefox both support headless screengrabs from command line, but I don't know how parameterised they can be? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Every datasette plugin on the ecosystem page should have a screenshot 455852801 | |
559207224 | https://github.com/simonw/datasette/issues/642#issuecomment-559207224 | https://api.github.com/repos/simonw/datasette/issues/642 | MDEyOklzc3VlQ29tbWVudDU1OTIwNzIyNA== | psychemedia 82988 | 2019-11-27T18:40:57Z | 2019-11-27T18:41:07Z | CONTRIBUTOR | Would cookie cutter approaches also work for creating various flavours of customised templates? I need to try to create a couple of sites for myself to get a feel for what sorts of thing are easily doable, and what cribbable cookie cutter items might be. I'm guessing https://simonwillison.net/2019/Nov/25/niche-museums/ is a good place to start from? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Provide a cookiecutter template for creating new plugins 529429214 | |
559632608 | https://github.com/simonw/datasette/issues/573#issuecomment-559632608 | https://api.github.com/repos/simonw/datasette/issues/573 | MDEyOklzc3VlQ29tbWVudDU1OTYzMjYwOA== | psychemedia 82988 | 2019-11-29T01:43:38Z | 2019-11-29T01:43:38Z | CONTRIBUTOR | In passing, it looks like a start was made on a datasette Jupyter server extension in https://github.com/lucasdurand/jupyter-datasette although the build fails in MyBinder. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Exposing Datasette via Jupyter-server-proxy 492153532 | |
571138093 | https://github.com/simonw/sqlite-utils/issues/73#issuecomment-571138093 | https://api.github.com/repos/simonw/sqlite-utils/issues/73 | MDEyOklzc3VlQ29tbWVudDU3MTEzODA5Mw== | psychemedia 82988 | 2020-01-06T13:28:31Z | 2020-01-06T13:28:31Z | NONE | I think I actually had several issues in play... The missing key was one, but I think there is also an issue as per below. For example, in the following: ```python def init_testdb(dbname='test.db'):
conn, db = init_testdb() c = conn.cursor() c.executescript('CREATE TABLE "test1" ("Col1" TEXT, "Col2" TEXT, PRIMARY KEY ("Col1"));') c.executescript('CREATE TABLE "test2" ("Col1" TEXT, "Col2" TEXT, PRIMARY KEY ("Col1"));') print('Test 1...') for i in range(3): db['test1'].upsert_all([{'Col1':'a', 'Col2':'x'},{'Col1':'b', 'Col2':'x'}], pk=('Col1')) db['test2'].upsert_all([{'Col1':'a', 'Col2':'x'},{'Col1':'b', 'Col2':'x'}], pk=('Col1')) print('Test 2...') for i in range(3): db['test1'].upsert_all([{'Col1':'a', 'Col2':'x'},{'Col1':'b', 'Col2':'x'}], pk=('Col1')) db['test2'].upsert_all([{'Col1':'a', 'Col2':'x'},{'Col1':'b', 'Col2':'x'}, {'Col1':'c','Col2':'x'}], pk=('Col1')) print('Done...') Test 1... Test 2... IndexError: list index out of range IndexError Traceback (most recent call last) <ipython-input-763-444132ca189f> in <module> 22 print('Test 2...') 23 for i in range(3): ---> 24 db['test1'].upsert_all([{'Col1':'a', 'Col2':'x'},{'Col1':'b', 'Col2':'x'}], pk=('Col1')) 25 db['test2'].upsert_all([{'Col1':'a', 'Col2':'x'},{'Col1':'b', 'Col2':'x'}, 26 {'Col1':'c','Col2':'x'}], pk=('Col1')) /usr/local/lib/python3.7/site-packages/sqlite_utils/db.py in upsert_all(self, records, pk, foreign_keys, column_order, not_null, defaults, batch_size, hash_id, alter, extracts) 1157 alter=alter, 1158 extracts=extracts, -> 1159 upsert=True, 1160 ) 1161 /usr/local/lib/python3.7/site-packages/sqlite_utils/db.py in insert_all(self, records, pk, foreign_keys, column_order, not_null, defaults, batch_size, hash_id, alter, ignore, replace, extracts, upsert) 1097 # self.last_rowid will be 0 if a "INSERT OR IGNORE" happened 1098 if (hash_id or pk) and self.last_rowid: -> 1099 row = list(self.rows_where("rowid = ?", [self.last_rowid]))[0] 1100 if hash_id: 1101 self.last_pk = row[hash_id] IndexError: list index out of range ``` the first test works but the second fails. Is the length of the list of items being upserted leaking somewhere? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
upsert_all() throws issue when upserting to empty table 545407916 | |
573047321 | https://github.com/simonw/sqlite-utils/issues/73#issuecomment-573047321 | https://api.github.com/repos/simonw/sqlite-utils/issues/73 | MDEyOklzc3VlQ29tbWVudDU3MzA0NzMyMQ== | psychemedia 82988 | 2020-01-10T14:02:56Z | 2020-01-10T14:09:23Z | NONE | Hmmm... just tried with installs from pip and the repo (v2.0.0 and v2.0.1) and I get the error each time (start of second run through the second loop). Could it be sqlite3? I'm on 3.30.1. UPDATE: just tried it on jupyter.org/try and I get the error there, too. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
upsert_all() throws issue when upserting to empty table 545407916 | |
580745213 | https://github.com/simonw/sqlite-utils/issues/73#issuecomment-580745213 | https://api.github.com/repos/simonw/sqlite-utils/issues/73 | MDEyOklzc3VlQ29tbWVudDU4MDc0NTIxMw== | psychemedia 82988 | 2020-01-31T14:02:38Z | 2020-01-31T14:21:09Z | NONE | So the conundrum continues.. The simple test case above now runs, but if I upsert a large number of new records (successfully) and then try to upsert a fewer number of new records to a different table, I get the same error. If I run the same upserts again (which in the first case means there are no new records to add, because they were already added), the second upsert works correctly. It feels as if the number of items added via an upsert >> the number of items I try to add in an upsert immediately after, I get the error. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
upsert_all() throws issue when upserting to empty table 545407916 | |
586599424 | https://github.com/simonw/datasette/issues/417#issuecomment-586599424 | https://api.github.com/repos/simonw/datasette/issues/417 | MDEyOklzc3VlQ29tbWVudDU4NjU5OTQyNA== | psychemedia 82988 | 2020-02-15T15:12:19Z | 2020-02-15T15:12:33Z | CONTRIBUTOR | So could the polling support also allow you to call sqlite_utils to update a database with csv files? (Though I'm guessing you would only want to handle changed files? Do your scrapers check and cache csv datestamps/hashes?) |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Datasette Library 421546944 | |
604328163 | https://github.com/simonw/datasette/issues/573#issuecomment-604328163 | https://api.github.com/repos/simonw/datasette/issues/573 | MDEyOklzc3VlQ29tbWVudDYwNDMyODE2Mw== | psychemedia 82988 | 2020-03-26T09:41:30Z | 2020-03-26T09:41:30Z | CONTRIBUTOR | Fixed by @simonw; example here: https://github.com/simonw/jupyterserverproxy-datasette-demo |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Exposing Datasette via Jupyter-server-proxy 492153532 | |
714657366 | https://github.com/simonw/datasette/issues/1033#issuecomment-714657366 | https://api.github.com/repos/simonw/datasette/issues/1033 | MDEyOklzc3VlQ29tbWVudDcxNDY1NzM2Ng== | psychemedia 82988 | 2020-10-22T17:51:29Z | 2020-10-22T17:51:29Z | CONTRIBUTOR | How does |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
datasette.urls.static_plugins(...) method 725099777 | |
716066000 | https://github.com/simonw/datasette/issues/1033#issuecomment-716066000 | https://api.github.com/repos/simonw/datasette/issues/1033 | MDEyOklzc3VlQ29tbWVudDcxNjA2NjAwMA== | psychemedia 82988 | 2020-10-24T22:58:33Z | 2020-10-24T22:58:33Z | CONTRIBUTOR | From the docs, I note:
What about the proxy case? Eg if I am using jupyter-server-proxy on a MyBinder or local Jupyter notebook server site,
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
datasette.urls.static_plugins(...) method 725099777 | |
716123598 | https://github.com/simonw/datasette/issues/838#issuecomment-716123598 | https://api.github.com/repos/simonw/datasette/issues/838 | MDEyOklzc3VlQ29tbWVudDcxNjEyMzU5OA== | psychemedia 82988 | 2020-10-25T10:20:12Z | 2020-10-25T10:53:24Z | CONTRIBUTOR | I'm trying to run something behind a MyBinder proxy, but seem to have something set up incorrectly and not sure what the fix is? I'm starting datasette with jupyter-server-proxy setup: ``` init.pydef setup_nbsearch():
``` where the ``` init.pyfrom datasette import hookimpl @hookimpl def extra_css_urls(database, table, columns, view_name, datasette): return [ "/-/static-plugins/nbsearch/prism.css", "/-/static-plugins/nbsearch/nbsearch.css", ] ``` but these seem to also need a base_url prefix set somehow? Currently, the generated HTML loads properly but internal links are incorrect; eg they take the form The main css is loaded correctly: |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Incorrect URLs when served behind a proxy with base_url set 637395097 | |
718528252 | https://github.com/simonw/datasette/pull/1049#issuecomment-718528252 | https://api.github.com/repos/simonw/datasette/issues/1049 | MDEyOklzc3VlQ29tbWVudDcxODUyODI1Mg== | psychemedia 82988 | 2020-10-29T09:20:34Z | 2020-10-29T09:20:34Z | CONTRIBUTOR | That workaround is probably fine. I was trying to work out whether there might be other situations where a pre-external package load might be useful but couldn't offhand bring any other examples to mind. The static plugins option also looks interesting. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add template block prior to extra URL loaders 729017519 | |
720354227 | https://github.com/simonw/datasette/issues/838#issuecomment-720354227 | https://api.github.com/repos/simonw/datasette/issues/838 | MDEyOklzc3VlQ29tbWVudDcyMDM1NDIyNw== | psychemedia 82988 | 2020-11-02T09:33:58Z | 2020-11-02T09:33:58Z | CONTRIBUTOR | Thanks; just a note that the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Incorrect URLs when served behind a proxy with base_url set 637395097 | |
752098906 | https://github.com/simonw/datasette/issues/417#issuecomment-752098906 | https://api.github.com/repos/simonw/datasette/issues/417 | MDEyOklzc3VlQ29tbWVudDc1MjA5ODkwNg== | psychemedia 82988 | 2020-12-29T14:34:30Z | 2020-12-29T14:34:50Z | CONTRIBUTOR | FWIW, I had a look at Not a production thing, just an experiment trying to explore what might be possible... |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Datasette Library 421546944 | |
1010947634 | https://github.com/simonw/datasette/issues/1591#issuecomment-1010947634 | https://api.github.com/repos/simonw/datasette/issues/1591 | IC_kwDOBm6k_c48QdYy | psychemedia 82988 | 2022-01-12T11:32:17Z | 2022-01-12T11:32:17Z | CONTRIBUTOR | Is it possible to parse things like |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Maybe let plugins define custom serve options? 1100015398 | |
1033641009 | https://github.com/simonw/sqlite-utils/pull/203#issuecomment-1033641009 | https://api.github.com/repos/simonw/sqlite-utils/issues/203 | IC_kwDOCGYnMM49nBwx | psychemedia 82988 | 2022-02-09T11:06:18Z | 2022-02-09T11:06:18Z | NONE | Is there any progress elsewhere on the handling of compound / composite foreign keys, or is this PR still effectively open? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
changes to allow for compound foreign keys 743384829 | |
1041313679 | https://github.com/simonw/sqlite-utils/issues/406#issuecomment-1041313679 | https://api.github.com/repos/simonw/sqlite-utils/issues/406 | IC_kwDOCGYnMM4-ES-P | psychemedia 82988 | 2022-02-16T09:59:51Z | 2022-02-16T10:00:10Z | NONE | The When creating the table, you could then error, or at least warn, if someone wasn't setting a column on a |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Creating tables with custom datatypes 1128466114 | |
1041325398 | https://github.com/simonw/sqlite-utils/issues/402#issuecomment-1041325398 | https://api.github.com/repos/simonw/sqlite-utils/issues/402 | IC_kwDOCGYnMM4-EV1W | psychemedia 82988 | 2022-02-16T10:12:48Z | 2022-02-16T10:18:55Z | NONE |
Other possible pairs: unconventional date/datetime and timezone pairs eg |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Advanced class-based `conversions=` mechanism 1125297737 | |
1041363433 | https://github.com/simonw/sqlite-utils/issues/406#issuecomment-1041363433 | https://api.github.com/repos/simonw/sqlite-utils/issues/406 | IC_kwDOCGYnMM4-EfHp | psychemedia 82988 | 2022-02-16T10:57:03Z | 2022-02-16T10:57:19Z | NONE | Wondering if this actually relates to https://github.com/simonw/sqlite-utils/issues/402 ? I also wonder if this would be a sensible approach for eg registering |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Creating tables with custom datatypes 1128466114 | |
1248204219 | https://github.com/simonw/datasette/issues/1810#issuecomment-1248204219 | https://api.github.com/repos/simonw/datasette/issues/1810 | IC_kwDOBm6k_c5KZhW7 | psychemedia 82988 | 2022-09-15T14:44:47Z | 2022-09-15T14:46:26Z | CONTRIBUTOR | A couple+ of possible use case examples:
In many cases, I suspect the raw content will be in one table, but the search table will be a second (eg FTS) table. Generally, the search may be over one or more joined tables, and the results constructed from one or more tables (which may or may not be distinct from the search tables). |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Featured table(s) on the homepage 1374626873 | |
1248440137 | https://github.com/simonw/sqlite-utils/issues/406#issuecomment-1248440137 | https://api.github.com/repos/simonw/sqlite-utils/issues/406 | IC_kwDOCGYnMM5Kaa9J | psychemedia 82988 | 2022-09-15T18:13:50Z | 2022-09-15T18:13:50Z | NONE | I was wondering if you have any more thoughts on this? I have a tangible use case now: adding a "vector" column to a database to support semantic search using doc2vec embeddings (example; note that the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Creating tables with custom datatypes 1128466114 | |
1420941334 | https://github.com/simonw/datasette/pull/564#issuecomment-1420941334 | https://api.github.com/repos/simonw/datasette/issues/564 | IC_kwDOBm6k_c5UsdgW | psychemedia 82988 | 2023-02-07T15:14:10Z | 2023-02-07T15:14:10Z | CONTRIBUTOR | Is this feature covered by any more recent updates to |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
First proof-of-concept of Datasette Library 473288428 | |
783560017 | https://github.com/simonw/datasette/issues/1166#issuecomment-783560017 | https://api.github.com/repos/simonw/datasette/issues/1166 | MDEyOklzc3VlQ29tbWVudDc4MzU2MDAxNw== | thorn0 94334 | 2021-02-22T18:00:57Z | 2021-02-22T18:13:11Z | NONE | Hi! I don't think Prettier supports this syntax for globs: Tested it. Apparently, it works as a negated character class in regexes (like
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Adopt Prettier for JavaScript code formatting 777140799 | |
1315853097 | https://github.com/simonw/datasette/pull/1893#issuecomment-1315853097 | https://api.github.com/repos/simonw/datasette/issues/1893 | IC_kwDOBm6k_c5OblMp | bgrins 95570 | 2022-11-15T20:55:40Z | 2022-11-15T20:55:40Z | CONTRIBUTOR | Should also minify the bundled output |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Upgrade to CodeMirror 6, add SQL autocomplete 1450363982 | |
1315869040 | https://github.com/simonw/datasette/pull/1893#issuecomment-1315869040 | https://api.github.com/repos/simonw/datasette/issues/1893 | IC_kwDOBm6k_c5ObpFw | bgrins 95570 | 2022-11-15T21:11:42Z | 2022-11-15T21:11:42Z | CONTRIBUTOR | extraKeys is done - Shift+Enter is added in the helper function, and it appears that the Tab behavior now defaults to what the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Upgrade to CodeMirror 6, add SQL autocomplete 1450363982 | |
1315869946 | https://github.com/simonw/datasette/pull/1893#issuecomment-1315869946 | https://api.github.com/repos/simonw/datasette/issues/1893 | IC_kwDOBm6k_c5ObpT6 | bgrins 95570 | 2022-11-15T21:12:38Z | 2022-11-15T21:12:38Z | CONTRIBUTOR | https://github.com/Sphinxxxx/cm-resize isn't compatible with 6. There's a suggestion to try using CSS resize in https://discuss.codemirror.net/t/resizing-codemirror-6/3265/2 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Upgrade to CodeMirror 6, add SQL autocomplete 1450363982 | |
1316041828 | https://github.com/simonw/datasette/pull/1893#issuecomment-1316041828 | https://api.github.com/repos/simonw/datasette/issues/1893 | IC_kwDOBm6k_c5OcTRk | bgrins 95570 | 2022-11-15T23:51:35Z | 2022-11-15T23:51:35Z | CONTRIBUTOR | I experimented with autocompleting the actual schema in https://github.com/bgrins/datasette/commit/8431c98850c7a552dbcde2a4dd0c3dc942a97d25, but it would need some work (current problems with it listed in the commit message there) |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Upgrade to CodeMirror 6, add SQL autocomplete 1450363982 | |
1316243602 | https://github.com/simonw/datasette/pull/1893#issuecomment-1316243602 | https://api.github.com/repos/simonw/datasette/issues/1893 | IC_kwDOBm6k_c5OdEiS | bgrins 95570 | 2022-11-16T03:11:46Z | 2022-11-16T03:11:46Z | CONTRIBUTOR | Was just reviewing the SQL options and there's an upperCaseKeywords if we'd rather have SELECT vs select. Datasette seems to prefer lowercase so probably best to keep it as-is |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Upgrade to CodeMirror 6, add SQL autocomplete 1450363982 | |
1316256386 | https://github.com/simonw/datasette/pull/1893#issuecomment-1316256386 | https://api.github.com/repos/simonw/datasette/issues/1893 | IC_kwDOBm6k_c5OdHqC | bgrins 95570 | 2022-11-16T03:18:06Z | 2022-11-16T03:18:06Z | CONTRIBUTOR |
This version "sort of" works when on the main database page where the template passes the relevant data https://github.com/bgrins/datasette/commit/8431c98850c7a552dbcde2a4dd0c3dc942a97d25 by doing this and passing that into the ```
let TABLES_DATA = [];
{% if tables is defined %} // Turn into an object, shaped like https://github.com/codemirror/lang-sql/blob/ebf115fffdbe07f91465ccbd82868c587f8182bc/test/test-complete.ts#L27. const TABLES_SCHEMA = Object.fromEntries( new Map( TABLES_DATA.map((table) => { return [table.name, table.columns]; }) ).entries() ); ``` But there are a number of papercuts with it - it's not escaping table names with spaces (likely be fixable from the data being passed into the view) but mainly it doesn't seem to autocomplete columns. I think it might only want to do it when you first type the table name from my read of https://github.com/codemirror/lang-sql/blob/ebf115fffdbe07f91465ccbd82868c587f8182bc/test/test-complete.ts#L37. It's possible I'm just passing something wrong, but it may end up being something that needs feature work upstream. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Upgrade to CodeMirror 6, add SQL autocomplete 1450363982 | |
1316318961 | https://github.com/simonw/datasette/pull/1893#issuecomment-1316318961 | https://api.github.com/repos/simonw/datasette/issues/1893 | IC_kwDOBm6k_c5OdW7x | bgrins 95570 | 2022-11-16T04:27:51Z | 2022-11-16T04:27:51Z | CONTRIBUTOR |
The default focus styles appear to be
Which I also see on desktop. Would be nice to changed to whatever the default UA textarea styles are to blend in better but I wouldn't recommend removing it entirely - just to keep the visual indication that the element is focused. Maybe followup material to have a theming pass |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Upgrade to CodeMirror 6, add SQL autocomplete 1450363982 | |
1316320521 | https://github.com/simonw/datasette/pull/1893#issuecomment-1316320521 | https://api.github.com/repos/simonw/datasette/issues/1893 | IC_kwDOBm6k_c5OdXUJ | bgrins 95570 | 2022-11-16T04:29:23Z | 2022-11-16T04:29:23Z | CONTRIBUTOR | UI issue I see on the autocomplete popup with overlapping icon & text. Screenshot's from Firefox, it seems even a little more pronounced on Safari |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Upgrade to CodeMirror 6, add SQL autocomplete 1450363982 | |
1316339035 | https://github.com/simonw/datasette/pull/1893#issuecomment-1316339035 | https://api.github.com/repos/simonw/datasette/issues/1893 | IC_kwDOBm6k_c5Odb1b | bgrins 95570 | 2022-11-16T04:47:11Z | 2022-11-16T04:47:11Z | CONTRIBUTOR |
I was thinking of the BigQuery console, like But they must be doing something pretty custom & appears to be using Monaco anyway. I suspect some kind of lower level autocomplete integration could make this work, but if the table completion is a good-enough starting point I think it's not too hard. The main issue is that we don't pass the relevant table data down to QueryView. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Upgrade to CodeMirror 6, add SQL autocomplete 1450363982 | |
1316387382 | https://github.com/simonw/datasette/pull/1893#issuecomment-1316387382 | https://api.github.com/repos/simonw/datasette/issues/1893 | IC_kwDOBm6k_c5Odno2 | bgrins 95570 | 2022-11-16T05:33:55Z | 2022-11-16T05:33:55Z | CONTRIBUTOR | I added a commit to make our own dialect at https://github.com/simonw/datasette/pull/1893/commits/e273fc8ed5341bdf0b622e722d761bd2acc30a90. Pulled in the full list of keywords from https://www.sqlite.org/lang_keywords.html but haven't gone through and pruned it to only include common select keywords. @simonw you'll have better knowledge than me on that - do you want to take a first shot at narrowing that down to the set that people will be using in the editor? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Upgrade to CodeMirror 6, add SQL autocomplete 1450363982 | |
1317281292 | https://github.com/simonw/datasette/pull/1893#issuecomment-1317281292 | https://api.github.com/repos/simonw/datasette/issues/1893 | IC_kwDOBm6k_c5OhB4M | bgrins 95570 | 2022-11-16T16:19:16Z | 2022-11-16T16:19:16Z | CONTRIBUTOR | Ha, nice idea! Updating the dialect with that list. I'm thinking of also adding |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Upgrade to CodeMirror 6, add SQL autocomplete 1450363982 | |
1317314064 | https://github.com/simonw/datasette/pull/1893#issuecomment-1317314064 | https://api.github.com/repos/simonw/datasette/issues/1893 | IC_kwDOBm6k_c5OhJ4Q | bgrins 95570 | 2022-11-16T16:36:46Z | 2022-11-16T16:36:46Z | CONTRIBUTOR | With
we get table autocompletion and column completion if you name the table in the query (see screencast). I do see bugs with escaped table names like https://user-images.githubusercontent.com/95570/202238521-e613b4e2-ba92-4418-9068-fc022edaee93.mp4 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Upgrade to CodeMirror 6, add SQL autocomplete 1450363982 |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [issue_comments] ( [html_url] TEXT, [issue_url] TEXT, [id] INTEGER PRIMARY KEY, [node_id] TEXT, [user] INTEGER REFERENCES [users]([id]), [created_at] TEXT, [updated_at] TEXT, [author_association] TEXT, [body] TEXT, [reactions] TEXT, [issue] INTEGER REFERENCES [issues]([id]) , [performed_via_github_app] TEXT); CREATE INDEX [idx_issue_comments_issue] ON [issue_comments] ([issue]); CREATE INDEX [idx_issue_comments_user] ON [issue_comments] ([user]);
user >30