issue_comments
616 rows where author_association = "CONTRIBUTOR" sorted by user
This data as json, CSV (advanced)
Suggested facets: reactions, updated_at (date)
issue 330
- Upgrade to CodeMirror 6, add SQL autocomplete 21
- feat: Javascript Plugin API (Custom panels, column menu items with JS actions) 14
- Database page loads too slowly with many large tables (due to table counts) 13
- Stream all results for arbitrary SQL and canned queries 10
- docker image is duplicating db files somehow 10
- Handle spatialite geometry columns better 7
- base_url configuration setting 7
- Exceeding Cloud Run memory limits when deploying a 4.8G database 7
- create-index should run analyze after creating index 7
- Add new spatialite helper methods 7
- Add register_output_renderer hook 6
- Helper methods for working with SpatiaLite 6
- Plugin hook for dynamic metadata 6
- clean checkout & clean environment has test failures 6
- [WIP] Add publish to heroku support 5
- Scripted exports 5
- datasette publish lambda plugin 4
- Build Dockerfile with recent Sqlite + Spatialite 4
- Documentation with recommendations on running Datasette in production without using Docker 4
- bpylist.archiver.CircularReference: archive has a cycle with uid(13) 4
- Add insert --truncate option 4
- Make it easier to insert geometries, with documentation and maybe code 4
- Advanced class-based `conversions=` mechanism 4
- query result page is using 400mb of browser memory 40x size of html page and 400x size of csv data 4
- Proposal: datasette query 4
- Writable canned queries fail with useless non-error against immutable databases 4
- Ability to merge databases and tables 4
- API to insert a single record into an existing table 4
- Exclude virtual tables from datasette inspect 4
- Call for birthday presents: if you're using Datasette, let us know how you're using it here 4
- array facet: don't materialize unnecessary columns 4
- Proposal: Combine settings, metadata, static, etc. into a single `datasette.yaml` File 4
- Ship a Docker image of the whole thing 3
- Support for units 3
- Don't duplicate simple primary keys in the link column 3
- Add version number support with Versioneer 3
- Integration with JupyterLab 3
- Datasette Library 3
- Exposing Datasette via Jupyter-server-proxy 3
- Don't auto-format SQL on page load 3
- Try out ExifReader 3
- Consider dropping explicit CSRF protection entirely? 3
- Handle case where subsequent records (after first batch) include extra columns 3
- For 1.0 update trove classifier in setup.py 3
- Support linking to compound foreign keys 3
- register_output_renderer() should support streaming data 3
- Feature or Documentation Request: Individual table as home page template 3
- Make row available to `render_cell` plugin hook 3
- Serve all db files in a folder 3
- Allow routes to have extra options 3
- Datasette feature for publishing snapshots of query results 3
- CLI eats my cursor 3
- don't use immutable=1, only mode=ro 3
- Datasette is not compatible with SQLite's strict quoting compilation option 3
- Plugin system 3
- De-tangling Metadata before Datasette 1.0 3
- Dockerfile should build more recent SQLite with FTS5 and spatialite support 2
- Add NHS England Hospitals example to wiki 2
- add support for ?field__isnull=1 2
- Travis should push tagged images to Docker Hub for each release 2
- Column types in inspected metadata 2
- "datasette publish cloudrun" command to publish to Google Cloud Run 2
- [WIP] Add primary key to the extra_body_script hook arguments 2
- Define mechanism for plugins to return structured data 2
- Rename metadata.json to config.json 2
- Mechanism for turning nested JSON into foreign keys / many-to-many 2
- Every datasette plugin on the ecosystem page should have a screenshot 2
- Fix static mounts using relative paths and prevent traversal exploits 2
- Handle spaces in DB names 2
- allow leading comments in SQL input field 2
- Test failures on openSUSE 15.1: AssertionError: Explicit other_table and other_column 2
- base_url doesn't entirely work for running Datasette inside Binder 2
- Option to automatically configure based on directory layout 2
- Expose scores from ZCOMPUTEDASSETATTRIBUTES 2
- Incorrect URLs when served behind a proxy with base_url set 2
- Skip counting hidden tables 2
- asgi_wrapper plugin hook is crashing at startup 2
- Improved (and better documented) support for transactions 2
- insert_all(..., alter=True) should work for new columns introduced after the first 100 records 2
- datasette.urls.static_plugins(...) method 2
- Refactor .csv to be an output renderer - and teach register_output_renderer to stream all rows 2
- Fix footer not sticking to bottom in short pages 2
- Remove unneeded exists=True for -a/--auth flag. 2
- Improve the display of facets information 2
- Archive import appears to be broken on recent exports 2
- Fix archive imports 2
- Ability for plugins to collaborate when adding extra HTML to blocks in default templates 2
- Use Data from SQLite in other commands 2
- Tests are very slow. 2
- Installing datasette via docker: Path 'fixtures.db' does not exist 2
- Async support 2
- Better default display of arrays of items 2
- Dockerfile: use Ubuntu 20.10 as base 2
- DRAFT: A new plugin hook for dynamic metadata 2
- unordered list is not rendering bullet points in description_html on database page 2
- `publish cloudrun` should deploy a more recent SQLite version 2
- Rename Datasette.__init__(config=) parameter to settings= 2
- base logo link visits `undefined` rather than href url 2
- Update pyyaml requirement from ~=5.3 to >=5.3,<7.0 2
- Add new `"sql_file"` key to Canned Queries in metadata? 2
- Writable canned queries fail to load custom templates 2
- Redesign CSV export to improve usability 2
- Allow to set `facets_array` in metadata (like current `facets`) 2
- if csv export is truncated in non streaming mode set informative response header 2
- Add SpatiaLite helpers to CLI 2
- Document how to add a primary key to a rowid table using `sqlite-utils transform --pk` 2
- Add SpatiaLite helpers to CLI 2
- `with db:` for transactions 2
- [plugins][documentation] Is it possible to serve per-plugin static folders when writing one-off (single file) plugins? 2
- ?_trace=1 fails with datasette-geojson for some reason 2
- Utilities for duplicating tables and creating a table with the results of a query 2
- feature request: pivot command 2
- google cloudrun updated their limits on maxscale based on memory and cpu count 2
- Add new entrypoint option to `--load-extension` 2
- SITE-BUSTING ERROR: "render_template() called before await ds.invoke_startup()" 2
- [insert_all, upsert_all] IntegrityError: constraint failed 2
- Clicking within the CodeMirror area below the SQL (i.e. when there's only a single line) doesn't cause the editor to get focused 2
- render_cell plugin hook's row object is not a sqlite.Row 2
- Show referring tables and rows when the referring foreign key is compound 2
- Repeated calls to `Table.convert()` fail 2
- Add paths for homebrew on Apple silicon 2
- Aliased ROWID option for tables created from alter=True commands 2
- Plugin hook for adding new output formats 2
- Bump sphinx, furo, blacken-docs dependencies 2
- Fix query for suggested facets with column named value 2
- Datasette Plugins 1
- Ability to plot a simple graph 1
- :fire: Removes DS_Store 1
- Consider data-package as a format for metadata 1
- Plot rows on a map with Leaflet and Leaflet.markercluster 1
- Ability to bundle and serve additional static files 1
- Document the querystring argument for setting a different time limit 1
- datasette skeleton command for kick-starting database and table metadata 1
- More metadata options for template authors 1
- Hide Spatialite system tables 1
- Raise 404 on nonexistent table URLs 1
- Support filtering with units and more 1
- Figure out a way to have /-/version return current git commit hash 1
- inspect should record column types 1
- Render boolean fields better by default 1
- datasette publish heroku fails without name provided 1
- Default built image size over Zeit Now 100MiB limit 1
- Interface should show same JSON shape options for custom SQL queries 1
- datasette publish digitalocean plugin 1
- Linked Data(sette) 1
- Default to opening files in mutable mode, special option for immutable files 1
- ?_where=sql-fragment parameter for table views 1
- Datasette doesn't reload when database file changes 1
- Refactor facets to a class and new plugin, refs #427 1
- Add inspect and prepare_sanic hooks 1
- Coalesce hidden table count to 0 1
- Installing installs the tests package 1
- Exporting sqlite database(s)? 1
- Get Datasette tests passing on Windows in GitHub Actions 1
- "about" parameter in metadata does not appear when alone 1
- Show total/unfiltered row count when filtering 1
- "Too many SQL variables" on large inserts 1
- First proof-of-concept of Datasette Library 1
- Escape the table name in populate_fts and search. 1
- Add triggers while enabling FTS 1
- importing CSV to SQLite as library 1
- Databases on index page should display in order they were passed to "datasette serve"? 1
- Offer to format readonly SQL 1
- `import` command fails on empty files 1
- updating metadata.json without recreating the app 1
- Provide a cookiecutter template for creating new plugins 1
- Validate metadata json on startup 1
- Display of the column definitions 1
- Use inspect-file, if possible, for total row count 1
- --reload sould reload server if code in --plugins-dir changes 1
- Import EXIF data into SQLite - lens used, ISO, aperture etc 1
- Update pytest-asyncio requirement from ~=0.10.0 to >=0.10,<0.12 1
- Replace "datasette publish --extra-options" with "--setting" 1
- sqlite3.OperationalError: too many SQL variables in insert_all when using rows with varying numbers of columns 1
- Import machine-learning detected labels (dog, llama etc) from Apple Photos 1
- Only install osxphotos if running on macOS 1
- Enable wildcard-searches by default 1
- Allow to specify a URL fragment for canned queries 1
- Redesign register_facet_classes plugin hook 1
- Having trouble getting writable canned queries to work 1
- sqlite-utils insert: options for column types 1
- Add pull requests 1
- Support column descriptions in metadata.json 1
- Update black requirement from ~=19.10b0 to >=19.10,<21.0 1
- Bug when first record contains fewer columns than subsequent records 1
- favorites --stop_after=N stops after min(N, 200) 1
- github-to-sqlite should handle rate limits better 1
- Wide tables should scroll horizontally within the page 1
- Update utils.py to fix sqlite3.OperationalError 1
- "Edit SQL" button on canned queries 1
- Include LICENSE in sdist 1
- Add template block prior to extra URL loaders 1
- Allow iterables other than Lists in m2m records 1
- changes to allow for compound foreign keys 1
- Accessing a database's `.json` is slow for very large SQLite files 1
- Fix --metadata doc usage 1
- --load-extension=spatialite not working with datasetteproject/datasette docker image 1
- Make it easier to theme Datasette with CSS 1
- Use YAML examples in documentation by default, not JSON 1
- Update for Big Sur 1
- Modernize code to Python 3.6+ 1
- Add Prettier to contributing documentation 1
- Mechanism for storing metadata in _metadata tables 1
- Prettier package not actually being cached 1
- Install Prettier via package.json 1
- ?_size=10 option for the arbitrary query page would be useful 1
- A lot of open(file) functions are used without a context manager thus producing ResourceWarning: unclosed file <_io.TextIOWrapper 1
- Use context manager instead of plain open 1
- photo-to-sqlite: command not found 1
- ensure immutable databses when starting in configuration directory mode with 1
- Ability to increase size of the SQL editor window 1
- Custom pages don't work with base_url setting 1
- Minor type in IP adress 1
- Allow canned query params to specify default values 1
- Plugin hook that could support 'order by random()' for table view 1
- Support for HTTP Basic Authentication 1
- Might there be some way to comment metadata.json? 1
- Ability to run CI against multiple SQLite versions 1
- improve table horizontal scroll experience 1
- Publishing to cloudrun with immutable mode? 1
- Bump black from 20.8b1 to 21.4b0 1
- Bump black from 20.8b1 to 21.4b1 1
- Bump black from 20.8b1 to 21.4b2 1
- Bump black from 21.4b2 to 21.5b0 1
- DRAFT: add test and scan for docker images 1
- Research: syntactic sugar for using --get with SQL queries, maybe "datasette query" 1
- Ensure db.path is a string before trying to insert into internal database 1
- Idea: import CSV to memory, run SQL, export in a single command 1
- Support db as first parameter before subcommand, or as environment variable 1
- Serve using UNIX domain socket 1
- "invalid reference format" publishing Docker image 1
- render_cell() hook should support returning an awaitable 1
- Bump black from 21.7b0 to 21.8b0 1
- Error: Use either --since or --since_id, not both - still broken 1
- Add scientists to target groups 1
- Test against 3.10-dev 1
- Bump black from 21.9b0 to 21.10b0 1
- Bump black from 21.9b0 to 21.11b0 1
- Deploy a live instance of demos/apache-proxy 1
- Support STRICT tables 1
- Command for creating an empty database 1
- Allow passing a file of code to "sqlite-utils convert" 1
- add hash id to "_memory" url if hashed url mode is turned on and crossdb is also turned on 1
- introduce new option for datasette package to use a slim base image 1
- when hashed urls are turned on, the _memory db has improperly long-lived cache expiry 1
- don't set far expiry if hash is '000' 1
- consider adding deletion step of cloudbuild artifacts to gcloud publish 1
- Maybe let plugins define custom serve options? 1
- Update pytest-asyncio requirement from <0.17,>=0.10 to >=0.10,<0.18 1
- Add KNN and data_licenses to hidden tables list 1
- Move canned queries closer to the SQL input area 1
- Try again with SQLite codemirror support 1
- Tweak mobile keyboard settings 1
- Support for generated columns 1
- Optional Pandas integration 1
- Mechanism for disabling faceting on large tables only 1
- Update jinja2 requirement from <3.1.0,>=2.10.3 to >=2.10.3,<3.2.0 1
- Bump black from 22.1.0 to 22.3.0 1
- Show foreign key label when filtering 1
- .extract() doesn't set foreign key when extracted columns contain NULL value 1
- Research: demonstrate if parallel SQL queries are worthwhile 1
- Bump furo from 2022.4.7 to 2022.6.4.1 1
- Reading rows from a file => AttributeError: '_io.StringIO' object has no attribute 'readinto' 1
- Mechanism for ensuring a table has all the columns 1
- Research an upgrade to CodeMirror 6 1
- search_sql add include_rank option 1
- Update aiofiles requirement from <0.9,>=0.4 to >=0.4,<22.2 1
- Featured table(s) on the homepage 1
- missing next and next_url in JSON responses from an instance deployed on Fly 1
- Expose `sql` and `params` arguments to various plugin hooks 1
- [SPIKE] Don't truncate query CSVs 1
- Tiny typographical error in install/uninstall docs 1
- fix: enable-fts permanently save triggers 1
- feat: recreate fts triggers after table transform 1
- API explorer tool 1
- conn.execute: UnicodeEncodeError: 'utf-8' codec can't encode character 1
- Allow surrogates in parameters 1
- Cannot enable FTS5 despite it being available 1
- Autocomplete text entry for filter values that correspond to facets 1
- Serve schema JSON to the SQL editor to enable autocomplete 1
- Incorrect link from the API explorer to the JSON API documentation 1
- /db/table/-/upsert 1
- Bump sphinx from 5.3.0 to 6.0.0 1
- rows_from_file() raises confusing error if file-like object is not in binary mode 1
- Bump sphinx from 5.3.0 to 6.1.0 1
- Bump sphinx from 5.3.0 to 6.1.1 1
- Document datasette.urls.row and row_blob 1
- Bump sphinx from 5.3.0 to 6.1.2 1
- Make CustomJSONEncoder a documented public API 1
- rewrite_sql hook 1
- Feature request: trim all leading and trailing white space for all columns for all tables in a database 1
- Bump black from 22.12.0 to 23.1.0 1
- Transformation type `--type DATETIME` 1
- `Table.convert()` skips falsey values 1
- Error: Invalid setting 'hash_urls' in settings.json in 0.64.1 1
- Microsoft line endings 1
- `datasette install -r requirements.txt` 1
- Bump furo from 2022.12.7 to 2023.3.23 1
- rows: --transpose or psql extended view-like functionality 1
- Bump sphinx from 6.1.3 to 6.2.0 1
- Bump sphinx from 6.1.3 to 6.2.1 1
- Bump sphinx from 6.1.3 to 7.0.0 1
- Bump sphinx from 6.1.3 to 7.0.1 1
- Bump furo from 2023.3.27 to 2023.5.20 1
- Filter table by a large bunch of ids 1
- Support storing incrementally piped values 1
- `--settings settings.json` option 1
- Bump blacken-docs from 1.14.0 to 1.15.0 1
- Tables starting with an underscore should be treated as hidden 1
- Bump sphinx from 6.1.3 to 7.1.0 1
- Bump furo from 2023.3.27 to 2023.7.26 1
- Bump sphinx from 6.1.3 to 7.1.1 1
- Bump sphinx from 6.1.3 to 7.1.2 1
- Bump the python-packages group with 1 update 1
- Bump the python-packages group with 2 updates 1
- Bump the python-packages group with 3 updates 1
- If a row has a primary key of `null` various things break 1
- Bump the python-packages group with 3 updates 1
- Fix hupper.start_reloader entry point 1
- Proposal: Make the `_internal` database persistent, customizable, and hidden 1
- Bump the python-packages group with 2 updates 1
- `datasette.yaml` plugin support 1
- Plugin Hooks for "compile to SQL" languages 1
- Raise an exception if a "plugins" block exists in metadata.json 1
- Detailed upgrade instructions for metadata.yaml -> datasette.yaml 1
- Bump the python-packages group with 1 update 1
- Bump the python-packages group with 1 update 1
- No suggested facets when a column named 'value' is included 1
- Add more STRICT table support 1
- feature request: gzip compression of database downloads 1
user 67
- fgregg 82
- eyeseast 74
- russss 39
- dependabot[bot] 36
- abdusco 26
- asg017 25
- psychemedia 24
- bgrins 24
- mroswell 22
- chapmanjacobd 20
- cldellow 18
- brandonrobertz 15
- jacobian 14
- RhetTbull 14
- wragge 12
- rixx 11
- hydrosquall 11
- rgieseke 10
- bobwhitelock 9
- amjith 6
- jefftriplett 6
- tsibley 6
- simonwiles 6
- mcarpenter 6
- jaywgraves 6
- davidbgk 5
- dependabot-preview[bot] 5
- bollwyvl 4
- ctb 4
- r4vi 4
- jsfenfen 4
- glasnt 4
- kbaikov 4
- JBPressac 4
- benpickles 3
- ghing 3
- macropin 3
- blairdrummond 3
- kevindkeogh 3
- daniel-butler 3
- jayvdb 2
- ingenieroariel 2
- camallen 2
- gfrmin 2
- mcint 2
- frosencrantz 2
- adipasquale 2
- davidleejy 2
- raynae 2
- tkhattra 2
- MichaelTiemannOSC 2
- danp 1
- tomdyson 1
- adamwolf 1
- rubenv 1
- spookylukey 1
- aslakr 1
- meatcar 1
- mikepqr 1
- adamjonas 1
- louispotok 1
- chris48s 1
- eumiro 1
- rprimet 1
- mattiaborsoi 1
- abeyerpath 1
- b0b5h4rp13 1
id | html_url | issue_url | node_id | user ▼ | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
735436014 | https://github.com/simonw/datasette/issues/1114#issuecomment-735436014 | https://api.github.com/repos/simonw/datasette/issues/1114 | MDEyOklzc3VlQ29tbWVudDczNTQzNjAxNA== | danp 2182 | 2020-11-29T18:33:30Z | 2020-11-29T18:33:30Z | CONTRIBUTOR | Thank you! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
--load-extension=spatialite not working with datasetteproject/datasette docker image 752966476 | |
995296725 | https://github.com/simonw/datasette/issues/1552#issuecomment-995296725 | https://api.github.com/repos/simonw/datasette/issues/1552 | IC_kwDOBm6k_c47UwXV | davidbgk 3556 | 2021-12-15T23:29:32Z | 2021-12-15T23:29:32Z | CONTRIBUTOR | @simonw thank you for your fast answer and your guidance! While digging into the code, I found an undocumented way of doing it:
The only remaining problem with that solution is here: https://github.com/simonw/datasette/blob/250db8192cb8aba5eb8cd301ccc2a49525bc3d24/datasette/facets.py#L33 We have:
But it requires to cast the
I guess it's an unspotted bug? (I mean, independently of the facets-with-arrays issue.) |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Allow to set `facets_array` in metadata (like current `facets`) 1078702875 | |
996229007 | https://github.com/simonw/datasette/issues/1552#issuecomment-996229007 | https://api.github.com/repos/simonw/datasette/issues/1552 | IC_kwDOBm6k_c47YT-P | davidbgk 3556 | 2021-12-16T22:04:39Z | 2021-12-16T22:04:39Z | CONTRIBUTOR | Wow, that was fast, thank you so much @simonw !
I agree that it's not ideal, my initial naive approach was to detect if it's an array, like what is done here: But it requires an extra query to determine the type, which is a bit problematic, especially for big tables I guess. Taking a look at #510, I wonder if a To be consistent with the Throwing a lot of ideas without knowing the big picture… but sometimes newcomers have superpowers :). |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Allow to set `facets_array` in metadata (like current `facets`) 1078702875 | |
1309650806 | https://github.com/simonw/datasette/issues/1871#issuecomment-1309650806 | https://api.github.com/repos/simonw/datasette/issues/1871 | IC_kwDOBm6k_c5OD692 | davidbgk 3556 | 2022-11-10T01:38:58Z | 2022-11-10T01:38:58Z | CONTRIBUTOR |
I agree (that's what I did with the previous insert plugin), maybe a complete example using |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
API explorer tool 1427293909 | |
1339906969 | https://github.com/simonw/datasette/issues/1929#issuecomment-1339906969 | https://api.github.com/repos/simonw/datasette/issues/1929 | IC_kwDOBm6k_c5P3VuZ | davidbgk 3556 | 2022-12-06T19:34:20Z | 2022-12-06T19:34:20Z | CONTRIBUTOR | I confirm that it works 👍 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Incorrect link from the API explorer to the JSON API documentation 1473659191 | |
1339916064 | https://github.com/simonw/datasette/pull/1931#issuecomment-1339916064 | https://api.github.com/repos/simonw/datasette/issues/1931 | IC_kwDOBm6k_c5P3X8g | davidbgk 3556 | 2022-12-06T19:42:45Z | 2022-12-06T19:42:45Z | CONTRIBUTOR | The |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
/db/table/-/upsert 1473814539 | |
754004715 | https://github.com/simonw/datasette/pull/1170#issuecomment-754004715 | https://api.github.com/repos/simonw/datasette/issues/1170 | MDEyOklzc3VlQ29tbWVudDc1NDAwNDcxNQ== | benpickles 3637 | 2021-01-04T14:25:44Z | 2021-01-04T14:25:44Z | CONTRIBUTOR | I was going to re-add the filter to only run Prettier when there have been changes in |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Install Prettier via package.json 778126516 | |
754007242 | https://github.com/simonw/datasette/issues/1169#issuecomment-754007242 | https://api.github.com/repos/simonw/datasette/issues/1169 | MDEyOklzc3VlQ29tbWVudDc1NDAwNzI0Mg== | benpickles 3637 | 2021-01-04T14:29:57Z | 2021-01-04T14:29:57Z | CONTRIBUTOR | I somewhat share your reluctance to add a package.json to seemingly every project out there but ultimately if they're project dependencies it's important they're managed within the codebase. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Prettier package not actually being cached 777677671 | |
754619930 | https://github.com/simonw/datasette/issues/1167#issuecomment-754619930 | https://api.github.com/repos/simonw/datasette/issues/1167 | MDEyOklzc3VlQ29tbWVudDc1NDYxOTkzMA== | benpickles 3637 | 2021-01-05T12:57:57Z | 2021-01-05T12:57:57Z | CONTRIBUTOR | Not sure where exactly to put the actual docs (presumably somewhere in docs/contributing.rst) but I've made a slight change to make it easier to run locally (copying the approach in excalidraw): https://github.com/simonw/datasette/compare/main...benpickles:prettier-docs |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add Prettier to contributing documentation 777145954 | |
573388052 | https://github.com/simonw/sqlite-utils/issues/74#issuecomment-573388052 | https://api.github.com/repos/simonw/sqlite-utils/issues/74 | MDEyOklzc3VlQ29tbWVudDU3MzM4ODA1Mg== | jayvdb 15092 | 2020-01-12T06:51:30Z | 2020-01-12T06:51:30Z | CONTRIBUTOR | Thanks. That showed me that there was a click cli runner error, and setting |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Test failures on openSUSE 15.1: AssertionError: Explicit other_table and other_column 546073980 | |
573389669 | https://github.com/simonw/sqlite-utils/issues/74#issuecomment-573389669 | https://api.github.com/repos/simonw/sqlite-utils/issues/74 | MDEyOklzc3VlQ29tbWVudDU3MzM4OTY2OQ== | jayvdb 15092 | 2020-01-12T07:21:17Z | 2020-01-12T07:21:17Z | CONTRIBUTOR | I guess there is some extra flag for |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Test failures on openSUSE 15.1: AssertionError: Explicit other_table and other_column 546073980 | |
344430689 | https://github.com/simonw/datasette/issues/88#issuecomment-344430689 | https://api.github.com/repos/simonw/datasette/issues/88 | MDEyOklzc3VlQ29tbWVudDM0NDQzMDY4OQ== | tomdyson 15543 | 2017-11-14T23:08:22Z | 2017-11-14T23:08:22Z | CONTRIBUTOR |
Sorry about that - here's a working version on Netlify: |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add NHS England Hospitals example to wiki 273775212 | |
344710204 | https://github.com/simonw/datasette/pull/104#issuecomment-344710204 | https://api.github.com/repos/simonw/datasette/issues/104 | MDEyOklzc3VlQ29tbWVudDM0NDcxMDIwNA== | jacobian 21148 | 2017-11-15T19:57:50Z | 2017-11-15T19:57:50Z | CONTRIBUTOR | A first basic stab at making this work, just to prove the approach. Right now this requires a Heroku CLI plugin, which seems pretty unreasonable. I think this can be replaced with direct API calls, which could clean up a lot of things. But I wanted to prove it worked first, and it does. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
[WIP] Add publish to heroku support 274284246 | |
345452669 | https://github.com/simonw/datasette/pull/104#issuecomment-345452669 | https://api.github.com/repos/simonw/datasette/issues/104 | MDEyOklzc3VlQ29tbWVudDM0NTQ1MjY2OQ== | jacobian 21148 | 2017-11-18T16:18:45Z | 2017-11-18T16:18:45Z | CONTRIBUTOR | I'd like to do a bit of cleanup, and some error checking in case heroku/heroku-builds isn't installed. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
[WIP] Add publish to heroku support 274284246 | |
346116745 | https://github.com/simonw/datasette/pull/104#issuecomment-346116745 | https://api.github.com/repos/simonw/datasette/issues/104 | MDEyOklzc3VlQ29tbWVudDM0NjExNjc0NQ== | jacobian 21148 | 2017-11-21T18:23:25Z | 2017-11-21T18:23:25Z | CONTRIBUTOR | @simonw ready for a review and merge if you want. There's still some nasty duplicated code in cli.py and utils.py, which is just going to get worse if/when we start adding any other deploy targets (and I want to do one for cloud.gov, at least). I think there's an opportunity for some refactoring here. I'm happy to do that now as part of this PR, or if you merge this first I'll do it in a different one. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
[WIP] Add publish to heroku support 274284246 | |
346124073 | https://github.com/simonw/datasette/pull/104#issuecomment-346124073 | https://api.github.com/repos/simonw/datasette/issues/104 | MDEyOklzc3VlQ29tbWVudDM0NjEyNDA3Mw== | jacobian 21148 | 2017-11-21T18:49:55Z | 2017-11-21T18:49:55Z | CONTRIBUTOR | Actually hang on, don't merge - there are some bugs that #141 masked when I tested this out elsewhere. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
[WIP] Add publish to heroku support 274284246 | |
346124764 | https://github.com/simonw/datasette/pull/104#issuecomment-346124764 | https://api.github.com/repos/simonw/datasette/issues/104 | MDEyOklzc3VlQ29tbWVudDM0NjEyNDc2NA== | jacobian 21148 | 2017-11-21T18:52:14Z | 2017-11-21T18:52:14Z | CONTRIBUTOR | OK, now this should work. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
[WIP] Add publish to heroku support 274284246 | |
346244871 | https://github.com/simonw/datasette/issues/14#issuecomment-346244871 | https://api.github.com/repos/simonw/datasette/issues/14 | MDEyOklzc3VlQ29tbWVudDM0NjI0NDg3MQ== | jacobian 21148 | 2017-11-22T05:06:30Z | 2017-11-22T05:06:30Z | CONTRIBUTOR | I'd also suggest taking a look at stevedore, which has a ton of tools for doing plugin stuff. I've had good luck with it in the past. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Datasette Plugins 267707940 | |
552134876 | https://github.com/dogsheep/twitter-to-sqlite/issues/29#issuecomment-552134876 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/29 | MDEyOklzc3VlQ29tbWVudDU1MjEzNDg3Ng== | jacobian 21148 | 2019-11-09T20:33:38Z | 2019-11-09T20:33:38Z | CONTRIBUTOR | ❤️ thanks! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`import` command fails on empty files 518725064 | |
558687342 | https://github.com/simonw/datasette/issues/639#issuecomment-558687342 | https://api.github.com/repos/simonw/datasette/issues/639 | MDEyOklzc3VlQ29tbWVudDU1ODY4NzM0Mg== | jacobian 21148 | 2019-11-26T15:40:00Z | 2019-11-26T15:40:00Z | CONTRIBUTOR | A bit of background: the reason So there's one option -- change @pkoppstein - what you suggested seems like it ought to work (you don't need maintenance mode, though). I'm not sure why it doesn't. You could also look into using the slugs API to download the slug, change Ultimately though I think I think @simonw's idea of reading Hope this helps a bit! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
updating metadata.json without recreating the app 527670799 | |
754721153 | https://github.com/dogsheep/twitter-to-sqlite/issues/54#issuecomment-754721153 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/54 | MDEyOklzc3VlQ29tbWVudDc1NDcyMTE1Mw== | jacobian 21148 | 2021-01-05T15:51:09Z | 2021-01-05T15:51:09Z | CONTRIBUTOR | Correction: the failure is on |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Archive import appears to be broken on recent exports 779088071 | |
754728696 | https://github.com/dogsheep/twitter-to-sqlite/pull/55#issuecomment-754728696 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/55 | MDEyOklzc3VlQ29tbWVudDc1NDcyODY5Ng== | jacobian 21148 | 2021-01-05T16:02:55Z | 2021-01-05T16:02:55Z | CONTRIBUTOR | This now works for me, though I'm entirely ensure if it's a just-my-export thing or a wider issue. Also, this doesn't contain any tests. So I'm not sure if there's more work to be done here, or if this is good enough. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Fix archive imports 779211940 | |
754729035 | https://github.com/dogsheep/twitter-to-sqlite/issues/54#issuecomment-754729035 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/54 | MDEyOklzc3VlQ29tbWVudDc1NDcyOTAzNQ== | jacobian 21148 | 2021-01-05T16:03:29Z | 2021-01-05T16:03:29Z | CONTRIBUTOR | I was able to fix this, at least enough to get my archive to import. Not sure if there's more work to be done here or not. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Archive import appears to be broken on recent exports 779088071 | |
760950128 | https://github.com/dogsheep/twitter-to-sqlite/pull/55#issuecomment-760950128 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/55 | MDEyOklzc3VlQ29tbWVudDc2MDk1MDEyOA== | jacobian 21148 | 2021-01-15T13:44:52Z | 2021-01-15T13:44:52Z | CONTRIBUTOR | I found and fixed another bug, this one around importing the tweets table. @simonw let me know if you'd prefer this broken out into multiple PRs, happy to do that if it makes review/merging easier. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Fix archive imports 779211940 | |
799002993 | https://github.com/simonw/datasette/issues/236#issuecomment-799002993 | https://api.github.com/repos/simonw/datasette/issues/236 | MDEyOklzc3VlQ29tbWVudDc5OTAwMjk5Mw== | jacobian 21148 | 2021-03-14T23:41:51Z | 2021-03-14T23:41:51Z | CONTRIBUTOR | Now that Lambda supports Docker, this probably is a bit easier and may be able to build on top of the existing package command. There are weirdnesses in how the command actually gets invoked; the aws-lambda-python image shows a bit of that. So Datasette would probably need some sort of Lambda-specific entry point to make this work. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
datasette publish lambda plugin 317001500 | |
799003172 | https://github.com/simonw/datasette/issues/236#issuecomment-799003172 | https://api.github.com/repos/simonw/datasette/issues/236 | MDEyOklzc3VlQ29tbWVudDc5OTAwMzE3Mg== | jacobian 21148 | 2021-03-14T23:42:57Z | 2021-03-14T23:42:57Z | CONTRIBUTOR | Oh, and the container image can be up to 10GB, so the EFS step might not be needed except for pretty big stuff. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
datasette publish lambda plugin 317001500 | |
489105665 | https://github.com/simonw/datasette/pull/434#issuecomment-489105665 | https://api.github.com/repos/simonw/datasette/issues/434 | MDEyOklzc3VlQ29tbWVudDQ4OTEwNTY2NQ== | eyeseast 25778 | 2019-05-03T14:01:30Z | 2019-05-03T14:01:30Z | CONTRIBUTOR | This is exactly what I needed. Thank you. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
"datasette publish cloudrun" command to publish to Google Cloud Run 434321685 | |
618126449 | https://github.com/simonw/datasette/issues/731#issuecomment-618126449 | https://api.github.com/repos/simonw/datasette/issues/731 | MDEyOklzc3VlQ29tbWVudDYxODEyNjQ0OQ== | eyeseast 25778 | 2020-04-23T01:38:55Z | 2020-04-23T01:38:55Z | CONTRIBUTOR | I've almost suggested this same thing a couple times. I tend to have Makefile (because I'm doing other |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Option to automatically configure based on directory layout 605110015 | |
618758326 | https://github.com/simonw/datasette/issues/731#issuecomment-618758326 | https://api.github.com/repos/simonw/datasette/issues/731 | MDEyOklzc3VlQ29tbWVudDYxODc1ODMyNg== | eyeseast 25778 | 2020-04-24T01:55:00Z | 2020-04-24T01:55:00Z | CONTRIBUTOR | Mounting |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Option to automatically configure based on directory layout 605110015 | |
787121933 | https://github.com/simonw/sqlite-utils/issues/242#issuecomment-787121933 | https://api.github.com/repos/simonw/sqlite-utils/issues/242 | MDEyOklzc3VlQ29tbWVudDc4NzEyMTkzMw== | eyeseast 25778 | 2021-02-27T19:18:57Z | 2021-02-27T19:18:57Z | CONTRIBUTOR | I think HTTPX gets it exactly right, with a clear separation between sync and async clients, each with a basically identical API. (I'm about to switch feed-to-sqlite over to it, from Requests, to eventually make way for async support.) |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Async support 817989436 | |
853895159 | https://github.com/simonw/datasette/issues/1356#issuecomment-853895159 | https://api.github.com/repos/simonw/datasette/issues/1356 | MDEyOklzc3VlQ29tbWVudDg1Mzg5NTE1OQ== | eyeseast 25778 | 2021-06-03T14:03:59Z | 2021-06-03T14:03:59Z | CONTRIBUTOR | (Putting thoughts here to keep the conversation in one place.) I think using datasette for this use-case is the right approach. I usually have both datasette and sqlite-utils installed in the same project, and that's where I'm trying out queries, so it probably makes the most sense to have datasette also manage the output (and maybe the input, too). It seems like both ```sh run an arbitrary querydatasette query covid.db "select * from ny_times_us_counties limit 1" --format yaml run a canned querydatasette get covid.db some-canned-query --format yaml ``` |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: syntactic sugar for using --get with SQL queries, maybe "datasette query" 910092577 | |
857298526 | https://github.com/simonw/datasette/pull/1370#issuecomment-857298526 | https://api.github.com/repos/simonw/datasette/issues/1370 | MDEyOklzc3VlQ29tbWVudDg1NzI5ODUyNg== | eyeseast 25778 | 2021-06-09T01:18:59Z | 2021-06-09T01:18:59Z | CONTRIBUTOR | I'm happy to grab some or all of these in this PR, if you want. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Ensure db.path is a string before trying to insert into internal database 914130834 | |
861944202 | https://github.com/simonw/sqlite-utils/issues/272#issuecomment-861944202 | https://api.github.com/repos/simonw/sqlite-utils/issues/272 | MDEyOklzc3VlQ29tbWVudDg2MTk0NDIwMg== | eyeseast 25778 | 2021-06-16T01:41:03Z | 2021-06-16T01:41:03Z | CONTRIBUTOR | So, I do things like this a lot, too. I like the idea of piping in from stdin. Something like this would be nice to do in a makefile:
If you assumed that you're always piping out the same format you're piping in, the option names don't have to change. Depends how much you want to change formats. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Idea: import CSV to memory, run SQL, export in a single command 921878733 | |
869191854 | https://github.com/simonw/datasette/issues/1101#issuecomment-869191854 | https://api.github.com/repos/simonw/datasette/issues/1101 | MDEyOklzc3VlQ29tbWVudDg2OTE5MTg1NA== | eyeseast 25778 | 2021-06-27T16:42:14Z | 2021-06-27T16:42:14Z | CONTRIBUTOR | This would really help with this issue: https://github.com/eyeseast/datasette-geojson/issues/7 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
register_output_renderer() should support streaming data 749283032 | |
953911245 | https://github.com/simonw/sqlite-utils/issues/242#issuecomment-953911245 | https://api.github.com/repos/simonw/sqlite-utils/issues/242 | IC_kwDOCGYnMM4424fN | eyeseast 25778 | 2021-10-28T14:37:55Z | 2021-10-28T14:37:55Z | CONTRIBUTOR | I've been thinking about this a bit lately, doing a project that involves moving a lot of data in and out of SQLite files, datasette and GeoJSON. This has me leaning toward the idea that something like I know there's a lot of overlap in sqlite-utils and datasette, and maybe keeping sqlite-utils synchronous would let datasette be entirely async and give a cleaner separation of implementations. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Async support 817989436 | |
983155079 | https://github.com/simonw/sqlite-utils/issues/348#issuecomment-983155079 | https://api.github.com/repos/simonw/sqlite-utils/issues/348 | IC_kwDOCGYnMM46mcGH | eyeseast 25778 | 2021-12-01T00:28:40Z | 2021-12-01T00:28:40Z | CONTRIBUTOR | I'd use this. Right now, I tend to do |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Command for creating an empty database 1067771698 | |
1012158895 | https://github.com/simonw/sqlite-utils/issues/79#issuecomment-1012158895 | https://api.github.com/repos/simonw/sqlite-utils/issues/79 | IC_kwDOCGYnMM48VFGv | eyeseast 25778 | 2022-01-13T13:55:59Z | 2022-01-13T13:55:59Z | CONTRIBUTOR | Came here to add this. I might pick it up. Would also add a utility to create (and update and delete?) a spatial index. It's not much code but I have to look it up every time. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Helper methods for working with SpatiaLite 557842245 | |
1012230212 | https://github.com/simonw/sqlite-utils/issues/79#issuecomment-1012230212 | https://api.github.com/repos/simonw/sqlite-utils/issues/79 | IC_kwDOCGYnMM48VWhE | eyeseast 25778 | 2022-01-13T15:15:13Z | 2022-01-13T15:15:13Z | CONTRIBUTOR | Some proposals I'd add to sqlite-utils: Some version of this, from geojson-to-sqlite:
Also a function for creating a spatial index:
I don't know the nuances of updating a spatial index, or checking if one already exists. This could be a CLI method like:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Helper methods for working with SpatiaLite 557842245 | |
1012253198 | https://github.com/simonw/sqlite-utils/issues/79#issuecomment-1012253198 | https://api.github.com/repos/simonw/sqlite-utils/issues/79 | IC_kwDOCGYnMM48VcIO | eyeseast 25778 | 2022-01-13T15:39:14Z | 2022-01-13T15:39:14Z | CONTRIBUTOR | Other thing: If there get to be enough utils, I think it's worth moving all the spatialite stuff into its own file ( |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Helper methods for working with SpatiaLite 557842245 | |
1012413729 | https://github.com/simonw/sqlite-utils/issues/79#issuecomment-1012413729 | https://api.github.com/repos/simonw/sqlite-utils/issues/79 | IC_kwDOCGYnMM48WDUh | eyeseast 25778 | 2022-01-13T18:50:00Z | 2022-01-13T18:50:00Z | CONTRIBUTOR | One more thing I'm going to add: A method to add a geometry column, which I'll need to do to create a spatial index on a table. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Helper methods for working with SpatiaLite 557842245 | |
1013698557 | https://github.com/simonw/sqlite-utils/issues/79#issuecomment-1013698557 | https://api.github.com/repos/simonw/sqlite-utils/issues/79 | IC_kwDOCGYnMM48a8_9 | eyeseast 25778 | 2022-01-15T15:15:22Z | 2022-01-15T15:15:22Z | CONTRIBUTOR | @simonw I have a PR here https://github.com/simonw/sqlite-utils/pull/385 that adds Spatialite helpers on the Python side. Please let me know how it looks. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Helper methods for working with SpatiaLite 557842245 | |
1016651485 | https://github.com/simonw/datasette/issues/1601#issuecomment-1016651485 | https://api.github.com/repos/simonw/datasette/issues/1601 | IC_kwDOBm6k_c48mN7d | eyeseast 25778 | 2022-01-19T16:39:03Z | 2022-01-19T16:39:03Z | CONTRIBUTOR | I think both of these are Spatialite specific. They get generated when you first initialize the extension. KNN is actually deprecated in favor of KNN2, as I understand it. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add KNN and data_licenses to hidden tables list 1105916061 | |
1016994329 | https://github.com/simonw/datasette/issues/1605#issuecomment-1016994329 | https://api.github.com/repos/simonw/datasette/issues/1605 | IC_kwDOBm6k_c48nhoZ | eyeseast 25778 | 2022-01-20T00:27:17Z | 2022-01-20T00:27:17Z | CONTRIBUTOR | Right now, I usually have a line in a Makefile like this:
That all assumes I've loaded whatever I need into It works, but as you can see, it's a lot to manage, a lot of boilerplate, and it wasn't obvious how to get there. If there's an error in the canned query, I get an HTML error page, so that's hard to debug. And it's only one query, so each output needs a line like this. Make isn't ideal, either, for that reason. The thing I really liked with
And then do:
For HTML export, maybe there's a |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Scripted exports 1108671952 | |
1018741262 | https://github.com/simonw/datasette/issues/1605#issuecomment-1018741262 | https://api.github.com/repos/simonw/datasette/issues/1605 | IC_kwDOBm6k_c48uMIO | eyeseast 25778 | 2022-01-21T18:05:09Z | 2022-01-21T18:05:09Z | CONTRIBUTOR | Thinking about this more, as well as #1356 and various other tickets related to output formats, I think there's a missing plugin hook for formatting results, separate from Right now, as I understand it, getting output in any format goes through the normal view stack -- a table, a row or a query -- and so by the time It's possible this could be done using Another option is some kind of registry of serializers, which |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Scripted exports 1108671952 | |
1018778667 | https://github.com/simonw/datasette/issues/1605#issuecomment-1018778667 | https://api.github.com/repos/simonw/datasette/issues/1605 | IC_kwDOBm6k_c48uVQr | eyeseast 25778 | 2022-01-21T19:00:01Z | 2022-01-21T19:00:01Z | CONTRIBUTOR | Let me know if you want help prototyping any of this, because I'm thinking about it and trying stuff out. Happy to be a sounding board, if it helps. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Scripted exports 1108671952 | |
1029175907 | https://github.com/simonw/sqlite-utils/pull/385#issuecomment-1029175907 | https://api.github.com/repos/simonw/sqlite-utils/issues/385 | IC_kwDOCGYnMM49V_pj | eyeseast 25778 | 2022-02-03T16:36:54Z | 2022-02-03T16:36:54Z | CONTRIBUTOR | @simonw Not sure if you've seen this, but any chance you can run the tests? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add new spatialite helper methods 1102899312 | |
1029180984 | https://github.com/simonw/sqlite-utils/pull/385#issuecomment-1029180984 | https://api.github.com/repos/simonw/sqlite-utils/issues/385 | IC_kwDOCGYnMM49WA44 | eyeseast 25778 | 2022-02-03T16:42:04Z | 2022-02-03T16:42:04Z | CONTRIBUTOR | Fixed my spelling. That's a useful thing. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add new spatialite helper methods 1102899312 | |
1029306428 | https://github.com/simonw/sqlite-utils/pull/385#issuecomment-1029306428 | https://api.github.com/repos/simonw/sqlite-utils/issues/385 | IC_kwDOCGYnMM49Wfg8 | eyeseast 25778 | 2022-02-03T19:03:43Z | 2022-02-03T19:03:43Z | CONTRIBUTOR | I thought about adding these as methods on I do sort of like having all the Spatialite stuff in its own module, just because it's built around an extension you might not have or want, but I don't know if that's a good reason to have a different API. You could have |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add new spatialite helper methods 1102899312 | |
1029317527 | https://github.com/simonw/sqlite-utils/issues/79#issuecomment-1029317527 | https://api.github.com/repos/simonw/sqlite-utils/issues/79 | IC_kwDOCGYnMM49WiOX | eyeseast 25778 | 2022-02-03T19:18:02Z | 2022-02-03T19:18:02Z | CONTRIBUTOR | Taking part of the conversation from #385 here.
Yes. And also |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Helper methods for working with SpatiaLite 557842245 | |
1029326568 | https://github.com/simonw/sqlite-utils/pull/385#issuecomment-1029326568 | https://api.github.com/repos/simonw/sqlite-utils/issues/385 | IC_kwDOCGYnMM49Wkbo | eyeseast 25778 | 2022-02-03T19:28:26Z | 2022-02-03T19:28:26Z | CONTRIBUTOR |
This is fixed now. I had to take out the type annotations for |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add new spatialite helper methods 1102899312 | |
1029338360 | https://github.com/simonw/sqlite-utils/pull/385#issuecomment-1029338360 | https://api.github.com/repos/simonw/sqlite-utils/issues/385 | IC_kwDOCGYnMM49WnT4 | eyeseast 25778 | 2022-02-03T19:43:56Z | 2022-02-03T19:43:56Z | CONTRIBUTOR | Works for me. I was just looking at how the FTS extensions work and they're just methods, too. So this can be consistent with that. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add new spatialite helper methods 1102899312 | |
1029370537 | https://github.com/simonw/sqlite-utils/pull/385#issuecomment-1029370537 | https://api.github.com/repos/simonw/sqlite-utils/issues/385 | IC_kwDOCGYnMM49WvKp | eyeseast 25778 | 2022-02-03T20:25:58Z | 2022-02-03T20:25:58Z | CONTRIBUTOR | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add new spatialite helper methods 1102899312 | ||
1030002502 | https://github.com/simonw/sqlite-utils/pull/385#issuecomment-1030002502 | https://api.github.com/repos/simonw/sqlite-utils/issues/385 | IC_kwDOCGYnMM49ZJdG | eyeseast 25778 | 2022-02-04T13:50:19Z | 2022-02-04T13:50:19Z | CONTRIBUTOR | Awesome. Thanks for your help getting it in. Will now look at adding CLI versions of this. It's going to be super helpful on a bunch of my projects. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add new spatialite helper methods 1102899312 | |
1030629879 | https://github.com/simonw/sqlite-utils/issues/398#issuecomment-1030629879 | https://api.github.com/repos/simonw/sqlite-utils/issues/398 | IC_kwDOCGYnMM49bin3 | eyeseast 25778 | 2022-02-05T13:57:33Z | 2022-02-05T19:49:38Z | CONTRIBUTOR | I'm mostly using geojson-to-sqlite at the moment. Even with shapefiles, I'm usually converting to GeoJSON and projecting to EPSG:4326 (with ogr2ogr) first. I think an open question here is how much you want to leave to external libraries and how much you want here. My thinking has been that adding Spatialite helpers here would make external stuff easier, but it would be nice to have some standard way to insert geometries. I'm in the middle of adding GeoJSON and Spatialite support to geocode-sqlite, and that will probably use WKT. Since that's all points, I think I can just make the string inline. But for polygons, I'd generally use Shapely, which probably isn't a dependency you want to add to sqlite-utils. I've also been trying to get some of the approaches here to work, but haven't had any success so far. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add SpatiaLite helpers to CLI 1124237013 | |
1030740653 | https://github.com/simonw/sqlite-utils/issues/399#issuecomment-1030740653 | https://api.github.com/repos/simonw/sqlite-utils/issues/399 | IC_kwDOCGYnMM49b9qt | eyeseast 25778 | 2022-02-06T02:57:17Z | 2022-02-06T02:57:17Z | CONTRIBUTOR | I like the idea of having stock conversions you could import. I'd actually move them to a dedicated module (call it ```python from sqlite_utils.conversions import LongitudeLatitude db["places"].insert( { "name": "London", "lng": -0.118092, "lat": 51.509865, }, conversions={"point": LongitudeLatitude("lng", "lat")}, ) ``` I would definitely use that for every CSV I get with lat/lng columns where I actually need GeoJSON. |
{ "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Make it easier to insert geometries, with documentation and maybe code 1124731464 | |
1030740826 | https://github.com/simonw/sqlite-utils/issues/399#issuecomment-1030740826 | https://api.github.com/repos/simonw/sqlite-utils/issues/399 | IC_kwDOCGYnMM49b9ta | eyeseast 25778 | 2022-02-06T02:59:10Z | 2022-02-06T02:59:10Z | CONTRIBUTOR | All this said, I don't think it's unreasonable to point people to dedicated tools like |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Make it easier to insert geometries, with documentation and maybe code 1124731464 | |
1030741289 | https://github.com/simonw/sqlite-utils/issues/399#issuecomment-1030741289 | https://api.github.com/repos/simonw/sqlite-utils/issues/399 | IC_kwDOCGYnMM49b90p | eyeseast 25778 | 2022-02-06T03:03:43Z | 2022-02-06T03:03:43Z | CONTRIBUTOR |
Off the top of my head:
Some of this is easy enough with SQL functions, some is easier in Python. Maybe that's where having pre-built classes gets really handy, because it saves you from thinking about which way it's implemented. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Make it easier to insert geometries, with documentation and maybe code 1124731464 | |
1031779460 | https://github.com/simonw/sqlite-utils/issues/402#issuecomment-1031779460 | https://api.github.com/repos/simonw/sqlite-utils/issues/402 | IC_kwDOCGYnMM49f7SE | eyeseast 25778 | 2022-02-07T18:24:56Z | 2022-02-07T18:24:56Z | CONTRIBUTOR | I wonder if there's any overlap with the goals here and the I'm not sure that's exactly what we're talking about here, but it might be a parallel with some useful ideas to borrow. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Advanced class-based `conversions=` mechanism 1125297737 | |
1031791783 | https://github.com/simonw/sqlite-utils/issues/402#issuecomment-1031791783 | https://api.github.com/repos/simonw/sqlite-utils/issues/402 | IC_kwDOCGYnMM49f-Sn | eyeseast 25778 | 2022-02-07T18:37:40Z | 2022-02-07T18:37:40Z | CONTRIBUTOR | I've never used it either, but it's interesting, right? Feel like I should try it for something. I'm trying to get my head around how this conversions feature might work, because I really like the idea of it. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Advanced class-based `conversions=` mechanism 1125297737 | |
1032732242 | https://github.com/simonw/sqlite-utils/issues/402#issuecomment-1032732242 | https://api.github.com/repos/simonw/sqlite-utils/issues/402 | IC_kwDOCGYnMM49jj5S | eyeseast 25778 | 2022-02-08T15:26:59Z | 2022-02-08T15:26:59Z | CONTRIBUTOR | What if you did something like this: ```python class Conversion: def init(self, args, *kwargs): "Put whatever settings you need here"
``` This way, you're always passing an instance, which has methods that do the conversion. (Or you're passing a SQL string, as you would now.) The You'd then use it like this: ```python subclass might be unneeded here, if methods are presentclass LngLatConversion(Conversion): def init(self, x="longitude", y="latitude"): self.x = x self.y = y
table.insert_all(rows, conversions={"point": LngLatConversion("lng", "lat"))} ``` I haven't thought through all the implementation details here, and it'll probably break in ways I haven't foreseen, but wanted to get this idea out of my head. Hope it helps. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Advanced class-based `conversions=` mechanism 1125297737 | |
1035057014 | https://github.com/simonw/sqlite-utils/issues/402#issuecomment-1035057014 | https://api.github.com/repos/simonw/sqlite-utils/issues/402 | IC_kwDOCGYnMM49sbd2 | eyeseast 25778 | 2022-02-10T15:30:28Z | 2022-02-10T15:30:40Z | CONTRIBUTOR | Yeah, the CLI experience is probably where any kind of multi-column, configured setup is going to fall apart. Sticking with GIS examples, one way I might think about this is using the fiona CLI: ```sh assuming a database is already created and has SpatiaLitefio cat boundary.shp | sqlite-utils insert boundaries --conversion geometry GeometryGeoJSON - ``` Anyway, very interested to see where you land here. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Advanced class-based `conversions=` mechanism 1125297737 | |
1038336591 | https://github.com/simonw/sqlite-utils/issues/398#issuecomment-1038336591 | https://api.github.com/repos/simonw/sqlite-utils/issues/398 | IC_kwDOCGYnMM4948JP | eyeseast 25778 | 2022-02-13T18:48:21Z | 2022-02-13T18:49:49Z | CONTRIBUTOR | Been chipping away at this between other things and realized I think it's probably worth keeping a SpatiaLite flag on |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add SpatiaLite helpers to CLI 1124237013 | |
1040580250 | https://github.com/simonw/sqlite-utils/pull/407#issuecomment-1040580250 | https://api.github.com/repos/simonw/sqlite-utils/issues/407 | IC_kwDOCGYnMM4-Bf6a | eyeseast 25778 | 2022-02-15T17:40:00Z | 2022-02-15T17:40:00Z | CONTRIBUTOR | @simonw I think this is ready for a look. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add SpatiaLite helpers to CLI 1138948786 | |
1040998433 | https://github.com/simonw/sqlite-utils/pull/407#issuecomment-1040998433 | https://api.github.com/repos/simonw/sqlite-utils/issues/407 | IC_kwDOCGYnMM4-DGAh | eyeseast 25778 | 2022-02-16T01:29:39Z | 2022-02-16T01:29:39Z | CONTRIBUTOR | Happy to do it and have it in the library. Going to use it a bunch. This whole SpatiaLite toolchain become a huge part of my work in the past year. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add SpatiaLite helpers to CLI 1138948786 | |
1059647114 | https://github.com/simonw/sqlite-utils/issues/412#issuecomment-1059647114 | https://api.github.com/repos/simonw/sqlite-utils/issues/412 | IC_kwDOCGYnMM4_KO6K | eyeseast 25778 | 2022-03-05T01:54:24Z | 2022-03-05T01:54:24Z | CONTRIBUTOR | I haven't tried this, but it looks like Pandas has a method for this: https://pandas.pydata.org/docs/reference/api/pandas.read_sql_query.html |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Optional Pandas integration 1160182768 | |
1065477258 | https://github.com/simonw/sqlite-utils/issues/411#issuecomment-1065477258 | https://api.github.com/repos/simonw/sqlite-utils/issues/411 | IC_kwDOCGYnMM4_geSK | eyeseast 25778 | 2022-03-11T20:14:59Z | 2022-03-11T20:14:59Z | CONTRIBUTOR | Good call on adding this to |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Support for generated columns 1160034488 | |
1067981656 | https://github.com/simonw/sqlite-utils/issues/131#issuecomment-1067981656 | https://api.github.com/repos/simonw/sqlite-utils/issues/131 | IC_kwDOCGYnMM4_qBtY | eyeseast 25778 | 2022-03-15T13:21:42Z | 2022-03-15T13:21:42Z | CONTRIBUTOR | Just ran into this issue last night. I have a big table that's mostly numbers, but also a zip code column in a state where ZIP codes start with 0. Would be great to run something like this:
Maybe I'll take a crack at this one. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
sqlite-utils insert: options for column types 675753042 | |
1077671779 | https://github.com/simonw/sqlite-utils/issues/399#issuecomment-1077671779 | https://api.github.com/repos/simonw/sqlite-utils/issues/399 | IC_kwDOCGYnMM5AO_dj | eyeseast 25778 | 2022-03-24T14:11:33Z | 2022-03-24T14:11:43Z | CONTRIBUTOR | Coming back to this. I was about to add a utility function to datasette-geojson to convert lat/lng columns to geometries. Thankfully I googled first. There's a SpatiaLite function for this: MakePoint.
I'm not sure if that would work with |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Make it easier to insert geometries, with documentation and maybe code 1124731464 | |
1092357672 | https://github.com/simonw/datasette/issues/1699#issuecomment-1092357672 | https://api.github.com/repos/simonw/datasette/issues/1699 | IC_kwDOBm6k_c5BHA4o | eyeseast 25778 | 2022-04-08T01:39:40Z | 2022-04-08T01:39:40Z | CONTRIBUTOR |
That's my thinking, too. It's really the thing I've been wanting since writing
I think this probably needs either a new plugin hook separate from |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Proposal: datasette query 1193090967 | |
1092370880 | https://github.com/simonw/datasette/issues/1699#issuecomment-1092370880 | https://api.github.com/repos/simonw/datasette/issues/1699 | IC_kwDOBm6k_c5BHEHA | eyeseast 25778 | 2022-04-08T02:07:40Z | 2022-04-08T02:07:40Z | CONTRIBUTOR | So maybe
And stream gets an iterator, instead of a list of rows, so it can efficiently handle large queries. Maybe it also gets passed a destination stream, or it returns an iterator. I'm not sure what makes more sense. Either way, that might cover both CLI exports and streaming responses. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Proposal: datasette query 1193090967 | |
1092386254 | https://github.com/simonw/datasette/issues/1699#issuecomment-1092386254 | https://api.github.com/repos/simonw/datasette/issues/1699 | IC_kwDOBm6k_c5BHH3O | eyeseast 25778 | 2022-04-08T02:39:25Z | 2022-04-08T02:39:25Z | CONTRIBUTOR | And just to think this through a little more, here's what
Alternately, that could be an async generator, like this:
Not sure which makes more sense, but I think this pattern would open up a lot of possibility. If you had your stream_indented_json function, you could do |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Proposal: datasette query 1193090967 | |
1094453751 | https://github.com/simonw/datasette/issues/1699#issuecomment-1094453751 | https://api.github.com/repos/simonw/datasette/issues/1699 | IC_kwDOBm6k_c5BPAn3 | eyeseast 25778 | 2022-04-11T01:32:12Z | 2022-04-11T01:32:12Z | CONTRIBUTOR | Was looking through old issues and realized a bunch of this got discussed in #1101 (including by me!), so sorry to rehash all this. Happy to help with whatever piece of it I can. Would be very excited to be able to use format plugins with exports. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Proposal: datasette query 1193090967 | |
1099540225 | https://github.com/simonw/datasette/issues/1713#issuecomment-1099540225 | https://api.github.com/repos/simonw/datasette/issues/1713 | IC_kwDOBm6k_c5BiacB | eyeseast 25778 | 2022-04-14T19:09:57Z | 2022-04-14T19:09:57Z | CONTRIBUTOR | I wonder if this overlaps with what I outlined in #1605. You could run something like this:
And maybe that does what you need. Of course, that plugin isn't built yet. But that's the idea. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Datasette feature for publishing snapshots of query results 1203943272 | |
1105588651 | https://github.com/simonw/datasette/issues/1101#issuecomment-1105588651 | https://api.github.com/repos/simonw/datasette/issues/1101 | IC_kwDOBm6k_c5B5fGr | eyeseast 25778 | 2022-04-21T18:15:39Z | 2022-04-21T18:15:39Z | CONTRIBUTOR | What if you split rendering and streaming into two things:
That way current plugins still work, and streaming is purely additive. A |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
register_output_renderer() should support streaming data 749283032 | |
1105642187 | https://github.com/simonw/datasette/issues/1101#issuecomment-1105642187 | https://api.github.com/repos/simonw/datasette/issues/1101 | IC_kwDOBm6k_c5B5sLL | eyeseast 25778 | 2022-04-21T18:59:08Z | 2022-04-21T18:59:08Z | CONTRIBUTOR | Ha! That was your idea (and a good one). But it's probably worth measuring to see what overhead it adds. It did require both passing in the database and making the whole thing Just timing the queries themselves:
Looking at the network panel:
I'm not sure how best to time the GeoJSON generation, but it would be interesting to check. Maybe I'll write a plugin to add query times to response headers. The other thing to consider with async streaming is that it might be well-suited for a slower response. When I have to get the whole result and send a response in a fixed amount of time, I need the most efficient query possible. If I can hang onto a connection and get things one chunk at a time, maybe it's ok if there's some overhead. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
register_output_renderer() should support streaming data 749283032 | |
1125342229 | https://github.com/simonw/datasette/issues/741#issuecomment-1125342229 | https://api.github.com/repos/simonw/datasette/issues/741 | IC_kwDOBm6k_c5DE1wV | eyeseast 25778 | 2022-05-12T19:21:16Z | 2022-05-12T19:21:16Z | CONTRIBUTOR | Came here to check if this had been flagged already. Was helping a colleague get something on Cloud Run and had to dig to find If I get some time next week, maybe I'll try to tackle it. Would definitely make things easier to be able to do something like this:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Replace "datasette publish --extra-options" with "--setting" 607223136 | |
1128049716 | https://github.com/simonw/datasette/issues/1742#issuecomment-1128049716 | https://api.github.com/repos/simonw/datasette/issues/1742 | IC_kwDOBm6k_c5DPKw0 | eyeseast 25778 | 2022-05-16T19:24:44Z | 2022-05-16T19:24:44Z | CONTRIBUTOR | Where is |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
?_trace=1 fails with datasette-geojson for some reason 1237586379 | |
1128064864 | https://github.com/simonw/datasette/issues/1742#issuecomment-1128064864 | https://api.github.com/repos/simonw/datasette/issues/1742 | IC_kwDOBm6k_c5DPOdg | eyeseast 25778 | 2022-05-16T19:42:13Z | 2022-05-16T19:42:13Z | CONTRIBUTOR | Just to add a wrinkle here, this loads fine: https://alltheplaces-datasette.fly.dev/alltheplaces/places.geojson?_trace=1 But also, this doesn't add any trace data: https://alltheplaces-datasette.fly.dev/alltheplaces/places.json?_trace=1 What am I missing? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
?_trace=1 fails with datasette-geojson for some reason 1237586379 | |
1151887842 | https://github.com/simonw/datasette/issues/1528#issuecomment-1151887842 | https://api.github.com/repos/simonw/datasette/issues/1528 | IC_kwDOBm6k_c5EqGni | eyeseast 25778 | 2022-06-10T03:23:08Z | 2022-06-10T03:23:08Z | CONTRIBUTOR | I just put together a version of this in a plugin: https://github.com/eyeseast/datasette-query-files. Happy to have any feedback. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add new `"sql_file"` key to Canned Queries in metadata? 1060631257 | |
1258508215 | https://github.com/simonw/sqlite-utils/issues/491#issuecomment-1258508215 | https://api.github.com/repos/simonw/sqlite-utils/issues/491 | IC_kwDOCGYnMM5LA0-3 | eyeseast 25778 | 2022-09-26T19:22:14Z | 2022-09-26T19:22:14Z | CONTRIBUTOR | This might be fairly straightforward using SQLite's backup utility: https://docs.python.org/3/library/sqlite3.html#sqlite3.Connection.backup |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Ability to merge databases and tables 1383646615 | |
1258712931 | https://github.com/simonw/sqlite-utils/issues/491#issuecomment-1258712931 | https://api.github.com/repos/simonw/sqlite-utils/issues/491 | IC_kwDOCGYnMM5LBm9j | eyeseast 25778 | 2022-09-26T22:31:58Z | 2022-09-26T22:31:58Z | CONTRIBUTOR | Right. The backup command will copy tables completely, but in the case of conflicting table names, the destination gets overwritten silently. That might not be what you want here. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Ability to merge databases and tables 1383646615 | |
1290615599 | https://github.com/simonw/datasette/issues/1851#issuecomment-1290615599 | https://api.github.com/repos/simonw/datasette/issues/1851 | IC_kwDOBm6k_c5M7Tsv | eyeseast 25778 | 2022-10-25T14:05:12Z | 2022-10-25T14:05:12Z | CONTRIBUTOR | This could use a new plugin hook, too. I don't want to complicate your life too much, but for things like GIS, I'd want a way to turn regular JSON into SpatiaLite geometries or combine X/Y coordinates into point geometries and such. Happy to help however I can. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
API to insert a single record into an existing table 1421544654 | |
1291228502 | https://github.com/simonw/datasette/issues/1851#issuecomment-1291228502 | https://api.github.com/repos/simonw/datasette/issues/1851 | IC_kwDOBm6k_c5M9pVW | eyeseast 25778 | 2022-10-25T23:02:10Z | 2022-10-25T23:02:10Z | CONTRIBUTOR | That's reasonable. Canned queries and custom endpoints are certainly going to give more room for specific needs. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
API to insert a single record into an existing table 1421544654 | |
1292592210 | https://github.com/simonw/datasette/issues/1851#issuecomment-1292592210 | https://api.github.com/repos/simonw/datasette/issues/1851 | IC_kwDOBm6k_c5NC2RS | eyeseast 25778 | 2022-10-26T20:03:46Z | 2022-10-26T20:03:46Z | CONTRIBUTOR | Yeah, every time I see something cool done with triggers, I remember that I need to start using triggers. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
API to insert a single record into an existing table 1421544654 | |
1309735529 | https://github.com/simonw/datasette/issues/1884#issuecomment-1309735529 | https://api.github.com/repos/simonw/datasette/issues/1884 | IC_kwDOBm6k_c5OEPpp | eyeseast 25778 | 2022-11-10T03:57:23Z | 2022-11-10T03:57:23Z | CONTRIBUTOR | Here's how to get a list of virtual tables: https://stackoverflow.com/questions/46617118/how-to-fetch-names-of-virtual-tables |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Exclude virtual tables from datasette inspect 1439009231 | |
1313962183 | https://github.com/simonw/datasette/issues/1884#issuecomment-1313962183 | https://api.github.com/repos/simonw/datasette/issues/1884 | IC_kwDOBm6k_c5OUXjH | eyeseast 25778 | 2022-11-14T15:46:32Z | 2022-11-14T15:46:32Z | CONTRIBUTOR | It does work, though I think it's probably still worth excluding virtual tables that will always be zero. Here's the same inspection as before, now with
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Exclude virtual tables from datasette inspect 1439009231 | |
1314066229 | https://github.com/simonw/datasette/issues/1884#issuecomment-1314066229 | https://api.github.com/repos/simonw/datasette/issues/1884 | IC_kwDOBm6k_c5OUw81 | eyeseast 25778 | 2022-11-14T16:48:35Z | 2022-11-14T16:48:35Z | CONTRIBUTOR | I'm realizing I don't know if a virtual table will ever return a count. Maybe it depends on the implementation. For these three, just checking now, it'll always return zero. That said, I'm not sure there's any downside to having them return zero and caching that. (They're hidden, too.) |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Exclude virtual tables from datasette inspect 1439009231 | |
1314241058 | https://github.com/simonw/datasette/issues/1886#issuecomment-1314241058 | https://api.github.com/repos/simonw/datasette/issues/1886 | IC_kwDOBm6k_c5OVboi | eyeseast 25778 | 2022-11-14T19:06:35Z | 2022-11-14T19:06:35Z | CONTRIBUTOR | This probably counts as a case study: https://github.com/eyeseast/spatial-data-cooking-show. Even has video. Seriously, though, this workflow has become integral to my work with reporters and editors across USA TODAY Network. Very often, I get sent a folder of data in mixed formats, with a vague ask of how we should communicate some part of it to users. Datasette and its constellation of tools makes it easy to get a quick look at that data, run exploratory queries, map it and ask questions to figure out what's important to show. And then I export a version of the data that's exactly what I need for display. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Call for birthday presents: if you're using Datasette, let us know how you're using it here 1447050738 | |
1331187551 | https://github.com/simonw/datasette/issues/1605#issuecomment-1331187551 | https://api.github.com/repos/simonw/datasette/issues/1605 | IC_kwDOBm6k_c5PWE9f | eyeseast 25778 | 2022-11-29T19:29:42Z | 2022-11-29T19:29:42Z | CONTRIBUTOR | Interesting. I started a version using metadata like I outlined up top, but I realized that there's no documented way for a plugin to access either metadata or canned queries. Or at least, I couldn't find a way. There is this method: https://github.com/simonw/datasette/blob/main/datasette/app.py#L472 but I don't want to rely on it if it's not documented. Same with this: https://github.com/simonw/datasette/blob/main/datasette/app.py#L544 If those are safe, I'll build on them. I'm also happy to document them, if that greases the wheels. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Scripted exports 1108671952 | |
1332310772 | https://github.com/simonw/datasette/issues/1605#issuecomment-1332310772 | https://api.github.com/repos/simonw/datasette/issues/1605 | IC_kwDOBm6k_c5PaXL0 | eyeseast 25778 | 2022-11-30T15:06:37Z | 2022-11-30T15:06:37Z | CONTRIBUTOR | I'll add issues for both and do a documentation PR. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Scripted exports 1108671952 | |
1375708725 | https://github.com/simonw/datasette/issues/1978#issuecomment-1375708725 | https://api.github.com/repos/simonw/datasette/issues/1978 | IC_kwDOBm6k_c5R_6Y1 | eyeseast 25778 | 2023-01-09T14:30:00Z | 2023-01-09T14:30:00Z | CONTRIBUTOR | Totally missed that issue. I can close this as a duplicate. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Document datasette.urls.row and row_blob 1522778923 | |
1375810027 | https://github.com/simonw/datasette/issues/1983#issuecomment-1375810027 | https://api.github.com/repos/simonw/datasette/issues/1983 | IC_kwDOBm6k_c5SATHr | eyeseast 25778 | 2023-01-09T15:35:58Z | 2023-01-09T15:35:58Z | CONTRIBUTOR | Yes please, and thank you. I realized I was maybe getting myself in trouble using that, but I think it's a good way to standardize JSON handling. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Make CustomJSONEncoder a documented public API 1525815985 | |
1419357290 | https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1419357290 | https://api.github.com/repos/simonw/sqlite-utils/issues/524 | IC_kwDOCGYnMM5Umaxq | eyeseast 25778 | 2023-02-06T16:21:44Z | 2023-02-06T16:21:44Z | CONTRIBUTOR | SQLite doesn't have a native |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Transformation type `--type DATETIME` 1572766460 | |
1457172180 | https://github.com/simonw/datasette/issues/2033#issuecomment-1457172180 | https://api.github.com/repos/simonw/datasette/issues/2033 | IC_kwDOBm6k_c5W2q7U | eyeseast 25778 | 2023-03-06T22:54:52Z | 2023-03-06T22:54:52Z | CONTRIBUTOR | This would be a nice feature to have with |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`datasette install -r requirements.txt` 1612296210 | |
1465315726 | https://github.com/simonw/sqlite-utils/pull/531#issuecomment-1465315726 | https://api.github.com/repos/simonw/sqlite-utils/issues/531 | IC_kwDOCGYnMM5XVvGO | eyeseast 25778 | 2023-03-12T22:21:56Z | 2023-03-12T22:21:56Z | CONTRIBUTOR | Exactly, that's what I was running into. On my M2 MacBook, SpatiaLite ends up in what is -- for the moment -- a non-standard location, so even when I passed in the location with What I learned doing this originally is that SQLite needs to load the extension for each connection, even if all the SpatiaLite stuff is already in the database. So that's why Here's the code where I hit the error: https://github.com/eyeseast/boston-parcels/blob/main/Makefile#L30 It works using this branch. I'm not attached to this solution if you can think of something better. And I'm not sure, TBH, my test would actually catch what I'm after here. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add paths for homebrew on Apple silicon 1620164673 | |
1501017004 | https://github.com/simonw/sqlite-utils/pull/531#issuecomment-1501017004 | https://api.github.com/repos/simonw/sqlite-utils/issues/531 | IC_kwDOCGYnMM5Zd7Os | eyeseast 25778 | 2023-04-09T01:49:43Z | 2023-04-09T01:49:43Z | CONTRIBUTOR | I'm going to close this in favor of #536. Will try a cleaner approach to custom paths once that one is merge. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add paths for homebrew on Apple silicon 1620164673 | |
1642808866 | https://github.com/simonw/sqlite-utils/issues/567#issuecomment-1642808866 | https://api.github.com/repos/simonw/sqlite-utils/issues/567 | IC_kwDOCGYnMM5h60Yi | eyeseast 25778 | 2023-07-19T21:54:27Z | 2023-07-19T21:54:27Z | CONTRIBUTOR | Would this possibly make a bunch of |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Plugin system 1801394744 | |
1646656283 | https://github.com/simonw/sqlite-utils/issues/567#issuecomment-1646656283 | https://api.github.com/repos/simonw/sqlite-utils/issues/567 | IC_kwDOCGYnMM5iJfsb | eyeseast 25778 | 2023-07-22T19:32:24Z | 2023-07-22T19:32:24Z | CONTRIBUTOR | Cool. I might try to add a geojson plugin that handles both input and output. That would help me out a lot. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Plugin system 1801394744 | |
1648339661 | https://github.com/simonw/sqlite-utils/issues/578#issuecomment-1648339661 | https://api.github.com/repos/simonw/sqlite-utils/issues/578 | IC_kwDOCGYnMM5iP6rN | eyeseast 25778 | 2023-07-24T17:44:30Z | 2023-07-24T17:44:30Z | CONTRIBUTOR |
This is my goal, to have one plugin that handles input and output symmetrically. I'd like to be able to do something like this: ```sh sqlite-utils insert data.db table file.geojson --format geojson ... explore and manipulate in Datasettesqlite-utils query data.db ... --format geojson > output.geojson ``` This would work especially well with datasette-query-files, since I already have the queries I need saved in standalone SQL files. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Plugin hook for adding new output formats 1818838294 | |
1668113177 | https://github.com/simonw/sqlite-utils/issues/578#issuecomment-1668113177 | https://api.github.com/repos/simonw/sqlite-utils/issues/578 | IC_kwDOCGYnMM5jbWMZ | eyeseast 25778 | 2023-08-07T15:41:49Z | 2023-08-07T15:41:49Z | CONTRIBUTOR | I wonder if this should be two hooks: input and output. The current |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Plugin hook for adding new output formats 1818838294 |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [issue_comments] ( [html_url] TEXT, [issue_url] TEXT, [id] INTEGER PRIMARY KEY, [node_id] TEXT, [user] INTEGER REFERENCES [users]([id]), [created_at] TEXT, [updated_at] TEXT, [author_association] TEXT, [body] TEXT, [reactions] TEXT, [issue] INTEGER REFERENCES [issues]([id]) , [performed_via_github_app] TEXT); CREATE INDEX [idx_issue_comments_issue] ON [issue_comments] ([issue]); CREATE INDEX [idx_issue_comments_user] ON [issue_comments] ([user]);
created_at (date) 379 ✖