issues_fts
3,044 rows
This data as json, CSV (advanced)
| Link | rowid ▼ | title | body | issues_fts | rank |
|---|---|---|---|---|---|
| 267513424 | 267513424 | Addressable pages for every row in a table | /database-name-7sha256/table-name/compound-pk /database-name-7sha256/table-name/compound-pk.json Tricky part will be figuring out what the private key is - especially since it could be a compound primary key and it might involve different data types. | 1287 | |
| 267513523 | 267513523 | Initial proof-of-concept | Implemented in https://github.com/simonw/stateless-datasets/commit/de04d7a854d71003ffcf98028eab976a936c2dba | 1287 | |
| 267515678 | 267515678 | Make individual column valuables addressable, with smart content types | Some SQLite databases embed images in columns. It would be cool if these had URLs. /database-name-7sha256/table-name/compound-pk/column /database-name-7sha256/table-name/compound-pk/column.json /database-name-7sha256/table-name/compound-pk/column.png /database-name-7sha256/table-name/compound-pk/column.gif /database-name-7sha256/table-name/compound-pk/column.txt The one without an explicit file extension auto-detects the correct extension. | 1287 | |
| 267515836 | 267515836 | Make URLs immutable | Absolutely everything should have a far-future expires header Part of the URL will be the truncated sha1 hash of the database file itself, calculated at build time | 1287 | |
| 267516066 | 267516066 | Implement sensible query pagination | 1287 | ||
| 267516329 | 267516329 | Better JSON response options | Default returns this: { “Columns”: [“id”, “name”, “age”], “Rows”: [ [45, “Simon”, 36] ] } .jsono instead returns a list of objects each duplicating the headers in its keys. They both probably share the same pagination mechanism so it might not be a jsono flat list. | 1287 | |
| 267516650 | 267516650 | Framework where by every page is JSON plus a template | Every single page of my interface should be implemented as a function that returns JSON. I can then build my jinja templates on top of the exact data that would be returned by the API version. | 1287 | |
| 267517314 | 267517314 | Attempting an INSERT or UPDATE should return a sane error message | 1287 | ||
| 267517348 | 267517348 | Initial test suite | 1287 | ||
| 267517381 | 267517381 | Set up Travis | 1287 | ||
| 267522549 | 267522549 | Code that generates compile-time properties about the database | At a minimum this will include: * sha hash of each database file * list of tables with row counts for each database file | 1287 | |
| 267523511 | 267523511 | Make it so you can override templates | The app will ship with default templates but, just like with the Django admin, you will be able to override them using either explicit configuration settings or just by dropping in templates with certain file names. Template inheritance should work here, both allowing you to override just the base template and allowing you to customize tiny bits of others. | 1287 | |
| 267542338 | 267542338 | Add a syntax highlighting SQL editor | https://ace.c9.io/#nav=embedding looks like a good option | 1287 | |
| 267707940 | 267707940 | Datasette Plugins | It would be neat if additional functionality could be opted-in to the system in the form of easy-to-add plugins, hosted as separate packages. First example: a Google Analytics plugin, which adds GA tracking code with your tracking ID to the web interface for your dataset. This may be an opportunity to experiment with entry points: http://amir.rachum.com/blog/2017/07/28/python-entry-points/ | 1287 | |
| 267713226 | 267713226 | Support multiple databases | I'm going to loop through every database file in the app root directory and bundle all of them. Each one will be accessible at /databasename Note this is without the file extension, and we will disallow multiple files with the same name but different extensions. Supported extensions to start with will be `.db` and `.sqlite` and `.sqlite3` | 1287 | |
| 267726219 | 267726219 | Default HTML/CSS needs to look reasonable and be responsive | Version one should have the following characteristics: - Looks OK - Works great on mobile - Loads extremely fast - No JavaScript! At least not in v1. | 1287 | |
| 267732005 | 267732005 | In development mode, should still pick up new .db files | Follow on from #11 | 1287 | |
| 267739593 | 267739593 | See if I can get a websockets interface working | Since I am already running on Sanic, how hard would it be to add a websocket ebdpoint that lets you talk to sqlite interactively? Could this be used to efficiently support streaming in answers to giant queries? | 1287 | |
| 267741262 | 267741262 | Efficient url for downloading the raw database file | Use Sanic support for steaming large files http://sanic.readthedocs.io/en/latest/sanic/response.html#file-streaming | 1287 | |
| 267759136 | 267759136 | Config file with support for defining canned queries | Probably using YAML because then we get support for multiline strings: bats: db: bats.sqlite3 name: "Bat sightings" queries: specific_row: | select * from Bats where a = 1; | 1287 | |
| 267769034 | 267769034 | Use Sanic configuration mechanism | http://sanic.readthedocs.io/en/latest/sanic/config.html | 1287 | |
| 267769431 | 267769431 | Refactor to use class based views | http://sanic.readthedocs.io/en/latest/sanic/class_based_views.html | 1287 | |
| 267788884 | 267788884 | Support Django-style filters in querystring arguments | e.g /database/table?name__contains=Simon&age__gte=4 Same format as Django: double underscore as the split. If you need to match against a column that happens to contain a double underscore in its official name, do this: /database/table?weird__column__exact=Simon __exact is the default operation if none is supplied. | 1287 | |
| 267828746 | 267828746 | Implement full URL design | Full URL design: /database-name /database-name.json /database-name-7sha256 /database-name-7sha256.json /database-name/table-name /database-name/table-name.json /database-name-7sha256/table-name /database-name-7sha256/table-name.json /database-name-7sha256/table-name/compound-pk /database-name-7sha256/table-name/compound-pk.json | 1287 | |
| 267857622 | 267857622 | Endpoint that returns SQL ready to be piped into DB | It would be cool if I could figure out a way to generate both the create table statements and the inserts for an individual table or the entire database and then stream them down to the client. | 1287 | |
| 267861210 | 267861210 | Command line tool for uploading one or more DBs to Now | Uploading files appears to be undocumented, but I found it in their code here: https://github.com/zeit/now-cli/blob/0ca7d1fe44ebdf460b64fdc38ba543b8e295ac40/src/providers/sh/util/index.js#L291 | 1287 | |
| 267886330 | 267886330 | Ability to plot a simple graph | Might be as simple as: pick he type of chart (bar, line) and then pick the column for the X axis and the column for the Y axis. Maybe also allow a pie chart. It’s up to the user to come up with SQL that gets the right values. | 1287 | |
| 267886865 | 267886865 | /database?sql= should redirect correctly | Needs to redirect to the location with the hash while retaining the query string. This should also work with the .json extension. | 1287 | |
| 268050821 | 268050821 | Handle bytestring records encoding to JSON | http://localhost:8006/northwind-40d049b/Categories.json 500s right now The string representation of one of the values looks like this: b"\x15\x1c/\x00\x02\x00 This is a bytestring from the database which cannot be naively converted to a unicode string. | 1287 | |
| 268078453 | 268078453 | Do something neat with foreign keys | https://www.sqlite.org/pragma.html#pragma_foreign_key_list SQLite has robust support for introspecting foreign keys. I could use that to automatically link to the corresponding record from my tables. | 1287 | |
| 268087542 | 268087542 | Idea: colour scheme based on sha256 of db | 1287 | ||
| 268106803 | 268106803 | Try running SQLite queries in a separate thread | https://pymotw.com/3/asyncio/executors.html Would be good to have some actual benchmarks so I can evaluate if this is worth it or not. | 1287 | |
| 268110769 | 268110769 | Use locust for benchmarking and load tests | https://github.com/locustio/locust Needed for #32 | 1287 | |
| 268176505 | 268176505 | Support CSV export with a .csv extension | Maybe do this using streaming with multiple pagination SQL queries so we can support arbritrarily large exports. How would this work against a view which doesn’t have an obvious efficient pagination mechanism? Maybe limit views to up to 1000 exported records? Relates to #5 | 1287 | |
| 268262480 | 268262480 | date, year, month and day querystring lookups | - [ ] `?timestamp___date=2017-07-17` - return every item where the timestamp falls on that date - [ ] `?timestamp___year=2017` - return every item where the timestamp falls within 2017 - [ ] `?timestamp___month=1` - return every item where the month component is January - [ ] `?timestamp___day=10` - return every item where the day-of-the-month component is 10 Follow on from #23 | 1287 | |
| 268453968 | 268453968 | Ability to serialize massive JSON without blocking event loop | We run the risk of someone attempting a select statement that returns thousands of rows and hence takes several seconds just to JSON encode the response, effectively blocking the event loop and pausing all other traffic. The Twisted community have a solution for this, can we adapt that in some way? http://as.ynchrono.us/2010/06/asynchronous-json_18.html?m=1 | 1287 | |
| 268462768 | 268462768 | Experiment with patterns for concurrent long running queries | I want to understand how the system could perform under load with many concurrent long-running queries. Can we serve these without blocking the event loop? | 1287 | |
| 268469569 | 268469569 | Protect against malicious SQL that causes damage even though our DB is immutable | I’m currently operating under the assumption that it’s safe to allow arbitrary SQL statements because we are dealing with an immutable database. But this might not be the case - there are some pretty weird SQLite language extensions (ATTACH, PRAGMA etc) and I’m not certain they cannot be used to break things in a way that would affect future requests to the API. Solution: provide a “safe mode” option which disables the ?sql= mechanism. This still leaves the URL filter lookups, so I need to make sure that those are “safe”. In the future I may also implement a whitelist option where datasets can be configured to only allow specific filters against specific columns. | 1287 | |
| 268470572 | 268470572 | Implement command-line tool interface | The first version needs to take one or more file names or URLs, then generate and deploy an app to Now. It will assume you already have the now command installed and configured. | 1287 | |
| 268590777 | 268590777 | Homepage should show summary of databases | I sch database should have a name, optional description, download link and a summary of the tables Flights.db Flights and suchlike blah. URL? License? 577373 rows across 14 tables airports, routes, airlines... Title of the homepage is derived from the databases or can be manually overridden e. “Datasets of Flights, NHS, Blah...” - or if only one database just the title of that. | 1287 | |
| 268591332 | 268591332 | Homepage UI for editing metadata file | Since we are going to have a metadata file which sets the title/description/etc for each database, why not allow you to run the app in —dev mode which makes the homepage into a WYSIWYG editor that can save to that file format. | 1287 | |
| 268592894 | 268592894 | While running, server should spot new db files added to its directory | Maybe in each request it checks the time and if 5s has elapsed since t last scanned the directory it scans it again This would allow people with dedicated hosting to run the app there and just upload new datasets whenever they want. It would also be very convenient for development. | 1287 | |
| 269731374 | 269731374 | ?_group_count=country - return counts by specific column(s) | Imagine if this: https://stateless-datasets-jykibytogk.now.sh/flights-07d1283/airports.jsono?country__contains=gu&_group_count=country Turned into this: https://stateless-datasets-jykibytogk.now.sh/flights-07d1283?sql=select%20country,%20count(*)%20as%20group_count_country%20from%20airports%20where%20country%20like%20%27%gu%%27%20group%20by%20country%20order%20by%20group_count_country%20desc This would involve introducing a new precedent of query string arguments that start with an _ having special meanings. While we're at it, could try adding _fields=x,y,z Tasks: - [x] Get initial version working - [ ] Refactor code to not just "pretend to be a view" - [ ] Get foreign key relationships expanded | 1287 | |
| 271242824 | 271242824 | Run SQLite operations in a thread pool | Let's run SQLite operations in threads, so we don't end up blocking our core event loop. These articles are helpful: * https://pymotw.com/3/asyncio/executors.html * https://marlinux.wordpress.com/2017/05/19/python-3-6-asyncio-sqlalchemy/ | 1287 | |
| 271301468 | 271301468 | Dockerfile should build more recent SQLite with FTS5 and spatialite support | The SQLite bundled with Python 3 doesn't support the FTS5 search extension. It would be nice if the SQLite built by our Dockerfile could support as many modern SQLite features as possible. https://web.archive.org/web/20170212034155/http://charlesleifer.com/blog/using-the-sqlite-json1-and-fts5-extensions-with-python/ has instructions on building a more recent SQLite and the pysqlite package. Our Dockerfile could carry out an updated version of this process. | 1287 | |
| 271831408 | 271831408 | Create neat example database | How about data from open elections eg https://github.com/openelections/openelections-data-ca?files=1 | 1287 | |
| 272391665 | 272391665 | Switch to ujson | ujson is already a dependency of Sanic, and should be quite a bit faster. | 1287 | |
| 272661336 | 272661336 | Pick a name | Options so far: * immutabase * datasite * sqlstatic * dbserve * sqlserve Terms to play with: * immutable * sqlite * dataset * json * static * serve | 1287 | |
| 272694136 | 272694136 | Unit tests against application itself | Use Sanic’s testing mechanism. Test should create a temporary SQLite database file on disk by executing sql that is stored in the test themselves. For the moment we can just test the JSON API more thoroughly and just sanity check that the HTML output doesn’t throw any errors. | 1287 | |
| 272735257 | 272735257 | Make a proper README | Include instructions on building a local Docker container - currently detailed here: https://gist.github.com/simonw/0ea5c960608c2d876e4637a5e48aa95d (those instructions don't work now that we have removed the Dockerfile in favour of a template generated by `datasette publish`) | 1287 | |
| 273026602 | 273026602 | Solution for temporarily uploading DB so it can be built by docker | For the `datasette publish` command I ideally need a way of uploading the specified DB to somewhere temporary on the internet so that when the Dockerfile is built by the final hosting location it can download that database as part of the build process. | 1287 | |
| 273054652 | 273054652 | Implement a better database index page | This view isn't great. I should do a better job of separating out tables from views and indexes, showing the count of rows in each table, and maybe move the SQL to the individual table pages. <img width="871" alt="flights" src="https://user-images.githubusercontent.com/9599/32423242-1b4458ce-c25a-11e7-910f-2dc1de909b8f.png"> | 1287 | |
| 273121803 | 273121803 | Views should not attempt to link to records / use rowids | http://localhost:8001/parlgov-development-25f9855/view_variable <img width="837" alt="parlgov-development__view_variable" src="https://user-images.githubusercontent.com/9599/32686757-5b40f6a8-c660-11e7-88de-5e8dfb12ccf1.png"> | 1287 | |
| 273127117 | 273127117 | Ship first version to PyPI | Just before doing this, update the Dockerfile template to `pip install datasette` https://github.com/simonw/datasette/blob/65e350ca2a4845c25752a62c16ba58cfe2c14b9b/datasette/utils.py#L125 | 1287 | |
| 273127443 | 273127443 | Easy way to block search engine crawling in robots.txt | For people who don't want their datasets to be crawled by search engines. | 1287 | |
| 273127694 | 273127694 | Ship a Docker image of the whole thing | The generated Docker images can then just inherit from that. This will speed up deploys as no need to `pip install` anything. - [x] Ship that image to Docker Hub - [ ] Update the generated Dockerfile to use it | 1287 | |
| 273128608 | 273128608 | publish command should detect if "now" is installed | If now is not installed, it should tell you where to get it. | 1287 | |
| 273157085 | 273157085 | datasette publish hyper | This is a bit tricky, because unlike Now there doesn't seem to be a way to tell Hyper to "build this Dockerfile and deploy the resulting image". They expect you to build a container and publish it to a registry instead. https://docs.hyper.sh/Reference/CLI/load.html allows you to publish an image directly from a tarball, but that still leaves the challenge of creating that image. The nice thing about the Now integration is that you don't need to have Docker installed on your local machine. | 1287 | |
| 273163905 | 273163905 | Rethink how metadata is generated and stored | I broke the existing mechanism in 407795b61217205625f2d4e084afbf69f1db781b In order to get unit tests for the sanic app working. I think i should ditch the build-metadata.json cache file entirely and calculate the SHA hashes on startup. Not sure what to do about the table row counts. | 1287 | |
| 273173116 | 273173116 | Common header and footer | Split from #16 - [x] A link to the homepage from some kind of navigation bar in the header - [x] link to github.com/simonw/datasette in the footer - [x] Slightly better titles (maybe ditch the visited link colours for titles only? should keep those for primary key links) | 1287 | |
| 273174397 | 273174397 | Link to .json and .jsono versions on various pages | 1287 | ||
| 273174447 | 273174447 | Review design of JSON output | 1287 | ||
| 273181020 | 273181020 | Support for ?field__isnull=1 or similar | 1287 | ||
| 273191608 | 273191608 | Re-implement ?sql= mode | Here's the code I removed: async def data(self, request, name, hash): sql = 'select * from sqlite_master' custom_sql = False params = {} if request.args.get('sql'): params = request.raw_args sql = params.pop('sql') validate_sql_select(sql) custom_sql = True rows = await self.execute(name, sql, params) columns = [r[0] for r in rows.description] return { 'database': name, 'rows': rows, 'columns': columns, 'query': { 'sql': sql, 'params': params, } }, { 'database_hash': hash, 'custom_sql': custom_sql, } | 1287 | |
| 273191806 | 273191806 | Show table SQL on table page | Let's do the SQL for the table you are looking at, plus SQL for any indexes that mention that table. The page for a view should show the SQL for that view. | 1287 | |
| 273192789 | 273192789 | Command that builds a local docker container | Be nice to indicate that this isn't just for Now. Shouldn't be too hard either. | 1287 | |
| 273247186 | 273247186 | Support for title/source/license metadata | I've decided this is important for launch: I want to set a precedent for people citing, licensing and documenting their datasets. Not sure how best to go about supporting this. I'd like to allow for the following data to be optionally attached to any given database: - Title - Description, potentially in markdown? - Original source URL - License I'd also like the ability to attach descriptions to individual tables - and maybe even to table columns? The question then becomes: how should this information be stored. A few options: - In the SQLite database itself, in a specially named table. Problem here is that this means having to modify SQLite databases before publishing them. - In a separate SQLite database that can be published alongside the databases we are publishing. - In a JSON file. This is neat, but JSON files are not a great editing experience once you start including multiple lines (e.g. a markdown description). - In a YAML file. This is a better format for multi-line descriptions, but still isn't a great editing experience. Whatever the format, it can be made much more usable by offering a web-based editing UI for populating it (a special mode the server can be run in). | 1287 | |
| 273248366 | 273248366 | Enforce pagination (or at least limits) for arbitrary custom SQL | It's way too easy to accidentally trigger a page that returns 100,000 rows at the moment. I need to use the LIMIT clause on views and custom SQL - I can support pagination "next" links using offset as well. | 1287 | |
| 273267081 | 273267081 | Paginate views using OFFSET/LIMIT | As with #69 these should obey a maximum offset setting, which can be over-ridden. | 1287 | |
| 273278840 | 273278840 | Set up some example datasets on a Cloudflare-backed domain | To better demonstrate the caching and HTTP/2 features, I'd like to go live with some demos that are hosted behind Cloudflare. - [x] Redirect https://datasettes.com/ and https://www.datasettes.com/ to https://github.com/simonw/datasette - [x] Have `now domain add -e datasettes.com` run without errors (hopefully just a matter of waiting for the DNS to update) - [x] Alias an example dataset hosted on Now on a datasettes.com subdomain - [x] Confirm that HTTP caching and HTTP/2 redirect pushing works as expected - this may require another page rule | 1287 | |
| 273283166 | 273283166 | publish command should take an optional --name argument | To set the directory name so that now will inherit it as the name of the app. Defaults to datasette | 1287 | |
| 273296178 | 273296178 | _nocache=1 query string option for use with sort-by-random | The one place where we wouldn’t want cdching is if we have something which uses sort by random to return random items. We can offer a _nocache=1 querystring argument to support this. | 1287 | |
| 273296684 | 273296684 | Send a 302 redirect to the new hash for hits to old hashes | 1287 | ||
| 273509159 | 273509159 | Add --cors argument to serve | 1287 | ||
| 273510781 | 273510781 | publish should have required argument specifying publisher | Initially the only argument will be “now” - but “hyper” can be added in the future | 1287 | |
| 273537940 | 273537940 | Add Travis CI badge to README | Also fix this newline issue: <img width="647" alt="simonw_datasette__instant_json_api_for_your_sqlite_database" src="https://user-images.githubusercontent.com/9599/32743234-ae81b224-c860-11e7-98a9-980b7b448ffc.png"> | 1287 | |
| 273554949 | 273554949 | Rename after to next and provide a next_url | 1287 | ||
| 273569068 | 273569068 | Add more detailed API documentation to the README | Need to document: - [ ] The ?column__gt=4 style filter arguments for tables - [ ] The ?sql= API, and how named parameters work - [ ] How API pagination works - [ ] How redirects and cache headers work | 1287 | |
| 273569477 | 273569477 | Deploy final versions of fivethirtyeight and parlgov datasets (with view pagination) | Final versions should be deployed using the first released version of datasette. | 1287 | |
| 273595473 | 273595473 | :fire: Removes DS_Store | 1287 | ||
| 273596159 | 273596159 | Post a blog entry announcing it to the world | 1287 | ||
| 273626815 | 273626815 | Individual row view is broken | https://parlgov.datasettes.com/parlgov-25f9855/viewcalc_parliament_composition/18 <img width="822" alt="cursor_and_localhost_8002_parlgov-25f9855_viewcalc_parliament_composition_18" src="https://user-images.githubusercontent.com/9599/32756593-c439c71c-c88f-11e7-9243-b6e1b778c8fa.png"> | 1287 | |
| 273660425 | 273660425 | datasette package --metadata does not work with a relative path | $ datasette package ~/parlgov-db/parlgov.db --metadata=~/parlgov-db/parlgov.json Usage: datasette package [OPTIONS] FILES... Error: Invalid value for "-m" / "--metadata": Could not open file: ~/parlgov-db/parlgov.json: No such file or directory simonw-07542:~ simonw$ cd ~/parlgov-db/ simonw-07542:parlgov-db simonw$ datasette package ~/parlgov-db/parlgov.db --metadata=parlgov.json Sending build context to Docker daemon 4.46MB Step 1/7 : FROM python:3 | 1287 | |
| 273678673 | 273678673 | Detect foreign keys and use them to link HTML pages together | https://stackoverflow.com/a/44430157/6083 documents the PRAGMA needed to extract foreign key references for a table. At a minimum we can link column values known to be foreign keys to the corresponding row page. We could try to summarize the linked row in some way too - somehow extracting a sensible link title, maybe based on additional configuration in the metadata.json file. Still todo: - [x] Fix it to csvs-to-sqlite refactoring command correctly creates primary key on generated tables - [x] Ship new csvs-to-sqlite with refactoring command - [x] Refactor column logic to be more predictable in our templates (the rowid special case) - [x] Mechanism by which table metadata can specify the "label" column for a table - [x] Automatically set the label column as the first column that isn't a primary key (falling back on primary key) - [x] Code which runs a "select id, label from table where id in (...)" query as part of the tableview and populates a lookup dictionary - [x] Modify templates to use values from that lookup dictionary | 1287 | |
| 273703829 | 273703829 | Filter UI on table page | A UI for building up simple table queries by adding additional filter rules that get executed as query parameters in the URL. | 1287 | |
| 273709194 | 273709194 | Configure Travis to release new tags to PyPI | https://docs.travis-ci.com/user/deployment/pypi/ | 1287 | |
| 273775212 | 273775212 | Add NHS England Hospitals example to wiki | https://nhs-england-hospitals.now.sh and an associated map visualisation: http://run.plnkr.co/preview/cj9zlf1qc0003414y90ajkwpk/ Datasette is wonderful! | 1287 | |
| 273816720 | 273816720 | SQL syntax highlighting with CodeMirror | Addresses #13 Future enhancements could include autocompletion of table and column names, e.g. with ```javascript extraKeys: {"Ctrl-Space": "autocomplete"}, hintOptions: {tables: { users: ["name", "score", "birthDate"], countries: ["name", "population", "size"] }} ``` (see https://codemirror.net/doc/manual.html#addon_sql-hint and source at http://codemirror.net/mode/sql/) | 1287 | |
| 273846123 | 273846123 | datasette publish heroku | Heroku has Docker container support so this should not be too hard: https://devcenter.heroku.com/articles/container-registry-and-runtime See also #59 This should work exactly like the existing “datasette publish now....” command except it would be “datasette publish heroku...” | 1287 | |
| 273878873 | 273878873 | Option to serve databases from a different prefix, serve regular content elsewhere | It would be useful if the databases themselves could be served from a prefix e.g. datasette serve mydb.db --path-prefix=db Now my database is at `http://localhost:8001/db/mydb-23423` This would free up the rest of the URL namespace for other things. Maybe we could have an option to serve static content from a known folder e.g. datasette serve mydb.db --path-prefix=db --root-content=~/my-project/static Now a hit to `http://localhost:8001/news/` serves content from `~/my-project/static/news/index.html` This would make it trivial to package up entire HTML/CSS/JS apps with one or more underlying SQLite databases. Running without `--cors` would be fine here because any JS apps would be hosted on the same origin. | 1287 | |
| 273895344 | 273895344 | Add --license --license_url --source --source_url --title arguments to datasette publish | I keep on using the `echo '{"source": "..."}' | datasette publish now --metadata=-` pattern, which suggests it makes sense for us to support these as optional arguments. https://gist.github.com/simonw/9f8bf23b37a42d7628c4dcc4bba10253 | 1287 | |
| 273944952 | 273944952 | Package as standalone binary | hint: more than the docker image a standalone and multiplatform binary (containing the app and the database) could be simpler to distribute. i would like to investigate the possibility to package everything with [pyinstaller](http://www.pyinstaller.org/) adding the database as a [data file](https://pythonhosted.org/PyInstaller/spec-files.html#adding-data-files) | 1287 | |
| 273961179 | 273961179 | Initial add simple prod ready Dockerfile refs #57 | Multi-stage build based off official python:3.6-slim Example usage: ``` docker run --rm -t -i -p 9000:8001 -v $(pwd)/db:/db datasette datasette serve /db/chinook.db ``` | 1287 | |
| 273998513 | 273998513 | Allow shorter time limits to be set using a ?_sql_time_limit_ms =20 query string limit | This cannot be greater than the configured time limit. | 1287 | |
| 274001453 | 274001453 | UI for editing named parameters | On any page displaying a custom query that includes named parameters, we should show HTML form fields for editing those parameters. Eg the breed parameter on https://australian-dogs.now.sh/australian-dogs-3ba9628?sql=select+name%2C+count%28*%29+as+n+from+%28%0D%0A%0D%0Aselect+upper%28%22Animal+name%22%29+as+name+from+%5BAdelaide-City-Council-dog-registrations-2013%5D+where+Breed+like+%3Abreed%0D%0A%0D%0Aunion+all%0D%0A%0D%0Aselect+upper%28Animal_Name%29+as+name+from+%5BAdelaide-City-Council-dog-registrations-2014%5D+where+Breed_Description+like+%3Abreed%0D%0A%0D%0Aunion+all+%0D%0A%0D%0Aselect+upper%28Animal_Name%29+as+name+from+%5BAdelaide-City-Council-dog-registrations-2015%5D+where+Breed_Description+like+%3Abreed%0D%0A%0D%0Aunion+all%0D%0A%0D%0Aselect+upper%28%22AnimalName%22%29+as+name+from+%5BCity-of-Port-Adelaide-Enfield-Dog_Registrations_2016%5D+where+AnimalBreed+like+%3Abreed%0D%0A%0D%0Aunion+all%0D%0A%0D%0Aselect+upper%28%22Animal+Name%22%29+as+name+from+%5BMitcham-dog-registrations-2015%5D+where+Breed+like+%3Abreed%0D%0A%0D%0Aunion+all%0D%0A%0D%0Aselect+upper%28%22DOG_NAME%22%29+as+name+from+%5Bburnside-dog-registrations-2015%5D+where+DOG_BREED+like+%3Abreed%0D%0A%0D%0Aunion+all+%0D%0A%0D%0Aselect+upper%28%22Animal_Name%22%29+as+name+from+%5Bcity-of-playford-2015-dog-registration%5D+where+Breed_Description+like+%3Abreed%0D%0A%0D%0Aunion+all%0D%0A%0D%0Aselect+upper%28%22Animal+Name%22%29+as+name+from+%5Bcity-of-prospect-dog-registration-details-2016%5D+where%22Breed+Description%22+like+%3Abreed%0D%0A%0D%0A%29+group+by+name+order+by+n+desc%3B&breed=pug | 1287 | |
| 274022950 | 274022950 | Link to JSON for the list of tables | https://twitter.com/yschimke/status/930606210855854080 | 1287 | |
| 274023417 | 274023417 | Default to 127.0.0.1 not 0.0.0.0 | https://twitter.com/yschimke/status/930606210855854080 | 1287 | |
| 274023625 | 274023625 | Start a change log | 1287 | ||
| 274160723 | 274160723 | TemplateAssertionError: no filter named 'tojson' | A 500 error is raised upon clicking on the name of a table on the homepage, say _http://0.0.0.0:8001/_ to _http://0.0.0.0:8001/test_check-c1f4771/users_ The API part seems to function as intended, though... ``` 2017-11-15 14:33:57 - (sanic)[ERROR]: Traceback (most recent call last): File "/usr/local/lib/python3.5/dist-packages/sanic/app.py", line 503, in handle_request response = await response File "/usr/local/lib/python3.5/dist-packages/datasette/app.py", line 155, in get return await self.view_get(request, name, hash, **kwargs) File "/usr/local/lib/python3.5/dist-packages/datasette/app.py", line 219, in view_get **context, File "/usr/local/lib/python3.5/dist-packages/sanic_jinja2/__init__.py", line 84, in render return html(self.render_string(template, request, **context)) File "/usr/local/lib/python3.5/dist-packages/sanic_jinja2/__init__.py", line 81, in render_string return self.env.get_template(template).render(**context) File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 812, in get_template return self._load_template(name, self.make_globals(globals)) File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 786, in _load_template template = self.loader.load(self, name, globals) File "/usr/lib/python3/dist-packages/jinja2/loaders.py", line 125, in load code = environment.compile(source, name, filename) File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 565, in compile self.handle_exception(exc_info, source_hint=source_hint) File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 754, in handle_exception reraise(exc_type, exc_value, tb) File "/usr/lib/python3/dist-packages/jinja2/_compat.py", line 37, in reraise raise value.with_traceback(tb) File "/usr/local/lib/python3.5/dist-packages/datasette/templates/table.html", line 29, in template <pre>params = {{ query.params|tojson(4) }}</pre> File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 515, i… | 1287 | |
| 274161964 | 274161964 | TemplateAssertionError: no filter named 'tojson' | I get an exception clicking on the table link: ``` 2017-11-15 08:40:10 - (sanic)[ERROR]: Traceback (most recent call last): File "/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/sanic/app.py", line 503, in handle_request response = await response File "/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/datasette/app.py", line 155, in get return await self.view_get(request, name, hash, **kwargs) File "/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/datasette/app.py", line 219, in view_get **context, File "/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/sanic_jinja2/__init__.py", line 84, in render return html(self.render_string(template, request, **context)) File "/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/sanic_jinja2/__init__.py", line 81, in render_string return self.env.get_template(template).render(**context) File "/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/environment.py", line 812, in get_template return self._load_template(name, self.make_globals(globals)) File "/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/environment.py", line 786, in _load_template template = self.loader.load(self, name, globals) File "/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/loaders.py", line 125, in load code = environment.compile(source, name, filename) File "/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/environment.py", line 565, in compile self.handle_exception(exc_info, source_hint=source_hint) File "/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/environment.py", line 754, in handle_exception reraise(exc_type, exc_value, tb) File "/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/jinja2/_compat.py", line 37, in reraise raise value.with_traceback(tb) File "/Users/e/anaconda3-4.2.0/lib/python3.5/site-packages/datasette/templates/table.html", line 29, in template <pre>params = {{ query.params|tojson(4) }}</pre> File "/Users/e/… | 1287 |
Advanced export
JSON shape: default, array, newline-delimited
CREATE VIRTUAL TABLE [issues_fts] USING FTS5 (
[title], [body],
content=[issues]
);