issues

16 rows where assignee = 9599 sorted by updated_at descending

View and edit SQL

Suggested facets: user, milestone, comments, author_association, created_at (date), updated_at (date), closed_at (date)

type

state

repo

id node_id number title user state locked assignee milestone comments created_at updated_at ▲ closed_at author_association pull_request body repo type active_lock_reason
455965174 MDU6SXNzdWU0NTU5NjUxNzQ= 508 Ability to set default sort order for a table or view in metadata.json simonw 9599 closed 0 simonw 9599   1 2019-06-13T21:40:51Z 2020-05-28T18:53:03Z 2020-05-28T18:53:02Z OWNER  

It can go here in the documentation: https://datasette.readthedocs.io/en/stable/metadata.html#setting-which-columns-can-be-used-for-sorting

Also need to fix this sentence which is no longer true:

By default, database views in Datasette do not support sorting

datasette 107914493 issue  
449931899 MDU6SXNzdWU0NDk5MzE4OTk= 494 --reload should only trigger for -i databases simonw 9599 closed 0 simonw 9599   1 2019-05-29T17:28:43Z 2020-02-24T19:45:05Z 2020-02-24T19:45:05Z OWNER  

Right now it's triggering any time a mutable database changes.

datasette 107914493 issue  
453639196 MDU6SXNzdWU0NTM2MzkxOTY= 504 Remove TableView ?_group_count= feature simonw 9599 closed 0 simonw 9599   0 2019-06-07T18:25:18Z 2019-11-06T05:13:10Z 2019-11-06T05:13:10Z OWNER  

This feature really doesn't warrant continuing to exist. For reference: #150 and #44

Don't forget to remove it from the docs: https://github.com/simonw/datasette/blob/172da009d890aa029cff7138b4dcfd4f60948525/docs/json_api.rst#L322-L324

datasette 107914493 issue  
453243459 MDU6SXNzdWU0NTMyNDM0NTk= 503 Handle SQLite databases with spaces in their names? chrismp 7936571 closed 0 simonw 9599   1 2019-06-06T21:20:59Z 2019-11-04T23:16:30Z 2019-11-04T23:16:30Z NONE  

I named my SQLite database "Government workers" and published it to Heroku. When I clicked the "Government workers" database online it lead to a 404 page: Database not found: Government%20workers.

I believe this is because the database name has a space.

datasette 107914493 issue  
406055201 MDU6SXNzdWU0MDYwNTUyMDE= 406 Support nullable foreign keys in _labels mode simonw 9599 closed 0 simonw 9599   2 2019-02-03T05:34:20Z 2019-11-02T22:39:28Z 2019-11-02T22:30:27Z OWNER  

Currently if there's a null in a foreign key we get "None" displayed in the inflated view:

datasette 107914493 issue  
440437037 MDU6SXNzdWU0NDA0MzcwMzc= 454 Plugin for allowing CORS from specified hosts simonw 9599 closed 0 simonw 9599   5 2019-05-05T12:05:02Z 2019-10-03T23:59:57Z 2019-10-03T23:59:56Z OWNER  

It would be useful if Datasette could be configured to allow CORS requests from one or more origins, as opposed to only allowing either none or "*".

This is slightly tricky because the Access-Control-Allow-Origin: https://foo.example header is only allowed to return one value per request - and according to https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS "The Access-Control-Allow-Origin header should contain the value that was sent in the request's Origin header."

This means the application code needs to have a whitelist of allowed hosts and code that dynamically changes the outgoing Access-Control-Allow-Origin header based on the Origin header from the incoming request.

datasette 107914493 issue  
456569067 MDU6SXNzdWU0NTY1NjkwNjc= 510 Ability to facet by delimiter (e.g. comma separated fields) simonw 9599 open 0 simonw 9599   1 2019-06-15T19:34:41Z 2019-07-08T15:44:51Z   OWNER  

E.g. if a field contains "Tags,With,Commas" be able to facet them in the same way as _facet_array= lets you facet ["Tags", "With", "Commas"]

datasette 107914493 issue  
453829910 MDU6SXNzdWU0NTM4Mjk5MTA= 505 Add white-space: pre-wrap to SQL create statement simonw 9599 closed 0 simonw 9599 Datasette 0.29 4471010 0 2019-06-08T19:59:56Z 2019-07-07T20:26:55Z 2019-07-07T20:26:55Z OWNER  

Right now a super-long CREATE TABLE statement causes the table page to be even wider than the table itself:

Adding white-space: pre-wrap to that <pre> element is an easy fix:

datasette 107914493 issue  
459598080 MDU6SXNzdWU0NTk1OTgwODA= 520 asgi_wrapper plugin hook simonw 9599 closed 0 simonw 9599   3 2019-06-23T17:16:45Z 2019-07-03T04:40:34Z 2019-07-03T04:06:28Z OWNER  

After #272 we can finally add this hook. It will allow plugins to wrap their own ASGI middleware around Datasette. Potential use-cases include:

  • adding authentication
  • custom CORS headers (see #454)
  • maybe gzip support?
  • possibly defining entirely new routes, though that may be better handled by a separate hook
datasette 107914493 issue  
459714943 MDU6SXNzdWU0NTk3MTQ5NDM= 525 Add section on sqite-utils enable-fts to the search documentation simonw 9599 closed 0 simonw 9599   2 2019-06-24T06:39:16Z 2019-06-24T16:36:35Z 2019-06-24T16:29:43Z OWNER  

https://datasette.readthedocs.io/en/stable/full_text_search.html already has a section about csvs-to-sqlite, sqlite-utils is even more relevant.

datasette 107914493 issue  
459587155 MDExOlB1bGxSZXF1ZXN0MjkwODk3MTA0 518 Port Datasette from Sanic to ASGI + Uvicorn simonw 9599 closed 0 simonw 9599 Datasette 1.0 3268330 12 2019-06-23T15:18:42Z 2019-06-24T13:42:50Z 2019-06-24T03:13:09Z OWNER simonw/datasette/pulls/518

Most of the code here was fleshed out in comments on #272 (Port Datasette to ASGI) - this pull request will track the final pieces:

  • Update test harness to more correctly simulate the raw_path issue
  • Use raw_path so table names containing / can work correctly
  • Bug: JSON not served with correct content-type
  • Get ?_trace=1 working again
  • Replacement for @app.listener("before_server_start")
  • Bug: /fixtures/table%2Fwith%2Fslashes.csv?_format=json downloads as CSV
  • Replace Sanic request and response objects with my own classes, so I can remove Sanic dependency
  • Final code tidy-up before merging to master
datasette 107914493 pull  
456568880 MDU6SXNzdWU0NTY1Njg4ODA= 509 Support opening multiple databases with the same stem simonw 9599 open 0 simonw 9599   1 2019-06-15T19:32:00Z 2019-06-24T06:56:57Z   OWNER  

e.g. I should be able to do this:

datasette App/data.db Other_App/data.db

This currently errors because you can't have two databases taking the /data URL path.

Instead, how about in this particular case assigning the second database /data-1?

datasette 107914493 issue  
447469253 MDU6SXNzdWU0NDc0NjkyNTM= 485 Improvements to table label detection simonw 9599 open 0 simonw 9599   8 2019-05-23T06:19:49Z 2019-06-24T06:56:56Z   OWNER  

Label detection doesn't work if the primary key is called pk rather than id, so this page doesn't work: https://latest.datasette.io/fixtures/roadside_attraction_characteristics

Code is here:

https://github.com/simonw/datasette/blob/cccea85be6aaaeadb31f3b588ec7f732628815f5/datasette/app.py#L644-L653

datasette 107914493 issue  
324188953 MDU6SXNzdWUzMjQxODg5NTM= 272 Port Datasette to ASGI simonw 9599 closed 0 simonw 9599 Datasette 1.0 3268330 42 2018-05-17T21:16:32Z 2019-06-24T04:54:15Z 2019-06-24T03:33:06Z OWNER  

Datasette doesn't take much advantage of Sanic, and I'm increasingly having to work around parts of it because of idiosyncrasies that are specific to Datasette - caring about the exact order of querystring arguments for example.

Since Datasette is GET-only our needs from a web framework are actually pretty slim.

This becomes more important as I expand the plugins #14 framework. Am I sure I want the plugin ecosystem to depend on a Sanic if I might move away from it in the future?

If Datasette wasn't all about async/await I would use WSGI, but today it makes more sense to use ASGI. I'd like to be confident that switching to ASGI would still give me the excellent performance that Sanic provides.

https://github.com/django/asgiref/blob/master/specs/asgi.rst

datasette 107914493 issue  
309471814 MDU6SXNzdWUzMDk0NzE4MTQ= 189 Ability to sort (and paginate) by column simonw 9599 closed 0 simonw 9599   31 2018-03-28T18:04:51Z 2018-04-15T18:54:22Z 2018-04-09T05:16:02Z OWNER  

As requested in https://github.com/simonw/datasette/issues/185#issuecomment-376614973

I've previously avoided this for performance reasons: sort-by-column on a column without an index is likely to perform badly for hundreds of thousands of rows.

That's not a good enough reason to avoid the feature entirely though. A few options:

  • Allow sort-by-column by default, give users the option to disable it for specific tables/columns
  • Disallow sort-by-column by default, give users option (probably in metadata.json) to enable it for specific tables/columns
  • Automatically detect if a column either has an index on it OR a table has less than X rows in it

We already have the mechanism in place to cut off SQL queries that take more than X seconds, so if someone DOES try to sort by a column that's too expensive it won't actually hurt anything - but it would be nice to not show people a "sort" option which is guaranteed to throw a timeout error.

The vast majority of datasette usage that I've seen so far is on smaller datasets where the performance penalties of sort-by-column are extremely unlikely to show up.


Still left to do:

  • UI that shows which sort order is currently being applied (in HTML and in JSON)
  • UI for applying a sort order (with rel=nofollow to avoid Google crawling it)
  • Sort column names should be escaped correctly in generated SQL
  • Validation that the selected sort order is a valid column
  • Throw error if user attempts to apply _sort AND _sort_desc at the same time
  • Ability to disable sorting (or sort only for specific columns) in metadata.json
  • Fix "201 rows where sorted by sortable_with_nulls " bug
datasette 107914493 issue  
267759136 MDU6SXNzdWUyNjc3NTkxMzY= 20 Config file with support for defining canned queries simonw 9599 closed 0 simonw 9599 Custom templates edition 2949431 9 2017-10-23T17:53:06Z 2017-12-05T19:05:35Z 2017-12-05T17:44:09Z OWNER  

Probably using YAML because then we get support for multiline strings:

bats:
  db: bats.sqlite3
  name: "Bat sightings"
  queries:
    specific_row: |
      select * from Bats
      where a = 1;
datasette 107914493 issue  

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issues] (
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [number] INTEGER,
   [title] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [state] TEXT,
   [locked] INTEGER,
   [assignee] INTEGER REFERENCES [users]([id]),
   [milestone] INTEGER REFERENCES [milestones]([id]),
   [comments] INTEGER,
   [created_at] TEXT,
   [updated_at] TEXT,
   [closed_at] TEXT,
   [author_association] TEXT,
   [pull_request] TEXT,
   [body] TEXT,
   [repo] INTEGER REFERENCES [repos]([id]),
   [type] TEXT
, [active_lock_reason] TEXT);
CREATE INDEX [idx_issues_repo]
                ON [issues] ([repo]);
CREATE INDEX [idx_issues_milestone]
                ON [issues] ([milestone]);
CREATE INDEX [idx_issues_assignee]
                ON [issues] ([assignee]);
CREATE INDEX [idx_issues_user]
                ON [issues] ([user]);
Powered by Datasette · Query took 43.236ms · About: github-to-sqlite