home / github

Menu
  • Search all tables
  • GraphQL API

issue_comments

Table actions
  • GraphQL API for issue_comments

52 rows where "updated_at" is on date 2020-10-15 sorted by updated_at descending

✖
✖

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: issue_url, reactions, created_at (date), updated_at (date)

issue 12

  • ?sort=colname~numeric to sort by by column cast to real 19
  • Figure out how to run an environment that exercises the base_url proxy setting 9
  • Mechanism for plugins to add action menu items for various things 4
  • datasette.urls.table() / .instance() / .database() methods for constructing URLs, also exposed to templates 4
  • base_url doesn't seem to work when adding criteria and clicking "apply" 3
  • Fix last remaining links to "/" that do not respect base_url 3
  • New pattern for views that return either JSON or HTML, available for plugins 2
  • Some links don't honor base_url 2
  • Mechanism for plugins to construct URLs that respect base_url 2
  • Add documentation on serving Datasette behind a proxy using base_url 2
  • Incorrect URLs when served behind a proxy with base_url set 1
  • How should datasette.client interact with base_url 1

user 1

  • simonw 52

author_association 1

  • OWNER 52
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions issue performed_via_github_app
709647525 https://github.com/simonw/datasette/issues/1027#issuecomment-709647525 https://api.github.com/repos/simonw/datasette/issues/1027 MDEyOklzc3VlQ29tbWVudDcwOTY0NzUyNQ== simonw 9599 2020-10-15T23:49:51Z 2020-10-15T23:51:39Z OWNER

I'll install Apache on macOS to figure this out using https://formulae.brew.sh/formula/httpd

brew install httpd output this at the end:

``` ==> httpd DocumentRoot is /usr/local/var/www.

The default ports have been set in /usr/local/etc/httpd/httpd.conf to 8080 and in /usr/local/etc/httpd/extra/httpd-ssl.conf to 8443 so that httpd can run without sudo.

To have launchd start httpd now and restart at login: brew services start httpd Or, if you don't want/need a background service you can just run: apachectl start ```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Add documentation on serving Datasette behind a proxy using base_url 722758132  
709646865 https://github.com/simonw/datasette/issues/1027#issuecomment-709646865 https://api.github.com/repos/simonw/datasette/issues/1027 MDEyOklzc3VlQ29tbWVudDcwOTY0Njg2NQ== simonw 9599 2020-10-15T23:47:08Z 2020-10-15T23:47:08Z OWNER

It should cover both nginx and Apache. nginx config is here: https://github.com/simonw/datasette/issues/1024#issuecomment-709598324

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Add documentation on serving Datasette behind a proxy using base_url 722758132  
709636372 https://github.com/simonw/datasette/issues/1026#issuecomment-709636372 https://api.github.com/repos/simonw/datasette/issues/1026 MDEyOklzc3VlQ29tbWVudDcwOTYzNjM3Mg== simonw 9599 2020-10-15T23:09:34Z 2020-10-15T23:09:34Z OWNER

I'm inclined to say that internal requests should ignore base_url - since that seems like the right thing for plugins that need to access default Datasette APIs.

The one catch here is plugins that might want to proxy the current incoming URL for some reason - where that incoming request.path could include the base_url.

Actually those should be fine - because it will have been stripped off earlier:

https://github.com/simonw/datasette/blob/4f7c0ebd85ccd8c1853d7aa0147628f7c1b749cc/datasette/app.py#L963-L968

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
How should datasette.client interact with base_url 722738988  
709635276 https://github.com/simonw/datasette/issues/904#issuecomment-709635276 https://api.github.com/repos/simonw/datasette/issues/904 MDEyOklzc3VlQ29tbWVudDcwOTYzNTI3Ng== simonw 9599 2020-10-15T23:05:54Z 2020-10-15T23:05:54Z OWNER

Could have instance_url() take an optional path argument which is then turned into the correct path.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
datasette.urls.table() / .instance() / .database() methods for constructing URLs, also exposed to templates 663228985  
709635021 https://github.com/simonw/datasette/issues/904#issuecomment-709635021 https://api.github.com/repos/simonw/datasette/issues/904 MDEyOklzc3VlQ29tbWVudDcwOTYzNTAyMQ== simonw 9599 2020-10-15T23:05:11Z 2020-10-15T23:05:11Z OWNER

I think this should be a family of functions:

  • instance_url() - the root URL of the instance (usually / unless base_url is set)
  • database_url(database_name) - already got this
  • table_url(database_name, table_name)
  • row_url(database_name, table_name, row) - not sure about this one. The idea would be for row to be correctly turned into a URL by introspecting the primary keys for that table, then pulling those values out of the SQLite row object. Might not be necessary though.

I also need a way for plugins to link to e.g. /-/configure-fts - or even /-/configure-fts/database-name/table-name. What should that look like?

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
datasette.urls.table() / .instance() / .database() methods for constructing URLs, also exposed to templates 663228985  
709634261 https://github.com/simonw/datasette/issues/904#issuecomment-709634261 https://api.github.com/repos/simonw/datasette/issues/904 MDEyOklzc3VlQ29tbWVudDcwOTYzNDI2MQ== simonw 9599 2020-10-15T23:02:43Z 2020-10-15T23:02:43Z OWNER

Here's the current implementation of database_url - on the BaseView class, but only because it needs access to a datasette instance (to read base_url): https://github.com/simonw/datasette/blob/8f97b9b58e77f82fef1f10e9c9f6754b993544b6/datasette/views/base.py#L102-L108

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
datasette.urls.table() / .instance() / .database() methods for constructing URLs, also exposed to templates 663228985  
709633823 https://github.com/simonw/datasette/issues/904#issuecomment-709633823 https://api.github.com/repos/simonw/datasette/issues/904 MDEyOklzc3VlQ29tbWVudDcwOTYzMzgyMw== simonw 9599 2020-10-15T23:01:13Z 2020-10-15T23:01:13Z OWNER

Tracking ticket: #1023

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
datasette.urls.table() / .instance() / .database() methods for constructing URLs, also exposed to templates 663228985  
709633762 https://github.com/simonw/datasette/issues/988#issuecomment-709633762 https://api.github.com/repos/simonw/datasette/issues/988 MDEyOklzc3VlQ29tbWVudDcwOTYzMzc2Mg== simonw 9599 2020-10-15T23:01:01Z 2020-10-15T23:01:01Z OWNER

This is a dupe of https://github.com/simonw/datasette/issues/904

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Mechanism for plugins to construct URLs that respect base_url 713209404  
709633080 https://github.com/simonw/datasette/issues/865#issuecomment-709633080 https://api.github.com/repos/simonw/datasette/issues/865 MDEyOklzc3VlQ29tbWVudDcwOTYzMzA4MA== simonw 9599 2020-10-15T22:58:51Z 2020-10-15T22:58:51Z OWNER

It looks like there are places where Datasette might return a redirect that doesn't take base_url into account - I'm planning on fixing those here, after which I think ProxyPassReverse should no longer be necessary. https://github.com/simonw/datasette/issues/1025#issuecomment-709632136

{
    "total_count": 1,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 1,
    "rocket": 0,
    "eyes": 0
}
base_url doesn't seem to work when adding criteria and clicking "apply" 644582921  
709632765 https://github.com/simonw/datasette/issues/900#issuecomment-709632765 https://api.github.com/repos/simonw/datasette/issues/900 MDEyOklzc3VlQ29tbWVudDcwOTYzMjc2NQ== simonw 9599 2020-10-15T22:57:55Z 2020-10-15T22:57:55Z OWNER

I believe this particular bug has been fixed, based on my testing here: https://github.com/simonw/datasette/issues/1024#issuecomment-709622973

Please re-open the ticket if you are still experiencing it.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Some links don't honor base_url 661605489  
709632314 https://github.com/simonw/datasette/issues/1025#issuecomment-709632314 https://api.github.com/repos/simonw/datasette/issues/1025 MDEyOklzc3VlQ29tbWVudDcwOTYzMjMxNA== simonw 9599 2020-10-15T22:56:25Z 2020-10-15T22:56:34Z OWNER

That utils/asgi.py line is the default path for setting cookies. That should likely take base_url into account too: https://github.com/simonw/datasette/blob/4f7c0ebd85ccd8c1853d7aa0147628f7c1b749cc/datasette/utils/asgi.py#L331-L342

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Fix last remaining links to "/" that do not respect base_url 722724086  
709632136 https://github.com/simonw/datasette/issues/1025#issuecomment-709632136 https://api.github.com/repos/simonw/datasette/issues/1025 MDEyOklzc3VlQ29tbWVudDcwOTYzMjEzNg== simonw 9599 2020-10-15T22:55:44Z 2020-10-15T22:55:44Z OWNER

It looks like there are also some generated redirect responses that don't take base_url into account: datasette % git grep '"/' -- '*.py' ':(exclude)*test_*.py' ':(exclude)datasette/app.py' datasette/_version.py: for i in cfg.versionfile_source.split("/"): datasette/utils/asgi.py: path="/", datasette/views/base.py: should_redirect = "/{}-{}".format(name, expected) datasette/views/base.py: should_redirect += "/" + urllib.parse.quote_plus(kwargs["table"]) datasette/views/base.py: should_redirect += "/" + kwargs["pk_path"] datasette/views/special.py: response = Response.redirect("/") datasette/views/special.py: return Response.redirect("/") datasette/views/special.py: response = Response.redirect("/") datasette/views/special.py: return Response.redirect("/")

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Fix last remaining links to "/" that do not respect base_url 722724086  
709629920 https://github.com/simonw/datasette/issues/1025#issuecomment-709629920 https://api.github.com/repos/simonw/datasette/issues/1025 MDEyOklzc3VlQ29tbWVudDcwOTYyOTkyMA== simonw 9599 2020-10-15T22:48:20Z 2020-10-15T22:48:20Z OWNER

Also these: datasette % git grep '"/' -- '*.html' ':(exclude)*/patterns.html' datasette/templates/allow_debug.html:<form action="/-/allow-debug" method="get"> datasette/templates/base.html: <form action="/-/logout" method="post"> datasette/templates/error.html: <a href="/">home</a> datasette/templates/logout.html:<form action="/-/logout" method="post"> datasette/templates/messages_debug.html:<form action="/-/messages" method="post"> datasette/templates/query.html: <a href="/">home</a> /

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Fix last remaining links to "/" that do not respect base_url 722724086  
709626786 https://github.com/simonw/datasette/issues/865#issuecomment-709626786 https://api.github.com/repos/simonw/datasette/issues/865 MDEyOklzc3VlQ29tbWVudDcwOTYyNjc4Ng== simonw 9599 2020-10-15T22:38:38Z 2020-10-15T22:38:38Z OWNER

I managed to recreate proxying using nginx in #1024 - but I could not replicate this bug. I did NOT use ProxyPassReverse though. I think that may be what caused the problem.

I'll add a section to the documentation about this shortly.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
base_url doesn't seem to work when adding criteria and clicking "apply" 644582921  
709625063 https://github.com/simonw/datasette/issues/1024#issuecomment-709625063 https://api.github.com/repos/simonw/datasette/issues/1024 MDEyOklzc3VlQ29tbWVudDcwOTYyNTA2Mw== simonw 9599 2020-10-15T22:33:22Z 2020-10-15T22:33:22Z OWNER

Of those errors...

http://localhost:8000/robots.txt 404 is fine.

http://localhost:8000/datasette/%5C%22https://www.openstreetmap.org/copyright%5C%22 looks to me like a wget parsing bug where it got confused by this JavaScript:

window.DATASETTE_CLUSTER_MAP_TILE_LAYER_OPTIONS = {"maxZoom": 19, "detectRetina": true, "attribution": "© <a href=\"https://www.openstreetmap.org/copyright\">OpenStreetMap</a> contributors"};</script>

http://localhost:8000/-/static-plugins/datasette_cluster_map/datasette-cluster-map.js is a real bug. It's a bug in datasette-cluster-map but also requires me to solve #988 - mechanism for plugins to construct URLs that obey base_url.

I'm not sure why I'm getting a hit to http://localhost:8000/ since I wouldn't expect to link to / anywhere.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Figure out how to run an environment that exercises the base_url proxy setting 722674708  
709622973 https://github.com/simonw/datasette/issues/1024#issuecomment-709622973 https://api.github.com/repos/simonw/datasette/issues/1024 MDEyOklzc3VlQ29tbWVudDcwOTYyMjk3Mw== simonw 9599 2020-10-15T22:27:31Z 2020-10-15T22:27:31Z OWNER

Here's how I tested it: time wget -r 'http://localhost:8000/datasette/' 2>&1 | grep -i -C 5 "failed\|error" > /tmp/errors.txt This wrote out any errors (plus context) to the errors.txt log - and reported that the full crawl took 33s.

Here's what I got in errors.txt: ```

 0K .                                                      71.6M=0s

2020-10-15 15:23:09 (71.6 MB/s) - ‘localhost:8000/datasette/index.html’ saved [1276]

Loading robots.txt; please ignore errors. --2020-10-15 15:23:09-- http://localhost:8000/robots.txt Reusing existing connection to localhost:8000. HTTP request sent, awaiting response... 404 Not Found -- --2020-10-15 15:23:09-- http://localhost:8000/robots.txt Reusing existing connection to localhost:8000. HTTP request sent, awaiting response... 404 Not Found 2020-10-15 15:23:09 ERROR 404: Not Found.

--2020-10-15 15:23:09-- http://localhost:8000/datasette/-/static/app.css?b576be Reusing existing connection to localhost:8000. HTTP request sent, awaiting response... 200 OK Length: 8563 (8.4K) [text/css] -- -- 2020-10-15 15:23:13 (7.90 MB/s) - ‘localhost:8000/datasette/fixtures/primary_key_multiple_columns_explicit_label.json?_shape=object’ saved [58]

--2020-10-15 15:23:13-- http://localhost:8000/-/static-plugins/datasette_cluster_map/datasette-cluster-map.js Reusing existing connection to localhost:8000. HTTP request sent, awaiting response... 404 Not Found 2020-10-15 15:23:13 ERROR 404: Not Found.

--2020-10-15 15:23:13-- http://localhost:8000/datasette/fixtures?sql=select+pk%2C+name%2C+address%2C+latitude%2C+longitude+from+roadside_attractions+order+by+pk+limit+101 Reusing existing connection to localhost:8000. HTTP request sent, awaiting response... 200 OK Length: unspecified [text/html] -- -- 2020-10-15 15:23:13 (84.3 MB/s) - ‘localhost:8000/datasette/fixtures/roadside_attractions.json?_shape=object’ saved [619]

--2020-10-15 15:23:13-- http://localhost:8000/datasette/fixtures/%5C%22https://www.openstreetmap.org/copyright%5C%22 Reusing existing connection to localhost:8000. HTTP request sent, awaiting response... 404 Not Found 2020-10-15 15:23:13 ERROR 404: Not Found.

--2020-10-15 15:23:13-- http://localhost:8000/datasette/fixtures?sql=select+pk%2C+text1%2C+text2%2C+%5Bname+with+.+and+spaces%5D+from+searchable+order+by+pk+limit+101 Reusing existing connection to localhost:8000. HTTP request sent, awaiting response... 200 OK Length: unspecified [text/html] -- -- 2020-10-15 15:23:14 (28.6 MB/s) - ‘localhost:8000/datasette/fixtures/searchable_view_configured_by_metadata.json?_shape=array&_nl=on’ saved [180]

--2020-10-15 15:23:14-- http://localhost:8000/ Reusing existing connection to localhost:8000. HTTP request sent, awaiting response... 404 Not Found 2020-10-15 15:23:14 ERROR 404: Not Found.

--2020-10-15 15:23:14-- http://localhost:8000/datasette/fixtures?sql=select+pk1%2C+pk2%2C+pk3%2C+content+from+compound_three_primary_keys+order+by+pk1%2C+pk2%2C+pk3+limit+101&_hide_sql=1 Reusing existing connection to localhost:8000. HTTP request sent, awaiting response... 200 OK Length: unspecified [text/html] -- -- 2020-10-15 15:23:21 (64.1 MB/s) - ‘localhost:8000/datasette/fixtures.csv?sql=select+pk,+name,+address,+latitude,+longitude+from+roadside_attractions+order+by+pk+limit+101&_size=max’ saved [403]

--2020-10-15 15:23:21-- http://localhost:8000/datasette/%5C%22https://www.openstreetmap.org/copyright%5C%22 Reusing existing connection to localhost:8000. HTTP request sent, awaiting response... 404 Not Found 2020-10-15 15:23:21 ERROR 404: Not Found.

--2020-10-15 15:23:21-- http://localhost:8000/datasette/fixtures?sql=select+pk%2C+name%2C+address%2C+latitude%2C+longitude+from+roadside_attractions+order+by+pk+desc+limit+101 Reusing existing connection to localhost:8000. HTTP request sent, awaiting response... 200 OK Length: unspecified [text/html] ```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Figure out how to run an environment that exercises the base_url proxy setting 722674708  
709600335 https://github.com/simonw/datasette/issues/1024#issuecomment-709600335 https://api.github.com/repos/simonw/datasette/issues/1024 MDEyOklzc3VlQ29tbWVudDcwOTYwMDMzNQ== simonw 9599 2020-10-15T21:28:02Z 2020-10-15T22:25:43Z OWNER

This is working OK so far:

I'll try crawling it with wget -r to see if I get any errors.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Figure out how to run an environment that exercises the base_url proxy setting 722674708  
709598324 https://github.com/simonw/datasette/issues/1024#issuecomment-709598324 https://api.github.com/repos/simonw/datasette/issues/1024 MDEyOklzc3VlQ29tbWVudDcwOTU5ODMyNA== simonw 9599 2020-10-15T21:23:33Z 2020-10-15T21:26:55Z OWNER

Combining these two examples, here's the config file I am going to use for this. I'll save this as nginx.conf: ``` daemon off;

events { worker_connections 1024; }

http { server { listen 8000;

location /datasette {
    proxy_pass              http://127.0.0.1:8001;
    proxy_set_header        X-Real-IP $remote_addr;
    proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
}

} } Then start the server with: nginx -p pwd -c pwd/nginx.conf And start Datasette like this: datasette fixtures.db --config base_url:/datasette/ ```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Figure out how to run an environment that exercises the base_url proxy setting 722674708  
709597589 https://github.com/simonw/datasette/issues/1024#issuecomment-709597589 https://api.github.com/repos/simonw/datasette/issues/1024 MDEyOklzc3VlQ29tbWVudDcwOTU5NzU4OQ== simonw 9599 2020-10-15T21:21:53Z 2020-10-15T21:23:25Z OWNER

Here's a recipe for running nginx against a custom config file: https://gist.github.com/simonw/35f0ebf9c1d6df158759

``` daemon off;

events { worker_connections 1024; }

http { access_log /dev/stdout; error_log /dev/stderr;

types { text/html html htm shtml; text/css css; image/gif gif; image/jpeg jpeg jpg; application/javascript js; }

server { listen 8002; index index.html; root app; } } nginx -p pwd -c pwd/nginx.conf ```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Figure out how to run an environment that exercises the base_url proxy setting 722674708  
709595960 https://github.com/simonw/datasette/issues/1024#issuecomment-709595960 https://api.github.com/repos/simonw/datasette/issues/1024 MDEyOklzc3VlQ29tbWVudDcwOTU5NTk2MA== simonw 9599 2020-10-15T21:18:14Z 2020-10-15T21:18:14Z OWNER

Typing nginx starts it running as a daemon listening on port http-alt aka 8080. It uses the config file from /usr/local/etc/nginx/nginx.conf.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Figure out how to run an environment that exercises the base_url proxy setting 722674708  
709590941 https://github.com/simonw/datasette/issues/1024#issuecomment-709590941 https://api.github.com/repos/simonw/datasette/issues/1024 MDEyOklzc3VlQ29tbWVudDcwOTU5MDk0MQ== simonw 9599 2020-10-15T21:07:47Z 2020-10-15T21:07:47Z OWNER

On macOS I ran brew install nginx. I'm going to try running it on port 8000 so I don't have to run it as root.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Figure out how to run an environment that exercises the base_url proxy setting 722674708  
709590337 https://github.com/simonw/datasette/issues/1024#issuecomment-709590337 https://api.github.com/repos/simonw/datasette/issues/1024 MDEyOklzc3VlQ29tbWVudDcwOTU5MDMzNw== simonw 9599 2020-10-15T21:06:24Z 2020-10-15T21:07:19Z OWNER

From https://stackoverflow.com/questions/32549684/nginx-proxy-and-remove-proxy-pass-prefix/32550251 it looks like the config I should use is: ``` server { listen 80; server_name example.com;

location /datasette/ {
    proxy_pass              http://127.0.0.1:8001;
    proxy_set_header        X-Real-IP $remote_addr;
    proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header        X-Forwarded-Proto $scheme;
    proxy_read_timeout      90;
}

} ```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Figure out how to run an environment that exercises the base_url proxy setting 722674708  
709589297 https://github.com/simonw/datasette/issues/1024#issuecomment-709589297 https://api.github.com/repos/simonw/datasette/issues/1024 MDEyOklzc3VlQ29tbWVudDcwOTU4OTI5Nw== simonw 9599 2020-10-15T21:04:31Z 2020-10-15T21:04:31Z OWNER

I think nginx or Apache would be the best tools for this. I'm inclined to try with nginx first since I know it better.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Figure out how to run an environment that exercises the base_url proxy setting 722674708  
709588425 https://github.com/simonw/datasette/issues/838#issuecomment-709588425 https://api.github.com/repos/simonw/datasette/issues/838 MDEyOklzc3VlQ29tbWVudDcwOTU4ODQyNQ== simonw 9599 2020-10-15T21:02:37Z 2020-10-15T21:02:37Z OWNER

Tracking ticket: #1023

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Incorrect URLs when served behind a proxy with base_url set 637395097  
709588373 https://github.com/simonw/datasette/issues/865#issuecomment-709588373 https://api.github.com/repos/simonw/datasette/issues/865 MDEyOklzc3VlQ29tbWVudDcwOTU4ODM3Mw== simonw 9599 2020-10-15T21:02:31Z 2020-10-15T21:02:31Z OWNER

Tracking ticket: #1023

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
base_url doesn't seem to work when adding criteria and clicking "apply" 644582921  
709588322 https://github.com/simonw/datasette/issues/900#issuecomment-709588322 https://api.github.com/repos/simonw/datasette/issues/900 MDEyOklzc3VlQ29tbWVudDcwOTU4ODMyMg== simonw 9599 2020-10-15T21:02:26Z 2020-10-15T21:02:26Z OWNER

Tracking ticket: #1023

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Some links don't honor base_url 661605489  
709588290 https://github.com/simonw/datasette/issues/988#issuecomment-709588290 https://api.github.com/repos/simonw/datasette/issues/988 MDEyOklzc3VlQ29tbWVudDcwOTU4ODI5MA== simonw 9599 2020-10-15T21:02:21Z 2020-10-15T21:02:21Z OWNER

Tracking ticket: #1023

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Mechanism for plugins to construct URLs that respect base_url 713209404  
709575818 https://github.com/simonw/datasette/issues/894#issuecomment-709575818 https://api.github.com/repos/simonw/datasette/issues/894 MDEyOklzc3VlQ29tbWVudDcwOTU3NTgxOA== simonw 9599 2020-10-15T20:35:03Z 2020-10-15T20:35:03Z OWNER

Prototype so far: ```diff diff --git a/datasette/views/table.py b/datasette/views/table.py index ea11a51..d61f8bd 100644 --- a/datasette/views/table.py +++ b/datasette/views/table.py @@ -497,17 +497,32 @@ class TableView(RowTableShared): if sort and sort_desc: raise DatasetteError("Cannot use _sort and _sort_desc at the same time")

  • def parse_sort(sort):
  • if "~" in sort:
  • if sort.endswith("~default"):
  • col = sort.rsplit("~", 1)[0]
  • return col, escape_sqlite(col)
  • elif sort.endswith("~numeric"):
  • col = sort.rsplit("~", 1)[0]
  • return col, "cast(nullif({}, '') as real)".format(escape_sqlite(col))
  • else:
  • return sort, escape_sqlite(sort)
  • else:
  • return sort, escape_sqlite(sort) + if sort:
  • if sort not in sortable_columns:
  • raise DatasetteError("Cannot sort table by {}".format(sort))
  • sort_column, sort_clause = parse_sort(sort)
  • if sort_column not in sortable_columns:
  • raise DatasetteError("Cannot sort table by {}".format(sort_column))

  • order_by = escape_sqlite(sort)

  • order_by = sort_clause

     if sort_desc:
    
    • if sort_desc not in sortable_columns:
    • raise DatasetteError("Cannot sort table by {}".format(sort_desc))
    • sort_column, sort_clause = parse_sort(sort_desc)
    • if sort_column not in sortable_columns:
    • raise DatasetteError("Cannot sort table by {}".format(sort_column))
  • order_by = "{} desc".format(escape_sqlite(sort_desc))

  • order_by = "{} desc".format(sort_clause)
     from_sql = "from {table_name} {where}".format(
         table_name=escape_sqlite(table),
    

    ```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
?sort=colname~numeric to sort by by column cast to real 657572753  
709572425 https://github.com/simonw/datasette/issues/894#issuecomment-709572425 https://api.github.com/repos/simonw/datasette/issues/894 MDEyOklzc3VlQ29tbWVudDcwOTU3MjQyNQ== simonw 9599 2020-10-15T20:28:18Z 2020-10-15T20:28:18Z OWNER

Also need to rethink this template logic that decides if to show a column as sorted or not: https://github.com/simonw/datasette/blob/4f7c0ebd85ccd8c1853d7aa0147628f7c1b749cc/datasette/templates/_table.html#L10-L14

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
?sort=colname~numeric to sort by by column cast to real 657572753  
709571143 https://github.com/simonw/datasette/issues/894#issuecomment-709571143 https://api.github.com/repos/simonw/datasette/issues/894 MDEyOklzc3VlQ29tbWVudDcwOTU3MTE0Mw== simonw 9599 2020-10-15T20:25:35Z 2020-10-15T20:25:35Z OWNER

cast(nullif(colname, '') as real) can fix this - it will treat '' the same as null.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
?sort=colname~numeric to sort by by column cast to real 657572753  
709569951 https://github.com/simonw/datasette/issues/894#issuecomment-709569951 https://api.github.com/repos/simonw/datasette/issues/894 MDEyOklzc3VlQ29tbWVudDcwOTU2OTk1MQ== simonw 9599 2020-10-15T20:23:02Z 2020-10-15T20:23:02Z OWNER

Something to watch out for: "" empty strings cast to 0.0:

select cast("100" as real), "100", cast(null as real), cast("" as real)

cast("100" as real) | "100" | cast(null as real) | cast("" as real) -- | -- | -- | -- 100.0 | 100 |   | 0.0

https://latest.datasette.io/fixtures?sql=select+cast%28%22100%22+as+real%29%2C+%22100%22%2C+cast%28null+as+real%29%2C+cast%28%22%22+as+real%29

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
?sort=colname~numeric to sort by by column cast to real 657572753  
709562940 https://github.com/simonw/datasette/issues/894#issuecomment-709562940 https://api.github.com/repos/simonw/datasette/issues/894 MDEyOklzc3VlQ29tbWVudDcwOTU2Mjk0MA== simonw 9599 2020-10-15T20:08:16Z 2020-10-15T20:08:16Z OWNER

Relevant code: https://github.com/simonw/datasette/blob/4f7c0ebd85ccd8c1853d7aa0147628f7c1b749cc/datasette/views/table.py#L485-L510

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
?sort=colname~numeric to sort by by column cast to real 657572753  
709546976 https://github.com/simonw/datasette/issues/894#issuecomment-709546976 https://api.github.com/repos/simonw/datasette/issues/894 MDEyOklzc3VlQ29tbWVudDcwOTU0Njk3Ng== simonw 9599 2020-10-15T19:35:55Z 2020-10-15T19:36:38Z OWNER

Much easier solution: if the suffix is ~numeric then treat it as the column name sorted numerically. If the suffix is missing OR the suffix is ~default, sort without casting. Only add the ~default suffix if the column name itself contains at least one ~ symbol.

Using ~ because it doesn't need to be URL-encoded.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
?sort=colname~numeric to sort by by column cast to real 657572753  
709539257 https://github.com/simonw/datasette/issues/894#issuecomment-709539257 https://api.github.com/repos/simonw/datasette/issues/894 MDEyOklzc3VlQ29tbWVudDcwOTUzOTI1Nw== simonw 9599 2020-10-15T19:19:29Z 2020-10-15T19:34:07Z OWNER

Urgh this isn't going to work. %7E~%7E gets decoded as ~~~ so I wouldn't be able to tell the difference.

I could use double-percentage-encoding here instead. I feel like there's a simpler solution that I'm missing (and that may well be in use within Datasette already, I'm not doing great thinking this morning).

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
?sort=colname~numeric to sort by by column cast to real 657572753  
709534197 https://github.com/simonw/datasette/issues/894#issuecomment-709534197 https://api.github.com/repos/simonw/datasette/issues/894 MDEyOklzc3VlQ29tbWVudDcwOTUzNDE5Nw== simonw 9599 2020-10-15T19:08:53Z 2020-10-15T19:17:55Z OWNER

Even better solution: use URL encoding in the parameter details. This is consistent with how ?_next= tokens work, e.g. ?_next=0.291861560261786%2Ce%2Cj.

So the format can be:

  • mycolumn
  • urlencoded-mycolumn$castname

For most columns this will look like: ?_sort=score$numeric

For columns with a $ in their name it will be ?_sort=score%24hasdollar$numeric

Problem: both $ and , are usually URL encoded anyway. I need a character which isn't encoded by default, so that I can use its encoded form to show it is part of the column name and its un-encoded form to split the cast indicator.

_ is a candidate here - not encoded by default, but can be encoded as %5F.

The other unreserved non-alphanumeric characters are -, ., _, ~.

Of these, ~ is least likely to show up in a column name. So I'll use that.

  • mycolumn
  • mycolumn~numeric
  • mycolumn%7Ewith%7Etildes~numeric
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
?sort=colname~numeric to sort by by column cast to real 657572753  
709532369 https://github.com/simonw/datasette/issues/894#issuecomment-709532369 https://api.github.com/repos/simonw/datasette/issues/894 MDEyOklzc3VlQ29tbWVudDcwOTUzMjM2OQ== simonw 9599 2020-10-15T19:05:07Z 2020-10-15T19:07:35Z OWNER

Simpler option: ?_sort= column values look like this:

  • mycolumn - for sort by column
  • mycolumn$numeric - for sort by column after cast to float
  • mycolumn$morename$default - for the edge case where the column name itself contains a $ symbol
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
?sort=colname~numeric to sort by by column cast to real 657572753  
709531343 https://github.com/simonw/datasette/issues/894#issuecomment-709531343 https://api.github.com/repos/simonw/datasette/issues/894 MDEyOklzc3VlQ29tbWVudDcwOTUzMTM0Mw== simonw 9599 2020-10-15T19:03:12Z 2020-10-15T19:03:12Z OWNER

The Sort by <select> menu needs a rethink: https://github.com/simonw/datasette/blob/4f7c0ebd85ccd8c1853d7aa0147628f7c1b749cc/datasette/templates/table.html#L80-L87

If it's going to include sort by numeric options it needs a different format - since ?_sort=colname needs to also support ?_sort=numeric_colname - but shouldn't clash with a weird column already named numeric_colname.

Can I come up with a value syntax for this that is guaranteed not to clash with a weirdly named existing column?

I think so. I could use {"type": "numeric", "column": "mycolumn"} as the value - then if a column has a name that is itself valid JSON (weird but possible) the _sort= value would be {"type": "default", "column": "{\"type\": \"numeric\", \"name\": \"column\"}"}.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
?sort=colname~numeric to sort by by column cast to real 657572753  
709525082 https://github.com/simonw/datasette/issues/894#issuecomment-709525082 https://api.github.com/repos/simonw/datasette/issues/894 MDEyOklzc3VlQ29tbWVudDcwOTUyNTA4Mg== simonw 9599 2020-10-15T18:51:59Z 2020-10-15T18:51:59Z OWNER

This is enough of a design to build a working prototype.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
?sort=colname~numeric to sort by by column cast to real 657572753  
709505147 https://github.com/simonw/datasette/issues/894#issuecomment-709505147 https://api.github.com/repos/simonw/datasette/issues/894 MDEyOklzc3VlQ29tbWVudDcwOTUwNTE0Nw== simonw 9599 2020-10-15T18:18:45Z 2020-10-15T18:50:50Z OWNER

This needs querystring parameter design. Some options:

  • ?_sort_clause=cast(mycol+as+integer) and ?_sort_clause_desc=cast(mycol+as+integer) - allowing any expression. This would need to be disabled if arbitrary SQL was turned off, similar to the restrictions on ?_where=.
  • ?_sort_numeric=mycol and ?_sort_numeric_desc=mycol - this would cast to real which would work on integer values as well. It could be allowed even when arbitrary SQL was disabled.
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
?sort=colname~numeric to sort by by column cast to real 657572753  
709524123 https://github.com/simonw/datasette/issues/894#issuecomment-709524123 https://api.github.com/repos/simonw/datasette/issues/894 MDEyOklzc3VlQ29tbWVudDcwOTUyNDEyMw== simonw 9599 2020-10-15T18:50:25Z 2020-10-15T18:50:25Z OWNER

For the "Sort by X" select menu case... I could automatically expand that menu to contain extra options for "Sort numerically by X" for each TEXT column in the table. That's a pretty good option.

For the action cog menu, I can add the extra options to the cog menu - and rely on the fact that the title of the page will say "Sorted numerically by colname descending".

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
?sort=colname~numeric to sort by by column cast to real 657572753  
709513483 https://github.com/simonw/datasette/issues/894#issuecomment-709513483 https://api.github.com/repos/simonw/datasette/issues/894 MDEyOklzc3VlQ29tbWVudDcwOTUxMzQ4Mw== simonw 9599 2020-10-15T18:31:56Z 2020-10-15T18:31:56Z OWNER

I think the first version of this feature involves implementing ?_sort_numeric=col and ?_sort_numeric_desc=col plus the JavaScript to detect if those values should be shown in the column actions menu.

One question: how to reflect that this is happening in the current sort UI. This menu here for example:

And this interface: how should it indicate that a text is currently sorted numerically v.s. sorted alphabetically, and allow the user to switch from one to the other?

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
?sort=colname~numeric to sort by by column cast to real 657572753  
709511399 https://github.com/simonw/datasette/issues/894#issuecomment-709511399 https://api.github.com/repos/simonw/datasette/issues/894 MDEyOklzc3VlQ29tbWVudDcwOTUxMTM5OQ== simonw 9599 2020-10-15T18:28:09Z 2020-10-15T18:28:09Z OWNER

The simplest solution would be for Python code to scan all of the visible values on the current page and show the column action for "sort by this numeric" based purely on that. I already do that in the JavaScript for "are there any blank values in the first page?" here:

https://github.com/simonw/datasette/blob/4f7c0ebd85ccd8c1853d7aa0147628f7c1b749cc/datasette/static/table.js#L106-L118

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
?sort=colname~numeric to sort by by column cast to real 657572753  
709510422 https://github.com/simonw/datasette/issues/894#issuecomment-709510422 https://api.github.com/repos/simonw/datasette/issues/894 MDEyOklzc3VlQ29tbWVudDcwOTUxMDQyMg== simonw 9599 2020-10-15T18:26:25Z 2020-10-15T18:26:25Z OWNER

There's something interesting about figuring out which sort options should be offered in the column actions menu.

Two options:

  • Try to detect if a text column is all-integers or all-floats, either by scanning the entire table (if it's small enough) or by scanning the first X rows, where X might be the size of the first page or maybe the first 1,000 or similar.
  • Could also let users define this in metadata.yml for the table.
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
?sort=colname~numeric to sort by by column cast to real 657572753  
709509635 https://github.com/simonw/datasette/issues/894#issuecomment-709509635 https://api.github.com/repos/simonw/datasette/issues/894 MDEyOklzc3VlQ29tbWVudDcwOTUwOTYzNQ== simonw 9599 2020-10-15T18:24:57Z 2020-10-15T18:24:57Z OWNER

This does feel like a weird plugin hook just because there aren't really THAT many different use-cases that plugins could solve. The ones I can think of are:

  • Sort numeric
  • Sort by parsed date
  • Sort by FTS rank

Could this work if I just allow _sort_clause=?

One possible solution for the no-arbitrary-SQL case: users can define sort orders in metadata.json/yml. So if you want to enable sort-by-distance without enabling arbitrary SQL you could add something like this:

yaml databases: mydb: tables: museums: sort_clause: bm25(fts)

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
?sort=colname~numeric to sort by by column cast to real 657572753  
709505953 https://github.com/simonw/datasette/issues/894#issuecomment-709505953 https://api.github.com/repos/simonw/datasette/issues/894 MDEyOklzc3VlQ29tbWVudDcwOTUwNTk1Mw== simonw 9599 2020-10-15T18:20:13Z 2020-10-15T18:20:13Z OWNER

Sorting by date when the column has a junk date format in it is such a column need it should maybe ship in Datasette by default - though I've been trying to avoid adding heavy dependencies like dateutil if I can get away with it.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
?sort=colname~numeric to sort by by column cast to real 657572753  
709505608 https://github.com/simonw/datasette/issues/894#issuecomment-709505608 https://api.github.com/repos/simonw/datasette/issues/894 MDEyOklzc3VlQ29tbWVudDcwOTUwNTYwOA== simonw 9599 2020-10-15T18:19:35Z 2020-10-15T18:19:35Z OWNER

I could even let plugins define new sort types. Imagine a plugin that enables this:

?_sort_date_desc=mycol - where it knows how to handle specific date formats, or even uses dateutil.parser.parse.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
?sort=colname~numeric to sort by by column cast to real 657572753  
709503359 https://github.com/simonw/datasette/issues/878#issuecomment-709503359 https://api.github.com/repos/simonw/datasette/issues/878 MDEyOklzc3VlQ29tbWVudDcwOTUwMzM1OQ== simonw 9599 2020-10-15T18:15:28Z 2020-10-15T18:15:28Z OWNER

I think this is blocking #619

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
New pattern for views that return either JSON or HTML, available for plugins 648435885  
709502889 https://github.com/simonw/datasette/issues/878#issuecomment-709502889 https://api.github.com/repos/simonw/datasette/issues/878 MDEyOklzc3VlQ29tbWVudDcwOTUwMjg4OQ== simonw 9599 2020-10-15T18:14:34Z 2020-10-15T18:14:34Z OWNER

The BaseView class does this for Datasette internals at the moment, but I'm not convinced it works as well as it could.

I'd like to turn this into a class that is documented and available to plugins as well.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
New pattern for views that return either JSON or HTML, available for plugins 648435885  
709497595 https://github.com/simonw/datasette/issues/690#issuecomment-709497595 https://api.github.com/repos/simonw/datasette/issues/690 MDEyOklzc3VlQ29tbWVudDcwOTQ5NzU5NQ== simonw 9599 2020-10-15T18:04:35Z 2020-10-15T18:12:15Z OWNER

For the table actions: attaching it to a cog icon next to the table name could make sense.

This is the column action icon at twice the size, color #666.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Mechanism for plugins to add action menu items for various things 573755726  
709500715 https://github.com/simonw/datasette/issues/690#issuecomment-709500715 https://api.github.com/repos/simonw/datasette/issues/690 MDEyOklzc3VlQ29tbWVudDcwOTUwMDcxNQ== simonw 9599 2020-10-15T18:10:32Z 2020-10-15T18:10:32Z OWNER

Row action menus are a question mark for me. Adding them to the table page itself could get really noisy - though maybe they're a vertical ... menu at the end of the row? They could also go on the row.html template, though that page is rarely used at the moment.

I'm going to skip row actions for the moment and concentrate on the other four, which I know I have use-cases for.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Mechanism for plugins to add action menu items for various things 573755726  
709499944 https://github.com/simonw/datasette/issues/690#issuecomment-709499944 https://api.github.com/repos/simonw/datasette/issues/690 MDEyOklzc3VlQ29tbWVudDcwOTQ5OTk0NA== simonw 9599 2020-10-15T18:09:06Z 2020-10-15T18:09:06Z OWNER

I'm going to dedicate this issue to thinking about action menus. #981 added column action menus, others to build could be:

  • Table action menus (for things like configure FTS, edit schema)
  • Database action menus (import CSV file into this table)
  • Instance action menus (manage users, import CSV into this instance)
  • Column action menus (already there, plugins could include extract this column)
  • Row action menus?
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Mechanism for plugins to add action menu items for various things 573755726  
709498425 https://github.com/simonw/datasette/issues/690#issuecomment-709498425 https://api.github.com/repos/simonw/datasette/issues/690 MDEyOklzc3VlQ29tbWVudDcwOTQ5ODQyNQ== simonw 9599 2020-10-15T18:06:08Z 2020-10-15T18:06:08Z OWNER

And for instance-level actions (linking to datasette-import-csv for example) an actions menu anchored against a burger-bar menu icon in the navigation bar.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Mechanism for plugins to add action menu items for various things 573755726  

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
, [performed_via_github_app] TEXT);
CREATE INDEX [idx_issue_comments_issue]
                ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
                ON [issue_comments] ([user]);
Powered by Datasette · Queries took 510.294ms · About: github-to-sqlite
  • Sort ascending
  • Sort descending
  • Facet by this
  • Hide this column
  • Show all columns
  • Show not-blank rows