home / github

Menu
  • Search all tables
  • GraphQL API

issue_comments

Table actions
  • GraphQL API for issue_comments

24 rows where user = 193185 sorted by updated_at descending

✖
✖

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: issue_url, reactions, created_at (date), updated_at (date)

issue 10

  • Transformation type `--type DATETIME` 6
  • array facet: don't materialize unnecessary columns 4
  • Datasette is not compatible with SQLite's strict quoting compilation option 3
  • feat: Javascript Plugin API (Custom panels, column menu items with JS actions) 3
  • datasette publish lambda plugin 2
  • render_cell plugin hook's row object is not a sqlite.Row 2
  • Improve the display of facets information 1
  • Show foreign key label when filtering 1
  • rewrite_sql hook 1
  • Error: Invalid setting 'hash_urls' in settings.json in 0.64.1 1

author_association 2

  • CONTRIBUTOR 18
  • NONE 6

user 1

  • cldellow · 24 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions issue performed_via_github_app
1548617257 https://github.com/simonw/datasette/pull/2052#issuecomment-1548617257 https://api.github.com/repos/simonw/datasette/issues/2052 IC_kwDOBm6k_c5cTgYp cldellow 193185 2023-05-15T21:32:20Z 2023-05-15T21:32:20Z CONTRIBUTOR

Were you picturing that the whole plugin config object could be returned as a promise, or that the individual hooks (like makeColumnActions or makeAboveTablePanelConfigs supported returning a promise of arrays instead only returning plain arrays?

The latter - that you could return a promise of arrays, so it parallels the "await me maybe" pattern in Datasette, where you can return either a value, a callable or an awaitable.

I have a hunch that what you're describing might be achievable without adding Promises to the API with something

Oops, I did a poor job explaining. Yes, this would work - but it requires me to continue to communicate the column names out of band (in order to fetch the facet data per-column before registering my plugin), vs being able to re-use them from the plugin implementation.

This isn't that big of a deal - it'd be a nice ergonomic improvement, but nowhere near as a big of an improvement as having an officially sanctioned way to add stuff to the column menus in the first place.

This could also be layered on in a future commit without breaking v1 users, too, so it's not at all urgent.

especially if those lines are encapsulated by a function we provide (maybe something that's available on the window provided by Datasette as an inline script tag

Ah, this is maybe the the key point. Since it's all hosted inside Datasette, Datasette can provide some arbitrary sugar to make it easier to work with.

My experience with async scripts in JS is that people sometimes don't understand the race conditions inherent to them. If they copy/paste from a tutorial, it does just work. But then they'll delete half the code, and by chance it still works on their machine/Datasette templates, and now someone's headed for an annoying debugging session -- maybe them, maybe someone else who tries to re-use their plugin.

Again, a fairly minor thing, though.

{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
feat: Javascript Plugin API (Custom panels, column menu items with JS actions) 1651082214  
1530822437 https://github.com/simonw/datasette/pull/2052#issuecomment-1530822437 https://api.github.com/repos/simonw/datasette/issues/2052 IC_kwDOBm6k_c5bPn8l cldellow 193185 2023-05-02T03:35:30Z 2023-05-02T16:02:38Z CONTRIBUTOR

Also, just checking - is this how I'd write bulletproof plugin registration code that is robust against the order in which the script tags load (eg if both my code and the Datasette code are loaded via a <script async src='...'/> tag)?

```js if (window.DATASETTE) go(window.DATASETTE); else document.addEventListener("datasette_init", (evt) => go(evt.detail));

function go(manager) { manager.registerPlugin(...) } ```

I don't know if it'd make sense, but you could also consider the asynchronous queuing pattern that Google Analytics uses (see this Stack Overflow post for more details):

```js DATASETTE = DATASETTE || []; DATASETTE.push(go);

function go(manager) { manager.registerPlugin(...); } ```

{
    "total_count": 2,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 1
}
feat: Javascript Plugin API (Custom panels, column menu items with JS actions) 1651082214  
1530817667 https://github.com/simonw/datasette/pull/2052#issuecomment-1530817667 https://api.github.com/repos/simonw/datasette/issues/2052 IC_kwDOBm6k_c5bPmyD cldellow 193185 2023-05-02T03:24:53Z 2023-05-02T03:24:53Z CONTRIBUTOR

Thanks for putting this together! I've been slammed with work/personal stuff so haven't been able to actually prototype anything with this. :(

tl;dr: I think this would be useful immediately as is. It might also be nice if the plugins could return Promises.

The long version: I read the design notes and example plugin. I think I'd be able to use this in datasette-ui-extras for my lazy-facets feature.

The lazy-facets feature tries to provide a snappier user experience. It does this by altering how suggested facets work.

First, at page render time: (A) it lies to Datasette and claims that no columns support facets, this avoids the lengthy delays/timeouts that can happen if the dataset is large. (B) there's a python plugin that implements the extra_body_script hook, to write out the list of column names for future use by JavaScript

Second, at page load time: there is some JavaScript that: (C) makes AJAX requests to suggest facets for each column - it makes 1 request per column, using the data from (B) (D) wires up the column menus to add Facet-by-this options for each facet

With the currently proposed plugin scheme, I think (D) could be moved into the plugin. I'd do the ajax requests, then register the plugin.

If the plugin scheme also supported promises, I think (B) and (C) could also be moved into the plugin.

Does that make sense? Sorry for the wall of text!

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
feat: Javascript Plugin API (Custom panels, column menu items with JS actions) 1651082214  
1425974877 https://github.com/simonw/datasette/issues/2023#issuecomment-1425974877 https://api.github.com/repos/simonw/datasette/issues/2023 IC_kwDOBm6k_c5U_qZd cldellow 193185 2023-02-10T15:32:41Z 2023-02-10T15:32:41Z CONTRIBUTOR

I think this feature was removed in Datasette 0.61 and moved to a plugin. People who want hashed URLs can use the datasette-hashed-urls plugin to achieve the same affect.

It looks like you're trying to disable hashed urls, so I think you can just remove that config setting and things will work.

{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Error: Invalid setting 'hash_urls' in settings.json in 0.64.1 1579695809  
1421081939 https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1421081939 https://api.github.com/repos/simonw/sqlite-utils/issues/524 IC_kwDOCGYnMM5Us_1T cldellow 193185 2023-02-07T16:42:25Z 2023-02-07T16:43:42Z NONE

Ha, yes, I might end up making something very niche. That's OK.

I'm building a UI for Datasette that lets users make schema changes, so it's important to me that the tool work in a non-surprising way -- if you ask for a column of type X, you should get type X. If the column or table previously had CHECK constraints, they shouldn't be silently removed. And so on. I had hoped that I could just lean on sqlite-utils, but I think it's a little too surprising.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Transformation type `--type DATETIME` 1572766460  
1421033725 https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1421033725 https://api.github.com/repos/simonw/sqlite-utils/issues/524 IC_kwDOCGYnMM5Us0D9 cldellow 193185 2023-02-07T16:12:13Z 2023-02-07T16:12:13Z NONE

I think the bigger issue is that sqlite-utils mixes mechanism (it implements the 12-step way to alter SQLite tables) and policy (it has an opinionated stance on what column types should be used).

That might be a design choice to make it accessible to users by providing a reasonable set of defaults, but it doesn't quite fit my use case.

It might make sense to extract a separate library that provides just the mechanisms, and then sqlite-utils would sit on top of that library with its opinionated set of policies.

That would be a very big change, though.

I might take a stab at extracting the library, but just for the table schema migration piece, not all the other features that sqlite-utils supports. I wouldn't expect sqlite-utils to depend on it.

Part of my motivation is that I want to provide some other abilities, too, like support for CHECK constraints. I see that the issue in this repo (https://github.com/simonw/sqlite-utils/issues/358) proposes a bunch of short-hand constraints, which I wouldn't want to accidentally expose to people -- I want a layer that is a 1:1 mapping to SQLite.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Transformation type `--type DATETIME` 1572766460  
1420992261 https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1420992261 https://api.github.com/repos/simonw/sqlite-utils/issues/524 IC_kwDOCGYnMM5Usp8F cldellow 193185 2023-02-07T15:45:58Z 2023-02-07T15:45:58Z NONE

I'd support that, but I'm not the author of this library.

One challenge is that would be a breaking change. Do you see a way to enable it without affecting existing users or bumping the major version number?

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Transformation type `--type DATETIME` 1572766460  
1420809773 https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1420809773 https://api.github.com/repos/simonw/sqlite-utils/issues/524 IC_kwDOCGYnMM5Ur9Yt cldellow 193185 2023-02-07T13:53:01Z 2023-02-07T13:53:01Z NONE

Ah, it looks like that is controlled by this dict: https://github.com/simonw/sqlite-utils/blob/main/sqlite_utils/db.py#L178

I suspect you could overwrite the datetime entry to achieve what you want

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Transformation type `--type DATETIME` 1572766460  
1419734229 https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1419734229 https://api.github.com/repos/simonw/sqlite-utils/issues/524 IC_kwDOCGYnMM5Un2zV cldellow 193185 2023-02-06T20:53:28Z 2023-02-06T21:16:29Z NONE

I think it's not currently possible: sqlite-utils requires that it be one of integer, text, float, blob (see code)

IMO, this is a bit of friction and it would be nice if it was more permissive. SQLite permits developers to use any data type when creating a table. For example, this is a perfectly cromulent sqlite session that creates a table with columns of type baz and bar:

``` sqlite> create table foo(column1 baz, column2 bar); sqlite> .schema foo CREATE TABLE foo(column1 baz, column2 bar); sqlite> select * from pragma_table_info('foo'); cid name type notnull dflt_value pk


0 column1 baz 0 0
1 column2 bar 0 0
```

The idea is that the application developer will know what meaning to ascribe to those types. For example, I'm working on a plugin to Datasette. Dates are tricky to handle. If you have some existing rows, you can look at the values in them to know how a user is serializing the dates -- as an ISO 8601 string? An RFC 3339 string? With millisecond precision? With timezone offset? But if you don't yet have any rows, you have to guess. If the column is of type TEXT, you don't even know that it's meant to hold a date! In this case, my plugin will look to see if the column is of type DATE or DATETIME, and assume a certain representation when writing.

Perhaps there is an argument that sqlite-utils is trying to conform to SQLite's strict mode, and that is why it limits the choices. In strict mode, SQLite requires that the data type be one of INT, INTEGER, REAL, TEXT, BLOB, ANY. But that can't be the case -- sqlite-utils supports FLOAT, which is not one of the valid types in strict mode, and it rejects INT, REAL and ANY, which are valid.

{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Transformation type `--type DATETIME` 1572766460  
1419740776 https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1419740776 https://api.github.com/repos/simonw/sqlite-utils/issues/524 IC_kwDOCGYnMM5Un4Zo cldellow 193185 2023-02-06T20:59:01Z 2023-02-06T20:59:01Z NONE

That said, it looks like the check is only enforced at the CLI level. If you use the API directly, I think it'll work.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Transformation type `--type DATETIME` 1572766460  
1407767434 https://github.com/simonw/datasette/issues/1696#issuecomment-1407767434 https://api.github.com/repos/simonw/datasette/issues/1696 IC_kwDOBm6k_c5T6NOK cldellow 193185 2023-01-29T20:56:20Z 2023-01-29T20:56:20Z CONTRIBUTOR

I did some horrible things in https://github.com/cldellow/datasette-ui-extras/issues/2 to enable this in my plugin -- example here: https://dux-demo.fly.dev/cooking/posts?_facet=owner_user_id&owner_user_id=67

The implementation relies on two things:

  • a filters_from_request hook that adds a good human description (unfortunately, without the benefit of the CSS styling you mention)
  • doing something evil to hijack the exact and not operators in the Filters class. We can't leave them as is, or we'll get 2 human descriptions -- the built-in Datasette one and the one from my plugin. We can't remove them, or the filters UI will stop supporting the = and != operators

This got me thinking: it'd be neat if the list of operators that the filters UI supported wasn't a closed set.

A motivating example: adding a geospatial NEAR operator. Ideally it'd take two arguments - a target point and a radius, so you could express a filter like find me all rows whose lat/lng are within 10km of 43.4516° N, 80.4925° W. (Optionally, the UI could be enhanced if the geonames database was loaded and queried, so a user could say find me all rows whose lat/lng are within 10km of Kitchener, ON, and the city gets translated to a lat/lng for them)

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Show foreign key label when filtering 1186696202  
1407716963 https://github.com/simonw/datasette/pull/2008#issuecomment-1407716963 https://api.github.com/repos/simonw/datasette/issues/2008 IC_kwDOBm6k_c5T6A5j cldellow 193185 2023-01-29T17:04:03Z 2023-01-29T17:04:03Z CONTRIBUTOR

Performance tests - I think most places don't have them as a formal gate enforced by CI. TypeScript and scalac seem to have tests that run to capture timings. The timings are included by a bot as a comment or build check, and also stored in a database so you can graph changes over time to spot regressions. Probably overkill for Datasette!

Window functions - oh, good point. Looks like Ubuntu shipped JSON1 support as far back as sqlite 3.11. I'll let this PR linger until there's a way to run against different SQLite versions. For now, I'm shipping this with datasette-ui-extras, since I think it's OK for a plugin to enforce a higher minimum requirement.

Tests - there actually did end up being test changes to capture the undercount bug of the current implementation, so the current implementation would fail against the new tests.

Perhaps a non-window function version could be written that uses random() instead of row_number() over () in order to get a unique key. It's technically not unique, but in practice, I imagine it'll work well.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
array facet: don't materialize unnecessary columns 1560982210  
1407561308 https://github.com/simonw/datasette/pull/2008#issuecomment-1407561308 https://api.github.com/repos/simonw/datasette/issues/2008 IC_kwDOBm6k_c5T5a5c cldellow 193185 2023-01-29T04:50:50Z 2023-01-29T04:50:50Z CONTRIBUTOR

I pushed a revised version which ends up being faster -- the example which currently takes 4 seconds now runs in 500ms.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
array facet: don't materialize unnecessary columns 1560982210  
1407558284 https://github.com/simonw/datasette/pull/2008#issuecomment-1407558284 https://api.github.com/repos/simonw/datasette/issues/2008 IC_kwDOBm6k_c5T5aKM cldellow 193185 2023-01-29T04:23:58Z 2023-01-29T04:24:27Z CONTRIBUTOR

Ack, this PR is broken. I see now that the inner.* is necessary for ensuring the correct count in the face of rows having duplicate values in views.

That fixes the overcounting, but I think can undercount when the rows have the same data, eg a view like:

sql SELECT '["bar"]' tags UNION ALL SELECT '["bar"]'

will produce a count of {"bar": 1 }, when it should be {"bar": 2}. In fact, this could apply in tables without primary keys, too.

If inner came from a base table that had a primary key or a rowid, we could use those column(s) to solve that case.

I guess a general solution would be to compute a window function so we have a distinct ID for each row. Will fiddle to see if I can get that working.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
array facet: don't materialize unnecessary columns 1560982210  
1407523547 https://github.com/simonw/datasette/issues/1973#issuecomment-1407523547 https://api.github.com/repos/simonw/datasette/issues/1973 IC_kwDOBm6k_c5T5Rrb cldellow 193185 2023-01-29T00:40:31Z 2023-01-29T00:40:31Z CONTRIBUTOR

A +1 for switching to CustomRow: I think you currently only get a CustomRow if the result set had a column that was an fkey (this code)

Otherwise you get vanilla sqlite3.Rows, which will fail if you try to access .columns or lookup the cell by name, which surprised me recently

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
render_cell plugin hook's row object is not a sqlite.Row 1515815014  
1407470429 https://github.com/simonw/datasette/pull/2008#issuecomment-1407470429 https://api.github.com/repos/simonw/datasette/issues/2008 IC_kwDOBm6k_c5T5Etd cldellow 193185 2023-01-28T19:34:29Z 2023-01-28T19:34:29Z CONTRIBUTOR

I don't know how/if you do automated tests for performance, so I haven't changed any of the tests.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
array facet: don't materialize unnecessary columns 1560982210  
1403084856 https://github.com/simonw/datasette/issues/2001#issuecomment-1403084856 https://api.github.com/repos/simonw/datasette/issues/2001 IC_kwDOBm6k_c5ToWA4 cldellow 193185 2023-01-25T04:31:02Z 2023-01-25T04:31:02Z CONTRIBUTOR

Aha, it's user error on my part.

Adding

sqlite3_db_config.argtypes = [ctypes.c_void_p, ctypes.c_int, ctypes.c_int, ctypes.c_int]

makes it work reliably both on the CLI and from datasette, and now I can reproduce the errors you mentioned in the issue description.

{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Datasette is not compatible with SQLite's strict quoting compilation option 1553615704  
1403078134 https://github.com/simonw/datasette/issues/2001#issuecomment-1403078134 https://api.github.com/repos/simonw/datasette/issues/2001 IC_kwDOBm6k_c5ToUX2 cldellow 193185 2023-01-25T04:20:43Z 2023-01-25T04:22:28Z CONTRIBUTOR

I'm on Ubuntu, unfortunately. :( Would it still be relevant?

I think I've narrowed things down a bit more.

Even sqlite3_free(sqlite3_malloc(128)) segfaults -- this suggests to me that it's something about the sqlite3 library that was loaded, vs, say, getting the wrong db handle when I go spelunking in the Connection object.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Datasette is not compatible with SQLite's strict quoting compilation option 1553615704  
1403053144 https://github.com/simonw/datasette/issues/2001#issuecomment-1403053144 https://api.github.com/repos/simonw/datasette/issues/2001 IC_kwDOBm6k_c5ToORY cldellow 193185 2023-01-25T03:34:53Z 2023-01-25T03:34:53Z CONTRIBUTOR

Your comment introduced me to this issue in sqlite and to the ctypes module - thanks!

I also hope that the datasette developers will enable this mode in a test environment [...] perhaps we could figure out how to invoke it using ctypes

I'm not a Datasette developer, but I am curious to learn more about getting unholy access to the sqlite C APIs inside of Datasette. (Such access could also help #1293, and if done without grovelling inside of pysqlite's Connection object for the db handle, could even be relatively safe.)

I experimented a bit. I came up with https://gist.github.com/cldellow/85bba507c314b127f85563869cd94820

If you run python3 enable-strict-quoting-sqlite3.py, it seems to set those flags correctly -- SELECT "foo" fails where it would normally succeed.

But if you put it in a plugins/ dir and run datasette --plugins-dir plugins/, it segfaults when it tries to call sqlite3_db_config on the connections created by Datasette.

I am... confused. I'm pretty sure I'm using the same python and the same libsqlite3 in both scenarios, so I would expect it to work.

@gwk do you know anything that might help me debug the segfault? I gather that my approach of going grovelling inside of a PyObject is particularly dangerous, but I was thinking (a) it's necessary in order to test Datasette's use of the sqlite3 library and (b) even if it's not portable, it'd be good enough for running the tests on a single machine.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Datasette is not compatible with SQLite's strict quoting compilation option 1553615704  
1399847946 https://github.com/simonw/datasette/issues/2000#issuecomment-1399847946 https://api.github.com/repos/simonw/datasette/issues/2000 IC_kwDOBm6k_c5Tb_wK cldellow 193185 2023-01-23T06:08:00Z 2023-01-23T06:08:00Z CONTRIBUTOR

Actually, I discovered your post showing how a plugin can add a Datasette hook. That's wild! I've released datasette-rewrite-sql that adds this ability, albeit via monkey patching.

I had hoped to be able to expose request to the hook (or, even better actor) when the SQL was being run as a result of a user's HTTP request.

But some spelunking in the code makes me suspect that would actually require co-operation from Datasette itself. I'd be happy to be wrong and pointed in the right direction, though!

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
rewrite_sql hook 1552368054  
1399589414 https://github.com/simonw/datasette/pull/1159#issuecomment-1399589414 https://api.github.com/repos/simonw/datasette/issues/1159 IC_kwDOBm6k_c5TbAom cldellow 193185 2023-01-22T19:48:41Z 2023-01-22T19:48:41Z CONTRIBUTOR

Hey @lovasoa, I hope you don't mind - I pulled this PR into datasette-ui-extras, a plugin I'm making that collects UI tweaks to Datasette.

You can apply it to your own Datasette instance by running datasette install datasette-ui-extras

{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Improve the display of facets information 774332247  
1369044959 https://github.com/simonw/datasette/issues/1973#issuecomment-1369044959 https://api.github.com/repos/simonw/datasette/issues/1973 IC_kwDOBm6k_c5Rmfff cldellow 193185 2023-01-02T15:41:40Z 2023-01-02T15:41:40Z CONTRIBUTOR

Thanks for the response!

Yes, it does seem like a pretty nice developer experience--both the automagical labelling of fkeys, and the ability to index the row by column name in addition to column index.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
render_cell plugin hook's row object is not a sqlite.Row 1515815014  
612216820 https://github.com/simonw/datasette/issues/236#issuecomment-612216820 https://api.github.com/repos/simonw/datasette/issues/236 MDEyOklzc3VlQ29tbWVudDYxMjIxNjgyMA== cldellow 193185 2020-04-10T21:03:38Z 2020-04-10T21:03:38Z CONTRIBUTOR

I made a repo at https://github.com/code402/datasette-lambda to demonstrate the idea, and scratch my personal itch for this.

The demo relies on some central authority having already published a public, reusable Lambda layer with Datasette & its dependencies. I think that differs from the other publish plugins which seem to mainly publish Dockerfiles that the host will interpret to install deps from a requirements.txt file.

I chose that approach because uvloop appears to be a dependency with native code that needs to be compiled for the target runtime environment. In this case, that's Amazon Linux 2. I'm not 100% clear on whether that's still required, because:

  • maybe uvloop is only needed for uvicorn, which the demo doesn't actually use since HTTP routing is handled by API Gateway
  • it seems like uvloop may be an optional, drop-in optimization for asyncio in any case (but I may be misreading this; I'm very much a Python noob)

If it's the case that uvloop is truly optional, then I think the publish plugin could do the packaging on the user's machine, regardless of what flavour of operating system they're on. That'd be a bit slower for the user, but would provide the most long-term flexibility in terms of supporting plugins.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
datasette publish lambda plugin 317001500  
608716819 https://github.com/simonw/datasette/issues/236#issuecomment-608716819 https://api.github.com/repos/simonw/datasette/issues/236 MDEyOklzc3VlQ29tbWVudDYwODcxNjgxOQ== cldellow 193185 2020-04-03T22:19:00Z 2020-04-03T22:19:00Z CONTRIBUTOR

Hi Simon,

I'm thinking of attempting this. Can you clarify some questions I have?

1) I assume the goal is to have a CORS-friendly HTTPS endpoint that hosts the datasette service + user's db.

2) If that's the goal, I think Lambda alone is insufficient. Lambda provides the compute fabric, but not the HTTP routing. You'd also need to add Application Load Balancer or API Gateway to provide an HTTP endpoint that routes to the lambda function.

Do you have a preference between ALB or API GW? ALB has better economics at scale, but has a minimum monthly cost. API GW has worse per-request economics, but scales to zero when no requests are happening.

3) Does Datasette have any native components, or is it all pure python? If it has native bits, they'll likely need to be recompiled to work on Amazon Linux 2.

4) There are a few disparate services that need to be wired together to expose a Python service securely to the web. If I was doing this outside of the datasette publish system, I'd use an AWS CloudFormation template. Even within datasette, I think it still makes sense to use a CloudFormation template and just have the publish plugin invoke it (via the standard aws cli) with user-specified parameters. Does that sound reasonable to you?

Thanks for your help!

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
datasette publish lambda plugin 317001500  

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
, [performed_via_github_app] TEXT);
CREATE INDEX [idx_issue_comments_issue]
                ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
                ON [issue_comments] ([user]);
Powered by Datasette · Queries took 19.402ms · About: github-to-sqlite
  • Sort ascending
  • Sort descending
  • Facet by this
  • Hide this column
  • Show all columns
  • Show not-blank rows