issue_comments

I have yet to build Datasette plugin and am unfamiliar with Pluggy. Since browsers have event handling builtin Datasette could communicate with plugins through it. Handlers register as listeners for custom Datasette events and Datasette's JS can then trigger said events.

I was also wondering if you had looked at Javascript Modules for JS plugins? With services like Skypack (https://www.skypack.dev) NPM libraries can be loaded directly into browser, no build step needed. Same goes for local JS if you adhere to ES Module spec.

If minification is required then tools such as Snowpack (https://www.snowpack.dev) could fit better. It uses https://github.com/evanw/esbuild for bundling and minification.

On plugins you'd simply:

javascript import {register} from '/assets/js/datasette' register.on({'click' : my_func})

In Datasette HTML pages' head you'd merely import these files as modules one by one.

I should configure the action to only run if changes have been made within the datasette/static directory.

https://github.com/mishoo/UglifyJS/issues/1905#issuecomment-300485490 says:

sourceMappingURL aren't added by default in 3.x due to one of the feature requests not to - some users are putting them within HTTP response headers instead.

So the command line for that would be:

js $ uglifyjs main.js -cmo main.min.js --source-map url=main.min.js.map

I had to add this as the first line in table.min.js for the source mapping to work: //# sourceMappingURL=/-/static/table.min.js.map

I started with a 7K table.js file.

npx uglifyjs table.js --source-map -o table.min.js gave me a 5.6K table.min.js file.

npx uglifyjs table.js --source-map -o table.min.js --compress --mangle gave me 4.5K.

I'm trying to minify table.js and I ran into a problem:

Uglification failed. Unexpected character '`'

It turns out uglify-js doesn't support ES6 syntax!

But uglify-es does:

npm install uglify-es

Annoyingly it looks like uglify-es uses the same CLI command, uglifyjs. So after installing it this seemed to work:

npx uglifyjs table.js --source-map -o table.min.js

I really don't like how npx uglifyjs could mean different things depending on which package was installed.

For inlining the plugins.min.js file into the Jinja templates I could use the trick described here: https://stackoverflow.com/a/41404611 - which adds a {{ include_file('file.txt') }} function to Jinja.

You'll be able to add JavaScript plugins using a bunch of different mechanisms:

• In a custom template, dropping the code in to a <script> block
• A bookmarklet that injects an extra script (I'm really excited to try this out)
• A separate script.js file that's loaded into Datasette using the "extra_js_urls" metadata option, documented here: https://docs.datasette.io/en/stable/custom_templates.html#custom-css-and-javascript
• A plugin you can install, like datasette-vega or datasette-cluster-map - since plugins can bundle their own script files that then get loaded on pages via this hook: https://docs.datasette.io/en/stable/plugin_hooks.html#extra-js-urls-template-database-table-columns-view-name-request-datasette

Amazing work! And you've put in far more work than I'd expect to reduce the payload (which is admirable).

So, to add a plugin with the current design, it goes in (a) the template or (b) a bookmarklet, right?

If I'm going to do that, it would be good if subsequent plugins that register against the load event are executed straight away. That's a bit of a weird edge-case in plugin world - it would involve the bulkier code that gets loaded redefining how datasette.plugins.register works to special-case the 'load' hook.

Maybe the tiny bootstrap code could define a datasette.plugins.onload(callbackFunction) method which gets upgraded later into something that fires straight away? Would add more bytes though.

Eventually I'd like to provide a whole bunch of other datasette.X utility functions that plugins can use - things like datasette.addTabbedContentPane() or similar.

But I don't want to inline those into the page.

So... I think the basic plugin system remains inline - maybe from an inlined file called plugins-bootstrap.js. Then a separate plugins.js contains the rest of the API functionality.

If a plugin wants to take advantage of those APIs, maybe it registers itself using datasette.plugins.register('load', () => ...) - that load hook can then be fired once the bulkier plugin code has been loaded.

I'm going to use class="plugin-content-pre-table" rather than id= - just because I still want to be able to display all of this stuff on the single https://latest.datasette.io/-/patterns page so duplicate IDs are best avoided.

I think I need to keep the mechanism whereby a plugin can return undefined in order to indicate that it has nothing to say for that specific item - that's borrowed from Pluggy and I've used it a bunch in my Python plugins. That makes the code a bit longer.

I'll write some example plugins to help me decide if the filtering-out-of-undefined mechanism is needed or not.

Using object destructuring like that is a great idea. I'm going to play with your version - it's delightfully succinct.

Wrote a TIL about this: https://til.simonwillison.net/github-actions/prettier-github-actions

https://github.com/simonw/datasette/runs/1631682372?check_suite_focus=true failed!

Oops, committed that bad formatting test to main instead of a branch!

Tests passed in https://github.com/simonw/datasette/runs/1631677726?check_suite_focus=true

I'm going to try submitting a pull request with badly formatted JavaScript to see if it gets caught.

I think this should work: - uses: actions/cache@v2 with: path: ~/.npm key: ${{ runner.os }}-node-${{ hashFiles('**/prettier.yml' }} I'll use the prettier.yml workflow that I'm about to create as the cache key for the NPM cache.

Maybe not that action actually - I wanted to use a pre-built action to avoid installing Prettier every time, but that's what it seems to do: https://github.com/creyD/prettier_action/blob/bb361e2979cff283ca7684908deac8f95400e779/entrypoint.sh#L28-L37

This action looks good - tag 3.2 is equivalent to this commit hash: https://github.com/creyD/prettier_action/tree/bb361e2979cff283ca7684908deac8f95400e779

I want a CI check that confirms that files conform to prettier - but only datasette/static/*.js files that are not already minified.

This seems to do the job:

npx prettier --check 'datasette/static/*[!.min].js'

Sorry to go on about it, but it's my only example ;) And thought it might be of interest/use. Here is FixMyStreet's Cypress workflow https://github.com/mysociety/fixmystreet/blob/master/.github/workflows/cypress.yml with the master script that sets up server etc at https://github.com/mysociety/fixmystreet/blob/master/bin/browser-tests (that has features such as working inside/outside Vagrant, and can do JS code coverage) and then the tests are at https://github.com/mysociety/fixmystreet/tree/master/.cypress/cypress/integration

If you're using arrow functions, you can presumably use default parameters, not much difference in support. That would save you 9 bytes. But OTOH you need "use strict"; to use arrow functions etc, and that's 13 bytes.

Your latest 250-byte one, with use strict, gzips to 199 bytes. The following might be 292 bytes, but compresses to 204, basically the same, and works in any browser (well, IE9+) at all:

var datasette=datasette||{};datasette.plugins=function(){var d={};return{register:function(b,c,e){d[b]||(d[b]=[]);d[b].push([c,e])},call:function(b,c){c=c||{};var e=[];(d[b]||[]).forEach(function(a){a=a[0].apply(a[0],a[1].map(function(a){return c[a]}));void 0!==a&&e.push(a)});return e}}}();

Source for that is below; I replaced the [fn,parameters] because closure-compiler includes a polyfill for that, and I ran closure-compiler --language_out ECMASCRIPT3:

js var datasette = datasette || {}; datasette.plugins = (() => { var registry = {}; return { register: (hook, fn, parameters) => { if (!registry[hook]) { registry[hook] = []; } registry[hook].push([fn, parameters]); }, call: (hook, args) => { args = args || {}; var results = []; (registry[hook] || []).forEach((data) => { /* Call with the correct arguments */ var result = data[0].apply(data[0], data[1].map(parameter => args[parameter])); if (result !== undefined) { results.push(result); } }); return results; } }; })();

If you could say that all hook functions had to accept one options parameter (and could use object destructuring if they wished to only see a subset), you could have this, which minifies (to all-browser-JS) to 200 bytes, gzips to 146, and works practically the same:

js var datasette = datasette || {}; datasette.plugins = (() => { var registry = {}; return { register: (hook, fn) => { registry[hook] = registry[hook] || []; registry[hook].push(fn); }, call: (hook, args) => { var results = (registry[hook] || []).map(fn => fn(args||{})); return results; } }; })();

var datasette=datasette||{};datasette.plugins=function(){var b={};return{register:function(a,c){b[a]=b[a]||[];b[a].push(c)},call:function(a,c){return(b[a]||[]).map(function(a){return a(c||{})})}}}();

Called the same, definitions tiny bit different:

js datasette.plugins.register('numbers', ({a, b}) => a + b) datasette.plugins.register('numbers', o => o.a * o.b) datasette.plugins.call('numbers', {a: 4, b: 6})

https://github.com/PostHog/posthog/tree/master/cypress/integration has some useful examples, linked from this article: https://posthog.com/blog/cypress-end-to-end-tests

Also useful: their workflow https://github.com/PostHog/posthog/blob/master/.github/workflows/e2e.yml

Important to absorb the slightly bizarre assertion syntax from Chai - docs here https://www.chaijs.com/api/bdd/

I got Cypress working! I added the datasette.plugins code to the table template and ran a test called plugins.spec.js using the following: javascript context('datasette.plugins API', () => { beforeEach(() => { cy.visit('/fixtures/compound_three_primary_keys') }); it('should exist', () => { let datasette; cy.window().then(win => { datasette = win.datasette; }).then(() => { expect(datasette).to.exist; expect(datasette.plugins).to.exist; }); }); it('should register and execute plugins', () => { let datasette; cy.window().then(win => { datasette = win.datasette; }).then(() => { expect(datasette.plugins.call('numbers')).to.deep.equal([]); // Register a plugin datasette.plugins.register("numbers", (a, b) => a + b, ['a', 'b']); var result = datasette.plugins.call("numbers", {a: 1, b: 2}); expect(result).to.deep.equal([3]); // Second plugin datasette.plugins.register("numbers", (a, b) => a * b, ['a', 'b']); var result2 = datasette.plugins.call("numbers", {a: 1, b: 2}); expect(result2).to.deep.equal([3, 2]); }); }); });